Processing large JSON files in PHP -
i trying process big (possibly 200m) json files. construction of file array of objects.
so along lines of:
[ {"property":"value", "property2":"value2"}, {"prop":"val"}, ... {"foo":"bar"} ] each object has arbitrary properties , not necessary share them other objects in array (as in, having same).
i want apply processing on each object in array , file potentially huge, cannot slurp whole file content in memory, decoding json , iterating on php array.
so ideally read file, fetch plenty info each object , process it. sax-type approach ok if there similar library available json.
any suggestion on how deal problem best?
i decided on working on event based parser. it's not quite done yet , edit question link work when roll out satisfying version.
edit:
i worked out version of parser satisfied with. it's available on github:
https://github.com/kuma-giyomu/jsonparser
there's room improvement , welcoming feedback.
php json large-files
No comments:
Post a Comment