Wallogit.com
2017 © Pedro Peláez
Lightweight incremental streaming JSON parser
A really simple and lightweight, incremental parser for JSON streaming (concatenated JSON and newline-delimited JSON, in PHP. You can use this library to process a stream of data that consists of multiple JSON documents., (*2)
Table of contents, (*3)
We invest a lot of time developing, maintaining and updating our awesome open-source projects. You can help us sustain this high-quality of our work by becoming a sponsor on GitHub. Sponsors get numerous benefits in return, see our sponsoring page for details., (*4)
Let's take these projects to the next level together! 🚀, (*5)
A newline-delimited JSON (NDJSON) example stream consisting of 3 individual JSON documents could look like this:, (*6)
{ "id": 1, "name": "first" }
{ "id": 3, "name": "third" }
{ "id": 6, "name": "sixth" }
Less commonly, the same format is referred to as JSON lines (JSONL) or line-delimited JSON (LDJSON), which is not to be confused with JSON-LD. To avoid confusion, we consistently refer to this as newline-delimited JSON (NDJSON). If you control the generating side, we highly recommend going for NDJSON instead of using concatenated JSON as discussed below. See also clue/reactphp-ndjson., (*7)
For this project, the whitespace between the individual JSON documents is entirely optional. Instead of newlines, you can use any number of whitespace or none at all., (*8)
A concatenated JSON example stream consisting of 3 individual JSON documents could look like this:, (*9)
{ "id": 1, "name": "first" }{ "id": 3, "name": "third" }{ "id": 6, "name": "sixth"}
The input stream can be of arbitrary size and can be interrupted at any time. This is often useful for processing network streams, where the chunk/buffer size is not under your control and you could potentially read single bytes only., (*10)
Please note that this library is about processing a stream that can contain any number of JSON documents. It is assumed that each document has a reasonable size and fits into memory. This is not to be confused with a streaming parser for processing a single, huge JSON document that is too big to fit into memory., (*11)
Once installed, you can use the following sample code to parse a stream of JSON chunks:, (*12)
$parser = new StreamingJsonParser();
assert($parser->push('[ 1, 2') === array());
assert($parser->push('3 ]') === array(array(1, 2, 3));
assert($parser->push('{} {}') === array(array(), array());
This is actually only a really simple hack to call a normal document based parser whenever it thinks a full document has been found in the input stream. Because the normal parser is implemented as an extension (instead of userland PHP), this turns out to be pretty fast for streams that contain common, rather small objects., (*13)
You might want to use this if, (*14)
You probably don't want to use this if, (*15)
The recommended way to install this library is through Composer. New to Composer?, (*16)
This project does not currently follow SemVer. This will install the latest supported version:, (*17)
$ composer require clue/json-stream:^0.1.1
See also the CHANGELOG for details about version upgrades., (*18)
This project aims to run on any platform and thus does not require any PHP extensions and supports running on legacy PHP 5.3 through current PHP 8+ and HHVM. It's highly recommended to use PHP 7+ for this project., (*19)
To run the test suite, you first need to clone this repo and then install all dependencies through Composer:, (*20)
$ composer install
To run the test suite, go to the project root and run:, (*21)
$ php vendor/bin/phpunit
This project is released under the permissive MIT license., (*22)
Did you know that I offer custom development services and issuing invoices for sponsorships of releases and for contributions? Contact me (@clue) for details., (*23)
.ndjson file extension).Lightweight incremental streaming JSON parser
MIT
json streaming parser incremental parser
Lightweight incremental streaming JSON parser
MIT
json streaming parser incremental parser