ragel-based json parser

[WillAbides]

I've been working on a ragel-based json parser for go.

I started on this to answer the question of why there isn't already a ragel-based generalized json parser for go. I expected to find blocker or performance issues along the way. Instead it's turned into a very efficient parser.

As I'm new to ragel, I am interested in any feedback I can get and anything I could be doing better.

The most interesting machine definitions are skipper and handleObjectValues. They are for finding the end of a json value and running a handler on each value in a json object.

[cjheath]

I'm also new to Ragel, and would like to hear feedback from someone who knows. JSON has unlimited nesting, which means it cannot be parsed by any finite state machine or regular language. Does your implementation overcome this using some feature of Ragel, or by adding an additional (unlimited) state stack?

[WillAbides]

rjson recursively calls either the array or object machine for each nested item. It isn't using ragel's state stack, but it is adding to the call stack at each level. I would have liked to try doing the recursion within the ragel machine, but I didn't find a way to expose the API I wanted doing that.

Also, it's not unlimited nesting. The rfc describing JSON allows for a maximum depth to be set. rjson has the same limit as the json parser in the go standard library which is 10,000 IIRC.

[cjheath]

Understandable, though disappointing. I'm working on a related tool that has more powerful parsing capability (using a VM approach ala Pike rather than a code generator), and wondered if Ragel had added parsing already. But now I see that's what Colm does :)

[WillAbides]

If what you're working on is open source, I would be interested to see it.

[cjheath]

Sure. My regex implementation is part of my Unicode strings library for C++. The implementation and style of C++ I use is very skewed by the desire to target embedded hardware like ESP32 - smallest memory use and fewest allocations to reduce fragmentation. I still have a fair bit of tuning to do - in particular the "maximum parallelism" estimates are very conservative (these are used to predict maximum memory requirements for matching). My current plan is to implement recursion (in the form of regex subroutines), and build an alternate compiler for a prefix-BNF syntax, for power akin to PEG parsers. I already have a limited counter stack for counted repetitions, so I just need to extend that to function as an unlimited context stack. I'd be happy to receive any feedback on my C++, as I haven't written significant C++ for fifteen years until this.

https://github.com/cjheath/strpp