Merge the LazySystemReader and the LazyExpandingReader
#675
Labels
enhancement
New feature or request
Comments
zslayton
added a commit
that referenced
this issue
Nov 2, 2023
This PR enables the evaluation of user-defined template macros. At a high level, the changes involved include: * Introducing a `TemplateCompiler` that reads TDL and produces an in-memory representation that can be efficiently evaluated. * Consolidating the e-expression evaluator and the template macro evaluator into a unified `MacroEvaluator`. * Moving `SystemReader` functionality into `ExpandingReader` to consolidate (temporary) uses of `unsafe`. A later PR will merge these types. (See #675.) * Removing the `MacroInvocation` trait in favor of the `MacroExpr` enum. * Renaming `RawMacroInvocation` to `RawEExpression`, which is more accurate. * Renaming `LazyDecoder::RawMacroInvocation to `_::EExpression`. * Removing 'data lifetime from <'top, 'data, D> signatures. * Removing 'data lifetime from the `LazyDecoder` trait. * Caching nested expressions during parsing. Regarding expression caching: The text reader performs parsing in two phases: first, it matches values and expressions. Upon request, it can then read the expression it matched. Prior to this PR, structural information detected during the matching phase would be discarded. For example, when the reader would match a list, it would also match all of the list's child expressions in the process. However, the child expressions would not be stored; when the application began reading the list, each one would need to be matched again. This redundant matching effort would also happen for the fields of a struct or the arguments of an e-expression. The `TextEncoding_1_1` decoder now caches child and argument expressions in the bump allocator, which makes iterating over the container or e-expression "free" during the reading phase. As part of this change, the lifetime associated with `TextBufferView` was reduced from `'data` to `'top`. Readers now hold a `&[u8]` directly and construct a transient `TextBufferView` at read time. This allows the `TextBufferView` to hold a reference to the bump allocator, making the allocator available in all of the parsing methods without having to manually plumb it into each method.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
PR #674 hollowed out the
LazySystemReader, moving nearly all of its functionality to theLazyExpandingReader, which had visibility into when the raw reader was between top-level expressions.We can merge the two and name the result
LazySystemReader.The text was updated successfully, but these errors were encountered: