diff --git a/Cargo.toml b/Cargo.toml index c027eed..bee9dac 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "preinterpret" -version = "0.1.0" +version = "0.1.1" edition = "2021" license = "MIT OR Apache-2.0" authors = ["David Edey "] @@ -10,7 +10,7 @@ keywords = ["macros", "declarative-macros", "toolkit", "interpreter", "preproces # Categories have to come from https://crates.io/category_slugs # And whilst there's a development-tools::procedural-macro-helpers, there's no declarative macro category. categories = ["development-tools", "compilers"] -# MSRV is the start of Edition 2021 +# MSRV 1.56.0 is the start of Edition 2021 # If changing this, update the local-check-msrv.sh script, ci.yml, and README.md rust-version = "1.56.0" diff --git a/README.md b/README.md index cae9606..1ae88a9 100644 --- a/README.md +++ b/README.md @@ -234,30 +234,43 @@ To create idents from these methods, simply nest them, like so: > > Such characters get dropped in camel case conversions. This could break up grapheme clusters and cause other non-intuitive behaviour. See the [tests in string_conversion.rs](https://www.github.com/dhedey/preinterpret/blob/main/src/string_conversion.rs) for more details. -### Integer commands (COMING SOON!) +## Contributing + +If you have an idea for additional commands, please raise an issue or a PR. + +Any commands should be simple, composable, and likely to see use in declarative macros, to keep code bloat down. + +## Future Extension Reflections + +### Possible extension: Integer commands Each of these commands functions in three steps: * Apply the interpreter to the token stream, which recursively executes preinterpret commands. * Iterate over each token (recursing into groups), expecting each to be an integer literal. * Apply some command-specific mapping to this stream of integer literals, and output a single integer literal without its type suffix. The suffix can be added back manually if required with a wrapper such as `[!literal! [!add! 1 2] u64]`. -The supported integer commands are: +Integer commands under consideration are: * `[!add! 5u64 9 32]` outputs `46`. It takes any number of integers and outputs their sum. The calculation operates in `u128` space. * `[!sub! 64u32 1u32]` outputs `63`. It takes two integers and outputs their difference. The calculation operates in `i128` space. * `[!mod! $length 2]` outputs `0` if `$length` is even, else `1`. It takes two integers `a` and `b`, and outputs `a mod b`. We also support the following assignment commands: -* `[!increment! #i]` is shorthand for `[!set! #i [!add! #i 1]]` and outputs no tokens. -### Boolean commands (COMING SOON!) +* `[!increment! #i]` is shorthand for `[!set! #i [!add! #i 1]]` and outputs no tokens. + +We could even support: + +* `[!math! (5 + 10) / mod(4, 2)]` outputs `7` + +### Possible extension: Boolean commands Each of these commands functions in three steps: * Apply the interpreter to the token stream, which recursively executes preinterpret commands. * Expects to read exactly two token trees (unless otherwise specified) * Apply some command-specific comparison, and outputs the boolean literal `true` or `false`. -The supported comparison commands are: +Comparison commands under consideration are: * `[!eq! #foo #bar]` outputs `true` if `#foo` and `#bar` are exactly the same token tree, via structural equality. For example: * `[!eq! (3 4) (3 4)]` outputs `true` because the token stream ignores spacing. * `[!eq! 1u64 1]` outputs `false` because these are different literals. @@ -268,44 +281,29 @@ The supported comparison commands are: * `[!not! #foo]` expects a single boolean literal, and outputs the negation of `#foo` * `[!str_contains! "needle" [!string! haystack]]` expects two string literals, and outputs `true` if the first string is a substring of the second string. -### Control flow commands (COMING SOON!) +### Possible extension: Token stream commands +* `[!skip! 4 from [#stream]]` reads and drops the first 4 token trees from the stream, and outputs the rest +* `[!ungroup! (#stream)]` outputs `#stream`. It expects to receive a single group (i.e. wrapped in brackets), and unwraps it. + +### Possible extension: Control flow commands + +#### If statement -Currently, only `if` is supported: -* `[!if! #cond then { #a } else { #b }]` outputs `#a` if `#cond` is `true`, else `#b` if `#cond` is false. +`[!if! #cond then { #a } else { #b }]` outputs `#a` if `#cond` is `true`, else `#b` if `#cond` is false. The `if` command works as follows: * It starts by only interpreting its first token tree, and expects to see a single `true` or `false` literal. * It then expects to reads an unintepreted `then` ident, following by a single `{ .. }` group, whose contents get interpreted and output only if the condition was `true`. * It optionally also reads an `else` ident and a by a single `{ .. }` group, whose contents get interpreted and output only if the condition was `false`. -## Contributing +#### For loop -If you have an idea for additional commands, please raise an issue or a PR. +* `[!for! #token_tree in [#stream] { ... }]` -Any commands should be simple, composable, and likely to see use in declarative macros, to keep code bloat down. +#### Goto and label -## Future Extension Reflections - -### Eager expansion of macros -When [eager expansion of macros returning literals](https://github.com/rust-lang/rust/issues/90765) is stabilized, it would be nice to include a command to do that, which could be used to include code, for example: `[!expand_literal_macros! include!("my-poem.txt")]`. - -### Removing syn - -The heavy `syn` library is only needed for literal parsing, and error conversion into compile errors. This could be removed to speed up compile times a lot for stacks which don't have a `syn` dependency. - -### Further Control Flow - -In theory, some additional control flow or lazy evaluation could be added to bring turing completeness. This would also allow the preinterpret to find some niche uses outside of declarative macros. - -I'm not sure if the complexity is warranted though. For declarative macro use-cases, the looping logic is currently provided by the meta-variables from the declarative macro. Mixing this with a separate layer of control flow might be confusing. - -But in theory, we could add a naive control flow with `[!label! loop_start]` and `[!goto! loop_start]`, where: -* `goto` could only jump backwards to labels which have already been read. -* A label's position is defined as the position in the input stream after the currently executing command stack. e.g. `Hello [!label! loop] World [!goto! loop]` -* For the jump to make consistent sense in terms of keeping brackets balanced, the label and its goto must exist in the same parent group / stream. -* It would need a bit of a rearchitecture of the token streaming logic to support holding a clone of the input stream at the label point to enable jumping backwards in the stream to that point. - -Potential syntax: +* `[!label! loop_start]` - defines a label which can be returned to. Effectively, it takes a clones of the remaining token stream after the label in the interpreter. +* `[!goto! loop_start]` - jumps to the last execution of `[!label! loop_start]`. It unrolls the preinterpret stack (dropping all unwritten token streams) until it finds a stackframe in which the interpreter has the defined label, and continues the token stream from there. ```rust preinterpret::preinterpret!{ @@ -315,4 +313,11 @@ preinterpret::preinterpret!{ [!set! #i = [!add! i 1]] [!if! [!lte! #i 100] then { [!goto! loop] }] } -``` \ No newline at end of file +``` + +### Possible extension: Eager expansion of macros +When [eager expansion of macros returning literals](https://github.com/rust-lang/rust/issues/90765) is stabilized, it would be nice to include a command to do that, which could be used to include code, for example: `[!expand_literal_macros! include!("my-poem.txt")]`. + +### Possible extension: Removing syn + +The heavy `syn` library is only needed for literal parsing, and error conversion into compile errors. This could be removed to speed up compile times a lot for stacks which don't have a `syn` dependency. diff --git a/src/lib.rs b/src/lib.rs index 717833d..39010dd 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -97,7 +97,7 @@ //! [!set! #current_index = 0] //! $( //! [!ignore! $item] // Loop over the items, but don't output them -//! [!increment! #current_index] +//! [!set! #current_index = #current_index + 1] //! )* //! [!set! #count = #current_index] //! #count @@ -112,16 +112,17 @@ //! preinterpret::preinterpret!{ //! [!set! #current_index = 0] //! [!ignore! a] -//! [!increment! #current_index] +//! [!set! #current_index = #current_index + 1] //! [!ignore! = b] -//! [!increment! #current_index] +//! [!set! #current_index = #current_index + 1] //! [!ignore! = c] -//! [!increment! #current_index] +//! [!set! #current_index = #current_index + 1] //! [!set! #count = #current_index] //! #count //! } //! ``` -//! Now the `preinterpret!` macro runs, resulting in `#count` equal to the literal `3`. +//! Now the `preinterpret!` macro runs, resulting in `#count` equal to the token stream `0 + 1 + 1 + 1`. +//! This will be improved in future releases by adding support for mathematical operations on integer literals. //! //! ### Heightened sensibility //! @@ -223,50 +224,9 @@ //! > //! > Such characters get dropped in camel case conversions. This could break up grapheme clusters and cause other non-intuitive behaviour. See the [tests in string_conversion.rs](https://www.github.com/dhedey/preinterpret/blob/main/src/string_conversion.rs) for more details. //! -//! ### Integer commands (COMING SOON!) +//! ## Future plans //! -//! Each of these commands functions in three steps: -//! * Apply the interpreter to the token stream, which recursively executes preinterpret commands. -//! * Iterate over each token (recursing into groups), expecting each to be an integer literal. -//! * Apply some command-specific mapping to this stream of integer literals, and output a single integer literal without its type suffix. The suffix can be added back manually if required with a wrapper such as `[!literal! [!add! 1 2] u64]`. -//! -//! The supported integer commands are: -//! -//! * `[!add! 5u64 9 32]` outputs `46`. It takes any number of integers and outputs their sum. The calculation operates in `u128` space. -//! * `[!sub! 64u32 1u32]` outputs `63`. It takes two integers and outputs their difference. The calculation operates in `i128` space. -//! * `[!mod! $length 2]` outputs `0` if `$length` is even, else `1`. It takes two integers `a` and `b`, and outputs `a mod b`. -//! -//! We also support the following assignment commands: -//! * `[!increment! #i]` is shorthand for `[!set! #i [!add! #i 1]]` and outputs no tokens. -//! -//! ### Boolean commands (COMING SOON!) -//! -//! Each of these commands functions in three steps: -//! * Apply the interpreter to the token stream, which recursively executes preinterpret commands. -//! * Expects to read exactly two token trees (unless otherwise specified) -//! * Apply some command-specific comparison, and outputs the boolean literal `true` or `false`. -//! -//! The supported comparison commands are: -//! * `[!eq! #foo #bar]` outputs `true` if `#foo` and `#bar` are exactly the same token tree, via structural equality. For example: -//! * `[!eq! (3 4) (3 4)]` outputs `true` because the token stream ignores spacing. -//! * `[!eq! 1u64 1]` outputs `false` because these are different literals. -//! * `[!lt! #foo #bar]` outputs `true` if `#foo` is an integer literal and less than `#bar` -//! * `[!gt! #foo #bar]` outputs `true` if `#foo` is an integer literal and greater than `#bar` -//! * `[!lte! #foo #bar]` outputs `true` if `#foo` is an integer literal and less than or equal to `#bar` -//! * `[!gte! #foo #bar]` outputs `true` if `#foo` is an integer literal and greater than or equal to `#bar` -//! * `[!not! #foo]` expects a single boolean literal, and outputs the negation of `#foo` -//! * `[!str_contains! "needle" [!string! haystack]]` expects two string literals, and outputs `true` if the first string is a substring of the second string. -//! -//! ### Control flow commands (COMING SOON!) -//! -//! Currently, only `if` is supported: -//! * `[!if! #cond then { #a } else { #b }]` outputs `#a` if `#cond` is `true`, else `#b` if `#cond` is false. -//! -//! The `if` command works as follows: -//! * It starts by only interpreting its first token tree, and expects to see a single `true` or `false` literal. -//! * It then expects to reads an unintepreted `then` ident, following by a single `{ .. }` group, whose contents get interpreted and output only if the condition was `true`. -//! * It optionally also reads an `else` ident and a by a single `{ .. }` group, whose contents get interpreted and output only if the condition was `false`. - +//! Future plans for extensions are discussed in the [preinterpret README](https://crates.io/crates/preinterpret). mod command; mod commands; mod internal_prelude;