RELEASES.md

v1.0.0 - June 5th, 2023

• Added ability to read from stdin in ictcc and generated binaries by specifying a filename of "-".
• Inherited attributes in an SDTS are now considered unsupported and trying to use them will cause an error. Inherited attribute use can be enabled by passing --exp inherited-attributes to ictcc, but it is unlikely to function properly.
• SDTS hook implementation functions can now return an error instead of just panicking when something goes wrong.
• Disconnected dependency graphs in simulated SDTSs are downgraded to warning, and only become error if there are multiple graphs that lead to the IR.
• Token classes can now be retrieved by class ID by using ByID() in the generated frontend's token package.
• Generated frontends can now detect when they are being fed empty input and will return a different error than just "unexpected end-of-text token".
• Codegen for binaries now clears old code before generating, preventing issues from arising when new generation doesn't completely replace every file.
• Added integration testing framework in tests and example testing framework in examples, both via shell.
• Significantly improved code style; replaced magic literals with proper constants, removed many unused functions, and limited the exported functions to only those necessary, among other things.
• References to loading a parser or other frontend components from 'cache' have been completely removed.
• The environment variable ICTIOBUS_SOURCE may now be used to give an alternate location for local Ictiobus source code when calling ictcc with the --dev flag.
• Fixed lack of support for built-in IR types as args to --ir.
• Fixed error message for impossible attribute reference pre-check giving the name of the attribute instead of the symbol it is on.
• Fixed referring to attribute that doesn't exist at translation time causing a panic. Now it's a normal error.
• Fixed epsilon productions completely breaking parser generation. This fixes the issue of SLR parsers being broken as it was the root cause.
• Fixed some generated token class variable names being generated as just "TC" if relying on rune names.
• Fixed generated SDTSs not checking the error returned by Bind() while initializing.
• Fixed SDTSs being applied in non-deterministic way. Now they will prefer translating the left-most nodes of parse trees, preventing some nasty possible use-before-set errors.
• Fixed (or removed) broken unit tests that were previously commented out.
• Fixed syntax error messages for token classes that start with newline sequences. Before, they were pointing to the apparently blank end of line where the token started, which while technically true, is next to useless for a human attempting to read the error.
• Updated CLI flags for ictcc:
  • Added -S/--suppress and -F/--fatal flags to control suppression and promotion of warnings.
  • Added --debug-sdts to debug the FISHI SDTS.
  • Added -P/--preproc to output a preprocessed FISHI spec before it is sent to the frontend.
  • Added -C/--command to provide FISHI code via CLI args.
  • Added -D/--dfa to output the generated parser's deterministic finite automaton.
  • Added -T/--parse-table to output the generated parser's parsing table.
  • Added --exp to enable experimental features.
  • Removed -p/--parser, --no-cache, and --no-cache-out flags.
• Updated CLI flags for generated binaries:
  • Added -S/--suppress and -F/--fatal flags to control suppression and promotion of warnings.
  • Added -s/--debug-sdts for debugging the SDTS.
  • Added -C/--command to provide input source code via CLI args.
  • Added -P/--preproc to output preprocessed source intput before it is sent to the frontend. This is only included if the generated binary performs a formatting/preprocessing step.
  • Changed -t to --tree and made -t be the shorthand version.
• Improved documentation:
  • Updated README.md to actually have example uses
  • Added new ictcc.md file to serve as the complete ictcc command reference.
  • Added new fishi-usage.md file to give full tutorial on FISHI usage.
  • Significatly improved code docs.
  • Added this version history file.

v0.7.0 - April 20th, 2023

• Ictiobus is now self-hosted with a frontend for FISHI specs generated by ictcc.
• Added the --dev flag to ictcc was added to make it look to local source code for Ictiobus when it needs to refer to them during binary generation. The local source is assumed to be where ictcc is executed from.
• The generated token classes are now put into a sub-package of the output package, to allow hook functions to refer to it separately. This might be changed in a later release.
• The import path for the generated frontend files can now be assumed by checking for module presence in the directory that they are output to. Token classes being a separate package necessitates this.

v0.6.1 - April 15th, 2023

• Diagnostics binary generation added. This binary contains a fully-functional generated frontend and is capable of analyzing source written in the language defined by the spec. Optionally, this binary can instead be instructed to perform input simulation of the frontend instead of analyzing input files.
• All flags parsing now uses the spf13/pflag library for GNU-style options.
• Added new flags to ictcc:
  • -d/--diag for generating a diagnostics binary for a language.
  • -f/--diag-format-pkg and -c/--diag-format-call for handling preprocessing of analyzed source in the diagnostics binary.
  • -q/--quiet to enable quiet mode to suppress progress output and other supplementary output.
  • --preserve-bin-source to keep source code output when it is generated as part of creating a binary.
  • --prefix to set a prefix for all generated source files. Combines well with --preserve-bin-source for debugging.
  • --tmpl-main to allow specification of the template used for the main.go file in generated binaries.
• Updated flags for ictcc:
  • added -v as shortcut for --lang-ver
  • changed --val-sdts-off, --val-sdts-trees, --val-sdts-graphs, --val-sdts-first, and --val-sdts-skip options to --sim-off, --sim-trees, --sim-graphs, --sim-first-err, and --sim-skip-errs respectively.
  • changed --pre-format to --debug-templates to betta demonstr8 its purpose.
• The current frontend for FISHI is now implicitly trusted as valid and will not have input simulation run on it; --sim-X options now apply to the language being analyzed.
• Fixed test of non-deterministic output of grammar.CreateFewestNonTermsAlternationsTable by adding additional possible success conditions for flaky cases.
• Added additional checks of SDTS sections of input specs during spec interpretation (%%actions section):
  • Specs can no longer define new attributes starting with $.
  • Attribute refs must point to an existing symbol in their action's associated grammar rule.
  • Attribute refs to terminal symbols can only refer to the built-in attributes $text and $ft.
  • Attribute refs to non-terminal symbols cannot refer to $text.
• Improved error reporting during checks of SDTS sections of specs.
• Fixed boostrap lexer error where attribute refs to terminal symbols were lexed as TCTerminal instead of TCAttrRef.

v0.6.0 - April 3rd, 2023

• The ictcc binary can completely parse an entire FISHI spec and generate a frontend. Yay! 38D
• Testing compiler for running SDTS now is put in .sim directory.
• Fixed several issues with fishi.md spec.

v0.5.3 - April 3rd, 2023

• Re-release of v0.5.2 but with ictcc reporting the correct version.

v0.5.2 - April 3rd, 2023

• Panics in SDTS hook functions are now caught and reported as errors.
• Code generation can now build all components of frontend.
• Added flags -tmpl-* for specifying tokens, lexer, parser, SDTS, and frontend template files to use instead of the embedded default.
• Added flags -slr, -ll, -clr, and -lalr to select generation of that type of parser as opposed to simply trying them until one is found that works.
• Added -no-ambig flag to disable the generation of LR parsers for languages that have an ambiguous grammar.
• Added -hooks, -hooks-table, and -ir flags for specifying type information and connecting external code to the generated frontend.

v0.5.1 - April 1st, 2023

• Code generation started with templates for tokens, lexer, parser, SDTS, and frontend.
• Add new flags for meta info during codegens, -pkg, -dest, -l/-lang, and -lang-ver.
• New flag -pre-format added to output filled codegen template prior to running gofmt on it.
• Renamed translation package to trans to keep it short.

v0.5.0 - March 30th, 2023

• The ictcc binary can now convert a FISHI AST into a language Spec. Use the -s/-spec CLI option to enable outputting it.
• First token of a grammar construct is now available to SDTS as the $ft ("First Token") built-in attribute.
• Various fixes to fishi.md.

v0.4.1 - March 26th, 2023

• Renamed FISHI %action directive to %set so there is less ambiguity between %%actions and %action.
• Added shorthand versions ->, (, :, and = for %prod, %with, %set, and %hook respectively to FISHI.
• Added ignored tokens ), ,, and () in %actions of FISHI for parity with ).
• Improved error reporting for SDTS validation errors.
• Split hooks into map of strings to hook function which is then passed to the SDTS.
• Release candidate for complete fishi spec in fishi.md.

v0.4.0 - March 25th, 2023

• Added new flags to ictcc binary:
  • -a/-ast to print the AST it reads.
  • -t/-tree to print the parse tree it reads.
  • -n/-no-gen to disable generation of the frontend.
  • -p/-parser to specify the CFF parser to use.
  • -no-cache to disable loading the parser from a CFF file.
  • -no-cache-out to disable writing the parser to a CFF file.
  • -version to print the version number and exit.
  • -val-sdts-off to disable SDTS validation, -val-sdts-trees to show the parse trees that caused SDTS errors, -val-sdts-graphs to show full dep graphs when they are broken, and -val-sdts-first and -val-sdts-skip to control which SDTS validation errors are shown.
  • -debug-lexer and -debug-parser to show additional output from the parser and lexer while reading FISHI input.
• Fixed dozens of mistakes in bootstrap grammar to be able to parse an AST.

v0.3.0 - March 24th, 2023

• Created ictcc binary which can read FISHI-containing markdown files and output the resulting AST.
• FISHI language and component changes:
  • Full ictiobus.Frontend creation for bootstrap frontend split into own file.
  • Added initial FISHI AST to use as IR.
  • Completed initial implementation of FISHI SDTS and hooks.
• SDTS functionality is now validated by automatic generation of valid blocks of source text which are then fed to the SDTS.
• Updated terminology referring to "SDD" to correctly refer to SDTS.
• ictiobus.Frontend now holds language metadata and debug options.
• Fixed unit tests and commented out harder ones to be fixed later.
• Frontend components can now be encoded to disk as bytes in "Compiled FISHI Format" (CFF). Used for the parser to increase testing speed.
• Added CI validation of gofmt and test execution.

v0.2.0 - March 4th, 2023

• Moved fishi.go into own sub-package, ictiobus/fishi.
• Split the bootstrap lexer, parser, and tokens from fishi.go into their own respective files in ictiobus/fishi.
• Created stubbed bootstrap SDTS in own file in ictiobus/fishi.
• Added ability to 'prioritize' which token is lexed first.
• Rough fixes to how syntax errors display the source line they were found on.

v0.1.0 - March 4th, 2023

• Moved entire in-progress codebase over from its origin as a sub-project of github.com/dekarrin/tunaq.