Typescript CALM Module Refactoring - Visualizer, Doc Publisher, Schema Directory, CALM Model

[LeighFinegold]

Background

Over recent discussions with developers (@aidanm3341, @oliviajanejohns, @willosborne, @Thels, @Budlee, @YoofiTT96, @Adwoa-Konadu-Appiah), we’ve been reviewing the current CALM codebase and future pull requests around documentation generation, CALM visualisation and the CLI server.

The current structure of the project uses the following organization:

• Workspaces:
  • shared: Contains submodules like generate, validate, and visualize.
  • cli: Published separately to npm and includes CLI-specific logic.
  • docs: Documentation-related functionality.

Relevant future contributions include:

• Visualization PR #786: Recent improvements to visualization.
• Docify prototype branch calm-docify as part of Issue #478.

---

Refactoring Discussed

1. Visualizer as a module to remain.:

   • The CALM Visualizer tool is an essential component to help users build CALM documents. It is due to be committed as its own workspace. The current visualiser module under shared was due to be deleted. However, there is consensus that a separate visualizer module should be created to support and integrate with docify workflows. This modular visualizer would allow docify and other tools to leverage visualization capabilities independently of the CLI and the tool and would make Feature Request: Markdown Parser for Inline CALM Document Rendering #782 possible.
   • The translator module, currently written in Java, was also discussed. We agreed it would be imported into the visualizer module
   • It is possible that the visualiser cli endpoint is not needed as will be exposed by document generator commands.

2. SchemaDirectory Duplication:

   • The SchemaDirectory functionality (responsible for resolving CALM references and obtaining additional related schema files) is duplicated across multiple submodules in shared. The Validator module uses AJV to load schemas. The Generator using the SchemaDirectory to load calm schemas from the file system leveraging document ids to make appropriate guess as to expected file. The Visualizer only supports a file upload currently and does not have any ref resolving and the Docify POC relies on some registering URL links to file locations as to where to resolve referenced documents.
   • We all agreed that the SchemaDirectory should be moved into its own module such that it can be leveraged by multiple modules in the future. The SchemaDirectory should abstract away from Validate and Generator functions about where the CALM references are resolved from e.g. CalmHub vs Local Directory
   • There was also discussion around the term bundling and whether we should remove from liturgy to avoid confusion with typescript bundling.

3. Introduce a New Doc Publisher Module under shared:

   • We discussed that to reduce dependency, the visualizer and generator should assume already validated documents have been published. We agreed we need to add a new Doc Publisher module.
   • This could be rolled out into two phases:-
     Phase 1: Implement endpoints for publishing CALM documents into the SchemaDirectory (See point1). This would be supported both from the CLI and CLI Server. This would allow for folks to leverage the generator and validator without needing a CALM Hub to get started e.g. they would publish documents first to CLI Server and then call validate on the top level architecture document.
     Phase 2: Extend functionality to support CALM Hub integration, enabling both registry publishing and CALM Hub interaction.

4. Define one CALM Model under shared:

   • Submodules like generate, validate, and visualize each define their own versions of the CALM model (flows, controls, patterns, etc.).
   • This inconsistency complicates interactions between modules and hinders collaboration.
   • General agreement is that we would consolidate on a typescript model. We would not use generator as discussed on Explore Options for CALM Schema to TypeScript Generation - Library Evaluation #701

Next Steps

• Review this discussion in the next working group meeting. Discuss when the work above should be prioritised. The General thought is that this would occur after Milestone 1
• Raise issue for detailed spec issue for the new Doc Publisher Module and CLI Server endpoints. (@Budlee, @Thels )
• Discussed use of AJV as unclear if plan is to continue to use

The following diagram as drawn to show module interaction during call (#not an official or completed diagram)

[LeighFinegold]

Discussed the model refactoring in more detail on #899 and concluded to maintain a central model and base it off classes rather than interfaces.