Text parser 2

Text parser 2

This is a sort of scrap book for the work on text-parser2.

Research approaches based on AST/TST

Research into using a universal AST (Abstract Syntax Tree) to represent a document. A complex language parser can provide a rich and complete AST while a simplistic parser can simply build a flat tree.

Example implementation of a patching AST parser for JavaScript

Current work and results are sometimes updated here: https://gist.github.com/764184

Outline:

• The text parser runs in the embedded nodejs runtime thread.
• Kod can send an "edited text" message to the parser at any given time.
• The parser can send a "tree updated" message to kod at any given time.

Informal protocol

Kod sending a "text edited" message to the parser system:

[kod -> node] "text edited" {
                document: [KDocument],
                modifiedRange: [123, 4],
                changeDelta: 4
              }

• document is an object which represents the document which text was altered. This document can be queried for:
  • language type
  • modification timestamp
  • complete source text

Ideas

Mathematica-style semantic expansion of the cursor

In Mathematica frontend, a user can press a key [Ctrl+.], and the token the cursor is on will be selected (highlighted), when user presses the key again, the selection expands to highlight the next smallest semantic unit. When the key is pressed again, it extends further.

-- http://xahlee.org/emacs/syntax_tree_walk.html