Write functions to give Dune_file.Stanzas.t from ASTs

[shonfeder]

The need for this came up in discussion around #1448 in #1448 (comment).

I expect we'll want a function Dune_File.Stanzas.of_ast, likely implemented on top of a Dune_File.Stanza.of_ast.

[shonfeder]

I would be happy to take this on, as it will help me prepare for expanding dune init to support updating the fields of existing stanzas.

[ghost]

Isn't it basically DUne_file.Stanzas.parse?

[shonfeder]

Dune_file.Stanzas.parse is a function from a file, a syntax, a project, and some additional AST objects to Stanzas.t:

dune/src/dune_file.mli

Lines 426 to 431 in d7274c2

	val parse
	: file:Path.t
	-> kind:Dune_lang.Syntax.t
	-> Dune_project.t
	-> Dune_lang.Ast.t list
	-> t

That is implemented on top of Dune_project.stanza_parser and Dune_lang.Decoder.parse, and neither of those seem to provide a ready means of converting an already existent Ast.t to a Stanzas.t either.

To recap the context of #1448 (comment): The use case is one where we have synthesized an Ast.t (or have one already parsed out from a file) and we want to inspect its constituent Stanza.ts for their semantic content without having to reinvent the parser.

After looking at the code a bit more, I suspect the desired function can be derived from a bit of refactoring of Dune_file.Stanzas.parse and it's recursive helper.

@diml Do you think the added context helps justify the need? Or, alternatively, do you have a preferred alternative for meeting the use case?

[ghost]

Thanks for the recap. To be clear, the file argument is only used to detect cycles, the Dune_lang.Ast.t list argument is the contents of the file that is parsed by parse. I guess parse could use a documentation comment. As long as you have a Dune_project.t value at hand, then you can use parse. You can always pass a dummy value for the file argument.

One more question though: are you planning to implement field updates using the following pipeline: source text -> Ast.t list -> Stanzas.t -> Ast.t list -> source text? I'm asking because the step Stanzas.t -> Ast.t list is not implemented yet. It's nothing complicated, though it is quite a bit of boilerplate.

[shonfeder]

Thanks for correcting my misunderstanding, @diml. This is a big help.

I actually added a documentation comment in #1448, but it is apparently in need of correction due to my misunderstanding. I'll fix that. While correcting the documentation, I'll move to make the file argument optional: this will help formalize its role.

[shonfeder]

One more question though: are you planning to implement field updates using the following pipeline: source text -> Ast.t list -> Stanzas.t -> Ast.t list -> source text? I'm asking because the step Stanzas.t -> Ast.t list is not implemented yet. It's nothing complicated, though it is quite a bit of boilerplate.

The plan for this isn't finalized, but it is complicated by the fact that we should be be able to update user-written dune files, which means we need to preserve comments. I am therefore anticipating working through a pipeline source text -> (Cst.t list * Stanzas.t) -> Cst.t list -> source text. Cst.t list * Stanzas.t will coordinate the concrete syntax tree with it's semantic information, allowing us to make all decision by inspecting Stanza.t objects (saving the need for ad hoc parsing), but letting us insert our updates into the Cst.t (allowing us to preserve the user's comments).

[ghost]

Ok, that should work. Though depending on what kind of edits you want to do, it might be just as easy to do them on the Ast directly without parsing it.

BTW, this is the pipeline I implemented in src/upgrader.ml to make dune upgrade preserve comments, it might apply here as well:

• extract the comments from the Cst.t list with Dune_lang.Cst.extract_comments
• convert the Cst.t list to an Ast.t list with Dune_lang.Cst.abstract
• process the Ast.t list
• convert the new Ast.t list to a Cst.t list with Dune_lang.Cst.concrete
• re-insert the comments with Dune_lang.insert_comments

[shonfeder]

Super helpful, @diml. Thank you!