How do you feel about hammers?

[timholy]

Hi @Keno,

Over the holidays I got sucked back into Revise development because I wanted to fix timholy/Revise.jl#146. Since that involved a fairly significant redesign of all the path-handling, it triggered other thoughts and ambitions. The immediate prompt for this message is that I've come to the conclusion that I need to add a new class of "backedges." Currently Revise splits your package code up into top-level blocks and then, when triggered by a file watching event, identifies those top-level blocks with changes and then evals them. While this works pretty well in practice (because it incorporates changes without recompiling more than it needs to), it's unfortunately trivial to defeat with code like

b = false
if b
    f() = 1
else
    f() = 2
end

where editing the value of b (and then triggering a revise) does not result in f being commensurately redefined (because they are in separate top-level blocks, and each such block is treated as an independent entity). My assessment is the best way to handle this is to parse the CodeInfos produced by lowering, and compute a new class of backedges using the SSAValues. So Revise is about to get into the business of interpreting lowered code, which is relevant to this package.

Once the lid is off the can of lowered code, lots more could change, specifically integration and synergies between Revise, Rebugger, and ASTInterpreter2:

• ASTInterpreter2, at its lowest level as a pure lowered-code interpreter, can in theory handle anything because Julia already caches the lowered code. But its use as a debugger is greatly enhanced by having access the user-supplied source code. Unfortunately, it relies on line numbers, and these are almost useless if you're Revising your code without restarting your session (unless you want to recompile everything so that all the line numbers get updated). The pairing of a debugger with the ability to change the code and try again is really convenient, so I think it should be a major goal when this package next gets attention.
• Revise/Rebugger are good at keeping track of code in a relocatable manner, but quite bad, frankly, at digging it out in response to an arbitrary method signature. Currently only about 1/3 of all Base method signatures are successfully linked to their original source expressions---many of the failures are for conditional or loop-@evaled methods, which are not really parsed (because they are not top-level). While this doesn't hurt Revise's core mission, it significantly compromises Rebugger. I've made a halfway attempt to fix this in Major enhancements to parsing timholy/Revise.jl#230, but this is not fully satisfying. A more robust approach would be to assemble these links by "stepping through" the lowered code that defines the module, executing all the for-loops etc that generate code, but instead of evaling the method definitions we just extract the signature and link it to the original source expression. This strategy requires that one build a lowered-code interpreter, which ASTInterpreter2 already excels at.

So, my proposal is that we marry these: work towards the ability of ASTInterpreter2 to "follow" code with mutable line numbers (using Revise to look up a "line number correction" from the method signature), and conversely to use lowered-code interpretation to make Revise more correct (through backedges) and a better facilitator for debuggers (so that we can build all those links between method signatures and and their defining source text). Further on down the line, one could also imagine marrying Rebugger's "editable" REPL interface with ASTInterpreter's stepping abilities.

To do all this, we might benefit from some changes here, which is the main purpose of this message. In particular, I'm thinking it might be good to split out the "pure lowered-code interpreter" from all the higher-level functionality and REPL mode. There will also need to be some generalizations: currently it's assumed that you're executing the lowered code inside a method, but I'd need to add the ability to interpret top-level code. Hence the question, how do you feel about taking a hammer to this package and splitting it into chunks? Is this an area you'd be interested in collaborating on?

[Keno]

All the GUI parts are essentially already in https://github.com/Keno/DebuggerFramework.jl, so there may be less hammering required than you might imagine. One thing that you may be interested in and I've been thinking about for a really long time is that we need a better way to track source expressions through lowering, i.e. given a location (or a byte range of changes) in a file, identify all the lowered expressions that correspond to it. ASTInterpreter will need that to give more complete location information and it sounds like you may need it as well to actually identify how lowering mangles the source changes you're tracking. The good news of course is that https://github.com/ZacLN/CSTParser.jl already does half of that.

[timholy]

we need a better way to track source expressions through lowering, i.e. given a location (or a byte range of changes) in a file, identify all the lowered expressions that correspond to it

Agreed. While Revise doesn't currently use lowered code at all, it could probably fake this by splitting blocks of surface-AST and lowering them individually. It kind of already does the splitting part, see https://github.com/timholy/Revise.jl/blob/97d3560783176c6ec32b9e660aee51a8e333c455/src/parsing.jl#L38-L70. But it seems that it might make more sense to just build this in to lowering.

A closely-related thing I want are backedges for macros so that we can convert timholy/Revise.jl#20 from a wontfix to a willfix. For most macros you could guess from the line numbers, but that's a fragile way to do it. And some macros don't leave fingerprints.

All the GUI parts are essentially already in https://github.com/Keno/DebuggerFramework.jl, so there may be less hammering required than you might imagine.

Might want to consider making ASTInterpreter a dependency of DebuggerFramework rather than vice-versa, if we really want the GUI parts to be an independent layer?

[timholy]

Of course, another option is to move the core interpreter to base Julia. This is probably relevant for compile-time latency and JITing less code, no?

[jpsamaroo]

Doesn't Julia already have an interpreter, namely the one invoked with --compile=no?

[timholy]

Duh, of course you're right---the C version (src/interpreter.c) can't be replaced by pure Julia code, and there's no reason to have two.

Interestingly, though, on my local branch of this package (which now has quite a few performance optimizations) I can run a simple summation demo at nearly 3x the speed of running it under --compile=no. It may be mostly a question of this still being a less-comprehensive interpreter---cost is partly a function of how many branches you have. EDIT: more likely it's due to non-recursive interpretation. The demo I ran shouldn't go that "deep" but it still could result in less dynamic lookup.

[jpsamaroo]

I think this is reasonable behavior, given that (to my knowledge) the initial goal for the C interpreter was to handle code which can't yet be compiled, which I suppose is anything considered "toplevel". Since it only needs to run for a brief period, it can get by being a bit slower than a heavily optimized interpreter.

However, I don't think that we should end up choosing between one or the other; I think it would be wonderful if one could make use of ASTInterpreter2 as the "next tier" of interpretation, which would benefit both from being itself JIT-compilable (and thus maybe user-extensible by dispatch?), and additionally decoupled from Julia itself, providing opportunity to accrue all sorts of features.

Edit: And in relation to your OP: I personally agree that freeing this package from any GUI or debugging functionality, and just have it be a lowered code interpreter, would be great not just for Revise or Rebugger, but also for anyone else who might want this functionality for their own packages (myself included).