Useful features to consider

[edgardozoppi]

1. Add a flag to enable/disable an optimization that automatically detects ld r8, expr where expr evaluates to zero so it generates the more efficient xor r8 instruction. This allows you to be more declarative when writing code without sacrificing performance. For example in the following common scenario:

DEF rNR52 EQU $FF26
DEF rAUDENA EQU rNR52

DEF AUDENA_OFF   EQU %00000000

; turn audio off
ld a, AUDENA_OFF
ld [rAUDENA], a

To be translated to a more efficient code like this:

DEF rNR52 EQU $FF26
DEF rAUDENA EQU rNR52

DEF AUDENA_OFF   EQU %00000000

; turn audio off
xor a
ld [rAUDENA], a

2. Add an inline keyword or similar to allow inlining a function body at a specified call site easily without having to create a macro version of the same subroutine. So we can write the subroutine only once as usual while being able to choose at each individual call site if we are going to call it or inline it. Example:

DEF rLY EQU $FF44

wait_vblank:
  ld a, [rLY]
  cp 144
  jr nz, wait_vblank
  ret

...
call wait_vblank
...
; this will be expanded the same way a macro does
; properly renaming local labels and
; replacing rets by jumps to the next instruction
inline wait_vblank

[pinobatch]

1. How would automatic ld a, 0 to xor a optimization ensure that ld a, 0 wouldn't overwrite flags in contexts where they need to be preserved? Would it scan forward for another instruction that writes to flags before the next push af, rl, rr, adc, sbc, ccf, daa, or end of the basic block?
2. How would inline distinguish the end of a subroutine from an early return?

[edgardozoppi]

1. You are actually correct! Sorry I just started writing code for the Game Boy in assembly and I didn't realize that ld instruction does not modify the flags while xor does.
2. Not sure if I'm understanding your question correclty. I was thinking that maybe it can replace all rets reachable from the subroutine label with a jump to a new automatically generated label defined at the address of the next instruction in the inlined call site.

[Rangi42]

For 2, a macro will suffice:

MACRO inline_wait_vblank
.loop\@
    ld a, [rLY]
    cp 144
    jr nz, .loop\@
ENDM

wait_vblank:
    inline_wait_vblank
    ret

[evie-calico]

By the way, you can use ASSERT to show that xor is being used in place of a constant: ASSERT AUDENA_OFF == 0 \ xor a.

[Rangi42]

Or even:

MACRO lda0
    assert (\1) == 0
    xor a
ENDM

    lda0 AUDENA_OFF

[aaaaaa123456789]

This is a really bad idea for many reasons, including the fact that it is both obscure and redundant :P

The assertion is really the way to go here, if there's any possible confusion about whether the value will be zero or not. (In this case, being a hardware constant, I doubt it is necessary.)

[edgardozoppi]

The idea behind point 2 is not about not knowing the value of the constant, but more like a general idea where the developer (in this case myself) doesn't need to know or care about the actual constant value. Just use it and assign it to a register like will do with any other constant while relying in the assembler to still produce efficient code when possible. It's more about declarativity than anything else.

I find much more declarative to write and read ld a, AUDIO_OFF than xor a, even when we already know that xor a clears the register. We are loosing valuable information regarding the semantic meaning of zero constant in that particuar case (which I found it's pretty common like ld a, LCDCF_OFF which can also be optimized using xor a instead.

[ISSOtm]

I agree, but I think comments are more appropriate for that purpose: xor a ; ld a, LCDCF_OFF. And, for the purpose of documentation, I treat asserts as comments (just assembler-enforced ones, as an extra seatbelt!)

[aaaaaa123456789]

If you're just starting in assembly, it's probably a good idea to get into the mindset that assembly pretty much requires if you want to be successful at it.
Assembly gives you full control of your code. That means that the optimizer is you. There is no optimizer, and this is a feature, not a bug; you're expected to optimize your own code. The counterpart to this is that you can ruthlessly optimize your code without having to worry about running afoul of some compiler's rules (e.g., undefined behavior in many languages).
Sometimes you make code deliberately larger or slower: for instance, you could be writing a timing loop. The toolchain doesn't need to know, because the whole point about writing assembly is that you have full control of the code in your binary — for better or worse.

There's no rule that says that all callable blocks of code have to end with ret, or that it has to only appear once in a function. I've personally written code many times that violates every single rule mentioned in this thread so far, and for good reason: the code is optimized to do something else. There are no functions, just like there are no variables, control structures (if, while, etc.), objects, and so on. All of these higher-level constructions are native to higher-level languages (that's what makes them higher level), but in assembly, they arise semantically from the code you write. And one of the reasons to write assembly in the first place is that it allows you to build these constructions however you like, deviating from "the usual rules" when needed; that's why a toolchain cannot optimize your code for you: it would have to know things about your code that it doesn't, and requiring you to write code that obeys the constraints that those rules would imply would lose the main advantage of assembly.

Toolchains don't optimize your assembly code because you are expected to do it. Your code is as lean and fast as you make it, and no better or worse.

[edgardozoppi]

I understand that and agree with most of what you mentioned, that's why I suggested to add this kind of optimizations behind an assembler flag, the same way the already existing flags to optimize ld [$ff00 + n8], a with ldh [$ff00 + n8], a currently works. Another somehow similar example is the flag --nop-after-halt in the sense that generates different code in comparison to what the developer wrote.

[Rangi42]

We're getting rid of both of those flags; they're in the process of being deprecated now.

[ISSOtm]

Put another way, the choice to use assembly is that you're giving up the convenience of a compiler for increased control; RGBASM optimising your code automatically seems antithetical to that.

As far as implementation goes, this is made more difficult. The prime example of that is "auto-ldh", which can only be performed when RGBASM can compute the operand; this somewhat infamously leads to HRAM labels being always missed. (There are several internal reasons why this is the case, and unfortunately it's either that or another feature that has to give.)

Finally, a silver lining: while we don't think RGBASM itself should perform those changes, linters or other helper tools can point them out, and then you can apply them to your own code. What we, the RGBDS maintainers, can do, is help making those tools easier to write (think like an AST export; a pipe dream of mine...) and perhaps eventually bundling some with RGBDS itself, or at least highlighting some of them on our website.

[edgardozoppi]

That would be great to have. I can imagine myself writing some helper tools for the features I would like to have.

[edgardozoppi]

I have workarounded this like follows, but I would like more to be able to name the macro ld instead of ldz because its easy to forget and just write ld.

macro ldz
  if (\2) == 0
    xor \1
  else
    ld \1, \2
  endc
endm

ldz a, AUDENA_OFF
ld [rNR52], a

[calc84maniac]

Be careful with this macro, it doesn't act like you would expect if you did ldz b, AUDENA_OFF because xor b is actually shorthand for xor a, b and not xor b, b. This optimization only works on the A register.

[Rangi42]

I would like more to be able to name the macro ld instead of ldz because its easy to forget and just write ld.

No, you absolutely do not want ld to automatically turn into xor, because of the flag difference that was already discussed. Plus the issue of no longer being able to refer to the built-in ld; that macro would be infinitely recursive.

[edgardozoppi]

Makes a lot of sense both comments. Thanks a lot!

[edgardozoppi]

I can imagine that the following idea can also be considered a bad one, but I would like to know your opinion anyways. I learn from other points of views.

3. Add some flag to automatically detect unused "functions" so they are not included in the binary to save space. I think that maybe the linker knows if a particular memory address is reachable or not. Could be the target of a jump, call, etc. or by falling through previous instruction. But I also know that the target destination of a jump can be computed at run-time so it's not possible to detect those cases statically.
   I'm thinking in the scenario where the developer includes a library with a lot of useful functions defined but only use a few of them. What can be done to avoid including unused library code with the current version of the tools?

[Rangi42]

I agree that "function" isn't well-defined, but thought it was understood that conditional ret/jp/jr don't end one.

On the other hand, what about this:

Foo:
    call Bar
    jr z, _MoreFoo
    jp FinishFoo

Foo2:
    call Bar2
    jr nz, FinishFoo2
    ; fallthrough
_MoreFoo:
    call Spam
    ret

Where does Foo2 end?

[edgardozoppi]

I would say Foo2 finish at the ret of last line and also will have to see the code of FinishFoo2 for additional rets.

[Rangi42]

The point is, you can write a definite and explicit macro to do whatever you have in mind. The assembler doesn't need heuristics built in to inline chunks of code, that's exactly what macros are for.

[ISSOtm]

Further complicating this is that a ret doesn't necessarily end a function, but a jump may (if it's conceptually a tail call, then do you want to inline the tail'd function as well?)

.execSubFunc ; @param de: Pointer to the sub-function to jump to.
    push de
    ret

[Rangi42]

A contrived example:

MyFunction:
    push hl
    push de
    ld hl, $1234
    ld de, .continue
    push de
    ld a, [wMode]
    cp MODE_ONE
    ld de, DoOneToHL
    jr z, .got_de
    ld de, DoTwoToHL
.got_de
    push de
    ret
.continue
    call CleanupAfterOneOrTwo
    pop de
    pop hl
    ret