Implement successions of keys

[TeXitoi]

Implement an action that correspond to a succession of key, i.e. one keypress to write a sentence.

[gilescope]

that would be cool.

[riskable]

Frankly I'm surprised it doesn't already have this feature! Add me to the list of folks that want this primarily so I can have the keyboard send unicode characters (e.g. keydown:LCtrl-LShift,U,1,F,4,A,F,keyup:LCtrl-Lshift to send 💯). It needs to be able to keep a key held down while sending the keycodes for other keys somehow. The important part being I need to be able to hold some keys down, type a bunch of keys, then release the keys that were held.

[TeXitoi]

That's not implemented because I didn't need this feature, and, for now, only me has implemented new features in keyberon ;-)

@riskable That's a bit more complicated than planned, but the feature request is noted.

I don't think I'll do that short term. If any of you is interested to do it, I can guide you in the code to do the feature.

[riskable]

If you give me some tips I'll do my best and hopefully submit a PR! I never intended to, but I've become quite familiar with the way Keyberon works developing my hall effect firmware =)

[TeXitoi]

First, let's design the interface. Do you have any idea?

[riskable]

Well let me state that I wrote a tool (https://github.com/liftoff/HumanInput) for JavaScript that takes strings as input and converts them to keystrokes (that it listens for--as opposed to executing them) and its interface is super easy to use: ctrl-a n would listen for the key combo, ctrl-a followed by the n key. This might not make sense for Rust but it's where I'm coming from.

For sending keystrokes we need to be able to program separate keydown and keyup events and possibly specify how long each lasts. So a naive approach might be an underlying array of arrays of unicode characters (u32) might work. Here's how we might execute ctrl-a n:

[
    [LCtrl, A,],
    [N,]
]

...where each array in the array represents a keydown event. However, this doesn't specify how long each key gets held down for. So for something like that we may need an array of array of a custom struct or something like that. I'm still new to Rust so I'm not sure how it would work but maybe something like KeyEvent::new([LCtrl,A,], 100); (hold ctrl-a for 400ms).

[riskable]

Almost forgot: If possible, we also need a way to serialize the macro format as well so we can store it in a config somewhere (e.g. EEPROM). Not sure if we need to take that into consideration when designing the interface.

[TeXitoi]

But you can't model your Unicode typing example like this.

[riskable]

How about something that accumulates? Like:

event_delay = 10; // ms between keydown/keypress/keyrelease
ke = KeyEvent::new(event_delay);
ke.add_keydown(LCtrl);
ke.add_keydown(LShift);
ke.add_keydown(U);
ke.add_keypress(Kb1);
ke.add_keypress(F);
ke.add_keypress(Kb4);
ke.add_keypress(A);
ke.add_keypress(F);
ke.add_keyrelease(LCtrl); // Or maybe just add_keyrelease() with no args to release all keydowns at once
ke.add_keyrelease(LShift);
ke.add_keyrelease(U);

[riskable]

Just realized a better way would be to just chain the calls:

ke = KeyEvent::new()
    .keydown(LCtrl)
    .keydown(LShift)
    .keydown(U)
    .keypress(Kb2)
    .keypress(Kb6)
    .keypress(Kb3)
    .keypress(A)
    .keyrelease();

[TeXitoi]

You can't really have such a builder pattern as we don't have allocation, but we can have something like:

use KeyAction::*; // I don't like this name
const HUNDRED_POINTS = Action::Succession {
    delay: 10,
    succession: &[
        Press(LCtrl),
        Press(LShift),
        Press(U),
        Tap(Kb1),
        Tap(F),
        Tap(Kb4),
        Tap(A),
        Tap(F),
        Release(LCtrl),
        Release(LShift),
        Release(U),
    ],
};

We might add in the implementation a "release everything at the end".

[riskable]

That'll work! For a mod name how about, "KeyMacro" since that's basically what it is?

[riskable]

I started trying to implement this myself but I've run into an issue: The way the keydown State seems to get handled via keycode(&self) but a key release gets handled via a coordinate, release(&self, c: (u8, u8)). So without a specific location/coordinate there's no way to track the "release" of an artificially-generated keydown event.

The WaitingState also seems to rely on coord in order to function. There also seems to be assumptions elsewhere in the code that any given keydown/keyrelease event is always going to be tied to a specific key coordinate.

Any recommendations as to how to proceed without refactoring a ton of code in layout.rs (and potentially action.rs)?

[TeXitoi]

You don't have to care of WaitingState, in fact, the only thing you care is layout::State.

You have to add an enum to manage a KeyMacro (name seems fine). You also have a bit of refactoring to change the signature of State::keycode from returning a Option<KeyCode> to return a Vec<KeyCode, U8> (let's say we manage maximum 8 simultaneous keycode for the moment) and, maybe add an s to the method.

Then, let's just handle Press and Release event. We can easily add some simple macro to transform some_name!(K, E, Y) to Press(K), Release(K), Press(E), Release(E), Press(Y), Release(Y). Now, in State::tick, you have to advance your state, and in State::keycodes to return the current state. When the macro is finished, tick will return None, and it will disapear, closing all the remaining keycode that are pressed.

Hope it's clear.

[riskable]

Your hints are definitely helpful. I've never written a Rust macro before but it doesn't look like rocket science (more like "dollar sign science" hehe) but I'll try to figure it out. Thanks.

The plan is to get you a PR that doesn't make you do more work than you would have if you implemented this on your own 👍 . Reasoning about the code without an allocator is definitely a challenge for my brain (that was free-range raised on dynamic, interpreted language feed).

[TeXitoi]

I don't care if it takes more of my time than if I implement it myself. I don't need it for the moment, so I'm not so motivated to do it. But mentoring is important to me. Don't be afraid of PR with lots of comments ;-)

[riskable]

That's great, actually. If you're taking the time to get me up-to-speed I'll do my best to make it worth your while 💯

[riskable]

I've got it working! I wasn't able to figure out how to get it working with just states though. Basically, I couldn't figure out a way to differentiate between Press and Release events when trying to perform sequences where keys needed to be held while other keys were pressed then released. Here's how I set it up to define a sequence:

const MACROTEST: Action = Sequence {
    delay: 10, // Currently unused
    actions: &[
        SequenceEvent::Press(LCtrl), SequenceEvent::Press(LShift), SequenceEvent::Press(U), // Long form
        tap(Kb1), tap(F), tap(Kb4), tap(A), tap(F), // Short form
        kr(LCtrl), kr(LShift), kr(U), // Couldn't figure out how to get ReleaseAll() working--need some help there
    ],
};

I also made a short video demonstrating a Sequence in action using my pretend hall effect keyboard I have laid out on a breadboard:

https://youtu.be/M2OXrNP3fgI

I'll be submitting a PR for review in a moment. It'll probably require changes 😄