Procedure for testing/comparing this branch vs others

[RedBearAK]

@joshgoebel

Readme says the installation instructions aren't correct yet. I just want to establish the best way to quickly go back and forth between this branch and (in my case) the Kinto branch.

When I was messing with the mainline branch all I had to do was sudo pip3 install --upgrade xkeysnail, which would pull down the normal xkeysnail and replace the patched Kinto version. Then I would just reinstall Kinto to revert back to the "held keys" patched version and get everything back to normal.

But, this is a destructive process. Not only does the Kinto install overwrite any customized config file with the default, but on my main system I'm running xkeysnail as user rather than root, and the Kinto installer would attempt to set up xkeysnail to run from the systemd service file that runs xkeysnail as root, which I would have to disable each time.

So I want to use this thread to record the commands needed to download and install this branch, and then easily download and reinstall the Kinto patched branch without re-running the entire Kinto installer. Right now I'm not exactly sure how to do that, but I should be able to pull the right command out of the setup.py file to pull the right branch down and install it.

Will add to this later.

[joshgoebel]

Checkout the source and then setup a python VENV:

git clone https://github.com/joshgoebel/xkeysnail.git
cd xkeysnail
python3 -m venv .venv
# you must activate the venv every time you open a new terminal
source .venv/bin/activate
pip3 install
./bin/xkeysnail -c path_to_config

Venvs build an isolated python environment. You can have lots of venvs and run 20 different versions of xkeysnail if you wanted...

To run the kinto config just point it's config.

source .venv/bin/activate
./bin/xkeysnail -c path_to_kinto_sh_config

You'd of course need to STOP the systemd service first.

I'm running xkeysnail as user

Then for tested I'd just run my version in a terminal window....

[RedBearAK]

@joshgoebel

This is exactly what I needed, and smarter than what I was thinking about doing, I'm sure.

Will give this a try as soon as I can.

The pip3 install near the end, just before running the actual xkeysnail command, what exactly is that doing?

And if I run this as a script like this, the git clone should automatically overwrite or update the folder with the latest code each time, right? Or is there a better/smarter way to do that repeatedly after the first clone?

#!/usr/bin/env bash

cd ~/Downloads
git clone https://github.com/joshgoebel/xkeysnail.git
cd xkeysnail
python3 -m venv .venv
# you must activate the venv every time you open a new terminal
source .venv/bin/activate
pip3 install
./bin/xkeysnail -c ~/.config/kinto/kinto.py

[joshgoebel]

The pip3 install near the end,

You need to install the dependencies into the python venv.

And if I run this as a script like this,

I wasn't suggesting you need to do ALL that over and over, you don't... each time minimally you might want something like:

git pull
source .venv/bin/activate
./bin/xkeysnail -c ~/.config/kinto/kinto.py

[RedBearAK]

@joshgoebel

available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

sudo apt install python3-venv brings in a couple of other packages too.

Error:

ERROR: You must give at least one requirement to install (see "pip help install")
Traceback (most recent call last):
  File "/home/kris/Downloads/xkeysnail/./bin/xkeysnail", line 5, in <module>
    from xkeysnail.cli import main
ModuleNotFoundError: No module named 'xkeysnail'

I tried adding a dot after the pip3 install and got further, but then:

Collecting six>=1.10.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Using legacy 'setup.py install' for xkeysnail, since package 'wheel' is not installed.
Using legacy 'setup.py install' for evdev, since package 'wheel' is not installed.
Using legacy 'setup.py install' for inotify_simple, since package 'wheel' is not installed.
Installing collected packages: evdev, appdirs, six, ordered_set, inotify_simple, python-xlib, xkeysnail
  Running setup.py install for evdev ... done
  Running setup.py install for inotify_simple ... done
  Running setup.py install for xkeysnail ... done
Successfully installed appdirs-1.4.4 evdev-1.5.0 inotify_simple-1.3.5 ordered_set-4.1.0 python-xlib-0.31 six-1.16.0 xkeysnail-0.4.99
keyszer v0.4.99
Traceback (most recent call last):
  File "/home/kris/Downloads/xkeysnail/./bin/xkeysnail", line 6, in <module>
    main()
  File "/home/kris/Downloads/xkeysnail/.venv/lib/python3.10/site-packages/xkeysnail/cli.py", line 71, in main
    if not has_access_to_uinput():
  File "/home/kris/Downloads/xkeysnail/.venv/lib/python3.10/site-packages/xkeysnail/cli.py", line 26, in has_access_to_uinput
    from xkeysnail.output import _uinput  # noqa: F401
  File "/home/kris/Downloads/xkeysnail/.venv/lib/python3.10/site-packages/xkeysnail/output.py", line 5, in <module>
    from .models.action import Action 
ModuleNotFoundError: No module named 'xkeysnail.models'

git pull
source .venv/bin/activate
./bin/xkeysnail -c ~/.config/kinto/kinto.py

Right, that's what I was looking for. git pull from inside the folder created by the clone.

[joshgoebel]

I'm a bit confused, do you have the models/action.py file? On my system:

❯ ./bin/xkeysnail
keyszer v0.4.99
(--) CONFIG: /home/jgoebel/.config/xkeysnail/config.py
(+K) Grabbing Apple, Inc Apple Keyboard (/dev/input/event3)
(--) Ready to process input.

[RedBearAK]

As far as I can tell:

ll xkeysnail/models 
total 15K
-rw-rw-r-- 1 kris kris  322 Jun  5 22:47 action.py
-rw-rw-r-- 1 kris kris 1.5K Jun  5 22:47 combo.py
-rw-rw-r-- 1 kris kris 2.7K Jun  5 22:47 modifier.py

[RedBearAK]

Is it supposed to be pip3 install .?

[joshgoebel]

I forget... but xkeysnail shouldn't be installed in the venv... it should be a link:

.venv/lib/python3.10/site-packages/xkeysnail.egg-link

[joshgoebel]

Maybe remove (and recreate) the venv and try again with pip install -e . ?

[joshgoebel]

I pushed the fix, but since you might end up hacking on things I think you still want -e... so that your testing is linked to the real-time git files, not installed into .venv as it a static library.

[RedBearAK]

Progress. It complained about a missing module (beepy), so now it's at least getting to the point where it's trying to use my config file. I removed the offending import line, then:

...
Successfully installed appdirs-1.4.4 evdev-1.5.0 inotify_simple-1.3.5 ordered_set-4.1.0 python-xlib-0.31 six-1.16.0 xkeysnail-0.4.99
keyszer v0.4.99
 
 
 
     ######################################  
    ######################################## 
   #### KINTO/XKEYSNAIL RUNNING NORMALLY ####
    ######################################## 
     ######################################  
 
 
 
Traceback (most recent call last):
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/./bin/xkeysnail", line 6, in <module>
    main()
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/cli.py", line 78, in main
    eval_config(args.config)
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/cli.py", line 15, in eval_config
    exec(compile(config_code, path, 'exec'), globals())
  File "/home/kris/.config/kinto/kinto.py", line 155, in <module>
    # - IBM
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/config_api.py", line 196, in define_conditional_modmap
    condition_fn = old_style_condition_to_fn(condition)
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/config_api.py", line 178, in old_style_condition_to_fn
    if len(signature(condition).parameters) == 1:
NameError: name 'signature' is not defined

This line 155 is not part of anything I've ever messed with, and I don't know what "signature" is referring to.

Pay no attention to the big KINTO/XKEYSNAIL RUNNING NORMALLY. I got tired of not having visually clear feedback in the log every time I restarted Kinto. So some time back I did this in the config:

import subprocess

# Insert blindingly obvious indication into log terminal that Kinto has restarted and is running successfully
subprocess.run('echo " \n \n "', shell=True)
subprocess.run('echo "     ######################################  "', shell=True)
subprocess.run('echo "    ######################################## "', shell=True)
subprocess.run('echo "   #### KINTO/XKEYSNAIL RUNNING NORMALLY ####"', shell=True)
subprocess.run('echo "    ######################################## "', shell=True)
subprocess.run('echo "     ######################################  "', shell=True)
subprocess.run('echo " \n \n "', shell=True)

[joshgoebel]

Pushed fix.

[RedBearAK]

Still getting the same error. Fix should be live already? git pull says "Already up to date."

[joshgoebel]

Sorry check again.

[RedBearAK]

Now it doesn't like the Combo function you had me import from key:

Traceback (most recent call last):
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/./bin/xkeysnail", line 6, in <module>
    main()
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/cli.py", line 78, in main
    eval_config(args.config)
  File "/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/cli.py", line 15, in eval_config
    exec(compile(config_code, path, 'exec'), globals())
  File "/home/kris/.config/kinto/kinto.py", line 612, in <module>
    K("RC-Shift-Left_Brace"):   K("C-Shift-Tab"),   # Go to prior tab (Left)
ImportError: cannot import name 'Combo' from 'xkeysnail.key' (/home/kris/Downloads/josh-xkeysnail/xkeysnail/xkeysnail/key.py)

I can remove that easily enough.

[joshgoebel]

Because now it's in .models.combo

[RedBearAK]

Heeyyyy:

Successfully installed xkeysnail-0.4.99
keyszer v0.4.99
 
 
      ########################################       
     ##########################################      
    ####                                    ####     
   ####   KINTO/XKEYSNAIL RUNNING NORMALLY   ####    
    ####                                    ####     
     ##########################################      
      ########################################       
 
 
(--) CONFIG: /home/kris/.config/kinto/kinto.py
(+K) Grabbing AT Translated Set 2 keyboard (/dev/input/event4)
(--) Ready to process input.

We be processing input 'n' stuff.

Okay, well, that started out OK, but then I opened a new Firefox window and made some new tabs and opened preferences with the macro and so on, then I tried to close a tab with Cmd+W and... Well, I'm not entirely sure what it did. I think it may have switched app windows? It seemed to close the window with two tabs and preferences open, but then I did Cmd+N to try to bring up a new window and the two tabs were still there, so that was very weird.

Things kind of went downhill from there and after a few more shortcuts it seemed to lock up, to the point that I had to use the mouse to close the terminal tab it was running from, to kill the process. It wouldn't respond to Cmd+dot (mapped to Ctrl+C in terminals) or the actual Ctrl+C.

It's late and I don't think I can do a coherent analysis of what's happening right now. At least it's at a point where it's definitely loading the config and working, to a point.

I'll take another look at it tomorrow. Thanks for all your guidance. I should be able to test any new updates rapidly and repeatedly now.

[RedBearAK]

OK, things go badly after I open a bunch of tabs, then try to do the preferences macro (Cmd+comma), which doesn't work until I release the Cmd key after opening all the tabs, then the macro works, but then when I do Cmd+W after that it appears to close the entire window with multiple tabs open, which is abnormal. Cmd+W should always just close a tab. And then some other stuff happened after that, like it opened the Firefox downloads window for some reason, which has a shortcut of Ctrl+Shift+Y that shouldn't have had anything to do with what I was doing.

The first failed macro attempt seems to be a critical point. I tried all the same operations with Kinto and had no trouble, as usual.

That's the best analysis I can do at the moment.

Here's the whole output from start to finish during that run, until I killed it with Cmd+dot.

keyszer-output-2022-06-06_000001.txt

EDIT: Removed pasted output, substitute text file.

[joshgoebel]

Ok lots of good stuff here, thanks. Seems I have a lot more testing to do. I couldn't reproduce your crash, but it could only be one thing... and I do see the strange behavior with preference, because shift is getting stuck down... I yhink I'm missing a few key pieces with multi-sequences...

If you can figure out explicitely how to reproduce the crash I'd love to know.

[joshgoebel]

Are you still constructing Combos manually, that's a potential issue as well as the internals are a bit different now.

[RedBearAK]

I disabled the Combo import and shortcuts even before the first successful run.

[joshgoebel]

Were you using marks for anything in all that?

[RedBearAK]

Marks?

[joshgoebel]

yeah set mark, etc... That crash is from a mismatch of mods/ordered_mods in the combo but I'm trying to figure out how you could put the system in such a state :)

[RedBearAK]

I haven’t messed with any code outside of my own Kinto config file, if that’s what you mean.

I’ll have to try again in the morning and see if it behaves better when I release the Cmd key after every shortcut. I really think it’s down to the macro that didn’t work the first time on each run, until I released the Cmd key.

Of course Cmd is physical Alt, logical Ctrl. Acer laptop.

[joshgoebel]

I'd entirely comment out _sticky = simple_sticky(input_combo, command) for now (in transform)... that will fix a lot of things. :)

[RedBearAK]

A number of other devs might be happy to tell you I have a talent for triggering bugs. Better be prepared. 😂

[RedBearAK]

Just did a new git pull this morning and ended up with nothing but exceptions for all shortcuts I tried. Couldn't even switch away from the terminal or open new tabs in the terminal.

Output:

keyszer-output-2022-06-06_112100.txt

Commented out the recommended _sticky line. No more exceptions and the macro works without releasing "Cmd" key, but of course this breaks Alt+Tab/Grave, limiting them to just switching between the same two windows.

[joshgoebel]

Can you upload your config? Does the test suite pass on your PC?

[joshgoebel]

15 subprocess.run('echo " \n \n "', shell=True)
16 subprocess.run('echo " ######################################## "', shell=True)
17 subprocess.run('echo " ########################################## "', shell=True)

Yikes, why don't you just use print in python?

[RedBearAK]

Yikes, why don't you just use print in python?

Um... I think because print wouldn't produce any output in the log window of the Kinto GUI app?

Which I actually haven't used in quite a while, at least on this particular machine, since it doesn't show anything useful after converting to running as user.

[joshgoebel]

Um... I think because print wouldn't produce any output in the log window of the Kinto GUI app?

Oh I dunno anything about that. I fixed the sticky bug.

[RedBearAK]

Will try it out when I get back.

[joshgoebel]

I wonder if I've broken my setup somehow.... now I always get an immediate repeat, even on the shortest of keypresses... hmmm... coming straight from the input device...

[joshgoebel]

Weird, replugging the keyboard helped.

[joshgoebel]

Would colored logging help? :) I'm spending a lot of time just trying to make the logs more readable to see what's happening.

[RedBearAK]

Colored output can be nice but of course it gets filtered out if you need to grep or something. Or you wind up with all the control characters when sending output to a file.

The formatting and symbols used in your output already seemed pretty clear. But the Kinto GUI log window would only show a few lines of the journal for the systemd service, and it was just never very clear which lines were before or after the restart.

Symbols and possibly some strategic indentation and/or blank lines are what I usually find most helpful.

[joshgoebel]

Yeah I removed some logging and adding blank line before combo triggers... I think it's a bit tighter now...

[joshgoebel]

We need a magic kill key for debugging I think... that's low-level... thoughts? Like you pick a key on the keyboard you never/rarely use and set that... and then (at the lowest input level) if we see that key we crash.

I went with F16, but you can hack the code to change it.

[RedBearAK]

That’s not going to be on the laptop keyboards I’m using. Does it have to be a single key?

Where is the line to change it, exactly?

[joshgoebel]

Does it have to be a single key?

Yes.

Where is the line to change it, exactly?

input.py ~166

[joshgoebel]

Well, I'm going to try running it all the time now, but man I'm suffering from Whiplash finally just starting to get used to Linux keys and now Mac keys work again, but my WM keys are all screwed up... what a fund challenge. :)

I think I need a hyper key, LOL.

[RedBearAK]

Did another pull and it seems to be working well this time. Can't replicate the issue with the failed macro. Holding down Cmd while switching between all sorts of shortcuts. New tabs, tab nav, Firefox prefs, closing tabs.

Looking good so far. Will continue playing with it later.

This has the WM_NAME support already?

[joshgoebel]

#2

No, would you like to add it? :)

[RedBearAK]

@joshgoebel

would you like to add it?

Ahem... "LOL". I mean, I'd love to, but it's most likely far beyond my ability at the moment.

So, one of the things that I do frequently in web browsers is Cmd+click links to open them in new background tabs. That does not appear to be working with your branch active. I'm holding Alt (Ctrl) like I usually do and it just opens the link in the same tab.

I suspect this is because you aren't remapping the modifier keys except when a shortcut is in use, unlike Kinto which moves logical Ctrl to physical Alt while it is active. As usual, I'm just guessing.

No matter what the cause, this is a problem for me. It's one of those second nature things that I do a lot.

Strangely, if what I was assuming was correct, it also doesn't work to hold the physical Ctrl key while clicking.

[joshgoebel]

So, one of the things that I do frequently in web browsers is Cmd+click links to open them in new background tabs.

What does the log say? Shortest length possible would be good. Try holding it longer (over a second).... does that fix it? You may be running into the suspension... that's definitely going to be an issue trying to fix keystrokes with mouse events that we don't have access to see.

Suspend means the keys won't register (on their own) unless you hold them a full second.

[joshgoebel]

You should join the Discord if that would be any easier.

[RedBearAK]

Output from start to trying to Ctrl+click a link. I held Ctrl for only a fraction of a second, which is how I frequently do it.

(--) CONFIG: /home/kris/.config/kinto/kinto.py
(+K) Grabbing AT Translated Set 2 keyboard (/dev/input/event4)
(--) Ready to process input.
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) suspending keys [<Key.RIGHT_CTRL: 97>]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) modmap: LEFT_ALT => RIGHT_CTRL [define_conditional_modmap (old API)]
(DD) resume because of mod release

You should join the Discord if that would be any easier.

Not a big fan of the more "chatty" type applications like Discord, but if that's easier for you...

[joshgoebel]

Suspend is how we fix #9... I'm not sure how I feel about reading all mouse events also just so we can capture mouse buttons, ugh... what a hard problem.

[RedBearAK]

Does appear to work if I hold for at least a second.

But that is not an expected behavior based on the way macOS or Kinto react. Not sure how well I could live with that if it's an essential feature of the way you're doing things.

[joshgoebel]

Read #9... I suppose we could start having flags - so people could pick and choose what type of broken they want, but I'd prefer to have no broken at all.

The problem is when you hit a modifier we have to wait until we know what you want to do with it (so we don't send the WRONG key to the output)... this works great with keys, but since the mouse is invisible we can't know you want to do a multi-key gesture using the mouse.

0.4.0 just sends the wrong keys to the output and then unpresses them later, but that's why it can trigger unintentional keypresses.

If we turn off suspend then #9 reasserts itself.

[joshgoebel]

but if that's easier for you...

It's just that I'm often there and it would allow for more real-time back and forth... up to you.

[RedBearAK]

Moved text to issue #16.

[joshgoebel]

Is this covered well enough by the readme now?

[RedBearAK]

Is this covered well enough by the readme now?

This?

https://github.com/joshgoebel/keyszer#for-testinghackingcontributing

That seems to cover it pretty well . Want to add a line for installing the pytest stuff? Could be helpful.

There’s also no note there about how the xkeysnail import line will cause a problem. That really needs a good solution if you want people to test with their existing Kinto config file.