Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement mouse and monitor functionality for X11 & Mac OS #140

Merged
merged 8 commits into from
Sep 29, 2019

Conversation

drmfinlay
Copy link
Member

@drmfinlay drmfinlay commented Sep 24, 2019

Re: #8.

This PR implements mouse support for Linux and Mac OS via pynput.

I've moved all internal mouse event classes, flags and functions into modules under a new dragonfly/actions/mouse sub-package, similar to the keyboard sub-package. Windows functionality should be completely unaffected by these changes. From my testing, it seems to work just as it did before.

Mouse button and scroll events all work in X11. pynput doesn't seem to support scrolling or the "four"/"five" buttons on Mac OS. The "left", "right" and "middle" buttons seem to work properly though.

I won't merge this just yet because I want to get the Monitor class working on Linux so that relative mouse movements work correctly.

Mouse movements do not yet work properly on Mac OS because we don't have a Window class for that platform. That can wait for another day because I'm not very familiar with the Mac OS APIs.

- Add new 'dragonfly/actions/mouse' sub-package similar to the
  'keyboard' sub-package.
- Add pynput as a Linux requirement in setup.py.
- Move all internal mouse event classes, flags and functions into
  modules under the new 'mouse' sub-package.
- Add pynput mouse implementation for X11 (Linux) and Mac OS.
- Unmock the Mouse action.
@drmfinlay drmfinlay added Enhancement Enhancement existing feature Linux/X11 Issue regarding Linux or X11 support MacOS Issue regarding Apple PC OS labels Sep 24, 2019
@drmfinlay drmfinlay self-assigned this Sep 24, 2019
- Move most monitor code into BaseMonitor class.
- Put Windows code in Win32Monitor sub-class & confirm it works.
- Put X11 boilerplate code in place (not implemented yet).
- Adjust a few modules to import the monitor list and class for
  the current platform.
- Remove monitor-related mocking.
@drmfinlay
Copy link
Member Author

I've moved monitor code into some new modules, added boilerplate code for X11Monitor, removed mocking, adjusted imports and verified that the old Windows Monitor class still works perfectly (tested with two virtual monitors on Windows 7).

All I should have to do now is parse output from xrandr to get the info for X11. I think identifying connected monitors via the port ID should be good enough. For example, xrandr is showing my two monitors with input ports HDMI-1 and DP-1.

The monitors list will be updated whenever the rectangle is accessed. This is kind of hacky, but it works fine on Windows and xrandr seems to run quick enough that I doubt it will cause any noticeable latency.

@drmfinlay
Copy link
Member Author

Getting the list of monitors seems to be pretty easy with AppKit on Mac OS: https://stackoverflow.com/questions/3129322/how-do-i-get-monitor-resolution-in-python/3129567#3129567
So I'll add a Monitor class for Mac OS too while I'm at it.

NSScreen.screens() doesn't update on monitor changes though. Not sure why.

These functions should work with each Window implementation once
the Monitor classes are implemented.

This commit also adds docstrings for each method.
AppKit doesn't currently update monitor information after it is
retrieved, so the interpreter needs to be restarted after each
monitor change.
- Add pyobjc as a required package on Mac OS (for AppKit).
- Keep platform dependencies separate in setup.py.
@drmfinlay drmfinlay mentioned this pull request Sep 26, 2019
7 tasks
@drmfinlay drmfinlay changed the title Implement the mouse action, events and functions for X11 & Mac OS Implement mouse and monitor functionality for X11 & Mac OS Sep 27, 2019
@drmfinlay
Copy link
Member Author

Monitor classes for Mac OS and X11 are now implemented:

  • The Mac OS class requires the pyobjc package. I've added that to the dependencies for Mac OS. It can take a while to install.
  • The X11 monitor class shells out to xrandr, which we should be able to rely on.

Like the Windows class, both of these implementations put the primary monitor first on the list. This affects where the mouse will move when using absolute movements (e.g. Mouse("[0, 0]")).

The monitors list currently gets updated, but only if a monitor's rectangle is retrieved, which is pretty hacky (I take full blame for that). A better way would be to change monitors into an object that acts like a list and updates when accessed. Caching updates for a few seconds should make that efficient. I'll implement that before merging.

- Add FakeMonitor class and import it on unsupported platforms.
- Add Monitor.get_monitor() class method.
- Add read-only, self updating MonitorList class.
- Make the global 'monitors' list an instance of MonitorList.
- Replace update_monitors_list() with get_all_monitors() and
  implement it for each platform.
@drmfinlay
Copy link
Member Author

drmfinlay commented Sep 29, 2019

This is ready to be merged now. The monitors list is now an instance of a new MonitorList class. MonitorList does what I described above except for the caching, which doesn't really seem necessary. List updates can be avoided by taking a copy of the list: list(monitors).

There are two new class methods:

  • get_all_monitors(cls) — Gets a list of all connected monitors. Replaces update_monitors_list().
  • get_monitor(cls, handle, rectangle) — Gets a new or existing Monitor object depending on the handle. Updates the rectangle if reusing an object.

These are based on similar methods in Dragonfly's Window classes and have greatly simplified platform-specific code.

A FakeMonitor class has also been added to be consistent with the Window classes, rather than just using the base class.

@drmfinlay drmfinlay merged commit d88bd71 into master Sep 29, 2019
@drmfinlay drmfinlay deleted the feat/multiplatform-mouse branch September 29, 2019 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Enhancement existing feature Linux/X11 Issue regarding Linux or X11 support MacOS Issue regarding Apple PC OS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant