From c91cb1029b34133dbb8b854c1ae709f7341a86fe Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Thu, 23 May 2024 17:22:55 +0200 Subject: [PATCH 01/12] TMP: Add draft of roadmap article --- content/blog/2024-05-23-linebender-roadmap.md | 191 ++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 content/blog/2024-05-23-linebender-roadmap.md diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md new file mode 100644 index 0000000..c822a9c --- /dev/null +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -0,0 +1,191 @@ ++++ +title = "Roadmap for the next three months of work on Linebender" +authors = ["Olivier Faure"] +date = "2024-05-23 18:00:00" ++++ + +On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL conference](https://2024.rustnl.org/) for two days, followed by two days of Unconference. + +By that point, we all felt fairly exhausted, but Raph Levien being the cruel taskmaster that he is, he still made us work one extra day, and so we came together in our hotel's conference room to discuss the Linebender project, and what we wanted from it. + +We'd initially thought of that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. + +What follows is a comprehensive summary of what exactly we're planning to do. We've tried to focus on blockers, where a missing feature in on project is stalling progress on other project, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. + +- **Priority:** +- **Assigned:** +- **Description:** +- **Issue:** + + +## Vello + +### Robust dynamic memory + +- **Priority:** Highest. +- **Assigned:** Daniel McNab. +- **Description:** Allocating dynamic memory in a robust way is essential to display large scenes without graphical artifacts. We have several possible solutions, we need to implement one. +- **Issue:** [vello#366](https://github.com/linebender/vello/issues/366) + +### Reliable snapshot tests + +- **Priority:** High. +- **Assigned:** Daniel McNab. +- **Description:** Snapshot tests are essential to test regressions in graphic software. Masonry's test suite heavily depends on them. But it's flaky right now because even when the fonts and inputs are exactly the same, renderers can have subtle differences that make the snapshots slightly different. We either need to create a fully deterministic renderer or add testing infrastructure that can tolerate small differences. [`nv-flip`](https://docs.rs/nv-flip/latest/nv_flip/) might have what we want. +- **Issue:** TODO + +### Sparse strips + +- **Priority:** High. +- **Assigned:** Raph Levien. +- **Description:** Sparse strips are a new research direction that might significantly improve our rendering pipeline and our memory footprint. We want to write a prototype implementation so we can confirm the performance benefits. +- **Issue:** TODO + +### Add tests + +- **Priority:** Medium. +- **Assigned:** Olivier Faure. +- **Description:** Adding tests is often a "we'll do it later" item, but it has short-term benefits as well. Adding tests would help Vello contributors check that their modifications don't add regressions to the crate, and help us catch hidden bugs. As a first step, we'd want to at least add some extremely basic tests, such as checking that Vello doesn't crash when we run a sample image through the CPU shaders. +- **Issue:** + + +## Masonry + +### Complete text input + +- **Priority:** Highest. +- **Assigned:** Olivier Faure, Aaron Muir Hamilton. +- **Description:** We'd like to bring up Masonry's textbox widget to be best-in-class among Rust frameworks. We want to support native-feeling interactions out of the box, such as cursor movement (go to next word, go to beginning of line), selection (select to end of text) and edition (IME, deleting an entire word, etc). This is something other framework developers have expressed interest in, so we want to implement it in a composable way. +- **Issue:** TODO + +### Better tracing for debugging + +- **Priority:** High. +- **Assigned:** Olivier Faure. +- **Description:** Masonry currently uses the `tracing` crate to produce logs usable for debugging. These logs could be improved, so that a user finding a bug can quickly narrow in on information relevant to the bug, without having to restart the app with changed log levels. The bulk of the improvement would be to implement a new subscriber with multiple layers. +- **Issue:** TODO + +### Fix glaring documentation problems + +- **Priority:** High. +- **Assigned:** Olivier Faure. +- **Description:** Masonry's documentation hasn't been updated since the port to Winit. While a lot of the code is still in flux, and thus hard to document, we should at least do a quick rewrite pass to remove references to Piet, Glazier, etc. +- **Issue:** TODO + +### Finish repository port + +- **Priority:** High. +- **Assigned:** Daniel McNab. (?) +- **Description:** We've ported the `masonry-rs` repository to the `linebender` organization, but that repository isn't where the crate's code lives anymore. We should update the README to reflect that information, port the issues to `linebender/xilem`, and archive `linebender/masonry`. +- **Issue:** TODO + +### Write "Widgets in Arena" RFC + +- **Priority:** High. +- **Assigned:** Olivier Faure. +- **Description:** This was something alluded to in the January blog posts. We want to move Masonry from storing widgets in WidgetPod to storing them in a slotmap-like arena. This is a large enough change to write an RFC about it. +- **Issue:** TODO + +### Write "Pass order" RFC + +- **Priority:** High. +- **Assigned:** Olivier Faure. +- **Description:** Right now widgets can use context methods in their different passes to request that other passes be ran. What passes can run what other passes isn't clearly documented, especially around edge cases. We should write an RFC to nail down the passes, how they can communicate, and the rationale around them. +- **Issue:** TODO + +### Rewrite documentation + +- **Priority:** Medium. +- **Assigned:** Olivier Faure. +- **Description:** Masonry's documentation hasn't been updated since the port to Winit. Once other major changes are implemented, the documentation should be brought up to date. We should avoid procrastinating on documentation too much just because the architecture is still in progress. +- **Issue:** TODO + +### Improve focus handling + +- **Priority:** Medium. +- **Assigned:** Olivier Faure. +- **Description:** We should improve tab focus in Masonry. Right now tab focus has some bugs, doesn't cover buttons, isn't updated by clicks, and uses architecture inherited from Druid that could be strongly simplified once other architectural changes are completed. +- **Issue:** TODO + +### Improve pointer status handling + +- **Priority:** Medium. +- **Assigned:** Olivier Faure. +- **Description:** We should centralize and improve how Masonry handles pointers. In particular, we should add an explicit "pointer capture" API that makes other widgets not get events from that pointer until the pointer is released or lost. +- **Issue:** [xilem#312](https://github.com/linebender/xilem/issues/312) + + +## Parley + +### Create abstractions for text selection and editing + +- **Priority:** Highest. +- **Assigned:** Chad Brokaw, Aaron Muir Hamilton. +- **Description:** We want Parley to implement types and methods that Masonry and other editors will be able to use to create a text-editing widget, or to handle selection in non-editable text. The initial implementation may be located in Masonry. +- **Issue:** [parley#52](https://github.com/linebender/parley/issues/52) + +### Inline block layout + +- **Priority:** High. +- **Assigned:** Chad Brokaw. +- **Description:** Parley should support laying out inline content the way web browsers do, where blocks are splices into your text and should be laid out according to available inline space the same way words are. +- **Issue:** TODO + +### Write adequate documentation + +- **Priority:** High. +- **Assigned:** Nico Burns. +- **Description:** Parley currently has virtually no documentation. If we want it to become a fundational crate, we need to explain to people how to use it. +- **Issue:** + +### Software renderer + +- **Priority:** Low. +- **Assigned:** Chad Brokaw. +- **Description:** Create a software renderer for the glyphs created by Parley, so that users can easily import Parley in their library without depending on a graphics API like Vello. The software renderer should be black-and-white for simplicity. We'll call for volunteers to implement COLRv1 and other advanced features. +- **Issue:** + + +## Xilem + +### Android bringup + +- **Priority:** Highest. +- **Assigned:** Daniel McNab. +- **Description:** We want Xilem to reliably work on Android. Running your Xilem app should be as simple as doing `cargo apk run` in your project. +- **PR:** [xilem#309](https://github.com/linebender/xilem/pull/309) + +### Update repository layout. + +- **Priority:** High. +- **Assigned:** Olivier Faure. +- **Description:** We've agreed on a layout for Linebender repository. We should move projects in the Xilem repo to match that layout. +- **PR:** [xilem#302](https://github.com/linebender/xilem/pull/302) + +### Rework xilem_core to be compatible with Masonry backend + +- **Priority:** High. +- **Assigned:** Daniel McNab. +- **Description:** The version of Xilem that was ported to Masonry used a different `View` trait that is incompatible with the current `xilem_core` crate. We need to rework `xilem_core` to get traits that can be used by both Masonry and `xilem_web`. Doing so while keeping readable code is non-trivial. +- **PR:** [xilem#235](https://github.com/linebender/xilem/pull/235), [xilem#310](https://github.com/linebender/xilem/pull/310) + +### Write Documentation + +- **Priority:** High. +- **Assigned:** ??? +- **Description:** Xilem's lack of documentation is one of the problems new users complain about the most often. We need to avoid staying in that state for too long. +- **Issue:** TODO. + +### Re-implement async integration + +- **Priority:** Medium. +- **Assigned:** Olivier Faure. +- **Description:** One of Xilem's selling point is the ability to handle virtual lists and async sources of data. That ability was temporarily removed for the Masonry port, but we'd like to add it back it. We don't want Masonry to be aware of async stuff, but the AppDriver trait should at least provide a method which lets Xilem access Masonry internals on its own initiative. +- **Issue:** TODO + +### Add unit tests + +- **Priority:** Low. +- **Assigned:** Olivier Faure. +- **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. +- **Issue:** TODO From e44d0bf3a146887cd5e9e0306299f4e05a0a0610 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Thu, 23 May 2024 17:45:10 +0200 Subject: [PATCH 02/12] Add long-term roadmap --- content/wiki/long-term-roadmap.md | 43 +++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 content/wiki/long-term-roadmap.md diff --git a/content/wiki/long-term-roadmap.md b/content/wiki/long-term-roadmap.md new file mode 100644 index 0000000..08c6784 --- /dev/null +++ b/content/wiki/long-term-roadmap.md @@ -0,0 +1,43 @@ +# Post-July Linebender roadmap + +## Vello + +* Emoji +* Image filters / blur / shadows + * Render graph (Raph to drive, see [thread](https://xi.zulipchat.com/#narrow/stream/197075-gpu/topic/Render.20Graph)) +* Gpu embedding / wgpu engine abstraction (see [Servo thread](https://xi.zulipchat.com/#narrow/stream/147922-new-members/topic/Greg.20from.20Servo)) +* Glyph caching + * Render graph?  + * or CPU side?  + * (sparse strips probably) +* Frame pacing +* Damage regions + * MVP: just render sub-rects +* CPU impl + * Testing + * Downlevel +* Fuzzing? + +## Masonry + +* ‘Compositing’ +* Frame pacing (relevant to Vello and AccessKit) +* Damage regions +* Styling options +* Copy features from [Tomorrow Corporation Tech Demo](https://www.youtube.com/watch?v=72y2EC5fkcE) + +## Parley + +* IME support + * Suggestion regions? +* Split out blob & other vocabulary types + * Blob may be prioritized by other project needs + +## Xilem + +* Android + * Material UI style widgets/views? Requires styling +* Styling +* Hot reloading +* Async behavior in Masonry widgets (image loading etc) +* Cleanup run for issues and PRs From 75692ccaa048ba077640a83b3a88afd6a3627cf0 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Wed, 12 Jun 2024 19:15:11 +0200 Subject: [PATCH 03/12] Add links to roadmap items --- content/blog/2024-05-23-linebender-roadmap.md | 45 ++++++++++--------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index c822a9c..08a3b6b 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -32,22 +32,21 @@ What follows is a comprehensive summary of what exactly we're planning to do. We - **Priority:** High. - **Assigned:** Daniel McNab. - **Description:** Snapshot tests are essential to test regressions in graphic software. Masonry's test suite heavily depends on them. But it's flaky right now because even when the fonts and inputs are exactly the same, renderers can have subtle differences that make the snapshots slightly different. We either need to create a fully deterministic renderer or add testing infrastructure that can tolerate small differences. [`nv-flip`](https://docs.rs/nv-flip/latest/nv_flip/) might have what we want. -- **Issue:** TODO +- **Issue:** [vello#608](https://github.com/linebender/vello/issues/608) ### Sparse strips - **Priority:** High. - **Assigned:** Raph Levien. - **Description:** Sparse strips are a new research direction that might significantly improve our rendering pipeline and our memory footprint. We want to write a prototype implementation so we can confirm the performance benefits. -- **Issue:** TODO +- **Zulip thread:** [Sparse strip path rendering](https://xi.zulipchat.com/#narrow/stream/197075-gpu/topic/Sparse.20strip.20path.20rendering) ### Add tests - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** Adding tests is often a "we'll do it later" item, but it has short-term benefits as well. Adding tests would help Vello contributors check that their modifications don't add regressions to the crate, and help us catch hidden bugs. As a first step, we'd want to at least add some extremely basic tests, such as checking that Vello doesn't crash when we run a sample image through the CPU shaders. -- **Issue:** - +- **PR:** [vello#439]'https://github.com/linebender/vello/pull/439' ## Masonry @@ -56,56 +55,56 @@ What follows is a comprehensive summary of what exactly we're planning to do. We - **Priority:** Highest. - **Assigned:** Olivier Faure, Aaron Muir Hamilton. - **Description:** We'd like to bring up Masonry's textbox widget to be best-in-class among Rust frameworks. We want to support native-feeling interactions out of the box, such as cursor movement (go to next word, go to beginning of line), selection (select to end of text) and edition (IME, deleting an entire word, etc). This is something other framework developers have expressed interest in, so we want to implement it in a composable way. -- **Issue:** TODO +- **Issue:** [xilem#388](https://github.com/linebender/xilem/issues/388) ### Better tracing for debugging - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Masonry currently uses the `tracing` crate to produce logs usable for debugging. These logs could be improved, so that a user finding a bug can quickly narrow in on information relevant to the bug, without having to restart the app with changed log levels. The bulk of the improvement would be to implement a new subscriber with multiple layers. -- **Issue:** TODO +- **Issue:** [xilem#250](https://github.com/linebender/xilem/issues/250) ### Fix glaring documentation problems - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Masonry's documentation hasn't been updated since the port to Winit. While a lot of the code is still in flux, and thus hard to document, we should at least do a quick rewrite pass to remove references to Piet, Glazier, etc. -- **Issue:** TODO +- **PR:** [xilem#329](https://github.com/linebender/xilem/pull/329) ### Finish repository port - **Priority:** High. - **Assigned:** Daniel McNab. (?) - **Description:** We've ported the `masonry-rs` repository to the `linebender` organization, but that repository isn't where the crate's code lives anymore. We should update the README to reflect that information, port the issues to `linebender/xilem`, and archive `linebender/masonry`. -- **Issue:** TODO +- **Zulip thread:** [Finishing the Masonry repository transition](https://xi.zulipchat.com/#narrow/stream/317477-masonry/topic/.E2.9C.94.20Finishing.20the.20Masonry.20repository.20transition) ### Write "Widgets in Arena" RFC - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** This was something alluded to in the January blog posts. We want to move Masonry from storing widgets in WidgetPod to storing them in a slotmap-like arena. This is a large enough change to write an RFC about it. -- **Issue:** TODO +- **PR:** [rfcs#6](https://github.com/linebender/rfcs/pull/6) ### Write "Pass order" RFC - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Right now widgets can use context methods in their different passes to request that other passes be ran. What passes can run what other passes isn't clearly documented, especially around edge cases. We should write an RFC to nail down the passes, how they can communicate, and the rationale around them. -- **Issue:** TODO +- **PR:** TBD. ### Rewrite documentation - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** Masonry's documentation hasn't been updated since the port to Winit. Once other major changes are implemented, the documentation should be brought up to date. We should avoid procrastinating on documentation too much just because the architecture is still in progress. -- **Issue:** TODO +- **Issue:** [xilem#389](https://github.com/linebender/xilem/issues/389) ### Improve focus handling - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** We should improve tab focus in Masonry. Right now tab focus has some bugs, doesn't cover buttons, isn't updated by clicks, and uses architecture inherited from Druid that could be strongly simplified once other architectural changes are completed. -- **Issue:** TODO +- **Issue:** [xilem#390](https://github.com/linebender/xilem/issues/390) ### Improve pointer status handling @@ -129,21 +128,22 @@ What follows is a comprehensive summary of what exactly we're planning to do. We - **Priority:** High. - **Assigned:** Chad Brokaw. - **Description:** Parley should support laying out inline content the way web browsers do, where blocks are splices into your text and should be laid out according to available inline space the same way words are. -- **Issue:** TODO +- **Issue:** [parley#25](https://github.com/linebender/parley/issues/25) +- **PRs:** [parley#67](https://github.com/linebender/parley/pull/67), [parley#76](https://github.com/linebender/parley/pull/76) ### Write adequate documentation - **Priority:** High. - **Assigned:** Nico Burns. - **Description:** Parley currently has virtually no documentation. If we want it to become a fundational crate, we need to explain to people how to use it. -- **Issue:** +- **PR:** [parley#26](https://github.com/linebender/parley/pull/26) ### Software renderer - **Priority:** Low. - **Assigned:** Chad Brokaw. - **Description:** Create a software renderer for the glyphs created by Parley, so that users can easily import Parley in their library without depending on a graphics API like Vello. The software renderer should be black-and-white for simplicity. We'll call for volunteers to implement COLRv1 and other advanced features. -- **Issue:** +- **PRs:** [parley#54](https://github.com/linebender/parley/pull/54), [parley#55](https://github.com/linebender/parley/pull/55) ## Xilem @@ -152,14 +152,14 @@ What follows is a comprehensive summary of what exactly we're planning to do. We - **Priority:** Highest. - **Assigned:** Daniel McNab. -- **Description:** We want Xilem to reliably work on Android. Running your Xilem app should be as simple as doing `cargo apk run` in your project. +- **Description:** We want Xilem to reliably work on Android. Running your Xilem app should require a small number of steps. Ideally, it should be as simple as running `cargo some-command` in your project. - **PR:** [xilem#309](https://github.com/linebender/xilem/pull/309) ### Update repository layout. - **Priority:** High. - **Assigned:** Olivier Faure. -- **Description:** We've agreed on a layout for Linebender repository. We should move projects in the Xilem repo to match that layout. +- **Description:** We've agreed on a layout for Linebender repositories. We should move projects in the Xilem repo to match that layout. - **PR:** [xilem#302](https://github.com/linebender/xilem/pull/302) ### Rework xilem_core to be compatible with Masonry backend @@ -172,20 +172,21 @@ What follows is a comprehensive summary of what exactly we're planning to do. We ### Write Documentation - **Priority:** High. -- **Assigned:** ??? +- **Assigned:** Olivier Faure. - **Description:** Xilem's lack of documentation is one of the problems new users complain about the most often. We need to avoid staying in that state for too long. -- **Issue:** TODO. +- **Issue:** [xilem#392](https://github.com/linebender/xilem/issues/392) ### Re-implement async integration - **Priority:** Medium. -- **Assigned:** Olivier Faure. +- **Assigned:** Daniel McNab. - **Description:** One of Xilem's selling point is the ability to handle virtual lists and async sources of data. That ability was temporarily removed for the Masonry port, but we'd like to add it back it. We don't want Masonry to be aware of async stuff, but the AppDriver trait should at least provide a method which lets Xilem access Masonry internals on its own initiative. -- **Issue:** TODO +- **Issue:** [xilem#340](https://github.com/linebender/xilem/issues/340) ### Add unit tests - **Priority:** Low. - **Assigned:** Olivier Faure. - **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. -- **Issue:** TODO +- **Issue:** [xilem#393](https://github.com/linebender/xilem/issues/393) + From dfb66779f7747d63c26bc370bc7cbd44535c118a Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Wed, 12 Jun 2024 19:15:25 +0200 Subject: [PATCH 04/12] Complete roadmap draft --- content/blog/2024-05-23-linebender-roadmap.md | 67 +++++++++++++++++-- content/wiki/long-term-roadmap.md | 46 +++++++++++++ 2 files changed, 107 insertions(+), 6 deletions(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index 08a3b6b..4182bea 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -10,12 +10,8 @@ By that point, we all felt fairly exhausted, but Raph Levien being the cruel tas We'd initially thought of that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. -What follows is a comprehensive summary of what exactly we're planning to do. We've tried to focus on blockers, where a missing feature in on project is stalling progress on other project, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. - -- **Priority:** -- **Assigned:** -- **Description:** -- **Issue:** +What follows is a comprehensive summary of what exactly we're planning to do. +We've tried to focus on blockers, where a missing feature in one project is stalling progress on other projects, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. ## Vello @@ -190,3 +186,62 @@ What follows is a comprehensive summary of what exactly we're planning to do. We - **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. - **Issue:** [xilem#393](https://github.com/linebender/xilem/issues/393) +### Add basic benchmarks + +- **Priority:** Low. +- **Assigned:** Olivier. +- **Description:** Right now we're mostly working off vibes to know whether or not any of our work improved performance or introduced regressions. +We should write at least *one* benchmark that would test a basic Xilem use-case, so we get a general idea how our performance is progressing. +- **Issue:** [xilem#362](https://github.com/linebender/xilem/issues/362) + + +## AccessKit + +Matt Campbell's contract with Google hasn't started yet. +When it does, top priority will likely be the Android backend. + +The plan is to study existing Android accessibility implementations for non-Java toolkits, including Flutter, Chromium, and Gio, to get a better understanding of what's required to implement Android accessibility from scratch. + +Beyond Android support, we've only had cursory discussions of what work Matt will do for the rest of the year. +Possible themes include scrollable containers, virtualized lists, richer text support (e.g. exposing font and style attributes) across platforms, and better integration into Xilem/Masonry. + + +## Android-specific work + +Getting the Android port right involves a lot of small changes and making things feel non-broken. + +Some notable sub-goals: + +### Subclassing View/Application/Activity + +- **Priority:** Highest. +- **Assigned:** Aaron Muir Hamilton, Raph Levien. +- **Description:** The Java classes of an Android application are where the rubber meets the road. We're investigating what we want our boilerplate to look like. + +### Handling IME + +- **Priority:** Highest. +- **Assigned:** Aaron Muir Hamilton, Raph Levien. +- **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without IME is a non-starter. Our design choices here will have influence on accessibility as well. + +### Gesture recognition + +- **Priority:** Low. +- **Assigned:** Aaron Muir Hamilton. +- **Description:** When the user swipes on the screen, we want to send some kind of scrolling events instead of a normal pointer event. Ideally there should be first-class support for that event to handle momentum scrolling, but a dumber solution would just emulate mouse wheel ticks. + + +## Conclusion + +Whew! We have our work cut out for us there. + +Overall, I'm feeling very optimistic about the coming months. +In the weeks since we first drafted this roadmap, we've already made a lot of progress on our respective tasks (for instance Daniel is now running on what async experts call a "work-stealing" algorithm), so it does look like we'll meet our deadlines. + +What comes after that is a bit hazier. +Our three areas of focus are likely to be Android support, accessibility, and developer experience. + +As we fill out gaps in our feature set, Xilem is moving closer to becoming a stable, mature framework for writing GUI applications. +We're not GUI yet, but we hope we'll be in the near future. + +In the meantime, stay tuned! diff --git a/content/wiki/long-term-roadmap.md b/content/wiki/long-term-roadmap.md index 08c6784..e7f1ccc 100644 --- a/content/wiki/long-term-roadmap.md +++ b/content/wiki/long-term-roadmap.md @@ -1,5 +1,7 @@ # Post-July Linebender roadmap +Unstructured notes from our last roadmap brainstorming that were left out of the formal roadmap blog post. + ## Vello * Emoji @@ -41,3 +43,47 @@ * Hot reloading * Async behavior in Masonry widgets (image loading etc) * Cleanup run for issues and PRs + +## AccessKit + +* Haven’t talked about it a lot + * Improve documentation (see [#402](https://github.com/AccessKit/accesskit/issues/402)) + * Point to / create a repository of accessibility best practices. + * Not just code, also social: Have people with disabilities test your stuff! + * Point to resources, forums, etc? Need to talk to Matt about this. (What were the best practices at Microsoft?) + * Accessibility isn’t just screen readers. Also color contrast, customizing size, IME support, motor disabilities (avoid stuff that requires simultaneous input). + +* “Accessibility is more than adding a dependency to AccessKit” + * (Olivier: Note to self: include GMTK video about Celeste Assist Mode) + +## Android specific work +* Need to prioritize Android + * Masonry: + * Hot state for pointer pressed? + * Gesture recognition (scrolling) P3 + * There is a dumb solution for scrolling without momentum + * Potentially adapt code from egui (used in vello demo) + * Pencil gestures + * Multitouch + * IME + * Landing the Lucas Meurer winit PR P1 + * Extend winit IME PR for richer functionality + * Review glazier IME API & come up with design proposal P3 (aaron) + * Plumbing suggestion spans etc (winit) + * Accessibility (matt, jul 1+) P-1 + * Study Tao + * To what extent do we tap into Android View etc + * winit future + * short term plan: PR functionality into winit + * look for natural extension points + * these will be platform-specific + * Mac: subclassing the NSView/NSApplication subclasses + * wayland: messy / smithay has message dispatcher + * Android: do we subclass View/Application/Activity? raph will investigate P1 + * Accessibility (matt, jul 1+) + * Can we access accessibility from native-activity / game-activity? + * Note: React Native requires writing Java-land activity + * What minimum platform version? + * Look at Makepad’s implementation (momentum scrolling etc) + + From 243c94aff59580aea1ba5b43a6e685cfa1def4a3 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Wed, 12 Jun 2024 19:16:34 +0200 Subject: [PATCH 05/12] Change roadmap title --- content/blog/2024-05-23-linebender-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index 4182bea..bdd57cd 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -1,5 +1,5 @@ +++ -title = "Roadmap for the next three months of work on Linebender" +title = "May-July roadamp for Linebender" authors = ["Olivier Faure"] date = "2024-05-23 18:00:00" +++ From 4452f909536e7c1eb6a5e8cbd005df01bae64d5a Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Wed, 12 Jun 2024 19:18:05 +0200 Subject: [PATCH 06/12] Fix small typos --- content/blog/2024-05-23-linebender-roadmap.md | 2 +- content/wiki/long-term-roadmap.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index bdd57cd..6a40382 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -1,5 +1,5 @@ +++ -title = "May-July roadamp for Linebender" +title = "May-July roadmap for Linebender" authors = ["Olivier Faure"] date = "2024-05-23 18:00:00" +++ diff --git a/content/wiki/long-term-roadmap.md b/content/wiki/long-term-roadmap.md index e7f1ccc..52e2d6a 100644 --- a/content/wiki/long-term-roadmap.md +++ b/content/wiki/long-term-roadmap.md @@ -1,4 +1,6 @@ -# Post-July Linebender roadmap ++++ +title = "Post-July Linebender roadmap" ++++ Unstructured notes from our last roadmap brainstorming that were left out of the formal roadmap blog post. From ec3854ec32feff10bdc4a065f49500a0d34994ef Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Wed, 12 Jun 2024 19:32:36 +0200 Subject: [PATCH 07/12] Small tweaks to roadmap --- content/blog/2024-05-23-linebender-roadmap.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index 6a40382..39bd5dd 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -42,7 +42,8 @@ We've tried to focus on blockers, where a missing feature in one project is stal - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** Adding tests is often a "we'll do it later" item, but it has short-term benefits as well. Adding tests would help Vello contributors check that their modifications don't add regressions to the crate, and help us catch hidden bugs. As a first step, we'd want to at least add some extremely basic tests, such as checking that Vello doesn't crash when we run a sample image through the CPU shaders. -- **PR:** [vello#439]'https://github.com/linebender/vello/pull/439' +- **PR:** [vello#439](https://github.com/linebender/vello/pull/439) + ## Masonry @@ -156,7 +157,7 @@ We've tried to focus on blockers, where a missing feature in one project is stal - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** We've agreed on a layout for Linebender repositories. We should move projects in the Xilem repo to match that layout. -- **PR:** [xilem#302](https://github.com/linebender/xilem/pull/302) +- **PRs:** [xilem#302](https://github.com/linebender/xilem/pull/302), [vello#590](https://github.com/linebender/vello/pull/590) ### Rework xilem_core to be compatible with Masonry backend From 9359d727db81f3cc4ca7b834b46f49b523102f60 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Thu, 13 Jun 2024 10:36:58 +0200 Subject: [PATCH 08/12] Include changes from code review --- content/blog/2024-05-23-linebender-roadmap.md | 16 +++++++++------- content/wiki/long-term-roadmap.md | 4 +++- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-linebender-roadmap.md index 39bd5dd..5817f7a 100644 --- a/content/blog/2024-05-23-linebender-roadmap.md +++ b/content/blog/2024-05-23-linebender-roadmap.md @@ -1,7 +1,6 @@ +++ title = "May-July roadmap for Linebender" authors = ["Olivier Faure"] -date = "2024-05-23 18:00:00" +++ On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL conference](https://2024.rustnl.org/) for two days, followed by two days of Unconference. @@ -13,6 +12,9 @@ We'd initially thought of that this day would be an informal day of unstructured What follows is a comprehensive summary of what exactly we're planning to do. We've tried to focus on blockers, where a missing feature in one project is stalling progress on other projects, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. +This is a snapshot of the project status at the time we're writing this. +It includes some work we already finished since RustNL, but will not be updated when further work finishes. + ## Vello @@ -51,7 +53,7 @@ We've tried to focus on blockers, where a missing feature in one project is stal - **Priority:** Highest. - **Assigned:** Olivier Faure, Aaron Muir Hamilton. -- **Description:** We'd like to bring up Masonry's textbox widget to be best-in-class among Rust frameworks. We want to support native-feeling interactions out of the box, such as cursor movement (go to next word, go to beginning of line), selection (select to end of text) and edition (IME, deleting an entire word, etc). This is something other framework developers have expressed interest in, so we want to implement it in a composable way. +- **Description:** We'd like to bring up Masonry's textbox widget to be best-in-class among Rust frameworks. We want to support native-feeling interactions out of the box, such as cursor movement (go to next word, go to beginning of line), selection (select to end of text) and editing (IME, deleting an entire word, etc). This is something other framework developers have expressed interest in, so we want to implement it in a composable way. - **Issue:** [xilem#388](https://github.com/linebender/xilem/issues/388) ### Better tracing for debugging @@ -71,7 +73,7 @@ We've tried to focus on blockers, where a missing feature in one project is stal ### Finish repository port - **Priority:** High. -- **Assigned:** Daniel McNab. (?) +- **Assigned:** Daniel McNab. - **Description:** We've ported the `masonry-rs` repository to the `linebender` organization, but that repository isn't where the crate's code lives anymore. We should update the README to reflect that information, port the issues to `linebender/xilem`, and archive `linebender/masonry`. - **Zulip thread:** [Finishing the Masonry repository transition](https://xi.zulipchat.com/#narrow/stream/317477-masonry/topic/.E2.9C.94.20Finishing.20the.20Masonry.20repository.20transition) @@ -184,13 +186,13 @@ We've tried to focus on blockers, where a missing feature in one project is stal - **Priority:** Low. - **Assigned:** Olivier Faure. -- **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. +- **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. At some point we should hopefully switch to a "new test for every new feature" workflow. - **Issue:** [xilem#393](https://github.com/linebender/xilem/issues/393) ### Add basic benchmarks - **Priority:** Low. -- **Assigned:** Olivier. +- **Assigned:** Olivier Faure. - **Description:** Right now we're mostly working off vibes to know whether or not any of our work improved performance or introduced regressions. We should write at least *one* benchmark that would test a basic Xilem use-case, so we get a general idea how our performance is progressing. - **Issue:** [xilem#362](https://github.com/linebender/xilem/issues/362) @@ -223,7 +225,7 @@ Some notable sub-goals: - **Priority:** Highest. - **Assigned:** Aaron Muir Hamilton, Raph Levien. -- **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without IME is a non-starter. Our design choices here will have influence on accessibility as well. +- **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without support for virtual keyboard and other input methods is a non-starter. Our design choices here will have influence on accessibility as well. ### Gesture recognition @@ -243,6 +245,6 @@ What comes after that is a bit hazier. Our three areas of focus are likely to be Android support, accessibility, and developer experience. As we fill out gaps in our feature set, Xilem is moving closer to becoming a stable, mature framework for writing GUI applications. -We're not GUI yet, but we hope we'll be in the near future. +We're not GUI yet, but we certainly hope to be one day. In the meantime, stay tuned! diff --git a/content/wiki/long-term-roadmap.md b/content/wiki/long-term-roadmap.md index 52e2d6a..901aea6 100644 --- a/content/wiki/long-term-roadmap.md +++ b/content/wiki/long-term-roadmap.md @@ -2,7 +2,9 @@ title = "Post-July Linebender roadmap" +++ -Unstructured notes from our last roadmap brainstorming that were left out of the formal roadmap blog post. +Unstructured notes from the RustNL roadmap brainstorming that were left out of the May-July roadmap blog post. + +This may become a living document, eg a long-term roadmap for the project. For now it's the post-July roadmap. ## Vello From 7285d5f451c05efd49a1ee42cb8664a4a9def5e2 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Thu, 13 Jun 2024 10:37:34 +0200 Subject: [PATCH 09/12] Rename roadmap article --- ...05-23-linebender-roadmap.md => 2024-05-23-roadmap-may-2024.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename content/blog/{2024-05-23-linebender-roadmap.md => 2024-05-23-roadmap-may-2024.md} (100%) diff --git a/content/blog/2024-05-23-linebender-roadmap.md b/content/blog/2024-05-23-roadmap-may-2024.md similarity index 100% rename from content/blog/2024-05-23-linebender-roadmap.md rename to content/blog/2024-05-23-roadmap-may-2024.md From 631c6ca6805ba22c252c43af82f33c3e5aa80a9d Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Sat, 15 Jun 2024 13:20:08 +0200 Subject: [PATCH 10/12] Wording changes in roadmap article to clarify intent --- content/blog/2024-05-23-roadmap-may-2024.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/blog/2024-05-23-roadmap-may-2024.md b/content/blog/2024-05-23-roadmap-may-2024.md index 5817f7a..f8ca014 100644 --- a/content/blog/2024-05-23-roadmap-may-2024.md +++ b/content/blog/2024-05-23-roadmap-may-2024.md @@ -5,7 +5,7 @@ authors = ["Olivier Faure"] On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL conference](https://2024.rustnl.org/) for two days, followed by two days of Unconference. -By that point, we all felt fairly exhausted, but Raph Levien being the cruel taskmaster that he is, he still made us work one extra day, and so we came together in our hotel's conference room to discuss the Linebender project, and what we wanted from it. +By that point, we all felt fairly exhausted, but ~~because Raph is a cruel taskmaster~~ since we wanted to use the opportunity we had, we came together in our hotel's conference room to discuss the Linebender project, and what we wanted from it. We'd initially thought of that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. @@ -244,7 +244,7 @@ In the weeks since we first drafted this roadmap, we've already made a lot of pr What comes after that is a bit hazier. Our three areas of focus are likely to be Android support, accessibility, and developer experience. -As we fill out gaps in our feature set, Xilem is moving closer to becoming a stable, mature framework for writing GUI applications. +As we fill out gaps in our feature set, Xilem is moving towards its faraway goal: becoming a stable, mature framework for GUI applications. We're not GUI yet, but we certainly hope to be one day. In the meantime, stay tuned! From 92795d931b84ee61397fe2e33bf4b25752956f69 Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Sat, 15 Jun 2024 13:28:35 +0200 Subject: [PATCH 11/12] Fix typos in roadmap --- content/blog/2024-05-23-roadmap-may-2024.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/content/blog/2024-05-23-roadmap-may-2024.md b/content/blog/2024-05-23-roadmap-may-2024.md index f8ca014..023548c 100644 --- a/content/blog/2024-05-23-roadmap-may-2024.md +++ b/content/blog/2024-05-23-roadmap-may-2024.md @@ -7,7 +7,7 @@ On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL confere By that point, we all felt fairly exhausted, but ~~because Raph is a cruel taskmaster~~ since we wanted to use the opportunity we had, we came together in our hotel's conference room to discuss the Linebender project, and what we wanted from it. -We'd initially thought of that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. +We'd initially thought that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. What follows is a comprehensive summary of what exactly we're planning to do. We've tried to focus on blockers, where a missing feature in one project is stalling progress on other projects, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. @@ -88,7 +88,7 @@ It includes some work we already finished since RustNL, but will not be updated - **Priority:** High. - **Assigned:** Olivier Faure. -- **Description:** Right now widgets can use context methods in their different passes to request that other passes be ran. What passes can run what other passes isn't clearly documented, especially around edge cases. We should write an RFC to nail down the passes, how they can communicate, and the rationale around them. +- **Description:** Right now widgets can use context methods in their different passes to request that other passes be run. What passes can run what other passes isn't clearly documented, especially around edge cases. We should write an RFC to nail down the passes, how they can communicate, and the rationale around them. - **PR:** TBD. ### Rewrite documentation @@ -126,7 +126,7 @@ It includes some work we already finished since RustNL, but will not be updated - **Priority:** High. - **Assigned:** Chad Brokaw. -- **Description:** Parley should support laying out inline content the way web browsers do, where blocks are splices into your text and should be laid out according to available inline space the same way words are. +- **Description:** Parley should support laying out inline content the way web browsers do, where blocks are spliced into your text and will be laid out according to available inline space the same way words are. - **Issue:** [parley#25](https://github.com/linebender/parley/issues/25) - **PRs:** [parley#67](https://github.com/linebender/parley/pull/67), [parley#76](https://github.com/linebender/parley/pull/76) @@ -134,7 +134,7 @@ It includes some work we already finished since RustNL, but will not be updated - **Priority:** High. - **Assigned:** Nico Burns. -- **Description:** Parley currently has virtually no documentation. If we want it to become a fundational crate, we need to explain to people how to use it. +- **Description:** Parley currently has virtually no documentation. If we want it to become a foundational crate, we need to tell people how to use it. - **PR:** [parley#26](https://github.com/linebender/parley/pull/26) ### Software renderer @@ -179,7 +179,7 @@ It includes some work we already finished since RustNL, but will not be updated - **Priority:** Medium. - **Assigned:** Daniel McNab. -- **Description:** One of Xilem's selling point is the ability to handle virtual lists and async sources of data. That ability was temporarily removed for the Masonry port, but we'd like to add it back it. We don't want Masonry to be aware of async stuff, but the AppDriver trait should at least provide a method which lets Xilem access Masonry internals on its own initiative. +- **Description:** One of Xilem's selling points is the ability to handle virtual lists and async sources of data. That ability was temporarily removed for the Masonry port, but we'd like to add it back. We don't want Masonry to be aware of async stuff, but the AppDriver trait should at least provide a method which lets Xilem access Masonry internals on its own. - **Issue:** [xilem#340](https://github.com/linebender/xilem/issues/340) ### Add unit tests @@ -194,14 +194,14 @@ It includes some work we already finished since RustNL, but will not be updated - **Priority:** Low. - **Assigned:** Olivier Faure. - **Description:** Right now we're mostly working off vibes to know whether or not any of our work improved performance or introduced regressions. -We should write at least *one* benchmark that would test a basic Xilem use-case, so we get a general idea how our performance is progressing. +We should write at least *one* benchmark that would test a basic Xilem use-case, so we get a general idea of how our performance is progressing. - **Issue:** [xilem#362](https://github.com/linebender/xilem/issues/362) ## AccessKit Matt Campbell's contract with Google hasn't started yet. -When it does, top priority will likely be the Android backend. +When it does, the top priority will likely be the Android backend. The plan is to study existing Android accessibility implementations for non-Java toolkits, including Flutter, Chromium, and Gio, to get a better understanding of what's required to implement Android accessibility from scratch. @@ -225,13 +225,13 @@ Some notable sub-goals: - **Priority:** Highest. - **Assigned:** Aaron Muir Hamilton, Raph Levien. -- **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without support for virtual keyboard and other input methods is a non-starter. Our design choices here will have influence on accessibility as well. +- **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without support for the virtual keyboard and other input methods is a non-starter. Our design choices here will have some influence on accessibility as well. ### Gesture recognition - **Priority:** Low. - **Assigned:** Aaron Muir Hamilton. -- **Description:** When the user swipes on the screen, we want to send some kind of scrolling events instead of a normal pointer event. Ideally there should be first-class support for that event to handle momentum scrolling, but a dumber solution would just emulate mouse wheel ticks. +- **Description:** When the user swipes on the screen, we want to send some kind of scrolling event instead of a normal pointer event. Ideally there should be first-class support for that event to handle momentum scrolling, but a dumber solution would just emulate mouse wheel ticks. ## Conclusion From 0aac356379a9ad97db5743841b9873014012035b Mon Sep 17 00:00:00 2001 From: Olivier FAURE Date: Mon, 17 Jun 2024 12:13:58 +0200 Subject: [PATCH 12/12] Change roadmap date --- ...4-05-23-roadmap-may-2024.md => 2024-06-17-roadmap-may-2024.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename content/blog/{2024-05-23-roadmap-may-2024.md => 2024-06-17-roadmap-may-2024.md} (100%) diff --git a/content/blog/2024-05-23-roadmap-may-2024.md b/content/blog/2024-06-17-roadmap-may-2024.md similarity index 100% rename from content/blog/2024-05-23-roadmap-may-2024.md rename to content/blog/2024-06-17-roadmap-may-2024.md