Testbeds

[danth]

In lieu of #50, which was never finished, it would be useful to provide utilities to test and preview changes without modifying your real configuration. This makes things easier when testing a different desktop environment, or something like a login screen which requires logging out to see what changed.

The following are some ideas I've had and a rough order in which to attempt them. Any suggestions are welcome.

• NixOS configurations as virtual machines for manual testing. Each configuration should run either:

  • A desktop environment with only its default applications
  • An application in a minimal environment such as cage

• Build the above on GitHub Actions.

  This won't test that the theme appears or looks good, but it should pick up on more build errors than the current CI does.

• Use the NixOS testing framework or another method to automatically screenshot each virtual machine.

  Benefits could include:

  • Before and after screenshots generated by GitHub Actions
  • Mass producing showcases for the documentation

  I don't think this one is worth implementing right now, because:

  • Many targets will require specific code to detect when they are ready for a screenshot
  • Creating a system to generate relevant screenshots and post them to GitHub would be a significant amount of work
  • Without such a system, the screenshots would have to be built locally, in which case you can just use the virtual machine interactively

• Specially crafted color schemes to make edge cases more noticeable.

  For example, a scheme with certain colors equal to each other would highlight cases where we expect two colors to contrast, but the specification doesn't require them to be that way.

  Similarly, a scheme with vastly different colors would catch cases where two similar colors were mistaken as being the same.

Current tasks:

• stylix: add testbeds for desktop environments #320
• Add testbeds for individual applications
  • Each /modules/firefox variant should have a testbed 1
• doc: add documentation for testbeds #347
• Design color schemes for edge cases