Changes to the Workshop

[LuckyJosh]

An Issue for collecting ideas and arguments for
changes to the Workshop, including but not limited to:

• the topics
• the material
• the schedule
• the goal/purpose

of the Workshop.
Once wie have formulated some actionable ideas we can split them into multiple 'bite sized' issues.

[maxnoe]

We might be lucky that most problems with WSL are gone?

https://devblogs.microsoft.com/commandline/the-windows-subsystem-for-linux-in-the-microsoft-store-is-now-generally-available-on-windows-10-and-11/

[maxnoe]

Let me write down the general principals that should guide us in these discussions:

• Prefer open source software whenever possible
• Follow best practices
• Show things that scale beyond the lab courses to bachelor / master thesis / research projects
• Teach people how to work in a reproducible, collaborative way

In our previous discussions, we forgot the "collaborative" part a bit, when it was proposed to switch from WSL to installing software on windows natively. Any solution we should come up with should ideally work

• On all OSes basically the same way after the installation (so we don't have to teach / be aware of multiple ways to do things)
• Work when two people want to collaborate across OSes (e.g. one person on Mac / one person on Linux / one person on Windows)

In that context, we wanted to discuss three main things:

• How to deal with Windows and the problems it's currently creating
• Replacing Make with something more modern
• Replacing uncertainties

At least the first 2 are linked: if we do not want to use WSL anymore on Windows, Make does not work (or we need to go back to git bash / mingw / something else). Snakemake unfortunately does not support windows (they recommend WSL).
Ninja supports windows, but writing ninja rule files that work on both windows and linux will be hard.

In my opinion, we thus don't really have an alternative to WSL. Indepent of the make question.
My hope would be that the recent WSL release

• Make installation much easier (just click download in the MS Store)
• Fixes the problems we saw (hopefully... TBC)
• Finally brings the GUI interoperability also to Windows 10, so with that, the installation of Win 10 and Win 11 are identical again, at least much more similar and the custom Xserver is not required anymore on Win10 (also to be checked).

With uncertainties, I am a bit conflicted. For the usecase of the lab courses, it's much more straight forward than anything else I think. The criticisms are valid though...

Here, I would propose to remove the more complex things like the curve_fit wrappers and just use it "bare-bones" where needed.

My main point would be, that we should prepare a real "Introduction" lecture explaining the principles in detail.

[chrbeckm]

Regarding the last point, I would add that some students treat the programms and languages we present as a black box, see last emails. We should think about ways how to address this. Although I'm not sure if everyone behind those mails visited a workshop.

[LuckyJosh]

In my opinion, we thus don't really have an alternative to WSL.

I recently dabbeld with VMs using VirtualBox and the experience was not bad.
Using a Ubuntu VM (maybe Xubuntu to conserve on resources) for the workshop would have some advantages:

• a clear separation between host OS and Linux
  (its like a "dualboot light" ... kind of)
  this would be a small step towards "working on a
  remote machine", that would not be to intrusive for beginners)
• no WSL shenanigans (or generally being at the mercy
  of Microsofts plans) just linux.
• the typical Problem of typing a comand into the wrong shell would be close to eliminated
• The length of instructions would be (approx.) more than halfed
• The participants would learn how to install linux
  without any risk, again a small step in the right direction

[LuckyJosh]

And to add a silly idea at the end:
We could go so far as to letting the mac users install
the vm too, then everybody would,have the exact same setup.

[maxnoe]

I recently dabbeld with VMs using VirtualBox and the experience was not bad.

WSL is essentially that, with extra niceties for better inter-op (data exchange!) with the windows host, think sending an email with the protocol. So I think WSL, if the problems are addressen in this recent update, is the preferable solution to a full, custom virtualbox.

[LuckyJosh]

Absolutly, if the WSL gets its stuff together, I'd say it is surly the way to go.

Just for the record, the data exchange is no problem, if you have a USB-Stick. You can change the USB connection on the fly from VM to host and back.

[maxnoe]

I just installed the new WSL 1.0 on Windows 10 and everything seems to work just fine, including plots popping up on windows without doing anything special and installing TeX Live and building the toolbox material.

I'd thus recommend keeping WSL for the windows users.

[SepplL]

Does everything also work on WIN11? Last time that was the bigger issue of the two Windows OS's.
Especially finding a solution to the same problem on more than one machine with similar steps was close to impossible.
And trying out multiple different possible solutions is not an enjoyable experience for both parties.

Do we have access to a couple of WIN11 machines so that we can test that behavior thoroughly?
If there everything behaves the way it should, then maybe the WSL is something we can keep for this year.

[LuckyJosh]

I was going to put a Win11 Partition on my laptop, but it is not officially supported. You can however alter the installer to remove the requirements it is missing.
So, soon we will have atleast one 😅

[chrbeckm]

I can try to test it on an old pc running the insider previews. If it runs there, every laptop can handle the workload.

[maxnoe]

I can try to test it on an old pc

Keep in mind that Windows 11 is incompatible with most older PCs (It requires TPM 2.0): https://support.microsoft.com/en-us/windows/enable-tpm-2-0-on-your-pc-1fd5a332-360d-4f46-a1e7-ae6b0c90645c#bkmk_enable_tpm

[chrbeckm]

I know, but that PC is running it already.

[LuckyJosh]

I can try to test it on an old pc

Keep in mind that Windows 11 is incompatible with most older PCs (It requires TPM 2.0): https://support.microsoft.com/en-us/windows/enable-tpm-2-0-on-your-pc-1fd5a332-360d-4f46-a1e7-ae6b0c90645c#bkmk_enable_tpm

That is the requirement I was talking about,
you can patch the installer to not check this requirement.
Whether it is save to do so, I am not sure but I wouldnt use Win11 for long anyway 🤷‍♂️

[Kevin2]

I recently installed a dual boot with Win11 and ubuntu (Win11 was pre-installed). After disabling the disk encryption in Win11, the steps in our dual boot instructions worked fine.

[maxnoe]

This is mainly about the WSL installation, though it's good to know. Could you maybe also test (and maybe do screenshots along the way) if the WSL install works as simple as advertised by MSFT? I.e. one should be able to install ubuntu from the store or do wsl --install in an admin powershell.

[jpwgnr]

This actually works awesome in Windows 11.

We can explain how to install the terminal (which I already had), but then we can start from the powershell and everything will work just like in linux.
When you had wsl already you need to wsl --update and wsl --shutdown once, but then it works like a charm.
I was able to install python with mambaforge as Max suggested, full texlive and vscodium.
Codium gives an error message, because it thinks it needs to be installed in Windows with the WSL remote add-on, but that is not true anymore.
One can set DONT_PROMPT_WSL_INSTALL = 1 and then this error message does not come anymore.
One can install okular and evince and other linux GUIs and they look very fine. It's great!

Installtion of Everything in WSL

As already mentioned I did not install the terminal, which is very simple.

Install Ubuntu

First I installed WSL and Ubuntu 22.04 LTS. (I had WSL, so I had to update it, but usually this would be enough)

Then I had to give a new name and password.

In the explorer I am able to find my home directory.

When downloading something from the web this is actually not shown so I usually downloaded to the Windows Desktop and drag and dropped it then to the Linux home folder.

Installation of Codium

This way I downloaded the codium .deb and the mambaforge .sh files.

I installed codium with

sudo apt install ./codium-*-amd64.deb

When I tried to start it there was this warning.

It was possible to disable with the following which we should let them add to their .bashrc.

Then Codium looked like the following.

Install Mambaforge

Mambaforge I installed almost analogous to our Linux installation of anaconda.

As Max said it would be nice to have an environment for them but I just install some packages in my base env.

pip install numpy, matplotlib, uncertainties, sympy, jupyterlab

With these I was able to do the test from our installation page.

When I saved it to a file and opened it with evince, that I installed with

sudo apt install evince

this also opened and looked very nice.

Install LaTeX

Just installed it analogous to the Linux installation page without any problem.

So very nice. W11 seams to be no problem anymore.
Just snap is not working without hacking around, but we don't need this for our workshop anyways. That's just a personal thing that annoys me.

[maxnoe]

Cool.

The only thing I am not sure about is whether it's better to have code in wsl or windows side.

[chrbeckm]

The only thing I am not sure about is whether it's better to have code in wsl or windows side.

If it runs well on the WSL side, some of the language support stuff might be easier. If it is installed on the Windows side it obviously can't find the python install, which is required by some addons.

[chrbeckm]

@jpherdi good to know that it works on your system. Now we would need to test it on Windows 10 and WSL2 as well. And discuss how we would like to handle errors, e.g., the previous ones, no internet, weird Desktop placement, changing default starting dirs of the terminal.

[jpwgnr]

If it runs well on the WSL side, some of the language support stuff might be easier. If it is installed on the Windows side it obviously can't find the python install, which is required by some addons.

I don't think this is true. You could just give it the right path to the python installation. The right terminal it finds as well because you can choose between these.

The reason why I did not install it in Windows is that the Codiums add-on WSL remote does not work properly.
If we would use VSCode this is no issue. Then it makes definitely sense to use the VSCode on Windows side.

My result was: When we want to use Codium we should do it via Linux, when we are fine with Code then Windows side would be best.

What would you like to discuss about the errors @chrbeckm? I mean these errors are not present anymore (for now).

[maxnoe]

If we would use VSCode this is no issue. Then it makes definitely sense to use the VSCode on Windows side.

The concerns about telemetry are somewhat absurd on windows... so I am fine with using vs code on windws as having windows is the main problem then

[jpwgnr]

True, I am also fine with that. And this definitely works.

[SepplL]

This really looks great.
And the fact, that the Linux home directory is now accessible via the Explorer will make things a lot easier in finding the "right" directory from Windows to Linux and the other way round.
Drag and Dropping downloaded files will be more manageable thing to do for entry level users with the WSL.

So maybe we can change the instructions to moving the downloaded files to the WSL home/Downloads and then executing all the commands there instead of using the Windows Desktop?

With the VSCode vs Codium I agree to what @maxnoe said. On Windows telemetry should not be that big of a concern so VSCode should be fine.

[maxnoe]

As Max said it would be nice to have an environment for them but I just install some packages in my base env.

pip install numpy, matplotlib, uncertainties, sympy, jupyterlab

That would be as simple: mamba create -n toolbox python=3.11 numpy matplotlib scipy sympy jupyter notebook uncertainties ipython

[SepplL]

You mean like you mentioned in #319 ?
I'd like to point out, that we should introduce the whole concept of environments in the first place then.
So maybe add a couple of slides and explain why, when and how to use these environments.
With the impression of people using this workshop as a black box, it could be unintuitive of python to not work "out of the box" with other projects or ideas or when changing the environment as an accident.
Also then just installing new packages at later stages for individual users should be done in different environments which requires a basic understanding of what is going on with the python install.

I have a neutral opinion on whether this is a good idea, but I'd like to prevent confusion for our attendees.

[maxnoe]

that we should introduce the whole concept of environments in the first place then.

Yes, of course. That is an integral part of "reproducibility"

[chrbeckm]

What would you like to discuss about the errors @chrbeckm? I mean these errors are not present anymore (for now).

When we tested it for the last years, we didn't run into all the errors we got per e-mail. And most of them are hard to resolve if you do not use Windows 11 or even 10 yourself.
If it works, I agree, that it is the best solution. But I would still like to work on the VM installation, as a maybe not as nice, but likelier to work alternative.

[maxnoe]

If it works, I agree, that it is the best solution. But I would still like to work on the VM installation, as a maybe not as nice, but likelier to work alternative.

WSL is a VM. I really don't think we should advocate for using something like an Ubuntu Virtual Box on Windows.

[chrbeckm]

I meant a VirtualBox or similar alternative. And not to have it actively on our website, but as a guide for people with, for us unsolvable, WSL problems. Or should we advocate for a dual-boot in that situation? Or do we trust in the WSL after its release?

[jpwgnr]

I mean, if we test W10 as well and it works, I don't understand, why we should not trust it.

[maxnoe]

I did that, works fine.

[jpwgnr]

Then I don't see any problem. @chrbeckm

[chrbeckm]

Ok, do you want to change then the installation notes? As you have a modern Windows 11 running laptop, it might be the best.
Or is the installation now similar between 10 and 11?

[LuckyJosh]

I fell like we are running in circles on the WSL issue.
My summary of the current state is:

If the WSL is working correctly and without the hassle we had last year it is the preferred solution. (Especially because the instalation is even easier after the release in the WindowsStore).

The two example installations discussed here, are reasons to be optimistic about the WSL. Though it still is some kind of "It works on my machine"-senario with such a small sample size¹.
Yet I have to say after the chaos the WSL has caused last year, I would like to be prepared if the need for an alternative arises. The two alternatives we had for years are:

1. Dual boot

2. Migrating to Linux

3. is a kind of hard core solution, that most partitipants
   wont choose ... atleast not after just 9 days of exposure to "linux".

4. Is a more argeeable solution then 2) but not without its on problems, especially if the hardware vendor tries everything to prevent it (a certain windows-tablet in last years workshop comes to mind).

The new alternative that has come up is the VM via VirtualBox (or whatever).
Reading through the comments, I get the impression
that VMs are kind of polarizing.

I layed out some arguments from my perspective
#317 (comment)

On the other hand

WSL is a VM. I really don't think we should advocate for using something like an Ubuntu Virtual Box on Windows

is a clear contra-standpoint. But I am missing the reasons for it:
Why is a VM categorically worse than the WSL (being a VM aswell) or a dual boot installation?

¹One idea that came to mind while tiping this is the following: We could ask for students with Windows PCs from semester 3/4 and up to test the of the new WSL-Installation (under our supervision). This way we could extend the sample size and get a wide range of different devices tested. Idealy it would be an improvmant for the testers as well.

[LuckyJosh]

Sorry fat-fingered the wrong button