Timeout launching browser installed as a Snap package

[ankur22]

Original community forum post: https://community.k6.io/t/xk6-browser-error-getting-devtools-url-context-deadline-exceeded/4410

xk6-browser version: https://github.com/grafana/xk6-browser/releases/tag/v0.4.0
Platform: Windows 11, WSL2, Ubuntu 20.04

Issue

The user reported that they were unable to run the example that was posted in this blog post. They were able to install and compile a binary using the commands that were in the blog post, but when it came to running the example test, the test times out and fails with:

ERRO[0030] launching browser: getting DevTools URL: context deadline exceeded
running at reflect.methodValueCall (native)
default at file:/xk6-browser-examples/browser-test.js:4:19(5)
        at native executor=per-vu-iterations scenario=default source=stacktrace

I was able to reproduce the issue on my Windows laptop.

[imiric]

Update and correction: After discussions with Ankur, we determined that the issue mentioned in the description happens in WSL2/Ubuntu because the chromium-browser package was installed, which is a Snap package. Snaps are containers, and as such don't work in the same way as a native binary, and are not supported by xk6-browser. The error described below happens with a native Chrome package, so the causes are different.

This issue was also seen on Linux, but only on Cloud test runs. See grafana/xk6-browser#559. It makes sense, since both EC2 and WSL2 run Linux VMs. I haven't been able to reproduce it natively on my local machine, and it's very rare even in the Cloud, but if it can be consistently reproduced in WSL2, that might be a good testbed for troubleshooting this.

The error is:

launching browser: getting DevTools URL: timed out after 30s
	at reflect.methodValueCall (native)
	at file:///tmp/PvwCGT/script.js:28:34(6)
	at native
 executor=constant-vus scenario=default		test_run_id: 141246

It's the same error, just friendlier since we don't mention the context anymore.

As mentioned in grafana/xk6-browser#559:

There's no reason this particular timeout should be that long, so we can shorten it, but also fix the root cause, which seems to be because of race condition between when the process starts, and us attaching the stdout listener to get the DevTools URL. Also, try to look into a more robust way of getting the URL that doesn't involve parsing stdout.

[imiric]

Given that we now know the root cause of this (see the updated comment above), the fix for this issue should ensure that we fail quickly, and, if possible, printing an informative error that mentions that only native binaries are supported (or maybe printing the process' stderr?).

Checking the behavior on Ubuntu 20.04, /usr/bin/chromium-browser is a shell script that exits with code 1, asking to run snap install chromium. So maybe we're not handling exit codes properly. The test run should be aborted in this case, as it doesn't make sense to keep trying to run other VUs and iterations.

And maybe part of this issue, but we should clarify in our README and documentation that Snap/Flatpak and any other way of running the browser inside a container, is not supported. We should specifically mention Ubuntu, as most people are likely unaware that the chromium-browser package is "transitional", and that Ubuntu officially only supports it as a Snap.

[inancgumus]

@grafana/k6-browser Is this still relevant?

[ankur22]

Needs to be evaluated.