Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support WebAssembly #280

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Support WebAssembly #280

wants to merge 13 commits into from

Conversation

magodo
Copy link

@magodo magodo commented Sep 15, 2023

This PR supports running go-plugin based application in the browser.

The basic idea is to leverage the Shared Web Worker API to replace the tcp/uds transport layer, but not touch the application protocol layer (i.e. netrpc/grpc). Meanwhile, the Shared Web Worker also fits well to the parent/child process model, so there is a new runner, wasmrunner, as a replacement for the cmdrunner.

The changes of this PR can be categaried to the following parts:

  • The client/server code change: In order to make the changes manageble, we only touch the minimal pieces of code that require to be replaced by a wasm implementation. Those code are moved out to a file appended with _other.go, with no change. Meanwhile, there are new files with the same prefix, but ends with _wasm.go, that contains the wasm implementation. These files are conditionally built by using Go build constraint.
  • The wasmrunner package: There is a new wasmrunner package added that implements the Runner interface, but using a wasm file (instead of a OS executable), and starts the target plugin as a Web Shared Worker.
  • server_net_wasm.go: This file implements the net.Listener, net.Conne and net.Addr, for the WASM abstract.
  • The examples/basic_wasm and examples/grpc_wasm: Almost the same content as the basic and grpc example (some files are even not changed at all, but using a symlink). They are meant to show case how it works in practice.

Unfortunately, due to the limitation of the WASM environment, we only implemented the basic functionalities of the go-plugin, with (at least) below limitations:

  • No bidirectional connection supported. This is due to we can not create a nested shared worker inside a shared worker

  • Client has to define the SyncStdout and SyncStderr in the ClientConfig, so that it can read the stdout/stderr from the plugin, including the logs that are printed via log (log-like) API.

    This is because when using WASM, we are not reassigning the os.Stdout/Stderr, but reimplement the writeSync used underlying. This means:

    • Previously, the log package (and the hclog) used in the provider will write to the plugin process's original stderr. Only those explicit write to stdout/stderr (e.g. via fmt.Fprint()), routes to the client via RPC.
    • Now, no matter using log or explicit write, the logs all route to the client via RPC.

    See the grpc_wasm example for details.

Beside these limitations, it actually works pretty well. We have been using it in some Terraform based projects, where we used this lightweighted TF client for importing resources from any provider (though some huge provider might causes WASM build failure due to its size, where needs to be trimmed a bit), in the browser, with only one line change in the go.mod to replace the go-plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant