Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update short guide of dev setup #3285

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
225 changes: 78 additions & 147 deletions docs/src/guides/setup-dev.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,21 @@
# Installing dependencies
# Installing development setup for ZKsync

## TL;DR

This is a shorter version of setup guide to make it easier subsequent initializations. If it's the first time you're
This is a shorter version of setup guide to make it easier for initializations. If it's the first time you're
initializing the workspace, it's recommended that you read the whole guide below, as it provides more context and tips.

If you run on 'clean' Ubuntu on GCP:
Just run the following command on Linux:

```bash
# For VMs only! They don't have SSH keys, so we override SSH with HTTPS
git config --global url."https://github.com/".insteadOf [email protected]:
git config --global url."https://".insteadOf git://

# Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash

# All necessary stuff
sudo apt-get update
sudo apt-get install -y build-essential pkg-config cmake clang lldb lld libssl-dev libpq-dev apt-transport-https ca-certificates curl software-properties-common

# Install docker
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo apt install docker-ce
sudo usermod -aG docker ${USER}

# Start docker.
sudo systemctl start docker

## You might need to re-connect (due to usermod change).

# Node & yarn
nvm install 20
# Important: there will be a note in the output to load
# new paths in your local session, either run it or reload the terminal.
npm install -g yarn
yarn set version 1.22.19

# For running unit tests
cargo install cargo-nextest
# SQL tools
cargo install sqlx-cli --version 0.8.1

# Foundry ZKsync
curl -L https://raw.githubusercontent.com/matter-labs/foundry-zksync/main/install-foundry-zksync | bash
foundryup-zksync

# Non CUDA (GPU) setup, can be skipped if the machine has a CUDA installed for provers
# Don't do that if you intend to run provers on your machine. Check the prover docs for a setup instead.
echo "export ZKSYNC_USE_CUDA_STUBS=true" >> ~/.bashrc
# You will need to reload your `*rc` file here

# Clone the repo to the desired location
git clone [email protected]:matter-labs/zksync-era.git
cd zksync-era
git submodule update --init --recursive
curl -L https://raw.githubusercontent.com/matter-labs/zksync-era/main/docs/src/guides/setup-dev.sh | bash
```

Don't forget to look at [tips](#tips).

## Supported operating systems
## Installation guide

### Supported operating systems

ZKsync currently can be launched on any \*nix operating system (e.g. any linux distribution or macOS).

Expand All @@ -77,84 +30,27 @@ Rosetta may cause problems that are hard to spot and debug, so make sure to chec

If you are a NixOS user or would like to have a reproducible environment, skip to the section about `nix`.

## Docker
### Basic packages

Install `docker`. It is recommended to follow the instructions from the
[official site](https://docs.docker.com/install/).

Note: currently official site proposes using Docker Desktop for Linux, which is a GUI tool with plenty of quirks. If you
want to only have CLI tool, you need the `docker-ce` package and you can follow
[this guide](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04) for Ubuntu.

Installing `docker` via `snap` or from the default repository can cause troubles.

You need to install both `docker` and `docker compose`.

**Note:** `docker compose` is installed automatically with `Docker Desktop`.

**Note:** On linux you may encounter the following error when you’ll try to work with `zksync`:

```
ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.
```

If so, you **do not need** to install `docker-machine`. Most probably, it means that your user is not added to
the`docker` group. You can check it as follows:

```bash
docker-compose up # Should raise the same error.
sudo docker-compose up # Should start doing things.
```

If the first command fails, but the second succeeds, then you need to add your user to the `docker` group:

```bash
sudo usermod -a -G docker your_user_name
```

After that, you should logout and login again (user groups are refreshed after the login). The problem should be solved
at this step.

If logging out does not resolve the issue, restarting the computer should.

## Node.js & Yarn

1. Install `Node` (requires version `v20`). The recommended way is via [nvm](https://github.com/nvm-sh/nvm).
2. Install `yarn`. Can be done via `npm install -g yarn`. Make sure to get version 1.22.19 - you can change the version
by running `yarn set version 1.22.19`.

## clang

In order to compile RocksDB, you must have LLVM available. On debian-based linux it can be installed as follows:
Install the basic packages:

On debian-based linux:

```bash
sudo apt-get install build-essential pkg-config cmake clang lldb lld
sudo apt install git build-essential pkg-config cmake clang lldb lld libssl-dev libpq-dev apt-transport-https ca-certificates curl software-properties-common
```

On macOS:

You need to have an up-to-date `Xcode`. You can install it directly from `App Store`. With Xcode command line tools, you
get the Clang compiler installed by default. Thus, having XCode you don't need to install `clang`.

## OpenSSL

Install OpenSSL:

On mac:

```bash
brew install openssl
brew install git build-essential pkg-config cmake clang lldb lld libssl-dev libpq-dev apt-transport-https ca-certificates curl software-properties-common
```

On debian-based linux:

```bash
sudo apt-get install libssl-dev
```
Note that in order to install `clang` on MacOS you need to have an up-to-date `Xcode`. You can install it directly from
`App Store`. With `Xcode` command line tools, you get the Clang compiler installed by default. Thus, having `Xcode` you
don't need to install `clang`.

## Rust
### Rust

Install `Rust`'s toolchain version reported in `/rust-toolchain.toml` (also a later stable version should work).

Expand Down Expand Up @@ -190,62 +86,85 @@ If you see `x86_64` mentioned in the output, probably you're running (or used to
that's the case, you should probably change the way you run terminal, and/or reinstall your IDE, and then reinstall the
Rust toolchain as well.

## PostgreSQL Client Library
#### Cargo nextest

For development purposes, you typically only need the PostgreSQL client library, not the full server installation.
Here's how to install it:

On macOS:
[cargo-nextest](https://nexte.st/) is the next-generation test runner for Rust projects. `zkstack dev test rust` uses
`cargo nextest` by default.

```bash
brew install libpq
cargo install cargo-nextest --locked
```

On Debian-based Linux:
#### SQLx CLI

SQLx is a Rust library we use to interact with Postgres, and its CLI is used to manage DB migrations and support several
features of the library.

```bash
sudo apt-get install libpq-dev
cargo install sqlx-cli --version 0.8.1
```

### Cargo nextest
### Docker

[cargo-nextest](https://nexte.st/) is the next-generation test runner for Rust projects. `zkstack dev test rust` uses
`cargo nextest` by default.
Install `docker`. It is recommended to follow the instructions from the
[official site](https://docs.docker.com/install/).

```bash
cargo install cargo-nextest
```
Note: currently official site proposes using Docker Desktop for Linux, which is a GUI tool with plenty of quirks. If you
want to only have CLI tool, you need the `docker-ce` package and you can follow
[this guide](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04) for Ubuntu.

### SQLx CLI
Installing `docker` via `snap` or from the default repository can cause troubles.

SQLx is a Rust library we use to interact with Postgres, and its CLI is used to manage DB migrations and support several
features of the library.
You need to install both `docker` and `docker compose`.

**Note:** `docker compose` is installed automatically with `Docker Desktop`.

**Note:** On linux you may encounter the following error when you’ll try to work with `zksync`:

```
ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.
```

If so, you **do not need** to install `docker-machine`. Most probably, it means that your user is not added to
the`docker` group. You can check it as follows:

```bash
cargo install --locked sqlx-cli --version 0.8.1
docker-compose up # Should raise the same error.
sudo docker-compose up # Should start doing things.
```

## Easier method using `nix`
If the first command fails, but the second succeeds, then you need to add your user to the `docker` group:

Nix is a tool that can fetch _exactly_ the right dependencies specified via hashes. The current config is Linux-only but
it is likely that it can be adapted to Mac.
```bash
sudo usermod -a -G docker your_user_name
```

Install `nix`. Enable the nix command and flakes.
After that, you should logout and login again (user groups are refreshed after the login). The problem should be solved
at this step.

Install docker, rustup and use rust to install SQLx CLI like described above. If you are on NixOS, you also need to
enable nix-ld.
If logging out does not resolve the issue, restarting the computer should.

Go to the zksync folder and run `nix develop`. After it finishes, you are in a shell that has all the dependencies.
### Node.js & Yarn

1. Install `Node` (requires version `v20`). The recommended way is via [nvm](https://github.com/nvm-sh/nvm).
2. Install `yarn`. Can be done via `npm install -g yarn`. Make sure to get version 1.22.19 - you can change the version
by running `yarn set version 1.22.19`.

## Foundry ZKsync
### Foundry ZKsync

ZKSync depends on Foundry ZKsync (which is is a specialized fork of Foundry, tailored for ZKsync). Please follow this
ZKsync depends on Foundry ZKsync (which is is a specialized fork of Foundry, tailored for ZKsync). Please follow this
[installation guide](https://foundry-book.zksync.io/getting-started/installation) to get started with Foundry ZKsync.

Foundry ZKsync can also be used for deploying smart contracts. For commands related to deployment, you can pass flags
for Foundry integration.

## Non-GPU setup
### ZK Stack CLI

Toolkit for creating and managing ZK Stack chains. `ZK Stack CLI` facilitates the creation and management of ZK Stacks.
Commands are interactive but can also accept arguments via the command line. Follow the installation instructions
[here](https://github.com/matter-labs/zksync-era/tree/main/zkstack_cli#installation).

### Non-GPU setup

Circuit Prover requires a CUDA bindings to run. If you still want to be able to build everything locally on non-CUDA
setup, you'll need use CUDA stubs.
Expand All @@ -270,6 +189,18 @@ RUSTFLAGS as env var, or pass it in `config.toml` (either project level or globa
rustflags = ["--cfg=no_cuda"]
```

## Easier method using `nix`

Nix is a tool that can fetch _exactly_ the right dependencies specified via hashes. The current config is Linux-only but
it is likely that it can be adapted to Mac.

Install `nix`. Enable the nix command and flakes.

Install docker, rustup and use rust to install SQLx CLI like described above. If you are on NixOS, you also need to
enable nix-ld.

Go to the zksync folder and run `nix develop`. After it finishes, you are in a shell that has all the dependencies.

## Tips

### Tip: `mold`
Expand Down
71 changes: 71 additions & 0 deletions docs/src/guides/setup-dev.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env bash

main() {
say "Setting up the development environment for ZKsync"

# All necessary stuff
say "Installing apt dependencies..."
sudo apt update
sudo apt install --yes git build-essential pkg-config cmake clang lldb lld libssl-dev libpq-dev apt-transport-https ca-certificates curl software-properties-common

# Rust
say "Installing Rust..."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y
source "$HOME/.cargo/env"
# For running unit tests
say "Installing Nextest..."
cargo install cargo-nextest --locked
# SQL tools
say "Installing SQLx CLI..."
cargo install sqlx-cli --version 0.8.1

# Install Docker
say "Installing Docker..."
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository --yes "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo apt install --yes docker-ce

# Start Docker
say "Starting Docker..."
sudo systemctl start docker

# NVM
say "Installing NVM..."
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
nvm install 20

# Yarn
say "Installing Yarn..."
npm install -g yarn
yarn set version 1.22.19

# Foundry ZKsync
say "Installing Foundry ZKsync..."
curl -L https://raw.githubusercontent.com/matter-labs/foundry-zksync/main/install-foundry-zksync | bash

# ZK Stack CLI
say "Installing ZK Stack CLI..."
curl -L https://raw.githubusercontent.com/matter-labs/zksync-era/main/zkstack_cli/zkstackup/install | bash
"$HOME/.local/bin/zkstackup"

# Non CUDA (GPU) setup, can be skipped if the machine has a CUDA installed for provers
# Don't do that if you intend to run provers on your machine. Check the prover docs for a setup instead.
say "Setting up the non-CUDA setup..."
echo "export ZKSYNC_USE_CUDA_STUBS=true" >> "$HOME/.bashrc"

# Clone the repo
say "Cloning the ZKsync repository..."
git clone --recurse-submodules https://github.com/matter-labs/zksync-era.git

say "Installation of the development environment for ZKsync complete!"
say "Please reload your shell configuration by running 'source ~/.bashrc'"
}

say() {
echo -e "\033[1;32m$1\033[0m"
}

main "$@"
5 changes: 0 additions & 5 deletions zkstack_cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,6 @@
Toolkit for creating and managing ZK Stack chains. `ZK Stack CLI` facilitates the creation and management of ZK Stacks.
Commands are interactive but can also accept arguments via the command line.

### Dependencies

Follow [these instructions](https://github.com/matter-labs/zksync-era/blob/main/docs/guides/setup-dev.md) to set up
dependencies on your machine. Ignore the Environment section for now.

### Installation

You can use `zkstackup` to install and manage `zkstack`:
Expand Down
Loading