docs: contributing

CONTRIBUTING.md

@@ -1,81 +1,54 @@
 # Developer's Guide
 
-This doc describes how you can get started at developing ConnectorX.
+## Development environment
 
-## Environment Setup
+This project specifies development environment using a [nix shell](./flake.nix).
 
-### Install tools and dependencies
+To load the shell:
 
-Please check out [here](https://sfu-db.github.io/connector-x/install.html#build-from-source-code)
+1. [Install nix (the package manager)](https://nixos.org/download). (only first
+   time)
 
+2. Enable flakes, which are a (pretty stable) experimental feature of nix. (only
+   first time)
 
-### Run tests
+   For non-NixOS users:
 
-* Set up environment variables by creating a `.env` file under project directory. Here is an example:
-```
-# postgres
-POSTGRES_URL=postgresql://username:password@hostname:5432/db
-
-# mysql
-MYSQL_HOST=hostname
-MYSQL_PORT=3306
-MYSQL_DB=db
-MYSQL_USER=username
-MYSQL_PASSWORD=password
-MYSQL_URL=mysql://${MYSQL_USER}:${MYSQL_PASSWORD}@${MYSQL_HOST}:${MYSQL_PORT}/${MYSQL_DB}
-
-# sqlite
-SQLITE_URL=sqlite://db_dir
-
-# mssql
-MSSQL_HOST=hostname
-MSSQL_PORT=1433
-MSSQL_USER=username
-MSSQL_PASSWORD=password
-MSSQL_DB=db
-MSSQL_URL=mssql://username:password@hostname:1433/db
-
-# log
-RUST_LOG=connectorx=debug,connectorx_python=debug
-
-# benchmark related
-TPCH_TABLE=lineitem
-MODIN_ENGINE=dask
+   ```sh
+   mkdir -p ~/.config/nix/
+   tee 'experimental-features = nix-command flakes' >> ~/.config/nix/nix.conf
+   ```
 
-```
+   For NixOs users, follow instructions [here](https://nixos.wiki/wiki/Flakes).
+
+3. Run:
 
-* Seed database: `just seed-db`
-* Run Rust tests: `just test`
-* Run Python tests: `just test-python [-k {test case keyword}]`
+   ```sh
+   nix develop
+   ```
 
-### Other commands
+Optionally, you can install [direnv](https://direnv.net/), to automatically load
+the shell when you enter this repo. The easiest way is to also install
+[direnv-nix](https://github.com/nix-community/nix-direnv) and configure your
+`.envrc` with:
 
-* Format the code: `cargo fmt`
+```sh
+# .envrc
+use flake .#full
+```
 
-## How to Add a New Source
+Additionally, Docker is required for running the test databases.
 
-* Implement source related logics, including:
-  * Define the type system of the new source
-  * Implement data fetching and parsing logics
-  * Examples can be found [here](https://github.com/sfu-db/connector-x/blob/main/connectorx/src/sources)
-* Define the conversion between the new source and existing destinations
-  * Examples can be found [here](https://github.com/sfu-db/connector-x/tree/main/connectorx/src/transports) and [here](https://github.com/sfu-db/connector-x/tree/main/connectorx-python/src/pandas/transports)
-* Make the new source visable to destinations, including:
-  * Add the source to the [source_router](https://github.com/sfu-db/connector-x/blob/main/connectorx-python/src/source_router.rs)
-  * Add the source to writing functions of each destination. Here are examples for [pandas](https://github.com/sfu-db/connector-x/blob/main/connectorx-python/src/pandas/mod.rs) and [arrow](https://github.com/sfu-db/connector-x/blob/main/connectorx-python/src/arrow.rs)
-* Add corresponding unit tests under `connectorx/tests` for Rust and `connectorx-python/connectorx/tests` for Python
+## Running and testing
 
-**Please check out [here](https://sfu-db.github.io/connector-x/connectorx/#extending-connectorx) for more detailed implementation instructions of how to extend ConnectorX.**
+This project specifies commands for running and testing using [Just](https://just.systems/man/en/).
 
-## How to Add a New Destination
+To run the full test suite, run:
 
-* Implement destination related logics, including:
-  * Define the type system of the new destination
-  * Implement data writing logics
-  * Implement the writing interface of destination
-  * Here are examples for [arrow](https://github.com/sfu-db/connector-x/tree/main/connectorx/src/destinations/arrow) and [pandas](https://github.com/sfu-db/connector-x/tree/main/connectorx-python/src/pandas)
-* Define the conversion between existing source and the new destination
-  * Examples can be found [here](https://github.com/sfu-db/connector-x/tree/main/connectorx/src/transports) and [here](https://github.com/sfu-db/connector-x/tree/main/connectorx-python/src/pandas/transports)
-* Add corresponding unit tests under `connectorx/tests` for Rust and `connectorx-python/connectorx/tests` for Python
+```
+just dbs/start
+just dbs/seed
+just test
+```
 
-**Please check out [here](https://sfu-db.github.io/connector-x/connectorx/#extending-connectorx) for more detailed implementation instructions of how to extend ConnectorX.**
+See [Justfile](./Justfile) for all commands available.