diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 750355dfd1..796bc85234 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,22 +3,26 @@ This repo is built and maintained by the Galoy team. We welcome and appreciate new contributions and encourage you to check out the repo and [join our community](https://chat.galoy.io/) to get started. To help get you started, we will explain a bit about how we've laid out the code here and some of the practices we use. Our layout details fall mostly into two categories: + - [Working with Types](#working-with-types) - [Understanding our Architecture](#architecture) --- ## Working with Types + This codebase is implemented using a well-typed approach where we rely on Typescript for catching & enforcing certain kinds of checks we would like to perform. This approach helps to strenghthen guarantees that bugs/issues don't get to runtime unhandled if our types are implemented properly. ### Our Approach + We define all our types in declaration files (`*.d.ts`) where typescript automatically adds any types to the context without needing to explicitly import. Declaration files have limitations on what can be imported & instantiated within them, so in the few cases where types derive from instances of things we would usually have to use special tricks to derive them. The majority of our types are defined in the `domain` layer (see Architecture section below). We generally create types for the following scenarios: #### Symbol types + These are unique types that are alternatives to generically typing things as Typescript's primitive types (e.g. `string`, `number`, `boolean`). Doing this helps us to add context to primitive types that we pass around and it allows the type checked to distinguish between different kinds of primitives throughout the code. For example, an onchain address and a lightning network payment request are both strings, but they aren't interchangeable as a data type. Instead of using `string` type for these types we would define as follows using a "unique symbol": @@ -27,7 +31,6 @@ For example, an onchain address and a lightning network payment request are both type EncodedPaymentRequest = string & { readonly brand: unique symbol } type OnChainAddress = string & { readonly brand: unique symbol } - ``` #### Error types @@ -35,6 +38,7 @@ type OnChainAddress = string & { readonly brand: unique symbol } These are types mostly used in function signature type definitions. They derive from implemented error classes and need to be imported in a special way to their type declaration file. For example, an error may be defined in a module's `error.ts` file like this: + ``` export class LightningError extends DomainError { name = this.constructor.name @@ -42,16 +46,17 @@ export class LightningError extends DomainError { ``` and then it is imported to its `index.types.d.ts` declaration file and made into a type like this: + ``` type LightningError = import("./errors").LightningError ``` - #### Imported Library types In the places where we would like to work with types defined in imported libraries, we have two options. When we would like to use them directly, we can simply import them. If we would like to re-use a type to create our own types, we would re-import to get those types into our declaration files. For example, for a result type from the `lightning` library, we would import and re-use it like: + ``` type GetPaymentResult = import("lightning").GetPaymentResult type RawPaths = NonNullable["paths"] @@ -59,11 +64,12 @@ type RawPaths = NonNullable["paths"] In this example, we need the type definition for the `payment.paths` property of the `GetPaymentResult` type from the library that we otherwise would not have direct access to. - #### Function signatures and argument types + Whenever we implement a new function, the argument and return types are assigned at the point of the function's implementation. For example: + ```ts const myFunction = async (myArg: MyArg): Promise => { return doThing(myArg) @@ -78,7 +84,6 @@ const myOtherFunction = async ({ }): Promise => { return doThing(myFirstArg, mySecondArg) } - ``` In cases where the function has a complex set of arguments, the argument type can be defined in a declaration file and then assigned at the point of function implementation. @@ -100,11 +105,13 @@ const myFunction = async ({ ``` #### Defining objects with methods + We use functional constructors to define certain types of objects that we can call method on. The intention here is to instantiate the object first and then call methods on that object with method-specific args to execute some functionality. For objects like these, the interface for the object is defined using a `type` declaration and methods are typed at this point. For example, our fee calculator is typed as follows: + ```ts // In '.d.ts' file type DepositFeeCalculator = { @@ -134,14 +141,17 @@ export const DepositFeeCalculator = (): DepositFeeCalculator => { Note that the top-level function arguments (for `DepositFeeCalculator`) would still be typed at the point of implementation like with any other function, and it is only the method signatures that are included in declaration files. #### Interfaces + We use the `interface` keyword to define objects with methods that are intended to be implemented outside of the domain. This most often happens with service implementations like our `ILightningService` interface that gets implemented as the `LndService` function. Everything else about how we go about typing and implementing things with the `interface` keyword is the same as with how we handle "objects with methods" in our domain as described above. #### "Enums" via object constants + Typescript's `enum` keyword has drawbacks that don't fully meet our typing needs. To get around this, we use the trick where we define our intended enum as a standard object and then import it into our type declarations. For example: + ```ts // In implementation file export const AccountStatus = { @@ -153,58 +163,68 @@ export const AccountStatus = { type AccountStatus = typeof import("./index").AccountStatus[keyof typeof import("./index").AccountStatus] ``` + ## Architecture + We use hexagonal architecture pattern ([context](https://blog.ndepend.com/hexagonal-architecture/)). Code goes into one of four layers: + - Domain layer (`./src/domain`) - Services layer (`./src/services`) - Application layer (`./src/app`) - Application Access layer (`./src/servers`) ### Domain layer + Defines the models and business logic with related interfaces. #### Responsibility + - Define all data types - Implement operations on the data types that depends on conditionals or data transformations - Define interfaces of external services #### Dependencies + - Internal: None - External: only utility libraries but must be as clean as possible - ### Services layer + Implements all the adapters (specific implementations of the interfaces defined in domain layer) that use external services/resources #### Responsibility + - Implement interfaces defined in domain layer #### Examples + - Access to external resources (database, redis, bitcoind, lnd) - Consumption of external services/APIs (twilio, geetest, price, hedging) #### Dependencies + - Internal: Domain Layer - External: all required dependencies - ### Application layer + Implements the API of our solution, i.e. will be the access point for components/servers in the access application layer. #### Responsibility + - Implement application logic #### Examples + - Wallet methods (pay, getTransactions, …) #### Dependencies + - Internal: Domain Layer. Keep in mind that access to services implementation must be through “indirect access”, i.e. the access application layer or the wiring up code/layer must create/inject them. (_AR `TODO`: double-check this_) - External: only utility libraries that do not go against domain layer definition (Ex: lodash) - - ### Application Access layer Responsible for wiring up the other layers and/or exposing the application to external clients or consumers. @@ -212,19 +232,34 @@ Responsible for wiring up the other layers and/or exposing the application to ex This layer will have the entry points used by the infrastructure (pods). #### Responsibility + - Entrypoint for the various use-case methods defined in the Application layer #### Examples + - Cron jobs - Http servers: Middleware related logic (JWT, graphql, …) - Triggers #### Dependencies + - Internal: Domain Layer, Application Layer and Services Layer. (_AR `TODO`: Confirm that this is not just Application layer_) - External: all required dependencies required to expose the application (expressjs, apollo server, …) - - ## Spans Notes on our instrumentation approach and how this is reflected in the codebase. + + + +## Opening a Pull Request + +Before opening a PR: + +1. Make sure you are following the conventions described in this document + +2. Make sure you have successfully ran the Test suite described in the [DEV.md](./DEV.md#testing) + +3. Please pay attention to having a [clean git history](https://medium.com/@catalinaturlea/clean-git-history-a-step-by-step-guide-eefc0ad8696d) with standard commit messages. We use the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) format for our commits. + +4. It is the responsibility of the PR author to resolve merge conflicts before a merge can happen. If the PR is open for a long time a rebase may be requested. diff --git a/DEV.md b/DEV.md index 407b82f03b..56cb13cda1 100644 --- a/DEV.md +++ b/DEV.md @@ -24,6 +24,7 @@ ## Setup This setup was last tested with the following tools: + ``` $ node --version v20.10.0 @@ -44,10 +45,12 @@ To use the correct node version, you can install nvm and run `nvm use 20`. Then ### Clone the repo: ``` -$ git clone git@github.com:GaloyMoney/galoy.git -$ cd galoy +$ git clone git@github.com:lnflash/flash.git +$ cd flash ``` +*Flash is a fork of [Blink](https://github.com/GaloyMoney/blink) at commit `0a52b0673` (tag: 0.13.92)* + ### Set the Environment [direnv](https://direnv.net) is required to load environment variables. Make sure it is installed and that the [direnv hook](https://direnv.net/docs/hook.html) is added to your `shell.rc` file. @@ -64,23 +67,28 @@ $ direnv reload (...) ``` -#### Testing the ibex-webhook +#### Testing the ibex-webhook + You'll need a web gateway that forwards traffic to your local server (default http://localhost:4008). This can be done with Ngrok. After installing the ngrok cli and creating an account, do the following: 1. Start ngrok tunnel: -``` -ngrok http http://localhost:4008 -``` -2. Copy the provided URL ("forwarding" field) -3. Add the URL to your `IBEX_EXTERNAL_URI` environment variable. E.g -``` -export IBEX_EXTERNAL_URI="https://1911-104-129-24-147.ngrok-free.app" -``` -Note: To avoid repeating steps 2 & 3 everytime you restart the web gateway, you can get a static domain (e.g [ngrok domains](https://dashboard.ngrok.com/cloud-edge/domains)) and then set the `IBEX_EXTERNAL_URI` in your `.env.local` + + ``` + ngrok http http://localhost:4008 + ``` +2. Copy the provided URL ("forwarding" field) +3. Add the URL to your `IBEX_EXTERNAL_URI` environment variable. E.g + + ``` + export IBEX_EXTERNAL_URI="https://1911-104-129-24-147.ngrok-free.app" + ``` + + Note: To avoid repeating steps 2 & 3 everytime you restart the web gateway, you can get a static domain (e.g [ngrok domains](https://dashboard.ngrok.com/cloud-edge/domains)) and then set the `IBEX_EXTERNAL_URI` in your `.env.local` ### Install dependencies + ``` $ yarn install ``` @@ -92,16 +100,19 @@ $ make start-deps # or $ make reset-deps ``` + Everytime the dependencies are re-started the environment must be reloaded via `direnv reload`. When using the [make command](../Makefile) this will happen automatically. ## Development - + To start the GraphQL server and its dependencies: + ``` $ make start ``` To run in debug mode: + ``` DEBUG=* make start ``` @@ -111,6 +122,7 @@ After running `make start-deps` or `make reset-deps`, the lightning network - ru You can then login with the following credentials to get an account with an existing balance: `phone: +16505554328`, `code: 000000` ### Config + There is a sample configuration file `galoy.yaml`. This is the applications default configuration and contains settings for LND, test accounts, rate limits, fees and more. If you need to customize any of these settings you can create a `custom.yaml` file in the path `/var/yaml/custom.yaml`. This file will be merged with the default config. Here is an example of a custom.yaml file that configures fees: @@ -154,6 +166,7 @@ To run the full test suite you can run: ```bash $ make test ``` + Executing the full test suite requires [runtime dependencies](#runtime-dependencies). ### Run unit tests @@ -177,6 +190,7 @@ $ make integration ``` The integration tests are *not* fully idempotent (yet) so currently to re-run the tests, run: + ``` $ make reset-integration ``` @@ -194,6 +208,7 @@ $ TEST=utils yarn test:unit # or $ TEST=utils make unit ``` + where `utils` is the name of the file `utils.spec.ts` #### Integration @@ -207,6 +222,7 @@ $ TEST=01-connection make integration ``` if within a specific test suite you want to run/debug only a describe or it(test) block please use: + * [describe.only](https://jestjs.io/docs/api#describeonlyname-fn): just for debug purposes * [it.only](https://jestjs.io/docs/api#testonlyname-fn-timeout): just for debug purposes * [it.skip](https://jestjs.io/docs/api#testskipname-fn): use it when a test is temporarily broken. Please don't commit commented test cases @@ -217,6 +233,7 @@ if within a specific test suite you want to run/debug only a describe or it(test Migrations are stored in the `src/migrations` folder. When developing migrations the best way to test them on a clean database is: + ``` make test-migrate ``` @@ -231,6 +248,7 @@ npx migrate-mongo create \ ``` Write the migration in the newly created migration file and then test/run with the following: + ```bash # Migrate npx migrate-mongo up \ @@ -246,13 +264,18 @@ When testing, to isolate just the current migration being worked on in local dev ### Known issues * **Test suite timeouts**: increase jest timeout value. Example: + ```bash # 120 seconds $ JEST_TIMEOUT=120000 yarn test:integration ``` + * **Integration tests running slow**: we use docker to run dependencies (redis, mongodb, bitcoind and 4 lnds) so the entire test suite is disk-intensive. + * Please make sure that you are running docker containers in a solid state drive (SSD) + * Reduce lnd log disk usage: change debuglevel to critical + ``` # ./dev/lnd/lnd.conf debuglevel=critical @@ -279,6 +302,4 @@ $ yarn prettier -w . ## Contributing -When opening a PR please pay attention to having a [clean git history](https://medium.com/@catalinaturlea/clean-git-history-a-step-by-step-guide-eefc0ad8696d) with standard commit messages. We use the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) format for our commits. - -It is the responsibility of the PR author to resolve merge conflicts before a merge can happen. If the PR is open for a long time a rebase may be requested. +See the [CONTRIBUTING.md](./CONTRIBUTING.md)