Skip to content

Commit

Permalink
Merge pull request #3 from wowsims/fix/refactor-sod-build
Browse files Browse the repository at this point in the history
Fix/refactor sod build
  • Loading branch information
ericbtaylorjr authored Nov 20, 2024
2 parents 3a451a9 + 35e1583 commit 3d057a9
Show file tree
Hide file tree
Showing 588 changed files with 9,780 additions and 10,687 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:

- name: Build 🔧
run: |
make dist/sod/.dirstamp
make dist/classic/.dirstamp
- name: Test
run: |
Expand All @@ -41,5 +41,5 @@ jobs:
uses: JamesIves/[email protected]
with:
branch: gh-pages
folder: dist/sod
folder: dist/classic
single-commit: true
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ jobs:
artifacts: |
wowsimcli-amd64-linux.zip
wowsimcli-windows.exe.zip
wowsimsod-amd64-darwin.zip
wowsimsod-arm64-darwin.zip
wowsimsod-amd64-linux.zip
wowsimsod-windows.exe.zip
wowsimclassic-amd64-darwin.zip
wowsimclassic-arm64-darwin.zip
wowsimclassic-amd64-linux.zip
wowsimclassic-windows.exe.zip
2 changes: 1 addition & 1 deletion .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:

- name: Build 🔧
run: |
make dist/sod/.dirstamp
make dist/classic/.dirstamp
- name: Test
run: |
Expand Down
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,21 @@ ui/*/index.html
dist
binary_dist
sim/web/__debug_bin
/wowsimclassiccli*
/wowsimclassic*
/wowsimsodcli*
/wowsimsod*
chromedriver*
google-chrome*

# temporary files
*.results.tmp

#.dockerignore
/classic/
/sod/
*.code-workspace
wowsimclassic
wowsimsod

# old result file
Expand All @@ -34,6 +39,9 @@ TestAPL.results
tmp/

# lib
wowsimclassic*.so
wowsimclassic*.dll
wowsimclassic*.h
wowsimsod*.so
wowsimsod*.dll
wowsimsod*.h
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

FROM golang:1.21

WORKDIR /sod
WORKDIR /classic
COPY . .
COPY gitconfig /etc/gitconfig

Expand Down
50 changes: 25 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ The primary goal of this project is to provide a framework that makes it easy to

This project is licensed with MIT license. We request that anyone using this software in their own project to make sure there is a user visible link back to the original project.

[Live sims can be found here.](https://wowsims.github.io/sod "https://wowsims.github.io/sod")
[Live sims can be found here.](https://wowsims.github.io/classic "https://wowsims.github.io/classic")

[Support our devs via Patreon.](https://www.patreon.com/wowsims)

# Downloading Sim

Links for latest Sim build:
- [Windows Sim](https://github.com/wowsims/sod/releases/latest/download/wowsimsod-windows.exe.zip)
- [MacOS Sim](https://github.com/wowsims/sod/releases/latest/download/wowsimsod-amd64-darwin.zip)
- [Linux Sim](https://github.com/wowsims/sod/releases/latest/download/wowsimsod-amd64-linux.zip)
- [Windows Sim](https://github.com/wowsims/classic/releases/latest/download/wowsimclassic-windows.exe.zip)
- [MacOS Sim](https://github.com/wowsims/classic/releases/latest/download/wowsimclassic-amd64-darwin.zip)
- [Linux Sim](https://github.com/wowsims/classic/releases/latest/download/wowsimclassic-amd64-linux.zip)

Then unzip the downloaded file, then open the unzipped file to open the sim in your browser!

Alternatively, you can choose from a specific relase on the [Releases](https://github.com/wowsims/sod/releases) page and click the suitable link under "Assets"
Alternatively, you can choose from a specific relase on the [Releases](https://github.com/wowsims/classic/releases) page and click the suitable link under "Assets"
# Local Dev Installation

This project has dependencies on Go >=1.21, protobuf-compiler and the corresponding Go plugins, and node >= 14.0.
Expand All @@ -35,7 +35,7 @@ echo 'export GOPATH=$HOME/go' >> $HOME/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin' >> $HOME/.bashrc
source $HOME/.bashrc

cd sod
cd classic

# Install protobuf compiler and Go plugins
sudo apt update && sudo apt upgrade
Expand All @@ -54,29 +54,29 @@ npm install
## Docker
Alternatively, install Docker and your workflow will look something like this:
```sh
git clone https://github.com/wowsims/sod.git
cd sod
git clone https://github.com/wowsims/classic.git
cd classic

# Build the docker image and install npm dependencies (only need to run these once).
docker build --tag wowsims-sod .
docker run --rm -v $(pwd):/sod wowsims-sod npm install
docker build --tag wowsims-classic .
docker run --rm -v $(pwd):/classic wowsims-classic npm install

# Now you can run the commands as shown in the Commands sections, preceding everything with, "docker run --rm -it -p 8080:8080 -v $(pwd):/sod wowsims-sod".
# Now you can run the commands as shown in the Commands sections, preceding everything with, "docker run --rm -it -p 8080:8080 -v $(pwd):/classic wowsims-classic".
# For convenience, set this as an environment variable:
SOD_CMD="docker run --rm -it -p 8080:8080 -v $(pwd):/sod wowsims-sod"
CLASSIC_CMD="docker run --rm -it -p 8080:8080 -v $(pwd):/classic wowsims-classic"

#For the watch commands assign this environment variable:
SOD_WATCH_CMD="docker run --rm -it -p 8080:8080 -p 3333:3333 -p 5173:5173 -e WATCH=1 -v $(pwd):/sod wowsims-sod"
CLASSIC_WATCH_CMD="docker run --rm -it -p 8080:8080 -p 3333:3333 -p 5173:5173 -e WATCH=1 -v $(pwd):/classic wowsims-classic"

# ... do some coding on the sim ...

# Run tests
$(echo $SOD_CMD) make test
$(echo $CLASSIC_CMD) make test

# ... do some coding on the UI ...

# Host a local site
$(echo $SOD_CMD) make host
$(echo $CLASSIC_CMD) make host
```

## Windows
Expand All @@ -103,8 +103,8 @@ make test
make update-tests

# Host a local version of the UI at http://localhost:8080. Visit it by pointing a browser to
# http://localhost:8080/sod/YOUR_SPEC_HERE, where YOUR_SPEC_HERE is the directory under ui/ with your custom code.
# Recompiles the entire client before launching using `make dist/sod`
# http://localhost:8080/classic/YOUR_SPEC_HERE, where YOUR_SPEC_HERE is the directory under ui/ with your custom code.
# Recompiles the entire client before launching using `make dist/classic`
make host

# With file-watching so the server auto-restarts and recompiles on Go or TS changes:
Expand All @@ -116,7 +116,7 @@ make clean
# Recompiles the ts only for the given spec (e.g. make host_elemental_shaman)
make host_$spec

# Recompiles the `wowsimsod` server binary and runs it, hosting /dist directory at http://localhost:3333/sod.
# Recompiles the `wowsimclassic` server binary and runs it, hosting /dist directory at http://localhost:3333/classic.
# This is the fastest way to iterate on core go simulator code so you don't have to wait for client rebuilds.
# To rebuild client for a spec just do 'make $spec' and refresh browser.
make rundevserver
Expand All @@ -125,7 +125,7 @@ make rundevserver
WATCH=1 make rundevserver


# The same as rundevserver, recompiles `wowsimsod` binary and runs it on port 3333. Instead of serving content from the dist folder,
# The same as rundevserver, recompiles `wowsimclassic` binary and runs it on port 3333. Instead of serving content from the dist folder,
# this command also runs `vite serve` to start the Vite dev server on port 5173 (or similar) and automatically reloads the page on .ts changes in less than a second.
# This allows for more rapid development, with sub second reloads on TS changes. This combines the benefits of `WATCH=1 make rundevserver` and `WATCH=1 make host`
# to create something that allows you to work in any part of the code with ease and speed.
Expand All @@ -142,16 +142,16 @@ make webworkers
# With file watch enabled
WATCH=1 make webworkers

# Creates the 'wowsimsod' binary that can host the UI and run simulations natively (instead of with wasm).
# Creates the 'wowsimclassic' binary that can host the UI and run simulations natively (instead of with wasm).
# Builds the UI and the compiles it into the binary so that you can host the sim as a server instead of wasm on the client.
# It does this by first doing make dist/sod and then copying all those files to binary_dist/sod and loading all the files in that directory into its binary on compile.
make wowsimsod
# It does this by first doing make dist/classic and then copying all those files to binary_dist/classic and loading all the files in that directory into its binary on compile.
make wowsimclassic

# Using the --usefs flag will instead of hosting the client built into the binary, it will host whatever code is found in the /dist directory.
# Use --wasm to host the client with the wasm simulator.
# The server also disables all caching so that refreshes should pickup any changed files in dist/. The client will still call to the server to run simulations so you can iterate more quickly on client changes.
# make dist/sod && ./wowsimsod --usefs would rebuild the whole client and host it. (you would have had to run `make devserver` to build the wowsimsod binary first.)
./wowsimsod --usefs
# make dist/classic && ./wowsimclassic --usefs would rebuild the whole client and host it. (you would have had to run `make devserver` to build the wowsimclassic binary first.)
./wowsimclassic --usefs

# Generate code for items. Only necessary if you changed the items generator.
make items
Expand Down Expand Up @@ -185,7 +185,7 @@ The UI and sim can be done in either order, but it is generally recommended to b

No .html is needed, it will be generated based on `ui/index_template.html` and the `$SPEC` name.

When you're ready to try out the site, run `make host` and navigate to `http://localhost:8080/sod/$SPEC`.
When you're ready to try out the site, run `make host` and navigate to `http://localhost:8080/classic/$SPEC`.

## Implement the Sim
This step is where most of the magic happens. A few highlights to start understanding the sim code:
Expand Down
2 changes: 1 addition & 1 deletion assets/database/loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package database
import (
_ "embed"

"github.com/wowsims/sod/sim/core/proto"
"github.com/wowsims/classic/sim/core/proto"
googleProto "google.golang.org/protobuf/proto"
)

Expand Down
4 changes: 2 additions & 2 deletions cmd/wowsimcli/cli_main.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package main

import (
"github.com/wowsims/sod/cmd/wowsimcli/cmd"
"github.com/wowsims/sod/sim"
"github.com/wowsims/classic/cmd/wowsimcli/cmd"
"github.com/wowsims/classic/sim"
)

func init() {
Expand Down
4 changes: 2 additions & 2 deletions cmd/wowsimcli/cmd/basic_sim.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"os"

"github.com/spf13/cobra"
"github.com/wowsims/sod/sim/core"
"github.com/wowsims/sod/sim/core/proto"
"github.com/wowsims/classic/sim/core"
"github.com/wowsims/classic/sim/core/proto"
"google.golang.org/protobuf/encoding/protojson"
)

Expand Down
4 changes: 2 additions & 2 deletions cmd/wowsimcli/cmd/bulk_replace.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
"time"

"github.com/spf13/cobra"
"github.com/wowsims/sod/sim/core"
"github.com/wowsims/sod/sim/core/proto"
"github.com/wowsims/classic/sim/core"
"github.com/wowsims/classic/sim/core/proto"
"google.golang.org/protobuf/encoding/protojson"
)

Expand Down
2 changes: 1 addition & 1 deletion cmd/wowsimcli/cmd/decode_link.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"strings"

"github.com/spf13/cobra"
"github.com/wowsims/sod/sim/core/proto"
"github.com/wowsims/classic/sim/core/proto"
"google.golang.org/protobuf/encoding/protojson"
goproto "google.golang.org/protobuf/proto"
)
Expand Down
33 changes: 16 additions & 17 deletions docs/database_updates.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
# Database Updates

This doc outlines the various methods for updating database data in WoWSims Season of Discovery.
This doc outlines the various methods for updating database data in WoWSims Classic.

## Introduction

WoWSims uses a number of JSON database files to store our data.
These can be found in [assets/database/](https://github.com/wowsims/sod/tree/master/assets/database):
These can be found in [assets/database/](https://github.com/wowsims/classic/tree/master/assets/database):

- [db.json](https://github.com/wowsims/sod/blob/master/assets/database/db.json) - Our primary database file. This is where we store the majority of our item, spell, and other data used in the sims.
- [leftover_db.json](https://github.com/wowsims/sod/blob/master/assets/database/leftover_db.json) - This is a leftover database file where items filtered out for having invalid data (determined in `main.go`) are placed.
- [db.json](https://github.com/wowsims/classic/blob/master/assets/database/db.json) - Our primary database file. This is where we store the majority of our item, spell, and other data used in the sims.
- [leftover_db.json](https://github.com/wowsims/classic/blob/master/assets/database/leftover_db.json) - This is a leftover database file where items filtered out for having invalid data (determined in `main.go`) are placed.

We also store scraped data from external sources like Wowhead, Atlasloot, etc. in order to build database data in the first place.
These can be found in [assets/db_inputs/](https://github.com/wowsims/sod/tree/master/assets/db_inputs):
These can be found in [assets/db_inputs/](https://github.com/wowsims/classic/tree/master/assets/db_inputs):

- [atlasloot_db.json](https://github.com/wowsims/sod/blob/master/assets/db_inputs/atlasloot_db.json) - Scraped data from the AtlasLoot addon's GitHub repo. This is primarily used for item source data and faction restrictions.
- [wago_db2_items.csv](https://github.com/wowsims/sod/blob/master/assets/db_inputs/wago_db2_items.csv) - Scraped item data from Wago DB. This is currently used for additional item set and faction restriction information.
- [wowhead_gearplannerdb.txt](https://github.com/wowsims/sod/blob/master/assets/db_inputs/wowhead_gearplannerdb.txt) - Scraped item data from Wowhead Gear Planner. This is one of our primary sources of item data along with Wowhead item tooltips.
- [wowhead_item_tooltips.csv](https://github.com/wowsims/sod/blob/master/assets/db_inputs/wowhead_item_tooltips.csv) - Scraped item tooltips from Wowhead. We store the full tooltip along with the item IDs.
- [wowhead_rune_tooltips.csv](https://github.com/wowsims/sod/blob/master/assets/db_inputs/wowhead_rune_tooltips.csv) - Scraped rune tooltips from Wowhead. We store the full tooltip along with the rune engraving spell IDs.
- [wowhead_spell_tooltips.csv](https://github.com/wowsims/sod/blob/master/assets/db_inputs/wowhead_spell_tooltips.csv) - Scraped spell tooltips from Wowhead. We store the full tooltip along with the spell IDs.
- [atlasloot_db.json](https://github.com/wowsims/classic/blob/master/assets/db_inputs/atlasloot_db.json) - Scraped data from the AtlasLoot addon's GitHub repo. This is primarily used for item source data and faction restrictions.
- [wago_db2_items.csv](https://github.com/wowsims/classic/blob/master/assets/db_inputs/wago_db2_items.csv) - Scraped item data from Wago DB. This is currently used for additional item set and faction restriction information.
- [wowhead_gearplannerdb.txt](https://github.com/wowsims/classic/blob/master/assets/db_inputs/wowhead_gearplannerdb.txt) - Scraped item data from Wowhead Gear Planner. This is one of our primary sources of item data along with Wowhead item tooltips.
- [wowhead_item_tooltips.csv](https://github.com/wowsims/classic/blob/master/assets/db_inputs/wowhead_item_tooltips.csv) - Scraped item tooltips from Wowhead. We store the full tooltip along with the item IDs.
- [wowhead_rune_tooltips.csv](https://github.com/wowsims/classic/blob/master/assets/db_inputs/wowhead_rune_tooltips.csv) - Scraped rune tooltips from Wowhead. We store the full tooltip along with the rune engraving spell IDs.
- [wowhead_spell_tooltips.csv](https://github.com/wowsims/classic/blob/master/assets/db_inputs/wowhead_spell_tooltips.csv) - Scraped spell tooltips from Wowhead. We store the full tooltip along with the spell IDs.

The entry point for running database scripts is [tools/database/gen_db/main.go](https://github.com/wowsims/sod/blob/master/tools/database/gen_db/main.go).
The entry point for running database scripts is [tools/database/gen_db/main.go](https://github.com/wowsims/classic/blob/master/tools/database/gen_db/main.go).
This file is executed by running the `make items` command, or by running one of a set of commands listed in the comments of `gen_db/main.go`.

- `go run ./tools/database/gen_db -outDir=assets -gen=atlasloot`
Expand All @@ -45,16 +45,15 @@ After performing any of these commands, you should then use `make items` in orde
In addition to our db inputs, we can also define overrides for both adding and removing data.
We have three different overrides files:

- [tools/database/overrides.go]https://github.com/wowsims/sod/blob/master/tools/database/overrides.go has several different types of overrides for items, including allowlists and denylists.
- [tools/database/enchant_overrides.go](https://github.com/wowsims/sod/blob/master/tools/database/enchant_overrides.go) has all of our enchant data and is where new enchant entries should be added
- [tools/database/rune_overrides.go](https://github.com/wowsims/sod/blob/master/tools/database/rune_overrides.go) has overrides related to runes, including overrides that fix some rune information scraped from wowhead, and a way to block runes from the sim until they can be implemented.
- [tools/database/overrides.go]https://github.com/wowsims/classic/blob/master/tools/database/overrides.go has several different types of overrides for items, including allowlists and denylists.
- [tools/database/enchant_overrides.go](https://github.com/wowsims/classic/blob/master/tools/database/enchant_overrides.go) has all of our enchant data and is where new enchant entries should be added
- [tools/database/rune_overrides.go](https://github.com/wowsims/classic/blob/master/tools/database/rune_overrides.go) has overrides related to runes, including overrides that fix some rune information scraped from wowhead, and a way to block runes from the sim until they can be implemented.

These overrides should be used sparingly when possible, but are often a necessary part of filtering data in the sim's database.

## Updating the Item Database

With the frequent patch cadence of SoD we often need to pull in new and updated item data.
This typically involves performing several commands to update the multiple sources of item data.
This involves performing several commands to update the multiple sources of item data.

1. Run the Wowhead Gear Planner scraper
2. Run the Wowhead Item Tooltip scraper with the range of IDs you want to add/update, deleting any existing entries in that range if there are any.
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/wowsims/sod
module github.com/wowsims/classic

go 1.21

Expand Down
Loading

0 comments on commit 3d057a9

Please sign in to comment.