Update conda env files and BuildingEnv docs

docs/buildingenv.md

@@ -3,13 +3,13 @@
 BuildingEnv considers the control of the heat flow in a multi-zone building so as to maintain a desired temperature setpoint.  Building temperature simulation uses first-principled physics models. Users can either choose from a pre-defined list of buildings (Office small, School primary, Apartment midrise, and Office large) and three climate types and cities (San Diego, Tucson, New York) provided by the Building Energy Codes Program or define a customized BuildingEnv environment by importing any self-defined EnergyPlus building models. Each episode runs for 1 day, with 5-minute time intervals ($H = 288$, $\tau = 5/60$ hours).
 
 ## Observation Space
-For a building with $M$ indoor zones, the state $s(t) \in \R^{3M+2}$ contains observable properties of the building environment at timestep $t$:
+For a building with $M$ indoor zones, the state $s(t) \in \R^{M+4}$ contains observable properties of the building environment at timestep $t$:
 
 $$
-s(t) = (T_1(t), ...,T_{M}(t), N_1(t), ..., N_{M}(t), Q_{1}^{GHI}(t), ..., Q_{M}^{GHI}(t), T_G(t), T_{E}(t)),
+    s(t) = (T_1(t), \dotsc, T_M(t), T_\mathrm{E}(t), T_\mathrm{G}(t), Q^\mathrm{GHI}(t), \bar{Q}^\mathrm{p}(t)),
 $$
 
-where $T_i(t)$ is zone $i$'s temperature at time step $t$, $N_i(t)$ is the number of occupants, $Q_{i}^{GHI}(t)$ is the heat gain from the solar irradiance, and $T_G(t)$ and $T_E(t)$ denote the ground and outdoor environment temperature. In practice, the agent may have access to all or part of the state variables for decision-making depending on the sensor setup. Note that the outdoor/ground temperature, room occupancy, and heat gain from solar radiance are time-varying uncontrolled variables from the environment.
+where $T_i(t)$ is zone $i$'s temperature at time step $t$, $\bar{Q}^\mathrm{p}(t)$ is the heat acquisition from occupant's activities, $Q^\mathrm{GHI}(t)$ is the heat gain from the solar irradiance, and $T_\mathrm{G}(t)$ and $T_\mathrm{E}(t)$ denote the ground and outdoor environment temperature. In practice, the agent may have access to all or part of the state variables for decision-making depending on the sensor setup. Note that the outdoor/ground temperature, room occupancy, and heat gain from solar radiance are time-varying uncontrolled variables from the environment.
 
 ## Action Space
 The action $a(t) \in [-1, 1]^M$ sets the controlled heating supplied to each of the $M$ zones, scaled to $[-1, 1]$.
@@ -18,13 +18,13 @@ The action $a(t) \in [-1, 1]^M$ sets the controlled heating supplied to each of
 The objective is to reduce energy consumption while keeping the temperature within a given comfort range. The default reward function is a weighted $\ell_2$ reward, defined as
 
 $$
-    r(t) = - (1-\beta) \|a(t)\|_2 - \beta \|T^{obj}(t)-T(t)\|_2
+    r(t) = - (1-\beta) \|a(t)\|_2 - \beta \|T^\mathrm{target}(t)-T(t)\|_2
 $$
 
-where $T^{obj}(t)=[T^{obj}_{1}(t),...,T^{obj}_{M}(t)]$ are the target temperatures and $T(t)=[T_{1}(t),...,T_{M}(t)]$ are the actual zonal temperature. BuildingEnv also allows users to customize reward functions using environment states $s(t)$, actions $a(t)$, target values $T^{obj}(t)$, and a weight term $\beta$. Users can also customize the reward function to take CO<sub>2</sub> emissions into consideration.
+where $T^\mathrm{target}(t)=[T^\mathrm{target}_{1}(t),\cdots,T^\mathrm{target}_{M}(t)]$ are the target temperatures and $T(t)=[T_1(t),\cdots,T_M(t)]$ are the actual zonal temperature. BuildingEnv also allows users to customize reward functions by changing the weight term $\beta$ or the parameter $p$ defining the $\ell_p$ norm. Users can also customize the reward function to take CO<sub>2</sub> emissions into consideration.
 
 ## Distribution Shift
-BuildingEnv features distribution shifts in the ambient outdoor temperature profile $T_E$ which varies with different seasons.
+BuildingEnv features distribution shifts in the ambient outdoor temperature profile $T_\mathrm{E}$ which varies with different seasons.
 
 ## Multiagent Setting
 In the multiagent setting for BuildingEnv, we treat each building as an independent agent whose action is the building's heat control decisions. It must coordinate with other building agents to maximize overall reward, which is the summation of each agent's reward. Each agent obtains either the global observation or individual building states.
@@ -33,11 +33,25 @@ In the multiagent setting for BuildingEnv, we treat each building as an independ
 
 ### Installation
 
-Coming soon!
+SustainGym is designed for Linux machines. SustainGym is hosted on [PyPI](https://pypi.org/project/sustaingym/) and can be installed with `pip`:
+
+```bash
+pip install sustaingym[building]
+```
 
 ### Using our training script
 
-Coming soon!
+1. Install [miniconda3](https://docs.conda.io/en/latest/miniconda-other-installer-links.html).
+2. (Optional, but recommended) If you are using a conda version `<=23.9.0`, set the conda solver to libmamba for faster dependency solving. Starting from conda version [`23.10.0`](https://github.com/conda/conda/releases/tag/23.10.0), libmamba is the default solver.
+    ```bash
+    conda config --set solver libmamba
+    ```
+3. Install the libraries necessary for runnning the BuildingEnv environment.
+    ```bash
+    conda env update --file env_building.yml --prune
+    ```
+
+More instructions coming soon!
 
 ### Custom RL Loop
 

docs/evchargingenv.md

@@ -92,4 +92,4 @@ optional arguments:
 
 ## References
 
-[1] Z. J. Lee et al., "Adaptive Charging Networks: A Framework for Smart Electric Vehicle Charging," in _IEEE Transactions on Smart Grid_, vol. 12, no. 5, pp. 4339-4350, Sept. 2021, doi: 10.1109/TSG.2021.3074437. URL [https://ieeexplore.ieee.org/document/9409126](https://ieeexplore.ieee.org/document/9409126).
+[1] Z. J. Lee et al., "Adaptive Charging Networks: A Framework for Smart Electric Vehicle Charging," in _IEEE Transactions on Smart Grid_, vol. 12, no. 5, pp. 4339-4350, Sept. 2021, doi: 10.1109/TSG.2021.3074437. URL [https://ieeexplore.ieee.org/document/9409126](https://ieeexplore.ieee.org/document/9409126).

env_cogen.yml

@@ -5,19 +5,13 @@
 # conda remove --name sustaingym_cogen --all
 #
 # Notes
-# - TensorFlow 2.14:
-#   - the GPU version only works with Python <=3.10 (see https://github.com/tensorflow/tensorflow/issues/61986)
-#   - TensorFlow 2.15 should fix this issue
 # - Ray 2.8:
 #   - officially only supports up to Python 3.10 (see https://docs.ray.io/en/latest/ray-overview/installation.html)
 #   - only supports gymnasium 0.28.1 (see https://github.com/ray-project/ray/blob/ray-2.8.0/python/setup.py#L305)
 #   - officially seems to only supports pettingzoo 1.23.1 (see https://github.com/ray-project/ray/blob/ray-2.8.0/python/requirements/ml/rllib-test-requirements.txt),
 #     but empirically seems to work with pettingzoo 1.24.*
-# - Onnxruntime 1.16:
-#   - pre-built pip package only supports up to CUDA 11.8 (see https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)
-#   - PyTorch 2.1 supports both CUDA 11.8 and 12.1
 #
-# last updated: November 13, 2023
+# last updated: November 16, 2023
 name: sustaingym_cogen
 channels:
 - pytorch           # for pytorch
@@ -47,11 +41,10 @@ dependencies:
   - gymnasium==0.28.1
   - pettingzoo==1.24.1
   - ray[rllib]==2.8.*
+  - onnxruntime==1.16.*    # the ONNX model for CogenEnv is small and runs sufficiently fast on CPU
 
   # uncomment for CPU-only
-  # - onnxruntime==1.16.*
-  # - tensorflow==2.14.*
+  # - tensorflow==2.15.*
 
   # for GPU. comment out for CPU-only.
-  - onnxruntime-gpu==1.16.*
-  - tensorflow[and-cuda]==2.14.*
+  - tensorflow[and-cuda]==2.15.*

env_ev.yml

@@ -10,16 +10,13 @@
 #      and Pandas 1.1 only supports up to Python 3.9
 #   - we install from the ACNPortal dev branch, which includes a fix that supports Pandas 2
 #
-# - TensorFlow 2.14:
-#   - the GPU version only works with Python <=3.10 (see https://github.com/tensorflow/tensorflow/issues/61986)
-#   - TensorFlow 2.15 should fix this issue
 # - Ray 2.8:
 #   - officially only supports up to Python 3.10 (see https://docs.ray.io/en/latest/ray-overview/installation.html)
 #   - only supports gymnasium 0.28.1 (see https://github.com/ray-project/ray/blob/ray-2.8.0/python/setup.py#L305)
 #   - officially seems to only supports pettingzoo 1.23.1 (see https://github.com/ray-project/ray/blob/ray-2.8.0/python/requirements/ml/rllib-test-requirements.txt),
 #     but empirically seems to work with pettingzoo 1.24.*
 #
-# last updated: November 13, 2023
+# last updated: November 16, 2023
 name: sustaingym_ev
 channels:
 - pytorch           # for pytorch
@@ -57,7 +54,7 @@ dependencies:
   - stable_baselines3>=2
 
   # uncomment for CPU-only
-  # - tensorflow==2.14.*
+  # - tensorflow==2.15.*
 
   # for GPU. comment out for CPU-only.
-  - tensorflow[and-cuda]==2.14.*
+  - tensorflow[and-cuda]==2.15.*