Merge branch 'staging' into fix-history-panel-colors

HowToRun_vLLM_Models.md

@@ -79,7 +79,23 @@ HF_TOKEN=hf_********
 
 ## 4. Run the Setup Script (vLLM Llama3.1-70B only)
 
-Navigate to the `model` folder within the `tt-inference-server` and run the automated setup script. You can find step-by-step instructions [here](https://github.com/tenstorrent/tt-inference-server/tree/main/vllm-tt-metal-llama3-70b#5-automated-setup-environment-variables-and-weights-files:~:text=70b/docs/development-,5.%20Automated%20Setup%3A%20environment%20variables%20and%20weights%20files,-The%20script%20vllm).
+Follow these step-by-step instructions for a smooth automated process of model weights setup.
+
+1. **Navigate to the `vllm-tt-metal-llama3-70b/` folder** within the `tt-inference-server`. This folder contains the necessary files and scripts for model setup.
+
+2. **Run the automated setup script** as outlined in the [official documentation](https://github.com/tenstorrent/tt-inference-server/tree/main/vllm-tt-metal-llama3-70b#5-automated-setup-environment-variables-and-weights-files:~:text=70b/docs/development-,5.%20Automated%20Setup%3A%20environment%20variables%20and%20weights%20files,-The%20script%20vllm). This script handles key steps such as configuring environment variables, downloading weight files, repacking weights, and creating directories.
+
+**Note** During the setup process, you will see the following prompt:
+
+   ```
+   Enter your PERSISTENT_VOLUME_ROOT [default: tt-inference-server/tt_inference_server_persistent_volume]:
+   ```
+
+   **Do not accept the default path.** Instead, set the persistent volume path to `tt-studio/tt_studio_persistent_volume`. This ensures the configuration matches TT-Studio’s directory structure. Using the default path may result in incorrect configuration.
+
+By following these instructions, you will have a properly configured model infrastructure, ready for inference and further development.
+
+
 
 ---
 
@@ -112,18 +128,24 @@ During the model weights download process, an `.env` file will be automatically
 /path/to/tt-inference-server/vllm-tt-metal-llama3-70b/.env
 ```
 
-To ensure the model can be deployed via the TT-Studio GUI, copy this `.env` file to the model's persistent storage location. For example:
+To ensure the model can be deployed via the TT-Studio GUI, this `.env` file must be copied to the model's persistent storage location. For example:
 
 ```bash
 /path/to/tt_studio_persistent_volume/volume_id_tt-metal-llama-3.1-70b-instructv0.0.1/copied_env
 ```
 
-Following command can be used as a reference (*replace paths as necessary*):
+The following command can be used as a reference (*replace paths as necessary*):
 
 ```bash
-cp /$USR/tt-inference-server/vllm-tt-metal-llama3-70b/.env /$USR/tt_studio/tt_studio_persistent_volume/volume_id_tt-metal-llama-3.1-70b-instructv0.0.1/**copied_env
+sudo cp /$USR/tt-inference-server/vllm-tt-metal-llama3-70b/.env /$USR/tt_studio/tt_studio_persistent_volume/volume_id_tt-metal-llama-3.1-70b-instructv0.0.1/.env
 ```
 
+### Step 2: Point to the Copied Environment File
+The `VLLM_LLAMA31_ENV_FILE` variable within the TT-Studio `$USR/tt-studio/app/.env` file must point to *this* copied `.env` file. This should be a **relative path**, for example it can be set as follows:
+
+```
+VLLM_LLAMA31_ENV_FILE="/tt_studio_persistent_volume/volume_id_tt-metal-llama-3.1-70b-instructv0.0.1/.env"
+```
 ---
 
 ### Step 2: Update the TT-Studio Environment File

app/api/README.md

@@ -56,7 +56,7 @@ example JSON response:
             ]
         },
         "networks": {
-            "llm_studio_network": {
+            "tt_studio_network": {
                 "DNSNames": [
                     "dummy_echo_model_p8013",
                     "1d1a274a7126"

app/api/shared_config/backend_config.py

@@ -32,7 +32,7 @@ class BackendConfig:
         "backend_volume"
     ),
     django_deploy_cache_name="deploy_cache",
-    docker_bridge_network_name="llm_studio_network",
+    docker_bridge_network_name="tt_studio_network",
     weights_dir="model_weights",
     model_container_cache_root="/home/user/cache_root",
     jwt_secret=os.environ["JWT_SECRET"],

app/docker-compose.yml

@@ -16,7 +16,7 @@ services:
 
     # note that `network_mode: host` does not work on mac OS
     networks:
-      - llm_studio_network
+      - tt_studio_network
     ports:
       - "8000:8000"
     # command: bash
@@ -66,7 +66,7 @@ services:
       tt_studio_backend:
         condition: service_healthy
     networks:
-      - llm_studio_network
+      - tt_studio_network
     ports:
       - "3000:3000"
     # volumes:
@@ -100,12 +100,12 @@ services:
     container_name: tt_studio_chroma
 
     networks:
-      - llm_studio_network
+      - tt_studio_network
 
 networks:
-  llm_studio_network:
+  tt_studio_network:
     # need external flag to allow for the backend to manage the docker network
     # otherwise, docker compose will create an app_* network for the backend container
     # to avoid colliding existing docker networks
     external: true
-    name: llm_studio_network
+    name: tt_studio_network

startup.sh

@@ -17,7 +17,7 @@ usage() {
     echo -e "This script sets up the TT-Studio environment by performing the following steps:"
     echo -e "  1. 🧭 Detects the OS."
     echo -e "  2. 🛠️  Sets the TT_STUDIO_ROOT variable in .env based on the running directory."
-    echo -e "  3. 🌐 Checks for and creates a Docker network named 'llm_studio_network' if not present."
+    echo -e "  3. 🌐 Checks for and creates a Docker network named 'tt_studio_network' if not present."
     echo -e "  4. 🚀 Runs Docker Compose to start the TT Studio services."
     echo
     echo -e "Options:"
@@ -160,7 +160,7 @@ fi
 source "${ENV_FILE_PATH}"
 
 # Step 3: Check if the Docker network already exists
-NETWORK_NAME="llm_studio_network"
+NETWORK_NAME="tt_studio_network"
 if docker network ls | grep -qw "${NETWORK_NAME}"; then
     echo "Network '${NETWORK_NAME}' exists."
 else