Instructions to run KG_RAG on mac

.devcontainer/Dockerfile

@@ -0,0 +1,11 @@
+FROM mcr.microsoft.com/devcontainers/anaconda:0-3
+
+# Copy environment.yml (if found) to a temp location so we update the environment. Also
+# copy "noop.txt" so the COPY instruction does not fail if no environment.yml exists.
+COPY environment.yml* .devcontainer/noop.txt /tmp/conda-tmp/
+RUN if [ -f "/tmp/conda-tmp/environment.yml" ]; then umask 0002 && /opt/conda/bin/conda env update -n base -f /tmp/conda-tmp/environment.yml; fi \
+    && rm -rf /tmp/conda-tmp
+
+# [Optional] Uncomment this section to install additional OS packages.
+# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
+#     && apt-get -y install --no-install-recommends <your-package-list-here>

.devcontainer/devcontainer.json

@@ -0,0 +1,24 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda
+{
+	"name": "Anaconda (Python 3)",
+	"build": { 
+		"context": "..",
+		"dockerfile": "Dockerfile"
+	}
+
+	// Features to add to the dev container. More info: https://containers.dev/features.
+	// "features": {},
+
+	// Use 'forwardPorts' to make a list of ports inside the container available locally.
+	// "forwardPorts": [],
+
+	// Use 'postCreateCommand' to run commands after the container is created.
+	// "postCreateCommand": "python --version",
+
+	// Configure tool-specific properties.
+	// "customizations": {},
+
+	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+	// "remoteUser": "root"
+}

.devcontainer/noop.txt

@@ -0,0 +1,3 @@
+This file copied into the container along with environment.yml* from the parent
+folder. This file is included to prevents the Dockerfile COPY instruction from 
+failing if no environment.yml is found.

.github/dependabot.yml

@@ -0,0 +1,12 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for more information:
+# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+# https://containers.dev/guide/dependabot
+
+version: 2
+updates:
+ - package-ecosystem: "devcontainers"
+   directory: "/"
+   schedule:
+     interval: weekly

.gpt_config.env

@@ -0,0 +1,8 @@
+# Uncomment the following 3 lines and add the azure API_KEY, RESOURCE_ENDPOINT and API_VERSION, if using GPT_API_TYPE="azure" in the config.yaml file. 
+# API_KEY=<API_KEY>
+# RESOURCE_ENDPOINT=<RESOURCE_ENDPOINT>
+# API_VERSION=<API_VERSION> # Can default to "2024-02-01"
+
+
+# Uncomment the following and add the openai api key, if using GPT_API_TYPE="open_ai" in the config.yaml file. Make sure to comment out the variables for azure endpoints above.
+# API_KEY=<OPENAI_API_KEY>

README.md

@@ -15,11 +15,12 @@
 
 [How to run KG-RAG](https://github.com/BaranziniLab/KG_RAG#how-to-run-kg-rag)
  - [Step 1: Clone the repo](https://github.com/BaranziniLab/KG_RAG#step-1-clone-the-repo)
- - [Step 2: Create a virtual environment](https://github.com/BaranziniLab/KG_RAG#step-2-create-a-virtual-environment)
- - [Step 3: Install dependencies](https://github.com/BaranziniLab/KG_RAG#step-3-install-dependencies)
- - [Step 4: Update config.yaml](https://github.com/BaranziniLab/KG_RAG#step-4-update-configyaml)
- - [Step 5: Run the setup script](https://github.com/BaranziniLab/KG_RAG#step-5-run-the-setup-script)
- - [Step 6: Run KG-RAG from your terminal](https://github.com/BaranziniLab/KG_RAG#step-6-run-kg-rag-from-your-terminal)
+ - [Step 2: Setup Dev Container](https://github.com/niraj17singh/KG_RAG#step-2-setup-devcontainers) 
+ - [Step 3: Create a virtual environment](https://github.com/BaranziniLab/KG_RAG#step-2-create-a-virtual-environment)
+ - [Step 4: Install dependencies](https://github.com/BaranziniLab/KG_RAG#step-3-install-dependencies)
+ - [Step 5: Update config.yaml](https://github.com/BaranziniLab/KG_RAG#step-4-update-configyaml)
+ - [Step 6: Run the setup script](https://github.com/BaranziniLab/KG_RAG#step-5-run-the-setup-script)
+ - [Step 7: Run KG-RAG from your terminal](https://github.com/BaranziniLab/KG_RAG#step-6-run-kg-rag-from-your-terminal)
     - [Using GPT](https://github.com/BaranziniLab/KG_RAG#using-gpt)
     - [Using GPT interactive mode](https://github.com/BaranziniLab/KG_RAG/blob/main/README.md#using-gpt-interactive-mode)
     - [Using Llama](https://github.com/BaranziniLab/KG_RAG#using-llama)
@@ -75,31 +76,39 @@ You can see that, KG-RAG was able to give the correct information about the FDA
 
 **Note: At the moment, KG-RAG is specifically designed for running prompts related to Diseases. We are actively working on improving its versatility.**
 
+
 ### Step 1: Clone the repo
 
 Clone this repository. All Biomedical data used in the paper are uploaded to this repository, hence you don't have to download that separately.
 
-### Step 2: Create a virtual environment
+### Step 2: Setup Devcontainers
+
+Setup dev containers to create a linux environment to run your code if using macOS. Click on the Remote Host button on VSCode (a button on the left-bottom of the VSCode) and select "Reopen in Container". For more information on setting this up refer to [official documentation](https://code.visualstudio.com/docs/devcontainers/containers). 
+
+### Step 3: Create a virtual environment
 Note: Scripts in this repository were run using python 3.10.9
 ```
 conda create -n kg_rag python=3.10.9
 conda activate kg_rag
 cd KG_RAG
 ```
 
-### Step 3: Install dependencies
+### Step 4: Install dependencies
 
 ```
 pip install -r requirements.txt
 ```
 
-### Step 4: Update config.yaml 
+### Step 5: Update config.yaml 
 
 [config.yaml](https://github.com/BaranziniLab/KG_RAG/blob/main/config.yaml) holds all the necessary information required to run the scripts in your machine. Make sure to populate [this](https://github.com/BaranziniLab/KG_RAG/blob/main/config.yaml) yaml file accordingly.
 
+[.gpt_config.env](https://github.com/niraj17singh/KG_RAG/blob/main/.gpt_config.yaml)
+Update the values in the `.gpt_config.env` file.
+
 Note: There is another yaml file called [system_prompts.yaml](https://github.com/BaranziniLab/KG_RAG/blob/main/system_prompts.yaml). This is already populated and it holds all the system prompts used in the KG-RAG framework.
 
-### Step 5: Run the setup script
+### Step 6: Run the setup script
 Note: Make sure you are in KG_RAG folder
 
 Setup script runs in an interactive fashion.
@@ -108,12 +117,15 @@ Running the setup script will:
 
 - create disease vector database for KG-RAG
 - download Llama model in your machine (optional, you can skip this and that is totally fine)
+  - If using the Llmaa model from huggingface, make sure you run `huggingface-cli login` and add the access token. For more info follow the [official documentation](https://huggingface.co/docs/huggingface_hub/en/quick-start#login-command)
+  - Make sure to request access to the gated repo in the `config["LLAMA_MODEL_NAME"]`. For the default example, it's [`meta-llama/Llama-2-13b-chat-hf`](https://huggingface.co/meta-llama/Llama-2-13b-hf). 
+  - Make sure to check `Read access to contents of all public gated repos you can access` by going to `Access Tokens > [Select the access token] > Manage > Edit Permissions`.
 
 ```
 python -m kg_rag.run_setup
 ```
 
-### Step 6: Run KG-RAG from your terminal
+### Step 7: Run KG-RAG from your terminal
 Note: Make sure you are in KG_RAG folder
 
 You can run KG-RAG using GPT and Llama model. 

config.yaml

@@ -8,38 +8,39 @@ SENTENCE_EMBEDDING_MODEL_FOR_NODE_RETRIEVAL : 'sentence-transformers/all-MiniLM-
 SENTENCE_EMBEDDING_MODEL_FOR_CONTEXT_RETRIEVAL : 'pritamdeka/S-PubMedBert-MS-MARCO'
 
 # VectorDB hyperparameters
-VECTOR_DB_DISEASE_ENTITY_PATH : '/data/somank/KG_RAG/data/disease_with_relation_to_genes.pickle'
-VECTOR_DB_PATH : '/data/somank/KG_RAG/data/vectorDB/disease_nodes_db'
+VECTOR_DB_DISEASE_ENTITY_PATH : '/workspaces/KG_RAG/data/disease_with_relation_to_genes.pickle'
+VECTOR_DB_PATH : '/workspaces/KG_RAG/data/vectorDB/disease_nodes_db'
 VECTOR_DB_CHUNK_SIZE : 650
 VECTOR_DB_CHUNK_OVERLAP : 200
 VECTOR_DB_BATCH_SIZE : 200
 VECTOR_DB_SENTENCE_EMBEDDING_MODEL : 'sentence-transformers/all-MiniLM-L6-v2'
 
 # Path for context file from SPOKE KG
-NODE_CONTEXT_PATH : '/data/somank/KG_RAG/data/context_of_disease_which_has_relation_to_genes.csv'
+NODE_CONTEXT_PATH : '/workspaces/KG_RAG/data/context_of_disease_which_has_relation_to_genes.csv'
 
 # Just note that, this assumes your GPT config file is in the $HOME path, if not, change it accordingly
 # Also, GPT '.env' file should contain values for API_KEY, and optionally API_VERSION and RESOURCE_ENDPOINT. We are not including those parameters in this yaml file
-GPT_CONFIG_FILE : '$HOME/.gpt_config.env'
+GPT_CONFIG_FILE : '/workspaces/KG_RAG/.gpt_config.env'
 # Can be 'azure' or 'open_ai'.
-GPT_API_TYPE : 'azure'
+# GPT_API_TYPE : 'azure'
+GPT_API_TYPE : 'open_ai'
 
 # Llama model name (Refer Hugging face to get the correct name for the model version you would like to use, also make sure you have the right permission to use the model)
 LLAMA_MODEL_NAME : 'meta-llama/Llama-2-13b-chat-hf'
 LLAMA_MODEL_BRANCH : 'main'
 
 # Path for caching LLM model files (When the model gets downloaded from hugging face, it will be saved in this path)
-LLM_CACHE_DIR : '/data/somank/llm_data/llm_models/huggingface'
+LLM_CACHE_DIR : '/workspaces/KG_RAG/llm_data/llm_models/huggingface'
 LLM_TEMPERATURE : 0
 
 # Path to save results
-SAVE_RESULTS_PATH : '/data/somank/kg_rag_fork/KG_RAG/data/results'
+SAVE_RESULTS_PATH : '/workspaces/KG_RAG/data/results'
 
 # File paths for test questions
-MCQ_PATH : '/data/somank/kg_rag_fork/KG_RAG/data/benchmark_data/mcq_questions.csv'
-TRUE_FALSE_PATH : '/data/somank/kg_rag_fork/KG_RAG/data/benchmark_data/true_false_questions.csv'
-SINGLE_DISEASE_ENTITY_FILE : '/data/somank/KG_RAG/data/hyperparam_tuning_data/single_disease_entity_prompts.csv'
-TWO_DISEASE_ENTITY_FILE : '/data/somank/KG_RAG/data/hyperparam_tuning_data/two_disease_entity_prompts.csv'
+MCQ_PATH : '/workspaces/KG_RAG/data/benchmark_data/mcq_questions.csv'
+TRUE_FALSE_PATH : '/workspaces/KG_RAG/data/benchmark_data/true_false_questions.csv'
+SINGLE_DISEASE_ENTITY_FILE : '/workspaces/KG_RAG/data/hyperparam_tuning_data/single_disease_entity_prompts.csv'
+TWO_DISEASE_ENTITY_FILE : '/workspaces/KG_RAG/data/hyperparam_tuning_data/two_disease_entity_prompts.csv'
 
 # SPOKE-API params
 BASE_URI : 'https://spoke.rbvi.ucsf.edu'

kg_rag/run_setup.py

@@ -4,10 +4,14 @@
 def download_llama(method):
     from kg_rag.utility import llama_model
     try:
+        if not os.path.exists(config_data["LLM_CACHE_DIR"]):
+            print(f"LLM_CACHE_DIR:  {config_data['LLM_CACHE_DIR']} doesn't exists. Creating one.." )
+            os.makedirs(config_data["LLM_CACHE_DIR"], exist_ok=True)
         llama_model(config_data["LLAMA_MODEL_NAME"], config_data["LLAMA_MODEL_BRANCH"], config_data["LLM_CACHE_DIR"], method=method)
         print("Model is successfully downloaded to the provided cache directory!")
-    except:
+    except Exception as e:
         print("Model is not downloaded! Make sure the above mentioned conditions are satisfied")
+        raise ValueError(e)
 
 
 print("")

kg_rag/utility.py

@@ -144,6 +144,10 @@ def get_prompt(instruction, new_system_prompt):
     return prompt_template
 
 def llama_model(model_name, branch_name, cache_dir, temperature=0, top_p=1, max_new_tokens=512, stream=False, method='method-1'):
+    if not os.path.exists(cache_dir):
+        print("Cache directory does not exist. Creating a new one.")
+        os.mkdir(cache_dir)
+
     if method == 'method-1':
         tokenizer = AutoTokenizer.from_pretrained(model_name,
                                                  revision=branch_name,
@@ -392,5 +396,6 @@ def interactive(question, vectorstore, node_context_df, embedding_function_for_c
         output = llm_chain.run(context=node_context_extracted, question=question)
     elif "gpt" in llm_type:
         enriched_prompt = "Context: "+ node_context_extracted + "\n" + "Question: " + question
-        output = get_GPT_response(enriched_prompt, system_prompt, llm_type, llm_type, temperature=config_data["LLM_TEMPERATURE"])
+        chat_model_id, chat_deployment_id = get_gpt35()
+        output = get_GPT_response(enriched_prompt, system_prompt, chat_model_id, chat_deployment_id, temperature=config_data["LLM_TEMPERATURE"])
         stream_out(output)

requirements.txt

@@ -11,7 +11,7 @@ asttokens==2.4.0
 async-lru==2.0.4
 async-timeout==4.0.3
 attrs==23.1.0
-auto-gptq==0.4.2
+# auto-gptq==0.4.2
 Babel==2.12.1
 backcall==0.2.0
 backoff==2.2.1
@@ -102,17 +102,17 @@ notebook==7.0.4
 notebook_shim==0.2.3
 numexpr==2.8.6
 numpy==1.26.0
-nvidia-cublas-cu11==11.10.3.66
-nvidia-cuda-cupti-cu11==11.7.101
-nvidia-cuda-nvrtc-cu11==11.7.99
-nvidia-cuda-runtime-cu11==11.7.99
-nvidia-cudnn-cu11==8.5.0.96
-nvidia-cufft-cu11==10.9.0.58
-nvidia-curand-cu11==10.2.10.91
-nvidia-cusolver-cu11==11.4.0.1
-nvidia-cusparse-cu11==11.7.4.91
-nvidia-nccl-cu11==2.14.3
-nvidia-nvtx-cu11==11.7.91
+# nvidia-cublas-cu11==11.10.3.66
+# nvidia-cuda-cupti-cu11==11.7.101
+# nvidia-cuda-nvrtc-cu11==11.7.99
+# nvidia-cuda-runtime-cu11==11.7.99
+# nvidia-cudnn-cu11==8.5.0.96
+# nvidia-cufft-cu11==10.9.0.58
+# nvidia-curand-cu11==10.2.10.91
+# nvidia-cusolver-cu11==11.4.0.1
+# nvidia-cusparse-cu11==11.7.4.91
+# nvidia-nccl-cu11==2.14.3
+# nvidia-nvtx-cu11==11.7.91
 onnxruntime==1.16.0
 openai==0.28.1
 overrides==7.4.0
@@ -185,7 +185,7 @@ tornado==6.3.3
 tqdm==4.66.1
 traitlets==5.10.0
 transformers==4.33.2
-triton==2.0.0
+# triton==2.0.0
 typer==0.9.0
 typing-inspect==0.9.0
 typing_extensions==4.8.0