This directory contains configuration files for training setups:
- deepspeed.yaml: Configuration file for DeepSpeed, a deep learning optimization library, which helps in efficient training of large models.
zero_stage1/2/3_config.json: Configurations for different stages of ZeRO (Zero Redundancy Optimizer) optimization stages 1, 2, and 3, helping in efficient memory usage during model training.- multiple_gpu.yaml: Configuration file for training the model on multiple GPUs.
- mixtral_7b_tangshi.yml: This is the primary configuration file for training the "mixtral7b" model on the Tangshi dataset. It contains settings specific to this model and dataset, including model parameters, training settings, and data handling instructions.
This directory contains scripts and notebooks for data handling, model training, and testing for mixtral model:
- build_data.py: A script for generating training and validation data. It preprocesses the raw data to create train.csv and val.csv.
- mistral-ft.ipynb: A Jupyter notebook demonstrating how to fine-tune the "mixtral7b" model. It includes steps for loading the model, training it on the dataset, and saving the fine-tuned version.
- mis-result.ipynb: This notebook is used for merging LoRA (Low-Rank Adaptation) layers and analyzing the results post-merging.
- test_merged.ipynb: A notebook for testing the model after the LoRA layers have been merged. It evaluates the model's performance on test data. train.csv, val.csv: CSV files containing the training and validation data, respectively. These are generated by build_data.py.
Several Jupyter notebooks are included in the project root directory for demonstration purposes:
- learn_qlora_finetune.ipynb: Demonstrates how to fine-tune the "llama2" model using QLoRA.
- inference_qlora_finetune.ipynb: Used for testing the fine-tuning results of the "llama2" model.
- quantize.ipynb: Demonstrates the process of quantizing the "mixtral 8x7B Moe" model using llama.cpp.
- inference-quantize.ipynb: A notebook to test the quantized "mixtral 8x7B Moe" model in the GGUF format.
- learn_nanogpt.ipynb: Demonstrates building a GPT-2 model from scratch.
To get started with the project, particularly for fine-tuning the "mixtral7b" model using LoRA, you should run the trl_finetune.py script. You can execute this script by navigating to the project root directory in your terminal and running:
python3 trl_finetune.py --config configs/mixtral_7b_tangshi.yml
This command uses the mixtral_7b_tangshi.yml configuration file to fine-tune the "mixtral7b" model on the Tangshi dataset.
python3 merge.py --config configs/merge_mixtral_7b.yml
Merge lora with base model.
python3 inference.py --config configs/inference_mixtral.yml
Inference on merged model.
To set up the project, follow these steps:
Create a Virtual Environment: It's recommended to use a virtual environment to manage dependencies. You can create one using the following command:
python3 -m venv venv
This command will create a new virtual environment named venv in your project directory.
Install Dependencies: With the virtual environment activated, install the required dependencies using:
pip3 install -r requirements.txt
This will install all the necessary Python packages as specified in the requirements.txt file.
Eval with lm-evaluation-harness:
lm_eval --model hf --model_args pretrained=HenryJJ/Instruct_Yi-6B_Dolly15K,parallelize=True,dtype=auto,trust_remote_code=True --tasks truthfulqa --num_fewshot 0 --batch_size 8
lm_eval --model vllm --model_args pretrained="/home/ubuntu/learn-llm/phi2_dolly/merged",trust_remote_code=True,gpu_memory_utilization=0.8 --tasks gsm8k --num_fewshot 5 --output_path phi2_dolly/gsm8k.json