[P1] Not supportive to all Hugging Face models.

[krishnardt]

Hi,

I tried to train below models with LorEFT for instruction fine tuning of text2SQL-

1. Qwen/Qwen2-7B-Instruct-AWQ
2. microsoft/phi-2

But these models are not able to load to GPU.
I tried this code base to load the model and finetune: https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/train.py

Command to run the script in above mentioned link:
python train.py --task "instruct" --model "meta-llama/Llama-2-7b-hf" --data_dir /datasets --train_dataset /datasets/text_2_sql/train_final.json --seed 42 --layers 'all' --rank 4 --position "f10+l10" --epochs 16 --save_model --intervention_type "LoreftIntervention" --gradient_accumulation_steps 4 --batch_size 4 --output_dir /finetuned_models/reft_models/test_layers_all_rank4_fl1010_epochs_16_lr_5e3 --lr 5e-3 --schedule "cosine" --warmup_ratio 0.03 --logging_steps 10

Am I missing anything here?
Could someone look into this issue for further model support?

[frankaging]

Hey, it is because the script is for Llama models! Not because of the library, i think. For all other types, please add them to this dict:
https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/train.py#L57