Error while importing "from unsloth import FastLanguageModel"

[thesillystudent]

Trace -

File "/home/ec2-user/SageMaker/mistral-finetune-unsloth/multi-run-compare/run_model_qwen.py", line 1, in <module> from unsloth import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/__init__.py", line 174, in <module> from .models import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/__init__.py", line 15, in <module> from .loader import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/loader.py", line 16, in <module> from .llama import FastLlamaModel, logger File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/llama.py", line 31, in <module> from ..kernels import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/__init__.py", line 15, in <module> from .cross_entropy_loss import ( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/cross_entropy_loss.py", line 33, in <module> def _cross_entropy_forward( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 882, in jit return decorator(fn) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 871, in decorator return JITFunction( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 717, in __init__ self.src = self.src[re.search(r"^def\s+\w+\s*\(", self.src, re.MULTILINE).start():] AttributeError: 'NoneType' object has no attribute 'start'

unsloth version - 2024.11.7
triton - 3.0.0
torch - 2.4.0
xformers - 0.0.27.post2

Tried uninstalling everything -
pip uninstall -y accelerate bitsandbytes captum deepspeed flash-attn peft torchaudio torchdata torchtext torchvision unsloth_zoo xformers unsloth

Then installing -

 pip install --upgrade setuptools packaging
 pip install torch==2.4.0 torchvision torchaudio
 pip install "unsloth @ git+https://github.com/unslothai/unsloth.git"
 pip install "transformers @ git+https://github.com/huggingface/transformers.git"
 pip install  --no-deps "xformers==0.0.27.post2"
 pip install git+https://github.com/microsoft/DeepSpeed
 pip install git+https://github.com/huggingface/accelerate
 pip install git+https://github.com/huggingface/peft
 pip install trl bitsandbytes datasets protobuf sentencepiece scikit-learn flash_attn unsloth-zoo

But still doesn't work

[thesillystudent]

Also doing this - wget -qO- https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/_auto_install.py | python -

and installing -

pip install --upgrade pip && pip install "unsloth[cu121-ampere-torch240] @ git+https://github.com/unslothai/unsloth.git"

doesn;t work

I'm on Amazon Linux 2

It's not able to install "xformers==0.0.28.post2" but can install "xformers==0.0.27.post2"

[thesillystudent]

It seems that this line fails

self.src = self.src[re.search(r"^def\s+\w+\s*\(", self.src, re.MULTILINE).start():]

due to these decorators

@triton.heuristics({ "DO_SOFTCAPPING": lambda args: bool(args["DO_SOFTCAPPING" ]), "DO_LOGIT_SCALING": lambda args: bool(args["DO_LOGIT_SCALING"]), })