FSDP2 and peft

[psinger]

Hey, sorry if this is the wrong place. Feel free to move it to discussion.

I am trying to get peft working with fsdp2 and am wondering if someone else attempted that already?

The issue is that Im always getting errors along the lines of:
RuntimeError: aten.mm.default: got mixed torch.Tensor and DTensor, need to convert all torch.Tensor to DTensor before calling distributed operators!

Happy for any pointers.

[BenjaminBossan]

Could you please provide environment info (package versions, hardware, etc), a reproducer and the full error message? In case you're using accelerate (maybe indirectly via transformers Trainer), please take note that FSDP2 is not supported yet.

[psinger]

No, Im not using accelerate. But Im following torchtriton:
https://github.com/pytorch/torchtune/blob/main/recipes/lora_dpo_distributed.py

This is more an explorative question if someone has successfully run fsdp2 with peft explicitly? I am not finding any info out there on it.

I think it mostly boils down which layers etc youre sharding at what point.

[BenjaminBossan]

But Im following torchtriton:
https://github.com/pytorch/torchtune/blob/main/recipes/lora_dpo_distributed.py

Unless you've rewritten that script, note that it uses torchtune, not PEFT.

This is more an explorative question if someone has successfully run fsdp2 with peft explicitly?

Personally, I haven't tried it. I'll probably do that once FSDP2 is supported by accelerate.

[psinger]

That's why Im asking here, I tried using peft with pretty much their script. It seems to work for them with their lora implementations.