LTXVPromptEnhancer: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)

[Skol600ml]

The node is very cool, it works sometimes, but most of the times i get this error. Sometimes it works if you insist

[RicardoAGG]

@Skol600ml Hey there! I ran into the same "Expected all tensors to be on the same device" error. Here’s how I fixed it by explicitly moving models and images to the same device. Just add the lines marked with # <-- add this line:

---

In the prompt enhancer loader (e.g., down_load_llm_model and down_load_image_captioner):

def down_load_llm_model(self, llm_name, load_device):
model_path = ...
llm_model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16)
llm_model = llm_model.to(load_device) # <-- add this line
llm_tokenizer = AutoTokenizer.from_pretrained(model_path)
return llm_model, llm_tokenizer

def down_load_image_captioner(self, image_captioner, load_device):
model_path = ...
image_caption_model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
image_caption_model = image_caption_model.to(load_device) # <-- add this line
image_caption_processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
return image_caption_model, image_caption_processor

---

And in the enhance() method:

def enhance(...):
...
if image_prompt is not None:
permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :].to(model.device) # <-- add .to(model.device)
image_conditioning = [(permuted_image, 0, 1.0)]
...

After making these changes, I no longer see the device mismatch error. Hope it helps!

[gxbsyxh]

嘿，你好！我遇到了相同的“预期所有张量都位于同一设备上”错误。以下是我通过将模型和图像显式移动到同一设备来修复它的方法。只需添加标有 # < 的行 -- 添加以下行：

在提示增强器加载器（例如，down_load_llm_model 和 down_load_image_captioner）：

def down_load_llm_model（self， llm_name， load_device）： model_path = ... llm_model = AutoModelForCausalLM.from_pretrained（model_path， torch_dtype=torch.bfloat16） llm_model = llm_model.to（load_device） # <-- 添加这一行 llm_tokenizer = AutoTokenizer.from_pretrained（model_path） return llm_model， llm_tokenizer

def down_load_image_captioner（self， image_captioner， load_device）： model_path = ... image_caption_model = AutoModelForCausalLM.from_pretrained（model_path， trust_remote_code=True） image_caption_model = image_caption_model.to（load_device） # <-- 添加这一行 image_caption_processor = AutoProcessor.from_pretrained（model_path， trust_remote_code=True） return image_caption_model， image_caption_processor

在 enhance（） 方法中：

防御增强（...）：

...
如果 image_prompt 不是 None：
permuted_image = image_prompt.permute（3， 0， 1， 2）[None， ：].to（model.device） # <-- add .to（model.device）
image_conditioning = [（permuted_image， 0， 1.0）]
...
进行这些更改后，我不再看到 device mismatch 错误。希望它有帮助！

Thank you, but I don't know in which file you modified them

[dan-just]

nevermind as it only works once on my side

Sadly the solution only works once and then on a new generation gives the same error.

My recommendation is to use Ollama Generate node for the prompt enhancement until the problem is fixed.

---

Doesnt work as suppose to.
To (temp) Fix the issue

• Go inside the ComfyUI-LTXVideo folder
• open Prompt_enhancer_node.py < with an editor to edit the code (of course, like MS visual code/notepad++).
• On line 121 add: llm_model = llm_model.to(load_device)
• Then AFTER editing the previous line, go to line 133 and add: image_caption_model = image_caption_model.to(load_device)
• As last on line 199 add: .to(model.device) behind: permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :]. So that it looks like: permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :].to(model.device)

This is all done in the Prompt_enhancer_node.py file

Final code should look like:

def down_load_llm_model(self, llm_name, load_device):
    model_path = self.model_path_download_if_needed(llm_name)
    llm_model = AutoModelForCausalLM.from_pretrained(
        model_path,
        torch_dtype=torch.bfloat16,
    )
    llm_model = llm_model.to(load_device)
    llm_tokenizer = AutoTokenizer.from_pretrained(
        model_path,
    )

    return llm_model, llm_tokenizer

def down_load_image_captioner(self, image_captioner, load_device):
    model_path = self.model_path_download_if_needed(image_captioner)
    image_caption_model = AutoModelForCausalLM.from_pretrained(
        model_path, trust_remote_code=True
    )
    image_caption_model = image_caption_model.to(load_device)
    image_caption_processor = AutoProcessor.from_pretrained(
        model_path, trust_remote_code=True
    )

The last part looks like this:

def enhance(
    self,
    prompt,
    prompt_enhancer: comfy.model_patcher.ModelPatcher,
    image_prompt: torch.Tensor = None,
    max_resulting_tokens=256,
):
    comfy.model_management.free_memory(
        prompt_enhancer.memory_required([]),
        comfy.model_management.get_torch_device(),
    )
    comfy.model_management.load_model_gpu(prompt_enhancer)
    model = prompt_enhancer.model
    image_conditioning = None
    if image_prompt is not None:
        permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :].to(model.device)
        image_conditioning = [(permuted_image, 0, 1.0)]

    enhanced_prompt = model(prompt, image_conditioning, max_resulting_tokens)
    return (enhanced_prompt[0],)

[patrice74]

Hello all,
I encountered the same problem and here is how I solved it:
The file at: "ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LTXVideo\prompt_enhancer_nodes.py"
Just modify the enhance procedure as below and you will see a noticeable speed improvement:
Before: 17' After: 6' on my 3080.

def enhance(
        self,
        prompt,
        prompt_enhancer: comfy.model_patcher.ModelPatcher,
        image_prompt: torch.Tensor = None,
        max_resulting_tokens=256,
    ):
        comfy.model_management.free_memory(
            prompt_enhancer.memory_required([]),
            comfy.model_management.get_torch_device(),
        )
        
        # comfy.model_management.load_model_gpu(prompt_enhancer)
        device = "cuda:0" if torch.cuda.is_available() else "cpu"   # <-- add
        model = prompt_enhancer.model.to(device)                    # <-- add
        
        image_conditioning = None
        if image_prompt is not None:
            permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :].to(device) # <-- add .to(device)
            image_conditioning = [(permuted_image, 0, 1.0)]

        enhanced_prompt = model(prompt, image_conditioning, max_resulting_tokens)
        return (enhanced_prompt[0],)

[altoiddealer]

Hello all, I encountered the same problem and here is how I solved it: The file at: "ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LTXVideo\prompt_enhancer_nodes.py" Just modify the enhance procedure as below and you will see a noticeable speed improvement: Before: 17' After: 6' on my 3080.

Unbelievable - I just came here to Issues to find an answer you just just solved this 7 minutes before I visited.

I tested this and it fixed the issue for me!

Edit For those who may be code illiterate, def enhance( needs to be indented after you copy/paste the code from @patrice74 verbatim.

[barclayslb]

Hello all, I encountered the same problem and here is how I solved it: The file at: "ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LTXVideo\prompt_enhancer_nodes.py" Just modify the enhance procedure as below and you will see a noticeable speed improvement: Before: 17' After: 6' on my 3080.

def enhance(
        self,
        prompt,
        prompt_enhancer: comfy.model_patcher.ModelPatcher,
        image_prompt: torch.Tensor = None,
        max_resulting_tokens=256,
    ):
        comfy.model_management.free_memory(
            prompt_enhancer.memory_required([]),
            comfy.model_management.get_torch_device(),
        )
        
        # comfy.model_management.load_model_gpu(prompt_enhancer)
        device = "cuda:0" if torch.cuda.is_available() else "cpu"   # <-- add
        model = prompt_enhancer.model.to(device)                    # <-- add
        
        image_conditioning = None
        if image_prompt is not None:
            permuted_image = image_prompt.permute(3, 0, 1, 2)[None, :].to(device) # <-- add .to(device)
            image_conditioning = [(permuted_image, 0, 1.0)]

        enhanced_prompt = model(prompt, image_conditioning, max_resulting_tokens)
        return (enhanced_prompt[0],)

you saved me a lot of time ! thank u