Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is the override for device='cpu' input really needed? #27

Open
innokean opened this issue Jan 6, 2025 · 1 comment
Open

is the override for device='cpu' input really needed? #27

innokean opened this issue Jan 6, 2025 · 1 comment

Comments

@innokean
Copy link

innokean commented Jan 6, 2025

@xhinker thanks a lot for sharing your code!
Why do you override device='cpu' input
https://github.com/xhinker/sd_embed/blob/main/src/sd_embed/embedding_funcs.py#L1532-L1533
I find that copying from GPU to CPU is rather slow so swapping a transformer model with the embedding models and back takes time comparable to running the embedding models on CPU.
So why prevent using CPU if someone wants to?

@xhinker
Copy link
Owner

xhinker commented Jan 8, 2025

This is because CPU will be extremely slow, feel free to change the code if your CPU is capable :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants