Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to use MPS instead of CPU on MacOS? #772

Open
yikchunnnn opened this issue Oct 7, 2024 · 2 comments
Open

How to use MPS instead of CPU on MacOS? #772

yikchunnnn opened this issue Oct 7, 2024 · 2 comments
Labels
algorithm Algorithm improvements & issues documentation Improvements or additions to documentation performance Running speed & quality

Comments

@yikchunnnn
Copy link

It seems there is an experimental feature of using MPS, may I know how to turn it on?
Thank you for your guidance in advance!
@fumiama
Copy link
Member

fumiama commented Oct 9, 2024

MPS is slow and has memory leak now so we're not recommending you to use it. You can turn it on by passing experimental=True to Chat.load()

@fumiama fumiama added documentation Improvements or additions to documentation algorithm Algorithm improvements & issues performance Running speed & quality labels Oct 9, 2024
@BurnieLiang
Copy link

Errors occur after passing experimental=True to Chat.load()
masked_input_ids: torch.Tensor = input_ids[text_mask_inv].to(device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
algorithm Algorithm improvements & issues documentation Improvements or additions to documentation performance Running speed & quality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@BurnieLiang @fumiama @yikchunnnn