Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

125G of memory, executing merge-weights.py on 30B will oom #5

Open
Chting opened this issue Mar 10, 2023 · 4 comments
Open

125G of memory, executing merge-weights.py on 30B will oom #5

Chting opened this issue Mar 10, 2023 · 4 comments

Comments

@Chting
Copy link

Chting commented Mar 10, 2023

No description provided.

@ortegaalfredo
Copy link

ortegaalfredo commented Mar 10, 2023 •
edited
Loading

Set up at least a 40G swap, it need about 130G of memory for merging 30B

@Chting
Copy link
Author

Chting commented Mar 10, 2023

Yes, I set 100G SWP, I don't know if 65B is enough

@CaullenOmdahl
Copy link

CaullenOmdahl commented Mar 10, 2023 •
edited
Loading

I was successful. My system has 3 separate swap files. 1 ZRAM (ram compression), 1 on an NVME SSD (32GB), and 1 on a PCI-e SSD (4GB). Set the Zram and NVME priority 1 and PCI-e Priority -1. Ubuntu 22.04, Ryzen 5900x, 64GB RAM.

@randaller
Copy link
Owner

randaller commented Mar 10, 2023 •
edited
Loading

Mine swap is set to 256 Gb in addition to a 128 Gb of RAM. Max swap usage was about 180 Gb while merging 65B model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ortegaalfredo @Chting @randaller @CaullenOmdahl