New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I am having problem merging GPT-Neo #409

Open

2625554780 opened this issue Aug 26, 2024 · 1 comment

2625554780 commented Aug 26, 2024

It seems like mergekit didn't support the merge method of GPT-Neo, could anyone help me or just realize the function? I appreciate it !

metric-space mentioned this issue

support for GPT-Neo needed! #408

Closed

Contributor

metric-space commented Sep 4, 2024 •

edited

Loading

@2625554780 apologies for the wait. To get this into mergekit/just for your purpose, start with defining the json based model architecture template (for GPT-Neo) similar to the ones described here https://github.com/arcee-ai/mergekit/tree/main/mergekit/_data/architectures

Once you have that in there, to test it out try merging GPT-neo based models

If everything checks out, do consider submitting a PR on here with the template

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment