Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Code LLaMA 13B Variant #18

Merged
merged 6 commits into from
Apr 22, 2024
Merged

Code LLaMA 13B Variant #18

merged 6 commits into from
Apr 22, 2024

Conversation

JRosenkranz
Copy link
Collaborator

@JRosenkranz JRosenkranz commented Apr 21, 2024 •
edited
Loading

  • Added a code llama 13b variant (PagedLLaMA)
  • mlp speculator code llama 13b variant
  • Added code prompts to speculative decoding inference script.

@JRosenkranz JRosenkranz requested review from nairbv and daviswer April 21, 2024 02:38
@JRosenkranz JRosenkranz self-assigned this Apr 21, 2024
daviswer
daviswer approved these changes Apr 22, 2024
View reviewed changes
Copy link
Collaborator

@daviswer daviswer left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Straightforward set of changes, looks good. Seems MyPy doesn't like some of the stuff associated with the recent QKV/MLP fusion changes, though

@JRosenkranz JRosenkranz merged commit c52037a into foundation-model-stack:main Apr 22, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daviswer daviswer daviswer approved these changes

@nairbv nairbv nairbv approved these changes

Assignees

@JRosenkranz JRosenkranz

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JRosenkranz @nairbv @daviswer