Skip to content

@neuralmagic Neural Magic

Change the repository type filter

All

    Repositories list

    59 repositories

    • nm-vllm-certs

      Public
      General Information, model certifications, and benchmarks for nm-vllm enterprise distributions
      vllm
      1•7•1•0•Updated Nov 18, 2024Nov 18, 2024

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •4.6k•6•0•15•Updated Nov 17, 2024Nov 17, 2024

    • nm-actions

      Public
      Neural Magic GHA
      Python
      •
      Apache License 2.0
      •0•0•0•3•Updated Nov 15, 2024Nov 15, 2024

    • graphs

      Public
      Apache License 2.0
      •0•0•0•0•Updated Nov 15, 2024Nov 15, 2024

    • temp-llm-compressor

      Public
      Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
      Python
      •
      Apache License 2.0
      •58•0•0•0•Updated Nov 14, 2024Nov 14, 2024

    • upstream-transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •27k•1•0•0•Updated Nov 13, 2024Nov 13, 2024

    • alpaca_eval

      Public
      An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
      Jupyter Notebook
      •
      Apache License 2.0
      •245•0•0•0•Updated Nov 12, 2024Nov 12, 2024

    • compressed-tensors

      Public
      A safetensors extension to efficiently store sparse quantized tensors on disk
      Python
      •
      Apache License 2.0
      •2•49•1•8•Updated Nov 12, 2024Nov 12, 2024

    • evalplus

      Public
      NeuralMagic fork of EvalPlus (Rigourous evaluation of LLM-synthesized code - NeurIPS 2023)
      Python
      •
      Apache License 2.0
      •108•0•0•0•Updated Nov 5, 2024Nov 5, 2024

    • guidellm

      Public
      Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
      Python
      •
      Apache License 2.0
      •12•165•9•9•Updated Nov 5, 2024Nov 5, 2024

    • yolov5

      Public
      YOLOv5 in PyTorch > ONNX > CoreML > TFLite
      Python
      •
      GNU General Public License v3.0
      •16k•20•0•2•Updated Oct 31, 2024Oct 31, 2024

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1.3k•0•0•0•Updated Oct 25, 2024Oct 25, 2024

    • upstream-llm-foundry

      Public
      LLM training code for MosaicML foundation models
      Python
      •
      Apache License 2.0
      •528•0•0•0•Updated Oct 24, 2024Oct 24, 2024

    • mistral-evals

      Public
      Python
      •4•0•0•0•Updated Oct 23, 2024Oct 23, 2024

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •1.9k•2•0•1•Updated Oct 23, 2024Oct 23, 2024

    • nm-vllm

      Public archive
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Other
      •4.6k•252•0•0•Updated Oct 11, 2024Oct 11, 2024

    • mteb

      Public
      MTEB: Massive Text Embedding Benchmark
      Jupyter Notebook
      •
      Apache License 2.0
      •272•0•0•1•Updated Oct 2, 2024Oct 2, 2024

    • transformers

      Public
      🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.
      Python
      •
      Apache License 2.0
      •27k•9•0•13•Updated Oct 1, 2024Oct 1, 2024

    • AutoFP8

      Public
      Python
      •
      Apache License 2.0
      •20•157•10•3•Updated Oct 1, 2024Oct 1, 2024

    • OmniQuant

      Public
      [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
      Python
      •
      MIT License
      •56•0•0•1•Updated Sep 27, 2024Sep 27, 2024

    • quant_kernel_benchmarks

      Public
      Benchmarking code for running quantized kernels from vLLM and other libraries
      Python
      •0•1•0•0•Updated Sep 24, 2024Sep 24, 2024

    • temp-AutoGPTQ

      Public
      An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
      Python
      •
      MIT License
      •484•0•0•0•Updated Sep 16, 2024Sep 16, 2024

    • upstream-composer

      Public
      Supercharge Your Model Training
      Python
      •
      Apache License 2.0
      •419•0•0•0•Updated Aug 27, 2024Aug 27, 2024

    • MixEval

      Public
      NM fork of MixEval compatible with SparseAutoModel.
      Python
      •34•0•0•1•Updated Aug 20, 2024Aug 20, 2024

    • mamba

      Public
      Mamba SSM architecture
      Python
      •
      Apache License 2.0
      •1.1k•0•0•0•Updated Aug 12, 2024Aug 12, 2024

    • causal-conv1d

      Public
      Causal depthwise conv1d in CUDA, with a PyTorch interface
      Cuda
      •
      BSD 3-Clause "New" or "Revised" License
      •61•0•0•0•Updated Aug 8, 2024Aug 8, 2024

    • sparseml

      Public
      Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
      sparsitykerasdeep-learning-algorithmsdeep-learning-librarypruningobject-detectioncomputer-vision-algorithmsonnxdeep-learning-modelssparsification
      Python
      •
      Apache License 2.0
      •148•2.1k•7•60•Updated Aug 1, 2024Aug 1, 2024

    • inference

      Public
      Reference implementations of MLPerf™ inference benchmarks
      Python
      •
      Apache License 2.0
      •536•1•0•1•Updated Jul 24, 2024Jul 24, 2024

    • examples

      Public
      Notebooks using the Neural Magic libraries 📓
      Jupyter Notebook
      •7•41•0•3•Updated Jul 24, 2024Jul 24, 2024

    • sparsezoo

      Public
      Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
      nlpcomputer-visiondeep-learning-algorithmsyoloresnetpruningtransfer-learningpretrained-modelsquantizationmobilenet
      Python
      •
      Apache License 2.0
      •25•370•2•6•Updated Jul 19, 2024Jul 19, 2024