FusionBench: A Comprehensive Benchmark/Toolkit of Deep Model Fusion

This repository serves as a template for creating new projects based on FusionBench. It includes all the necessary configurations and boilerplate code to get started quickly.

Tools for merging pretrained large language models.

Exploring Model Kinship for Merging Large Language Models

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. arXiv:2408.07666.

[ECCV'24] MaxFusion: Plug & Play multimodal generation in text to image diffusion models

A model merging project for generalizing Featured Finite State Machines (FFSMs) to unify behaviors across Software Product Lines (SPLs)

AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.

Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]

SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery. Arxiv, 2024.

Representation Surgery for Multi-Task Model Merging. ICML, 2024.

The code used in the paper "DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging"

flow-merge is a powerful Python library that enables seamless merging of multiple transformer-based language models using the most popular merge methods such as model soups, SLERP, ties-MERGING or DARE.

Mergecraft is a simple library to streamline model merging operations, with seamless integration with HuggingFace🤗

Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic

[ECCV 2024] MagMax: Leveraging Model Merging for Seamless Continual Learning (official repository)

DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling

An easy-to-use Python library for merging PyTorch models.

All-in-one UI for merged LLMs in Hugging Face

Advanced Transfer Learning project with the purpose of obtaining the best model by mixing three twice-fine-tuned models.