🔥A curated list of awesome Generative Models for Decision Making🔥
Please feel free to offer your suggestions in the Issues and pull requests to add links.
[ Last updated at 2025/03/04 ]
Generative Models in Decision Making: A Survey (arxiv)
The organization of this survey is as follows:
- Section 2 introduces sequential decision-making formulation and provides the basics of all examined methods.
- We offer a detailed introduction to seven types of generative models and compare their performance with traditional approaches.
- Section 3 presents the proposed taxonomy for categorizing generative decision-making methods.
- In Section 4, we review and analyze existing literature according to the introduced taxonomy.
- Section 5 showcases practical applications of generative models in decision-making.
- Finally, Section 6 discusses future directions of generative models in decision-making.
- We conclude the paper in Section 7 with an overall summary.
- Comparison of seven generative models in decision-making: training stability, generation diversity, and computational efficiency.
- Larger bubbles represent higher computational efficiency, with different models indicated by distinct colors. Best viewed in color.
-
🔥 Family
- Energy Based Models (EBMs)
- Generative Adversarial Networks (GANs)
- Variational Autoencoders (VAEs)
- Normalizing Flow (NFs)
- Diffusion Models (DMs)
- GFlowNets (GFNs)
- Autoregressive Models (AMs)
-
🔥 Function
- Controller
- Modeler
- Optimizer
-
🔥 Structure
- EBM: EBM
- GAN: GAN / WGAN / DCGAN / C-GAN.
- VAE: VAE / CVAE / seq2seq CVAE /
$\beta$ VAE / MAE. - NF: NF / Coupling Flow / Autoregressive Flow / Continuous Flow.
- DM: DDPM / DDIM / LDM / EDM.
- GFN: GFN / CFN.
- AM: Decoder Only / Encoder Decoder.
-
🔥 Expertise
- Imitation Learning
- online RL
- offline RL
- Robotics
- Generation
- Others
-
🔥 Application
- Robotic Control
- Structure Generation
- Games
- Autonomous Driving
- Optimization
Table
Taxonomy
Survey: Methodology
🔥🔥🔥 Generative Models as Controller
Energy-Based Imitation Learning
Reinforcement Learning with Deep Energy-Based Policies
Generative Adversarial Imitation Learning
InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
Fail-Safe Adversarial Generative Imitation Learning
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning
Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning
Wasserstein Adversarial Imitation Learning
Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model
SC-AIRL: Share-Critic in Adversarial Inverse Reinforcement Learning for Long-Horizon Task
Learning Latent Plans from Play
Latent Plans for Task-Agnostic Offline Reinforcement Learning
OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data
Masked Autoencoding for Scalable and Generalizable Decision Making
Learning Structured Output Representation using Deep Conditional Generative Models
Density estimation using Real NVP
Normalizing Flows for Probabilistic Modeling and Inference
Normalizing Flows: An Introduction and Review of Current Methods
Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies
Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
Guided Flows for Generative Modeling and Decision Making
Imitating Human Behaviour with Diffusion Models
Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
Policy Representation via Diffusion Probability Model for Reinforcement Learning
Planning with Diffusion for Flexible Behavior Synthesis
AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
Is Conditional Generative Modeling all you need for Decision-Making?
Decision Stacks: Flexible Reinforcement Learning via Modular Generative Models
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Learning Universal Policies via Text-Guided Video Generation
Imitating Human Behaviour with Diffusion Models
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation
CFlowNets: Continuous Control with Generative Flow Networks
A Theory of Non-Acyclic Generative Flow Networks
A theory of continuous generative flow networks
Decision Transformer: Reinforcement Learning via Sequence Modeling
Dynamics-Augmented Decision Transformer for Offline Dynamics Generalization
Offline Reinforcement Learning as One Big Sequence Modeling Problem
Bootstrapped Transformer for Offline Reinforcement Learning
Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RL
RvS: What is Essential for Offline RL via Supervised Learning?
RT-1: Robotics Transformer for Real-World Control at Scale
🔥🔥🔥 Generative Models as Modeler
A learning algorithm for boltzmann machines
Reinforcement Learning with Deep Energy-Based Policies
Generative Modeling by Estimating Gradients of the Data Distribution
Enhanced Experience Replay Generation for Efficient Reinforcement Learning
S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learning
Selective Data Augmentation for Improving the Performance of Offline Reinforcement Learning
NICE: Non-linear Independent Components Estimation
Variational Inference with Normalizing Flows
Scaling Robot Learning with Semantically Imagined Experience
GenAug: Retargeting behaviors to unseen situations via Generative Augmentation
Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning
Unifying Generative Models with GFlowNets and Beyond
Generative Flow Networks for Discrete Probabilistic Modeling
Improving GFlowNets for Text-to-Image Diffusion Alignment
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Autoregressive Policies for Continuous Control Deep Reinforcement Learning
🔥🔥🔥 Generative Models as Optimizer
End-to-End Stochastic Optimization with Energy-Based Model
A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences
Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs)
Augmenting High-dimensional Nonlinear Optimization with Conditional GANs
GANs and DCGANs for generation of topology optimization validation curve through clustering analysis
Bayesian Optimization with Hidden Constraints via Latent Decision Models
Learning a Latent Search Space for Routing Problems using Variational Autoencoders
Adaptive Monte Carlo augmented with normalizing flows
Diffusion Models for Black-Box Optimization
Diffusion Model for Data-Driven Black-Box Optimization
Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
Ant Colony Sampling with GFlowNets for Combinatorial Optimization
Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling
Generative Pretraining for Black-Box Optimization
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Applications
There are lots of applications of generative models in decision making scenarios. We consider five typical applications including: robot control, autonomous driving, games, structural generation, and optimization.
🤖🤖🤖 Robotic Control
Benchmarking deep reinforcement learning for continuous control
End-to-end training of deep visuomotor policies
Optimal consensus control of the Cucker-Smale model
Algorithmic trading, stochastic control, and mutually exciting processes
Continuous-time mean--variance portfolio selection: A reinforcement learning framework
Sim2Real in robotics and automation: Applications and challenges
Understanding Domain Randomization for Sim-to-real Transfer
Provable Sim-to-real Transfer in Continuous Domain with Partial Observations
Adversarially approximated autoencoder for image generation and manipulation
Trajectory balance: Improved credit assignment in gflownets
Rapid locomotion via reinforcement learning
🧬🧬🧬 Structure Generation
Multi-objective de novo drug design with conditional graph generative model
Graphite: Iterative Generative Modeling of Graphs
Biological Sequence Design with GFlowNets
Generative model-based document clustering: a comparative study
Rethinking Reinforcement Learning for Recommendation: A Prompt Perspective
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
TEMPERA: Test-Time Prompting via Reinforcement Learning
🎮🎮🎮 Games
A Survey of Deep Reinforcement Learning in Video Games
Unbounded: A Generative Infinite Game of Character Life Simulation
Multi-Game Decision Transformers
Simulating Life: The Application of Generative Agents in Virtual Environments
Generative Agents: Interactive Simulacra of Human Behavior
Expected flow networks in stochastic environments and two-player zero-sum games
🚗🚗🚗 Autonomous Driving
Parallel planning: A new motion planning framework for autonomous driving
Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning
AGen: Adaptable Generative Prediction Networks for Autonomous Driving
WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models
Recurrent Conditional Generative Adversarial Networks for Autonomous Driving Sensor Modelling
Learning a Decision Module by Imitating Driver's Control Behaviors
TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios
Conformal Prediction for Uncertainty-Aware Planning with Diffusion Dynamics Model
SAD-GAN: Synthetic Autonomous Driving using Generative Adversarial Networks
🏷️🏷️🏷️ Optimization
Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon
A Survey for Solving Mixed Integer Programming via Machine Learning
Learning Combinatorial Optimization Algorithms over Graphs
Attention, Learn to Solve Routing Problems!
POMO: Policy Optimization with Multiple Optima for Reinforcement Learning
An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
Generalize a Small Pre-trained Model to Arbitrarily Large TSP Instances
DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
The Power of Scale for Parameter-Efficient Prompt Tuning
OpenPrompt: An Open-source Framework for Prompt-learning
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
TEMPERA: Test-Time Prompting via Reinforcement Learning
GFlowOut: Dropout with Generative Flow Networks
Robust Scheduling with GFlowNets
Citing
If you find this work useful, please cite our paper:
@article{li2025generative,
title={Generative Models in Decision Making: A Survey},
author={Li, Yinchuan and Shao, Xinyu and Zhang, Jianping and Wang, Haozhi and Brunswic, Leo Maxime and Zhou, Kaiwen and Dong, Jiqian and Guo, Kaiyang and Li, Xiu and Chen, Zhitang and others},
journal={arXiv preprint arXiv:2502.17100},
year={2025}
}