Refactor cvi projection marginal rule (with proposal distribution)

[Nimrais]

Conjugate Variational Inference Enhancement PR

Overview

This PR enhances the Conjugate Variational Inference (CVI) projection method by adding proposal distribution support and flexible sampling strategies. These improvements address practical limitations in the current implementation, specifically for deterministic nodes with multiple interfaces.

Key Features

1. Proposal Distribution Support

• Added ProposalDistributionContainer to maintain and update proposal distributions across iterations
• Improved convergence by recycling posterior approximations as proposals for subsequent iterations
• Resolved the issue where sampling directly from messages isn't always possible

2. Flexible Sampling Strategies

• Implemented CVISamplingStrategy enum with two options:
  • FullSampling: Traditional approach with multiple samples (better accuracy)
  • MeanBased: Uses only distribution means (significantly faster)
• Added helper functions to generate samples based on chosen strategy

Benefits

• Performance: Mean-based sampling provides substantial speed improvements (benchmarks show ~50-70% faster execution)
• Convergence: Improved approximation quality over multiple iterations with recycled proposals
• Flexibility: Works better with deterministic nodes with many interfaces
• Stability: More robust in complex models where direct sampling is problematic

Testing

The PR includes some tests covering:

• Convergence quality analysis across multiple iterations
• Performance benchmarking comparing sampling strategies
• Correctness verification for different sampling approaches
• Integration tests with typical use cases

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.75%. Comparing base (b88de4d) to head (d099a27).
Report is 38 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #430      +/-   ##
==========================================
+ Coverage   74.42%   74.75%   +0.33%     
==========================================
  Files         195      198       +3     
  Lines        5623     5721      +98     
==========================================
+ Hits         4185     4277      +92     
- Misses       1438     1444       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bvdmitri]

@Nimrais ping

[bvdmitri]

@Nimrais do we want to merge this?

[Nimrais]

@bvdmitri This PR works in principle. I have an example (in a model notebook) that this PR allows to run. I also added tests that show the convergence with the marginal approach. However, I am currently refactoring the ExponentialFamilyManifolds to address one of the reviews.

If you could review it and suggest the minimal possible changes needed before merging, it would be nice, as I am using this branch locally because CVIProjection works better in this branch for deterministic nodes with many interfaces.

More importantly, this PR solves an important problem: it's not always possible to sample from the messages; however, in some sense, it's always possible from marginals.