Skip to content

Commit

Permalink
Merge 'simulator: add counterexample minimization' from Alperen Keleş
Browse files Browse the repository at this point in the history
This PR introduces counterexample minimization(shrinking) in the
simulator. It will require changes to the current structure in various
places, so I've opened it as a draft PR for now, in order to not
overwhelm the reviewers all at once.
- [x] Turn interactions plans into sequences of properties instead of
sequences of interactions, adding a semantic layer.
- [x] Add assumptions to the properties, rendering a property invalid if
its assumptions are not valid.
- [x] Record the failure point in a failing assertion, as shrinking by
definition works when the same assertion fails for a smaller input.
- [ ] Add shrinking at three levels,
  - [x] top level(removing whole properties),
  - [x] property level(removing interactions within properties),
  - [ ] interaction level(shrinking values in interactions to smaller
ones).
- [ ] Add [marauders](https://github.com/alpaylan/marauders) as a dev
dependency, inject custom mutations to a testing branch for evaluating
the simulator performance.
- [ ] Integrate the simulator evaluation with the CI.

Closes #623
  • Loading branch information
penberg committed Jan 15, 2025
2 parents d0b5f50 + ea6ad8d commit ffe6514
Show file tree
Hide file tree
Showing 13 changed files with 1,480 additions and 432 deletions.
27 changes: 25 additions & 2 deletions simulator/generation/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,41 @@ use anarchist_readable_name_generator_lib::readable_name_custom;
use rand::{distributions::uniform::SampleUniform, Rng};

pub mod plan;
pub mod property;
pub mod query;
pub mod table;

/// Arbitrary trait for generating random values
/// An implementation of arbitrary is assumed to be a uniform sampling of
/// the possible values of the type, with a bias towards smaller values for
/// practicality.
pub trait Arbitrary {
fn arbitrary<R: Rng>(rng: &mut R) -> Self;
}

/// ArbitraryFrom trait for generating random values from a given value
/// ArbitraryFrom allows for constructing relations, where the generated
/// value is dependent on the given value. These relations could be constraints
/// such as generating an integer within an interval, or a value that fits in a table,
/// or a predicate satisfying a given table row.
pub trait ArbitraryFrom<T> {
fn arbitrary_from<R: Rng>(rng: &mut R, t: &T) -> Self;
fn arbitrary_from<R: Rng>(rng: &mut R, t: T) -> Self;
}

/// Frequency is a helper function for composing different generators with different frequency
/// of occurences.
/// The type signature for the `N` parameter is a bit complex, but it
/// roughly corresponds to a type that can be summed, compared, subtracted and sampled, which are
/// the operations we require for the implementation.
// todo: switch to a simpler type signature that can accomodate all integer and float types, which
// should be enough for our purposes.
pub(crate) fn frequency<
'a,
T,
R: rand::Rng,
N: Sum + PartialOrd + Copy + Default + SampleUniform + SubAssign,
>(
choices: Vec<(N, Box<dyn FnOnce(&mut R) -> T + 'a>)>,
choices: Vec<(N, Box<dyn Fn(&mut R) -> T + 'a>)>,
rng: &mut R,
) -> T {
let total = choices.iter().map(|(weight, _)| *weight).sum::<N>();
Expand All @@ -37,6 +54,7 @@ pub(crate) fn frequency<
unreachable!()
}

/// one_of is a helper function for composing different generators with equal probability of occurence.
pub(crate) fn one_of<'a, T, R: rand::Rng>(
choices: Vec<Box<dyn Fn(&mut R) -> T + 'a>>,
rng: &mut R,
Expand All @@ -45,15 +63,20 @@ pub(crate) fn one_of<'a, T, R: rand::Rng>(
choices[index](rng)
}

/// pick is a helper function for uniformly picking a random element from a slice
pub(crate) fn pick<'a, T, R: rand::Rng>(choices: &'a [T], rng: &mut R) -> &'a T {
let index = rng.gen_range(0..choices.len());
&choices[index]
}

/// pick_index is typically used for picking an index from a slice to later refer to the element
/// at that index.
pub(crate) fn pick_index<R: rand::Rng>(choices: usize, rng: &mut R) -> usize {
rng.gen_range(0..choices)
}

/// gen_random_text uses `anarchist_readable_name_generator_lib` to generate random
/// readable names for tables, columns, text values etc.
fn gen_random_text<T: Rng>(rng: &mut T) -> String {
let big_text = rng.gen_ratio(1, 1000);
if big_text {
Expand Down
Loading

0 comments on commit ffe6514

Please sign in to comment.