salsa 3.0

book/src/SUMMARY.md

@@ -16,6 +16,7 @@
   - [Defining the checker](./tutorial/checker.md)
   - [Defining the interpreter](./tutorial/interpreter.md)
 - [Reference](./reference.md)
+  - [Durability](./reference/durability.md)
   - [Algorithm](./reference/algorithm.md)
 - [Common patterns](./common_patterns.md)
   - [Selection](./common_patterns/selection.md)

book/src/overview.md

@@ -114,9 +114,12 @@ Finally, you can also modify the value of an input field by using the setter met
 Since this is modifying the input, the setter takes an `&mut`-reference to the database:
 
 ```rust
-file.set_contents(&mut db, String::from("fn foo() { /* add a comment */ }"));
+file.set_contents(&mut db).to(String::from("fn foo() { /* add a comment */ }"));
 ```
 
+Note that the setter method `set_contents` returns a "builder".
+This gives the ability to set the [durability](./reference/durability.md) and other advanced concepts.
+
 ## Tracked functions
 
 Once you've defined your inputs, the next thing to define are **tracked functions**:
@@ -147,12 +150,13 @@ Tracked functions can return any clone-able type. A clone is required since, whe
 
 **Tracked structs** are intermediate structs created during your computation.
 Like inputs, their fields are stored inside the database, and the struct itself just wraps an id.
-Unlike inputs, they can only be created inside a tracked function, and their fields can never change once they are created.
-Getter methods are provided to read the fields, but there are no setter methods[^specify]. Example:
+Unlike inputs, they can only be created inside a tracked function, and their fields can never change once they are created (until the next revision, at least).
+Getter methods are provided to read the fields, but there are no setter methods. 
+Example:
 
 ```rust
 #[salsa::tracked]
-struct Ast {
+struct Ast<'db> {
     #[return_ref]
     top_level_items: Vec<Item>,
 }

book/src/plumbing/tracked_structs.md

@@ -10,12 +10,12 @@ For a single tracked struct we create multiple ingredients.
 The **tracked struct ingredient** is the ingredient created first.
 It offers methods to create new instances of the struct and therefore
 has unique access to the interner and hashtables used to create the struct id.
-It also shares access to a hashtable that stores the `TrackedStructValue` that
+It also shares access to a hashtable that stores the `ValueStruct` that
 contains the field data.
 
 For each field, we create a **tracked field ingredient** that moderates access
 to a particular field. All of these ingredients use that same shared hashtable
-to access the `TrackedStructValue` instance for a given id. The `TrackedStructValue`
+to access the `ValueStruct` instance for a given id. The `ValueStruct`
 contains both the field values but also the revisions when they last changed value.
 
 ## Each tracked struct has a globally unique id
@@ -26,13 +26,13 @@ This will begin by creating a *globally unique, 32-bit id* for the tracked struc
 * a u64 hash of the `#[id]` fields;
 * a *disambiguator* that makes this hash unique within the current query. i.e., when a query starts executing, it creates an empty map, and the first time a tracked struct with a given hash is created, it gets disambiguator 0. The next one will be given 1, etc.
 
-## Each tracked struct has a `TrackedStructValue` storing its data
+## Each tracked struct has a `ValueStruct` storing its data
 
 The struct and field ingredients share access to a hashmap that maps
 each field id to a value struct:
 
 ```rust,ignore
-{{#include ../../../components/salsa-2022/src/tracked_struct.rs:TrackedStructValue}}
+{{#include ../../../components/salsa-2022/src/tracked_struct.rs:ValueStruct}}
 ```
 
 The value struct stores the values of the fields but also the revisions when 

book/src/reference/durability.md

@@ -0,0 +1,13 @@
+# Durability
+
+"Durability" is an optimization that can greatly improve the performance of your salsa programs.
+Durability specifies the probably that an input's value will change.
+The default is "low durability".
+But when you set the value of an input, you can manually specify a higher durability,
+typically `Durability::HIGH`.
+Salsa tracks when tracked functions only consume values of high durability
+and, if no high durability input has changed, it can skip traversing their
+dependencies.
+
+Typically "high durability" values are things like data read from the standard library
+or other inputs that aren't actively being edited by the end user.

components/salsa-2022-macros/Cargo.toml

@@ -10,5 +10,6 @@ proc-macro = true
 heck = "0.4"
 proc-macro2 = "1.0"
 quote = "1.0"
-syn = { version = "1.0", features = ["full", "extra-traits", "visit-mut"] }
 eyre = "0.6.5"
+syn = { version = "2.0.64", features = ["visit-mut"] }
+synstructure = "0.13.1"

components/salsa-2022-macros/src/configuration.rs

@@ -1,20 +1,22 @@
 pub(crate) struct Configuration {
+    pub(crate) db_lt: syn::Lifetime,
     pub(crate) jar_ty: syn::Type,
     pub(crate) salsa_struct_ty: syn::Type,
-    pub(crate) key_ty: syn::Type,
+    pub(crate) input_ty: syn::Type,
     pub(crate) value_ty: syn::Type,
     pub(crate) cycle_strategy: CycleRecoveryStrategy,
-    pub(crate) backdate_fn: syn::ImplItemMethod,
-    pub(crate) execute_fn: syn::ImplItemMethod,
-    pub(crate) recover_fn: syn::ImplItemMethod,
+    pub(crate) backdate_fn: syn::ImplItemFn,
+    pub(crate) execute_fn: syn::ImplItemFn,
+    pub(crate) recover_fn: syn::ImplItemFn,
 }
 
 impl Configuration {
     pub(crate) fn to_impl(&self, self_ty: &syn::Type) -> syn::ItemImpl {
         let Configuration {
+            db_lt,
             jar_ty,
             salsa_struct_ty,
-            key_ty,
+            input_ty,
             value_ty,
             cycle_strategy,
             backdate_fn,
@@ -24,9 +26,9 @@ impl Configuration {
         parse_quote! {
             impl salsa::function::Configuration for #self_ty {
                 type Jar = #jar_ty;
-                type SalsaStruct = #salsa_struct_ty;
-                type Key = #key_ty;
-                type Value = #value_ty;
+                type SalsaStruct<#db_lt> = #salsa_struct_ty;
+                type Input<#db_lt> = #input_ty;
+                type Value<#db_lt> = #value_ty;
                 const CYCLE_STRATEGY: salsa::cycle::CycleRecoveryStrategy = #cycle_strategy;
                 #backdate_fn
                 #execute_fn
@@ -56,16 +58,16 @@ impl quote::ToTokens for CycleRecoveryStrategy {
 
 /// Returns an appropriate definition for `should_backdate_value` depending on
 /// whether this value is memoized or not.
-pub(crate) fn should_backdate_value_fn(should_backdate: bool) -> syn::ImplItemMethod {
+pub(crate) fn should_backdate_value_fn(should_backdate: bool) -> syn::ImplItemFn {
     if should_backdate {
         parse_quote! {
-            fn should_backdate_value(v1: &Self::Value, v2: &Self::Value) -> bool {
+            fn should_backdate_value(v1: &Self::Value<'_>, v2: &Self::Value<'_>) -> bool {
                 salsa::function::should_backdate_value(v1, v2)
             }
         }
     } else {
         parse_quote! {
-            fn should_backdate_value(_v1: &Self::Value, _v2: &Self::Value) -> bool {
+            fn should_backdate_value(_v1: &Self::Value<'_>, _v2: &Self::Value<'_>) -> bool {
                 false
             }
         }
@@ -74,13 +76,13 @@ pub(crate) fn should_backdate_value_fn(should_backdate: bool) -> syn::ImplItemMe
 
 /// Returns an appropriate definition for `recover_from_cycle` for cases where
 /// the cycle recovery is panic.
-pub(crate) fn panic_cycle_recovery_fn() -> syn::ImplItemMethod {
+pub(crate) fn panic_cycle_recovery_fn() -> syn::ImplItemFn {
     parse_quote! {
-        fn recover_from_cycle(
-            _db: &salsa::function::DynDb<Self>,
+        fn recover_from_cycle<'db>(
+            _db: &'db salsa::function::DynDb<'db, Self>,
             _cycle: &salsa::Cycle,
-            _key: Self::Key,
-        ) -> Self::Value {
+            _key: salsa::Id,
+        ) -> Self::Value<'db> {
             panic!()
         }
     }

components/salsa-2022-macros/src/db_lifetime.rs

@@ -0,0 +1,62 @@
+//! Helper functions for working with fns, structs, and other generic things
+//! that are allowed to have a `'db` lifetime.
+
+use proc_macro2::Span;
+use syn::spanned::Spanned;
+
+/// Normally we try to use whatever lifetime parameter the use gave us
+/// to represent `'db`; but if they didn't give us one, we need to use a default
+/// name. We choose `'db`.
+pub(crate) fn default_db_lifetime(span: Span) -> syn::Lifetime {
+    syn::Lifetime {
+        apostrophe: span,
+        ident: syn::Ident::new("db", span),
+    }
+}
+
+/// Require that either there are no generics or exactly one lifetime parameter.
+pub(crate) fn require_optional_db_lifetime(generics: &syn::Generics) -> syn::Result<()> {
+    if generics.params.is_empty() {
+        return Ok(());
+    }
+
+    require_db_lifetime(generics)?;
+
+    Ok(())
+}
+
+/// Require that either there is exactly one lifetime parameter.
+pub(crate) fn require_db_lifetime(generics: &syn::Generics) -> syn::Result<()> {
+    if generics.params.is_empty() {
+        return Err(syn::Error::new_spanned(
+            generics,
+            "this definition must have a `'db` lifetime",
+        ));
+    }
+
+    for (param, index) in generics.params.iter().zip(0..) {
+        let error = match param {
+            syn::GenericParam::Lifetime(_) => index > 0,
+            syn::GenericParam::Type(_) | syn::GenericParam::Const(_) => true,
+        };
+
+        if error {
+            return Err(syn::Error::new_spanned(
+                param,
+                "only a single lifetime parameter is accepted",
+            ));
+        }
+    }
+
+    Ok(())
+}
+
+/// Return the `'db` lifetime given be the user, or a default.
+/// The generics ought to have been checked with `require_db_lifetime` already.
+pub(crate) fn db_lifetime(generics: &syn::Generics) -> syn::Lifetime {
+    if let Some(lt) = generics.lifetimes().next() {
+        lt.lifetime.clone()
+    } else {
+        default_db_lifetime(generics.span())
+    }
+}

components/salsa-2022-macros/src/debug.rs

@@ -0,0 +1,40 @@
+use std::io::Write;
+use std::process::{Command, Stdio};
+use std::sync::OnceLock;
+
+use proc_macro2::TokenStream;
+
+static SALSA_DEBUG_MACRO: OnceLock<Option<String>> = OnceLock::new();
+
+pub(crate) fn debug_enabled(input_name: impl ToString) -> bool {
+    let Some(env_name) = SALSA_DEBUG_MACRO.get_or_init(|| std::env::var("SALSA_DEBUG_MACRO").ok())
+    else {
+        return false;
+    };
+
+    let input_name = input_name.to_string();
+    env_name == "*" || env_name == &input_name[..]
+}
+
+pub(crate) fn dump_tokens(input_name: impl ToString, tokens: TokenStream) -> TokenStream {
+    if debug_enabled(input_name) {
+        let token_string = tokens.to_string();
+
+        let _: Result<(), ()> = Command::new("rustfmt")
+            .arg("--emit=stdout")
+            .stdin(Stdio::piped())
+            .spawn()
+            .and_then(|mut rustfmt| {
+                rustfmt
+                    .stdin
+                    .take()
+                    .unwrap()
+                    .write_all(token_string.as_bytes())?;
+                rustfmt.wait_with_output()
+            })
+            .map(|output| eprintln!("{}", String::from_utf8_lossy(&output.stdout)))
+            .or_else(|_| Ok(eprintln!("{token_string}")));
+    }
+
+    tokens
+}

components/salsa-2022-macros/src/debug_with_db.rs

@@ -0,0 +1,95 @@
+use proc_macro2::{Literal, Span, TokenStream};
+
+pub(crate) fn debug_with_db(input: syn::DeriveInput) -> syn::Result<proc_macro2::TokenStream> {
+    // Figure out the lifetime to use for the `dyn Db` that we will expect.
+    // We allow structs to have at most one lifetime -- if a lifetime parameter is present,
+    // it should be `'db`. We may want to generalize this later.
+
+    let num_lifetimes = input.generics.lifetimes().count();
+    if num_lifetimes > 1 {
+        return syn::Result::Err(syn::Error::new(
+            input.generics.lifetimes().nth(1).unwrap().lifetime.span(),
+            "only one lifetime is supported",
+        ));
+    }
+
+    let db_lt = match input.generics.lifetimes().next() {
+        Some(lt) => lt.lifetime.clone(),
+        None => syn::Lifetime::new("'_", Span::call_site()),
+    };
+
+    // Generate the type of database we expect. This hardcodes the convention of using `jar::Jar`.
+    // That's not great and should be fixed but we'd have to add a custom attribute and I am too lazy.
+
+    #[allow(non_snake_case)]
+    let DB: syn::Type = parse_quote! {
+        <crate::Jar as salsa::jar::Jar< #db_lt >>::DynDb
+    };
+
+    let structure: synstructure::Structure = synstructure::Structure::new(&input);
+
+    let fmt = syn::Ident::new("fmt", Span::call_site());
+    let db = syn::Ident::new("db", Span::call_site());
+
+    // Generic the match arm for each variant.
+    let fields: TokenStream = structure
+        .variants()
+        .iter()
+        .map(|variant| {
+            let variant_name = &variant.ast().ident;
+            let variant_name = Literal::string(&variant_name.to_string());
+
+            // Closure: given a binding, generate a call to the `salsa_debug` helper to either
+            // print its "debug with db" value or just use `std::fmt::Debug`. This is a nice hack that
+            // lets us use `debug_with_db` when available; it won't work great for generic types unless we add
+            // `DebugWithDb` bounds though.
+            let binding_tokens = |binding: &synstructure::BindingInfo| {
+                let field_ty = &binding.ast().ty;
+                quote!(
+                    &::salsa::debug::helper::SalsaDebug::<#field_ty, #DB>::salsa_debug(
+                        #binding,
+                        #db,
+                    )
+                )
+            };
+
+            // Create something like `fmt.debug_struct(...).field().field().finish()`
+            // for each struct field; the values to be debugged are created by
+            // the `binding_tokens` closure above.
+            let fields = match variant.ast().fields {
+                syn::Fields::Named(_) => variant.fold(
+                    quote!(#fmt.debug_struct(#variant_name)),
+                    |tokens, binding| {
+                        let binding_name =
+                            Literal::string(&binding.ast().ident.as_ref().unwrap().to_string());
+                        let binding_data = binding_tokens(binding);
+                        quote!(#tokens . field(#binding_name, #binding_data))
+                    },
+                ),
+
+                syn::Fields::Unnamed(_) | syn::Fields::Unit => variant.fold(
+                    quote!(#fmt.debug_tuple(#variant_name)),
+                    |tokens, binding| {
+                        let binding_data = binding_tokens(binding);
+                        quote!(#tokens . field(#binding_data))
+                    },
+                ),
+            };
+
+            quote!(#fields . finish(),)
+        })
+        .collect();
+
+    let tokens = structure.gen_impl(quote! {
+        gen impl ::salsa::debug::DebugWithDb<#DB> for @Self {
+            fn fmt(&self, #fmt: &mut std::fmt::Formatter<'_>, #db: & #DB) -> std::fmt::Result {
+                use ::salsa::debug::helper::Fallback as _;
+                match self {
+                    #fields
+                }
+            }
+        }
+    });
+
+    Ok(crate::debug::dump_tokens(&input.ident, tokens))
+}