streaming dynamic queries

[cemoktra]

I implemented a pretty convenient way to handle dynamic queries. This works fine, except that i cannot get streaming to work as i need the reference to the sql string being 'static.

Basically i have a SQL string and the arguments buffer. While i can easily have a static executor i cannot have a static SQL query string for dynamic queries.

Is there a way to put the SQL string as owned data into the stream implementation?

[CommanderStorm]

Could you please share an example based on your current code?
That makes recommending good suggestions much simpler

[cemoktra]

The initial idea was to own the sql statement in this struct and impl the Stream trait, so that the Stream owns its data, which did not work.

So i tried to have a reference with query lifetime 'q. But now the fetch results in:

247 |             inner: query.fetch(executor),
    |                    ^^^^^^^^^^^^^^^^^^^^^ cast requires that `'q` must outlive `'static`

struct DynamicStream<'q, T> {
    sql: &'q str,
    inner: Pin<Box<dyn Stream<Item = Result<T, sqlx::Error>>>>,
}

impl<'q, T> DynamicStream<'q, T> {
    pub fn new<E>(sql: &'q str, arguments: PgArguments, executor: E) -> Self
    where
        E: sqlx::Executor<'static, Database = sqlx::Postgres>,
        T: for<'r> sqlx::FromRow<'r, sqlx::postgres::PgRow> + Send + Unpin,
    {
        let query = sqlx::query_as_with::<'q, Postgres, T, PgArguments>(&sql, arguments);

        Self {
            sql,
            inner: query.fetch(executor),
        }
    }
}

But i can't get a static reference to the query, as it is a dynamic query. If i'm not mistaken this whole stuff would work, if the stream impl in the executor owns the sql string

[CommanderStorm]

I am somewhat doubtfull, that the complexity of the requried bounds does pay off, but this is your call. (keep it simple, even though overingeneering is fun)

Lets think this through..

If sql cannot go via the 'static escape hatch, we need to actually think about where which livetimes live.
=> we need to track the returned lifetime
=> we need to make the executor not 'static

my version should help you to a compiling solution. My bounds might be too coupled for your taste..

// 'e => livetime of output and executor
// 'q => lifetime of inner borrowed str

struct DynamicStream<'e,'q, T> {
    sql: &'q str,
    inner: Pin<Box<dyn Stream<Item = Result<T, sqlx::Error>>+'e>>,
}

impl<'e,'q:'e, T: 'e> DynamicStream<'e,'q, T> {
    pub fn new<E>(sql: &'q str, arguments: PgArguments, executor: E) -> Self
    where
        E: sqlx::Executor<'e, Database = sqlx::Postgres> + 'e,
        T: for<'r> sqlx::FromRow<'r, sqlx::postgres::PgRow> + Send + Unpin,
    {
        let query = sqlx::query_as_with::<'q, Postgres, T, PgArguments>(&sql, arguments);

        Self {
            sql,
            inner: query.fetch(executor),
        }
    }
}

[abonander]

My proposed API to resolve this is in #3364

It needs more work still, and it's a breaking change anyway, but this is the direction I want to go in.