Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a custom SELECT with SAMPLE BY support #27

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

jerrinot
Copy link
Contributor

@jerrinot jerrinot commented Jan 3, 2025

TODO:

  • align to ... + offsets
  • polish API
  • tests, including both sqlalchemy 1.4 and 2.0 and in combination with other clauses (GROUP BY etc)
  • documentation

Example usage:

from sqlalchemy import create_engine, MetaData, Table, Column
from questdb_connect import (
    Timestamp,
    Double,
    Symbol,
)
from sqlalchemy import func
from questdb_connect import select

engine = create_engine('questdb://admin:quest@localhost:8812/main')
metadata = MetaData()

# Define a table for sensor readings
sensors = Table(
    'sensors',
    metadata,
    Column('ts', Timestamp),
    Column('temperature', Double),
    Column('humidity', Double),
    Column('location', Symbol),
)

def main():
    metadata.create_all(engine)

    location_samples = select(
        sensors.c.ts,
        func.avg(sensors.c.temperature).label('avg_temp'),
        func.min(sensors.c.temperature).label('min_temp'),
        func.max(sensors.c.temperature).label('max_temp')
    ).where(
        sensors.c.location == 'warehouse'
    ).sample_by(1, 'd');

    with engine.connect() as conn:
        for row in conn.execute(location_samples).fetchall():
            print(f"Time: {row.ts}, Average Temp: {row.avg_temp}, Minimal Temp: {row.min_temp}, Maximal Temp: {row.max_temp}")

if __name__ == '__main__':
    main()

TODO:
- tests, including both sqlalchemy 1.4 and 2.0 and in combination with other clauses (GROUP BY etc)
- documentation

Example usage:
```python
from sqlalchemy import create_engine, MetaData, Table, Column
from questdb_connect import (
    Timestamp,
    Double,
    Symbol,
)
from sqlalchemy import func
from questdb_connect import select

engine = create_engine('questdb://admin:quest@localhost:8812/main')
metadata = MetaData()

# Define a table for sensor readings
sensors = Table(
    'sensors',
    metadata,
    Column('ts', Timestamp),
    Column('temperature', Double),
    Column('humidity', Double),
    Column('location', Symbol),
)

def main():
    metadata.create_all(engine)

    location_samples = select(
        sensors.c.ts,
        func.avg(sensors.c.temperature).label('avg_temp'),
        func.min(sensors.c.temperature).label('min_temp'),
        func.max(sensors.c.temperature).label('max_temp')
    ).where(
        sensors.c.location == 'warehouse'
    ).sample_by(1, 'd');

    with engine.connect() as conn:
        for row in conn.execute(location_samples).fetchall():
            print(f"Time: {row.ts}, Average Temp: {row.avg_temp}, Minimal Temp: {row.min_temp}, Maximal Temp: {row.max_temp}")

if __name__ == '__main__':
    main()
```
this impl no longer depends on SQL Text post-processing
and uses clause rendering instead. this makes it more robust.
@jerrinot
Copy link
Contributor Author

@nwoolmer I'm making this a non-draft even docs is still missing, I believe impl is now alright.

your review is very much appreciated!
some points:

  1. sample_by() receives a lot of arguments. most of them are optional and given Python supports named parameters it's probably fine. wdyt? You certainly know the Python ecosystem and customers better than I do
  2. not sure syntex for sampling intervals. right now the arguments are separated as value and unit. wdyt?
  3. I did not implement any kind of a validation. for example FROM-TO does not support some FILL options, etc. so the extension could validate this on a client side. otoh: we would need to change this if/when we implement it on server-side.

@jerrinot jerrinot marked this pull request as ready for review January 14, 2025 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant