v1.4.4
v1.4.0 – v1.4.4
-
Added in-place syncing for SQL pipes.
This feature is big (enough to warrant a new point release). When pipes with the same instance connector and data source connector are synced, the methodsync_pipe_inplace()
is invoked. For SQL pipes, this means the entire syncing process will now happen entirely in SQL, which can lead to massive performance improvements.import meerschaum as mrsm import pandas as pd conn = mrsm.get_connector('sql', 'local') conn.to_sql(pd.DataFrame([{'a': 1}]), 'foo') pipe = mrsm.Pipe( "sql:local", "foo", instance = "sql:local", parameters = { "query": "SELECT * FROM foo" }, ) ### This will no longer load table data into memory. pipe.sync()
This applies even when the source table's schema changes, just like the dynamic columns feature added in v1.3.0.
To disable this behavior, run the command
edit config system
and set the value under the keysexperimental:inplace_sync
tofalse
. -
Added negation to
--params
.
Thebuild_where()
function now allows you to negate certain values when prefixed with an underscore (_
):### Show recent data, excluding where `id` equals `2`. mrsm show data --params id:_2
-
Added
--params
to SQL pipes' queries.
Specifying parameters when syncing SQL pipes will add those constraints to the fetch stage. -
Skip invalid parameters in
--params
.
If a column does not exist in a pipe's table, the value will be ignored in--params
. -
Fixed environment issue when starting the Web API with
gunicorn
. -
Added an emoji to the SQL Query option of the web console.
-
Fixed an edge case with data type enforcement.
-
Other bugfixes