⚡️ v2.6.14 Speed up dtype enforcement, fix Oracle auto-incrementing IDs.
v2.6.10 – v2.6.14
-
Improve datetime timezone-awareness enforcement performance.
Datetime columns are only parsed for timezone awareness if the desired awareness differs. This drastically speeds up sync times. -
Switch to
tz_localize()
when stripping timezone information.
The previous method of using a lambda to replace individualtzinfo
attributes did not scale well. Usingtz_localize()
can be vectorized and greatly speeds up syncs, especially with large chunks. -
Add
enforce_dtypes
toPipe.filter_existing()
.
You may optionally enforce dtype information duringfilter_existing()
. This may be useful when implementing custom syncs for instance connectors. Note this may impact memory and compute performance.import meerschaum as mrsm import pandas as pd pipe = mrsm.Pipe('a', 'b', instance='sql:local') pipe.sync([{'a': 1}]) df = pd.DataFrame([{'a': '2'}]) ### `enforce_dtypes=True` will suppress the differing dtypes warning. unseen, update, delta = pipe.filter_existing(df, enforce_dtypes=True) print(delta)
-
Fix
query_df()
for null parameters.
This is useful for when you may usequery_df()
with onlyselect_columns
oromit_columns
. -
Fix autoincrementing IDs for Oracle SQL.
-
Enforce security settings for creating jobs.
Jobs and remote actions will only be accessible to admin users when running with--secure
(system:permissions:actions:non_admin
in config).