Skip to content

v1.5.6

Compare
Choose a tag to compare
@bmeares bmeares released this 16 Jan 21:22
· 151 commits to main since this release
a3202fe

v1.5.3 – v1.5.6

  • Pipes now support syncing dictionaries and lists.
    Complex columns (dicts or lists) will now be preserved:

    import meerschaum as mrsm
    pipe = mrsm.Pipe('a', 'b')
    pipe.sync([{'a': {'b': 1}}])
    df = pipe.get_data()
    print(df['a'][0])
    # {'b': 1}

    You can also force strings to be parsed by setting the data type to json:

    import meerschaum as mrsm
    pipe = mrsm.Pipe(
        'foo', 'bar',
        columns = {'datetime': 'id'},
        dtypes = {'data': 'json', 'id': 'Int64'},
    )
    docs = [{'id': 1, 'data': '{"foo": "bar"}'}]
    pipe.sync(docs)
    df = pipe.get_data()
    print(df['data'][0])
    # {'foo': 'bar'}

    For PostgreSQL-like databases (e.g. TimescaleDB), this is stored as JSONB under the hood. For all others, it's stored as the equivalent for TEXT.

  • Fixed determining the version when installing plugins.
    Like the required list, the __version__ string must be explicitly set in order for the correct version to be determined.

  • Automatically cast postgres to postgresql
    When a SQLConnector is built with a flavor of postgres, it will be automatically set to postgresql.