You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a analyzer I want to be able to work with data in CSV files so I can standardize my analysis
Description
We want to be able to read data from weird CSV and TSV formats from an experiment. We will use the Pandas library as the basis for this transformer.
We will use the python code generator with the following assumptions:
The function will be called with an open file handle
The function will return a Pandas dataframe
There will be a new python code generator based on the uproot python code generator. Its transform_single_file.py script will write the data frame to parquet using the dataframe to_parquet method.
Story
As a analyzer I want to be able to work with data in CSV files so I can standardize my analysis
Description
We want to be able to read data from weird CSV and TSV formats from an experiment. We will use the Pandas library as the basis for this transformer.
We will use the python code generator with the following assumptions:
There will be a new python code generator based on the uproot python code generator. Its
transform_single_file.py
script will write the data frame to parquet using the dataframe to_parquet method.It looks like it is possible to stream directly from the Dataframe to a parquet object in minio https://stackoverflow.com/a/57838851
The text was updated successfully, but these errors were encountered: