-
Notifications
You must be signed in to change notification settings - Fork 11
Comparing changes
Open a pull request
base repository: fox-it/flow.record
base: 3.9
head repository: fox-it/flow.record
compare: main
Commits on Mar 16, 2023
-
Configuration menu - View commit details
-
Copy full SHA for cb20bb9 - Browse repository at this point
Copy the full SHA cb20bb9View commit details
Commits on Mar 17, 2023
-
Always write Avro header (#62)
* Always write Avro header * Added avro and test extras to pyproject.toml * Also skip lz4 and zstandard tests when running under PyPy due to incompatibilities * Remove avro[snappy] from test due to missing wheels * Add unit test --------- Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for d182309 - Browse repository at this point
Copy the full SHA d182309View commit details
Commits on Apr 13, 2023
-
Fix invalid expression when parsing datetime type in Avro adapter (#63)
Co-authored-by: Jan Starke <jan.starke@t-systems.com>
Configuration menu - View commit details
-
Copy full SHA for 059c300 - Browse repository at this point
Copy the full SHA 059c300View commit details
Commits on Apr 26, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 1db1743 - Browse repository at this point
Copy the full SHA 1db1743View commit details
Commits on May 8, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 1de369f - Browse repository at this point
Copy the full SHA 1de369fView commit details
Commits on May 15, 2023
-
Improve the path field type (#66)
The path class will now allow being instantiated using multiple path parts, making it behave more like its parent pathlib.PurePath class. The inference on whether to return a windows_path or posix_path class instance is now done based on the first path part it encounters that is an instance of pathlib.PurePath. For path parts that are a custom subclass of pathlib.PurePath (so not an instance of PureWindowsPath or PurePosixPath), the presence of a '\' as either separator or alternative separator will result in returning a windows_path class instance. (DIS-1977)
Configuration menu - View commit details
-
Copy full SHA for 4b51ec6 - Browse repository at this point
Copy the full SHA 4b51ec6View commit details
Commits on May 16, 2023
-
Add Requirements section with URL to the supported Python versions. Update links to main documentation. (DIS-1986)
Configuration menu - View commit details
-
Copy full SHA for ee0cb69 - Browse repository at this point
Copy the full SHA ee0cb69View commit details
Commits on May 23, 2023
-
Add documentation testing tooling (#69)
* Add documentation testing tooling Add tox commands to generate API documentation for previewing in browser and automatic checking of broken URLs. (DIS-1888) * Remove unnecessary newline from tox.ini Make the formatting consistent with the other sections.
Configuration menu - View commit details
-
Copy full SHA for a1a94b2 - Browse repository at this point
Copy the full SHA a1a94b2View commit details
Commits on May 31, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 6aa3aa5 - Browse repository at this point
Copy the full SHA 6aa3aa5View commit details
Commits on Jun 16, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 6c1da47 - Browse repository at this point
Copy the full SHA 6c1da47View commit details
Commits on Jun 27, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 549d118 - Browse repository at this point
Copy the full SHA 549d118View commit details
Commits on Jul 21, 2023
-
Add hash method to datetime fieldtype (#77)
* Add hash method to datetime fieldtype (DIS-2036)
Configuration menu - View commit details
-
Copy full SHA for d52f2b5 - Browse repository at this point
Copy the full SHA d52f2b5View commit details
Commits on Aug 1, 2023
-
Add --skip flag to rdump (#76)
Adds the ability to skip a number of records when reading or writing records using rdump. Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for 464914e - Browse repository at this point
Copy the full SHA 464914eView commit details
Commits on Aug 24, 2023
-
Make datetime fieldtypes timezone aware (#78)
- Record datetime fields are now offset-aware by default - Naive datetime fields are converted to UTC - Support for packing/unpacking aware datetimes Note that comparing to naive datetime objects will now break and is also in line with default Python behaviour. To ensure uniform datetime field output they are always displayed in UTC. To use a different display timezone you can set the environment variable `FLOW_RECORD_TZ`. Examples: - `FLOW_RECORD_TZ=UTC` to display datetime fields in UTC, this is the default - `FLOW_RECORD_TZ=Europe/Amsterdam` to display datetime fields in local time of the Netherlands - `FLOW_RECORD_TZ=NONE` to disable the datetime display normalisation --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 7cc4440 - Browse repository at this point
Copy the full SHA 7cc4440View commit details -
Configuration menu - View commit details
-
Copy full SHA for ddea907 - Browse repository at this point
Copy the full SHA ddea907View commit details
Commits on Aug 25, 2023
-
Add path type __eq__ and __repr__ QOL changes (#79)
Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for b2818b0 - Browse repository at this point
Copy the full SHA b2818b0View commit details -
Add GitHub workflow to test extra compatibility (#81)
As flow.record still targets Python 3.7 as a minimum, we also test: - Python 3.7 - Python 3.8 - Windows, Python 3.9+
Configuration menu - View commit details
-
Copy full SHA for 8a1c74f - Browse repository at this point
Copy the full SHA 8a1c74fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6358ba3 - Browse repository at this point
Copy the full SHA 6358ba3View commit details
Commits on Sep 13, 2023
-
Support file-like inputs for RecordReader (#59)
Peek into the file to find the right adapter by checking the file magic --------- Co-authored-by: Max Groot <max.groot@fox-it.com> Co-authored-by: Yun Zheng Hu <hu@fox-it.com> Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 2e2eb62 - Browse repository at this point
Copy the full SHA 2e2eb62View commit details
Commits on Oct 11, 2023
-
Add behaviour to always use datetime.UTC if there is no zoneinfo avai…
…lable (#86) Add behaviour to always use datetime.UTC if there is no zoneinfo available --------- Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for ccfa214 - Browse repository at this point
Copy the full SHA ccfa214View commit details
Commits on Oct 13, 2023
-
Python 3.7 is EOL since 27 June 2023. Minimal supported Python version for flow.record is now 3.8. Windows tests are now tested via the main github dissect-workflow-template.
Configuration menu - View commit details
-
Copy full SHA for f0a2608 - Browse repository at this point
Copy the full SHA f0a2608View commit details
Commits on Oct 16, 2023
-
Speedup parsing of datetime fieldtypes initialization by string (#87)
This change mainly removes the use of expensive regexes and exception handling when parsing datetime strings. It speeds up parsing significantly on Python versions below 3.11. --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 53c744b - Browse repository at this point
Copy the full SHA 53c744bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ecbd912 - Browse repository at this point
Copy the full SHA ecbd912View commit details
Commits on Oct 26, 2023
-
Configuration menu - View commit details
-
Copy full SHA for fdcecba - Browse repository at this point
Copy the full SHA fdcecbaView commit details -
Update
elastic.py
adapter (#92)Fix AttributeError if invalid uri is given, add `verify_certs` flag to optional arguments. Also adds `hash_record` argument for making every document unique.
Configuration menu - View commit details
-
Copy full SHA for 6144cf4 - Browse repository at this point
Copy the full SHA 6144cf4View commit details
Commits on Oct 27, 2023
-
Fix RecordReader not reading from stdin by default (#94)
Calling `RecordReader()` without arguments should always default to stdin
Configuration menu - View commit details
-
Copy full SHA for 9cad89f - Browse repository at this point
Copy the full SHA 9cad89fView commit details
Commits on Nov 15, 2023
-
This adds support for reading from and writing to SQLite database via `sqlite://`. Columns are dynamically added if a RecordDescriptor changes for the table. --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 67f71d5 - Browse repository at this point
Copy the full SHA 67f71d5View commit details
Commits on Nov 17, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 64263bc - Browse repository at this point
Copy the full SHA 64263bcView commit details
Commits on Nov 20, 2023
-
Refactor and improve CSV adapter (#96)
* Refactor and improve CSV adapter This change allows the CSV adapter to: - ability to read reserved fields (eg: _generated, _source, etc) - use `normalize_fieldname` to normalize field names in flow.record - deduce format of csv file automatically by using `csv.Sniffer`
Configuration menu - View commit details
-
Copy full SHA for 676d61c - Browse repository at this point
Copy the full SHA 676d61cView commit details
Commits on Dec 20, 2023
-
Use reprlib to limit the warning message (#101)
The warning message could get very long (single line) due to extra data in msgpack parsing. For example due to corrupt file, or incorrect use. By using reprlib we limit the length of the warning message.
Configuration menu - View commit details
-
Copy full SHA for a9808ec - Browse repository at this point
Copy the full SHA a9808ecView commit details
Commits on Jan 4, 2024
-
Add Python 3.12 compatibility for path fieldtype (#91)
--------- Co-authored-by: pyrco <105293448+pyrco@users.noreply.github.com> Co-authored-by: Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 58d5915 - Browse repository at this point
Copy the full SHA 58d5915View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9670a38 - Browse repository at this point
Copy the full SHA 9670a38View commit details
Commits on Jan 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 9a6829b - Browse repository at this point
Copy the full SHA 9a6829bView commit details
Commits on Feb 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 09ed812 - Browse repository at this point
Copy the full SHA 09ed812View commit details -
Configuration menu - View commit details
-
Copy full SHA for 67a36e8 - Browse repository at this point
Copy the full SHA 67a36e8View commit details
Commits on Feb 19, 2024
-
This adds DuckDB reader and writer support. Because DuckDB is mostly compatible with the SQLite API, we just subclass from the existing SQLite adapter with minimal changes. Changes done the SQLite adapter and tests are: * backtick quoting does not work in DuckDB so we use double quotes now * DuckDB has strict typing so some tests are not applicable and skipped * `executescript()` does not exist in DuckDB so we avoid using it * Switched SQLite to `isolation_level=None` for manual transactions --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 2f52023 - Browse repository at this point
Copy the full SHA 2f52023View commit details
Commits on Feb 20, 2024
-
Move lru_cache definitions to __init__ (#109)
Using the lru_cache decorators on class methods, the ones that have a reference to `self`, will also cache self. So we move it to the __init__ of the class (DIS-2913) --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for abae08c - Browse repository at this point
Copy the full SHA abae08cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 523b96c - Browse repository at this point
Copy the full SHA 523b96cView commit details
Commits on Mar 7, 2024
-
--------- Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for a3b5310 - Browse repository at this point
Copy the full SHA a3b5310View commit details
Commits on Mar 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5933d34 - Browse repository at this point
Copy the full SHA 5933d34View commit details
Commits on Mar 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4d267dd - Browse repository at this point
Copy the full SHA 4d267ddView commit details
Commits on Mar 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 256733d - Browse repository at this point
Copy the full SHA 256733dView commit details
Commits on Apr 5, 2024
-
Add ignore_fields_for_comparison() context manager (#115)
It behaves the same as set_ignored_fields_for_comparison() but only for the duration of the context manager.
Configuration menu - View commit details
-
Copy full SHA for 5b9e62a - Browse repository at this point
Copy the full SHA 5b9e62aView commit details
Commits on Apr 11, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c1c3abf - Browse repository at this point
Copy the full SHA c1c3abfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9eb1557 - Browse repository at this point
Copy the full SHA 9eb1557View commit details
Commits on Apr 12, 2024
-
Add support for Splunk HTTP Event Collector (#85)
--------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com> Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for 4a47670 - Browse repository at this point
Copy the full SHA 4a47670View commit details
Commits on May 3, 2024
-
This command type splits an executable (path) from its arguments (list). There is a best effort detection for windows type commands. This is because windows executables handles its own argument parsing. (DIS-2977)
Configuration menu - View commit details
-
Copy full SHA for e0586ef - Browse repository at this point
Copy the full SHA e0586efView commit details
Commits on May 15, 2024
-
Add metadata fields to elastic adapter (#121)
This adds metadata fields to the elastic adapter and repairs `elastic+[PROTOCOL]://` behaviour. It also enables users to authenticate to Elasticsearch with an API key. You can now write arbitrary metadata to the `document._source._record_metadata` dict using the following syntax: rdump -w "elastic+https://localhost:9200?_meta_foo=bar" This will result in the following `_record_metadata` dict: { ... "foo": "bar" } --------- Co-authored-by: Yun Zheng Hu <hu@fox-it.com>
Configuration menu - View commit details
-
Copy full SHA for 43a5656 - Browse repository at this point
Copy the full SHA 43a5656View commit details
Commits on May 16, 2024
-
Add support for empty value in path fieldtype (#122)
* Add support for empty value in path fieldtype Normally an empty path would be normalized to a "." (dot) character. This change allows you to initialize a path field with an empty string. This is useful to represent a path that is empty. Fixes DIS-2557 * Also initialise subclass * Move empty_path attribute to __new__ * Fix comparison between windows and posix classes
Configuration menu - View commit details
-
Copy full SHA for 0865b50 - Browse repository at this point
Copy the full SHA 0865b50View commit details
Commits on May 20, 2024
-
Fix ValueError: I/O operation on closed file during tests (#123)
Sometimes the `stdout` file object is closed by `flow.record` internals as it is sometimes mocked and swapped by `pytest` during tests which in turn can confuse `is_stdout()` to return `False` causing the file to be closed. This is now fixed by adding two custom methods for getting the stdio streams: * `flow.record.utils.get_stdout()` * `flow.record.utils.get_stdin()` These methods are the preferred way to get the stdio streams as they also set an extra attribute on the returned file object that is checked by `is_stdout()`. --------- Co-authored-by: Erik Schamper <1254028+Schamper@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for a8fd59d - Browse repository at this point
Copy the full SHA a8fd59dView commit details
There are no files selected for viewing