Python version 3.1.1

What's Changed

• Limit service client operations to 10 minutes by @AsafMah in #384
• Add retry mechanism for potentially throttled processes. by @alonadam in #381
• Added error recovery when parsing oneapi errors by @AsafMah in #387

New Contributors

• @alonadam made their first contribution in #381

Full Changelog: v3.1.0...v3.1.1

Python version 3.1.0

Features

• Added proxy support for all ingest clients, using the set_proxy method

Improvements

• All ingestion types (including JSON and AVRO) no longer require a mapping to work. In case a mapping is not provided, kusto will infer one based on the fields.
• dataframe_from_result_table will now save space by using int32 as the type for ADX's int

Python version 3.0.1

Changes

• DataFormat is now directly exported from azure.kusto.data, and is recommended to be imported this way.

Python version 3.0.0

This is a major release.
It brings many new features, but also includes breaking changes.

Breaking Changes

• The minimum supported Python version is now Python 3.7. Python 3.5's EOL was a year ago, so you should upgrade regardless of this release.
• Exception messages may have changed. Make sure to test it if you're reliant on the exact message.
• Stricter checks on formats and mapping types. If your code breaks due to this, you had a bug.
• Renamed IngestionMappingType to IngestionMappingKind to align with other SDKs.
• Renamed ingestion_mapping parameter to column_mappings for clarity and alignment.
• KustoMissingMappingReferenceError and KustoMappingAndMappingReferenceError are now KustoMissingMappingError.

Data Format

• DataFormat is now exported directly from azure-kusto-data, and no longer exported from azure-kusto-ingest (use azure-kusto-data).
• DataFormat's internal representation changed from a string to an enum, including the Kusto service's formatting, whether a mapping is required, relevant IngestionMappingKind, and whether it's compressible.
• Added missing formats and mappings (SStream, ApacheAvro).

Major Features

Quickstart App

• Inside the repo there is now a new module for a Quickstart app. It serves as a sample and a tutorial for the entire flow for working with the SDK, including creating tables, ingesting data and querying.

Streaming Query

• A new API was added to KustoClient (sync and async) - execute_streaming_query.
• The regular execute call waits for all the data to arrive, parse it in memory, and provide Python objects to access it.
• Streaming query lets you work with the data as it arrives, saving time and memory.
• For usage, see the streaming query section in azure-kusto-data/tests/sample.py.

Managed Streaming Ingestion

• Adds a new type of ingestion client - ManagedStreamingIngestClient.
• The client will default to using streaming ingestion, but will be more robust to errors.
• If it fails to use streaming ingestion, it will fall back to queued when:
  • Retrying streaming ingest fails multiple times, getting transient errors
  • The size of the ingested object is more than 4MB
  • Using ingest_from_blob
• For usage, see the managed streaming section in azure-kusto-ingest/tests/sample.py.

Ingest Clients Changes

• All of the ingestion clients now inherit from BaseIngestClient.
• All of the ingestion clients now support the following methods - ingest_from_file, ingest_from_stream, ingest_from_dataframe (only Queued and ManagedStreaming support ingest_from_blob).
• IngestionResult is now returned from ingest_* methods. The result contains some details about the ingestion.
• For ManagedStreamingIngestClient, you can query whether the data was ingested via queued or streaming using IngestionStatus (== QUEUED for queued, == SUCCESS for streaming).

More Detailed Exceptions

• KustoServiceError now has two new subclasses - KustoApiError, KustoMultiApiError.
  • They both utilize the new OneApiError class, which has detailed information about the exception, for a consistent structure between SDKs.
  • KustoApiError is raised on a server error in the web request, and contains within it 1 OneApiError.
  • KustoMultiApiError is raised while reading the data, and can contain multiple OneApiErrors within.
• KustoClientError has a new subclass KustoBlobError, for when uploading a blob fails.
• Since these classes are subclasses of exceptions that already exist, they will not break existing code unless it parses and relies on the exact message.

Proxy

• Kusto Client now supports connecting with a proxy using the set_proxy(proxy_url: str) method.

Minor Features and Bug Fixes

• More type annotations in the code, to help provide completions and error detections.
• Binary types in ingestions will no longer be compressed.
• Providing the size for blobs for ingestion is now optional.
• Dataframe - support numpy types, including int, long, real and decimal.
• Dataframe - handle invalid date types.
• Relax Azure Identity version to support more dependencies.
• The client now sends a specific API version to avoid conflicts.
• Send correct scopes to managed identity and AZ CLI providers.
• Only retrieve CloudInfo when needed.
• Source IDs are always generated, and are always the correct type for improved telemetry.
• Added advanced checks for column mappings.
• Improved E2E test performance by only clearing cache once.

Kusto Python version 2.3.2

What's Changed

• Removed NoReturn reference, Solves #340

Kusto Python version 2.3.1

Improvements:

• Widened supported msal versions (>=1.9.0, <2) and azure-identity vesions (>=1.5.0,<1.7.0). Closes #333 #336
• Widened supported asyncio (>=3.4.4,<4) and asgrief (>=3.2.3,<4) versions for the aio option. Closes #337

Fixes:

• Async resources will now never be created for sync clients. That solves issues with creating clients in enviornments that don't have event loops, such as new threads. Fixes #332

Kusto Python version 2.3.0

Features:

• Added support for async callback to retrieve a token. use KustoConnectionStringBuilder.with_async_token_provider

Fixes:

• Locks are now local to an instance in TokenProviderBase. This should fix event-loop related issues. (#329)

Kusto Python version 2.2.1

Fixes:

• Fixed possible race condition when running multiple async queries at once

Kusto Python version 2.2.0

Improvements:

• Kusto now queries the provided cluster for metadata. This fixes incompatibility issues with other clouds, or services like Synapse. The results are cached and don't require any changes to the user's code or behavior.
  master

Kusto Python version 2.1.3

• Fixed status addressing between async and sync (#326)