Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[FEAT] Add initial Spark Connect support (#3261)
Implements basic Spark Connect functionality in Daft with the following: - Add `daft-connect` crate to handle Spark Connect protocol - Implement configuration management via Spark Connect API - Add Python bindings with `connect_start()` function - Add integration tests for config operations Currently supports: - Basic session management - Config operations (Set, Get, GetWithDefault, GetOption, GetAll, Unset) - Error handling and status reporting Notable changes: - New dependency on `spark-connect` protocol - Added `tracing` for debugging and monitoring - Integration with existing Daft infrastructure Some operations like execute_plan, analyze_plan, and artifact handling are currently unimplemented and will return appropriate error messages.
- Loading branch information