Skip to content

Commit

Permalink
add an additional post for esp32-micropython3
Browse files Browse the repository at this point in the history
  • Loading branch information
paul356 committed Dec 26, 2023
1 parent e21a670 commit ce27262
Show file tree
Hide file tree
Showing 4 changed files with 252 additions and 19 deletions.
15 changes: 15 additions & 0 deletions _org/2022-12-31-spark.org
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,21 @@ SparkCatalog还依赖一些其他类实现其功能
- OutputFileFactory - Factory responsible for generating unique but recognizable data file names and file objects.
- SparkFileWriteFactory - Return data file writer for Parquet, ORC or AVRO

** 创建QueryExecution
- SessionState.executePlan(logicalPlan: LogicalPlan)

** Analalyze UnresolvedRelation(multipartIdentifier, ...)
- Dataset.ofRows
- SessionState.executePlan(LogicalPlan)
- createQueryexecution(LogicalPlan, mode)
- QueryExecution.assertAnalyzed()
- lazy val analyzed
- RuleExecutor.execute(LogicalPlan)
- batches.foreach(...)
- ResolveRelations.apply
- ResolveRelations.lookupRelations
- new Dataset[Row](QueryExecution, ...)

** TODO 支持分层的数据湖格式(HTAP)
- 基于lsm结构组织文件
+ 第一层采用tiering的方式排布
Expand Down
Loading

0 comments on commit ce27262

Please sign in to comment.