Skip to content

Commit

Permalink
update spark notes
Browse files Browse the repository at this point in the history
  • Loading branch information
paul356 committed Apr 29, 2024
1 parent 7ffdc94 commit e1dad27
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions _org/2022-12-31-spark.org
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,9 @@ SparkCatalog还依赖一些其他类实现其功能

** spark写流程
- AppendData --(ExtendedV2Writes strategy)--> AppendData(write = SparkWriteBuilder.build())
- Spark --> WriteToDataSourceV2Exec.run() --> V2TableWriteExec.writeWithV2
- Spark --> AppendDataExec.run() --> V2TableWriteExec.writeWithV2
+ SparkWrite.asBatchAppend()
+ BaseWrite.createBatchWriterFactory()
+ BatchWrite.createBatchWriterFactory()
+ WriterFactory.createWriter()
+ DataWriter<InternalRow>.write(row)
* UnpartitionedDataWriter.write(row)
Expand Down

0 comments on commit e1dad27

Please sign in to comment.