Skip to content
This repository has been archived by the owner on Dec 15, 2022. It is now read-only.

Streaming with PubSub and BigQuery
Compare
Choose a tag to compare
@mattcasters mattcasters released this 14 Jan 09:14
0.2.0
2c93836

Issue #20 : Google Pub/Sub Publish and Subscribe
Issue #21 : Goole BigQuery Input and Output
Issue #22 : Beam Timestamp (for streaming bounded data sources)

Releasing the hundreds of MB in libraries for this release is not optimal for GitHub so you need to download the plugin elsewhere. Please download this archive and unzip it in the <PDI>/plugins/ folder.

Then patch your Kettle CE version 8.2 by unzipping pdi-engine-configuration-8.2.0.0-342.zip in and over your Kettle distribution root folder <PDI>. Doing this adds "Beam" as a Run Configuration option besides "Pentaho" and "Spark". The source code for this new "Beam Run Configuration" can be found by getting branch 8.2.0.1 from pentaho-kettle in the engine-configuration plugin

For configuration and usage please see the README.md file in this project.

Assets 4
Loading