Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor create a bigquery connector section docs. #367

Merged
merged 1 commit into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 31 additions & 19 deletions connectors/google-bigquery-sink/v2.10.5.9/google-bigquery-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,28 +49,40 @@ The prerequisites for connecting an Google BigQuery sink connector to external s

### 1. Create a connector

Depending on the environment, there are several ways to create an Google BigQuery sink connector:
The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector,
you need to replace `--sink-type bigquery` with `--archive /path/to/pulsar-io-bigquery.jar`. You can find the button to download the `jar` package at the beginning of the document.

- [Create Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create).
- [Create Connector with Function worker](https://pulsar.apache.org/docs/3.0.x/io-quickstart/).
Using this way requires you to download a **JAR** package to create a built-in or non-built-in connector. You can download the version you need from [here](https://github.com/streamnative/pulsar-io-bigquery/releases).
- [Create Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector).
Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-bigquery/tags)

No matter how you create an Google BigQuery sink connector, the minimum configuration contains the following parameters.
{% callout title="For StreamNative Cloud User" type="note" %}
If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first.
{% /callout %}

```yaml
configs:
projectId: {{Your BigQuery project id}}
datasetName: {{Your Bigquery DataSet name}}
tableName: {{The name of the table you want to write data to is automatically created by default}}
credentialJsonString: {{Public key certificate you created above}}
```
```bash
pulsarctl sinks create \
--sink-type bigquery \
--name bigquery-sink \
--tenant public \
--namespace default \
--inputs "Your topic name" \
--parallelism 1 \
--sink-config \
'{
"projectId": "Your BigQuery project Id",
"datasetName": "Your Bigquery DataSet name",
"tableName": "The name of the table you want to write data to is automatically created by default",
"credentialJsonString": "Public key certificate you created above"
}'
```

The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own.
If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.

> * The configuration structure varies depending on how you create the Google BigQuery sink connector.
> For example, some are **JSON**, some are **YAML**, and some are **Kubernetes YAML**. You need to adapt the configs to the corresponding format.
>
> * If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.
{% callout title="Note" type="note" %}
You can also choose to use a variety of other tools to create a connector:
- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ).
- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document.
{% /callout %}

### 2. Send messages to the topic

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,28 +49,40 @@ The prerequisites for connecting an Google BigQuery sink connector to external s

### 1. Create a connector

Depending on the environment, there are several ways to create an Google BigQuery sink connector:
The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector,
you need to replace `--sink-type bigquery` with `--archive /path/to/pulsar-io-bigquery.jar`. You can find the button to download the `jar` package at the beginning of the document.

- [Create Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create).
- [Create Connector with Function worker](https://pulsar.apache.org/docs/3.0.x/io-quickstart/).
Using this way requires you to download a **JAR** package to create a built-in or non-built-in connector. You can download the version you need from [here](https://github.com/streamnative/pulsar-io-bigquery/releases).
- [Create Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector).
Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-bigquery/tags)

No matter how you create an Google BigQuery sink connector, the minimum configuration contains the following parameters.
{% callout title="For StreamNative Cloud User" type="note" %}
If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first.
{% /callout %}

```yaml
configs:
projectId: {{Your BigQuery project id}}
datasetName: {{Your Bigquery DataSet name}}
tableName: {{The name of the table you want to write data to is automatically created by default}}
credentialJsonString: {{Public key certificate you created above}}
```
```bash
pulsarctl sinks create \
--sink-type bigquery \
--name bigquery-sink \
--tenant public \
--namespace default \
--inputs "Your topic name" \
--parallelism 1 \
--sink-config \
'{
"projectId": "Your BigQuery project Id",
"datasetName": "Your Bigquery DataSet name",
"tableName": "The name of the table you want to write data to is automatically created by default",
"credentialJsonString": "Public key certificate you created above"
}'
```

The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own.
If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.

> * The configuration structure varies depending on how you create the Google BigQuery sink connector.
> For example, some are **JSON**, some are **YAML**, and some are **Kubernetes YAML**. You need to adapt the configs to the corresponding format.
>
> * If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.
{% callout title="Note" type="note" %}
You can also choose to use a variety of other tools to create a connector:
- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ).
- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document.
{% /callout %}

### 2. Send messages to the topic

Expand Down
50 changes: 31 additions & 19 deletions connectors/google-bigquery-sink/v3.0.2.1/google-bigquery-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,28 +49,40 @@ The prerequisites for connecting an Google BigQuery sink connector to external s

### 1. Create a connector

Depending on the environment, there are several ways to create an Google BigQuery sink connector:
The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector,
you need to replace `--sink-type bigquery` with `--archive /path/to/pulsar-io-bigquery.jar`. You can find the button to download the `jar` package at the beginning of the document.

- [Create Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create).
- [Create Connector with Function worker](https://pulsar.apache.org/docs/3.0.x/io-quickstart/).
Using this way requires you to download a **JAR** package to create a built-in or non-built-in connector. You can download the version you need from [here](https://github.com/streamnative/pulsar-io-bigquery/releases).
- [Create Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector).
Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-bigquery/tags)

No matter how you create an Google BigQuery sink connector, the minimum configuration contains the following parameters.
{% callout title="For StreamNative Cloud User" type="note" %}
If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first.
{% /callout %}

```yaml
configs:
projectId: {{Your BigQuery project id}}
datasetName: {{Your Bigquery DataSet name}}
tableName: {{The name of the table you want to write data to is automatically created by default}}
credentialJsonString: {{Public key certificate you created above}}
```
```bash
pulsarctl sinks create \
--sink-type bigquery \
--name bigquery-sink \
--tenant public \
--namespace default \
--inputs "Your topic name" \
--parallelism 1 \
--sink-config \
'{
"projectId": "Your BigQuery project Id",
"datasetName": "Your Bigquery DataSet name",
"tableName": "The name of the table you want to write data to is automatically created by default",
"credentialJsonString": "Public key certificate you created above"
}'
```

The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own.
If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.

> * The configuration structure varies depending on how you create the Google BigQuery sink connector.
> For example, some are **JSON**, some are **YAML**, and some are **Kubernetes YAML**. You need to adapt the configs to the corresponding format.
>
> * If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.
{% callout title="Note" type="note" %}
You can also choose to use a variety of other tools to create a connector:
- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ).
- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document.
{% /callout %}

### 2. Send messages to the topic

Expand Down
50 changes: 31 additions & 19 deletions connectors/google-bigquery-sink/v3.1.1.1/google-bigquery-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,28 +49,40 @@ The prerequisites for connecting an Google BigQuery sink connector to external s

### 1. Create a connector

Depending on the environment, there are several ways to create an Google BigQuery sink connector:
The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector,
you need to replace `--sink-type bigquery` with `--archive /path/to/pulsar-io-bigquery.jar`. You can find the button to download the `jar` package at the beginning of the document.

- [Create Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create).
- [Create Connector with Function worker](https://pulsar.apache.org/docs/3.0.x/io-quickstart/).
Using this way requires you to download a **JAR** package to create a built-in or non-built-in connector. You can download the version you need from [here](https://github.com/streamnative/pulsar-io-bigquery/releases).
- [Create Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector).
Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-bigquery/tags)

No matter how you create an Google BigQuery sink connector, the minimum configuration contains the following parameters.
{% callout title="For StreamNative Cloud User" type="note" %}
If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first.
{% /callout %}

```yaml
configs:
projectId: {{Your BigQuery project id}}
datasetName: {{Your Bigquery DataSet name}}
tableName: {{The name of the table you want to write data to is automatically created by default}}
credentialJsonString: {{Public key certificate you created above}}
```
```bash
pulsarctl sinks create \
--sink-type bigquery \
--name bigquery-sink \
--tenant public \
--namespace default \
--inputs "Your topic name" \
--parallelism 1 \
--sink-config \
'{
"projectId": "Your BigQuery project Id",
"datasetName": "Your Bigquery DataSet name",
"tableName": "The name of the table you want to write data to is automatically created by default",
"credentialJsonString": "Public key certificate you created above"
}'
```

The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own.
If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.

> * The configuration structure varies depending on how you create the Google BigQuery sink connector.
> For example, some are **JSON**, some are **YAML**, and some are **Kubernetes YAML**. You need to adapt the configs to the corresponding format.
>
> * If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference.
{% callout title="Note" type="note" %}
You can also choose to use a variety of other tools to create a connector:
- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ).
- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector).
- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document.
{% /callout %}

### 2. Send messages to the topic

Expand Down
Loading