Skip to content

Commit

Permalink
Add sensitive column for configuration properties (#380)
Browse files Browse the repository at this point in the history
  • Loading branch information
@jiangpengcheng
jiangpengcheng authored Dec 18, 2023
1 parent 803cfb5 commit 0d7ca12
Show file tree
Hide file tree
Showing 19 changed files with 278 additions and 278 deletions.
20 changes: 10 additions & 10 deletions connectors/canal-source/v3.1.1.1/canal-source.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,16 @@ The configuration of Canal source connector has the following properties.

## Property

| Name | Required | Default | Description |
|------|----------|---------|-------------|
| `username` | true | None | Canal server account (not MySQL).|
| `password` | true | None | Canal server password (not MySQL). |
|`destination`|true|None|Source destination that Canal source connector connects to.
| `singleHostname` | false | None | Canal server address.|
| `singlePort` | false | None | Canal server port.|
| `cluster` | true | false | Whether to enable cluster mode based on Canal server configuration or not.<br/><br/><li>true: **cluster** mode.<br/>If set to true, it talks to `zkServers` to figure out the actual database host.<br/><br/><li>false: **standalone** mode.<br/>If set to false, it connects to the database specified by `singleHostname` and `singlePort`. |
| `zkServers` | true | None | Address and port of the Zookeeper that Canal source connector talks to figure out the actual database host.|
| `batchSize` | false | 1000 | Batch size to fetch from Canal. |
| Name | Required | Sensitive | Default | Description |
|------------------|----------|-----------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `username` | true | true | None | Canal server account (not MySQL). |
| `password` | true | true | None | Canal server password (not MySQL). |
| `destination` | true | false | None | Source destination that Canal source connector connects to. |
| `singleHostname` | false | false | None | Canal server address. |
| `singlePort` | false | false | None | Canal server port. |
| `cluster` | true | false | false | Whether to enable cluster mode based on Canal server configuration or not.<br/><br/><li>true: **cluster** mode.<br/>If set to true, it talks to `zkServers` to figure out the actual database host.<br/><br/><li>false: **standalone** mode.<br/>If set to false, it connects to the database specified by `singleHostname` and `singlePort`. |
| `zkServers` | true | false | None | Address and port of the Zookeeper that Canal source connector talks to figure out the actual database host. |
| `batchSize` | false | false | 1000 | Batch size to fetch from Canal. |

## Example

Expand Down
28 changes: 14 additions & 14 deletions connectors/debezium-mongodb-source/v3.1.1.1/debezium-mongodb-source.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,20 +120,20 @@ key:[eyJpZCI6IjQifQ==], properties:[], content:{"after":"{\"_id\": {\"$numberLon
## Configuration Properties
The configuration of Debezium Mongodb source connector has the following properties.

| Name | Required | Default | Description |
|------|----------|---------|-------------|
| `mongodb.hosts` | true | null | The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set. The list contains a single hostname and a port pair. If mongodb.members.auto.discover is set to false, the host and port pair are prefixed with the replica set name (e.g., rs0/localhost:27017). |
| `mongodb.name` | true | null | A unique name that identifies the connector and/or MongoDB replica set or shared cluster that this connector monitors. Each server should be monitored by at most one Debezium connector, since this server name prefixes all persisted Kafka topics emanating from the MongoDB replica set or cluster. |
| `mongodb.user` | false | null | Name of the database user to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.password` | false | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |
| `database.whitelist` | false | null | A list of all databases hosted by this server which is monitored by the connector.<br/><br/> By default, all databases are monitored. |
| `key.converter` | false | null | The converter provided by Kafka Connect to convert record key. |
| `value.converter` | false | null | The converter provided by Kafka Connect to convert record value. |
| `database.history.pulsar.topic` | false | null | The name of the database history topic where the connector writes and recovers DDL statements. <br/><br/>**Note: this topic is for internal use only and should not be used by consumers.** |
| `database.history.pulsar.service.url` | false | null | Pulsar cluster service URL for history topic. |
| `offset.storage.topic` | false | null | Record the last committed offsets that the connector successfully completes. By default, it's `topicNamespace + "/" + sourceName + "-debezium-offset-topic"`. eg. `persistent://public/default/debezium-mongodb-source-debezium-offset-topic`|
| `json-with-envelope`| false | false | The`json-with-envelope` config is valid only for the JsonConverter. By default, the value is set to false. When the `json-with-envelope` value is set to false, the consumer uses the schema `Schema.KeyValue(Schema.AUTO_CONSUME(), Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message only consists of the payload. When the `json-with-envelope` value is set to true, the consumer uses the schema `Schema.KeyValue(Schema.BYTES, Schema.BYTES)`, and the message consists of the schema and the payload. |
| Name | Required | Sensitive | Default | Description |
|---------------------------------------|----------|-----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `mongodb.hosts` | true | false | null | The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set. The list contains a single hostname and a port pair. If mongodb.members.auto.discover is set to false, the host and port pair are prefixed with the replica set name (e.g., rs0/localhost:27017). |
| `mongodb.name` | true | false | null | A unique name that identifies the connector and/or MongoDB replica set or shared cluster that this connector monitors. Each server should be monitored by at most one Debezium connector, since this server name prefixes all persisted Kafka topics emanating from the MongoDB replica set or cluster. |
| `mongodb.user` | false | true | null | Name of the database user to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.password` | false | true | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | false | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |
| `database.whitelist` | false | false | null | A list of all databases hosted by this server which is monitored by the connector.<br/><br/> By default, all databases are monitored. |
| `key.converter` | false | false | null | The converter provided by Kafka Connect to convert record key. |
| `value.converter` | false | false | null | The converter provided by Kafka Connect to convert record value. |
| `database.history.pulsar.topic` | false | false | null | The name of the database history topic where the connector writes and recovers DDL statements. <br/><br/>**Note: this topic is for internal use only and should not be used by consumers.** |
| `database.history.pulsar.service.url` | false | false | null | Pulsar cluster service URL for history topic. |
| `offset.storage.topic` | false | false | null | Record the last committed offsets that the connector successfully completes. By default, it's `topicNamespace + "/" + sourceName + "-debezium-offset-topic"`. eg. `persistent://public/default/debezium-mongodb-source-debezium-offset-topic` |
| `json-with-envelope` | false | false | false | The`json-with-envelope` config is valid only for the JsonConverter. By default, the value is set to false. When the `json-with-envelope` value is set to false, the consumer uses the schema `Schema.KeyValue(Schema.AUTO_CONSUME(), Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message only consists of the payload. When the `json-with-envelope` value is set to true, the consumer uses the schema `Schema.KeyValue(Schema.BYTES, Schema.BYTES)`, and the message consists of the schema and the payload. |

For more configuration properties, plesae see [Debezium MongoDB connector configuration properties](https://debezium.io/documentation/reference/1.9/connectors/mongodb.html#mongodb-connector-properties)

Expand Down
Loading

0 comments on commit 0d7ca12

Please sign in to comment.