Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[source-bigquery] when using sync mode "incremental | Append" on a stream which references a cursor field that contains the keyword "Timestamp", subsequent syncs after the first to fail with a Syntax error: Expected end of input but got keyword TIMESTAMP #51566

Open
1 task
Conall-airbyte opened this issue Jan 15, 2025 · 0 comments
Open
1 task

[source-bigquery] when using sync mode "incremental | Append" on a stream which references a cursor field that contains the keyword "Timestamp", subsequent syncs after the first to fail with a Syntax error: Expected end of input but got keyword TIMESTAMP #51566

Conall-airbyte opened this issue Jan 15, 2025 · 0 comments
Labels
area/connectors Connector related issues autoteam team/connectors-python type/bug Something isn't working

Comments

@Conall-airbyte
Copy link
Contributor

Conall-airbyte commented Jan 15, 2025 •
edited
Loading

Connector Name

source-bigquery

Connector Version

0.4.2

What step the error happened?

During the sync

Relevant information

It looks like there's a parsing error with field names here when the keyword "Timestamp" is included in a column name.
e.g adding a prefix like: "a_Timestamp" will sync fine, but a field called "Timestamp" will not.

Reproduction steps are below:

  1. Create a table in Big Query that has a column full of timestamps and a header of "Updated Timestamp", the keyword "Timestamp" matters here: Image

Image

  1. set the “Load Type” as Append . Pick "Updated Timestamp" as your Cursor field

Image

  1. the initial full sync will succeed:

Image

  1. add another row to your Big Query table/ or just re-sync it will fail.

Image

Table after insert:

Image

  1. failure condition on sync attempt manifests:

Image

Logs:
default_workspace_logs_25_txt.txt.zip

Relevant log output

2025-01-08 18:18:59 �[43mdestination�[0m > INFO i.a.i.d.e.TestingDestinations(main):71 starting destination: class io.airbyte.integrations.destination.e2e_test.TestingDestinations

2025-01-08 18:18:59 �[43mdestination�[0m > INFO i.a.c.i.b.IntegrationCliParser(parseOptions):126 integration args: {catalog=/dest/catalog.json, write=null, config=/dest/connectorConfig.json}

2025-01-08 18:18:59 �[43mdestination�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):132 Running integration: io.airbyte.integrations.destination.e2e_test.TestingDestinations

2025-01-08 18:18:59 �[43mdestination�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):133 Command: WRITE

2025-01-08 18:18:59 �[43mdestination�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):134 Integration config: IntegrationConfig{command=WRITE, configPath='/dest/connectorConfig.json', catalogPath='/dest/catalog.json', statePath='null'}

2025-01-08 18:18:59 �[44msource�[0m > INFO i.a.i.s.b.BigQuerySource(main):219 starting source: class io.airbyte.integrations.source.bigquery.BigQuerySource

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.b.IntegrationCliParser(parseOptions):126 integration args: {read=null, catalog=/source/catalog.json, state=/source/inputState.json, config=/source/connectorConfig.json}

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):132 Running integration: io.airbyte.integrations.source.bigquery.BigQuerySource

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):133 Command: READ

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):134 Integration config: IntegrationConfig{command=READ, configPath='/source/connectorConfig.json', catalogPath='/source/catalog.json', statePath='/source/inputState.json'}

2025-01-08 18:19:00 �[44msource�[0m > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.s.r.s.StateManagerFactory(createStateManager):57 Stream state manager selected to manage state object with type STREAM.

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.s.r.s.CursorManager(createCursorInfoForStream):199 Cursor field set in state but not present in catalog. Stream: conall_bigquery_Updated Timestamp. Original Cursor Field: Updated_Timestamp. Original value: 2025-01-08T10:00:00Z. Resetting cursor.

2025-01-08 18:19:00 �[44msource�[0m > INFO i.a.c.i.s.r.s.CursorManager(createCursorInfoForStream):179 Found matching cursor in state. Stream: conall_bigquery_timestamp_only. Cursor Field: Updated Timestamp Value: 2025-01-08T10:00:00Z Count: 1

2025-01-08 18:19:02 �[44msource�[0m > INFO i.a.c.d.b.BigQuerySourceOperations(getQueryParameter):148 Query parameter for set : 2025-01-08T10:00:00.000000 UTC. Type: TIMESTAMP

2025-01-08 18:19:03 �[44msource�[0m > INFO i.a.c.i.s.r.AbstractDbSource(lambda$read$1):184 Closing database connection pool.

2025-01-08 18:19:03 �[44msource�[0m > INFO i.a.c.i.s.r.AbstractDbSource(lambda$read$1):186 Closed database connection pool.

2025-01-08 18:19:03 �[44msource�[0m > ERROR i.a.c.i.b.AirbyteExceptionHandler(uncaughtException):64 Something went wrong in the connector. See the logs for more details. java.lang.RuntimeException: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:203) ~[io.airbyte.airbyte-integrations.connectors-source-bigquery.jar:?]
	at io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:41) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46) ~[guava-31.1-jre.jar:?]
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:127) ~[airbyte-cdk-db-sources-0.13.2.jar:?]
	at io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:22) ~[airbyte-cdk-db-sources-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46) ~[guava-31.1-jre.jar:?]
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:74) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42) ~[airbyte-cdk-airbyte-commons-0.13.2.jar:?]
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) ~[guava-31.1-jre.jar:?]
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) ~[guava-31.1-jre.jar:?]
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132) ~[?:?]
	at io.airbyte.cdk.integrations.base.IntegrationRunner.produceMessages(IntegrationRunner.java:236) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.integrations.base.IntegrationRunner.readSerial(IntegrationRunner.java:276) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:173) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:220) ~[io.airbyte.airbyte-integrations.connectors-source-bigquery.jar:?]
Caused by: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:209) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:193) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:143) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:121) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:105) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:200) ~[io.airbyte.airbyte-integrations.connectors-source-bigquery.jar:?]
	... 30 more
Caused by: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:694) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1437) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1432) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) ~[gax-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1431) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1415) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job$1.call(Job.java:338) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job$1.call(Job.java:335) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) ~[gax-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:334) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job.waitFor(Job.java:244) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:207) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:193) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:143) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:121) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:105) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:200) ~[io.airbyte.airbyte-integrations.connectors-source-bigquery.jar:?]
	... 30 more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
GET https://www.googleapis.com/bigquery/v2/projects/tse-playground/queries/98ee9aad-5bfe-4cd3-ba9e-ad8fd8e522a5?location=US&maxResults=0&prettyPrint=false
{
  "code": 400,
  "errors": [
    {
      "domain":"**********",
      "location": "q",
      "locationType": "parameter",
      "message": "Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]",
      "reason": "invalidQuery"
    }
  ],
  "message": "Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]",
  "status": "INVALID_ARGUMENT"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:466) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111) ~[google-http-client-1.43.0.jar:1.43.0]
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:552) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:493) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:603) ~[google-api-client-2.2.0.jar:2.2.0]
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:692) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1437) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1432) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) ~[gax-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1431) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1415) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job$1.call(Job.java:338) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job$1.call(Job.java:335) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) ~[gax-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:334) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at com.google.cloud.bigquery.Job.waitFor(Job.java:244) ~[google-cloud-bigquery-2.23.2.jar:2.23.2]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:207) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:193) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:143) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:121) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:105) ~[airbyte-cdk-core-0.13.2.jar:?]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:200) ~[io.airbyte.airbyte-integrations.connectors-source-bigquery.jar:?]
	... 30 more

Stack Trace: java.lang.RuntimeException: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:203)
	at io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:41)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46)
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:127)
	at io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:22)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46)
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:74)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.produceMessages(IntegrationRunner.java:236)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.readSerial(IntegrationRunner.java:276)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:173)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
	at io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:220)
Caused by: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:209)
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:193)
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:143)
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:121)
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:105)
	at io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:200)
	... 30 more
Caused by: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:694)
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1437)
	at com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1432)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86)
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49)
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1431)
	at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1415)
	at com.google.cloud.bigquery.Job$1.call(Job.java:338)
	at com.google.cloud.bigquery.Job$1.call(Job.java:335)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86)
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49)
	at com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:334)
	at com.google.cloud.bigquery.Job.waitFor(Job.java:244)
	at io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:207)
	... 35 more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
GET https://www.googleapis.com/bigquery/v2/projects/tse-playground/queries/98ee9aad-5bfe-4cd3-ba9e-ad8fd8e522a5?location=US&maxResults=0&prettyPrint=false
{
  "code": 400,
  "errors": [
    {
      "domain":"**********",
      "location": "q",
      "locationType": "parameter",
      "message": "Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]",
      "reason": "invalidQuery"
    }
  ],
  "message": "Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]",
  "status": "INVALID_ARGUMENT"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:466)
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:552)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:493)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:603)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:692)
	... 50 more


2025-01-08 18:19:03 �[46mreplication-orchestrator�[0m > readFromSource: source exception
java.lang.Throwable: Source process exited with non-zero exit code 1
	at io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(Unknown Source)
	at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(Unknown Source)
	at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)


2025-01-08 18:19:03 �[46mreplication-orchestrator�[0m > readFromSource: done. (source.isFinished:true, fromSource.isClosed:false)

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > processMessage: done. (fromSource.isDone:true, forDest.isClosed:false)

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > writeToDestination: done. (forDest.isDone:true, isDestRunning:true)

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > thread status... timeout thread: false , replication thread: true

2025-01-08 18:19:08 �[43mdestination�[0m > INFO i.a.c.i.b.IntegrationRunner(runInternal):231 Completed integration: io.airbyte.integrations.destination.e2e_test.TestingDestinations

2025-01-08 18:19:08 �[43mdestination�[0m > INFO i.a.i.d.e.TestingDestinations(main):73 completed destination: class io.airbyte.integrations.destination.e2e_test.TestingDestinations

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > Destination finished successfully — exiting read dest...

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > readFromDestination: done. (writeToDestFailed:false, dest.isFinished:true)

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > thread status... timeout thread: false , replication thread: true

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > Closing StateCheckSumCountEventHandler

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > sync summary: {
  "status" : "failed",
  "startTime" : 1736360338071,
  "endTime" : 1736360348430,
  "totalStats" : {
    "bytesEmitted" : 0,
    "destinationStateMessagesEmitted" : 0,
    "destinationWriteEndTime" : 1736360348392,
    "destinationWriteStartTime" : 1736360338240,
    "meanSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBeforeSourceStateMessageEmitted" : 0,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "recordsEmitted" : 0,
    "recordsFilteredOut" : 0,
    "bytesFilteredOut" : 0,
    "replicationEndTime" : 1736360348424,
    "replicationStartTime" : 1736360338071,
    "sourceReadEndTime" : 0,
    "sourceReadStartTime" : 1736360338242,
    "sourceStateMessagesEmitted" : 0
  },
  "streamStats" : [ ],
  "performanceMetrics" : {
    "processFromSource" : {
      "elapsedTimeInNanos" : 2078845,
      "executionCount" : 1,
      "avgExecTimeInNanos" : 2078845.0
    },
    "readFromSource" : {
      "elapsedTimeInNanos" : 5007776915,
      "executionCount" : 72,
      "avgExecTimeInNanos" : 6.955245715277778E7
    },
    "processFromDest" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "writeToDest" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "readFromDest" : {
      "elapsedTimeInNanos" : 10111378815,
      "executionCount" : 335,
      "avgExecTimeInNanos" : 3.0183220343283582E7
    }
  }
}

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > failures: [ {
  "failureOrigin" : "source",
  "failureType" : "system_error",
  "internalMessage" : "java.lang.RuntimeException: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]",
  "externalMessage" : "Something went wrong in the connector. See the logs for more details.",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 25,
    "from_trace_message" : true,
    "connector_command" : "read"
  },
  "stacktrace" : "java.lang.RuntimeException: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]\n\tat io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:203)\n\tat io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:41)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46)\n\tat io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:127)\n\tat io.airbyte.cdk.integrations.source.relationaldb.StateDecoratingIterator.computeNext(StateDecoratingIterator.java:22)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46)\n\tat io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:74)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:42)\n\tat com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)\n\tat com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)\n\tat java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.produceMessages(IntegrationRunner.java:236)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.readSerial(IntegrationRunner.java:276)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:173)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)\n\tat io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:220)\nCaused by: java.lang.RuntimeException: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:209)\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:193)\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.executeQuery(BigQueryDatabase.java:143)\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:121)\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.query(BigQueryDatabase.java:105)\n\tat io.airbyte.integrations.source.bigquery.BigQuerySource.lambda$queryTableWithParams$4(BigQuerySource.java:200)\n\t... 30 more\nCaused by: com.google.cloud.bigquery.BigQueryException: Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]\n\tat com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)\n\tat com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:694)\n\tat com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1437)\n\tat com.google.cloud.bigquery.BigQueryImpl$36.call(BigQueryImpl.java:1432)\n\tat com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)\n\tat com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86)\n\tat com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49)\n\tat com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1431)\n\tat com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1415)\n\tat com.google.cloud.bigquery.Job$1.call(Job.java:338)\n\tat com.google.cloud.bigquery.Job$1.call(Job.java:335)\n\tat com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)\n\tat com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86)\n\tat com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49)\n\tat com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:334)\n\tat com.google.cloud.bigquery.Job.waitFor(Job.java:244)\n\tat io.airbyte.cdk.db.bigquery.BigQueryDatabase.waitForQuery(BigQueryDatabase.java:207)\n\t... 35 more\nCaused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request\nGET https://www.googleapis.com/bigquery/v2/projects/tse-playground/queries/98ee9aad-5bfe-4cd3-ba9e-ad8fd8e522a5?location=US&maxResults=0&prettyPrint=false\n{\n  \"code\": 400,\n  \"errors\": [\n    {\n      \"domain\": \"global\",\n      \"location\": \"q\",\n      \"locationType\": \"parameter\",\n      \"message\": \"Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]\",\n      \"reason\": \"invalidQuery\"\n    }\n  ],\n  \"message\": \"Syntax error: Expected end of input but got keyword TIMESTAMP at [1:82]\",\n  \"status\": \"INVALID_ARGUMENT\"\n}\n\tat com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)\n\tat com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)\n\tat com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)\n\tat com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:466)\n\tat com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)\n\tat com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:552)\n\tat com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:493)\n\tat com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:603)\n\tat com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:692)\n\t... 50 more\n",
  "timestamp" : 1736360343190
}, {
  "failureOrigin" : "source",
  "internalMessage" : "Source process exited with non-zero exit code 1",
  "externalMessage" : "Something went wrong within the source connector",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 25,
    "connector_command" : "read"
  },
  "stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:364)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:222)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n",
  "timestamp" : 1736360343328
} ]

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > 

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > ----- END REPLICATION -----

2025-01-08 18:19:08 �[46mreplication-orchestrator�[0m > 

2025-01-08 18:19:10 �[46mreplication-orchestrator�[0m > Returning output...

2025-01-08 18:19:10 �[46mplatform�[0m > Closing Segment analytics client...

2025-01-08 18:19:10 �[46mplatform�[0m > Waiting for Segment analytic client to flush enqueued messages...

2025-01-08 18:19:10 �[46mplatform�[0m > Segment analytic client flush complete.

2025-01-08 18:19:10 �[46mplatform�[0m > Segment analytics client closed.  No new events will be accepted.

2025-01-08 18:19:32 �[46mplatform�[0m > Workload aa5db4da-84df-48de-8552-eef3a5d90ae2_25_0_sync has returned a terminal status of failure.  Fetching output...

2025-01-08 18:19:33 �[46mplatform�[0m > No post-replication operation(s) to perform.

Contribute

  • Yes, I want to contribute
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/connectors Connector related issues autoteam team/connectors-python type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@marcosmarxm @octavia-squidington-iii @Conall-airbyte