Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clickhouse Native(Binary) client #931

Open
wants to merge 186 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
186 commits
Select commit Hold shift + click to select a range
8101d06
Added clickhouse InitCommand handler
smagellan Feb 15, 2021
622b2fb
Fixed typo
smagellan Feb 15, 2021
95a624e
implemented SimpleQueryCodec
smagellan Feb 21, 2021
74ed583
added unit test
smagellan Feb 22, 2021
240cb5a
added UInt16, UInt64 types support
smagellan Feb 22, 2021
50a11ac
renamed nItems to nRows, removed reflection API calls
smagellan Feb 22, 2021
3be7ed8
initial writeSettings implementation; initial PreparedStatement imple…
smagellan Feb 26, 2021
52da104
refactored PacketReader
smagellan Feb 27, 2021
057ca73
added ExtendedClickhouseTest
smagellan Feb 28, 2021
ab430b3
added testIntegerRanges test
smagellan Mar 1, 2021
9df29ad
added Int128 type
smagellan Mar 2, 2021
9edb5e1
basic lz4 compression/decompression [WIP]
smagellan Mar 7, 2021
1b90509
added DateTime/DateTime64 column support
smagellan Mar 7, 2021
b6403a3
added UUID column support
smagellan Mar 7, 2021
e36625f
added Decimal column support
smagellan Mar 8, 2021
88e3b2c
added TABLE_COLUMNS packet-type support
smagellan Mar 9, 2021
217be04
added Replay test
smagellan Mar 9, 2021
837bb65
added Enum column support
smagellan Mar 10, 2021
c4a4c66
added LowCardinality column support
smagellan Mar 12, 2021
d345fa5
Array support [WIP]
smagellan Mar 13, 2021
0d48b63
skip bytes if element is NULL
smagellan Mar 14, 2021
b898cf9
Added Date,Float32,Float64 columns; added ClickhouseNativeBinaryDataT…
smagellan Mar 15, 2021
0004994
added ClickhouseNativeCollectorTest
smagellan Mar 15, 2021
331072a
added ClickhouseNativeConnectionTest
smagellan Mar 15, 2021
010cfe2
added ClickhouseNativeConnectionAutoRetryTest
smagellan Mar 16, 2021
98aaaa4
added ClickhouseNativeBinaryDataTypeEncodeTest and ClickhouseNativeDr…
smagellan Mar 16, 2021
ec3da62
reworked ArrayColumn to recode elements if needed
smagellan Mar 17, 2021
940df68
reworked ArrayColumn to handle partial reads
smagellan Mar 17, 2021
f1c615d
initial batch insert support
smagellan Mar 21, 2021
623cb2f
splitted columns into readers and writers
smagellan Mar 21, 2021
f386e03
finished tck test set
smagellan Mar 22, 2021
bb1c42f
added LowCardinality encoder
smagellan Mar 28, 2021
e8ebbf6
added Array encoder
smagellan Mar 28, 2021
1234e7c
added DateColumnWriter and DateTimeColumnWriter
smagellan Mar 30, 2021
6a11da8
reworked AllTypesTest to use ColumnChecker
smagellan Mar 31, 2021
c65f52b
reworked ArrayColumnReader
smagellan Apr 3, 2021
070aafc
reworked AllTypesTest
smagellan Apr 3, 2021
91632f5
added IPv4 column test
smagellan Apr 3, 2021
3330970
added IPv6 column test
smagellan Apr 3, 2021
712b52b
added Int16/32/64 tests
smagellan Apr 3, 2021
e79de39
fixed fragmented read issue
smagellan Apr 4, 2021
0a3839e
added DateTime64 test
smagellan Apr 4, 2021
4af3384
added Enum8Test
smagellan Apr 4, 2021
be34c67
added Enum16Test
smagellan Apr 4, 2021
429e4de
added Interval array tests
smagellan Apr 4, 2021
07e3f94
added enum_resolution option
smagellan Apr 5, 2021
ad2c556
added FixedString test
smagellan Apr 6, 2021
c421deb
added UInt16/32/64 test
smagellan Apr 6, 2021
f68a178
added Decimal32/64 and Float32 tests
smagellan Apr 6, 2021
108a965
added Decimal128/256 tests
smagellan Apr 6, 2021
7f8e10a
fixed TODO: reuse cached key columns for LowCardinality reader
smagellan Apr 7, 2021
225ced8
fixed TODO: do not store array descriptor as lined list of nested types
smagellan Apr 7, 2021
f0005e4
do not do extra nested array copy
smagellan Apr 7, 2021
66e6212
removed recursion from ArrayColumnWriter
smagellan Apr 7, 2021
e9136d5
do not reslice every row if recoding is required
smagellan Apr 8, 2021
1395271
added StringCache for LowCardinality columns
smagellan Apr 8, 2021
6fdbe02
added IntervalArray tests
smagellan Apr 8, 2021
225dd3d
added array deduplication test
smagellan Apr 8, 2021
b7be852
reworked array deduplication test
smagellan Apr 8, 2021
cef98c2
use columnreader as accessor if array recoding is reqired
smagellan Apr 9, 2021
da9062c
use less reflection for empty arrays
smagellan Apr 9, 2021
831b3a1
added more LowCardinality checks, fixed TODOs
smagellan Apr 9, 2021
ce4fdf0
moved INSERT info extraction into QueryInfo
smagellan Apr 9, 2021
0643b1f
added NestedExceptionsTest
smagellan Apr 10, 2021
ee30664
replaced LOG.info with LOG.debug where needed
smagellan Apr 10, 2021
da8e34c
added copyright headers
smagellan Apr 10, 2021
e9af16e
fixed copyright headers
smagellan Apr 10, 2021
5735b68
fixed copyright headers, reworked array to use int[] arrays as slices
smagellan Apr 10, 2021
8a45ce2
fixed TODOs in pom.xml
smagellan Apr 12, 2021
1e0e95a
reworked ArrayColumnReader to support primitive array
smagellan Apr 13, 2021
dbd1084
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan May 7, 2021
daea1c0
added charset parameter to readPascalString
smagellan May 8, 2021
c8b538d
removed TestRunner
smagellan May 8, 2021
9212824
rebasing onto master
smagellan May 9, 2021
5d9a37a
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan May 9, 2021
4329849
Merge branch 'feature/clickhouse-native-client' of https://github.com…
smagellan May 9, 2021
eede5dd
rebasing onto master
smagellan May 9, 2021
413caf9
throw ClassCastException in get(Class<T> type, int position) if can't…
smagellan May 10, 2021
fd972ed
added column index into ClassCastException message
smagellan May 10, 2021
ad76c64
replaced confusing 'native' to 'binary'
smagellan May 11, 2021
5ff7b1f
replaced confusing 'native' to 'binary' in pom.xml
smagellan Jun 2, 2021
ba517dd
replaced confusing 'native' to 'binary' in tests and package-info
smagellan Jun 2, 2021
3157dab
replaced confusing 'native' to 'binary' in tests and package-info
smagellan Jun 2, 2021
1491473
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Jun 2, 2021
59be4d2
replaced confusing 'native' to 'binary' in pom description
smagellan Jun 2, 2021
ffe427e
replaced confusing 'native' to 'binary' in pom description
smagellan Jun 2, 2021
f39a54c
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Oct 24, 2021
7a6f899
Added clickhouse InitCommand handler
smagellan Feb 15, 2021
f68bb39
Fixed typo
smagellan Feb 15, 2021
3e7c29c
implemented SimpleQueryCodec
smagellan Feb 21, 2021
372f6d8
added unit test
smagellan Feb 22, 2021
1386a5d
added UInt16, UInt64 types support
smagellan Feb 22, 2021
a2e88ce
renamed nItems to nRows, removed reflection API calls
smagellan Feb 22, 2021
e4c97cc
initial writeSettings implementation; initial PreparedStatement imple…
smagellan Feb 26, 2021
f7a096e
refactored PacketReader
smagellan Feb 27, 2021
05411b4
added ExtendedClickhouseTest
smagellan Feb 28, 2021
27fdd76
added testIntegerRanges test
smagellan Mar 1, 2021
c46f3e0
added Int128 type
smagellan Mar 2, 2021
91a0371
basic lz4 compression/decompression [WIP]
smagellan Mar 7, 2021
da0ee4c
added DateTime/DateTime64 column support
smagellan Mar 7, 2021
d5106f3
added UUID column support
smagellan Mar 7, 2021
8841c24
added Decimal column support
smagellan Mar 8, 2021
957ece6
added TABLE_COLUMNS packet-type support
smagellan Mar 9, 2021
92f38b1
added Replay test
smagellan Mar 9, 2021
e16e995
added Enum column support
smagellan Mar 10, 2021
fac35ba
added LowCardinality column support
smagellan Mar 12, 2021
500f03f
Array support [WIP]
smagellan Mar 13, 2021
f1e0636
skip bytes if element is NULL
smagellan Mar 14, 2021
d97b16b
Added Date,Float32,Float64 columns; added ClickhouseNativeBinaryDataT…
smagellan Mar 15, 2021
cc27188
added ClickhouseNativeCollectorTest
smagellan Mar 15, 2021
2871542
added ClickhouseNativeConnectionTest
smagellan Mar 15, 2021
d820e90
added ClickhouseNativeConnectionAutoRetryTest
smagellan Mar 16, 2021
be0635f
added ClickhouseNativeBinaryDataTypeEncodeTest and ClickhouseNativeDr…
smagellan Mar 16, 2021
cf8c171
reworked ArrayColumn to recode elements if needed
smagellan Mar 17, 2021
d13538c
reworked ArrayColumn to handle partial reads
smagellan Mar 17, 2021
22bb068
initial batch insert support
smagellan Mar 21, 2021
d37f743
splitted columns into readers and writers
smagellan Mar 21, 2021
e16e1ab
finished tck test set
smagellan Mar 22, 2021
672b998
added LowCardinality encoder
smagellan Mar 28, 2021
c15b884
added Array encoder
smagellan Mar 28, 2021
233fd4f
added DateColumnWriter and DateTimeColumnWriter
smagellan Mar 30, 2021
4c6f944
reworked AllTypesTest to use ColumnChecker
smagellan Mar 31, 2021
41c68fd
reworked ArrayColumnReader
smagellan Apr 3, 2021
13267df
reworked AllTypesTest
smagellan Apr 3, 2021
888985f
added IPv4 column test
smagellan Apr 3, 2021
0ef566d
added IPv6 column test
smagellan Apr 3, 2021
c7228b2
added Int16/32/64 tests
smagellan Apr 3, 2021
4ddb4d4
fixed fragmented read issue
smagellan Apr 4, 2021
f43cf87
added DateTime64 test
smagellan Apr 4, 2021
dff9104
added Enum8Test
smagellan Apr 4, 2021
82c9ec6
added Enum16Test
smagellan Apr 4, 2021
b4fce3d
added Interval array tests
smagellan Apr 4, 2021
923f44b
added enum_resolution option
smagellan Apr 5, 2021
8456b0f
added FixedString test
smagellan Apr 6, 2021
8aa88a5
added UInt16/32/64 test
smagellan Apr 6, 2021
73369b3
added Decimal32/64 and Float32 tests
smagellan Apr 6, 2021
08be2bf
added Decimal128/256 tests
smagellan Apr 6, 2021
14ae1c6
fixed TODO: reuse cached key columns for LowCardinality reader
smagellan Apr 7, 2021
7c92fbe
fixed TODO: do not store array descriptor as lined list of nested types
smagellan Apr 7, 2021
930a424
do not do extra nested array copy
smagellan Apr 7, 2021
3851d06
removed recursion from ArrayColumnWriter
smagellan Apr 7, 2021
73113d8
do not reslice every row if recoding is required
smagellan Apr 8, 2021
32ddad9
added StringCache for LowCardinality columns
smagellan Apr 8, 2021
a2f099e
added IntervalArray tests
smagellan Apr 8, 2021
8569966
added array deduplication test
smagellan Apr 8, 2021
7e206ff
reworked array deduplication test
smagellan Apr 8, 2021
2252717
use columnreader as accessor if array recoding is reqired
smagellan Apr 9, 2021
33b839c
use less reflection for empty arrays
smagellan Apr 9, 2021
d0185ec
added more LowCardinality checks, fixed TODOs
smagellan Apr 9, 2021
af722ff
moved INSERT info extraction into QueryInfo
smagellan Apr 9, 2021
d974e2e
added NestedExceptionsTest
smagellan Apr 10, 2021
4472783
replaced LOG.info with LOG.debug where needed
smagellan Apr 10, 2021
4254305
added copyright headers
smagellan Apr 10, 2021
4c7cb93
fixed copyright headers
smagellan Apr 10, 2021
75cdc8a
fixed copyright headers, reworked array to use int[] arrays as slices
smagellan Apr 10, 2021
57f1d3b
fixed TODOs in pom.xml
smagellan Apr 12, 2021
ad58bce
reworked ArrayColumnReader to support primitive array
smagellan Apr 13, 2021
df4df78
added charset parameter to readPascalString
smagellan May 8, 2021
4eff81e
removed TestRunner
smagellan May 8, 2021
009ec4d
rebasing onto master
smagellan May 9, 2021
8724b82
rebasing onto master
smagellan May 9, 2021
77f21e6
throw ClassCastException in get(Class<T> type, int position) if can't…
smagellan May 10, 2021
39714dd
added column index into ClassCastException message
smagellan May 10, 2021
9a076bc
replaced confusing 'native' to 'binary'
smagellan May 11, 2021
8b1b28a
replaced confusing 'native' to 'binary' in pom.xml
smagellan Jun 2, 2021
6d3e8e3
replaced confusing 'native' to 'binary' in tests and package-info
smagellan Jun 2, 2021
6c9faf1
replaced confusing 'native' to 'binary' in tests and package-info
smagellan Jun 2, 2021
152261c
replaced confusing 'native' to 'binary' in pom description
smagellan Jun 2, 2021
3d91c33
replaced confusing 'native' to 'binary' in pom description
smagellan Jun 2, 2021
7eb874f
rebase to 4.2.0-SNAPSHOT
smagellan Oct 24, 2021
e9c19a6
Merge remote-tracking branch 'origin/feature/clickhouse-native-client…
smagellan Oct 24, 2021
a0dd2a1
handle 'dangling' exceptions (e.g. non-existent DB exception after su…
smagellan Oct 24, 2021
5cba677
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Dec 25, 2021
ec30819
rebase over master
smagellan Dec 26, 2021
6e686cc
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Oct 2, 2022
365247c
1) rebase over 4.3.5-SNAPSHOT; 2) fix arrays deserialization
smagellan Oct 23, 2022
f081249
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Jan 15, 2023
b15d646
rebase over 4.4.0-SNAPSHOT
smagellan Jan 19, 2023
f366718
add Boolean type support
smagellan Jan 31, 2023
86cf5e4
separate Signed/Unsigned column readers
smagellan Feb 4, 2023
b5b120c
separate Signed/Unsigned column readers
smagellan Feb 4, 2023
a3c95d0
finalize initial DB switching machinery
smagellan Feb 4, 2023
d7e6296
enable non-compressed connections again
smagellan Feb 4, 2023
df25675
remove traces of old Exception-after-HELLO(Metadata) handling code
smagellan Feb 5, 2023
fbee08e
simplify DB switch logic
smagellan Feb 5, 2023
5a0cb44
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan Feb 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@
<module>vertx-mysql-client</module>
<module>vertx-mssql-client</module>
<module>vertx-db2-client</module>
<module>vertx-clickhouse-binary-client</module>
<module>vertx-sql-client-templates</module>
<module>vertx-oracle-client</module>
</modules>
Expand Down
204 changes: 204 additions & 0 deletions vertx-clickhouse-binary-client/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
<?xml version="1.0"?>
<!--
~ Copyright (C) 2017 Julien Viet
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.vertx</groupId>
<artifactId>vertx-sql-client-parent</artifactId>
<version>4.4.0-SNAPSHOT</version>
</parent>

<artifactId>vertx-clickhouse-binary-client</artifactId>

<name>Vertx Clickhouse binary Client</name>
<url>https://github.com/eclipse-vertx/vertx-sql-client</url>
<description>The Reactive Clickhouse Client</description>

<properties>
<doc.skip>false</doc.skip>
<docs.dir>${project.basedir}/src/main/docs</docs.dir>
<generated.dir>${project.basedir}/src/main/generated</generated.dir>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<!-- Set to a value for testing with a specific database -->
<embedded.clickhouse.version />
<log4j.version>2.19.0</log4j.version>
</properties>

<dependencies>

<!-- Vert.x dependencies -->
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-core</artifactId>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-codegen</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-docgen</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-sql-client</artifactId>
</dependency>

<dependency>
<groupId>org.lz4</groupId>
<artifactId>lz4-java</artifactId>
<version>1.8.0</version>
</dependency>

<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-sql-client</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>clickhouse</artifactId>
<version>${testcontainers.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.3.2-patch11</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.3</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>${log4j.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jul</artifactId>
<version>${log4j.version}</version>
<scope>test</scope>
</dependency>


<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1024M</argLine>
<systemPropertyVariables>
<target.dir>${project.build.directory}</target.dir>
<embedded.clickhouse.version>${embedded.clickhouse.version}</embedded.clickhouse.version>
<connection.uri>${connection.uri}</connection.uri>
<!-- TODO: implement encryption -->
<!--tls.connection.uri>${tls.connection.uri}</tls.connection.uri-->
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.bsc.maven</groupId>
<artifactId>maven-processor-plugin</artifactId>
<executions>
<execution>
<id>generate-sources</id>
<configuration>
<optionMap>
<docgen.source>${project.basedir}/../vertx-sql-client/src/main/asciidoc/*.adoc,${asciidoc.dir}/*.adoc</docgen.source>
</optionMap>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<!-- Override sources to use custom descriptor to have all adoc files -->
<execution>
<id>package-sources</id>
<configuration>
<descriptors>
<descriptor>${project.basedir}/../assembly/sources.xml</descriptor>
</descriptors>
<descriptorRefs>
<descriptorRef>none</descriptorRef>
</descriptorRefs>
<ignoreMissingDescriptor>true</ignoreMissingDescriptor>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package io.vertx.clickhouseclient.binary;

import io.vertx.core.json.JsonObject;
import io.vertx.core.json.JsonArray;
import io.vertx.core.json.impl.JsonUtil;
import java.time.Instant;
import java.time.format.DateTimeFormatter;
import java.util.Base64;

/**
* Converter and mapper for {@link io.vertx.clickhouseclient.binary.ClickhouseBinaryConnectOptions}.
* NOTE: This class has been automatically generated from the {@link io.vertx.clickhouseclient.binary.ClickhouseBinaryConnectOptions} original class using Vert.x codegen.
*/
public class ClickhouseBinaryConnectOptionsConverter {


private static final Base64.Decoder BASE64_DECODER = JsonUtil.BASE64_DECODER;
private static final Base64.Encoder BASE64_ENCODER = JsonUtil.BASE64_ENCODER;

public static void fromJson(Iterable<java.util.Map.Entry<String, Object>> json, ClickhouseBinaryConnectOptions obj) {
for (java.util.Map.Entry<String, Object> member : json) {
switch (member.getKey()) {
}
}
}

public static void toJson(ClickhouseBinaryConnectOptions obj, JsonObject json) {
toJson(obj, json.getMap());
}

public static void toJson(ClickhouseBinaryConnectOptions obj, java.util.Map<String, Object> json) {
}
}
Loading