add mysql8 support for 3.0.x branch

README.md

@@ -9,10 +9,11 @@
 5. [Metrics](#metrics)   
 6. [Management Database](#management-database)
 7. [Error Handling And Retry Mechanism](#error-handling-and-retry-mechanism)
-8. [Developer Guide](#developer-guide)
+8. [MySQL 8 Support](#mysql-8-support)
+9. [Developer Guide](#developer-guide)
     1. [Build](#build)
     2. [Tests](#tests)
-9. [Building Custom Applications](docs/custom/README.md)
+10. [Building Custom Applications](docs/custom/README.md)
 
 # Introduction
 This project aims at providing a mechanism to track low-level changes in an OpenMRS database based on [Debezium](https://debezium.io) and 
@@ -81,6 +82,25 @@ The Management DB has been tested with MySQL and H2, you should be able to use a
 by hibernate, it is highly recommended that the management DB resides on the same physical machine as the application to 
 eliminate any possibility of being unreachable and lower latency hence better performance.
 
+# MySQL 8 Support
+
+## Public Key Retrieval
+To avoid the known issue `Unable to connect: Public Key Retrieval is not allowed` with an OpenMRS MySQL 8 database, set the MySQL URL parameter `allowPublicKeyRetrieval` to `true`. This parameter should be added to the following application properties:
+- `debezium.extraParameters` ( extra parameters given to Debezium)
+- `spring.openmrs-datasource.jdbcUrl`
+- `spring.mngt-datasource.jdbcUrl` ( only if you use MySQL 8 for management database)
+
+Please, see the file [example-app/application.properties](example-app/application.properties) for examples.
+
+## Authentication protocol
+
+For some users (debezium user), the connection could fail with the error `AuthenticationException: Client does not support authentication protocol`.
+To fix this, use this following command to create a user compatible with MySQL 8 ( notice the use of `WITH mysql_native_password`);
+
+```sql
+CREATE USER '${DEBEZIUM_USERNAME}'@'%' IDENTIFIED WITH mysql_native_password BY '${DEBEZIUM_PASSWORD}';
+```
+
 # Error Handling And Retry Mechanism
 The `openmrs-watcher` module has a built-in error handling and retry mechanism in case something goes wrong when the 
 sender is processing a DB event, this also applies to any custom application built on top of the openmrs-watcher, the 

example-app/application.properties

@@ -65,7 +65,9 @@ spring.openmrs-datasource.driverClassName=com.mysql.jdbc.Driver
 # Dialect of the openMRS datasource (should not be changed in a usual use)
 spring.openmrs-datasource.dialect=org.hibernate.dialect.MySQLDialect
 
-# Url of the openMRS datasource, you don't have to change this value as long you've set the placeholder property values
+# URL of the openMRS datasource, you don't have to change this value as long you've set the placeholder property values. 
+# For compatibility with MySQL 8, append the allowPublicKeyRetrieval=true parameter to the URL. For example;
+# spring.openmrs-datasource.jdbcUrl=jdbc:mysql://${openmrs.db.host}:${openmrs.db.port}/${openmrs.db.name}?allowPublicKeyRetrieval=true
 spring.openmrs-datasource.jdbcUrl=jdbc:mysql://${openmrs.db.host}:${openmrs.db.port}/${openmrs.db.name}
 
 # User name of the openMRS datasource
@@ -84,7 +86,9 @@ spring.mngt-datasource.driverClassName=org.h2.Driver
 # Dialect of the management datasource (should not be changed in a usual use)
 spring.mngt-datasource.dialect=org.hibernate.dialect.H2Dialect
 
-# Url of the management datasource
+# URL of the management datasource
+# When using MySQL 8 for the management database, append the allowPublicKeyRetrieval=true parameter to the URL. For example;
+# spring.mngt-datasource.jdbcUrl==jdbc:mysql://mysqlUrl:3306/openmrs_eip_mgt?allowPublicKeyRetrieval=true
 spring.mngt-datasource.jdbcUrl=jdbc:h2:file:${eip.home}/db/custom/openmrs_eip_mgt
 
 # User name of the management datasource
@@ -107,6 +111,11 @@ debezium.db.serverId=2
 # database.server.name, DO NOT change after setting it
 debezium.db.serverName=Example
 
+# For a detailed explanation of usable Debezium Connector Component options/parameters including the one used in this example, please refer to https://camel.apache.org/components/3.20.x/debezium-mysql-component.html#_endpoint_query_option_additionalProperties
+# To add an extra parameter allowPublicKeyRetrieval required by mysql 8, additionalProperties.database option documented in the above link will be used. 
+# debezium.extraParameters will be appended to the overall Debezium Connector component URL. Additionally, Debezium Connector Component options/parameters should be prefixed by & symbol. For example;
+# debezium.extraParameters=&additionalProperties.database.allowPublicKeyRetrieval=true
+
 # Database username for debezium user account you created to access the MySQL binlog, maps to the debezium MySQL
 # connector property named database.user, it's highly recommended to create separate user account as described at
 # https://debezium.io/documentation/reference/connectors/mysql.html#setting-up-mysql

openmrs-watcher/src/main/java/org/openmrs/eip/mysql/watcher/DebeziumRoute.java

@@ -30,7 +30,7 @@ public void configure() {
 		RouteDefinition routeDef = from(
 		    "debezium-mysql:extract?databaseServerId={{debezium.db.serverId}}&databaseServerName={{debezium.db.serverName}}&databaseHostname={{openmrs.db.host}}&databasePort={{openmrs.db.port}}&databaseUser={{debezium.db.user}}&databasePassword={{debezium.db.password}}&databaseWhitelist={{openmrs.db.name}}&offsetStorageFileName={{debezium.offsetFilename}}&databaseHistoryFileFilename={{debezium.historyFilename}}&tableWhitelist={{debezium.tablesToSync}}&offsetFlushIntervalMs=0&snapshotMode={{debezium.snapshotMode}}&snapshotFetchSize=1000&snapshotLockingMode={{debezium.snapshotLockingMode}}&includeSchemaChanges=false&maxBatchSize={{debezium.reader.maxBatchSize}}&offsetStorage={{"
 		            + WatcherConstants.PROP_DBZM_OFFSET_STORAGE_CLASS + "}}&databaseHistory={{"
-		            + WatcherConstants.PROP_DBZM_OFFSET_HISTORY_CLASS + "}}&offsetCommitTimeoutMs=15000")
+		            + WatcherConstants.PROP_DBZM_OFFSET_HISTORY_CLASS + "}}&offsetCommitTimeoutMs=15000"+"{{debezium.extraParameters:}}")
 		                    .routeId(DEBEZIUM_ROUTE_ID);
 
 		logger.info("Setting debezium route handler to: " + WatcherConstants.SHUTDOWN_HANDLER_REF);

pom.xml

@@ -20,12 +20,12 @@
     <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
-        <mysqlConnectorVersion>8.0.16</mysqlConnectorVersion>
+        <mysqlConnectorVersion>8.0.25</mysqlConnectorVersion>
         <sprintBootVersion>2.3.0.RELEASE</sprintBootVersion>
         <hibernateVersion>5.4.21.Final</hibernateVersion>
         <hibernateValidatorVersion>6.1.5.Final</hibernateValidatorVersion>
         <camelVersion>3.3.0</camelVersion>
-        <liquibaseVersion>4.2.0</liquibaseVersion>
+        <liquibaseVersion>4.21.1</liquibaseVersion>
         <h2Version>1.4.200</h2Version>
         <guava.version>29.0-jre</guava.version>
         <activeMq.version>5.15.8</activeMq.version>