use circe as serde lib for json: 200

[TebaleloS]

Release notes:

• Added Circe Json serde module library for Json for serialization.
• Added 'Circe generic` in the serde dependencies.
• Added CirceJsonImplicit class and defined implicit conversion for DTO's.
• Integrated Circe with the server module.
• Removed play Json as part of the server dependencies.
• Used Circe to implicitly convert DTO's to Json throughout the project.
• Modified the test cases to suit the Circe format in the for http.
• Replaced jsonbArrayPutUsingString with jsonbArrayPut in DoobieImplicits for both Json and Jsonb objects.
• Reimplemented SerislizationUtils using circe format.
• Replaced Json4s with circe in the agent module.
• Modified SerializationUtilsUnitTest.
• Implemented implicit decoding and encoding in the companion objects of the DTOs.
• Removed jacksonModuleScala dependency.

closes #200

[github-actions]

JaCoCo model module code coverage report - scala 2.13.11

Overall Project	81.12% -423.78%	🍏
Files changed	76.71%	❌

File	Coverage
CheckpointDTO.scala	100% -348.48%	🍏
MeasurementDTO.scala	100% -326.67%	🍏
PartitioningSubmitDTO.scala	100% -333.33%	🍏
MeasureResultDTO.scala	100% -281.82%	🍏
MeasureDTO.scala	100% -326.67%	🍏
PartitionDTO.scala	100% -326.67%	🍏
AdditionalDataSubmitDTO.scala	100% -333.33%	🍏
package.scala	100%	🍏
AtumContextDTO.scala	100% -333.33%	🍏
JsonSyntaxExtensions.scala	63.86% -36.14%	❌
ResultValueType.scala	62.5% -925%	❌
CheckpointQueryDTO.scala	0% -2772.22%	❌

[github-actions]

JaCoCo agent module code coverage report - scala 2.13.11

Overall Project	85.84% -2.25%	🍏
Files changed	63.83%	❌

File	Coverage
MeasureResult.scala	100%	🍏
MeasurementBuilder.scala	94.74%	🍏
Measure.scala	88.28%	🍏
HttpDispatcher.scala	45.68% -13.99%	❌

[github-actions]

JaCoCo server module code coverage report - scala 2.13.11

Overall Project	64.2% -63.35%	🍏
Files changed	59.04%	❌

File	Coverage
SuccessResponse.scala	100% -122.5%	🍏
FlowControllerImpl.scala	100%	🍏
BaseController.scala	100%	🍏
PartitioningForDB.scala	95.45% -345.45%	❌
PartitioningControllerImpl.scala	88.89%	🍏
WriteCheckpoint.scala	77.46%	🍏
CheckpointControllerImpl.scala	76%	🍏
GetPartitioningMeasures.scala	71.3%	🍏
GetPartitioningAdditionalData.scala	70.69%	🍏
CreateOrUpdateAdditionalData.scala	69.69% -37.63%	❌
ErrorResponse.scala	69.39% -2932.65%	❌
CreatePartitioningIfNotExists.scala	69.37% -38.03%	❌
GetPartitioningCheckpoints.scala	54.65%	🍏
GetFlowCheckpoints.scala	0% -5.9%	❌

[TebaleloS]

Release notes:

Added Circe Json serde module library for Json for serialization.
Added 'Circe genericin the serde dependencies. AddedCirceJsonImplicitclass and defined implicit conversion for DTO's. Integrated Circe with the server module. Removed play Json as part of the server dependencies. Used for Circe to implicitly convert DTO's to Json throughout the project. Modified the test cases to suit the Circe format in the for http. ReplacedjsonbArrayPutUsingStringwithjsonbArrayPutinDoobieImplicitsfor both Json and Jsonb objects. ReimplementedSerislizationUtilsusing circe format. Replaced Json4s with circe in the agent module. ModifiedSerializationUtilsUnitTest`.
Implemented implicit decoding and encoding in the companion objects of the DTOs.
Removed jacksonModuleScala dependency.

[salamonpavel]

The whole object SerializationUtils should be removed (or kept only for third party types serde implicits). It's customary to define serde implicits in companion objects - for default de/serialization. If needed one could then provide alternative implementation(s) either in direct scope where the de/serialization is performed or create yet another object for non-default implementations inside the companion object and in import into direct scope only the desired implementation (the preffered approach in my opinion).

A simple example would be:

import io.circe.{Decoder, Encoder, HCursor, Json}
import io.circe.generic.semiauto.{deriveDecoder, deriveEncoder}

case class Person(name: String, age: Int)

object Person {
  implicit val defaultPersonEncoder: Encoder[Person] = deriveEncoder[Person]
  implicit val defaultPersonDecoder: Decoder[Person] = deriveDecoder[Person]

  object JsonImplicits {
    implicit val nonDefaultPersonEncoder: Encoder[Person] = new Encoder[Person] {
      final def apply(a: Person): Json = Json.obj(
        ("name", Json.fromString(a.name)),
        ("isAdult", Json.fromBoolean(a.age >= 18))
      )
    }

    implicit val nonDefaultPersonDecoder: Decoder[Person] = new Decoder[Person] {
      final def apply(c: HCursor): Decoder.Result[Person] =
        for {
          name <- c.downField("name").as[String]
          isAdult <- c.downField("isAdult").as[Boolean]
        } yield {
          val age = if (isAdult) 18 else 17
          Person(name, age)
        }
    }
  }
}

Usage would look like this:

import io.circe.syntax._
import io.circe.parser._

val person = Person("John Doe", 20)

// Using default serde from companion (always available in direct scope without the need to import it)
val defaultJson = person.asJson // implicitly looked up from companion Person.defaultPersonEncoder

// Using non-default serde, passed implicitly
import Person.JsonImplicits.nonDefaultPersonEncoder
val nonDefaultJson = person.asJson

// Using non-default serde, passed explicitly
val nonDefaultJson = person.asJson(Person.JsonImplicits.nonDefaultPersonEncoder)

[TebaleloS]

Thank you for the clarification.

[benedeki]

This seems unused

[salamonpavel]

Yeah, we have defined our serde in companions, therefore we don't need generic.auto.

[benedeki]

Unused

[benedeki]

Dead code

[benedeki]

Dead code

[benedeki]

Dead code

[benedeki]

For discussion:
I would rename these methods to
asJsonString, asJsonStringPretty and fromJsonString
To make ti clear it accepts String, instead of Circe's Json and actually not to confuse with functions of the same name in Circe.

[TebaleloS]

Yah, I can change them. I don't see anything wrong with that.

[salamonpavel]

@benedeki good suggestion, it's better to have them named differently than methods in Circe.

[salamonpavel]

I find this a bit inconsistent. If you intent to have the JsonUtils class with fromJsonString method, then I think you should use the asJsonString method instead of calling asJson.noSpaces. And this way it should be done all over the code base.

[salamonpavel]

Alternatively we could use extension methods.

object JsonSyntaxExtensions {
  implicit class JsonSerializationSyntax[T: Encoder](val obj: T) extends AnyVal {
    def asJsonString: String = obj.asJson.noSpaces
    def asJsonStringPretty: String = obj.asJson.spaces2
  }
  implicit class JsonDeserializationSyntax[T: Decoder](val jsonStr: String) extends AnyVal {
    def as: T = {
      decode[T](jsonStr) match {
        case Right(value) => value
        case Left(error) => throw new RuntimeException(s"Failed to decode JSON: $error")
      }
    }
  }
}

val person = Person("Alice", 30)
 
// Convert a Person instance to a compact JSON string
val jsonString = person.asJsonString
println(jsonString) // {"name":"Alice","age":30}
 
// Convert a Person instance to a pretty JSON string
val jsonPrettyString = person.asJsonStringPretty

val jsonStr = """{"name":"Bob","age":25}"""
 
// Convert a JSON string back to a Person instance
val personFromJson = jsonStr.as[Person]

[TebaleloS]

This I tried to do, but it keeps saying that JsonSerializationSyntax and JsonDeserializationSyntax must have one input and it must be val. So, I switched back to our old implementation.

[salamonpavel]

Working solution

object JsonSyntaxExtensions {
  implicit class JsonSerializationSyntax[T: Encoder](val obj: T)  {
    def asJsonString: String = obj.asJson.noSpaces
    def asJsonStringPretty: String = obj.asJson.spaces2
  }
  implicit class JsonDeserializationSyntax(val jsonStr: String) {
    def as[T: Decoder]: T = {
      decode[T](jsonStr) match {
        case Right(value) => value
        case Left(error) => throw new RuntimeException(s"Failed to decode JSON: $error")
      }
    }
  }
}

[salamonpavel]

Do we really need this 'pretty' version?

[TebaleloS]

It is not used, I guess we don't need it until further notice.

[salamonpavel]

these conversions are not needed, it can be passed as not serialized ... the import sttp.client3.circe._ brings circeBodySerializer method into scope which generate BodySerializer given an implicit Encoder in scope for a given type ...

[salamonpavel]

this doesn't work... I am currently testing these implicits in fa-db

[TebaleloS]

Okay, I'm checking it.

[benedeki]

Good from my point of view.
The little suggestion is not worth its own commit. So once clarified the possible problem @salamonpavel pointed out, I am happy to approve and merge.

[benedeki]

TIny:

Suggested change

	handleResponseBody(response).as[AtumContextDTO]
	handleResponseBody(response).as[AtumContextDTO]

[benedeki]

Love this. Makes it so clear at the usage places.. 😎