Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools #258

AndreiKingsley · 2024-10-08T14:07:05Z

Solves Add Jupyter integration(s) #244.

Add Jupyter Integration plot-api and toolkit/geotools modules. All the old logic (imports/init/renders) from descriptors has been moved into them. Updated descriptors are in this PR.

A new type returned by rendering has also been added, which allows you to use the following features that are now available for Kandy plots in Kotlin Notebook:

Native rendering in Swing;
Dynamic IDEA theme application for plots (only for plots where flavor is not specified);
Export plot from cell in different formats (PNG, SVG, JPG, HTML);
Copy plot (as PNG) from cell;

Added letsPlotNotebookConfig variable (in notebooks only) which allows to disable native rendering in KTNB.

These changes do not affect Jupyter and Datalore notebooks.

Also this rendering adds static SVG in the output, hidden by script which allows to show static part of plot on GitHub/Gist.

Several simple tests for Jupyter Integration are also added.

…tools

AndreiKingsley · 2024-10-08T14:38:23Z

I've run into the problem that the latest versions of the Kotlin jupyter api use java 11, which makes it completely incompatible with geotools module, and partially with plot-api. In particular, I had to add different versions of the plugin to them because of this. But it's clear that this is a bad solution. @ileasile suggested the following: Kotlin/kotlin-jupyter-libraries#421 (comment)

ileasile · 2024-10-08T14:56:05Z

I suggest adding lets-plot-jupyter and lets-plot-geotools-jupyter modules. They should have all dependencies that are now listed in corresponding descriptors, and also integration classes. Also, they should target 11th version of JVM bytecode. I'm ready to pair-program it with you if there are any problems in implementation

AndreiKingsley · 2024-10-08T15:07:18Z

Shouldn't be difficult, but I'm interested in @alshan opinion.

alshan · 2024-10-08T17:02:47Z

Jupyter integration doesn't belong to the "plot-api" project in my opinion. Let's move it to a separate project nested in the 'toolkit' folder? Can have java 11 as a target there.

plot-api/src/commonMain/kotlin/org/jetbrains/letsPlot/frontend/NotebookFrontendContext.kt

AndreiKingsley · 2024-10-10T13:46:45Z

@alshan, we can fix the version of the lets-plot library (and, but not necessarily - js) when compiling the artifact with integration. This will help user to avoid version incompatibility. That is, there will be only one version in the descriptor - Lets-Plot Kotlin, one dependency that will load the lpk of this version and lp corresponding to it (bundled at compilation). What do you think?

AndreiKingsley · 2024-10-10T14:34:19Z

And of course it will avoid cases when the integration code is written on a version incompatible with the specified in argument.

alshan · 2024-10-10T19:08:31Z

Do you mean drop arguments in %use lets-plot ?

AndreiKingsley · 2024-10-10T19:14:58Z

Yes. There will be only one dependency on lets-plot-kotlin-jupyter, that will load lets-plot-kotlin with the same version and lets-plot artifacts with tge corresonding for this lpk versions.

alshan · 2024-10-10T19:43:43Z

Sounds good.

…K implementations; extract spec serialization to util module

AndreiKingsley · 2024-10-12T14:43:35Z

Extracted integrations to separated modules jupyter and geotools-jupyter. These modules creates artifacts with all LP and LPK implementation (and all geotools implementations for geotools-jupyter) and can be used in notebooks with updated descriptors (this PR with descriptors should be merged with the versions of integration artifacts inserted).
Also added "util" module and artifact - it contains Lets-Plot spec serialization and deserialization methods. It's used internally in Integration and should be used externally in Kotlin Notebook plugin (now it uses "kandy-util" similar artifact, but it's better to store it here, since it's related to Lets-Plot, for future changes; also it can be very useful for users who want to have Lets-Plot spec serialization).

alshan · 2024-10-15T16:49:15Z

toolkit/util/build.gradle.kts

@@ -0,0 +1,66 @@
+plugins {


We like self-descriptive code. Could we rename "utils" to something more specific? Who's the utils client?

Well, this module is responsible for spec serialization. Also, I'd like to move here a specific logic, that is used for example in Kotlin Notebook - applying the flavour for spec if it is not set (this is necessary for auto-applying the environment theme).
So generally, this is useful stuff for those who create a Lets-plot spec on the client and send it (like Kotlin Notebook).

The package is package org.jetbrains.letsPlot.toolkit.jupyter.util.

If this project is jupyter specific I would propose to rename it to "jupyter-util".

On the other hand, if it's not jupyter specific then we better remove "jupyter" from the package name and rename the project to smth like "kotlinx-serialization" (and the package - accordingly).

Ok, may be just "serialization"?

I'm not sure now. This is generic serialization code not related to LP or Jupyter. Strictly speaking it shouldn't be here at all. Do you use it in Jupyter integration?

Ok, let's remove mentioning of letsPlot and spec:

typealias LetsPlotSpec = Map<String, Any> fun serializeSpec(spec: LetsPlotSpec): JsonElement { return serialize(spec) }

Next, this module is looking as a patch/missing feature for kotlinx.serilization.json wherefore we can name the package as "org.jetbrains.letsPlot.toolkit.kotlinx.json" and the project name would be "kotlinx-json".
What do you think?

But why would we remove the mention of lets-plot? We can more generally say that it is a serialisation of a Map that contains nested only Map, List and primitives, but it is exactly a serialisation of Lets-Plot spec. Yes, the module does not use Lets-plot directly, but this function is restricted to work specifically with Lets-Plot spec. If the spec structure is changed in the future (like adding a new possible type, or something like that), then we will also have to change this logic. So this implementation depends directly on Lets-Plot.

But JSON is essentially just Maps, Lists and primitives. I don't see anything that is specific for lets-plot.

Renamed to json. All Lets-plot mentions are removed.

alshan · 2024-10-15T20:59:12Z

toolkit/util/src/main/kotlin/org/jetbrains/letsPlot/toolkit/jupyter/util/serializeSpec.kt

+        is String -> JsonPrimitive(obj)
+        is Boolean -> JsonPrimitive(obj)
+        is Number -> JsonPrimitive(obj)
+        else -> error("Don't know how to parse object [$obj] of class ${obj::class}")


The error message: Don't know how to serialize object .. ?

alshan · 2024-10-15T21:08:56Z

toolkit/util/src/main/kotlin/org/jetbrains/letsPlot/toolkit/jupyter/util/serializeSpec.kt

+    }
+}
+
+fun deserializeSpec(json: JsonElement): LetsPlotSpec {


Can be replaced with deserializeMap with an additional parameter controlling handling of null (i.e. keep/drop/error).

ileasile · 2024-10-18T01:28:20Z

toolkit/util/src/main/kotlin/org/jetbrains/letsPlot/toolkit/jupyter/util/serializeSpec.kt

+        json is JsonNull -> null
+        json.isString -> json.content
+        else -> {
+            json.booleanOrNull ?: json.doubleOrNull ?: error("Unknown JSON primitive type: [$json]")


@alshan we don't try to parse int here (and most likely it will result in an error) so I would say it's not completely universal

AFAIK there is only one numeric type in JSON - double-precision floating-point and decimal point in the string representation is just an option. Accordingly, the code above looks right and universal.

Agg jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geo…

9fadc51

…tools

AndreiKingsley mentioned this pull request Oct 8, 2024

Remove imports/init/renders from lets-plot descriptors (move to Integ… Kotlin/kotlin-jupyter-libraries#421

Merged

alshan reviewed Oct 8, 2024

View reviewed changes

plot-api/src/commonMain/kotlin/org/jetbrains/letsPlot/frontend/NotebookFrontendContext.kt Outdated Show resolved Hide resolved

AndreiKingsley added 2 commits October 12, 2024 17:54

Extract integrations to separate modules and artifacts with all LP/LP…

831de94

…K implementations; extract spec serialization to util module

little code clean up

7599b93

AndreiKingsley requested a review from alshan October 12, 2024 14:43

AndreiKingsley changed the title ~~Agg jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools~~ Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools Oct 14, 2024

alshan requested changes Oct 15, 2024

View reviewed changes

ileasile reviewed Oct 18, 2024

View reviewed changes

AndreiKingsley added 3 commits November 4, 2024 14:28

rename util to json & remove lets-plot mention

bda81dd

Merge branch 'master' into jupyter_integration

5da00aa

move serialization version

90ea70f

AndreiKingsley requested a review from alshan November 4, 2024 11:18

alshan merged commit c81d496 into JetBrains:master Nov 4, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools #258

Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools #258

AndreiKingsley commented Oct 8, 2024

AndreiKingsley commented Oct 8, 2024

ileasile commented Oct 8, 2024

AndreiKingsley commented Oct 8, 2024

alshan commented Oct 8, 2024

AndreiKingsley commented Oct 10, 2024

AndreiKingsley commented Oct 10, 2024

alshan commented Oct 10, 2024

AndreiKingsley commented Oct 10, 2024

alshan commented Oct 10, 2024

AndreiKingsley commented Oct 12, 2024 •

edited

Loading

alshan Oct 15, 2024

AndreiKingsley Oct 15, 2024

alshan Oct 15, 2024 •

edited

Loading

AndreiKingsley Oct 15, 2024

alshan Oct 15, 2024

alshan Oct 15, 2024 •

edited

Loading

AndreiKingsley Oct 15, 2024

alshan Oct 15, 2024

AndreiKingsley Oct 15, 2024

AndreiKingsley Nov 4, 2024

alshan Oct 15, 2024

alshan Oct 15, 2024 •

edited

Loading

ileasile Oct 18, 2024

alshan Oct 18, 2024

Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools #258

Add jupyter integration for lets-plot-kotlin and lets-plot-kotlin-geotools #258

Conversation

AndreiKingsley commented Oct 8, 2024

AndreiKingsley commented Oct 8, 2024

ileasile commented Oct 8, 2024

AndreiKingsley commented Oct 8, 2024

alshan commented Oct 8, 2024

AndreiKingsley commented Oct 10, 2024

AndreiKingsley commented Oct 10, 2024

alshan commented Oct 10, 2024

AndreiKingsley commented Oct 10, 2024

alshan commented Oct 10, 2024

AndreiKingsley commented Oct 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alshan Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alshan Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alshan Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndreiKingsley commented Oct 12, 2024 •

edited

Loading

alshan Oct 15, 2024 •

edited

Loading

alshan Oct 15, 2024 •

edited

Loading

alshan Oct 15, 2024 •

edited

Loading