Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eyetracking utilization #523

Draft
wants to merge 63 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
ac16240
utilize eyetracking in sciview
RuoshanLan Aug 2, 2021
c3bc1b4
a working version and adjust the size of some messages
RuoshanLan Sep 9, 2021
f1f00c7
code clean
RuoshanLan Sep 9, 2021
9eb458e
let the volume show before starting the eye tracking
RuoshanLan Oct 7, 2021
785eb2c
update bionic tracking
RuoshanLan Oct 12, 2021
0c8afc7
solve conflict
RuoshanLan Oct 12, 2021
b4f7188
add headset for cell tracking
RuoshanLan Nov 15, 2021
0d613f6
add test without VR headset
RuoshanLan Nov 29, 2021
3ddfcfc
a working version of utilizing VRHeadset for cell tracking - part of …
RuoshanLan Dec 3, 2021
efe9902
add function of tracking with controllers
RuoshanLan Dec 17, 2021
7370519
replace numerictype with realtype
RuoshanLan May 19, 2022
7e4c128
resolve conflicts
RuoshanLan May 19, 2022
9fb6c22
fix issue in controller version
RuoshanLan Jul 26, 2022
e1c6013
improve the VRHeadsSettTrackingDemo and fix a bug caused by UV coordi…
RuoshanLan Oct 28, 2022
9348a25
add VR interaction to VRHeadSetTrackingDemo and add nodetaggedevent
RuoshanLan Nov 9, 2022
8aa32f4
code clean
RuoshanLan Nov 9, 2022
3eb8741
merge
RuoshanLan Nov 9, 2022
1d59a30
Fix line endings
RuoshanLan Nov 15, 2022
a7057b0
Gradle: Add missing transitive dependencies
RuoshanLan Nov 15, 2022
12cd940
SciView: Revert failed merge for addVolume functions
RuoshanLan Nov 15, 2022
e909c79
clean and add comment
RuoshanLan Dec 19, 2022
954ae8e
Merge branch 'master' into eyetracking-utilization
smlpt Jul 4, 2023
3735f6c
Gradle Build: update to scenery 0.9.0
smlpt Jul 4, 2023
1ecaf0f
Gradle Build: fixing outdated code
smlpt Jul 6, 2023
0a78f66
SciView: bring back openDirTiff and openTrackFile
smlpt Jul 6, 2023
9efcb08
Gradle: fix directories dep issue
smlpt Jul 11, 2023
80aeccd
build: update gradle and make the build compatible with java 21
smlpt Mar 28, 2024
f116a80
Merge branch 'main' into eyetracking-utilization
smlpt Mar 28, 2024
8b2839d
improve volume loading from scene
smlpt Apr 12, 2024
6136a3c
fix filtering for volumes in the scene
smlpt Apr 12, 2024
d883c68
Merge branch 'main' into eyetracking-utilization
smlpt Apr 15, 2024
279a846
Merge remote-tracking branch 'origin/eyetracking-utilization' into ey…
smlpt Apr 15, 2024
a754d09
SciView: fix VR toggling changes to stereo rendering
smlpt May 7, 2024
bbca486
EyeTrackingDemo: catch if no volume is found
smlpt May 7, 2024
6fe0805
HedgehogAnalysis: add docs and comments
smlpt May 7, 2024
da9979a
Add: test object for loading a volume and starting the eye tracking d…
smlpt Aug 23, 2024
6d94bbd
Merge branch 'main' into eyetracking-utilization
smlpt Aug 26, 2024
8c6c93a
Make newest sciview compatible with local scenery (stupid jackson)
smlpt Aug 26, 2024
fc6dd31
Fix toggleVRRendering
smlpt Aug 26, 2024
4b4bd77
Minor changes to fix finding local maxima, adding info logs
smlpt Aug 26, 2024
5e6f99c
Increase volume size for testing dataset
smlpt Aug 27, 2024
f58cb0b
Clean up logging
smlpt Aug 27, 2024
e1fe8a3
Change showMessage to centered, try to fix track instancing
smlpt Aug 27, 2024
8560296
Fix track cylinder radius
smlpt Sep 2, 2024
1d64c4c
EyeTrackingDemo: update sciview API usage
smlpt Sep 3, 2024
7b8031d
Make eye tracking compatible with mastodon bridge
smlpt Sep 6, 2024
153dabf
Change log to logger for consistency
smlpt Sep 10, 2024
b03c042
Add: gaussian smoothing function
smlpt Sep 11, 2024
b5e5eda
Change to sampleRayGridTraversal and add samplePos list to SpineMetadata
smlpt Sep 11, 2024
1d77ea3
HedgehogAnalysis: get spine vertex position from grid traversal
smlpt Sep 13, 2024
34b76f8
Fix: ignore children in AABB intersection
smlpt Sep 17, 2024
7325742
EyeTrackingDemo: fix vertex positions
smlpt Sep 19, 2024
f51de82
Create CellTrackingBase, let other classes inherit most of the (other…
smlpt Oct 8, 2024
8f05cc4
Make sure that cam is not inside volume when tracking
smlpt Oct 9, 2024
9bcafe8
Add names to scene objects to make it easier to find and remove them …
smlpt Oct 9, 2024
e6c8612
CellTrackingBase: moved file to demo/advanced
smlpt Oct 9, 2024
886997a
Refactor the eye tracking logic into EyeTracking, so that EyeTracking…
smlpt Oct 9, 2024
6d93dc0
Add proper eyetracking shutdown handling
smlpt Oct 10, 2024
9cafe64
Unified naming scheme, extracted tracking functionality into separate…
smlpt Oct 11, 2024
f072138
Use lazylogger instead of Logservice
smlpt Oct 11, 2024
7c2f2c3
StartEyeTrackingDirectly: Rename to EyeTrackingCommand
smlpt Oct 14, 2024
d2d2618
SciviewBridge: wrap eyeTracking into a separate thread to fix blank s…
smlpt Nov 6, 2024
317734b
Add TimepointObserver to unify timepoint updates across VR and bridge
smlpt Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 38 additions & 22 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ dependencies {
}

implementation("net.java.dev.jna:jna-platform:5.14.0")
// implementation("net.clearvolume:cleargl")
implementation("org.janelia.saalfeldlab:n5")
implementation("org.janelia.saalfeldlab:n5-imglib2")
implementation("org.apache.logging.log4j:log4j-api:2.20.0")
Expand Down Expand Up @@ -148,6 +149,19 @@ dependencies {
val isRelease: Boolean
get() = System.getProperty("release") == "true"

//kotlin {
// jvmToolchain(21)
//// compilerOptions {
//// jvmTarget = JvmTarget.JVM_21
//// freeCompilerArgs = listOf("-Xinline-classes", "-opt-in=kotlin.RequiresOptIn")
//// }
//}
//
//java {
// targetCompatibility = JavaVersion.VERSION_21
// sourceCompatibility = JavaVersion.VERSION_21
//}

tasks {
withType<KotlinCompile>().all {
val version = System.getProperty("java.version").substringBefore('.').toInt()
Expand Down Expand Up @@ -414,30 +428,32 @@ tasks {

register<JavaExec>(name = "run") {
classpath = sourceSets.main.get().runtimeClasspath
if (project.hasProperty("target")) {
project.property("target")?.let { target ->
classpath = sourceSets.test.get().runtimeClasspath

println("Target is $target")
// if(target.endsWith(".kt")) {
// main = target.substringAfter("kotlin${File.separatorChar}").replace(File.separatorChar, '.').substringBefore(".kt")
// } else {
// main = target.substringAfter("java${File.separatorChar}").replace(File.separatorChar, '.').substringBefore(".java")
// }

mainClass.set("$target")
val props = System.getProperties().filter { (k, _) -> k.toString().startsWith("scenery.") }

val additionalArgs = System.getenv("SCENERY_JVM_ARGS")
allJvmArgs = if (additionalArgs != null) {
allJvmArgs + props.flatMap { (k, v) -> listOf("-D$k=$v") } + additionalArgs
} else {
allJvmArgs + props.flatMap { (k, v) -> listOf("-D$k=$v") }
}
var target: Any? = null
if (project.hasProperty("target"))
target = project.property("target")
target = "StartEyeTrackingDirectlyKt"
if (target != null) {
classpath = sourceSets.test.get().runtimeClasspath

println("Target is $target")
// if(target.endsWith(".kt")) {
// main = target.substringAfter("kotlin${File.separatorChar}").replace(File.separatorChar, '.').substringBefore(".kt")
// } else {
// main = target.substringAfter("java${File.separatorChar}").replace(File.separatorChar, '.').substringBefore(".java")
// }

mainClass.set("$target")
val props = System.getProperties().filter { (k, _) -> k.toString().startsWith("scenery.") }

println("Will run target $target with classpath $classpath, main=${mainClass.get()}")
println("JVM arguments passed to target: $allJvmArgs")
val additionalArgs = System.getenv("SCENERY_JVM_ARGS")
allJvmArgs = if (additionalArgs != null) {
allJvmArgs + props.flatMap { (k, v) -> listOf("-D$k=$v") } + additionalArgs
} else {
allJvmArgs + props.flatMap { (k, v) -> listOf("-D$k=$v") }
}

println("Will run target $target with classpath $classpath, main=${mainClass.get()}")
println("JVM arguments passed to target: $allJvmArgs")
}
}

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
org.gradle.jvmargs=-XX:MaxMetaspaceSize=2g
org.gradle.caching=true
jvmTarget=21
#useLocalScenery=true
useLocalScenery=true
kotlinVersion=1.9.23
dokkaVersion=1.9.10
scijavaParentPOMVersion=37.0.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;



public class DefaultLogger extends Logger {
@SuppressWarnings("UnusedParameters")
public DefaultLogger(String category) { }
Expand Down
85 changes: 85 additions & 0 deletions src/main/java/sc/iview/commands/file/OpenDirofTif.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*-
* #%L
* Scenery-backed 3D visualization package for ImageJ.
* %%
* Copyright (C) 2016 - 2021 SciView developers.
* %%
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
* #L%
*/
package sc.iview.commands.file;

import org.scijava.command.Command;
import org.scijava.io.IOService;
import org.scijava.log.LogService;
import org.scijava.plugin.Menu;
import org.scijava.plugin.Parameter;
import org.scijava.plugin.Plugin;
import sc.iview.SciView;

import java.io.File;
import java.io.IOException;

import static sc.iview.commands.MenuWeights.FILE;
import static sc.iview.commands.MenuWeights.FILE_OPEN;

/**
* Command to open a file in SciView
*
* @author Kyle Harrington
*
*/
@Plugin(type = Command.class, menuRoot = "SciView", //
menu = { @Menu(label = "File", weight = FILE), //
@Menu(label = "Open Directory of tif...", weight = FILE_OPEN) })
public class OpenDirofTif implements Command {

@Parameter
private IOService io;

@Parameter
private LogService log;

@Parameter
private SciView sciView;

// TODO: Find a more extensible way than hard-coding the extensions.
@Parameter(style = "directory")
private File file;

@Parameter
private int onlyFirst = 0;

@Override
public void run() {
try {
if(onlyFirst > 0) {
sciView.openDirTiff(file.toPath(), onlyFirst);
} else {
sciView.openDirTiff(file.toPath(), null);
}
}
catch (final IOException | IllegalArgumentException exc) {
log.error( exc );
}
}
}
79 changes: 79 additions & 0 deletions src/main/java/sc/iview/commands/file/OpenTrackFile.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*-
* #%L
* Scenery-backed 3D visualization package for ImageJ.
* %%
* Copyright (C) 2016 - 2021 SciView developers.
* %%
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
* #L%
*/
package sc.iview.commands.file;

import org.scijava.command.Command;
import org.scijava.io.IOService;
import org.scijava.log.LogService;
import org.scijava.plugin.Menu;
import org.scijava.plugin.Parameter;
import org.scijava.plugin.Plugin;
import sc.iview.SciView;

import java.io.File;
import java.io.IOException;

import static sc.iview.commands.MenuWeights.FILE;
import static sc.iview.commands.MenuWeights.FILE_OPEN;

/**
* Command to open a file in SciView
*
* @author Kyle Harrington
*
*/
@Plugin(type = Command.class, menuRoot = "SciView", //
menu = { @Menu(label = "File", weight = FILE), //
@Menu(label = "Open Track File", weight = FILE_OPEN) })
public class OpenTrackFile implements Command {

@Parameter
private IOService io;

@Parameter
private LogService log;

@Parameter
private SciView sciView;

// TODO: Find a more extensible way than hard-coding the extensions.
@Parameter(style = "open,extensions:csv")
private File file;

@Override
public void run() {
try {
sciView.openTrackFile(file);

}
catch (final IOException | IllegalArgumentException exc) {
log.error( exc );
}
}
}
10 changes: 10 additions & 0 deletions src/main/java/sc/iview/event/NodeTaggedEvent.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package sc.iview.event;

import graphics.scenery.Node;


public class NodeTaggedEvent extends NodeEvent {
public NodeTaggedEvent(final Node node ) {
super( node );
}
}
Loading
Loading