Feature/243 (#275)

* #255 feat: Allow adding populations to existent experiment * #261 feat: Add instant feedback on population updates * #246 feat: WIP - Add residential populations * #246 fix: Fix residential populations race condition * #246 fix: Fix UI misalignments on resident populations * #246 fix: Fix color picker not opening * #246 fix: Fix residential populations race condition * #243 feat: WIP - Update heatmap to contour plot * #246 feat: Update sidebar design * #246 feat: Update sidebar width * #246 feat: Add complete data for residential populations; Update migration * #246 chore: Remove unnecessary print * #243 feat: WIP - Update contour creator * #242 feat: WIP - Add atlas upscale service * #243 feat: WIP - Update probability map calculations * #242 fix: Fix interpolate_atlas_section * #242 chore: Rename annotation to grey_and_white_matter * #242 fix: Fix canal image offset * #242 feat: Upscale grid dimensions * #242 fix: Fix gray and white matter image generation * #242 fix: Update hidden canvas dimensions * #242 fix: Update white and grey matter image generation * #242 fix: Fix centroids plot points position * #242 fix: Update centroids plot points size * #243 feat: Add contour plot image creator * #243 feat: Add overlay to contours * #243 feat: Rename probability map overlay to contour plot * #243 chore: Improve resources management for plotting * #243 chore: Rename density map type * #243 chore: Add management command to update static images of all populations * chore: Update geppetto-meta * chore: Update codefresh-dev * chore: Update codefresh-dev * chore: Add missing icon * chore: Update local setup script * #243 feat: Make contour plot creator generate 2 images * #243 feat: Add grid to contour plot UI control * #242 feat: Update 3D meshes * #243 fix: Update contour plot creator * #243 feat: Update residential populations * #243 fix: Fix typo in neuronal location switch * #243 feat: Update contour levels * #243 feat: Consider intensity on drawColoredImage * #242 chore: Add 3D meshes * #chore: Remove deprecated annotation changes * #274 fix: Update download populations to work with residential populations * #243 feat: Update static images creators * #243 chore: Update residential populations data * #243 fix: Update drawColoredImageCallback to allow for skipping intensity changes * #243 chore: Update residential populations * #243 fix: Fix background image generation * #284 changes made * #257 feat: Update cordmap * #284 population entry ui changed * #284 code cleaning * #284 small changes * #284 changes made * #284 Integrating the populations with hierarchy accordion * fix: Fix indentation typo * #285 chore: Fix itk * #284 cleanup and split function into two * #284 disable subpopulation color change * #285 fix: Fix indentation typo * feat: Add residential populations back * #243 chore: WIP - Merge with develop * #284 feat: Connect show all * #284 feat: Connect parent switch * #284 feat: Connect color picker * #284 feat: Connect download populations * #284 fix: Fix population grouping * #284 chore: Fix linting * #243-b ui changes made * 290 Add z-axis position in the 2D viewer (#311) * #develop changed some styling * Revert "#develop changed some styling" This reverts commit a85b1f8. * #290 added indicator cord image and label --------- Co-authored-by: afonso pinto <[email protected]> * Feature/276 (#315) * #276 chore: Add mesh splitting to build pipeline * #276 chore: Update residential populations migration * #276 chore: Undo local dev change * fix: Import real model instead of historic version --------- Co-authored-by: Aiga115 <[email protected]> Co-authored-by: D. Gopal Krishna <[email protected]> Co-authored-by: Aigul Zhyldyzbekova <[email protected]>
MetaCell · Oct 31, 2023 · 387024b · 387024b
1 parent b7e67a4
commit 387024b
Show file tree

Hide file tree

Showing 100 changed files with 223,437 additions and 494,726 deletions.
diff --git a/applications/portal/Dockerfile b/applications/portal/Dockerfile
@@ -8,7 +8,6 @@ RUN mkdir -p ${APP_DIR}
 
 WORKDIR ${APP_DIR}/..
 COPY cordmap ./cordmap
-RUN python3 -m pip install itk==5.2.1.post1 --no-cache-dir --upgrade
 RUN python3 -m pip install -e cordmap --no-cache-dir --upgrade
 
 ADD local-atlas/salk_cord_10um_v1.0.tar.gz /root/.brainglobe/
@@ -29,7 +28,8 @@ RUN mkdir -p ${APP_DIR}/static/www
 RUN python3 manage.py generate_canal_images salk_cord_10um
 RUN python3 manage.py generate_grid_images salk_cord_10um
 RUN python3 manage.py generate_lamina_images salk_cord_10um
-RUN python3 manage.py generate_annotation_images salk_cord_10um
+RUN python3 manage.py generate_grey_and_white_matter_images salk_cord_10um
+RUN python3 manage.py split_meshes salk_cord_10um
 
 # compile frontend
 FROM node:16-alpine as frontend

diff --git a/applications/portal/api/openapi.yaml b/applications/portal/api/openapi.yaml
@@ -399,6 +399,22 @@ paths:
           description: ''
       tags:
       - api
+  /api/population/residential/:
+    get:
+      operationId: residentialPopulation
+      description: This viewset automatically provides `list` actions.
+      parameters: []
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '#/components/schemas/Population'
+          description: Successful retrieval of the residential populations
+      tags:
+      - api
   /api/population/{id}/:
     get:
       operationId: retrievePopulation
@@ -540,9 +556,9 @@ paths:
           description: ''
       tags:
       - api
-  /api/population/{id}/probability_map/{subdivision}/:
+  /api/population/{id}/contour_plot/{subdivision}/{type}/:
     get:
-      operationId: probabilityMapPopulation
+      operationId: contourPlotPopulation
       description: This viewset automatically provides `list` actions.
       parameters:
       - name: id
@@ -557,6 +573,12 @@ paths:
         description: ''
         schema:
           type: string
+      - name: type
+        in: path
+        required: true
+        description: ''
+        schema:
+          type: string
       responses:
         '200':
           content:
@@ -1302,6 +1324,7 @@ components:
                 maxLength: 7
               experiment:
                 type: integer
+                nullable: true
               atlas:
                 enum:
                 - salk_cord_10um
@@ -1362,6 +1385,7 @@ components:
           maxLength: 7
         experiment:
           type: integer
+          nullable: true
         atlas:
           enum:
           - salk_cord_10um

diff --git a/applications/portal/backend/api/constants.py b/applications/portal/backend/api/constants.py
@@ -6,7 +6,8 @@
 
 class PopulationPersistentFiles(Enum):
     CSV_FILE = ".csv"
-    PROBABILITY_MAP_IMG = "_probability_map.png"
+    CONTOUR_PLOT_IMG = "_contour_plot.png"
+    CONTOUR_PLOT_WITH_OVERLAY_IMG = "_contour_plot_with_overlay.png"
     CENTROIDS_IMG = "_centroids.png"
 
 

diff --git a/applications/portal/backend/api/helpers/bg_atlasapi_wrapper.py b/applications/portal/backend/api/helpers/bg_atlasapi_wrapper.py
@@ -1,48 +1,48 @@
 import numpy as np
 
+from api.constants import ROSTRAL
 from api.helpers.icustom_atlas import ICustomAtlas
+from workspaces.settings import POSITION_WITHIN_SUBDIVISION
 
 
 class SalkAtlas(ICustomAtlas):
 
     def __init__(self, atlas_name):
         super().__init__(atlas_name)
-        self._annotation = self._get_annotation_with_merged_layers_1_to_4()
+        self._update_annotation()
 
     def _get_atlas_merge_layers_1_to_4(self):
-        return np.logical_and(self.annotation >= self._get_structure_annotation_value('1Sp'),
-                              self.annotation <= self._get_structure_annotation_value('4Sp'))
+        return np.logical_and(self.annotation >= self.get_structure_annotation_value('1Sp'),
+                              self.annotation <= self.get_structure_annotation_value('4Sp'))
 
-    def _get_annotation_with_merged_layers_1_to_4(self):
-        annotation_copy = self.annotation.copy()
-        annotation_copy[self._get_atlas_merge_layers_1_to_4()] = self._get_structure_annotation_value('1-4Sp')
-        return annotation_copy
-
-    @property
-    def canal(self):
-        """
-        Generates an image volume of the central canal
-        """
-        return self.get_image_volume(self.structures["CC"]["id"])
+    def _update_annotation(self):
+        self.annotation[self._get_atlas_merge_layers_1_to_4()] = self.get_structure_annotation_value('1-4Sp')
 
     def get_image_volume(self, region_key):
         """
         Generates 3D numpy array mask of the input region.
         """
         region_id = self.structures[region_key]["id"]
-        img = self.annotation == self._get_structure_annotation_value(region_key)
+        img = self.annotation == self.get_structure_annotation_value(region_key)
         for child in self.hierarchy.children(region_id):
             img = np.logical_or(img, self.annotation == (child.identifier - 1))
         return img
 
-    def get_annotation(self, structure_key_list):
-        new_annotation = np.empty_like(self.annotation)
-        for region_key in structure_key_list:
-            img = self.get_image_volume(region_key)
-            locs = np.where(img)
-            new_annotation[locs] = self.structures[region_key]["id"] - 1
-        return new_annotation
-
-    def _get_structure_annotation_value(self, structure_key: str) -> int:
+    def get_structure_annotation_value(self, structure_key: str) -> int:
         structure_id = self.structures[structure_key]["id"]
         return structure_id - 1
+
+    def get_subdivision_limits(self, subdivision: str) -> tuple:
+        segment, part = subdivision.split("-")
+        for s in self.metadata["atlas_segments"]:
+            if s["Segment"] == segment:
+                if part == ROSTRAL:
+                    return s["Start"], (s["Start"] + s["End"]) / 2
+                else:
+                    return (s["Start"] + s["End"]) / 2, s["End"]
+
+    def get_section_idx(self, subdivision, position_within_segment=POSITION_WITHIN_SUBDIVISION):
+        segment_start, segment_end = self.get_subdivision_limits(subdivision)
+        return int(
+            (segment_end - segment_start) * position_within_segment + segment_start
+        )
diff --git a/applications/portal/backend/api/helpers/density_map/centroids_creator.py b/applications/portal/backend/api/helpers/density_map/centroids_creator.py
@@ -1,103 +1,30 @@
-import matplotlib.image as mimage
+from typing import Dict
+
 import numpy as np
+from PIL.Image import Image
 from matplotlib import pyplot as plt
-from matplotlib import rcParams
-from PIL import Image
 
-from api.constants import FULLY_OPAQUE
-from api.helpers.density_map.common_density_helpers import get_subdivision_limits
-from api.helpers.density_map.generate_image import get_annotation_array, shift_image_array, get_canal_offset, \
-    get_pad_from_offset
+from api.constants import PopulationPersistentFiles
+from api.helpers.density_map.common_plot_helpers import setup_matplotlib_figure, plot_to_shifted_image
+from api.helpers.density_map.generate_image import get_grey_and_white_matter_image_array
 from api.helpers.density_map.ipopulation_image_creator import IPopulationImageCreator
 from api.helpers.icustom_atlas import ICustomAtlas
-from api.helpers.image_manipulation import fig_to_img, fig_to_numpy, get_image_from_array, black_to_transparent, \
-    pad_image
-from workspaces.settings import FIGURE_DPI
+from workspaces.settings import UPSCALE_FACTOR
 
 
 class CentroidsCreator(IPopulationImageCreator):
     def create(
-            self, bg_atlas: ICustomAtlas, subdivision: str, points: np.array
-    ) -> Image:
-        return _generate_centroids(bg_atlas, subdivision, points)
+            self, bg_atlas: ICustomAtlas, subdivision: str, points: np.array, is_residential: bool
+    ) -> Dict[PopulationPersistentFiles, Image]:
+        return {PopulationPersistentFiles.CENTROIDS_IMG: _generate_centroids(bg_atlas, subdivision, points)}
 
 
 def _generate_centroids(
         bg_atlas: ICustomAtlas, subdivision: str, points: np.array
 ) -> Image:
-    subdivision_limits = get_subdivision_limits(bg_atlas, subdivision)
-    points_slice = points[
-        np.logical_and(
-            subdivision_limits[0] <= points[:, 0], points[:, 0] <= subdivision_limits[1]
-        )
-    ]
-    points_slice = points_slice[:, 1:]
-    im = get_annotation_array(bg_atlas, subdivision)
-    dpi = FIGURE_DPI
-    h, w = im.shape
-    xmin, xmax, ymin, ymax = -0.5, w + 0.5, -0.5, h + 0.5
-    fig = plt.figure(figsize=(w / dpi, h / dpi), dpi=dpi)
-    ax = plt.Axes(fig, (0, 0, 1, 1))
-    ax.set_xlim(xmin, xmax)
-    ax.set_ylim(ymin, ymax)
-    ax.invert_yaxis()
-    fig.add_axes(ax)
-    fig.set_facecolor((0, 0, 0, 0))
-    _imshow(ax, im)
-    plt.scatter(x=points_slice[:, 1], y=points_slice[:, 0], c="y", s=1)
-    plt.grid(False)
-    plt.axis("off")
-    img = fig_to_img(fig)
-    return pad_image(img, *get_pad_from_offset(get_canal_offset(bg_atlas, subdivision)))
-
-
-def _imshow(
-        axis,
-        X,
-        cmap=None,
-        norm=None,
-        aspect=None,
-        interpolation=None,
-        alpha=None,
-        vmin=None,
-        vmax=None,
-        origin=None,
-        extent=None,
-        *,
-        interpolation_stage=None,
-        filternorm=True,
-        filterrad=4.0,
-        resample=None,
-        url=None,
-        **kwargs
-):
-    if aspect is None:
-        aspect = rcParams["image.aspect"]
-    axis.set_aspect(aspect)
-    im = mimage.AxesImage(
-        axis,
-        cmap,
-        norm,
-        interpolation,
-        origin,
-        extent,
-        filternorm=filternorm,
-        filterrad=filterrad,
-        resample=resample,
-        interpolation_stage=interpolation_stage,
-        **kwargs
-    )
-
-    im.set_data(X)
-    im.set_alpha(alpha)
-    if im.get_clip_path() is None:
-        # image does not already have clipping set, clip to axes patch
-        im.set_clip_path(axis.patch)
-    im._scale_norm(norm, vmin, vmax)
-    im.set_url(url)
-
-    # update ax.dataLim, and, if autoscaling, set viewLim
-    # to tightly fit the image, regardless of dataLim.
-    im.set_extent(im.get_extent())
+    points_slice = points[:, 1:] * UPSCALE_FACTOR
+    im = get_grey_and_white_matter_image_array(bg_atlas, subdivision)
 
-    return im
+    fig, ax = setup_matplotlib_figure(im)
+    plt.scatter(x=points_slice[:, 1], y=points_slice[:, 0], c="k", s=1 * UPSCALE_FACTOR)
+    return plot_to_shifted_image(fig, bg_atlas, subdivision)