Metadata Processing

[orenbenkiki]

This is a big topic. Main issue is per-cell metadata.
Functionally, we need to project the per-cell metadata to per-metacell metadata in one of several ways:

• Mean/Median (for continuous values). Possibly also variance/stdev?
• Most frequent value (with optional threshold on minimal %, e.g. threshold=0.5 would be "majority")
• Fraction of each value (for discrete values - generates one per-metacell annotation for each value)
  Note we can compute several same per-metacell properties for the same per-cell property.
  (The MC2 python package has functions that do this)

Main options:

• Do this in MC2 so all the per-metacell data would be in metacells.h5ad. Downside: Create a process where someone can go back to MC2, compute additional per-metacell properties, and export just them into MCView.
• Export the per-cell data to MCView (in addition to metacells.h5ad) - this is just per-cell data, not the per-cell-per-gene data, so it isn't too bad. Then within MCView, allow the user to create the projections he needs.

Usage:
Either way, MCView should allow visualizing the per-metacell annotations (on the UMAP) and possibly also allow using them for thing-thing projections (where "thing" could be either a gene or a per-metacell property) and for thing-on-UMAP projection (where "thing" could be either a gene or a per-metacell property). This would allow selecting groups of metacells using properties as well as (and in combination of) genes.

[orenbenkiki]

Additional thoughts:

• Do we really need the import section in config.yaml? That is, wouldn't it better to have an explicit "import" function which takes the name of a CSV file, or the name of an h5ad property, and import them as an annotation into MCView?
• We also need the inverse - an "export" function which takes the name of an existing annotation and creates a CSV file or adds it as an annotation inside an h5ad file.
• Projecting cell properties to metacell properties - in the context of issue Log file of manual annotation actions #3, when we get a new metacells dataset for the same cells, it would be useful to be able to say "project the same properties in the same way" as for the old data set, rather than having to manually re-do all the projections. (This would require keeping a log of the projections similarly to the log of manual annotations.)

[orenbenkiki]

DAF point of view would be to have everything in one dataset and MCView on top of it. We'll need to discuss.