Writing Skeleton Question

[tathey1]

Hi,

I'm having issues creating skeleton objects locally using cloud-volume that can be seen in neuroglancer. Take this toy code:

from cloudvolume import CloudVolume, Skeleton
import numpy as np

path = "file://<path-to-segmentation-layer>"

info = CloudVolume.create_new_info(
    num_channels    = 1,
    layer_type      = 'segmentation',
    data_type       = 'uint64',
    encoding        = 'raw', 
    resolution      = [1, 1, 1], 
    voxel_offset    = [0, 0, 0], 
    skeletons = "skeletons",
    chunk_size      = [ 5, 5, 5 ],
    volume_size     = [ 10, 10, 10 ],
)
vol = CloudVolume(path, info=info)
vol.commit_info() 
vol.skeleton.meta.commit_info() #commit skeleton info file

#create image and skeleton representing line through center of volume
seg = np.zeros(vol.volume_size, dtype=vol.data_type)
seg[5,5,:] = 1

vol[:,:,:] = seg
skel = Skeleton(segid = 1, vertices=[[5,5,0], [5,5,9]], edges=[[0,1]])
vol.skeleton.upload(skel)
vol.viewer()

When I use the neuroglancer live demo: https://neuroglancer-demo.appspot.com/ and enter the layer precomputed://<address-from-viewer()>, I can see the segmentation volume properly, but I cannot load the skeleton (see attached screenshots). What's weird is that CloudVolume has no issue reading the skeleton object, and further it appears that the browser is correctly locating it (I get HTTP 200 message), but nothing is showing.

• I noticed that neuroglancer displayed an error "Data type not supported by WebGL: UINT8" which is the data_type of "vertex_types" in the skeleton info file, not sure if that's related.
• I am using Ubuntu, and cloudvolume version 8.5.5.

Anyway, is there anything I'm obviously doing wrong? I've tried things like upload_raw for the skeleton, and changing the compress parameter in the CloudVolume object. I would appreciate any tips related to writing skeleton objects using CloudVolume.

[tathey1]

Thanks so much in advance for any help!!

[william-silversmith]

Thanks for writing in! This is 100% related to:

I noticed that neuroglancer displayed an error "Data type not supported by WebGL: UINT8" which is the data_type of "vertex_types" in the skeleton info file, not sure if that's related.

I should probably remove that as a default option. Neuroglancer can't handle integer attributes at this time. @jbms do you know if it would be easy to add integer support for Skeleton attributes?

In the mean time, do the following:

for skel in skeletons:
    skel.extra_attributes = [ attr for attr in skel.extra_attributes if attr['data_type'] == 'float32' ]

That will prevent Neuroglancer from encountering indigestible attribute types. The "radius" attribute will still be present and decodable.

[tathey1]

Thanks so much!

Oddly, those attributes seem to reappear sometime during skeleton.upload i.e. after my original codeblock above I run:

skel = Skeleton(segid = 1, vertices=[[5,5,0], [5,5,9]], edges=[[0,1]])
print(f"Original attributes: {skel.extra_attributes}")
skeletons = [skel]
for skel in skeletons:
    skel.extra_attributes = [ attr for attr in skel.extra_attributes if attr['data_type'] == 'float32' ]
print(f"After removing: {skel.extra_attributes}")
vol.skeleton.upload(skel)
print(f"After uploading: {vol.skeleton.get(1).extra_attributes}")

and this gives the output:

Original attributes: [{'id': 'radius', 'data_type': 'float32', 'num_components': 1}, {'id': 'vertex_types', 'data_type': 'uint8', 'num_components': 1}]
After removing: [{'id': 'radius', 'data_type': 'float32', 'num_components': 1}]
After uploading: [{'id': 'radius', 'data_type': 'float32', 'num_components': 1}, {'id': 'vertex_types', 'data_type': 'uint8', 'num_components': 1}]

I'll look into why this might be tomorrow, but let me know if you have any ideas.

[jbms]

Yes, it would be fairly easy to add support for integer attributes.

[william-silversmith]

Thanks Jeremy, I'll have to look into it more closely!

[tathey1]

@william-silversmith I noticed that when committing the skeleton info file for the first time, it appears that the default skeleton info file includes radius and vertex_type attributes (here). Sounds like the vertex_type attribute is what is incompatible with neuroglancer. Indeed, when I edit the skeleton info file in a text editor and remove this attribute (along with your suggestion of removing it from the skeleton object), then things work (screenshot attached).

So, sounds like the default skeleton metadata is incompatible with neuroglancer, and looks like you think the best way forward is to change neuroglancer to accept integers. Is there anything I can do to help?

[william-silversmith]

I made a first attempt at adding integer support to neuroglancer here but haven't tested it yet: https://github.com/william-silversmith/neuroglancer/tree/wms_int_skel