-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error during Inference. ValueError: Found Multiple nodes named (name). #1918
Comments
Hi @hemanyaradadia, We have run into this before, and the issue seems to stem from the "py/state": {"name": "nose", "weight": 1.0} when it should contain "py/state": {"py/tuple": ["nose", 1.0]} . While I can (and will shortly) write a file converter to reformat the file into the expected format, this still does not explain the root cause of the behavior. We use the
I am testing this on my computer as well and will report back, but would you be able to share the environment information for the Windows computer that does not have this Thanks, Related |
TLDRThe change in Action Plan (developer)The underlying theme in these discussions is that "pickling across versions" is an "antipattern" and we should perhaps find a different way to serialize our Action Plan (user)Report to us that you are having this problem (including your environment info if possible), until a new release is published, use this script to convert incompatible skeleton.json files (that use a dictionary for the Journaling the possibility that something might be happening inside
|
# Encode to JSON | |
graph = json_graph.node_link_data(indexed_node_graph) | |
# SLEAP v1.3.0 added `description` and `preview_image` to `Skeleton`, but saving | |
# these fields breaks data format compatibility. Currently, these are only | |
# added in our custom template skeletons. To ensure backwards data format | |
# compatibilty of user data, we only save these fields if they are not None. | |
if self.is_template: | |
data = { | |
"nx_graph": graph, | |
"description": self.description, | |
"preview_image": self.preview_image, | |
} | |
else: | |
data = graph | |
json_str = jsonpickle.encode(data) |
that looks something like this:
{'directed': True, 'multigraph': True, 'graph': {'name': 'Fly', 'num_edges_inserted': 4}, 'nodes': [{...}, {...}, {...}, {...}, {...}], 'links': [{...}, {...}, {...}, {...}, {...}, {...}]}
. Thus, jsonpickle's Pickler._get_flattener
returns Pickler._flatten_dict_obj
.
Then, jsonpickle calls Pickler._flatten_key_value_pair
on each key-value pair in the dictionary, which basically finds a way to flatten both the key and the value.
- If the
obj
is a primitive type, then the value is returned as is. - If the
obj
is a dictionary, then the we again callPickler._flatten_dic_obj
. - If the
obj
is an "object" (i.e. an instance of a SLEAP class, e.g. aNode
), then we callPickler._ref_obj_instance
.- if the object has not been seen before, it is registered in
Pickler._objs
using theid(obj)
as the key andobj
as the value - then
Pickler._flatten_obj_instance
is called which- reads a few dunder attributes, finds the class (or type) of the object and
- if
Pickler.unpickable
(as in the case of aNode
)- sets
data = {'py/object': 'sleap.skeleton.Node'}
, - reads the attributes of the
obj
withobj.__getstate__()
to return("head", 1.0)
, - serializes the tuple state to
value = {'py/tuple': ['head', 1.0]}
, and - appends the state to the
data = {'py/object': 'sleap.skeleton.Node', 'py/state': {'py/tuple': ['head', 1.0]}}
.
- sets
- if the object has not been seen before, it is registered in
Thus, since the formatting of the "py/state"
is the problem, we need to pay attention to any changes to either jsonpickle's Pickler._getstate
method (which also calls Pickler._flatten
) or the code block within Pickler._flatten_obj_instance
that calls Pickler._getstate
. It could very likely be a change to attrs
which creates a custom __getstate__
method (which returns a tuple for attrs ==21.4.0
). We expect that the error occurs when the state is read in as a dict
instead of a tuple
.
Now pointing fingers at attr
In #1009 of attrs, I found that they changed the nested slots_getstate
function (inside the _ClassBuilder._make_getstate_setstate
method) to return a dictionary instead of a tuple as reportedly returned in attrs ==21.4.0
.
In the aforementioned attrs PR, one user commented:
I feel like this may be a breaking change. Do objects serialized with the previous version can be deserialized with the new version?
to which an/the attrs' maintainer replied
They cannot. A fix has been added in #1085 that will be in 23.1, but (de)serializing classes across versions is risky (in general, not just the attrs case) so we'd encourage to look into avoiding that.
Hi @roomrys, Thanks for the reply. The skeleton file I am using was created on the mentioned Windows 10 computer. I had installed SLEAP using conda from the package. conda env info for the windows computer
Interestingly, this environment has the same version of |
Hi @hemanyaradadia, This is a file format compatibility issue that will be officially addressed in our future release (by constraining the version of A (temporary) peace offeringApologies you have run into this, but thank you for reporting! Until this issue is officially addressed in a release, you can run this script to convert the incompatible skeleton.json that uses a To run the script:
Results: Converting your skeletonI set your current skeleton.json as the {"directed": true, "graph": {"name": "Skeleton-3", "num_edges_inserted": 5}, "links": [{"edge_insert_idx": 0, "key": 0, "source": {"py/object": "sleap.skeleton.Node", "py/state": {"name": "nose", "weight": 1.0}}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"name": "right-ear", "weight": 1.0}}, "type": {"py/reduce": [{"py/type": "sleap.skeleton.EdgeType"}, {"py/tuple": [1]}]}}, {"edge_insert_idx": 1, "key": 0, "source": {"py/id": 1}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"name": "left-ear", "weight": 1.0}}, "type": {"py/id": 3}}, {"edge_insert_idx": 2, "key": 0, "source": {"py/id": 2}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"name": "thorax", "weight": 1.0}}, "type": {"py/id": 3}}, {"edge_insert_idx": 3, "key": 0, "source": {"py/id": 4}, "target": {"py/id": 5}, "type": {"py/id": 3}}, {"edge_insert_idx": 4, "key": 0, "source": {"py/id": 5}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"name": "tail-base", "weight": 1.0}}, "type": {"py/id": 3}}], "multigraph": true, "nodes": [{"id": {"py/id": 1}}, {"id": {"py/id": 2}}, {"id": {"py/id": 4}}, {"id": {"py/id": 5}}, {"id": {"py/id": 6}}]} and this is the output skeleton.json that was saved as {"directed": true, "graph": {"name": "Skeleton-3", "num_edges_inserted": 5}, "links": [{"edge_insert_idx": 0, "key": 0, "source": {"py/object": "sleap.skeleton.Node", "py/state": {"py/tuple": ["nose", 1.0]}}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"py/tuple": ["right-ear", 1.0]}}, "type": {"py/reduce": [{"py/type": "sleap.skeleton.EdgeType"}, {"py/tuple": [1]}]}}, {"edge_insert_idx": 1, "key": 0, "source": {"py/id": 1}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"py/tuple": ["left-ear", 1.0]}}, "type": {"py/id": 3}}, {"edge_insert_idx": 2, "key": 0, "source": {"py/id": 2}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"py/tuple": ["thorax", 1.0]}}, "type": {"py/id": 3}}, {"edge_insert_idx": 3, "key": 0, "source": {"py/id": 4}, "target": {"py/id": 5}, "type": {"py/id": 3}}, {"edge_insert_idx": 4, "key": 0, "source": {"py/id": 5}, "target": {"py/object": "sleap.skeleton.Node", "py/state": {"py/tuple": ["tail-base", 1.0]}}, "type": {"py/id": 3}}], "multigraph": true, "nodes": [{"id": {"py/id": 1}}, {"id": {"py/id": 2}}, {"id": {"py/id": 4}}, {"id": {"py/id": 5}}, {"id": {"py/id": 6}}]} (notice the differences in the Then when loading each skeleton.json as a In [1]: from sleap import Skeleton
In [2]: input_json = "dict_state.json"
In [3]: output_json = "tuple_state.json"
In [4]: skeleton_input = Skeleton.load_json(input_json)
In [5]: skeleton_input
Out[5]: Skeleton(name='Skeleton-3', description='None', nodes=['name', 'name', 'name', 'name', 'name'], edges=[('name', 'name'), ('name', 'name'), ('name', 'name'), ('name', 'name'), ('name', 'name')],
symmetries=[])
In [6]: skeleton_output = Skeleton.load_json(output_json)
In [7]: skeleton_output
Out[7]: Skeleton(name='Skeleton-3', description='None', nodes=['nose', 'right-ear', 'left-ear', 'thorax', 'tail-base'], edges=[('nose', 'right-ear'), ('nose', 'left-ear'), ('right-ear', 'thorax'), ('left-ear', 'thorax'), ('thorax', 'tail-base')], symmetries=[]) Let us know if you run into further trouble. Thanks, |
Yes, I also originally thought it might be a I see that your Although, this is still strange because, while we do not have a pin on the Line 58 in 864c994
I am wondering if perhaps the Windows computer might have a "conda from source" installation which uses the development branch in which Again, this is our fault as we list dependency constraints, I am working on building a 1.3.4 mac conda packages as those have no upper bound on the Thanks, |
Hi @roomrys, Thanks for the script to convert the skeleton |
We have released SLEAP 1.3.4 (off #1927) which adds additional dependency constraints to prevent this from happening, but we are also switching to manually handling the I will leave this issue open until our next release (where we include a conversion script as well). |
Bug description
Running Inference and Tracking throws up 'ValueError: Found multiple nodes named (name).' on a Linux system. While running the same command on a Windows 10 system, it works without any error.
Expected behaviour
A .slp output file with predictions/track in each frame.
Actual behaviour
The error shows up in the terminal. While .slp file is generated but without any predictions or frames. Essentially an empty .slp project file.
Your personal set up
Environment packages
Logs
CLI command and resulting outputSkeleton .json file used
The text was updated successfully, but these errors were encountered: