Unable to read Branches

[DebajyotiS]

I have a root file with the following structure (image attached)

I load the root file with uproot, in order to access the "m_event_number", however when I try file["CollectionTree"]["Eventnfo_p4_McEventInfo"]["m_event_ID"] - I get keyInFileError - Available keys: None.

Am I doing something wrong?

[tamasgal]

Do you have the possibility to upload the file? That would make thing a bit easier.

Meanwhile: what is the output of:

file["CollectionTree"]["Eventnfo_p4_McEventInfo"].show()

[DebajyotiS]

I will check if I have the rights to share the file, until then I can share the result for this

[tamasgal]

OK, it seems you have a custom streamer there. Do you have any luck with file["CollectionTree"]["Eventnfo_p4_McEventInfo"].array()?

I fear you need to provide the interpretations on your own, but let's see 🤔

[tamasgal]

Yes, this means that you need to tell uproot how a specific class is serialised into the ROOT format. This can be quite complicated as you can imagine since you need to provide the field types and the types/structure of sub-structures etc. uproot is usually able to figure out a lot but sometimes it chokes on structures which still needs to be understood and implemented. For this, a sample file would be very helpful to see where it fails (obviously the TriggerInfo stuff is one of them) and maybe is an easy fix but it can also be much more difficult, especially without access to the C++ code or an actual dump of the original data to be able to compare.

[jpivarski]

Actually, "cannot be Awkward" is weaker than "cannot be read." If the error message doesn't already say it, it should say "try library="np". Very complex objects, such as histograms, can't be represented as Awkward Arrays, but they can be read into Python objects, and library="np" makes NumPy arrays of Python objects.

According to .show(), it looks like Uproot identified an interpretation for this type. (We don't know, until you successfully read it, that the interpretation is correct.) So start with library="np" because there's a good chance that the streamers that Uproot found and interpreted might be correct.

[tamasgal]

Ah interesting, I always thought that "cannot be awkward" is kind of a dead-end, since my gut feeling is that if something cannot be awkward, it cannot be numpy either (like numpy being a subset of awkard), but indeed I completely forgot that numpy is able to hold Python objects 😉

Edit: that being said, yes, I think there should be a hint to try library="np" and additionally maybe a warning that the performance might not be optimal.

[DebajyotiS]

with HITS_1["CollectionTree"]["EventInfo_p4_McEventInfo"].array(library="np"), I get a deserialisation error (looks like there is a version clash?):

[jpivarski]

It wasn't interpreted correctly. From this error message, it looks like the interpretation thought there would be a header indicating the number of bytes in EventType_p3 (version 1), but that header doesn't exist and it's picking something else up as the number of bytes, thereby thinking that there ought to be 391380996 bytes in this object, which is contained within EventInfo_p4 (version 1), which has 54 bytes. Very likely, only the 54 bytes is correct. The 4-byte number "391380996" is [23, 84, 0, 4] as single bytes, and that [0, 4] looks a lot like a version number.

More fundamentally, I think the main error is, as you've pointed out, related to the "version 0" versus "version 1." These are version numbers of the classes, not ROOT. If Uproot encounters an object with a specific version number but doesn't find a class with that version number in the ROOT file's streamers, then it takes the highest version and hopes that's correct. It's not correct in this case. Sometimes, the class streamers that are absolutely required for knowing how to deserialize an object just aren't in the ROOT streamers, and then we can't do anything. This might be such a case.

If it's possible to recreate these files with a high "split-level," like splitLevel=99, doing so would alleviate all of these problems. This m_event_number is just a bunch of integers, the m_bit_mask is just std::vector<bool>, etc. With a high split-level, they would each be separate TBranches, separate arrays, and not only avoid the difficulties of deserializing complex objects, but would also be much faster. (This deserialization code is Python for loops—interpreting an array of integers or std::vector<bool> is NumPy.)