Code in datasets.py

[ling-cai]

in datasets.py 'while id_timeline < len(self.events) and self.events[id_timeline][0] <= eval_events[id_event][3]:' should it be 'eval_events[id_event][0]' instead of 'eval_events[id_event][3]'?

[ling-cai]

Yes. i guess this is to filter out some positive predictions to get filtered evaluation metrics. However,
self.events[id_timeline][0] --> this is the date (start/end) associated with the event id 'id_timeline'.
but I checked on the eval_events[id_event][3],which is the object id. So i am confused. Not sure whether this is a bug or I misunderstood something.

[timlacroix]

Actually you're right.
let me fix this and check what happens

[ling-cai]

yes. i agree that it would not make a big difference to the results. Just wanted to make sure i understood it correctly. thanks for your quick response.

[timlacroix]

So, this would only wikidata results, since this eval is not used for ICEWS.
Evaluating on wikidata would take more time. I will try to see if the results change drastically.
It is very possible that there is very little to filter on these datasets anyway so this fix wouldn't change the results significantly.

[timlacroix]

yeah, sorry for the confusion in my answers, it's been a while since I've looked at the code. Thanks for spotting the mistake !