fix: mark signal v0.2.4 as failed (#831)

* fix: mark signal v0.2.4 as failed

The ooni/probe-cli#1421 PR trimmed the endpoints and bumped signal's version to v0.2.5.

So we need to ignore versions of signal lower than v0.2.5.

I am wondering whether we should also use a time window because otherwise what happens when we reprocess measurements. If my analysis is correct, then we have an additional issue that there's another check in this codepath that seems to mark signal measurements as failed without any temporal constraints. This would also cause reprocessing to cause downstream issues.

Part of ooni/probe#2636

* Add more specific constraints to the signal scoring logic

* Bump changelog entry

Fix changelog entry to avoid conflict with: https://github.com/ooni/backend/pull/793/files

---------

Co-authored-by: Arturo Filastò <[email protected]>

fastpath/debian/changelog

@@ -1,3 +1,10 @@
+fastpath (0.86) unstable; urgency=medium
+
+  * Mark signal measurements as failed
+  * Disable torsf test via check-in
+
+ -- Arturo Filastò <[email protected]>  Mon, 25 Mar 2024 12:14:54 +0100
+
 fastpath (0.84) unstable; urgency=medium
 
   * Support ooni_run_link_id

fastpath/fastpath/core.py

@@ -1353,32 +1353,46 @@ def score_signal(msm: dict) -> dict:
     try:
         # https://github.com/ooni/probe/issues/2344
         tv = g_or(msm, "test_version", "0.0.0")
-        if parse_version(tv) <= parse_version("0.2.3"):
+
+        msmt_start_time = msm.get("measurement_start_time")
+        if msmt_start_time is None:
+            scores["accuracy"] = 0.0
+            return scores
+        start_time = datetime.strptime(msmt_start_time, "%Y-%m-%d %H:%M:%S")
+
+        if tv == "0.2.4":
+            # the breakage here is affecting just v0.2.4 of the test
+            # See https://github.com/ooni/probe/issues/2636
+            # See https://github.com/ooni/probe-cli/pull/1421
+            scores["accuracy"] = 0.0
+            return scores
+
+        if parse_version(tv) <= parse_version("0.2.3") and start_time >= datetime(
+            2023, 11, 7
+        ):
             # https://github.com/ooni/probe/issues/2627
             scores["accuracy"] = 0.0
             return scores
 
-        if parse_version(tv) < parse_version("0.2.2"):
-            start_time = msm.get("measurement_start_time")
-            if start_time is None:
-                scores["accuracy"] = 0.0
-            else:
-                start_time = datetime.strptime(start_time, "%Y-%m-%d %H:%M:%S")
-                if start_time >= datetime(2022, 10, 19):
-                    scores["accuracy"] = 0.0
+        if parse_version(tv) < parse_version("0.2.2") and start_time >= datetime(
+            2022, 10, 19
+        ):
+            scores["accuracy"] = 0.0
+            return scores
 
         # https://github.com/ooni/backend/issues/679
         # engine_version < 3.17.2 and measurement_start_time > 2023-05-02
         annot = g_or(msm, "annotations", {})
         ev = g_or(annot, "engine_version", "0.0.0")
-        if parse_version(ev) < parse_version("3.17.2"):
-            st = g_or(msm, "measurement_start_time", "2023-05-05 00:00:00")
-            start_time = datetime.strptime(st, "%Y-%m-%d %H:%M:%S")
-            if start_time >= datetime(2023, 5, 2):
-                scores["accuracy"] = 0.0
+        if parse_version(ev) < parse_version("3.17.2") and start_time >= datetime(
+            2023, 5, 2
+        ):
+            scores["accuracy"] = 0.0
+            return scores
 
     except Exception:
         scores["accuracy"] = 0.0
+        return scores
 
     st = tk.get("signal_backend_status")
     if st == "ok":