Merge pull request #40 from lsst/tickets/DM-41530

DM-41530: Add individual flag fields to APDB schema

README.rst

@@ -31,20 +31,21 @@ Adding a new schema
 Steps to update the alert schema (for example, when the APDB schema is updated).
 
 * Decide what the new schema version will be, following the guidelines given in `DMTN-093 <https://dmtn-093.lsst.io/#management-and-evolution>`_, referring to the current version number directories in ``python/lsst/alert/packet/schema``.
-* ``setup -r .`` in this package's root.
 * Checkout the ticket branch for your schema changes.
 * Update the default ``schema_root`` kwarg in ``python/lsst/alert/packet/schemaRegistry.py:from_filesystem()`` to your new schema version number.
 
-* New schemas are built from the apdb. Any changes to a field should be done in sdm_schemas/yml/apdb.yaml, and any changes to what is included/excluded should be made in updateSchema.py.
+* New schemas are built from the apdb. Any changes to a field should be done in ``sdm_schemas/yml/apdb.yaml``, and any changes to what is included/excluded should be made in ``updateSchema.py``.
 * To update the schema, you must have the path to the apdb.yaml file and have chosen a version number. If the directory for your version does not already exist, ``updateSchema.py`` will create it.
 
-   * run ``python updateSchema.py /path/to/LSST/code/sdm_schemas/yml/apdb.yaml Path/To/alert_packet/lsst/alert/packet/schema "6.0"`` All Generated files do not need to be altered.
+    * set up ``sdm_schemas`` and ``cd python/lsst/alert/packet/``.
+    * run ``python updateSchema.py $SDM_SCHEMAS_DIR/yml/apdb.yaml ./schema "6.0"`` All Generated files do not need to be altered.
     * Navigate to the new schema. Copy in the previous ``lsst.vX_X.alert.avsc`` file and ``lsst.vX_X.diaNondetectionLimit.avsc``.
     * Within the two copied files, update ``"namespace": "lsst.vX_X",`` line at the top of each ``*.avsc`` file to the new version.
     * Update the contents of those avro schema files to reflect the new schema.
 
 * Update the sample alert packet in ``sample_data``:
 
+    * Copy ``sample_data`` from previous schema into the new version's ``X_X`` directory.
     * Update ``alert.json`` to reflect the new schema.
     * Change the ``schema_root`` and ``get_by_version`` parameters in ``generate.py`` to your new version number.
     * Run ``python generate.py`` to produce a new ``fakeAlert.avro`` file with data filled in from the updated json file above and using the new schema files you made earlier.

python/lsst/alert/packet/schema/7/0/lsst.v7_0.alert.avsc

@@ -0,0 +1,24 @@
+{
+	"namespace": "lsst.v7_0",
+	"type": "record",
+	"name": "alert",
+	"doc": "Rubin Avro alert schema v7.0",
+	"fields": [
+		{"name": "alertId", "type": "long", "doc": "unique alert identifer"},
+		{"name": "diaSource", "type": "lsst.v7_0.diaSource"},
+		{"name": "prvDiaSources", "type": ["null", {
+				"type": "array",
+				"items": "lsst.v7_0.diaSource"}], "default": null},
+		{"name": "prvDiaForcedSources", "type": ["null", {
+				"type": "array",
+				"items": "lsst.v7_0.diaForcedSource"}], "default": null},
+		{"name": "prvDiaNondetectionLimits", "type": ["null", {
+				"type": "array",
+				"items": "lsst.v7_0.diaNondetectionLimit"}], "default": null},
+		{"name": "diaObject", "type": ["null", "lsst.v7_0.diaObject"], "default": null},
+		{"name": "ssObject", "type": ["null", "lsst.v7_0.ssObject"], "default": null},
+		{"name": "cutoutDifference", "type": ["null", "bytes"], "default": null},
+		{"name": "cutoutScience", "type": ["null", "bytes"], "default": null},
+		{"name": "cutoutTemplate", "type": ["null", "bytes"], "default": null}
+     ]
+}

python/lsst/alert/packet/schema/7/0/lsst.v7_0.diaForcedSource.avsc

@@ -0,0 +1,77 @@
+{
+  "type": "record",
+  "namespace": "lsst.v7_0",
+  "name": "diaForcedSource",
+  "fields": [
+    {
+      "doc": "Unique id.",
+      "name": "diaForcedSourceId",
+      "type": "long"
+    },
+    {
+      "doc": "Id of the DiaObject that this DiaForcedSource was associated with.",
+      "name": "diaObjectId",
+      "type": "long"
+    },
+    {
+      "doc": "Id of the visit where this forcedSource was measured.",
+      "name": "visit",
+      "type": "long"
+    },
+    {
+      "doc": "Id of the detector where this forcedSource was measured.",
+      "name": "detector",
+      "type": "int"
+    },
+    {
+      "default": null,
+      "doc": "Point Source model flux.",
+      "name": "psfFlux",
+      "type": [
+        "null",
+        "float"
+      ]
+    },
+    {
+      "default": null,
+      "doc": "Uncertainty of psfFlux.",
+      "name": "psfFluxErr",
+      "type": [
+        "null",
+        "float"
+      ]
+    },
+    {
+      "default": null,
+      "doc": "x position at which psfFlux has been measured.",
+      "name": "x",
+      "type": [
+        "null",
+        "float"
+      ]
+    },
+    {
+      "default": null,
+      "doc": "y position at which psfFlux has been measured.",
+      "name": "y",
+      "type": [
+        "null",
+        "float"
+      ]
+    },
+    {
+      "doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.",
+      "name": "midpointMjdTai",
+      "type": "double"
+    },
+    {
+      "default": null,
+      "doc": "Filter band this source was observed with.",
+      "name": "band",
+      "type": [
+        "null",
+        "string"
+      ]
+    }
+  ]
+}

python/lsst/alert/packet/schema/7/0/lsst.v7_0.diaNondetectionLimit.avsc

@@ -0,0 +1,11 @@
+{
+	"namespace": "lsst.v7_0",
+	"name": "diaNondetectionLimit",
+	"type": "record",
+	"fields": [
+		{"name": "ccdVisitId",  "type": "long"},
+		{"name": "midpointMjdTai", "type": "double"},
+		{"name": "band", "type": "string"},
+		{"name": "diaNoise", "type": "float"}
+	]
+}