Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feedback_reason column to the notification table #2443

Merged
merged 4 commits into from
Feb 10, 2025
Merged

Add feedback_reason column to the notification table #2443

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4df97f6
Add feedback_reason column to the notification table
whabanks Feb 5, 2025
b9f45a2
Fix tests
whabanks Feb 5, 2025
9fde272
Merge branch 'main' into task/add-feedback-reason-column
whabanks Feb 6, 2025
1c34dfa
Merge branch 'main' into task/add-feedback-reason-column
whabanks Feb 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions app/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1548,6 +1548,7 @@ def check_code(self, cde):
NOTIFICATION_TECHNICAL_FAILURE = "technical-failure"
NOTIFICATION_TEMPORARY_FAILURE = "temporary-failure"
NOTIFICATION_PERMANENT_FAILURE = "permanent-failure"
NOTIFICATION_PINPOINT_FAILURE = "pinpoint-failure"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's name the status to untie from a specific technology. We could go back to AWS SES or we could support Twilio as well and other providers. Is there a way to name this without depending on the implementation (and extend the model on an abstract name)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently all the failures are SES related. This is now Piinpoint related and we wanted to store that and make it obv. I think as we grow and add more providers/ clients/ - we should add failures according to those implementations so we might know exactly where the failure came from and not have to traceback through logs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with not tying statuses to functionality or vendor specific terminology. That said, properly achieving vendor agnostic terminology for statuses, while still improving observability would take some effort.

Since we're handling a conflict between a Request - send to intl. #, and the configuration of the target provider resource - pinpoint not configured for intl. sending, we could use something more generic like:

  • PROVIDER_FAILURE
  • PROVIDER_CONFLICT_FAILURE
  • PROVIDER_VALIDATION_FAILURE

Then capture the vendor specific info + failure reason in the feedback_reason. To Jumana's point - the current solution does provide added observability while requiring minimal changes to existing infrastructure for the time being.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey I missed these replies. I am not big on tying our implementations with vendor specific names, especially when changing that to be generic would have been minimal effort IMO. We are starting a technical debt here that could have been avoided easily. This breaks abstractions and good design in general. I would bring this to the code review sessions.

NOTIFICATION_PENDING_VIRUS_CHECK = "pending-virus-check"
NOTIFICATION_VALIDATION_FAILED = "validation-failed"
NOTIFICATION_VIRUS_SCAN_FAILED = "virus-scan-failed"
Expand All @@ -1558,6 +1559,7 @@ def check_code(self, cde):
NOTIFICATION_TECHNICAL_FAILURE,
NOTIFICATION_TEMPORARY_FAILURE,
NOTIFICATION_PERMANENT_FAILURE,
NOTIFICATION_PINPOINT_FAILURE,
NOTIFICATION_VALIDATION_FAILED,
NOTIFICATION_VIRUS_SCAN_FAILED,
NOTIFICATION_RETURNED_LETTER,
Expand Down Expand Up @@ -1605,6 +1607,7 @@ def check_code(self, cde):
NOTIFICATION_TECHNICAL_FAILURE,
NOTIFICATION_TEMPORARY_FAILURE,
NOTIFICATION_PERMANENT_FAILURE,
NOTIFICATION_PINPOINT_FAILURE,
NOTIFICATION_PENDING_VIRUS_CHECK,
NOTIFICATION_VALIDATION_FAILED,
NOTIFICATION_VIRUS_SCAN_FAILED,
Expand Down Expand Up @@ -1739,6 +1742,7 @@ class Notification(BaseModel):
feedback_subtype = db.Column(db.String, nullable=True)
ses_feedback_id = db.Column(db.String, nullable=True)
ses_feedback_date = db.Column(db.DateTime, nullable=True)
feedback_reason = db.Column(db.String, nullable=True)

# SMS columns
sms_total_message_price = db.Column(db.Numeric(), nullable=True)
Expand Down
1 change: 1 addition & 0 deletions app/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"temporary-failure",
"permanent-failure",
"technical-failure",
"pinpoint-failure",
"virus-scan-failed",
"validation-failed",
]
Expand Down
41 changes: 41 additions & 0 deletions migrations/versions/0474_add_feedback_reason_column.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""

Revision ID: 0474_add_feedback_reason_column
Revises: 0473_change_pt_support_email
Create Date: 2025-02-05 14:40:00 EST

"""

import sqlalchemy as sa
from alembic import op

revision = "0474_add_feedback_reason_column"
down_revision = "0473_change_pt_support_email"

new_notification_status = "pinpoint-failure"


def upgrade():
op.add_column("notifications", sa.Column("feedback_reason", sa.String(length=255), nullable=True))
op.add_column("notification_history", sa.Column("feedback_reason", sa.String(length=255), nullable=True))

op.create_index(
op.f("ix_notifications_feedback_reason"),
"notifications",
["feedback_reason"],
)
op.create_index(
op.f("ix_notification_history_feedback_reason"),
"notification_history",
["feedback_reason"],
)

op.execute("INSERT INTO notification_status_types (name) VALUES ('{}')".format(new_notification_status))


def downgrade():
op.drop_index(op.f("ix_notifications_feedback_reason"), table_name="notifications")
op.drop_index(op.f("ix_notification_history_feedback_reason"), table_name="notification_history")
op.drop_column("notifications", "feedback_reason")
op.drop_column("notification_history", "feedback_reason")
op.execute("DELETE FROM notification_status_types WHERE name = '{}'".format(new_notification_status))
2 changes: 1 addition & 1 deletion tests/app/v2/notifications/test_get_notifications.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,7 @@ def test_get_all_notifications_filter_by_status_invalid_status(client, sample_no
assert len(json_response["errors"]) == 1
assert (
json_response["errors"][0]["message"] == "status elephant is not one of [cancelled, created, sending, "
"sent, delivered, pending, failed, technical-failure, temporary-failure, permanent-failure, "
"sent, delivered, pending, failed, technical-failure, temporary-failure, permanent-failure, pinpoint-failure, "
"pending-virus-check, validation-failed, virus-scan-failed, returned-letter, "
"pii-check-failed, accepted, received]"
)
Expand Down
4 changes: 2 additions & 2 deletions tests/app/v2/notifications/test_notification_schemas.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def test_get_notifications_request_invalid_statuses(invalid_statuses, valid_stat
partial_error_status = (
"is not one of "
"[cancelled, created, sending, sent, delivered, pending, failed, "
"technical-failure, temporary-failure, permanent-failure, pending-virus-check, "
"technical-failure, temporary-failure, permanent-failure, pinpoint-failure, pending-virus-check, "
"validation-failed, virus-scan-failed, returned-letter, pii-check-failed, accepted, received]"
)

Expand Down Expand Up @@ -103,7 +103,7 @@ def test_get_notifications_request_invalid_statuses_and_template_types():
for invalid_status in ["elephant", "giraffe"]:
assert (
"status {} is not one of [cancelled, created, sending, sent, delivered, "
"pending, failed, technical-failure, temporary-failure, permanent-failure, "
"pending, failed, technical-failure, temporary-failure, permanent-failure, pinpoint-failure, "
"pending-virus-check, validation-failed, virus-scan-failed, returned-letter, "
"pii-check-failed, accepted, received]".format(invalid_status) in error_messages
)
Expand Down
Loading