feat(webhook): Add is_delivered filter to list initial attempts endpoint #7344

AmeyWale · 2025-02-21T13:09:41Z

Type of Change

Description

This PR features addition of new filter called is_delivered which will allow merchants to filter out webhook events based on the fact that if a particular event was ultimately delivered or not.

The is_delivery_successful field now takes data from is_overall_delivery_successful column of the events table and not is_webhook_notified to represent the eventual delivery of the event.

Additional Changes

This PR modifies the API contract
This PR modifies the database schema
This PR modifies application configuration/environment variables

Motivation and Context

Upon merging this PR, the merchants would be able to filter webhook events based on ultimate delivery status.

How did you test it?

API request to fetch webhook events with no filter

curl --location 'http://localhost:8080/events/merchant_1740390998' \
--header 'Content-Type: application/json' \
--header 'admin_api_key: test_admin' \
--header 'api-key: test_admin' \
--data ''

Response from the above call

{"events":[{"event_id":"evt_01953774e7eb7c9190e280ea6756d032","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_QlQAl1DOJug8H0kk9FtW","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774e7eb7c9190e280ea6756d032","created":"2025-02-24T10:16:03.308Z"},{"event_id":"evt_01953774d6b57762a2ff4fd96e27cce3","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_lKLVcse8HzUlytYrQtKJ","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774d6b57762a2ff4fd96e27cce3","created":"2025-02-24T10:15:58.901Z"},{"event_id":"evt_01953774c1147c41a855a13536beafd7","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_TpPAwHW9pb1VtDoQR1fn","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774c1147c41a855a13536beafd7","created":"2025-02-24T10:15:53.365Z"},{"event_id":"evt_019537748a317841814d292122fe384b","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_Wfg8KwLIKpNukRyI7LGN","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537748a317841814d292122fe384b","created":"2025-02-24T10:15:39.313Z"},{"event_id":"evt_019537728f3b7000b60bbf18cb8b1373","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_ISS739fX9arr9CNf6w73","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537728f3b7000b60bbf18cb8b1373","created":"2025-02-24T10:13:29.532Z"},{"event_id":"evt_01953771f29d7911be93a3753f1eb0c4","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_PEAL9Kt0lSt3tllLtSD6","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_01953771f29d7911be93a3753f1eb0c4","created":"2025-02-24T10:12:49.438Z"},{"event_id":"evt_019537707b8b70838e753daf51eb1d67","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_hXCyBkIAVf00lf03jyMQ","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537707b8b70838e753daf51eb1d67","created":"2025-02-24T10:11:13.420Z"},{"event_id":"evt_0195376867a57282999dd9ac75b137f9","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_g2TOloPWFFrhm02n9kuv","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376867a57282999dd9ac75b137f9","created":"2025-02-24T10:02:24.038Z"},{"event_id":"evt_0195376750287e53bb08204068a6a902","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_47LGuikk3tqhnHNxrQfL","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376750287e53bb08204068a6a902","created":"2025-02-24T10:01:12.489Z"},{"event_id":"evt_01953764d84a7811b03e83c906fd8009","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_d4MTTk0iuQzjQcChWHsa","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_01953764d84a7811b03e83c906fd8009","created":"2025-02-24T09:58:30.731Z"},{"event_id":"evt_0195376421b477909d06c2b227c734f7","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_9yrvfEwJyUYS1BScPn8Y","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376421b477909d06c2b227c734f7","created":"2025-02-24T09:57:43.988Z"},{"event_id":"evt_0195376386e97a93a6b464285a262d8f","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_43vhcdhitA2NdA7cALAx","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376386e97a93a6b464285a262d8f","created":"2025-02-24T09:57:04.361Z"}],"total_count":12}

API call to fetch webhook events with is_delivered having false value.

curl --location 'http://localhost:8080/events/merchant_1740390998?is_delivered=false' \
--header 'Content-Type: application/json' \
--header 'admin_api_key: test_admin' \
--header 'api-key: test_admin' \
--data ''

Response from above call

{"events":[{"event_id":"evt_019537748a317841814d292122fe384b","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_Wfg8KwLIKpNukRyI7LGN","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537748a317841814d292122fe384b","created":"2025-02-24T10:15:39.313Z"},{"event_id":"evt_019537728f3b7000b60bbf18cb8b1373","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_ISS739fX9arr9CNf6w73","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537728f3b7000b60bbf18cb8b1373","created":"2025-02-24T10:13:29.532Z"},{"event_id":"evt_01953771f29d7911be93a3753f1eb0c4","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_PEAL9Kt0lSt3tllLtSD6","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_01953771f29d7911be93a3753f1eb0c4","created":"2025-02-24T10:12:49.438Z"},{"event_id":"evt_019537707b8b70838e753daf51eb1d67","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_hXCyBkIAVf00lf03jyMQ","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_019537707b8b70838e753daf51eb1d67","created":"2025-02-24T10:11:13.420Z"},{"event_id":"evt_0195376867a57282999dd9ac75b137f9","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_g2TOloPWFFrhm02n9kuv","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376867a57282999dd9ac75b137f9","created":"2025-02-24T10:02:24.038Z"},{"event_id":"evt_0195376750287e53bb08204068a6a902","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_47LGuikk3tqhnHNxrQfL","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376750287e53bb08204068a6a902","created":"2025-02-24T10:01:12.489Z"},{"event_id":"evt_01953764d84a7811b03e83c906fd8009","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_d4MTTk0iuQzjQcChWHsa","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_01953764d84a7811b03e83c906fd8009","created":"2025-02-24T09:58:30.731Z"},{"event_id":"evt_0195376421b477909d06c2b227c734f7","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_9yrvfEwJyUYS1BScPn8Y","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376421b477909d06c2b227c734f7","created":"2025-02-24T09:57:43.988Z"},{"event_id":"evt_0195376386e97a93a6b464285a262d8f","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_43vhcdhitA2NdA7cALAx","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":false,"initial_attempt_id":"evt_0195376386e97a93a6b464285a262d8f","created":"2025-02-24T09:57:04.361Z"}],"total_count":9}

API call to fetch webhook events with is_delivered filter having true value.

curl --location 'http://localhost:8080/events/merchant_1740390998?is_delivered=true' \
--header 'Content-Type: application/json' \
--header 'admin_api_key: test_admin' \
--header 'api-key: test_admin' \
--data ''

Response from above call

{"events":[{"event_id":"evt_01953774e7eb7c9190e280ea6756d032","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_QlQAl1DOJug8H0kk9FtW","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774e7eb7c9190e280ea6756d032","created":"2025-02-24T10:16:03.308Z"},{"event_id":"evt_01953774d6b57762a2ff4fd96e27cce3","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_lKLVcse8HzUlytYrQtKJ","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774d6b57762a2ff4fd96e27cce3","created":"2025-02-24T10:15:58.901Z"},{"event_id":"evt_01953774c1147c41a855a13536beafd7","merchant_id":"merchant_1740390998","profile_id":"pro_IzLLBXNTTbYTO6TehKsK","object_id":"pay_TpPAwHW9pb1VtDoQR1fn","event_type":"payment_succeeded","event_class":"payments","is_delivery_successful":true,"initial_attempt_id":"evt_01953774c1147c41a855a13536beafd7","created":"2025-02-24T10:15:53.365Z"}],"total_count":3}

Checklist

I formatted the code cargo +nightly fmt --all
I addressed lints thrown by cargo clippy
I reviewed the submitted code
I added unit tests for my changes where possible

semanticdiff-com · 2025-02-21T13:09:45Z

Review changes with

Changed Files

File	Status
api-reference/openapi_spec.json	23% smaller
crates/router/src/core/webhooks/outgoing.rs	8% smaller
crates/router/src/db/events.rs	1% smaller
api-reference-v2/openapi_spec.json	0% smaller
crates/api_models/src/webhook_events.rs	0% smaller
crates/diesel_models/src/events.rs	0% smaller
crates/diesel_models/src/query/events.rs	0% smaller
crates/diesel_models/src/schema.rs	0% smaller
crates/diesel_models/src/schema_v2.rs	0% smaller
crates/openapi/src/routes/webhook_events.rs	0% smaller
crates/router/src/core/webhooks/webhook_events.rs	0% smaller
crates/router/src/db/kafka_store.rs	0% smaller
crates/router/src/types/domain/event.rs	0% smaller
crates/router/src/types/transformers.rs	0% smaller
crates/router/src/workflows/outgoing_webhook_retry.rs	0% smaller
migrations/2025-02-19-072457_add_is_overall_delivery_successful_column_in_events_table/down.sql	Unsupported file format
migrations/2025-02-19-072457_add_is_overall_delivery_successful_column_in_events_table/up.sql	Unsupported file format

…hema

…nto is-delivered-filter

SanchithHegde · 2025-02-25T11:19:26Z

crates/api_models/src/webhook_events.rs

+    // Filter all events by overall_delivery_status.
+    pub is_delivered: Option<bool>,


Please make this a documentation comment instead.

And update the documentation comment to reflect the current field name, in case you think we actually need to reference the field name here.

SanchithHegde · 2025-02-25T11:19:39Z

crates/api_models/src/webhook_events.rs

@@ -68,7 +72,7 @@ pub struct EventListItemResponse {
    /// Specifies the class of event (the type of object: Payment, Refund, etc.)
    pub event_class: EventClass,

-    /// Indicates whether the webhook delivery attempt was successful.
+    /// Indicates whether the webhook was ultimately delivery.


Nit:

Suggested change

/// Indicates whether the webhook was ultimately delivery.

/// Indicates whether the webhook was ultimately delivered.

SanchithHegde · 2025-02-25T11:22:25Z

migrations/2025-02-19-072457_add_is_overall_delivery_successful_column_in_events_table/up.sql

@@ -0,0 +1,2 @@
+-- Your SQL goes here
+ALTER TABLE events ADD COLUMN IF NOT EXISTS is_overall_delivery_successful BOOLEAN NOT NULL DEFAULT FALSE;


Why is this column non-nullable? Why can't the old data have null values?

SanchithHegde · 2025-02-25T11:53:43Z

crates/router/src/core/webhooks/outgoing.rs

+    let parent_update = domain::EventUpdate::ParentUpdate {
+        is_overall_delivery_successful: status_code.is_success(),
+    };
+
+    let parent_event_id = current_event.initial_attempt_id.clone();
+    let delivery_attempt = current_event.delivery_attempt;
+
+    if let Some((
+        parent_event_id,
+        enums::WebhookDeliveryAttempt::InitialAttempt
+        | enums::WebhookDeliveryAttempt::AutomaticRetry,
+    )) = parent_event_id.zip(delivery_attempt)
+    {
+        state
+            .store
+            .update_event_by_merchant_id_event_id(
+                key_manager_state,
+                merchant_id,
+                parent_event_id.as_str(),
+                parent_update,
+                &merchant_key_store,
+            )
+            .await
+            .change_context(errors::WebhooksFlowError::WebhookEventUpdationFailed)
+            .attach_printable("Failed to update parent event")?;
+    }


Can we extract updating of the initial attempt to a different function, and call it in trigger_webhook_to_merchant()? We'd prefer to keep the update_event_in_storage() function responsible for updating only the current event that we're working with.

Ideally, we'd want to keep a function to have only one responsibility...

SanchithHegde · 2025-02-25T11:55:17Z

crates/router/src/core/webhooks/outgoing.rs

+    let parent_update = domain::EventUpdate::ParentUpdate {
+        is_overall_delivery_successful: status_code.is_success(),
+    };


Once we've extracted it to a separate function, let's name this enum variant to indicate which field is being updated, rather than which event is being updated: something along the lines of OverallDeliveryStatusUpdate.

SanchithHegde · 2025-02-25T11:57:49Z

crates/router/src/types/domain/event.rs

+    /// Reference to the object type for which the webhook was created.
    pub primary_object_type: EventObjectType,


Suggested change

/// Reference to the object type for which the webhook was created.

pub primary_object_type: EventObjectType,

/// Type of the object for which the webhook was created.

pub primary_object_type: EventObjectType,

SanchithHegde · 2025-02-25T12:14:52Z

crates/router/src/types/domain/event.rs

+    /// Merchant identifier to which the webhook was sent.
    pub merchant_id: Option<common_utils::id_type::MerchantId>,
+
+    /// Business Profile identifier to which the webhook was sent.
    pub business_profile_id: Option<common_utils::id_type::ProfileId>,


These actually are the identifiers of the merchant account and business profile the object is associated with.

feat(webhook): Add is_delivered filter to list initial attempts endpoint

041cf3a

AmeyWale requested review from a team as code owners February 21, 2025 13:09

AmeyWale changed the base branch from main to return-total-events-count February 21, 2025 13:26

AmeyWale self-assigned this Feb 21, 2025

hyperswitch-bot bot added the M-database-changes Metadata: This PR involves database schema changes label Feb 21, 2025

docs(openapi): re-generate OpenAPI specification

23e06a8

hyperswitch-bot bot added the M-api-contract-changes Metadata: This PR involves API contract changes label Feb 21, 2025

AmeyWale added 3 commits February 24, 2025 12:13

refactor(webhook): add is_overall_delivery_successful column in v2 sc…

579dd65

…hema

refactor(webhook): remove spaces in diesel migration folder name

d5ba8ff

Merge branch 'is-delivered-filter' of github.com:juspay/hyperswitch i…

13a666a

…nto is-delivered-filter

SanchithHegde reviewed Feb 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(webhook): Add is_delivered filter to list initial attempts endpoint #7344

feat(webhook): Add is_delivered filter to list initial attempts endpoint #7344

AmeyWale commented Feb 21, 2025 •

edited

Loading

semanticdiff-com bot commented Feb 21, 2025 •

edited

Loading

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

SanchithHegde Feb 25, 2025

		// Filter all events by overall_delivery_status.
		pub is_delivered: Option<bool>,

	/// Indicates whether the webhook was ultimately delivery.
	/// Indicates whether the webhook was ultimately delivered.

		@@ -0,0 +1,2 @@
		-- Your SQL goes here
		ALTER TABLE events ADD COLUMN IF NOT EXISTS is_overall_delivery_successful BOOLEAN NOT NULL DEFAULT FALSE;

		/// Reference to the object type for which the webhook was created.
		pub primary_object_type: EventObjectType,

feat(webhook): Add is_delivered filter to list initial attempts endpoint #7344

Are you sure you want to change the base?

feat(webhook): Add is_delivered filter to list initial attempts endpoint #7344

Conversation

AmeyWale commented Feb 21, 2025 • edited Loading

Type of Change

Description

Additional Changes

Motivation and Context

How did you test it?

Checklist

semanticdiff-com bot commented Feb 21, 2025 • edited Loading

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

SanchithHegde Feb 25, 2025

Choose a reason for hiding this comment

AmeyWale commented Feb 21, 2025 •

edited

Loading

semanticdiff-com bot commented Feb 21, 2025 •

edited

Loading