You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Consider the following flow case:
Simple Trigger -> SOAP API call that returns large array -> Splitter -> ...
Imagine that the SOAP API call returns an array with total size 1 MB with 9000 entries. Given that for each object produced by Splitter includes the entire result of the SOAP API call in every message emitted from the splitter, the above flow would produce 9 GB of data on execution easily exceeding the queue size limits. (This is further exasterbasted by elasticio/sailor-nodejs#80)
However, if these messages are 100% independent of each other, then there is no reason to attach each result to the larger SOAP body. It would be complex to change passthrough logic or message structure to handle the above case. In theory, the above could be tuned to emit things slower, but that also introduces problems. However, if we understand that the above is a common pairing, then it may make sense to add the following two inputs to the SOAP API component:
Body JSONata transform expression (optional, default to idempotent $)
Split result if it is an array (optional, default to false)
The component behavior would be changed to:
After the component has a JSON representation of the response body that they are going to emit, then:
Evaluate the provided JSON expression if it exists to the body of the HTTP response.
If the result is an array and splitting is true, emit a message per array entry. Otherwise emit the transformed result.
The text was updated successfully, but these errors were encountered:
Implements the functionality described in https://github.com/elasticio/elasticio/issues/2793 at a component level (assuming that it is not implemented at a platform level).
SOAP equivalent of this functionality in the REST Component: elasticio/rest-api-component#91 & elasticio/rest-api-component#68
Consider the following flow case:
Simple Trigger -> SOAP API call that returns large array -> Splitter -> ...
Imagine that the SOAP API call returns an array with total size 1 MB with 9000 entries. Given that for each object produced by Splitter includes the entire result of the SOAP API call in every message emitted from the splitter, the above flow would produce 9 GB of data on execution easily exceeding the queue size limits. (This is further exasterbasted by elasticio/sailor-nodejs#80)
However, if these messages are 100% independent of each other, then there is no reason to attach each result to the larger SOAP body. It would be complex to change passthrough logic or message structure to handle the above case. In theory, the above could be tuned to emit things slower, but that also introduces problems. However, if we understand that the above is a common pairing, then it may make sense to add the following two inputs to the SOAP API component:
The component behavior would be changed to:
After the component has a JSON representation of the response body that they are going to emit, then:
The text was updated successfully, but these errors were encountered: