Using cumulus etl and cumulus library to curate Bundle Resources from a FHIR API request

[wfclark5]

Hello! I am incredibly impressed by this project and was hoping to use it for non-bulk operations.

So for example, is it possible to use the cumulus project (etl, library) to curate and normalize FHIR bundles for a single patient from an API call that's not the bulk export api? If so where would be a good place to start? If this is not possible due to the strict use case of only bulk fhir exports could you point me to a few places in the code where this sort of feature could be added?

Thank you again for the incredible work on this project and for sharing it with the community.

[mikix]

Hello! Thanks for your kind words and interest!

Cumulus ETL works on piles of ndjson - it doesn't really care how you got them. So if you could format your FHIR pile as ndjson, you're good to go.

I deal with the single patient API a lot less than the bulk API, but I think it often provides you with actual Bundle objects, yeah? Cumulus isn't really set up to process those, so there'd have to be some splitting of a bundle into separate ndjson files per resource.

Maybe a reasonable feature request, but not something that we've needed yet. You ask about the code for this kind of thing - it might make sense as a "loader", like a BundleLoader or something. Or maybe it could just live as an optional feature of the NdjsonLoader if it detects a bundle. I dunno what makes most sense off the top of my head.

But in the meantime, if you can just massage your json into separate ndjson files per-resource, you're good to experiment with Cumulus ETL today.