ETL default task changes & separating out study-specific workflows #11
mikix
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In anticipation of additional future studies & tasks, it makes sense to re-examine our default set of tasks and services and how you interact with studies. There are two notable changes that just landed:
Dropping covid symptom tasks from the default set
Specifically, the
covid_symptom__nlp_results
task will no longer be run by default (i.e. if you specify no--task
or--task-filter
parameter). You can still run it by name or by thecovid_symptom
orgpu
filters.Going forward, we won't add specific study tasks to the default set. We'll leave that for core FHIR resource tasks.
Renaming the
etl-support
docker compose profilePreviously, in preparation for running ETL, you would run
docker compose --profile etl-support up -d
to start the cTAKES and cNLPT services.But with multiple studies, it doesn't make sense to have a global profile that runs every service that any task could want. One study might need llama2, another might need cTAKES. Instead, we're renaming the
etl-support
profile to be study-specific:covid-symptom
.It's exactly the same, except you type
docker compose --profile covid-symptom up -d
(or useetl-support-gpu
).Going forward, as studies are added, we'll make new profiles for the services they specifically need.
tl;dr;
docker compose --profile etl-support up -d
before ETL runsdocker compose --profile covid-symptom-gpu up -d
firstBeta Was this translation helpful? Give feedback.
All reactions