diff --git a/.dockstore.yml b/.dockstore.yml index 8ffe380ecbc..7787628d022 100644 --- a/.dockstore.yml +++ b/.dockstore.yml @@ -164,7 +164,7 @@ workflows: branches: - master - ah_var_store - - vs_1334_vcf_max_alt_alleles + - vs_1351_retry_with_more_memory_vcf_extract tags: - /.*/ - name: GvsImportGenomes diff --git a/scripts/variantstore/docs/aou/AOU_DELIVERABLES.md b/scripts/variantstore/docs/aou/AOU_DELIVERABLES.md index 14ba7fb25ac..31993322c71 100644 --- a/scripts/variantstore/docs/aou/AOU_DELIVERABLES.md +++ b/scripts/variantstore/docs/aou/AOU_DELIVERABLES.md @@ -93,7 +93,7 @@ - This workflow does not use the Terra Data Entity Model to run, so be sure to select the `Run workflow with inputs defined by file paths` workflow submission option. - Specify the same `call_set_identifier`, `dataset_name`, `project_id`, `extract_table_prefix`, and `interval_list` that were used in the `GvsPrepareRangesCallset` run documented above. - Specify the `interval_weights_bed` appropriate for the PGEN / VCF extraction run you are performing. `gs://gvs_quickstart_storage/weights/gvs_full_vet_weights_1kb_padded_orig.bed` is the interval weights BED used for Quickstart. - - For `GvsExtractCallsetPgenMerged` only, select the workflow option "Retry with more memory" and choose a "Memory retry factor" of 1.5 + - For both `GvsExtractCallset` and `GvsExtractCallsetPgenMerged`, select the workflow option "Retry with more memory" and choose a "Memory retry factor" of 1.5 - For `GvsExtractCallset`, make sure to specify the appropriate `maximum_alternate_alleles` value (currently 100). - These workflows do not use the Terra Data Entity Model to run, so be sure to select the `Run workflow with inputs defined by file paths` workflow submission option. 1. `GvsCalculatePrecisionAndSensitivity` workflow diff --git a/scripts/variantstore/wdl/GvsExtractCallset.wdl b/scripts/variantstore/wdl/GvsExtractCallset.wdl index af8286810d0..e9d12312c64 100644 --- a/scripts/variantstore/wdl/GvsExtractCallset.wdl +++ b/scripts/variantstore/wdl/GvsExtractCallset.wdl @@ -371,7 +371,18 @@ task ExtractTask { --filter-set-name ~{filter_set_name}' fi - gatk --java-options "-Xmx~{memory_gib - 3}g" \ + # This tool may get invoked with "Retry with more memory" with a different amount of memory than specified in + # the input `memory_gib`, so use the memory-related environment variables rather than the `memory_gib` input. + # https://support.terra.bio/hc/en-us/articles/4403215299355-Out-of-Memory-Retry + if [[ ${MEM_UNIT} == "GB" ]] + then + memory_mb=$(python3 -c "from math import floor; print(floor((${MEM_SIZE} - 3) * 1000))") + else + echo "Unexpected memory unit: ${MEM_UNIT}" 1>&2 + exit 1 + fi + + gatk --java-options "-Xmx${memory_mb}m" \ ExtractCohortToVcf \ --vet-ranges-extract-fq-table ~{fq_ranges_cohort_vet_extract_table} \ ~{"--vet-ranges-extract-table-version " + vet_extract_table_version} \