Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add crabs/dbimport from readsimulator pipeline and rename to crabs/import #6584

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
768b952
add crabs/dbimport from readsimulator pipeline
famosab Sep 5, 2024
07c49d2
fix linting
famosab Sep 5, 2024
6be4add
Merge branch 'master' into crabs-2
famosab Sep 5, 2024
115f539
remove defaults
famosab Sep 5, 2024
eec926c
update crabs with wave containers
famosab Sep 5, 2024
35a9c29
update meta
famosab Sep 5, 2024
344aee7
Merge branch 'master' into crabs-2
SPPearce Sep 6, 2024
d545ed4
Merge branch 'master' into crabs-2
SPPearce Sep 6, 2024
a2a89f9
Merge branch 'master' into crabs-2
ewels Sep 8, 2024
141c1d0
remove name
famosab Sep 9, 2024
a5480ab
Merge branch 'master' into crabs-2
famosab Sep 9, 2024
a530697
update env
famosab Sep 9, 2024
4b69ae0
prettier
famosab Sep 9, 2024
bc32c2d
fix pip version
famosab Sep 9, 2024
99daaea
Merge branch 'master' into crabs-2
famosab Oct 15, 2024
90ea385
update to 1.0.4 new structure needs resolving
famosab Oct 25, 2024
f157b0b
update
famosab Nov 18, 2024
65baa69
Merge branch 'master' into crabs-2
famosab Nov 18, 2024
5ed8b33
change ext args to new convention
famosab Nov 20, 2024
f03faa7
Merge branch 'master' into crabs-2
famosab Nov 20, 2024
b525e17
Merge branch 'master' into crabs-2
famosab Feb 3, 2025
29d8509
remove tags and move config file
famosab Feb 3, 2025
d0335ba
add tests with download taxonomy
famosab Feb 3, 2025
ff5d661
change to test data
famosab Feb 4, 2025
0d6c3c3
add test file
famosab Feb 17, 2025
6c0c017
Merge branch 'master' into crabs-2
famosab Feb 17, 2025
23b17ea
change version accession
famosab Feb 18, 2025
d662f22
Merge branch 'master' into crabs-2
famosab Feb 18, 2025
52b57fb
update paths and correct meta
famosab Feb 18, 2025
2918805
Merge branch 'master' into crabs-2
famosab Feb 18, 2025
fc0fb10
Update modules/nf-core/crabs/dbimport/main.nf
famosab Feb 18, 2025
bd4a3cd
rename to import
famosab Feb 18, 2025
3b321b0
update snaps
famosab Feb 18, 2025
2da40fc
Merge branch 'master' into crabs-2
famosab Feb 19, 2025
bffb12e
Remove v from version string
jasmezz Feb 19, 2025
dbbf9e1
Test out old version 1.0.3 for version string capture
jasmezz Feb 19, 2025
f44ca45
Revert back to version 1.0.7
jasmezz Feb 19, 2025
ea80fd1
Merge branch 'master' into crabs-2
famosab Feb 19, 2025
6a08dbe
Merge branch 'master' into crabs-2
famosab Feb 20, 2025
78780c5
Merge branch 'master' into crabs-2
famosab Feb 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions modules/nf-core/crabs/import/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
channels:
- conda-forge
- bioconda
dependencies:
- bioconda::crabs=1.0.7
59 changes: 59 additions & 0 deletions modules/nf-core/crabs/import/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
process CRABS_IMPORT {
tag "$meta.id"
label 'process_medium'

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/crabs:1.0.7--pyhdfd78af_0':
'biocontainers/crabs:1.0.7--pyhdfd78af_0' }"

input:
tuple val(meta), path(fasta)
tuple val(meta2), path(accession2taxid)
tuple val(meta3), path(names)
tuple val(meta4), path(nodes)

output:
tuple val(meta), path("*.fa"), emit: fasta
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
def is_compressed = fasta.name.endsWith(".gz")
def fasta_name = fasta.name.replace(".gz", "")
"""
if [ "${is_compressed}" == "true" ]; then
gzip -c -d ${fasta} > ${fasta_name}
fi

crabs --import \\
--input ${fasta_name} \\
--output ${prefix}.crabsdb.fa \\
--acc2tax ${accession2taxid} \\
--names ${names} \\
--nodes ${nodes} \\
$args

rm ${fasta_name}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
crabs: \$(crabs --help 2>/dev/null | grep 'CRABS |' | sed 's/.*CRABS | v\\([0-9.]*\\).*/\\1/')
END_VERSIONS
"""

stub:
def prefix = task.ext.prefix ?: "${meta.id}"
"""
touch ${prefix}.fa

cat <<-END_VERSIONS > versions.yml
"${task.process}":
crabs: \$(crabs --help 2>/dev/null | grep 'CRABS |' | sed 's/.*CRABS | v\\([0-9.]*\\).*/\\1/')
END_VERSIONS
"""
}
79 changes: 79 additions & 0 deletions modules/nf-core/crabs/import/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: crabs_import
description: In-house generated or curated data can be imported into CRABS.
keywords:
- insilico
- amplicon
- sequencing
- inhouse
tools:
- crabs:
description: |
Crabs (Creating Reference databases for Amplicon-Based Sequencing)
is a program to download and curate reference databases
for eDNA metabarcoding analyses
homepage: https://github.com/gjeunen/reference_database_creator
documentation: https://github.com/gjeunen/reference_database_creator?tab=readme-ov-file#running-crabs
tool_dev_url: https://github.com/gjeunen/reference_database_creator
doi: 10.1111/1755-0998.13741
licence: ["MIT License"]
identifier: ""

input:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- fasta:
type: file
description: In-house sequencing data
pattern: "*.{fa,fasta}"
- - meta2:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- accession2taxid:
type: file
description: Linking accession numbers to taxonomic IDs
pattern: "*.accession2taxid"
- - meta3:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- names:
type: file
description: Containing information about the phylogenetic name associated with each taxonomic ID
pattern: "*.dmp"
- - meta4:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- nodes:
type: file
description: Containing information how taxonomic IDs are linked
pattern: "*.dmp"
output:
- fasta:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- "*.fa":
type: file
description: Reverse complemented Sequence
pattern: "*.{fa,fasta}"
- versions:
- versions.yml:
type: file
description: File containing software versions
pattern: "versions.yml"
authors:
- "@famosab"
- "@a4000"
maintainers:
- "@famosab"
- "@a4000"
90 changes: 90 additions & 0 deletions modules/nf-core/crabs/import/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
nextflow_process {

name "Test Process CRABS_IMPORT"
script "../main.nf"
process "CRABS_IMPORT"

tag "modules"
tag "modules_nfcore"
tag "crabs"
tag "crabs/import"

test("sarscov2 - fasta") {

config "./nextflow.config"

when {
params {
module_args = "--import-format embl --ranks 'superkingdom;phylum;class;order;family;genus;species' \\"
}
process {
"""
input[0] = [
[ id:'test' ], // meta map
file(params.modules_testdata_base_path +'genomics/sarscov2/genome/genome-ena.fasta', checkIfExists: true)
]
input[1] = [
[ id:'accession2taxid' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/accession2taxid/nucl_gb.accession2taxid', checkIfExists: true)
]
input[2] = [
[ id:'names' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/taxdmp/names.dmp', checkIfExists: true)
]
input[3] = [
[ id:'nodes' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/taxdmp/nodes.dmp', checkIfExists: true)
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}

}

test("sarscov2 - fasta - stub") {

options "-stub"

when {
process {
"""
input[0] = [
[ id:'test' ], // meta map
file(params.modules_testdata_base_path +'genomics/sarscov2/genome/genome-ena.fasta', checkIfExists: true)
]
input[1] = [
[ id:'accession2taxid' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/accession2taxid/nucl_gb.accession2taxid', checkIfExists: true)
]
input[2] = [
[ id:'names' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/taxdmp/names.dmp', checkIfExists: true)
]
input[3] = [
[ id:'nodes' ], // meta map
file(params.modules_testdata_base_path +'genomics/prokaryotes/metagenome/taxonomy/taxdmp/nodes.dmp', checkIfExists: true)
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(
process.out,
path(process.out.versions[0]).yaml
).match() }
)
}

}

}
73 changes: 73 additions & 0 deletions modules/nf-core/crabs/import/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{
"sarscov2 - fasta - stub": {
"content": [
{
"0": [
[
{
"id": "test"
},
"test.fa:md5,d41d8cd98f00b204e9800998ecf8427e"
]
],
"1": [
"versions.yml:md5,d21925bd41f67039425ac09278c847d0"
],
"fasta": [
[
{
"id": "test"
},
"test.fa:md5,d41d8cd98f00b204e9800998ecf8427e"
]
],
"versions": [
"versions.yml:md5,d21925bd41f67039425ac09278c847d0"
]
},
{
"CRABS_IMPORT": {
"crabs": "1.0.7"
}
}
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2025-02-19T13:58:38.53029566"
},
"sarscov2 - fasta": {
"content": [
{
"0": [
[
{
"id": "test"
},
"test.crabsdb.fa:md5,22c8c613e67ce3f5237a6c4646a0abb1"
]
],
"1": [
"versions.yml:md5,d21925bd41f67039425ac09278c847d0"
],
"fasta": [
[
{
"id": "test"
},
"test.crabsdb.fa:md5,22c8c613e67ce3f5237a6c4646a0abb1"
]
],
"versions": [
"versions.yml:md5,d21925bd41f67039425ac09278c847d0"
]
}
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2025-02-19T13:58:30.649755841"
}
}
7 changes: 7 additions & 0 deletions modules/nf-core/crabs/import/tests/nextflow.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
process {

withName: CRABS_IMPORT {
ext.args = params.module_args
}

}
Loading