Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add coptr module #6935

Merged
merged 45 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
dad8548
Initial addition
ramirobarrantes Nov 5, 2024
77c71db
Updated snapshot
ramirobarrantes Nov 5, 2024
0416062
Updated snapshot
ramirobarrantes Nov 6, 2024
9579e60
Updated tests.
ramirobarrantes Nov 6, 2024
bcec266
Updated tests.
ramirobarrantes Nov 6, 2024
d117ef5
Updated snapshot
ramirobarrantes Nov 6, 2024
03e33ec
removed extra directory creation
ramirobarrantes Nov 6, 2024
0a502b6
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
b5da004
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
3a7c330
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
78546ff
Output filename now also contains index id so as to not to repeat fil…
ramirobarrantes Nov 7, 2024
41b1ba3
Updated snapshot
ramirobarrantes Nov 7, 2024
014d1d2
Updated snapshot
ramirobarrantes Nov 7, 2024
e3c26af
Added paired end test
ramirobarrantes Nov 7, 2024
786ca40
Added paired end test
ramirobarrantes Nov 7, 2024
ad56b3b
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
7d37343
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
01022eb
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
eb8364b
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
14457c3
Merge branch 'master' into coptr_map2
jfy133 Nov 12, 2024
98faf7a
Updated container used
ramirobarrantes Nov 13, 2024
15f0771
Added number of lines test for review
ramirobarrantes Nov 13, 2024
b3f5757
Added number of lines test for review
ramirobarrantes Nov 13, 2024
1becbd0
Updated nft-bam plugin
ramirobarrantes Nov 14, 2024
5f11a65
Updated container
ramirobarrantes Nov 14, 2024
0990246
Updated counting of reads
ramirobarrantes Nov 14, 2024
08e3e90
Updated counting of reads
ramirobarrantes Nov 14, 2024
56825bb
removed
ramirobarrantes Nov 14, 2024
b5413b5
updated snapshot
ramirobarrantes Nov 14, 2024
861c997
Updated test and snapshot
ramirobarrantes Nov 15, 2024
d36a0dc
Updated test and snapshot
ramirobarrantes Nov 15, 2024
d36035e
fixed versions issue
ramirobarrantes Nov 15, 2024
3098168
fixed versions issue
ramirobarrantes Nov 15, 2024
163041c
fixed versions issue
ramirobarrantes Nov 15, 2024
a453a1d
fixed versions issue
ramirobarrantes Nov 15, 2024
e6da7da
fixed versions issue
ramirobarrantes Nov 15, 2024
c52eff1
fixed versions issue
ramirobarrantes Nov 15, 2024
2704cc4
Updated test
ramirobarrantes Nov 15, 2024
463c469
Updated test
ramirobarrantes Nov 15, 2024
4ff3bbc
Updated snapshot
ramirobarrantes Nov 15, 2024
f2b04cc
Updated tag
ramirobarrantes Nov 15, 2024
ef7c169
Updated tests
ramirobarrantes Nov 15, 2024
1b963eb
Updated tests
ramirobarrantes Nov 15, 2024
7511481
Merge branch 'master' into coptr_map2
itrujnara Nov 19, 2024
d5f55e8
Improved descriptions
ramirobarrantes Nov 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions modules/nf-core/coptr/index/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ process COPTR_INDEX {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_1':
'biocontainers/coptr:1.1.4--pyhdfd78af_1' }"
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_3':
'biocontainers/coptr:1.1.4--pyhdfd78af_3' }"

input:
tuple val(meta), path(indexfasta, stageAs: "fastafolder/*")
Expand Down
12 changes: 6 additions & 6 deletions modules/nf-core/coptr/index/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@
}
],
"meta": {
"nf-test": "0.9.1",
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:20:08.028567181"
"timestamp": "2024-11-13T13:32:37.002823715"
},
"coptr index with multiple fastqs - stub": {
"content": [
Expand Down Expand Up @@ -92,10 +92,10 @@
}
],
"meta": {
"nf-test": "0.9.1",
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:28:00.318435045"
"timestamp": "2024-11-13T13:32:49.415406628"
},
"coptr index with multiple fastqs": {
"content": [
Expand Down Expand Up @@ -141,9 +141,9 @@
}
],
"meta": {
"nf-test": "0.9.1",
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:24:04.847842182"
"timestamp": "2024-11-13T13:32:43.654868575"
}
}
7 changes: 7 additions & 0 deletions modules/nf-core/coptr/map/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json
channels:
- conda-forge
- bioconda
dependencies:
- "bioconda::coptr=1.1.4"
62 changes: 62 additions & 0 deletions modules/nf-core/coptr/map/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
process COPTR_MAP {
tag "$meta.id"
label 'process_low'

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_3':
'biocontainers/coptr:1.1.4--pyhdfd78af_3' }"

input:
tuple val(meta), path(fasta, stageAs: "fastafolder/*")
tuple val(meta2), path(index)

output:
tuple val(meta), path("*.bam"), emit: bam
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
def prefix2 = task.ext.prefix ?: "${meta2.id}"

def paired_end = ""
if ( ! meta.single_end ) {
paired_end = "--paired"
}
"""
INDEX=`find -L ./ -name "*.rev.1.bt2" | sed "s/\\.rev.1.bt2\$//"`
[ -z "\$INDEX" ] && INDEX=`find -L ./ -name "*.rev.1.bt2l" | sed "s/\\.rev.1.bt2l\$//"`
[ -z "\$INDEX" ] && echo "Bowtie2 index files not found" 1>&2 && exit 1

coptr \
map \
$args $paired_end \
--threads $task.cpus \
\$INDEX \
fastafolder \
.

mv ${prefix}.bam ${prefix}_${prefix2}.bam
cat <<-END_VERSIONS > versions.yml
"${task.process}":
coptr: \$(coptr |& sed -E '11!d ; s/CoPTR.*?\\(v(.*?)\\).*/\\1/')
END_VERSIONS
"""

stub:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
"""
touch ${prefix}.bam

cat <<-END_VERSIONS > versions.yml
"${task.process}":
"${task.process}":
coptr: \$(coptr |& sed -E '11!d ; s/CoPTR.*?\\(v(.*?)\\).*/\\1/')
END_VERSIONS
"""
}
65 changes: 65 additions & 0 deletions modules/nf-core/coptr/map/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json
name: "coptr_map"
description: write your description here
keywords:
- coptr
- mapping
- ptr
tools:
- "coptr":
description: "Accurate and robust inference of microbial growth dynamics from
metagenomic sequencing reads."
homepage: "https://github.com/tyjo/coptr"
documentation: "https://coptr.readthedocs.io/"
tool_dev_url: "https://github.com/tyjo/coptr"
doi: "10.1101/gr.275533.121"
licence: ["GPL v3"]
identifier: biotools:coptr

input:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- fasta:
type: file
description: fastq file with reads
pattern: "*.{.fastq,.fq,.fastq.gz,fq.gz}"
ontologies:
- edam: "http://edamontology.org/format_1929"
- - meta2:
type: map
description: |
Groovy Map containing index genome id and path
e.g. [ id:'test', 'bowtie2' ]
- index:
type: file
description: Directory with Bowtie2 genome index files
pattern: "*.ebwt"

output:
- bam:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- "*.bam":
type: file
description: BAM file
pattern: "*.{bam}"
ontologies:
- edam: "http://edamontology.org/format_2572"

- versions:
- "versions.yml":
type: file
description: File containing software versions
pattern: "versions.yml"

authors:
- "@ramirobarrantes"
maintainers:
- "@ramirobarrantes"
100 changes: 100 additions & 0 deletions modules/nf-core/coptr/map/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
nextflow_process {

name "Test Process COPTR_MAP"
script "../main.nf"
process "COPTR_MAP"

tag "modules"
tag "modules_nfcore"
tag "coptr"
tag "coptr/index"
tag "coptr/map"

setup {
run("COPTR_INDEX") {
script "../../index/main.nf"
process {
"""
input[0] = [
[id:'test_0'],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
"""
}
}
}

test("sarscov2 - single end test") {

when {
process {
"""
input[0] = [
[ id:'test_1', single_end: true ],
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true)
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out.bam.collect{bam(it[1]).getReads(10)}).match() },
{ assert snapshot(process.out.bam.collect{bam(it[1]).getStatistics(include:["readCount"])}).match() }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{ assert snapshot(process.out.bam.collect{bam(it[1]).getReads(10)}).match() },
{ assert snapshot(process.out.bam.collect{bam(it[1]).getStatistics(include:["readCount"])}).match() }
{ assert snapshot(
process.out.bam.collect{bam(it[1]).getReads(10),
process.out.bam.collect{bam(it[1]).getStatistics(include:"readCount"])}
).match() }

This should fix your issue :) nf-test currently had no way to distinguish between both snapshot but my suggestion is the preferred way to fix this since we like to have one snapshot per test in nf-core

)
}
}

test("sarscov2 - paired end") {

when {
process {
"""
input[0] = [
[ id:'test', single_end: false ],
[
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true),
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_2.fastq.gz", checkIfExists: true)
]
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out.bam.collect{bam(it[1]).getReads(2)}).match() }
)
}
}


test("sarscov2 - single end - stub") {

options "-stub"

when {
process {
"""
input[0] = [
[ id:'test', single_end: true ],
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true)
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out.bam.collect{bam(it[1]).getReads(1)}).match() }
)
}
}

}
54 changes: 54 additions & 0 deletions modules/nf-core/coptr/map/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"sarscov2 - paired end": {
"content": [
[
[
"ACGCACATTGCTAACTAAGGGCACACTAGAACCAGAATATTTCAATTCAGTGTGTAGACTTATGAAAACTATAGGTCCAGACATGTTCCTCGGAACTTGTCGGCGTTGTCCTGCTGAAATTGTTGACACTGTGAGTGCTTTGGTTTATGA",
"ATGTGTACATTGGCGACCCTGCTCAATTACCTGCACCACGCACATTGCTAACTAAGGGCACACTAGAACCAGAATATTTCAATTCAGTGTGTAGACTTATGAAAACTATAGGTCCAGACATGTTCCTCGGAACTTGTCGGCGTTGTCCTG"
]
]
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-14T14:59:54.146815"
},
"sarscov2 - single end test": {
"content": [
[
[
"ACGCACATTGCTAACTAAGGGCACACTAGAACCAGAATATTTCAATTCAGTGTGTAGACTTATGAAAACTATAGGTCCAGACATGTTCCTCGGAACTTGTCGGCGTTGTCCTGCTGAAATTGTTGACACTGTGAGTGCTTTGGTTTATGA",
"GCATAGACGGTGCTTTACTTACAAAGTCCTCAGAATACAAAGGTCCTATTACGGATGTTTTCTACAAAGAAAACAGT",
"TAGGTGAGTTAGGTGATGTTAGAGAAACAATGAGTTACTTGTTTCAACATGCCAATTTAGATTCTTGCAAAAGAGTCTTGAACGTGGTGTGTAAAACTTGTGGACAACAGCAGACAACCCTTAAGGGTGTAGAAGCTGTTATGTAC",
"GTCTACAAGCTGGTAATGCAACAGAAGTGCCTGCCAATTCAACTGTATTATCTTTCTGTGCTTTTGCTGTAGATGCTGCTAAAGCTTACAAAGATTATCTAGCTAGTGGGGGACAACCAATCACTAATTGTG",
"AACCAGAATATTTCAATTCAGTGTGTAGACTTATGAAAACTATAGGTCCAGACATGTTCCTCGGAACTTGTCGGCGTTGT",
"ACTTTCCAAAGTGCAGTCAAAAGAACAATCACGGGTACACACCACTGGTTGTTACTCACAATTTTGACTTCACTTTTAG",
"CCTGTTGTCTATGTGATAGACGTGCCACATGCTTTTCCACTGCTTCAGACACTTATGCCTGTTGGCATCATTCTATTGGATTTGATTACGTCTATAATCCGTTTATGATTGATGTTCAACAATGGGGTTTTACAGGTAACCTACAAAGCA",
"GCTCCATATATAGTGGGTGATGTTGTTCAAGAGGGTGTTTTAACTGCTGTGGTTATACCTACTAAAAAGGCTGGTGGCACTACTGAAATGCTAGCGAAAGCTTTGAGAAAAGTGCCAACAGACAATTATATAACCACTTACCCGGGTCAGG",
"CAGTAGTCTTACTCTCAGTTTTGCAACAACTCAGAGTAGAATCATCATCTAAATTGTGGGCTCAATGTGTCCAGTTACACAATGACATTCTCTTAGCTAAAGATACTACTGAATCCTTTGAAAAAAAGGTTTCACTACTTTCGGTTTTG",
"CCTTATGACAGCAAGAACTGTGTATGATGATGGTGCTAGGAGAGTGTGGACACTTATGAATGTCTTGACACTCGTTTATAAAGTTTATTATGGTAATGCTTTAGATCAAGCCATTTCCATGTGGGCTCTTATAATCTCTGTTACTTC"
]
]
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-14T14:59:43.606198"
},
"sarscov2 - single end - stub": {
"content": [
[
[

]
]
],
"meta": {
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-14T15:00:02.115"
}
}
2 changes: 1 addition & 1 deletion nf-test.config
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ config {

// load the necessary plugins
plugins {
load "nft-bam@0.3.0"
load "nft-bam@0.5.0"
load "[email protected]"
load "[email protected]"
}
Expand Down
Loading