Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Errors when running tests on Mac OS X #9

Open
terrycojones opened this issue Mar 7, 2014 · 2 comments
Open

Errors when running tests on Mac OS X #9

terrycojones opened this issue Mar 7, 2014 · 2 comments

Comments

@terrycojones
Copy link
Contributor

There are quite a few errors hit when running the tests on OS X:

.................E...EEEE....E..........E.....EEEEEEE..............................E........................estscan not available for this platform: darwin
EEE..........
======================================================================
ERROR: test_orf_annotator (test.test_annotation.AnnotationTest)
It tests orf annotator
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_annotation.py", line 46, in test_orf_annotator
    seq_records = orf_annotator(seq_records)
  File "crumbs/annotation.py", line 89, in __call__
    self._usage_matrix)
  File "crumbs/annotation.py", line 37, in _run_estscan
    binary = get_binary_path('estscan')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: estscan not available for this platform: darwin

======================================================================
ERROR: test_blast_search (test.test_blast.BlastTest)
It does a blast search
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 67, in test_blast_search
    dbtype='nucl')
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:


Building a new DB, current time: 03/07/2014 11:05:07
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_Ej1JWM/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_Ej1JWM/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_Ej1JWM/arabidopsis_genes does not exist



======================================================================
ERROR: test_blastdb (test.test_blast.BlastTest)
It creates a blast database.
----------------------------------------------------------------------


======================================================================
ERROR: test_blastdb (test.test_blast.BlastTest)
It creates a blast database.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 50, in test_blastdb
    dbtype='nucl')
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:


Building a new DB, current time: 03/07/2014 11:05:07
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_ctE7Vy/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_ctE7Vy/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_ctE7Vy/arabidopsis_genes does not exist



======================================================================
ERROR: test_get_or_create_blastdb (test.test_blast.BlastTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 105, in test_get_or_create_blastdb
    get_or_create_blastdb(blastdb, NUCL, directory.name)
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:

Building a new DB, current time: 03/07/2014 11:05:07
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/DFtHxB/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/DFtHxB/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/DFtHxB/arabidopsis_genes does not exist



======================================================================
ERROR: test.test_bulk_filters (__main__.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 64, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_bulk_filters
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 288, in _find_tests
    module = self._get_module_from_name(name)
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 266, in _get_module_from_name
    __import__(name)
  File "test/test_bulk_filters.py", line 31, in <module>
    from crumbs.bulk_filters import (filter_duplicates, _read_pairs,
  File "crumbs/bulk_filters.py", line 19, in <module>
    from crumbs.pairs import group_pairs_by_name, group_pairs
  File "crumbs/pairs.py", line 18, in <module>
    from toolz import first
ImportError: No module named toolz


======================================================================
ERROR: test.test_filters (__main__.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 64, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_filters
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 288, in _find_tests
    module = self._get_module_from_name(name)
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 266, in _get_module_from_name
    __import__(name)
  File "test/test_filters.py", line 33, in <module>
    from crumbs.filters import (FilterByLength, FilterById, FilterByQuality,
  File "crumbs/filters.py", line 37, in <module>
    from crumbs.pairs import group_pairs, group_pairs_by_name
  File "crumbs/pairs.py", line 18, in <module>
    from toolz import first
ImportError: No module named toolz


======================================================================
ERROR: test_key (test.test_iterutils.IterutilsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_iterutils.py", line 162, in test_key
    assert list(unique_items) == [(1, 'a'), (2, 'a')]
  File "crumbs/iterutils.py", line 234, in <genexpr>
    return (first(groups[1]) for groups in groupby(items, key))
  File "crumbs/utils/optional_modules.py", line 37, in FakeRequiredfunct
    raise OptionalRequirementError(msg)
OptionalRequirementError: A python package to run this executable is required, but it is not installed: toolz

======================================================================
ERROR: test_sorted_items (test.test_iterutils.IterutilsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_iterutils.py", line 146, in test_sorted_items
    max_items_in_memory=3)
  File "crumbs/iterutils.py", line 257, in sorted_items
    sorted_items = merge_sorted(*sorted_groups)
  File "crumbs/utils/optional_modules.py", line 37, in FakeRequiredfunct
    raise OptionalRequirementError(msg)
OptionalRequirementError: A python package to run this executable is required, but it is not installed: toolz

======================================================================
ERROR: test_unique_items (test.test_iterutils.IterutilsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_iterutils.py", line 155, in test_unique_items
    assert list(unique_items) == [1, 2, 3, 4]
  File "crumbs/iterutils.py", line 234, in <genexpr>
    return (first(groups[1]) for groups in groupby(items, key))
  File "crumbs/utils/optional_modules.py", line 37, in FakeRequiredfunct
    raise OptionalRequirementError(msg)
OptionalRequirementError: A python package to run this executable is required, but it is not installed: toolz

======================================================================
ERROR: test_get_or_create_index (test.test_mapping.Bowtie2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_mapping.py", line 42, in test_get_or_create_index
    index_fpath = get_or_create_bowtie2_index(seq_fpath, directory.name)
  File "crumbs/mapping.py", line 131, in get_or_create_bowtie2_index
    binary = get_binary_path('bowtie2-build')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: bowtie2-build not available for this platform: darwin

======================================================================
ERROR: test_map_with_bowtie2 (test.test_mapping.Bowtie2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_mapping.py", line 59, in test_map_with_bowtie2
    directory.name)
  File "crumbs/mapping.py", line 131, in get_or_create_bowtie2_index
    binary = get_binary_path('bowtie2-build')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: bowtie2-build not available for this platform: darwin

======================================================================
ERROR: test_get_or_create_index (test.test_mapping.Bwa2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_mapping.py", line 74, in test_get_or_create_index
    index_fpath = get_or_create_bwa_index(seq_fpath, directory.name)
  File "crumbs/mapping.py", line 64, in get_or_create_bwa_index
    _create_bwa_index(index_fpath)
  File "crumbs/mapping.py", line 33, in _create_bwa_index
    binary = get_binary_path('bwa')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: bwa not available for this platform: darwin

======================================================================
ERROR: test_map_with_bwa (test.test_mapping.Bwa2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_mapping.py", line 90, in test_map_with_bwa
    index_fpath = get_or_create_bwa_index(reference_fpath, directory.name)
  File "crumbs/mapping.py", line 64, in get_or_create_bwa_index
    _create_bwa_index(index_fpath)
  File "crumbs/mapping.py", line 33, in _create_bwa_index
    binary = get_binary_path('bwa')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: bwa not available for this platform: darwin


======================================================================
ERROR: test.test_pairing (__main__.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 64, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_pairing
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 288, in _find_tests
    module = self._get_module_from_name(name)
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 266, in _get_module_from_name
    __import__(name)
  File "test/test_pairing.py", line 25, in <module>
    from crumbs.pairs import (match_pairs, interleave_pairs, deinterleave_pairs,
  File "crumbs/pairs.py", line 18, in <module>
    from toolz import first
ImportError: No module named toolz


======================================================================
ERROR: test.test_small_bins (__main__.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 64, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_small_bins
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 288, in _find_tests
    module = self._get_module_from_name(name)
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 266, in _get_module_from_name
    __import__(name)
  File "test/test_small_bins.py", line 25, in <module>
    from crumbs.utils import BZ2File
ImportError: cannot import name BZ2File


======================================================================
ERROR: test_bin_transcrip_orientator (test.test_transcriptome_orientation.TestTranscriptomeOrientator)
it tests the transcript orientator binary
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_transcriptome_orientation.py", line 152, in test_bin_transcrip_orientator
    check_output(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['/Users/terry/s/net/seq_crumbs/bin/orientate_transcripts', '-u', '/Users/terry/s/net/seq_crumbs/crumbs/test_data/Arabidopsis_thaliana.smat', '-d', '/Users/terry/s/net/seq_crumbs/crumbs/test_data/blastdbs/arabidopsis_genes', '-d', '/Users/terry/s/net/seq_crumbs/crumbs/test_data/blastdbs/calabaza', '-g', 'blastn', '-g', 'blastn', '-v', '0.0001', '-v', '0.0001', '/Users/terry/s/net/seq_crumbs/crumbs/test_data/seqs_to_orientate.fasta', '-o', '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpuuiw4L', '--polya_min_len', '4']' returned non-zero exit status 10


======================================================================
ERROR: test_transcriptome_orientator (test.test_transcriptome_orientation.TestTranscriptomeOrientator)
tests the orientator class
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_transcriptome_orientation.py", line 124, in test_transcriptome_orientator
    seqs = orientator(seqrecords)
  File "crumbs/transcript_orientations.py", line 138, in __call__
    annot_seqrecords = annotator(seqs_to_analyze)
  File "crumbs/annotation.py", line 89, in __call__
    self._usage_matrix)
  File "crumbs/annotation.py", line 37, in _run_estscan
    binary = get_binary_path('estscan')
  File "crumbs/utils/bin_utils.py", line 243, in _get_binary_path
    raise MissingBinaryError(msg)
MissingBinaryError: estscan not available for this platform: darwin

======================================================================
ERROR: test.test_trim (__main__.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 64, in testFailure
    raise exception
ImportError: Failed to import test module: test.test_trim
Traceback (most recent call last):
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 288, in _find_tests
    module = self._get_module_from_name(name)
  File "/Users/terry/.virtualenvs/dm/lib/python2.7/site-packages/discover.py", line 266, in _get_module_from_name
    __import__(name)
  File "test/test_trim.py", line 25, in <module>
    from crumbs.trim import (TrimLowercasedLetters, TrimEdges, TrimOrMask,
  File "crumbs/trim.py", line 32, in <module>
    from crumbs.pairs import group_pairs_by_name, group_pairs
  File "crumbs/pairs.py", line 18, in <module>
    from toolz import first
ImportError: No module named toolz


----------------------------------------------------------------------
Ran 121 tests in 11.129s

FAILED (errors=18)
@terrycojones
Copy link
Contributor Author

OK, I've sorted a few of these out (mainly by installing things). My errors are down to

$ PYTHONPATH=. python -m discover
.....................EEE.............�BC����J�0�3�AE�|���Y�n��W��)�:���J��-Әd��|��[3��A
                                                                                       ���w�_��ko5NO��4���]$i
6B7�q'�A7�i���~FIfѼnqB*�HB�CXi����>���|TR��kAh�O��C�g��x!g!���rVq;�lUd\Z�%(F��h��,�m��$��,����
��\�%*�8���Dd=�R�
�*pk$�5J��~�FT��
                e��FL��}��34Lnyɠ(�T�P��15�,'-��}P�+�!�5�r!&\ҝ�:`�핀���B�t�˺y�����fi��m;���f����y��8v��BCEAn unexpected error happened.
The seq_crumbs developers would appreciate your feedback.
Please send them the error log: filter_by_bowtie2.error

file header is empty (mode='r') - is it SAM/BAM format?Traceback (most recent call last):
  File "/Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2", line 87, in <module>
    sys.exit(main(filter_by_bowtie2))
  File "/Users/terry/s/net/seq_crumbs/crumbs/utils/bin_utils.py", line 61, in main
    return(funct())
  File "/Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2", line 79, in filter_by_bowtie2
    args['out_format'], workers=workers)
  File "/Users/terry/s/net/seq_crumbs/crumbs/seqio.py", line 136, in write_filter_packets
    seqs_diverted=SEQS_FILTERED_OUT)
  File "/Users/terry/s/net/seq_crumbs/crumbs/seqio.py", line 112, in _write_filter_trim_packets
    for packet in packets:
  File "/Users/terry/s/net/seq_crumbs/crumbs/utils/seq_utils.py", line 94, in __call__
    processed_packet = map_function(processed_packet)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 67, in __call__
    self._setup_checks(filterpacket)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 310, in _setup_checks
    self.mapped_reads = _get_mapped_reads(bam_fhand.name, self.min_mapq)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 184, in _get_mapped_reads
    bam = pysam.Samfile(bam_fpath)
  File "csamtools.pyx", line 736, in pysam.csamtools.Samfile.__cinit__ (pysam/csamtools.c:8722)
  File "csamtools.pyx", line 781, in pysam.csamtools.Samfile._open (pysam/csamtools.c:9359)
  File "csamtools.pyx", line 891, in pysam.csamtools.Samfile._open (pysam/csamtools.c:10460)
ValueError: file header is empty (mode='r') - is it SAM/BAM format?
�o�P����tk[]�#�{?�e���b��#;p`����/5�bm>�tω��BC..�BC�sr�e�������g�b�njjdf�g���gefi��1u7��03���q!i24�4�JY�˘Be,�M`2��)�9�ɉ%�)�E���y Ff���������Ԣ���DAjbI1�2s.6��y���
                                                                                                                                                                 [�`bpg3�!�A��`ƌ"r)�2�P
                                                                                                                                                                                       �x��r�\ǐ4`vz_�BCE....................................................................................................
======================================================================
ERROR: test_blast_search (test.test_blast.BlastTest)
It does a blast search
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 67, in test_blast_search
    dbtype='nucl')
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:


Building a new DB, current time: 03/07/2014 12:20:09
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_D1EkdN/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_D1EkdN/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_D1EkdN/arabidopsis_genes does not exist



======================================================================
ERROR: test_blastdb (test.test_blast.BlastTest)
It creates a blast database.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 50, in test_blastdb
    dbtype='nucl')
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:


Building a new DB, current time: 03/07/2014 12:20:09
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_zfQ5Y_/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_zfQ5Y_/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/blast_dbs_zfQ5Y_/arabidopsis_genes does not exist



======================================================================
ERROR: test_get_or_create_blastdb (test.test_blast.BlastTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_blast.py", line 105, in test_get_or_create_blastdb
    get_or_create_blastdb(blastdb, NUCL, directory.name)
  File "crumbs/blast.py", line 109, in get_or_create_blastdb
    _makeblastdb_plus(dbpath, dbtype)
  File "crumbs/blast.py", line 58, in _makeblastdb_plus
    check_process_finishes(process, binary=cmd[0])
  File "crumbs/utils/bin_utils.py", line 162, in check_process_finishes
    raise ExternalBinaryError(msg)
ExternalBinaryError: makeblastdb had a problem running
stdout:


Building a new DB, current time: 03/07/2014 12:20:09
New DB name:   /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/QIo4FE/arabidopsis_genes
New DB title:  /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/QIo4FE/arabidopsis_genes
Sequence type: Nucleotide
Keep Linkouts: T
Keep MBits: T
Maximum file size: 1000000000B

stderr:
BLAST options error: File /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/QIo4FE/arabidopsis_genes does not exist



======================================================================
ERROR: test_filter_by_bowtie2 (test.test_filters.FilterBowtie2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_filters.py", line 522, in test_filter_by_bowtie2
    filter_packets = filter_(filter_packet)
  File "crumbs/filters.py", line 67, in __call__
    self._setup_checks(filterpacket)
  File "crumbs/filters.py", line 310, in _setup_checks
    self.mapped_reads = _get_mapped_reads(bam_fhand.name, self.min_mapq)
  File "crumbs/filters.py", line 184, in _get_mapped_reads
    bam = pysam.Samfile(bam_fpath)
  File "csamtools.pyx", line 736, in pysam.csamtools.Samfile.__cinit__ (pysam/csamtools.c:8722)
  File "csamtools.pyx", line 781, in pysam.csamtools.Samfile._open (pysam/csamtools.c:9359)
  File "csamtools.pyx", line 891, in pysam.csamtools.Samfile._open (pysam/csamtools.c:10460)
ValueError: file header is empty (mode='r') - is it SAM/BAM format?

======================================================================
ERROR: test_filter_by_bowtie2_bin (test.test_filters.FilterBowtie2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_filters.py", line 544, in test_filter_by_bowtie2_bin
    check_output(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['/Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2', '-i', '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/ieQCGK/arabidopsis_genes', '-o', '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpcD7Tth.seqs', '-e', '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpND27Ej.seqs', '/Users/terry/s/net/seq_crumbs/crumbs/test_data/arabidopsis_reads.fastq']' returned non-zero exit status 1

======================================================================
ERROR: test_map_with_bwa (test.test_mapping.Bwa2Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_mapping.py", line 92, in test_map_with_bwa
    map_with_bwasw(index_fpath, bam_fhand.name, unpaired_fpath=reads_fpath)
  File "crumbs/mapping.py", line 121, in map_with_bwasw
    raise RuntimeError(open(stderr.name).read())
RuntimeError: [bsw2_aln] read 4 sequences/pairs (728 bp) ...
[main] Version: 0.7.7-r441
[main] CMD: bwa bwasw -t 1 /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/YPSJbQ/arabidopsis_genes /Users/terry/s/net/seq_crumbs/crumbs/test_data/arabidopsis_reads.fastq
[main] Real time: 0.003 sec; CPU: 0.005 sec
[samopen] SAM header is present: 6 sequences.
[main_samview] random alignment retrieval only works for indexed BAM files.


----------------------------------------------------------------------
Ran 171 tests in 30.582s

FAILED (errors=6)

Yes, the test suite is spewing binary output on my tty :-)

I'll try to fix more later. I'm doing my work in https://github.com/terrycojones/seq_crumbs/tree/fix-tests-9 (just edited the README and added a requirements.txt file).

@terrycojones
Copy link
Contributor Author

Here are the contents of the filter_by_bowtie2.error file

<type 'exceptions.ValueError'>
Python 2.7.2: /Users/terry/.virtualenvs/seq_crumbs/bin/python
Fri Mar  7 12:20:16 2014

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /Users/terry/s/net/seq_crumbs/crumbs/utils/bin_utils.py in main(funct=<function filter_by_bowtie2>)
   59         if fail:
   60             raise RuntimeError('Generating a test error')
   61         return(funct())
   62     except FileNotFoundError, error:
   63         stderr.write(str(error) + '\n')
funct = <function filter_by_bowtie2>

 /Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2 in filter_by_bowtie2()
   77
   78     write_filter_packets(passed_fhand, filtered_fhand, filter_packets,
   79                          args['out_format'], workers=workers)
   80     flush_fhand(passed_fhand)
   81     if filtered_fhand is not None:
args = {'fail_drags_pair': None, 'filtered_fhand': <open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpND27Ej.seqs', mode 'wt'>, 'in_fhands': [<open file '/Users/terry/s/net/seq_crumbs/crumbs/test_data/arabidopsis_reads.fastq', mode 'rt'>], 'index': '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/ieQCGK/arabidopsis_genes', 'min_mapq': 0, 'original_in_fhands': [<open file '/Users/terry/s/net/seq_crumbs/crumbs/test_data/arabidopsis_reads.fastq', mode 'rt'>], 'out_fhand': <open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpcD7Tth.seqs', mode 'wt'>, 'out_format': 'fastq-illumina', 'paired_reads': False, 'processes': 1, ...}
workers = None

 /Users/terry/s/net/seq_crumbs/crumbs/seqio.py in write_filter_packets(passed_fhand=<open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpcD7Tth.seqs', mode 'wt'>, filtered_fhand=<open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpND27Ej.seqs', mode 'wt'>, filter_packets=<itertools.imap object>, file_format='fastq-illumina', workers=None)
  134     _write_filter_trim_packets(passed_fhand, filtered_fhand, filter_packets,
  135                                file_format=file_format, workers=workers,
  136                                seqs_diverted=SEQS_FILTERED_OUT)
  137
  138
seqs_diverted undefined
global SEQS_FILTERED_OUT = 'seqs_filtered_out'

 /Users/terry/s/net/seq_crumbs/crumbs/seqio.py in _write_filter_trim_packets(passed_fhand=<open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpcD7Tth.seqs', mode 'wt'>, diverted_fhand=<open file '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpND27Ej.seqs', mode 'wt'>, packets=<itertools.imap object>, file_format='fastq-illumina', workers=None, seqs_diverted='seqs_filtered_out')
  110
  111     flatten_pairs = lambda pairs: (seq for pair in pairs for seq in pair)
  112     for packet in packets:
  113         try:
  114             write_seqs(flatten_pairs(packet[SEQS_PASSED]), fhand=passed_fhand,
packet undefined
packets = <itertools.imap object>

 /Users/terry/s/net/seq_crumbs/crumbs/utils/seq_utils.py in __call__(self=<crumbs.utils.seq_utils._FunctionRunner object>, seq_packet={'seqs_filtered_out': [], 'seqs_passed': [(SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),)]})
   92         processed_packet = seq_packet
   93         for map_function in self.map_functions:
   94             processed_packet = map_function(processed_packet)
   95         return processed_packet
   96
processed_packet = {'seqs_filtered_out': [], 'seqs_passed': [(SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),)]}
map_function = <crumbs.filters.FilterBowtie2Match object>

 /Users/terry/s/net/seq_crumbs/crumbs/filters.py in __call__(self=<crumbs.filters.FilterBowtie2Match object>, filterpacket={'seqs_filtered_out': [], 'seqs_passed': [(SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),)]})
   65
   66     def __call__(self, filterpacket):
   67         self._setup_checks(filterpacket)
   68         reverse = self.reverse
   69         failed_drags_pair = self.failed_drags_pair
self = <crumbs.filters.FilterBowtie2Match object>
self._setup_checks = <bound method FilterBowtie2Match._setup_checks of <crumbs.filters.FilterBowtie2Match object>>
filterpacket = {'seqs_filtered_out': [], 'seqs_passed': [(SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),)]}

 /Users/terry/s/net/seq_crumbs/crumbs/filters.py in _setup_checks(self=<crumbs.filters.FilterBowtie2Match object>, filterpacket={'seqs_filtered_out': [], 'seqs_passed': [(SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),), (SeqWrapper(kind='seqitem', object=SeqItem(name='...), annotations={}), file_format='fastq-illumina'),)]})
  308                          extra_params=extra_params)
  309
  310         self.mapped_reads = _get_mapped_reads(bam_fhand.name, self.min_mapq)
  311
  312     def _do_check(self, seq):
self = <crumbs.filters.FilterBowtie2Match object>
self.mapped_reads undefined
global _get_mapped_reads = <function _get_mapped_reads>
bam_fhand = <open file '<fdopen>', mode 'w+b'>
bam_fhand.name = '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpQp7Ybz.bam'
self.min_mapq = 0

 /Users/terry/s/net/seq_crumbs/crumbs/filters.py in _get_mapped_reads(bam_fpath='/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpQp7Ybz.bam', min_mapq=0)
  182
  183 def _get_mapped_reads(bam_fpath, min_mapq=0):
  184     bam = pysam.Samfile(bam_fpath)
  185     return [read.qname for read in bam if not read.is_unmapped and (not min_mapq or read.mapq > min_mapq)]
  186
bam undefined
global pysam = <module 'pysam' from '/Users/terry/.virtualenvs/.../lib/python2.7/site-packages/pysam/__init__.pyc'>
pysam.Samfile = <type 'pysam.csamtools.Samfile'>
bam_fpath = '/var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpQp7Ybz.bam'

 /Users/terry/s/net/seq_crumbs/csamtools.pyx in pysam.csamtools.Samfile.__cinit__ (pysam/csamtools.c:8722)()


 /Users/terry/s/net/seq_crumbs/csamtools.pyx in pysam.csamtools.Samfile._open (pysam/csamtools.c:9359)()


 /Users/terry/s/net/seq_crumbs/csamtools.pyx in pysam.csamtools.Samfile._open (pysam/csamtools.c:10460)()

<type 'exceptions.ValueError'>: file header is empty (mode='r') - is it SAM/BAM format?
    __class__ = <type 'exceptions.ValueError'>
    __delattr__ = <method-wrapper '__delattr__' of exceptions.ValueError object>
    __dict__ = {}
    __doc__ = 'Inappropriate argument value (of correct type).'
    __format__ = <built-in method __format__ of exceptions.ValueError object>
    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ValueError object>
    __getitem__ = <method-wrapper '__getitem__' of exceptions.ValueError object>
    __getslice__ = <method-wrapper '__getslice__' of exceptions.ValueError object>
    __hash__ = <method-wrapper '__hash__' of exceptions.ValueError object>
    __init__ = <method-wrapper '__init__' of exceptions.ValueError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of exceptions.ValueError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ValueError object>
    __repr__ = <method-wrapper '__repr__' of exceptions.ValueError object>
    __setattr__ = <method-wrapper '__setattr__' of exceptions.ValueError object>
    __setstate__ = <built-in method __setstate__ of exceptions.ValueError object>
    __sizeof__ = <built-in method __sizeof__ of exceptions.ValueError object>
    __str__ = <method-wrapper '__str__' of exceptions.ValueError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of exceptions.ValueError object>
    args = ("file header is empty (mode='r') - is it SAM/BAM format?",)
    message = "file header is empty (mode='r') - is it SAM/BAM format?"

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "/Users/terry/s/net/seq_crumbs/crumbs/utils/bin_utils.py", line 61, in main
    return(funct())
  File "/Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2", line 79, in filter_by_bowtie2
    args['out_format'], workers=workers)
  File "/Users/terry/s/net/seq_crumbs/crumbs/seqio.py", line 136, in write_filter_packets
    seqs_diverted=SEQS_FILTERED_OUT)
  File "/Users/terry/s/net/seq_crumbs/crumbs/seqio.py", line 112, in _write_filter_trim_packets
    for packet in packets:
  File "/Users/terry/s/net/seq_crumbs/crumbs/utils/seq_utils.py", line 94, in __call__
    processed_packet = map_function(processed_packet)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 67, in __call__
    self._setup_checks(filterpacket)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 310, in _setup_checks
    self.mapped_reads = _get_mapped_reads(bam_fhand.name, self.min_mapq)
  File "/Users/terry/s/net/seq_crumbs/crumbs/filters.py", line 184, in _get_mapped_reads
    bam = pysam.Samfile(bam_fpath)
  File "csamtools.pyx", line 736, in pysam.csamtools.Samfile.__cinit__ (pysam/csamtools.c:8722)
  File "csamtools.pyx", line 781, in pysam.csamtools.Samfile._open (pysam/csamtools.c:9359)
  File "csamtools.pyx", line 891, in pysam.csamtools.Samfile._open (pysam/csamtools.c:10460)
ValueError: file header is empty (mode='r') - is it SAM/BAM format?


The command was:
/Users/terry/s/net/seq_crumbs/bin/filter_by_bowtie2 -i /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/ieQCGK/arabidopsis_genes -o /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpcD7Tth.seqs -e /var/folders/cv/v6h2zl0s56z5xnvnvw6lp1rw0000gn/T/tmpND27Ej.seqs /Users/terry/s/net/seq_crumbs/crumbs/test_data/arabidopsis_reads.fastq

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@terrycojones