Read.spc leads to error while reading any horiba spc files From 2018 version till now.

[b8e5n]

So far, I had no luck reading horiba labspec generated files using any version after 20171005.
The error message is the following:

Error in x[[jj]][iseq] <- vjj : replacement has length zero

Looking at the code, I haven't found anything that could fix that.

[cbeleites]

Hi @b8e5n
in order to help us track down what is happening here, we need some more information.

• We need to know the exact function call you use
• the output of traceback() after the error occured
• Please attach the output of sessionInfo()
• Can you send us a small(!) test file for which this error occurs together with a description what it should contain

Many thanks.

[b8e5n]

Hello Caudia,
Here are the lines ran and traceback.
Sorry some R text are translated in the language of my system.
I have the same issue with R-4.0.0, and the session or packages loaded do not help. I would like to share a file, but it is a 140k spectra file (>400mb), and I am certainly not allowed to share it. It it was a 1 spectra sample, that would be fine though. I have some, but there are actually loading fine.

> read.spc("file.spc")
Error in x[[jj]][iseq] <- vjj :
  l'argument de remplacement est de longueur nulle
De plus : Warning message:
In .spc.filehdr(f) :
  w planes found! This is not yet tested as the developer didn't have access to such files.
Please contact the package maintainer Claudia Beleites <[email protected]> stating whether the file was imported successfully or not.

> traceback()
3: `[<-.data.frame`(`*tmp*`, s, "w", value = NULL)
2: `[<-`(`*tmp*`, s, "w", value = NULL)
1: read.spc(file.spc")

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=French_Switzerland.1252  LC_CTYPE=French_Switzerland.1252
[3] LC_MONETARY=French_Switzerland.1252 LC_NUMERIC=C
[5] LC_TIME=French_Switzerland.1252

attached base packages:
[1] grid      stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
[1] hyperSpec_0.99-20200213.1 xml2_1.3.1
[3] ggplot2_3.3.0             lattice_0.20-41

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6        magrittr_1.5        tidyselect_1.0.0
 [4] munsell_0.5.0       colorspace_1.4-1    R6_2.4.1
 [7] jpeg_0.1-8.1        rlang_0.4.5         fansi_0.4.1
[10] dplyr_0.8.5         tools_3.6.1         gtable_0.3.0
[13] png_0.1-7           latticeExtra_0.6-29 cli_2.0.2
[16] withr_2.1.2         ellipsis_0.3.0      lazyeval_0.2.2
[19] assertthat_0.2.1    tibble_3.0.0        lifecycle_0.2.0
[22] crayon_1.3.4        RColorBrewer_1.1-2  purrr_0.3.3
[25] vctrs_0.2.4         testthat_2.3.2      glue_1.4.0
[28] compiler_3.6.1      pillar_1.4.3        scales_1.1.0
[31] pkgconfig_2.0.3

[ximeg]

Hi @b8e5n , if you still have access to the instrument, the easiest would be to acquire a couple of new random spectra, save in this file format, and share with us. Please also indicate the name and model of the instrument you use. Thank you!

[cbeleites]

It looks as if these are files in a particular subformat of .spc that I've never had access to so far. So @ximeg is right, we need a small test file to look into this.

If the problematic data is a time series or depth scan, a very small such file of maybe 3 or 5 spectra or a 2 x 3 map/image would be sufficient. But we'd need it to be of so unimportant content that you can contribute it to the hyperSpec project as material for unit testing.

[b8e5n]

This is not up to me, and I do not have direct access to the machines and software. I will share it once I got a sample.

[b8e5n]

sample.spc.pdf
Here is a sample file.
Notice that I renamed it was PDF so github would let me upload it.
I just tried and I cannot read it with the latestest version of hyperspec, which gives me the same issues as I have with other files. The file was generated by horiba labspec 6.

[cbeleites]

@b8e5n, many thanks. Could you leave a short description of what we should see in the file?

[b8e5n]

It can be imported easily with 2017 version:

install.packages("https://cran.r-project.org/src/contrib/Archive/hyperSpec/hyperSpec_0.99-20171005.tar.gz", repo=NULL)
Important notice, that version of hyperspec has a bug, and label needs to be instantiated as follow:
label=list()

---

Here is the print, and a plot of the spectra. Not sure if this is what you are asking for.

spc.test
hyperSpec object
12 spectra
5 data columns
737 data points / spectrum
wavelength: Delta * tilde(nu)/cm^-1 [numeric] 550.1374 552.2841 ... 1999.798
data: (12 rows x 5 columns)

1. z: lambda/(mu * m) [numeric] 277.3399 282.3346 ... 292.324
2. z.end: lambda/(mu * m) [numeric] 282.3346 287.3293 ... 277.3399
3. w: [numeric] -4021.368 -4021.368 ... -4011.38
4. spc: I/I[0] [matrix, array737] 22594.29 21009.22 ... 7627.913
5. filename: filename [character] ../../../Downloads/sample.spc ../../../Downloads/sample.spc ... ../../../Downloads/sample.spc
   

[ximeg]

Thank you @b8e5n , your comment that the old version from Oct 2017 was working is very valuable. It allowed me to track down the exact code change that changed this behavior.

Here is what I did

I checked out a commit from Oct 2017, which was a1d2bd85ec684. Somehow I could not build it, but I could reload all scripts from the hyperSpec/R directory using devtools::load_all function. To figure out, what commit changed the behavior, I used git bisect with the following script named issue-124-test.R:

#!/usr/bin/env Rscript

library(hyperSpec)

devtools::load_all('~/dev/hyperSpec/hyperSpec/')
label=list()
read.spc('~/tmp/sample.spc')

The script exit code is 0 on success and 1 in case of error.

Then I used git bisect

$ git bisect start
$ git bisect bad
$ git bisect good a1d2bd85ec684
Bisecting: 85 revisions left to test after this (roughly 7 steps)
[14029d81babb8b0e6688f22ba3e56576b6426bdd] skip file input tests if test files directory is not available
$ git bisect run ../issue-124-test.R

......

ba438343d0896637588e6ce902ac9a2db4263980 is the first bad commit
commit ba438343d0896637588e6ce902ac9a2db4263980
Author: Claudia Beleites <[email protected]>
Date:   Sun Mar 25 17:40:21 2018 +0200

    more spc w-planes

:040000 040000 47fc76b56ed321cbfc126b461e659d4b5f76a883 9aad60f572f887b0585471fb3261fe8becaed4d8 M	hyperSpec
bisect run success

Here are the code changes between working 142cb and broken ba4383

The change was in the function .spc.subhdr, file R/read.spc.R.

@cbeleites , could you please comment on that? I have no idea what this code actually does :)

[cbeleites]

I'll check - but first need to submit hotfix #134 to CRAN.

[GegznaV]

I think, the discussion must continue in r-hyperspec/hySpc.read.spc#3 as the functions related to spc were moved to package hySpc.read.spc.