Implement minor changes from Weather branch into Auxiliary for PR

[gmanatole]

Weather branch is a huge PR coming up. We break it down into smaller PRs starting with the changes to Auxiliary.
Here is a small list of the modifications :
utils/timestamp_utils :
Function to_timestamp():
Existing timestamp handling of : "%Y-%m-%dT%H:%M:%S.%fZ" or "%Y-%m-%dT%H-%M-%S_%fZ"
We add handling of "%Y-%m-%dT%H:%M:%S.%f%z", handling of data already in timestamp format.
We can now apply this to_timestamp() to any df regardless of is the data is already into timestamp.
The check_epoch function applies the to_timestamp function to any df and creates an epoch column (POSIX time).

Spectrogram :
We switch audio_foldername from a variable to an attribute that will be used in Auxiliary

Auxiliary :
Added a saving method. As of now joined dataframes are stored in data/auxiliary/{self.spectro_duration}_{self.samplerate}.csv (ie self.audio_foldername)
Note that the dataframe only depends on spectro_duration.
Moved fetch_data method from Weather class to join_acoustic method in Auxiliary

[cazaudo]

please format your code using black , poetry run black .

[cazaudo]

i cannot see the acoustic features in the saved dataframe , eg /home/datawork-osmose/dataset/glider_WHOI_2014_we04/data/auxiliary/59_32000.csv , we just have to set " , acoustic = True)" right ?

[cazaudo]

not easy to see how you will make your input params fcs settable by the user , but it will be necessary , making available to the user the method join_acoustics ? that should bypass the automatic_join when defined from the exterior ? dont know

[cazaudo]

suggestion : another possible commit on the implementation of a dropna method inst.df = inst.df.dropna() in the Auxiliary class ?

[cazaudo]

not sure it is a good idea to name dataframe column with a numerical value (eg fcs=[8000] will give a column name 8000 -> inst.df.8000 does not work..)

[gmanatole]

not sure it is a good idea to name dataframe column with a numerical value (eg fcs=[8000] will give a column name 8000 -> inst.df.8000 does not work..)

Do you have any suggestions ? I don't mind calling inst.df['8000'] as I find it intuitive, but we could change column name

[gmanatole]

i cannot see the acoustic features in the saved dataframe , eg /home/datawork-osmose/dataset/glider_WHOI_2014_we04/data/auxiliary/59_32000.csv , we just have to set " , acoustic = True)" right ?

Modified acoustic join to create full_band acoustic join.

[cazaudo]

"Do you have any suggestions ? I don't mind calling inst.df['8000'] as I find it intuitive, but we could change column name"

i would use SPL_8000 , by default SPL_broadband

[cazaudo]

Modified acoustic join to create full_band acoustic join.

sorry dedicated term is more broadband rather than fullband

[gmanatole]

"Do you have any suggestions ? I don't mind calling inst.df['8000'] as I find it intuitive, but we could change column name"

i would use SPL_8000 , by default SPL_broadband

@cazaudo Are all npz storing sound pressure level data ? Scipy's welch function returns power spectral densities.
We can use NL for noise level ?

[cazaudo]

@cazaudo Are all npz storing sound pressure level data ? Scipy's welch function returns power spectral densities.
We can use NL for noise level ?

good point we need to more rigorous on our voc , i let you watch the method gen_spectro to see details on what is saved , if doubt talk about it with @GabrielDubus he is the last contributor to the code

[GabrielDubus]

@cazaudo Are all npz storing sound pressure level data ? Scipy's welch function returns power spectral densities.
We can use NL for noise level ?

good point we need to more rigorous on our voc , i let you watch the method gen_spectro to see details on what is saved , if doubt talk about it with @GabrielDubus he is the last contributor to the code

If self.data_normalization is instrument, and considering that input data are in Pascal, gen_spectro return:

• Pa**2 (scaling=spectrum), wich correspond to Power spectrum
• Pa**2/Hz (scaling = density) wich correspond to DSP

If self.data_normalization is zscore, gen_spectro return the square of adimensionned values, so "log_spectro = 10 * np.log10(Sxx + (1e-20))" is scaled around 0dB