Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG - Science start and end date is used for dependency #1316

Open
tech3371 opened this issue Jan 29, 2025 · 0 comments · May be fixed by #1318
Open

BUG - Science start and end date is used for dependency #1316

tech3371 opened this issue Jan 29, 2025 · 0 comments · May be fixed by #1318
Assignees
Labels
bug Something isn't working
Milestone

Comments

@tech3371
Copy link
Contributor

Description of the issue

Because we are using self.start_date and self.end_date, it's causing to assume that other dependency will have same date as current instrument and its data level's date. This becomes an issue since start and end date of dependency could range longer than current processing data.

Steps to reproduce the issue

Run this command
imap_cli --instrument swapi --data-level l1 --descriptor sci --start-date 20241122 --version v001 --dependency "[{'instrument': 'swapi', 'data_level': 'l0', 'descriptor': 'raw', 'version': 'v001','start_date': '20241122'}, {'instrument': 'swapi', 'data_level': 'l1', 'descriptor': 'hk', 'version': 'v001','start_date': '20240924'}]"

Expected behavior (What should happen)

Download dependency of different start and end date and it's parameters.

Actual behavior (What does happen)

INFO:imap_processing.cli:Processing Swapi level l1
INFO:imap_processing.cli:Beginning preprocessing (download dependencies)
INFO:imap_data_access.io:Querying data archive for {'instrument': 'swapi', 'data_level': 'l0', 'descriptor': 'raw', 'start_date': '20241122', 'end_date': '20241122', 'version': 'v001'} with url https://api.dev.imap-mission.com/query?instrument=swapi&data_level=l0&descriptor=raw&start_date=20241122&end_date=20241122&version=v001
INFO:imap_data_access.io:The file /Users/tech3371/git/imap_processing/data/imap/swapi/l0/2024/11/imap_swapi_l0_raw_20241122_v001.pkts already exists, skipping download
INFO:imap_data_access.io:Querying data archive for {'instrument': 'swapi', 'data_level': 'l1', 'descriptor': 'hk', 'start_date': '20241122', 'end_date': '20241122', 'version': 'v001'} with url https://api.dev.imap-mission.com/query?instrument=swapi&data_level=l1&descriptor=hk&start_date=20241122&end_date=20241122&version=v001
Traceback (most recent call last):
  File "/Users/tech3371/Library/Caches/pypoetry/virtualenvs/imap-processing-52JUrpuW-py3.10/bin/imap_cli", line 8, in <module>
    sys.exit(main())
  File "/Users/tech3371/Library/Caches/pypoetry/virtualenvs/imap-processing-52JUrpuW-py3.10/lib/python3.10/site-packages/imap_processing/cli.py", line 899, in main
    instrument.process()
  File "/Users/tech3371/Library/Caches/pypoetry/virtualenvs/imap-processing-52JUrpuW-py3.10/lib/python3.10/site-packages/imap_processing/cli.py", line 346, in process
    dependencies = self.pre_processing()
  File "/Users/tech3371/Library/Caches/pypoetry/virtualenvs/imap-processing-52JUrpuW-py3.10/lib/python3.10/site-packages/imap_processing/cli.py", line 366, in pre_processing
    self._dependency_list = self.download_dependencies()
  File "/Users/tech3371/Library/Caches/pypoetry/virtualenvs/imap-processing-52JUrpuW-py3.10/lib/python3.10/site-packages/imap_processing/cli.py", line 302, in download_dependencies
    raise FileNotFoundError(
FileNotFoundError: File not found for required dependency {'instrument': 'swapi', 'data_level': 'l1', 'descriptor': 'hk', 'version': 'v001', 'start_date': '20240924'} while attempting to create file.This should never occur in normal processing.

Code Snippet:

Code

Additional notes, affected areas, and suggested fixes

No response

@tech3371 tech3371 added the bug Something isn't working label Jan 29, 2025
@tech3371 tech3371 added this to the Feb 2025 milestone Jan 29, 2025
@tech3371 tech3371 self-assigned this Jan 29, 2025
@tech3371 tech3371 added this to IMAP Jan 29, 2025
@tech3371 tech3371 linked a pull request Jan 29, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

1 participant