Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Connection hangups properly #464

Open
MaxwellDPS opened this issue Sep 26, 2023 · 0 comments
Open

Handle Connection hangups properly #464

MaxwellDPS opened this issue Sep 26, 2023 · 0 comments
Labels
feature use for describing a new feature to develop

Comments

@MaxwellDPS
Copy link

Description

Any hiccup to the graphql API will cause an API error, and makes calling .list() at scale impractical as it cant hold page / state.

This error handling is in-scope of the SDK

Environment

  1. OS (where OpenCTI server runs): CentOS Stream 9
  2. OpenCTI version: 10.5.2
  3. Other environment details: Scaled Cluster Deployment

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Use the SDK.

Expected Output

Automatically handle retries and connection hangups rather then raise an exception if a retry works

Actual Output

{"timestamp": "2023-09-26T17:04:31.821244Z", "level": "INFO", "name": "pycti.entities", "message": "Listing StixCyberObservables after <REMOVED>"}
5
{"timestamp": "2023-09-26T17:04:32.182713Z", "level": "INFO", "name": "pycti.entities", "message": "Listing StixCyberObservables after <REMOVED>"}
4
{"timestamp": "2023-09-26T17:04:32.325378Z", "level": "INFO", "name": "pycti.api", "message": "upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: delayed connect error: 111"}
3
{"timestamp": "2023-09-26T17:04:32.398060Z", "level": "ERROR", "name": "root", "message": "Error in message processing, reporting error to API", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.11/site-packages/pycti/connector/opencti_connector_helper.py\", line 235, in _process_message\n    json_data[\"internal\"][\"work_id\"], message_task.result()\n                                      ^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/concurrent/futures/thread.py\", line 58, in run\n    result = self.fn(*self.args, **self.kwargs)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/opencti-connector-export-file-csv/export-file-csv.py\", line 333, in _process_message\n    entities_list = do_list(\n                    ^^^^^^^^\n  File \"/usr/local/lib/python3.11/site-packages/pycti/entities/opencti_stix_cyber_observable.py\", line 401, in list\n    result = self.opencti.query(\n             ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/site-packages/pycti/api/opencti_api_client.py\", line 365, in query\n    raise ValueError(r.text)\nValueError: upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: delayed connect error: 111"}
2
{"timestamp": "2023-09-26T17:04:32.398405Z", "level": "INFO", "name": "pycti.api", "message": "Reporting work update_processed work_622e884a-b3fe-401d-8426-6b27e12f283e_2023-09-26T17:03:46.665Z"}

Additional information

Present since release

@SamuelHassine SamuelHassine added the feature use for describing a new feature to develop label Oct 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature use for describing a new feature to develop
Projects
None yet
Development

No branches or pull requests

2 participants