jsonschema errors - No way to know which stream name failed

[visch]

2023-09-29T18:56:53.270886Z [info     ] 2023-09-29 18:56:53,270 | INFO     | snowflake.connector.cursor | Number of results in first chunk: 1 cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.271316Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.271668Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/bin/target-snowflake", line 8, in <module> cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.272072Z [info     ]     sys.exit(TargetSnowflake.cli()) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.272406Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/click/core.py", line 1157, in __call__ cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.272720Z [info     ]     return self.main(*args, **kwargs) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.273074Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/click/core.py", line 1078, in main cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.273485Z [info     ]     rv = self.invoke(ctx)      cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.273831Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/click/core.py", line 1434, in invoke cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.274157Z [info     ]     return ctx.invoke(self.callback, **ctx.params) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.274471Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/click/core.py", line 783, in invoke cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.274805Z [info     ]     return __callback(*args, **kwargs) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.275122Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 550, in invoke cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.275445Z [info     ]     target.listen(file_input)  cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.275742Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/io_base.py", line 34, in listen cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.276050Z [info     ]     self._process_lines(file_input) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.276343Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 291, in _process_lines cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.276789Z [info     ]     counter = super()._process_lines(file_input) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.277120Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/io_base.py", line 93, in _process_lines cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.277426Z [info     ]     self._process_record_message(line_dict) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.277724Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 341, in _process_record_message cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.278017Z [info     ]     sink._validate_and_parse(transformed_record) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.278309Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/sinks/core.py", line 317, in _validate_and_parse cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.278603Z [info     ]     self._validator.validate(record) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.278914Z [info     ]   File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/jsonschema/validators.py", line 435, in validate cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.279209Z [info     ]     raise error                cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.279504Z [info     ] jsonschema.exceptions.ValidationError: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' is too long cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.279798Z [info     ]                                cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.280089Z [info     ] Failed validating 'maxLength' in schema['properties']['fieldname']: cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.280392Z [info     ]     {'maxLength': 20, 'type': ['string', 'null']} cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.280697Z [info     ]                                cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.281026Z [info     ] On instance['fieldname']:        cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-09-29T18:56:53.281332Z [info     ]     'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake

We could wrap /project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/sinks/core.py with a try catch and output the sink name. Ideally the error would include the stack trace, and jsonschema error, along the with the stream_name, and key properties of the row that fails (if one exists)

We could do this in the SDK as well 🤷‍♂️

[visch]

Going to close this one as #128 covers it