Improve server-client communication error handling

[MetRonnie]

In #6499 we added a new field to the trigger GraphQL mutation, and this unfortunately broke the ability for cylc trigger at 8.4 to work on workflows running in 8.3 (I shall term this "inter-version server-client comms").

This PR does not fix that (we don't think a fix is feasible at this stage).

An additional problem is that the error handling for server-client comms is broken - that is what this PR fixes. And it attempts to ensure we have defined format for server-client comms which should reduce the chance of breaking inter-version server-client comms in future.

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• No dependency changes
• Tests are included (or explain why tests are not needed).
• Changelog entry included if this is a change that can affect users
• No docs needed
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[MetRonnie]

For future/historical reference, I originally tried mocking the poll method of client.poller, i.e.

mp.setattr(client.poller, 'poll', Mock())

but this resulted in pytest hanging near the end of running all integration tests; some strange interaction due to the ZMQ poller using threading or something, dunno.

Anyway, the reason for this monkeypatching is to get client.poller.poll() to return a truthy value so the socket can receive the mock response

cylc-flow/cylc/flow/network/client.py

Lines 298 to 300 in dd468d6

	# receive response
	if self.poller.poll(timeout):
	res = await self.socket.recv()

[wxtim]

I know that you are trying to catch as much as possible for the future, but might we be able to identify particular problems and provide a more obvious error?

--- a/cylc/flow/network/multi.py
+++ b/cylc/flow/network/multi.py
@@ -38,6 +38,13 @@ from cylc.flow.id_cli import parse_ids_async
 from cylc.flow.terminal import DIM
 
 
+KNOWN_ERRS = {
+    'Unknown argument "onResume" on field "trigger" of type "Mutations".':
+        'Tasks in workflows running using versions of Cylc older than 8.4.0'
+        ' cannot be triggered using Cylc 8.4.0.'
+}
+
+
 def call_multi(*args, **kwargs):
     """Call a function for each workflow in a list of IDs.
 
@@ -251,6 +258,10 @@ def _report(
             if isinstance(response, list) and response[0].get('error'):
                 # If operating on workflow running in older Cylc version,
                 # may get a error response like [{'error': '...'}]
+                if response[0].get('error').get('message') in KNOWN_ERRS:
+                    raise CylcError(
+                        KNOWN_ERRS[response[0].get('error').get('message')]
+                    )
                 raise Exception(response)
             raise Exception(f"Unexpected response: {response}")
         for mutation_response in response.values():

[wxtim]

Probably over pedantic (and should be a separate commit to any other changes to avoid mixing with code change), but I think it'd be nicer if these tests were in tests/i/network/.

[wxtim]

I've made two suggestions, but neither should block this PR. :)