Detect cluster version mismatches

[agoddard]

Feature Suggestion

In rare cases where a user is unable to upgrade all backends at the same time, such as due to a network or config management failure, a sensu-backed cluster might end up with mismatched sensu-backend version members. Depending on the version mismatch, this could cause unexpected results w/ the API, backend, or Web UI.

Possible Implementation

Perhaps it would suffice for backends to write an error to their logs if they detect a member of their cluster with a version mismatch. If this could be raised to the /health endpoint or somewhere similar, the Web UI might also be able to catch this state and display an error

Context

We saw a situation where mismatched clusters resulted in an error in the UI when the UI was viewed via a load balancer, presumably because a version of the web UI requested web assets from a backend which didn't have the correct assets, this resulted in a consistent error state and unusable web UI.