Improve efficiency of shard store delete checks

[fcofdez]

The IndicesStore class acts as a cluster state listener to monitor changes in the routing table and determine if shard data on the local node can be safely deleted. To achieve this, it broadcasts a message to ensure the shard data is no longer needed. However, this approach has a significant drawback: if the routing table frequently changes and nodes take time to respond, the local node may repeatedly send the same ShardActiveRequest.

This behavior becomes particularly problematic in large clusters, especially during node shutdowns, as it leads to unnecessary work and potential inefficiencies. To address this, we should introduce a deduplication mechanism to avoid redundant ShardActiveRequest messages and reduce wasted effort.

[DaveCTurner]

The ShardActiveRequest mechanism was introduced in v1.3, at which point ClusterState updates were very much a best-effort thing and nowhere near reliable enough to use as a source of truth for data deletion decisions. These days it seems unnecessary to me. Rather than adding complexity in this area, we should consider whether to drop the ShardActiveRequest mechanism entirely and trust that the RoutingTable is accurate.