Soft-deletes not respected by connector framework logic

[jedrazb]

Bug Description

Connectors defined in the service config.yml can be soft-deleted. Since the logic for fetching/listing connectors is not yet using APIs we might still show deleted connectors as ready do sync.

To Reproduce

Steps to reproduce the behavior:

1. Add connector to config yml
2. Start service
3. Delete the connector in Kibana UI (or DELETE _connector/{id} with api)
4. Connector service still reports this connector as valid, e.g. in periodic log lines ("connector {} is not configured yet ....")

Expected behavior

Soft deleted connector should be treated as deleted connector.

Screenshots

Suggested workaround

Quick fix:

• Listing connectors defined in the service config, add filter by deleted true flag in here
• Handle deleted flag in connector_index.fetch_by_id

Long term fix:

• Switch to Connector APIs