Connectors CLI tool

[vidok]

Part of https://github.com/elastic/enterprise-search-team/issues/6245

This PR will introduce a new connector CLI.

Motivation

This tool aims to streamline data management processes, providing engineers with a faster and more efficient way to work with Connectors and Search indices, ultimately enhancing their productivity and workflow.

Usage

Installation

Before you start using you need to regenerate scripts by running make clean install

Print help page

Run ./bin/connectors --help to get detailed information about the tool.

Usage: connectors [OPTIONS] COMMAND [ARGS]...

Options:
  -v, --version          Show the version and exit.
  -c, --config FILENAME
  --help                 Show this message and exit.

Commands:
  connector  Connectors management
  index      Search indices management
  job        Sync jobs management
  login      Authenticate Connectors CLI with an Elasticsearch instance

You can also print a help page for specific commands e.g. ./bin/connectors job --help

Connecting to an Elasticsearch instance

To start using the tool you need to authenticate with your Elasticsearch instance. Run ./bin/connector login and provide credentials. The result of the successful command is a file in .ci/config.yml. It's up to you where to store this file, however, you need to specify it using ./bin/connectors -c path_to_your_configuration command.

Known limitations

• Supports only connectors-clients
• Does not support API keys for authentication
• Does not support all Kibana UI functionality

Checklists

Pre-Review Checklist

• this PR has a meaningful title
• this PR links to all relevant github issues that it fixes or partially addresses
• if there is no GH issue, please create it. Each PR should have a link to an issue
• this PR has a thorough description
• Covered the changes with automated tests
• Tested the changes locally
• Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)
• Considered corresponding documentation changes
• Contributed any configuration settings changes to the configuration reference

Changes Requiring Extra Attention

• Security-related changes (encryption, TLS, SSRF, etc)
• New external service dependencies added.

Release Note

We have added a new Connectors CLI (technical preview). The CLI is designed to streamline and automate the management of Elastic Connectors. You can use it for scripting and automation in your own infrastructure as it can connect to any Elasticsearch instance.

Known limitations:

• Does not support API keys for authentication
• Does not support sync job scheduling configuration

[artem-shelkovnikov]

CI seems to be red due to failing tests :/

I'll start the review of actual code, cause it seems like a problem in tests, but won't put approval cause it'll be dismissed anyway for now :)

[artem-shelkovnikov]

Amazing PR! I left a couple questions/nits/suggestions, but I love the direction of it and what it allows us to do

[navarone-feekery]

🔥

[github-actions]

💔 Failed to create backport PR(s)

The backport operation could not be completed due to the following error:
There are no branches to backport to. Aborting.

The backport PRs will be merged automatically after passing CI.

To backport manually run:
backport --pr 1616 --autoMerge --autoMergeMethod squash