Define GitHub Actions and Workflows according to the convention

[vikman90]

Description

We need to define and implement GitHub Actions and Workflows following the established naming and usage conventions. This includes setting up Actions for various use cases and designing corresponding Workflows to ensure consistency and efficiency in our CI/CD processes.

Convention

Actions

Naming convention

• Major: Product major version.
• Prefix: Prefixes depicted in the use cases section.
• Target: The Action target. It can be a component, module, subsystem, programming language, code analysis tool, etc.

Actions use the following naming convention:

<major>_<prefix>_<target>

Use cases

Actions are meant to cover the following cases:

• Code analysis. Every tool used to do code analysis (either static or dynamic).
  Action name prefix: codeanalysis
  Available targets: code analysis tool.
  Name example: 4_codeanalysis_coverity
• Linter. Programming language linter or automatic documentation generation.
  Action name prefix: codelinter
  Available targets: linter.
  Name example: 5_codelinter_clangformat
• Unit tests.
  Action name prefix: testunit
  Available targets: module
  Name example: 5_testunit_engine
• Component tests.
  Action name prefix: testcomponent
  Available targets: component/module
  Name example: 5_testcomponent_indexerconnector
• Integration tests.
  Action name prefix: testintegration
  Available targets: module
  Name example: 4_testintegration_cluster
• Package builder. Subsystem package builder.
  Action name prefix: builderpackage
  Available targets: subsystem
  Name example: 4_builderpackage_server
• Precompiled object builder. Any needed precompiled object.
  Action name prefix: builderprecompiled
  Available targets: subsystem
  Name example: 5_builderprecompiled_agent

Jobs

• A job step must be solely comprised of Actions.
• A job step cannot contain Actions with different prefixes.
• A job step must use matrices whenever possible.

Workflows

Pull Request

PR workflows are run against protected branches. They should not last longer than 30m.

PR workflows are meant to cover the following use cases:

• Code quality. All available codeanalysis and linter type of Actions in that repository.
  Workflow name prefix: codequality
  Available targets: repository
  Name example: 4_codequality
  Trigger: Any change.
• Unit tests.
  Workflow name prefix: testunit
  Available targets: component/module
  Name example: 5_testunit_engine
  Trigger: Any change within the target component/module.
• Component tests.
  Workflow name prefix: testcomponent
  Available targets: component/module
  Name example: 5_testcomponent_comms-api
  Trigger: Any change within the target module.
• Integration tests.
  Workflow name prefix: testintegration
  Available targets: module
  Name example: 5_testintegration_management-api
  Trigger: Any change within the target module.
• Packages.
  Workflow name prefix: builderpackage
  Available targets: subsystem
  Name example: 5_builderpackage_agent
  Trigger: Any code change.

Manual (workflow dispatch)

• Precompiled.
  Workflow name prefix: builderprecompiled
  Available targets: subsystem
  Name example: 5_builderprecompiled_dashboard
  Trigger: Any change requiring new precompilation (mostly related to dependencies changes).

Tasks

• Define Actions according to the specification:
  • Ensure the naming convention is followed (<major>_<prefix>_<target>).
  • Categorize Actions into the appropriate use cases (code analysis, linters, tests, builders).
  • Ensure that job steps only contain Actions with the same prefix and use matrices whenever possible.
• Define Workflows according to the specification:
  • Use the correct workflow naming prefix based on its purpose.
  • Define appropriate triggers for each workflow (PR workflows or manual dispatch).
  • Ensure PR workflows do not exceed the 30-minute execution limit.

Definition of Done

• All required Actions are defined and adhere to the naming convention.
• Workflows are implemented and correctly triggered based on repository changes.
• Job steps within workflows are structured correctly, following the prefix restrictions and using matrices where applicable.
• PR workflows are optimized to complete within the 30-minute limit.

Plan

For each Wazuh Indexer repository:

• List the existing workflows.
• Describe its usage.
• Write down if they are from OpenSearch or created by us.
• Propose what to do with OpenSearch's workflows: adapt and keep or remove (default).
• Fit the workflows we keep (including these created by us) on one of categories described on this issue.
• Rename the workflows according to the naming convention defined in this issue.
• Configure the workflows to run under the conditions defined in this issue.

[abbonno]

• List the existing workflows.
• Describe its usage.
• Write down if they are from OpenSearch or created by us.
• Propose what to do with OpenSearch's workflows: adapt and keep or remove (default).

Workflow list

Wazuh indexer reporting

Workflow	Author	When	Action	Decission
backport.yml	Opensearch	It runs when merging	It runs VachaShah/backport action to backport changes to other branches	Delete
build.yml	Wazuh	It runs manually or is called by others	Builds the Wazuh indexer packages	Keep
ci.yml	Wazuh	It runs when a pull is made on branches named: ci/* or migrate-*	Responsible for running all tests, calls build	Delete, redundant use of build
delete_backport_branch.yml	Opensearch	It runs when a pull request is closed	Deletes the used backport branch	Delete
draft-release-notes-workflow.yml	Opensearch	It runs when there is a push on the main branch	Updates the release notes	Delete
link-checker.yml	Opensearch	It runs when there is a push on the main branch or when a pull request is created or updated to the main branch	Checks for errors in links	Keep
maven-publish.yml	Opensearch	It runs when there is a push on the main, 1.3, and 2.x branches	Configures and publishes snapshots to a Maven repository using Gradle	Delete
reports-scheduler-test-and-build-workflow.yml	Opensearch	It runs when there is a push or pull	Gets the CI image tag, then builds and tests the project on Linux, Windows, and macOS. After that it Runs compatibility and code coverage tests and finally uploads build artifacts.	Delete or Adapt

Wazuh indexer

Workflow	Author	When	Action	Decission
assemble.yml	Opensearch	It runs when a pull is made	Only runs in the OpenSearch repository, but it sets up different Java environments and runs the Gradle assemble task for pull requests on various operating systems	Delete or Adapt
auto-release.yml	Opensearch	It activates when there is a push on any tag	Automates the release process for new Opensearch versions	Delete
backport.yml	Opensearch	It runs when merging	It runs VachaShah/backport action to backport changes to other branches	Delete
build-push-docker-image.yml	Wazuh	It runs when: manually run or invoked	Is used to build and push a Docker image for the Wazuh Indexer	Keep
build.yml	Wazuh	It runs when: manually run or invoked	It is used to build Wazuh indexer packages	Keep
chanelog_verifier.yml	Opensearch	It runs when a pull request is done with one of this types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]	Enforces the update of a changelog file on every pull request	Delete
ci.yml	Wazuh	It runs when a pull is made on branches named: ci/* or migrate-*	Responsible for running all tests, calls build	Keep
codeql.yml	Wazuh	It is scheduled to run	Is used to run CodeQL analysis on the repository to identify vulnerabilities and errors in the code	Keep
copy-linked-issue-labels.yml	Opensearch	It runs when a pull request is done with one of this types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]	Checks if the repository is OpenSearch repo and then uses the michalvankodev/[email protected] action to copy labels from linked issues	Delete
create-documentation-issue.yml	Opensearch	Runs when a pull request is labeled with needs-documentation	Creates an issue in the opensearch-project/documentation-website repository to add related documentation.	Delete
dco.yml	Opensearch	It triggers on pull requests	Retrieves the pull request commits using and checks the Developer Certificate of Origin (DCO)	Delete
delete_backport_branch.yml	Opensearch	It triggers when a pull request is closed in Opensearch repo	Delete merged branches of backport pull requests	Delete
dependabot_pr.yml	Opensearch	It triggers on pull requests	Automates Dependabot pull request	Delete
detect-breaking-change.yml	Opensearch	It triggers on pull requests	The workflow detects breaking changes in Java code during pull requests by running a Gradle task and uploading a compatibility report if failures occur	Delete
links.yml	Opensearch	It is scheduled	It verifies the validity of links in the repository	Adapt
lucene-snapshots.yml	Opensearch	It's manually triggered	Checks out, builds, and publishes snapshots of the Lucene project	Delete
poc-checklist.yml	Opensearch	Triggered when an issue is labeled with "poc"	The workflow adds a comment to the issue containing a POC (Proof of Concept) checklist	Delete
precommit.yml	OpenSearch	Runs on pull request	Precommit changes (validates code before the commit is made)	Delete
publish-maven-snapshots.yml	OpenSearch	Runs manually or when a main or version branch is pushed	Builds and publishes snapshots to Maven	Delete
version.yml	OpenSearch	Runs when a version ('..*') tag is pushed	Automatically changes the version and generates the needed files and pull requests	Delete
version_check.yml	Wazuh	Runs when VERSION file is pushed	Checks the version using build-scripts/check-version.sh	Might not be needed
wrapper.yml	OpenSearch	Runs on pull request	Executes the gradle/wrapper-validation-action@v3 action	Delete

Wazuh indexer plugins

Workflow	Author	When	Action	Decission
build.yml	Wazuh	It runs manually or is called by others	Builds the plugin packages	Delete, redundant use of build
build_on_push.yml	Wazuh	It runs when branches named after ci/* are pushed	Executes build.yml	Keep
ci.yml	Wazuh	Runs when branches named after " ci/* " and "migrate-* " are pushed	Executes build.yml and test.yml	Keep
deploy_docs.yml	Wazuh	Runs when main branch is pushed	Deploys the documents of the branch to GitHub pages	Keep
generate_ecs_mappings.yml	Wazuh	Runs when ecs/**/*.json or .yml documents are pushed	Generates ecs-templates based on the modified files with generate-and-push-templates.sh script	Keep
test.yml	Wazuh	Runs when called by ci.yml or when plugins/**/*.java or .gradle files are pulled	Runs tests for modified files	Keep
build.yml	Wazuh	It runs manually or is called by others	Builds the plugin packages	Keep

[abbonno]

• Fit the workflows we keep (including these created by us) on one of categories described on this issue.

Category

Wazuh indexer reporting

Workflow	Category	Need changes
build.yml	Package builder	No
ci.yml	Integration test	No
link-checker.yml	Code quality	No

Wazuh indexer

Workflow	Category	Need changes
build-push-docker-image.yml	Package builder	No
build.yml	Package builder	No
ci.yml	Integration test	No
codeql.yml	Code Quality	No
links.yml	Code Quality	Yes

Wazuh indexer plugins

Workflow	Category	Need changes
build.yml	Package builder	No
build-on-push.yml	Package builder	No
ci.yml	Integration test	No
deploy_docs.yml	Package builder	No
generate_ecs_mappings.yml	Package Builder	No
test.yml	Integration test	No

[abbonno]

• Rename the workflows according to the naming convention defined in this issue.

Rename

Wazuh indexer reporting

• build.yml ---> 5_builderpackage_wazuh-indexer-reporting
• ci.yml ---> 5_testintegration_ci
• link-checker.yml ---> 5_codequality_links

Wazuh indexer

• build-push-docker-image.yml ---> 5_builderpackage_pushdockerimage
• build.yml ---> 5_builderpackage_wazuh-indexer
• ci.yml ---> 5_testintegration_ci
• codeql.yml ---> 5_codequality_codeql
• links.yml ---> 5_codequality_links
• version_check.yml ---> 5_testunit_version

Wazuh indexer plugins

• build.yml ---> 5_builderpackage_wazuh-indexer-plugins
• build-on-push.yml ---> 5_builderpackage_onpush
• ci.yml ---> 5_testintegration_ci
• deploy_docs.yml ---> 5_builderpackage_deploydocs
• generate_ecs_mappings.yml ---> 5_builderpackage_ecsmapping
• test.yml ---> 5_testintegration_gradlecheck

[f-galland]

@abbonno, @Jorgesnchz I've reviewed the file changes and they seem ok.
Let's meet tomorrow and go through your test runs in your forked repos just to double check nothing is broken.