Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RC 1 draft #21

Merged
merged 21 commits into from
Sep 17, 2021
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f15f4ee
(Draft) GitOps Principles 1.0.0-rc.1
scottrigby Sep 15, 2021
adc0af3
Add context sentence for the principle headers grammar
scottrigby Sep 15, 2021
0d845cc
Remove incomplete summary from introduction. Unecessary now because p…
scottrigby Sep 15, 2021
f1c3094
Move glossary to a standalone file
scottrigby Sep 15, 2021
d2610a0
First attempt at linking terms in principles to glossary items
scottrigby Sep 15, 2021
3626449
Address markdown version / git revision mismatch. Fixes #20
scottrigby Sep 15, 2021
c5fb6f4
Typo fix suggested by @RedbackThomson
scottrigby Sep 15, 2021
cf6b40f
Add note about configuration data excluding other external data
scottrigby Sep 15, 2021
22ce664
Fix formatting of software system glossary item. Clarify one list item
scottrigby Sep 15, 2021
e48298c
glossary item links
scottrigby Sep 17, 2021
94ec893
text format and glossary item links
scottrigby Sep 17, 2021
db1c6d1
glossary item links
scottrigby Sep 17, 2021
20a8430
glossary item links
scottrigby Sep 17, 2021
d3aad38
Broaden user data example to just database contents. Fix formatting
scottrigby Sep 17, 2021
0e357fd
Remove versioned from reconciliation glossary item. Link to other ite…
scottrigby Sep 17, 2021
db8d703
Grammar fixes, and link to other glossary item
scottrigby Sep 17, 2021
a76e8ba
Lowercase formatting
scottrigby Sep 17, 2021
17bfdea
Lowercase formatting
scottrigby Sep 17, 2021
c44ad22
Lowercase formatting, oxford comma
scottrigby Sep 17, 2021
365e68d
Fix case and remove trailing space
scottrigby Sep 17, 2021
f871777
Reviews and prior work attribution
scottrigby Sep 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions GLOSSARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# GitOps Glossary

This glossary accompanies the [GitOps Principles](./PRINCIPLES.md), and other supporting documents in this repository.

- ## Break Glass

The temporary suspension of GitOps principles, often accomplished by pausing automated [reconciliation](#reconciliation).
While these principles apply to typical operations, it may at times be necessary to temporarily pause reconciliation, for example during incident management activities.
In these cases, other modes of operations should be considered (e.g. manual intervention), followed by any necessary updates to the desired state declarations, and finally resuming reconciliation of the system with the updated declarations.
Pragmatic exceptions to these guiding principles are expected from time to time during the journey toward a system being fully managed by GitOps.

- ## Continuous

By "continuous" we adopt the industry standard to mean that [reconciliation](#reconciliation) continues to happen, not that it must be instantaneous.

- ## Declarative Description

Describing the desired state or behavior of a system without specifying how that state will be achieved, thereby separating configuration (the desired state) from the implementation (commands, API calls, scripts etc.) that actually achieves the desired state described in the declarative description.

- ## Desired State

The aggregate of all configuration data for a system form its desired state which is defined as data sufficient to recreate the system so that instances of the system are behaviourally indistinguishable, but do not include the state of any data stored within the system, eg. database contents.

- ## Drift

When a system's actual state changes for any reason other than its versioned [desired state](#desired-state) declarations having changed, we say that the system has drifted from its desired state.

- ## Reconciliation

The process of ensuring that the actual state of a system matches its [desired state](#desired-state) declarations.
Contrary to CIops, any divergence between the two will trigger reconciliation, regardless of where changes occured.
Divergence could be due to the actual state unintentionally [drifting](#drift) from the desired state declarations, or a new desired state declaration version having been changed intentionally.

- ## Software System

We currently understand a software system to include:

- One or more runtime environments consisting of resources under management
- In each runtime, the management agents which act on resources according to security policies
- One or more software repositories for storing deployable artifacts that may be loaded into the runtime environments, eg. configuration files, code, binaries, and packages
- One or more Administrators who are responsible for operating the runtime environments ie. installing, starting, stopping and updating software, code, configuration, etc
- A set of policies controlling access and management of repositories, deployments, runtimes

- ## State Store

A system for storing immutable versions of [Desired State](#desired-state) declarations.
This state store should provide access control and auditing on the changes to the Desired State.
Git, from which GitOps derives its name, is the canonical example used as this state store but any other system that meets these criteria may be used.
In all cases, these state stores must be properly configured and special precautions must be taken to comply with requirements set out in the GitOps Principles.
74 changes: 11 additions & 63 deletions PRINCIPLES.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,22 @@
# GitOps Principles v0.1.0

## Summary
# GitOps Principles {{version}}

GitOps is a set of principles for operating and managing software systems.
These principles are derived from modern software operations, but are also rooted in pre-existing and widely adopted best practices.

When using GitOps, the _Desired State_ of a system or subsystem is defined declaratively as versioned, immutable data, and the running system's configuration is continuously derived from this data.

These principles were derived from modern software operations but are rooted in pre-existing and widely adopted best practices.

## Principles

1. **The principle of declarative desired state**

A system managed by GitOps must have its _Desired State_ expressed declaratively as data in a format writable and readable by both humans and machines.

2. **The principle of immutable desired state versions**

_Desired State_ is stored in a way that supports versioning, immutability of versions, and retains a complete version history.

3. **The principle of continuous state reconciliation**

Software agents continuously, and automatically, compare a system's _Actual State_ to its _Desired State_.
If the actual and desired states differ for any reason, automated actions to reconcile them are initiated.

4. **The principle of operations through declaration**

The only mechanism through which the system is intentionally operated on is through these principles.

## Glossary

- ### Break Glass

The temporary suspension of GitOps principles, often accomplished by pausing automated _Reconciliation_.
While these principles apply to typical operations, it may at times be necessary to temporarily pause reconciliation, for example during incident management activities.
In these cases, other modes of operations should be considered (e.g. manual intervention), followed by any necessary updates to the desired state declarations, and finally resuming reconciliation of the system with the updated declarations.
Pragmatic exceptions to these guiding principles are expected from time to time during the journey toward a system being fully managed by GitOps.

- ### Continuous

By "continuous" we adopt the industry standard to mean that _Reconciliation_ continues to happen, not that it must be instantaneous.

- ### Declarative Description

Describing the desired state or behavior of a system without specifying how that state will be achieved, thereby separating configuration (the desired state) from the implementation (commands, API calls, scripts etc.) that actually achieves the desired state described in the declarative description.

- ### Desired State

The aggregate of all configuration data for a system form its _Desired State_ which is defined as data sufficient to recreate the system so that instances of the system are behaviourally indistinguishable.
The [desired state](./GLOSSARY.md#desired-state) of a GitOps managed system must be:

- ### Drift
1. **Declarative**

When a system's _Actual State_ changes for any reason other than its versioned _Desired State_ declarations having changed, we say that the system has drifted from its _Desired State_.
A [system](./GLOSSARY.md#software-system) managed by GitOps must have its desired state expressed [declaratively](./GLOSSARY.md#declarative-description).

- ### Reconciliation
2. **Versioned and Immutable**

The process of ensuring that the _Actual State_ of a sytem matches its versioned _Desired State_ declarations.
Contrary to CIops, any divergence between the two will trigger reconciliation, regardless of where changes occured.
Divergence could be due to the actual state unintentionally _Drifting_ from the desired state declarations, or a new desired state declaration version having been changed intentionally.
Desired state is [stored](./GLOSSARY.md#state-store) in a way that enforces immutability, versioning and retains a complete version history.

- ### Software System
3. **Pulled Automatically**

One or more Runtime environments consisting of resources under management.
In each Runtime, management Agents to act on resources according to security policies.
One or more software Repositories for storing deployable artifacts that may be loaded into the runtime environments, eg. configuration files, code, binaries and packages.
One or more Administrators who are responsible for operating the runtime environments ie. installing, starting, stopping and updating software, code, configuration, etc.
A set of policies controlling access and management of repositories, deployments, runtimes.
Software agents automatically pull the desired state declarations from the source.

- ### State Store
4. **Continuously Reconciled**

A system for storing immutable versions of _Desired State_ declarations.
This state store should provide access control and auditing on the changes to the Desired State.
Git is the canonical example used as this State Store, and where GitOps derived its name, but but any other system that meets this criteria may be used.
In all cases these must be properly configured, and special precautions must be taken to comply with requirements set out in the GitOps Principles.
Software agents [continuously](./GLOSSARY.md#continuous) observe actual system state and [attempt to apply](./GLOSSARY.md#reconciliation) the desired state.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The attempt to apply might not communicate the imperative need for the software agent to actually work towards the reconciliation.

What about:
...observe actual system state and on drift reconcile towards the desired state.

Copy link
Member Author

@scottrigby scottrigby Sep 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to hear what you think about this section of the PR description above: "items left out of this PR" > "existing principal 5" > "5. Operated in a closed loop"? This is the principle that addresses a feedback loop based on configurations in the desired state and the output of attempts to apply to the actual state.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thinking about it I think I'm more in favor of attempt to apply which can mean a lot. We could potentially add more to the glossary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like this is not a blocker for RC 1. @williamcaban let's continue to discuss this during the process when we send out RC 1 for feedback for folks. Most likely that will be in the form of a GitHub Discussion ❤️

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern with attempt to apply is that it could open the interpretation that a series of remediations were attempted (lets say remediation A & B) and even if not successful it can attempt additional ones (eg. remediations C & D) which might lead to a situation of unrecoverable drift state (which brings us back to the snowflakes configurations results and desired state vs running state open to divergence)

How can we phrase it in a way that an attempt is not enough, but that it must continue trying remediate?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WRT leaving principle 5, agree as it might be covered by principle 3. At least I have not found any extremely compelling example not covers by the other principles. The more concise the better.