diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000000..a3ee6a8a5e --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,3 @@ +## Community Code of Conduct + +This project follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dddc350834..bf5ae92600 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -88,9 +88,10 @@ questions: what changed and why. The subject line should feature the what and the body of the commit should describe the why. ``` -ceph: update MON to use rocksdb +aws: add support for replication to RDS -this enables us to remove leveldb from the codebase. +this commit enables fail-over logic for RDS, across multiple zones. it +enables setting up replication between databases across zones. ``` The format can be described more formally as follows: diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000000..a9aaf65af0 --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,133 @@ +# Crossplane Governance + +This document defines governance policies for the Crossplane project. + +## Roles + +Crossplane uses a two-tiered system of maintainer roles: + +* Senior Maintainers + * Have the most experience with the Crossplane project and are expected to have the knowledge and insight to lead the project's growth and improvement + * Represent their organization within the Crossplane community + * Oversee the process for adding new maintainers and provides guidance for the standard maintainers + * Receive **two votes** in the [conflict resolution and voting process](#conflict-resolution-and-voting) described below +* Standard Maintainers + * Have less experience with the Crossplane project than senior maintainers, but are also expected to provide significant value to the project, helping it grow and improve + * Receive **one vote** in the voting process + +## Becoming a maintainer + +The current list of maintainers is published and updated in [OWNERS.md](OWNERS.md). + +### Maintainer Pre-requisites + +To become a maintainer you need to demonstrate the following: + +* A strong commitment to the project + * Participate in design and technical discussions + * Contribute non-trivial pull requests + * Perform code reviews on other's pull requests + * Answer user questions and troubleshoot issues in the field +* Ability to write good solid code +* Ability to collaborate with the team +* Understanding of how the team works (policies, processes for testing and code review, etc.) +* Understanding of the project's code base and coding style + +### Your organization is not yet a maintainer + +* Express interest to the [senior maintainers](OWNERS.md#senior-maintainers) directly that your + organization is interested in becoming a maintainer. Becoming a maintainer generally means that + you are going to be spending substantial time (>25%) on Crossplane for the foreseeable future. You + should have domain expertise and be extremely proficient with Kubernetes and Golang. Ultimately + your goal is to become a senior maintainer that will represent your organization. +* You should likely have already been commenting on pull requests and issues with the intent of solving + user problems and improving the overall quality of the project. +* We will expect you to start contributing increasingly complicated PRs, under the guidance + of the existing senior maintainers. +* We may ask you to do some PRs from our backlog. +* As you gain experience with the code base and our standards, we will ask you to do code reviews + for incoming PRs (i.e., all maintainers are expected to shoulder a proportional share of + community reviews). +* After a period of approximately 2-3 months of working together and making sure we see eye to eye, + the existing senior maintainers will confer and decide whether to grant maintainer status or not. + We make no guarantees on the length of time this will take, but 2-3 months is the approximate + goal. + +### Your organization is currently a maintainer + +* First decide whether your organization really needs more people with maintainer access. Valid + reasons are "blast radius", a large organization that is working on multiple unrelated projects, + etc. +* Contact a senior maintainer for your organization and express interest. +* Start doing PRs and code reviews under the guidance of your senior maintainer. +* After a period of 1-2 months the existing senior maintainers will discuss granting "standard" + maintainer access. +* "Standard" maintainer access can be upgraded to "senior" maintainer access after another 1-2 + months of work and another conference of the existing senior committers. + +## Removing a maintainer + +If a maintainer is no longer interested or cannot perform the maintainer duties listed above, they +should volunteer to be moved to emeritus status. In extreme cases this can also occur by a vote of +the maintainers per the voting process below. + +## Maintainer responsibilities + +* Monitor email aliases. +* Monitor Slack (delayed response is perfectly acceptable). +* Attend the regularly recurring [community meetings](README.md#community-meeting). +* Triage GitHub issues and perform pull request reviews for other maintainers and the community. + The areas of specialization listed in [OWNERS.md](OWNERS.md) can be used to help with routing + an issue/question to the right person. +* During GitHub issue triage, apply all applicable [labels](https://github.com/crossplaneio/crossplane/labels) + to each new issue. Labels are extremely useful for future issue follow up. Which labels to apply + is somewhat subjective so just use your best judgment. A few of the most important labels that are + not self explanatory are: + * **beginner**: Mark any issue that can reasonably be accomplished by a new contributor with + this label. + * **help wanted**: Unless it is immediately obvious that someone is going to work on an issue (and + if so assign it), mark it help wanted. + * **question**: If it's unclear if an issue is immediately actionable, mark it with the + question label. Questions are easy to search for and close out at a later time. Questions + can be promoted to other issue types once it's clear they are actionable (at which point the + question label should be removed). +* Make sure that ongoing PRs are moving forward at the right pace or closing them. +* In general continue to be willing to spend at least 25% of ones time working on Crossplane (~1.25 + business days per week). + +### Approving PRs + +PRs may be merged after receiving at least **1 approval from a maintainer** (either senior or standard) +that is **not the author** of the PR, and preferably from a **different organization** than the PR author. +As complexity of a PR increases, such as design changes or major PRs, the need for an approval from +a different organization also increases. This should be a judgement call from the maintainers, +and it is expected that all maintainers act in good faith to seek approval from a different +organization when appropriate. + +### Github Project Administration + +Maintainers will be added to the Crossplane GitHub organization (if they are not already) and added to +the GitHub Maintainers team. + +After 6 months, **senior** maintainers will be made an "owner" of the Crossplane GitHub organization. + +## Conflict resolution and voting + +As previously mentioned, senior maintainers receive **2 votes** each and standard maintainers +receive 1 vote each. + +In general, we prefer that technical issues and maintainer membership are amicably worked out +between the persons involved. If a dispute cannot be decided independently, the maintainers can be +called in to decide an issue. If the maintainers themselves cannot decide an issue, the issue will +be resolved by voting. The voting process is a simple majority (except for maintainer changes, +which require 2/3 majority as described below) in which each senior maintainer receives two votes +and each standard maintainer receives one vote. + +For formal votes, a specific statement of what is being voted on should be added to the relevant +GitHub issue or PR. Maintainers should indicate their yes/no vote on that issue or PR, and after a +suitable period of time (goal is by 5 business days), the votes will be tallied and the outcome +noted. If any maintainers are unreachable during the voting period, postponing the completion of +the voting process should be considered. + +Additions and removals of maintainers require a **2/3 majority**, while other decisions and changes +require only a simple majority. diff --git a/INSTALL.md b/INSTALL.md index ec5b67bc00..380bd91981 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,7 +1,6 @@ # Building and Installing Crossplane -Crossplane is composed of golang project and can be built directly with standard `golang` tools. We currently support -two different platforms for building: +Crossplane is composed of golang project and can be built directly with standard `golang` tools. We currently support two different platforms for building: * Linux: most modern distros should work although most testing has been done on Ubuntu * Mac: macOS 10.6+ is supported @@ -28,8 +27,7 @@ First ensure that you have the build submodule synced and updated: git submodule sync && git submodule update --init --recursive ``` -You can then build the Crossplane binaries and all container images for the host platform by simply running the -command below. Building in parallel with the `-j` option is recommended. +You can then build the Crossplane binaries and all container images for the host platform by simply running the command below. Building in parallel with the `-j` option is recommended. ``` make -j4 @@ -45,13 +43,11 @@ Official Crossplane builds are done inside a build container. This ensures that > build/run make -j4 ``` -The first run of `build/run` will build the container itself and could take a few -minutes to complete. +The first run of `build/run` will build the container itself and could take a few minutes to complete, but subsequent builds should go much faster. ### Resetting the build container -If you're running the build container on the Mac using Docker for Mac, the build -container will rely on rsync to copy source to and from the host. To reset the build container and it's persistent volumes, you can run the below command. You should not have to do this often unless something is broken or stale with your build container: +If you're running the build container on macOS using Docker for Mac, the build container will rely on rsync to copy source to and from the host. To reset the build container and it's persistent volumes, you can run the below command. You should not have to do this often unless something is broken or stale with your build container: ``` build/reset @@ -130,13 +126,13 @@ systemctl enable update-binfmt.service # Install ## Local -Please refer to [Local Build & Install](/cluster/local/README.md) documentation on how to deploy locally built Crossplane image onto Kubernetes cluster running on Minikube +Please refer to [Local Build & Install](/cluster/local/README.md) documentation on how to deploy locally built Crossplane image onto Kubernetes cluster running on minikube or using docker for mac. # Improving Build Speed ## Image Caching and Pruning -Doing a complete build of Crossplane and the dependent packages can take a long time (more than an hour on a typical macbook). To speed things up we rely heavily on image caching in docker. Docker support content-addressable images by default and we use that effectively when building images. Images are factored to increase reusability across builds. We also tag and timestamp images that should remain in the cache to help future builds. +Doing a complete build of Crossplane and the dependent packages can take a long time. To speed things up we rely heavily on image caching in docker. Docker support content-addressable images by default and we use that effectively when building images. Images are factored to increase reusability across builds. We also tag and timestamp images that should remain in the cache to help future builds. ### Pruning the cache @@ -146,8 +142,12 @@ To prune the number of cached images run `make prune`. There are two options tha - `PRUNE_KEEP` - the minimum number of cached images to keep in the cache. Default is 24 images. ## CI workflow and options -Every PR and every merge to master triggers the CI process in [TBD](http://TBD). -The Jenkins CI will build, run unit tests, run integration tests and Publish artifacts- On every commit to PR and master. -If any of the CI stages fail, then the process is aborted and no artifacts are published. -On every successful build Artifacts are pushed to a [s3 bucket](https://release.crossplane.io/). On every successful master build, -images are uploaded to docker hub in addition. + +Every PR and every merge to master triggers the CI process in [Jenkins](https://jenkinsci.upbound.io/blue). +The Jenkins CI will build, run unit tests, run integration tests and Publish artifacts- On every commit to PR and master. If any of the CI stages fail, then the process is aborted and no artifacts are published. + +On every successful build Artifacts are pushed to a [s3 bucket](https://releases.crossplane.io/). On every successful master build, images are uploaded to docker hub in addition. + +Based on nature of the PR, it may not be required to run full regression or users may want to skip build all together for trivial changes like documentation changes. Based on the PR body text, Jenkins will skip the build or skip some tests: + + * [skip ci] - if this text is found in the body of PR, then Jenkins will skip the build process and accept the commit diff --git a/OWNERS.md b/OWNERS.md new file mode 100644 index 0000000000..10ef7a4ea9 --- /dev/null +++ b/OWNERS.md @@ -0,0 +1,31 @@ +# Crossplane Maintainers + +This page lists all active maintainers and their areas of expertise. This can be used for routing PRs, questions, etc. to the right place. + +* See [CONTRIBUTING.md](CONTRIBUTING.md) for general contribution guidelines. +* See [GOVERNANCE.md](GOVERNANCE.md) for governance guidelines and maintainer responsibilities. + +## Senior maintainers + +Here's the current list of senior maintainers: + +* Bassam Tabbara ([bassam](https://github.com/bassam)) + * Catch-all, architecture, strategy, build and packaging +* Illya Chekrygin ([ichekrygin](https://github.com/ichekrygin)) + * Core, Controllers, GCP, AWS, Azure +* Jared Watts ([jbw976](https://github.com/jbw976)) + * Core, Controllers, GCP, AWS, Azure + +## Maintainers + +There are currently no maintainers. Maintainers will be added according to the [process for adding a maintainer](GOVERNANCE.md#becoming-a-maintainer) in the [governance](GOVERNANCE.md). + +## Emeritus maintainers + +There are currently no emeritus maintainers. This list will be updated according to the [process for removing a maintainer](GOVERNANCE.md#removing-a-maintainer) in the [governance](GOVERNANCE.md). + +## Friends of Crossplane + +This section lists people that are not currently maintainers but have demonstrated value to the community. +They typically help out with technical, code and design reviews and also with support and troubleshooting. +Feel free to loop them in as needed. diff --git a/README.md b/README.md index 32e746d8b8..c94ae6a4cc 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,48 @@ -# Project Crossplane (codename) +

Crossplane

-## What is Crossplane? +[![Build Status](https://jenkinsci.upbound.io/buildStatus/icon?job=crossplane/build/master)](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplane%2Fbuild/activity) +[![GitHub release](https://img.shields.io/github/release/crossplane/crossplane/all.svg?style=flat-square)](https://github.com/crossplaneio/crossplane/releases) +[![Docker Pulls](https://img.shields.io/docker/pulls/crossplane/crossplane.svg)](https://img.shields.io/docker/pulls/crossplane/crossplane.svg) +[![Go Report Card](https://goreportcard.com/badge/github.com/crossplaneio/crossplane)](https://goreportcard.com/report/github.com/crossplaneio/crossplane) +[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fcrossplaneio%2Fcrossplane.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fcrossplaneio%2Fcrossplane?ref=badge_shield) +[![Slack](https://crossplane-slackin.herokuapp.com/badge.svg)](https://crossplane-slackin.herokuapp.com/badge.svg) +[![Twitter Follow](https://img.shields.io/twitter/follow/crossplane_io.svg?style=social&label=Follow)](https://twitter.com/intent/follow?screen_name=crossplane_io&user_id=788180534543339520) -Crossplane is an open source **external-resource-definition** for Kubernetes , providing the platform, framework, and support for a diverse set of the managed resources offered by major cloud providers (Currently focused on AWS and GCP) +## Overview -Crossplane turns storage software into self-managing, self-scaling, and self-healing of managed cloud resources. Crossplane extends the facilities provided by Kubernetes such container management, scheduling and orchestration to the external resources. +Crossplane is an open source multicloud control plane. It introduces workload and resource abstractions on-top of existing managed services that enables a high degree of workload portability across cloud providers. A single crossplane enables the provisioning and full-lifecycle management of services and infrastructure across a wide range of providers, offerings, vendors, regions, and clusters. Crossplane offers a universal API for cloud computing, a workload scheduler, and a set of smart controllers that can automate work across clouds. -Crossplane integrates deeply into cloud native environments leveraging extension points and providing a seamless experience for scheduling, lifecycle management, resource management, security, monitoring, and user experience. +

Crossplane

-For more details about the cloud providers and resources currently supported by Crossplane, please refer to the [project status section](#project-status) below. -We plan to continue adding support for other cloud providers and resource based on community demand and engagement in future releases. See our [roadmap](ROADMAP.md) for more details. +Crossplane presents a declarative management style API that covers a wide range of portable abstractions including databases, message queues, buckets, data pipelines, serverless, clusters, and many more coming. It’s based on the declarative resource model of the popular [Kubernetes](https://github.com/kubernetes/kubernetes) project, and applies many of the lessons learned in container orchestration to multicloud workload and resource orchestration. + +Crossplane supports a clean separation of concerns between developers and administrators. Developers define workloads without having to worry about implementation details, environment constraints, and policies. Administrators can define environment specifics, and policies. The separation of concern leads to a higher degree of reusability and reduces complexity. + +Crossplane includes a workload scheduler that can factor a number of criteria including capabilities, availability, reliability, cost, regions, and performance while deploying workloads and their resources. The scheduler works alongside specialized resource controllers to ensure policies set by administrators are honored. + +For a deeper dive into Crossplane, see the [architecture](https://docs.google.com/document/d/1whncqdUeU2cATGEJhHvzXWC9xdK29Er45NJeoemxebo/edit?usp=sharing) document. + +## Getting Started and Documentation + +For getting started guides, installation, deployment, and administration, see our [Documentation](https://crossplane.io/docs). ## Contributing -We welcome contributions. See [Contributing](CONTRIBUTING.md) to get started. +Crossplane is a community driven project and we welcome contributions. See [Contributing](CONTRIBUTING.md) to get started. ## Report a Bug For filing bugs, suggesting improvements, or requesting new features, please open an [issue](https://github.com/crossplaneio/crossplane/issues). +## Contact + +Please use the following to reach members of the community: + +- Slack: Join our [slack channel](https://slack.crossplane.io) +- Forums: [crossplane-dev](https://groups.google.com/forum/#!forum/crossplane-dev) +- Twitter: [@crossplane_io](https://twitter.com/crossplane_io) +- Email: [info@crossplane.io](mailto:info@crossplane.io) + ## Community Meeting A regular community meeting takes place every other [Tuesday at 9:00 AM PT (Pacific Time)](https://zoom.us/j/425148449). @@ -34,19 +58,30 @@ Anyone who wants to discuss the direction of the project, design and implementat ## Project Status -The status of each storage provider supported by Crossplane can be found in the table below. -Each API group is assigned its own individual status to reflect their varying maturity and stability. -More details about API versioning and status in Kubernetes can be found on the Kubernetes [API versioning page](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning), but the key difference between the statuses are summarized below: +The project is an early preview. We realize that it's going to take a village to arrive at the vision of a multicloud control plane, and we wanted to open this up early to get your help and feedback. Please see the [Roadmap](ROADMAP.md) for details on what we are planning for future releases. + +### API Status + +Each API supported by Crossplane is assigned its own individual status to reflect the varying maturity and stability. More details about API versioning and status in Kubernetes can be found on the Kubernetes [API versioning page](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning), but the key difference between the statuses are summarized below: * **Alpha:** The API may change in incompatible ways in a later software release without notice, recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support. * **Beta:** Support for the overall features will not be dropped, though details may change. Support for upgrading or migrating between versions will be provided, either through automation or manual steps. * **Stable:** Features will appear in released software for many subsequent versions and support for upgrading between versions will be provided with software automation in the vast majority of scenarios. -| Name | Details | API Group | Status | -| ----- | --------- | ----------- | -------- | -| AWS Database | Database storage services in AWS | database.aws.crossplane.io/v1alpha1 | Alpha | -| GCP Database | Database storage services in GCP | database.gcp.crossplane.io/v1alpha1 | Alpha | +| Cloud | Name | Details | API Group | Status | +| ----- | ----- | --------- | ----------- | -------- | +| All | Compute | Compute services | compute.crossplane.io/v1alpha1 | Alpha | +| All | Storage | Storage services | storage.crossplane.io/v1alpha1 | Alpha | +| AWS | Compute | Compute services | compute.aws.crossplane.io/v1alpha1 | Alpha | +| AWS | Database | Database services | database.aws.crossplane.io/v1alpha1 | Alpha | +| AWS | Storage | Storage services | storage.aws.crossplane.io/v1alpha1 | Alpha | +| Azure | Compute | Compute services | compute.azure.crossplane.io/v1alpha1 | Alpha | +| Azure | Database | Database services | database.azure.crossplane.io/v1alpha1 | Alpha | +| Azure | Storage | Storage services | storage.azure.crossplane.io/v1alpha1 | Alpha | +| GCP | Compute | Compute services | compute.gcp.crossplane.io/v1alpha1 | Alpha | +| GCP | Database | Database services | database.gcp.crossplane.io/v1alpha1 | Alpha | +| GCP | Storage | Storage services | storage.gcp.crossplane.io/v1alpha1 | Alpha | ### Official Releases diff --git a/ROADMAP.md b/ROADMAP.md index 9e985cd842..e33edea168 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,29 +1,35 @@ # Roadmap -This document defines a high level roadmap for Crossplane development and upcoming releases. -The features and themes included in each milestone are optimistic in the sense that many do not have clear owners yet. -Community and contributor involvement is vital for successfully implementing all desired items for each release. -We hope that the items listed below will inspire further engagement from the community to keep Crossplane progressing and shipping exciting and valuable features. +This document defines a high level roadmap for Crossplane development and upcoming releases. Community and contributor involvement is vital for successfully implementing all desired items for each release. We hope that the items listed below will inspire further engagement from the community to keep Crossplane progressing and shipping exciting and valuable features. -Any dates listed below and the specific issues that will ship in a given milestone are subject to change but should give a general idea of what we are planning. -We use the [milestone](https://github.com/crossplaneio/crossplane/milestones) feature in Github so look there for the most up-to-date and issue plan. +Any dates listed below and the specific issues that will ship in a given milestone are subject to change but should give a general idea of what we are planning. We use the [milestone](https://github.com/crossplaneio/crossplane/milestones) feature in Github so look there for the most up-to-date and issue plan. -## v0.1 +## v0.1 - Proof of Concept -* MySQL support for AWS, GCP, and Azure +* Resource Claims, Resource Classes, and Resources +* Basic Container Workload + * Support for Deployments / Services + * Resource Usage and Secret management +* Cloud Providers * Provider CRDs, credentials management, API/SDK consumption - * Provider specific MySQL CRDs (Amazon RDS, Google Cloud SQL, Microsoft Azure Database for MySQL) - * All 3 big cloud providers will be supported for all resources going forward -* PostgreSQL support for AWS, GCP, and Azure - * same work items as MySQL support -* Controller depth and reliability - * Full CRUD support for all resources (robust lifecycle management) + * AWS, GCP, and Azure +* Managed Kubernetes Clusters + * Support for EKS, AKS and GKE + * Generic Kubernetes Cluster Resource Claim + * Status and Conditions for Clusters + * Static and Dynamic Provisioning +* MySQL Support + * Static and Dynamic Provisioning + * Provider specific MySQL CRDs (AWS RDS, GCP CloudSQL, Azure MySQL) + * Connection strings and firewall support +* Resource Controller depth and reliability + * CRUD support and robust lifecycle management * CRD status Conditions for status of resources * Event recording * Normalized logging using single logging solution (with configurable levels) * Retry/recovery from failure, idempotence, dealing with partial state * CI builds/tests/releases - * New isolated jenkins instance (similar to Rook's jenkins) + * New jenkins instance (similar to Rook's jenkins) * Developer unit testing with high code coverage * Integration testing pipeline * Artifact publishing (container images, crossplane helm chart, etc.) @@ -31,18 +37,28 @@ We use the [milestone](https://github.com/crossplaneio/crossplane/milestones) fe * User guides, quick-starts, walkthroughs * Godocs developer docs for source code/packages/libraries * Open source project management - * [CII best practices checklist](https://bestpractices.coreinfrastructure.org/en/projects/1599#) * Governance * Contributor License Agreement (CLA) or Developer Certificate of Origin (DCO) -## v0.2 +## v0.2 - Run Real-world Applications -* Support for other SockShop resources +* Support for a few realworld applications on-top of Crossplane + * GitLab and others coming +* Workload Scheduling + * Region and cloud provider aware scheduling + * Delayed binding of resources to support co-location in same region +* Resource Pool and Auto-Scalers + * Support for automatically creating Kubernetes Clusters +* New Stateful managed services across AWS, Azure, and GCP + * PostgresSQL * MongoDB * Redis - * RabbitMQ -* Support for Clusters, deploy and manage clusters lifecycle via CRDs -* Federation, deploy resources to target clusters (possibly a new project) + * ActiveMQ + * Memcached +* Managed Services using the Kubernetes Operator Pattern + * Support for running Managed Services based on the Operator pattern + * Early support for Rook and others + * Seamless integration into the Resource Claim and Resource Class model * Performance and Efficiency * 2-way reconciliation with external resources * Events/notifications from cloud provider on changes to external resources to trigger reconciliation