A rubric to rate the maturity of open source software projects
The OSS Health and Sustainability rubric was developed with the following in mind:
- Measure the current health and maturity of an open source project
- Identify areas that need improvements
- Provide a path to success and sustainability for new and existing open source projects
- Set clear indicators that define levels of product growth and maturation
This project was built by identifying success criteria across communities of practice. Measureable practices were researched from the following projects: CHAOSS Community Metrics, Red Hat Open Source Health Checklist, Open Source Maturity Model, GitHub Insights, and the Apereo Foundation Incubation Process.
The metric criteria used to rate the health included 5 areas of concentration:
- Community: maturity, openness, diversity, code of conduct, responsiveness
- Governance: management, documentation, roadmaps, resolution practices, funding
- Development: development activity, license agreements, integrations, standards, accessibility
- Support: documentation, backwards compatibility, mobile friendly, services and support
- Security / Privacy: security review and privacy policy
Over 40 performance criteria, are grouped into the 5 areas of concentration and then wrapped into a total score. Each criteria has a metric to determine level of project maturity as described above. The scoring was determined through research to define rating level indicators.
Performance criteria scored on whether the task exists or based on the maturity level.
Examples of how tasks are scored depends on the following factors
- Number of years in practice
- Number of implementations
- Consistency of product releases
- Number of unique contributors
- Response time in the community (developers, users, etc..)
- Code of Conduct available
Based on the total calculated score from the 40 tasks, the project is rated by maturity level.
- Emerging: A relatively new project and/or a project with multiple areas to improve before becoming "Established".
- Established: A project that has met criteria related to contributors and stability. There is room for improvement, but there’s a viable community that continues to have momentum.
- Mature: A project that has met most criteria and has a well established community, governance, and funding model is considered “Mature”. An active project that continues to have contributions and significant adoptions in its market.
In testing the rubric tool, we surfaced areas that could be improved, such as:
- Data dictionary for each topic
- More example assessments done on a variety of OSS projects
- Generalize topic options that are too specific
- Add more topics in the Security / Privacy category
- Add Discoverability topic to Community
- Add Fame / Reputation topic to Community
- Add Test Code Coverage topic to Development
This work is licensed under a Creative Commons Attribution 4.0 International License.
