Skip to content

Commit

Permalink
refresh contributor instructions (#1417)
Browse files Browse the repository at this point in the history
  • Loading branch information
pnadolny13 authored Jul 7, 2023
1 parent 87c0a7e commit b0e7f8d
Show file tree
Hide file tree
Showing 7 changed files with 130 additions and 384 deletions.
86 changes: 86 additions & 0 deletions .github/ISSUE_TEMPLATE/new_plugin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: 🚰 New Plugin
description: Request a new Plugin to Meltano Hub
title: "Add Plugin: <insert plugin name>"
labels: []
assignees: [taylormurphy, pnadolny13]

body:
- type: markdown
attributes:
value: |
## New Plugin Request
Please fill out the following information to request a new plugin to be added to the Meltano Hub.
If you'd like you can follow the steps in the [CONTRIBUTING.md](https://github.com/meltano/hub/blob/main/CONTRIBUTING.md) to add it yourself,
otherwise someone on the Meltano team will get it added.
- type: dropdown
id: plugin_type
attributes:
label: What is the plugin type?
multiple: true
options:
- extractor
- loader
- utility
- mapper
- files

- type: input
id: name
attributes:
label: Repository URL
description: The URL for the repository that contains the plugin code
placeholder: ex. https://github.com/MeltanoLabs/tap-github
validations:
required: true

- type: input
id: name
attributes:
label: Extension Repository URL
description: The URL for the plugin extension repository. Only for utility plugins.
placeholder: ex. https://github.com/meltano/airflow-ext
validations:
required: false

- type: input
id: description
attributes:
label: Description
description: General description of what the company behind the API or database does
placeholder: ex. Online Marketing Platform
validations:
required: true

- type: input
id: url
attributes:
label: Domain URL
description: URL of the developer documentation or website
placeholder: ex. https://developers.hubspot.com/docs/api/overview
validations:
required: true

- type: textarea
attributes:
label: Logo Image
description: |
Upload a logo image for the new plugin. The ideal image is an SVG or PNG, has no text over the logo, and has a transparent background.
Sometimes and image already exists, for example if an extractor is listed on the MeltanoHub and you want to add a loader.
In that case, you can leave a note of what existing plugin to use.
Tip: You can attach images by clicking this area to highlight it and then dragging files in.
validations:
required: false

- type: textarea
attributes:
label: Any other notes?
description: |
Links? References? Anything that will give us more context.
validations:
required: false
122 changes: 0 additions & 122 deletions .github/ISSUE_TEMPLATE/new_tap.yml

This file was deleted.

46 changes: 0 additions & 46 deletions .github/PULL_REQUEST_TEMPLATE.md

This file was deleted.

67 changes: 41 additions & 26 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,52 @@ Let's build together! Please see our [Contributor Guide](https://docs.meltano.co
for more information on contributing to Meltano.

We believe that everyone can contribute and we welcome all contributions.
If you're not sure what to work on, here are some [ideas to get you started](https://github.com/meltano/meltano/issues?q=is%3Aissue+is%3Aopen+label%3A%22accepting+merge+requests%22+).
If you're not sure what to work on, here are some [ideas to get you started](https://github.com/meltano/hub/issues?q=is%3Aissue+is%3Aopen+label%3A%22Accepting+Pull+Requests%22)

Chat with us in [#contributing](https://meltano.slack.com/archives/C013Z450LCD) on [Slack](https://meltano.com/slack).

Contributors are expected to follow our [Code of Conduct](https://docs.meltano.com/contribute/#code-of-conduct).

## hub-utils CLI

MeltanoHub has some conventions and patterns that need to be followed in order for the plugin pages to render properly and CI tests to pass.
This can sometimes make it difficult for our team and community members to contribute changes.
To help with this we created a [hub-utils CLI](https://github.com/meltano/hub-utils) that offers an interactive CLI interface for common operations.
For example it will help you add or update plugins, along the way it will prompt for information it needs to fulfil the plugin definition and if the plugin is SDK based it will scrape most of the information for you so you don't need to provide it.
It does it's best to default to the correct answers and fill common descriptions (i.e. start_date) and labels for you.

This assumes you already have pipx installed, see the meltano [install-pipx docs](https://docs.meltano.com/guide/installation-guide#install-pipx) for details.

```bash
pipx install git+https://github.com/meltano/hub-utils.git
```

The CLI assumes your terminal is in the root of the hub repository, if you need to run it outside the hub repository root you can set the path using the `HUB_ROOT_PATH` environment variable.

## Add or Updating Plugins

To add a new plugin or variant of an existing one, run the following command and provide any input that it prompts for.

```bash
hub-utils add
```

To update an existing variant, run the following command and provide any input that it prompts for.

```bash
hub-utils update-definition
```

If you chose to make any manual changes to the yaml files, make sure you run the yaml linters to fix any linting violations before creating a PR.

```bash
# Automatically attempt to fix any lint violations
hub-utils yamllint fix _data/meltano/extractors/tap-3plcentral/bytecodeio.yml

# Check for lint violations
hub-utils yamllint lint _data/meltano/extractors/tap-3plcentral/bytecodeio.yml
```

## Automated plugin testing

Repo maintainers with `write` access are able to perform automated plugin testing using slash commands in any issue or pull request.
Expand All @@ -35,31 +75,6 @@ yarn
yarn add --global @gridsome/cli
```

### Linters

You can use `pre-commit` to run the linters before committing.

```console
pipx install pre-commit
pre-commit install
```

This will run the linters on all files that are staged for commit. Included linters:

- [yamlllint](https://yamllint.readthedocs.io/en/stable/)

Theres a utility script that makes a best effort to help reformat yaml files to conform to the
yamllint rules.
To use the script run:

```bash
# Run on a single file
poetry run python utility_scripts/plugin_definitions/yaml_lint_fix.py _data/meltano/extractors/tap-3plcentral/bytecodeio.yml

# Run on all .yml files in the `_data/` directory including subdirectories
poetry run python utility_scripts/plugin_definitions/yaml_lint_fix.py
```

## Build and serve the project

Build for development and get live updates as files are changed:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ _Not familiar with Meltano?_

- **Starts simple**: Meltano is pip-installable and comes in a prepackaged docker container, you can have your first ELT pipeline running within minutes.
- **Has DataOps out-of-the-box**: Meltano provides tools that make DataOps best practices easy to use in every project.
- **Integrates with everything**: 350+ natively supported data sources & targets, as well as additional plugins like great expectations or dbt are natively available.
- **Integrates with everything**: 600+ natively supported data sources & targets, as well as additional plugins like great expectations or dbt are natively available.
- **Is Easily customizable**: Meltano isn't just extensible, it's built to be extended! The SDK for Singer Connectors & EDK for Meltano Components are easy to use. Meltano Hub helps you find all of the connectors and components created across the data community.
- **Is a Mature system**: Developed since [2018](https://handbook.meltano.com/timeline), runs in production at large companies like GitLab, and currently powers over a million pipeline runs monthly.
- **Has first class ELT tooling built-in**: Extract data from any data source, load into any target, use inline maps to transform on data on the fly, and test the incoming data, all in one package.
Expand Down
Loading

0 comments on commit b0e7f8d

Please sign in to comment.