Jobs Updater

The jobs updater is a simple GitHub action application to, given some trigger and a yaml file with a list of jobs (or other links):

- name: My Job
  ...
  url: https://my-job.org/12345

The action will inspect the file to determine lines that are newly added (compared to a parent commit or second file) for a field of interest (e.g., the "url" attribute in a list of jobs), extract this field, and then post to a Slack channel, a Discord Channel, Twitter, or Mastodon.

This is custom made to help the hpc.social and US-RSE site to have job updates posted to slack! If you'd like help setting this up for your group, please ping @vsoch.

Usage

You'll generally want to:

1. Generate needed credentials for your apps of choice.
2. Add a GitHub workflow file, as shown below, with your desired triggers.

More specifically, add a GitHub workflow file in .github/workflows to specify the following. Note that the workflow below will do the check and update on any push to main (e.g., a merged pull request).

Deploy to Slack

You'll want to follow the instructions here to create a webhook for your slack community and channel of interest. This usually means first creating an application and selecting your slack community.

1. For the kind of app, you'll want to select the first box for incoming webhooks.

You can then use the example to test the webhook with curl

curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' YOUR_WEBHOOK_URL_HERE

Click on "Add new webhook to workspace" and then test the provided url with the bot. Copy the webhook URL and save this to SLACK_WEBHOOK in your repository secrets.

on:
  push:
    paths:
      - '_data/jobs.yaml'
    branches:
      - main

jobs:
  slack-poster:
    runs-on: ubuntu-latest
    name: Run Jobs Slack Poster
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 2

      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@main
        with:
          filename: "_data/jobs.yaml"
          keys: "url,name"
          unique: "url"
          slack_webhook: ${{ secrets.SLACK_WEBHOOK }}
          slack_deploy: true

      - run: echo ${{ steps.updater.outputs.fields }}
        name: Show New Jobs
        shell: bash

In the above, we will include the url and name fields, and use the url field to determine uniqueness (default). Given that you have the slack webhook as a secret provided to the action and deploy_slack is true, the default deploy variable (to indicate all services) is true and deployment will happen. If you just want to test, then do:

...
      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@main
        with:
          filename: "_data/jobs.yaml"
          keys: "url"
          deploy: false
          slack_webhook: ${{ secrets.SLACK_WEBHOOK }}
          slack_deploy: true

If you want to run a test run (meaning a random number of jobs will be selected that aren't necessarily new) then add test:

...
      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@main
        with:
          filename: "_data/jobs.yaml"
          keys: "url"
          test: true
          slack_deploy: true

If test is true, deploy will always be set to false.

Deploy to Twitter

To deploy to Twitter (in addiction to slack) you are required to set deploy_twitter to true, and also define all the needed environment variables in your repository secrets.

...
      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@add/deploy-arg
        with:
          filename: "_data/jobs.yaml"
          keys: "url,name"
          test: false
          slack_deploy: true

          # Also deploy to Twitter (all secrets required in repository secrets)
          twitter_deploy: true
          twitter_api_secret: ${{ secrets.TWITTER_ACCESS_SECRET }}
          twitter_api_key: ${{ secrets.TWITTER_ACCESS_TOKEN }}
          twitter_consumer_secret: ${{ secrets.TWITTER_CONSUMER_API_SECRET }}
          twitter_consumer_key: ${{ secrets.TWITTER_CONSUMER_API_KEY }}

Deploy to Mastodon

To deploy to Mastodon you are required to set deploy_mastodon to true, and also define all the needed environment variables in your repository secrets.

      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@main
        with:
          filename: "_data/jobs.yaml"
          key: "url"
          test: false

          # Also deploy to Mastodon (all secrets required in repository secrets)
          # The access token can be generated by:
          # 1) visiting https://YOUR.MASTODON.SITE.HERE/settings/applications/new (after logging into it)
          # 2) choosing any name for the new application, only 'write:statuses' privileges are
          #    required for this purpose
          # 3) copying the value of 'Your access token' into a secret named MASTODON_ACCESS_TOKEN
          #    in the GitHub repository where you want to run the updater
          # 4) also add a secret named MASTODON_API_BASE_URL with the value of, e.g.,
          #    https://YOUR.MASTODON.SITE.HERE/
          mastodon_deploy: true
          mastodon_access_token: ${{ secrets.MASTODON_ACCESS_TOKEN }}
          mastodon_api_base_url: ${{ secrets.MASTODON_API_BASE_URL }}

Deploy to Discord

To deploy to Discord you will need to create a webhook and then set deploy_discord to true, along with adding the webhook to your repository secrets as DISCORD_WEBHOOK.

      - id: updater
        name: Job Updater
        uses: rseng/jobs-updater@main
        with:
          filename: "_data/jobs.yaml"
          key: "url"
          test: false
          discord_deploy: true
          discord_webhook: ${{ secrets.DISCORD_WEBHOOK }}

Variables

Inputs

The following variables are available. You can also look at the action.yml.

Name	Description	Required	Default
filename	The filename for the jobs	true	unset
previous_filename	The previous filename (for manual tesing or running alongside update)	false	unset
keys	Comma separated list of keys to post (defaults to url)	false	url
unique	Field to use to determine uniqueness	true	url
hashtag	A hashtag to use (defaults to #Rseng)	false	#RSEng
test	Test the updater (ensure there are jobs)	true	false
deploy	Global deploy across any service set to true?	true	true
slack_deploy	Deploy to Slack?	true	false
slack_webhook	Slack webhook to deploy to.	false	unset
discord_deploy	Deploy to Discord?	true	false
discord_webhook	Discord webhook to deploy to.	false	unset
twitter_deploy	Deploy to Twitter?	false	unset
twitter_api_key	API key generated for the user account to tweet	false	unset
twitter_api_secret	API secret generated for the user account to tweet	false	unset
twitter_consumer_key	Consumer key generated for the entire app	false	unset
twitter_consumer_secret	Consumer secret generated for the entire app	false	unset
mastodon_deploy	Boolean to deploy to Mastodon	false	unset
mastodon_access_token	API key generated for the user account to tweet	false	unset
mastodon_api_base_url	Base URL of the Mastodon instance to post to, e.g., https://fosstodon.org/	false	unset

Outputs

Name	Description
Fields (keys) parsed	The fields that are parsed in the jobs
Matrix	Matrix (list of lists) with value (index 1), icon (index 2) and full message (index 3)
Empty Matrix	true if empty, false otherwise