Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add donors scraping in GH actions #442

Merged
merged 7 commits into from
Jan 30, 2025
Merged

Add donors scraping in GH actions #442

merged 7 commits into from
Jan 30, 2025

Conversation

Xpirix
Copy link
Collaborator

@Xpirix Xpirix commented Sep 23, 2024

Fix for #426

Summary:

  • Get the script stripe_donor_logger.cgi and its dependencies from the old repo
  • Update and test them locally
  • It should be set up with nginx on the server side and ensure that the webhook link on stripe points to it. Please find the proposed change for that at: https://github.com/qgis/QGIS-Sysadmin-Documentation/pull/66

The cgi script will:

  • Be called whenever a donation has been made on stripe
  • Receive the donation information
  • Add the donor name to the file donors.json
  • Pull from the origin repo, commit and push this change to the main branch

For this change, I kept the process of manual deployment for now.

@Xpirix Xpirix requested a review from timlinux September 23, 2024 11:43
@timlinux
Copy link
Member

Let me chat with @andreasneumann a bit about this - I am not too mad about turning our static site back into a dynamic site. Maybe we can implement this on a GH action rather or on a server that is not exposed to the internet....

@timlinux
Copy link
Member

Can we not do this as a gh action rather to do the scraping and update the deployment bundle (when we have teh site as a nixos package) e.g. once a day?

Copy link
Member

@timlinux timlinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to find another way e.g. as a gh action. I don't want to run code and cgi scripts on our server.

@Xpirix Xpirix self-assigned this Jan 23, 2025
@Xpirix Xpirix added the Size 4 It will take me between 2 hours to half a day. label Jan 24, 2025
@Xpirix Xpirix changed the title Add donors scraping CGI scripts Add donors scraping in GH actions Jan 24, 2025
@Xpirix Xpirix marked this pull request as draft January 24, 2025 06:26
@Xpirix Xpirix marked this pull request as ready for review January 27, 2025 12:33
@Xpirix
Copy link
Collaborator Author

Xpirix commented Jan 27, 2025

Latest Updates according to previous comments

  • Added a script update_donors.py: it fetches new donors from Stripe API and updates the donors list accordingly.
  • Run update_donors.py in GitHub actions with the update-donors.yml workflow
  • The script requires the variable STRIPE_API_KEY from the secrets
  • The schedule is similar to the update feed workflow, i.e each day at noon (UTC)
  • Running the workflow for the first time will create a file at scripts/latest_donation.txt to store the timestamp of the latest donation. In the future, it will be used to fetch new donors. So that you know, running this workflow for the first time might take a while (depending on the number of existing donors) since it doesn't have this reference yet and will retrieve all donors from Stripe.

Test on my account: https://github.com/Xpirix/QGIS-Website/actions/runs/12988979981/job/36221080846

@Xpirix Xpirix requested a review from timlinux January 27, 2025 12:50
Copy link
Member

@timlinux timlinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@timlinux timlinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Xpirix Xpirix merged commit 401220f into qgis:main Jan 30, 2025
2 checks passed
@Xpirix Xpirix mentioned this pull request Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Size 4 It will take me between 2 hours to half a day.
Projects
Development

Successfully merging this pull request may close these issues.

2 participants