Add sample_data script

[dacook]

It can be executed by ofn-admin or ofn-deploy user.

[mkllnk]

I see where you are going with this. All the sudo commands work for ofn-admin but not for ofn-deploy. That user can execute only one single command as root at the moment:

# Grant ofn-deploy sudo access for deploy script
ofn-deploy ALL=(ALL) NOPASSWD: /home/ofn-deploy/deploy

/etc/sudoers.d/ofn-deploy

[dacook]

Thanks, so I take it you have no concerns about heading in this direction. 👍

Hmm, actually there's another piece of the puzzle I hadn't realised before: the ability to trigger the script is tied to the specific deployment SSH key.
https://github.com/openfoodfoundation/ofn-install/blob/master/roles/remote_deployment/tasks/main.yml#L67

So I guess we would need to create a separate SSH key for this new script. I see why you had reservations about heading down this path :)

[mkllnk]

Also, currently, this is just replacing the current data with the sample data. And then there are requests to expand the sample data, or apply only some of the sample data. Each of these may be different commands. If we don't want to end up with lots of repetition then we would need to open our SSH setup. I think that it would be easier to allow Github to execute SSH commands as the openfoodnetwork user. It would make all this much easier.

But maybe it would be better to have a more generic way of creating snapshots and use them. In that case, testers could create whatever test data they want and then store it for repeating tests. That data doesn't have to be managed in code then either.

I reckon it may be easier to implement a simple admin UI for this than managing all kinds of sample data in code. Then we don't need any SSH or Github Actions for this. Example how this could work:

• New directory in the app: db/snapshots.
• New admin controller admin/snapshots.
• Index: list files in that directory, each with a button to restore or delete.
• A big Create button lives at the top of the index page. It has an input field for naming the snapshot.
• Creating a snapshot takes a database dump and stores it under db/snapshots/2025-01-03_18:20_#{name}.sql.gz.
• Restoring a snapshot resets the database and loads the snapshot file. This may need to disconnect the running puma server. I don't know if it can reconnect after. If there's no better solution, we have to execute the command in the background as a separate process and tell the user to reload the page.
• Deleting a snapshot just deleted the file. Done.
• Optional bonus: Before creating a snapshot, save the snapshot name in ar_internal_metadata to reference later. After a snapshot has been restored, we can display that on the index page. So people know on which snapshot the database is currently based on. The created_at and updated_at values can be used for snapshot creation and restoration and displayed in the UI. So you know that restoring the snapshot worked or the current database has been worked with for days.

Estimate: 3 days.

And I don't think that we would have less work messing around with Github Actions, ofn-install, managing SSH keys and creating new sample data factories. What do you think?

[dacook]

Wow, thanks for thinking through this. You're right that we need to design a solution before jumping in to it.
That's a clever system and sounds great, we should take that to the testing team and see what they think. I would add the current system disk usage percentage to the screen too, so you can see at a glance if you're filling up the hard drive.

---

I thought this PR just needed another hour or two maybe, but then maybe I didn't think it all through.

1. add sudoers entry
2. Add ssh key for sample_data command
3. run the task on 3 servers, test it out
4. set up the GH action

Optional extension:

1. Set up individual rake tasks for each factory (eg ofn:sample_data:users
2. add a parameter to the GH Action, passed through to the script (like we do for deploy). Eg choose "users" and it is appended to the command: rake ofn:sample_data:$1 (need to deal with the colon, but hopefully you see what I mean)

---

Again, it's worth talking to testers to see what the most pressing needs are. Filipe seems offline today, maybe we could bring it up next delivery circle, or after delivery circle (he's usually online then).

[filipefurtad0]

Thanks so much for the time invested and sharing your insights ✨

A big Create button lives at the top of the index page. It has an input field for naming the snapshot.

This is an interesting solution indeed. Two thoughts:

• would this replace this approach Add scripts to save and restore baseline data openfoodnetwork#12947? It currently does not allow for baseline versioning (it sets one baseline only)
• really like the fact that this would be managed through the UI. However, I have concerns that some DB changes, under some conditions, would throw error 500s, and make the index page inaccessible. This would probably make it harder to revert the changes

Yes, lets discuss it on the next delivery circle, thanks again 👍

[mkllnk]

would this replace this approach openfoodfoundation/openfoodnetwork#12947?

Yes, it would.

[mkllnk]

I have concerns that some DB changes, under some conditions, would throw error 500s, and make the index page inaccessible.

This is true. So we could keep a backup restore script for the console. Maybe if we move this new UI into its own engine then it wouldn't depend on the OFN database and may be more robust. It depends if the bad database prevents Rails from loading altogether.

[dacook]

Closed in favour of triggering as an option within the existing deploy script.