Skip to content

Commit

Permalink
add SQL Upgrade Scripts section to Dev Guide #5186
Browse files Browse the repository at this point in the history
  • Loading branch information
@pdurbin
pdurbin committed Oct 16, 2018
1 parent 7c67734 commit b802c85
Show file tree
Hide file tree
Showing 7 changed files with 46 additions and 5 deletions.
2 changes: 1 addition & 1 deletion PULL_REQUEST_TEMPLATE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Welcome! New contributors should at least glance at [CONTRIBUTING.md](/CONTRIBUT
- [ ] Merged latest from "develop" [branch][] and resolved conflicts

[tests]: http://guides.dataverse.org/en/latest/developers/testing.html
[SQL updates]: https://github.com/IQSS/dataverse/tree/develop/scripts/database/upgrades
[SQL updates]: http://guides.dataverse.org/en/latest/developers/sql-upgrade-scripts.html
[Solr updates]: https://github.com/IQSS/dataverse/blob/develop/conf/solr/7.3.0/schema.xml
[docs]: http://guides.dataverse.org/en/latest/developers/documentation.html
[branch]: http://guides.dataverse.org/en/latest/developers/branching-strategy.html
1 change: 1 addition & 0 deletions doc/sphinx-guides/source/developers/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Developer Guide
tips
troubleshooting
version-control
sql-upgrade-scripts
testing
documentation
debugging
Expand Down
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/developers/making-releases.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Upload the following artifacts to the draft release you created:

- war file (``mvn package`` from Jenkins)
- installer (``cd scripts/installer && make``)
- database migration script
- database migration script (see also the :doc:`sql-upgrade-scripts` section)
- other files as needed, such as an updated Solr schema

Publish Release
Expand Down
40 changes: 40 additions & 0 deletions doc/sphinx-guides/source/developers/sql-upgrade-scripts.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
===================
SQL Upgrade Scripts
===================

The database schema for Dataverse is constantly evolving. As other developers make changes to the database schema you will need to keep up with these changes to keep your development environment in working order. Additionally, as you make changes to the database schema, you must help write SQL upgrade scripts as needed and communicate with your fellow developers when the scripts must be applied.

.. contents:: |toctitle|
:local:

Location of SQL Upgrade Scripts
-------------------------------

``scripts/database/upgrades`` is the directory where we keep or SQL upgrade scripts.

How to Determine if You Need to Create or Update a SQL Upgrade Script
---------------------------------------------------------------------

If you are creating a new database table (which maps to an ``@Entity`` in JPA), you do not need to create or update a SQL upgrade script. The reason for this is that we use ``create-tables`` in ``src/main/resources/META-INF/persistence.xml`` so that new tables are automatically created by Glassfish when you deploy your war file.

If you are doing anything other than creating a new database table such as adding a column to an existing table, you must create or update a SQL upgrade script.

This comment has been minimized.

Sign in to view

Copy link
@qqmyers

qqmyers Oct 16, 2018

Member

Note that scripts should be idempotent - they should only make changes the first time they run. This is true by definition for statements such as 'ALTER TABLE', but changes to database values should include a check to verify that the check has not been previously made.

This comment has been minimized.

Sign in to view

Copy link
@pdurbin

pdurbin Oct 17, 2018

Author Member

@qqmyers I agree that SQL scripts should be idempotent but I think it would be better to mention this in the next section.

Somewhere we should probably also mention reference_data.sql but we don't update that one very frequently. It's called by the installer.
How to Create or Update a SQL Upgrade Script
--------------------------------------------

We assume you have already read the :doc:`version-control` section and have been keeping your feature branch up to date with the "develop" branch.

First, check https://github.com/IQSS/dataverse/tree/develop/scripts/database/upgrades to see if a SQL upgrade script for the next release already exists. For example, if the current release is 4.9.4 and the next release will be 4.10, the script will be named ``upgrade_v4.9.4_to_v4.10.sql``. If the script exists, just add your changes to the bottom of it.

If no SQL upgrade script exists, look at https://github.com/IQSS/dataverse/milestones to figure out the name of the next milestone and create a script using the naming convention above.

As with any task related to Dataverse development, if you need any help writing SQL upgrade scripts, please reach out using any of the channels mentioned under "Getting Help" in the :doc:`intro` section.

Communicating the Need to Run SQL Updates
-----------------------------------------

If you have made a pull request that contains SQL updates and that pull request is merged into the "develop" branch, you are responsible for communicating to other developers that when then pull the latest code from "develop" they must run your SQL updates. Post a message to the "dataverse-dev" mailing list at https://groups.google.com/forum/#!forum/dataverse-dev

This comment has been minimized.

Sign in to view

Copy link
@qqmyers

qqmyers Oct 16, 2018

Member

Given that the developer doesn't choose when a merge happens, it's tricky for them to be responsible to alert other developers. I'm OK with leaving this as is, but wonder if having developers do something to flag that a db update is needed (e.g. in the template where we link the PR to an issue) and having Dataverse send an email when that's set would be better.

This comment has been minimized.

Sign in to view

Copy link
@pdurbin

pdurbin Oct 17, 2018

Author Member

@qqmyers sorry, I'm having trouble understanding what you mean by "having Dataverse send an email" because I'm not sure who that would be. I would rather it not always be me. I think it's better to spread out the responsibility. Maybe you're talking about some fancy automation? I'm not opposed to that in the future but it's out of scope for this pull request, which is just a little cleanup and some docs.

This comment has been minimized.

Sign in to view

Copy link
@qqmyers

qqmyers Oct 17, 2018

Member

@pdurbin - I'm OK with what you wrote and didn't have any automation ideas. Just worried about people (me :-) ) missing the cue to send an email since the merge can be far off from the day the change was created.

----

Previous: :doc:`version-control` | Next: :doc:`testing`
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/developers/testing.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -290,4 +290,4 @@ Future Work on Load/Performance Testing

----

Previous: :doc:`version-control` | Next: :doc:`documentation`
Previous: :doc:`sql-upgrade-scripts` | Next: :doc:`documentation`
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/developers/tips.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ For faster iteration while working on JSF pages, it is highly recommended that y
Database Schema Exploration
---------------------------

With over 100 tables, the Dataverse PostgreSQL database ("dvndb") can be somewhat daunting for newcomers. Here are some tips for coming up to speed.
With over 100 tables, the Dataverse PostgreSQL database ("dvndb") can be somewhat daunting for newcomers. Here are some tips for coming up to speed. (See also the :doc:`sql-upgrade-scripts` section.)

pgAdmin
~~~~~~~~
Expand Down
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/developers/version-control.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -145,4 +145,4 @@ GitHub documents how to make changes to a fork at https://help.github.com/articl
----

Previous: :doc:`troubleshooting` | Next: :doc:`testing`
Previous: :doc:`troubleshooting` | Next: :doc:`sql-upgrade-scripts`

0 comments on commit b802c85

Please sign in to comment.