Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename mutable to managed #109

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ An [Ansible][ansible] role for installing and managing [Galaxy][galaxyproject] s
Requirements
------------

This role has the same dependencies as the git module. In addition, [pip][pip] and [Python virtualenv][venv] are required. These can easily be installed via a pre-task in
the same play as this role:
This role has the same dependencies as the git module. In addition, [pip][pip] and [Python virtualenv][venv] are required.
These can easily be installed via a pre-task in the same play as this role:

```yaml
- hosts: galaxyservers
Expand Down Expand Up @@ -206,8 +206,8 @@ Several variables control which functions this role will perform (all default to
well as set up a [virtualenv][virtualenv] from which it can be run.
- `galaxy_manage_static_setup`: Manage "static" Galaxy configuration files - ones which are not modifiable by the Galaxy
server itself. At a minimum, this is the primary Galaxy configuration file, `galaxy.ini`.
- `galaxy_manage_mutable_setup`: Manage "mutable" Galaxy configuration files - ones which are modifiable by Galaxy (e.g.
as you install tools from the Galaxy Tool Shed).
- `galaxy_manage_managed_setup`: Manage "managed" Galaxy configuration files - ones which are modifiable by Galaxy
(e.g. as you install tools from the Galaxy Tool Shed).
- `galaxy_manage_database`: Upgrade the database schema as necessary, when new schema versions become available.
- `galaxy_fetch_dependencies`: Fetch Galaxy dependent modules to the Galaxy virtualenv.
- `galaxy_build_client`: Build the Galaxy client application (web UI).
Expand All @@ -225,7 +225,10 @@ Options for configuring Galaxy and controlling which version is installed.
- `galaxy_config`: The contents of the Galaxy configuration file (`galaxy.ini` by default) are controlled by this
variable. It is a hash of hashes (or dictionaries) that will be translated in to the configuration
file. See the Example Playbooks below for usage.
- `galaxy_config_files`: List of hashes (with `src` and `dest` keys) of files to copy from the control machine. For example, to set job destinations, you can use the `galaxy_config_dir` variable followed by the file name as the `dest`, e.g. `dest: "{{ galaxy_config_dir }}/job_conf.xml"`. Make sure to add the appropriate setup within `galaxy_config` for each file added here (so, if adding `job_conf.xml` make sure that `galaxy_config.galaxy.job_config_file` points to that file).
- `galaxy_config_files`: List of hashes (with `src` and `dest` keys) of files to copy from the control machine. For example,
to set job destinations, you can use the `galaxy_config_dir` variable followed by the file name as the `dest`, e.g.
`dest: "{{ galaxy_config_dir }}/job_conf.xml"`. Make sure to add the appropriate setup within `galaxy_config` for each file
added here (so, if adding `job_conf.xml` make sure that `galaxy_config.galaxy.job_config_file` points to that file).
- `galaxy_config_templates`: List of hashes (with `src` and `dest` keys) of templates to fill from the control machine.
- `galaxy_local_tools`: List of local tool files or directories to copy from the control machine, relative to
`galaxy_local_tools_src_dir` (default: `files/galaxy/tools` in the playbook).
Expand Down Expand Up @@ -257,9 +260,9 @@ Options for controlling where certain Galaxy components are placed on the filesy
to use when creating the virtualenv. For Galaxy < 20.01, use python2.7 (if it is not the default), for Galaxy >=
20.01, use `python3.5` or higher.
- `galaxy_config_dir` (default: `<galaxy_server_dir>`): Directory that will be used for "static" configuration files.
- `galaxy_mutable_config_dir` (default: `<galaxy_server_dir>`): Directory that will be used for "mutable" configuration
- `galaxy_managed_config_dir` (default: `<galaxy_server_dir>`): Directory that will be used for "managed" configuration
files, must be writable by the user running Galaxy.
- `galaxy_mutable_data_dir` (default: `<galaxy_server_dir>/database`): Directory that will be used for "mutable" data
- `galaxy_managed_data_dir` (default: `<galaxy_server_dir>/database`): Directory that will be used for "managed" data
and caches, must be writable by the user running Galaxy.
- `galaxy_config_file` (default: `<galaxy_config_dir>/galaxy.ini`): Galaxy's primary configuration file.

Expand Down Expand Up @@ -329,7 +332,7 @@ $ sh run.sh

Install Galaxy as per the current production server best practices:

- Galaxy code (clone) is "clean": no configs or mutable data live underneath the clone
- Galaxy code (clone) is "clean": no configs or managed data live underneath the clone
- Galaxy code and static configs are privilege separated: not owned/writeable by the user that runs Galaxy
- Configuration files are not world-readable
- PostgreSQL is used as the backing database
Expand Down
34 changes: 17 additions & 17 deletions defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ galaxy_manage_paths: no
galaxy_manage_clone: yes
galaxy_manage_download: no
galaxy_manage_static_setup: yes
galaxy_manage_mutable_setup: yes
galaxy_manage_managed_setup: yes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
galaxy_manage_managed_setup: yes
galaxy_manage_managed_setup: "{{ galaxy_manage_mutable_setup | default(yes) }}"

maybe something like this to ensure that people who don't read the upgrade notice can use the legacy value? @natefoo what are your opinions on that. "Read the release notes or else?"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no objections. @natefoo - up to you?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and we'll need to default all the other changed vars as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coolie. Sorry for the extra work, but figure it'll be a bit more userfriendly.

Copy link
Member

@natefoo natefoo Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be too bad as you can do:

galaxy_managed_data_dir: "{{ galaxy_mutable_data_dir }}"

Rather than defaulting each instance of its usage. Hopefully that all plays nicely with the layout stuff.

galaxy_manage_database: yes
galaxy_fetch_dependencies: yes
galaxy_build_client: yes
Expand Down Expand Up @@ -84,8 +84,8 @@ galaxy_create_privsep_user: "{{ galaxy_create_user if __galaxy_privsep_user_name

# Directories to create as the Galaxy user if galaxy_manage_paths is enabled
galaxy_dirs:
- "{{ galaxy_mutable_data_dir }}"
- "{{ galaxy_mutable_config_dir }}"
- "{{ galaxy_managed_data_dir }}"
- "{{ galaxy_managed_config_dir }}"
- "{{ galaxy_cache_dir }}"
- "{{ galaxy_shed_tools_dir }}"
- "{{ galaxy_tool_dependency_dir }}"
Expand Down Expand Up @@ -147,7 +147,7 @@ galaxy_tool_config_files:

# The shed tool conf path is needed separate from the value of tool_conf for instantiation. The 'conf' version is a
# legacy variable name and should not be used.
galaxy_shed_tool_conf_file: "{{ galaxy_mutable_config_dir }}/shed_tool_conf.xml"
galaxy_shed_tool_conf_file: "{{ galaxy_managed_config_dir }}/shed_tool_conf.xml"
galaxy_shed_tool_config_file: "{{ galaxy_shed_tool_conf_file }}"

# The Python `requirements.txt` file that should be used to install Galaxy dependent modules using pip
Expand Down Expand Up @@ -184,17 +184,17 @@ galaxy_app_config_default:
openid_consumer_cache_path: "{{ galaxy_cache_dir }}/openid_consumer_cache"
citation_cache_data_dir: "{{ galaxy_cache_dir }}/citations/data"
citation_cache_lock_dir: "{{ galaxy_cache_dir }}/citations/locks"
new_file_path: "{{ galaxy_mutable_data_dir }}/tmp"
mulled_resolution_cache_data_dir: "{{ galaxy_mutable_data_dir }}/mulled/data"
mulled_resolution_cache_lock_dir: "{{ galaxy_mutable_data_dir }}/mulled/lock"
new_file_path: "{{ galaxy_managed_data_dir }}/tmp"
mulled_resolution_cache_data_dir: "{{ galaxy_managed_data_dir }}/mulled/data"
mulled_resolution_cache_lock_dir: "{{ galaxy_managed_data_dir }}/mulled/lock"

# Mutable config files
integrated_tool_panel_config: "{{ galaxy_mutable_config_dir }}/integrated_tool_panel.xml"
migrated_tools_config: "{{ galaxy_mutable_config_dir }}/migrated_tools_conf.xml"
shed_data_manager_config_file: "{{ galaxy_mutable_config_dir }}/shed_data_manager_conf.xml"
shed_tool_data_table_config: "{{ galaxy_mutable_config_dir }}/shed_tool_data_table_conf.xml"
# Managed config files
integrated_tool_panel_config: "{{ galaxy_managed_config_dir }}/integrated_tool_panel.xml"
migrated_tools_config: "{{ galaxy_managed_config_dir }}/migrated_tools_conf.xml"
shed_data_manager_config_file: "{{ galaxy_managed_config_dir }}/shed_data_manager_conf.xml"
shed_tool_data_table_config: "{{ galaxy_managed_config_dir }}/shed_tool_data_table_conf.xml"

# Directories that would often be stored outside the mutable data dir, these values are either from the layout or set by the user
# Directories that would often be stored outside the managed data dir, these values are either from the layout or set by the user
file_path: "{{ galaxy_file_path }}"
job_working_directory: "{{ galaxy_job_working_directory }}"
tool_dependency_dir: "{{ galaxy_tool_dependency_dir }}"
Expand All @@ -210,13 +210,13 @@ galaxy_app_config_default:
galaxy_config_default: "{{ {} | combine({galaxy_app_config_section: galaxy_app_config_default}) }}"
galaxy_config_merged: "{{ galaxy_config_default | combine(galaxy_config | default({}), recursive=True) }}"

# Automatically instantiate mutable config files from copies in role if they don't exist
galaxy_mutable_config_files:
# Automatically instantiate managed config files from copies in role if they don't exist
galaxy_managed_config_files:
- src: "shed_data_manager_conf.xml"
dest: "{{ galaxy_config_merged[galaxy_app_config_section].shed_data_manager_config_file }}"
- src: "shed_tool_data_table_conf.xml"
dest: "{{ galaxy_config_merged[galaxy_app_config_section].shed_tool_data_table_config }}"
galaxy_mutable_config_templates:
galaxy_managed_config_templates:
- src: "shed_tool_conf.xml.j2"
dest: "{{ galaxy_config_merged[galaxy_app_config_section].migrated_tools_config }}"
- src: "shed_tool_conf.xml.j2"
Expand All @@ -227,7 +227,7 @@ galaxy_config_instance_resource_url: "{{ galaxy_config_merged[galaxy_app_config_
galaxy_errordocs_server_name: Galaxy
galaxy_errordocs_prefix: /error

# These are lists of hashes in the same format as galaxy_mutable_configs that can be used to cause extra files and
# These are lists of hashes in the same format as galaxy_managed_configs that can be used to cause extra files and
# templates to be installed on the managed host.
galaxy_config_files: []
galaxy_config_templates: []
Expand Down
2 changes: 1 addition & 1 deletion tasks/_inc_galaxy_version.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
# Determine Galaxy version

# Currently only used by mutable config setup but placed in an include because it'll probably be used by more
# Currently only used by managed config setup but placed in an include because it'll probably be used by more

- name: Collect Galaxy version file
slurp:
Expand Down
4 changes: 2 additions & 2 deletions tasks/layout.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
- galaxy_venv_dir
- galaxy_server_dir
- galaxy_config_dir
- galaxy_mutable_data_dir
- galaxy_mutable_config_dir
- galaxy_managed_data_dir
- galaxy_managed_config_dir
- galaxy_shed_tools_dir
- galaxy_cache_dir
- galaxy_local_tools_dir
Expand Down
8 changes: 4 additions & 4 deletions tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,11 @@
tags:
- galaxy_fetch_dependencies

- name: Include mutable config setup tasks
include_tasks: mutable_setup.yml
when: galaxy_manage_mutable_setup
- name: Include managed config setup tasks
include_tasks: managed_setup.yml
when: galaxy_manage_managed_setup
tags:
- galaxy_manage_mutable_setup
- galaxy_manage_managed_setup

- name: Include database management tasks
include_tasks: database.yml
Expand Down
12 changes: 6 additions & 6 deletions tasks/mutable_setup.yml → tasks/managed_setup.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
---
# Instantiate mutable config files
# Instantiate managed config files

- name: Mutable config setup
- name: Managed config setup
block:

- name: Ensure Galaxy version is set
include_tasks: _inc_galaxy_version.yml
when: __galaxy_major_version is undefined

# force: no in the following 2 tasks will not overwrite existing configs
- name: Instantiate mutable configuration files
- name: Instantiate managed configuration files
copy:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
force: no
with_items: "{{ galaxy_mutable_config_files }}"
with_items: "{{ galaxy_managed_config_files }}"

- name: Instantiate mutable configuration templates
- name: Instantiate managed configuration templates
template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
force: no
with_items: "{{ galaxy_mutable_config_templates }}"
with_items: "{{ galaxy_managed_config_templates }}"

remote_user: "{{ galaxy_remote_users.galaxy | default(__galaxy_remote_user) }}"
become: "{{ true if galaxy_become_users.galaxy is defined else __galaxy_become }}"
Expand Down
2 changes: 1 addition & 1 deletion tests/test_playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
galaxy_privsep_user: gxpriv
galaxy_config:
galaxy:
database_connection: sqlite:///{{ galaxy_mutable_data_dir }}/universe.sqlite
database_connection: sqlite:///{{ galaxy_managed_data_dir }}/universe.sqlite
# needed because we've disabled facts
ansible_user_uid: "{{ lookup('env', 'UID') }}"
ansible_user_id: "{{ lookup('env', 'USER') }}"
Expand Down
14 changes: 7 additions & 7 deletions vars/layout-custom.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@
# - galaxy_server_dir
# - galaxy_venv_dir
# - galaxy_config_dir
# - galaxy_mutable_data_dir
# - galaxy_managed_data_dir

# Layout defaults
__galaxy_mutable_config_dir: "{{ galaxy_mutable_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_mutable_data_dir }}/cache"
__galaxy_managed_config_dir: "{{ galaxy_managed_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_managed_data_dir }}/cache"
__galaxy_local_tools_dir: null
__galaxy_shed_tools_dir: "{{ galaxy_mutable_data_dir }}/shed_tools"
__galaxy_shed_tools_dir: "{{ galaxy_managed_data_dir }}/shed_tools"

# These correspond to galaxy config options
__galaxy_tool_dependency_dir: "{{ galaxy_mutable_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_mutable_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_mutable_data_dir }}/jobs"
__galaxy_tool_dependency_dir: "{{ galaxy_managed_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_managed_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_managed_data_dir }}/jobs"
14 changes: 7 additions & 7 deletions vars/layout-legacy-improved.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
# Layout defaults
__galaxy_venv_dir: "{{ galaxy_server_dir }}/.venv"
__galaxy_config_dir: "{{ galaxy_server_dir }}/config"
__galaxy_mutable_data_dir: "{{ galaxy_server_dir }}/database"
__galaxy_mutable_config_dir: "{{ galaxy_server_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_mutable_data_dir }}"
__galaxy_managed_data_dir: "{{ galaxy_server_dir }}/database"
__galaxy_managed_config_dir: "{{ galaxy_server_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_managed_data_dir }}"
__galaxy_local_tools_dir: null
__galaxy_shed_tools_dir: "{{ galaxy_mutable_data_dir }}/shed_tools"
__galaxy_shed_tools_dir: "{{ galaxy_managed_data_dir }}/shed_tools"

# These correspond to galaxy config options
__galaxy_tool_dependency_dir: "{{ galaxy_mutable_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_mutable_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_mutable_data_dir }}/jobs"
__galaxy_tool_dependency_dir: "{{ galaxy_managed_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_managed_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_managed_data_dir }}/jobs"
12 changes: 6 additions & 6 deletions vars/layout-legacy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
# Layout defaults
__galaxy_venv_dir: "{{ galaxy_server_dir }}/.venv"
__galaxy_config_dir: "{{ galaxy_server_dir }}"
__galaxy_mutable_data_dir: "{{ galaxy_server_dir }}/database"
__galaxy_mutable_config_dir: "{{ galaxy_server_dir }}"
__galaxy_cache_dir: "{{ galaxy_mutable_data_dir }}"
__galaxy_managed_data_dir: "{{ galaxy_server_dir }}/database"
__galaxy_managed_config_dir: "{{ galaxy_server_dir }}"
__galaxy_cache_dir: "{{ galaxy_managed_data_dir }}"
__galaxy_local_tools_dir: null
__galaxy_shed_tools_dir: "{{ galaxy_server_dir }}/../shed_tools"

# These correspond to galaxy config options
__galaxy_tool_dependency_dir: "{{ galaxy_mutable_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_mutable_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_mutable_data_dir }}/jobs"
__galaxy_tool_dependency_dir: "{{ galaxy_managed_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_managed_data_dir }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_managed_data_dir }}/jobs"
12 changes: 6 additions & 6 deletions vars/layout-opt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
__galaxy_server_dir: "/opt/galaxy"
__galaxy_venv_dir: "/var/opt/galaxy/venv"
__galaxy_config_dir: "/etc/opt/galaxy"
__galaxy_mutable_data_dir: "/var/opt/galaxy/data"
__galaxy_mutable_config_dir: "{{ galaxy_mutable_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_mutable_data_dir }}/cache"
__galaxy_managed_data_dir: "/var/opt/galaxy/data"
__galaxy_managed_config_dir: "{{ galaxy_managed_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_managed_data_dir }}/cache"
__galaxy_local_tools_dir: "/var/opt/galaxy/local_tools"
__galaxy_shed_tools_dir: "{{ galaxy_mutable_data_dir }}/shed_tools"
__galaxy_shed_tools_dir: "{{ galaxy_managed_data_dir }}/shed_tools"

# These correspond to galaxy config options
__galaxy_tool_dependency_dir: "{{ galaxy_mutable_data_dir }}/dependencies"
__galaxy_tool_dependency_dir: "{{ galaxy_managed_data_dir }}/dependencies"
__galaxy_file_path: "/srv/galaxy/datasets"
__galaxy_job_working_directory: "{{ galaxy_mutable_data_dir }}/jobs"
__galaxy_job_working_directory: "{{ galaxy_managed_data_dir }}/jobs"
10 changes: 5 additions & 5 deletions vars/layout-root-dir.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
__galaxy_server_dir: "{{ galaxy_root }}/server"
__galaxy_venv_dir: "{{ galaxy_root }}/venv"
__galaxy_config_dir: "{{ galaxy_root }}/config"
__galaxy_mutable_data_dir: "{{ galaxy_root }}/var"
__galaxy_mutable_config_dir: "{{ galaxy_mutable_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_mutable_data_dir }}/cache"
__galaxy_managed_data_dir: "{{ galaxy_root }}/var"
__galaxy_managed_config_dir: "{{ galaxy_managed_data_dir }}/config"
__galaxy_cache_dir: "{{ galaxy_managed_data_dir }}/cache"
__galaxy_local_tools_dir: "{{ galaxy_root }}/local_tools"
__galaxy_shed_tools_dir: "{{ galaxy_mutable_data_dir }}/shed_tools"
__galaxy_shed_tools_dir: "{{ galaxy_managed_data_dir }}/shed_tools"

# These correspond to galaxy config options
__galaxy_tool_dependency_dir: "{{ galaxy_mutable_data_dir }}/dependencies"
__galaxy_tool_dependency_dir: "{{ galaxy_managed_data_dir }}/dependencies"
__galaxy_file_path: "{{ galaxy_root }}/datasets"
__galaxy_job_working_directory: "{{ galaxy_root }}/jobs"