Build infrastructure for creating platform-specific builds of OpenFlight projects with Omnibus.
The OpenFlight Omnibus Builder repository provides build instructions and configuration to allow complex OpenFlight projects to be built for distribution for particular platforms (primarily Enterprise Linux) using Omnibus. This allows them to be shipped along with their necessary dependencies on versions of upstream projects that aren't made available as part of the distribution, for e.g. more advanced Ruby versions, latest VNC servers etc.
The build infrastructure requires that you have a working Vagrant installation.
Firstly, clone the repo!
git clone https://github.com/openflighthpc/openflight-omnibus-builder
Bring up the EL7 build VM:
cd openflight-omnibus-builder
vagrant up
The EL7 build VM will be provisioned with the required tools including required distribution dependencies via yum and an RVM installation of Ruby 2.6.
Access the build VM using vagrant ssh. The build infrastructure is mounted within the guest at /vagrant.
This repository contains a Thor template designed to ease the initialisation of an Omnibus builder for a new project.
vagrant ssh
cd /vagrant/generator
bin/thor generate_builder EXECUTABLE REPO_NAME FRIENDLY_NAME
where, in the case of, for example, Flight Desktop, the required arguments resemble:
bin/thor generate_builder desktop "flight-desktop" "Flight Desktop"
A simplistic builder will be generated in the working directory. Copy this directory to /vagrant/builders and make any project-specific changes to it before continuing to build. See the builder docs for more info with regards to further configuration options.
Log in to the build VM and set up the Omnibus build infrastructure.
vagrant ssh
cd /vagrant/builders/flight-example
bundle install
The build infrastructure is set up so as not to need root access. This is important because, when building and installing complex projects, the installation of any software dependencies that attempt to write outside of the /opt/flight hierarchy will fail fast allowing potential problems to be corrected.
To build a project, enter the project directory within /vagrant/builders and issue the omnibus build command, e.g.:
cd /vagrant/builders/flight-example
bin/omnibus build flight-example
Once the build has completed, the RPM package will be available in the pkg/ subdirectory.
Sometimes (mostly, in React apps), you may wish to include a static JS file that has been created by the Flight UMD Builder. To do so, a new step should be added to the Omnibus builder file to download, build, and import the static JS file. Please note that the changes made here assume that: the package being imported has the UMD builder set up as part of the branch being used (see UMD builder repo for setup info); the package being built with Omnibus contains the logic to import the files generated by the UMD builder (for example, a webapp would need script tags in the headers to import the JS file).
The step should be added to config/projects/<package>.rb among the dependencies. The name you give the dependency should match a file you will create in the config/software/ directory.
In the config/software/ directory, create a .rb with the same name as the dependency step you added before. The goal of this file is to clone the desired package from Github; clone the UMD builder into the directory; and build the static copy of the component(s). An example may look like the following, where <umd-buildable-package> is the package being imported:
require 'zlib'
require 'minitar'
name '<umd-buildable-package>'
default_version '0.0.0'
# Git repository URL
source git: "https://github.com/openflighthpc/<umd-buildable-package>"
skip_transitive_dependency_licensing true
build do
env = with_standard_compiler_flags(with_embedded_path)
# Build flight-webapp-components and the login menu
block do
# Build <umd-buildable-package>
command "cd #{project_dir} && /opt/flight/bin/yarn install", env: env
command "cd #{project_dir} && /opt/flight/bin/yarn run build", env: env
# Build <static-js-file>
command "cd #{project_dir}/builder && /opt/flight/bin/yarn install", env: env
command "cd #{project_dir}/builder && /opt/flight/bin/yarn run build", env: env
# Move newly generated static js to install directory
FileUtils.mkdir_p(File.join(install_dir, "content", "js"))
command "cp " \
"#{project_dir}/builder/build/static/js/main.js " \
"#{install_dir}/content/js/imported-package.js"
# OPTIONAL: Move newly generated CSS file to install directory
command "cp " \
"#{project_dir}/builder/build/static/css/main.css " \
"#{install_dir}/content/styles/importedpackage.css"
end
# Clean up imported package dir
FileUtils.rm_rf(project_dir)
endThe above code block will:
- Clone the Github repository containing the UMD buildable package
- Create a Yarn build of the package
- Use the Yarn build to build/export a static JS file containing the desired component(s)
- Copy the built static files into the
content/[js/styles]/directory(s) in the built Omnibus package - Continue building the Omnibus package as normal.
The Vagrantfile also contains a definition for a simple VM you can use to test built packages. Manage and access the test VM using the test definition:
vagrant up test
vagrant ssh test
Similarly to the build VM, the build infrastructure is mounted within the guest at /vagrant where you can find the packages you built within the build VM, for e.g.:
sudo -s
cd /vagrant/builders/flight-example
yum install pkg/flight-example-someversion.el7.x86_64.rpm
Once a package is built, the result can be uploaded to a yum repository.
OpenFlightHPC contributors who have been provided with permissions to publish to the OpenFlight yum repositories on Amazon S3, can use the scripts/publish-rpm.sh script to push an RPM to the development repository.
First, add your AWS credentials:
aws configure
Next, publish the RPM that you've built:
scripts/publish-rpm.sh builders/flight-example/pkg/flight-example-someversion.rpm
Users of the openflight-dev repository will then have access to install the new RPM via yum on their systems.
Once the RPM has been tested, you can promote it to the live/production repo using the scripts/promote-rpm.sh script:
scripts/promote-rpm.sh flight-example-someversion
Note that the script does not take a file path, but takes a name pattern to match one or more RPMs that are present in the development repo.
Once the RPM is published to the production repo, users of the openflight repository will have access to install/upgrade to the new RPM via yum on their systems.
Fork the project. Make your feature addition or bug fix. Send a pull request. Bonus points for topic branches.
Read CONTRIBUTING.md for more details.
Eclipse Public License 2.0, see LICENSE.txt for details.
Copyright (C) 2019-present Alces Flight Ltd.
This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at https://www.eclipse.org/legal/epl-2.0, or alternative license terms made available by Alces Flight Ltd - please direct inquiries about licensing to [email protected].
OpenFlight Omnibus Builder is distributed in the hope that it will be useful, but WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. See the Eclipse Public License 2.0 for more details.