- Adds the new parameter
--use_private_docker_repository
to launch jobs using private docker images from a linked docker.io account.
- Updates GET workflows endpoint to v3.
- Adds the new
--accelerate-file-staging
parameter to job submission to add support for AWS S3 mountpoint for quicker file staging.
- Adds new
--archived
flag tocloudos job list
to allow getting archived jobs list.
- Updates GET job list to use API v2 endpoint.
- Adds the new parameter
--nextflow-version
to select the Nextflow version for job submissions. - Now
--cloudos-url
can also take URLs with a trailing/
- Updates API requests to only use API key via HTTP header. Done in preparation for the upcoming deprecation of API key via parameters in CloudOS API.
- Updates queue support to disallow queue selection on fixed-queue workflows.
- Now, the default
cloudos job run
command will save job process logs. To prevent saving process logs, you can use the new flag--do-not-save-logs
. - Removes unsupported
--spot
option fromcloudos job run
.
- Adds the new parameter
--workflow-docs-link
to add a documentation link to the imported workflow.
- Adds
workflow import
command, allowing user to import Nextflow workflows to CloudOS.
- Adds support for using CloudOS HPC executor.
- Changes the default Nextflow executor to be AWSbatch. Now, the
--batch
flag is no longer necessary (although it's maintained for backwards compatibility) and a new--ignite
flag is created to support ignite if available. - Selects the CloudOS workspace default queue, when no valid
--job-queue
is provided.
- Discards archived worflows from the executable ones, as they will always produce an error.
- Adds an error strategy to retry GET/POST requests on time-out errors.
- Improve parsing of parameters added with
-p
. Now they can include internal=
symbols.
- adds Azure support
- adds job queue support to batch runs
- add queue list command
- add run-curated-examples command
- add commitizen support
- add project list
- add workflows list --curated option
- Feature:
cloudos job list
has the new parameter--last-n-jobs n
, if used, the lastn
jobs from the user will be collected. Default is last 30, which was the previous behaviour.
- Removes some default fields returned from
cloudos job list
command in preparation for its deprecation from the CloudOS API. In particular, the following fields were removed:resumeWorkDir
project.user
project.team
- Remove all cohort browser functionality that will be maintained in a separated repository.
- Patch: fixes problems with CloudOS environments using the new API specification for
projects
endpoint while maintaining backwards compatibility.
- Patch: fixes
BarRequestException
andTimeOutException
messages when the response from the API server is empty.
- All Cromwell functionality works now with personal API key. The
--cromwell-token
argument is maintained for backwards compatibility, but can be completely substituted by--apikey
. - Changes
--wdl-importsfile
parameter to be optional even when running a WDL pipeline asimportsFiles
are not always present in WDL pipelines. - Fixes some incomplete error messages.
- Modifies default
--cost-limit
from infinite (-1
) to30.0
. This will prevent wasting resources without a purpose of running a pipeline.
- Adds
--disable-ssl-verification
new flag to be able to disable SSL certificate verification when required. It also disablesurllib3
associated warning messages. - Adds
--ssl-cert
new option to specify the path to the corresponding SSL certificate file.
- Adds
--request-interval
new parameter to allow the custom time specification for job status request. This will be useful for big jobs, to specify a bigger interval since a smaller one is causing the API to consider it as spam or simply to crash. - Changes
REQUEST_INTERVAL
forREQUEST_INTERVAL_CROMWELL
. This is only used in thecromwell
workflows.
- Adds
--parameter / -p
new argument to allow to specify the job parameters using the command-line. This version introduces a backwards incompatible change The -p flag is now used for parameters and not for the nextflow profile. Commands that utilised -p for denoting a profile will break with this release.
- Unittests added for method
load
andcreate
from classCloudos
- Adds
--cost-limit <float>
tocloudos job run
command. It is used to indicate the job cost limit, in $.
- Adds worked example of CohortBrowser to README
- Adds WDL pipeline support, iteration 2: WDL workflows can be run
using the regular
cloudos job run
using the new arguments:--wdl-mainfile
--wdl-importsfile
--cromwell-token
- Adds the new argument
--repository-platform
to specify the repository platform (Default: 'github').
- Adds WDL pipeline support, iteration 1: cromwell server managing.
Now, a new command
cloudos cromwell
is available, with the following subcommands:- status
- start
- stop
- Adds
cloudos workflow list
command. This command allows to collect all the workflows data from a given workspace. - Adds JSON output for
cloudos job list
andcloudos workflow list
commands.
- Adds support for lustre storage with the new
--storage-mode
and--lustre-size
parameters.
- Adds
--nextflow-profile
parameter to accept nextflow profiles. It also makes--job-config
parameter optional, as a run with only profiles is possible.
- Hotfix: extends the wait time from 1s to 60s when checking for job
status (
--wait-completion true
). This helps preventing API call errors from CloudOS API server.
- Adds support for aborted jobs
- Adds
--batch
option tojob
subtool to be able to usebatch
executor instead of the defaultignite
in CloudOS.
- Unittests added for method
process_job_list
from classCloudos
- Unittests added for method
convert_nextflow_to_json
from classJobs
- Adds Cohort class
- Adds
git-commit
and--git-tag
optional arguments tocloudos job run
to be able to set the github commit or tag to run.
- Changes
--job-params
to--job-config
- Removes the collection of the
project.description
column from the returned json when listing all jobs, as this column is not available in all the CloudOS workspaces.
- Adds
cloudos job list
command. - Minor changes in
stdout
of the other commands to improve readability. - Adds a small docstring to each command.
- Refactors
runjob
andjobstatus
commands. Now, the maincloudos
tool have thejob
subtool which in turn has itsrun
andstatus
commands performing the previous functionality. This way, now the tool can be used with:cloudos job run [OPTIONS]
andcloudos job status [OPTIONS]
. - Adding
--wait-completion
option tocloudos job run
command, to be able to wait until job completion or failure.
Initial implementation of the cloudos
python package:
- Implements
runjob
andjobstatus
commands to send jobs and get their status, respectively.