Have you ever written a shell script and found yourself thinking "if only the Bash had an include mechanism", while copy-pasting the command line parser you wrote for a different shell script? Then you should try toolbox for your next shell script!
Toolbox is a simple framework for Bash that lets you modularize
your scripts. At the heart of toolbox is the include() function,
which works much like the import statement in Python. Once you
source toolbox.sh, you can use include() to load the modules
that ship with toolbox, or create your own ones.
All you have to do is source toolbox.sh and you can start using
the include() function. For example, the following code is a
minimal publisher using the uipc module.
. toolbox.sh
include "uipc"
endp=$(uipc_endpoint_open)
uipc_endpoint_publish "$endp" "mytopic" "Hello world"
And this is the corresponding subscriber.
. toolbox.sh
include "uipc"
endp=$(uipc_endpoint_open)
uipc_endpoint_subscribe "$endp" "mytopic"
if msg=$(uipc_endpoint_recv "$endp"); then
data=$(uipc_msg_get_data "$msg")
echo "Received message: $data"
fi
Toolbox comes with a number of modules that solve common problems. The following is a list of the default modules and what they are used for.
| Module | Purpose |
|---|---|
| array | Comparing, searching, and sorting arrays |
| conf | Configuration handling |
| git | Interacting with git repositories |
| inst | Writing daemons |
| ipc | Message-based IPC (GPG-signed messages) |
| is | Checking the value of variables |
| json | Handling JSON data |
| log | Logging and debugging |
| mutex | Locks for protecting shared resources (similar to pthread mutexes) |
| opt | Command line parsing |
| queue | "Thread"-safe FIFOs |
| sem | Process synchronization (similar to POSIX semaphores) |
| uipc | Message-based IPC (unsigned messages) |
| wmutex | Weak (owner-less) locks |
Modules are described in detail in the module reference
There are two ways to install toolbox: using apt or from the source.
If you are using a Debian-based distribution, you can install toolbox through apt.
First, import the GPG key used to sign packages in the repository and make sure you
have apt-transport-https installed.
# curl --get https://deb.m10k.eu/deb.m10k.eu.gpg > /etc/apt/trusted.gpg.d/deb.m10k.eu.gpg
# apt-get install apt-transport-https
Then add the following line to your /etc/apt/sources.lst.
deb https://deb.m10k.eu stable main
If you prefer a more recent (and maybe slightly more unstable version), use the
unstable suite instead.
deb https://deb.m10k.eu unstable main
Next, update your package index using the following command.
# apt-get update
Now you can install and update toolbox as you're used to.
# apt-get install toolbox
The packages in the repository are automatically built from the stable and unstable branches, so the Debian packages are usually no more than a few minutes older than the sources.
To install toolbox from git, check out the sources and run make install.
$ git clone https://github.com/m10k/toolbox
$ cd toolbox
$ sudo make install
You will also need to create the toolbox and toolbox_ipc groups and change ownership on /var/lib/toolbox and /var/lib/toolbox/ipc.
# chown root:toolbox /var/lib/toolbox
# chown root:toolbox_ipc /var/lib/toolbox/ipc
To use the ipc module, membership in toolbox and toolbox_ipc is necessary. You can add a user to these groups with the following command.
# usermod -a -G toolbox,toolbox_ipc USERNAME
The ipc module uses GPG to sign and verify messages, so you also need to make sure that you have a keypair. You can generate one with the following command.
$ gpg --full-generate-key
When toolbox is installed, toolbox.sh is placed in PATH, where it will
be picked up by the source (aka .) shell command, allowing you to load
toolbox without worrying about the path of any other files.
When loading a module, include() will search the following paths (in
order) for modules:
$HOME/.toolbox/include/usr/share/toolbox/include
This allows you to place custom modules in your home directory, while the default modules are in a system-wide path.
Bash does not have a notion of modules, so toolbox tries its best to provide something that looks like a module, feels like a module, and tastes like a module, but is not actually a module. For this to work, modules have to follow strict guidelines.
A module is a single shell script with one or more of the following:
| Element | Description |
|---|---|
| Constructor | Module initialization function called by include() |
| Public functions | Functions that may be called by users |
| Private functions | Functions that users must not call |
| Private variables | Variables that users must not use |
In shell, everything shares the same namespace. Thus, the most important thing about a module is that it does not declare functions or variables outside of its own namespace. The following table summarizes the naming convention for modules.
| Element | Naming convention | Example |
|---|---|---|
| Module name | Lower-case letters | example |
| Module file name | Module name + .sh |
example.sh |
| Constructor | __init() |
__init() |
| Public function | Module name + _ + function name |
example_open() |
| Private function | _ + module name + _ + function name |
_example_write() |
| Private variable | __ + module name + _ + variable name |
__example_mode |
Functions intended to be called by users are prefixed with the module name and an
underscore. Functions that users must not call because they might change between
versions at the developers' whim (that is, private functions) are prefixed with an
underscore, the module name, and another underscore. Variables are prefixed with two
underscores, the module name, and another underscore. Variables declared by modules
are always considered private and must not be used directly by users. Further,
because users may execute functions in sub-shells, modules must export their
variables (declare -x). Module developers are also encouraged to declare their
variables as integers (declare -i), arrays (declare -a), or associative arrays
(declare -A) when they are any of those types. Variables that are not expected to
change (i.e. constants) must be declared as such using declare -r or readonly.
All global variables must be declared in the module constructor, unless that is
absolutely not feasible. All variables used in module functions must be declared
using local unless they are intended to be global.
The following code example illustrates the coding guidelines for modules.
#
# logrun.sh - Toolbox module for logging commands
#
__init() {
if ! include "log"; then
return 1
fi
declare -gxr __logrun_timestamp_format="%Y-%m-%d %H:%M:%S"
return 0
}
_logrun_get_timestamp() {
if ! date +"$__logrun_timestamp_format"; then
log_error "Call to date failed. Out of memory?"
return 1
fi
return 0
}
_logrun_write_line() {
local data="$1"
local timestamp
if ! timestamp=$(_logrun_get_timestamp); then
return 1
fi
echo "$timestamp $data" 1>&2
return 0
}
_logrun_log() {
local data="$1"
local line
if (( $# < 1 )); then
data=$(< /dev/stdin)
fi
while read -r line; do
if ! _logrun_write_line "$line"; then
return 1
fi
done <<< "$data"
return 0
}
logrun_exec() {
local cmd="$1"
local args=("${@:2}")
local stdout
local retval
_logrun_log "Executing: $cmd ${args[*]}"
stdout=$("$cmd" "${args[@]}")
retval="$?"
_logrun_log "$cmd returned $retval"
_logrun_log "===== BEGIN stdout ====="
_logrun_log <<< "$stdout"
_logrun_log "===== END stdout ====="
return "$retval"
}
There are additional modules in the following repositories. These modules have been removed from this repository to reduce the dependencies of the toolbox package.
| Repository | Purpose |
|---|---|
| toolbox-goodies | Nice-to-have modules |
| toolbox-linux | Linux-specific modules |
| toolbox-restapis | Modules for interacting with REST-ful APIs |