Skip to content

Latest commit

 

History

History
124 lines (100 loc) · 9.97 KB

README.md

File metadata and controls

124 lines (100 loc) · 9.97 KB
Raw

jaws-libp CI PyPI Docker

Reusable Python classes for JAWS plus admin scripts to setup and manage a deployment.

Quick Start with Compose

  1. Grab project
git clone https://github.com/JeffersonLab/jaws-libp
cd jaws-libp
  1. Launch Compose
docker compose up
  1. Monitor active alarms
docker exec -it cli list_activations --monitor
  1. Trip an alarm
docker exec cli set_activation alarm1

Note: The docker-compose services require significant system resources - tested with 4 CPUs and 4GB memory.

See: Docker Compose Strategy

Install

Requires Python 3.9+

pip install jaws-libp

Note: Using newer versions of Python may be problematic because the dependency confluent-kafka uses librdkafka, which often does not have a wheel file prepared for later versions of Python, meaning setuptools will attempt to compile it for you, and that often doesn't work (especially on Windows). Python 3.9 does have a wheel file for confluent-kafka so that's your safest bet. Wheel files also generally only are prepared for Windows, MacOS, and Linux. Plus only for architectures x86_64 and arm64, also only for glibc. If you use with musl libc or linux-aarch64 then you'll likely have to compile librdkafka yourself from source.

API

Sphinx Docs

Configure

Environment variables are used to configure jaws-libp:

Name Description
BOOTSTRAP_SERVER Host and port pair pointing to a Kafka server to bootstrap the client connection to a Kafka Cluster; example: kafka:9092
SCHEMA_REGISTRY URL to Confluent Schema Registry; example: http://registry:8081

The Docker container can optionally handle the following environment variables as well:

Name Description
ALARM_LOCATIONS Path to an alarm locations file to import (example file), else an https URL to a file, else a comma separated list of location definitions with fields separated by the pipe symbol. Example Inline CSV: name|parent
ALARM_SYSTEMS Path to an alarm categories file to import (example file), else an https URL to a file, else a comma separated list of system definitions with fields. Example Inline CSV: name
ALARM_ACTIONS Path to an alarm classes file to import (example file), else an https URL to a file, else a comma separated list of class definitions with fields separated by the pipe symbol. Example Inline CSV: name|category|priority|rationale|correctiveaction|latching|filterable|ondelayseconds|offdelayseconds
ALARMS Path to an alarm registration instances file to import (example file), else an https URL to a file, else a comma separated list of instance definitions with fields separated by the pipe symbol. Leave epicspv field empty for SimpleProducer. Example Inline CSV: name|action|epicspv|location|maskedby|screencommand
ALARMS_URL_CSV If provided, is a comma separated list of file names to append to ALARMS; ignored if ALARMS doesn't start with https; Example.
ALARM_OVERRIDES Path to an alarm overrides file to import (example file), else an https URL to a file.

Build

This Python 3.9+ project is built with setuptools and may be run using the Python virtual environment feature to isolate dependencies. The pip tool can be used to download dependencies.

git clone https://github.com/JeffersonLab/jaws-libp
cd jaws-libp
python -m venv .venv_dev --upgrade-deps

Activate the virtual env using your shell specific command, then install in editable mode with dev deps and run build:

# Windows
.venv_dev\Scripts\activate.bat
# UNIX (SH Shell)
source .venv_dev/bin/activate
# UNIX (CSH Shell)
source .venv_dev/bin/activate.csh


pip install -e ."[dev]"
python -m build
pylint --recursive=y src/*

Note for JLab On-Site Users: Jefferson Lab has an intercepting proxy

See: Python Development Notes

Develop

Set up the build environment following the Build instructions.

In order to iterate rapidly when making changes it's often useful to run the Python scripts directly on the local workstation, perhaps leveraging an IDE. In this scenario run the service dependencies with Docker Compose:

docker compose -f deps.yaml up

Note: The environment variable defaults work in this scenario and are defined as: BOOTSTRAP_SERVERS=localhost:9094 and SCHEMA_REGISTRY=http://localhost:8081

Test

The integration tests depend on a running Kafka instance, generally in Docker. The tests run automatically via the CI GitHub Action on every commit (unless [no ci] is included in the commit message). The tests can be run locally during development. Set up the development environment following the Develop instructions. Then with the deps.yaml Docker containers running and the build virtual environment activated run:

pytest

Release

  1. Bump the version number in the VERSION file and commit and push to GitHub (using Semantic Versioning).
  2. The CD GitHub Action should run automatically invoking:
    • The Create release GitHub Action to tag the source and create release notes summarizing any pull requests. Edit the release notes to add any missing details.
    • The Publish artifact GitHub Action to create a deployment artifact on PyPi.
    • The Publish docs GitHub Action to create Sphinx docs.
    • The Publish docker image GitHub Action to create a new demo Docker image.

See Also