Cross-Facility Federated Learning (xFFL) is a federated learning (FL) framework based on the StreamFlow workflow management system (WMS) developed in the Parallel Computing [Alpha] research group at the University of Turin, Italy.
xFFL is designed to be a research-oriented software, enabling FL workloads deployment on geographically distributed computing infrastructures including, but not limited to, high-performance computers (HPCs).
We aim to democratize extensive ML workload development, allowing researchers and SMEs to be competitive with big tech companies by exploiting sparse computing power.
Clone this repository on the computing infrastructure that will act as the federation orchestrator:
git clone --depth 1 --branch main https://github.com/alpha-unito/xffl.gitNote that this machine should have internet access and be capable of reaching the federation clients through SSH.
Create and activate a Python virtual environment (both venv and conda are valid options):
python3 -m venv xFFL
source xFFL/bin/activateInstall xFFL and its requirements:
cd xffl/
python3 -m pip install .Create an xFFL configuration file in a guided way through the xFFL CLI:
xffl config --w [workdir] --p [project]Where workdir is the name of the working directory to create (defaults to the local directory), and project is the name of the created deployment.
This process will produce a directory containing the necessary StreamFlow and xFFL configuration files and the chosen model.
Python>=3.9
Singularity/Docker
NodeJS
xFFL implements a centralized FL schema in which a central server coordinates, distributes, and aggregates the learning tasks deployed on the clients. Such a process is orchestrated by StreamFlow, which handles the data movement between the computing infrastructures and the deployment of workloads.
| Name | Role | |
|---|---|---|
| Gianluca Mittone | [email protected] | Creator and maintainer |
| Alberto Mulone | [email protected] | Developer and maintainer |
| Iacopo Colonnelli | [email protected] | Investigator |
| Robert Birke | [email protected] | Investigator |
| Marco Aldinucci | [email protected] | Principal investigator |