Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate from Gramine to Intel TDX #1

Merged
merged 7 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 57 additions & 97 deletions .github/workflows/build-and-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,100 +13,60 @@ jobs:
build-and-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build Docker image
uses: docker/build-push-action@v4
with:
context: .
load: true
tags: |
my-proof:${{ github.run_number }}
my-proof:latest
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Clone and set up GSC
run: |
git clone https://github.com/gramineproject/gsc.git
cd gsc
python3 -m pip install --no-cache-dir 'docker>=7.1.0' 'jinja2>=3.1.4' 'tomli>=2.0.1' 'tomli-w>=1.0.0' 'pyyaml>=6.0.2'

- name: Create signing key
run: |
echo "${{ secrets.SIGNING_KEY }}" > signing_key.pem
chmod 600 signing_key.pem

- name: Build GSC image
run: |
cd gsc
./gsc build my-proof ../my-proof.manifest.template -c ../config.yaml

- name: Sign GSC image
run: |
cd gsc
./gsc sign-image my-proof ../signing_key.pem -c ../config.yaml

- name: Export GSC image to file
run: |
docker save gsc-my-proof:latest | gzip > gsc-my-proof-${{ github.run_number }}.tar.gz

- name: Generate verification data
run: |
cd gsc
./gsc info-image gsc-my-proof > ../sigstruct.txt

- name: Upload image
uses: actions/upload-artifact@v3
with:
name: gsc-my-proof-image
path: gsc-my-proof-${{ github.run_number }}.tar.gz

- name: Upload verification data
uses: actions/upload-artifact@v3
with:
name: gsc-my-proof-sigstruct
path: sigstruct.txt

- name: Generate release body
run: |
echo "MRSIGNER: $(grep -oP 'mr_signer = "\K[^"]*' sigstruct.txt)" >> release_body.txt
echo "MRENCLAVE: $(grep -oP 'mr_enclave = "\K[^"]*' sigstruct.txt)" >> release_body.txt
echo "Image SHA256: $(sha256sum gsc-my-proof-${{ github.run_number }}.tar.gz | cut -d' ' -f1)" >> release_body.txt

- name: Create Release and Upload Assets
uses: softprops/action-gh-release@v1
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: v${{ github.run_number }}
name: Release v${{ github.run_number }}
body_path: release_body.txt
draft: false
prerelease: false
files: |
./gsc-my-proof-${{ github.run_number }}.tar.gz
./sigstruct.txt

- name: Cleanup signing key
if: always()
run: |
rm -f signing_key.pem

- name: Log build result
if: always()
run: |
if [ ${{ job.status }} == "success" ]; then
echo "Build and release completed successfully"
else
echo "Build and release failed"
fi
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build Docker image
uses: docker/build-push-action@v4
with:
context: .
load: true
tags: |
my-proof:${{ github.run_number }}
my-proof:latest
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Export image to file
run: |
docker save my-proof:latest | gzip > my-proof-${{ github.run_number }}.tar.gz

- name: Generate release body
run: |
echo "Image SHA256: $(sha256sum my-proof-${{ github.run_number }}.tar.gz | cut -d' ' -f1)" >> release_body.txt

- name: Upload image
uses: actions/upload-artifact@v3
with:
name: my-proof-image
path: my-proof-${{ github.run_number }}.tar.gz

- name: Create Release and Upload Assets
uses: softprops/action-gh-release@v1
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: v${{ github.run_number }}
name: Release v${{ github.run_number }}
body_path: release_body.txt
draft: false
prerelease: false
files: |
./my-proof-${{ github.run_number }}.tar.gz

- name: Log build result
if: always()
run: |
if [ ${{ job.status }} == "success" ]; then
echo "Build and release completed successfully"
else
echo "Build and release failed"
fi
5 changes: 0 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
FROM python:3.12-slim

# Install any Python dependencies your application needs, e.g.:
RUN pip install --no-cache-dir requests

RUN mkdir /sealed && chmod 777 /sealed

WORKDIR /app

COPY . /app
Expand Down
103 changes: 27 additions & 76 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,112 +27,64 @@ This template provides a basic structure for building proof tasks that:
}
```

The project is designed to work with [Gramine](https://gramine.readthedocs.io/en/latest/), a lightweight library OS that enables running unmodified applications in secure enclaves, such as Intel SGX (Software Guard Extensions). This allows the code to run in a trusted execution environment, ensuring confidentiality and integrity of the computation.
The project is designed to work with Intel TDX (Trust Domain Extensions), providing hardware-level isolation and security guarantees for confidential computing workloads.

## Project Structure

- `my_proof/`: Contains the main proof logic
- `proof.py`: Implements the proof generation logic
- `__main__.py`: Entry point for the proof execution
- `proof.py`: Implements the proof generation logic
- `__main__.py`: Entry point for the proof execution
- `models/`: Data models for the proof system
- `demo/`: Contains sample input and output for testing
- `.github/workflows/`: CI/CD pipeline for building and releasing
- `Dockerfile`: Defines the container image for the proof task
- `my-proof.manifest.template`: Gramine manifest template for running securely in an Intel SGX enclave
- `config.yaml`: Configuration file for Gramine Shielded Containers (GSC)
- `requirements.txt`: Python package dependencies

## Getting Started

To use this template:

1. Fork this repository
2. Modify the `my_proof/proof.py` file to implement your specific proof logic
3. Update the `my-proof.manifest.template` if you need to add any additional files or change the configuration
3. Update the project dependencies in `requirements.txt` if needed
4. Commit your changes and push to your repository

## Customizing the Proof Logic

The main proof logic is implemented in `my_proof/proof.py`. To customize it, update the `Proof.generate()` function to change how input files are processed.

The proof can be configured using environment variables. When running in an enclave, the environment variables must be defined in the `my-proof.manifest.template` file as well. The following environment variables are used for this demo proof:
The proof can be configured using environment variables:

- `USER_EMAIL`: The email address of the data contributor, to verify data ownership

If you want to use a language other than Python, you can modify the Dockerfile to install the necessary dependencies and build the proof task in the desired language.

## Local Development

To run the proof locally, without Gramine, you can use Docker:
To run the proof locally for testing, you can use Docker:

```
```bash
docker build -t my-proof .
docker run \
--rm \
--volume $(pwd)/demo/sealed:/sealed \
--volume $(pwd)/demo/input:/input \
--volume $(pwd)/demo/output:/output \
--env [email protected] \
my-proof
--rm \
--volume $(pwd)/input:/input \
--volume $(pwd)/output:/output \
--env [email protected] \
my-proof
```

## Building and Releasing

This template includes a GitHub Actions workflow that automatically:

1. Builds a Docker image with your code
2. Creates a Gramine-shielded container (GSC) image
3. Publishes the GSC image as a GitHub release

**Important:** To use this workflow, you must generate a signing key and add it to your GitHub secrets. Follow these steps:

1. Generate a signing key (see instructions below)
2. Add the key as a GitHub secret named `SIGNING_KEY`
3. Push your changes to the `main` branch or create a pull request

### Generating the Gramine Signing Key (Required)

Before building and signing your graminized Docker image, you must generate a signing key. This key is crucial for creating secure SGX enclaves. Here's how to generate it:

1. If you have Gramine installed:

```
gramine-sgx-gen-private-key enclave-key.pem
```

2. If you don't have Gramine, use OpenSSL:
## Running with Intel TDX

```
openssl genrsa -3 -out enclave-key.pem 3072
```
Intel TDX (Trust Domain Extensions) provides hardware-based memory encryption and integrity protection for virtual machines. To run this container in a TDX-enabled environment, follow your infrastructure provider's specific instructions for deploying confidential containers.

After generating the key:
Common volume mounts and environment variables:

1. Keep this key secure, as it will be used to sign your enclaves.
2. Add the contents of `enclave-key.pem` as a GitHub secret named `SIGNING_KEY`.

This key is essential for the `gsc sign-image` step in the GSC workflow.

## Running with SGX

Intel SGX (Software Guard Extensions) is a set of security-related instruction codes built into modern Intel CPUs. It allows parts of a program to be executed in a secure enclave, isolated from the rest of the system.

To load a released image with docker, copy the URL from the release and run:

```
curl -L https://address/of/gsc-my-proof.tar.gz | docker load
```

To run the image:

```
```bash
docker run \
--rm \
--volume /gsc-my-proof/input:/input \
--volume /gsc-my-proof/output:/output \
--device /dev/sgx_enclave:/dev/sgx_enclave \
--volume /var/run/aesmd:/var/run/aesmd \
--volume /mnt/gsc-my-proof/sealed:/sealed \
--env [email protected] \
gsc-my-proof
--rm \
--volume /path/to/input:/input \
--volume /path/to/output:/output \
--env [email protected] \
my-proof
```

Remember to populate the `/input` directory with the files you want to process.
Expand All @@ -141,10 +93,9 @@ Remember to populate the `/input` directory with the files you want to process.

This template leverages several security features:

1. **Secure Enclaves**: The proof runs inside an SGX enclave, isolating it from the rest of the system.
2. **Encrypted Storage**: The `/sealed` directory is automatically encrypted/decrypted by Gramine, providing secure storage for sensitive data.
3. **Input/Output Isolation**: Input and output directories are mounted separately, ensuring clear data flow boundaries.
4. **Minimal Attack Surface**: The Gramine manifest limits the files and resources accessible to the enclave, reducing potential vulnerabilities.
1. **Hardware-based Isolation**: The proof runs inside a TDX-protected environment, isolating it from the rest of the system
2. **Input/Output Isolation**: Input and output directories are mounted separately, ensuring clear data flow boundaries
3. **Minimal Container**: Uses a minimal Python base image to reduce attack surface

## Customization

Expand All @@ -156,4 +107,4 @@ If you have suggestions for improving this template, please open an issue or sub

## License

[MIT License](LICENSE)
[MIT License](LICENSE)
55 changes: 0 additions & 55 deletions config.yaml

This file was deleted.

Loading