Skip to content

Commit

Permalink
v1.1
Browse files Browse the repository at this point in the history
- rate limiting
- systemd DynamicUser compatibility (replaced SQLITE_FILE_NAME with SQLITE_FILE_PATH)
- Update readme
- Version info in api docs
  • Loading branch information
KerekesDavid committed Sep 3, 2024
1 parent f43d364 commit 9560822
Show file tree
Hide file tree
Showing 5 changed files with 65 additions and 28 deletions.
4 changes: 2 additions & 2 deletions .env
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Name of the database file to write into.
SQLITE_FILE_NAME="database.db"
SQLITE_FILE_PATH="databases/database.db"

# A list of secret keys. The server won't accept events that do not contain one of these keys in the request body.
API_KEYS=["I-am-an-unsecure-dev-key-REPLACE_ME"]

# Requests from the same IP above this rate will be rejected.
# See https://limits.readthedocs.io/en/stable/quickstart.html#rate-limit-string-notation
RATE_LIMIT="60/minute"
RATE_LIMIT="60/minute"
61 changes: 42 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,42 +122,58 @@ It can be opened using any sqlite database browser, or in python using the built
My personal choice for performing data analytics is a [jupyter notebook](https://jupyter.org/) using [pandas](https://pandas.pydata.org/). They have a wonerful cheat sheet [here](https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf).
## Launching a Production Server
Setting up a permanent server as a service is also quite simple:
- Generate an API key:
```
openssl rand -base64 24
```
This will stop random people from logging events in your database, it will not stop a script kiddie who can decompile the key from your app, or pluck it from network traffic. I'd suggest creating a new one for every version of your application, and retiring old ones after a while.
Setting up a permanent server as a service is also quite simple.
The method I share here has some minimal extra complications, but it ensures some level of separation from other parts of the system using systemd's `DynamicUser` parameter. It might come handy in case there is a vulnerability in FastAPI.
- Replace `API_KEYS=["I-am-an-unsecure-dev-key-REPLACE_ME"]` in the .env file with the newly generated key.
(Contributions to this section are very welcome, I'm barely a fledgeling server admin.)
- Install pycolytics, and set up a virtualenv.
A usual place for this would be `/srv/pycolytics` for example.
- Create a systemd service file: `/etc/systemd/system/pycolytics.service`
```
[Unit]
Description=Gunicorn instance to serve pycolytics
Description=Uvicorn instance serving Pycolytics
After=network.target
[Service]
User=your_username
Group=your_groupname
WorkingDirectory=/path/to/pycolytics
Environment="PATH=/path/to/venv/bin"
ExecStart=/path/to/venv/bin/gunicorn app -w 4 -k uvicorn.workers.UvicornWorker
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true
Type=simple
DynamicUser=yes
User=pycolytics
WorkingDirectory=/srv/pycolytics
StateDirectory=pycolytics/databases
ExecStart=/srv/pycolytics/.venv/bin/uvicorn \
--workers=4 \
--host=0.0.0.0 \
--port=8080 \
app.main:app
ExecReload=/bin/kill -HUP ${MAINPID}
RestartSec=15
Restart=always
[Install]
WantedBy=multi-user.target
```
- Generate an API key:
```
openssl rand -base64 24
```
This will stop random people from logging events in your database, it will not stop a script kiddie who can decompile the key from your app, or pluck it from network traffic. I'd suggest creating a new one for every version of your application, and retiring old ones after a while.
- Setup the .env file:
- Replace `API_KEYS=["I-am-an-unsecure-dev-key-REPLACE_ME"]` with the newly generated key.
- Set the database path to the systemd state directory: `SQLITE_FILE_NAME="/var/lib/pycolytics/databases/database.db"`
- Reload the systemd to read the config:
- Run read the new config:
```sudo systemctl daemon-reload```
- Enable the service start on boot:
- Make the service start on boot:
```sudo systemctl enable pycolytics```
Expand All @@ -169,6 +185,13 @@ Setting up a permanent server as a service is also quite simple:
```sudo systemctl status pycolytics```
- In case you need to fix configurations and restart the service use:
```sudo systemctl daemon-reload
sudo systemctl restart pycolytics
```
Most online guides also recommend setting up fastapi behind an nginx reverse proxy, in case somebody tries to DDOS your server. I've never been successful enough for this to happen, so I'll leave it to you to figure out the details.
## Planned Features
- HTTPS communication for you security nerds out there
Expand Down
7 changes: 4 additions & 3 deletions app/config.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
from functools import lru_cache
from pydantic_settings import BaseSettings, SettingsConfigDict
import typing
import pydantic


class Settings(BaseSettings):
sqlite_file_name: str = "fallback.db"
sqlite_file_path: str = "databases/fallback.db"
api_keys: list[str] = []
rate_limit: str = "60/minute"

Expand All @@ -16,6 +15,8 @@ def get_settings():
return Settings()


# Stored in hex so somebody a bit overeager doesn't replace it here by accident.
# This will be publicly available on the API doc so that wouldn't be the best.
default_dev_key = bytes.fromhex(
"492d616d2d616e2d756e7365637572652d6465762d6b65792d5245504c4143455f4d45"
).decode("utf-8")
2 changes: 1 addition & 1 deletion app/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

settings = get_settings()

sqlite_url = f"sqlite+aiosqlite:///databases/{settings.sqlite_file_name}"
sqlite_url = f"sqlite+aiosqlite:///{settings.sqlite_file_path}"
connect_args = {"check_same_thread": False}
engine = sqlalchemy.ext.asyncio.create_async_engine(
sqlite_url, echo=True, connect_args=connect_args
Expand Down
19 changes: 16 additions & 3 deletions app/main.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
__version__ = "v1.1.0"
__api_version__ = "v1.0.0"

from contextlib import asynccontextmanager

import fastapi
Expand All @@ -24,9 +27,19 @@ async def lifespan(app: fastapi.FastAPI):
await create_db_and_tables()
yield

description = f"""
Running Pycolitics __{__version__}__
Serving API version: __{__api_version__}__
"""

limiter = slowapi.Limiter(key_func=slowapi.util.get_remote_address)
app = fastapi.FastAPI(lifespan=lifespan)
app = fastapi.FastAPI(
lifespan=lifespan,
title="Pycolytics Event API",
version=__api_version__,
description=description,
)
app.state.limiter = limiter
app.add_exception_handler(
slowapi.errors.RateLimitExceeded, slowapi._rate_limit_exceeded_handler
Expand All @@ -39,7 +52,7 @@ async def log_event(
*,
session: AsyncSession = fastapi.Depends(get_session),
event: EventCreate,
request: fastapi.Request
request: fastapi.Request,
):
db_event = Event.model_validate(event)
session.add(db_event)
Expand All @@ -52,7 +65,7 @@ async def log_events(
*,
session: AsyncSession = fastapi.Depends(get_session),
events: list[EventCreate],
request: fastapi.Request
request: fastapi.Request,
):
db_events = [Event.model_validate(event).model_dump() for event in events]
# Pylance freaks out if I use exec here, says it can't take an Executable
Expand Down

0 comments on commit 9560822

Please sign in to comment.