Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Hello Endpoint #41

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ab66a58
feat(api/cargo): pass through features to wire crate
gierens Sep 12, 2024
b11bf13
feat(api/cargo): add wire crate dependency
gierens Sep 12, 2024
6d1497f
feat(api/hello): add simplified message in hello endpoint and remove …
gierens Sep 12, 2024
b55595e
feat(api/openstack): derive Clone to ProjectMinimal
gierens Sep 12, 2024
106fa95
test(api/hello): adjust hello_works_with_valid_token
gierens Sep 12, 2024
752a0bd
feat(env): add .env file with DATABASE_URL
gierens Sep 12, 2024
b731040
refactor(env): revise for mariadb
gierens Sep 12, 2024
28e4116
feat(mariadb): add .mariadb.cnf with client settings
gierens Sep 12, 2024
f72a80f
feat(scripts): revise init_db.sh for mariadb
gierens Sep 12, 2024
0f705ec
feat(scripts): add mariadb.sh
gierens Sep 12, 2024
cfd2c04
feat(api/configuration): revise database settings for mariadb
gierens Sep 12, 2024
e65f27b
feat(api/cargo): change sqlx feature postgres to mariadb
gierens Sep 12, 2024
1f5c5f5
feat(api/configuration): revise DatabaseSettings and functions for ma…
gierens Sep 12, 2024
e0a1656
feat(api/startup): revise for mariadb, use sqlx mysql types
gierens Sep 12, 2024
7a01c1e
test(api/helpers): revise TestApp and configure_database for MariaDB
gierens Sep 12, 2024
882f96c
feat(ci/test): revise test job for mariadb
gierens Sep 12, 2024
b87073f
fix(env): replace db user use mariadb by root
gierens Sep 12, 2024
a7a0fd2
fix(ci/test): add DATABASE_URL to env in test job
gierens Sep 12, 2024
13a8d62
feat(ci/test): change SQLX_VERSION to latest
gierens Sep 12, 2024
fd96bf5
feat(api/migrations): add project table
gierens Sep 12, 2024
82b4bf9
test(api/helpers): remove leading underscore TestApp.db_pool
gierens Sep 12, 2024
4b39727
test(api/hello): add database_insert_works
gierens Sep 12, 2024
863d3f4
chore(deny): add ignore for rsa crate
gierens Sep 12, 2024
cc52a55
fix(ci/test): remove SQLX_VERSION from env
gierens Sep 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .deny.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ ignore = [
#{ crate = "[email protected]", reason = "you can specify why you are ignoring the yanked crate" },
{ id = "RUSTSEC-2024-0320", reason = "Nothing we can do about it now." },
{ id = "RUSTSEC-2024-0370", reason = "Nothing we can do about it now." },
{ id = "RUSTSEC-2023-0071", reason = "Nothing we can do about it now." },
]
# If this is true, then cargo deny will use the git executable to fetch advisory database.
# If this is false, then it uses a built-in git library.
Expand Down
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DATABASE_URL=mysql://root:password@localhost:3306/lrzcc
21 changes: 8 additions & 13 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,23 +19,18 @@ on:

env:
CARGO_TERM_COLOR: always
SQLX_VERSION: 0.7.3
SQLX_FEATURES: "rustls,postgres"
SQLX_FEATURES: "rustls,mysql"
DATABASE_URL: "mysql://root:[email protected]:3306/lrzcc"

jobs:
test:
name: test
runs-on: ubuntu-latest
services:
postgres:
image: postgres:14
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
POSTGRES_DB: postgres
ports:
- 5432:5432
steps:
- uses: getong/[email protected]
with:
mysql database: 'lrzcc'
mysql root password: 'password'
- name: Check out repository code
uses: actions/checkout@v4
- name: Rust Cache Action
Expand All @@ -49,8 +44,8 @@ jobs:
--features ${{ env.SQLX_FEATURES }}
--no-default-features
--locked
- name: Install postgresql-client and mold
run: sudo apt update && sudo apt install postgresql-client mold -y
- name: Install mariadb-client and mold
run: sudo apt update && sudo apt install mariadb-client mold -y
- name: Migrate database
run: SKIP_DOCKER=true ./scripts/init_db.sh
- name: Check sqlx offline data is up to date
Expand Down
7 changes: 7 additions & 0 deletions .mariadb.cnf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[client]
protocol = tcp
host = 127.0.0.1
port = 3306
user = root
password = password
database = lrzcc
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 9 additions & 8 deletions api/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@ path = "src/main.rs"
[features]
default = ["all"]
all = ["accounting", "budgeting", "hello", "pricing", "quota", "resources", "user"]
accounting = []
budgeting = []
hello = []
pricing = []
quota = []
resources = []
user = []
accounting = ["lrzcc-wire/accounting"]
budgeting = ["lrzcc-wire/budgeting"]
hello = ["lrzcc-wire/hello"]
pricing = ["lrzcc-wire/pricing"]
quota = ["lrzcc-wire/quota"]
resources = ["lrzcc-wire/resources"]
user = ["lrzcc-wire/user"]

[dependencies]
actix-web = "4"
Expand All @@ -49,6 +49,7 @@ config = "0.14"
uuid = { version = "1.10", features = ["v4", "serde"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
jzon = "0.12"
lrzcc-wire = { version = "1.0", path = "../wire" }

[dependencies.sqlx]
version = "0.8"
Expand All @@ -57,7 +58,7 @@ features = [
"runtime-tokio",
"tls-rustls",
"macros",
"postgres",
"mysql",
"uuid",
"chrono",
"migrate",
Expand Down
4 changes: 2 additions & 2 deletions api/configuration/base.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ application:
port: 8000
database:
host: "127.0.0.1"
port: 5432
username: "postgres"
port: 3306
username: "root"
password: "password"
database_name: "lrzcc"
openstack:
Expand Down
6 changes: 6 additions & 0 deletions api/migrations/20240912151726_create_project_table.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
CREATE TABLE project (
id INT PRIMARY KEY AUTO_INCREMENT,
name TEXT(255) UNIQUE NOT NULL,
openstack_id TEXT(255) UNIQUE NOT NULL,
user_class SMALLINT UNSIGNED NOT NULL
);
12 changes: 6 additions & 6 deletions api/src/configuration.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use secrecy::{ExposeSecret, Secret};
use serde_aux::field_attributes::deserialize_number_from_string;
use sqlx::postgres::{PgConnectOptions, PgSslMode};
use sqlx::mysql::{MySqlConnectOptions, MySqlSslMode};

#[derive(Clone, serde::Deserialize)]
pub struct Settings {
Expand Down Expand Up @@ -41,21 +41,21 @@ pub struct OpenStackSettings {
}

impl DatabaseSettings {
pub fn without_db(&self) -> PgConnectOptions {
pub fn without_db(&self) -> MySqlConnectOptions {
let ssl_mode = if self.require_ssl {
PgSslMode::Require
MySqlSslMode::Required
} else {
PgSslMode::Prefer
MySqlSslMode::Preferred
};
PgConnectOptions::new()
MySqlConnectOptions::new()
.host(&self.host)
.port(self.port)
.ssl_mode(ssl_mode)
.username(&self.username)
.password(self.password.expose_secret())
}

pub fn with_db(&self) -> PgConnectOptions {
pub fn with_db(&self) -> MySqlConnectOptions {
self.without_db().database(&self.database_name)
}
}
Expand Down
2 changes: 1 addition & 1 deletion api/src/openstack.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ pub struct OpenStack {
token: TokenHandler,
}

#[derive(Debug, serde::Deserialize)]
#[derive(Clone, Debug, serde::Deserialize)]
pub struct ProjectMinimal {
pub id: String,
pub name: String,
Expand Down
25 changes: 11 additions & 14 deletions api/src/routes/hello.rs
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
use crate::openstack::ProjectMinimal;
use actix_web::web::ReqData;
use actix_web::HttpResponse;
use lrzcc_wire::hello::Hello;

#[tracing::instrument(name = "hello")]
pub async fn hello() -> Result<HttpResponse, actix_web::Error> {
// TODO implement
Ok(HttpResponse::Ok().finish())
}

#[cfg(test)]
mod tests {
use super::*;

#[tokio::test]
async fn hello_works() {
let response = hello().await.unwrap();
assert!(response.status().is_success())
}
pub async fn hello(
project: ReqData<ProjectMinimal>,
) -> Result<HttpResponse, actix_web::Error> {
Ok(HttpResponse::Ok()
.content_type("application/json")
.json(Hello {
message: format!("Hello, user {}!", project.name),
}))
}
10 changes: 5 additions & 5 deletions api/src/startup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ use crate::routes::{health_check, hello};
use actix_web::{
dev::Server, middleware::from_fn, web, web::Data, App, HttpServer,
};
use sqlx::postgres::PgPoolOptions;
use sqlx::PgPool;
use sqlx::mysql::MySqlPoolOptions;
use sqlx::MySqlPool;
use std::net::TcpListener;
use tracing_actix_web::TracingLogger;

Expand Down Expand Up @@ -50,7 +50,7 @@ pub struct ApplicationBaseUrl(pub String);

async fn run(
listener: TcpListener,
db_pool: PgPool,
db_pool: MySqlPool,
base_url: String,
openstack: OpenStack,
) -> Result<Server, anyhow::Error> {
Expand Down Expand Up @@ -78,6 +78,6 @@ async fn run(
Ok(server)
}

pub fn get_connection_pool(configuration: &DatabaseSettings) -> PgPool {
PgPoolOptions::new().connect_lazy_with(configuration.with_db())
pub fn get_connection_pool(configuration: &DatabaseSettings) -> MySqlPool {
MySqlPoolOptions::new().connect_lazy_with(configuration.with_db())
}
28 changes: 27 additions & 1 deletion api/tests/api/hello.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use lrzcc_wire::hello::Hello;
use uuid::Uuid;

use crate::helpers::spawn_app;
Expand Down Expand Up @@ -52,7 +53,8 @@ async fn hello_works_with_valid_token() {
let app = spawn_app().await;
let client = reqwest::Client::new();
let token = Uuid::new_v4().to_string();
app.mock_keystone_auth(&token, "project_id", "project_name")
let project_name = "project_name";
app.mock_keystone_auth(&token, "project_id", project_name)
.mount(&app.keystone_server)
.await;

Expand All @@ -66,4 +68,28 @@ async fn hello_works_with_valid_token() {

// assert
assert_eq!(response.status().as_u16(), 200);
assert_eq!(
response.headers().get("Content-Type").unwrap(),
"application/json"
);
let hello =
serde_json::from_str::<Hello>(&response.text().await.unwrap()).unwrap();
assert_eq!(hello.message, format!("Hello, user {}!", project_name));
}

#[tokio::test]
async fn database_insert_works() {
// arrange
let app = spawn_app().await;

// act and assert
sqlx::query!(
"INSERT INTO project (name, openstack_id, user_class) VALUES (?, ?, ?)",
"test",
"some-uuid",
4,
)
.execute(&app.db_pool)
.await
.expect("Failed to insert user.");
}
24 changes: 14 additions & 10 deletions api/tests/api/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use lrzcc_api::startup::{get_connection_pool, Application};
use lrzcc_api::telemetry::{get_subscriber, init_subscriber};
use once_cell::sync::Lazy;
use serde_json::json;
use sqlx::{Connection, Executor, PgConnection, PgPool};
use sqlx::{Connection, Executor, MySqlConnection, MySqlPool};
use uuid::Uuid;
use wiremock::matchers::{header, method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
Expand Down Expand Up @@ -31,7 +31,7 @@ static TRACING: Lazy<()> = Lazy::new(|| {
pub struct TestApp {
pub address: String,
pub _port: u16,
pub _db_pool: sqlx::PgPool,
pub db_pool: sqlx::MySqlPool,
pub _api_client: reqwest::Client,
pub keystone_server: MockServer,
pub keystone_token: String,
Expand Down Expand Up @@ -70,7 +70,7 @@ pub async fn spawn_app() -> TestApp {

let configuration = {
let mut c = get_configuration().expect("Failed to read configuration.");
c.database.database_name = Uuid::new_v4().to_string();
c.database.database_name = Uuid::new_v4().simple().to_string();
c.application.port = 0;
c.openstack.keystone_endpoint = keystone_server.uri();
c
Expand Down Expand Up @@ -102,30 +102,34 @@ pub async fn spawn_app() -> TestApp {
let test_app = TestApp {
address: format!("http://127.0.0.1:{}", application_port),
_port: application_port,
_db_pool: get_connection_pool(&configuration.database),
db_pool: get_connection_pool(&configuration.database),
_api_client: client,
keystone_server,
keystone_token,
};
test_app
}

async fn configure_database(config: &DatabaseSettings) -> PgPool {
async fn configure_database(config: &DatabaseSettings) -> MySqlPool {
// Create database
let mut connection = PgConnection::connect_with(&config.without_db())
let mut connection = MySqlConnection::connect_with(&config.without_db())
.await
.expect("Failed to connect to Postgres.");
.expect("Failed to connect to MariaDB.");
connection
.execute(
format!(r#"CREATE DATABASE "{}";"#, config.database_name).as_str(),
format!(
"CREATE DATABASE IF NOT EXISTS `{}`;",
config.database_name
)
.as_str(),
)
.await
.expect("Failed to create database.");

// Migrate database
let connection_pool = PgPool::connect_with(config.with_db())
let connection_pool = MySqlPool::connect_with(config.with_db())
.await
.expect("Failed to connect to Postgres.");
.expect("Failed to connect to MariaDB.");
sqlx::migrate!("./migrations")
.run(&connection_pool)
.await
Expand Down
34 changes: 16 additions & 18 deletions scripts/init_db.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
set -x
set -eo pipefail

if ! [ -x "$(command -v psql)" ]; then
echo >&2 "Error: psql is not installed."
if ! [ -x "$(command -v mariadb)" ]; then
echo >&2 "Error: mariadb-client is not installed."
exit 1
fi

Expand All @@ -15,32 +15,30 @@ if ! [ -x "$(command -v sqlx)" ]; then
exit 1
fi

DB_USER=${POSTGRES_USER:=postgres}
DB_PASSWORD="${POSTGRES_PASSWORD:=password}"
DB_NAME="${POSTGRES_DB:=lrzcc}"
DB_PORT="${POSTGRES_PORT:=5432}"
DB_HOST="${MARIADB_HOST:=127.0.0.1}"
DB_USER=${MARIADB_USER:=root}
DB_PASSWORD="${MARIADB_PASSWORD:=password}"
DB_NAME="${MARIADB_DB:=lrzcc}"
DB_PORT="${MARIADB_PORT:=3306}"

if [[ -z "${SKIP_DOCKER}" ]]
then
docker run \
-e POSTGRES_USER="${DB_USER}" \
-e POSTGRES_PASSWORD="${DB_PASSWORD}" \
-e POSTGRES_DB="${DB_NAME}" \
-p "${DB_PORT}":5432 \
-d postgres \
postgres -N 1000
-e MARIADB_ROOT_PASSWORD="${DB_PASSWORD}" \
-e MARIADB_DB="${DB_NAME}" \
-p "${DB_PORT}":3306 \
-d mariadb:latest
fi

export PGPASSWORD="${DB_PASSWORD}"
until psql -h "localhost" -U "${DB_USER}" -p "${DB_PORT}" -c '\q'; do
>&2 echo "Postgres is still unavailable - sleeping"
until mariadb -h "${DB_HOST}" -P "${DB_PORT}" -u "${DB_USER}" -p"${DB_PASSWORD}" -D "" -e "QUIT"; do
>&2 echo "MariaDB is still unavailable - sleeping"
sleep 1
done

>&2 echo "Postgres is up and running on port ${DB_PORT}!"
>&2 echo "MariaDB is up and running on ${DB_HOST} on port ${DB_PORT}!"

export DATABASE_URL=postgresql://${DB_USER}:${DB_PASSWORD}@localhost:${DB_PORT}/${DB_NAME}
export DATABASE_URL=mysql://${DB_USER}:${DB_PASSWORD}@${DB_HOST}:${DB_PORT}/${DB_NAME}
sqlx database create
sqlx migrate run

>&2 echo "Postgres has been migrated, ready to go!"
>&2 echo "MariaDB has been migrated, ready to go!"
Loading
Loading