Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add start script and local db config #13

Merged
merged 4 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions newswires/app/AppComponents.scala
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,11 @@ class AppComponents(context: Context)
.build()

if (context.environment.mode == Mode.Dev) {
// TODO run against a local DB?
Database.configureRemoteDevDb(ssmClient)
if (sys.env.contains("USE_CODE_DB")) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not super happy about using an environment variable here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we discussed in-person and ideally we might instead have an environment variable more specific to connect differences, such as USE_IAM_AUTH but let's save for a future PR

Database.configureRemoteDevDb(ssmClient)
} else {
Database.configureLocalDevDB()
}
} else if (context.environment.mode == Mode.Prod) {
Database.configureDeployedDb(configuration)
}
Expand Down
21 changes: 21 additions & 0 deletions newswires/app/conf/Database.scala
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,25 @@ object Database extends Logging {

ConnectionPool.singleton(new DataSourceConnectionPool(ds))
}

def configureLocalDevDB(): Unit = {
logger.info("building DB config for connecting to local DB")

val username = "postgres"
val port = "5432"
val address = "localhost"
val databaseName = "newswires"

val ds = new AwsWrapperDataSource()
ds.setJdbcProtocol("jdbc:postgresql:")
ds.setServerName(address)
ds.setDatabase(databaseName)
ds.setServerPort(port)
ds.setTargetDataSourceClassName("org.postgresql.ds.PGSimpleDataSource")
ds.setPassword("postgres")
ds.setUser(username)

ConnectionPool.singleton(new DataSourceConnectionPool(ds))
}

}
152 changes: 152 additions & 0 deletions scripts/start
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#!/usr/bin/env bash

set -e

green='\x1B[0;32m'
red='\x1B[0;31m'
plain='\x1B[0m' # No Color

AWS_PROFILE='editorial-feeds'
POSTGRES_PORT='5432'
APP_NAME='newswires'
EXISTING_TUNNELS=$(ps -ef | grep ssh | grep ${POSTGRES_PORT} | grep -v grep || true)

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
ROOT_DIR=${DIR}/..


# copied over from the Grid. Not 100% sure of the rationale, but perhaps: https://stackoverflow.com/questions/56520354/getting-an-amazonkinesisexception-status-code-502-when-using-localstack-from-ja
export AWS_CBOR_DISABLE=true

for arg in "$@"; do
if [ "$arg" == "--debug" ]; then
IS_DEBUG=true
shift
fi
if [ "$arg" == "--use-CODE" ]; then
USE_CODE=true
shift
fi
done

isInstalled() {
hash "$1" 2>/dev/null
}

hasCredentials() {
STATUS=$(aws sts get-caller-identity --profile ${AWS_PROFILE} 2>&1 || true)
if [[ ${STATUS} =~ (ExpiredToken) ]]; then
echo -e "${red}Credentials for the ${AWS_PROFILE} profile are expired. Please fetch new credentials and run this again.${plain}"
exit 1
elif [[ ${STATUS} =~ ("could not be found") ]]; then
echo -e "${red}Credentials for the ${AWS_PROFILE} profile are missing. Please ensure you have the right credentials.${plain}"
exit 1
fi
}

checkRequirement() {
if ! isInstalled "$1"; then
echo -e "${red}[MISSING DEPENDENCY] $1 not found. Please install $1${plain}"
exit 1
fi
}

checkRequirements() {
# server side
checkRequirement java
checkRequirement sbt

# client side
checkRequirement npm

# used for postgres and AWS localstack
checkRequirement docker

# other
checkRequirement nginx
checkRequirement aws
}

tunnelToAwsDb() {
if (docker stats --no-stream &> /dev/null); then
docker compose down
fi
if [[ -n $EXISTING_TUNNELS ]]; then
echo "RE-USING EXISTING TUNNEL TO CODE POSTGRES (on port ${POSTGRES_PORT})"
else
ssm ssh -t ${APP_NAME},CODE -p ${AWS_PROFILE} -x --newest --rds-tunnel 5432:${APP_NAME},CODE
echo "TUNNEL ESTABLISHED TO CODE POSTGRES (on port ${POSTGRES_PORT})"
fi
}

startDockerContainers() {
if [[ $EXISTING_TUNNELS ]]; then
echo "KILLING EXISTING TUNNEL TO CODE POSTGRES (on port ${POSTGRES_PORT})"
# shellcheck disable=SC2046
kill $(echo "${EXISTING_TUNNELS}" | awk '{print $2}')
fi
docker compose up -d
}

startIngestionLambda() {
pushd "$ROOT_DIR/ingestion-lambda"
npm install
npm run dev
popd
}

startPlayApp() {
# pushd to find build.sbt and allow this script to be executed from any location (but ideally from the project root)
echo "========================================================="
echo "= Press cmd-C then cmd-D to stop"
echo "========================================================="
pushd "$ROOT_DIR/${APP_NAME}"
if [ "$IS_DEBUG" == true ] ; then
SBT_OPTS="-jvm-debug 5005"
fi
if [[ $USE_CODE == true ]]; then
sbt "${SBT_OPTS} run"
else
sbt "${SBT_OPTS} run"
fi
popd
}

checkNodeVersion() {
runningNodeVersion=$(node -v)
requiredNodeVersion=$(cat "$ROOT_DIR/.nvmrc")

if [ "$runningNodeVersion" != "$requiredNodeVersion" ]; then
echo -e "${red}Using wrong version of Node. Required ${requiredNodeVersion}. Running ${runningNodeVersion}.${plain}"
exit 1
fi
}

checkForJavaHome() {
echo "Checking JAVA_HOME"
if [[ -z "$JAVA_HOME" ]]; then
echo " JAVA_HOME not set, please set it before continuing"
echo " This can be done by adding \"export JAVA_HOME=\$(/usr/libexec/java_home)\" to ~/.profile"
exit 1
else
echo " JAVA_HOME is set to $JAVA_HOME"
fi
}

main() {
checkForJavaHome
hasCredentials
checkRequirements
checkNodeVersion
if [ "$USE_CODE" == "true" ]; then
echo "Using CODE. Starting play app with tunnel to AWS DB."
tunnelToAwsDb && USE_CODE_DB=true startPlayApp
else
echo "Using the local stack. Starting Docker, Ingestion Lambda and Play App."
# run all three, and terminate together when the user presses `ctrl+c`: https://stackoverflow.com/a/52033580
(trap 'kill 0' SIGINT; startDockerContainers && startIngestionLambda & startPlayApp & wait)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to broadly work as expected, but I've noticed a couple of bugs with my shell when subsequently doing interactive stuff in the same shell session, so I think this needs a bit more investigation..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had trouble with backgrounding via & before, let's see how painful it gets when using day-to-day and improve in a future PR if needs be

fi

}

main