bring-custom-container.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab: Bring your own custom container with Amazon SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "Here, we’ll show how to package a simple Python example which showcases the decision tree algorithm from the widely used scikit-learn machine learning package. The example is purposefully fairly trivial since the point is to show the surrounding structure that you’ll want to add to your own code so you can train and host it in Amazon SageMaker.\n",
    "\n",
    "The ideas shown here will work in any language or environment. You’ll need to choose the right tools for your environment to serve HTTP requests for inference, but good HTTP environments are available in every language these days.\n",
    "\n",
    "In this example, we use a single image to support training and hosting. This is easy because it means that we only need to manage one image and we can set it up to do everything. Sometimes you’ll want separate images for training and hosting because they have different requirements. Just separate the parts discussed below into separate Dockerfiles and build two images. Choosing whether to have a single image or two images is really a matter of which is more convenient for you to develop and manage.\n",
    "\n",
    "If you’re only using Amazon SageMaker for training or hosting, but not both, there is no need to build the unused functionality into your container."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the container\n",
    "Docker provides a simple way to package arbitrary code into an image that is totally self-contained. Once you have an image, you can use Docker to run a container based on that image. Running a container is just like running a program on the machine except that the container creates a fully self-contained environment for the program to run. Containers are isolated from each other and from the host environment, so the way you set up your program is the way it runs, no matter where you run it.\n",
    "\n",
    "Amazon SageMaker uses Docker to allow users to train and deploy arbitrary algorithms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parts of the container\n",
    "In the container directory are all the components you need to package the sample algorithm for Amazon SageMager:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    ".\n",
    "|-- Dockerfile\n",
    "|-- build_and_push.sh\n",
    "`-- decision_trees\n",
    "    |-- nginx.conf\n",
    "    |-- predictor.py\n",
    "    |-- serve\n",
    "    |-- train\n",
    "    `-- wsgi.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s discuss each of these in turn:\n",
    "\n",
    "- Dockerfile describes how to build your Docker container image. More details below.\n",
    "- build_and_push.sh is a script that uses the Dockerfile to build your container images and then pushes it to ECR. We’ll invoke the commands directly later in this notebook, but you can just copy and run the script for your own algorithms.\n",
    "- decision_trees is the directory which contains the files that will be installed in the container.\n",
    "- local_test is a directory that shows how to test your new container on any computer that can run Docker, including an Amazon SageMaker notebook instance. Using this method, you can quickly iterate using small datasets to eliminate any structural bugs before you use the container with Amazon SageMaker. We’ll walk through local testing later in this notebook.\n",
    "\n",
    "In this simple application, we only install five files in the container.\n",
    "\n",
    "The files that we’ll put in the container are:\n",
    "\n",
    "- nginx.conf is the configuration file for the nginx front-end. Generally, you should be able to take this file as-is.\n",
    "- predictor.py is the program that actually implements the Flask web server and the decision tree predictions for this app. You’ll want to customize the actual prediction parts to your application. Since this algorithm is simple, we do all the processing here in this file, but you may choose to have separate files for implementing your custom logic.\n",
    "- serve is the program started when the container is started for hosting. It simply launches the gunicorn server which runs multiple instances of the Flask app defined in predictor.py. You should be able to take this file as-is.\n",
    "- train is the program that is invoked when the container is run for training. You will modify this program to implement your training algorithm.\n",
    "- wsgi.py is a small wrapper used to invoke the Flask app. You should be able to take this file as-is.\n",
    "\n",
    "In summary, the two files you will probably want to change for your application are train and predictor.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Dockerfile\n",
    "The Dockerfile describes the image that we want to build. You can think of it as describing the complete operating system installation of the system that you want to run. A Docker container running is quite a bit lighter than a full operating system, however, because it takes advantage of Linux on the host machine for the basic operations.\n",
    "\n",
    "For the Python science stack, we will start from a standard Ubuntu installation and run the normal tools to install the things needed by scikit-learn. Finally, we add the code that implements our specific algorithm to the container and set up the right environment to run under."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #1\n",
    "\n",
    "!curl http://d1rwcpsuqsa5hl.cloudfront.net/scikit_bring_your_own.zip --output scikit_bring_your_own.zip\n",
    "!unzip scikit_bring_your_own.zip\n",
    "!mv scikit_bring_your_own/data ./\n",
    "!mv scikit_bring_your_own/container ./\n",
    "!rm scikit_bring_your_own.zip\n",
    "!rm -rf scikit_bring_your_own\n",
    "!cat container/Dockerfile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building and registering the container"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%sh\n",
    "\n",
    "# Cell #2\n",
    "\n",
    "\n",
    "# The name of our algorithm\n",
    "algorithm_name=sagemaker-decision-trees\n",
    "\n",
    "cd container\n",
    "\n",
    "chmod +x decision_trees/train\n",
    "chmod +x decision_trees/serve\n",
    "\n",
    "account=$(aws sts get-caller-identity --query Account --output text)\n",
    "\n",
    "# Get the region defined in the current configuration (default to us-west-2 if none defined)\n",
    "region=$(aws configure get region)\n",
    "region=${region:-us-west-2}\n",
    "\n",
    "fullname=\"${account}.dkr.ecr.${region}.amazonaws.com/${algorithm_name}:latest\"\n",
    "\n",
    "# If the repository doesn't exist in ECR, create it.\n",
    "aws ecr describe-repositories --repository-names \"${algorithm_name}\" > /dev/null 2>&1\n",
    "\n",
    "if [ $? -ne 0 ]\n",
    "then\n",
    "    aws ecr create-repository --repository-name \"${algorithm_name}\" > /dev/null\n",
    "fi\n",
    "\n",
    "# Get the login command from ECR and execute it directly\n",
    "$(aws ecr get-login --region ${region} --no-include-email)\n",
    "\n",
    "# Build the docker image locally with the image name and then push it to ECR\n",
    "# with the full name.\n",
    "\n",
    "docker build  -t ${algorithm_name} .\n",
    "docker tag ${algorithm_name} ${fullname}\n",
    "\n",
    "docker push ${fullname}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the container\n",
    "Here we specify a bucket to use and the role that will be used for working with SageMaker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #3\n",
    "\n",
    "# S3 prefix\n",
    "prefix = 'DEMO-scikit-byo-iris'\n",
    "\n",
    "# Define IAM role\n",
    "import boto3\n",
    "import re\n",
    "\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The session remembers our connection parameters to SageMaker. We’ll use it to perform all of our SageMaker operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #4\n",
    "\n",
    "import sagemaker as sage\n",
    "from time import gmtime, strftime\n",
    "\n",
    "sess = sage.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When training large models with huge amounts of data, you’ll typically use big data tools, like Amazon Athena, AWS Glue, or Amazon EMR, to create your data in S3. For the purposes of this example, we’re using some the classic Iris dataset, which we have included.\n",
    "\n",
    "We can use use the tools provided by the SageMaker Python SDK to upload the data to a default bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #5\n",
    "\n",
    "WORK_DIRECTORY = 'data'\n",
    "\n",
    "data_location = sess.upload_data(WORK_DIRECTORY, key_prefix=prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to use SageMaker to fit our algorithm, we’ll create an Estimator that defines how to use the container to train. This includes the configuration we need to invoke SageMaker training:\n",
    "\n",
    "- The container name. This is constructed as in the shell commands above.\n",
    "- The role. As defined above.\n",
    "- The instance count which is the number of machines to use for training.\n",
    "- The instance type which is the type of machine to use for training.\n",
    "- The output path determines where the model artifact will be written.\n",
    "- The session is the SageMaker session object that we defined above.\n",
    "\n",
    "Then we use fit() on the estimator to train against the data that we uploaded above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #6\n",
    "\n",
    "account = sess.boto_session.client('sts').get_caller_identity()['Account']\n",
    "region = sess.boto_session.region_name\n",
    "image = '{}.dkr.ecr.{}.amazonaws.com/sagemaker-decision-trees:latest'.format(account, region)\n",
    "\n",
    "tree = sage.estimator.Estimator(image,\n",
    "                       role, train_instance_count=1, train_instance_type='ml.c4.2xlarge',\n",
    "                       output_path=\"s3://{}/output\".format(sess.default_bucket()),\n",
    "                       sagemaker_session=sess)\n",
    "\n",
    "tree.fit(data_location)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hosting your model\n",
    "You can use a trained model to get real time predictions using HTTP endpoint. Follow these steps to walk you through the process.\n",
    "\n",
    "Deploying the model to SageMaker hosting just requires a deploy call on the fitted model. This call takes an instance count, instance type, and optionally serializer and deserializer functions. These are used when the resulting predictor is created on the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #7\n",
    "\n",
    "from sagemaker.predictor import csv_serializer\n",
    "predictor = tree.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge', serializer=csv_serializer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing test data to run inferences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to do some predictions, we’ll extract some of the data we used for training and do predictions against it. This is, of course, bad statistical practice, but a good way to see how the mechanism works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #8\n",
    "\n",
    "shape=pd.read_csv(\"data/iris.csv\", header=None)\n",
    "shape.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #9\n",
    "\n",
    "# drop the label column in the training set\n",
    "shape.drop(shape.columns[[0]],axis=1,inplace=True)\n",
    "shape.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #10\n",
    "\n",
    "import itertools\n",
    "\n",
    "a = [50*i for i in range(3)]\n",
    "b = [40+i for i in range(10)]\n",
    "indices = [i+j for i,j in itertools.product(a,b)]\n",
    "\n",
    "test_data=shape.iloc[indices[:-1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run predictions\n",
    "\n",
    "Prediction is as easy as calling predict with the predictor we got back from deploy and the data we want to do predictions with. The serializers take care of doing the data conversions for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #11\n",
    "\n",
    "print(predictor.predict(test_data.values).decode('utf-8'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup\n",
    "After completing the lab, use these steps to [delete the endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-cleanup.html) or run the following code\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cell #12\n",
    "sess.delete_endpoint(predictor.endpoint)"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7-final"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python37364bitcd2bc172d6a4400c81a493876fe75516",
   "display_name": "Python 3.7.3 64-bit"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}