opencv.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1a721ae2241bbcdf",
   "metadata": {},
   "source": [
    "# Course 1: Getting Started with Computer Vision\n",
    "\n",
    "In this notebook, we are learning OpenCV to perform some basic operations on images.\n",
    "\n",
    "The session starts at 13 a.m.\n",
    "\n",
    "## Before we start\n",
    "\n",
    "We should install OpenCV first. You can install it by running the following command in your terminal:\n",
    "\n",
    "```bash\n",
    "pip install opencv-python\n",
    "```\n",
    "\n",
    "### Parts\n",
    "\n",
    "1. Recognition: Recognise the object, including classification;\n",
    "2. Target Detection: Count the amount, check the position;\n",
    "3. Action Recognition: Recognise the action, including the gesture;\n",
    "4. Generate: Generate the image via prompts.\n",
    "\n",
    "### Application\n",
    "\n",
    "Facial Recognition, Facial Detection (please tell recognition (recognize who the person is) and detection (determine whether the object is a person) apart), Object Detection, Image Classification, Image Generation, etc.\n",
    "\n",
    "The Computer Vision is the process of the machine automatically processing the image or video and get the information from it.\n",
    "\n",
    "### Coordination System\n",
    "\n",
    "The left-top corner is the origin, and the x-axis is from left to right, the y-axis is from top to bottom, starting from `0`.\n",
    "\n",
    "We can regard the image as a matrix. Typically, the Gray image is a 2D matrix, and the RGB image is a 3D matrix.\n",
    "\n",
    "In `OpenCV`, it uses `BGR`, whose order is [Blue, Green, Red], instead of `RGB`.\n",
    "\n",
    "We sometimes use the coordinate system (x, y) to check out a point, but in a matrix, the same point is (y, x).\n",
    "\n",
    "## OpenCV - A Brief Introduction\n",
    "\n",
    "The basic operation of OpenCV includes the reading image, converting color, etc. It is quite easy so that I won't expand it here.\n",
    "\n",
    "### The Basic Action of Image\n",
    "\n",
    "1. Read the image;\n",
    "2. Show the image;\n",
    "3. Save the image;\n",
    "4. Close the window.\n",
    "5. Convert the color.\n",
    "6. Get the pixel value.\n",
    "7. Set the pixel value.\n",
    "8. Get the image information.\n",
    "9. Cut the image.\n",
    "10. Merge the image.\n",
    "11. Resize the image.\n",
    "12. Rotate the image.\n",
    "13. Flip the image.\n",
    "14. Draw the shape.\n",
    "15. Add the text.\n",
    "16. Add the border.\n",
    "\n",
    "#### File I/O"
   ]
  },
  {
   "cell_type": "code",
   "id": "e65a4eb2467a7676",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:05.482402Z",
     "start_time": "2024-07-15T07:42:04.706650Z"
    }
   },
   "source": [
    "import cv2\n",
    "\n",
    "image = cv2.imread('opencv/cats.jpeg')\n",
    "\n",
    "cv2.imshow('Cats', image)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()\n",
    "\n",
    "cv2.imwrite('opencv/cats.png', image)"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 16
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Convert the Color\n",
    "\n",
    "Via `cv2.cvtColor`, we can convert the color of the image. The first parameter is the image, and the second parameter is the color space conversion code."
   ],
   "id": "38ce1942253aefc5"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:06.054792Z",
     "start_time": "2024-07-15T07:42:05.483497Z"
    }
   },
   "cell_type": "code",
   "source": [
    "gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
    "hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n",
    "\n",
    "cv2.imshow('Gray', gray)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "e03085de79bd3e5f",
   "outputs": [],
   "execution_count": 17
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Pixel Access\n",
    "\n",
    "Because the image is a matrix, we can access the pixel value via the index of the matrix."
   ],
   "id": "93486039e4252af3"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:06.565858Z",
     "start_time": "2024-07-15T07:42:06.055427Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import numpy as np\n",
    "\n",
    "pixel_114_514 = image[114, 514]\n",
    "print(pixel_114_514)\n",
    "image[114, 514] = np.array([191, 98, 10])\n",
    "\n",
    "cv2.imshow('Cats', image)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "cfd23502a133ec60",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[252 252 252]\n"
     ]
    }
   ],
   "execution_count": 18
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Image Information\n",
    "\n",
    "We can get the image information via the `shape` attribute.\n",
    "\n",
    "As for EXIF, we can use the `exifread` library."
   ],
   "id": "7da7a9479def71ef"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:06.569075Z",
     "start_time": "2024-07-15T07:42:06.567028Z"
    }
   },
   "cell_type": "code",
   "source": "image.shape",
   "id": "95ad12cd01fcdba3",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(768, 1024, 3)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 19
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Image Cropping\n",
    "\n",
    "We can crop the image via the slicing operation."
   ],
   "id": "28d980827691ceaa"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:07.433187Z",
     "start_time": "2024-07-15T07:42:06.569629Z"
    }
   },
   "cell_type": "code",
   "source": [
    "target = image[100:200, 200:300]\n",
    "\n",
    "cv2.imshow('Target', target)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "a5bf3c93f2464f68",
   "outputs": [],
   "execution_count": 20
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Image Enhancement\n",
    "\n",
    "We can perform the image enhancement via rotating, flipping, resizing, etc."
   ],
   "id": "3d01d887f7f5946d"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:42:09.663927Z",
     "start_time": "2024-07-15T07:42:07.434700Z"
    }
   },
   "cell_type": "code",
   "source": [
    "rotated = cv2.rotate(image, cv2.ROTATE_90_CLOCKWISE)\n",
    "flipped = cv2.flip(image, 1)\n",
    "resized = cv2.resize(image, (300, 300))\n",
    "\n",
    "cv2.imshow('Rotated', rotated)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()\n",
    "\n",
    "cv2.imshow('Flipped', flipped)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()\n",
    "\n",
    "cv2.imshow('Resized', resized)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "ff1ee8f0228671c6",
   "outputs": [],
   "execution_count": 21
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Drawing\n",
    "\n",
    "There are commonly 3 types of drawing: line, rectangle, and text.\n",
    "\n",
    "- Line: `cv2.line`\n",
    "- Rectangle: `cv2.rectangle`\n",
    "- Text: `cv2.putText`"
   ],
   "id": "d6e08e9d5ab586c1"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:43:14.526190Z",
     "start_time": "2024-07-15T07:43:12.583134Z"
    }
   },
   "cell_type": "code",
   "source": [
    "with_line = image.copy()\n",
    "cv2.line(with_line, (0, 0), (200, 200), (255, 0, 0), 5)\n",
    "\n",
    "cv2.imshow('With Line', with_line)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()\n",
    "\n",
    "with_rectangle = image.copy()\n",
    "cv2.rectangle(with_rectangle, (100, 100), (200, 200), (0, 255, 0), 3)\n",
    "\n",
    "cv2.imshow('With Rectangle', with_rectangle)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()\n",
    "\n",
    "with_text = image.copy()\n",
    "cv2.putText(with_text, 'Hello, OpenCV!', (100, 100), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)\n",
    "\n",
    "cv2.imshow('With Text', with_text)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "ceb3c472408530a5",
   "outputs": [],
   "execution_count": 22
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Video Operations\n",
    "\n",
    "Via `cv2.VideoCapture`, we can read the video and perform some operations on it. If your laptop has a camera, you can use it to capture the video as id `0`.\n",
    "\n",
    "We should release the camera after using it."
   ],
   "id": "4af464b0d2d9e8c8"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:44:44.636465Z",
     "start_time": "2024-07-15T07:44:37.541227Z"
    }
   },
   "cell_type": "code",
   "source": [
    "camera = cv2.VideoCapture(0)\n",
    "\n",
    "while camera.isOpened():\n",
    "    ret, frame = camera.read()\n",
    "    if not ret:\n",
    "        break\n",
    "    cv2.imshow('Frame', frame)\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break\n",
    "\n",
    "camera.release()"
   ],
   "id": "eb540fb4cd600a8f",
   "outputs": [],
   "execution_count": 24
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Video Reading\n",
    "\n",
    "We can read the video via `cv2.VideoCapture` and get the frame via `read`."
   ],
   "id": "77b6ddf9378efe69"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:45:42.586721Z",
     "start_time": "2024-07-15T07:45:38.892448Z"
    }
   },
   "cell_type": "code",
   "source": [
    "video = cv2.VideoCapture('opencv/orange.mp4')\n",
    "\n",
    "while video.isOpened():\n",
    "    ret, frame = video.read()\n",
    "    if not ret:\n",
    "        break\n",
    "    cv2.imshow('Frame', frame)\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break\n",
    "\n",
    "video.release()"
   ],
   "id": "afe2d043c67b049",
   "outputs": [],
   "execution_count": 25
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Color Detection\n",
    "\n",
    "We can detect the color via the HSV color space. The HSV color space is more suitable for color detection.\n",
    "\n",
    "The HSV color space includes three components:\n",
    "\n",
    "- Hue: The color type, which is represented by the angle.\n",
    "- Saturation: The purity of the color.\n",
    "- Value: The brightness of the color.\n",
    "\n",
    "The range of HSV is:\n",
    "\n",
    "- Hue: [0, 179]\n",
    "- Saturation: [0, 255]\n",
    "- Value: [0, 255]\n",
    "\n",
    "We can use the trackbar to adjust the value of the color."
   ],
   "id": "ef46675b5f235a22"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "outputs": [],
   "execution_count": null,
   "source": "hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)",
   "id": "e928d2c07283e5af"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Masks\n",
    "\n",
    "We can use the mask to filter the color. The mask is a binary image, which is used to filter the color.\n",
    "\n",
    "We can use the `cv2.inRange` function to get the mask.\n",
    "\n",
    "The first parameter is the image, and the second and third parameters are the lower bound and upper bound of the color.\n",
    "\n",
    "The function will return a binary image, which is the mask.\n",
    "\n",
    "We can use the `cv2.bitwise_and` function to get the result.\n",
    "\n",
    "For example, we are fetching the green one in OpenCV logo."
   ],
   "id": "c3750ad122618b17"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "",
   "id": "7f6c40e2c51d65a7"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T07:52:21.449477Z",
     "start_time": "2024-07-15T07:52:20.579205Z"
    }
   },
   "cell_type": "code",
   "source": [
    "lower = np.array([40, 50, 50])\n",
    "upper = np.array([80, 255, 255])\n",
    "\n",
    "image = cv2.imread('opencv/opencvlogo.jpg')\n",
    "\n",
    "hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n",
    "\n",
    "mask = cv2.inRange(hsv, lower, upper)\n",
    "\n",
    "result = cv2.bitwise_and(image, image, mask=mask)\n",
    "\n",
    "cv2.imshow('Mask', mask)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "dcba5c0f038c6185",
   "outputs": [],
   "execution_count": 29
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Edge Detection\n",
    "\n",
    "Via convolution, we can detect the border of the object.\n",
    "\n",
    "Edge detection is an important research area in image processing and computer vision, especially in feature extraction.The purpose of edge detection is to identify the points in digital images with more obvious brightness changes, which usually exist between the target and background regions, and are an important basis for image segmentation.\n",
    "Commonly used algorithms for edge detection are: Sobel algorithm, Laplace algorithm, Canny algorithm and so on.\n",
    "\n",
    "#### Sobel algorithm\n",
    "\n",
    "![sobel](./opencv/sobel.png)\n",
    "\n",
    "Sobel algorithm simulates the first-order derivatives by weighting the gray levels of the upper, lower, left and right neighboring points in the spatial neighborhood of a pixel, and the larger the derivatives are, the more drastic the change is, and the more likely it is to be an edge.\n",
    "\n",
    "Sobel edge detection is usually directional and can detect only horizontal or vertical edges or both."
   ],
   "id": "f85a2188b30e2576"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T08:07:21.637147Z",
     "start_time": "2024-07-15T08:07:18.301878Z"
    }
   },
   "cell_type": "code",
   "source": [
    "image = cv2.imread('opencv/cats.jpeg')\n",
    "\n",
    "gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "core = np.array([\n",
    "    [-1, 0, 1],\n",
    "    [-2, 0, 2],\n",
    "    [-1, 0, 1]\n",
    "])\n",
    "\n",
    "result = cv2.filter2D(gray, -1, core)\n",
    "\n",
    "cv2.imshow('Result', result)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "8ea3f6b4777657c",
   "outputs": [],
   "execution_count": 31
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "The cat is so cute!",
   "id": "6176b5b7268231a6"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Contour Detection\n",
    "\n",
    "A contour is a series of curves of connected points that represent the basic shape of an object.\n",
    "When we talk about contours, we think about edges, and they are indeed very similar. Simply put, contours are continuous and edges are not all continuous (below). Edges are mainly used as features of an image, while contours are mainly used to analyze the shape of an object, such as the perimeter and area of an object, etc. It can be said that edges include contours. The operation of finding contours is generally used for black and white images, so usually threshold segmentation or Canny edge detection is used to get a binary image first.\n",
    "\n",
    "![](./opencv/contour.png)\n",
    "\n",
    "Before the detection, you should binarization the image via `cv2.threshold` or masks."
   ],
   "id": "3ab5a92c7d2408c7"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T08:13:50.384651Z",
     "start_time": "2024-07-15T08:13:50.317007Z"
    }
   },
   "cell_type": "code",
   "source": [
    "masked = cv2.imread('opencv/mask.png')\n",
    "gray = cv2.cvtColor(masked, cv2.COLOR_BGR2GRAY)\n",
    "ret, thresh = cv2.threshold(gray, 240, 255, cv2.THRESH_BINARY)\n",
    "\n",
    "contours, hierarchy = cv2.findContours(\n",
    "    thresh,\n",
    "    cv2.RETR_EXTERNAL,\n",
    "    cv2.CHAIN_APPROX_SIMPLE)\n",
    "\n",
    "cv2.drawContours(masked, contours, -1, (0, 255, 0), 3)\n",
    "\n",
    "x, y, w, h = cv2.boundingRect(contours[0])\n",
    "cv2.rectangle(masked, (x, y), (x + w, y + h), (0, 255, 255), 2)\n",
    "cv2.circle(masked, (x + w // 2, y + h // 2), 5, (255, 0, 0), -1)\n",
    "\n",
    "cv2.imshow('Contours', masked)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "d32db3da0c41ab4",
   "outputs": [],
   "execution_count": 36
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Then you can count the area, which can reduce some noise.",
   "id": "a3b59ebae0b5d7"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T08:14:25.718538Z",
     "start_time": "2024-07-15T08:14:25.715600Z"
    }
   },
   "cell_type": "code",
   "source": "cv2.contourArea(contours[0])",
   "id": "9a7fcfa50e7f6922",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3630.5"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 37
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Tasks\n",
    "\n",
    "### Task 1: Trace the ping-pong ball\n",
    "\n",
    "The task is to trace the ping-pong ball in the video.\n",
    "\n",
    "The color of the ping-pong ball is `orange`, and the color space is `HSV`.\n",
    "\n",
    "#### Color Detection\n",
    "\n",
    "First of all, we should handle the mouse action to know the color range:"
   ],
   "id": "b09292f371c8fb5b"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T08:30:35.858555Z",
     "start_time": "2024-07-15T08:30:32.192542Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import cv2\n",
    "import numpy as np\n",
    "\n",
    "cv2.namedWindow('mouse trace')\n",
    "\n",
    "def get_color(event, x, y, flags, param):\n",
    "    if event == cv2.EVENT_LBUTTONDOWN:\n",
    "        print(hsv[y, x])\n",
    "\n",
    "video = cv2.VideoCapture('opencv/orange.mp4')\n",
    "\n",
    "while video.isOpened():\n",
    "    ret, frame = video.read()\n",
    "    if not ret:\n",
    "        break\n",
    "    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)\n",
    "    cv2.imshow('mouse trace', frame)\n",
    "    cv2.setMouseCallback('mouse trace', get_color)\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break"
   ],
   "id": "f6406733454cff95",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[151  32 231]\n",
      "[  5 175 179]\n"
     ]
    }
   ],
   "execution_count": 2
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "##### The Color Range\n",
    "\n",
    "The color range is:\n",
    "\n",
    "- Lower Bound: [ 0, 50, 50]\n",
    "- Upper Bound: [15, 255, 255]\n",
    "\n",
    "Then we can detect the orange ball via the mask."
   ],
   "id": "b9f205ed124b0d59"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T08:40:33.329245Z",
     "start_time": "2024-07-15T08:40:29.071312Z"
    }
   },
   "cell_type": "code",
   "source": [
    "lower = np.array([0, 160, 160])\n",
    "upper = np.array([10, 200, 200])\n",
    "\n",
    "video = cv2.VideoCapture('opencv/orange.mp4')\n",
    "\n",
    "while video.isOpened():\n",
    "    ret, frame = video.read()\n",
    "    if not ret:\n",
    "        break\n",
    "    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)\n",
    "    mask = cv2.inRange(hsv, lower, upper)\n",
    "    result = cv2.bitwise_and(frame, frame, mask=mask)\n",
    "    \n",
    "    ret, thresh = cv2.threshold(mask, 127, 255, 0)\n",
    "    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n",
    "    cv2.drawContours(frame, contours, -1, (0, 255, 0), 3)\n",
    "    \n",
    "    x, y, w, h = cv2.boundingRect(contours[0])\n",
    "    cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 255), 2)\n",
    "    \n",
    "    cv2.imshow('Result', frame)\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break"
   ],
   "id": "a94a3e61d0e294b1",
   "outputs": [],
   "execution_count": 8
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Task 2: Traffic Light Detection\n",
    "\n",
    "The task is to detect the traffic light in the image, which includes Red, Yellow, and Green, and reflect whether the light is on.\n",
    "\n",
    "#### Color Detection\n",
    "\n",
    "Through the color detection, we can get the mask of the traffic light."
   ],
   "id": "aa7794e1b52ee1e0"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T10:21:58.166035Z",
     "start_time": "2024-07-15T10:21:50.555556Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import cv2\n",
    "\n",
    "image = cv2.imread('opencv/redlight.jpg')\n",
    "\n",
    "hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n",
    "\n",
    "def onMouse(event, x, y, flags, param):\n",
    "    if event == cv2.EVENT_LBUTTONDOWN:\n",
    "        print(hsv[y, x])\n",
    "\n",
    "cv2.namedWindow('Traffic Light')\n",
    "cv2.setMouseCallback('Traffic Light', onMouse)\n",
    "\n",
    "cv2.imshow('Traffic Light', image)\n",
    "cv2.waitKey(0)\n",
    "cv2.destroyAllWindows()"
   ],
   "id": "8b7832388f3d3833",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 27 219 100]\n",
      "[ 27 219 100]\n",
      "[ 26 214 105]\n",
      "[ 26 222 108]\n",
      "[ 26 222 108]\n",
      "[ 26 222 108]\n",
      "[ 23 220 108]\n",
      "[12 93 63]\n",
      "[ 23 209  84]\n",
      "[ 23 143  82]\n",
      "[ 24 211 105]\n",
      "[ 24 226 106]\n",
      "[ 28 244 203]\n",
      "[ 29 242 198]\n",
      "[ 28 239 197]\n"
     ]
    }
   ],
   "execution_count": 40
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Handler for detecting the traffic light\n",
    "\n",
    "Because the active light is like white, we decided to recognize the topest 2 inactive light."
   ],
   "id": "2cd74e5644565a59"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T10:45:06.052839Z",
     "start_time": "2024-07-15T10:45:06.039771Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from cv2.typing import MatLike\n",
    "\n",
    "def determine_color(image: MatLike):\n",
    "    image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n",
    "    \n",
    "    image = image[50:200, 720:1083]\n",
    "    \n",
    "    color_list = {\n",
    "        'red1': ([170, 90, 30], [180, 200, 80]),\n",
    "        'red2': ([0, 90, 30], [10, 200, 100]),\n",
    "        'yellow': ([20, 150, 90], [30, 230, 100]),\n",
    "        'green': ([50, 150, 20], [65, 230, 80])\n",
    "    }\n",
    "    \n",
    "    def detect_light(image, color):\n",
    "        lower = np.array(color_list[color][0])\n",
    "        upper = np.array(color_list[color][1])\n",
    "        \n",
    "        mask = cv2.inRange(image, lower, upper)\n",
    "    \n",
    "        result = cv2.bitwise_and(image, image, mask=mask)\n",
    "        \n",
    "        ret, thresh = cv2.threshold(mask, 127, 255, 0)\n",
    "        contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n",
    "        cv2.drawContours(image, contours, -1, (255, 255, 255), 3)\n",
    "    \n",
    "        if len(contours) == 0:\n",
    "            return image, 0\n",
    "        \n",
    "        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
    "        \n",
    "        # Get the biggest contour\n",
    "        contours = sorted(contours, key=cv2.contourArea, reverse=True)\n",
    "        \n",
    "        amount = cv2.contourArea(contours[0])\n",
    "        \n",
    "        if len(contours) > 0:\n",
    "            x, y, w, h = cv2.boundingRect(contours[0])\n",
    "            cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 255), 2)\n",
    "        \n",
    "        return image, amount\n",
    "    \n",
    "    result1 = detect_light(image, 'red1')\n",
    "    result2 = detect_light(image, 'red2')\n",
    "    result3 = detect_light(image, 'yellow')\n",
    "    result4 = detect_light(image, 'green')\n",
    "    \n",
    "    red = result2[1]\n",
    "    yellow = result3[1]\n",
    "    green = result4[1]\n",
    "    \n",
    "    return red, yellow, green\n",
    "\n",
    "(r, y, g) = determine_color(cv2.imread('opencv/greenlight.jpg'))\n",
    "\n",
    "if r == min(r, y, g):\n",
    "    print('Red')\n",
    "elif y == min(r, y, g):\n",
    "    print('Yellow')\n",
    "else:\n",
    "    print('Green')"
   ],
   "id": "a7c361f05387ab99",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Green\n"
     ]
    }
   ],
   "execution_count": 114
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-07-15T10:20:17.066748Z",
     "start_time": "2024-07-15T10:20:17.064843Z"
    }
   },
   "cell_type": "code",
   "source": "",
   "id": "42c92ad7f7bd87ce",
   "outputs": [],
   "execution_count": 32
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}