SEMI_SUPERVISED_LEARNING_FASHION_MNIST_DATASET.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **SEMI-SUPERVISED LEARNING ON FASHION MNIST DATASET**"
      ],
      "metadata": {
        "id": "tXeFVE5-m10S"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install ipython-autotime --quiet\n",
        "%load_ext autotime"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WS5W6JqX5uA3",
        "outputId": "91439e4a-7a49-48dd-f5f8-bebd33c408d8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The autotime extension is already loaded. To reload it, use:\n",
            "  %reload_ext autotime\n",
            "time: 12 s (started: 2024-04-21 14:18:26 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#  Import Necessary Models"
      ],
      "metadata": {
        "id": "S4A7pq04Xsg3"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4g41_RIHfFdR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "459db3a2-cf40-415d-b8e0-d27a3fbf937c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 1.08 ms (started: 2024-04-21 14:18:57 +00:00)\n"
          ]
        }
      ],
      "source": [
        "# import necessary models\n",
        "from tqdm import tqdm\n",
        "import numpy as np\n",
        "from tensorflow import keras\n",
        "from sklearn.cluster import KMeans, MiniBatchKMeans\n",
        "from sklearn.linear_model import LogisticRegression"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Load Fashion MNIST data"
      ],
      "metadata": {
        "id": "cDKPCEMAX2ct"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Fashion MNIST dataset can be easily accessed in Python through the Keras library, which provides a simple interface to download and load the dataset."
      ],
      "metadata": {
        "id": "AzD_2Vm2g9i-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Load Fashion MNIST data\n",
        "\n",
        "(X_train, y_train), (X_test, y_test) = keras.datasets.fashion_mnist.load_data()"
      ],
      "metadata": {
        "id": "p3ansHdijQzT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "70b38f64-f668-44e0-8d64-0fcb1b6a13f0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "29515/29515 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26421880/26421880 [==============================] - 1s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "5148/5148 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4422102/4422102 [==============================] - 1s 0us/step\n",
            "time: 3.84 s (started: 2024-04-21 14:19:06 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Data Preprocessing"
      ],
      "metadata": {
        "id": "sMQSgax6X9KI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The following preprocessing steps standardize the pixel values and reshape the image data into a format suitable for feeding into a machine learning model, typically for tasks such as image classification."
      ],
      "metadata": {
        "id": "o0yCzNddhWeb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Preprocess data\n",
        "\n",
        "X_train = X_train.astype('float32') / 255.0\n",
        "X_test = X_test.astype('float32') / 255.0\n",
        "\n",
        "X_train = X_train.reshape(X_train.shape[0], -1)\n",
        "X_test = X_test.reshape(X_test.shape[0], -1)"
      ],
      "metadata": {
        "id": "ynIz89G0jbBj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9f9c42c0-c166-4a3a-9080-817eec0c7d43"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 179 ms (started: 2024-04-21 14:19:13 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# A function to print a particular image"
      ],
      "metadata": {
        "id": "o39UTkhcYCUo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Printing a particular image\n",
        "\n",
        "get_original_array = lambda arr: np.array([int(x * 255) for x in arr], dtype=np.uint8)\n",
        "\n",
        "def print_image(i, data=\"train\"):\n",
        "  if type(i) is int:\n",
        "    if data == 'train':\n",
        "      return get_original_array(X_train[i]).reshape(28,28)\n",
        "    else:\n",
        "      return get_original_array(X_test[i]).reshape(28,28)\n",
        "  else:\n",
        "    return get_original_array(i).reshape(28,28)\n",
        "#example\n",
        "print_image(48)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        },
        "id": "su7e6nQN3jt5",
        "outputId": "66f9979d-3a9d-47fd-80c2-72155f947c60"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  0,   0,   0,   0,   0,   0,   0,   0,  39,  90, 135, 200, 132,\n",
              "         60,  64,  83, 183, 195,  97,  29,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,  35, 125, 185, 200, 191, 159, 176, 233,\n",
              "        237, 255, 240, 212, 164, 173, 201, 191, 128,  10,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,  29, 172, 171, 159, 150, 145, 155, 132, 128,\n",
              "        149, 152, 138, 129, 143, 151, 147, 156, 171, 169,  40,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0, 158, 164, 151, 152, 151, 151, 150, 143, 142,\n",
              "        136, 135, 145, 150, 152, 150, 155, 155, 151, 164, 142,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,  31, 164, 153, 161, 154, 150, 149, 151, 169, 172,\n",
              "        171, 168, 175, 155, 144, 150, 149, 150, 156, 159, 175,  51,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,  95, 173, 162, 157, 153, 149, 149, 174, 115,  36,\n",
              "         12,  23,  55, 146, 170, 150, 148, 145, 154, 166, 179, 106,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0, 151, 170, 174, 173, 171, 151, 175,  93,  21, 108,\n",
              "        144, 139,  69,   9, 124, 174, 143, 150, 162, 172, 180, 156,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0, 179, 162, 180, 180, 148, 171, 126,   0, 129, 142,\n",
              "        179, 167, 213, 133,   5, 147, 157, 138, 177, 185, 177, 179,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,  22, 163, 167, 173, 198, 165, 190,  28,  34, 142, 181,\n",
              "        177, 145, 140, 210,   0, 103, 181, 147, 208, 198, 165, 175,  44,\n",
              "          0,   0],\n",
              "       [  0,   0,  82, 176, 171, 175, 212, 178, 202,  37,  21, 116,  63,\n",
              "         71,  68,  63,  94,   0, 129, 176, 168, 229, 191, 163, 179,  82,\n",
              "          0,   0],\n",
              "       [  0,   0,  60, 163, 174, 212, 186, 124, 210,  47,  20, 190, 147,\n",
              "        152, 165, 212, 169,   0, 171, 176,  98, 208, 215, 170, 174, 104,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,  49,  13, 101, 185, 157,  52, 150, 208,\n",
              "        198, 193, 170,  63,  90, 183, 171, 112,   0,  45,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   1,   0,   0,   0,   0, 139, 155, 159, 138,  49,  21,\n",
              "        118,  97,  55,  85, 169, 163, 158, 136,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   1,   4,   0, 111, 160, 147, 183, 166, 101,\n",
              "        112, 100, 144, 180, 162, 158, 161, 130,   0,   0,   2,   2,   2,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  90, 153, 154, 155, 168, 186,\n",
              "        185, 186, 174, 160, 158, 156, 162, 130,   0,   0,   0,   1,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  85, 156, 158, 159, 156, 156,\n",
              "        158, 158, 155, 157, 162, 152, 165, 132,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  78, 162, 159, 158, 162, 164,\n",
              "        162, 164, 162, 158, 162, 155, 165, 148,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  84, 164, 157, 159, 165, 165,\n",
              "        161, 164, 163, 157, 160, 161, 162, 161,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  95, 167, 155, 158, 161, 163,\n",
              "        161, 168, 166, 158, 162, 163, 162, 158,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 169, 155, 160, 162, 164,\n",
              "        163, 168, 163, 161, 162, 157, 163, 161,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 170, 154, 159, 160, 162,\n",
              "        164, 167, 161, 162, 159, 154, 162, 169,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  94, 167, 155, 160, 162, 164,\n",
              "        161, 168, 164, 158, 159, 164, 158, 174,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  96, 169, 156, 159, 162, 163,\n",
              "        162, 168, 171, 163, 159, 163, 155, 176,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 167, 157, 165, 164, 163,\n",
              "        165, 168, 169, 164, 161, 162, 155, 175,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0, 103, 168, 158, 166, 163, 161,\n",
              "        160, 168, 170, 155, 163, 166, 156, 173,   0,   0,   3,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0, 109, 169, 157, 163, 164, 164,\n",
              "        172, 172, 166, 161, 165, 163, 155, 169,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   3,   0, 126, 177, 163, 174, 178, 184,\n",
              "        185, 175, 169, 172, 176, 173, 158, 173,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,  88, 152, 138, 138, 136, 135,\n",
              "        138, 141, 143, 143, 145, 144, 149, 186,   0,   0,   2,   0,   0,\n",
              "          0,   0]], dtype=uint8)"
            ],
            "text/html": [
              "<style>\n",
              "      .ndarray_repr .ndarray_raw_data {\n",
              "        display: none;\n",
              "      }\n",
              "      .ndarray_repr.show_array .ndarray_raw_data {\n",
              "        display: block;\n",
              "      }\n",
              "      .ndarray_repr.show_array .ndarray_image_preview {\n",
              "        display: none;\n",
              "      }\n",
              "      </style>\n",
              "      <div id=\"id-4deb7f47-d644-41dd-aa61-87d10a6dcb20\" class=\"ndarray_repr\"><pre>ndarray (28, 28) <button style=\"padding: 0 2px;\">show data</button></pre><img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAACMElEQVR4nF2RO2hUURCG/5lz7t3n3bzcJCyr0SQaEYOaFCpE0Faw8xkrwVIstNBCsFYLrbRLoYWdkMoQAiLERhBFYwjBQnGNZnfRmNzsuvfec8Zi7wY20wzMxz+PfwhxjIyPnpqb0T1Xs1Nz3+MaAQCGJic66l+7juvuv6UPqWLt9f3aFtz5UPx6mF6N8mq9nktGqQ57d6kFp9wNEkhDAkk7bB1EXvkGAA1gwN1MNyLhghKTqG66AbnhnvF3TXgyZUMR9L2Z/pXZfblQlUin9dEYHjNZsdX+FzMXhsPZWzfHviWSLo0BYADFf25idcf7Z9cGh4qP7tz7kfM7PNPXhL1rTqO7q2dq0Bux5ZdnwoX+1IDnRwcBDUx4Iee8bGptZTppwo8ohPn8stX7FqCB8kZm78rb8+k/pwcc79Dnpwcwu7+/EvYCGpifx5EvF39ef7J0ohYuPp7M+rnDo59ih0gAPOjUleelK5XFXWeDsHy76a0GBKSiUn5jtPgqUJeGf0cqABiw0AAgBpVGouSdE/Kr1moAlqRpAgCEisn3GYadWorjWS3YcI24bNlGUOiMixxnly0pEpBY13jYWggAkFGWIoa2xCSmTUnQDCZiGIWAVPznlhIkYsFiyRhJtikFoSFiNqQipCNnW1tmEggh4UAMt0FAWXIZCpZYINugJiKxIajOmmzbQgKXxJBKInDZSISmuAnZdCWDjHapplxtwtYJzWywXFiHFaUYLE4xhv8B88DhUZQunKYAAAAASUVORK5CYII=\" class=\"ndarray_image_preview\" /><pre class=\"ndarray_raw_data\">array([[  0,   0,   0,   0,   0,   0,   0,   0,  39,  90, 135, 200, 132,\n",
              "         60,  64,  83, 183, 195,  97,  29,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,  35, 125, 185, 200, 191, 159, 176, 233,\n",
              "        237, 255, 240, 212, 164, 173, 201, 191, 128,  10,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,  29, 172, 171, 159, 150, 145, 155, 132, 128,\n",
              "        149, 152, 138, 129, 143, 151, 147, 156, 171, 169,  40,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0, 158, 164, 151, 152, 151, 151, 150, 143, 142,\n",
              "        136, 135, 145, 150, 152, 150, 155, 155, 151, 164, 142,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,  31, 164, 153, 161, 154, 150, 149, 151, 169, 172,\n",
              "        171, 168, 175, 155, 144, 150, 149, 150, 156, 159, 175,  51,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,  95, 173, 162, 157, 153, 149, 149, 174, 115,  36,\n",
              "         12,  23,  55, 146, 170, 150, 148, 145, 154, 166, 179, 106,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0, 151, 170, 174, 173, 171, 151, 175,  93,  21, 108,\n",
              "        144, 139,  69,   9, 124, 174, 143, 150, 162, 172, 180, 156,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0, 179, 162, 180, 180, 148, 171, 126,   0, 129, 142,\n",
              "        179, 167, 213, 133,   5, 147, 157, 138, 177, 185, 177, 179,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,  22, 163, 167, 173, 198, 165, 190,  28,  34, 142, 181,\n",
              "        177, 145, 140, 210,   0, 103, 181, 147, 208, 198, 165, 175,  44,\n",
              "          0,   0],\n",
              "       [  0,   0,  82, 176, 171, 175, 212, 178, 202,  37,  21, 116,  63,\n",
              "         71,  68,  63,  94,   0, 129, 176, 168, 229, 191, 163, 179,  82,\n",
              "          0,   0],\n",
              "       [  0,   0,  60, 163, 174, 212, 186, 124, 210,  47,  20, 190, 147,\n",
              "        152, 165, 212, 169,   0, 171, 176,  98, 208, 215, 170, 174, 104,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,  49,  13, 101, 185, 157,  52, 150, 208,\n",
              "        198, 193, 170,  63,  90, 183, 171, 112,   0,  45,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   1,   0,   0,   0,   0, 139, 155, 159, 138,  49,  21,\n",
              "        118,  97,  55,  85, 169, 163, 158, 136,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   1,   4,   0, 111, 160, 147, 183, 166, 101,\n",
              "        112, 100, 144, 180, 162, 158, 161, 130,   0,   0,   2,   2,   2,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  90, 153, 154, 155, 168, 186,\n",
              "        185, 186, 174, 160, 158, 156, 162, 130,   0,   0,   0,   1,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  85, 156, 158, 159, 156, 156,\n",
              "        158, 158, 155, 157, 162, 152, 165, 132,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  78, 162, 159, 158, 162, 164,\n",
              "        162, 164, 162, 158, 162, 155, 165, 148,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  84, 164, 157, 159, 165, 165,\n",
              "        161, 164, 163, 157, 160, 161, 162, 161,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  95, 167, 155, 158, 161, 163,\n",
              "        161, 168, 166, 158, 162, 163, 162, 158,   0,   0,   1,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 169, 155, 160, 162, 164,\n",
              "        163, 168, 163, 161, 162, 157, 163, 161,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 170, 154, 159, 160, 162,\n",
              "        164, 167, 161, 162, 159, 154, 162, 169,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0,  94, 167, 155, 160, 162, 164,\n",
              "        161, 168, 164, 158, 159, 164, 158, 174,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  96, 169, 156, 159, 162, 163,\n",
              "        162, 168, 171, 163, 159, 163, 155, 176,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0,  99, 167, 157, 165, 164, 163,\n",
              "        165, 168, 169, 164, 161, 162, 155, 175,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   2,   0, 103, 168, 158, 166, 163, 161,\n",
              "        160, 168, 170, 155, 163, 166, 156, 173,   0,   0,   3,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   1,   0, 109, 169, 157, 163, 164, 164,\n",
              "        172, 172, 166, 161, 165, 163, 155, 169,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   3,   0, 126, 177, 163, 174, 178, 184,\n",
              "        185, 175, 169, 172, 176, 173, 158, 173,   0,   0,   2,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,  88, 152, 138, 138, 136, 135,\n",
              "        138, 141, 143, 143, 145, 144, 149, 186,   0,   0,   2,   0,   0,\n",
              "          0,   0]], dtype=uint8)</pre></div><script>\n",
              "      (() => {\n",
              "      const titles = ['show data', 'hide data'];\n",
              "      let index = 0\n",
              "      document.querySelector('#id-4deb7f47-d644-41dd-aa61-87d10a6dcb20 button').onclick = (e) => {\n",
              "        document.querySelector('#id-4deb7f47-d644-41dd-aa61-87d10a6dcb20').classList.toggle('show_array');\n",
              "        index = (++index) % 2;\n",
              "        document.querySelector('#id-4deb7f47-d644-41dd-aa61-87d10a6dcb20 button').textContent = titles[index];\n",
              "        e.preventDefault();\n",
              "        e.stopPropagation();\n",
              "      }\n",
              "      })();\n",
              "    </script>"
            ]
          },
          "metadata": {},
          "execution_count": 17
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 21 ms (started: 2024-04-21 15:20:15 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Standard Logistic Regression"
      ],
      "metadata": {
        "id": "EARQ0IHVYMFd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "At first, we fit a standard logistic regression model on the Fashion MNIST dataset and observe the accuracy score."
      ],
      "metadata": {
        "id": "7KjZf9qGh-kj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Standard logistic model has a very high accuracy for the dataset.\n",
        "\n",
        "logistic_regressor_clf = LogisticRegression(random_state=69, max_iter=10000).fit(X_train, y_train)\n",
        "logistic_regressor_clf.score(X_test, y_test)"
      ],
      "metadata": {
        "id": "O63DSIzaAGVh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "32a25f67-d0b9-4110-b757-b2497cc9fb5d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8442"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 9min 57s (started: 2024-04-21 14:37:10 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note that the standard logistic regression model gives an accuracy score of 0.8442 for the Fashion MNIST dataset (which is a pretty high score)."
      ],
      "metadata": {
        "id": "nWxLwO0iiOZK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Now, we will perform semi-supervised learning on the Fashion MNIST dataset and compare the accuracy scores.**\n",
        "\n",
        "1.   First, we will take 100 random samples from the dataset use it to train the\n",
        " logistic regression model.\n",
        "2.   Next, we will use k-means clustering to get 100 clusters and take the nearest points as centroids. Then we will use those 100 data-points to train the logistic regression classifier.\n",
        "3. Then we will propagate the labelling to the whole cluster and do the same.\n",
        "4. Repeat the same, but only propagate to the 20% of the dataset now.\n",
        "\n"
      ],
      "metadata": {
        "id": "fWWkaA1aDnCx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Perform Logistic Regression On Randomly Selected 100 Samples"
      ],
      "metadata": {
        "id": "-s6TwixhYltG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Randomly select 100 samples from the training data\n",
        "idx = np.random.choice(X_train.shape[0], 100, replace=False)\n",
        "\n",
        "# Get the selected samples and their corresponding labels\n",
        "X_sample = X_train[idx]\n",
        "y_sample = y_train[idx]\n",
        "\n",
        "logistic_regressor_clf = LogisticRegression(random_state=69, max_iter=10000).fit(X_sample, y_sample)\n",
        "logistic_regressor_clf.score(X_test, y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fF7gEyZaDZhI",
        "outputId": "8f00a9d9-8b11-439a-c305-18ca1eb05b6e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6925"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 662 ms (started: 2024-04-21 14:19:49 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "An accuracy score of 0.6925 is obtained when we use randomly selected 100 samples from the dataset to train the model. Note that the accuracy is lesser than the standard logistic regression model."
      ],
      "metadata": {
        "id": "kzKR4Ka4jY48"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Perform Logistic Regression On 100 Cluster Representative Points"
      ],
      "metadata": {
        "id": "ceAPTiSaamMP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Use KMeans to select 100 representative samples and then train the classifier\n",
        "\n",
        "kmeans = MiniBatchKMeans(n_clusters=100, random_state=42)\n",
        "kmeans.fit(X_train)\n",
        "\n",
        "cluster_centers = kmeans.cluster_centers_\n",
        "\n",
        "nearest_points_to_cluster = [-1]*100\n",
        "distance_from_nearest_points_to_cluster = [10e9]*100\n",
        "\n",
        "# Select those representative points\n",
        "for i, cluster_center in enumerate(cluster_centers):\n",
        "  for j, data_point in enumerate(X_train):\n",
        "    distance = np.linalg.norm(cluster_center-data_point)\n",
        "    if distance < distance_from_nearest_points_to_cluster[i]:\n",
        "      distance_from_nearest_points_to_cluster[i] = distance\n",
        "      nearest_points_to_cluster[i] = j\n",
        "\n",
        "X_sample_ = X_train[nearest_points_to_cluster]\n",
        "y_sample_ = y_train[nearest_points_to_cluster]\n",
        "\n",
        "# Train and test the logistic regression classifier\n",
        "logistic_regressor_clf = LogisticRegression(random_state=69, max_iter=10000).fit(X_sample_, y_sample_)\n",
        "logistic_regressor_clf.score(X_test, y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Ot1S6IqjkC5",
        "outputId": "63480fd8-4211-4525-c9de-db05e96f9af8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 3 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7078"
            ]
          },
          "metadata": {},
          "execution_count": 13
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 49.2 s (started: 2024-04-21 14:19:54 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " We observe that the accuracy  score for the above model has improved but is still less than the standard logistic regression model."
      ],
      "metadata": {
        "id": "J0-x52B-kKFf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Perform Logistic Regression On 100 Cluster Representative Points After Propagating the Labels"
      ],
      "metadata": {
        "id": "nw-Br4bTa2Ip"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Use KMeans to select 100 representative samples, propagate the labels and then train the classifier\n",
        "\n",
        "kmeans = MiniBatchKMeans(n_clusters=100, random_state=42)\n",
        "kmeans.fit(X_train)\n",
        "\n",
        "cluster_centers = kmeans.cluster_centers_\n",
        "datapoints_labels = kmeans.labels_\n",
        "\n",
        "# Nearest points of each cluster\n",
        "nearest_points_to_cluster = [-1]*100\n",
        "distance_from_nearest_points_to_cluster = [10e9]*100\n",
        "\n",
        "# Mapping between labels and nearest points\n",
        "nearest_point_of_a_label = [-1]*100\n",
        "\n",
        "# Select those representative points\n",
        "for i, cluster_center in enumerate(cluster_centers):\n",
        "  for j, data_point in enumerate(X_train):\n",
        "    distance = np.linalg.norm(cluster_center-data_point)\n",
        "    if distance < distance_from_nearest_points_to_cluster[i]:\n",
        "      distance_from_nearest_points_to_cluster[i] = distance\n",
        "      nearest_points_to_cluster[i] = j\n",
        "      nearest_point_of_a_label[datapoints_labels[j]] = j\n",
        "\n",
        "y_propagated = [-1]*(X_train.shape[0])\n",
        "for i, datapoint in enumerate(X_train):\n",
        "  cluster_label = datapoints_labels[i]\n",
        "  y_propagated[i] = y_train[nearest_point_of_a_label[cluster_label]]\n",
        "\n",
        "# Train and test the logistic regression classifier\n",
        "logistic_regressor_clf = LogisticRegression(random_state=69, max_iter=10000).fit(X_train, y_propagated)\n",
        "logistic_regressor_clf.score(X_test, y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OY6EXCVXc4HU",
        "outputId": "f6773225-fdc1-4075-b35b-70eec3159b70"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 3 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6992"
            ]
          },
          "metadata": {},
          "execution_count": 14
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 9min 51s (started: 2024-04-21 14:20:43 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Perform Logistic Regression On 100 Cluster Representative Points After Propagating the Labels Partially"
      ],
      "metadata": {
        "id": "wHukIs8qbCga"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Use KMeans to select 100 representative samples, propagate the labels partially and then train the classifier\n",
        "\n",
        "kmeans = MiniBatchKMeans(n_clusters=100, random_state=42)\n",
        "kmeans.fit(X_train)\n",
        "\n",
        "cluster_centers = kmeans.cluster_centers_\n",
        "datapoints_labels = kmeans.labels_\n",
        "\n",
        "# Nearest points of each cluster\n",
        "nearest_points_to_cluster = [-1]*100\n",
        "distance_from_nearest_points_to_cluster = [10e9]*100\n",
        "\n",
        "# Mapping between labels and nearest points\n",
        "nearest_point_of_a_label = [-1]*100\n",
        "\n",
        "# Select those representative points\n",
        "for i in tqdm(range(len(cluster_centers))):\n",
        "  for j, data_point in enumerate(X_train):\n",
        "    distance = np.linalg.norm(cluster_centers[i]-data_point)\n",
        "    if distance < distance_from_nearest_points_to_cluster[i]:\n",
        "      distance_from_nearest_points_to_cluster[i] = distance\n",
        "      nearest_points_to_cluster[i] = j\n",
        "      nearest_point_of_a_label[datapoints_labels[j]] = j\n",
        "\n",
        "closest_points_from_centroid = np.array([[1]*120]*100)\n",
        "\n",
        "for i in tqdm(range(len(nearest_points_to_cluster))):\n",
        "  distance_point_pair = np.empty((0, 2))\n",
        "\n",
        "  for j, data_point in enumerate(X_train):\n",
        "\n",
        "    dist = np.linalg.norm(data_point-X_train[nearest_points_to_cluster[i]])\n",
        "    to_append = np.array([dist, j])\n",
        "    distance_point_pair = np.concatenate((distance_point_pair, [to_append]))\n",
        "\n",
        "  sorted_indices = np.argsort(distance_point_pair[:, 0])\n",
        "  distance_point_pair = (distance_point_pair[sorted_indices])[:120]\n",
        "\n",
        "  closest_points_idx = distance_point_pair[:, 1]\n",
        "  closest_points_from_centroid[i] = closest_points_idx\n",
        "\n",
        "X_sample = []\n",
        "y_propagated = []\n",
        "\n",
        "for i, id in enumerate(closest_points_from_centroid):\n",
        "  for j in id:\n",
        "    X_sample.append(X_train[j])\n",
        "    y_propagated.append(y_train[nearest_points_to_cluster[i]])\n",
        "\n",
        "# Train and test the logistic regression classifier\n",
        "logistic_regressor_clf = LogisticRegression(random_state=37, max_iter=10000).fit(X_sample, y_propagated)\n",
        "logistic_regressor_clf.score(X_test, y_test)"
      ],
      "metadata": {
        "id": "UQezwWXmxJRX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3b7c18d5-ab53-4568-b7da-27f22bdcdca2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 3 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
            "  warnings.warn(\n",
            "100%|██████████| 100/100 [00:38<00:00,  2.62it/s]\n",
            "100%|██████████| 100/100 [04:37<00:00,  2.77s/it]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7041"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "time: 6min 34s (started: 2024-04-21 14:30:35 +00:00)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Report:\n",
        "\n",
        "- NOTE THAT THE NUMBER OF CLUSTERS TO BE MADE WAS CHOSEN BASED ON THE SIZE OF THE DATASET.\n",
        "- Standard logistic model has a very high accuracy for the dataset. (`0.8442`)\n",
        "- Now, we choose randomly 100 datapoints (out of 60000) and use standard logistic regression to get (`0.6925`). The performance was poor.\n",
        "- Then, instead of choosing randomly, we cluster the dataset into 100 clusters and used the nearest points to the centroids as the representative points to train the standard logistic model. The accuracy was `0.7078`. The accuracy is greatly improved as we are now using the best of its kind to train.\n",
        "- Next, we propagate the labels to every cluster point to get the accuracy as `0.6992`. Performance drops as there might be spurious labelling.\n",
        "- Now, instead of propagating to all the points, we propagate to the 20% data (`7041`). Surprisingly, the performance improves implying labelling small dataset accordingly (semi-supervised learning) performs greatly."
      ],
      "metadata": {
        "id": "eqbsroOPWoZz"
      }
    }
  ]
}