Assignment 1, 2A, 2B completed. Added files to respective folders.

assMath/ass2B/PranjalGautam/1a&4.ipynb

@@ -0,0 +1,35 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1a\n",
+    "When we talk about $p(y|x)$, we need to find some estimate of y given the different values of x. We can achieve this by using the mean of $p(y|x)$ (the expectation of y given x) to find out the most likely value of y given the input x. It acts as a n unbiased and consistent estimator of the true mean."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4\n",
+    "### a.\n",
+    "When we have a lot of data points to work with and the number of predictors is small, a flexible statistical learning  method is expected to perform better than an inflexible one. As the number of data points is large, it can easily find the true pattern and any noise can easily be offset by the number of samples. However, it might overfit the trainset and fail to provide accurate predictions on new data. We expect the model to have low bias (high if the model was overfit to the training set) and rvery low variance.\n",
+    "### b.\n",
+    "When the number of points are small, it generally becomes harder for flexible learning models to provide reliable results, they might not be able to estimate the true values and might overfit to the data very quickly. It becomes very hard to estimate the true values because the model will be much more prone to noise in the data due to the high number of predictors. We expect the model to have high bias and low variance.\n",
+    "### c.\n",
+    "If the relationship between the predictors and response variable is highly non linear and complex, we expect a flexible model to work better than an inflexible one. Flexible models are generally better at recogonising more complex relationships, and can use approximation to get a decent estimate of the response. Considering the biar variance tradeoff, we expect the model to be able to find the patterns and thus have a relatively low bias (again if we assume the model is not fitting the training set) however the variance in the predictions might be higher due to the complex nature of the model."
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

assMath/ass2B/PranjalGautam/1c.ipynb

@@ -0,0 +1,141 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "list2=np.random.exponential(20,100)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "list1=np.random.normal(0,0.5,100)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "list1+=list2\n",
+    "list1.sort()\n",
+    "# list1 is our final database where x follows some exponential distribution E(theta)\n",
+    "# we will estimate theta using gradient descent"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# y=(1/b)e^(-x/b), take 1/b as w,\n",
+    "#y=prod(we^(-wxi))\n",
+    "#log(y)=sum(log(w)-wxi)\n",
+    "#dlog(y)/dw=n/w-sum(xi)\n",
+    "#we use this as the loss function for gradient descent\n",
+    "#Now we define the function\n",
+    "def gradient_descent(x,w,L):\n",
+    "    grad_w=-len(x)/w\n",
+    "    for i in x:\n",
+    "        grad_w+=i\n",
+    "    if(grad_w<0):\n",
+    "        L*=10\n",
+    "    return max(w-grad_w*L,0.0001)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "w=1 #initial value\n",
+    "L=0.00001 #learning rate\n",
+    "epochs=100000"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for i in range(epochs):\n",
+    "    w=gradient_descent(list1,w,L)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.05008954922152509"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "w"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Here is a list of theta and w(predicted value of 1/theta)\n",
+    "$\\theta\\space\\space\\space\\space\\space\\space\\space\\space\\space\\space w$\\\n",
+    "$10\\space\\space\\space\\space\\space\\space\\space 0.09462341857288536$\\\n",
+    "$0.1 \\space\\space\\space\\space\\space 9.130099007007797$\\\n",
+    "$0.2\\space\\space\\space\\space\\space 4.922704924280886$\\\n",
+    "$5\\space\\space\\space\\space\\space\\space\\space\\space 0.19044194086134064$\\\n",
+    "for the final test using a high theta and some noise\\\n",
+    "$20\\space\\space\\space\\space\\space 0.045793632194344173$"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.1"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}