2 Linear Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性分类器 Linear Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 良/恶性肿瘤预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(683, 11)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入pandas和numpy工具包\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# 注意这里事先要知道数据集的结构，此处创建特征列表\n",
    "column_names=['Sample code number', 'Clump Thickness', 'Uniformity of Cell Size', 'Uniformity of Cell Shape', 'Marginal Adhesion',\n",
    "              'Single Epithelial Cell Size', 'Bare Nuclei', 'Bland Chromatin', 'Normal Nucleoli', 'Mitoses', 'Class']\n",
    "\n",
    "# 使用pandas.read_csv从互联网读取指定数据\n",
    "data = pd.read_csv('http://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data',\n",
    "                   names = column_names)\n",
    "\n",
    "# 在事先知道的数据集结构中有？数据的存在，将它们换为标准缺失值表示并丢弃，只要有一个维度有缺失\n",
    "data = data.replace(to_replace = '?', value = np.nan)\n",
    "data = data.dropna(how = 'any')\n",
    "\n",
    "# 输出data的数据量和维度\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "承上启下：\n",
    "\n",
    "- 经过处理，无缺失值的样本数据一共683条，特征一共有9个维度，特征均被量化为1~10之间的数值（已经处理过）。其中第一个和最后一个不是特征。\n",
    "\n",
    "\n",
    "- 下面分割数据一般是按1：3来分割的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2    344\n",
      "4    168\n",
      "Name: Class, dtype: int64\n",
      "2    100\n",
      "4     71\n",
      "Name: Class, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# 下面准备训练数据，用sklearn.cross_validation中的train_test_split分割数据，其中随机采样25%测试\n",
    "from sklearn.cross_validation import train_test_split as tts\n",
    "X_train, X_test, y_train, y_test = tts(data[column_names[1:10]], data[column_names[10]], test_size = 0.25, random_state = 33)\n",
    "# 这里random_state是干什么的？记得去问！\n",
    "\n",
    "# 查看训练样本的数量和类别分布\n",
    "print(y_train.value_counts())\n",
    "\n",
    "# 然后是测试样本的数量和类别分布\n",
    "print(y_test.value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面输出的意思是：\n",
    "\n",
    "- 训练样本一共512条，包括344条良性肿瘤数据，168条恶性肿瘤数据。\n",
    "\n",
    "\n",
    "- 测试样本一共171条，包括内容类似。\n",
    "\n",
    "下面就要具体地用Logistic Regression和随机梯度参数估计两种方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从sklearn.preprocessing导入StandardScaler。\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# From Sklearn.linear_model import LogisticRegression & SGDClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "\n",
    "### Note that here's a function to handle the data. ###\n",
    "### To STANDARDIZE the data, we try to make 每个维度特征数据方差为1，means is 0. ###\n",
    "### This can let prediction away from some too big features! ###\n",
    "# Initial ss\n",
    "ss = StandardScaler()\n",
    "# Standardize it\n",
    "X_train = ss.fit_transform(X_train)\n",
    "X_test = ss.fit_transform(X_test)\n",
    "\n",
    "# Now initial LogisticRegression and SGDClassifier\n",
    "lr = LogisticRegression()\n",
    "sgdc = SGDClassifier()\n",
    "\n",
    "# Then we train the models and predit\n",
    "lr.fit(X_train, y_train)\n",
    "lr_y_predict = lr.predict(X_test)\n",
    "sgdc.fit(X_train, y_train)\n",
    "sgdc_y_predict = sgdc.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Since we have finished the training procedure, we can analyze the result below.\n",
    "\n",
    "\n",
    "- Apart from accuracy, we have another three predictors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of Linear Regression Classifier: 0.9707602339181286\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "     Benign       0.96      0.99      0.98       100\n",
      "  Malignant       0.99      0.94      0.96        71\n",
      "\n",
      "avg / total       0.97      0.97      0.97       171\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# import the evaluation mode classification_report form sklearn.metrics\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "# Use default function score\n",
    "print 'Accuracy of Linear Regression Classifier:', lr.score(X_test, y_test)\n",
    "\n",
    "# Another three predictors\n",
    "print classification_report(y_test, lr_y_predict, target_names = ['Benign', 'Malignant'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of SGD Classifier: 0.9649122807017544\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "     Benign       0.96      0.98      0.97       100\n",
      "  Malignant       0.97      0.94      0.96        71\n",
      "\n",
      "avg / total       0.97      0.96      0.96       171\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Use default function score\n",
    "print 'Accuracy of SGD Classifier:', sgdc.score(X_test, y_test)\n",
    "\n",
    "# Another three predictors\n",
    "print classification_report(y_test, sgdc_y_predict, target_names = ['Benign', 'Malignant'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the results above, we can get:\n",
    "\n",
    "- Logistic Regression have higher accuracy on test data, this is because it use analitical ways to calculate the paraments.\n",
    "\n",
    "\n",
    "- Usually, Logistic is much slower but more accurate."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}