{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8797cb42",
   "metadata": {},
   "source": [
    "## First view of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "391c54d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(116, 10)\n",
      "   Age        BMI  Glucose  Insulin      HOMA   Leptin  Adiponectin  Resistin  \\\n",
      "0   48  23.500000       70    2.707  0.467409   8.8071     9.702400   7.99585   \n",
      "1   83  20.690495       92    3.115  0.706897   8.8438     5.429285   4.06405   \n",
      "2   82  23.124670       91    4.498  1.009651  17.9393    22.432040   9.27715   \n",
      "3   68  21.367521       77    3.226  0.612725   9.8827     7.169560  12.76600   \n",
      "4   86  21.111111       92    3.549  0.805386   6.6994     4.819240  10.57635   \n",
      "\n",
      "     MCP.1  Classification  \n",
      "0  417.114               1  \n",
      "1  468.786               1  \n",
      "2  554.697               1  \n",
      "3  928.220               1  \n",
      "4  773.920               1  \n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 116 entries, 0 to 115\n",
      "Data columns (total 10 columns):\n",
      " #   Column          Non-Null Count  Dtype  \n",
      "---  ------          --------------  -----  \n",
      " 0   Age             116 non-null    int64  \n",
      " 1   BMI             116 non-null    float64\n",
      " 2   Glucose         116 non-null    int64  \n",
      " 3   Insulin         116 non-null    float64\n",
      " 4   HOMA            116 non-null    float64\n",
      " 5   Leptin          116 non-null    float64\n",
      " 6   Adiponectin     116 non-null    float64\n",
      " 7   Resistin        116 non-null    float64\n",
      " 8   MCP.1           116 non-null    float64\n",
      " 9   Classification  116 non-null    int64  \n",
      "dtypes: float64(7), int64(3)\n",
      "memory usage: 9.2 KB\n",
      "The features are : ['Age', 'BMI', 'Glucose', 'Insulin', 'HOMA', 'Leptin', 'Adiponectin', 'Resistin', 'MCP.1']\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 77840 characters]",
      "text/plain": [
       "<Figure size 1200x800 with 12 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Import of the libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# Load data\n",
    "data = pd.read_csv('dataR2.csv')\n",
    "\n",
    "# View data\n",
    "print(data.shape)\n",
    "print(data.head())\n",
    "data.info()\n",
    "\n",
    "# Separate features and targets\n",
    "features = data.columns[:-1]\n",
    "print(\"The features are :\", list(features))\n",
    "target = 'Classification'\n",
    "\n",
    "# View the distribution of the features\n",
    "data.hist(bins=50, figsize=(12, 8))\n",
    "plt.suptitle(\"Barplot of the features\", fontsize=16)\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62b9137",
   "metadata": {},
   "source": [
    "### The dataset\n",
    "\n",
    "The dataset has 116 observations on 9 explicative features and 1 target variable.\n",
    "The features are continuous and have the following labels: `Age`, `BMI`, `Glucose`, `Insulin`, `HOMA`, `Leptin`, `Adiponectin`, `Resistin`, `MCP.1`.\n",
    "The target variable is called `Classification` and takes the values 2 if the patient has breast cancer and 1 if the patient hasn't.\n",
    "\n",
    "There are no missing values.\n",
    "\n",
    "### Distribution of the variables\n",
    "Certain variables have an **asymmetrical distribution** and extreme values, more specifically `Insulin`, `HOMA`, `Leptin`, `Adiponectin`, `Resistin` and `MCP.1`. We will transform and standardise the data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cb261cd",
   "metadata": {},
   "source": [
    "## Splitting dataset between target and features\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3eddff01",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data.drop(columns='Classification')\n",
    "y = data['Classification'].map({2: 1, 1: 0})\n",
    "# Mapping : 1 = healthy patient (negative), 2 = sick patient(positive)\n",
    "# We want : 0 = healthy patient (negative), 1 = sick patient(positive)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ffc678",
   "metadata": {},
   "source": [
    "## Visualisation of ouliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "14be5c48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 136012 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_cols = 3\n",
    "n_rows = int(np.ceil(len(features) / n_cols))\n",
    "\n",
    "# Boxplots\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(X.columns):\n",
    "    sns.boxplot(x=y, y=X[col], ax=axes[i])\n",
    "    axes[i].set_title(f\"Boxplot of {col}\")\n",
    "    axes[i].set_xlabel(\"Class\")\n",
    "    axes[i].set_ylabel(col)\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Boxplots of (before tranformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eec1d6e",
   "metadata": {},
   "source": [
    "we start by separating explicative features and target, and recode the latter the following way:\n",
    "- `1` : sick patient (positive),\n",
    "- `0` : healthy patient (negative).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "556f426b",
   "metadata": {},
   "source": [
    "## Logarithmic transformation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "af21af16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 47960 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 52396 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 48488 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 50256 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 135456 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 242036 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# List of variables we want to transform\n",
    "variables_log = ['Insulin', 'HOMA', 'MCP.1', 'Resistin']\n",
    "\n",
    "# Comparison between regular and transformed data\n",
    "for col in variables_log:\n",
    "    fig, axes = plt.subplots(1, 2, figsize=(12, 4))\n",
    "\n",
    "    # before transformation\n",
    "    axes[0].hist(X[col], bins=30, alpha=0.6, color='salmon', edgecolor='black')\n",
    "    axes[0].set_title(f\"{col} (before transformation)\")\n",
    "    axes[0].set_xlabel(col)\n",
    "    axes[0].set_ylabel(\"Fequency\")\n",
    "\n",
    "    # after log1p transformation\n",
    "    transformed = np.log1p(X[col])\n",
    "    axes[1].hist(transformed, bins=30, alpha=0.6, color='steelblue', edgecolor='black')\n",
    "    axes[1].set_title(f\"{col} after transformation log({col} + 1)\")\n",
    "    axes[1].set_xlabel(f\"log1p({col})\")\n",
    "    axes[1].set_ylabel(\"Frequency\")\n",
    "\n",
    "    plt.suptitle(f\"Comparaison between regular and transformed data : {col}\", fontsize=14)\n",
    "    plt.tight_layout(rect=[0, 0, 1, 0.93])\n",
    "    plt.show()\n",
    "\n",
    "# Transformation of X\n",
    "X_transformed = X.copy()\n",
    "for col in variables_log:\n",
    "    X_transformed[col + '_log'] = np.log1p(X[col])\n",
    "X_transformed = X_transformed.drop(columns=variables_log)\n",
    "\n",
    "# Update list of features\n",
    "features = X_transformed.columns.tolist()\n",
    "\n",
    "# Create transformed data\n",
    "data2 = X_transformed.copy()\n",
    "data2['Classification'] = y.values\n",
    "\n",
    "# Boxplots according to target class\n",
    "\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(features):\n",
    "    sns.boxplot(x='Classification', y=col, data=data2, ax=axes[i])\n",
    "    axes[i].set_title(f\"Boxplot of {col}\")\n",
    "    axes[i].set_xlabel(\"Class\")\n",
    "    axes[i].set_ylabel(col)\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Boxplots according to class (after transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n",
    "\n",
    "# Scatter plots jittered according to target class\n",
    "\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(features):\n",
    "    for classe in [0, 1]:\n",
    "        values = data2[data2['Classification'] == classe][col]\n",
    "        jittered_y = np.random.normal(loc=classe, scale=0.02, size=values.shape)\n",
    "        axes[i].scatter(values, jittered_y, alpha=0.5, label=f\"Class {classe}\")\n",
    "    axes[i].set_title(f\"{col} (scatter jittered)\")\n",
    "    axes[i].legend()\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Scatter plots (jittered) according to class (after transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c582a520",
   "metadata": {},
   "source": [
    "\n",
    "### Objective of the transformation\n",
    "\n",
    "In order to ensure these variables are distributed more symmetrically and to reduce the impact of outliers, we have applied a logarithmic transformation using `log(x+1)`with the function `np.log1p`\n",
    "\n",
    "### Comparison\n",
    "\n",
    "We realise that the barplots are less steep and the distributions become more compact, closer to a normal distribution curve. The dataset is now more homogeneous and further standardisation will be more impactful. These graphics are a visual representation of the impact of the logarithmic transformation.\n",
    "\n",
    "### Updated dataset\n",
    "\n",
    "We have added the transformed features `Insulin_log`, `HOMA_log`, `MCP.1_log` and `Resistin_log` in favor of the old, less-balanced features.\n",
    "\n",
    "### Boxplots\n",
    "The logarithmic transformation severly reduces the number of outliers.\n",
    "\n",
    "### Scatter plots jittered\n",
    "The scatter plots show the datapoints are well and horizontally separated according to class.\n",
    "\n",
    "\n",
    "### Conclusion\n",
    "The logarithmic transformation has reduced the impact of outliers and stabilised the standard deviation by making the distribution more compact and centered around the mean."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "184a592b",
   "metadata": {},
   "source": [
    "## Standardisation (StandardScaler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6dbb4fee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Age', 'BMI', 'Glucose', 'Leptin', 'Adiponectin', 'Insulin_log',\n",
      "       'HOMA_log', 'MCP.1_log', 'Resistin_log'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(X_transformed.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "4b77fbe5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>1.493059e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BMI</th>\n",
       "      <td>-6.121779e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Glucose</th>\n",
       "      <td>1.990745e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Leptin</th>\n",
       "      <td>1.339924e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Adiponectin</th>\n",
       "      <td>2.201304e-17</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Insulin_log</th>\n",
       "      <td>-2.277871e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HOMA_log</th>\n",
       "      <td>-1.378208e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MCP.1_log</th>\n",
       "      <td>-6.661338e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Resistin_log</th>\n",
       "      <td>3.292386e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      mean       std\n",
       "Age           1.493059e-16  1.004338\n",
       "BMI          -6.121779e-16  1.004338\n",
       "Glucose       1.990745e-16  1.004338\n",
       "Leptin        1.339924e-16  1.004338\n",
       "Adiponectin   2.201304e-17  1.004338\n",
       "Insulin_log  -2.277871e-16  1.004338\n",
       "HOMA_log     -1.378208e-16  1.004338\n",
       "MCP.1_log    -6.661338e-16  1.004338\n",
       "Resistin_log  3.292386e-16  1.004338"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Initialisation dof the scaler\n",
    "scaler = StandardScaler()\n",
    "scaler_orig = StandardScaler()\n",
    "\n",
    "# Standardisation of the transformed data\n",
    "X_scaled = scaler.fit_transform(X_transformed)\n",
    "X_scaled = pd.DataFrame(X_scaled, columns=X_transformed.columns, index=X_transformed.index)\n",
    "X_scaled.describe().T[['mean', 'std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cdf1fb2",
   "metadata": {},
   "source": [
    "We will also create standardised data without a logarithmic transformation to look at the impact of the logarithmic transformation on the model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "4ca93d3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>1.493059e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BMI</th>\n",
       "      <td>-6.121779e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Glucose</th>\n",
       "      <td>1.990745e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Insulin</th>\n",
       "      <td>3.158393e-17</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HOMA</th>\n",
       "      <td>1.990745e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Leptin</th>\n",
       "      <td>1.339924e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Adiponectin</th>\n",
       "      <td>2.201304e-17</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Resistin</th>\n",
       "      <td>2.737274e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MCP.1</th>\n",
       "      <td>-1.282499e-16</td>\n",
       "      <td>1.004338</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     mean       std\n",
       "Age          1.493059e-16  1.004338\n",
       "BMI         -6.121779e-16  1.004338\n",
       "Glucose      1.990745e-16  1.004338\n",
       "Insulin      3.158393e-17  1.004338\n",
       "HOMA         1.990745e-16  1.004338\n",
       "Leptin       1.339924e-16  1.004338\n",
       "Adiponectin  2.201304e-17  1.004338\n",
       "Resistin     2.737274e-16  1.004338\n",
       "MCP.1       -1.282499e-16  1.004338"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Standardisation of the original data\n",
    "X_scaled_orig = scaler_orig.fit_transform(X)\n",
    "X_scaled_orig = pd.DataFrame(X_scaled_orig, columns=X.columns, index=X.index)\n",
    "X_scaled_orig.describe().T[['mean', 'std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e525eb2",
   "metadata": {},
   "source": [
    "### Objectif\n",
    "\n",
    "We want to center the variables around 0 and get their variance around 1 in order to approximate a gaussian distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dc8763f",
   "metadata": {},
   "source": [
    "## Why use a Gaussian Naive Bayes?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26514e2a",
   "metadata": {},
   "source": [
    "The data of the features is continuous and non-binary. As such, we use a GaussianNB, the performance will nevertheless suffer as the features aren't all Normal distributed and the dimension is rather small, we cannot suppose normal distribution through size."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd9b3f0",
   "metadata": {},
   "source": [
    "## Train/Test split for the evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "d58417ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train : (81, 9) (81,)\n",
      "Test  : (35, 9) (35,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Stratified split to conserve the distribution of the variables\n",
    "X_transf_train, X_transf_test, y_transf_train, y_transf_test = train_test_split(\n",
    "    X_scaled, y,\n",
    "    test_size=0.3,\n",
    "    random_state=42,\n",
    "    stratify=y)\n",
    "\n",
    "print(\"Train :\", X_transf_train.shape, y_transf_train.shape)\n",
    "print(\"Test  :\", X_transf_test.shape, y_transf_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a53cc076",
   "metadata": {},
   "source": [
    "A stratified split help preserve the original proportion of healthy and sick patients in train and test sample (to ensure the model isn't trained with an overbearing class)\n",
    "The sets have the following size:\n",
    "- **Training set** : 81 observations\n",
    "- **Test set** : 24 observations\n",
    "\n",
    "This is a standard 80/20 split"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8891271",
   "metadata": {},
   "source": [
    "## Final evaluation on the dataset with transformed data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "d26b8326",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy : 0.714\n",
      "F1-score : 0.688\n",
      "Recall : 0.579\n",
      "\n",
      "Classification Report with log-transformation:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.64      0.88      0.74        16\n",
      "           1       0.85      0.58      0.69        19\n",
      "\n",
      "    accuracy                           0.71        35\n",
      "   macro avg       0.74      0.73      0.71        35\n",
      "weighted avg       0.75      0.71      0.71        35\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20340 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.metrics import accuracy_score, f1_score, recall_score, classification_report, confusion_matrix\n",
    "\n",
    "#Initialize and train a Naive Bayes classifier\n",
    "nb_classifier = GaussianNB()\n",
    "nb_classifier.fit(X_transf_train, y_transf_train)\n",
    "\n",
    "# Make predictions on the test set\n",
    "y_pred = nb_classifier.predict(X_transf_test)\n",
    "\n",
    "# Accuracy\n",
    "accuracy = accuracy_score(y_transf_test, y_pred)\n",
    "print(f\"Accuracy : {accuracy:.3f}\")\n",
    "\n",
    "# F1-score\n",
    "f1 = f1_score(y_transf_test, y_pred)\n",
    "print(f\"F1-score : {f1:.3f}\")\n",
    "\n",
    "# Recall\n",
    "rec = recall_score(y_transf_test, y_pred)\n",
    "print(f\"Recall : {rec:.3f}\")\n",
    "\n",
    "# Complete report\n",
    "print(\"\\nClassification Report with log-transformation:\")\n",
    "print(classification_report(y_transf_test, y_pred))\n",
    "\n",
    "## Confusion matrix\n",
    "conf = sns.heatmap(confusion_matrix(y_true=y_transf_test, y_pred=y_pred), annot=True, cmap=\"Blues\", xticklabels=[\"Negatives\", \"Positives\"], yticklabels=[\"Negatives\", \"Positives\"])\n",
    "conf.set_xlabel(\"Predictions\")\n",
    "conf.set_ylabel(\"Test data\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d666c0d",
   "metadata": {},
   "source": [
    "We want to maximise the recall, even if it might compromise the accuracy: it is more important to find every sick patient than every healthy one. In our case, the model doesn't mark enough people as sick: only 8 out of the 19 sick patients are identified as such (58%)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f654b0",
   "metadata": {},
   "source": [
    "## Final evaluation on the dataset with only standardised data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "9696c47f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train : (81, 9) (81,)\n",
      "Test  : (35, 9) (35,)\n"
     ]
    }
   ],
   "source": [
    "# Stratified split to conserve the distribution of the variables\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_scaled_orig, y,\n",
    "    test_size=0.3,\n",
    "    random_state=42,\n",
    "    stratify=y)\n",
    "\n",
    "print(\"Train :\", X_train.shape, y_train.shape)\n",
    "print(\"Test  :\", X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "5eb2fd7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy : 0.686\n",
      "F1-score : 0.621\n",
      "Recall : 0.474\n",
      "\n",
      "Classification Report without log-transformation :\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.60      0.94      0.73        16\n",
      "           1       0.90      0.47      0.62        19\n",
      "\n",
      "    accuracy                           0.69        35\n",
      "   macro avg       0.75      0.71      0.68        35\n",
      "weighted avg       0.76      0.69      0.67        35\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20520 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Initialize and train a Naive Bayes classifier\n",
    "nb_classifier = GaussianNB()\n",
    "nb_classifier.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions on the test set\n",
    "y_pred = nb_classifier.predict(X_test)\n",
    "\n",
    "# Accuracy\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f\"Accuracy : {accuracy:.3f}\")\n",
    "\n",
    "# F1-score\n",
    "f1 = f1_score(y_test, y_pred)\n",
    "print(f\"F1-score : {f1:.3f}\")\n",
    "\n",
    "# Recall\n",
    "rec = recall_score(y_test, y_pred)\n",
    "print(f\"Recall : {rec:.3f}\")\n",
    "\n",
    "# Complete report\n",
    "print(\"\\nClassification Report without log-transformation :\")\n",
    "print(classification_report(y_test, y_pred))\n",
    "\n",
    "## Confusion matrix\n",
    "conf = sns.heatmap(confusion_matrix(y_true=y_test, y_pred=y_pred), annot=True, cmap=\"Blues\", xticklabels=[\"Negatives\", \"Positives\"], yticklabels=[\"Negatives\", \"Positives\"])\n",
    "conf.set_xlabel(\"Predictions\")\n",
    "conf.set_ylabel(\"Test data\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f72c4e0",
   "metadata": {},
   "source": [
    "So the performance is worse without the logarithmic transformation, underlining the utility of said transformation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1595de0",
   "metadata": {},
   "source": [
    "## ROC and AUC"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b70ca49b",
   "metadata": {},
   "source": [
    "We get the follow ROC curve:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "4845ebda",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 56872 characters]",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Courbe ROC et AUC\n",
    "from sklearn.metrics import roc_curve, auc\n",
    "\n",
    "fpr_transf, tpr_transf, thresholds = roc_curve(y_transf_test, y_pred)\n",
    "roc_auc_transf = auc(fpr_transf, tpr_transf)\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_pred)\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(fpr_transf, tpr_transf, label=f'AUC with log= {roc_auc_transf:.2f}', color='blue')\n",
    "plt.scatter(fpr, tpr, label=f'AUC without log = {roc_auc:.2f}', color='green')\n",
    "plt.plot([0, 1], [0, 1], linestyle='--', color='red')\n",
    "plt.title(\"ROC curve with and without log-transformation\")\n",
    "plt.xlabel(\"Rate of FP\")\n",
    "plt.ylabel(\"Rate of TP\")\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ad43508",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
