{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8797cb42",
   "metadata": {},
   "source": [
    "# Step 1 — Initial loading and exploration\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "391c54d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(116, 10)\n",
      "   Age        BMI  Glucose  Insulin      HOMA   Leptin  Adiponectin  Resistin  \\\n",
      "0   48  23.500000       70    2.707  0.467409   8.8071     9.702400   7.99585   \n",
      "1   83  20.690495       92    3.115  0.706897   8.8438     5.429285   4.06405   \n",
      "2   82  23.124670       91    4.498  1.009651  17.9393    22.432040   9.27715   \n",
      "3   68  21.367521       77    3.226  0.612725   9.8827     7.169560  12.76600   \n",
      "4   86  21.111111       92    3.549  0.805386   6.6994     4.819240  10.57635   \n",
      "\n",
      "     MCP.1  Classification  \n",
      "0  417.114               1  \n",
      "1  468.786               1  \n",
      "2  554.697               1  \n",
      "3  928.220               1  \n",
      "4  773.920               1  \n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 116 entries, 0 to 115\n",
      "Data columns (total 10 columns):\n",
      " #   Column          Non-Null Count  Dtype  \n",
      "---  ------          --------------  -----  \n",
      " 0   Age             116 non-null    int64  \n",
      " 1   BMI             116 non-null    float64\n",
      " 2   Glucose         116 non-null    int64  \n",
      " 3   Insulin         116 non-null    float64\n",
      " 4   HOMA            116 non-null    float64\n",
      " 5   Leptin          116 non-null    float64\n",
      " 6   Adiponectin     116 non-null    float64\n",
      " 7   Resistin        116 non-null    float64\n",
      " 8   MCP.1           116 non-null    float64\n",
      " 9   Classification  116 non-null    int64  \n",
      "dtypes: float64(7), int64(3)\n",
      "memory usage: 9.2 KB\n",
      "The explanatory variables are: ['Age', 'BMI', 'Glucose', 'Insulin', 'HOMA', 'Leptin', 'Adiponectin', 'Resistin', 'MCP.1']\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 78036 characters]",
      "text/plain": [
       "<Figure size 1200x800 with 12 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# Load data\n",
    "data = pd.read_csv('breast+cancer+coimbra/dataR2.csv')\n",
    "\n",
    "# Data overview\n",
    "print(data.shape)\n",
    "print(data.head())\n",
    "data.info()\n",
    "\n",
    "# List of explanatory variables\n",
    "features = data.columns[:-1]\n",
    "print(\"The explanatory variables are:\", list(features))\n",
    "target = 'Classification'\n",
    "\n",
    "# Histograms of variables\n",
    "data.hist(bins=50, figsize=(12, 8))\n",
    "plt.suptitle(\"Histograms of variables\", fontsize=16)\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62b9137",
   "metadata": {},
   "source": [
    "We use data from the *Breast Cancer Coimbra* dataset, which contains biometric measurements of 116 patients. The goal is to predict whether a patient has breast cancer or not, based on nine biological variables.\n",
    "\n",
    "### Dataset overview\n",
    "\n",
    "The dataset contains 116 observations and 10 variables, including:\n",
    "- **9 continuous explanatory variables**: `Age`, `BMI`, `Glucose`, `Insulin`, `HOMA`, `Leptin`, `Adiponectin`, `Resistin`, `MCP.1`.\n",
    "- **1 target variable**: `Classification`, which takes the value `1` if the patient has cancer (positive) and `2` otherwise (negative).\n",
    "\n",
    "There are no missing values in the data, allowing us to proceed directly with exploratory analysis.\n",
    "\n",
    "### Variable distributions\n",
    "\n",
    "The distribution of the variables was visualized using histograms:\n",
    "\n",
    "- Some variables exhibit **asymmetric distributions** with extreme values, notably `Insulin`, `HOMA`, `Leptin`, `Adiponectin`, `Resistin`, and `MCP.1`. This may motivate **data transformation** or **normalization** prior to modeling.\n",
    "- The `Classification` variable is **imbalanced**, with a majority of healthy patients (`2`). This is important information as it will affect the **choice of evaluation metrics**: simple accuracy will not be sufficient to assess model quality.\n",
    "\n",
    "We will now prepare the data for model training by separating features and target and performing a **stratified split** of the dataset.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cb261cd",
   "metadata": {},
   "source": [
    "# Step 2 — Split the dataset into training and testing sets\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "3eddff01",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train: (92, 9) (92,)\n",
      "Test : (24, 9) (24,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Separate features and target variable\n",
    "X = data.drop(columns='Classification')\n",
    "y = data['Classification'].map({2: 0, 1: 1})\n",
    "# Mapping: 1 = patient with cancer (positive class), 2 = healthy patient (negative class)\n",
    "# We recode to: 1 → positive, 0 → negative\n",
    "\n",
    "# Stratified split to preserve class distribution\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y,\n",
    "    test_size=0.2,\n",
    "    random_state=42,\n",
    "    stratify=y)\n",
    "\n",
    "# Check shapes of the splits\n",
    "print(\"Train:\", X_train.shape, y_train.shape)\n",
    "print(\"Test :\", X_test.shape, y_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eec1d6e",
   "metadata": {},
   "source": [
    "We start by separating the explanatory variables (features) from the target variable (`Classification`), recoding the latter to facilitate result interpretation:\n",
    "- `1`: patient with cancer (positive class),\n",
    "- `0`: healthy patient (negative class).\n",
    "\n",
    "A **stratified split** was performed using the `train_test_split` function from `scikit-learn` to preserve the original class proportions in both training and testing sets. This is particularly important given the observed **class imbalance**.\n",
    "\n",
    "The resulting dataset sizes are:\n",
    "- **Training set**: 92 observations\n",
    "- **Test set**: 24 observations\n",
    "\n",
    "This corresponds to a classical 80% / 20% split.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02d2e734",
   "metadata": {},
   "source": [
    "# Step 3 — Visualization of distributions by class in the training dataset\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "ac59107b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 114516 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 134456 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 218980 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Temporary merge for visualization\n",
    "train_data = X_train.copy()\n",
    "train_data['Classification'] = y_train\n",
    "\n",
    "variables = X_train.columns.tolist()\n",
    "n_cols = 3\n",
    "n_rows = int(np.ceil(len(variables) / n_cols))\n",
    "\n",
    "# Histograms\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(variables):\n",
    "    for cls in y_train.unique():\n",
    "        axes[i].hist(X_train[y_train == cls][col], bins=30, alpha=0.5, label=f\"Class {cls}\")\n",
    "    axes[i].set_title(f\"{col} by class\")\n",
    "    axes[i].legend()\n",
    "    \n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Histograms by class (before transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n",
    "\n",
    "# Boxplots\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(variables):\n",
    "    sns.boxplot(x=y_train, y=X_train[col], ax=axes[i])\n",
    "    axes[i].set_title(f\"Boxplot of {col}\")\n",
    "    axes[i].set_xlabel(\"Class\")\n",
    "    axes[i].set_ylabel(col)\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Boxplots by class (before transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n",
    "\n",
    "# Scatter plots (jittered)\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(variables):\n",
    "    for cls in y_train.unique():\n",
    "        axes[i].scatter(\n",
    "            X_train[y_train == cls][col],\n",
    "            np.random.normal(loc=cls, scale=0.02, size=X_train[y_train == cls][col].shape),\n",
    "            alpha=0.5,\n",
    "            label=f\"Class {cls}\"\n",
    "        )\n",
    "    axes[i].set_title(f\"{col} (jittered scatter plot)\")\n",
    "    axes[i].legend()\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Jittered scatter plots by class (before transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "066377f6",
   "metadata": {},
   "source": [
    "To better understand the differences between patients with cancer (Class 1) and healthy patients (Class 0), we represented the distributions of explanatory variables segmented by class. Three complementary types of visualizations were produced: histograms, boxplots, and jittered scatter plots.\n",
    "\n",
    "### Histograms by class\n",
    "\n",
    "Histograms allow us to observe the distribution of values within each class. Several variables show notable shifts between the two groups, notably:\n",
    "- **Age**: patients with cancer tend to be slightly older.\n",
    "- **Resistin**, **MCP.1**, and **Adiponectin** appear to exhibit density shifts between classes, suggesting potential discriminative power.\n",
    "- Conversely, variables such as **BMI** or **Leptin** show strong overlap between classes.\n",
    "\n",
    "These qualitative observations suggest that some variables may be informative, but **none alone allow a clear separation**.\n",
    "\n",
    "### Boxplots by class\n",
    "\n",
    "Boxplots provide a more precise visual assessment of distribution differences:\n",
    "- Most variables display **high dispersion** with numerous **outliers** visible in both classes, particularly for `Insulin`, `Leptin`, and `MCP.1`.\n",
    "- **Medians are close**, but some trends differ (e.g., `Resistin` is higher for class 1).\n",
    "\n",
    "These elements indicate **high intra-class variance**, which may affect the performance of models sensitive to unstandardized scales.\n",
    "\n",
    "### Jittered scatter plots\n",
    "\n",
    "These visualizations explicitly show vertical separation between classes (0 or 1):\n",
    "- No single variable fully separates the classes.\n",
    "- Some clusters appear (e.g., low values of `Insulin`, `HOMA`, `Adiponectin` associated with class 0), but with **significant overlap**.\n",
    "\n",
    "---\n",
    "\n",
    "### Summary of exploratory analysis\n",
    "\n",
    "- Variables seem to contain **partial signals**, but will need to be **combined in a multivariate model** to best exploit interactions.\n",
    "- The presence of **outliers**, **asymmetric distributions**, and **variables on very different scales** justifies:\n",
    "  - **prior standardization** of variables,\n",
    "  - and possibly a **logarithmic transformation** for some (e.g., `Insulin`, `Leptin`, `HOMA`).\n",
    "- Finally, the strong **inter-class overlap** implies that linear separation may be limited, motivating the use of nonlinear models.\n",
    "\n",
    "This exploratory step thus rigorously motivates the choices of **preprocessing** and **supervised modeling** to follow.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09e7c056",
   "metadata": {},
   "source": [
    "# Step 4 — Correlation matrix and scatter matrix (training set only)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "9ff82466",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 110608 characters]",
      "text/plain": [
       "<Figure size 1000x800 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 462576 characters]",
      "text/plain": [
       "<Figure size 1200x800 with 81 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from pandas.plotting import scatter_matrix\n",
    "\n",
    "# Correlation matrix (on training data)\n",
    "corr_matrix = X_train.corr()\n",
    "\n",
    "plt.figure(figsize=(10, 8))\n",
    "sns.heatmap(corr_matrix, annot=True, fmt=\".2f\", cmap='coolwarm', square=True)\n",
    "plt.title(\"Correlation matrix (X_train)\")\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Combined DataFrame for colored visualization\n",
    "X_train_visu = X_train.copy()\n",
    "X_train_visu['Classification'] = y_train.values\n",
    "\n",
    "# Colored scatter matrix\n",
    "scatter_matrix(X_train_visu.drop(columns='Classification'),\n",
    "               figsize=(12, 8),\n",
    "               diagonal='kde',\n",
    "               c=X_train_visu['Classification'],\n",
    "               cmap='coolwarm',\n",
    "               alpha=0.7)\n",
    "plt.suptitle(\"Colored scatter matrix (X_train)\", y=1.02)\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a358c6aa",
   "metadata": {},
   "source": [
    "To examine potential redundancy between variables and anticipate multicollinearity risks, we studied the linear relationships among explanatory variables using two complementary tools:\n",
    "\n",
    "### Correlation matrix\n",
    "\n",
    "The correlation matrix identifies linear relationships between continuous variables. Notably, we observe:\n",
    "\n",
    "- A **very strong correlation** between `Insulin` and `HOMA` (ρ ≈ 0.93), as expected since HOMA is a function of insulin and glucose (`HOMA = (Glucose × Insulin) / 405`).\n",
    "- Moderate correlations between:\n",
    "  - `Glucose` and `HOMA` (ρ ≈ 0.70),\n",
    "  - `BMI` and `Leptin` (ρ ≈ 0.60).\n",
    "- Other pairs exhibit relatively low correlations (ρ < 0.4), suggesting that **most variables provide complementary information**.\n",
    "\n",
    "This analysis suggests considering **dimensionality reduction** or **feature selection**, especially if certain methods are sensitive to collinearity (e.g., logistic regression).\n",
    "\n",
    "### Colored scatter matrix\n",
    "\n",
    "The scatter matrix, colored by class (`Classification`), provides a global view of pairwise interactions between variables while highlighting groupings by class.\n",
    "\n",
    "- There is **significant overlap** between the two classes in most 2D projections.\n",
    "- Some variable combinations, such as (`Resistin`, `MCP.1`) or (`Adiponectin`, `HOMA`), show partial separation between classes but not a clear boundary.\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusion of this step\n",
    "\n",
    "- There is no **obvious linear separation** between the two classes in the variable pairs.\n",
    "- The dataset structure suggests that multivariate methods could benefit from **complex**, possibly **nonlinear** relationships.\n",
    "- **Standardization** will be necessary before applying scale-sensitive methods (SVM, k-NN, PCA, etc.).\n",
    "\n",
    "We now proceed to the preprocessing step (data standardization) before comparing different supervised learning methods.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "556f426b",
   "metadata": {},
   "source": [
    "# Step 5 — Logarithmic transformation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "af21af16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 43064 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 48188 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 46112 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 49416 characters]",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 113228 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 131944 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 222464 characters]",
      "text/plain": [
       "<Figure size 1500x1200 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# List of variables to transform (asymmetric distributions)\n",
    "variables_log = ['Insulin', 'HOMA', 'MCP.1', 'Resistin']\n",
    "\n",
    "# Comparison before / after transformation (1 figure per variable)\n",
    "for col in variables_log:\n",
    "    fig, axes = plt.subplots(1, 2, figsize=(12, 4))\n",
    "\n",
    "    # Before transformation\n",
    "    axes[0].hist(X_train[col], bins=30, alpha=0.6, color='salmon', edgecolor='black')\n",
    "    axes[0].set_title(f\"{col} (before transformation)\")\n",
    "    axes[0].set_xlabel(col)\n",
    "    axes[0].set_ylabel(\"Frequency\")\n",
    "\n",
    "    # After log1p transformation\n",
    "    transformed = np.log1p(X_train[col])\n",
    "    axes[1].hist(transformed, bins=30, alpha=0.6, color='steelblue', edgecolor='black')\n",
    "    axes[1].set_title(f\"{col} after log({col} + 1) transformation\")\n",
    "    axes[1].set_xlabel(f\"log1p({col})\")\n",
    "    axes[1].set_ylabel(\"Frequency\")\n",
    "\n",
    "    plt.suptitle(f\"Comparison before / after transformation: {col}\", fontsize=14)\n",
    "    plt.tight_layout(rect=[0, 0, 1, 0.93])\n",
    "    plt.show()\n",
    "\n",
    "# Apply the transformation to X_train\n",
    "X_train_transformed = X_train.copy()\n",
    "for col in variables_log:\n",
    "    X_train_transformed[col + '_log'] = np.log1p(X_train[col])\n",
    "X_train_transformed = X_train_transformed.drop(columns=variables_log)\n",
    "\n",
    "# Update the list of transformed features\n",
    "features = X_train_transformed.columns.tolist()\n",
    "\n",
    "# Merge with y_train to facilitate visualization\n",
    "train_data2 = X_train_transformed.copy()\n",
    "train_data2['Classification'] = y_train.values\n",
    "\n",
    "# Histograms by class\n",
    "n_cols = 3\n",
    "n_rows = int(np.ceil(len(features) / n_cols))\n",
    "\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(features):\n",
    "    for cls in [0, 1]:\n",
    "        axes[i].hist(train_data2[train_data2['Classification'] == cls][col],\n",
    "                     bins=30, alpha=0.5, label=f\"Class {cls}\")\n",
    "    axes[i].set_title(f\"{col} by class\")\n",
    "    axes[i].legend()\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Histograms by class (after transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n",
    "\n",
    "# Boxplots by class\n",
    "\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(features):\n",
    "    sns.boxplot(x='Classification', y=col, data=train_data2, ax=axes[i])\n",
    "    axes[i].set_title(f\"Boxplot of {col}\")\n",
    "    axes[i].set_xlabel(\"Class\")\n",
    "    axes[i].set_ylabel(col)\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Boxplots by class (after transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n",
    "\n",
    "# Jittered scatter plots by class\n",
    "\n",
    "fig, axes = plt.subplots(n_rows, n_cols, figsize=(5 * n_cols, 4 * n_rows))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, col in enumerate(features):\n",
    "    for cls in [0, 1]:\n",
    "        values = train_data2[train_data2['Classification'] == cls][col]\n",
    "        jittered_y = np.random.normal(loc=cls, scale=0.02, size=values.shape)\n",
    "        axes[i].scatter(values, jittered_y, alpha=0.5, label=f\"Class {cls}\")\n",
    "    axes[i].set_title(f\"{col} (jittered scatter plot)\")\n",
    "    axes[i].legend()\n",
    "\n",
    "for j in range(i + 1, len(axes)):\n",
    "    fig.delaxes(axes[j])\n",
    "\n",
    "fig.suptitle(\"Jittered scatter plots by class (after transformation)\", fontsize=16)\n",
    "plt.tight_layout(rect=[0, 0, 1, 0.97])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c582a520",
   "metadata": {},
   "source": [
    "### Objective\n",
    "\n",
    "The initial exploratory analysis revealed a **marked skewness** (distribution heavily right-skewed) for several variables: `Insulin`, `HOMA`, `MCP.1`, and `Resistin`.  \n",
    "These distributions contained many extreme values that could hinder the learning of some models (logistic regression, k-NN, SVM, etc.) sensitive to scale differences or non-normality.\n",
    "\n",
    "To make these distributions more symmetric and reduce the impact of extreme values, we applied a **logarithmic transformation** of the form `log(x + 1)` using the `np.log1p` function.\n",
    "\n",
    "---\n",
    "\n",
    "### Comparison before / after transformation\n",
    "\n",
    "For each transformed variable, we compared the distribution **before and after transformation** using histograms:\n",
    "\n",
    "- The long right tails are **strongly reduced**.\n",
    "- The distributions become **more compact, less skewed**, and often closer to a Gaussian shape.\n",
    "- This improves data homogeneity and facilitates subsequent **standardization** steps.\n",
    "\n",
    "These figures provide a **clear visual justification** for the interest in logarithmic transformation.\n",
    "\n",
    "---\n",
    "\n",
    "### Dataset update\n",
    "\n",
    "The new transformed variables (`Insulin_log`, `HOMA_log`, `MCP.1_log`, `Resistin_log`) were added to the dataset, and the original variables were removed to avoid redundancy.\n",
    "\n",
    "---\n",
    "\n",
    "### Visualization of transformed variables by class\n",
    "\n",
    "To observe the effect of this transformation on the distribution of values within the two classes (positive / negative), we reproduced several visualizations on the transformed variables:\n",
    "\n",
    "#### • Histograms by class  \n",
    "These histograms allow visualization of relative densities within each class after transformation.  \n",
    "We observe that some variables, notably `Insulin_log` and `Resistin_log`, still present density differences between classes, but with a **much more regular distribution**.\n",
    "\n",
    "#### • Boxplots by class  \n",
    "The log transformation clearly reduces the impact of **visible outliers** in the boxplots.  \n",
    "Medians are more representative and interquartile ranges less spread.\n",
    "\n",
    "#### • Jittered scatter plots  \n",
    "These visualizations show that points remain well separated vertically between classes 0 and 1, while benefiting from a **more controlled horizontal distribution**, which favors distance-based models.\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "Applying the logarithmic transformation allowed to:\n",
    "- Reduce the effect of extreme values,\n",
    "- Stabilize variance,\n",
    "- Improve the geometric structure of the dataset in view of standardization and learning.\n",
    "\n",
    "These transformations constitute an **essential step in the preprocessing pipeline** before model construction.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "184a592b",
   "metadata": {},
   "source": [
    "# Step 6 — Standardization (StandardScaler)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "4b77fbe5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>1.037817e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BMI</th>\n",
       "      <td>-8.447349e-17</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Glucose</th>\n",
       "      <td>1.995309e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Leptin</th>\n",
       "      <td>6.033821e-17</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Adiponectin</th>\n",
       "      <td>1.436049e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Insulin_log</th>\n",
       "      <td>2.292852e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HOMA_log</th>\n",
       "      <td>-3.891814e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MCP.1_log</th>\n",
       "      <td>-7.406515e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Resistin_log</th>\n",
       "      <td>-1.680419e-16</td>\n",
       "      <td>1.005479</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      mean       std\n",
       "Age           1.037817e-16  1.005479\n",
       "BMI          -8.447349e-17  1.005479\n",
       "Glucose       1.995309e-16  1.005479\n",
       "Leptin        6.033821e-17  1.005479\n",
       "Adiponectin   1.436049e-16  1.005479\n",
       "Insulin_log   2.292852e-16  1.005479\n",
       "HOMA_log     -3.891814e-16  1.005479\n",
       "MCP.1_log    -7.406515e-16  1.005479\n",
       "Resistin_log -1.680419e-16  1.005479"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Initialize the scaler\n",
    "scaler = StandardScaler()\n",
    "\n",
    "# Fit the scaler on the transformed training set and transform\n",
    "X_train_scaled = pd.DataFrame(\n",
    "    scaler.fit_transform(X_train_transformed),\n",
    "    columns=X_train_transformed.columns,\n",
    "    index=X_train_transformed.index\n",
    ")\n",
    "\n",
    "# Display summary statistics of scaled data\n",
    "X_train_scaled.describe().T[['mean', 'std']]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e525eb2",
   "metadata": {},
   "source": [
    "### Objective\n",
    "\n",
    "After applying logarithmic transformation to asymmetric variables, we apply **standardization** to all explanatory variables in the training set.  \n",
    "This step aims to **bring all variables to the same scale** by centering them around zero and scaling them to unit variance.\n",
    "\n",
    "Standardization is particularly important for models sensitive to distances or scale differences, such as:\n",
    "- logistic regression (for more stable convergence),\n",
    "- support vector machines (SVM),\n",
    "- k-nearest neighbors (k-NN),\n",
    "- regularization methods (Ridge, Lasso),\n",
    "- and dimensionality reduction techniques (PCA, etc.).\n",
    "\n",
    "---\n",
    "### Method\n",
    "\n",
    "The transformation used is the classical Z-score standardization, defined as:\n",
    "\n",
    "$$\n",
    "X^{(i)}_{\\text{std}} = \\frac{X^{(i)} - \\mu^{(i)}}{\\sigma^{(i)}}\n",
    "$$\n",
    "\n",
    "where \\( \\mu^{(i)} \\) and \\( \\sigma^{(i)} \\) denote respectively the mean and standard deviation of variable \\( X^{(i)} \\) computed on the training set.\n",
    "\n",
    "We use the `StandardScaler` tool from `scikit-learn`, applied only on the training data (`fit_transform`) to avoid any data leakage.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d374b93f",
   "metadata": {},
   "source": [
    "# Step 7 — Preparation of X_test\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "28a4ffe0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy X_test\n",
    "X_test_transformed = X_test.copy()\n",
    "\n",
    "# Apply the same log-transformations\n",
    "for col in variables_log:\n",
    "    X_test_transformed[col + '_log'] = np.log1p(X_test_transformed[col])\n",
    "\n",
    "# Drop original columns\n",
    "X_test_transformed = X_test_transformed.drop(columns=variables_log)\n",
    "\n",
    "# Standardize using the scaler previously fitted on X_train\n",
    "X_test_scaled = pd.DataFrame(\n",
    "    scaler.transform(X_test_transformed),\n",
    "    columns=X_test_transformed.columns,\n",
    "    index=X_test_transformed.index\n",
    ")\n",
    "\n",
    "# Check that the columns of X_test are identical (and in the same order) as those of X_train to avoid prediction errors\n",
    "if list(X_train_scaled.columns) != list(X_test_scaled.columns):\n",
    "    print(\"Mismatch between train and test columns\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4a5689a",
   "metadata": {},
   "source": [
    "The transformations previously applied to the training set (logarithmic transformation + standardization) must be **reproduced exactly the same way** on the test set, without re-estimating any parameters.\n",
    "\n",
    "### Steps performed:\n",
    "1. **Logarithmic transformation** on the same variables (`Insulin`, `HOMA`, `MCP.1`, `Resistin`) using `log(x + 1)`;\n",
    "2. **Removal of raw variables** to avoid redundancy;\n",
    "3. **Standardization using the same parameters** (mean `μ`, standard deviation `σ`) learned from `X_train`, via `scaler.transform()`.\n",
    "\n",
    "We also verified that the **columns of `X_test_scaled` are exactly the same (and in the same order)** as those of `X_train_scaled`, ensuring compatibility with the learning models.\n",
    "\n",
    "---\n",
    "\n",
    "This step completes the preprocessing pipeline and allows us to proceed to **supervised modeling**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd9b3f0",
   "metadata": {},
   "source": [
    "# Step 8 — Modeling with LogisticRegression\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "d58417ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation of simple model (logistic regression)\n",
      "Accuracy: 0.792\n",
      "F1-score: 0.783\n",
      "\n",
      "Classification report:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.77      0.80        13\n",
      "           1       0.75      0.82      0.78        11\n",
      "\n",
      "    accuracy                           0.79        24\n",
      "   macro avg       0.79      0.79      0.79        24\n",
      "weighted avg       0.80      0.79      0.79        24\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score, f1_score, classification_report\n",
    "\n",
    "# Initialize and train the model without explicit regularization\n",
    "logreg_simple = LogisticRegression(random_state=42)\n",
    "logreg_simple.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Predictions on the test set\n",
    "y_pred_simple = logreg_simple.predict(X_test_scaled)\n",
    "\n",
    "# Basic performance evaluation\n",
    "print(\"Evaluation of simple model (logistic regression)\")\n",
    "print(\"Accuracy:\", round(accuracy_score(y_test, y_pred_simple), 3))\n",
    "print(\"F1-score:\", round(f1_score(y_test, y_pred_simple), 3))\n",
    "print(\"\\nClassification report:\\n\", classification_report(y_test, y_pred_simple))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76d3da3a",
   "metadata": {},
   "source": [
    "We begin by training a simple logistic regression model, without explicit regularization, on the preprocessed data (after log transformation and standardization).\n",
    "\n",
    "### Objective\n",
    "\n",
    "Predict the probability that a patient has cancer based on the nine explanatory variables.  \n",
    "Logistic regression is an interpretable linear model, particularly suited for a first supervised approach.\n",
    "\n",
    "### Results on the test set\n",
    "\n",
    "The model was evaluated on the test set after training on the standardized data. The results are as follows:\n",
    "\n",
    "- **Accuracy**: 0.792\n",
    "- **F1-score**: 0.783\n",
    "\n",
    "### Classification report details:\n",
    "\n",
    "| Class   | Precision | Recall | F1-score | Support |\n",
    "|---------|-----------|--------|----------|---------|\n",
    "| 0 (healthy) | 0.83      | 0.77   | 0.80     | 13      |\n",
    "| 1 (cancer)  | 0.75      | 0.82   | 0.78     | 11      |\n",
    "\n",
    "- **Macro avg F1**: 0.79  \n",
    "- **Weighted avg F1**: 0.79\n",
    "\n",
    "These results show a good balance between precision and recall, which is important in a medical context.  \n",
    "However, this simple model does not account for potential interactions between variables nor hyperparameter optimization, which will be addressed in the following steps.\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b7c1cdf",
   "metadata": {},
   "source": [
    "# Step 9 — Cross-validation on training data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "d8804f23",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10-fold cross-validation\n",
      "Mean F1-score : 0.696 ± 0.187\n",
      "Mean Accuracy : 0.731 ± 0.175\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 56732 characters]",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "# 10-fold cross-validation on the simple model\n",
    "f1_scores = cross_val_score(logreg_simple, X_train_scaled, y_train, cv=10, scoring='f1')\n",
    "acc_scores = cross_val_score(logreg_simple, X_train_scaled, y_train, cv=10, scoring='accuracy')\n",
    "\n",
    "# Summary\n",
    "print(\"10-fold cross-validation\")\n",
    "print(f\"Mean F1-score : {f1_scores.mean():.3f} ± {f1_scores.std():.3f}\")\n",
    "print(f\"Mean Accuracy : {acc_scores.mean():.3f} ± {acc_scores.std():.3f}\")\n",
    "\n",
    "# Visualization of scores by fold\n",
    "folds = range(1, len(f1_scores) + 1)\n",
    "plt.figure(figsize=(8, 5))\n",
    "plt.plot(folds, f1_scores, marker='o', label='F1-score')\n",
    "plt.plot(folds, acc_scores, marker='s', label='Accuracy')\n",
    "plt.title(\"Scores by fold (cross-validation)\")\n",
    "plt.xlabel(\"Fold\")\n",
    "plt.ylabel(\"Score\")\n",
    "plt.ylim(0, 1)\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8407908d",
   "metadata": {},
   "source": [
    "To evaluate the stability of the logistic regression model and its generalization capacity, we performed a **10-fold cross-validation** (k-fold CV).  \n",
    "Two main metrics were monitored: the **F1-score** (suitable for imbalanced class contexts) and **accuracy** (overall correctness).\n",
    "\n",
    "### Cross-validation results\n",
    "\n",
    "- **Mean F1-score**: 0.696 ± 0.187  \n",
    "- **Mean Accuracy**: 0.731 ± 0.175\n",
    "\n",
    "### Analysis\n",
    "\n",
    "The high variance in scores (standard deviations > 0.17) reflects a **sensitivity to fold composition**, likely due to the **small training set size** (92 observations).  \n",
    "This justifies the need for future **regularization or feature selection**.\n",
    "\n",
    "### Visualization\n",
    "\n",
    "The figure below shows individual scores obtained on each fold. There is **significant variability between folds**, especially in F1-scores (some folds dropping to 0.4–0.5, others exceeding 0.8), confirming the need to optimize the model for better robustness.\n",
    "\n",
    "---\n",
    "\n",
    "This step establishes a baseline for the simple model’s performance before considering **hyperparameter optimization**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c8f1ec9",
   "metadata": {},
   "source": [
    "# Step 10 — Hyperparameter Optimization (GridSearchCV)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "ba6b4fd9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters: {'C': 100.0, 'penalty': 'l1', 'solver': 'liblinear'}\n",
      "Best F1-score (cross-val): 0.737\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "# Hyperparameter grid\n",
    "param_grid = {\n",
    "    'penalty': ['l1', 'l2'],\n",
    "    'C': np.logspace(-2, 4, 10),  # from 0.01 to 10000 (log scale)\n",
    "    'solver': ['liblinear']  # required for 'l1' penalty\n",
    "}\n",
    "\n",
    "# Initialize the model + cross-validation grid search\n",
    "logreg_grid = LogisticRegression(random_state=42, max_iter=1000)\n",
    "grid_search = GridSearchCV(logreg_grid, param_grid, cv=10, scoring='f1')\n",
    "grid_search.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Best model\n",
    "best_logreg = grid_search.best_estimator_\n",
    "\n",
    "print(\"Best parameters:\", grid_search.best_params_)\n",
    "print(\"Best F1-score (cross-val):\", round(grid_search.best_score_, 3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "090475a0",
   "metadata": {},
   "source": [
    "To improve the model’s performance, we implemented a **grid search (GridSearchCV)** to identify the best combination of hyperparameters for logistic regression.  \n",
    "The goal is to automatically select the optimal level of regularization to balance the model’s learning capacity and generalization.\n",
    "\n",
    "---\n",
    "\n",
    "### Hyperparameter grid tested\n",
    "\n",
    "The following combinations were evaluated using 10-fold cross-validation:\n",
    "\n",
    "- **Regularization type (`penalty`)**:  \n",
    "  `'l1'` (Lasso, encouraging sparsity) and `'l2'` (Ridge, classical quadratic regularization),\n",
    "- **Regularization coefficient (`C`)**:  \n",
    "  `[0.01, 0.1, 1, 10, 100]`  \n",
    "  *(the larger `C`, the weaker the regularization; `C` is the inverse of the regularization strength)*,\n",
    "- **Solver (`solver`)**:  \n",
    "  `'liblinear'`, chosen because it is the only solver compatible with `l1` regularization in `scikit-learn`.\n",
    "\n",
    "The evaluation metric used to guide selection was the **F1-score**, well suited to binary classification problems with class imbalance.\n",
    "\n",
    "---\n",
    "\n",
    "### Search results\n",
    "\n",
    "- **Best hyperparameters identified**:  \n",
    "  `{'C': 100, 'penalty': 'l1', 'solver': 'liblinear'}`\n",
    "\n",
    "- **Mean F1-score in cross-validation (10 folds)**:  \n",
    "  **0.737**\n",
    "\n",
    "---\n",
    "\n",
    "### Analysis and interpretation\n",
    "\n",
    "The optimal model uses:\n",
    "- a **weak L1 regularization** (`C = 100`), which reduces the penalization effect while allowing automatic zeroing of some coefficients (sparsity),\n",
    "- a **more parsimonious formulation**, potentially limiting overfitting on a small dataset,\n",
    "- an **interpretable structure**, since non-zero coefficients can be directly analyzed as variable weights.\n",
    "\n",
    "The fact that the best value of `C` lies at the upper edge of the initial grid motivated extending the search range, but this confirmed that `C = 100` remains optimal.  \n",
    "This suggests that regularization, though controlled, **does not have a significant impact** on performance in this context.\n",
    "\n",
    "---\n",
    "\n",
    "### Expected impact\n",
    "\n",
    "The regularized model thus obtained will be evaluated in the next step (step 11) on the test set.  \n",
    "The goal is to check whether this optimization provides **a significant performance gain** compared to the baseline (non-regularized) model of step 8.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57f03133",
   "metadata": {},
   "source": [
    "# Step 11 — Evaluation of the optimized model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "bd749019",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Evaluation on test set:\n",
      "Accuracy: 0.792\n",
      "F1-score: 0.762\n",
      "\n",
      "Classification Report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.79      0.85      0.81        13\n",
      "           1       0.80      0.73      0.76        11\n",
      "\n",
      "    accuracy                           0.79        24\n",
      "   macro avg       0.79      0.79      0.79        24\n",
      "weighted avg       0.79      0.79      0.79        24\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 19836 characters]",
      "text/plain": [
       "<Figure size 600x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 44360 characters]",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix, roc_curve, auc\n",
    "\n",
    "# Predictions\n",
    "y_pred_best = best_logreg.predict(X_test_scaled)\n",
    "\n",
    "# Overall evaluation\n",
    "print(\"\\nEvaluation on test set:\")\n",
    "print(\"Accuracy:\", round(accuracy_score(y_test, y_pred_best), 3))\n",
    "print(\"F1-score:\", round(f1_score(y_test, y_pred_best), 3))\n",
    "print(\"\\nClassification Report:\")\n",
    "print(classification_report(y_test, y_pred_best))\n",
    "\n",
    "# Confusion matrix\n",
    "cm = confusion_matrix(y_test, y_pred_best)\n",
    "plt.figure(figsize=(6, 5))\n",
    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', cbar=False,\n",
    "            xticklabels=['Negative', 'Positive'], yticklabels=['Negative', 'Positive'])\n",
    "plt.title(\"Confusion Matrix\")\n",
    "plt.xlabel(\"Predictions\")\n",
    "plt.ylabel(\"True labels\")\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# ROC curve and AUC\n",
    "y_proba_best = best_logreg.predict_proba(X_test_scaled)[:, 1]\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_proba_best)\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(fpr, tpr, label=f'AUC = {roc_auc:.2f}', color='blue')\n",
    "plt.plot([0, 1], [0, 1], linestyle='--', color='red')\n",
    "plt.title(\"ROC Curve - Optimized Model\")\n",
    "plt.xlabel(\"False Positive Rate (FPR)\")\n",
    "plt.ylabel(\"True Positive Rate (TPR)\")\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bcbfdc8",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "After selecting the best logistic regression model by cross-validation (see Step 10), we evaluated it on the **test set**, which remained completely independent during training.\n",
    "\n",
    "---\n",
    "\n",
    "### Overall Results\n",
    "\n",
    "- **Accuracy**: 0.792  \n",
    "- **F1-score**: 0.762\n",
    "\n",
    "**Classification Report:**\n",
    "\n",
    "| Class        | Precision | Recall | F1-score | Support |\n",
    "|--------------|-----------|--------|----------|---------|\n",
    "| 0 (healthy)  | 0.79      | 0.85   | 0.81     | 13      |\n",
    "| 1 (diseased) | 0.80      | 0.73   | 0.76     | 11      |\n",
    "| **Macro avg**| 0.79      | 0.79   | 0.79     | 24      |\n",
    "\n",
    "The **balance between precision and recall is good**, and overall performance is comparable (slightly lower in F1-score) to that obtained with the simple model (Step 8), despite the imposed L1 regularization.\n",
    "\n",
    "---\n",
    "\n",
    "### Confusion Matrix\n",
    "\n",
    "The matrix below shows class-wise prediction performance:\n",
    "\n",
    "|           | Predicted: 0 | Predicted: 1 |\n",
    "|-----------|--------------|--------------|\n",
    "| **True: 0** | 11           | 2            |\n",
    "| **True: 1** | 3            | 8            |\n",
    "\n",
    "- **11 true negatives** and **8 true positives** correctly identified  \n",
    "- **2 false positives** and **3 false negatives**\n",
    "\n",
    "The model shows a **good ability to identify both classes**, although some diseased cases are still missed (which is critical in a medical context).\n",
    "\n",
    "---\n",
    "\n",
    "### ROC Curve and AUC\n",
    "\n",
    "The ROC curve on the test set confirms a **good discriminative ability**, with an **area under the curve (AUC) of 0.81**.\n",
    "\n",
    "This means that on average, the model assigns a higher probability to a diseased individual than to a healthy individual in **81% of cases**.\n",
    "\n",
    "The ROC curve lies well above the diagonal (random classifier), validating the quality of the model.\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "The optimized model offers a **robust trade-off between performance and sparsity**, featuring:\n",
    "- L1 regularization that limits overfitting,\n",
    "- stability demonstrated through cross-validation,\n",
    "- solid test set performance.\n",
    "\n",
    "This evaluation confirms that logistic regression is a **reliable and interpretable model** on this dataset, although nonlinear models could be explored to further improve results.\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "918960cc",
   "metadata": {},
   "source": [
    "# Step 12 — Comparison of Logistic Regression Variants\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "e9c2a95f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                          Model  Accuracy  F1-score    AUC\n",
      "0                 Simple LogReg     0.792     0.783  0.811\n",
      "1  Optimized LogReg (L1, C=100)     0.792     0.762  0.811\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 52888 characters]",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 1000x600 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 44728 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 33288 characters]",
      "text/plain": [
       "<Figure size 700x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y_test</th>\n",
       "      <th>Pred_simple</th>\n",
       "      <th>Pred_optimized</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     y_test  Pred_simple  Pred_optimized\n",
       "66        0            1               1\n",
       "5         1            0               0\n",
       "80        0            1               0\n",
       "16        1            1               0\n",
       "103       0            1               1\n",
       "13        1            0               0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "# 1. Compute predicted probabilities\n",
    "y_proba_simple = logreg_simple.predict_proba(X_test_scaled)[:, 1]\n",
    "y_proba_best = best_logreg.predict_proba(X_test_scaled)[:, 1]\n",
    "\n",
    "# 2. Summary of scores\n",
    "results = pd.DataFrame({\n",
    "    'Model': ['Simple LogReg', 'Optimized LogReg (L1, C=100)'],\n",
    "    'Accuracy': [\n",
    "        accuracy_score(y_test, y_pred_simple),\n",
    "        accuracy_score(y_test, y_pred_best)\n",
    "    ],\n",
    "    'F1-score': [\n",
    "        f1_score(y_test, y_pred_simple),\n",
    "        f1_score(y_test, y_pred_best)\n",
    "    ],\n",
    "    'AUC': [\n",
    "        roc_auc_score(y_test, y_proba_simple),\n",
    "        roc_auc_score(y_test, y_proba_best)\n",
    "    ]\n",
    "})\n",
    "print(results.round(3))\n",
    "\n",
    "# 3. Overlaid ROC curves\n",
    "fpr_simple, tpr_simple, _ = roc_curve(y_test, y_proba_simple)\n",
    "fpr_best, tpr_best, _ = roc_curve(y_test, y_proba_best)\n",
    "\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(fpr_simple, tpr_simple, '--', label=f'Simple (AUC={roc_auc_score(y_test, y_proba_simple):.2f})')\n",
    "plt.plot(fpr_best, tpr_best, '-', label=f'Optimized (AUC={roc_auc_score(y_test, y_proba_best):.2f})')\n",
    "plt.plot([0, 1], [0, 1], linestyle=':', color='grey')\n",
    "plt.title(\"ROC Curve Comparison\")\n",
    "plt.xlabel(\"False Positive Rate (FPR)\")\n",
    "plt.ylabel(\"True Positive Rate (TPR)\")\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# 4. Comparison of coefficients\n",
    "coef_df = pd.DataFrame({\n",
    "    'Feature': X_train_scaled.columns,\n",
    "    'Simple LogReg': logreg_simple.coef_[0],\n",
    "    'Optimized LogReg (L1)': best_logreg.coef_[0]\n",
    "}).set_index('Feature')\n",
    "\n",
    "plt.figure(figsize=(10, 6))\n",
    "coef_df.T.plot(kind='barh', colormap='viridis')\n",
    "plt.axvline(x=0, color='black', linestyle='--')\n",
    "plt.title(\"Comparison of Coefficients Between Models\")\n",
    "plt.xlabel(\"Coefficient\")\n",
    "plt.ylabel(\"Model / Feature\")\n",
    "plt.grid(True)\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# 5. Distribution of predicted probabilities (class 1)\n",
    "plt.figure(figsize=(7, 5))\n",
    "plt.hist(y_proba_simple, bins=10, alpha=0.5, label=\"Simple\", density=True)\n",
    "plt.hist(y_proba_best, bins=10, alpha=0.5, label=\"Optimized\", density=True)\n",
    "plt.title(\"Distribution of Predicted Probabilities (Positive Class)\")\n",
    "plt.xlabel(\"Predicted Probability\")\n",
    "plt.ylabel(\"Density\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# 6. Analysis of errors per individual\n",
    "errors = pd.DataFrame({\n",
    "    'y_test': y_test.values,\n",
    "    'Pred_simple': y_pred_simple,\n",
    "    'Pred_optimized': y_pred_best\n",
    "}, index=y_test.index)\n",
    "\n",
    "# Display only cases misclassified by at least one model\n",
    "diff = errors.query(\"Pred_simple != y_test or Pred_optimized != y_test\")\n",
    "display(diff)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a30c65a7",
   "metadata": {},
   "source": [
    "We compared two versions of the logistic regression model:\n",
    "- a **simple model**, without explicit regularization (default parameters),\n",
    "- an **optimized model**, with L1 regularization (`penalty='l1'`, `C=100`), determined via cross-validation.\n",
    "\n",
    "---\n",
    "\n",
    "### Overall Performance\n",
    "\n",
    "| Model                        | Accuracy | F1-score | AUC   |\n",
    "|-----------------------------|----------|----------|-------|\n",
    "| Simple LogReg                | 0.792    | 0.783    | 0.811 |\n",
    "| Optimized LogReg (L1, C=100) | 0.792    | 0.762    | 0.811 |\n",
    "\n",
    "Both models achieve **identical accuracy (0.792)** and **equivalent discriminative power (AUC = 0.811)**.  \n",
    "The **simple model attains a slightly higher F1-score**, indicating a better balance between recall and precision on the positive class.\n",
    "\n",
    "---\n",
    "\n",
    "### ROC Curves\n",
    "\n",
    "The ROC curves confirm that the two models are very close in terms of overall performance.  \n",
    "They share the same AUC (0.81) and follow similar trajectories on the test set.\n",
    "\n",
    "---\n",
    "\n",
    "### Coefficients Comparison\n",
    "\n",
    "The graph below compares the weights assigned to each feature by the two models.\n",
    "\n",
    "- The L1-regularized model **strongly modifies some coefficients**, especially those of `Insulin_log` and `HOMA_log`, indicating effective regularization.\n",
    "- Other features like `Glucose` or `BMI` retain a dominant role in both models.\n",
    "\n",
    "This comparison reinforces the idea that the **regularized model is more parsimonious**, while preserving the essential predictive signal.\n",
    "\n",
    "---\n",
    "\n",
    "### Distribution of `predict_proba` Probabilities\n",
    "\n",
    "We visualized the distributions of predicted probabilities for the positive class from both models.\n",
    "\n",
    "- Both distributions cover similar ranges, but the regularized model produces **slightly more extreme predictions** (closer to 0 or 1),\n",
    "- This may indicate **greater confidence in some decisions**, or **better class separation**.\n",
    "\n",
    "---\n",
    "\n",
    "### Errors Comparison\n",
    "\n",
    "We identified the cases where predictions differ between the two models.  \n",
    "Only a few individuals are classified differently, which explains the very close scores.  \n",
    "Example:\n",
    "\n",
    "| id | True Label | Simple | Optimized |\n",
    "|----|------------|--------|-----------|\n",
    "| 80 | 0          | 1      | 0         |\n",
    "| 5  | 1          | 0      | 0         |\n",
    "\n",
    "This confirms that the **error patterns differ structurally**, even though aggregated performances are close.\n",
    "\n",
    "---\n",
    "\n",
    "### Final Analysis\n",
    "\n",
    "The two models are close in terms of performance but show:\n",
    "- **structural differences in coefficients**,\n",
    "- **slight behavioral differences** on borderline individuals,\n",
    "- and a **smoother decision boundary** in the case of the L1 model.\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "| Model                      | Main Advantage                        |\n",
    "|----------------------------|-------------------------------------|\n",
    "| **Simple LogReg**          | Slightly better F1-score             |\n",
    "| **Optimized LogReg (L1)**  | Parsimony, regularized coefficients, better interpretability |\n",
    "\n",
    "In practice, both models can be considered depending on the desired trade-off between raw performance and structural simplicity.  \n",
    "The L1 model is especially relevant for generalization or automatic feature selection purposes.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d11139d7",
   "metadata": {},
   "source": [
    "# Step 13 (Bonus): Applying PCA to Try to Improve the Models\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "b3a8341b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 54324 characters]",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best hyperparameters: {'C': 1.6681005372000592, 'penalty': 'l2', 'solver': 'liblinear'}\n",
      "Mean F1-score (cross-validation): 0.715\n",
      "\n",
      "Test set evaluation:\n",
      "Accuracy: 0.833\n",
      "F1-score: 0.818\n",
      "AUC: 0.832\n",
      "\n",
      "Classification Report:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.85      0.85      0.85        13\n",
      "           1       0.82      0.82      0.82        11\n",
      "\n",
      "    accuracy                           0.83        24\n",
      "   macro avg       0.83      0.83      0.83        24\n",
      "weighted avg       0.83      0.83      0.83        24\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20408 characters]",
      "text/plain": [
       "<Figure size 600x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 44288 characters]",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# Step 1 — Select the best number of components via cross-validation\n",
    "scores = []\n",
    "components_range = range(2, X_train_scaled.shape[1] + 1)\n",
    "\n",
    "for n in components_range:\n",
    "    pca = PCA(n_components=n, svd_solver='full', random_state=42)\n",
    "    X_train_pca = pca.fit_transform(X_train_scaled)\n",
    "    model = LogisticRegression(random_state=42, max_iter=1000)\n",
    "    score = cross_val_score(model, X_train_pca, y_train, cv=10, scoring='f1').mean()\n",
    "    scores.append(score)\n",
    "\n",
    "# Plot performance\n",
    "plt.figure(figsize=(8, 5))\n",
    "plt.plot(components_range, scores, marker='o')\n",
    "plt.title(\"Selecting the Number of Components for PCA\")\n",
    "plt.xlabel(\"Number of Principal Components\")\n",
    "plt.ylabel(\"F1-score (Cross-Validation)\")\n",
    "plt.grid(True)\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Step 2 — Apply PCA with the best number of components\n",
    "n_best = components_range[np.argmax(scores)]\n",
    "pca_final = PCA(n_components=n_best, svd_solver='full', random_state=42)\n",
    "X_train_pca = pca_final.fit_transform(X_train_scaled)\n",
    "X_test_pca = pca_final.transform(X_test_scaled)\n",
    "\n",
    "# Step 3 — Model optimization via GridSearchCV\n",
    "param_grid = {\n",
    "    'penalty': ['l1', 'l2'],\n",
    "    'C': np.logspace(-2, 3, 10),\n",
    "    'solver': ['liblinear']  # required for L1 penalty\n",
    "}\n",
    "logreg = LogisticRegression(random_state=42, max_iter=1000)\n",
    "grid_search = GridSearchCV(logreg, param_grid, cv=10, scoring='f1')\n",
    "grid_search.fit(X_train_pca, y_train)\n",
    "\n",
    "best_model = grid_search.best_estimator_\n",
    "print(\"Best hyperparameters:\", grid_search.best_params_)\n",
    "print(\"Mean F1-score (cross-validation):\", round(grid_search.best_score_, 3))\n",
    "\n",
    "# Step 4 — Final evaluation on the test set\n",
    "y_pred_pca = best_model.predict(X_test_pca)\n",
    "y_proba_pca = best_model.predict_proba(X_test_pca)[:, 1]\n",
    "\n",
    "print(\"\\nTest set evaluation:\")\n",
    "print(\"Accuracy:\", round(accuracy_score(y_test, y_pred_pca), 3))\n",
    "print(\"F1-score:\", round(f1_score(y_test, y_pred_pca), 3))\n",
    "print(\"AUC:\", round(roc_auc_score(y_test, y_proba_pca), 3))\n",
    "print(\"\\nClassification Report:\\n\", classification_report(y_test, y_pred_pca))\n",
    "\n",
    "# Confusion matrix\n",
    "cm = confusion_matrix(y_test, y_pred_pca)\n",
    "plt.figure(figsize=(6, 5))\n",
    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', cbar=False,\n",
    "            xticklabels=['Healthy', 'Diseased'], yticklabels=['Healthy', 'Diseased'])\n",
    "plt.xlabel(\"Predicted\")\n",
    "plt.ylabel(\"Actual\")\n",
    "plt.title(\"Confusion Matrix (PCA)\")\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# ROC curve\n",
    "fpr, tpr, _ = roc_curve(y_test, y_proba_pca)\n",
    "auc_value = roc_auc_score(y_test, y_proba_pca)\n",
    "\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(fpr, tpr, label=f\"AUC = {auc_value:.2f}\", color='blue')\n",
    "plt.plot([0, 1], [0, 1], linestyle='--', color='red')\n",
    "plt.xlabel(\"False Positive Rate (FPR)\")\n",
    "plt.ylabel(\"True Positive Rate (TPR)\")\n",
    "plt.title(\"ROC Curve — Model after PCA\")\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "394a2a73",
   "metadata": {},
   "source": [
    "## Objective\n",
    "\n",
    "The goal of this step is to investigate whether dimensionality reduction using Principal Component Analysis (PCA) can improve the classification model’s performance.\n",
    "\n",
    "---\n",
    "\n",
    "### Justification\n",
    "\n",
    "Even after logarithmic transformation and standardization, explanatory variables may still contain redundant correlations or uninformative noise. PCA helps to:\n",
    "- Project the data into a lower-dimensional space,\n",
    "- Concentrate the majority of variance into a few principal components,\n",
    "- Stabilize learning on small datasets by reducing unnecessary variance.\n",
    "\n",
    "---\n",
    "\n",
    "### Methodology\n",
    "\n",
    "1. PCA is applied on the standardized data (`X_train_scaled`) using `sklearn.decomposition.PCA`.\n",
    "2. Classification performance is evaluated as a function of the number of principal components, ranging from 2 to 9 (limited by the original number of variables).\n",
    "3. For each iteration, a logistic regression model is trained with 10-fold cross-validation to measure the average F1-score.\n",
    "4. The optimal number of components is selected by maximizing this score.\n",
    "5. The optimal model is then retrained on the reduced dataset and evaluated on the test set.\n",
    "\n",
    "---\n",
    "\n",
    "### Selection of Number of Components\n",
    "\n",
    "The figure below shows the evolution of the average F1-score according to the number of principal components:\n",
    "\n",
    "> *Figure – Selection of number of components (PCA)*\n",
    "\n",
    "The maximum is reached at 7 components, with a cross-validation F1-score of 0.715. Beyond this, the score plateaus, indicating that the last components carry little useful discriminative information.\n",
    "\n",
    "---\n",
    "\n",
    "### Final Results on the Test Set\n",
    "\n",
    "After reducing to 7 components and optimizing logistic regression hyperparameters (`C`, `penalty`) via `GridSearchCV`, the results are:\n",
    "\n",
    "- Accuracy: 0.833\n",
    "- F1-score: 0.818\n",
    "- AUC: 0.832\n",
    "\n",
    "#### Classification Report:\n",
    "\n",
    "| Class         | Precision | Recall | F1-score | Support |\n",
    "|---------------|-----------|--------|----------|---------|\n",
    "| 0 (Healthy)   | 0.85      | 0.85   | 0.85     | 13      |\n",
    "| 1 (Diseased)  | 0.82      | 0.82   | 0.82     | 11      |\n",
    "| Macro avg     | 0.83      | 0.83   | 0.83     | 24      |\n",
    "\n",
    "---\n",
    "\n",
    "### Confusion Matrix\n",
    "\n",
    "|               | Predicted Healthy | Predicted Diseased |\n",
    "|---------------|-------------------|--------------------|\n",
    "| Actual Healthy| 11                | 2                  |\n",
    "| Actual Diseased| 2                 | 9                  |\n",
    "\n",
    "The PCA model balances the two classes well, with a reduced number of false positives and false negatives.\n",
    "\n",
    "---\n",
    "\n",
    "### ROC Curve\n",
    "\n",
    "The ROC curve confirms high discriminative power with an AUC of 0.83. This means the model correctly assigns a higher probability to a positive observation than to a negative one in 83% of cases.\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "Applying classical PCA with 7 principal components:\n",
    "- Slightly improves overall performance compared to models without dimensionality reduction,\n",
    "- Stabilizes learning on a small dataset,\n",
    "- Maintains good discriminative capacity while simplifying data representation.\n",
    "\n",
    "These results validate the use of linear PCA for dimensionality reduction and justify its integration into the final supervised learning pipeline.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
