{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d1317e8c",
   "metadata": {},
   "source": [
    "# Cours 5 : Machine Learning - Algorithme CATBOOST"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bb8268a",
   "metadata": {},
   "source": [
    "# Préambule"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ffe2699",
   "metadata": {},
   "source": [
    "Objectifs du cours :\n",
    "1. Comprendre les spécificités de CatBoost vs XGBoost et Random Forest\n",
    "2. Maîtriser le traitement des variables catégorielles\n",
    "3. Optimiser les hyperparamètres\n",
    "4. Appliquer CatBoost à un problème de classification financière"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "666d94aa",
   "metadata": {},
   "source": [
    "# Partie 1 : Introduction théorique"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e05ef8b4",
   "metadata": {},
   "source": [
    "## 1.1 CatBoost : Categorical Boosting\n",
    "\n",
    "### Rappel : Gradient Boosting\n",
    "- Vous avez vu XGBoost : optimisation de la fonction de cout en corrigeant de manière récurrente \n",
    "- CatBoost est aussi un algorithme de gradient boosting, mais avec des innovations :\n",
    "\n",
    "### Trois innovations majeures de CatBoost :\n",
    "1. **Ordered Boosting**\n",
    "   - XGBoost/LightGBM : peuvent souffrir de \"prediction shift\" (overfitting sur train)\n",
    "   - CatBoost utilise un ordre aléatoire différent pour chaque arbre\n",
    "   - Réduit le risque d'overfitting naturellement   \n",
    "2. **Gestion native des variables catégorielles**\n",
    "   - Pas besoin de One-Hot Encoding ou Label Encoding manuel\n",
    "   - Utilise \"Ordered Target Statistics\" pour encoder les catégories\n",
    "   - Évite le target leakage grâce à l'ordered boosting\n",
    "3. **Arbres symétriques (Oblivious Trees)**\n",
    "   - Même condition de split à chaque niveau\n",
    "   - Plus rapide en prédiction\n",
    "   - Régularisation naturelle\n",
    "\n",
    "### Quand utiliser CatBoost ?\n",
    "* Nombreuses variables catégorielles  \n",
    "* Dataset de taille moyenne à grande  \n",
    "* Besoin de robustesse sans trop de tuning  \n",
    "* Problèmes de classification ou régression  \n",
    "\n",
    "### CatBoost vs XGBoost vs Random Forest\n",
    "| Critère | CatBoost | XGBoost | Random Forest |\n",
    "|---------|----------|---------|---------------|\n",
    "| Variables catégorielles |  Natif |  Preprocessing |  Preprocessing |\n",
    "| Vitesse d'entraînement | Moyen | Rapide | Très rapide |\n",
    "| Overfitting | Faible | Moyen | Faible |\n",
    "| Tuning requis | Minimal | Important | Minimal |\n",
    "| Interprétabilité | Bonne | Bonne | Moyenne |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7def124e",
   "metadata": {},
   "source": [
    "# Partie 2 : Mise en pratique - Prédiction de défaut de crédit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ce8ff5c",
   "metadata": {},
   "source": [
    "## Préparation du workspace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8188cb04",
   "metadata": {},
   "source": [
    "### Import de librairies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e3d8e095",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from catboost import CatBoostClassifier, Pool\n",
    "from sklearn.metrics import (\n",
    "    classification_report,\n",
    "    confusion_matrix,\n",
    "    roc_auc_score,\n",
    "    roc_curve,\n",
    ")\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# Configuration des graphiques\n",
    "plt.style.use(\"seaborn-v0_8-darkgrid\")\n",
    "sns.set_palette(\"husl\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1919e0ae",
   "metadata": {},
   "source": [
    "### Import des données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1bbb653c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Chargement du dataset Adult Income ===\n",
      "\n",
      "Dataset classique de Kaggle/UCI qui illustre parfaitement les forces de CatBoost\n",
      "Objectif : Prédire si le revenu annuel > 50K$ basé sur des caractéristiques socio-démographiques\n",
      "\n",
      "Dataset chargé depuis UCI repository\n"
     ]
    }
   ],
   "source": [
    "## Chargement du dataset Adult Income (Census)\n",
    "\n",
    "print(\"=== Chargement du dataset Adult Income ===\\n\")\n",
    "print(\"Dataset classique de Kaggle/UCI qui illustre parfaitement les forces de CatBoost\")\n",
    "print(\"Objectif : Prédire si le revenu annuel > 50K$ basé sur des caractéristiques socio-démographiques\\n\")\n",
    "\n",
    "# Chargement depuis UCI\n",
    "url = \"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data\"\n",
    "\n",
    "column_names = [\n",
    "    'age', 'workclass', 'fnlwgt', 'education', 'education_num',\n",
    "    'marital_status', 'occupation', 'relationship', 'race', 'sex',\n",
    "    'capital_gain', 'capital_loss', 'hours_per_week', 'native_country', 'income'\n",
    "]\n",
    "\n",
    "try:\n",
    "    df = pd.read_csv(url, names=column_names, sep=r',\\s*', engine='python', na_values='?')\n",
    "    print(\"Dataset chargé depuis UCI repository\")\n",
    "except:  # noqa: E722\n",
    "    print(\"Impossible de charger depuis UCI, création d'un dataset simulé similaire...\")\n",
    "    # Fallback : créer un dataset similaire\n",
    "    np.random.seed(42)\n",
    "    n_samples = 32561\n",
    "\n",
    "    df = pd.DataFrame({\n",
    "        'age': np.random.randint(17, 90, n_samples),\n",
    "        'workclass': np.random.choice(['Private', 'Self-emp-not-inc', 'Local-gov', 'State-gov', 'Self-emp-inc',\n",
    "                                       'Federal-gov', 'Without-pay'], n_samples, p=[0.73, 0.08, 0.06, 0.04, 0.03, 0.03, 0.03]),\n",
    "        'fnlwgt': np.random.randint(12285, 1484705, n_samples),\n",
    "        'education': np.random.choice(\n",
    "            ['HS-grad', 'Some-college', 'Bachelors', 'Masters', 'Assoc-voc',\n",
    "                                       'Doctorate', '11th', '9th', '7th-8th'], n_samples,\n",
    "                                      p=[0.32, 0.22, 0.16, 0.05, 0.04, 0.01, 0.04, 0.03, 0.13]),\n",
    "        'education_num': np.random.randint(1, 16, n_samples),\n",
    "        'marital_status': np.random.choice(['Married-civ-spouse', 'Never-married', 'Divorced', 'Separated',\n",
    "                                            'Widowed'], n_samples, p=[0.46, 0.33, 0.14, 0.03, 0.04]),\n",
    "        'occupation': np.random.choice(['Prof-specialty', 'Craft-repair', 'Exec-managerial', 'Adm-clerical',\n",
    "                                        'Sales', 'Other-service', 'Machine-op-inspct', 'Tech-support'], n_samples,\n",
    "                                       p=[0.13, 0.13, 0.13, 0.12, 0.11, 0.10, 0.06, 0.22]),\n",
    "        'relationship': np.random.choice(['Husband', 'Not-in-family', 'Own-child', 'Unmarried', 'Wife', 'Other-relative'],\n",
    "                                         n_samples, p=[0.40, 0.26, 0.16, 0.10, 0.05, 0.03]),\n",
    "        'race': np.random.choice(['White', 'Black', 'Asian-Pac-Islander', 'Amer-Indian-Eskimo', 'Other'],\n",
    "                                 n_samples, p=[0.85, 0.10, 0.03, 0.01, 0.01]),\n",
    "        'sex': np.random.choice(['Male', 'Female'], n_samples, p=[0.67, 0.33]),\n",
    "        'capital_gain': np.where(np.random.random(n_samples) < 0.92, 0, np.random.randint(1, 99999, n_samples)),\n",
    "        'capital_loss': np.where(np.random.random(n_samples) < 0.95, 0, np.random.randint(1, 4356, n_samples)),\n",
    "        'hours_per_week': np.random.randint(1, 99, n_samples),\n",
    "        'native_country': np.random.choice(['United-States', 'Mexico', 'Philippines', 'Germany', 'Canada',\n",
    "                                            'India', 'Other'], n_samples, p=[0.90, 0.02, 0.01, 0.01, 0.01, 0.01, 0.04])\n",
    "    })\n",
    "\n",
    "    # Création de la cible avec logique réaliste\n",
    "    income_score = (\n",
    "        (df['age'] > 35).astype(int) * 20 +\n",
    "        (df['education_num'] > 12).astype(int) * 30 +\n",
    "        (df['hours_per_week'] > 40).astype(int) * 15 +\n",
    "        (df['capital_gain'] > 0).astype(int) * 25 +\n",
    "        (df['marital_status'] == 'Married-civ-spouse').astype(int) * 20 +\n",
    "        (df['occupation'].isin(['Exec-managerial', 'Prof-specialty'])).astype(int) * 15 +\n",
    "        np.random.normal(0, 15, n_samples)\n",
    "    )\n",
    "    df['income'] = (income_score > 60).map({True: '>50K', False: '<=50K'})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a68d80f3",
   "metadata": {},
   "source": [
    "### Première visualisation des données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bc254063",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Shape du dataset : (32561, 15)\n",
      " Nombre de features : 14\n"
     ]
    }
   ],
   "source": [
    "print(f\"\\n Shape du dataset : {df.shape}\")\n",
    "print(f\" Nombre de features : {df.shape[1] - 1}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "94727394",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Valeurs manquantes :\n",
      "workclass         1836\n",
      "occupation        1843\n",
      "native_country     583\n",
      "dtype: int64\n",
      " Dataset nettoyé : (30162, 15)\n"
     ]
    }
   ],
   "source": [
    "# Nettoyage des valeurs manquantes\n",
    "print(\"\\n Valeurs manquantes :\")\n",
    "print(df.isnull().sum()[df.isnull().sum() > 0])\n",
    "\n",
    "# Suppression des lignes avec valeurs manquantes (peu nombreuses)\n",
    "df = df.dropna()\n",
    "print(f\" Dataset nettoyé : {df.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e7685461",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Distribution de la cible (income) :\n",
      "income\n",
      "<=50K    22654\n",
      ">50K      7508\n",
      "Name: count, dtype: int64\n",
      "income\n",
      "<=50K    0.751078\n",
      ">50K     0.248922\n",
      "Name: proportion, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n Distribution de la cible (income) :\")\n",
    "print(df[\"income\"].value_counts())\n",
    "print(df[\"income\"].value_counts(normalize=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "647df2e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Premières lignes :\n",
      "   age         workclass  fnlwgt  education  education_num  \\\n",
      "0   39         State-gov   77516  Bachelors             13   \n",
      "1   50  Self-emp-not-inc   83311  Bachelors             13   \n",
      "2   38           Private  215646    HS-grad              9   \n",
      "3   53           Private  234721       11th              7   \n",
      "4   28           Private  338409  Bachelors             13   \n",
      "\n",
      "       marital_status         occupation   relationship   race     sex  \\\n",
      "0       Never-married       Adm-clerical  Not-in-family  White    Male   \n",
      "1  Married-civ-spouse    Exec-managerial        Husband  White    Male   \n",
      "2            Divorced  Handlers-cleaners  Not-in-family  White    Male   \n",
      "3  Married-civ-spouse  Handlers-cleaners        Husband  Black    Male   \n",
      "4  Married-civ-spouse     Prof-specialty           Wife  Black  Female   \n",
      "\n",
      "   capital_gain  capital_loss  hours_per_week native_country income  \n",
      "0          2174             0              40  United-States  <=50K  \n",
      "1             0             0              13  United-States  <=50K  \n",
      "2             0             0              40  United-States  <=50K  \n",
      "3             0             0              40  United-States  <=50K  \n",
      "4             0             0              40           Cuba  <=50K  \n"
     ]
    }
   ],
   "source": [
    "print(f\"\\n Premières lignes :\\n{df.head()}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8f65e4b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Encodage de la cible en 0/1\n",
    "df['income'] = (df['income'] == '>50K').astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "04bbcd27",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Types de données :\n",
      "age                int64\n",
      "workclass         object\n",
      "fnlwgt             int64\n",
      "education         object\n",
      "education_num      int64\n",
      "marital_status    object\n",
      "occupation        object\n",
      "relationship      object\n",
      "race              object\n",
      "sex               object\n",
      "capital_gain       int64\n",
      "capital_loss       int64\n",
      "hours_per_week     int64\n",
      "native_country    object\n",
      "income             int64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(f\"\\n Types de données :\\n{df.dtypes}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "267aa325",
   "metadata": {},
   "source": [
    "### Exploration rapide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "908f7dce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistiques des variables numériques :\n",
      "                age        fnlwgt  education_num  capital_gain  capital_loss  \\\n",
      "count  30162.000000  3.016200e+04   30162.000000  30162.000000  30162.000000   \n",
      "mean      38.437902  1.897938e+05      10.121312   1092.007858     88.372489   \n",
      "std       13.134665  1.056530e+05       2.549995   7406.346497    404.298370   \n",
      "min       17.000000  1.376900e+04       1.000000      0.000000      0.000000   \n",
      "25%       28.000000  1.176272e+05       9.000000      0.000000      0.000000   \n",
      "50%       37.000000  1.784250e+05      10.000000      0.000000      0.000000   \n",
      "75%       47.000000  2.376285e+05      13.000000      0.000000      0.000000   \n",
      "max       90.000000  1.484705e+06      16.000000  99999.000000   4356.000000   \n",
      "\n",
      "       hours_per_week  \n",
      "count    30162.000000  \n",
      "mean        40.931238  \n",
      "std         11.979984  \n",
      "min          1.000000  \n",
      "25%         40.000000  \n",
      "50%         40.000000  \n",
      "75%         45.000000  \n",
      "max         99.000000  \n"
     ]
    }
   ],
   "source": [
    "# Statistiques descriptives\n",
    "print(\"Statistiques des variables numériques :\")\n",
    "numeric_features = [\n",
    "    \"age\",\n",
    "    \"fnlwgt\",\n",
    "    \"education_num\",\n",
    "    \"capital_gain\",\n",
    "    \"capital_loss\",\n",
    "    \"hours_per_week\",\n",
    "]\n",
    "print(df[numeric_features].describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bfe561fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Variables catégorielles\n",
    "cat_features = [\n",
    "    \"workclass\",\n",
    "    \"education\",\n",
    "    \"marital_status\",\n",
    "    \"occupation\",\n",
    "    \"relationship\",\n",
    "    \"race\",\n",
    "    \"sex\",\n",
    "    \"native_country\",\n",
    "]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b52186fa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Cardinalité des variables catégorielles :\n",
      "   workclass: 7 catégories uniques\n",
      "   education: 16 catégories uniques\n",
      "   marital_status: 7 catégories uniques\n",
      "   occupation: 14 catégories uniques\n",
      "   relationship: 6 catégories uniques\n",
      "   race: 5 catégories uniques\n",
      "   sex: 2 catégories uniques\n",
      "   native_country: 41 catégories uniques\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n Cardinalité des variables catégorielles :\")\n",
    "for col in cat_features:\n",
    "    print(f\"   {col}: {df[col].nunique()} catégories uniques\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b4fca22b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Corrélations avec le revenu >50K :\n",
      "education_num     0.335286\n",
      "age               0.241998\n",
      "hours_per_week    0.229480\n",
      "capital_gain      0.221196\n",
      "capital_loss      0.150053\n",
      "fnlwgt           -0.008957\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# Corrélation avec la cible\n",
    "print(\"\\n Corrélations avec le revenu >50K :\")\n",
    "correlations = df[numeric_features].corrwith(df[\"income\"]).sort_values(ascending=False)\n",
    "print(correlations)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f617c5aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 154760 characters]",
      "text/plain": [
       "<Figure size 1600x1000 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualisation\n",
    "fig, axes = plt.subplots(2, 3, figsize=(16, 10))\n",
    "\n",
    "# Taux de revenu >50K par catégorie\n",
    "df.groupby(\"education\")[\"income\"].mean().sort_values(ascending=False).plot(\n",
    "    kind=\"barh\", ax=axes[0, 0], color=\"skyblue\"\n",
    ")\n",
    "axes[0, 0].set_title(\"Taux de revenu >50K par niveau d'éducation\")\n",
    "axes[0, 0].set_xlabel(\"Taux\")\n",
    "\n",
    "df.groupby(\"occupation\")[\"income\"].mean().sort_values(ascending=False).plot(\n",
    "    kind=\"barh\", ax=axes[0, 1], color=\"lightcoral\"\n",
    ")\n",
    "axes[0, 1].set_title(\"Taux de revenu >50K par occupation\")\n",
    "axes[0, 1].set_xlabel(\"Taux\")\n",
    "\n",
    "df.groupby(\"marital_status\")[\"income\"].mean().sort_values(ascending=False).plot(\n",
    "    kind=\"barh\", ax=axes[0, 2], color=\"lightgreen\"\n",
    ")\n",
    "axes[0, 2].set_title(\"Taux de revenu >50K par statut marital\")\n",
    "axes[0, 2].set_xlabel(\"Taux\")\n",
    "\n",
    "# Distribution des variables numériques par revenu\n",
    "df.boxplot(column=\"age\", by=\"income\", ax=axes[1, 0])\n",
    "axes[1, 0].set_title(\"Âge par niveau de revenu\")\n",
    "axes[1, 0].set_xlabel(\"Revenu >50K\")\n",
    "\n",
    "df.boxplot(column=\"hours_per_week\", by=\"income\", ax=axes[1, 1])\n",
    "axes[1, 1].set_title(\"Heures travaillées par niveau de revenu\")\n",
    "axes[1, 1].set_xlabel(\"Revenu >50K\")\n",
    "\n",
    "df.boxplot(column=\"education_num\", by=\"income\", ax=axes[1, 2])\n",
    "axes[1, 2].set_title(\"Années d'éducation par niveau de revenu\")\n",
    "axes[1, 2].set_xlabel(\"Revenu >50K\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbd53154",
   "metadata": {},
   "source": [
    "**Observations clés :**\n",
    "* L'éducation a un fort impact sur le revenu\n",
    "* Les personnes mariées ont des revenus plus élevés\n",
    "* Certaines occupations (Exec-managerial, Prof-specialty) sont associées à des revenus >50K\n",
    "* L'âge et les heures travaillées sont positivement corrélés au revenu"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b07b569d",
   "metadata": {},
   "source": [
    "## Modélisation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c37658c",
   "metadata": {},
   "source": [
    "### Sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "96b98795",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Préparation pour CatBoost ===\n",
      "\n",
      "Variables catégorielles : ['workclass', 'education', 'marital_status', 'occupation', 'relationship', 'race', 'sex', 'native_country']\n",
      "Variables numériques : ['age', 'fnlwgt', 'education_num', 'capital_gain', 'capital_loss', 'hours_per_week']\n"
     ]
    }
   ],
   "source": [
    "# Identification des variables catégorielles\n",
    "cat_features = [\n",
    "    \"workclass\",\n",
    "    \"education\",\n",
    "    \"marital_status\",\n",
    "    \"occupation\",\n",
    "    \"relationship\",\n",
    "    \"race\",\n",
    "    \"sex\",\n",
    "    \"native_country\",\n",
    "]\n",
    "\n",
    "print(\"\\n=== Préparation pour CatBoost ===\\n\")\n",
    "print(f\"Variables catégorielles : {cat_features}\")\n",
    "print(f\"Variables numériques : {numeric_features}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "344d3cab",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Séparation features / target\n",
    "X = df.drop(\"income\", axis=1)\n",
    "y = df[\"income\"]\n",
    "\n",
    "# Split train/test stratifié\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=42, stratify=y\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9266362a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Taille train : (24129, 14) (6006 positifs)\n",
      "Taille test : (6033, 14) (1502 positifs)\n"
     ]
    }
   ],
   "source": [
    "print(f\"\\nTaille train : {X_train.shape} ({len(y_train[y_train == 1])} positifs)\")\n",
    "print(f\"Taille test : {X_test.shape} ({len(y_test[y_test == 1])} positifs)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "15ae5c81",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Création des Pools CatBoost (structure optimisée)\n",
    "train_pool = Pool(X_train, y_train, cat_features=cat_features)\n",
    "test_pool = Pool(X_test, y_test, cat_features=cat_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cda3c3c",
   "metadata": {},
   "source": [
    "### Algorithme supervisé : CatBoost  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4a9aa383",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Premier modèle CatBoost avec paramètres par défaut\n",
    "# Initialisation\n",
    "model_baseline = CatBoostClassifier(\n",
    "    iterations=500,\n",
    "    learning_rate=0.1,\n",
    "    depth=6,\n",
    "    loss_function=\"Logloss\",\n",
    "    eval_metric=\"AUC\",\n",
    "    random_seed=42,\n",
    "    verbose=100,\n",
    "    early_stopping_rounds=50,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "ed755501",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\ttest: 0.8746563\tbest: 0.8746563 (0)\ttotal: 235ms\tremaining: 1m 57s\n",
      "100:\ttest: 0.9207882\tbest: 0.9207893 (99)\ttotal: 17.6s\tremaining: 1m 9s\n",
      "200:\ttest: 0.9248347\tbest: 0.9248575 (199)\ttotal: 23.4s\tremaining: 34.8s\n",
      "Stopped by overfitting detector  (50 iterations wait)\n",
      "\n",
      "bestTest = 0.9249125201\n",
      "bestIteration = 227\n",
      "\n",
      "Shrink model to first 228 iterations.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x119c30050>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Entrainement\n",
    "model_baseline.fit(train_pool, eval_set=test_pool, plot=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b62bc10b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prédictions\n",
    "y_pred_baseline = model_baseline.predict(X_test)\n",
    "y_pred_proba_baseline = model_baseline.predict_proba(X_test)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "dbc909b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Performance du modèle baseline ===\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.94      0.91      4531\n",
      "           1       0.78      0.65      0.71      1502\n",
      "\n",
      "    accuracy                           0.87      6033\n",
      "   macro avg       0.83      0.79      0.81      6033\n",
      "weighted avg       0.86      0.87      0.86      6033\n",
      "\n",
      "\n",
      "AUC-ROC : 0.9249\n"
     ]
    }
   ],
   "source": [
    "# Évaluation\n",
    "print(\"\\n=== Performance du modèle baseline ===\\n\")\n",
    "print(classification_report(y_test, y_pred_baseline))\n",
    "print(f\"\\nAUC-ROC : {roc_auc_score(y_test, y_pred_proba_baseline):.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "9a92a89d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25908 characters]",
      "text/plain": [
       "<Figure size 800x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Matrice de confusion\n",
    "cm = confusion_matrix(y_test, y_pred_baseline)\n",
    "plt.figure(figsize=(8, 6))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", cmap=\"Blues\")\n",
    "plt.title(\"Matrice de confusion\")\n",
    "plt.ylabel(\"Valeur réelle\")\n",
    "plt.xlabel(\"Valeur prédite\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "df84f1fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 44748 characters]",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Courbe ROC\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba_baseline)\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(fpr, tpr, label=f'CatBoost (AUC = {roc_auc_score(y_test, y_pred_proba_baseline):.3f})')\n",
    "plt.plot([0, 1], [0, 1], 'k--', label='Hasard')\n",
    "plt.xlabel('Taux de faux positifs')\n",
    "plt.ylabel('Taux de vrais positifs')\n",
    "plt.title('Courbe ROC')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "68a8d19d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Importance des variables ===\n",
      "\n",
      "           feature  importance\n",
      "10    capital_gain   22.184166\n",
      "7     relationship   21.515132\n",
      "0              age   11.989052\n",
      "11    capital_loss    8.099907\n",
      "12  hours_per_week    8.092746\n",
      "6       occupation    6.768665\n",
      "5   marital_status    5.823948\n",
      "4    education_num    5.064122\n",
      "3        education    3.166436\n",
      "2           fnlwgt    2.317912\n",
      "1        workclass    1.725724\n",
      "9              sex    1.702985\n",
      "8             race    0.822925\n",
      "13  native_country    0.726279\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40548 characters]",
      "text/plain": [
       "<Figure size 1000x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Importance des variables\n",
    "\n",
    "print(\"\\n=== Importance des variables ===\\n\")\n",
    "\n",
    "feature_importance = model_baseline.get_feature_importance(train_pool)\n",
    "feature_names = X_train.columns\n",
    "\n",
    "importance_df = pd.DataFrame(\n",
    "    {\"feature\": feature_names, \"importance\": feature_importance}\n",
    ").sort_values(\"importance\", ascending=False)\n",
    "\n",
    "print(importance_df)\n",
    "\n",
    "# Visualisation\n",
    "plt.figure(figsize=(10, 8))\n",
    "plt.barh(importance_df[\"feature\"][:15], importance_df[\"importance\"][:15])\n",
    "plt.xlabel(\"Importance\")\n",
    "plt.title(\"Top 15 variables les plus importantes\")\n",
    "plt.gca().invert_yaxis()\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8288a45",
   "metadata": {},
   "source": [
    "# Partie 3 : Exercices"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64e8d141",
   "metadata": {},
   "source": [
    "## Exercice 1 : Tuning des hyperparamètres"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86d7206e",
   "metadata": {},
   "source": [
    "**Objectif : Améliorer les performances du modèle en testant différentes combinaisons\n",
    "d'hyperparamètres.**\n",
    "\n",
    "Hyperparamètres à explorer :\n",
    "- learning_rate : [0.01, 0.05, 0.1, 0.2]\n",
    "- depth : [4, 6, 8, 10]\n",
    "- l2_leaf_reg : [1, 3, 5, 7] (régularisation L2)\n",
    "- bagging_temperature : [0, 0.5, 1] (randomisation de l'échantillonnage)\n",
    "\n",
    "Consignes :\n",
    "1. Utilisez une boucle ou GridSearchCV pour tester différentes combinaisons\n",
    "2. Comparez les AUC-ROC sur le test set\n",
    "3. Identifiez la meilleure configuration\n",
    "4. Entraînez le modèle final avec les meilleurs paramètres"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03e1ee55",
   "metadata": {},
   "source": [
    "## Exercice 2 : Analyse approfondie"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86594aca",
   "metadata": {},
   "source": [
    "**Contexte : Vous êtes data scientist dans une banque. Le coût d'un faux négatif \n",
    "(prêter à quelqu'un qui va faire défaut) est estimé à 10 000€. Le coût d'un \n",
    "faux positif (refuser un bon client) est de 500€ (perte d'opportunité).**\n",
    "\n",
    "**Objectifs :**\n",
    "1. Calculer le coût total pour différents seuils de classification (0.3, 0.5, 0.7)\n",
    "2. Déterminer le seuil optimal qui minimise le coût total\n",
    "3. Produire un rapport de recommandation\n",
    "\n",
    "Formule :\n",
    "Coût total = (FN × 10000) + (FP × 500)\n",
    "\n",
    "Consignes :\n",
    "- Utilisez y_pred_proba_baseline pour tester différents seuils\n",
    "- Créez une fonction pour calculer le coût\n",
    "- Visualisez le coût en fonction du seuil\n",
    "- Recommandez le meilleur seuil avec justification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "65693a22",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\ttest: 0.8746563\tbest: 0.8746563 (0)\ttotal: 172ms\tremaining: 1m 25s\n",
      "100:\ttest: 0.9207882\tbest: 0.9207893 (99)\ttotal: 8.11s\tremaining: 32s\n",
      "200:\ttest: 0.9248347\tbest: 0.9248575 (199)\ttotal: 12.4s\tremaining: 18.4s\n",
      "Stopped by overfitting detector  (50 iterations wait)\n",
      "\n",
      "bestTest = 0.9249125201\n",
      "bestIteration = 227\n",
      "\n",
      "Shrink model to first 228 iterations.\n",
      "\n",
      "=== Performance du modèle baseline ===\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.94      0.91      4531\n",
      "           1       0.78      0.65      0.71      1502\n",
      "\n",
      "    accuracy                           0.87      6033\n",
      "   macro avg       0.83      0.79      0.81      6033\n",
      "weighted avg       0.86      0.87      0.86      6033\n",
      "\n",
      "\n",
      "AUC-ROC : 0.9249\n"
     ]
    }
   ],
   "source": [
    "best_catboost = CatBoostClassifier(\n",
    "    iterations=500,\n",
    "    learning_rate=0.1,\n",
    "    depth=6,\n",
    "    loss_function=\"Logloss\",\n",
    "    eval_metric=\"AUC\",\n",
    "    random_seed=42,\n",
    "    verbose=100,\n",
    "    early_stopping_rounds=50,\n",
    ")\n",
    "\n",
    "# Entrainement\n",
    "best_catboost.fit(train_pool, eval_set=test_pool, plot=False)\n",
    "\n",
    "# Prédictions\n",
    "y_pred_baseline = best_catboost.predict(X_test)\n",
    "y_pred_proba_baseline = best_catboost.predict_proba(X_test)[:, 1]\n",
    "\n",
    "# Évaluation\n",
    "print(\"\\n=== Performance du modèle baseline ===\\n\")\n",
    "print(classification_report(y_test, y_pred_baseline))\n",
    "print(f\"\\nAUC-ROC : {roc_auc_score(y_test, y_pred_proba_baseline):.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "af1cb8af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25908 characters]",
      "text/plain": [
       "<Figure size 800x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Matrice de confusion\n",
    "cm = confusion_matrix(y_test, y_pred_baseline)\n",
    "plt.figure(figsize=(8, 6))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", cmap=\"Blues\")\n",
    "plt.title(\"Matrice de confusion\")\n",
    "plt.ylabel(\"Valeur réelle\")\n",
    "plt.xlabel(\"Valeur prédite\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fbbf84b",
   "metadata": {},
   "source": [
    "## Exercice 3 : Prédiction du montant de défaut (régression)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ba92dd4",
   "metadata": {},
   "source": [
    "**Objectif : Transformer le problème en régression pour prédire le montant \n",
    "du défaut potentiel (et pas seulement si/non).**\n",
    "\n",
    "Étapes :\n",
    "1. Créer une nouvelle variable cible : 'montant_defaut' \n",
    "   - Si défaut = 0 : montant_defaut = 0\n",
    "   - Si défaut = 1 : montant_defaut = montant_credit * ratio_defaut_simule\n",
    "   \n",
    "2. Entraîner un CatBoostRegressor\n",
    "\n",
    "3. Évaluer avec MAE, RMSE, R²\n",
    "\n",
    "4. Comparer l'importance des features avec le modèle de classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e74f071f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Création de la variable defaut (oubli dans l'énoncé)\n",
    "df[\"montant_credit\"] = np.random.uniform(3000, 450000)\n",
    "df[\"defaut\"] = df[\"capital_gain\"] < df[\"capital_loss\"]\n",
    "\n",
    "df = df.drop([\"capital_gain\", \"capital_loss\"], axis=1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "5a9ac689",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\tlearn: 49203.9724124\ttest: 50617.1901164\tbest: 50617.1901164 (0)\ttotal: 41ms\tremaining: 20.5s\n",
      "100:\tlearn: 48437.1893196\ttest: 50052.4073304\tbest: 50038.4930903 (73)\ttotal: 2.54s\tremaining: 10s\n",
      "200:\tlearn: 47905.0927458\ttest: 50121.0948143\tbest: 50038.4930903 (73)\ttotal: 6.04s\tremaining: 8.98s\n",
      "300:\tlearn: 47372.1749510\ttest: 50166.1469085\tbest: 50038.4930903 (73)\ttotal: 9.39s\tremaining: 6.21s\n",
      "400:\tlearn: 46855.4993995\ttest: 50270.6663358\tbest: 50038.4930903 (73)\ttotal: 13.2s\tremaining: 3.26s\n",
      "499:\tlearn: 46487.9636285\ttest: 50308.7451900\tbest: 50038.4930903 (73)\ttotal: 16.3s\tremaining: 0us\n",
      "\n",
      "bestTest = 50038.49309\n",
      "bestIteration = 73\n",
      "\n",
      "Shrink model to first 74 iterations.\n",
      "MAE : 20,089.78€\n",
      "RMSE : 50,038.49€\n",
      "R² : 0.0267\n",
      "\\nPerte totale réelle : 66,263,722.75€\n",
      "Perte totale prédite : 64,040,461.12€\n",
      "Erreur : 2,223,261.63€\n"
     ]
    }
   ],
   "source": [
    "from catboost import CatBoostRegressor\n",
    "from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n",
    "\n",
    "# Création de la cible de régression\n",
    "np.random.seed(42)\n",
    "df[\"montant_defaut\"] = df.apply(\n",
    "    lambda row: row[\"montant_credit\"] * np.random.uniform(0.3, 0.8)\n",
    "    if row[\"defaut\"] == 1\n",
    "    else 0,\n",
    "    axis=1,\n",
    ")\n",
    "\n",
    "# Préparation\n",
    "X_reg = df.drop([\"defaut\", \"montant_defaut\"], axis=1)\n",
    "y_reg = df[\"montant_defaut\"]\n",
    "\n",
    "X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(\n",
    "    X_reg, y_reg, test_size=0.2, random_state=42\n",
    ")\n",
    "\n",
    "# Pools\n",
    "train_pool_reg = Pool(X_train_reg, y_train_reg, cat_features=cat_features)\n",
    "test_pool_reg = Pool(X_test_reg, y_test_reg, cat_features=cat_features)\n",
    "\n",
    "# Modèle\n",
    "model_reg = CatBoostRegressor(\n",
    "    iterations=500, learning_rate=0.1, depth=6, random_seed=42, verbose=100\n",
    ")\n",
    "\n",
    "model_reg.fit(train_pool_reg, eval_set=test_pool_reg)\n",
    "\n",
    "# Prédictions\n",
    "y_pred_reg = model_reg.predict(X_test_reg)\n",
    "\n",
    "# Évaluation\n",
    "print(f\"MAE : {mean_absolute_error(y_test_reg, y_pred_reg):,.2f}€\")\n",
    "print(f\"RMSE : {np.sqrt(mean_squared_error(y_test_reg, y_pred_reg)):,.2f}€\")\n",
    "print(f\"R² : {r2_score(y_test_reg, y_pred_reg):.4f}\")\n",
    "\n",
    "# Perte attendue totale\n",
    "perte_totale_reelle = y_test_reg.sum()\n",
    "perte_totale_predite = y_pred_reg.sum()\n",
    "print(f\"\\\\nPerte totale réelle : {perte_totale_reelle:,.2f}€\")\n",
    "print(f\"Perte totale prédite : {perte_totale_predite:,.2f}€\")\n",
    "print(f\"Erreur : {abs(perte_totale_reelle - perte_totale_predite):,.2f}€\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "e71ef5a4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Importance des variables ===\n",
      "\n",
      "           feature  importance\n",
      "12          income   50.866106\n",
      "3        education    7.719029\n",
      "4    education_num    7.679601\n",
      "2           fnlwgt    5.390006\n",
      "6       occupation    4.938505\n",
      "5   marital_status    4.321928\n",
      "11  native_country    4.066952\n",
      "0              age    3.823869\n",
      "10  hours_per_week    3.527171\n",
      "7     relationship    2.400281\n",
      "1        workclass    2.350175\n",
      "8             race    2.238380\n",
      "9              sex    0.677997\n",
      "13  montant_credit    0.000000\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40564 characters]",
      "text/plain": [
       "<Figure size 1000x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Importance des variables\n",
    "\n",
    "print(\"\\n=== Importance des variables ===\\n\")\n",
    "\n",
    "feature_importance = model_reg.get_feature_importance(train_pool_reg)\n",
    "feature_names = X_reg.columns\n",
    "\n",
    "importance_df = pd.DataFrame(\n",
    "    {\"feature\": feature_names, \"importance\": feature_importance}\n",
    ").sort_values(\"importance\", ascending=False)\n",
    "\n",
    "print(importance_df)\n",
    "\n",
    "# Visualisation\n",
    "plt.figure(figsize=(10, 8))\n",
    "plt.barh(importance_df[\"feature\"][:15], importance_df[\"importance\"][:15])\n",
    "plt.xlabel(\"Importance\")\n",
    "plt.title(\"Top 15 variables les plus importantes\")\n",
    "plt.gca().invert_yaxis()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a54f79a5",
   "metadata": {},
   "source": [
    "# Partie 4 : Techniques avancées (bonus)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8849fdbc",
   "metadata": {},
   "source": [
    "## Gestion du déséquilibre des classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c9bbd9a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x1192a6710>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Option 1 : Pondération automatique\n",
    "model_weighted = CatBoostClassifier(\n",
    "    iterations=500,\n",
    "    learning_rate=0.1,\n",
    "    depth=6,\n",
    "    random_seed=42,\n",
    "    verbose=0,\n",
    "    auto_class_weights='Balanced',\n",
    "    eval_metric='AUC'\n",
    ")\n",
    "\n",
    "model_weighted.fit(train_pool, eval_set=test_pool)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "9f10f1a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_weighted = model_weighted.predict(X_test)\n",
    "y_pred_proba_weighted = model_weighted.predict_proba(X_test)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "89f57e71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance avec class weights :\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.94      0.83      0.88      4531\n",
      "           1       0.63      0.84      0.72      1502\n",
      "\n",
      "    accuracy                           0.84      6033\n",
      "   macro avg       0.78      0.84      0.80      6033\n",
      "weighted avg       0.86      0.84      0.84      6033\n",
      "\n",
      "AUC-ROC : 0.9253\n"
     ]
    }
   ],
   "source": [
    "print(\"Performance avec class weights :\")\n",
    "print(classification_report(y_test, y_pred_weighted))\n",
    "print(f\"AUC-ROC : {roc_auc_score(y_test, y_pred_proba_weighted):.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "dd70cf9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Scale pos weight calculé : 3.02\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x118d76780>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Option 2 : scale_pos_weight (pour déséquilibre important)\n",
    "scale = len(y_train[y_train == 0]) / len(y_train[y_train == 1])\n",
    "print(f\"\\nScale pos weight calculé : {scale:.2f}\")\n",
    "\n",
    "model_scaled = CatBoostClassifier(\n",
    "    iterations=500,\n",
    "    learning_rate=0.1,\n",
    "    depth=6,\n",
    "    random_seed=42,\n",
    "    verbose=0,\n",
    "    scale_pos_weight=scale,\n",
    "    eval_metric='AUC'\n",
    ")\n",
    "\n",
    "model_scaled.fit(train_pool, eval_set=test_pool)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81526670",
   "metadata": {},
   "source": [
    "## SHAP Values pour l'interprétabilité"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "c9a3f8ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# CatBoost a un support natif pour SHAP\n",
    "shap_values = model_baseline.get_feature_importance(\n",
    "    train_pool,\n",
    "    type='ShapValues'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "8c29907f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape des SHAP values : (24129, 15)\n"
     ]
    }
   ],
   "source": [
    "# Les shap_values ont la forme (n_samples, n_features + 1)\n",
    "# La dernière colonne est la valeur de base\n",
    "\n",
    "print(f\"Shape des SHAP values : {shap_values.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "d638560d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Importance SHAP moyenne :\n",
      "           feature  shap_importance\n",
      "7     relationship         0.833481\n",
      "0              age         0.763489\n",
      "10    capital_gain         0.484163\n",
      "5   marital_status         0.426608\n",
      "6       occupation         0.415689\n",
      "12  hours_per_week         0.378604\n",
      "3        education         0.281114\n",
      "4    education_num         0.271613\n",
      "9              sex         0.144403\n",
      "11    capital_loss         0.133650\n"
     ]
    }
   ],
   "source": [
    "# Importance SHAP moyenne\n",
    "shap_importance = np.abs(shap_values[:, :-1]).mean(axis=0)\n",
    "shap_df = pd.DataFrame({\n",
    "    'feature': X_train.columns,\n",
    "    'shap_importance': shap_importance\n",
    "}).sort_values('shap_importance', ascending=False)\n",
    "\n",
    "print(\"\\nImportance SHAP moyenne :\")\n",
    "print(shap_df.head(10))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "79488875",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 43200 characters]",
      "text/plain": [
       "<Figure size 1000x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualisation\n",
    "plt.figure(figsize=(10, 8))\n",
    "plt.barh(shap_df['feature'][:15], shap_df['shap_importance'][:15])\n",
    "plt.xlabel('|SHAP value| moyen')\n",
    "plt.title('Importance des features (SHAP)')\n",
    "plt.gca().invert_yaxis()\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "studies",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
