Fix execution counts and update policy array initialization in maze game notebooks

Update Python version and refine Jupyter Notebook formatting
- Bump Python version from 3.11 to 3.13 in .python-version file. - Reset execution counts to null in Jupyter Notebook for reproducibility. - Improve code readability by adjusting comments and formatting in the notebook. - Change the policy definition to use numpy.ndarray for better clarity. - Modify pyproject.toml to enable E501 rule for line length management.
2026-01-14 15:54:13 +01:00 · 2026-01-06 11:13:17 +01:00 · 2026-01-06 11:07:31 +01:00 · 2026-01-06 10:22:42 +01:00
8 changed files with 3846 additions and 101 deletions
--- a/Starter.ipynb
+++ b/Starter.ipynb
@@ -0,0 +1,456 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "172a7a9f",
+   "metadata": {},
+   "source": [
+    "# TP2 - Benchmark automatique\n",
+    "\n",
+    "Dans ce TP nous allons définir une fonction pour mesurer les performances d'un modèle de langage via l'exécution de plusieurs benchmarks. Nous avons vu en cours trois manières de mesurer la performance d'un modèle de langage qu'on peut résumer à:\n",
+    "1. **Évaluation automatique**: via un ensemble de questions dont on connait la réponse\n",
+    "2. **Évaluation humaine**: qualification humaine de la réponse d'un modèle à une question\n",
+    "3. **Évaluation par modèle de langage**: notation ou comparaison de réponse d'un ou plusieurs modèles par un autre modèle\n",
+    "\n",
+    "Nous nous intéressons ici au premier point, en particulier avec les benchmarks [GSM8K](https://huggingface.co/datasets/openai/gsm8k) et [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag).\n",
+    "Dans l'ensemble du notebook nous utiliserons la librairie LangChain.\n",
+    "\n",
+    "Il est à garder en tête que ce notebook n'a qu'une portée pédagogique et n'est pas forcément à jour puisque le domaine évolue rapidement, ni que les pratiques sont celles validées par l'industrie.\n",
+    "\n",
+    "## Uniformisation des benchmarks\n",
+    "\n",
+    "Pour chaque benchmark que l'on considère, nous avons besoin de plusieurs informations :\n",
+    "* **Dataset** : une fonction pour charger les questions du benchmark\n",
+    "* **Référence** : une fonction capable d'identifier la réponse attentue\n",
+    "* **Prompt** : un prompt qui permet de demander correctement au modèle de répondre à la question\n",
+    "* **Chaîne** : une fonction qui renvoie la chaîne de traitement de LangChain\n",
+    "* **Score** : une fonction qui score la performance d'un modèle sur une question\n",
+    "\n",
+    "Nous allons commencer par créer une classe qui regroupe ces desiderata :\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cd75374d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.runnables import Runnable\n",
+    "from langchain_core.prompts import PromptTemplate\n",
+    "\n",
+    "\n",
+    "class Benchmark:\n",
+    "    name: str\n",
+    "\n",
+    "    def __init__(self, prompt: PromptTemplate):\n",
+    "        self.prompt = prompt\n",
+    "\n",
+    "    def load_data(self):\n",
+    "        raise NotImplementedError\n",
+    "\n",
+    "    def build_chain(self, model) -> Runnable:\n",
+    "        raise NotImplementedError\n",
+    "\n",
+    "    def get_reference(self, sample):\n",
+    "        raise NotImplementedError\n",
+    "\n",
+    "    def score(self, prediction, reference):\n",
+    "        raise NotImplementedError"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e2ab41df",
+   "metadata": {},
+   "source": [
+    "Pour rendre cette classe plus concrète, commençons par travailler avec le benchmark [GSM8K](https://huggingface.co/datasets/openai/gsm8k).\n",
+    "\n",
+    "### Benchmark GSM8K\n",
+    "\n",
+    "On commence par charger le dataset et observer une question.\n",
+    "\n",
+    "**Consigne** : Résoudre la question *à la main* et vérifier votre réponse. On recommande d'explorer plusieurs questions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "93979ba0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np; np.random.seed(42)\n",
+    "from datasets import load_dataset\n",
+    "\n",
+    "dataset = load_dataset(\"gsm8k\", \"main\")\n",
+    "dataset = dataset[\"test\"]\n",
+    "\n",
+    "print(f\"Number of questions: {len(dataset)}\")\n",
+    "index = 0\n",
+    "print(\"Example of question:\\n\", dataset[index][\"question\"])\n",
+    "print(\"And its answer:\\n\", dataset[index][\"answer\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "82d797f0",
+   "metadata": {},
+   "source": [
+    "Après avoir inspecté plusieurs éléments du dataset, on remarque que la réponse finale est placée après la chaîne de caractères \"####\".\n",
+    "\n",
+    "**Consigne**: Construire une fonction `get_reference` qui prend en argument un élément de GMS8K (dictionnaire avec question et réponse) et renvoie la réponse attendue (string). On pourra utiliser la fonction [`search`](https://docs.python.org/3/library/re.html#re.search) de la librairie [`re`](https://docs.python.org/3/library/re.html#).\n",
+    "Puis tester cette fonction sur l'exemple précédent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b336056a",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4c137e6a",
+   "metadata": {},
+   "source": [
+    "Il nous reste maintenant à définir un prompt tel que l'on puisse appeler un modèle et tester notre mécanique."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0b899872",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.prompts import PromptTemplate\n",
+    "\n",
+    "prompt = PromptTemplate(\n",
+    "    input_variables=[\"question\"],\n",
+    "    template=(\n",
+    "        \"\"\"You are a careful mathematician. Solve the problem step by step, then display your answer in the end.\n",
+    "        Question: {question}\n",
+    "        Answer:\"\"\"\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "36433b53",
+   "metadata": {},
+   "source": [
+    "En intégrant l'appel à un modèle via Ollama sur notre ordinateur, on peut définir avec LangChain :"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2f0676b6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.runnables import RunnablePassthrough\n",
+    "from langchain_core.output_parsers import StrOutputParser\n",
+    "from langchain_ollama import OllamaLLM\n",
+    "\n",
+    "model = OllamaLLM(model=\"gemma3:4b\")\n",
+    "\n",
+    "chain = (\n",
+    "    {\"question\": RunnablePassthrough()}\n",
+    "    | prompt\n",
+    "    | model\n",
+    "    | StrOutputParser()\n",
+    ")\n",
+    "\n",
+    "index = 0\n",
+    "\n",
+    "question = dataset[index][\"question\"]\n",
+    "answer = get_reference(dataset[index])\n",
+    "response = chain.invoke(question)\n",
+    "print(f\"Model answer : {response}\")\n",
+    "print(f\"The answer was : {answer}\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "97dd7db7",
+   "metadata": {},
+   "source": [
+    "Il nous faut extraire la dernière valeur numérique pour obtenir automatiquement la réponse du modèle.\n",
+    "\n",
+    "**Consigne** : Définir une fonction `score` qui prend en paramètre la réponse du modèle et la réponse attendue puis renvoie si les deux réponses sont identiques (1 / 0). On pourra utiliser la fonction [`findall`](https://docs.python.org/3/library/re.html#re.findall) de la librairie `re`.\n",
+    "Puis l'appliquer sur l'exemple précédent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ad43cf84",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a2ec5088",
+   "metadata": {},
+   "source": [
+    "Nous avons l'ensemble des éléments nécessaire pour définir la classe `GSM8KBenchmark` depuis la classe `Benchmark` que nous avons défini précédemment.\n",
+    "\n",
+    "**Consigne** : Définir cette classe comme sous-classe de `Benchmark`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d83f4394",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dfc3cb78",
+   "metadata": {},
+   "source": [
+    "Il est maintenant temps de définir une fonction qui *fait* le benchmark.\n",
+    "\n",
+    "**Consigne** : Définir une fonction `run_benchmark` qui prend en paramètre :\n",
+    "* `model_name` : le nom du modèle Ollama que l'on veut tester\n",
+    "* `benchmark` : la classe benchmark que l'on souhaite tester\n",
+    "* `max_samples` : le nombre maximum de questions que l'on souhaite utiliser\n",
+    "\n",
+    "Puisque l'object avec lequel nous travaillons est un dataset HuggingFace, pour sélectionner $n$ lignes, on utilisera \n",
+    "```python\n",
+    "dataset = dataset.select(range(max_samples))\n",
+    "```\n",
+    "De cette manière on préserve la structure."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2d7125af",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "81de8940",
+   "metadata": {},
+   "source": [
+    "**Consigne** : Utiliser la fonction `run_benchmark` en définissant un prompt pour GSM8K."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f6bbeb53",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0c943124",
+   "metadata": {},
+   "source": [
+    "### HellaSwag\n",
+    "\n",
+    "Maintenant que nous avons réussi à le faire pour le dataset GMS8K, attaquons-nous à [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag).\n",
+    "\n",
+    "**Consigne** : En suivant la même approche que précédemment, implémenter une sous classe `HellaSwagBenchmark` à partir de la classe `Benchmark`. Puis utiliser la fonction `run_benchmark` pour valider votre travail."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "32886901",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "96a3031a",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c542783c",
+   "metadata": {},
+   "source": [
+    "## Réponses structurées\n",
+    "\n",
+    "Sur quelques exemples tout semble fonctionner ! Mais il y a au moins une fragilité dans notre travail : la récupération de la réponse est peu fiable et largement dépendante des prompts.\n",
+    "\n",
+    "\n",
+    "Par exemple pour GMS8K, on aimerait avoir une réponse sous la forme d'un JSON :\n",
+    "```json\n",
+    "{\n",
+    "  \"reasoning\": \"étapes de raisonnement\",\n",
+    "  \"final_answer\": 18\n",
+    "}\n",
+    "```\n",
+    "\n",
+    "De cette manière ce serait particulièrement simple d'extraire la réponse, sans pour autant ne pas avoir de *réflexion* du modèle. En revanche pour HellaSwag, un JSON extrêment simple suffit :\n",
+    "```json\n",
+    "{\n",
+    "    \"choice\": 2\n",
+    "}\n",
+    "```\n",
+    "\n",
+    "Pour forcer le modèle à suivre ces formats, nous allons utiliser l'option [Pydantic](https://docs.langchain.com/oss/python/langchain/structured-output). Elle s'utilise comme suit, pour GSM8K :"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "988dbca3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "\n",
+    "class GSM8KOutput(BaseModel):\n",
+    "    reasoning: str = Field(description=\"Step-by-step reasoning\")\n",
+    "    final_answer: float = Field(description=\"Final numeric answer\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d855adfe",
+   "metadata": {},
+   "source": [
+    "Concernant l'intégration dans le prompt :"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f25afddc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.output_parsers import PydanticOutputParser\n",
+    "\n",
+    "parser_gsm8k = PydanticOutputParser(pydantic_object=GSM8KOutput)\n",
+    "\n",
+    "prompt_gsm8k = PromptTemplate(\n",
+    "    input_variables=[\"question\"],\n",
+    "    partial_variables={\"format_instructions\": parser_gsm8k.get_format_instructions()},\n",
+    "    template=(\n",
+    "        \"\"\"You are a careful mathematician. Solve the problem step by step.\n",
+    "        Question: {question}\n",
+    "        {format_instructions}\"\"\"\n",
+    "    ),\n",
+    ")\n",
+    "\n",
+    "print(parser_gsm8k.get_format_instructions())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d1dcc480",
+   "metadata": {},
+   "source": [
+    "**Consigne** : Modifier la classe `Benchmark` et la sous-classe `GMS8KBenchmark` pour intégrer ces évolutions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "542a31d6",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c94f1dd1",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b2076f24",
+   "metadata": {},
+   "source": [
+    "**Consigne** : Utiliser la fonction `run_benchmark` et vérifier que tout fonctionne."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "31e433b0",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b7ed90cd",
+   "metadata": {},
+   "source": [
+    "**Consigne** : Réaliser la même modification pour HellaSwag, et vérifier que cela fonctionne."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e678bed2",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2455f816",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ba9acd54",
+   "metadata": {},
+   "source": [
+    "## Pour aller plus loin\n",
+    "\n",
+    "On pourrait implémenter d'autres benchmark, comparer vraiment des modèles entre eux, comparer des prompts entre eux..."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
--- a/Starter.ipynb
+++ b/Starter.ipynb
--- a/AI/TP1/data/ML.pdf
+++ b/AI/TP1/data/ML.pdf
--- a/M2/Reinforcement
+++ b/M2/Reinforcement
@@ -31,7 +31,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 73,
+   "execution_count": 1,
   "id": "100d1e0d",
   "metadata": {},
   "outputs": [],
@@ -42,7 +42,9 @@
    "np.set_printoptions(\n",
    "    precision=3,\n",
    "    suppress=True,\n",
-    ")  # (not mandatory) This line is for limiting floats to 3 decimal places, avoiding scientific notation (like 1.23e-04) for small numbers.\n",
+    ")\n",
+    "# (not mandatory) This line is for limiting floats to 3 decimal places,\n",
+    "# avoiding scientific notation (like 1.23e-04) for small numbers.\n",
    "\n",
    "# For reproducibility\n",
    "rng = np.random.default_rng(seed=42)  # This line creates a random number generator."
@@ -98,7 +100,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 74,
+   "execution_count": 2,
   "id": "f91cda05",
   "metadata": {},
   "outputs": [],
@@ -126,7 +128,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 75,
+   "execution_count": 3,
   "id": "564cb757-eefe-4be6-9b6f-bb77ace42a97",
   "metadata": {},
   "outputs": [
@@ -148,7 +150,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 76,
+   "execution_count": 4,
   "id": "26c821d3-2362-4b60-8c77-3d09296d130d",
   "metadata": {},
   "outputs": [
@@ -198,7 +200,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 77,
+   "execution_count": 5,
   "id": "7116044b-c134-43de-9f30-01ab62325300",
   "metadata": {},
   "outputs": [],
@@ -229,7 +231,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 78,
+   "execution_count": 6,
   "id": "a1258de4",
   "metadata": {},
   "outputs": [
@@ -249,18 +251,17 @@
    "pos_to_state = {}  # (i,j) -> s\n",
    "\n",
    "start_state = None  # will store the state index of start state\n",
-    "goal_states = []  # will store the state index of goal state # We use a list in case there are multiple goals\n",
-    "trap_states = []  # will store the state index of trap state # We use a list in case there are multiple traps\n",
+    "goal_states = []  # will store the state index of goal state\n",
+    "trap_states = []  # will store the state index of trap state\n",
    "\n",
    "s = 0\n",
    "for i in range(n_rows):  # i = row index\n",
    "    for j in range(n_cols):  # j = column index\n",
    "        cell = maze_str[i][j]  # cell = the character at that position (S, ., #, etc.)\n",
    "\n",
-    "        if (\n",
-    "            cell in FREE\n",
-    "        ):  # FREE contains: free cells \".\", start cell \"S\", goal cell \"G\" and trap cell \"X\"\n",
-    "            # Walls # are ignored, they are not MDP states.\n",
+    "        if cell in FREE:\n",
+    "        # FREE contains: free cells \".\", start cell \"S\", goal cell \"G\" and trap cell \"X\"\n",
+    "        # Walls # are ignored, they are not MDP states.\n",
    "            state_to_pos[s] = (i, j)\n",
    "            pos_to_state[(i, j)] = s\n",
    "\n",
@@ -291,7 +292,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 79,
+   "execution_count": 7,
   "id": "68744dd6-7278-4c20-8b82-34212685352f",
   "metadata": {},
   "outputs": [
@@ -365,7 +366,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 80,
+   "execution_count": 8,
   "id": "fc61ceef-217c-47f4-8eba-0353369210db",
   "metadata": {},
   "outputs": [
@@ -398,7 +399,7 @@
    "    for (\n",
    "        s,\n",
    "        (i, j),\n",
-    "    ) in state_to_pos.items():  # Calling .items() returns a list-like sequence of (key, value) pairs in the dictionary.\n",
+    "    ) in state_to_pos.items():\n",
    "        cell = maze_str[i][j]\n",
    "\n",
    "        if cell == \"S\":\n",
@@ -475,7 +476,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 81,
+   "execution_count": 9,
   "id": "f7f0b8e4-1f48-4d03-9e5f-a47e59c3e827",
   "metadata": {},
   "outputs": [],
@@ -487,7 +488,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 82,
+   "execution_count": 10,
   "id": "3773781c-a0cd-48db-967b-d4b432d17046",
   "metadata": {},
   "outputs": [
@@ -517,7 +518,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 83,
+   "execution_count": 11,
   "id": "4b06da5e-bc63-48e5-a336-37bce952443d",
   "metadata": {},
   "outputs": [],
@@ -596,7 +597,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 84,
+   "execution_count": 12,
   "id": "610253e7-f3f7-4a30-be3e-2ec5a1e2ed04",
   "metadata": {},
   "outputs": [],
@@ -627,7 +628,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 85,
+   "execution_count": 13,
   "id": "7a51f242-fe4e-4e74-8a1f-a8df32b194b8",
   "metadata": {},
   "outputs": [],
@@ -655,7 +656,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 86,
+   "execution_count": 14,
   "id": "49d54d1f-dc29-45b6-ad31-ad0e848f920d",
   "metadata": {},
   "outputs": [],
@@ -700,7 +701,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 87,
+   "execution_count": 15,
   "id": "b9b7495a-c233-425c-99c0-5bddaf6c3225",
   "metadata": {},
   "outputs": [],
@@ -726,7 +727,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 88,
+   "execution_count": 16,
   "id": "eca4c571-39c7-468b-af86-0bab9489415e",
   "metadata": {},
   "outputs": [],
@@ -757,7 +758,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 89,
+   "execution_count": 17,
   "id": "2d03276b-e206-4d1f-9024-f6948ca61523",
   "metadata": {},
   "outputs": [],
@@ -820,7 +821,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 90,
+   "execution_count": 18,
   "id": "341fe630-8f87-4773-84ad-92d3516e53e2",
   "metadata": {},
   "outputs": [
@@ -967,7 +968,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 91,
+   "execution_count": 19,
   "id": "2fffe0b7",
   "metadata": {},
   "outputs": [],
@@ -1036,7 +1037,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 92,
+   "execution_count": 20,
   "id": "b4a44e38",
   "metadata": {},
   "outputs": [
@@ -1065,7 +1066,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 93,
+   "execution_count": 21,
   "id": "c5f559b2-452a-477c-a1fa-258b40805670",
   "metadata": {},
   "outputs": [
@@ -1109,7 +1110,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 94,
+   "execution_count": 22,
   "id": "4c428327",
   "metadata": {},
   "outputs": [
@@ -1125,7 +1126,7 @@
    }
   ],
   "source": [
-    "def plot_values(V: np.ndarray, title=\"Value function\") -> None:\n",
+    "def plot_values(V: np.ndarray, title: str = \"Value function\") -> None:\n",
    "    \"\"\"Plot the value function V on the maze as a heatmap.\"\"\"\n",
    "    grid_values = np.full(\n",
    "        (n_rows, n_cols),\n",
@@ -1149,7 +1150,7 @@
    "    # For each state:\n",
    "    # Place the text label at (column j, row i).\n",
    "    # Display value to two decimals.\n",
-    "    # Use white text so it’s visible on the heatmap.\n",
+    "    # Use white text so it's visible on the heatmap.\n",
    "    # Center the text inside each cell.\n",
    "\n",
    "    for s, (i, j) in state_to_pos.items():\n",
@@ -1183,12 +1184,12 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 95,
+   "execution_count": 23,
   "id": "c1ab67f0-bd5e-4ffe-b655-aec030401b78",
   "metadata": {},
   "outputs": [],
   "source": [
-    "def plot_policy(policy: np.ndarray, title=\"Policy\") -> None:\n",
+    "def plot_policy(policy: np.ndarray, title: str =\"Policy\") -> None:\n",
    "    \"\"\"Plot the given policy on the maze.\"\"\"\n",
    "    _fig, ax = plt.subplots()\n",
    "    # draw walls as dark cells\n",
@@ -1253,7 +1254,7 @@
    "    ax.set_yticks(np.arange(-0.5, n_rows, 1))\n",
    "    ax.set_xticklabels([])\n",
    "    ax.set_yticklabels([])\n",
-    "    ax.grid(True)\n",
+    "    ax.grid(visible=True)\n",
    "    ax.set_title(title)\n",
    "    plt.show()"
   ]
@@ -1268,7 +1269,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 96,
+   "execution_count": 24,
   "id": "d452681c-c89c-41cc-95dc-df75993b0391",
   "metadata": {},
   "outputs": [
@@ -1297,7 +1298,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 97,
+   "execution_count": 25,
   "id": "929707e6-3022-4d86-96cc-12f251f890a9",
   "metadata": {},
   "outputs": [
@@ -1323,35 +1324,37 @@
    }
   ],
   "source": [
-    "my_policy = [\n",
-    "    A_RIGHT,\n",
-    "    A_RIGHT,\n",
-    "    A_RIGHT,\n",
-    "    A_DOWN,\n",
-    "    A_DOWN,  # First row\n",
-    "    A_UP,\n",
-    "    A_DOWN,\n",
-    "    A_DOWN,\n",
-    "    A_LEFT,  # Second row\n",
-    "    A_UP,\n",
-    "    A_RIGHT,\n",
-    "    A_DOWN,  # Third row\n",
-    "    A_UP,\n",
-    "    A_LEFT,\n",
-    "    A_RIGHT,\n",
-    "    A_RIGHT,\n",
-    "    A_RIGHT,  # Fourth row\n",
-    "    A_UP,\n",
-    "    A_LEFT,\n",
-    "    A_DOWN,\n",
-    "    A_RIGHT,\n",
-    "    A_UP,  # Fifth row\n",
-    "]\n",
+    "my_policy = np.array(\n",
+    "    [\n",
+    "        A_RIGHT,\n",
+    "        A_RIGHT,\n",
+    "        A_RIGHT,\n",
+    "        A_DOWN,\n",
+    "        A_DOWN,  # First row\n",
+    "        A_UP,\n",
+    "        A_DOWN,\n",
+    "        A_DOWN,\n",
+    "        A_LEFT,  # Second row\n",
+    "        A_UP,\n",
+    "        A_RIGHT,\n",
+    "        A_DOWN,  # Third row\n",
+    "        A_UP,\n",
+    "        A_LEFT,\n",
+    "        A_RIGHT,\n",
+    "        A_RIGHT,\n",
+    "        A_RIGHT,  # Fourth row\n",
+    "        A_UP,\n",
+    "        A_LEFT,\n",
+    "        A_DOWN,\n",
+    "        A_RIGHT,\n",
+    "        A_UP,  # Fifth row\n",
+    "    ],\n",
+    ")\n",
    "\n",
    "V_my_policy = policy_evaluation(policy=my_policy, P=P, R=R, gamma=gamma)\n",
    "\n",
    "plot_values(V=V_my_policy, title=\"Value function: my policy\")\n",
-    "plot_policy(policy=my_policy, title=\"My policy\")"
+    "plot_policy(policy=my_policy, title=\"My policy\")\n"
   ]
  },
  {
--- a/M2/Reinforcement
+++ b/M2/Reinforcement
@@ -14,7 +14,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 535,
+   "execution_count": 24,
   "id": "100d1e0d",
   "metadata": {},
   "outputs": [],
@@ -49,7 +49,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 536,
+   "execution_count": 25,
   "id": "f91cda05",
   "metadata": {},
   "outputs": [],
@@ -72,7 +72,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 537,
+   "execution_count": 26,
   "id": "564cb757-eefe-4be6-9b6f-bb77ace42a97",
   "metadata": {},
   "outputs": [],
@@ -93,7 +93,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 538,
+   "execution_count": 27,
   "id": "7116044b-c134-43de-9f30-01ab62325300",
   "metadata": {},
   "outputs": [],
@@ -116,7 +116,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 539,
+   "execution_count": 28,
   "id": "a1258de4",
   "metadata": {},
   "outputs": [
@@ -170,7 +170,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 540,
+   "execution_count": 29,
   "id": "fc61ceef-217c-47f4-8eba-0353369210db",
   "metadata": {},
   "outputs": [
@@ -250,7 +250,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 541,
+   "execution_count": 30,
   "id": "f7f0b8e4-1f48-4d03-9e5f-a47e59c3e827",
   "metadata": {},
   "outputs": [],
@@ -262,7 +262,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 542,
+   "execution_count": 31,
   "id": "4b06da5e-bc63-48e5-a336-37bce952443d",
   "metadata": {},
   "outputs": [],
@@ -330,7 +330,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 543,
+   "execution_count": 32,
   "id": "610253e7-f3f7-4a30-be3e-2ec5a1e2ed04",
   "metadata": {},
   "outputs": [],
@@ -341,7 +341,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 544,
+   "execution_count": 33,
   "id": "7a51f242-fe4e-4e74-8a1f-a8df32b194b8",
   "metadata": {},
   "outputs": [],
@@ -353,7 +353,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 545,
+   "execution_count": 34,
   "id": "49d54d1f-dc29-45b6-ad31-ad0e848f920d",
   "metadata": {},
   "outputs": [],
@@ -366,7 +366,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 546,
+   "execution_count": 35,
   "id": "b9b7495a-c233-425c-99c0-5bddaf6c3225",
   "metadata": {},
   "outputs": [],
@@ -382,7 +382,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 547,
+   "execution_count": 36,
   "id": "eca4c571-39c7-468b-af86-0bab9489415e",
   "metadata": {},
   "outputs": [],
@@ -397,7 +397,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 548,
+   "execution_count": 37,
   "id": "2d03276b-e206-4d1f-9024-f6948ca61523",
   "metadata": {},
   "outputs": [],
@@ -445,7 +445,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 549,
+   "execution_count": 38,
   "id": "341fe630-8f87-4773-84ad-92d3516e53e2",
   "metadata": {},
   "outputs": [
@@ -492,7 +492,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 550,
+   "execution_count": 39,
   "id": "2fffe0b7",
   "metadata": {},
   "outputs": [],
@@ -543,7 +543,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 551,
+   "execution_count": 40,
   "id": "4c428327",
   "metadata": {},
   "outputs": [],
@@ -595,7 +595,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 552,
+   "execution_count": null,
   "id": "c1ab67f0-bd5e-4ffe-b655-aec030401b78",
   "metadata": {},
   "outputs": [],
@@ -665,7 +665,7 @@
    "    ax.set_yticks(np.arange(-0.5, n_rows, 1))\n",
    "    ax.set_xticklabels([])\n",
    "    ax.set_yticklabels([])\n",
-    "    ax.grid(True)\n",
+    "    ax.grid(visible=True)\n",
    "    ax.set_title(title)\n",
    "    plt.show()"
   ]
@@ -680,7 +680,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 553,
+   "execution_count": 42,
   "id": "ceb5dfe2",
   "metadata": {},
   "outputs": [
@@ -702,7 +702,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 554,
+   "execution_count": 43,
   "id": "8f3e2ac2",
   "metadata": {},
   "outputs": [
@@ -729,7 +729,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 558,
+   "execution_count": 44,
   "id": "cf45291e",
   "metadata": {},
   "outputs": [
@@ -761,7 +761,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 557,
+   "execution_count": 45,
   "id": "5a82a3b7",
   "metadata": {},
   "outputs": [
@@ -787,7 +787,7 @@
    }
   ],
   "source": [
-    "my_policy = [\n",
+    "my_policy = np.array([\n",
    "    A_DOWN,\n",
    "    A_DOWN,\n",
    "    A_LEFT,\n",
@@ -850,7 +850,7 @@
    "    A_UP,\n",
    "    A_DOWN,\n",
    "    A_LEFT,\n",
-    "]\n",
+    "])\n",
    "\n",
    "V_my_policy = policy_evaluation(policy=my_policy, P=P, R=R, gamma=gamma)\n",
    "\n",
--- a/README.md
+++ b/README.md
@@ -31,6 +31,7 @@ The projects are organized into two main sections:
 - `M2`
  - `Data Visualisation`
  - `Deep Learning`
+  - `Generative AI`
  - `Linear Models`
  - `Machine Learning`
  - `Reinforcement Learning`
@@ -58,4 +59,5 @@ The projects are organized into two main sections:
 - [FactoMineR](https://factominer.free.fr/): An R package focused on multivariate exploratory data analysis (e.g., PCA, MCA, CA).
 - [ggplot2](https://ggplot2.tidyverse.org): A grammar-based graphics package for creating complex and elegant visualizations in R.
 - [RShiny](https://shiny.rstudio.com): A web application framework for building interactive web apps directly from R.
+- [LangChain](https://langchain.com): A framework for developing applications powered by language models.
 - and my 🧠.
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,24 +3,31 @@ name = "studies"
 version = "0.1.0"
 description = "A curated collection of mathematics and data science projects developed during my academic journey."
 readme = "README.md"
-requires-python = ">=3.12"
+requires-python = ">= 3.11"
 dependencies = [
    "catboost>=1.2.8",
+    "faiss-cpu>=1.13.2",
    "imblearn>=0.0",
    "ipykernel>=6.29.5",
-    "keras>=3.11.3",
+    "langchain>=1.2.0",
+    "langchain-community>=0.4.1",
+    "langchain-huggingface>=1.2.0",
+    "langchain-ollama>=1.0.1",
    "matplotlib>=3.10.1",
    "nbformat>=5.10.4",
    "numpy>=2.2.5",
    "opencv-python>=4.11.0.86",
-    "pandas>=2.2.3", 
+    "pandas>=2.2.3",
    "pandas-stubs>=2.3.2.250926",
    "plotly>=6.3.0",
+    "pypdf>=6.5.0",
    "scikit-learn>=1.6.1",
    "scipy>=1.15.2",
    "seaborn>=0.13.2",
+    "sentence-transformers>=5.2.0",
    "shap>=0.49.1",
    "tensorflow>=2.20.0",
+    "tf-keras>=2.20.1",
    "xgboost>=3.1.2",
    "yfinance>=0.2.66",
 ]
@@ -44,7 +51,7 @@ select = ["ALL"]

 # Désactiver certaines règles
 ignore = [
-    # "E501", # line too long, géré par le formatter
+    "E501", # line too long, géré par le formatter
    "E402", # Imports in top of file
    "T201", # Print
    "N806",
--- a/uv.lock
+++ b/uv.lock
Author	SHA1	Message	Date
Arthur DANJOU	0e65815e38	Fix execution counts and update policy array initialization in maze game notebooks	2026-01-06 11:13:17 +01:00
Arthur DANJOU	6eecdd6ab3	Update Python version and refine Jupyter Notebook formatting - Bump Python version from 3.11 to 3.13 in .python-version file. - Reset execution counts to null in Jupyter Notebook for reproducibility. - Improve code readability by adjusting comments and formatting in the notebook. - Change the policy definition to use numpy.ndarray for better clarity. - Modify pyproject.toml to enable E501 rule for line length management.	2026-01-06 11:07:31 +01:00
Arthur DANJOU	06bc1f28a9	Refactor code structure for improved readability and maintainability	2026-01-06 10:22:42 +01:00