{
"cells": [
{
"cell_type": "markdown",
"id": "78b68162",
"metadata": {},
"source": [
"# TP3 - Fine tuning\n",
"\n",
"L'objectif de ce TP est de présenter comment exploiter les modèles disponibles sur HuggingFace, plus particulièrement pour des tâches de classification. Nous mettrons également en pratique l'approche [*Two-stage fine-tuning*](https://arxiv.org/pdf/2207.10858).\n",
"\n",
"## Contexte\n",
"\n",
"Les flux RSS, de l'anglais *Rich Site Summary*, est un flux d'information pour partager simplement des informations. Largement utilisé dans le domaine de la communication par les médias, nous pouvons nous tenir au courant des informations que les journaux partagent par exemple.\n",
"Cependant, le volume d'informations peut être rapidement très important pour pouvoir être lu. On se propose de construire un filtre pour nous notifier uniquement quand une information sera jugée *d'intérêt*.\n",
"\n",
"Pour cela, nous avons sauvegardé puis labellisé les flux RSS de plusieurs journaux mondiaux en anglais, pendant plusieurs semaines. Importons et visualisons la bases de données à disposition."
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "7e9e1d29",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"7619 news with 10.59% of positive class\n"
]
},
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "index",
"rawType": "int64",
"type": "integer"
},
{
"name": "provider",
"rawType": "object",
"type": "string"
},
{
"name": "category",
"rawType": "object",
"type": "string"
},
{
"name": "link",
"rawType": "object",
"type": "string"
},
{
"name": "published",
"rawType": "datetime64[ns, UTC]",
"type": "unknown"
},
{
"name": "title",
"rawType": "object",
"type": "string"
},
{
"name": "summary",
"rawType": "object",
"type": "string"
},
{
"name": "target",
"rawType": "int64",
"type": "integer"
}
],
"ref": "46cf2b21-2a0e-4b5c-a13a-fad0480005c4",
"rows": [
[
"0",
"El Pais",
"World",
"https://english.elpais.com/climate/2025-12-27/a-floating-school-teaches-children-how-to-save-lake-atitlan.html",
"2025-12-27 04:00:00+00:00",
"A floating school teaches children how to save Lake Atitlán",
"This is the latest attempt by Guatemalan scientists to make kids aware that the once-pristine lake is in danger of ‘dying’ from the constant discharge of waste water, garbage and erosion",
"0"
],
[
"1",
"El Pais",
"World",
"https://english.elpais.com/climate/2025-12-31/david-king-chemist-there-are-scientists-studying-how-to-cool-the-planet-nobody-should-stop-these-experiments-from-happening.html",
"2025-12-31 13:53:41+00:00",
"David King, chemist: ‘There are scientists studying how to cool the planet; nobody should stop these experiments from happening’",
"The British researcher is calling for global regulations to test extreme measures against climate change, such as marine cloud brightening over the Arctic",
"0"
],
[
"2",
"El Pais",
"World",
"https://english.elpais.com/climate/2026-01-01/cartagena-de-indias-is-sinking-what-can-the-city-do-to-mitigate-it.html",
"2026-01-01 12:24:15+00:00",
"Cartagena de Indias is sinking: What can the city do to mitigate it?",
"Sea levels in the bay of the Colombian resort have risen seven millimeters per year for the past two decades, the second-highest rate in the Caribbean after Haiti",
"0"
],
[
"3",
"El Pais",
"World",
"https://english.elpais.com/climate/2026-01-03/the-secrets-of-the-washingtonia-palm-named-after-the-first-us-president.html",
"2026-01-03 04:45:00+00:00",
"The secrets of the Washingtonia palm, named after the first US president",
"They are among the most widely cultivated plants in the world, and can thrive in complete neglect, able to withstand both harsh summers and low temperatures",
"0"
],
[
"4",
"El Pais",
"World",
"https://english.elpais.com/climate/2026-01-05/a-recipe-for-resistance-indigenous-peoples-politicize-their-struggles-from-the-kitchen.html",
"2026-01-05 17:02:27+00:00",
"A recipe for resistance: Indigenous peoples politicize their struggles from the kitchen",
"A book that compiles the recovery of ancestral recipes from the Americas, from Canada to the Amazon, maps strategies in the face of climate change and agribusiness",
"0"
]
],
"shape": {
"columns": 7,
"rows": 5
}
},
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
provider
\n",
"
category
\n",
"
link
\n",
"
published
\n",
"
title
\n",
"
summary
\n",
"
target
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
El Pais
\n",
"
World
\n",
"
https://english.elpais.com/climate/2025-12-27/...
\n",
"
2025-12-27 04:00:00+00:00
\n",
"
A floating school teaches children how to save...
\n",
"
This is the latest attempt by Guatemalan scien...
\n",
"
0
\n",
"
\n",
"
\n",
"
1
\n",
"
El Pais
\n",
"
World
\n",
"
https://english.elpais.com/climate/2025-12-31/...
\n",
"
2025-12-31 13:53:41+00:00
\n",
"
David King, chemist: ‘There are scientists stu...
\n",
"
The British researcher is calling for global r...
\n",
"
0
\n",
"
\n",
"
\n",
"
2
\n",
"
El Pais
\n",
"
World
\n",
"
https://english.elpais.com/climate/2026-01-01/...
\n",
"
2026-01-01 12:24:15+00:00
\n",
"
Cartagena de Indias is sinking: What can the c...
\n",
"
Sea levels in the bay of the Colombian resort ...
\n",
"
0
\n",
"
\n",
"
\n",
"
3
\n",
"
El Pais
\n",
"
World
\n",
"
https://english.elpais.com/climate/2026-01-03/...
\n",
"
2026-01-03 04:45:00+00:00
\n",
"
The secrets of the Washingtonia palm, named af...
\n",
"
They are among the most widely cultivated plan...
\n",
"
0
\n",
"
\n",
"
\n",
"
4
\n",
"
El Pais
\n",
"
World
\n",
"
https://english.elpais.com/climate/2026-01-05/...
\n",
"
2026-01-05 17:02:27+00:00
\n",
"
A recipe for resistance: Indigenous peoples po...
\n",
"
A book that compiles the recovery of ancestral...
\n",
"
0
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" provider category link \\\n",
"0 El Pais World https://english.elpais.com/climate/2025-12-27/... \n",
"1 El Pais World https://english.elpais.com/climate/2025-12-31/... \n",
"2 El Pais World https://english.elpais.com/climate/2026-01-01/... \n",
"3 El Pais World https://english.elpais.com/climate/2026-01-03/... \n",
"4 El Pais World https://english.elpais.com/climate/2026-01-05/... \n",
"\n",
" published \\\n",
"0 2025-12-27 04:00:00+00:00 \n",
"1 2025-12-31 13:53:41+00:00 \n",
"2 2026-01-01 12:24:15+00:00 \n",
"3 2026-01-03 04:45:00+00:00 \n",
"4 2026-01-05 17:02:27+00:00 \n",
"\n",
" title \\\n",
"0 A floating school teaches children how to save... \n",
"1 David King, chemist: ‘There are scientists stu... \n",
"2 Cartagena de Indias is sinking: What can the c... \n",
"3 The secrets of the Washingtonia palm, named af... \n",
"4 A recipe for resistance: Indigenous peoples po... \n",
"\n",
" summary target \n",
"0 This is the latest attempt by Guatemalan scien... 0 \n",
"1 The British researcher is calling for global r... 0 \n",
"2 Sea levels in the bay of the Colombian resort ... 0 \n",
"3 They are among the most widely cultivated plan... 0 \n",
"4 A book that compiles the recovery of ancestral... 0 "
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"df = pd.read_csv(\"data/Complete.csv\")\n",
"\n",
"df[\"published\"] = pd.to_datetime(df[\"published\"])\n",
"df = df.loc[~df[\"target\"].isna()].copy()\n",
"df[\"target\"] = df[\"target\"].astype(int)\n",
"\n",
"\n",
"imbalanced_rate = df[\"target\"].mean()\n",
"print(f\"{df.shape[0]} news with {100 * imbalanced_rate:.2f}% of positive class\")\n",
"\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"id": "19931804",
"metadata": {},
"source": [
"## Préparation\n",
"\n",
"Nous n'allons exploiter que le titre et le contenu du flux RSS, et avant de commencer nous allons nettoyer un peu le texte."
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "45710a37",
"metadata": {},
"outputs": [],
"source": [
"def clean_text(title: str, summary: str) -> str:\n",
" \"\"\"Clean the title and summary strings.\"\"\"\n",
"\n",
" def clean_string(string: str) -> str:\n",
" \"\"\"Clean a single string.\"\"\"\n",
" string = string.lower()\n",
" string = string.replace(\"(Source: Bloomberg)\", \"\")\n",
" string = string.replace(\"“\", \"'\")\n",
" string = string.replace(\"”\", \"'\")\n",
" string = string.replace(\"\\n\", \"\")\n",
" string = string.replace(\" \", \" \")\n",
" return string.strip()\n",
"\n",
" title = clean_string(title)\n",
" summary = clean_string(summary)\n",
"\n",
" return f\"Title: {title}\\nSummary: {summary}\"\n",
"\n",
"\n",
"df[\"text\"] = df.apply(lambda row: clean_text(row[\"title\"], row[\"summary\"]), axis=1)"
]
},
{
"cell_type": "markdown",
"id": "467c2960",
"metadata": {},
"source": [
"Puisque le dataset est dépendant du temps, nous allons le découper en jeu d'entraînement et de validation en suivant cela."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "024281d1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training size: 6092, imbalanced rate: 10.69%\n",
"Validation size: 1523, imbalanced rate: 10.18%\n"
]
}
],
"source": [
"train_ratio = 0.8\n",
"\n",
"df = df.drop_duplicates(subset=[\"text\"])\n",
"df = df.sort_values(\"published\")\n",
"split_date = df[\"published\"].quantile(train_ratio)\n",
"\n",
"df_train = df[df[\"published\"] <= split_date][[\"text\", \"target\"]]\n",
"df_val = df[df[\"published\"] > split_date][[\"text\", \"target\"]]\n",
"\n",
"df_train = df_train.reset_index(drop=True)\n",
"df_val = df_val.reset_index(drop=True)\n",
"\n",
"print(\n",
" f\"Training size: {len(df_train)}, imbalanced rate: {100 * df_train.target.mean():.2f}%\",\n",
")\n",
"print(\n",
" f\"Validation size: {len(df_val)}, imbalanced rate: {100 * df_val.target.mean():.2f}%\",\n",
")"
]
},
{
"cell_type": "markdown",
"id": "2d9d353f",
"metadata": {},
"source": [
"Puisque nous avons un déséquilibre, nous allons le reporter dans la fonction de perte. Pour préparer cette étape, nous allons utiliser la fonction [`compute_class_weight`](https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html) et stocker son résultat dans un [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html) de PyTorch pour l'exploiter dans la fonction de perte."
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "71f1c0e3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0.5598, 4.6790])\n"
]
}
],
"source": [
"import numpy as np\n",
"\n",
"import torch\n",
"from sklearn.utils.class_weight import compute_class_weight\n",
"\n",
"labels = df_train[\"target\"].to_numpy()\n",
"class_weights = compute_class_weight(\n",
" class_weight=\"balanced\",\n",
" classes=np.array([0, 1]),\n",
" y=labels,\n",
")\n",
"\n",
"class_weights = torch.tensor(class_weights, dtype=torch.float)\n",
"print(class_weights)"
]
},
{
"cell_type": "markdown",
"id": "f093593f",
"metadata": {},
"source": [
"## Premier entraînement\n",
"\n",
"Il est maintenant temps de basculer du monde classique numpy, pandas, scikit-learn à celui de la librairies transformers d'HuggingFace.\n",
"\n",
"### Dataset d'entraînement\n",
"\n",
"On ne travaille plus avec des DataFrames pandas mais des [Datasets](https://huggingface.co/docs/datasets/use_with_pandas). "
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "2b32e4cc",
"metadata": {},
"outputs": [],
"source": [
"from datasets import Dataset\n",
"\n",
"ds_train = Dataset.from_pandas(df_train)\n",
"ds_val = Dataset.from_pandas(df_val)\n",
"\n",
"ds_train = ds_train.rename_column(\"target\", \"labels\")\n",
"ds_val = ds_val.rename_column(\"target\", \"labels\")"
]
},
{
"cell_type": "markdown",
"id": "824250f7",
"metadata": {},
"source": [
"Le texte contenu dans les datasets n'est pas *compréhensible* en l'état par un modèle: nous devons **tokeniser** le texte. Nous devons le faire en utilisant la même méthode qui a servit à entraîner le modèle que nous utiliserons par la suite : nous choisissons [DistilRoBERTa base](https://huggingface.co/distilbert/distilroberta-base). Le choix s'est fait sur les capacités du modèle et sa taille réduite pour permettre des itérations *rapide*."
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "55224f64",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Map: 100%|██████████| 6092/6092 [00:00<00:00, 14604.07 examples/s]\n",
"Map: 100%|██████████| 1523/1523 [00:00<00:00, 16717.37 examples/s]\n"
]
}
],
"source": [
"from transformers import AutoTokenizer\n",
"\n",
"MODEL_NAME = \"distilroberta-base\"\n",
"\n",
"tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\n",
"\n",
"\n",
"def tokenize(batch: dict) -> dict:\n",
" \"\"\"Tokenize a batch of texts.\"\"\"\n",
" return tokenizer(\n",
" batch[\"text\"],\n",
" truncation=True,\n",
" padding=\"max_length\",\n",
" max_length=256,\n",
" )\n",
"\n",
"\n",
"ds_train = ds_train.map(tokenize, batched=True)\n",
"ds_val = ds_val.map(tokenize, batched=True)"
]
},
{
"cell_type": "markdown",
"id": "301f82fc",
"metadata": {},
"source": [
"Dans cette cellule nous avons réutiliser le tokenizer du modèle d'intérêt puis l'avons appliquer à la colonne contenant le texte. A présent, le dataset n'est plus composé de deux colonnes (labels et target) mais :"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "4bc70018",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Dataset({\n",
" features: ['text', 'labels', 'input_ids', 'attention_mask'],\n",
" num_rows: 6092\n",
"})"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds_train"
]
},
{
"cell_type": "markdown",
"id": "d2a4ad98",
"metadata": {},
"source": [
"Avant de continuer, nous changeons le format des données en tenseur torch."
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "056a0baf",
"metadata": {},
"outputs": [],
"source": [
"ds_train.set_format(\n",
" type=\"torch\",\n",
" columns=[\"input_ids\", \"attention_mask\", \"labels\"],\n",
")\n",
"ds_val.set_format(\n",
" type=\"torch\",\n",
" columns=[\"input_ids\", \"attention_mask\", \"labels\"],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "c290b844",
"metadata": {},
"source": [
"La librairie transformers n'a pas de méthode `fit` comme dans scikit-learn ou Keras, mais une classe [`Trainer`](https://huggingface.co/docs/transformers/main_classes/trainer).\n",
"Nativement, cette classe ne prend pas en compte le déséquilibre : nous allons la modifier pour le faire. On se propose d'hériter de la classe `Trainer` et de modifier uniquement la méthode `compute_loss` pour qu'elle utilise la [`CrossEntropyLoss`](https://docs.pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) de PyTorch."
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "02a89166",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from transformers import Trainer\n",
"\n",
"\n",
"class WeightedTrainer(Trainer):\n",
" \"\"\"Custom Trainer with weighted loss to handle class imbalance.\"\"\"\n",
"\n",
" def compute_loss(\n",
" self,\n",
" model,\n",
" inputs,\n",
" return_outputs = False,\n",
" num_items_in_batch = None,\n",
" ):\n",
" \"\"\"Compute the weighted cross-entropy loss for class imbalance.\"\"\"\n",
" labels = inputs.pop(\"labels\")\n",
" outputs = model(**inputs)\n",
" logits = outputs.get(\"logits\")\n",
"\n",
" # Configuration de la perte avec les poids de classe\n",
" loss_fct = torch.nn.CrossEntropyLoss(weight=class_weights.to(logits.device))\n",
"\n",
" # Calcul de la perte\n",
" loss = loss_fct(logits.view(-1, self.model.config.num_labels), labels.view(-1))\n",
"\n",
" return (loss, outputs) if return_outputs else loss\n"
]
},
{
"cell_type": "markdown",
"id": "69e3c4c0",
"metadata": {},
"source": [
"Tout au long de l'entraînement du modèle, nous voulons être capable de suivre ses performances, en plus de la valeur de la fonction de perte. Nous définissons une fonction pour répondre à cet objectif :"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "48d9e14c",
"metadata": {},
"outputs": [],
"source": [
"import scipy.special as sp\n",
"\n",
"from sklearn.metrics import (\n",
" accuracy_score,\n",
" average_precision_score,\n",
" precision_recall_fscore_support,\n",
")\n",
"\n",
"\n",
"def compute_metrics(eval_pred: tuple) -> dict:\n",
" \"\"\"Compute evaluation metrics.\"\"\"\n",
" logits, y_true = eval_pred\n",
" probas = sp.softmax(logits, axis=1)[:, 1]\n",
"\n",
" threshold = 0.5\n",
" y_pred = (probas >= threshold).astype(int)\n",
"\n",
" precision, recall, f1, _ = precision_recall_fscore_support(\n",
" y_true,\n",
" y_pred,\n",
" average=\"binary\",\n",
" zero_division=0,\n",
" )\n",
"\n",
" auprc = average_precision_score(y_true, probas, average=\"weighted\")\n",
"\n",
" return {\n",
" \"precision\": precision,\n",
" \"recall\": recall,\n",
" \"f1\": f1,\n",
" \"accuracy\": accuracy_score(y_true, y_pred),\n",
" \"auprc\": auprc,\n",
" }\n"
]
},
{
"cell_type": "markdown",
"id": "0b96a2e4",
"metadata": {},
"source": [
"Pour paramétrer la classe `Trainer`, nous allons utiliser la classe [`TrainingArguments`](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments)."
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "69d6ac13",
"metadata": {},
"outputs": [],
"source": [
"from transformers import TrainingArguments\n",
"\n",
"training_args = TrainingArguments(\n",
" output_dir=\"./results\",\n",
" eval_strategy=\"epoch\",\n",
" save_strategy=\"epoch\",\n",
" per_device_train_batch_size=32,\n",
" per_device_eval_batch_size=32,\n",
" num_train_epochs=3,\n",
" learning_rate=1e-4,\n",
" weight_decay=0.0,\n",
" warmup_ratio=0.1,\n",
" lr_scheduler_type=\"linear\",\n",
" load_best_model_at_end=True,\n",
" metric_for_best_model=\"f1\",\n",
" greater_is_better=True,\n",
" logging_strategy=\"epoch\",\n",
" report_to=[],\n",
" save_total_limit=1,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "edf6abad",
"metadata": {},
"source": [
"Il est enfin temps de lancer l'entraînement !\n",
"\n",
"**Consigne** :\n",
"1. Initialiser le modèle à l'aide de la classe [`AutoModelForSequenceClassification`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForSequenceClassification), on valorisera le paramètre `num_labels` à 2 puisque nous réalisons une classification binaire.\n",
"2. Initialiser le `WeightedTrainer` avec :\n",
" * Le modèle définit à l'étape précédente\n",
" * Comme arguments (args) le paramètrage d'entraînement défini dans la cellule précédente\n",
" * Les datasets d'entraînement et de test\n",
" * La fonction de mesure de performance\n",
"3. Lancer l'entraînement avec la méthode `train` du `Trainer`"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "b730681f",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at distilroberta-base and are newly initialized: ['classifier.dense.bias', 'classifier.dense.weight', 'classifier.out_proj.bias', 'classifier.out_proj.weight']\n",
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
"/Users/arthurdanjou/Workspace/studies/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
" warnings.warn(warn_msg)\n"
]
},
{
"data": {
"text/html": [
"\n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/arthurdanjou/Workspace/studies/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
" warnings.warn(warn_msg)\n",
"/Users/arthurdanjou/Workspace/studies/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
" warnings.warn(warn_msg)\n"
]
},
{
"data": {
"text/plain": [
"TrainOutput(global_step=573, training_loss=0.40664318677226496, metrics={'train_runtime': 1411.5979, 'train_samples_per_second': 12.947, 'train_steps_per_second': 0.406, 'total_flos': 1210487088918528.0, 'train_loss': 0.40664318677226496, 'epoch': 3.0})"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from transformers import AutoModelForSequenceClassification\n",
"\n",
"classifier = AutoModelForSequenceClassification.from_pretrained(\n",
" MODEL_NAME,\n",
" num_labels=2,\n",
")\n",
"\n",
"trainer = WeightedTrainer(\n",
" model=classifier,\n",
" args=training_args,\n",
" train_dataset=ds_train,\n",
" eval_dataset=ds_val,\n",
" compute_metrics=compute_metrics,\n",
")\n",
"\n",
"trainer.train()"
]
},
{
"cell_type": "markdown",
"id": "9db1793d",
"metadata": {},
"source": [
"On aimerait visualiser la performance et la distribution des réponses du modèles. Pour cela, on produit deux graphes :\n",
"1. Graphique de distribution des probabilités du modèle sur le dataset de validation\n",
"2. Graphique de valeurs des principales métriques (précision, recall et f1-score) en fonction du seuil"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "fc202c3b",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"\n",
"sns.set_style(style=\"whitegrid\")\n",
"\n",
"\n",
"def metrics_vs_threshold(\n",
" y_true: np.ndarray,\n",
" y_proba: np.ndarray,\n",
" thresholds: np.ndarray,\n",
") -> tuple[np.ndarray, np.ndarray, np.ndarray]:\n",
" \"\"\"Compute precision, recall, and F1-score for various thresholds.\n",
"\n",
" Args:\n",
" y_true (np.ndarray): True binary labels.\n",
" y_proba (np.ndarray): Predicted probabilities for the positive class.\n",
" thresholds (np.ndarray): Array of thresholds to evaluate.\n",
"\n",
" Returns:\n",
" tuple[np.ndarray, np.ndarray, np.ndarray]: Precision, recall, and F1-score arrays.\n",
"\n",
" \"\"\"\n",
" precision, recall, f1 = [], [], []\n",
"\n",
" for t in thresholds:\n",
" y_pred = (y_proba >= t).astype(int)\n",
"\n",
" p, r, f, _ = precision_recall_fscore_support(\n",
" y_true,\n",
" y_pred,\n",
" average=\"binary\",\n",
" zero_division=0,\n",
" )\n",
"\n",
" precision.append(p)\n",
" recall.append(r)\n",
" f1.append(f)\n",
"\n",
" return np.array(precision), np.array(recall), np.array(f1)\n",
"\n",
"\n",
"def make_plot(y_true: np.ndarray, y_proba: np.ndarray) -> None:\n",
" \"\"\"Plot probability distribution and metrics vs. threshold.\n",
"\n",
" Args:\n",
" y_true (np.ndarray): True binary labels.\n",
" y_proba (np.ndarray): Predicted probabilities for the positive class.\n",
"\n",
" \"\"\"\n",
" thresholds = np.linspace(0.0, 1.0, 100)\n",
" dataframe = pd.DataFrame({\"target\": y_true, \"proba\": y_proba})\n",
"\n",
" precision, recall, f1 = metrics_vs_threshold(\n",
" y_true=y_true,\n",
" y_proba=y_proba,\n",
" thresholds=thresholds,\n",
" )\n",
"\n",
" precisions, recalls, f1_scores = [], [], []\n",
"\n",
" for threshold in thresholds:\n",
" y_pred = (y_proba >= threshold).astype(int)\n",
"\n",
" precision, recall, f1_score, _ = precision_recall_fscore_support(\n",
" y_true,\n",
" y_pred,\n",
" average=\"binary\",\n",
" zero_division=0,\n",
" )\n",
"\n",
" precisions.append(precision)\n",
" recalls.append(recall)\n",
" f1_scores.append(f1_score)\n",
"\n",
" precision, recall, f1 = np.array(precisions), np.array(recalls), np.array(f1_scores)\n",
"\n",
" plt.figure(figsize=(12, 5))\n",
"\n",
" plt.subplot(1, 2, 1)\n",
" sns.histplot(\n",
" data=dataframe,\n",
" x=\"proba\",\n",
" hue=\"target\",\n",
" common_norm=False,\n",
" stat=\"probability\",\n",
" )\n",
" plt.xlim(0, 1)\n",
" plt.title(\"Probability distribution\")\n",
"\n",
" plt.subplot(1, 2, 2)\n",
" plt.plot(thresholds, precision, label=\"Precision\")\n",
" plt.plot(thresholds, recall, label=\"Recall\")\n",
" plt.plot(thresholds, f1, label=\"F1-score\")\n",
" plt.xlabel(\"Decision Threshold\")\n",
" plt.ylabel(\"Score\")\n",
" plt.title(\"Threshold Optimization\")\n",
" plt.legend()\n",
"\n",
" plt.suptitle(\"Performance on validation dataset\")\n",
" plt.show()\n"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "b35f11f8",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/arthurdanjou/Workspace/studies/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
" warnings.warn(warn_msg)\n"
]
},
{
"data": {
"text/html": [],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA+4AAAHyCAYAAAB1b5R3AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAurFJREFUeJzs3Qd4U9X7B/BvRvegg72nzLKXiiJDBEXFjRPBgSK4FXEhKgjoz4ELRfiLgqgoLpaIgANBNhSQvWcptNDdZvyf96Q3JF0kbdqMfj88eTKa3NyctNz7nvOe9+isVqsVREREREREROST9N7eASIiIiIiIiIqHgN3IiIiIiIiIh/GwJ2IiIiIiIjIhzFwJyIiIiIiIvJhDNyJiIiIiIiIfBgDdyIiIiIiIiIfxsCdiIiIiIiIyIcxcCciIiIiIiLyYQzciYiIyK9YrVYEAm9/Dm+/PxERuY6BOxERFXL33XejefPmTpc2bdrgiiuuwLhx43D27FmPvM+JEydw5513IiEhARdffDGysrI8sl2qWEeOHFG/I/PmzVP35Vruy+OuvsZVc+fOxaRJk+z3XXmv8ibv//7777v1mt9//x2jR4+GN8jf3YMPPoijR4965f2JiMh9xlK8hoiIKoFWrVph7Nix9vt5eXnYtm0b3n77bfz333+YM2cOdDpdmd5j5syZ2LRpE958803UqFEDYWFhHthz8jbp4Pnmm29QvXp1j2/7448/RteuXSvkvcrT559/7rX3/ueff/DHH3947f2JiMh9DNyJiKhIkZGRaN++vdNjXbp0QUZGBqZMmYLNmzcX+rm7UlNTVcB19dVXl3FvyZfExcWpS6C9FxERkbcwVZ6IiNwiKfPi2LFj9seWLl2KG2+8UaW8X3rppXj99deRmZlp/7mkEV955ZX44IMP1Ghpjx490KlTJ5XmLNtxTDVOSkrCmDFj0LNnT7Rt2xY333yzSit2JM+Xbcl7ynPktmxL3n/dunW46aab1O2rrroKy5Ytw759+zBkyBC0a9dO7ceCBQuctrd27Vrcd999qmNCPl/v3r3V/lgsFqe07kWLFuHRRx9Fhw4d1Od48cUXnT6nzBmWkdQBAwao/ZL3mj59utNcYtm/u+66S+2LbEPSpc+cOXPBdl+4cKH6vPLe0sYvv/yy05QFrY1XrFiBa6+9Vn0O+fw//vhjsdv85Zdf1OfatWuX0+Pyfcrj27dvV/d37NiBkSNHonv37mjdujUuu+wy9R1nZ2cXud2i0teXLFmC6667TrXLDTfcoLZZ0IXeR74XSe/+4Ycf7Nsv6r1WrlyJO+64Q/2OdevWDU899RSOHz/utH+SUSKdT7fddpv6XenVq5f6ri5kzZo16jXy/Un7yuh1QbIvzz77rPo9l88h00DkfkpKin0qimxHLrLv//77r8vtLJ/t1ltvVb8H8vv68MMPY+/evYW+v+L+HuWzy9+X6NOnD5577rkLfmYiIvI+Bu5EROSW/fv3q+t69erZg79HHnkEjRs3xocffqgCj59//hkjRoxwClglQJf03HfeeUcFDl9++aUKzqtVq6ZSnW+55RYkJyerQF2C2yeeeEIFo3Xq1FHbl206mjp1qgpQZfRfAihhMplUkDZ48GCVUi2p908//TQeeughlVItr5ERfgmWZZ6vFizde++9iImJUfsmr+vcubPqDJBA3ZFMHZD9+eijj1Sg/91336nnayZPnqwuEmDKe8lneeutt/Dpp5/aOwjkvUJDQ/Huu+/i+eefV8HbPffcU2wQLOT9nnzySZXhIJ9X2uPXX39VAaDj606dOoVXX31VbU/es27duuqzFgzsNH379kV4eHihjoz58+ejWbNmKriVjhSpQyD1ByZOnIhp06bhmmuuUd/fF198AVdI54l0eEiQKr8j0rHxzDPPOD3HlfeR70R+X+T3prj0eOmoGDZsGGrVqqWmdcjv2saNG1Wwffr0afvzpFPm8ccfV9ke0lYdO3ZU391ff/1V7OeQqSKy7aioKPU9SDvL9+JI9l8elzaX3xfpDJD70sby+yXkcWlbucjnkCDdlc9/+PBh9XclnTLyezd+/Hj19yjz1bVOpgv9PcrfgQT7WnvK40RE5AesREREBdx1113WO++805qXl2e/JCcnWxcuXGjt2rWr9bbbbrNaLBZ1ufzyy6333Xef0+v/+ecf60UXXWRdvny5uj9lyhR1f+3atU7PGz16tLVXr172+5MnT7a2bt3aeuTIEafnDRkyxHrppZdazWazui/bksccff/99+rxr776yv7YggUL1GPvvvuu/bHExET12G+//abu//DDD9b777/fvm0htzt16mR96aWX1P3Dhw+r1zz99NNO73n33XdbBw4cqG6fPXvW2qpVK+v48eOdnvPaa6/Z20faTZ5vMpnsP9+3b5+1ZcuW1lmzZhX5XaSmplrbtGlj3xeNtKXsk/Y6rY2l7TVHjx5Vj02fPt1aHPkO+vbta7+fnp5ubdu2rfWTTz5R9//66y/1u5CWlub0Ovkcw4YNc2of+Q4cvwt5XNx4443WW265xen1sn3H17jyPkJ+X2SfNY7vJd+b/J44Pl8cPHhQ/V5NmjTJ6TXffvut/Tk5OTnWhIQE66uvvlpsW40aNUr9vufm5hb6HZP2F9u3b7fefvvt1kOHDjm9dvjw4darrrrK6W9MLhpXPv/8+fPVe504ccL+882bN1vffvtt9TpX/x4Lfj9EROT7OMediIiKJKPDMhLoSK/X45JLLlGjulKYTkYVZeR6+PDharRbIym8Mkde0nplhE/TsmXLEt9TRp8lBVhGtR1JirWMnErKe9OmTUvclrxeEx8fr64lrVkjI+vi3Llz6nrQoEHqkpOTo0YvDx48qIrvmc1mVZDPUcE5/TVr1rRX5pYie9IG/fr1c3qOpNMLGUmV1GwZqZeRT629JHOhSZMmqq1kxLUg2W5ubi4GDhzo9LhkBUg7SZs5vs5xH2X/hGM6f0HXX3+9Sj3fsmWLSmOXaQnyftLmQtK95SJtsWfPHtU+klov6f1aW5ZEMgJkpPqxxx5zelxG3f/3v//Z75f1fYR8f5J1IFkXjurXr69+L6StivtdCQ4OVnPlS2qr9evXq5T6oKAg+2PyfRsMBvt9+b386quv1Aj4gQMH1OeQzyO/u45/IwW58vnl9zgkJERlcvTv3x+XX365mgog35tw9++RiIj8BwN3IiIqkgTtsvSbkCBdAgZJP5YAwLG4nJDnac91JOm/jiIiIkp8T5mzraXgO6patapTsC0kxbsojvunKalavQSWr732Gn766ScV7Eh6uQR0RqOx0DrXBbcjHRnac7S2KK5Qmuy7BHOSAi2XgqR9i6LNY9fawJE8lpaWVuw+yv5daL1uCfykor+kcksAKNcy914L+mWfJeV89uzZKqiV3wF5XnH7W9T+y/vHxsY6PV4wzb2s7+P4HRTXVtqcfY1MWSju+yzusxT8HPJ7UvCx//u//1NTJWR/5H0ltV2+l4LflbufX343Z82apVL7ZZqGpNBHR0er+fyS9u/u3yMREfkPBu5ERFQkCbKluFVJJGgQUnjLcYkuTZUqVdx6T3m+jJgWpD1WMEDyBJknLPPFZc65ZBNoHQJSUMwdWlvICKnML3ac23/o0CEVvEkHiMxxl7nLrnYuaG0o8/8dt6u1S1EdHe6QYFVqBci8dqkFIKOyklGhkSBRCu5JICijyzK/W8iorytktFjeQ/bfkRZkeup9tPcSBd9La6uy/v7I9gtuWwJ9xyKBMsdc5qjLHH4pEKd15EjGQWJiYrHbdvXza8UYJStCMgBkjrx0ErRo0cKejeKpv0ciIvIdLE5HRESlJoGkpKNLFW0J8rWLjOBKGnTBEc4LkZReKSSmpZ9rpLiWFCVr0KCBhz+BLf1ZRp21Qm1i69atKgDXCn65QgIqSaFevny50+MzZsxQBcxk21KMTFKmHdtKisBJET6tsnhBkh4tadwSWDuSAn7SKSBF1cpK0uUlxVqKmUnat2O6v7SPBIRSqV8LJk+ePKnSuF1pHxkxlgwGqSrvOJotBescufo+WhZBURo1aqR+Twq2lRR1kykHZW0r6cz5888/1bQHjRSzc5xSIZ9DOnHuv/9+e9AuSyjK4yV9Dlc+vwT2kqovQbv8Tsj+SLaIkN8FV/8eS2pDIiLyTRxxJyKiUpMgT6q/y9JkcluCCkkJlyroEnQUnCN/IUOHDlVBuoxKSzVsGeGUKuGrV6/GhAkTyiXgkIBbqsfPmTNHzTWXKvNSsVtGxx0DtAuRIE2qh0twJUGVjHjKnHbZroyAyr5LAC8VwGUOtswhl3n0EtjL84qr7i1tIK+RoFo6BqSNJTB77733VKAnS6uV1UUXXWSfmy1zzx2nG0j7yPcpI8Iyf17mXn/yyScqeHS1feRzy3J88p1KdXeZiy6jxI5cfR8JiiUAlfnq2txujdbGUg9Ba2NZgk1GqGW0WX6/ykKqtctSa1KnQAJz6dyRTA3HOe+yT/Kdy6i7fFeSni6V5WWk3nHEWz6HdFKtWrVKdei48vllmThZpUD2Q5YUlL+5r7/+Wv2+yXu5+veoZYf89ttvap68/N4TEZFvY+BORERlIsu4SVr9Z599ptJ2ZWRZRjYlwHA3jVtGSyXokdFBWXtaRjIlBVgCD1lzujzIOtbyPhKASZAk84hluSwpECajwhJcu0rSo2XEU4IpaQ/Z1ksvvaSWpxNSfEyCOAkkZXk0CfgkmJI50QUL3zkaNWqUmist85uljSWYl+JkMq+5uLn+pRl1l2BTK0qnkUJnEvzKfGrpPJC51/Jc6diQwNKx7kBxpJCezOuXOdwSvEu7SEeMpOa78z4ScMpybPJaCZ6l3QqS9HT5fZTXSIArnRCyHroE9PL7VRYNGzZU34G0kwTI8l3LcntyXyMdKdKx8v3336uOEBntluXrZB66/C5IATkJlKWgoGR2PPDAA3jjjTdc+vzytyAdHvJz+TzyuylTMKTzR5tG4crfo2SYyLQQ+TuTjgNtuUIiIvJdOikt7+2dICIiIiIiIqKicZITERERERERkQ9j4E5ERERERETkwxi4ExEREREREfkwBu5EREREREREPoyBOxEREREREZEPY+BORERERERE5MMYuBMRERERERH5MAbuRERERERERD6MgTsRERERERGRD2PgTkREREREROTDGLgTERERERER+TAG7kREREREREQ+jIE7ERERERERkQ9j4E5ERERERETkwxi4ExEREREREfkwBu5EREREREREPoyBOxEVyWq1wlf5wr75wj4QERH5M387lvrb/lJgYeBOVMDdd9+N5s2bO13atGmDK664AuPGjcPZs2c98j69e/fGc889V+btzJs3T+3jkSNHin2O/EyeI88V77//vrrv+Jnlopk7dy4mTZqE8lDwc8t+yP64ytV9K/iZ3H2f4pw4cQIPPvggjh496vHvkoiIPE/+fy54XC940Y4XBY8dFc2VY7qrXDnuufp5c3Nz8eWXX+KWW25Bx44d1eWGG27AjBkzkJWVVar9++ijjzB9+nT7/YLnJqVVHt/huXPn8Oyzz2LdunXl+j5EJTGW+FOiSqpVq1YYO3as/X5eXh62bduGt99+G//99x/mzJkDnU4Hf1G9enV88803qF+/fpE/d/ys4uOPP0bXrl0rZN9kv2rWrOny813dt4KfyVP++ecf/PHHH06PffDBB4iMjCyX9yMiorIZMWIEBg8e7BQwbt++Xf3freH/4cVLS0vDAw88gB07duD222/Ho48+qs6BJIiVY/IPP/yAadOmuXUsF++99x5Gjhxpvy+dApdddlmZ97c8jv9y7vfTTz/hpptuKtf3ISoJA3eiIsgBvH379k6PdenSBRkZGZgyZQo2b95c6Oe+LDg4uMT9bdq0KbylvNqxIj+TdPQQEZFvkk5rx47ruLi4Cx4X6bwXXngBu3fvxtdff40WLVrYH+/Roweuv/56Fcw//fTTakS+LIMaEvi7G/x78/jvzXMnqpyYKk/kBkmZF8eOHVPXkiIlByvpfZYTgKFDh9p7p9944w307dsXCQkJGDhwIL777rtC25OR/Ndff111CnTu3BmjR4/GmTNnCqWG33jjjWr7bdu2VQfJRYsWFdrWhg0bMGjQILWP8n4LFy4sNlW+IMd0L0n7ljRw6UGX10imgXwGyTZwJKlxnTp1Ur3txZHeeWmTDh06oFevXvj5558vmMo3c+ZM9O/fX72n9Ly/8sorSE9PL3Lf5HPJZ5LAWdrp0ksvVaPxe/bsKTKFTbYj35fsz8UXX6za3jHFr6iUd8e0Rbk9ZswY9XifPn3szy34Ole+f3mNdAJJ2v8ll1yivtv77rsPBw4cKLY9iYioYuYxywiyTJGT/5tvu+02bNmyxf5zOWZdeeWVasRejjkSwGrT6ORYdM0119in2MlzzWaz/bVyjH/qqafU8UqOD3JM//HHHwvtgwwQSJaAPEe289lnnzn93NXzDEdy7iIj3HLslvf/v//7vwu2hQTsv/76K4YPH+4UtGsaNWqExx57DGvXrsXq1audjpvyGSSdXtrw2muvxeLFi+2v01LipQ2120VN43v55ZdVhoScD7Rr106N/CcnJ+P7779X34Ecz++9916nqQWOx39tX4q6OB63SzrX+vfff3HPPfeo23Jd3LSKnJwcfPjhh/ZzmH79+uHTTz+FxWJx2jfpCJHH5XuV58n37Pj7RVQcjrgTuWH//v3qul69evbH5D/26667TgWw8p9zdnY27rjjDpw+fVoF9HXq1MHSpUvVf9RysHnooYecXisHookTJ6qD+VtvvaWCzm+//RYGgwGzZ89WweWoUaPUgVZODORkQgs+HXum5eD28MMPo2XLliqwfeKJJ9SIghzU3SEHUZnDLcGwpBc2a9ZMbeOXX35R29R603/77TdkZmaqzoKinDx5EnfddRcaNmyIN998UwXN8vmkXYozf/589VzpwJCD6r59+1RgK8G1XBfcN5kCIOSkSObZjR8/HikpKWjSpEmR25fRgJ49e+Ldd99V3+U777yD48ePqwOtK+QgK20s37XjyYYjd77/L774Qn2vcvIl363sv3x2mT5ARETesX79ejWn+6WXXoLJZFLHaPm/X6ZJGY1GexAs9+U4kpqaiipVquCTTz5R9+XYJ528kl4twagcZyZMmKBe98wzz6jjg9TMkew+Sb+W//fleN69e3f7PkintRxDJCiWcwI5NsqxTTrB3TnOaOR4Lfsl+//aa69Br9erzuNDhw6p84ni/PXXX/bO5uJcffXV6vP8/vvvqlNcI8G+vKecO0inwuOPP67aSI7DcpyTDpGbb75ZpciXdF7QunVrdXyUGjOvvvqq2mZISIhqNzk/kPMfeVyC4aKO2wWPqdJhIe0lnQriQuda8v7ae8h1t27diuzskXbftGmT6hyRTg4J+OV84/Dhw6rNNdIRIt/liy++qF4n5zfy3suWLVPnfkTFYeBOVAT5j1QO1hr5T3zNmjUqYJP/xLWRdxEUFKQOWBIki6+++gq7du1SKWXawVB6imV70mssPasxMTHq8djYWFWYJTw83H7/kUcewZ9//qkOzvKfvYzCSpCqkQO09ArLiYX06mvkP315rrj88svVyK28n7uBuwTF8lkklVBLI5Q5XTKCLwch7cRCRghkpLhWrVpFbufzzz9XAbUcSGVbWs/8rbfeWux7SxvXrVsXd955pzqpkJEMaRttJKOofdPIAVMO0CWRA6UE6bJtOXGQTgg5mZLv66KLLrpg28j7aumW0kEi+1qQ9O67+v1HR0erx7QDtZxAyUmedD7I7wIREVU8Oc7IsUv7v1oKk0mQJR3r2qiz/J8ugaNky2kj4PL/uQSj8lwhI/GyDbkv2WfSES7HOTnOa8dmOc7Jc7RzCM2TTz6pUtCFHO+ks1xGtOXcwJ3jjEY69KWzQQJhLcVbBg5k1Lok2ki2nHsURzot5OJYtFUbXZbPqu2fBMpyDJbjr3YMlw6LkqYsyGeSjnLZvliyZInqTJDAWxtEkWBZOkCKO25r5yBC2lEC5+eff94egLtyrqW1mVwXlSIv521SA0eyE7VzM8lqCA0NVXP5ZaRevn/tM8m5n1ZXQaZhyu+SdPQ4nl8SFcRUeaIiSMqX9LBqFwlQ5SAq/6H+73//c5rD1bhxY6cDrhyU5T/8gj3YMiovaVSSOqaRg5cWtGs92tIbLu8vJI1LenzlpEE7MEnPsJDRgII93o7kpECK78gBoazk89euXdt+YJRe71WrVtl7q4siBzs5GDseMOUkQbZTHOkUkJFwOVjKgToxMVGl17lStVUC6QuR9DUJ2jWSxia09vYEd75/SZFz7F3XMihKW6GXiIjKTgIzx8BX66SV4Ly4487GjRvVSLgcxyUw0y7aSPXKlSvVtQSL0kErI+WSni0j5BK0SZV2R1qHgAgLC0PVqlXVuYC7xxmNFJKTjmfHoFM63i80z19b/kzLNCiO/LzgUmmO5why3iSdBJISLu3kKulw14J2Ie0gHduOmY/yXRX8boqbvieV4SVTUEt9d/dcqzjynUgbyHlGwe9E+7lGvgPHYog1atRQ1zz204VwxJ2oCBKsyyi6drCRlCw5wBVVdTYiIsLpvowOV6tWrdDz5GAjtAOvKPg8CSrlgKQ9R0ZgJS1LgmQZ2ZdOAq23v+ABUtu+Jj4+Xj1Hmx9eFrJfEkxLeplUUZWDmrRFST310g5FjUgX1TaOnQ8y3UCyFmTUQE5u5OREDqgFOyYKcuwAKU7B95Y2KvidlJU737+cjDnSOhUc58MREVHFKng8Ke7/Zsfjv6TLC5nOVZSkpCR1Lan0U6dOVVPlZORXti2d45KG7TiqXdTxQTvuu3Oc0chrisrkku1I50FxtH2S0XSZ+lYUOc+Q6X4FR+W16WwFz0tk/2Qk2hVFnXe5crwvSKYVyHQHOY/Szu807pxrFUdr34Kp7tr35NixwGM/lRYDd6IiyMFYRkNLQ3qGDx48WOjxU6dOqWvHA6d2oNdIarmkScvBTf4DlxMAOYjI3DDp2ZfeXEnVKyolTA4ajsG7HIjlACL7U9JB2VUSuEuKm6SDyQmHBNLSoVEc+ZxFvW/Bz1yQFNiRixzk/v77bzXPTOYEyrwzrVe6tAq+t/adaAG8cCwipM0LLK/vn4iIAoNMfRJSy6WoAFc7PkdFRaljmlykjovMC5eOagkmi5qj7anjjDxW1GsudEyWjAGZXy+F5YqaOy8k/VzOWaRoa8FtF3VeUjCNv7zJqLmk7GvF4xzPXdw91yrpO5HzNzmHcAzetQ4bHvvJE5gqT+RhUiFeeqYlbc6RVFSXA4NUK9VI6pzjXHrpfZf7kkonBwBJG5fCLdKJoKWpSeBcVM/sihUr7LflZ3KQldR0V3u1HTmmk2ukJ12KzkhBNZmHJYF8SSTtXdpAitRp5EAoc8mKI4VrtPlwcnIzYMAANedM2kQ7+BW1b67S2k6zYMEClVGhrQsvPfsyDaBgyr+jC72/O98/EREFBjneyv/xcsyTY7Z2kWO3zHuWueJybJApclp1dRnZlSrpMuKurVbjitIcZ+SYLPsgU9A0MkouqeElkX2UznQpKrd169ZCP5djunRWSNq+Y3E9IfPQNTJyLfPTpRNem15YluO5OyRTUPZdivEVXG7O1XOtCxWNk/MIOVdxrJwvtNV05HMTlRVH3Ik8TAJaSfWWAFTmsEm6uFQKlaVLpNKo1iuv9Y5LUTmZwy3F5OTgLsVMJECWgFKCZZlnJQcaeZ0UZJHAuai5UFK5VHp6JaV/zpw56kDkylIvRZH3kvnxMidLTgC04F8ObDLXX+acyUlKSYYMGaJ6r6Xgi3xG2TdJEZSTiuLIQV8OsFJhVQrsSTqdzHWX0Qstba3gvrlDTlik6q6chMhtOYjLZ9JGR6Toj5ycyEU+n3xv2vI2jm2jjTDIPhasYO/O909ERIFBRlTvv/9+VYhMUselA16CeLkvx3M5hkmHtBzPpYK5PEfmnEtAKdXppQK7q0pznJHlzeT8QX4uVd6lo1pbDedCpMK9dJ5LNXepZi/nKRJ0S8eBLOEqo+py/lIwEJ88ebIa5ZbCtDKff+/ever5GtlPWcpW6sw4zun3JCmUK8X8hg0bplLUHTsqpANBit66cq4l3502SCKj6wWXxpPzAfnOpRChfO/yczlPkaxBmevPNd/JExi4E3mYHBhk2TEpYqcdwKXHWpYykSDRkRwAJSVcDr5yAJFCbJI+pxW/k/Q5eZ0UTpGfy3/8cqCVSuhSaMaxaJssKSZL1kgqnFRIl4OFNpLsLjnAyXtI0C3Bv3ZA1SqxX2i0XTuJkQ4Ebf9l+oGc1DiuL1+QVMKVte2lUq6clEiHgXRiSJtoAX/BfXOHtLOcJEm6nxyEZX/kJEYjJ04yAiHVXmU/pEq97L/Mi9PIgVlGR+T7lflwBVMb3fn+iYgocEjWmMxpluOXrLsuAZ4cw6TDWwv8pDNaglw5Pshor3S2y3GouLnxRSnNcUbOISRoluOnPE+O5bLKixR5K2mZViH7Lsdb6RiQ6vQShEtnvHR6S8aArARTcN62FvBLR7iMykuALMu2OgbociyW8xzZRknnBmUhUxGEvLdcHEnALh0erpxrSUV46fSXAF8Ce6nO70jaUz6rDAhIZ4GcS0iHinz3sqIAkSforK5WXSCiSk8OrFKRVUYHHOeFExEREQkZ4ZZ17CVoLqpILRGVDkfcieiCZJ6apJbLSLiMtjNoJyIiIiKqOCxOR0QXJAVtJMVO1rGXtHUiIiIiIqo4TJUnIiIiIiIi8mEccSciIiIiIiLyYQzciYiIiIiIiHwYA3ciIiIiIiIiH8aq8gAsFgtMJhP0er19/WwiIiJvkhI0cnwyGo3q+ERlx+M9ERH567GegTugDuKy1BUREZGvSUhIQHBwsLd3IyDweE9ERP56rGfgLvMF8ns3WrVqxZMjDzGbzerkSH4JDQaDt3fH77E9PY9t6nlsU8/Kzc3F9u3bOdruQVpbeuJ3lL/v7mObuY9t5h62l/vYZt5tM21brhzrGbjLmnj56XLS8PyF9Sy2qWexPT2Pbep5bFPP0NqQKd2+fbzn77v72GbuY5u5h+3lPraZd9vMlWM9u/GJiIiIiIiIfBgDdyIiIiIiIiIfxsCdiIiIiIiIyIdxjjsREfn0MilSCVyKtwQamRcny79wDjsRERFdCAN3IiLy2arqx48fR2ZmJgJVeHg4atWqxRVNiIiIqEQM3ImIyOdYLBbs379fjUrXrl1bBbaBNDItmQTSMXHq1Cn1OZs1a8Zl34iIiKhYDNyJiMjnSFArwXu9evXUqHQgCgsLQ1BQEA4ePKg+b2hoqLd3iYiIiHwUu/eJiMhnBfoodKB/PiIiIvIMnjEQERERERER+TAG7kREFDD+++8/bNiwwSvvvWjRIpw+fdor7x0IZLrAwIED8e+//xb7nO3bt+OWW25Bu3btcNNNN2Hr1q0Vuo9ERETewsCdiIgCxiOPPIIDBw5U+PsePXoUjz/+OLKysir8vQNBTk4OnnzySezevbvY58jqAg8++CA6d+6MefPmoUOHDhg+fHhArzpARESkYeBORETkgSrxVDp79uzBrbfeikOHDpX4vIULFyIkJATPPvssmjRpghdeeAERERFYvHhxhe0rERGRtzBwJyKigHD33Xerke8xY8bgueeew++//45BgwYhISFBjdLKiG5GRoZ67vvvv48RI0bgzjvvRNeuXbFmzRpkZ2erYLBTp0647LLLMHfuXLRq1QpHjhxRr5E15R966CGVpt27d2988MEHMJvN6md9+vSxX8toMLlO2r5bt2745ptvSnze5s2b1XejLQso1x07dsSmTZsqaE+JiIi8h8vBERFRQJBg/Prrr8ewYcNUIHjzzTfj5ZdfxiWXXKLS559++ml8++23GDp0qHq+BPavvPIK2rdvj0aNGuH111/Hxo0bMX36dJhMJhXEa4G5jKiPHDkSLVq0wA8//KDWX5dtS/Ao6fkS5Mvca7m+6KKLvNwS/uWOO+5w6XnS5k2bNnV6LD4+vsT0+uJo32upWS3Ajw+j5cH10K0OhVX6EnR6WLsOh7X9nWXbdgDT2r3M7V+JsM3cw/ZyH9vMu23mzjYYuBMRUUCIiYmBwWBAVFSUWhP9xRdfVCnYom7duiqAdwzyqlatittvv13dlpH4H3/8EdOmTVOBvJDX33///er26tWrcezYMRWYyxJujRs3xujRo9XovgTucXFx6nlyzfXYy4fUDwgODnZ6TO5LUTt3JSYmlmlf9HkZaLf9J4RbcoG084/nLJuEbWhdpm1XBmVt/8qIbeYetpf72Ga+32YM3Eswa/VB3NW9gbd3g4iI3NSwYUMV1H388ccqWJeLzKWWEXlNnTp17Lf37duHvLw8lVavkeJnmr179yI1NVWlamssFotKr09JSamQz1TZyfz2gkG63C9NR4l8z9LJUxamJv/iwPqlKlvDkHUG+h8fREhOMtq3ayd5/GXadqCSkSU50fVE+1cWbDP3sL3cxzbzbptp23IFA/cS5Jos3t4FIiIqhR07dqjRdJmLLvPb7733XsycObNQIKgxGo0lFpyT1HkZZf/oo48KPU9G+LW581R+atSogeTkZKfH5H716tXd3pacaJX5BDWuAdKqdYK+aXvorSbgxwehM2XDkJMKRFQt27YDnEfav5Jhm7mH7eU+tpnvtxmL0xERUcD56aef0KVLF/zvf/9Tc6jbtm2LgwcPFlv9vX79+ggKCnJaF9zxtoyqSqq8pMI3aNBAXaRo3ZQpU9Q8d61gGpUfKQooNQi071CuN2zYoB73OmMIEFnDdvusrZghERGRJzFwJyKigBEeHq7S3qOjo7Fz505s2bIF+/fvx8SJE1UqWnHzoWVZsRtvvBHjx49X1culUrncFhKU9+jRQ6XWP/PMM2q769atw0svvYSwsDDV2y7X2kg/R989RwrSyXQE0b9/f5w7d059LzLtQa5l3vuAAQPgE6Lzp14wcCcionLAwJ2IiAKGpMfPnj1bjZZLkTlJkZcRdxktlyJy27dvL/a1UmyuefPm6jWjRo3CwIED1eMyEi/BucyXl3ntUvBOft6zZ09VwE7ISPx1112Hxx9/XBWwI8+QDhNZv11ERkbik08+wfr161Uni3SwfPrpp6qzxidUyQ/czx319p4QEVEA4hx3IiIKGLIuu1yK8+ijj6prCbwLWrVqFcaNG4e33npL3ZfRelmrPTY2Vt2vV6+eChSL8+abb6oLlZ5kM5R0X6Y8yHJ8PqlKPds1R9yJiKgcMHAnIiICVJC+fPlyPPjggyrdXYJwKW4nI+5EF8RUeSIiKkdMlSciIgLUSLsUnBs0aBCGDh2q1n7X5rkTXRBT5YmIqBxxxJ2IiAhA06ZNCy0Z5xVrZwBdhnl7L6jUqfIM3ImIyPMYuBMREfkSU46394DKkiqfdgwwmwADT7GIiPzRyXPZ+G79EYQFGVA9OgTVo0LRsGq4uvYmHlWIiIiIykrWcdcHAZY8IP0EUKWut/eIiIhK4d2luzFnzSGnx4x6HX585FK0qVMF3sI57kRERERlpdcD0bVst5kuT0Tkt3afTFPXHevHoEvDWIQHG2CyWLEnKd2r+8XAnYiIiMgTovNH2c8e9vaeEBFRKR04namuX72+DeY+dAk6NbAtC2u2WOFNDNyJiIiIPEFLj2dleSIiv5SeY0Jyuq3WTP34cHVt0OvUtdlaiQP3nJwcPP/88+jcuTN69OiBGTNmFPvcn3/+GVdddRXatm2LwYMHY8uWLU4/nz9/Pvr27Yt27drhkUcewZkzZyrgExAREREVWBKOqfJERH7p4OkMdR0fEYzo0CB126CzBe6WyjziPnnyZGzdulUtvzN27Fh88MEHWLx4caHnrVu3Di+88AJGjBiBBQsWoEOHDnjggQeQkWFrWAni5ecjR47EN998g3PnzmHMmDFe+ERERETudUxTAFaWP3vE23tCRESlcCDZlibfIH+0Xeh9ZMTda1XlMzMzMXfuXEybNg2tW7dWl927d2P27Nno37+/03NPnTqlgvbrr79e3ZcRdTkJ2rt3rxqBnzVrFgYMGIBBgwbZOwR69eqFw4cPo169/HVViYgoIFisVujze7999f0cO6aPHTuG0aNHo3bt2oWObxSga7mfY+BOROSPDuSPuDeMj7A/Zh9x927c7r3AfceOHTCZTGr0XNOpUydMnToVFosFeqnOmk+Cck12djY+//xzxMfHo0mTJuqxzZs3qxF4Ta1atdQJkjzOwJ2IKLBIEL0o8TjOZOSW+3vFRQRjQEJ+pfBy6JimAMNUeSKigEiVb+AYuOt9I1Xea4G7jKLHxsYiODjY/ljVqlVVemFqairi4uIKvWbVqlUYNmwYrFYr3nrrLURE2Bo0KSkJ1atXd3quBPYnTpxwa5/MZrO6aKxWi9N9cp3Wbmw/z2B7eh7b1LfbVLYh/9drF0c6nQ5nMnKQlGYrHlO+bO9dcB9K8t9//6mO6fbt29tf17FjR9UxLZ/LsWNa+3yOxx+d1QJrgeMR+VmqfGYykJcFBIV5e4+IiKgUFeUbVi0iVb6yBu5ZWVlOQbvQ7ufmFj2K0qxZM8ybNw/Lly/Hc889h7p166oTIxmFL2pbxW2nONu3b7ffNhgMOHUqG4mJaTx5KoPExERv70JAYXt6HtvUd9vUaDSqY4VkYWkk6A0LC1OBcV5eHsqbyWRQ13KccdyPkhw9ehQxMTFqH+UipKNZOqaPHz+uOq018ph8DslC0449tU6dwvHERB57/FFYLBAUAeRlAOeOAfG2zEAiIvLjEXfd+alzlTJwDwkJKRRYa/dDQ0OLfI2MyMulZcuWKg3+66+/VoF7cduSkzt3tGrVyqkDYH3aASQkNHRrG2QjJ5xy8p6QkKBORKls2J6exzb17TaVQPngwYPq//GijgkS1AcF2aq9lid5n5KOS0WRAF+OS+Hh53vrq1Spoq6lXRwfl44I+RxNmza1v4cucyWqJiSo45hjhzL5AZkHKenyybtsBeoYuBMR+Y3MXBNOnrNl8zV0LE6nq+Qj7jVq1EBKSooajdBOjCR9Xk5coqOjnZ4rVePlZEfmCWpkfrsUp9O2lZyc7PQauV+tWjW39knew/FkU6fT84S+jAq2KZUN29Pz2Ka+2abyekmJ1y6FyWMVUaDO9h5F70PRtM5kx9do2QHSEeH4uPb5nNpMp5cG4O+lP6fLS+DOtdyJiPzKoTO2NPkqYUGICQ/2uaryXlsOTkbNJWDftGmT/bH169erkRrH+X/iu+++w9tvv+302LZt29C4cWN1W9Zul9dqJBVRLvI4ERGRtzqmNcV1TFMAqlLXds0l4YiI/HIpuIYOo+2i0q/jLqMOsnzbK6+8okbUly5dqpZ4u+eee+wnOZIqKW677TasXr1aLatz4MABTJkyRb3m3nvvVT+//fbb8dNPP6kqvjJP8Nlnn8UVV1zBivJEROTTHdMUgBi4ExEFzPx25+J08CqvnkGMGTNGpb8PGTIE48aNw6hRo9CvXz/1sx49emDhwoXqtjzngw8+UCPv1113Hf744w9Mnz5djWoIWVLu1VdfxYcffqiCeJlL+MYbb3jzoxERUSV1oY5pqiSV5ZkqT0TknxXl4wuMuOdHzN5OlffaHHft5GbSpEnqUtDOnTud7vfq1UtdinPjjTeqCxERBT5ZX92X30c6piVwl47pyMhIp45pCnAccSciCqgRd4OPpMp7NXAnIiJylyzHMiChVoW+n1ZR1hMd01RZAneOuBMR+ZODRazhLip9cToiIqLScDeI9rf3owBJlc9NA7LPentviIjIBdl5Zhw7m+XTI+4M3ImIiIg8JTgcCIuz3Wa6PBGRXziSkgkZUI8MMSK+wDQ5g704HQN3IiIiosBRJX/UnenyRER+tRRcg/hw6Apk2jFVnoiIiCgQRefPcz/HEXciIn9wIL8wXcMCafKCqfJEREREgYiV5YmI/LIwXYMCS8EJjrgTERERBSKmyhMRBdyIu9kCr2LgTkRERORJsY1s18m7vL0nRERUxhF3Q37EbOWIOxEREVEAqZlgu07aDphN3t4bIiIqQa7JoqrKi4ZVI4pPleccdyIiIqIAG3EPjgJM2Rx1JyLycUdTsyAxeWiQHtWjQopPleeIOxERUeDJzc3FwIED8e+//3p7V6ii6fXnR91PbPH23hARkYvz2wsuBSf0rCpPRERUClaLz79fTk4OnnzySezevbtcdon8QK22tuvjDNyJiHzZgeSMYue3O1eVh1cZvfv2REREbtLpge0/AxnJ5f9eEVWBVte59ZI9e/bgqaee8noRG/KymvmBO0fciYh82l+7becTLWpGF/lzQ/4gvLdH3Bm4ExGR/5GgPf0kfNGaNWvQrVs3PPHEE2jfvr23d4e8PeIugbt04hSRfklERN6VkpGLP3edUrevbVeryOcYfKQ4HQN3IiIiD7rjjju8vQvkC6o2B/RBQPZZIPUgENvQ23tEREQFLNx6HCaLFS1rRaNp9SiUnCrPOe5EREREgcUYDFRvabvNee5ERD7p503H1PX17WsX+xytqry3U+UZuBMRERGVd7o8ERH5lONns7DmwBl1+9p2xQfuHHEnIiIiCmQ129muOeJORORz5m8+rkqQdGkYizoxYcU+z76OO0fcfVewQY/pf+1TFyIiIiK3cMSdiMhn/bzZliZ/XQmj7Y7F6SwccfdtOSaLuhARERG5pUYbWb8QSDsOpNuqFhMRkfftO5WOxKNnVVB+dULR1eQLpcqzqjwREVEp1lcPpPehwBQSCcQ3AU7vsY26N+3j7T0iIiKcH23v0bQq4iNDSnzu+eJ08CoG7kRE5F+sFqDVdRX7frrSJajt3LnT47tDfqZmAgN3IiIfYrVa7YF7SdXkNYb8UwAWpyMiInJHKYNov3k/Ciw18+e5s0AdEZFP2HbsHPadykCIUY9+rWte8Pl6FqcjIiIiCnAsUEdE5FPmrDmkrvu2qoHIkAsnoGvF6WSk3psYuBMRERGV95Jwp/cCOene3hsiokrtXHYefth4VN2+q1sDl17DddyJiIiIAl1kNSBKKhZbgZNbvb03RESV2vfrjyAz14yLakSie+M4l15zfh13eBUDdyIiIqLyxHnuREReZ7FY8eWqg+r23Rc3hC4/IHd1jru83psYuBMRkc/y9nyy8hbon4/y1elou97yjffXEyIiqqRW7k3GvuQMRIUYcWOHOi6/Ts+q8kREREULCgpS15mZmQhk2ufTPi8FqE73AsGRwNF1QOJcb+8NEVGlNPMf22j7TZ3qIsKFonSF13H3buDOddyJiMjnGAwGxMTEICkpSd0PDw93OaXNX0baJWiXzyefUz4vBbComsBlTwK/vwosfQVoORAIjvD2XhERVRqHz2Ri2Y6T6vZd3V0rSlewqry3R9wZuBMRkU+qWdO2tqoWvAciCdq1z0kBrvsjwPrPgdRDwMr3gF7Pe3uPiIgqjdn/HoIMmF/WrCqaVo9067X2qvIccSciIipMRthr1aqF6tWrIy8vD4FG0uM50l6JBIUCV74GzB1iC9w73A3E1PP2XhERBbysXDO+WWtbu/1uN0fbBVPliYiIXCDBLQNcCgitrgcaXAocXGlLmb95urf3iIgooOWYzBgxez1SMvNQJyYMfVrWcHsbvpIqz+J0RERERBVBRm36vyE3gK3fAWs/Ayxmb+8VEVHgBu2zNmD5zlMIDdLjf7e2swfh7tCWg+M67kRERESVRa12QOdhttsLngKm9QIOrfb2XhERBZRckwWPzN6I33ckIcSox4whXdC9cXyptqUF+xYWpyMiIiKqRAZMAuKbACsmAsc3AzOuAi4aYKs+r6naDOj6IGDgUoFERO6QueiPztmIpf+dVEH79CFdcEnTqigtg7aOO+e4ExEREVUiEoxf/AiQcCuw7FVgw5fArkWFn3dqJ3Dte7YUeyIicsmctYeweNsJBBv1mHZPZ/RoVvqg3TFVnsXpiIiIiCqjyGrAde8DXe4Hdv8GWPMnUOacA1Z9CGyYaRuZv/Qxb+8pEZFfSE7PwaRFO9Tt5/q3wOUXVSvzNn2lOB0DdyIiIvKanJwcjBs3DkuWLEFoaCiGDRumLkX57bff8Pbbb+PEiRNo0aIFXnzxRbRu3RoBMe9dLo6i6wKLRwO/vQzENABaD/LW3hER+Y0JC/7DuWwTWtWKxj0Xu7/0W8nF6VhVnoiIiCqpyZMnY+vWrZg5cybGjh2LDz74AIsXLy70vN27d+Opp57C8OHD8dNPP6Fly5bqdlZWFgJS94eArsNtt38YDhxZ5+09IiLyaav2nsa8jUfV7KLxN7SBUZuc7qERdy8PuDNwJyIiIu/IzMzE3Llz8cILL6iR8yuvvBL3338/Zs+eXei5K1euRNOmTTFo0CDUr18fTz75JE6dOoU9e/YgYMnScc2uAkzZwJzbgexz3t4jIiKfrSL/0k9b1e3bu9ZHh/qxHtu2r6TKM3AnIiIir9ixYwdMJhM6dOhgf6xTp07YvHkzLBbnBXNjYmJUkL5+/Xr1s3nz5iEyMlIF8QFLbwBungHENQYykoDNc7y9R0REPmnaX/uwJykd8RHBGH1VC49u21dS5TnHnYiIiLxCRsxjY2MRHBxsf6xq1apq3ntqairi4uLsj1999dVYtmwZ7rjjDhgMBuj1enzyySeoUqWK2+9rNpvLvO/aNjyxrRIZw6Dr9jD0i56B9d9PYOk0DND557hLhbVZAGGbuYftVTnbbOWeZLy3dJe6PWZAc0SG6D36eaxa4VAAeXkm+31PHktcwcCdiIiIvELmpzsG7UK7n5ub6/R4SkqKCvRffvlltGvXDnPmzMGYMWPwww8/ID4+3q33TUxM9MDee35bxdGjFdoaI2A4sxf7lkzDuRrd4M8qos0CDdvMPWyvytNm/yXn4rU/U5BrtqJbnRA0RBI2bTrl0fdIyz0fuG/YtAnG/NT5im4zBu5ERETkFSEhIYUCdO2+VJh39NZbb+Giiy7CnXfeqe6/9tprGDBgAL7//ns8+OCDbr1vQkKCGrUv6yiJnLR5Yluu0J25B/j3YzQ9vRSWq/KL1vmZim6zQMA2cw/bq3K12ZYjZzHx57XIMVtxWbOq+OSujggxej4jKS07D/jpd3U7IaEt5C081WZa+7uCgTsRERF5RY0aNdRIusxzNxptpyQyqi5Be3R0tNNzt23bhrvvvtt+X1LlZUm4Y8eOuf2+cqLlqRNUT26rRN0eBP6dCt3e32FI2QdUbQZ/VWFtFkDYZu5hewVem51Oz8F/x9NghW2eeXq2CWN+SER6jgldG8Xh07s7Iyy4fPY/yOgwt12vh8Gg80qbGf1l7dYVK1bgnXfewaFDh1C3bl08/vjj6NOnj/3nnTt3RlpamtNrNmzYgIiIiHL/HEREROQ+WdJNAvZNmzap47iQ4nMyiiGBuaPq1atj7969To/t379fPbdSkAJ1F/UHdi0C1nwKXP2mt/eIiKjcK8Uv25GE7zccwfIdSTAVURyufb0YzLi3S7kF7Y5V5e0F6vID94pm9JW1W6XHfPTo0ahduzb69+9fqOrsyJEj8eyzz6Jnz574+++/8dhjj+G7775Tve0nT55UQfvSpUudUuvCw8O98KmIiIjIFWFhYWp5t1deeQUTJkxAUlISZsyYgTfeeMM++h4VFaWO7bfeeiuee+45tGnTRlWhl2Xk5NzhhhtuQKXRbbgtcN/0FdD7RSDU/cJ8RETlzWq14u89yZj2134kncsu9XZOnMtGamae/X7D+HCEBp0P0JvViMLr17dBZEj5hrRaVXlRYMGTyhG4a2u3Tps2Ta3dKpfdu3ertVsLBu7z589H9+7dcc8996j7DRo0UJVlFy1apAJ36YGvVq0a6tWr56VPQ0RERKUhBeYkcB8yZIha3m3UqFHo16+f+lmPHj1UEH/jjTeqqvIZGRmqkvyJEyfUaL10/LtbmM6vNb4CqNYCOLXDFrx3f9jbe0RE5BSwr9xzGu8s3YX1B1M8ss3qUSG4oUMd3NSpLi6qEQVvcBpx9+Ja7kZfW7t16tSpan1WxxQ56U3Pyzvf26LRUuNlXddGjRpV0J4TERGRJ0fdJ02apC4F7dy50+n+Lbfcoi6Vloz6yKj7/CeAv94GYuoDza+2PU5E5EFHU7Pw8Yo9+Gt3MiwuBqt5JqsaJRdSJO7Obg3Qq0U16FC6/6Mk/b1d3SowGry7BKZD3O7VtdyN/rB2a5MmTZxeKyPzq1atwuDBg9V9GXGXJWWkaI3Md5Ne+Oeff97tYF6q+jmupWeFxaPr9FUmgbAmpC9he3oe29Tz2KaeobNaYC1wPCKya3sbsOpD4PQe4Os7gMa9gP4TgeotvL1nRORl2Xlm/Lz5GI6lZjk9Hh8ZotLMG8ZHoHZMmNMIckGHz2TioxV78d36w8gzux+kBquAvT4e7tkE1aOdVwfxVzqdTgXvErO72okRUIG7O2u3Ojpz5oxKo+vYsaO9ON2+fftw9uxZPPnkkyrNTtLv7733XixYsEDdd9X27dvtt6VCYHJyHs5m5qngPTExkydRlWhNSF/F9vQ8tqnnsU1LT449tU6dwvHERB5zqGjBEcCDK4C//mcL4PctBz6+BKjVVk4vbc/RG4GuDwJtK3F2AlElkplrwuzVh/DpX/twKi2nxOcGGXQIMToWcrPCbLbA8PNv6v8Q2ZY2qHxJk3jcf1kjxIY7x2wlaRAfgbgI15/vLwx6HSxma+UccXdn7VZNcnIyhg4dquZPTJkyxZ5OP336dJVKr1WQl7VepYjd8uXLce2117q8T61atXLqTEjMOoTgLFuKfkKC86g/Be6akL6I7el5bFPPY5t6hi5zJaomJKhjomOHMpFdSBTQ9xWgw93AkpeAnQuAYxudn3NkLRAcDrS4xlt7SVQpnTibja/+PYiT50oOoD1F5lxL5fUzGbY4qk5MGHo2r2ZP75Y4UwrEHTidiUOnM5FrtiDPbCq8IdP5zmJZE/3RPs3QpeH5DOjKTq+mJFXSwN2dtVuFVI7XitN98cUXTqn0Emw7BtzSKSBLxslr3FFwLT4d9NDpbJ0DPAkNzDUh/Q3b0/PYpp7HNi0jOe6wDckV8U2A278Cjm8Gzh0///j2H4HNc4Dv7weGLgRqn68nROQvjqRkYvuxc/mrdrtG6mTtP5qNpOCThZaU9DSpZN4gPhy1q4RBr9fh4OkMTP1DUsyPlCrFvKxkXx65oikGdaij0tWLIkHnyXPZyDOfL40uo+3//bcdLVu2gsGgV1XbawRIintxtiVvw4Q1E9AwuiEGNh6IrjW7wqAv+ZirTS+olKny7qzdKhXo77//fvW4BO1SQV4jo+9XXnklRowYoarOas8/ePAgGjduXMGfioiIiKiC1Wpnu2ia9gHSTwJ7lwFfDQbuXwrEcOUd8s052Y5rc0tQ+eu2E1iUeAKJR8+WfsP/FMhAKUcSJNeNDcOB5Ax7innXRnHoedH5eKW8NaoagX6talywiJsEnzLHvWC2XEqkrROiMnQYb0rahIeXPoz0vHRsObUFP+/9GdXCquHqRlfjvoT7EBsaW+TrDPlFQCvliLs7a7fK0i+HDh3Cl19+af+ZkJ/Jc6644gq8//77qFOnjhqJf++991CzZk2VLk9ERERUqRiCgFs+B2b0B5K2A3MGA0MXAaGFMxqJvGHdgTOY+sc+/L7jJIobwJQBzla1owvMxy6ZDOjJspEyfVYKipWn1MxcHDqTiVyTBftOZajHJFgf2bspU8x91LoT6zDi9xHIMmWhY/WOaBrTFIsPLMaprFOYuX0mMk2ZePnil4t8rWRVCC/G7d4L3N1Zu/XXX39FdnZ2oSVgZJm4iRMn4plnnlGj90899RTS09PVmu+ffvpppeg1IiIiIioktApwx7fAZ32Ak1uBqZcC/V4HWl7H5eOo1KRw2abDqaV+/en0XHz+z4Fi1/iWEWEpiDagTS30a10DVSND3Nq+jB5LNm/79u0rJA6Q0Vep4H7gdIZKL/fWOuN04Q6dVcdX4bFljyHbnI1utbphSq8pCA8Kx3Ndn8O3u77FxDUT8eeRP9Vzi+r0qdSp8u6s3bp48eIStyNz2p977jl1ISIiIiLY0uMleJdl41IPAd/eAzS8DBgwCajR2tt7R35GqpVf98HfOH7Wtk53WQQb9LipUx3c16MR6saGOwVHQV5es9sdsr/14sLVhXxHclYy3tvwHnac2YEzWWdwJucMTBZbQb5L61yKd694F6FG2zz+IEMQbmp2E95Z/w5OZp7EntQ9aBbbrJjidJU0VZ6IiIiIylnt9sDItcDf7wL/TAEO/AVM7QF0vg/o9TwQzpReujCLxYonv92kgvaY8CBUj3JvJFxj0OtVOvmwSxsGzBrf5FtWHF6Bsf+MxZnsM4V+JvPYX7v0NQQbnJerkyC+S80u+Pvo3+pSdOBuu2bgTkRERETlt/Z77xeADncBS14E/vsZWDsN2Pod0OsFoNNQwMBTQirex3/sxV+7kxEapMfc4RejGVPCycdk5mXirXVvYe6uuer+RbEXYWT7kagRUQNxoXGq6FyIofgOpx51etgD96Fthvpkqrz/5KIQERERUenFNgBu+xIY8gtQvRWQlQIsfBr45HJg/5/e3jvy4UJyb/+2S91+9bo2DNrJ5wL22f/NxqCfBtmD9ntb34s518xBr/q90Cq+FWpG1CwxaBeX1blMXW9I2oCMPFuxQUdMlSciIiKiitXocmD4X8D6/wOWjweStgEzr7UVrpMCdhLgEwFIycjFo3M2qmBlUPvauKVzXftc4dLINefivzP/YfOpzdictBm7U3fDbDHbf27UG1E3qq5aX7thlYaoHVEbep3erXXc96XtQ/rR9HJfx716eHU1qlve1etdlZ6bji3JW9QSZ2dzzi+lJ/vXMq4l+jboizCj81Jw5eFkxklsOrVJLbsm+yP7VVry+1A/qj4aVWmkLrUiatnXW5cicv8c+wdf7/za/nlrhNfA+B7jVfE5d9WPrq/e61DaIaw+vhp96vfxuRF3Bu5ERERElY2kxnd9AGhzE7B8ArBuui2FfvcSoOM9QHh8KTesA5r0Aup19fAOU0WSJc7mbzmGqX/sxbGz2Wqd8Lt7GnH7gtux7fS2cn1vCZwkICuTg6gQspzYoKaDcE3jaxATEoPdKbtVp0RiciLSctMqZB+ssOJI2hHsTd2rbhdnwr8TcFXDq9T+Nohu4FSJP82UhpTsFFWJX9Y3TzyVaA++D547WOJ2C5Kl1jxpV4ot26Mk9aLqqVH265pcZy86VxqSLv/Vjq9UunxxgbvZAq9h4E5ERERUWUlxumveAjoPBRaNthWvW/Np2ba5YoKtQ+DKV4EqdT21p1QKJrMF/+4/g7Rs10fJd55Iw6x/D6oq8iIq1IpLu6zBfb99BbP1/Oh4aVULq4Z21dqpS+uqrZ1GgbNN2SpwP3D2APaf24+kzCT3Nm4FMjMzER4ervqQyouM9u47u09VIJd51VKRXAqeeTpodVedyDqqXSU1XJffADnmHCw/vBxH04/i+93fq0uRdnhmHyRDonlsc7Sv3l7ti2QmlJb8PkjHgbT1/rOFfx9k27e3uF0F2dpIfFk4Bu4Fl4VjcToiIiIi8j5ZHk7mvu9YAOxbLpFJ6baTmQxs/xnY+j2wcxHQ4wmgYQ/bzywWRJ7eDRzKAso5jblCxDUGomrCV1Pcv157GF+uOqBGzN1nRbXYdFzSOgN7cn/ETwcOqEdlxPbJTk8iIiii1EFdZFBkienlnWt2RmlV5Dru53LPYfH+xfhpz08qJVyCdvlsCVUT0K562QJWd0nxNQmSq4ZVLfLnz3R5ButPrscPu3/A0kNLS+xgMOqMaBHXwh54N49rjiB9kFv7Iuuje8plsM09rwhSWV7mwp/IOKEyGJrGNrX/jKnyREREROQbJJhqOdB2KYvjm22j94dW2ebQ55MwqrncKGMWtM+QJaUuHglc9hQQEglfkGMyY8KC//DNugPINRyDIfQwouudUKPmLjNkwRx0EBmms1h22vZQfGg8Xur+Evo0cE4frsyig6Nxa/Nb1eXwucNqZLtxTGO35uRXFNknCUrlMt56/m+yuM4OX5m3X9FCjaGq42jl0ZVq1N0xcGdxOiIiIiIKLLXaAUMX2UbdV30I5Njm+srpbk52NkJCQ8szi7limHOA1EPA328Dm74CrhwHJNzq9UyCJ35YiOVJ/wdj4/0I0pvs7X7O3Q2ZbIXBWsS2QJdaXXBfm/tQJaRKeexyQKgXXQ/+omBQLvcdL5WdVJfXAvd729xbeI47R9yJiIiIKGBIAJBws+2Sz2I2Y1sFpTGXOzl537kQ+PV5IOUA8MNw4McRts/tQdIN0FHea0HJ283S6fBRlUj8Ex0JY6TtuVFmCxJyc9E6Nw9V3KioFWK1omVuHlrk5iFk/yEAS4AFzqO05SY4Eug2HLj0cSDYc+nWRO7Mcxfrk9arZeG0aSH2VHmOuBMRERER+QkJ0FtcAzTta8sq+Ot/gCx75eFzegkV9gUZsTM4uNjnZOp0mBZTBUeDbKf1V6VnYETqWTTMM6nA369kpwJ/TAI2zgb6vQa0vsHjnSFEJZGK+1Kl/nDaYfx7/F/0rt9bPc5UeSIiIiIif2UMAS57Euj+MJCV6rHN5llMWH78H8zZ9zPWJW9x6TW6vCh00N+MyXcNtgcZfufwamDJS8DZw8B3Q4G/3gYiq7n8cr3ViqZpadBviyr/gD+2oa2+AVdOCDgdq3dUgbssRacF7ixOR0RERETk74LCbJdSMllMqoq1rJG+NXkr/jj8B5KytKWvdAi3NIYeRVf2zjZZkHGuNurorsEHI/pAH+p6BXCfIyPsF/UHVk4B/n4HOJkInHT95RJaqZn4p1AxNn8N9HgSuGRkmb5/8i0xITHqOl2yaPIZ7CPuXtstBu5ERERERBXFYrXgwLkD2Ja8zR6o7zyzE9lm52XbYkPiEJFzCXbsboM0ky2QKE5kiBHTHrkUUf4ctGskAL5iNNDxbuDASsDqeqRksVhw8NBBNKjfAPryLBQo69lv+CJ/5YTXgY1fAB3uAQy+GFrpgEaXAXU6eXtH/EZUcJS6Ts87H7hrv04ccSciIiIi8gNWq1Wt82yBBbUiapW4/Jc8V1Jut5/eroJ0uchtKXpVkKwB3iq+FVpXbY06oS0x47cQ/HciC0Y98ES/i1A3rvhibd0axaNmlVAElOjaQNtb3HqJ1WzGGcsm1G/bHijvAojtbretnPDby7YVBiSA92VtbwP6jgOia3l7T3xepBRJBJCWa1sRQzBVnoiIiIjIx0fI151Yh1XHV6mgWy6pObb57GHGMDSMbqjW744NiXVKfd9/dj+2n9nudPKvCTWEomV8S7SOb20P1mU7JjOweNsJjP1+K1Iys1A1MhhPdo3EbT0b+38l/kBdOaH5AGDNNCB5N3xSVoptBYQt3wD/zQcufwpofMX5n1ssCE/dBRyzeH05w0JCooH4phVeoDAqf8Td8W+XxemIiIiIiHxQclYyftrzE77f/b0aNXdk1BnVmtdZpiz8d+Y/dSlOsD4YzeOaqyBdAnS5blSlkVonXXMgOQOTF+/Cd+sPIzk9Vz3Wtm4VfHRHB5zcv6McPyWVWXAE0ONx+LSjG4BFo4Eja4DfX7Vd8kl3UEu58Rd8k6zccNUbQLWLKuwtJfulUKo8A3ciIiIiIt9gtpjVyPq83fOw/NBymKwm9bis5dynfh+0rdpWBd/NYpvBoDPgSNoR7D27V42uF0x/rxtZVz23SUwTBOkLzz3PMZmxZNtJfL32EFbuOW1/vHpUCAZ3rY8RV8jr3KrNRlS0Oh2B+5YAW74F/nkfyD5r/5EVVuTm5iI4OBg6Vd7Ph6QdB/YsBfatALo9BPR8FghV5QcrfMSdqfJERERERF4ko+a7U3bj76N/44c9P6j565q21dri5mY346qGVyE8qPAc84ZVGqqLO/YkpeObtYfw/YajOJNhG12XwbzLm1XD7V3ro0/L6ggy2FKWzWZzmT8fkf2XrN1ttosDi9mMrZs2oX379r43HeP0XuDX54Fdi4FVHwD/fmJbgtGdDot+rwO12pW9OB2ryhMRERERle8c9X+P/4sVh1cgx5xjfzzTlKkC9n1n96nnaKKDo3Ftk2txY7MbcVGsZ9JzM3NNWJh4QgXsaw+k2B+vER2CWzvXU5d6JRSfI6qU4psAd3wD7F4K/DoGSN4F5Oa5/vr9fwKf9AQ63gP0eRmIqOpeqrzjcnD50//NHHEnIiIiIvIcSWP/ee/Pap76sYxjJT43LjQObaq2wTWNrkGfBn0QYnBjVK+EivKbj5zFt+sO45dNx5CWY7Kn3PZqXg2Du9THFc2rwahFBERUtGZ9gSa9gLOHXV8eMDcT+PsdYOt3wIaZwLYfgQaXnC90pzcA3R4GGl5a7Ii7LNGYZ85DkCHofKo857gTEREREZU+SF55bKWq/r7jzA51OZ19ft54VFAUrmp0lVq+TSPF4ZpUaaKqu1cLq6aKzbkrz2zBzhNpSDx6Fueyzo8EZuWZsSjxBHaePD9Htn5cOG7rUg83d6qLGtEBtnQbUXnTG4BY96al4ObpQJf7gUXPAie2ALsWOf/85HZg5LpC1fSlpoUmLS8NcYY4FqcjIiIiIioLKQo3btU4LNrvfFIu66t3qdEFNzS7QRWWCzV6JliWOeo/bDyC1fvOYOvRs8gxFT8CGGLUY0Cbmri1Sz10bxQPff6oHRFVkAYXAw+uAHb9CmQk2R6TdPffxgJn9gJ7fweaXen0EunUCzeGq+k0ki4vGTksTkdEREREVEo7z+zEU388hYPnDqoq7zI3vU18G7SIb4FmMc2KLChXGmnZefh58zF8t/4INh6yreGuiQ41ol29GFSPCnWuA1YvBte1q40qYYUryhNRBY/Wt7ja+bHTe/IL3k0tFLiLyOBIFbjLiLswcMSdiIiIiMh1mXmZ2JO6B2tOrMHHmz5GriUXNcJr4K2eb6F99fYef7+Ve5Lx+DebcCrNVthORt6uuKgark6ohQ71Y9AwPoIj6UT+psv9wKoPbcvNJe8GqjZz+rFMr0lCkn1JOO1vnMXpiIiIiKjS2pe6D7/s+wWJpxLVutJFkcdPZZ5So+uOz7m87uUYf+l4xITGeHSfTGYL3l26Gx+u2KMyaxvEh+Oubg1wfYfaTqPrROSH4hoBzQcAOxcCaz4Frn6z6CXh8ivLayPuLE5HRERERJVu5FzWTf9l7y/YdnqbW6+ND41Hi7gW6FWvF25pfouaz+5Jx1Kz8OicjVh30LZ0m6yv/vLAVggL9rF1romo9LoNtwXum74Cer8IhFZxSpUXhUbcuY47EREREVUWMrL+7J/P4kj6EXXfqDOiR50e6F2/N8KMYcW+TtZYvyjuIlQNc2095tJYlHgco7/fgnPZJkSFGDHhxgRc2652ub0fEXlJo55AtRbAqR224L37w06p8iI9L3/Eneu4ExEREVGgOJlxUqWYFlcUzmK1YPrW6fh488cwWU2oGVET97a+F/0b9kd8WDy8KTPXhNfmb8ecNYfV/XZ1q+D92zuifrxnCtwRkY/R6Wyj7vOfAP79BOg63L40XMERd6bKExEREVFArKP+5fYv8da6txBiCFEp7Fc3vhqX1r5UzUfff3Y/dpzegVkHZmFHxg71mqsaXoWXL35ZjaJ7k6zF/sfOU5iw6D/sO5WhzuUf6tkET155EYK0YTYiCkxtbwOWvgKk7Ad2LwGa93ea414wVZ7LwRERERGRXzJbzJi8djK+2vGVup9tzsaiA4vURdZCzjHnwGw1258vqfBjuo7BoKaDoMsfxaoIJ89lIz3HZL8vVeLnbzmGBVuOIyUzTz1WIzoE79zaHpc0Lb9UfCLyIcERQLs7gH8/BnYuKBS421PlteXgGLgTERERkb/JMmVh9J+jsfzwcnX/yU5PomvNrliwfwEW71+MU1mn7CfBsq56nCkOo3qMQuPYxhW2j+ey8/DKT9swb+PRYp9TNTJYzWMf1bsZ4iKCK2zfiMgH1Gpru049ZH8oMijSuaq8NuLOVHkiIiIi8odK8NtPb1frqMvl3+P/4sC5AwjWB2P8ZePVXHXRumprPNXpKexK2YXY0Fi1zrrFYsGmTZvQILpBhe3vqr2n8fTczTiamqVS4KNDg+w/CzbqcXmzari+fW1c0iQeRqbFE1VOVerZrlNt9S2KmuOuZQexqjwRERER+bSNSRvx+PLHcSb7jNPjVUKqYEqvKehYo6PT4wa9AS3jW8IbZDm36X/vx4yV++1rsL99azt0ahDnlf0hIh8WU992ffawDKmrAnVa7Y20vPzidPn9epzjTkREREQ+S9ZaH/vPWORZ8tRSbK3jW6NpTFM0jW2K7rW6l+vybK6SOesLthzD/C3H7euva2uwv3hNS0SE8LSXiIoQXRvQ6QFzLpCRBETVLJwqbx9xZ+BORERERD5Glm/7YOMHmJY4Td3vW78vxvcYX+xyb96y+XAq7pi2Ghm5tiJ4co7dpWEcHurZGL1b1PD27hGRLzMEAVG1gXNHbOnyErjnp8prxem0qvIsTkdEREREPiMlOwXLDi3Dz3t/xoakDeqx+xPux6gOo6CXkSkfcjYzD498tUEF7c1rROHWLvVwTUIt1KwS6u1dIyJ/Spc/J4H7QaBeF0QF2arKn8s9p5a75DruRERERFSu5MRTlmUz6p1P+46lH1PV4KXYnA46NSfdoDPgSNoRrDmxxr6Em7xu7MVj1fJtvkZOqJ/+bjOOpGShXlwYvn3oYlQJO1+AjojIJTH1gEP589wdloMzWUxqSUv7iDsDdyIiIiLy5Nrqfx75E7P+m6WCcKn63iSmiZqXLvPRVx1fhR1ndpS4jRZxLdCvQT/0b9Qf9aLyqy77GClA99v2kwg26PHRHZ0YtBNR2QrU5S8JJ9OBpEPTCqtKl9eWg2OqPBERERF5JGCfu2suvtj+BQ6nnV/aKNeSi//O/KcuGkl5b1+tPbrX7o4QQ4h6rclqQoQxAlfUuwL1o/NPZH3UhkMpmLjI1vnw4sCWSKhbxdu7REQBsiScXqdXBeqkqrwsCcdUeSIiIiLymHc3vIvPt32ubstyRjdfdDNua36bCsp3pe7C7pTdOJFxAu2rt8fldS9HXKh/LY8maaprD5zB4q0n8NOmozBZrGo++93dK25teCIK8CXh8kmBOi1w1+ttqfNm78XtDNyJiIiIAsFPe36yB+1PdHoCg5sPdqr+Xi+6HvrU7wNfDcjnrDmEpf+dLHEO6fZj53A6I9d+X4rRTbwpAbr80TAiojKnyks6vE6n5rkfzziuloQz6GyBO0fciYiIiKjUNiVtwrhV49Tt4W2HY1ibYfAXO0+k4bl5W7DxUKpLz5d57Fe2qoEBbWqiR7OqCDEayn0fiSjARdexXedlAplngIh4+1ruMupu0NdWt1mcjoiIiIhKRVLfH1/+OPIseWpEfUT7EfAHOSYz3v99D6b+sVelvEeGGPHwFU1QO6b4ZdxqRIWiS6M4BBl8a0k6IvJzQaFAZE0g/YRtSbiIeHtleRlx5zruRERERFRqWaYsPLrsUZzOPo2LYi/ChB4TfG6d9aLIqNWIWRvw+44kdV9G0F+9vjVqVQnz9q4RUWVeEi79hG2ee52Oao67kDnuoSxOR0RERESlXcP8lX9eUZXipcjc+73fd5rT7sv+t2SnCtpDjHq8c1t7lfbOeepE5PV57kfW2peEiwqKsqfKh+ePuFu8OOLu1S7ZnJwcPP/88+jcuTN69OiBGTNmFPvcFStW4Prrr0eHDh1w7bXX4vfff3f6+fz589G3b1+0a9cOjzzyCM6cOVMBn4CIiIjIO77a8RUW7l8Ig86At3q+hdqRtjmYvu6XLcfx0Yq96vbkm9vi6oRaDNqJyOeWhItySJXXloPzZlV5rwbukydPxtatWzFz5kyMHTsWH3zwARYvXlzoeTt27MDIkSNx00034ccff8TgwYPx2GOPqcfFli1b8MILL6jnfPPNNzh37hzGjBnjhU9EREREVP7WnViHt9a+pW4/1fkpdKnZBf5gX0oenpuXqG4P79kY17fPLwhFRORjS8JF5qfKp+elw6CvxKnymZmZmDt3LqZNm4bWrVury+7duzF79mz079+/0Gh69+7dcc8996j7DRo0wLJly7Bo0SK0aNECs2bNwoABAzBo0CB7h0CvXr1w+PBh1KuX33NCREREFABOZpzE0388DZPVhAGNBuCulnfBl+auy/ltwRF0k9mCdQfOYNLKFGTnWXBF82p49qoWXttPIqISl4STwF2rKp+bBp3tZuWsKi+j5SaTSaW+azp16oSpU6fCYrFArz+fDHDDDTcgLy+v0DbS0tLU9ebNm/HAAw/YH69VqxZq166tHmfgTkRERIEiIy8DT654UhWjaxbbDK9c/IrPpJlvPXoWt3+6Wt1uWTsarWpFo2F8ODYdTsWKXaeQmmk7l2tcNQLvDe5gH8EiIvKtwP2wuooOjrYH7obKXFX+1KlTiI2NRXBwsP2xqlWrqnnvqampiIuLsz/epEkTp9fKyPyqVatUyrxISkpC9erVnZ4THx+PEydOuLVPZrNZXTRWWGC1Wuw/I9dp7cV28wy2p+exTT2PbeoZOqsF1gLHo0Amx/1x48ZhyZIlCA0NxbBhw9SlKDt37sQrr7yCbdu2qew7mSYnGXmVRWp2Kkb8PgKJyYlq7uW7V7zrM8XosvPMePybTUjLMan7a/afURdHMWFBaFfdgNdu6azWYici8ilV6tquc84CWanOqfKVuap8VlaWU9AutPu5ubnFvk6Kzo0aNQodO3ZEnz591GPZ2dlFbquk7RRl+/bt9tsGgwHJyXk4m5mngvfExMxKcxLlSYmJtnls5BlsT89jm3oe27T05NhT69QpHE9MrDTHHMd6N8eOHcPo0aNV1lzBaXOSZScBfe/evTFx4kT89NNPqrbNr7/+qjrrA11SZhKG/zYce1L3ICYkBh/3/Rj1o/NHh3zA5MU7sScpHdWiQjD1rk44eDoD24+dw77kDDSrEYm+LWugbe0obE3cgjqxXPKNiHxQcAQQHg9knlbz3B1T5Sv1Ou4hISGFAmvtvvS4FyU5ORlDhw5Vy59MmTLFnk5f3LbCwtw7MLRq1cqpAyAx6xCCs2xpXQkJzqP+VDI54ZST94SEBHUiSmXD9vQ8tqnnsU09Q5e5ElUTEtRxzLFDORC5U+/mhx9+QHh4uBpxl9+vRx99FH/88YcK+nv27IlAdvjcYTzw2wM4mn4U1cOq49N+n6JJjO+cl6zck4wZK/fbq8R3ahCrLjd2dH5eZemMIiI/T5fPPK3S5aNqNlcPVfoR9xo1aiAlJUXNczcajfb0eQnao6Nt8wkcnTx50l6c7osvvnBKpZdtSVDvSO5Xq1bNrX2SEwHHk00d9NDpbJ0DPAktnYJtSmXD9vQ8tqnnsU3LSI47laQN3al3s2bNGpVp59gu33//PQJdrjnXHrTXi6qHT6/8FHWj8tM5fcDZrDw8PXezun1Ht/ro1dx56iIRkd8tCXdsoypQF1W/s305OK0kR6UccW/ZsqUK2Ddt2qTWcRfr169XIzWOB2qtR/7+++9Xj0vQXjAgl7Xb5bU33nijun/8+HF1kceJiIjIN7lT70ZWimnbti1eeukltbJMnTp1VFq9BPru8sTIb0XVdPhl7y8qaK8WVg0zrpyBauHVfGbk+lhqFiYs2oHjZ7NRPy4cz111UYn7xjoY7mObuYft5T62mTNdlXpqvXRL6kGEG2w1RMxWM3ItmbbbZqtH28ydbXgtcJc0dlm+TVLeJkyYoArMzZgxA2+88Yb9YB4VFaVG4D/55BMcOnQIX375pf1nQn4mz7n99ttx9913o3379irwHz9+PK644gpWlCciIvJh7tS7kU78Tz/9VGXfSWr9ggULcN9996mlYWU1GW/VYSjPmg4WqwWf7PlE3e5dpTeO7joK+edN+1PzsOJAFjadzMWRc7ZCdHKS+1D7UOz+b6tL22AdDPexzdzD9nIf28ymWroOUj3k7IFE7E3cAT30sMCCXftsU9cyMrPsbVXRbea1wF2MGTNGBe5DhgxBZGSkKjrXr18/9bMePXqoIF5G0aXwjBSgu+WWW5xeL8vESYEaSbF79dVX1bz3s2fP4tJLL8Vrr73mpU9FREREnq53Iynykq0nc9u1ujQrV65UReoeeught97XE3UYKqKmw4rDK3B823FVIGnU5aPsFY69JfHoWTw/bzVyzbZUUUkdbVcvBsMuaYirE2pe8PWsg+E+tpl72F7uY5sVEH4C2PoBYnRpKsaM2h2Fs7lnUat+NWDlKQSHhKi28lSbae3v84G7jLpPmjRJXYpa8kWzePHiC25LAnwtVZ6IiIh8nzv1bmSaXOPGjZ0ea9iwoZoa5806DOVZ0+Hz7Z+r69ua34YqYVXgTWnZeXjsm80qaO/aMA5DLmmIHk2rokq4+8u6sQ6G+9hm7mF7uY9tli+2gbrSnT2s2kOW3pTAPdeapR6X2nRaO1V0mzlPJiciIiLyQr0bTXH1bmQ6nGOnvti3b5+a6x6INiZtxKZTmxCkD8KdLe/06r7Iaj4v/rgVB09nok5MGKbd0xnXtK1VqqCdiMinxeRPtZbK8rkZKnAXOeYMrxenY+BOREREXq93s2XLFixdulTVu9FWkZHRd5kqJwYPHqwC9/fffx8HDx7Ee++9pwrWXX/99QhEMxJnqOvrmlynCtJ509x1R/DTpmMw6HWYcnt7BuxEFLhCq9guIvWwfYpStlaczovLwTFwJyIiIq/Wu5H126Xezbhx4wrVu1m4cKG6LSPrn332GZYvX46BAweqaylWJ+n2gWZPyh6sOLICOuhwb+t7vbsvSWl4+Wdb0bknr7wInRqcr/RPRBSQqtTPD9wPqRojIjt/xN3v1nHv1asXrrnmGlx99dWqOAwRERFReda7EbL027x58xDoPt9mm9vep34fNKzS0Cv7kGe24Pv1R/DO0l3IzrOo+ewP92zilX0hIqpQMfWBk4nA2UP2VPlsc7osWKrmuPtV4P7cc8+pgnF33nmn6umWAF4C+SZN+B86ERERUWkdSz+GBfsWqNtD2wyt8PeX0aT5icfxzm+7sD/ZNsLUID4cb9/WDnopI09EFOgiq9uuM04jKtwWuGeZvD/HvVSB+1VXXaUuMu9MUtWWLFmCO+64QwXxkr4mgXzdunU9v7dEREREAWzG1hkwWU3oXqs72lZrWyHvKXM21x44g8VbT+DXbSdw/KytrkB8RDBG9GqKO7vVR2gQq00TUSURYgvWkXMOkVVqqZtZ/poqr5HlWiSAj4mJQVxcHL777jt8/vnn+Oijj9CxY0e89NJLaNSokef2loiIiChAJWUm4YfdP6jbD7Z9sEKqxX+x6iDeX7Ybyem59sejQo148LLGGNqjESJDvLpyMBFRxQvJX440N92eKp9lSvfPEXeLxYLVq1erdHmpACsLx1955ZWYOnUqunXrhszMTIwdOxYPP/ywS2uwExEREVV2M7fNRK4lFx2qd0DnGp3L9b2y88x4/odEzNtwVN2vEhaEvi1roH+bmrisWVWOsBNR5RViK0iHnDR74J6pBe7+NuJ+8cUXIzc3F1dccQVeffVVXH755QgODrb/PDIyUgXymzdv9uS+EhEREQWkM9lnMHfXXPtou05XfvPJj5/NwvAv12PLkbNqibcxA1pgyCUNEWTgYkNERLCnyqfZq8pnmvw0Vf7FF19Enz59EB4eXuhnZ86cUWnz/fv3VxciIiIiKtms7bOQZcpCq/hWuLT2pR7bblauGXPXH8bRlCyk5ZiQnm3CP3uTVWp8THgQPryjIy5tWtVj70dEFFCBe7Bz4O53qfLPPvssVq5cWShwP3r0qCpOt3HjRk/tHxEREVFAO5tzFl/t+Mrjo+0yMjTyqw34fUdSoZ+1qBmFafd0Rr24woMwRESVWogWuKcjKsg5Vd5ige8H7j/++KN97VQpZvLII48gKCjI6TlJSUmoVq2a5/eSiIiIKEDN2TEHGXkZaBrTFL3q9fLYdqcs262C9mCjHnd1a4DoMKMqNlctKgT9WtVEWDDnsRMRFRJ8vqq8Nsc9PS/Nf0bcZc76kSNH1O01a9agffv2iIiIcHqOjMDL84iIiIjINb/s/UVdD2szDHqdZ+aZ//7fSby7dLe6PX5QG9zSuZ5HtktEVBlT5bNMmTLeDrNF7/uBuwTpI0eOVLfr1KmDa665xqkgHRERERG559C5QziUdghGndFjo+37kzPw+Deb1O27uzdg0E5EVMrAXRtxt8IK6HMBS6jXCtS5lSp/9dVXq2Bd5l4tXLiw2OcOGjTIU/tHREREFLD+OvqXuu5Qo4N9ZKcsMnJMePCLdUjLNqFzg1i8NLCVB/aSiKgSBu6WPIRYgSB9EPIsedDps2G1hHotXd7lwH3KlCno2bOnCtzldnEkqGfgTkRERHRhfx/9W133qNPDI9t7f9ke7E5KR/WoEHx0V0c1v52IiNzg2ImaP+ouS3bqDFmwmmJ8f8R92bJlRd4mIiIiIvdlm7Kx9sRadfuyOpeVeXsHT2dgxt/71e3xNySgelRombdJRFTp6PW24D033V6gTgXu+mz1Y58fcV+71nZguRAZce/cuXNZ9omIiIj8QFpaGn7++Wfs378fI0aMwObNm9GkSRPUr1/f27vmFyRozzHnoEZ4DVVRvqwmLPwPuWYLLmtWFX1bVvfIPhIRVdp0+VwJ3NMQGZQ/Am/ID9y9tCScy4H73Xff7XLg/t9//5Vln4iIiMjH7dq1C0OGDEGtWrXst5csWYLFixfjk08+QdeuXb29i36VJl/Wtdv/2ZuMX7edhEGvU/PaPbUWPBFRpQ3c046rtdy1wF2nz1HXFl8fcd+xY0f57gkRERH5jddffx233347Hn30UXTo0EE99sYbbyAuLg6TJ0/Gd9995+1d9JvAvaxp8maLFa/+sl3dvrNbfVxUI7+wEhERlW2ee04ajIb8kFln9o/A/dixY6pXXXpw5XZJateu7Yl9IyIiIh+VmJiogveCBg8ejNmzZ3tln/xyGTi9Ed1qdSvTtr5eewg7TqShSlgQnuh7kcf2kYio0go5vyRckC5I3dTlB+7SWerTgXvv3r2xcuVKxMfHq9sSwFsdehu0+0yVJyIiCnwysi5z2wvOZ9+wYYM6VyDXloHrWL1jmZaBO5edh/8t2aVuP963GWIjgj22j0RElVaIFrifUx2sis42ud3nq8r//vvv6iCt3SYiIqLK64EHHsCLL76Ihx56SHXcr169Gj/88ANmzpyJJ554wtu7V2mWgfu/vw/gTEYuGleLwF3dG3ho74iIKrmQaNt1brpax13otRF378TtrgfuderUKXRbetr37t2LoKAgNG7cGPXq1SufvSQiIiKfIinx1atXx/Tp0xEaGqrmtTdq1AivvfYarr76am/vnt8sA1eWwD0tOw/T/96nbj/e9yIEGbhmOxGRR4Q4zHEPsoXMer2fjLg7On78OJ599lm1RFyVKlVUT7ssCSMp9OPHj0dMTIzn95SIiIh8xmeffYaBAwdyPrsXl4Gb+c8BnMs2oUm1CFyTUMuj+0hEVKmFnJ/jbgwJUTd1+YG7t9ZxL1XXrKTGGQwGlTL/77//Ys2aNVi0aBFSUlLw8ssve34viYiIyKdMnToVeXl53t4Nv7Ty2Ep1fVndy0q9bFt6jgmf/b1f3X60TzO1DBwREZVD4J4/x92eKm/xo8BdRtoleHdMn2/YsKEK2v/8809P7h8RERH5IBlt//jjj3HgwAHk5uZ6e3f8yupjq9X1JbUvKdNoe2pmnprbPrAtV/MhIiqv4nTaHHed3k+Wg3PUpEkT7Nq1C02bOqd3HT582CmYJyIiosAkHfWyPKwUpCsKV5gpWnJWMvae3QsddOhSo0uptpEho+1/2ea2j+rdlKPtRESeFqwF7un2EXddflV5n18O7scff7Tf7t69O1544QVs374dCQkJKm1+586d+PzzzzF06NDy2lciIiLyERMnTvT2Lvilf4//q65bxLVATGjpagJ9seogUjLz0KhqBK7laDsRUcWkymvF6Xy9qvyUKVOc7sfGxmLhwoXqoomKisL333+PESNGeHYviYiIyKd07dpVXUuqvKwwY7FYVFX5gtl45GzNiTXqulutbqV6fWauCdPyR9tH9moKIyvJExFVSOAOL89xdzlwX7ZsWfnuCREREfmNc+fOYcyYMapQrawwYzabkZGRgS5duuDDDz9UnflU/Ih715q2jg93fbP2sFq3vX5cOK5vz9F2IqLyDtztc9z9JXAv6MyZM2odd+lhF7IknBSnkfT5Bx980JP7SERERD7m9ddfx4kTJ1TmXePGjdVje/bswXPPPYc33ngDEyZM8PYu+pwjaUdwNP0ojDojOtXo5Pbr88wWTPvTNto+vGdjjrYTEZV34J5beI67XxWn+/bbb/Hqq6/CZDKpZUwkaBdyu23btgzciYiIApxk4v3f//2fPWgXkiYvK8w88MADXt03X0+TT6iWgPCgcLdf//OmYzh2NhvVokJwU8e65bCHRERUaMRdZyg04m57pGLpS7t260MPPYQtW7YgPj4ey5cvx/z589GyZUtceeWVnt9LIiIi8ikhISHQ6wufRkgnvqTNU2Grj68u9fx2i8WKqX/sVbfv69EIoUHeOG0kIqpkgTusMFotTnPcvTXiXqrAPSkpCYMGDUJwcDBat26NTZs2qV72559/HnPnzvX8XhIREZFP6d27N8aNG4dDhw7ZH5NCdZJC37NnT6/umy+S7MQ1x9eUen777zuSsDspHVGhRtzZrX457CEREdkZQ4H8FPkgi9kn5riXKnCPi4tTc9yFpMhpa7XWqFEDJ0+e9OweEhERkc955pln1Kh7v3790K1bN3Xp37+/KlT30ksveXv3fM7e1L04nX0aoYZQtKvWzu2g/6MVe9Ttu7s3QFSorVASERGVE50OCI5UN41mU/5jfrIcnKMBAwZg9OjRGD9+PC677DI8++yzauRdUuYbNGjg+b0kIiIinxIdHY0vv/wSO3fuVMvBSRAvy8E5znmn8/49Yasm36F6BwQbgt167Zr9Z7DxUCqCjXoMvbRROe0hERE5CYkGslMdAnfvpsqXKnB/+umn1TIvKSkp6NOnD2666SaMHTsWMTExqpIsERERBTZZSebdd99FnTp1cOedd6rHbrzxRlxyySV47LHHEBTEUeEil4Gr5X6a/Mf5c9tv7VxXFaYjIqKKm+duNOfmP+CHqfJyMB45cqQK2sUTTzyB1atXY/HixejQoYOn95GIiIh8jMxl/+OPP9CiRQv7YyNGjMCKFSswadIkr+6brzFbzFh3Yp263b1Wd7dee+JsNlbsPKWyNh+4jNkMREQVHbgHmXKdR9z9bR33tWvX4uuvv1bpcRLIN2nSBEOGDFGV5YmIiCiwLVmyRC0H53jc79u3r6p3M3z4cLz44ote3T9f8t+Z/5CWl4aooCi0jHPvPGnx1uPqulP9WDSIjyinPSQiokJCtDnueU5z3M1emuNeqhH3WbNmYdiwYaqq/M0334xrr71Wrel+6623YsGCBZ7fSyIiIvIpUjAtJyenyMfz8vJPckhZe2Ktuu5UsxMMeveWcVu49YS6HpBQq1z2jYiILpAqb8rxiVT5Uo24T5s2Da+99ppaEs5R586d8fbbb+Oaa67x1P4RERGRD7rqqqtU9XipcdOqVSv12I4dO1QK/ZVXXunt3fMpB88dVNfujrYnpWVj7QHbKj7929Qsl30jIqILBe5Ztvs6k/+t456eno6EhIRCj0vgri0TR0RERIFrzJgxaNasmZom16lTJ3W55557VOr8Cy+84O3d8ynH0o+p61oR7o2a/7rtJOT8sH29GNSJCSunvSMiomKryssc9zzbiLvVH0fc77rrLrz55puYPHmyWg5GSLrcBx98oNLliYiIKHAlJycjNjZWZdmdO3cOBw4cULVvZEk4qSwfHh7u7V30KcczbPPUa0fWdut1ixJtr7s6gaPtREQVTlvHPS+7wDruPh649+7dGzopaZo/f+3YsWO4/PLLUa9ePej1ehw6dEgF7yxOR0REFJgyMjLw1FNPqWry8+fPV4Vpf//9dzXCXrNmTRW4S8G62bNnq/tkO2eyB+4Rrgfup9NzsHrfaXV7QBvObyci8lpV+TxbqrwVpvMj7raw2DcD91GjRpXvnhAREZFPe//993H06FFVpLZx48bIzMxUc9rbtm2LL7/8Uq0yI3Pe33rrLXUh4HT2aeSYc6CDDjUjXO/M+G37Sci5YZs60agXxwwGIiKvzXG3B+75qfJWHw/cb7jhhkKPZWVl4eDBg7BYLKhfvz4iI23pBK6SEfpx48apJWVCQ0NVpXq5lGTdunUYPXq06uEvOL8+LS3N6bENGzYgIoJLpxAREXmCHK8nTJig5rOLv//+W43C33333SpoF5IqL8vBkc3xdNtoe7Wwaggy2NrIrWryHG0nIvLuiHtupqoMpwXuFktpK8V5YY67LPMic9y/+uormM1mlQZmNBrVsnASiMsyca6QOfJbt27FzJkzVeq9BOS1a9dG//79i3z+zp078dhjj6lUPEcnT55UQfvSpUtVB4CGc+yIiIg859SpU6qjXvPPP//AYDCgR48e9seqVq2qOvbJ5lhGfmG6SNcD8NTMXPyzJ1ndHsBq8kRE3h1xz00HQgGrzrvF6UrVVzBp0iQsX74cH3/8sSpGs2bNGnz44YdqNPydd95xaRuSXjd37lw1L65169Zq6Zj7779fzYsrytdff43BgwcjPj6+0M/27t2LatWqqfn2cq1dtDn5REREVHY1atTA4cOH1W3ptJe57u3atUOVKlXsz9m4cSNq1eIoccERd3fmt0uavMliRYuaUWhczb1sRiIi8nTgnuk0x92vloOTgjQyp+2yyy5T6fFSWb5nz55qbfdffvnFpW3IWq8mkwkdOnSwPyapd5s3b1ap9wX9+eefqsPg3nvvLfSzPXv2oFGjRqX5KEREROSi66+/HuPHj1fT1SRl/vjx47jjjjucju1Sab64zLnKqDQj7gvzq8kzTZ6IyBdS5dP9dzk46WUvauQ7Li5OzXVzNd1OlpJxTKuX9DqZ956amqq25eijjz5S1/PmzStyxF3S8mSO3f79+1Vl++eff97tYF7S/uVi/5ywwGq1dSI4Pk4XprUX280z2J6exzb1PLapZ+isFlgLHI98xcMPP4z09HR1jJWstkcffRQDBw5UP5POdakof8UVV6jnUelG3I+fzcKfu21p8te0ZeBOROT1EfccCdzDVHQIWGzF6fwlcO/evbu9YqxWkE7WcZVe9m7durm0DQm0C86F1+7n5ua6tT/79u3D2bNn8eSTT6r9mTZtmhqZX7BggVsF87Zv326/LXP2kpPzcDYzTwXviYmZPnkS5esSExO9vQsBhe3peWxTz2Oblp4ce2qdOoXjiYk+ecyRejZjxoxRl4IGDRqkat20atXKK/vm6yPurq7hPmfNYTWa061RHJpWZ5o8EZHPrOMudBZY/GnEXXra77nnHpUqr41qy0i3zDGXee+ukAJzBQN07b5jgTlXTJ8+XRXM0yrIS4eCpO7LPHw5iXCVnGw4diYkZh1CcFaeup2Q0MStfars5IRTTt4TEhLUiSiVDdvT89imnsc29Qxd5kpUTUhQx0THDmVf17x5c2/vgm+PuLsQuOeZLZiz5pC6fVf3BuW+b0RE5MKIu+NjOrNaqtNvAveoqCg1z13mnctotwThEsBfeuml0Ov1Lhe4SUlJUfPcpQdfS5+XoF3mzLtDgm3HgFv2p27duqravDvkRNPxZFMHPXQ62+fhSWjpFGxTKhu2p+exTT2PbVpGctxhGwaEtNw0pOXZlqqtFXHhtPcl207iVFoOqkaG4KrWrCZPRORVhiDAGIYgk+NKKWb/Kk4n89l2796NPn364IEHHrCPvrsatAuZhy4B+6ZNm+yPrV+/Xo3UuLMdmW/ft29fp7nvUrFe1pdv3LixG5+KiIiIyHOOpdvS5GNCYhAedOElar9cfUBd3961HoKNXlgkmIiInIVEwbEbXacz+9dycBJYS2p6WYSFhan5cK+88gq2bNmi1mCfMWOG6gTQRt+zsx3mExRDiuNIIZz3338f//77r+pQePbZZ1GzZk2VLk9ERETkDcczjrs82r77ZBpW7zsDvU4C9/oVsHdERHRBIZGQBcaNuvzwXWfxr6ryEigPHToUvXr1Qp06dQoVmRs5cqRL25HiNhK4DxkyRBWRGzVqFPr166d+1qNHD7zxxhu48cYbL7idZ555Ro3eP/XUU6rarRTP+/TTT5lmSERERF4fcXdlfvvsf21z2/u0rIHaMVK9mIiIfGZJOJ0BJqsZ0Jm8lipfqsB9586daN26NZKSktSl4Ai4O6PusnyMXIp6j6JIIF8wmJc57c8995y6EBEREfnTiHtGjgnfrz+ibt/NonRERL4jxFZ7zZhf98w24g7fD9x/+ukn/Pbbb2q9dZnfrq3dSkRERESlG3H/adMxpOWY0DA+HD2aVq2gvSMiIpdH3FXCvG2Ou88Xp5s5c6ZaBk7mncsa7JLmLuu2ExEREVHxI+61I0oO3H/YaBttv7NbA+hlkjsREfnWWu7Q/m/2XnE6l0fcv/76a4wfP14VlBNLlixRwfsTTzzhVno8ERERUWUaca8VWXyqfGauCZsOp6rbXAKOiMhX13LX2e5LVXlfH3E/fPgwLr74Yvv93r17q5H3gnPciYiIiCq7HHMOTmefvuCI+4aDqcgzW1G7SijqxbEoHRGRb6bK2+h0Flh8fTk4k8mkKrdr5LYUhcvNzS2vfSMiIqIAl5OTo6bide7cWa0oI0vDXsiRI0fQoUMHtQysrzqebkuTDzOGoUpIlWKft3qfLbjv3jieGYxERL464m61en3EvVRV5YmIiIg8YfLkydi6dauqpXPs2DGMHj0atWvXRv/+/Yt9jSwlm5mZCV92LCO/MF1E7RIDcsfAnYiIfD9wt/hDVflFixap9dY1FotFVZmPi4tzep42D56IiIioOBJ8z507F9OmTVPLzMpl9+7dmD17drGB+88//4yMjAwEQkV5md+++YhtfjsDdyIiXw7cLedH3H29OJ30fhdMX4uPj8esWbOcHpNeZQbuREREdCE7duxQU/Ek7V3TqVMnTJ06VQ0O6PXOM/pSUlLw5ptvqvMRX1+S1pXAnfPbiYj8ZI671QKpT6fzh1T5ZcuWle+eEBERUaVy6tQpxMbGIjg42P5Y1apV1bz31NTUQhl9EydOxA033IBmzZqV6X3NZnOZXu+4jeK2pQXuNcNrFvucf/aeUtddG8WpjopAd6E2o8LYZu5he7mPbXYBQREwSNBsMUPdgAVms8XjxxJXcI47EREReYWsTuMYtAvtfsHit//88w/Wr1+P+fPnl/l9ExMTy7yNC21r98nd6jrnVA42bdpU5HOWJeZXnTemF/ucQOTJ9q8s2GbuYXu5j21WtPCUY2gpFd1NeYDBoFLlT585AyCmwtuMgTsRERF5RVGr02j3Q0ND7Y9lZ2fj5ZdfxtixY50eL62EhAQY5ASsjKMkctJW3LbS9qep64tbX4x21doVOb9977zf1e1berZH/bhwBLoLtRkVxjZzD9vLfWyzC0gOB/4GgtUcd1vgHh0TC8Dq0WOJKxi4ExERkVfUqFFDzVt3XHJW0uclOI+OjrY/b8uWLTh8+DAeffRRp9c/8MADqq7Oq6++6tb7yomWp05Qi9qWyWJCUmaSul03um6R77X5SIp9fnvDqpGVaik4T7Z/ZcE2cw/by31ss2KExairIItJreYuc9y1Ke4V3WYM3ImIiMgrWrZsqQJ2SROXddyFpMPLKIZjYbq2bdtiyZIlTq/t168fXn/9dVx66aXwNRK0m61mBOmDUDWsapHP4frtRET+uBycxfeL0xERERF5UlhYmBoxl3XZJ0yYgKSkJFUx/o033rCPvkdFRakR+AYNGhQ5Yi8r3Pgae2G6iJrQ65wr42u4fjsRkR8IjpBo3SFwN8GiloOr+A7Xoo8mRERERBVgzJgxav32IUOGYNy4cRg1apQaTRc9evTAwoUL4W+OZxxX17Ujil4Kjuu3ExH5CZ1OjboH5QfuOqkq7+vruBMRERGVx6j7pEmT1KWgnTt3Fvu6kn7mbedyz6nr2FApYFQY128nIvIjIVEIQpbtts4Mi5dS5TniTkRERORB6bnp6joiSFIsC+P8diIiPxIS5ZAqb0b+Mu4VjoE7ERERkQel59kC96hgW1GjgtbslzWAgW6N4yp0v4iIqBSCI2HUBtklcOeIOxEREVHgBO5FjbjnmMzYlD+/vUtDBu5ERD4vJPL8HHedJb84XcVj4E5ERETkQRm5Geo6Miiy0M8Sj5xFrsmCqpHBaFS16FR6IiLyIYYQGOGQKs8RdyIiIqLAHnFfc8CWJt+5QRzntxMR+QNjyPlUeZg54k5EREQU6HPc1+bPb+/SiGnyRER+wRjKEXciIiKiyjLiLqM06w6mqNtdOb+diMg/GIMd5rhzxJ2IiIgooOe47zyZhrRsEyKCDWhZq+iK80RE5IMj7tb82zoLl4MjIiIiCqgR92DnEfe1+fPbOzaIhdHAUzAiIr8pTmdlqjwRERFRwLBarcjIs424RwU5j6qvPWBLk+cycEREflacDvmYKk9ERETk/7JMWTBbzYXmuEtAby9Mx8CdiMh/GEOd57hzxJ2IiIjIv2mj7XqdHmHGMPvjR1KycOJcNoIMOrSvF+PFPSQiIrcYg8+nysMCLw24M3AnIiIiKo+K8o7rtK/JH21vU6cKwoINXts/IiIqxYi7dltngpmp8kRERET+LT03vcj57esO2gJ3LgNHRORnDOdH3HVSVZ6p8kRERESBWVFeG3HvzMCdiMhv57iDxemIiIiIAmeOu+Ma7qfTc7D3lO3xzg1ivbZvRERU9qryHHEnIiIiCqA57pp1B23LwF1UIxKxEcFe2zciIipl4M4RdyIiIqLAm+PuOOK+5Uiquu7E0XYiIv9jDIVRi9thgdnind1g4E5ERETk4RH3yODzgfuhM1nqunHV848REZGfMAQjCOdH3JkqT0RERBSAc9yPpGSq67qx59d1JyIifxpxt9puM1WeiIiIKDDnuB/OH3GvFxfutf0iIqJSMspycPm3OeJOREREFHhz3LNyzUhOz1G368UycCci8ssRd2jruHPEnYiIiCjg5rhrafJRIUZEh9kXFCIiIn9hDHFYx93CEXciIiKiQJvjfiTFliZfNy4cOp3Oq/tGRESlYJDAXbvjvRF3dv0SERH5glUfqZMDCqw57ofzR9zrsTAdEZH/p8rrvTfHnYE7ERGRLzDZ5kHLKrEUOHPcD5/RKspzfjsRkV8yBjuMuANmi9kru8FUeSIiIqJym+OuVZTniDsRkd8vByd0Fli9MOrOwJ2IiIjIA+REruAc9/Op8hxxJyLyS3ojjI41SqSyvBey5Rm4ExEREXlAlikLFqvFeY57/hrudTniTkTkn3Q6GPXna9CoJeEYuBMRERH5d5q8QWdAmDEM57LzcDYrTz3GEXciIv9lMIZAb18SrhIG7jk5OXj++efRuXNn9OjRAzNmzLjga9atW4c+ffoUenz+/Pno27cv2rVrh0ceeQRnzpwpp70mIiIiKrmivCz9diR/tD0uIhgRIawHTETkt4whMNqDdQsslW2O++TJk7F161bMnDkTY8eOxQcffIDFixcX+/ydO3fiscceK1QMYMuWLXjhhRcwcuRIfPPNNzh37hzGjBlTAZ+AiIiIyCYjt+j57XW5FBwRkf8H7tBG3E2Va8Q9MzMTc+fOVQF369atceWVV+L+++/H7Nmzi3z+119/jcGDByM+Pr7Qz2bNmoUBAwZg0KBBaNGiheoQ+OOPP3D48OEK+CREREREDiPuwRHOFeWZJk9EFDCV5XU6GXGvRIH7jh07YDKZ0KFDB/tjnTp1wubNm2Gx2Aq7OPrzzz8xadIk3HvvvYV+Jq+RdHtNrVq1ULt2bfU4ERERUYUuBVdwDXcWpiMi8m+GkPNruas57hUfuXttwtWpU6cQGxuL4OBg+2NVq1ZV895TU1MRFxfn9PyPPvpIXc+bN6/QtpKSklC9enWnx2Rk/sSJE27tk9lsVheNFbJGn60TwfFxujCtvdhunsH29Dy2qeexTctGJ8cbdczRwVrgeET+IT3XOXA/Yk+V54g7EVHgpMqbvTLi7rXAPSsryyloF9r93Nxct7aVnZ1d5Lbc3c727dvttw0GA5KT83A2M08F74mJmTyJKoXExERv70JAYXt6HtvU89im7pNjTo2TJ6ELCoNBb8DRxEQec/xQwTXcz6fKc8SdiMjvA3czKmfgHhISUiiw1u6HhoZ6ZFthYe4dKFu1auXUAZCYdQjB+cu4JCQ0cWtblZ2ccMrJe0JCgjohpbJhe3oe29Tz2KZloztXQ50YyIh7XEKCOo45diiTf81xl0K6Wqp8vTiOuBMR+TVjCIJM+XPcYYa5MgXuNWrUQEpKiprnbjQa7enzErRHR0e7va3k5GSnx+R+tWrV3NqOnGg6nmzqoIdOZysDwJPQ0inYplQ2bE/PY5t6Htu0lOR4o445OmlEtqGfp8qnZOYhI9c2PFMnhiPuRER+zeCYKl/JloNr2bKlCtg3bdpkf2z9+vVqpEavd2+3ZO12ea3m+PHj6iKPExEREVX0Ou7a/PbqUSEIDWInDBGR34+4W72bKu+1wF3S2GX5tldeeUWtw7506VLMmDED99xzj330Xeauu+L222/HTz/9pJaXk2r1zz77LK644grUq1evnD8FERERkfMc96jgKBw+kz+/nWnyREQBtRwcKlvgLsaMGaPWcB8yZAjGjRuHUaNGoV+/fupnPXr0wMKFC13ajiwp9+qrr+LDDz9UQXyVKlXwxhtvlPPeExERERU94n7YXlGeafJERH7PGGxPlddVtuXgtFF3WZtdLgXt3LmzyNfceOON6uLq40REREQVPcfdXpiOS8EREfk/Y6hDqrzMca9kI+5EREREgTnHXUuV54g7EZHfMwSfT5WHWStTV6EYuBMRERF5eo67PVWeI+5ERAExxx1wSJWv+F1g4E5ERETkwRH3cEP4+RF3Bu5ERAFYnM5a4bvAwJ2IiIiojKxWq33EPScvGLkmC/Q6oFZMqLd3jYiIysoYjKDKXFWeiIiIKBBkmbJgsVrU7ZR0nbquVSUMQQaeahERBVKqPBi4ExEREfl3mrxBZ0DSWdsZHZeCIyIKvOJ0OlaVJyIiIvLvpeCkonxyeq66XSOaafJERIEzxx02nONORERE5N8j7rKGe0qmLXCPiwj28l4REZFHGEMQZF8EjqnyPivEqMes1Qe9vRtERETk62u4B0fgTEaeuh0bzsCdiCggGEMKVJWv+F1g4O4iqQ5LREREVOIa7kFRSMnQRtyDvLxXRETksRF3LW5nqjwRERGR/89xP5OfKh/LVHkiosBgCIFRS5VncToiIiKiAJjjro24M1WeiCgwGEO5jjsRERFRIM1x14rTccSdiCiQ5rjDhoE7ERERkX/KyLXNcY80SlV5W3E6VpUnIgqgwB3567irqvKc405ERETktyPuRl04zPlDMTHhLE5HRBRoxenAOe5ERERE/h2466wh6joyxIgQo8HLe0VERB4rTucwx93MwJ2IiIgqk5ycHDz//PPo3LkzevTogRkzZhT73BUrVuD6669Hhw4dcO211+L333+HrwXuFnOouo7lUnBERIGZKq8zwQuZ8gzciYiIyHsmT56MrVu3YubMmRg7diw++OADLF68uNDzduzYgZEjR+Kmm27Cjz/+iMGDB+Oxxx5Tj/vSHHeL2TbizoryRESBWZxOrzN5ZY67scLfkYiIiAhAZmYm5s6di2nTpqF169bqsnv3bsyePRv9+/d3eu78+fPRvXt33HPPPep+gwYNsGzZMixatAgtWrSAr4y45+XZAnZWlCciCiAGmeNudQjcK34XGLgTERGRV8houclkUqnvmk6dOmHq1KmwWCzQ688nBt5www3Iy7NVa3eUlpYGX6AF7jm5kiJv5og7EVEgMRhh1NmOSXowcCciIqJK5NSpU4iNjUVw8Pkgt2rVqmree2pqKuLi4uyPN2nSxOm1MjK/atUqlTLvLrPZXMY9P78N7To91xa4Z2bJqZUZMeFGj7xPICnYZnRhbDP3sL3cxzZznVFnC511ettycJ48lrj0/mV+NyIiIqJSyMrKcgrahXY/Nze32NedOXMGo0aNQseOHdGnTx+33zcxMbEUe1v8tqxWKzLybHPcjxyXDIBgZJ89jU2bNnnsfQKJJ9u/smCbuYft5T622YXp88vD6fJH3Cu6zRi4ExERkVeEhIQUCtC1+6GhtursBSUnJ2Po0KEqWJ4yZYpTOr2rEhISYDAYyjxKIidtsq1sSzas22x5k0Gh1QCcRasmDdC+fb0yvUegcWyzsrZ/ZcE2cw/by31sM9f9u9J2XNLlr+PuyWOJKxi4ExERkVfUqFEDKSkpap670Wi0p89L0B4dHV3o+SdPnrQXp/viiy+cUundISdanjpBle1k5WXZbusMOJupU7erRoXwJLgC2r+yYJu5h+3lPrbZhQUZpIaJFcgvTlfRbcbl4IiIiMgrWrZsqQJ2x5Ty9evXq1GMgiPpUoH+/vvvV4/PmjVLBf2+QkuTjwiKQGqGrYBeLIvTEREFFKM+2GHEncvBERERUSURFhaGQYMG4ZVXXsGECROQlJSEGTNm4I033rCPvkdFRakR+E8++QSHDh3Cl19+af+ZkJ/JcyraisMr8GfSn1ibuBbJ2cnqscigSJzMtKX6x3E5OCKigGLUy4h7DqCT4nReeP+Kf0siIiIimzFjxqjAfciQIYiMjFRF5/r166d+1qNHDxXE33jjjfj111+RnZ2NW265xen1skzcxIkTK3Sf03LT8MQfT8AqKZNJ5x+PDY3Frqz8EXcG7kREASXIECJriMCaP8e9ojFwJyIi8rYNs1CZR90nTZqkLgXt3LnTfnvx4sXwFVHBURjdZTT+2fMP4uPjodfpodPp0LNWf6z+2zb6HhMmIzNERBQojEZbh6wVTJUnIiKqnEw53t4DctPg5oPRIqsF2rdvby9OtCdJloL7E1XCgmA0sIwQEVEgMcqIu0UmuXtnxJ1HFSIiIiIPOJNfmI7z24mIAo/RaFsOzsLAnYiIiMh/ncmwFaaLDWeaPBFRoAnKrypv1VkZuBMRERH5qxRWlCciCvgRd6vOCrNVcuYrFgN3IiIiIg+OuMdwDXciooBjNEpVeRuz1Vzh78/AnYiIiMgDUvIDd464ExEFniBjmFcDd1aVJyIiIvKAM/mp8rEccacimM1m5OXluf0akZ2dbV+9gMqnvYKCgtjGVKKgIAbuRERERAE04s7idHSe1WrFiRMnkJqaWqrXGo1GHDx4EDqdrlz2L5CUtb1iYmJQs2ZNtjUVyWiwzXEXZqsJFY2BOxEREZEHnMm0jaZyxJ0caUF79erVER4e7lZQKIFoVlYWwsLCGEyWY3vJ6zIzM5GUlKTu16pVqxz3kvyVLigUBqsVZp2OI+5ERERE/opz3Kmo1G0taI+Pj3f79RJQWiwWhIaGMnAv5/aSYF9I8C7fF9PmqRBjCIxa4A4WpyMiIiLy68A9loE75dPmtMtIO/k+7XtytxYBVRLGUGgToVhVnoiIiMgP5ZosSMuxzXmMY6o8FcDRcv/A74lKZAhWI+7emuPOwJ2IiIiojFLzK8rrdUB0GIvTEREFHGMojLa4HRamyvuuYIMes1Yf9PZuEBFRIFn1EbBmurf3gjy4FFxMeDAMEr0T+bnevXujefPm9kvr1q3Rv39/fP755x5/r/fffx933323x55HVC6MIQiCNuJuqfi3r/B39PM0OCIiIo8x5diuDSHe3hMqozPa/PZwjrZT4Hj++edx9dVXq9smkwmrV6/GCy+8oJZNGzRokMfeZ9iwYS4F5K4+j6g8i9MJCyq+DgJH3ImIiIjKKCXDdhLHivIUSKKiolCtWjV1kSXSbrjhBlx88cVYsmSJR98nIiJCdQZ46nlE5Re4w2sj7gzciYiIiDyUKs813MnldcNzTS5ezG48t/iLvKcnGI1GBAUFqZHv1157DX369MEVV1yB9PR0HD9+HA899BDatWunUu0/+OADtSSe5s8//1TBv/z8uuuuw6pVqwqlwEtF9xdffBHdunVDhw4d1PZOnjxZ6Hli48aNuP3229G+fXv1fl9//bX9Z8899xzeeOMNPP744+r9evbsiR9//NEjbUCVlCEExvxUeW/McWeqPBEREVEZcQ13cpUE0DdPXYX1B1Mq9H07N4jF3IcuLnXldAmoly9fjpUrV2LChAmYO3cu5s2bh+nTpyM4OFiNhg8ZMgQtWrTADz/8gFOnTuHll19W7/fII49g9+7dePjhh9VtSb+XUfsRI0YUGr2fPXs21q5dixkzZqj12F955RX1fu+9957T8/bu3ave795778X48eOxefNmjBs3DpGRkRg4cKB9W4899hieeuopfPHFFxg7dqzqaJBMAqJSLQenpcpbK1ngnpOTo/7A5A9W/jBl3opcirJ9+3b1x7Zr1y40bdpUva5Nmzb2n3fu3BlpaWlOr9mwYYP6T4SIiIioQua4M3AnF/hL+UI595ZRdZGdna3O1yVYltFyCdxlpL1jx47q5zJ6fuzYMfW4Xq9H48aNMXr0aIwZM0YF69999516rgTr4sEHH0RmZibOnTvn9J5HjhxBSEgI6tSpo9LiJ06ciNTU1EL79u2336JVq1Z48skn1X15PwnmZ86caQ/cpajeAw88oG5LAC/Bu3QgaPtMVNpUeQtMlStwnzx5MrZu3ar+wOQPXf64a9eurSpWOpI/avnjvvbaa9Uf75w5czB8+HD89ttvCA8PV+kzErQvXbpU/YeikZ8RERERlbeU/FR5ruFOFyIj0DLynZVndi2lPjML4eFhZV5jPCzI4PY2Hn30UfTr10/dlmBa5robDAb7zyW41kjQLAF2p06d7I9ZLBYV8KekpGD//v2qMr0jSWMv6LbbbsOCBQvQo0cPdO3aFX379sWNN95Y6Hnyfm3btnV6TFLrJU7QNGzY0H5bRuK1IntEpQ7cK2OqvATj0iM3bdo09UcsF+kBk5SWgoH7woUL1X8Wzz77rPoPR6pZyhyZxYsXqz9k+cOV/0jq1avnrY9DRERElVhKpq04HUfcyRVyPhsefOHTcDUv3WRQzy1r4F4a8fHxaNCgQbE/l/NzjQTEMur90UcfFXqepKbL3HhXNGvWDMuWLcOKFSvU5e2338b8+fNVjFDce2tkPr10FmhkLn5BnprrT5V0OThrJVzHfceOHeoPXHrGNNJDJ/NTHP/ghDwmP9P+w5JrSXHZtGmTur9nzx40atSofHc4eVf5bp+IiIgCYI47l4OjyknOxSWDNi4uTgX7cpG09ylTpqhzd7kv5/+OBg8erEbXHUkBOZlLP2DAAEyaNAmfffYZ1q9fj9OnTxd6P4kRHElsUFJHA1GZi9PZ57hXolR5KVgRGxurilloqlatqua9S5qN/NE7PlfmtRfsAZQReiEj7llZWarKpKThtGzZUq076W4wL710jpUvrbDAapULYLWY1W3Hn1PxtHZie3kG29Pz2KaexzZ1n06Wk1EXdaCxXaCDtcDxiPxpHXeOuFPlJKntkjr/zDPP4IknnlDTWF966SVccsklKr1eqr9LUbr/+7//UxXgJXNWzuWlTtW+ffvs25HXTZ06VcUJdevWxS+//IKaNWuq+47uuOMONWddRuSlUr0E7V999ZXK0CUqvxF3W+Cu88I67l4L3CXQdgzahXY/NzfXpedqz5M/9rNnz6riFDJ/RdLvpcKk9OBp81lcIQXwNPIfTHJyHs5m5iE0yIB4UzZSzpxBYmIaT6bckJiY6O1dCChsT89jm3oe29Q1cpypcfIkdEFhMIZEwJxxBlZjCAx6A44mJvJY469z3JkqT5X4/7SPP/5YFbO79dZbVa0pmf4qNaxE/fr11XJu//vf/1SwLSnxEqDXqFHDaTt33nknTpw4oToA5PxeilHLdh3n1gupi/XJJ5+omllSgV7uy3tJ4TyicmEMtQfPlSpwl3kpBQN07b5jgbmSnqs9T5ahkCUqtAryb731llqrUdJspKCdq6QypWMHQWLWIQRn5SHEqEdISgri4+KQkHC+yAUVT0445eQ9ISGh0H+05D62p+exTT2Pbeo+3bkaqgcfhlAgwmC7DR3iEhLUcc6xQ5l8V3aerLNt62jhHHcKFDLPvCRffvllocek3tSnn35a7Gt69eqlLgWNGjXKflsq0kvQLpeSnicuvvhitfScczG/THVbCloXtHPnzhI/E1GJDMH2VHk9nGPTgA7cpXdNKkzKPHetWIWkxEswHh0dXei5ycnJTo/J/erVq6vbEmw7BtwS6EtqjVSbd4ecaDqebOqgh06Xf5F/Oj1PRt1UsE2pbNiensc29Ty2qRt0+vyL7vxtWSiKbeiXo+1GvQ5RIV5dsIeIiMqLXg9j/mKOOmtu5SlOJ/PQJWDXCswJKTwhIzXS0+aoXbt22Lhxo70KpFzLGu3yuNyWZSLmzZtnf770tB08eFBVtiQiIiLytL/3JOOXXRmYsfIAPl95wD7a7o3K30REVDEM+eFzpUqVDwsLw6BBg/DKK69gwoQJSEpKUvNT3njjDfvouywdISPwMj9G5sOMHz9eVZ/8+uuv1bx3qTYpB8grrrhCzZmRghhS1O69995TRSwkXZ6IiIjIk85l5+Hez9epmoLYfL5KdrXIwstTERFR4DDqDJUvcBdjxoxRgfuQIUNUETmZt9KvXz97ZUoJ4mWddvmZFJ8YO3Ysvv32WzRv3lzNn5GiF0LmwMjo/VNPPYX09HR0795d/ZxphkRE5LM2zPL2HlApRYcG4Zl+F2HVf4dUpWsZRNDrdLilc11v7xoREZUjg5rSJoF7JVoOTht1l/UZ5XKh4hFt27Z1Kj7hSOa0P/fcc+pCRETkF0w53t4DKoPhlzdGt+hzaN++HQcKiIgq2Yg7dJVojjsRERERERGRvzDqjF4bcWfgTkRERERERHQBBvscdwbuRERERERERD7HkD/iDh0DdyIiIiIiIiKfE6TXappUsuJ0RERERETke3r37o2jR4/a78vqCdHR0ejUqRNefvll1KpVq9zeWys4PXHiRLXk85o1a/Dll1+W2/sRucqgC1LXOp0ZFY0j7kREREREVMjzzz+Pv//+W13++OMPvPPOO9i9ezdGjx7t7V0j8gqjwRa4AxUfuHPEnYiIiIiIComKikK1atXs92vUqIFHH30UzzzzDNLS0tTPiSpdVXmr3OKIu08LNugx/a996kJEREREVCpWK5Cb4eIl043nlnCR9/SA4OBgda3X63Hu3DkVxHfs2BE9evTAa6+9huzsbPtzt2zZgttvvx3t2rXDVVddhQULFth/NnfuXPTv3x9t2rRBt27dMG7cOJjNFR8MEbnDaLD9/sMLqfIccXdTjsni7V0gIiIiIn8lAfSMq4DD/17wqToAEZ5633rdgWGLZXJuqTdx6NAhfPrpp7jssssQERGBUaNGIS8vD3PmzEFOTg5ef/11vPrqq5gwYQJOnz6NYcOG4brrrsP48eOxadMmlWLfpEkTFfDLc9988020atUKW7duVR0AF198Mfr16+epT0zkcUa9LXC3MnAnIiIiIgp0pQ+eK9LYsWPVKLowmUwICgpCnz591Nx3CeKXLl2qCsdpKfPy3EGDBmHMmDFqdL1KlSp48cUX1eh848aNcfbsWTUiHx4eroJ5LUivW7cu/u///k/Nn2fgTv4w4m5FxQ/mMnAnIiIiIqooMuItI995mRd8qtVqRWZmFsLDw1RV9zIJCnd7tF3ms0sgnZGRoaq7S5X5p556CrGxsWoE3WKx4PLLL3d6jTx28OBB7N+/X42mS9CuGTp0qP12aGgopkyZgj179mDnzp3qNZJuT+QfqfKWin/vCn9HIiIiIqLKTALo4AjX0upN8lz3g25PiI+PR4MGDdTt9957DzfffDNGjBiBb775Rs1Hl5H277//vtDrpIid0Vh8mPHXX3/hkUceUaPzknYvt2WOO5GvCzKEqGuLFwJ3FqcjIiIiIqILFqWTeen//fcfPv/8czRq1EhVlpdMAAnu5SJp8JMnT0Zubi4aNmyoRtIla0Dz+OOP47PPPlOF6W666SY1H/6WW25R894l9d7xuUS+KMgY4rVUeQbuF3J8C3Bim5TOBJJ3eXtviIjIX62dcf72hlne3BMiolJp27atGnX/6KOPEBkZqUbLn376aVU9ftu2bWpue2ZmJqKjo3HttdciNTVVBfIHDhzAvHnz8Pvvv+PSSy9FTEwMNm7cqAJ7mdf+3HPP4dSpUyrgJ/JlBocR94ruaGLgfiFWqRiYXzXQworyRERUSqacom8TEfmRJ554QhWpk4rwEpRLYbl7771XzV+XUfi3335bPU+C908++QTr1q3DwIEDMW3aNPzvf/9Dy5YtMXLkSJWGf9ttt6nXhYSEqGXjZDSfyJcFG0PVtVVnhdlSsYE757gTEREREZGTZcuWFfl4XFycqiSv0QL1onTo0EGlxRdUvXp1TJ8+vdjXTZw40X5blpwj8hVBQWHq2iKBewXP7OCIOxEREREREZGLc9wtMsu9gkfcGbgTERERERERXUCQ0TbiblYj7gzciYiIiIiIiHxKcLAWuEv5MwbuPi/EqMes1Qe9vRtERERERERUQYKDwtU1R9z9SK6JFeaJiIiIiIgq3Yg7UOFV5Rm4ExEREREREV1AkH3EnanyRERERERERD4nKChCXVt0OuRaZNy94jBwJyIiIiIiIroAY/6IuzCZ81CRGLgTERERERERuRG451Rw4G6s0HcjIiIiIiKf17t3bxw9erTQ4x07dsScOXPs9z/++GMcPHgQEydOrOA9JKp4xvxUeZFrYuBORERERERe9vzzz+Pqq692eiwoKMh+e/78+Xj//fdx3XXXeWHviLw74p6Xm16x712h70ZERERERH4hKioK1apVK/S4yWTCa6+9hh9++AH16tXzyr4ReYMuKAxGqxUmnQ55eZkV+t6c4+5ta2d4ew+IiKg8rfoIWDPd23vhs3JyctSoXufOndGjRw/MmFH8cXH79u245ZZb0K5dO9x0003YunVrhe4rkadYrVZk5mW6dMkyZbn83JIu8p6ekpmZiZ07d+Lbb79Fhw4dLvj8vLw8vPjii+jWrZt6/kMPPYSTJ0/af/7TTz+hf//+6m978ODB6m9dM2/ePAwYMABt27bFjTfeiLVr1zql87/55pvq/45Bgwapz7hnzx7cc8896vlXXXUVZs+e7bHPTQRDEILy/5ZMFRy4c8Td20w53t4DIiKqkP/ndV7eEd80efJkFYDPnDkTx44dw+jRo1G7dm11El8wUHjwwQdx7bXXqrm0Msd2+PDh+O233xAefj51kcjXSXB5z6J7sOnUpgp93w7VO2Bm/5nQ6cr+f1F0dDS+/vprl58vwbME3NIxFxoaildeeQUTJkzAe++9h7/++gsvvPCCulxyySX48ssv1d/277//rlLxZWR/7NixKhCXIF7+H1i8eDFq1Kihtv3LL79g+vTpql2lI3DUqFEqwJfX7du3Dy+99BIiIiJUYE9UZjodDPl9YHm5DNyJiIioEpBgfO7cuZg2bRpat26tLrt371Yn+QUD94ULFyIkJATPPvusCjzkJP/PP/9UJ/Bykk7kTzwRPFcECZglAHa0cuVKtzvLjhw5ov5+69Spg5iYGNX5lpqaqn72zTffYODAgbj99tvVffkbl3n0Z8+eVUH83XffbQ+6n376adUBMGvWLDz11FPqMZlf37x5c3Vb/j+JjY3FY489ptq4YcOGqsDeF198wcCdPB5Am8wM3ImIiKgS2LFjh5or65hq26lTJ0ydOhUWiwV6/fkZfZs3b1Y/0wIeuZbq1ps2bWLgTn5Ffndl5FtS4C9ERpGzsrIQFhZW5mA/zOj+Nh599FH069fPeTthYSW+5ueff1YBv2bcuHG47bbbsGDBApXS3rVrV/Tt29f+d7t//36VHq8JDg5WmTdi7969eOSRR5y23759e/W4RjoDNPK4dP7J/w0as9kMg8Hg1ucmKokxf8TdlHfhv2FPYuBeSsEGPab/tU/dvu+yxt7eHSIiIr9z6tQpNTomJ+qaqlWrqnRXGY2Li4tzem7Tpk2dXh8fH69O0t0lJ/JlpW3DE9uqLCpjm8lnleBbuxQMpC9EvcZYuqC72O258Vz5G6xfv36J29Fua9e9evVSae2Of6eRkZEq9f2PP/7A8uXL8fbbb6s0eBk5NxqNRbaPkFH6gu8nberYrvL/h/Zz6Qjs0qWLSsUv2F6ufHZtm9p7VAaV8e+yrLRuoLzcrDK3mzuvZ+BeBjkmi7d3gYiIyG/JSKJj0C60+7m5uS49t+DzXJGYmFiq/S3vbVUWla3NJDCV31/JIikteX1FkwBW/r5kSosrgYf2PMmUKViJXorYSfq7FIu7/PLLccMNN+Dee+9VKfR169bFtm3b7K+X7V1//fV4/fXXVafBunXrcPHFF9u3tXHjRpWlI88vuI+yLekgkM4CbZRdRvql2N0zzzxzwc8snYZSSE+ygSqbyvZ3WRZV84KRZMiDJTtEZX1VFAbuRERE5BUymlYw8NbuSwErV55b8HmuSEhIKHPqrAQXcqLriW1VFpWxzbKzs3Hw4EGVXl6a31VPpsq7S95POscuNJ9d+y5Lep4ExFOmTEHNmjXV8nFSVFJuSyHKIUOG4L777lMV5yXFXea1C7ktj0s9ixYtWtiL0+3atUsVtZT3K7iPstrEJ598oubQy2sPHz6Mt956C0OHDnVpXr50OkgHg2T3lOb78keV8e+yrN6p9zNWrl2KgX1u9dixxBUM3ImIiMgrpCp0SkqKSm+VUUktJV5OmKVqdcHnJicnOz0m96tXr+72+8qJlqdOUD25rcqiMrWZfE4JLrVLaZX19WV5zwu9r2PdieLcddddavk3KTwnRefatGmDjz/+WP3dy5x3mRP/0Ucfqb9/+ZnUuZDOiquvvlr9nUvQLz9r2bKlqkzfpEmTIvdRUvLff/99vPPOO6oYnRTCu/POO1WVelfaT9tWZfod1VTGz1xa1eNro1HNthXeZgzcS5K8y/m+FMk5vgUICpKhACCyEUKMesyavxR3Dezr+rrtXYaVy+4SEZEPMwTb1nQ32OZsEtRJuJy4S6qhrOMu1q9fr0Z+HAvTCVnfWarPywiknFjL9YYNG9R60ETkecuWLXPpeTK6fSHy9yyp6sWlq998883qUhRZk10uru6j/L8ic+f9pXI/kaucj4rkrKi5SFaZx2N2+lludrbr2+S67URElZccA8w8DmhkRE1GxaSQ1JYtW7B06VI1mqadpMsIm6QaC1ke7ty5cxg/fjz27NmjriWFeMCAAV7+FEREROWPgbsHBAcFYdbqg97eDSIiIr8zZswYtX67zHOVZaNGjRplX35Klo6S9du1FFiZuyoj8rKMlCwP9+mnn7q9njQREZE/Yqq8h+SywjwREVGpRt0nTZqkLgXt3LnT6b4Up/rhhx8qcO+IiIh8AwN3D6/rHmzUqzk1d3Vv4O1dIiIiIiIiogDAVHkPr+uuRt6tUEG8XIiIiIiocpNiiuT7+D2RL2PgXo5BvFyIiIiIqHKSNcFFZmamt3eFXKB9T9r3RuRLmCpfWo5LwwUHF15GLrJR8csBOQqOtC0PJC4e4drScdrjjq8rahsFX+/JpeiK2+fKhG1Q8QJhOUV//r0JhPb31Gdy9XUbZpVqt4goMMgaz7KWeFJSkrovxRTdWaZMRoBzcnLUcmpc3qz82kteJ0G7fE/yfXE9c/JFDNzLQi0Np3dtGbl83ybXR1p+Cv19lzXGrKPVcVe1tKKXiStu6Tjtccefu/J6Ty5Fx2Xt2AbeEAht7s+fwZ/33dOfydXXBWKbEZFbatasqa614N0dElDm5eWpEWAG7uXfXhK0a98Xka/xauAuPWKy9MuSJUsQGhqKYcOGqUtRtm/fjrFjx2LXrl1o2rSpel2bNm3sP58/fz7effddtearLB/z2muvIS4uDr4mNzfPKYVe7ivGELWkHIvaEREREQUOCSBr1aqF6tWrq6DSHWazGTt27FDnvhwFLt/2kmCfbUy+zKuB++TJk7F161bMnDkTx44dw+jRo1G7dm3079/f6XmSuvLggw/i2muvxcSJEzFnzhwMHz4cv/32m0o52rJlC1544QUVzLdo0QLjx49X68LKeq/eFGLU49u1h9XttOy8/IrzBuef6Q3FLykXHGkvcCej80RE5OeMIYAhxNt7QUReIEGhu4GhBKJCBrgYVF4Y24sCmdeK00kwPnfuXBVwt27dGldeeSXuv/9+zJ49u9BzFy5ciJCQEDz77LNo0qSJes3/t3cn0DGd7x/AnyySKLXF0lO0SsVOFkpRHLSIfWuD1hZapehir6VESxtV0uqhWkrT04pa0lYRtLSl4l9LiNpia9PGEntFROT+z/f1u9ObBXNjzExmvp9zxmTuvbn3nddknne/RYoUkXXr1qn90dHR0q5dO+nSpYuquKNBYMuWLfLXX7cqzY6EyjgelhXnc+zL65ZyemU95vRDlt559MZbePnceo0CoJ1lS8dd4Atz8bYTXF2fiNyXcV0TNMb++ZB8dtL/1vomRERERM5ecccwlszMTAkKCrJsCwkJkYSEBMnKMUcc27BPn6uC5+DgYNmzZ49lf/369S3HYzgSeu6xvUDw+q8HRq+oozc+44ZhSH1elX78zp0q7zkXwsv5Oh9yjQq4i+s3bnJ1fQcy09BCRLaHdUz0XvaYlNJyPfNWb1B0MivvREREVAAq7piLXrJkSfExrMheunRpNe/94sWLuY7FvCAjf39/OXXqlGWxjzvtd2Y+hbxV78sXyWWtryB7+dzqnT/pr3pvckIP9xe/nZDoUxWybY9OediqNKHRwJkrfMZRCWb23Qtnzg9bNrQQ0d2/U9T3wR0aQvXvYNUAm21Nk8z/DtK0W5V3Y+OrPozeBo2sRERE5FocNsf92rVr2SrtoL/OyMiw6lj9uPT09Dvut2YFyryu61PYR/wyC4tPoULi43vr/H5+/722/OznI+KVJX7eIj7e//s9/WevW4W22+7zRTqvi4enh3qdbZ/hWA/RLOn74cKj4lMsSzI88d+XqbZ7+BS79V7ws2QJxiZ4eHmqfct3JkshLw/x8Pa2nCM24R/pXO+/ivwP+1LkasZNdRx4aIbrGfb5emm5rmeknxdzjLy9vcUP1/XMnbf6OaFnSPYGhpyQfv04/B7eH9YLWP5/J7K9B5VuuZXfxuvlfK85z6m/Vnnk4ZHrWPVZ8MzKdT1jHuR1jZzXu935c6bldv8nNR/wVe/LzJwt/f/rdvmQH8bzGNN4u7zLj9vliYefv8T8dsyqz83d4DOKhWjM5um9MvO5cZTbpcvD58Fcf/PW5qkt3qut8sv4+brbe8rr+8ZDuyk3/Eqqz2LOz77+HaX+ajVNfH195EahB0U8fXLFlBs3Mm/tA58iEnPcTwr5+oiHp7d0zsiw/O3qMYrunZ6X+jzYe6GfwxbnchfMM/OYZ+Ywv8xjnjk2z/RzWBPrPTQHlQjWrl0r06dPl61bt1q2HT16VEJDQyU+Pl7djkGHhekCAgJk1KhRlm2RkZHq+Pnz50tgYKBERUVJs2bNLPt79uypzjVgwIC7pgWFo3379tn0/REREdlCnTp1cjVOU/4w3hMRUUGN9Q7rcS9XrpxcuHBBzXNHz6w+JB6rQBYrVizXsampqdm24bU+PP52+8uUKWNVWnB9ZJanJ1Z95z0yiYjI8dCujjVf9BhJ947xnoiICmqsd1hpoEaNGiqBWGBOX1hu586dloBqVK9ePVm4cKF6Ywi0eN61a5cMGTLEsh+/261bN/U6JSVFPbDdGrgeezOIiIhcG+M9EREVVA5bnK5w4cLq9m1vvfWWug/7xo0bZdGiRdK3b19L7zvmrgPu63758mV1f/akpCT1jHnvuAUc9OrVS2JjY9Xt5bBaPW4b16JFC6lYsaKj3h4RERERERGRTThsjjug8o2Ke1xcnBQtWlTCw8Olf//+al+1atVkxowZll50VO6nTJmi5rVj39SpU6VmzZqWc61cuVLNc7906ZI0adJEIiIi1Kr1RERERERERAWZQyvuREREREREROSkQ+WJiIiIiIiI6O5YcSciIiIiIiJyYqy4ExERERERETkxVtyJiIiIiIiInJjbVNyvX78uEyZMUPeMb9q0qbr13O388ccf0rNnT3Uf+O7du0tiYqJd0+qKebp582bp3LmzBAUFSceOHWXTpk12Taur5acuOTlZ5Wl8fLxd0ujKeXro0CF1a8m6deuqz+j27dvtmlZXzNMNGzao23biM4q83b9/v13TWpBkZGRIhw4d7vi3zNhkfyw7mMeygXmM/+YwtpvH2O0i8VlzE9OmTdM6duyoJSYmanFxcVpQUJC2du3aXMddvXpVa9KkiTZz5kwtKSlJi4iI0Bo3bqy2U/7y9MCBA1qtWrW0JUuWaCdOnNCio6PVa2wn8/lpFB4ergUEBGjbt2+3WzpdMU8vX76s/s4nTpyoPqNz587VQkJCtNTUVIek2xXy9PDhw1qdOnW0VatWaSdPntSmTp2qvlvT0tIckm5nlp6erg0bNuyOf8uMTY7BsoN5LBuYx/hvDmO7eYzdrhGf3aLijozDh9CY4fPmzdOef/75XMcuX75ca9mypZaVlaVe4/npp5/WVqxYYdc0u1KeRkZGqgBjNHDgQG327Nl2Saur5acuNjZWCwsLc9vAbcs8RcGxdevWWmZmpmVbt27dtM2bN9stva6Wp4sXL9a6du1qeX3lyhX1Wd27d6/d0lsQHDlyROvUqZMqUN3pb5mxyf5YdjCPZQPzGP/NYWw3j7HbdeKzWwyVP3jwoGRmZqohH7qQkBBJSEiQrKysbMdiG/Z5eHio13gODg6WPXv22D3drpKnXbt2lVGjRuU6x5UrV+ySVlfLT7hw4YJERkbKtGnT7JxS18zTHTt2SKtWrcTLy8uybcWKFdK8eXO7ptmV8rREiRKSlJQkO3fuVPtWrlwpRYsWlUceecQBKXde+Ow1bNhQli1bdsfjGJvsj2UH81g2MI/x3xzGdvMYu10nPnuLGzh79qyULFlSfHx8LNtKly6t5ntcvHhRSpUqle3Yxx9/PNvv+/v7y5EjR+yaZlfK0ypVqmT7XeTlb7/9JmFhYXZNs6vkJ8ycOVMVeqpWreqA1Lpenv71119q/tukSZPkxx9/lPLly8vYsWPVFzHlL09DQ0NVXvbu3VsVmjw9PWXBggVSvHhxB6XeOSF/rMHYZH8sO5jHsoF5jP/mMLabx9jtOvHZLXrcr127lu3DCvprLDhgzbE5j3N3ZvLU6Pz58zJ8+HDVEoVWUDKfn9u2bVMtoUOHDrVrGl05T9PS0uSTTz6RMmXKyMKFC6VBgwYSHh4uKSkpdk2zK+UpeoUQzCZPniwxMTFqAarx48fLuXPn7JpmV8HYZH8sO5jHsoF5jP/mMLabx9h9f9nz+98tKu6+vr65Mk9/7efnZ9WxOY9zd2byVJeamir9+vXDugoSFRWlWvHIXH6mp6erL9MpU6bwM2nDzyhalWvUqCEjRoyQmjVryujRo6VSpUoSGxtr1zS7Up7OmjVLAgICpE+fPlK7dm2JiIiQwoULq2GKZB5jk/2x7GAeywbmMf6bw9huHmP3/WXP73+3+HYsV66cakHC/A4dWpOQocWKFct1LIKIEV6XLVvWbul1tTyF06dPqy8BfJCXLl2aa+iXu7M2P/fu3auGfiEIYa6SPl9p8ODBKqBT/j6jaI2vXLlytm0I7u7WKm/LPMXtY6pXr255jcI4Xv/zzz92TbOrYGyyP5YdzGPZwDzGf3MY281j7L6/7Pn97xYVd7S2eXt7Z1skAEON6tSpk6tlF/ff2717t2r5BTzv2rVLbaf85SmGKg0aNEhtj46OVh9wyl9+Yq5WXFycrF692vKA6dOny8iRIx2Sdlf4jAYGBqp7vRodO3ZMzYej/OUpAtbRo0ezbTt+/LhUqFDBbul1JYxN9seyg3ksG5jH+G8OY7t5jN33l12//zU3MWnSJK19+/ZaQkKCtmHDBi04OFhbv3692nfmzBnt2rVrltseNGrUSN2DD7cBwDPuzeeO92K1VZ7i1i5169ZVx2G7/sD9Ncl8fubkjreDsXWeJicna4GBgVpUVJS61+ucOXPU61OnTjn4HRTcPF2zZo3lXrDIU9z6yV3vn2utnH/LjE2Ox7KDeSwbmMf4bw5ju3mM3a4Rn92m4p6WlqaNGTNG/cE2bdpU3afQ+J9hvNcePtRdunRRH9wePXpo+/fvd1CqXSNP27Rpo17nfIwdO9aBqS/Yn1Ejdw3cts7T33//Xd27tHbt2lrnzp21HTt2OCjVrpOnMTExWtu2bdWxvXr10hITEx2U6oIh598yY5PjsexgHssG5jH+m8PYbh5jt2vEZw/8Y/t+fCIiIiIiIiKyBbeY405ERERERERUULHiTkREREREROTEWHEnIiIiIiIicmKsuBMRERERERE5MVbciYiIiIiIiJwYK+5ERERERERETowVdyIiIiIiIiInxoo7Ed03ycnJUq1aNfVMRETkDlq2bKliHx7Vq1eXoKAgCQsLk19++cUm51+5cqW6hq2OM2vcuHGW95fXIz4+Xh2Dhz3cy7XuVk65X3lIlB/e+fotIiIiIiLK04QJEyQ0NFSysrLk0qVLsnr1annppZfk008/lcaNG9/TuXHeFi1a2Ow4s958801544031M8//PCDLFq0SL755hvL/uLFi8uqVatsfl0id8eKOxERERGRDT344INSpkwZ9XO5cuVkzJgxcvbsWZkxY4Z8991393RuPz8/9bDVcfl5b3joP3t5eVneKxHdPxwqT0R5DhtDweKpp56S+vXry/Tp0yUzM1M+/PBDGTp0qPTp00eeeOIJ2bFjh1y/fl0iIyOlefPmEhgYKEOGDJGUlJRs51y3bp00a9ZMgoODZfLkyZKRkWHZt3z5cmnbtq3Url1bGjZsKFOnTpWbN2864J0TERHdP88995wcPnxYTp48qV5fvnxZRo8erWJj06ZNJSIiQtLT0y3H7927V3r16iX16tWTNm3ayJo1a/Icvj179mz1+3Xr1pUXXnhBjhw5kudxR48elfDwcHU9xPePPvpIjQgAxHf0ok+ZMkXtf/LJJ2XhwoX39H7//fdfee2111T60fNvbLBAulB2QLq7dOkimqapvEH68T7wfr/88kvL8cir4cOHqzJJgwYNZNSoUer81lzLmnKK7vTp0zJo0CB1XNeuXeXPP/+8pzwgsiVW3IkoTwjoH3zwgXqOi4tTQR02bdokHTp0kCVLlqjgiiC/YcMGeffdd+Xrr79WFXxU7vXCAMTExKhzzZ8/X37++WdZsGCB2o6KPxoFXn/9dVW5R6Udw+1wDSIiIldSpUoV9ZyUlGQZcn7lyhX56quv5OOPP5Z9+/bJtGnT1L5z587JwIEDpUaNGmrYOYbZjx07Vg4ePJjtnIi/y5Ytkzlz5sj3338vpUuXlvHjx+e69vnz56V3795StmxZ1WCO2B0dHS1Lly61HLN+/Xrx9fVV10MFf9asWXL8+PF8v1+krVatWipd7dq1U9MH8H51qFx/9tlnMnPmTFW5Hjx4sISEhMi3336r3ivyBFMMICoqSo1YQF4hzcgH7LfmWtaUU3QjR45U25FHSA/KOkTOgkPliShP6AVAy7YeyBDA0fKPQgGeAfP2YmNjVat8o0aN1DYch9burVu3ymOPPaa2IYAiGBvPhZbzBx54QN5++2155pln1L4KFSrI4sWLVW+Bvo2IiMgV6MPLr169qnpyN27cqBqw9e3ocUfvMyre6F3HXPGJEyeKp6enVK5cWcVcY488/P3331KoUCF5+OGH1WPSpEly7NixXNdGhbZw4cLqGt7e3qoRARXhefPmSf/+/dUxJUqUUBVmDH1HrzNie2JioiWWm4VF+XAeQEUZc+GRNvSKQ6dOndQIP0BF2d/fX1599VX1ulKlSuq9oZKOPMHPRYoUUeUEvI+5c+dadS2cx5pyCqDssXv3bvnpp59UXlatWlW9f3QsEDkDVtyJKE8YKqfDMHa01l+4cEHKly9v2X7ixAnVMq0HYT3wIxBiSJ4eENEzr6tZs6akpqaqAgjOi/l3aElHD8ShQ4fUEEIMnSMiInIl+tDuokWLqhiJ+IlpZEbYhjiInm7ES1TadQMGDFDPxop5+/btVc95q1at1PDu1q1bS48ePXJdG9dDjzQq7cbKLirvGIYOqBSj0q5DRRm90/lVsWJFy8964wR61nXG8gTeE3rRkSYdps3p6enbt6+qkGMIPx4YSt+xY8e7XsvacgqgHIJ9qLTr6tSpw4o7OQ1W3IkoT2jB1+nDyVCAwDA6nfFnIwRb4xA0Y8ED89j08+PWOMOGDVOt6Zhvh58xXJ6IiMjVoHEa0JOLn1HBXLFiRa7jsJidsYJ9J1gUbu3atar3GD3FGHqO6Wn6EPM7xWs9Tuvryhjjfs6YnR/GRoC8zmdMExoIUCHHOjh5wb4tW7aoqXSbN29Wx/3666+q9/xO17K2nJJX+m6XJ0SOwjnuRJSnAwcOWH7GUDHMi0NLtBFauFG42LNnj2UbeuXRW2BsxcaCM8bFdh566CE1TB5D47p3767m9PXs2VMN3cPwwXspKBARETkjVNLR643YiRiJOdgeHh7y6KOPqgeGwb/33ntqAVcM8Ubl3hgPMYwct5MzQiUWsRRDv9HwjWHh6GU2xl3A9fbv3y83btywbMOw8FKlSuWK7Y6A9GGUAXr99fxA2eKLL75Q+z///HOVfiwYh2HyWJ0f6+/cjbXlFAgICFCjAfXFA3OWhYgcjRV3IsoT5p5joZxt27apIImV5HPCMDpUuDFnLj4+Xg1zw9x4VMybNGliOQ77ExISVI8AhsUb59Oh4IDCCeaWjRs3Tg3bM646T0REVNCgUo54dubMGRXjEFNxz3PEOUBDNUaaYXV0NGijUoq57WlpaVKsWDE1DPzixYuqIo+KOFaIR2+zMbYCeo1xDBZfw11hcBzmgKPib4TzIbaipxpDxDG/HovOYs0aNB44Gua7o+FCTx9615FnmPcOp06dUo38qIAjP7CQHqYS3I215RT9/wQ9+1iXB8chjzANgchZcKg8EeUpNDRUrWKLQgEC+4svvqgWsckJC9lgpdYRI0aoQkHjxo1Vy7iPj4/lGPz+yy+/rFr6n332WenXr5/a/sorr6iCCm6Rgzl/uFULjmULNxERFWTvvPOOeqBSjF5tVDIRG/VFXwEVbtxZBY3Z6BVGRR6L0QEq77gDC86BXmf0HL///vtqlXljjMRt1RB/0QONhgIsYofV1rGwnRFiLHrrURnG9DSkCbEYcd4ZIH1YQA7vF+lDwz46DPT0YWFbNIagLIHGDdwSDrd4s4Y15RQd7oCDBf7CwsLUXHfcng6NIUTOwEPjmFQiMkCLPRa5Qcs+hqwREREREZFjcag8ERERERERkRNjxZ2IiIiIiIjIiXGoPBEREREREZETY487ERERERERkRNjxZ2IiIiIiIjIibHiTkREREREROTEWHEnIiIiIiIicmKsuBMRERERERE5MVbciYiIiIiIiJwYK+5ERERERERETowVdyIiIiIiIiInxoo7ERERERERkTiv/wffoQP9Bke7HwAAAABJRU5ErkJggg==",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"y_pred = trainer.predict(ds_val)\n",
"probas = sp.softmax(y_pred.predictions, axis=1)\n",
"y_proba = probas[:, 1]\n",
"\n",
"y_true = df_val[\"target\"].to_numpy()\n",
"make_plot(y_true, y_proba)"
]
},
{
"cell_type": "markdown",
"id": "4b1d7883",
"metadata": {},
"source": [
"Il y a deux principaux problèmes :\n",
"1. **Overfitting** : en regardant les performances au cours de l'entraînement, on voit que la loss pour le train baisse mais pas celle de la validation\n",
"2. **Sur-confiance** : les masses sont aux extrêmes des valeurs de probabilités, rendant difficile le choix du seuil\n",
"\n",
"Cela s'explique parce que nous avons ré-entraîné l'ensemble du modèle (les millions de paramètres) avec un learning rate trop aggressif. \n",
"\n",
"## Amélioration de la procédure\n",
"\n",
"Pour contrer cela, nous allons implémenter plusieurs changement:\n",
"* Découper l'entraînement en deux parties :\n",
" 1. Entraînement en gelant tous les poids, sauf ceux de la couche de classification sur quelques époques\n",
" 2. Entraînement avec l'ensemble des poids sur une à deux époques, avec un learning rate faible\n",
"* Ajouter à la fonction de perte du *label smoothing*\n",
"* Adapter les scheduler de learning rate, quand c'est nécessaire\n",
"* Ajouter du *weight decay*, quand c'est nécessaire\n",
"\n",
"Commençons !\n",
"\n",
"\n",
"\n",
"### Intégration du *label smoothing*\n",
"\n",
"Le *label smoothing* est une technique de régularisation introduite dans un [article](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.pdf) améliorant l'architecture Inception. L'idée est d'adresser à la fois le surapprentissage et la sur confiance dans la prédiction.\n",
"\n",
"Concrétement on remplace le vecteur cible $y$ de $K$ classes par :\n",
"\n",
"$$\\tilde{y} = (1 - \\alpha) y + \\frac{\\alpha}{K}$$\n",
"\n",
"Avec $\\alpha \\in [0,1]$ issue de la loi uniforme. De cette manière on évite au logits de devenir trop grand.\n",
"\n",
"**Consigne** : Reprendre la classe `WeightedTrainer` pour ajouter dans la couche de [`CrossEntropyLoss`](https://docs.pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) le label smoothin valorisé à 0.05"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "fdfcf227",
"metadata": {},
"outputs": [],
"source": [
"class WeightedTrainer(Trainer):\n",
" \"\"\"Custom Trainer with weighted loss and label smoothing to handle class imbalance.\"\"\"\n",
"\n",
" def compute_loss(\n",
" self,\n",
" model,\n",
" inputs,\n",
" return_outputs=False,\n",
" num_items_in_batch=None,\n",
" ):\n",
" \"\"\"Compute the weighted cross-entropy loss with label smoothing for class imbalance.\"\"\"\n",
" labels = inputs.pop(\"labels\")\n",
" outputs = model(**inputs)\n",
" logits = outputs.get(\"logits\")\n",
"\n",
" loss_fct = torch.nn.CrossEntropyLoss(\n",
" weight=class_weights.to(logits.device),\n",
" label_smoothing=0.05,\n",
" )\n",
" loss = loss_fct(logits.view(-1, self.model.config.num_labels), labels.view(-1))\n",
"\n",
" return (loss, outputs) if return_outputs else loss\n"
]
},
{
"cell_type": "markdown",
"id": "92f61fa3",
"metadata": {},
"source": [
"### Premier entraînement\n",
"\n",
"Nous avons besoin de réinitialiser le modèle et geler l'ensemble des couches, sauf celle de classification."
]
},
{
"cell_type": "code",
"execution_count": 45,
"id": "83a5631e",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at distilroberta-base and are newly initialized: ['classifier.dense.bias', 'classifier.dense.weight', 'classifier.out_proj.bias', 'classifier.out_proj.weight']\n",
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"592130 trainable parameters, 0.72% of total model parameters\n"
]
}
],
"source": [
"model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME, num_labels=2)\n",
"\n",
"for name, param in model.named_parameters():\n",
" if not name.startswith(\"classifier\"):\n",
" param.requires_grad = False\n",
"\n",
"n_total_parameters = sum(parameters.numel() for parameters in model.parameters())\n",
"n_trainable_parameters = sum(\n",
" parameters.numel() for parameters in model.parameters() if parameters.requires_grad\n",
")\n",
"rate = n_trainable_parameters / n_total_parameters\n",
"\n",
"print(\n",
" f\"{n_trainable_parameters} trainable parameters, {100 * rate:.2f}% of total model parameters\",\n",
")"
]
},
{
"cell_type": "markdown",
"id": "9f95be92",
"metadata": {},
"source": [
"**Consigne** : Reprendre le paramètrage de `TrainingArguments` et lancer l'entraînement. Stocker les paramétrages, modèles et entraînement dans des variables différentes."
]
},
{
"cell_type": "code",
"execution_count": 47,
"id": "f682e048",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/arthurdanjou/Workspace/studies/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:692: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
" warnings.warn(warn_msg)\n"
]
},
{
"data": {
"text/html": [
"\n",
"