diff --git a/M2/Reinforcement Learning/Lab 2 - Maze Game as a Markov Decision Process Part 1.ipynb b/M2/Reinforcement Learning/Lab 2 - Maze Game as a Markov Decision Process Part 1.ipynb new file mode 100644 index 0000000..38b504a --- /dev/null +++ b/M2/Reinforcement Learning/Lab 2 - Maze Game as a Markov Decision Process Part 1.ipynb @@ -0,0 +1,1400 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "44b75d44", + "metadata": {}, + "source": [ + "# Lab 2 - Maze Game as a Markov Decision Process Part 1\n", + "\n", + "## **1. Objectives**\n", + "\n", + "In this lab, we will:\n", + "\n", + "- Model a simple **maze game** as a **Markov Decision Process (MDP)** by defining:\n", + " - **States**\n", + " - **Actions**\n", + " - **Transition probabilities**\n", + " - **Rewards**\n", + "\n", + "- Implement **policy evaluation** to compute the value function of a given policy.\n", + "\n", + "This week, we **do not** improve the policy and search for an optimal one yet. \n", + "We will continue working on the Maze Game **next week**, where we will use these components to compute an **optimal policy**.\n", + "\n", + "We consider a **discounted MDP** with discount factor $\\gamma \\in (0,1)$.\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "100d1e0d", + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "np.set_printoptions(\n", + " precision=3, suppress=True\n", + ") # (not mandatory) This line is for limiting floats to 3 decimal places, avoiding scientific notation (like 1.23e-04) for small numbers.\n", + "\n", + "# For reproducibility\n", + "rng = np.random.default_rng(seed=42) # This line creates a random number generator.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1018deab", + "metadata": {}, + "source": [ + "## 2. Maze definition and MDP formulation\n", + "\n", + "We consider a small 2D maze on a grid. The agent is a **robot** that moves on the grid.\n", + "\n", + "- `S` : start state\n", + "- `G` : goal state, with positive reward\n", + "- `#` : wall (not accessible)\n", + "- `.` : empty cell\n", + "- `X` : \"trap\" (negative reward)\n", + "\n", + "At each step, the robot can choose among 4 actions:\n", + "\n", + "$$\n", + "\\mathcal{A} = \\{\\text{Up} \\uparrow, \\quad \\text{Right} \\rightarrow, \\quad \\text{Down} \\downarrow, \\quad \\text{Left}\\leftarrow\\}.\n", + "$$\n", + "\n", + "The movement is deterministic, but here we set a small probability of “error” to make the example more realistic.\n", + "- With probability $1 - p_{\\text{error}}$, it moves in the chosen direction.\n", + "- With probability $p_{\\text{error}}$, it moves in a random *other* direction.\n", + "- If the movement would hit a wall or go outside the grid, the agent stays in place.\n", + "\n", + "We will represent the MDP with:\n", + "\n", + "- A list of **states** $\\mathcal{S} = \\{0, \\dots, n_{S - 1}\\}$, **each corresponding to a grid cell.**\n", + "- For each action $a$, a transition matrix $P[a]$ of size $(n_S, n_S)$, where\n", + " $$\n", + " P[a][s, s'] = \\mathbb{P}(S_{t+1} = s' \\mid S_t = s, A_t = a).\n", + " $$\n", + "- A reward vector $R$ of length $n_S$, where $R[s]$ is the immediate reward obtained when **leaving** state $s$.\n", + "\n", + "We will use a discount factor $\\gamma = 0.95$.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ca4fa301-c14f-44ec-b04f-b01ca42d979a", + "metadata": {}, + "source": [ + "### 2.1 Define the maze \n", + "\n", + "Let us now define the maze as follows." + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "id": "f91cda05", + "metadata": {}, + "outputs": [], + "source": [ + "maze_str = [\n", + " \"#######\",\n", + " \"S...#.#\",\n", + " \"#.#...#\",\n", + " \"#.#..##\",\n", + " \"#..#..G\",\n", + " \"#..X..#\",\n", + " \"#######\",\n", + "]\n" + ] + }, + { + "cell_type": "markdown", + "id": "99820cf4-292d-49ba-b662-f9f05f901f62", + "metadata": {}, + "source": [ + "**Exercise 1.** Compute the dimensions of the maze (complete the “TO DO” parts):\n", + "- How many rows does the maze have?\n", + "- How many columns does the maze have?" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "564cb757-eefe-4be6-9b6f-bb77ace42a97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "7\n", + "7\n" + ] + } + ], + "source": [ + "n_rows = len(maze_str)\n", + "print(n_rows)\n", + "n_cols = len(maze_str[0])\n", + "print(n_cols)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "26c821d3-2362-4b60-8c77-3d09296d130d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Maze:\n", + "#######\n", + "S...#.#\n", + "#.#...#\n", + "#.#..##\n", + "#..#..G\n", + "#..X..#\n", + "#######\n" + ] + } + ], + "source": [ + "print(\"Maze:\")\n", + "for row in maze_str:\n", + " print(row)\n" + ] + }, + { + "cell_type": "markdown", + "id": "adc49d58-2730-41d8-96fb-ca7c9cb4fcdf", + "metadata": {}, + "source": [ + "### 2.2 Map each walkable cell (not a wall '#') to a state index\n", + "\n", + "Now we convert the maze grid into state indices for the MDP.\n", + "\n", + "\n", + "The cells where the robot is allowed to stand are \n", + "\n", + "- . : empty space\n", + "\n", + "- S : start\n", + "\n", + "- G : goal\n", + "\n", + "- X : trap\n", + "\n", + "Everything else (i.e., #) is a wall and cannot be a state in the MDP.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "7116044b-c134-43de-9f30-01ab62325300", + "metadata": {}, + "outputs": [], + "source": [ + "FREE = {\n", + " \".\",\n", + " \"S\",\n", + " \"G\",\n", + " \"X\",\n", + "} # The vector Free represents cells that the agent is allowed to move into.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c9ad05e-9c6c-4e00-918c-44b858f45298", + "metadata": {}, + "source": [ + "**Dictionaries to convert between grid and state index**\n", + "\n", + "We now want to identify all **valid states** of the maze (all non-wall cells). \n", + "To do this, we need two mappings:\n", + "\n", + "1. `state_to_pos[s] = (i, j)`: Given a state index $s$, return its grid coordinates (row, column).\n", + "2. `pos_to_state[(i, j)] = s`: Given coordinates (i, j), return the corresponding state index $s$.\n", + "\n", + "These two dictionaries allow easy conversion between **MDP state indices** and the **physical maze positions**. " + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "a1258de4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of states (non-wall cells): 22\n", + "Start state: 0 at (1, 0)\n", + "Goal states: [16] at (4, 6)\n", + "Trap states: [19] at (5, 3)\n" + ] + } + ], + "source": [ + "state_to_pos = {} # s -> (i,j)\n", + "pos_to_state = {} # (i,j) -> s\n", + "\n", + "start_state = None # will store the state index of start state\n", + "goal_states = [] # will store the state index of goal state # We use a list in case there are multiple goals\n", + "trap_states = [] # will store the state index of trap state # We use a list in case there are multiple traps\n", + "\n", + "s = 0\n", + "for i in range(n_rows): # i = row index\n", + " for j in range(n_cols): # j = column index\n", + " cell = maze_str[i][j] # cell = the character at that position (S, ., #, etc.)\n", + "\n", + " if (\n", + " cell in FREE\n", + " ): # FREE contains: free cells \".\", start cell \"S\", goal cell \"G\" and trap cell \"X\"\n", + " # Walls # are ignored, they are not MDP states.\n", + " state_to_pos[s] = (i, j)\n", + " pos_to_state[(i, j)] = s\n", + "\n", + " if cell == \"S\":\n", + " start_state = s\n", + " elif cell == \"G\":\n", + " goal_states.append(s)\n", + " elif cell == \"X\":\n", + " trap_states.append(s)\n", + "\n", + " s += 1\n", + "\n", + "n_states = s\n", + "\n", + "print(\"Number of states (non-wall cells):\", n_states)\n", + "print(\"Start state:\", start_state, \"at\", state_to_pos[start_state])\n", + "print(\"Goal states:\", goal_states, \"at\", state_to_pos[goal_states[0]])\n", + "print(\"Trap states:\", trap_states, \"at\", state_to_pos[trap_states[0]])\n" + ] + }, + { + "cell_type": "markdown", + "id": "721b968c-a355-46eb-aae4-5950441ba604", + "metadata": {}, + "source": [ + "*Hint.* If you don’t know what a dictionary is in Python, try the following code to help you understand." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "68744dd6-7278-4c20-8b82-34212685352f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "value2\n" + ] + } + ], + "source": [ + "my_dict = {\"key1\": \"value1\", \"key2\": \"value2\"}\n", + "print(my_dict[\"key2\"])\n" + ] + }, + { + "cell_type": "markdown", + "id": "0c76f4e1-b0ba-49c5-b9d5-cfb523024ba9", + "metadata": {}, + "source": [ + "**Exercise 2.** Read the program above and answer the following questions:\n", + "1. What is the purpose of state_to_pos and pos_to_state?\n", + "2. Why do we only assign states to cells in FREE?\n", + "3. What would happen if the maze had multiple goal cells?\n", + "4. What is the total number of states (n_states) in this maze? Does this match the number of non-wall cells you can count visually?" + ] + }, + { + "cell_type": "markdown", + "id": "4c26a18f-2d03-401c-8eae-f9a17ac55f6d", + "metadata": {}, + "source": [ + "1. What is the purpose of `state_to_pos` and `pos_to_state`? These dictionaries establish a bijective mapping between the mathematical representation of the state and its spatial representation:\n", + "\n", + " `state_to_pos`: Maps the scalar state index `s` (an integer used for matrix/vector operations in RL algorithms like Q-learning) to the grid coordinates (i,j).\n", + "\n", + " `pos_to_state`: Maps the grid coordinates (`i,j`) (used to calculate movement and dynamics within the 2D grid) back to the unique state index s.\n", + "\n", + "2. Why do we only assign states to cells in FREE? In a Markov Decision Process (MDP), walls (#) are obstructions, not valid states.\n", + "\n", + " The agent can never \"be\" in a wall, so assigning a state index to a wall would needlessly increase the dimensionality of the state space (∣S∣).\n", + "\n", + " Excluding walls ensures the transition matrices and value vectors remain compact and contain only reachable positions.\n", + "\n", + "3. What would happen if the maze had multiple goal cells?\n", + "\n", + " In the code: The logic is robust. Since goal_states is initialized as a list (`[]`), the code would simply append the state index `s` of every `G` cell found during the iteration. The list would contain multiple integers representing all terminal states.\n", + "\n", + " Caveat: While the logic holds, the final print statement in the provided script (`state_to_pos[goal_states[0]]`) would only display the coordinates of the first goal found, ignoring the others in the console output.\n", + "\n", + "4. What is the total number of states (`n_states`) in this maze? Does this match the number of non-wall cells you can count visually?\n", + "\n", + " `n_states` represents the total count of walkable cells (Start, Goal, Trap, and empty space).\n", + "\n", + " Yes, this value matches exactly the number of non-wall cells visible in the maze, as the counter s is incremented precisely when a cell is found in the FREE set." + ] + }, + { + "cell_type": "markdown", + "id": "6d0fa298-7b7c-44fc-bbed-15ea002037c2", + "metadata": {}, + "source": [ + "-----\n", + "\n", + "The following function `plot_maze_with_states` creates a figure showing:\n", + "- the maze walls and free cells\n", + "- the state index for each non-wall cell\n", + "- special labels and colors for S (start state), G (goal state), and X (trap state). " + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "fc61ceef-217c-47f4-8eba-0353369210db", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGbCAYAAAAr/4yjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAK99JREFUeJzt3Qd4VFX6x/E3CS1CEiSAofcS+spKEZCmgiBZKQqKiIuCrIDArmBBKSu661IUQYoNpFhoYhCpSlEEFekIiHThTwslEaRm/s97xjlMMAlRk9wp38/zXObOzZA5d+7k/O4pdybE5XK5BAAAEQl1ugAAAN9BKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUowHEhISEydOjQDD+2d+/eEoyaNGlilqz8nfv27TOv8ZQpUzL1eeA/CAUfoX+E+seoy5dffvmbn+unkZQoUcL8/O6775ZA9tVXX5mQOH36tF8+9/jx46lU4bdyOF0ApJQnTx557733pGHDhim2r1y5Un766SfJnTu3BJpffvlFcuTIkaJiHjZsmDz88MOSP3/+bC1LZjy3hkLBggXN78hMS5YskaxWqlQpczxy5syZ5c8F30RLwce0atVKZs2aJZcvX06xXYOidu3aEhMTI4EYhN6hgNTlypXLLFlJW6J6PMLCwrL0eeC7CAUfc//990tCQoIsXbrUbrt48aLMnj1bHnjggVT/z8iRI+XWW2+V6OhoCQ8PN+Ghj/emZ62e7qlrF+/+/AsXLsiQIUOkfPnyplWiXVYDBw4029Pz2muvmYrEu9tl1KhR5vf/85//tNuuXLkiERER8tRTT9lt3mXQ2wEDBpj1MmXK2DJqX7e3efPmSbVq1UwZq1atKosWLZKMGDt2rHn8DTfcIDfeeKP89a9/NYGbkeeePHmyNGvWTAoXLmyet0qVKjJhwoQUv7906dKybds207Lz/H/vPnt9ffr162deV/0d+jq//PLLkpyc/Lv7/1esWGF+/8yZM+XFF1+U4sWLmwq9efPm8uOPP/7m/7/xxhtSrlw58x6pU6eOfPHFF795TFpjCjt27JD77rtPChUqZP5/pUqVZNCgQSkec+jQIenWrZvcdNNN9ri88847v+sYwHmcnvkYrVTq168v77//vtx1111m28KFC+XMmTPSqVMnU/lea8yYMRIXFyedO3c2AfLBBx/IvffeK5988om0bt3aPOaxxx6T22+/PcX/04p0xowZppJTWjHp79ExjR49ekhsbKxs2bJFXnnlFfnhhx9MRZyWRo0amf+v/9cz5qGVTmhoaIrKZ8OGDfLzzz/LbbfdlurvadeunXku3X99Xu2GUVoZeehzzJ07Vx5//HETMPqatG/fXg4cOGCCMS1vvvmmPPHEE9KhQwfp27evnD9/XjZv3ixff/21CdzrPbcGgFZm+hppy2b+/PmmDLrfvXr1Mo959dVXpU+fPpIvXz5baWolqc6dOyeNGzc2lacej5IlS5ruqmeeeUb+7//+z/zfP+K///2veZ2ffPJJ8z753//+Z94Lul8eb7/9tnlOPXnQUNqzZ4/ZjwIFCpiASo++Rnp8tUtJ3xf6Ht29e7fZfw0jdfToUalXr56dCKCvmb5vH3nkEUlMTDTPmZFjAB+g36cA502ePFm/18L17bffusaNG+eKiIhwnTt3zvzs3nvvdTVt2tSslypVytW6desU/9fzOI+LFy+6qlWr5mrWrFmaz7dr1y5XVFSU64477nBdvnzZbJs2bZorNDTU9cUXX6R47MSJE03ZVq9enebvu3LliisyMtI1cOBAcz85OdkVHR1tyh4WFuZKSkoy20ePHm2e49SpU/b/6u8eMmSIvT9ixAizbe/evb95Ht2eK1cu148//mi3bdq0yWwfO3asKz1/+9vfXFWrVk33Mek997Wvs2rRooWrbNmyKbbpczRu3Pg3j33hhRdcefPmdf3www8ptj/99NPmNTpw4EC6ZdPf6f17ly9fbsoaGxvrunDhgt0+ZswYs33Lli32/VC4cGFXrVq1UjzujTfeMI/z/p2637pN348et912m3k/7t+/P0V59Bh7PPLII64iRYq4Tpw4keIxnTp1Mu8zz2uXkWMAZ9F95IO0ma6DfXqmn5SUZG7TO4vS5rzHqVOnzNmintmtX78+1cefPXtW2rZta5ruelbs6T/WsQxtHVSuXFlOnDhhF+0yUcuXL0+zDHqmqmehq1atMve3b99uusGefvppM3NqzZo1Zru2GrTb588MIGuLR7tBPGrUqCGRkZHm7Dc9+pw6WP/tt9/+oef1fp31NdbXRs/89Xn1/vXo66vHRV9379dX90e71Tyv3e/197//PcVYgz6H8rwe69atk2PHjknPnj1TPE67FKOiotL93cePHzfl0m4hbdl401aB0uM7Z84cadOmjVn33rcWLVqY18bzXvyzxwBZj+4jH6RNb60otJ9Vuxy0wtDmdlo0NIYPHy4bN25M0ffv+aO9Vvfu3U3zX7suvLtbdu3aZSpz764ab1qxpEcrI+2X10DTyr9IkSJy8803S82aNc39O+64w3T9aOj9GddWTkorWg3E9Og4xrJly0x/uvbl33nnnSZsGzRokKHnXb16tRlv0YDT4+JNK77rVbD6+mpXyR99fTP6euhroTyvx/79+81thQoVUjxOu4PKli2b7u/2BIsGeXrBoWMlOmahS3r79mePAbIeoeCj9A9FK+8jR46YsYW0zqy1stW+Ye2j16mQWhHrH7sOiqY2eKfjD9o6mD59utSqVSvFz7RvvHr16jJ69OhUn+t6fc86jfbSpUum0tRyec5Y9Vbv62ClViCe7X9UWjNjrvfNstoK2rlzpwlRHU/Rs1t9zQYPHmymoaZHQ1QHcLUVpa+PvhZ61v3pp5+a8YeMDBTrYzQYdeA+NRUrVpTsfD0yi2ffH3zwQenatWuqj9HW3J89BsgehIKP0u4dHRhcu3atfPjhh2k+Tv+odMbJ4sWLU1zDoKFwLa2YdTBSB/10IPJa2iWzadMmU/ml1cpIj579aUWpz6OLZyaPBpYOMH722Wf2fnr+yHNnVN68eaVjx45m0UF5HVzWwVId7NXXMa3n1kFVbYXFx8enODNPrUstrd+hr68Osl874J8d1x54WiqerkClAb53717TkkuLpyWxdevWNB+jLR8d8NcWbUb27XrHAM5iTMFH6ewVne2i3THaV5veWaJWQvoH6T2t8NqZQjq7Rbtt9Gx+xIgRqf4u/bnOjNEK/FraJaRjEenRP+hbbrnFtER0JpB3S0H/v84S0opRWzPXqzRUZl/RrGMc3jTAdFqpnlFrBZnec3vOxr3PvrXLKLXw1d+RWtn19dVWlAb4tfTx116bkll0yqdW3BMnTjSVsIdOO73ea6z/T0Ncp5bqMfXmeS30tdHZX3qCklp4aOvw9xwDOIuWgg9LqynuTaecandGy5YtTZeT9t2+/vrrpr9W+689dBqg/nFq14VOWb22aa9Lly5dzJx3HZDUM2Dt59Ww0W4f3a6VmVYw6dEA0CmS2r+uXVFKp7zqvHbtNsjIVb56nYXSKZ06DVe7wzQYPRX2H6X913rxn+6XThPV8ZNx48aZ11DPdNN7bv2/WoHpurbg9Ixfw1P3TQP32vJroOs4jx4HfYyeoWvLSVsaOmVXXwd9nAatTvvV60o0zD3TYDOT7oOWRcut5dAzdG0haKBdb0xBaZjryYSOD+mUVL2GQ8u6YMECM46l9Jjre6Zu3bqm21Mr+pMnT5oBZh1D0PWMHgM4zOHZT0hlSmp6UpuS+vbbb7sqVKjgyp07t6ty5crmd+kUT+/Dq9MO9X5qi/d0UJ2++PLLL5tpg/r7brzxRlft2rVdw4YNc505c+a6+7FgwQLzO++6664U2x999FGzXct6rWvL4Jm+WaxYMTN91XuKqK736tUr1dela9eu6ZZt0qRJZnqlTpXVfStXrpxrwIABv9mvtJ47Pj7eVaNGDVeePHlcpUuXNq/TO++885sprEeOHDHHSKdxXjvlU6fmPvPMM67y5cubqbUFCxZ03Xrrra6RI0ea1/6PTEmdNWtWiselNq1UjR8/3lWmTBmz73/9619dq1at+s3vTOv/bt261dW2bVtX/vz5zf5XqlTJ9fzzz6d4zNGjR82xKVGihCtnzpyumJgYV/Pmzc3U1997DOCcEP3H6WACAPgGxhQAABahAACwCAUAgEUoAAAsQgEA8PuuU9DL2A8fPmzmEWfl1aYAgKyhE031AzaLFi1qPsDyT4WCBsL1PvcGAOD7Dh48aL6Q6U+FgudKQ/1cE742EQD8j36Mil5dfr0rxzNUw3u6jDQQ+EJvAPBf1xsCYKAZAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAK4f8Dh9++KFERkb+nv8C4BpxcXESaOLj4yXQxAXgccoIWgoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAAH/sA/GywsUrF2XkVyNl+ubpsv/MfgkLCZPCeQtL9Zuqy9DGQ6VmTE2niwgAQcPxlsKAJQNk0OeDZPuJ7VIsopiUzl9ajp09JvN2zJNdJ3c5XTwACCqOtxQ+3PahuR1822AZ1nSYWXe5XPLVwa9MiwEAEEShkOxKNrdL9iyRW4rdIrcUvUVuyneTNCjZwOmiAUDQcbz76PFbHje3a39aK23ebyMxo2Kk8rjK8sLKF+T85fNOFw8AgorjoTC0yVCZe99caVOxjUTmdn+r286EnTJ4xWDp+UlPp4sHAEHF8VBQbWPbSvz98XLqqVPyzaPfSPXC1c12HWwGAARRKDz3+XOy8chGd2FCQs24QsXoiuZ+VJ4oh0sHAMHF8YHmt9a/JS9+8aIUvKGglIwqaaaj/pT4k/nZA9UecLp4ABBUHG8pDG82XP5W6W8SkStCdpzYYUKhUnQlGdJ4iLzQ7AXxV6tWrZJWrVpJoUKFJCQkxCwTJ04UfzVq1Chp0qSJFClSRHLnzi2lSpWSrl27yp49e8Sfvfrqq1KzZk3Jnz+/2a/ixYvLvffeK5s3b3a6aEjFfffdZ/+eOnXq5HRxApLjLYVHb37ULIFm/fr1snTpUilbtqycOHFC/N3YsWPlwIEDUqlSJQkPD5e9e/fK1KlTZcmSJbJz506JjHRPEvA3K1eulOPHj5vjdP78ebMvs2fPls8//9zsb968eZ0uIn41efJkmTVrltPFCHiOtxQCVZcuXSQxMVEWL14sgaB79+6yb98+2b59u2kd9OvXz2w/cuSIfPbZZ+Kv3n//fTl8+LAJ8e+//16effZZs/3kyZOyY8cOp4uHX+3evVueeOIJqV+/vmnNIesQClkkOjranFEHikGDBknJkiXt/UaNGtl17XbxV3ny5JGPPvpI6tWrJ1WqVJGXXnrJbNduv4oV3RMe4KzLly9L586dJTQ0VGbMmCFhYWFOFymgOd59BP9z5coVeeONN8y6drs0b95c/NnRo0fl66+/tvfLlCkj8+fPl4iICEfLBbdhw4aZ4zN9+nRzbJC1aCngdzl79qy0bdvWdIvFxMSYytOfWwqqZ8+ekpycLPv375eOHTua8RK9TUpKcrpoQW/dunXyn//8Rx588EHTWkDWIxSQYTp+0LhxYxME2rWyevVq0+USCHQ2i3aPecYUtm3bZsYb4KytW7ealqkO/ufLl88sOgFAzZkzx9w/c+aM08UMKIQCMkQrSe13/+6778x4wpo1a0zXkT9LSEiQadOmycWLF+22Tz/9NEWrCL5BZ4bp8dBFP0XZM9bgfR8BFgofbP1Abp50s4S/GC4FXi4gHWZ2kN0nd4u/mjt3rpQvX97M7fcYPHiw2eaPzeB27dqZ7hWl3Sp6DYaGhC5vvfWW+CPdj4ceeshco1C9enXTUnjmmWfMz3Q8QfcZznr44YdNpe+96DUySrv49L4ePwTYQPPb69+WR+e7r1Uok7+MJPySIHO2z5EvDnwhm3pukph8MeJvdDqqTqPzpvPhdfHHKXUXLlyw6xs3uj+WxKNly5bij7Qy0QugvvnmG3OsLl26JCVKlDBdZNqN5Kl8gGAS4spA20sruKioKNN3l9kXKenXcRYbXUxOnDsh7WPby+z7ZsvhpMPm47OTLiZJnzp95LW7XsvU5wScFBcXJ4EmPj5eAk1cgB0nPelZtGjRdetxx7uPvj30rQkEpaGgikYUlXrF65n1RT8ucrR8ABBMHA+Fg4kH7br312/qt6+pA2fcMw0AAEEQCmlhRgEABGEolIgsYdf1E1KvXdeP0wYABEko6JfqRIdHm3WdcaR0oFm/s1m1LO+fM1sAwB85Hgq5wnLJS81fsqFQdkxZiX091sw80i/eebrh004XEQCChuOhoHrU7iHT206XWjG1TCshREKkXWw7+arbV2YmEgAgiC5eU51rdDYLACDIWwoAAN9AKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAIA/9oF4HTt2lJw5c0qg4MvGfV8gHqNA3CcEDloKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahkAWGDh0qISEhqS6XL192uni4xvHjx6VPnz5SqlQpyZUrlxQsWFCaN28ue/bsEX+zb9++NN97uuh709+cPXtWBg4cKBUqVJAbbrhBoqKipEaNGjJixAhxuVxOFy/g5HC6AIFMK5dy5cql2KZ/mPAdJ06ckLp168revXtNIFSsWNFUNGvWrJHDhw9L2bJlxZ/kzp3b7I+306dPy86dO816kSJFxN/06tVL3n33XbNetWpVOXPmjGzZssUERZ48eUygI/MQClmodevWMmXKFKeLgXQ899xzJhC0slm6dKmtNC9evOiXZ6Fa/rVr16bY1rt3bxMKN954o3Tu3Fn8zZdffmluW7ZsKQsXLpRffvlFChQoIOfPn5f9+/c7XbyAQ/dRFpozZ46Eh4ebP9S7775bNmzY4HSR4EUr/ZkzZ5r1EiVKyB133CF58+aVmjVrmmOnZ93+LiEhQSZPnmzW//GPf0i+fPnE3zRq1MjcLlq0SKpVq2ZacxoIuv1f//qX08ULOIRCFgkLC5OYmBgpXbq0HDlyRBYsWCD169cnGHxsLOHUqVO2wtFuFj2b3rx5szzwwAMye/Zs8Xfjx4+Xc+fOmYDz126WiRMnykMPPWTWt23bJj/99JPp6tNxBT1eyFyEQhbQCuXYsWOya9cu2b59u6lw1IULF+T11193unj4lfegf2xsrBlY1kXX1bhx48Sfeb/fHnzwQXOS4o9eeeUVmTZtmjRo0MD8XWkwREREmH17+umnnS5ewCEUsoA2b7XP06NFixYSHR1t1g8cOOBgyeCtUKFC5oxTaZeRruui656ZPP5s6tSpcvToUTO5wV+7WbSV8/zzz5uuvvbt25tjVqVKFRMQatmyZU4XMeAQClng5ZdfTlH56wCm9u0q7U6Cb8iZM6fcdtttZl27jC5dumQWXVc6BdJfaSU6atQoO+HB0/rxx1DwtOi+++47c6vjCdpaUDoGhMxFKGSBCRMmmMpf573rWY22FDxv4H79+jldPHgZPny4aR18//33UqZMGbPouo4JPfvss+Kv5s+fb6ehDhgwQPx5WrcnuGfMmGGCWv+2du/ebbZ17drV4RIGHkIhC2hlohc/6Vmn9lFrOOhUQD3T0ZCA79A5/Z9//rk0adLEDDrrWejtt98uq1evlqZNm4q/GjlypLmtU6eOrVT91bx588w1Cdotq9eO6HRhPW7Tp0+Xxx9/3OniBZwQVwYmYycmJpqrCHWesDa5A0V8fLzTRch0cXFxEkgC8RjBP8QF2N+SnqTqpBe9+C8yMjLNx9FSAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALByXF0FgD8m0L7kPpjRUgAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoZJGff/5ZhgwZIpUrV5bw8HApWrSo/OMf/5BTp045XbSgtWrVKmnVqpUUKlRIQkJCzDJx4sQUj7l06ZIMGzZMypYtK7ly5ZLixYtL//79zfH0x/2ZNGmSNGzYUPLmzWsfs2PHDsfKDN9HKGSRNm3ayL///W/58ccfpWLFiqZS0T/YO++8Uy5fvux08YLS+vXrZenSpVKgQIE0H9OtWzcZOnSo7N+/3wTDsWPH5NVXX5W7775bkpOTxd/2Z+HChbJhwwYTHEBGEApZ4Pvvv5cVK1aY9TFjxsimTZvku+++M/fXrVsnM2fOdLiEwalLly6SmJgoixcvTrOSnT59uj1uekY9Z84cc3/lypUyb9488af9UePHjzeP0aADMoJQyALeZ5ShoaEpbtWyZcscKVewi46ONl156Z1Ve7Rv397ctm7dWvLkyWPWFy1aJP60P0q7LcPCwrKtTPB/hEIWiI2NlWrVqpn1Pn36SK1ateTmm2+2Pz906JCDpUNaDh48aNcLFy5sw7xgwYJm/cCBA46VDcguhEIW0DMzPevs3LmzqVD27NkjjRo1knLlypmf58yZ0+ki4ndwuVxOFwHINjmy76mCi85a8fRPq/Pnz0tMTIxZr1SpkoMlQ1pKlChh13WAuUiRIqYrMCEhwWwrWbKkg6UDsgcthSyig5ZJSUlm/cqVKzJgwAA5c+aMud+xY0eHS4fUtGzZ0q57BpgXLFhgAv3anwOBilDIIu+8847pl65evbppIYwbN85s79evn9SpU8fp4gWluXPnSvny5aVJkyZ22+DBg8027eqrXbu23H///WZ73759zdiQZ8BZu//uuece8af9UU899ZS5r7ceLVq0MNtee+01R8oN30b3URbRin/58uVmPEH7pLXC0YvXHnnkEaeLFrR0aubu3btTbDt+/LhZtLtPvfvuu1KhQgWZOnWqeazO7+/QoYMMHz48xQwyf9mfo0eP/uYxngHzkydPZmNp4S9CXBkYRdM3X1RUlGk+B9IgaXx8vASauLg4CSSBeIwCUaC97wKRXq2v06q1GzsyMjLNx/nWqQ8AwFGEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsHJcXUUgCLQvug/EL4QPtGMUqPsUaBITEyUqKuq6j6OlAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAIBsdv68yCuviNx6q0j+/CK5c4uULCly++0io0c7WzY+OhsAslFCgkjz5iKbNrnv33CDSMWKIklJIitXinz2mcg//+lc+WgpAEA26t37aiD07esOiS1bRPbtEzlxQmTyZGfLR0sBALLJ6dMis2a512vWdHcVhXqdmut34Dz8sDiKlgIAZJMffhC5csW93qjR1UC45x6RkJCry5QpzpWRUAAAB4R61b6VKrlbDr6AUACAbFKpkkhYmHv9q6+ubn/5ZZEPPhCfQCgAQDaJihK57z73+rp1IkOGXO1O8hWEQiZYtWqVtGrVSgoVKiQhISFmmThxov15UlKS9OvXT2rXri0FCxaU8PBwqVixojz//PPmZ/64T6pbt25SoUIFyZcvn+TNm1fKlSsnTzzxhJw8edKxcgeTjBwjD32f6fG53uP8YZ+aNGlif+a9NGzYUPzB2LEiNWq41//9b5ECBUT+8hfdL/EJhEImWL9+vSxdulQK6NFNRUJCgowZM0a2bdsmxYsXN5Xorl27ZPjw4dKxY0fxx31SH3/8sVy5ckUqV65swm7Pnj0yduxYeeCBB7K1rMEqI8fIo3fv3ub4BNI+lS1bVurWrWuXqlWrij+IjhZZu9bdZVS7tkhyssiOHSLh4SItWohoBurAs1MIhUzQpUsXSUxMlMWLF6f68zx58siIESPk+PHjsnHjRjl48KDUq1fP/GzhwoVy6tQp8bd9UocOHTIVzbp162T//v32TG316tXZWNLglZFjpGbOnClTp06V+zz9FgGwT0pb2mvXrrXLpEmTxF+Eh4sMHOjuQtLOgl9+Edm7V2TRIpHHHnNf5ewUQiETREdHmy6htMTExMiTTz4pERERNiRuueUWsx4aGio5cuTwu33y7If+YepZWunSpeXLL7802/2lGe/vMnKM9ATkscceM12X2jINhH3y6N+/v+TOndu0GHr06CFHjx7N8vIFA0LBAceOHZM5c+aY9U6dOtmw8EfaDfbNN9+YloK6/fbbzZkpnJecnGzOvC9duiTvvfee5MyZUwKFBkexYsXM2MPevXvlzTfflPr168vZs2edLprfIxSy2e7du82Z9OHDh6VBgwY+O+CXUR988IFcvHhRNmzYINWqVZNly5ZJr169nC4WRMw41sqVK82tTmwIFK+88orpct26datpCT3zzDNmu4bDRx995HTx/B6hkI3WrFljxhL07LpNmzayZMkSv24leOgZaK1ataR79+7m/rRp0+QHvXQTjtr06wfs9O3b10xu8B6I1dlwt+pHdPqhv/zlL6bbSOmsI++JDQcOHHCwZIGBUMgms2fPlmbNmsmJEyekT58+Mm/ePLlBPx7RT3377beyYsUKe19bC9pK8KAZ7zv0WOhy7tw5u+3ChQsp7vtT1+vo0aNTTOX+8MMP7bqObeHPIRQywdy5c6V8+fJm/rTH4MGDzbbOnTubriKd+XH+/HnJlSuX6YPXszRtNeii0/D8bZ90em3Tpk3N1EFtJRQpUkTmz59vHqf3a/rKNfsB7HrHaMqUKeJyueyi3SseEyZMMDPh/G2fNMj+9a9/mfddbGyslCxZ0g6g6/127dqJr1m1SqRVK5FCha5+tlFqvcabN4t06OB+XK5cIsWKXb3QLTv53rQXP6RT6HSswJtOP9VFr0vQs2j9o1S6/vXXX//m//vbPun4QcuWLU0Xxffffy9hYWHmj7J169by7LPPmllVcPYYBeI+6cDyoEGDTNerPu6XX34x18ncc889MnDgQDMjztesXy+ydKleV+H+aOzU6MS9O+90T02NjBTRnr6ff9ZrgbK7tCIhLk9tdZ0DFRUVZSqBQJrBEB8f73QRcB1xcXESaHjfBZeEBPcX6eiM2TJl3NsmTBDp2dO9rjVwlSruC9g6dxZ58033dQxKe8kya9jRU4+fOXNGIjV50sDpHABk8RXM4elceqHdRhoInoDQD83Tz0hq1sz9UdvZjVAAAAft3Hl1/b333K0KtXy5+/OQ9BvZshOhAAAOunz56vojj7hbDToHQD9iW8cVsvsLdwgFAHBQsWJX13/99Bsz9qCzkBQtBQAIInXquGccKf2APKWfGnP8uHu9QoXsLQ+hAABZaO5ckfLlU35fwuDB7m0620gHoYcOdW9/6y293sL91Zz65TsxMSI9emRveQkFAMhCiYn6mWfus38PbQXotkOH3Pf793cHQrVq7o/Q1mmoXbq4Ww6ebqTswsVrAJCFHn7YvVyPDjLr4jRaCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFh+IB5/Gl9zDKXFxcRJILl26lKHH0VIAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSggOC1fLhIaKhISIvLf/17dfuWKSL167u2lSokkJjpZSiDbEQoITk2bivTt614fMkRk82b3ugbE11+7Q+Hdd0UiIx0tJpDdCAUEr//8RyQ2VuTiRZEuXUS++UZk2DD3z/r1E2nSxOkSAtmOUEDwypNHZNo0kRw53C2Fxo31m0hEqlQReeklp0sHOIJQQHCrXVvkuefc6+fPi4SFuYNCAwMIQoQCsGtXyoHmffucLA3gKEIBwW3OHJEZM9zrOttIPfaYyNGjjhYLcAqhgOClFX/Pnu71Vq1E1qwRiY4WOXFCpHt3p0sHOIJQyASrVq2SVq1aSaFChSQkJMQsEydOtD+fMmWK3Z7asmLFCvG3fVK7d++WBx98UEqUKCG5c+eWggULSuPGjeXjjz8Wv/Doo+4AKFBA5K23RIoUEZkwwf2z+fNF3n5bfNmoUaOkSZMmUqRIEfP6lypVSrp27Sp79uyxj7l06ZIMGzZMypYtK7ly5ZLixYtL//795eeffxZ/3adJkyZJw4YNJW/evPa9uWPHDkfLHUgIhUywfv16Wbp0qRTQyiUVWrHWrVs3xaJveo+YmBjxt31yuVxyxx13yIwZM+T48eNStWpVuXLligmTtm3byqZNm8SnaQh88ol7ffx4dyCoe+8V6dzZvd6/v0+PL4wdO9a83vnz55dixYrJgQMHZOrUqdKgQQNJ/PWiu27dusnQoUNl//79JhiOHTsmr776qtx9992SnJws/rhPCxculA0bNpi/K2Q+QiETdOnSxbxhFy9enOrPW7duLWvXrk2x6Fm10oq1cuXK4m/7dOjQIdm7d69Z1zNRDZG5c+fawDh48KD4fCvB5XIvHTum/Nn06e7tWgmVLi2+qnv37rJv3z7Zvn27OZPup9dWiMiRI0fks88+M8dkuu6LiIwZM8acTc/RMRQRWblypcybN0/8bZ/U+PHjzXtTww6Zj1DIBNHR0RIeHp7hxy9atEi2bNli1gcMGCD+uE/a0ilfvrxZHzJkiNx8883Srl07yZEjhzk7veuuu7KxtMFp0KBBUrJkSXu/UaNGdl27XvSM2qN9+/b2BCXPr9Nt9X3ob/ukihYtKmE6dRhZglBwwIgRI8xtzZo1TUvBH+kf5fLly6V27dpy4cIF05w/ffq03HjjjSYg+KPNXtp198Ybb5h17SZq3rx5itZa4cKFzW1oaKhtpWrXjL/tE7IeoZDNtPL8/PPPzfqTTz4p/kr7o3v27Cnfffed9O3b1wxczpo1y4wv9O7d2ye7JgLV2bNnzTiOdvXp+NT8+fPtWXVqtHsv0PYJmYdQyGYjR440tzpjp1OnTuKvtH93wYIFZl1nh+hMkA4dOkjkrx8gt2zZModLGBy0r11nfGmlWbFiRVm9erVU0Y/p+PU95qEDzJ4wT0hIMOve3TT+sk/IeoRCNtLm+syZM826nl1r/7u/OnPmjF1ft26duf3hhx8kKSnJrGtIIGtt27ZN6tWrZ1pr2ve+Zs0a083i0bJlS7vuGWDWID+vH+dxzc/9ZZ+Q9QiFTKCzbnTQVedXewwePNhs6+yZ3ihipgJevnxZoqKipEePHuLP+9S0aVMzfqC0G6l69epmLEG7JnLmzCn333+/+JxVq9wXqelURv1obF2uufbCTEF9+GH31c06IFupksj//qen2OJrdGBfp5oqDWO9rkQrVF3eeustM97jOQ56EhIbG2sHnLXCveeee8Tf9kk99dRT5n2otx4tWrQw21577TXHyh4o/PdU1Yfo9Di9kMub9q3rohcLec6sPW9qDYSIiAjx533S2UnarH/xxRfliy++kF27dpmQ0Gb/c889J7Vq1RKfs369yNKlOmrpvmjtWsePi9Sp477Nl09Epwpv3aq1kMjhw5rq4kt0gN9j48aNKX7maQW8++67UqFCBTPXX4+nzu3Xbr7hw4ebQWdfk5F9Onr06G/em55B85MnT2ZLOQNZiCsDo05aQejZrR4UPQsMFPHx8U4XAdlJ+9JvuMH98RZlyri36RXMno+60IvYevVyr2sYVK0q8uabmuLuT0/V6zK8+ukR2OLi4iSQ6NXtOg1ZT1A9Y3+p8b1TBSCr6OcapXc9iXcXkecs2nOrn56qX+EJBDhCAfDQ8QbtNlJ164poF5inFaEOHXKsaEB2IRQADx1rWLLE/f3N2kLQcQQddNYBaRVAXadAWggFwFv9+iJ6ceHp0zq5Xz9Rzv05SEpnIgEBjlAAvH35pXv8QJ06pZedu9f1oyH4mAUEAUIBwUM/xVU/xM/r2gsZPNi9zXM9iY4haADUqCGi04m/+so980ivZ9CZS0CAIxQQPPSjsHV++68XRxl6TYJu8wwi33mniE7X27lTRK841/vanfTrRV9AoOPiNQQPHTTWJT2jR7sXIEjRUgAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsIL6A/EC7Yu5AeDPoqUAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAAKwckgEul8vcXr58OSMPBwD4GE/97anP/1QoJCUlmdtly5ZlRtkAAA7R+jwqKirNn4e4rhcbIpKcnCyHDx+WiIgICQkJyewyAgCymFb1GghFixaV0NDQPxcKAIDgwEAzAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAPH4f8EOdr53tgDzAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def plot_maze_with_states():\n", + " \"\"\"Plot the maze with state indices.\"\"\"\n", + " grid = np.ones(\n", + " (n_rows, n_cols)\n", + " ) # Start with a matrix of ones. Here 1 means “free cell”\n", + " for i in range(n_rows):\n", + " for j in range(n_cols):\n", + " if maze_str[i][j] == \"#\":\n", + " grid[i, j] = 0 # We replace walls (#) with 0\n", + "\n", + " fig, ax = plt.subplots()\n", + " ax.imshow(grid, cmap=\"gray\", alpha=0.7)\n", + "\n", + " # Plot state indices\n", + " for (\n", + " s,\n", + " (i, j),\n", + " ) in state_to_pos.items(): # Calling .items() returns a list-like sequence of (key, value) pairs in the dictionary.\n", + " cell = maze_str[i][j]\n", + "\n", + " if cell == \"S\":\n", + " label = f\"S\\n{s}\"\n", + " color = \"green\"\n", + " elif cell == \"G\":\n", + " label = f\"G\\n{s}\"\n", + " color = \"blue\"\n", + " elif cell == \"X\":\n", + " label = f\"X\\n{s}\"\n", + " color = \"red\"\n", + " else:\n", + " label = str(s)\n", + " color = \"black\"\n", + "\n", + " ax.text(\n", + " j,\n", + " i,\n", + " label, # Attention : matplotlib, text(x, y, ...) expects (column, row)\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=10,\n", + " fontweight=\"bold\",\n", + " color=color,\n", + " )\n", + "\n", + " ax.set_xticks([]) # remove numeric axes, we don't need.\n", + " ax.set_yticks([])\n", + " ax.set_title(\"Maze with state indices\")\n", + "\n", + " plt.show()\n", + "\n", + "\n", + "plot_maze_with_states()" + ] + }, + { + "cell_type": "markdown", + "id": "db078d86", + "metadata": {}, + "source": [ + "### 2.4 Actions and deterministic movement" + ] + }, + { + "cell_type": "markdown", + "id": "96e7f1f2-9d73-410b-853d-e39f40dfb5da", + "metadata": {}, + "source": [ + "We first define integer codes for each action. \n", + "\n", + "**Exercise 3.** How many possible actions can the agent take in the maze?" + ] + }, + { + "cell_type": "markdown", + "id": "22259ab4-527e-4d7c-bb30-98fb240da6d5", + "metadata": {}, + "source": [ + "We have four possible actions in the maze. \n", + "\n", + "In this following cell, each action is mapped to an integer (0,1,2,3). This makes it easy to store and use actions inside arrays and matrices\n", + "\n", + "Here we use Unicode arrow character:\n", + "\n", + "- \"\\u2191\" : ↑ (up arrow)\n", + "\n", + "- \"\\u2192\" : → (right arrow)\n", + "\n", + "- \"\\u2193\" : ↓ (down arrow)\n", + "\n", + "- \"\\u2190\" : ← (left arrow)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "f7f0b8e4-1f48-4d03-9e5f-a47e59c3e827", + "metadata": {}, + "outputs": [], + "source": [ + "A_UP, A_RIGHT, A_DOWN, A_LEFT = 0, 1, 2, 3\n", + "ACTIONS = [A_UP, A_RIGHT, A_DOWN, A_LEFT]\n", + "action_names = {A_UP: \"\\u2191\", A_RIGHT: \"\\u2192\", A_DOWN: \"\\u2193\", A_LEFT: \"\\u2190\"}" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "3773781c-a0cd-48db-967b-d4b432d17046", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "↑\n" + ] + } + ], + "source": [ + "print(action_names[0])" + ] + }, + { + "cell_type": "markdown", + "id": "4b957f5a-ee39-4437-abc1-4809105ad83c", + "metadata": {}, + "source": [ + "**Exercise 4.** Now we define a **deterministic movement function** `move_deterministic(i, j, a)`. \n", + "\n", + "This function simulates the robot trying to move from (i, j) in direction a.\n", + "\n", + "But if the movement hits a wall or boundary, the agent stays in place." + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "4b06da5e-bc63-48e5-a336-37bce952443d", + "metadata": {}, + "outputs": [], + "source": [ + "def move_deterministic(i: int, j: int, a: int) -> tuple[int, int]:\n", + " \"\"\"Deterministic movement on the grid. If the movement hits a wall or boundary, the agent stays in place.\n", + "\n", + " Args:\n", + " i (int): current row index\n", + " j (int): current column index\n", + " a (int): action to take (A_UP, A_DOWN, A_LEFT, A_RIGHT)\n", + "\n", + " Returns:\n", + " (tuple[int, int]): new (row, column) position after taking action a\n", + "\n", + " \"\"\"\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j,\n", + " ) # It means “Unless the action succeeds, the robot stays in place.”\n", + "\n", + " # Now each action changes the coordinates of the robot:\n", + " if a == A_UP:\n", + " candidate_i, candidate_j = (\n", + " i - 1,\n", + " j,\n", + " ) # if the action is UP, then row becomes row -1\n", + " elif a == A_DOWN:\n", + " candidate_i, candidate_j = (\n", + " i + 1,\n", + " j,\n", + " ) # if the action is DOWN, then row becomes row +1\n", + " elif a == A_LEFT:\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j - 1,\n", + " ) # if the action is LEFT, then column becomes column -1\n", + " elif a == A_RIGHT:\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j + 1,\n", + " ) # if the action is RIGHT, then column becomes column +1\n", + "\n", + " # Check boundaries\n", + " if not (0 <= candidate_i < n_rows and 0 <= candidate_j < n_cols):\n", + " # If the robot tries to move outside the maze\n", + " # It will not move and it stays at (i, j).\n", + " return i, j\n", + "\n", + " # Check wall\n", + " if maze_str[candidate_i][candidate_j] == \"#\":\n", + " # If the next cell is a wall, the robot stays in place.\n", + " return i, j\n", + "\n", + " return candidate_i, candidate_j # Otherwise, return the new position\n" + ] + }, + { + "cell_type": "markdown", + "id": "c9e620e6", + "metadata": {}, + "source": [ + "### 2.5 Transition probabilities and reward function" + ] + }, + { + "cell_type": "markdown", + "id": "80bd2bca-7717-4b5f-bffa-76fe86a51d35", + "metadata": {}, + "source": [ + "Recall that we set the discount factor $\\gamma \\in(0,1)$, that is, the future rewards are multiplied by $\\gamma$, so immediate rewards matter a little bit more than future ones. \n", + "\n", + "\n", + "Moreover, we consider a probability error $p_{\\text{error}}$, which means, with probability $p_{\\text{error}}$, the robot **does not** execute the intended action but one of the 3 other directions (chosen uniformly). With probability $1-p_{\\text{error}}$, the robot executes the action that we asked." + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "id": "610253e7-f3f7-4a30-be3e-2ec5a1e2ed04", + "metadata": {}, + "outputs": [], + "source": [ + "gamma = 0.95\n", + "p_error = 0.1 # probability of the error to a random other direction\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d1ceff8-86e0-4c45-83d3-af9fae974608", + "metadata": {}, + "source": [ + "Now we initialize the state–transition probability : the probability of reaching next state $s'$ after taking action $a$ in state $s$. \n", + "$$\n", + " p(s' \\mid s, a)\n", + " = \\mathbb{P} \\big[S_t=s'\\,|\\, S_{t-1}=s, \\,A_{t-1}=a\\big]\n", + "$$\n", + "\n", + "We store these transition probabilities in the 3D array `P` (`P[a][s, s_next]`), which has shape `(n_actions, n_states, n_states)`:\n", + "\n", + "`P[a, s, s_next] = P(S_{t+1} = s_next | S_t = s, A_t = a)`.\n", + "\n", + "We also initialize the reward vector `R`, which has length `n_states`, where `R[s]` is the reward received when the agent is in state `s`.\n", + "\n", + "In this maze game, we assume that the reward depends only on the current state, which is natural: in navigation tasks, being in a particular location is what matters, not the direction you used to reach it." + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "7a51f242-fe4e-4e74-8a1f-a8df32b194b8", + "metadata": {}, + "outputs": [], + "source": [ + "# Initialize transition matrices and reward vector\n", + "P = np.zeros((len(ACTIONS), n_states, n_states))\n", + "R = np.zeros(n_states)" + ] + }, + { + "cell_type": "markdown", + "id": "c08f4af5-a2a7-4baa-b5da-c7ce636d8a4a", + "metadata": {}, + "source": [ + "Now we assign the reward to each state. \n", + "\n", + "For each state index s:\n", + "\n", + "1. If s is a goal, then the reward = +1.0\n", + "2. If s is a trap, then the reward = −1.0\n", + "3. Otherwise for the normal cell, the reward = −0.01 every time you leave this cell.\n", + "\n", + "Recall that rewards are received at the moment the agent executes an action. Here when the agent moves out of the cell, we set reward −0.01. " + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "id": "49d54d1f-dc29-45b6-ad31-ad0e848f920d", + "metadata": {}, + "outputs": [], + "source": [ + "# Set rewards for each state\n", + "step_penalty = -0.01\n", + "goal_reward = 1.0\n", + "trap_reward = -1.0\n" + ] + }, + { + "cell_type": "markdown", + "id": "dd571ec8-c36a-4e20-bec6-9e6458dc622b", + "metadata": {}, + "source": [ + "**Exercise 5.** Why do we set the step penalty to -0.01 in this MDP?" + ] + }, + { + "cell_type": "markdown", + "id": "1e8ea171", + "metadata": {}, + "source": [ + "We set a small negative step penalty (`-0.01`) for two main reasons:\n", + "\n", + "- Incentivize Efficiency: It forces the agent to find the shortest path to the goal. By losing a small amount of reward at every step, the agent learns that the faster it reaches the goal, the higher its total cumulative return will be.\n", + "\n", + "- Prevent Loitering: It discourages infinite loops or wandering. Without this penalty (i.e., if step reward = 0), the agent might be indifferent between reaching the goal now or in 1000 steps, potentially leading to a policy that never terminates." + ] + }, + { + "cell_type": "markdown", + "id": "07bfb065-b1af-4df1-885e-780fe250f2fb", + "metadata": {}, + "source": [ + "**Exercise 6.** We now define the reward vector. Recall that we have already initialized\n", + "`R = np.zeros(n_states)`.\n", + "If a state belongs to `goal_states`, we assign the `goal_reward`.\n", + "If it belongs to `trap_states`, we assign the `trap_reward`.\n", + "Otherwise, we assign the `step_penalty`. " + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "id": "b9b7495a-c233-425c-99c0-5bddaf6c3225", + "metadata": {}, + "outputs": [], + "source": [ + "for s in range(n_states):\n", + " if s in goal_states:\n", + " R[s] = goal_reward\n", + " elif s in trap_states:\n", + " R[s] = trap_reward\n", + " else:\n", + " R[s] = step_penalty\n" + ] + }, + { + "cell_type": "markdown", + "id": "b90fb80c-9452-48a2-889f-286703c2ae93", + "metadata": {}, + "source": [ + "Now we define terminal states and a helper function. Here terminal_states is a set containing all absorbing states, which means, reaching them ends the episode conceptually. \n", + "\n", + "Moreover, `is_terminal(s)` is a small helper to check if a state is terminal." + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "id": "eca4c571-39c7-468b-af86-0bab9489415e", + "metadata": {}, + "outputs": [], + "source": [ + "terminal_states = set(goal_states + trap_states)\n", + "\n", + "\n", + "def is_terminal(s: int) -> bool:\n", + " \"\"\"Check if a state is terminal (goal or trap).\"\"\"\n", + " return s in terminal_states\n" + ] + }, + { + "cell_type": "markdown", + "id": "3a9a1d54-8339-402b-84e9-105961ed78d7", + "metadata": {}, + "source": [ + "Now we need to fill the transition matrices `P[a][s, s_next]`. \n" + ] + }, + { + "cell_type": "markdown", + "id": "d9cfd15c-12cc-48bb-bd88-07f3ae3db31c", + "metadata": {}, + "source": [ + "**Exercise 7.** **Complete the `# TO DO` part in the program below** to fill the transition matrices `P[a][s, s_next]`. " + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "id": "2d03276b-e206-4d1f-9024-f6948ca61523", + "metadata": {}, + "outputs": [], + "source": [ + "for s in range(n_states): # We loop over all states s.\n", + " i, j = state_to_pos[\n", + " s\n", + " ] # We recover the states to their coordinates (i, j) in the maze.\n", + "\n", + " # First, in a goal or trap state,\n", + " # No matter which action you “choose”, you stay in the same state with probability 1.\n", + " # This makes the terminal states as the absorbing states.\n", + " if is_terminal(s):\n", + " # Terminal states: stay forever\n", + " for a in ACTIONS:\n", + " P[a, s, s] = goal_reward\n", + " continue\n", + "\n", + " # If the state is non-terminal, we define the stochastic movement.\n", + " # For a given state s and intended action a,\n", + " # With probability 1 - p_error, the robot will move in direction a;\n", + " # With probability p_error, the robot will move in one of the other 3 directions, each with probability p_error / 3.\n", + " for a in ACTIONS:\n", + " # main action (intended action)\n", + " main_i, main_j = move_deterministic(i, j, a)\n", + " s_main = pos_to_state[\n", + " (main_i, main_j)\n", + " ] # s_main is the state index of that next cell.\n", + " P[a, s, s_main] += (\n", + " 1 - p_error\n", + " ) # We add probability 1 - p_error to P[a, s, s_main].\n", + "\n", + " # error actions\n", + " other_actions = [\n", + " a2 for a2 in ACTIONS if a2 != a\n", + " ] # other_actions = the 3 actions different from a.\n", + " for a2 in other_actions: # for each of the error action,\n", + " error_i, error_j = move_deterministic(i, j, a2)\n", + " s_error = pos_to_state[(error_i, error_j)] # get its state index s_error\n", + " P[a, s, s_error] += p_error / len(\n", + " other_actions\n", + " ) # add p_error / 3 to P[a, s, s_error]\n", + "# So for each (s,a), probabilities over all s_next sum to 1.\n" + ] + }, + { + "cell_type": "markdown", + "id": "7841b264-af00-4322-b728-adcffac0ef89", + "metadata": {}, + "source": [ + "Now we check if the transition matrices `P[a][s, s_next]` are computed correctly.\n", + "For each action `a`, we sum the transition probabilities over all possible next states `s_next` and verify that these sums are equal to 1.\n", + "\n", + "This is because the matrix `P[a, s, s_next]` stores the transition probability\n", + "\n", + "$\\mathbb{P} \\big[S_t=s_{\\text{next}}\\,|\\, S_{t-1}=s, \\,A_{t-1}=a\\big]$. \n", + "\n", + "Therefore, for each action $a$, and for each state $s$, the sum over $s_{\\text{next}}$ of $\\mathbb{P} \\big[S_t=s_{\\text{next}}\\,|\\, S_{t-1}=s, \\,A_{t-1}=a\\big]$ should be 1. " + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "id": "341fe630-8f87-4773-84ad-92d3516e53e2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Action ↑: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action →: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action ↓: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action ←: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n" + ] + } + ], + "source": [ + "for a in ACTIONS:\n", + " # For each action a:\n", + " # P[a] is a matrix of shape (n_states, n_states).\n", + " # P[a].sum(axis=1) sums over next states s_next, giving for each state s:\n", + " # We print these row sums.\n", + " # If everything is correct, they should be very close to 1.\n", + "\n", + " probs = P[a].sum(axis=1)\n", + " print(f\"Action {action_names[a]}:\", probs)\n" + ] + }, + { + "cell_type": "markdown", + "id": "46d23991", + "metadata": {}, + "source": [ + "## 3. Policy evaluation\n", + "\n", + "### 3.1 Bellman expectation equation" + ] + }, + { + "cell_type": "markdown", + "id": "305b047c-e83b-4f42-b64e-e2050d5deeff", + "metadata": {}, + "source": [ + "Recall that the value function under a policy $\\pi$ is defined as:\n", + "$$\n", + "V^{\\pi}(s)=\\mathbb{E}\\Big[\\:G_t \\:\\Big|\\: S_t=s\\:\\Big]\n", + "$$\n", + "where the return $G_t$ is\n", + "$$\n", + "G_t=R_t +\\gamma R_{t+1}+\\gamma^2 R_{t+2}+... . \n", + "$$\n", + "This means *The value of a state is the expected discounted sum of all future rewards\n", + "when following policy $\\pi$.*\n", + "\n", + "We know that $G_t=R_t+\\gamma G_{t+1}$, and plugging this equation into the definition of $V^{\\pi}(s)$, we get \n", + "$$\n", + "V^{\\pi}(s)=\\mathbb{E}\\Big[\\:R_t \\:\\Big|\\: S_t=s\\:\\Big]+\\gamma\\mathbb{E}\\Big[\\:G_{t+1} \\:\\Big|\\: S_t=s\\:\\Big]. \n", + "$$\n", + "This step shows simply ``The total future reward = immediate reward + discounted reward from next state.''" + ] + }, + { + "cell_type": "markdown", + "id": "88ea8d56-3b62-4690-9ff7-469e43726fbc", + "metadata": {}, + "source": [ + "For the expected immediate reward part $\\mathbb{E}[R_t| S_t=s]$, as we are in a maze problem, the reward depends only on the current state, not the time step, i.e., $\\mathbb{E}[R_t| S_t=s]=R(s)$. Hence we get \n", + "$$\n", + "V^{\\pi}(s)=R(s)+\\gamma\\mathbb{E}\\Big[\\:G_{t+1} \\:\\Big|\\: S_t=s\\:\\Big]. \n", + "$$\n", + "\n", + "Moreover, in this maze problem, we consider a deterministic policy $A_t=\\pi(s)$ (the action depends only on the state). Therefore, \n", + "$$\n", + "V^{\\pi}(s)=\\mathbb{E}\\Big[\\:R_t \\:\\Big|\\: S_t=s\\:\\Big]+\\gamma\\mathbb{E}\\Big[\\:G_{t+1} \\:\\Big|\\: S_t=s, A_t=\\pi(s)\\:\\Big]. \n", + "$$\n", + "\n", + "Now **given the state $S_t=s$ and $A_t=a$**, the next state is random (because of the error probability) and we know the transition probability \n", + "$$\n", + "\\mathbb{P}\\big(\\:S_{t+1}=s' \\:|\\:S_t=s, \\, A_t=a\\big)=P\\big(s'\\:\\big|\\:s, a\\big). \n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "c25e255d-8f58-4eaf-9485-cee6ab3bea6c", + "metadata": {}, + "source": [ + "Therefore,\n", + "$$\n", + "\\mathbb{E}\\big[\\,G_{t+1}\\,|\\,S_t=s,A_t=a\\,\\big] =\\sum_{s'}\\mathbb{E}\\big[\\,G_{t+1}\\,|\\,S_{t+1}=s'\\,\\big]\\times \\mathbb{P}\\big[S_{t+1}=s'\\,\\big|\\,S_t=s, A_t=a\\, \\big]\n", + "$$\n", + "$$\n", + "\\hspace{-1.2cm}=\\sum_{s'}V^{\\pi}(s')P\\big(s'\\:\\big|\\:s, a\\big),\n", + "$$\n", + "where here we use the Markov property. (**Question: Can you show the detailed computations here?**)" + ] + }, + { + "cell_type": "markdown", + "id": "9a2b6cff-e848-44a2-b504-973067b367b3", + "metadata": {}, + "source": [ + "In conclusion, we have (the Bellman expectation equation)\n", + "$$\n", + "V^{\\pi}(s)=R(s)+\\gamma \\sum_{s'}P\\big(\\,s'\\,\\big|\\,s, \\pi(s)\\,\\big)V^{\\pi}(s').\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "15049fdb-f3af-4f78-b556-817284260ed0", + "metadata": {}, + "source": [ + "### 3.2 Define a function which computes the value function $V^{\\pi}(s)$ for a given deterministic policy. \n", + "\n", + "\n", + "**Exercise $8^*$.** Now we define `policy_evaluation(...)`, which computes the value function $V^{\\pi}(s)$ for a given deterministic policy. \n", + "\n", + "The input of this function `policy_evaluation(...)` are:\n", + "1. policy: array of size `n_states`, each entry is an action 0,1,2,3, which correspond to UP, RIGHT, DOWN, LEFT.\n", + "2. `P`: the transition probabilities `P[a, s, s']`.\n", + "3. `R`: the reward vector `R[s]`.\n", + "4. gamma: the discount factor $\\gamma\\in(0,1)$.\n", + "5. theta: convergence threshold.\n", + "6. max_iter: which is used to avoid infinite loops.\n", + "\n", + "How can we apply the Bellman expectation equation\n", + "$$\n", + "V^{\\pi}(s)=R(s)+\\gamma \\sum_{s'}P\\big(\\,s'\\,\\big|\\,s, \\pi(s)\\,\\big)V^{\\pi}(s').\n", + "$$\n", + "here ?\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "5c48f489-3508-4981-8b35-5bedc2e5838c", + "metadata": {}, + "source": [ + "We start with an initial guess of $V^{\\pi}$(e.g., all values = 0) and repeatedly apply the Bellman equation to update each state:\n", + "$$\n", + "V_{k+1}^\\pi(s) \\leftarrow R(s)+\\gamma \\sum_{s'}P\\big(\\,s'\\,\\big|\\,s, \\pi(s)\\,\\big)V^{\\pi}_k(s').\n", + "$$\n", + "until values converge." + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "2fffe0b7", + "metadata": {}, + "outputs": [], + "source": [ + "def policy_evaluation( # noqa: PLR0913\n", + " policy: np.ndarray,\n", + " P: np.ndarray,\n", + " R: np.ndarray,\n", + " gamma: float,\n", + " theta: float = 1e-6,\n", + " max_iter: int = 10_000,\n", + ") -> np.ndarray:\n", + " \"\"\"Evaluate a deterministic policy for the given MDP.\n", + "\n", + " Args:\n", + " policy: array of shape (n_states,), with values in {0,1,2,3}\n", + " P: array of shape (n_actions, n_states, n_states)\n", + " R: array of shape (n_states,)\n", + " gamma: discount factor\n", + " theta: convergence threshold\n", + " max_iter: maximum number of iterations\n", + "\n", + " \"\"\"\n", + " n_states = len(R) # get the number of states\n", + " V = np.zeros(n_states) # initialize the value function\n", + "\n", + " for _it in range(max_iter): # Main iterative loop\n", + " V_new = np.zeros_like(\n", + " V\n", + " ) # Create a new value vector and we will compute an updated value for each state.\n", + "\n", + " # Now we update each state using the Bellman expectation equation\n", + " for s in range(n_states):\n", + " a = policy[s] # Extract the action chosen by the policy in state\n", + " V_new[s] = R[s] + gamma * np.sum(P[a, s, :] * V)\n", + "\n", + " delta = np.max(\n", + " np.abs(V_new - V)\n", + " ) # This measures how much the value function changed in this iteration:\n", + " # If delta is small, the values start to converge; otherwise, we need to keep iterating.\n", + " V = V_new # Update V, i.e. Set the new values for the next iteration.\n", + "\n", + " if delta < theta: # Check convergence: When changes are tiny, we stop.\n", + " break\n", + "\n", + " return V # Return the final value function, this is our estimate for V^{pi}(s), s in the state set.\n" + ] + }, + { + "cell_type": "markdown", + "id": "09ef3439", + "metadata": {}, + "source": [ + "### 3.3 Evaluating a random policy" + ] + }, + { + "cell_type": "markdown", + "id": "eecbca15-f89f-47bf-a13d-7d7c051699b8", + "metadata": {}, + "source": [ + "Now we use the policy evaluation function `policy_evaluation` to evaluate a random policy. \n", + "\n", + "We first generate a `random_policy`, which is an array like [2, 0, 1, 3, 0, 2, ...] and has the size `n_states`. (Recall that the policy is a mapping from states to actions)." + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "id": "b4a44e38", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0 3 2 1 1 3 0 2 0 0 2 3 2 3 2 3 2 0 3 1 2 1]\n" + ] + } + ], + "source": [ + "# Random policy: for each state, pick a random action\n", + "random_policy = rng.integers(low=0, high=len(ACTIONS), size=n_states)\n", + "\n", + "print(random_policy)" + ] + }, + { + "cell_type": "markdown", + "id": "3fe07992-ce82-4124-aebc-a6384d417f64", + "metadata": {}, + "source": [ + "Now we call the function `policy_evaluation(...)` to compute $V^{\\pi_{\\text{random}}}(s)$." + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "id": "c5f559b2-452a-477c-a1fa-258b40805670", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Value function under random policy:\n", + "[ -0.2 -0.2 -0.201 -0.204 -0.205 -0.202 -0.214 -0.429 -0.212\n", + " -0.207 -0.276 -0.459 -0.352 -0.366 -5.827 -4.605 20. -0.366\n", + " -0.999 -20. -6.4 -3.163]\n" + ] + } + ], + "source": [ + "V_random = policy_evaluation(policy=random_policy, P=P, R=R, gamma=gamma)\n", + "print(\"Value function under random policy:\")\n", + "print(V_random)\n" + ] + }, + { + "cell_type": "markdown", + "id": "f46c70ba-2932-49af-b568-b5477260bc94", + "metadata": {}, + "source": [ + "Here in this value vector of the policy, \n", + "- If it is a negative values, then the agent tends to move around aimlessly, fall in traps, or take too long.\n", + "- It it is a higher values, then the agent is closer to the goal or more likely to reach it" + ] + }, + { + "cell_type": "markdown", + "id": "1efcb076-467c-42d8-94e8-87453f688bbd", + "metadata": {}, + "source": [ + "Now we define a function `plot_values`, which displays the value function $V(s)$ and displays it on the maze grid. It helps students visually understand:\n", + "- which states are good (high value, near the goal),\n", + "- which states are bad (low value, near traps),\n", + "- how a policy affects the long-term expected reward." + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "id": "4c428327", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdcAAAGbCAYAAACWHtrWAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAARjhJREFUeJzt3Qd4FGX+B/DvptcNoSQhEHqHUAUUUWnSBEQ9FRtgvb/lUBEBGyJFBBThlAPvTkVQBPQUEAREBFHpJfROIAFSSCC9kez8n/dddskmm7rvbrLL9+MzLjsz+868O5v5zdtmdJqmaSAiIiJl3NQlRURERAKDKxERkWIMrkRERIoxuBIRESnG4EpERKQYgysREZFiDK5ERESKMbgSEREpxuBKRESkGIOrg507dw46nQ6LFi2qku2vX78eHTt2hI+Pj9yPlJQUVEdi3yZPnlzVu1GtiN+M+F7Eb8hViWMu8lhYo0aNMHr06CrbJ6LKYHAtxbBhw+Dn54f09PQS13nsscfg5eWF5ORkVHdiHx966CH4+vpi/vz5WLJkCfz9/atsf37++WcGUCJySR5VvQPVmQicP/30E3788UeMHDmy2PKsrCysWrUKAwcORK1atVDd7d69W14oTJ06Ff369avq3ZHBVQR5awE2OzsbHh78eRJw4sQJuLmxHEDOhb/YMkqugYGBWLp0qdXlIrBmZmbKIOwMEhMT5WuNGjVQ3Ylq6+ocXMXzLsQFANmft7c3PD09q3o3iCqEwbUUovr0/vvvx6ZNm8yBqTARdEXwFUH4ypUrGDduHCIjIxEQEAC9Xo9BgwbhwIEDZW6nV69ecipKtDOJ9qbCDAYD5s6di7Zt28oAFBoair///e+4evVqmdsYNWqU/HfXrl1lu5apHaukNq2i+7Vlyxb5uRUrVmD69OmoX7++3Ie+ffvi9OnTxT6/c+dODB48GMHBwbL6uX379pg3b545b6LUKog0TVNpba779++X36n4bsV3LLa7Y8cOq+2Sf/31F8aOHYs6derIbd933324fPmyxbqpqak4fvy4fC2L+I6GDBmCDRs24JZbbpG/jc8++0wu+/LLL9GnTx+EhITIQNCmTRssWLCgxDT+/PNPdOvWTX53TZo0weLFi4ute+TIEZmm2I74nqdNmyaPvTX/+te/5O9BbDs8PBwvvvhisbZ0cRzbtWuHgwcP4q677pLNHc2aNcP3338vl//+++/o3r273F7Lli3x66+/lvmdmH4Py5cvx5tvvomwsDD5XYu/h9jY2GLrf/fdd+jSpYvcRu3atfH444/j4sWLZW7H2u9T5O/VV1+Vy0S+xXckapeSkpKQkZEh9+Pll18ultaFCxfg7u6OGTNmlLldIpuIR85RyX755RfxSD7tk08+sZifnJyseXp6aiNHjpTvd+/erTVt2lSbOHGi9tlnn2lTpkzR6tWrpwUFBWkXL140fy46Olqm9+WXX5rn3XXXXXIqatSoUVrDhg0t5j3zzDOah4eH9uyzz2oLFy7UJkyYoPn7+2tdu3bV8vLySs3Hc889J7ct9m3JkiXatm3b5DKxDbGtooru1+bNm+XnO3XqpHXp0kX7+OOPtcmTJ2t+fn5at27dim3Py8tLpv3uu+9qCxYs0MaMGaP169dPLhfbvvvuu2V6Yl9Mk4mYLz5ncvjwYZnPunXralOnTtU++OADrXHjxpq3t7e2Y8cO83riezXtY58+feRxe+211zR3d3ftoYcesthH07qFj0VJRD6aNWumBQcHy2MsvnvxfQjiux89erT8PsT2+vfvL9P99NNPi6XRsmVLLTQ0VHvzzTfl8s6dO2s6nU7mzyQuLk6rU6eO3Jb4fmfPnq01b95ca9++vUxX/IZMxHck5onvVWz7pZdeknkt+nsQxzE8PFyLiIjQXn/9dblumzZt5LrLli3TwsLC5Lbmzp1r/t2mpaWV+p2Yfg+RkZFy3+bMmSO/Gx8fH61FixZaVlZWse9a7Jf4nsR6vr6+WqNGjbSrV68Wy0/R763w7zM9PV1r166d3HfxdyB+W+I3IdLev3+/XOexxx6T33N+fr5FWrNmzZLf9/nz58s85kS2YHAtg/jjFCf02267zWK+OLmKk8CGDRvk+5ycHK2goMBiHXESFCd/EcxUBNc//vhDfvabb76xWG/9+vVW5xdlOsGJC4HCKhpcW7dureXm5prnz5s3T84/dOiQ+TsTgU+kW/jEKRgMBvO/X3zxxWIn0pKC6/Dhw2WwPnPmjHnepUuXtMDAQO3OO+8slkcRbApv69VXX5Un45SUlEoHV7Gu+K6LKhxETAYMGKA1adLEahpbt241z0tMTJS/EXEBYPLKK6/I9Xbu3Gmxngh4hYOrmCe+ExHMC//2RNAW633xxRfmeeI4inlLly41zzt+/Lic5+bmZnGBIn7T5fleTL8HEYwLB+IVK1bI+eJ3IYggHxISIgNidna2eb01a9bI9SZNmlSh4CrWF+v88MMPxfbJdMxNeVi3bp3FcnERYO1vjUg1VguXQVQhjRgxAtu3b7cYAiGqhEWVrKiaFETVlKnTRUFBgeyZK6ouRRXbvn37lOyLqFYLCgrC3XffLau/TJOoahPb2rx5MxzhySeflD2kTe644w75evbsWXP1bXR0NF555ZVi7btFh1mUh/g+f/nlFwwfPlxWo5rUrVsXjz76qKxmTUtLs/jMc889Z7EtsY8infPnz5vniapGEcfLO8yjcePGGDBgQLH5oprTRFQxi2Miql7F91G0yllUGZu+L0FUW4vfiOm7M3X0uvXWW2XVceH1irbti6rbvLw8+T0X7vDz7LPPyqrztWvXWqwvfiPit2witiuOT+vWrWWVsInp34X3qTSiOlY0j5j87W9/k8dG5EPYs2ePbFZ54YUXZFW4yT333INWrVoV28+y/O9//0OHDh1kVX9RpmMuOuyJKvJvvvnGvOzw4cOyWlxURxPZG4NrOZhOaqaOTaLd5o8//pAnKhF8BdEe9vHHH6N58+Yy0Io2JXFCFH/M5WnTK49Tp07JtETbnki78CTamay1C9tDgwYNLN6LNlXB1O575swZ+Sra+FQQbaWiZ7YIBkWJwCC++6JtfGXtY2WI4GqNaN8VJ3PRzieClTgeog1SKHrsi+6Xad8K75e4ABC/o6KK5t90oVB0vrjwERchhS8kBNEuWfTiRlysRUREFJtXke+q6L6KbYj2XNPFaEn7KYjgWnQ/yyJ+X2X9tsTFhvi7XblypfztCCLQiuD+4IMPVmh7RJVRfbtjViOiZChOAt9++608aYpXUeIpXJJ4//338c477+Cpp56SQ11q1qwp/8BFqaKkjiiFT0bGmlBLoqRVmEhHBNbCV+OFiZN6ZZRUmhTbN108FGZtnmAtD1XFHvtYuIRa+EQvai/E72POnDkyUIngJkpt4mKr6LGvyu+upG07w/GsDFGinj17tgywjzzyiLw4Fh3KTBcPRPbE4FpOIpCK4ClKouKPVFyti163JqLXZe/evfH5558X69UoSrGlESUXa1VwRa/omzZtKqsCb7/9dqsn+soS27d2pyax/cLVsOUl9tNUDVfaeNryVhGLiwbRu1WMdyxK9PYVFzFFS1+OIsZB5+bmYvXq1RalUluq6Bs2bChrKYoqmn+xnml+4eMkqopFtbyjxjIX3VcRlEXvcdE7vOh+ih7QhYl5puUV+X2J31ZZROm2U6dO8mJUlNpjYmLwySefVGhbRJXFauFyMpVSJ02ahKioqGLtX+Lqv+iVvmgjLc9QA3GyEEGi8FARMYRHVDcWJu6uJEqTomRcVH5+fqVvZSi2L4a0iJOyyZo1a6wOpyiPzp07yypUMWSo6D4V/o5Md4cqa7/Fd9u/f385rrhwu3dCQoK80OnZs6dsY6yoigzFKW3fiuZLpCeG51SWGL4kjseuXbvM88Rvo2iNhQieopT8z3/+02L74gJP7INo03QEMZSo8F3MxIVmXFycHDYliKFLosZl4cKF8kLEZN26dTh27FiF9/OBBx6Qfx/i5i5FFf0bfOKJJ2R7vfgtihu9mPaJyN5Yci0nESx69OghT/BC0eAqqpumTJkiO/uI9Q4dOiRPhuUp+YmqZFGlKDrLPP3007LtVJyIxNjFwh11RCcZMaZVjNETAV4EHDG4XpQcRCAXY0hFZ5KKeuaZZ+QJUdxpSgRwUdX59ddfm0ugFSVKkmKc59ChQ+V9jMV3Ijq4iEAmxm+KsaKm6nZhzJgxMu+mzmPWiHGeGzdulIFUdIwRN5gQ40zFyXrWrFmV2k9xchb7JgJhZe9dK46BCHAir+LYiLbv//znPzKYiABTGePHj5e3phTHQ4zVFBch//73v2UJT9ScFC7Rv/HGG3jvvffkumJ8qSgJinGvolbFUR13RBOIOC7iuxQXPCKQiTZX0bFKEL/RmTNnyuXiNyyqaMV64vcqxqmK8aoV8frrr8vfq2g7FX874nckxpmL2gPxdyM6O5mIDm/i+xTH+vnnn+fNKMhxlPc/dmHz58+X3fuLjuk0DcURwynEsB0xfu/222/Xtm/fXmw4i7WhOMLXX38th26IoRUdO3aUQwmsjXMV/v3vf8txpmI7YiiKGGc4fvx4OTSlMkNxhI8++kgOqRDDQsS+79mzp8ShON99953FZ0vK059//inHsop9FGNUxTCIwuOFxZCdf/zjH3JMpxh7WPjnWHQojrBv3z45xCUgIECOre3du7d5rG5ZeTTtu2lsamWG4txzzz1Wl61evVrmTYzvFOM2Z86cKYfBFB2TWlIa1oZiHTx4UM4TaYrjIsZxfv7558XSNA29adWqlRx3LcZ2Pv/888WGQIm02rZtW+58ie2IoVKlMX2n3377rfbGG2/I4TbiNynSszaOdPny5XL8sfiN1axZU45FvXDhgsU65RmKYxpnLsb0iu9G/M3Ur19frpOUlFRsu4MHD5ZpFv2tENmTTvzPgbGciFyEuEOT6Gcgak0qU2PiKGLIjqhJsnYXMSJ7YZsrEbksUTUvxtGKtlciR2KbKxG5HNFbWnQI/O9//yvbWUV7OJEjseRKRC5HPIhAlFZFkP3qq6/kQwWIHIltrkRERIqx5EpERFQVba7iFm6XLl2SN+euzI3XiYioaolKSnGzD/FAg8IPelApJyfH4mY0thDjxws/6MElg6sIrFV1ezkiIlJH3HlN3A7SHoG1ceN6iI+/oiQ90U4u2sydNcCWK7iaHiclDkplbjNHRERVS9ztTRSSCj8eUCVRYhWB9dz5FdDr/WxKKy0tC40aPiTTdOngaqoKFoGVwZWIyHnZu2lPH+ADfYCNDxYp40lizoDjXImISB0RGA02BkcXCK7sLUxERKQYS65ERKQOS64SgysREakj7kuk2XhvIhe4txGrhYmIiBRjyZWIiNQxaAqqhZ2/5MrgSkRE6rDNVWK1MBERkWIsuRIRkTosuUoMrkREpA6Dq8TgSkRE6mgKgqtIw8mxzZWIiEgxllyJiEgZnWaQk61pODsGVyIiUodtrhKrhYmIiBRjyZWIiBTfoUmzPQ0nx+BKRETqsFpYYrUwERGRYiy5EhGROiy5SgyuRESk+HmuBtvTcHKsFiYiIlKMJVciIlKH1cISgysREanDoTgSgysREanDkqvENlciIiLFWHIlIiJ1+Mg5icGViIiU0RkMcrI1DWfHamEiIiLFWHIlIiLFN5HQbE/DyTG4EhGROuwtLLFamIiISDGWXImISB2WXCUGVyIiUod3aJIYXImISB2WXCW2uRIRESnGkisRESmuFjbYnoaTY3AlIiJ1OM5VYrUwERGRYiy5EhGROuzQJDG4EhGROqJK18BqYVYLExGRU9u6dSuGDh2K8PBw6HQ6rFy50mL56NGj5fzC08CBA+26Tyy5EhGRU1cLZ2ZmokOHDnjqqadw//33W11HBNMvv/zS/N7b2xv2xOBKREROHVwHDRokp9KIYBoWFgZHYbUwERFVS2lpaRZTbm5updPasmULQkJC0LJlSzz//PNITk6GPTG4EhGR+nsLG2ycAERERCAoKMg8zZgxo1K7JKqEFy9ejE2bNmHmzJn4/fffZUm3oKAA9sJqYSIiUkczGCdb0wAQGxsLvV5vczvpiBEjzP+OjIxE+/bt0bRpU1ma7du3L+yBJVciIqqWJVe9Xm8xqeqE1KRJE9SuXRunT5+GvTC4EhHRTeXChQuyzbVu3bp22warhYmIyKl7C2dkZFiUQqOjoxEVFYWaNWvK6b333sMDDzwgewufOXMG48ePR7NmzTBgwADYC4MrERE59cPS9+zZg969e5vfjx07Vr6OGjUKCxYswMGDB/HVV18hJSVF3miif//+mDp1ql3HulYouK7vPQV+7vYdeEvk6obsmg5Xs6bbW3A1rnicXFWvXr2glXLLxA0bNsDRWHIlIiJ1+DxXicGViIiculq4OmJvYSIiIsVYciUiIoUU3ERCpOHkGFyJiEgdVgtLrBYmIiJSjCVXIiJShyVXicGViIic+g5N1RGDKxERqcOSq8Q2VyIiIsVYciUiInVYcpUYXImISB22uUqsFiYiIlKMJVciIlJHPJ1Gs7Fa19bPVwMMrkREpA7bXCVWCxMRESnGkisREanDkqvE4EpEROqIJ+IYbOzta/NTdaoeq4WJiIgUY8mViIjUYbWwxOBKRETqiBpdg63BFU7PYcE1uH0DRE4YBv+IWsiIScahmauQcijW6roht7dE05F3ILBpGLT8AlzZfw5HPl6LnMQ08zqhd7VGm38MhE+IHqnHL+HA9B+ReT4JjuRqeXK1/LhqnlwRj5MLYcnVcW2unnpfdJ0zEudW7MCGvtNw/rsd6DZnJDwCfKyu7xHgjTOL/8CmobPw2/APcS0zF53fH2Fe7t+gNjpNeQhH5v6MDf2mI2nPWXT98HHo3B3XhOxqeXK1/LhqnlwRjxO5Iof82sJ6tUHO5TTErNoDw7UC+ZqbnC7nW3Npw0Ek/nUCBdl5KMi5huhlfyG4bYT5j6PeoI5I3nsWiX+egCEvH6c+3wyv4ADU7NjQEdlxyTy5Wn5cNU+uiMfJtWgGTcnk7BwSXPXNwpB2Ms5innivbx5Wrs/X6twY6ecuQyswWE1PzM+ITpTzHcXV8uRq+bG2D66QJ1fE4+Sitz/UbJycnEOCq7ufF/LTcyzmXUvPgYefd5mf1beoi5Z/74ejH681z/Pw85KfL5qeu3/Z6anianlytfy4ap5cEY8TuSK7dGiqN6ADIt+4V/47Oz4FSbvOyHYViw0H+CAvJbPUdAKbhqLbvFE4PPsnmYZJflZesfYYzwBvFGTmwl5cLU+ulh9XzZMr4nFycezQZL/genHDATmZRAzrgsYjehS74oxe+lepfzi3fvoUjs3fgIvrb6QlpJ2OR1CLuub3oq0loHEI0s4kwF5cLU+ulh9XzZMr4nFycQyujqsWjt9yFD4hQfKPSOfhLl99agcifssRq+sHNAmRfzgnFm7EhTX7ii2/uC4KtW5pgpAeLeDm6Y7mT/VCXmqW7JLvKK6WJ1fLj6vmyRXxOJEr0mla2S3HaWlpCAoKwvLOr8HPvXLtFsEdGiJyvHEcW2ZsEg59sBpXD8XIZT6hQei1/GVseXgechJS0eGd+1H/nk6yJ2BhpuWC6EnY+qUB8o8y9cQlHJj2g+PHULpYnlwtP9U1T0N2TYerWdPtLZs+z+Nkf6bzeGpqKvR6vd3Svzr7Seh9vWxLKzsPwa9/abd9dangSkSuedJWEVyrI1c7Tg4LrjNHqwmuExY5dXDlqGoiIiLFeG9hIiJSRlSGajZ2SCpHhWq1x+BKRETqsLewxOBKRETqMLhKbHMlIiKntnXrVgwdOhTh4eHQ6XRYuXJlsWrmSZMmoW7duvD19UW/fv1w6tQpu+4TgysREakvuRpsnCogMzMTHTp0wPz5860unzVrFv75z39i4cKF2LlzJ/z9/TFgwADk5FjeJlMlVgsTEZE6Km68r1Xs84MGDZKT9aQ0zJ07F2+//Tbuvdd4283FixcjNDRUlnBHjLjxuEKVWHIlIqJqKS0tzWLKza34/aGjo6MRHx8vq4JNxHjc7t27Y/v27bAXBlciIlJGM6iZhIiICBkITdOMGTMqvD8isAqipFqYeG9aZg+sFiYiomrZWzg2NtbiDk3e3s5zh0CWXImIqFrS6/UWU2WCa1hYmHxNSLB8KpJ4b1pmDwyuRETk1L2FS9O4cWMZRDdt2mSeJ9pvRa/h2267DfbCamEiIlKmcJtpZVX08xkZGTh9+rRFJ6aoqCjUrFkTDRo0wCuvvIJp06ahefPmMti+8847ckzs8OHDYS8MrkRE5NT27NmD3r17m9+PHTtWvo4aNQqLFi3C+PHj5VjY5557DikpKejZsyfWr18PHx8fu+0TgysREamjKajWreA41169epV6s39x16YpU6bIyVEYXImISB1RpWtQkIaTY3AlIiJlxOPmNFsfOccb9xMREVFRLLkSEZE6rBaWGFyJiEgdUaOrKUjDybFamIiIqCpLrgM3T7K4z6OzW9PtLbiaIbumw5W44jFa7YJ54lU6mbBDkxGrhYmISB22uUq84CQiIlKMJVciInLqewtXRwyuRESkDquFJVYLExERKcaSKxERKcNqYSMGVyIiUkeMojEoSMPJMbgSEZEy4slvmmOfOFctsc2ViIhIMZZciYhIGba5GjG4EhGROhyKI7FamIiISDGWXImISBlWCxsxuBIRkTLsLWzEamEiIiLFWHIlIiJ1DDrjZGsaTo7BlYiIlGGbqxGrhYmIiBRjyZWIiJTRNJ2cbE3D2TG4EhGRMqwWNmJwJSIitUNxDLan4ewYXG0Q3L4BIicMg39ELWTEJOPQzFVIORRrdd2Q21ui6cg7ENg0DFp+Aa7sP4cjH69FTmKaeZ3Qu1qjzT8GwidEj9Tjl3Bg+o/IPJ/kwBy5HpXHKLBJCFq/Mhg1WoXDq4Y/1veZivyMHAfnCKhZKE+ZMck4OHMVrpaQp8IaDu+KDm8Ox+E5a3F22TY5L+S2FmjzjwHwCQmSZzTxuzs892ekn0mAI7nicaKbGzs0VZKn3hdd54zEuRU7sKHvNJz/bge6zRkJjwAfq+t7BHjjzOI/sGnoLPw2/ENcy8xF5/dHmJf7N6iNTlMewpG5P2NDv+lI2nMWXT98HDp3HqLqcowM+QbE/XoIUVP+h6rMk8hD9IodWN93GqK/24HupeTJxLt2IJo+3hNpp+It5qeejMP2fyzC+n7TsGHgDCT8dQLdZj0GR3LF43QzM7W5ajZOzo5n7koK69UGOZfTELNqDwzXCuRrbnK6nG/NpQ0HkfjXCRRk56Eg5xqil/2F4LYR5uBZb1BHJO89i8Q/T8CQl49Tn2+GV3AAanZs6OCcuQ7VxygzJgmxq/c6vFRXWF0recpJTpfzS9N+/DCc/GIz8tKyLOaL70NMJprBAN+6NRx6UeeKx+mmZtBBs3HiONebmL5ZGNJOxlnME+/1zcPK9flanRsj/dxlaAUGq+mJ+RnRiXJ+8t5oxXt/c1B9jJw1T3X7tIWHvzcu/ByFBkO7FFvuGxqEXkv/AQ8/b0AHnPzyd4fm2RWPExGDayW5+3khP92yHedaeo7xBFUGfYu6aPn3ftj7xrfmeR5+XvLzRdNz9y87PXLMMaouebpWgTx5BvqgzZiB2PGPRSWmmZ2QinV9p8m0I+7pjJyEVDiSKx6nmxnvLWzE4FpO9QZ0QOQb98p/Z8enIGnXGdlWVJhoI8pLySw1ncCmoeg2bxQOz/5JpmGSn5VXrI3JM8AbBZm5SvPhyux9jKoqTx2u5ymrhDx5Bvggt4Q8tRkzCDGr9yIzNrnMbRVk5eHc9zsx8Jc3sXXUv5B16SrswRWPE93Aca5GDK7ldHHDATmZRAzrgsYjehS7io5e+lepJ4NbP30Kx+ZvwMX1N9IS0k7HI6hFXfN70X4U0DgEaWw3qjbHqDrkqcGwLmhiJU9nSshTna5NZZWw6TMiENdoXU+25e+ZaKW0pwPcvD1ku6u9gqsrHieiotihqZLitxyVwxfEiUHn4S5ffWoHIn7LEavrBzQJkSeDEws34sKafcWWX1wXhVq3NEFIjxZw83RH86d6IS81Sw4zoOpxjAQ3Lw+4eRqvSd283OV7R4q7nqcG1/PUoIw8/fH0Qmx57BP8/vincko5dhGnv/4TB2esksvD746Ef/2agE4nS4uRY4egIPuaHJLjKK54nG5mtnZm0kydmipg8uTJ0Ol0FlOrVq1QlfiLq6RradnY/doSRI4fhnbjhiIzNgm7xy4xt4f5iE4iy1/GlofnyTaspo/1hFewH9q8OlhOJqbloodj1Lvfoe3Ye+SJJvXEJZk+O2lUn2MkSnN9V71unt9//ZvyddO9s5Edl+KwPO16bYns/Rs5bigyYpOws1CeROek3stfxuaH58m21NzkDIvPi57oYsynuHAT/OoGo/WL/eEdHCB73149egHbX/oS+Q5sjnDF43Qzq6o217Zt2+LXX381v/fwqNrwptO0srORlpaGoKAgpKamQq/Xw1Ws6fYWXM2QXdPhSlzxGLni5ZIrVoG52t+Svc/jpvRPDXscgZ5eNqWVfi0PzVd/Xe59FSXXlStXIioqCtWFK/5NEBGRC9xEIi0tzWLKzS25RuXUqVMIDw9HkyZN8NhjjyEmJgZVicGViIiUMRh0SiYhIiJCloZN04wZM6xus3v37li0aBHWr1+PBQsWIDo6GnfccQfS02/cIMXR2OZKRETVss01NjbWolrY29v62OdBgwaZ/92+fXsZbBs2bIgVK1bg6aefRlVgcCUiompJr9dXqn24Ro0aaNGiBU6fPo2qwmphIiJyqRv3Z2Rk4MyZM6hb98a9AxyNwZWIiJw6uI4bNw6///47zp07h23btuG+++6Du7s7HnnkEVQVVgsTEZFTu3DhggykycnJqFOnDnr27IkdO3bIf1cVBlciIlLGoOnkZGsaFbFs2TJUNwyuRESkTGVuX1iUrZ+vDtjmSkREpBhLrkREpAyf52rE4EpERMoYoKDNVTz70MmxWpiIiEgxllyJiEgZFTeB0Gz8fHXA4EpERMqIwGhgcGVwJSIidVhyNWKbKxERkWIsuRIRkTKG65MtbP18dcDgSkREyrBa2IjVwkRERIqx5EpENhuya3pV7wJVEwat4jfet5aGs2NwJSIiZVgtbMRqYSIiIsVYciUiIsXVwrA5DWfH4EpERMqwWtiI1cJERESKseRKRERqHzkHPnKOwZWIiJThw9KNGFyJiEgZMcbVYPM4V+cvubLNlYiISDGWXImISBlNQZurxjZXIiKiG9jmasRqYSIiIsVYciUiImXYocmIwZWIiJQR7aUa21xZLUxERKQaS65ERKQMb9xvxOBKRETKsM3ViNXCREREirHkSkREyrBDkxGDKxERKcM2VyMGVyIiUoYlVyO2uRIRESnGkqsNgts3QOSEYfCPqIWMmGQcmrkKKYdira4bcntLNB15BwKbhkHLL8CV/edw5OO1yElMk8sDm4Sg9SuDUaNVOLxq+GN9n6nIz8hxcI5cj8pjJDR7shcaDL8FnoG+yLp4Bcc+3YCknacdmCOgZqE8ZcYk4+DMVbhaQp4Kazi8Kzq8ORyH56zF2WXbzPN9QvRo9+pg1OnWTL6/euQCdoxZZNc8kOtitbARS66V5Kn3Rdc5I3FuxQ5s6DsN57/bgW5zRsIjwMfq+h4B3jiz+A9sGjoLvw3/ENcyc9H5/RHm5YZ8A+J+PYSoKf9zYC5cm+pjFHpXazR5rCd2j12CDX2m4uzSv3DLrMfkdhyZJ5GH6BU7sL7vNER/twPdS8mTiXftQDR9vCfSTsVbzHf38USPfz2N1FPx+GXoLKzv/z6OL9ho51zQzTAUx2DjVFHz589Ho0aN4OPjg+7du2PXrl2oSgyulRTWqw1yLqchZtUeGK4VyNfc5HQ535pLGw4i8a8TKMjOQ0HONUQv+wvBbSOgczcegsyYJMSu3ov0MwkOzonrUn2M/OrVROrRC+ZjdHFdFNw83OR8R6lrJU85yelyfmnajx+Gk19sRl5alsX8iCGdkZeahVNfbEFBVh60AgNSjl20cy6I1Fq+fDnGjh2Ld999F/v27UOHDh0wYMAAJCYmoqowuFaSvlkY0k7GWcwT7/XNw8r1+VqdGyP93GV5MiPnOEZxGw/Bu1Yg9C3qAm461B/SGdmJaQ69IKpMnur2aQsPf29c+DnKah5zElPRfe4oDNz4Fu786gWE9Ghhl32nm4OmaBLS0tIsptzcXFgzZ84cPPvss3jyySfRpk0bLFy4EH5+fvjiiy9QVRhcK8ndzwv56ZZtotfSc+Dh513mZ8XJueXf++Hox2vtuIek+hjlXsmQJds7vnoBg/98D23H3oOD7/8IQ14+HJknkYfy5skz0AdtxgzEwQ9WWV3upfdF3V5tcf7HXdgwcIYs3d7ywSPwr++40ji54MPSNdsmU2/hiIgIBAUFmacZM2YU215eXh727t2Lfv36mee5ubnJ99u3b0dVYYemcqo3oAMi37hX/js7PgVJu84Ua2sT7V55KZmlphPYNBTd5o3C4dk/yTTIeY5R82f6IOT2Ftj8t4+RdekqanVqhC4fPIIdL36JtFNxdstTh+t5yiohT54BPsgtIU9txgxCzOq9yIxNtro8PzsPVw7FIP73Y/K9eE09fgl1ujdH5oWdyvNDVBGxsbHQ6/Xm997exS8ik5KSUFBQgNDQUIv54v3x48dRVRhcy+nihgNyMokY1gWNR/QoVtqJXvpXqSftWz99Csfmb8DF9TfSIuc4RkEt6+LSpsOyl7CQvC9adhCq3a2p3YJr0Tw1GNYFTazk6UwJearTtamsEjZ9RgTiGq3roWbHhtgz8VuknYxH7a5N7LLvdHMSjSgGBWkIIrAWDq7OhNXClRS/5Sh8QoLkCVzn4S5ffWoHIn7LEavrBzQJkSftEws34sKafVbXcfPygJun8XrHzctdvqfqc4zEcJe6fdrBN6yGeZhPjTb1i7WB2lPc9Tw1uJ6nBmXk6Y+nF2LLY5/g98c/lZPorHT66z9xcIaxmjj25/0IahmO0J4tAZ1Ovor3iTtOOSxP5Fo0Ua2r2T6VV+3ateHu7o6EBMu+D+J9WFj5+lfYA8/elXQtLRu7X1uCyPHD0G7cUGTGJskhGqb2MJ/QIPRa/jK2PDwPOQmpaPpYT3gF+6HNq4PlZGJa7lu3Bvquet08v//6N+XrpntnIzsupQpy6PxUH6Mzi7fKKtke/3kWngG+spfu8QW/IGn3GYfmaddrS2Tv38hxQ5ERm4SdhfLkGxqE3stfxuaH5yE7IRW5yRkWnxftw2L8tOghLIhSuCjBtn15ELpMexiZF65g94Sl5tI5UXXn5eWFLl26YNOmTRg+fLicZzAY5PuXXnqpyvZLp2lamcN1RS8t0ZicmprqtEV0a9Z0ewuuZsiu6XAlrniMXLF/+DAX+925Inufx03pL+08Dn7uZXcaLE1WQS4e3fdhufdVDMUZNWoUPvvsM3Tr1g1z587FihUrZJtr0bZYR2HJlYiInPoOTQ8//DAuX76MSZMmIT4+Hh07dsT69eurLLAKDK5EROT0N+5/6aWXqrQauCh2aCIiIlKMJVciIlKGN+43YnAlIiJl+DxXI1YLExERKcaSKxERKcNqYSMGVyIiUobB1YjVwkRERIqx5EpERMqwQ5MRgysRESkjbqhrsLFat+yb8lZ/rBYmIiJSjCVXIiKqls9zdWYMrkREpExFn8dqja2frw4YXImISBmWXI3Y5kpERKQYS65ERKQMbyJhxOBKRETKiLioKUjD2bFamIiISDGWXImISHG1sM7mNJzdTR1cXaFHWlGru70FVzJs13S4mjebToGrOdzS9fI0/uidcCX5BZkO2Q6rhY1YLUxERKTYTV1yJSIitdhb2IjBlYiIlOFNJIxYLUxERKQYS65ERKSMeFycxkfOMbgSEZE64kHnBj4sncGViIjUYcnViG2uREREirHkSkREyrC3sBGDKxERKcNxrkasFiYiIlKMJVciIlKG9xY2YnAlIiJlWC1sxGphIiIixVhyJSIiZTjO1YjBlYiIlOFQHCNWCxMR0U2jUaNG0Ol0FtMHH3ygfDssuRIR0U3VoWnKlCl49tlnze8DAwOVb4PBlYiIquVQnLS0NIv53t7ecrKVCKZhYWGwJ1YLExGR8pKrwcZJiIiIQFBQkHmaMWOGkn0U1cC1atVCp06dMHv2bOTn50M1llyJiKhaio2NhV6vN79XUWodM2YMOnfujJo1a2Lbtm144403EBcXhzlz5kAlBlcb1GzfAJEThsE/ohYyY5JxcOYqXD0Ua3XdoJbh6PDmcPiFB0PnpkN6dCKOzv8FV/afk8trdW6M2xc+g/ysXPNnYtfsx6EPf3LK/LSfeC/qD+xw4wNuOnj4eOH3J+Yj9cQlR2XJ5dSoF4TxW19Gbmaeed7ZHeew5LllJX7mloc64c7neiCwTgDSEtLx2ydbceCnw3JZeNsw3Pf+EATXNx7HxNOXsWHWJpzbHYOq0PGhzhg8dQg2vr8Bu7/aWeJ6gaGB6PfmADTu0US+v3TgIpY98415eaeHO6PH/90B3xq+iNl1Hmvf/gmZlzPssMc6uOmaQ6cLBuAJIA8GLQaaFn99uTvcdC2g09WSfWAN2kVo2vlS0itr/Yqm53jiWayaoue5isBaOLiWZOLEiZg5c2ap6xw7dgytWrXC2LFjzfPat28PLy8v/P3vf5elYhXB24TBtZI89b7oNmckjn6yHhd+3o/6gzuh+5yR+PW+j5CfkVNs/az4q9g9YSmy41Pk+7q92uDWOSOxfuD7MOQaqySupWdjXd9pcIX8HPxglZxMmj56Oxre15WBVZGZt3+MnPQbF2IlqdsmDMPeG4xFT34jg3DTHo0x8r+PIO5YPBJPJ+HqxVR88/x3SLmUKtdv278VRv33EUzv9hHyr/8uHSUgJAC3Pn0bEk8klLqep68nHls8EodWHsTaN1fjWs41hLWpa17e8NZG6D2uH5Y9/Q0un0pE/3cG4t4P78PSUUvssNciCOShwHAAgPg70cPdLRIGLRcarsrAC50nCgw7ZPB1d+sAA3KgadbzWNb6FU2vKmgKOiRpFVz/tddew+jRo0tdp0kT44VYUd27d5fVwufOnUPLli2hCttcK0kEk5zLaYhZtQeGawXyNSc5Xc635lpqtjkQQaeDZtDg4e8Nn1rqe6lVx/w0GHYLYn7aa88skBU169dAysUUGViFM9uikRqXipBmdeT77JRsc2DV6QCDwQDvAG9ZynW0AZMG489//SH3qTTt7+uArKvZ+GvBH8jLzINWoCHu0I2Ltvb3d8Th1Qdx6eBFXMu+hi0f/YYGXRuiRv0adthrUXoU363pAjQNGlKg0wXJ06tOFwKDIRqAuFDJliVNN92NCwFLZa1f0fRuHnXq1JGl0tImUUK1JioqCm5ubggJCVG6Tyy5VpK+WRjSTsZZzBPv9c1L74E2aNPbcPf1gpuHO2LX7kPWpavmZWJ+/7UTZKBK3h+No59skAHPWfNjEhwZAf8GtRC7Zp/y/b5Zvbzuebh5uOHCgYtYP/NXXD6bbHW9k3+cQa8Xe6LZ7U1wZttZNOvZFL6BPji3x7La95394+Hl5wV3Dzfs++EArl64fuHkIK0GtJZB/fCqg+jwQMdS123QrSHSE9Lw8H8eRXj7eki5cBVb523Bma2n5fKQliHYs2S3ef3M5ExkJmWgTssQpNg9X27QQQ+DlgjADzqdKL8Uqo7WMgBdgxI+W9b6FU2valTnoTjbt2/Hzp070bt3b9ljWLx/9dVX8fjjjyM4WFTtq8PgWknufl64lm5ZXSree/iVXmcvqn3dvD0Q3rst3LxFG41RxvnL+P3xT5F+7jK8g/3R9pXB6PbRE9g66l8OuReY6vwU1uDeW5Dw5wnkXslUus83o6yrWfjXff/FpaPx8PL1RO+X7sSTix/HvIELkJtxox3WRJTcolYewhP/flheAGkFBvxv4mpkJFkei6mdZsHD2wPtBraWr47ko/dBn/H98O1T35Rv/SBfNOzeCD/84zt89/wyNLurOe7/54P477CFuBpzVV4k5Bb5Leek5cDLX117WkncdC2hIQsaLoueCdC0AotKTk2WOEv6ft3LWL+s5dVDdX4qjre3N5YtW4bJkycjNzcXjRs3lsG1cDusKtXrqFRj9QZ0QIc37pX/zopPQdKuM7KdsjDPAB/kppQdQESb5IX1B9Br2RhknLuMKwfOIzc5Q06CeD3w/koM/u0dBDSohYzzSU6XHxNRqq3XNxJ731muPA83gw7D2mH4tCHy36J6d96ghbhw0FgFKtpc183YiI73RqJB5wic2nqm2Oe7PNgRPZ+5DQse+AIJJxIQ2jIUI/87AjlpuTix5ZTFuqKNNWrVIby87v9w+UwSzu+13pnNVm2HtsOg94x5Sr2UgotRF3Dg+yhcPX+lXJ/Py8rDxf0XcHLTCflevMYfiUPjnk1xdekeudw70DKQegf6IC+z7DZqWxg7Nvleb38VRCB0u94uawwXOnnKLaktu6z1K5oeFSV6Ce/YIdqr7Y/BtZwubjggJ5MGw7qgyYgeFuvoW9TFmaV/lTtNUZIQPXMLByMzO5dWHZWfev3b41pmLhK2nVS05zeXA6sPy6k0Wim/lfA2YTj5+2nEHzd2eBGvp/84ixZ3NS0WXE3cPd1Rq1FNuwXXIz8dlpPJC5vGyCrhrqO6y/fi33XbhSOiSwP8MOa7Yp9PPJ6ARrc2LjH9xBOJCG11oznDr6YfAuoE4PIJUVVrz8Cqvx5YRRAUsq4HQf8bVbk60ZZd0gVrWetXNL2qUZ2rhR2JHZoqKW7LUfiEBMmgpPNwl68+tQMRv+WI1fVDe7aEvlkodO5ucPf2RPPRd8EnRI9k01CcLo3lsBbBM8hXDmVJP5uIjNhkp8yPiUhHtMW6xF9LNVC/Qz3UaVpbDpvx8vPEgPF95fk2Zt8Fq+vH7L+A5nc2RUhzYwcm8dr8jqayWllo2bs5wlqGwM1dB08fD9z1fE/ow/QOHYrz1cOfyyrdz+/9TE5xh+Ow4/NtWPfuGqvri17CoW3C0KyX6DkL+Sren/3DWHI/+EMU2g6LRN3IcHj4eKDX2D6I2X3ebu2txsAadD2wFi5FGqBpiXBzExcC7gB84aarB4Nm2beh/OtXNL2qoSn6z9mx5FpJ19Kyseu1JWg/fhgixw1FRmwSdo5dYm639A0NQu/lL2Pzw/OQnZAKryA/tH15EHzq6FGQl4/00wnY+epiZF28Yh432nny3+Cp90N+Zi6S9p7FzrGLHRaUVOdHCGhcB8Ft62PfOysckoebQc0GNXD3q71lb17Rnhp74CK+HP01cjOMVZ5BdfV4ZcMLmDvgX0iNS5Ol3hrhQRj57xHwr+WPrJQs7P0+Cnu/i5Lr+9f0w+A374Y+VC+rhRNOJmLxM9/iSkzxjmn2klmk/Vf8nkR+sq8aew2LEqzovPRhZ+PN1VNir+KHl79Hvwl3Y/icB3A15oos4Yr5wvkd57Blzm944NOHZHuuCKyrxv1op733hptbPWiaAe5ut5nniqExBu0kDNopuKHF9WWmcak3hs24uUVC01KhacaLmbLWL2s5VR86rbQ6pUL3dxS3nkpNTS3XgF5nsbrbW1W9C1SGYbumw9W82XQKXE2AC16mjz96J1xJWlomatUcYrfzuClOvNzwDXi7+diUVq4hB/POz3DqmOOCfxJERFRVqnNvYUdicCUiImXYocmIHZqIiIgUY8mViIiUEb14NFvvLewCJVcGVyIiUsZwfbKFrZ+vDlgtTEREpBhLrkREpAw7NBkxuBIRkToK2lzhAsGV1cJERESKseRKRETKsEOTEYMrEREpw6E4RqwWJiIiUowlVyIiUobVwkYMrkREpIx40JpmY72urZ+vDhhciYhIGY5zNWKbKxERkWIsuRIRkTJ8nqsRgysRESnDamEjVgsTEREpxpIrEREpw5KrEYMrEREpbnPVbE7D2TG4upg8g2vV9Ot0rvcTHddoElxNkKcrnA4tebj3givxcE+r6l24qbjemYuIiKoMq4WNGFyJiEgZ3rjfiMGViIiUEe2tBpvbXJ0/urpWAx0REVE1wJIrEREpw2phIwZXIiJSho+cM2K1MBERkWIsuRIRkTJ8nqsRgysRESnDca5GrBYmIqKbxvTp09GjRw/4+fmhRo0aVteJiYnBPffcI9cJCQnB66+/jvz8/ApthyVXIiJSxqBgnKvBjuNc8/Ly8OCDD+K2227D559/Xmx5QUGBDKxhYWHYtm0b4uLiMHLkSHh6euL9998v93YYXImISO2N+zXb07CX9957T74uWrTI6vJffvkFR48exa+//orQ0FB07NgRU6dOxYQJEzB58mR4eXmVazusFiYiomopLS3NYsrNzbX7Nrdv347IyEgZWE0GDBggt3/kyJFyp8PgSkREyquFDTZOQkREBIKCgszTjBkz7L7/8fHxFoFVML0Xy8qL1cJERKT2Dk2wPQ0hNjYWer3ePN/b29vq+hMnTsTMmTNLTfPYsWNo1aoVHIXBlYiIqmWHJr1ebxFcS/Laa69h9OjRpa7TpEmTcm1bdGTatWuXxbyEhATzsvJicCUiIqdWp04dOakgehGL4TqJiYlyGI6wceNGGeTbtGlT7nQYXImISBmDpqDkasc7NIkxrFeuXJGvYthNVFSUnN+sWTMEBASgf//+Mog+8cQTmDVrlmxnffvtt/Hiiy+WWC1tDYMrEREpI57FqlXj57lOmjQJX331lfl9p06d5OvmzZvRq1cvuLu7Y82aNXj++edlKdbf3x+jRo3ClClTKrQdBlciIrppLFq0qMQxriYNGzbEzz//bNN2GFyJiEgZTcEj4zQ4PwZXG9Rs3wCRE4bBP6IWMmOScXDmKlw9FGt13aCW4ejw5nD4hQdD56ZDenQijs7/BVf2n5PL20+8F/UHdrjxATcdPHy88PsT85F64pLd8+JTKwCd37oXwa3D4VtHj42PzkfqydLHdIXf1RqRLw+Ab0ggUo7HYe/UlUg/n1Tu5aoNHjwYEya8jsjIdrh27Rq2bv0Dr7wyFhcvXjSvc++9wzB79kzUq1cP+/btxzPPPIcTJ06UmGZZ61c0PRV8Ar0x5K270e7uVnD3dMfl6GQseHgRruWUfu/TgeP6oO+LPbHoueU4svHGPt76SGf0efEO+Af74syO8/hu4k9Iv5wBR6ndOgy9Jw+Fvn4wdDodrpy9jO0f/4q4vTFW16/ZrA5uf70/6rQJh2+wH/5z2wfIS7e8uUBwk9roOWEAwjpGwJBvwNlNx7F50moH5ejmVt1vf+govIlEJXnqfdFtzkhEr9iB9X2nIfq7Heg+ZyQ8Anysrp8VfxW7JyzF+runY13faTjz9Z+4dc5IuHkbr28OfrAKP/eaYp5OLPwVGecvOySwmh7xFL/tFLaNW1qu9QMa1ka3aX/DwTk/Y3WfGUjcfRY95jwGnbtbuZbbQ1CQHjNnzkZERCM0btxM3lFlxYpl5uUtWrTAN98swauvjkPNmnXw22+bsWrVD7KNxZqy1q9oeirodMBTnz8CwzUDZvb5FJM6zMT3b6xBQX7pZYW6rUPRpm9zpCakW8xvelsjDJ7YD0te+h6Tb/kI6UkZeHTufXCk9EupWP/KCnx++yz8t8dMRC3ahiH/ehTu1/82ihLB8vSGo9j09kqry/3qBGD4F6PkOl/c+SG+7PURDn1rObSCyN4YXCupbq82yLmchphVe2C4ViBfc5LT5XxrrqVmIzs+xfhGp4Nm0ODh7w2fWoFW128w7BbE/LQXjpJ7JRNnv9+Fq0dulPJK03BQB1zeE424P0/CkJePY//dAu9gf9Tu2LBcy+3h22+XyXaSzMxMZGVlYe7cf6J7927mYPf4449h8+YtWLt2rbyN2tSp02RX+zvuuMNqemWtX9H0VGjZqzlqhAdh5eR1yE7NkYPtLx2NlwGnJKKm5MEZQ7By8noUXCuwWNb1wY7Yt/IgYqMu4lr2Nayb/RuadG+ImhHWnxZiD7mp2UiPS72+s4BWoMHL3xt+tQOsrp9yLhnHftiPK6cSrS7vOPI2XNgZLdcpyM2Xf59Jx8p/Zx1S8zxXzcbJ2bFauJL0zcKQdjLOYp54r29e+iDjQZvehruvF9w83BG7dh+yLl0ttk5wZAT8G9RC7Jp9qK6CmocipVD+tQID0qIT5fzLe6PLXO4Id911p7wri+huL7RvH4moqAPm5eIRUkePHpPzt2zZUuzzZa1f0fRUaNq9IZLPX8GIOfehxR1NZPXtls+2Ye8PB0v8zJ1P34q444k4u/N8sWV1W4Xgr692m99nJGXKNMX8K7HXLwYd5JltE+DpJ/423HB8VRTSL1Zu++G3NETS8Xjcv+RJBDeujStnLmPbhxuRcKh8F45kG1YLGzG4VpK7nxeupedYzBPvPfxKHwclqoRFVXB477Zw8/a0uk6De29Bwp8nZGmyuvLwLSH//t7lWm5vxidZvIcHHxxhnifGsKWkWJ6wxfvAQOu1B2WtX9H0VPCt4YNmPRrjx3fXYfm4lYhoXw9PL3oUVy6kIHpX8TZKUQLtMbIr5g75t9X0vP28kJ1meZzEe28HHafCRJWwqApuenfrEquEy8MnyBfNB7fDT//3DRIPXUTbB7vgnvmP4JshnyK3SF6J7IXBtZzqDeiADm/cK/+dFZ+CpF1nZLtrYZ4BPshNKTsgGnLzcWH9AfRaNgYZ5y7jyoEbJQpRqq3XNxJ731kOe4oY2B5d3hwm/50Zl4qND39Soc/nZ+fJ/BYm3udn5pZruQqPPvoIPvtsgfz3+fPn0a6dsUNYu3btsG7dGrz00hj52CiTjIwMefPvwsT79HTLdsjyrl/R9Cqj073t8MD0IfLfVy+m4NSfZ5FyKRXbFhtLm+f2xsrOSW36tLAaXP/2/hBs+GizrEK2JjcrT3aQKswn0Ae5Co9TUS3uiUSvd415Sr+Ugm+HG4+hIKpxT645hEdWPo+Us0mI22+9g2BprmXlIf5ALOKvf/bQt7vR6enbEdahPs7/cVphTsgallyNGFzL6eKGA3IyaTCsC5qM6GGxjr5FXZxZ+le50xRVw6KnceHgWq9/e1zLzEXCtpOwp9j1B+VUWamnEhDU4kYVuOiopG9cB6mnE8q1XIWlS7+VU2EisP766wZMnPgmvvnGsnPWwYOH0LHjjR7ZHh4eaNOmNQ4dOmw1/bLWr2h6lbF/1WE5mdzytw6IHNi63J9v3rMJwtuEYdg7A+R73yAfjPhoOHat2I+fpv0iq4vFchP/Wn7QhwTI+fZycu0hOZX1txHUsFalgmvSiXjZzkxVw/RcG1vY+vnqgB2aKiluy1H4hATJIKvzcJevPrUDEb/F+vP+Qnu2hL5ZqAwy7t6eaD76LviE6JF8fSiOiUhHtMXC4PgrNzcvDznJf3u6G/8tuqdacX7dAYR0bYKw25vLdVs/fRdyU7KQtP98uZbbg7hlmQisb789CYsW3bgDi8nXX3+DPn16Y9CgQfKBx2+99SaSkpKwdetWq+mVtX5F01Ph8Ibj8PD2wK2PdpEBJKJjPbTt1xJHfrU+/GfabR/j43s+M09pCelYPW0Dfv2ncR93fxeFzsMjEdEhHJ4+Hhj0eh/ZNuvI9taGdzVHrRYh0LmL4Wce6PJsT/iH6nFpb8m/FXcvd7hf/62KV/He5Oj3+9C4dyuERtaT31Hbh7rA3dMDcVEVD9RUtY+cc2YsuVbStbRs7HptCdqPH4bIcUOREZuEnWOXmNsZfUOD0Hv5y9j88DxkJ6TCK8gPbV8eBJ86ehTk5SP9dAJ2vroYWRevmNMMaFwHwW3rY987K6okT/dve9f8775f/Z98/f3vn+Py3nOyl2/Pfz6BlXdOk/Mzzidh1zvfo8Nr98AvRI+rJy5h29hvZMel8iy3h3Hjxsqbd3/88UdyMmnTJlI+uurkyZN4/PGRmDdvDurXry/HpQ4bdp+5w1PPnj1ldXJgoLGnbFnrl7XcHnLSc/HF09/ivvcGYehbdyMlPg0/vvszzu0xBo7GXRvg6S8fxdvtPpDvU+Mtq6gNBg1ZV7PN7axntp/Dulm/YeSCh+AX5IMzO89j6Ss/wpF8a/jh9nH9ERCqR35uPpJPJWDtC0uRFmvs7Fe3cwMMXfgY/t3N+CzPwPAgjPzlFfPnn/p9nHxd3H+uHNYjSrtb31+H/rMfgE+wH5JPJWLti0uLjYUlsiedVo4+z2K8oGhLSk1NLdfjf5zF6m5vwdXkGVyrMuLBvfZ/OLKjjWs0Ca6mkb/zlzSKevHwjYtNV2Dv87gp/e765+Ghs61DXL6Wi51pC5w65rDkSkREyhiu/2cLWz9fHbhWMYeIiKgaYMmViIiU0XQaNJ2tvYWdv5mBwZWIiJTRFPT21VwguLJamIiISDGWXImISBnRGUnHDk0MrkREpA7v0GTEamEiIiLFWHIlIiJlDDoDdDb2Fma1MBERUSFsczVicCUiImUYXI3Y5kpERKQYS65ERKQMewsbMbgSEZEyBhRAhwKb03B2rBYmIiJSjCVXIiJSRtwXWLO5Wtj57y3M4EpERMpwnKsRq4WJiIgUY8mViIgUd2hyszkNZ8fgSkRECtk+FEek4exYLUxERKTYTV1yHbZrelXvApVBw9Sq3gUiqgCDJqp03RSk4dxu6uBKRERq8Q5NRgyuRESkjIYCaDaWXEUazo5trkREdNOYPn06evToAT8/P9SoUcPqOjqdrti0bNmyCm2HJVciIlLGeAMIg4I07CMvLw8PPvggbrvtNnz++eclrvfll19i4MCB5vclBeKSMLgSEdFNc/vD9957T74uWrSo1PVEMA0LC6v0dlgtTERE1VJaWprFlJub67Btv/jii6hduza6deuGL774AppWsYDPkisRESmjaaJDk87mNISIiAiL+e+++y4mT54Me5syZQr69Okj22V/+eUXvPDCC8jIyMCYMWPKnQaDKxERVcs219jYWOj1evN8b29vq+tPnDgRM2fOLDXNY8eOoVWrVuXa/jvvvGP+d6dOnZCZmYnZs2czuBIRkfPT6/UWwbUkr732GkaPHl3qOk2aNKn0fnTv3h1Tp06V1dIlBfiiGFyJiEjxOFedzWlURJ06deRkL1FRUQgODi53YBUYXImISBlNU3CHJs1+Q3FiYmJw5coV+VpQUCADp9CsWTMEBATgp59+QkJCAm699Vb4+Phg48aNeP/99zFu3LgKbYfBlYiIbhqTJk3CV199ZdGmKmzevBm9evWCp6cn5s+fj1dffVX2EBZBd86cOXj22WcrtB2dVo7+xaILdFBQEFJTU8tV/01ERNWLvc/jpvRr67vDTWdbuc2g5SMpbadTxxyWXImIqFoOxXFmDK5ERHTT3KHJUXiHJiIiIsVYciUiIsW9hXU2p+HsGFyJiEgh0eZqexrOjtXCREREirHkSkREyhirdHUK0nBuDK5ERKQMg6sRq4WJiIgUY8mViIiUEY+L09l8437nL7kyuBIRkTKsFjZitTAREZFiLLkSEZEyKu4LrPHewkREREXvC2xQkIZzY3AlIiJlVLSXamxzJSIioqJYciUiImVYcjVicCUiImVUjFHVXGCcK6uFiYiIFGPJlYiIlGG1sBGDKxERKcPgasRqYSIiIsVYciUiIoVUlDqdv+TK4EpERMqwWtiI1cJERESKseRKRETKcJyrEYMrEREpo2kKbtwv03BuDK5ERKSQeFyczuayq7NjmysREZFiLLkSEZEyxp6+OhvTcP6SK4MrEREpZHtwZbUwERERFcOSKxERqaOgWhisFiYiIrpBU1Clq7FamIiIiIpiyZWIiBRihyaBwZWIiBTSFHT2ZbUwERERVabkahrQm5aWVp7ViYiomjGdv+1/gwbRHcn5S54OCa7p6enyNSIiwt77Q0REdiTO50FBQcrT9fLyQlhYGOLj45WkJ9ISaTornVaOyxiDwYBLly4hMDAQOp2td94gIiJHE6d6EVjDw8Ph5mafFsGcnBzk5eUpSUsEVh8fH7h0cCUiIqLyY4cmIiIixRhciYiIFGNwJSIiUozBlYiISDEGVyIiIsUYXImIiBRjcCUiIoJa/w/0ndzpKjipNgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def plot_values(V: np.ndarray, title=\"Value function\") -> None:\n", + " \"\"\"Plot the value function V on the maze as a heatmap.\"\"\"\n", + " grid_values = np.full(\n", + " (n_rows, n_cols), np.nan\n", + " ) # Initializes a grid the same size as the maze. Every cell starts as NaN.\n", + " for (\n", + " s,\n", + " (i, j),\n", + " ) in (\n", + " state_to_pos.items()\n", + " ): # recall that state_to_pos maps each state index to its maze coordinates (i,j).\n", + " grid_values[i, j] = V[\n", + " s\n", + " ] # For each reachable cell, we write the value V[s] in the grid.\n", + " # Walls # never get values, and they stay as NaN.\n", + "\n", + " fig, ax = plt.subplots()\n", + " im = ax.imshow(grid_values, cmap=\"magma\")\n", + " plt.colorbar(im, ax=ax)\n", + "\n", + " # For each state:\n", + " # Place the text label at (column j, row i).\n", + " # Display value to two decimals.\n", + " # Use white text so it’s visible on the heatmap.\n", + " # Center the text inside each cell.\n", + "\n", + " for s, (i, j) in state_to_pos.items():\n", + " ax.text(\n", + " j, i, f\"{V[s]:.2f}\", ha=\"center\", va=\"center\", color=\"white\", fontsize=9\n", + " )\n", + "\n", + " # Remove axis ticks and set title\n", + " ax.set_xticks([])\n", + " ax.set_yticks([])\n", + " ax.set_title(title)\n", + " plt.show()\n", + "\n", + "\n", + "plot_values(V_random, title=\"Value function: random policy\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "8275a1eb-b58e-4e05-ae5d-5635ff9a1556", + "metadata": {}, + "source": [ + "The next function `plot_policy` visualizes a policy on the maze.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "id": "c1ab67f0-bd5e-4ffe-b655-aec030401b78", + "metadata": {}, + "outputs": [], + "source": [ + "def plot_policy(policy: np.ndarray, title=\"Policy\") -> None:\n", + " \"\"\"Plot the given policy on the maze.\"\"\"\n", + " _fig, ax = plt.subplots()\n", + " # draw walls as dark cells\n", + " wall_grid = np.zeros((n_rows, n_cols))\n", + " for i in range(n_rows):\n", + " for j in range(n_cols):\n", + " if maze_str[i][j] == \"#\":\n", + " wall_grid[i, j] = 1\n", + " ax.imshow(wall_grid, cmap=\"Greys\", alpha=0.5)\n", + "\n", + " for s, (i, j) in state_to_pos.items():\n", + " cell = maze_str[i][j]\n", + " if cell == \"#\":\n", + " continue\n", + "\n", + " if s in goal_states:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"G\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"blue\",\n", + " )\n", + " elif s in trap_states:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"X\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"red\",\n", + " )\n", + " elif s == start_state:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"S\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"green\",\n", + " )\n", + " else:\n", + " a = policy[s]\n", + " ax.text(\n", + " j,\n", + " i,\n", + " action_names[a],\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " color=\"black\",\n", + " )\n", + "\n", + " ax.set_xticks(np.arange(-0.5, n_cols, 1))\n", + " ax.set_yticks(np.arange(-0.5, n_rows, 1))\n", + " ax.set_xticklabels([])\n", + " ax.set_yticklabels([])\n", + " ax.grid(True)\n", + " ax.set_title(title)\n", + " plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "48037254-dccc-4f9c-a4d7-349adba5c74f", + "metadata": {}, + "source": [ + "Now let’s visualize the `random_policy`. Does it seem like a good policy?" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "id": "d452681c-c89c-41cc-95dc-df75993b0391", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAGgCAYAAAC0SSBAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAH6JJREFUeJzt3Q1wlNWh//Ff1pCQxLw0AQUvEVulcm16UyJSShlem5bSoS1cKCUoOmqrcOWC5Ho7KUVIteoES6UjU65ixb6kV+xM6aU4XAIBW4tN9Z8/oyGASlHBvzLgSyIGQ0L2P+fETUKCxw0meXZPvp+Z4+4+2eA5eZ7d356XZ5+EcDgcFgAAHyH0UT8AAMAgKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIiiADiZNmmRLxCuvvKKEhARt3Lgx0HoBQSIoEPfMm7h5M4+UgQMH6rOf/axuu+02HTt2LOjqAXEvMegKAD3lxz/+sT796U/rgw8+0NNPP61f/OIXevLJJ1VTU6PU1NTz+jeHDx+uU6dOacCAAT1eXyBeEBTwxte//nWNHj3a3r/55puVk5OjNWvW6I9//KPmzZt3Xv9mpIcC9GcMPcFbU6ZMsbeHDx9Wc3Oz7rrrLl1++eVKTk7WZZddph/+8IdqbGx0/hsfNUdx4MABfec739HgwYOVkpKiK6+8UsuXL7c/27Vrl/2dP/zhD13+vfLycvuzZ555pkfbCvQmggLeOnTokL01PQvTw7jzzjtVUFCgn/3sZ5o4caLuvfdeffe73+32v/v888/ri1/8oiorK/W9731Pa9eu1be//W1t2bLF/txMhufm5uq3v/1tl98120xYfelLX+qBFgJ9g6EneKOurk4nTpywcxR//etf7ZyF+bQ/cuRI3XrrrTYsHn74YfvcRYsW6aKLLtL9999vewCTJ0+O+v+zePFimcu4VFdX69JLL23bft9999lb02O49tpr7bCXqVNmZqbdfvz4cW3fvr2t5wHEC3oU8MZXvvIVOxRkPs2bnsKFF15oh3/27Nljf75s2bKznl9cXGxvt27dGvX/w7zZ//nPf9aNN954VkhEAiJiwYIFdljr97//fdu2xx9/3A6BmRAB4gk9Cnhj3bp1dllsYmKiLr74YjtvEAqFbFiY2yuuuOKs5w8ZMkRZWVl69dVXo/5//OMf/7C3eXl5zueZXsw111xjh5puuukmu83cHzt2bJd6ALGOoIA3xowZ07bq6Vw6fuLvC6ZXsWTJEh09etT2Lv72t7/pwQcf7NM6AD2BoSd4z5wL0dLSopdeeums7eZkvHfffdf+PFqf+cxn7K05N+PjmOGvCy64QL/73e9sb8KcizF37tzzaAEQLIIC3ps+fbq9feCBB87abiabjW984xtR/1tmDmTChAn65S9/qddee+2sn5kJ7o4GDRpkz+34zW9+Y4Ni2rRpdhsQbxh6gvfy8/N1/fXX66GHHrI9CLM09u9//7see+wxu6y1OyuejJ///OcaP368XWr7/e9/354Nbs63MJPie/fu7TL8NHv2bHvfnMcBxCOCAv3Chg0b7LCROXHOTG6bieySkhKtXLnyvILHzDesWLHCfk2IWY5rhq/MCXidzZgxQ5/61Kfs0Nc3v/nNHmoN0LcSwp37ywB6jFkOe8kll9jAeOSRR4KuDnBemKMAetHmzZvtuRdmCAqIV/QogF5QVVVlv+rDzEuYCWxzFjcQr+hRAL3AzF0sXLjQfk3Ir371q6CrA3wi9CgAAE70KAAATgQFAKBnzqMw31XT8SIvZl3422+/bb/rv6+/QwcA8MmYWYf33nvPLt82X5rZI0FhLvJSWlr6CasGAIglR44c0bBhw3pmMrtzj8JckMV8H/8dd9zhzYXnTapeddVVqq2ttT2meOdbewzaFB9oU+xramrS6tWr7dfaRC6u9Yl7FOY6w6Z0ZkIiKSlJvhwIqamptj0+HAi+tcegTfGBNsWPaKYOmMwGADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAKTHaJzY2NtoSUV9fb29DoZAtPoi0g/bELtoUH2hT7OtOOxLC4XA4mieuWrVKpaWlXbaXl5crNTW1ezUEAASqoaFBRUVFqqurU0ZGRs8Exbl6FLm5uXrjjTeUk5MjHzQ1NamiokKFhYUaMGCA4p1v7fG9TTU1NWppaZEvn1bz8vLYTzHs9OnTuvvuu6MKiqiHnpKTk23pzBwEvhwIvrbJt/b42ibz5uPDG1BH7KfY1Z02+DHYBgDoNQQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEA6JkLF/W1/675bz2691HtfXOv3j71tlIHpCo7JVvDM4cr/+J8TR8xXV+74mtBVxMfWrJkiR5//HG9+eabQVcFQH8IigV/WKBfP//rs7bVN9bb8sq7r+ipV5/Sq3WvEhQxxFxO8dixY0FXA0B/CIptL287KySuHnq1vnb513Rh0oU63nBc1W9U65mjzwRaRwDoT2IuKLYf2t52/4rsK1R1c5UuCF1w1nNMz+KFYy8o3q5Pe8cdd+imm27SVVddFXR1cB4Xol+2bJndh8OHDw+6OjgH9lE/msxubmluu//uB+/aoabOMpIz9OVLv6x4cebMGV133XVas2aNHn300aCrg/NQXV2tDRs2aMKECTp06FDQ1cE5sI/6UVAUDC1ou3+i4YQ+++BndfVDV+vWP92qh//Pw3r57ZcVT5qbmzVv3jyVl5dr0aJFKisrC7pKOA9jx47Vli1bdPz4cftGdPDgwaCrhE7YR/1o6Onaf7lW655dp+f+33P2cUu4xc5LmBIx/tLxevDrDyp/SL5i3Zw5c7R582ZlZmYqISFBixcvjnoV0YgRI3q9fpBWrFihd955J6rn5uXl6dlnn9XEiRP11FNP6corr+z1+oF9FLSYC4rEUKIqF1Tq3qfv1S//7y917P2uK2mefu1pFf66UPsW7dPgtMGK5XmJ3bt3t60KWrduXdS/O3v2bIKij5jhwNdff71bv2NWeNXW1vIm1EfYR8GKuaEnIz05XfdMvUdvFL+hmoU1euSbj+j6/OuVnpTe9hyzAqrzEtpYEwqFtHPnTmVnZysrK0tVVVUKh8NRlUmTJgVd/X7j6NGjUe2TkydPtu2X0tJSzZw5M+iq9xvso2DFZFBEmKGaz130Od046kZt/PZGPb/weYUS2qv80lsvKdYVFBSosrJSiYmJKiws1J49e+TT5OH69eu7bN+/f7/Wrl0rn7z33nuaNm2a7SHed999uvPOO4OuEjphH/WjoafH9j6mD5o/0LzPz7OrmzpKG5Bmg8LMWxhZA7MUD/Lz87Vr1y5NnTrV9jDGjRsnHyxfvlzbtm1TQ0ND27Z9+/ZpypQpOnXqlGbNmqXc3Fz54OWXX9YLL7xgV67dfvvtQVcH58A+6kdBcfjdwyp9qlRL/3epnbT+wsVfsF/d8dapt/T72t+ftXx22hXTFC/MBJt5Ex00aJB8sWnTJk2fPl3FxcUaPLh1rmjy5Ml2Pfv27du9CQlj1KhR9o3Ip/3nG/ZRPwqKCNOr2PGPHbacy/cKvqeJl01UPPHtAE5PT7c9ihkzZtgeU2Q58I4dOzR69Gj5xrf95yP2UT8JiqVjl+rzF31elYcr9dwbz+nNk2/q+PvHdSZ8RoNTB+vqS662E9uz/nlW0FWFGQ5MS9PWrVv1rW99y85ZVFRU2E92APwRc0Fh5h3+9ap/tQXxISUlxQ41AfBTTK96AgAEj6AAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAA9MylUBsbG22JqK+vt7dNTU22+CDSDtoTu3xuUyjkz+e2SFvYT7GrO+1ICIfD4WieuGrVKpWWlnbZXl5ertTU1O7VEAAQqIaGBhUVFamurk4ZGRk906MoKSnRsmXLzupR5Obmqra2VklJSfIlYfPy8lRYWKgBAwbIh09AFRUVqqmpUUtLi3zg2z7quJ9oU2xr8uz1dPr06aifG3VQJCcn29KZ+YP58EfryBzYvhzcBvsoPtCm+NDiyeupO23wY7ANANBrCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAnTQ3N2v+/PkaOXKkXnzxRfliyZIlGjJkiHzh636KRQQF0OnykHPmzLHXgj948KAmTZqkAwcOyAfm2sjHjh2TD3zeT7GIoAA+1NjYqFmzZmnz5s1tF5s/efKkfRPat29f0NXDh9hPfY+g6MPr0xYXF6u2tjboquAjzJ07V1u3blVJSYlmzpxpt23fvl2nTp3S5MmTdfTo0aCrCPZTIBKD+d/2L2fOnNGCBQtsNzkUCmn16tVBVwnnsHTpUl1zzTVavny5brjhBrtt7Nixqqio0JYtWzRs2LCgqwj2UyAIij6YcCsqKtITTzyhRYsWqaysLOgq4SOYoQtTOhszZowtiA3sp75HUPQyM+FmxlIzMzOVkJCgxYsXR71CZcSIEb1ePwD4OARFL89L7N69u23Fybp166L+3dmzZxMUAGICk9m9yMxH7Ny5U9nZ2crKylJVVZXC4XBU5VxdawAIAkHRywoKClRZWanExEQVFhZqz549QVcJ/Uh1dbXWr1/fZfv+/fu1du3aQOqE+MPQUx/Iz8/Xrl27NHXqVNvDGDduXNBVQj9hVgZt27ZNDQ0NbdvMuQZTpkyxy0nN+Qi5ubmB1hGxj6DoI3l5efYFOmjQoKCrgn5k06ZNmj59uj2HZ/DgwXabOdfAnNlszj0gJBANhp76ECGBvpaenm57FCYcjh8/3rZke8eOHfbcAyAaBAXgubS0NHsms5kjy8nJscOfo0ePDrpaiCMEBXAOGzdutKvPfJGSkmKHmk6cOKFRo0bJF77tp1hFUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgFOiotTY2GhLRH19vb0NhUK2+CDSjqamJvkg0g5f9o+P+6hjW3xsU1lZmVpaWuTLsZeXl+fN66k77UgIR3nB2VWrVqm0tLTL9vLycqWmpnavhgCAQDU0NKioqEh1dXXKyMjomR5FSUmJli1bdlaPIjc3V7W1tUpKSpJPnxgKCws1YMAA+fCprqKiQjU1Nd59qvNlH3XcTz62ycdjr8aTNp0+fTrq50YdFMnJybZ0Zv5gPvzROjIvVl9esAb7KD742CYfj70WT9rUnTb4MdgGAOg1BAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIij6WHNzs+bPn6+RI0fqxRdfDLo6+AhLlizRkCFDgq4GEBMIij6+9OCcOXPsdcYPHjyoSZMm6cCBA0FXC+dgriN87NixoKsBxASCoo80NjZq1qxZ2rx5c9uFzE+ePGnDYt++fUFXDwA+EkHRR+bOnautW7eqpKREM2fOtNu2b9+uU6dOafLkyTp69GjQVYSHzHWRi4uLVVtbG3RVEMcSg65Af7F06VJdc801Wr58uW644Qa7bezYsaqoqNCWLVs0bNiwoKsIz5w5c0YLFiywQ52hUEirV68OukqIUwRFHzFDTKZ0NmbMGFuAnl40UVRUpCeeeEKLFi1SWVlZ0FVCHCMoAA+ZRRNmPiwzM1MJCQlavHhx1Ku9RowY0ev1Q3whKAAP5yV2797dtnpr3bp1Uf/u7NmzCQp0wWQ24BkzH7Fz505lZ2crKytLVVVVCofDUZVzDY8CBAXwoerqaq1fv77L9v3792vt2rWKJwUFBaqsrFRiYqIKCwu1Z8+eoKuEOMbQE/AhsyJt27ZtamhoaNtmznGZMmWKXcZszoPJzc1VvMjPz9euXbs0depU28MYN25c0FVCnCIogA9t2rRJ06dPt+cdDB482G4z57iYM+rNOS/xFBIReXl5NuwGDRoUdFUQxxh6Aj6Unp5uexQmHI4fP962zHTHjh32nJd4RUjgkyIogA7S0tLsGfRmXD8nJ8cO2YwePTroagGBIigCsHHjRrvCBLEpJSXFDjWdOHFCo0aNCro6QOAICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcEpUlBobG22JqK+vt7ehUMgWH0Ta0dTUJB9E2lFcXKwBAwbIlzZVVFSorKxMLS0t8uW4y8vL8+a48/3YK/akTW+99ZbuueeeqJ6bEI7y4s2rVq1SaWlpl+3l5eVKTU3tfi0BAIFpaGhQUVGR6urqlJGR0TNBca4eRW5urn70ox8pKSlJPn2yKyws9OITQ+QTkC/t6dimmpoa73oUPu4n2hTbPYqhQ4dGFRRRDz0lJyfb0pl5sfrygo0wB4EPB4Kv7TE47uIDbYpd3WmDH5MLAIBeQ1AAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAFYsmSJhgwZEnQ10M9w3MWut96Sysqkr35VuuQSaeBAcw0gaehQacIE6Y47pL/8RYruMnM9L+oLF6HnmCtKHTt2LOhqoJ/huItNDz0kLVsmvf9+15+9+WZrMSFx//3SG29IQWQ9QQEAAVm9WvrP/2x/nJAgTZ4sjR0rXXih9Pbb0t690tNPSx98EFw9CQoACMD+/VJJSfvjnBzpf/5HGjeu63NPnpR+/WspJUWBYI4C581cs7q4uFi1tbVBVwX9iC/H3c9/Lp050/54/fpzh4RhehcLF0qZmQoEPQqclzNnzmjBggUqLy9XKBTSatOHBnqZT8fdzp3t9z/1KWnWLMUsehTotubmZs2bN8++WBctWqQys1wD6GW+HXevv95+f8QIKdTh3fjAgdb5is7lhhsCqSo9CnTfnDlztHnzZmVmZiohIUGLFy+OennmCPOKAM6Dz8ddQoJiGkGBbo8P7969u2255bp166L+3dmzZ8f8Cxaxycfj7p/+SXrppdb75tacIxEJjIsual0RZaxcKTU0BFdPg6GnPlJdXa31Zraqk/3792vt2rWKF2ZceOfOncrOzlZWVpaqqqoUDoejKpMmTQq6+v0Ox13sHndTp7bfN8tgzYqniOxs6T/+o7UEtdKpI4KijyxfvlwLFy7UmjVr2rbt27fPHsQrVqzQkSNHFC8KCgpUWVmpxMREFRYWas+ePUFXCR+B4y523XabdMEF7Y9vvbX1nIlYRFD0kU2bNmn8+PF2Wd+TTz5pt02ePFmNjY3avn27cnNzFU/y8/O1a9cuDRw40H7SQ2ziuItdn/ucdNdd7Y/NGdijR0szZkirVkk/+Yl0881Sfb0CxxxFH0lPT9e2bds0Y8YMe6BHVnHs2LFDo83REYfy8vLsp9NBgwYFXRV8BI672FZSIqWltZ6d3djYel7Fn/7UWs7FnJQXBHoUfSgtLU1bt2613eacnBz7iSheX6wRPrxYfcdxF9v+/d+lw4dbexHjx0uDB0uJia1zE5deKhUWtv6sulr66U+DqSM9ij6WkpJiu/xAX+K4i21Dh7aubjIlFtGjAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICANAz18xubGy0JaK+vt7ehkIhW3wQaUdTU5N8EGmHL+3p2Jbi4mINGDBAvrSpoqLCy/3kY5vKysrU0tKieHf69Omon5sQDofD0Txx1apVKi0t7bK9vLxcqamp3ashACBQDQ0NKioqUl1dnTIyMnqmR1FSUqJly5ad1aPIzc1VbW2tkpKS5EuPIi8vT4WFhV58Wo18UvWlPQZtig8+t6mmpqbf9SiiDork5GRbOjN/MB/+aB2ZA9uXg9vH9hi0KT742KYWT97zutMGPyYXAAC9hqAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKPpYc3Oz5s+fr5EjR+rFF18MujowDh2SLrxQSkhoLV/9qtT5wo/mcWFh+3PS0qSXXlI8WbJkiYYMGRJ0NRCHCIo+vqLUnDlz7OVjDx48qEmTJunAgQNBVwuXXy799KftjysqpHXrzn7Ogw9KO3a0P77/fmnECMUTc8nLY8eOBV0NxCGCoo80NjZq1qxZ2rx5c9v1aU+ePGnDYt++fUFXD7fcIk2f3v74Bz+QIj0+c2seR0ybJi1c2Pd1BAJCUPSRuXPnauvWrfba4zNnzrTbtm/frlOnTmny5Mk6evRo0FXEhg1STk7r/YYGacECk/DSdddJp061bs/Olh55JNBqAn2NoOgjS5cu1d1336177rmnbdvYsWPtxdpvueUWDRs2TPF4zd3i4mLV1tbKC0OHSr/4RfvjqippzBjp739v32Z+fsklgVQPHz+0e9ttt+nVV18NuireISj6iBliWr58eZftY8aM0V133aV4c+bMGV133XVas2aNHn30UXljzhxp/vz2x88/336/qEj6zncCqRY+XnV1tTZs2KAJEybokFmggB5DUOC8Vm7NmzfPTsovWrRIZWVl8oqZuDa9i44uvrjrBDdiiumhb9myRcePH7dhYRaMoGck9tC/g37ErNwyk/KZmZlKSEjQ4sWLo16eOSIeVgqZ+aK33z57m3n8yivSF74QVK36tRUrVuidd96J6rl5eXl69tlnNXHiRD311FO68sore71+viMo0O15id27d7ctt1zXjU/Zs2fPjv2gaGpqnbw2k9jn2v7cc1JyclC167fM8Obrr7/erd8xS4HN/BlB8ckx9IRuCYVC2rlzp7Kzs5WVlaWqqiqFw+GoipmniXkrV0p797Y//rd/a79fUyP96EeKt3H79evXd9m+f/9+rV27VvHCrAqM5hiLLDk3SktL21YY4pMhKNBtBQUFqqysVGJiogoLC7Vnzx55wbSj43zLjTe2zlfcdFP7tjVrpL/8RfHCLKBYuHChXXQQYc7bMW+mZjjnyJEj8sV7772nadOm2R7vfffdpzvvvDPoKnmDoMB5yc/P165duzRw4EDbw4h7778vXX+9Wc7V+viyy6QHHmi9b24/85nW+y0trc87eVLxYNOmTRo/frxdxvzkk0/abea8HXMCqDmPJzc3V754+eWX9cILL9hQ/EHHEyTxiTFHgfNmJg3Np9NBgwYp7hUXm3ea1vuhkPTYY1J6eutj8z1Qv/qVNHFia5AcPizdfrv08MOKdenp6dq2bZtmzJhhgz2yam3Hjh0aPXq0fDJq1CgbFl4cjzGGHgU+ES9elNu2Sf/1X+2PTQhMmHD2c7785bO/xsOcxb11q+JBWlqa/VYAM0yYk5Nje4C+hYRXx2MMokcRgI0bN9qCGGG+u6nzt8Wey09+0lriUEpKih1qAs4HPQoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAOiZS6E2NjbaElFfX29vQ6GQLT6ItKOpqUk+iLTDl/YYtCk++NymkGfvd9FICIejuViwtGrVKpWWlnbZXl5ertTU1O7VEAAQqIaGBhUVFamurk4ZGRk906MoKSnRsmXLzupR5Obmqra2VklJSfIlYfPy8lRTU6OWlhbFO9/aY9Cm+ECbYt/p06ejfm7UQZGcnGxLZ+YP5sMfzec2+dYegzbFB9oUu7rTBj8G2wAAvYagAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwClRUWpsbLQloq6uzt42NTXJF6FQSA0NDTp9+rRaWloU73xrj0Gb4gNtin2R9+5wOPzxTw5HaeXKleZfo1AoFIr8KYcOHfrY9/8E85/z6VG8++67Gj58uF577TVlZmbKB/X19crNzdWRI0eUkZGheOdbewzaFB9oU+wzo0KXXnqp3nnnHWVlZfXM0FNycrItnZmQ8OGP1pFpj09t8q09Bm2KD7QpPobUPvY5fVITAEDcIigAAL0TFGYYauXKleccjopXvrXJt/YYtCk+0Ca/2hP1ZDYAoH9i6AkA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAufx/wkBHS6w2LYIAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_policy(policy=random_policy, title=\"Policy\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "cbad5bf1-0150-4c3f-8cce-c82e0f1d1695", + "metadata": {}, + "source": [ + "**Exercise 9.** Define your own policy and evaluate it using the functions `policy_evaluation(...)` and `plot_values(...)`. **Can you identify an optimal policy visually?** Plot your own policy using `plot_policy`. \n" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "id": "929707e6-3022-4d86-96cc-12f251f890a9", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdcAAAGbCAYAAACWHtrWAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAATRhJREFUeJzt3Ql8VNX5N/Dfmcm+EwgkgbALCLKJiigquCHiglvdRau2r0utW9X2X/e2VG2t1lq1dcHWvYtaN1xZVDYFQVkFZAlbWJOQPZl73s9zbiaZSSbJhJwsE35fP+Ms986Ze2fCPPd5zjl3lNZag4iIiKzx2GuKiIiIBIMrERGRZQyuREREljG4EhERWcbgSkREZBmDKxERkWUMrkRERJYxuBIREVnG4EpERGQZg2sE27hxI5RSmDFjRru8/syZMzFq1CjExcWZ7cjPz0dHJNt23333tfdmdFoTJkwwl47yd0nUETC4tpGzzjoLCQkJ2L9/f4PrXHrppYiJicGePXvQ0ck2/uhHP0J8fDyefPJJ/POf/0RiYmK7bc/777/PAEpEHUZUe2/AwUIC5zvvvIM333wTV1xxRb3lJSUlePvtt3Haaaeha9eu6Oi++uorc6Dw4IMP4uSTT27vzTHBVYJ8qABbWlqKqCj+qbeVPn36mPc8Ojq6vTeFqN0wc23DzDU5ORmvvPJKyOUSWIuLi00QjgQ7d+4012lpaejopGzN4Np2pCQs77nX623vTSFqNwyubUTKp+eeey4+/fTTmsAUSIKuBF8Jwnv37sXtt9+O4cOHIykpCSkpKZg8eTKWLVvW7P4vvyuvvBJ9+/YNesxxHDz22GMYNmyY+TLs0aMHfvrTn2Lfvn1Nvsa0adPM7SOPPNJ8mUr7Ql7Df7ux7Zo9e7Z53htvvIHf/va36NWrl9mGk046CevWrav3/IULF+L0009Hly5dTPl5xIgRePzxx2v2TbJWIW36L431uX7zzTfmPZX3Vt5jed0FCxYErSN9hvLcL7/8ErfeeisyMjLMa59zzjnYtWtX0LoFBQVYvXq1uW6KvEdnnHGGeQ+OOOII87chn7XcF//973/NfXk/xowZY7bV74UXXjDbFPiY3+9+9zsT0LZu3drga8v7IM+XbZWyvuy/VEp+/vOfo6ysLGjdqqoqU5kYMGAAYmNjzXb/6le/Qnl5eaP711Cfq/815X2UfR48eDD+7//+zyybNWuWeY5UdkL925Bl8+fPb/R1iToSBtc2JFmpfGFJQAkkwfTDDz80X9rypfPDDz/grbfeMl/Ajz76KH7xi1/gu+++wwknnIBt27ZZ2x4JpNL2scceawLVVVddhZdffhmTJk1CZWVlg8+TL8Sf/OQn5vYDDzxg+lulrQPx+9//3nyhysHEL3/5SxPg6mbvH3/8MY4//nisXLnSBIE//vGPmDhxIt59992a/TjllFPMbdkW/6UhK1aswHHHHWcOVu644w7cfffd2LBhgwn+EsTr+tnPfmbWvffee3HdddeZ8v6NN94YtI7sw6GHHhoyOIQiBxCXXHIJzjzzTEyfPt0c0Mhtef9vueUWXHbZZbj//vuxfv16E5DkQEicf/755m9E1qtLHpN96NmzZ5OvL21KMJXXloOWP//5zzWfqd8111yDe+65B4cffjj+9Kc/mb8/Wf+iiy5Cc3377bcYO3YsPvvsM1x77bXm723q1KnmvRSy3Tk5OQ3ulwT4cePGNft1idqN/J4rtY2qqiqdlZWlx40bF/T4008/Lb+pqz/88ENzv6ysTPt8vqB1NmzYoGNjY/UDDzwQ9Jg874UXXqh57IQTTjCXuqZNm6b79OlTc//zzz83z3355ZeD1ps5c2bIx+uS15T1vvrqq6DH5TXkteqqu12zZs0yzz/00EN1eXl5zeOPP/64efy7776rec/69etn2t23b19Qm47j1Ny+4YYbzPNCkcfvvffemvtTp07VMTExev369TWPbdu2TScnJ+vjjz++3j6efPLJQa91yy23aK/Xq/Pz8+utG/hZNET2RdadN29ezWPy2ctj8fHxetOmTTWPP/PMM+Zxeb/8Lr74Yp2dnR30N7JkyZKwXl/eB1nvrLPOCnr8+uuvN48vW7bM3F+6dKm5f8011wStd/vtt5vHP/vsswY/21B/l/K+yvsbuG8i8H395S9/af7GA9/XnTt36qioqKDPjygSMHNtQ1Kyk6N+KW9J6Syw7CUlWSlNCinBeTzuR+Pz+czIXCldShltyZIlVrblX//6F1JTU03Gt3v37pqLlCHltaRM1xYkW5YR0n6SUQrJ3oWUPyWrvPnmm+v17waWfsMl7+dHH31ksqb+/fvXPJ6VlWUyyS+++AKFhYVBz5GMLvC1ZBulnU2bNtU8JqVpieOhSuKhDB06NCgTk6xOnHjiiejdu3e9x/3vh5ABcVLBCPyMJLuTjPa8884L6/VvuOGGetm5f2BY4LWUwwPddttt5vq9995DuKSEPnfuXPz4xz8O2jcR+L7KfknJ+d///nfNY6+//rqp9kgmTxRJGFzbmL/k6R/YtGXLFnz++ecm6PoHgEgJUMpwhxxyiAm03bp1M/1UUloLp08vHGvXrjVtde/e3bQdeCkqKgrZL9wa6n7ZSp+q8Pf7SllUHHbYYVZeT77oZWS2HKjUJWVdee9zc3ObtY0Hom6bcqAjpDQa6vHA15IDIjkY8JdQZZtfffVVnH322abfPhzytxVIyq5yQOc/6JMDB7k/cODAoPUyMzPNQU7ggUVT/AcGTX2GQ4YMMX34gaVhuX300UfX2w6ijo5DKNuYZIbyJSJfhjI4RK4l4wnsZ5SBKdIPKEf6MqAkPT3dfNFJ9ubve2uIZAJuJTSYZFqBpB0JrKH6uIQE2QPRUDYprx9q9GhDI0pD7UN7aY1tbKjNcF5L1pEs++9//zv++te/mgFXksm2JLtr6HM7kOpAS0j2Kv3qctApWaz0wf/lL39p020gsoHBtR1IIJXgKZmoZLCSRcgRu5+UxWTAznPPPRf0PDkDkmSxjZGsKrCE6Fc305BM5ZNPPjGDmaScaIu8fqgzNcnrB5ZhwyXbKZYvX97ofNpwg4AcNMjJPNasWVNvmYxmlYOYutljRyRBSAZ2yYCgDz74wOyXDERrTuWiX79+QQOs5IDLP6Jc5qrKfVlPMnq/vLw88/nK8nD5P3f5DJsiFRwpRctBp3+u7IUXXhj2axF1FCwLtwN/liojMZcuXVpvdKxkJnWzIukjbWyKRWAwkiAROFVERrpKdlN3tKhkk5IZ1yV9XAd6KkN5fck2Kioqah6TUb11S63hkpGqEgRkylDdbQp8j/xnh2pqu+W9PfXUU8284sB+bwkacqAzfvx4Mz2luZozFccGmYokl2effRb/+c9/TFBqzlxe/9QlvyeeeMJcy/QkISOIhbzvgWT0upgyZUrYryWBX0Z7P//889i8eXPQsrp/53LwKNvw0ksvmaqKnFSlqQNKoo6ImWs7kGBxzDHHmC94UTe4yhQcmeIig31kPZmGI1804WR+UkqWL0DJYq6++mrTd/r000+buayBA3VkWoVMYZGpFRLgJeBIliCZigRymSoh0z6aS6ZvSOYtX4oSwKXPVL4o/Rloc0km+dRTT5lpKnIeY3lPpL9RAplMqZEpTP5yu7jpppvMvvsHj4Xym9/8xkzvkUB6/fXXm6D0zDPPmDLkww8/fEDbKVNwZNtkHmq4g5psZK8yhUk0tyQsg8RkTrV8TjLATj4jKTWPHDnSLJdrmcv8t7/9zRywyN/LokWL8OKLL5rBYFJZaQ6Z6iPvtxwsyQAx+TcgBzcyMEr+/urul/9vL9TBH1FEaO/hygerJ5980kxXOOqoo+otk6k4t912m5m2I1Mzjj32WD1//vywpjyIl156Sffv399MNxk1apSZ5lF3Ko7f3/72Nz1mzBjzOjJVYvjw4fqOO+4wU1MOZCqO+OMf/6h79uxpplXItn/99dcNTsX517/+FfTchvbpiy++0KeccorZxsTERD1ixAj9xBNP1CyXKTs/+9nPdEZGhlZKBU3LqTsVxz91ZdKkSTopKUknJCToiRMnBk2NaWwf/dseOD2muVNxpkyZUu9xeb5MKQr1fjzyyCP11t++fbuZEjRo0CAdLv9UnJUrV+rzzz/fvJ9dunTRN954oy4tLQ1at7KyUt9///1mKlR0dLTOyckx02Xk7zNQuH+Xy5cv1+ecc45OS0vTcXFxevDgwfruu++ut40yNUu2KTU1td42EUUKJf9r7wBPRM0nU6cki5fuBenDD4ecoUlOTiHdBh213CrdEtnZ2aZaUXfcAVGkYJ8rUYSS0wtKv/nll1+OzkTOTibBP9QPXBBFCva5EkUYOYWgnApSzsks/Z91zxkdqeTUkzKCXvpZR48ebfp5iSIVgytRhJHBbvPmzTPTqPyjfDsDGbgmA6tk4Bp/aJ0iHftciYiILGOfKxERUXuUheVMLXJ6NTlvaVufDo2IiFpOipT79+83I7H9Pwxim/yMYeAJZFpCftBDftO4UwdXCayRcEo4IiJqnJwtrVevXq0SWPv164kdO/ZaaU9+JEJOdhKpATas4Or/pQ35UA7k1HBERNS+5AxtkiSF+8tJzSUZqwTWjZveQEpKQovaKiwsQd8+PzJtdurg6i8FS2BlcCUiilyt3bWXkhSHlKQW/hhIE7/+FQk4FYeIiOyRwOi0MDh2guDK0cJERESWMXMlIiJ7mLkaDK5ERGSPnJdIt/DcRJ3g3EYsCxMREVnGzJWIiOxxtIWycORnrgyuRERkD/tcDZaFiYiILGPmSkRE9jBzNRhciYjIHgZXg8GViIjs0RaCq7QR4djnSkREZBkzVyIiskZpx1xa2kakY3AlIiJ72OdqsCxMRERkGTNXIiKyfIYm3fI2IhyDKxER2cOysMGyMBERkWXMXImIyB5mrgaDKxERWf49V6flbUQ4loWJiIgsY+ZKRET2sCxsMLgSEZE9nIpjMLgSEZE9zFwN9rkSERFZxsyViIjs4U/OGQyuRERkjXIcc2lpG5GOZWEiIiLLmLkSEZHlk0jolrcR4RhciYjIHo4WNlgWJiIisoyZKxER2cPM1WBwJSIie3iGJoPBlYiI7GHmarDPlYiIyDJmrkREZLks7LS8jQjH4EpERPZwnqvBsjAREZFlzFyJiMgeDmgyGFyJiMgeKek6LAuzLExERBFt7ty5OPPMM5GdnQ2lFN56662g5VdeeaV5PPBy2mmnteo2MXMlIqKILgsXFxdj5MiR+PGPf4xzzz035DoSTF944YWa+7GxsWhNDK5ERBTRwXXy5Mnm0hgJppmZmWgrLAsTEVGHVFhYGHQpLy8/4LZmz56N7t27Y/DgwbjuuuuwZ88etCYGVyIisn9uYaeFFwA5OTlITU2tuUyfPv2ANklKwv/4xz/w6aef4qGHHsKcOXNMpuvz+dBaWBYmIiJ7tONeWtoGgNzcXKSkpLS4n/Siiy6quT18+HCMGDECAwYMMNnsSSedhNbAzJWIiDpk5pqSkhJ0sTUIqX///ujWrRvWrVuH1sLgSkREB5UtW7aYPtesrKxWew2WhYmIKKJHCxcVFQVloRs2bMDSpUuRnp5uLvfffz/OO+88M1p4/fr1uOOOOzBw4EBMmjQJrYXBlYiIIvrH0r/++mtMnDix5v6tt95qrqdNm4annnoK3377LV588UXk5+ebE02ceuqpePDBB1t1rmuzgqvz4m1w4mPQacRGo7NRCZ3o8xHxrTvRuz2o0x9EZ6PfvxudTWf8nDqrCRMmQDdyysQPP/wQbY2ZKxER2cPfczUYXImIKKLLwh0RRwsTERFZxsyViIgssnASCWkjwjG4EhGRPSwLGywLExERWcbMlYiI7GHmajC4EhFRRJ+hqSNicCUiInuYuRrscyUiIrKMmSsREdnDzNVgcCUiInvY52qwLExERGQZM1ciIrJHfp1Gt7Cs29LndwAMrkREZA/7XA2WhYmIiCxj5kpERPYwczUYXImIyB75RRynhaN9W/yrOu2PZWEiIiLLmLkSEZE9LAsbDK5ERGSPVHSdlgZXRDz7wXXoCVCHjAXSs4HcldCf/K1mkRpzBtBnBJCWCaycA73gP023N/JUqCHjgbgkoCQfevY/gF0bgYy+UGOmAN16S8vA7k1ue/k7rO8SBo+HGnAUkJYNbFsFPfs59/G4JKgjpgLdBwLRcUDRbuhlHwBbVoRux+OFOuk6IK0H4IkGSgugV80G1s6vXT7+cqBrb6ikdDjyOrnfoVUMOAbocwSQkgnkrQbm/6N22djLgK59gagYoKIE2LgIWP1Zw23FpQBjzge69QcqioFVn7rP8et+CHDY6UBSN6A0H/j2HSDve7v702cs0Gs0kNQD2LUWWPJK7bLRFwFdegPeaKCyFMhdDKyf03Bb3QYAg08FEruazwirPgB2r6tdnpgBDD8bSM0CygqBVR8CO1fb3Z/OyubnJOsOOQ1IygB8FcDWpcCaT6TDLrzl1DqYubZScJUAuHQmVPYQIDEtaJEu3AUsegtqyLFhNaWOOAvIHAj9wROAPDcpHfBVuQtjE6C/XwB89jxQVQE1ejLUaTdAv36P/QnIJYXQ330MlTkoeJ+iYqH3bgWWvGPWQa+hUMddAf3+o0BBXv12tAP91X/cZdJhn9oD6pQboeX+zh/cVXZuAFbPBSTItqbSQmD1p+6BQXxq8LJVnwBFuwDHB8SnAeOvBor3AbnfhG7rqEuA4j3Au/e7wXr8NeZAA7t/ABLTgXFXAAtfAXasBjKHAEdfAXzyKFC8197+lO0H1s0BuvYH4ursz7pZQPFud39k2ZFXuEF+27L67cR3AQ6/GFj6BrBzrXtgIPc//wtQug9QHuCIS4Ft3wKLZrgHFKN+BHzxV6DE4v50VrY+JzmgHnMpsOFLYP7f3b/hsT8GSuTv9OswlhNF2oCmjcuATd8C5UX1l61dCGxZCVSUNd1ObAJw2InQc19yA6so2usGBSHt/LAYqCg1/xj1t5+YbM8EYNtyv3UzyPLi4MeL9gArZwElBe7RsGSsBTuBbn1DtyNBP3977Ug4cwyggeRu7n35Ulk9xw20rX2Gkm3LgW0rgPKS+ssKd7jb4t9I2RbJOkOR4Cn7u/wDwFcJ7Mt1g3DfI9zlPQYD+7YCO1a5bcn13lyg9xi7+5O3EshbBVSG2J/9ecH7IxfJSkPJOAQo3A7slMxau9cFW4Beo9zl6X2BmARg3WzAqXKX790I9KxeTm3zOUXHQcnnsEUO+LQbhHevB5J7hLecWo12tJVLpOu4fa4Z/dwvrwFHuJmu/KP7YTH04ncD/gEGyBoILYGiaB/ajZSuU3sA+7Y1upqaeC2QNQjKG+1mvptbqfTbEqOmmrKxioqBlgxzUwNH+1IalQOewIOp/G3AgHHubaXcSyC5n5qJNjXsDFOOVN4YaMletiwJvV7dbXUfBJKrt1e+nPfvDJ4qIAcj/NJu28+pshRaysY5Y4D1c93MVMr5K94Jbzm1Hp7+sIMH17gEqJh4IDUD+l8PmExWTboOqCwHls4MXjexC9SxF0Mv/G/7zY+S/tLjrgA2LXUzs0boWX83X+I6oz/QY6DbH9TRLH0LWPo2dFpPIHuo2wcWSlQsUFmnEiHryuMiby0wfAqQPQzYvgrIOhTo2sctGbelFe8CK96DloOB7kPqb7OfZDdDJgE9DgV2rgG6D3b77iQ7FbJfVY3sL7XN5yS2LweGTwUGToDyeKE3LnD7ccNdTnRQznOVICqBaPF7QFW56fPTy2dB9R4evF5CGtTpN0GvnAt8Xz0wqD0C6/FXAVWV0AteD//IbOd6qPgkYNiJ6JikjL3Fff+HnxF6FVkmg7kCRce7jwvpu134MnDoycAZ9wB9jwS2LAtdjm51GijY5m7boZNCryJ9ft+8AQycCJx0J9DrcPdLWgZ2CXlu3UAq++/fX2qbzymxGzDmEnew2YcPQH/6kDtwSQaihbOcWn9Ak9PCS4TruJnrnq1NryOBdcrPodd9BSz7EO0XWK8013r2s6FL1k09PzmjY49flEE8DfW5FmwH4lOA2MTaPum0LKAgYNT29pXuxW/ijcCmxWg3Hi+Q0EBfnpCRv4Gjf4/5CbBlaW2/4MAJ7nvir5KkyP423hVAlj8nKcPLSO0d1SPzpVtC+lcHHAes+ajp5dR6OFq4lTJX+dLxRrnX0n8lt+UfSb1lAbdDKdoDvXWVGQVshuYnpEINmwC9+Vt3udyf8nN3UNM371vfjXr75JH9kG1V1be95nETWKVfUqbNNBVYu/Q0fa1mf6TNnkOBfmOgtwV8kUu70n7g64bsB7S0T+b999+WL7M0IPswwBvj7mt6H2DgeCBvTeh2pD9290ZgWPXn1CUHyBkNbPyqdp20Xu7rSMY35GR3QFBDfbg29kfeL//+yKjTzKG1+5OWA/Q9OnhqTV2p2dV/nzFuII1OALZWj5SW8rCUgQee4H5WMgBKBjnJNA9qu8+pYCsQl+yW72V9+ZuSQWUyGC2c5UStTGnddM9xYWEhUlNTse/P1yAlPqbxBg8/HerwKUGP6e3fQ7/3ONTxl0MNOjp42fcLoOf+033ueb+GXvohsP6r2nmk4y8Bsge7fS/rFkF//Y6bMYw+HZ4xU6Cry8c17c18EshbH97ex0aHtZoacRrUyNOCX2fHOjOn1TPpZ9BVFUEd8Hr5x8BymU8HqDPvhJbbGxYD6TlQYy8AUrq7pa+ivdDffwmsnVf7Wufc4456DuB8+Qrww6LwtjWh8c+nxqGnQA09JXifdq0Hvn4dOPJid0qNfPnJ0f+mJcCaWbXzA0+5FVg9q3ZqTtA81xJ3Kk/gPFeZmpMu85G12wcr81xl/mg44sPsyzxkItQhweV1vWcDsOw/wKjzqwccKaB8P7B1mTvIxb8/x/3MnU8p02vEUdOA1F61fbCr3nffBz8pLx4m81yzq+e5zmzWPFd1+oPobPT7d7f95yR9sodMBBLS3cGP8lmtfL92JHJTyw+yz8n/PV5QUICUlJRWa3/fI1c1GSeabKu0Al1+8UKrbWtEBteIEmZwjSRhB9dIEW5wjSCd7Uu7WcE1gnS2z6nNgutDV9oJrnfOiOjg2nEHNBEREUWojjugiYiIIo4UQ3ULBySFUVDt8BhciYjIHo4WNhhciYjIHgZXg32uREQU0ebOnYszzzwT2dnZUErhrbfeqldmvueee5CVlYX4+HicfPLJWLu2dc/WxeBKREQRfYam4uJijBw5Ek8++WTI5Q8//DD+/Oc/4+mnn8bChQuRmJiISZMmoawsjB+ROUAsCxMRUUSfuH/y5MnmEropjcceewy//vWvcfbZZ5vH/vGPf6BHjx4mw73ooovQGpi5EhFRh1RYWBh0KS9v/jm8N2zYgB07dphSsJ/Mxx07dizmz2+989EzuBIRkTVyAj1t4SJycnJMIPRfpk+f3uztkcAqJFMNJPf9y1oDy8JERNQhRwvn5uYGnaEpNjZyztjGzJWIiDqklJSUoMuBBNfMzExznZeXF/S43Pcvaw0MrkRE1Gl/z7Vfv34miH766ac1j0n/rYwaHjduHFoLy8JERGRNYJ/pgWru84uKirBu3bqgQUxLly5Feno6evfujZtvvhm/+c1vcMghh5hge/fdd5s5sVOnTkVrYXAlIqKI9vXXX2PixIk192+99VZzPW3aNMyYMQN33HGHmQv7k5/8BPn5+Rg/fjxmzpyJuLi4VtsmBlciIrJHWyjrNnOe64QJExo92b+ctemBBx4wl7bC4EpERPZISdex0EaEY3AlIiJr5OfmdEt/co4n7iciIqK6mLkSEZE9LAsbDK5ERGSPVHS1hTYiHMvCRERE7Zm5eqb9EZ6A8zxGOufZG9DZqMsfQ2ei5/wOnY1e9Cg6ncT49t4C6iA4oMnFsjAREdnDPleDZWEiIiLLmLkSEVFEn1u4I2JwJSIie1gWNlgWJiIisoyZKxERWcOysIvBlYiI7JFZNI6FNiIcgysREVkjv/ym2/YX5zok9rkSERFZxsyViIisYZ+ri8GViIjs4VQcg2VhIiIiy5i5EhGRNSwLuxhciYjIGo4WdrEsTEREZBkzVyIissdR7qWlbUQ4BlciIrKGfa4uloWJiIgsY+ZKRETWaK3MpaVtRDoGVyIisoZlYReDKxER2Z2K47S8jUjH4BqOoSdAHTIWSM8GcldCf/K3mkVqzBlAnxFAWiawcg70gv803d7IU6GGjAfikoCSfOjZ/wB2bXSXpfWAOu5SoGsOULwPeuGbwObvWnHnOonsMUDmCCAxA9i7HlgR4nOITgSO/AlQXggsfi50O6k5wPALgx/zRANbvwbWf1z7WM44IHs0EJ0AlO8HVr8D7N9md5+6jwIyhgLx3YCCjcDa/9UuG3IBkJQV/C327QtAZXHD7WUcBmQeAcQkA1UlwKbZQP762vem36lAci+gqhTYthDY9V1kf04xScCg04G03kBlKbDpS2DHUvv7RBQCg2s4JAAunQmVPQRITAtapAt3AYveghpybFhNqSPOAjIHQn/wBCDPTUoHfFXVCz1Qp1wHrP8K+v0/A9lDoE78MfSb0911qWEVRe6XZ5e+QGxy6HUOORUoygOi4xtupyAX+OIPwV/0R98I7FpZ+1i/E4DU3sCyV4GyfUBsCqB9sK6yCNi6EEjt4waKunI/B/K+Ca+tjOFA5uHA+veAkl1AVALgja5dPmAKUJ4PfPOUG8wHn+vu2/4tiNjP6dCpQOk+YN7jbjAfcRFQuhco2Gxxh6gu9rm6OFo4HBuXAZu+BcqL6i9buxDYshKoKGu6ndgE4LAToee+VBssi/YCpYXu7axDgLhE6G8+cANu7nJg+1qogUdZ3qFOaPcaYM/3boYSStdDgKh4IK+Z2VjmcPcLunCrez8qDug1Fljznht8hGRYFY1kjAdq3zo3s5RMskUU0PMYN1OVwCokcy0vcG/HpgLJUpX5AnCqgOIdwJ7VQMYwROznFJcGpPYCNswGnEq3qpC3ws2aqXU5CrqFF85zpebJ6Od+eQ04ws10HR/ww2Loxe+6t6XsvG97cKlv7xb3cTpw3lhgwMnAd68BKb2a99zMkcD2gOwwpaf7GXYfCmSNdjPWnauAjXPafhRG9tFAz3FucN+xGNizKvR6cV2gYhKhE7sD/U42FRLkbwQ2zwGcCiAhwy0nS8D1kyDcfSQi9nOSfZUsObBMLtlw9uH2tpeoEQyubSkuASomHkjNgP7XAyaTVZOuAyrLgaUzgahYoKIkuDxSXgoVHddum9wp9D8R2PGtm9k050tb+vUkA8pbXvtYVBxUVBx0fDqw6Gm3dHnYjwBfBbD5S7SZLV8ApXvcQJ+SAww4w83QJNutS7JtkdIbWPFybRm4zwRgw0duX2VVefBzqsoAbwwi9nOSbQ+1T1FtvE8HIZ5b2MWycFuSICp/OIvfc//hy4Cl5bOgeg93l8tjEnwDqJg4oDKMkjM1/MUr5cHc+c1/rmRDe9YClQEHPL5K93rjXDeYSda49Su3nNmWira7AV2y5YJNwK5vgfRBodeV7RTbF7kBRi5yO61/7XLJGgPJgZ60H7GfU0Xofapqw306yPtcdQsvkY6Za1vaU90f1JC924BRk92ynb/E2LUXsDu3TTavU0rr62Y1425y7yuvO5DnmJuBr//ecF+pZD4ZQ4AV/w1+vDgPHVJjh/ql+6Alw22IlIBjEt2+Tn//bkJ3oHQ3Ivdz2gnEJrmjuf1BN6kHUMyBgdQ2mLmGQ4KdN8q9Vsq97fGGWBZwO5SiPdBbV0GNnux+cSSkQg2bAL35W3f59rVAeQnU6NMATxTQaxiQNQh63aK229eIpdwvZPPeB9zestAt3379nHuRjLNkj3u7Tgk+SPdh7qCbfT8EP15WAL1vA9BnvPsZySjenke4g3Raa5/kOnCfJCNL7ee+vjwuZeHuI4C9a0M3o6uA3auArCPd58pFbu+rnoYjA5tkwE+v6n1KzAS6DgF2LY/gzykfKNgC9Jvg7lNylrvujmWtsE8UqKWDmbR/UFMz3HfffVBKBV2GDBmC9sTMNQwS7NThU2rvX/U49Pbvod973MxJVYOOrl0mwfL7BdBz/+neP+/X0Es/NNNrhJ41A2r8JVCX/t4t90rgXFY9L0870B8/DXXcJVAjTgGK8836nIYThj7jofoeV3v/+Duh8zcBy14OLm9KSVSqAhX7ax874lpg8zxg54rgUqP0/4Wy6m1g0GRg3M/dtqWvL3eB/X3qeTSUDFjyO/Ln0IW5wLp3zTLEn+4+LqVpGZy0LyC4DjoH2L/VLf+KzbOAPicBI692B2FJMNo8u3b99e8D/U4BRl/nvkcyzcf2NJw2/5zeAgZPcbNf+bf2wyxOw+nEfa7Dhg3DJ598UnM/Kqp9w5vSuundKCwsRGpqKgoKCpCSkoLOwnn2BnQ2nmueRGei5/wOnU58JxygVtr5xgWoE36FzqS1v8f97a896zIkR7ds4Nj+ygoc8r+Xwt5WyVzfeustLF3acU4SwrIwERF1yAFNhYWFQZfy8jojwAOsXbsW2dnZ6N+/Py699FJs3ty+VQoGVyIissZxlJWLyMnJMdmw/zJ9+vSQrzl27FjMmDEDM2fOxFNPPYUNGzbguOOOw/79Ad0KbYx9rkRE1CH7XHNzc4PKwrGxdaZXVZs8eXLN7REjRphg26dPH7zxxhu4+uqr0R4YXImIqENKSUk5oP7htLQ0DBo0COvWhTipShthWZiIiDrVSSSKioqwfv16ZGVlob0wuBIRUUQH19tvvx1z5szBxo0bMW/ePJxzzjnwer24+OKL0V5YFiYiooi2ZcsWE0j37NmDjIwMjB8/HgsWLDC32wuDKxERWeNoZS4tbaM5XnvtNXQ0DK5ERGTNgZy+sK6WPr8jYJ8rERGRZcxciYjIGv6eq4vBlYiIrHFgoc/V/BJUZGNZmIiIyDJmrkREZI2Nk0DoFj6/I2BwJSIiayQwOgyuDK5ERGQPM1cX+1yJiIgsY+ZKRETWONWXlmjp8zsCBlciIrKGZWEXy8JERESWHdSZqy6tbO9NoKYkxqOz0YkJ6Gw8R93a3ptAHYSjm3/i/VBtRLqDOrgSEZFdLAu7WBYmIiKyjJkrERFZLgujxW1EOgZXIiKyhmVhF8vCREREljFzJSIiuz85B/7kHIMrERFZwx9LdzG4EhGRNTLH1WnxPNfIz1zZ50pERGQZM1ciIrJGW+hz1exzJSIiqsU+VxfLwkRERJYxcyUiIms4oMnF4EpERNZIf6lmnyvLwkRERLYxcyUiImt44n4XgysREVnDPlcXy8JERESWMXMlIiJrOKDJxeBKRETWsM/VxeBKRETWMHN1sc+ViIjIMmauYVAjJkINGQd06wlsWgHnvb/WLoyOg5p4KVS/EUBVJfS3s6C/eq/hxjJ6w3P8RW5bpUXQi96BXr3AXZbWHZ5jzgMy+wNR0cCebXDm/QfYvr71dzLSZYwEug0D4rsCBRuB9e/ULht8PpCYBWin9rHlM4DK4obb63YYkDkGiE4GqkqA3NlA/g+A8gKDzgHiugIeL1BRDOQtAXZ/Z3+fugyDSh0MxKYDxZuht3wUvDxtCFT6SCA6Eagqg877EijaFLIpNeASICre5ASGdqC/n1G7QmJPqO5jgehUoKoYOm8+UJxrf5+o02NZ2MXgGgZdnA/99ftQOYdCJXUJWqZOuAgqLhHOC3cBCcnwTL0V2L+nNmAGiomH56yboBf+D/q/nwPd+8Jz9s3QBbuB7euA2AToTcuhP/snUF4MNfRYeM68Cc4//g8oK2q7HY5EEii3LwRSegPRSfWXb/kC2PlNeG11Gw70GA2sfx8o3QVEJQCeaHeZBOjNs4DSvW6gikt3g3fZXqBoq919qiqB3r0EKrGnG0ADpR0KlT4ceusnQPkewBsPeBr/56y3fgoUbay/IDoZqtckt62izUBSb6hep0D/8C+gcr/dfaJOr72m4jz55JN45JFHsGPHDowcORJPPPEEjjrqKLQXloXDsf4b4IelJtMMEhUDNehIOAveBipKgfyd0Ms+gxo6PnQ7WQMAXxX08rnuzz7kbYBevwRqWPX6eRuhV3zuBlKtoVd84X6ZS5ZLjctfB+SvB6pKW9iQAnqOczNVCaxCMteKgurlGijdU5sB+sWmwrr9G9xg6Curt40q4wjovHluYBW+0gMPhEk5QNluN7AKuS7dCZU6qGXbT9RGXn/9ddx666249957sWTJEhNcJ02ahJ07d6K9MHNtibQeUN5oYFdt+UzvzoU6YnLo9VWIozGloLr2qvtV7eoqGUscsHe7vW0+WGWNBbKPBioK3TLunlWh14vrAhWdCJ3QA+hzMqA8bpk5dy7gVNSuN/BskyUrTxR0yS43sLeVmFSoqATouG5Qmce7f1dFudA75wNOZYNPU1nHATjevAd69+KAsm+oLEEBsV1bbReo85LvMm2hDVFYWBj0eGxsrLnU9eijj+Laa6/FVVddZe4//fTTeO+99/D888/jrrvuQntg5toSMbHQFWXBfXnlJUBMXOj1d/wARMeaPlzTX5c1AGrA6NDrSwl50rXQi98HSoL/wKiZtnwJLH8eWPaMWx7OmQikDQi9blT1ZyHl5VWvACtfcrPSnBOC11v3NrDkL9Cr3wD2rQWcKrQZr7uNUi7WG/8LveE/QEwyVI9jGnyK3vYZ9LpXode9BL1vOVSvU4G4DHdh8Rb3dlJfN6jKdUImIAeORAfyY+m6ZRf/aOGcnBykpqbWXKZPn17v9SoqKrB48WKcfPLJNY95PB5zf/78+WgvzFxboqIciI5xsxt/gI1NACTghlJWDOfdv8Bz7PlQR50J7NsOvXIelAxgqhtYpS92+zrohQEDc+jAFAdk/oWb3MFH6YNDZ5u+6sxv+yIzSKjmdv/TgXpjhbTbz5o+yB38JOu1hersVO9eWlMyltuq50kNP6d0R+3twnVAcl+o5H7QZbtMyVv6W6XUjOwTgJI8oHC9+3dN1I5yc3ORkpJScz9U1rp79274fD706NEj6HG5v3r1arQXBteWyM8DHB/QrRewy+2vUnJ7TyMDW7avh/Pvh2ruqtOuhd76fZ3A+nPovdugZ73Uqpt/0JL+7oaU7YVubhYqI4hjgwe6taqK/OZvY1PvQdEm6ICRxqrvVOiCgL9LojBJmuFYaENIYA0MrpGEh6bhkCN4b5TUGtyymbntBaoqoNd+Dc/RZ5ugiNTuUCNOhF75RcNtdctxR3V6o81AJtVzMPSyT9xl0XHuaOL8POhP/9Fmu9c5KDfIyZ+0qr5tPrdYILVv9UhaBSTnABnD3VJuKNrn9sdmHuk+Vy5y25/lxme4JWPzWgpI7QekDwEKN7bePpkMMmD/ZBsL1kJ1HQl4YszF3N7fwDZEJQHxWdXteIDk/iZz1YHrx3VzX0NGRXc73C095zO4UvNpKevqll/C1a1bN3i9XuTl5QU9LvczMzPRXpi5hkEdOQWesWfW3Pde/1foLWvgvPlH6NmvAideBs9VD7nBVua5BkzDMcFy21rorz9w2xp5otvPKl90O34wbaDYHYkqjysZUdytF1T/0TVtSAarv2+jkmOkyh4LlT2u9v6Ym6D35wLr3wOyjgb6p7uPlxe6g5MCg+shU4H9W4EdX7n3ZaRw7xOB4T92A5nMb82d4y6Tz63nsWbgkxl1IQOkZNneNdZ3SXU73C3V+u8PuQa6eBv05nfMSGGVOR5q4CXuNu7f5A5o8q/b/wLo3d+4JWBPFFTmMWYglOm+qC4Do6x2JKXKGAvEd3dL3cVboTe9A+g27EcmOkAxMTEYM2YMPv30U0ydOtU85jiOuX/jjTeivSitG6uR1Y7Yks7kgoKCiE3RQ/E98RN0Nt6f/Q2dif76T+hsdGICOhvPoT9t702gdv4e97f/yuG3I0EqPi1Q4ivHJUv+EPa2ylScadOm4ZlnnjFzWx977DG88cYbps+1bl9sW2HmSkREEX2GpgsvvBC7du3CPffcY04iMWrUKMycObPdAqtgcCUioog/cf+NN97YrmXgujigiYiIyDJmrkREZA1P3O9icCUiImv4e64uloWJiIgsY+ZKRETWsCzsYnAlIiJrGFxdLAsTERFZxsyViIis4YAmF4MrERFZIyfUdVpY1m36pLwdH8vCREREljFzJSKiDvl7rpGMwZWIiKxp7u+xhtLS53cEDK5ERGQNM1cX+1yJiIgsY+ZKRETW8CQSLgZXIiKyRuKittBGpGNZmIiIyDJmrkREZLksrFrcRqQ7qIOrLvOhs6l88Ep0JtF3z0Bnoze/hM7GV/A/dDY6KQWdSZWvuE1eh2VhF8vCRERElh3UmSsREdnF0cIuBlciIrKGJ5FwsSxMRERkGTNXIiKyRn4uTvMn5xhciYjIHvmhc4c/ls7gSkRE9jBzdbHPlYiIyDJmrkREZA1HC7sYXImIyBrOc3WxLExERGQZM1ciIrKG5xZ2MbgSEZE1LAu7WBYmIiKyjJkrERFZw3muLgZXIiKyhlNxXCwLExHRQaNv375QSgVdfv/731t/HWauRER0UA1oeuCBB3DttdfW3E9OTrb+GgyuRETUIafiFBYWBj0eGxtrLi0lwTQzMxOtiWVhIiKynrk6LbyInJwcpKam1lymT59uZRulDNy1a1eMHj0ajzzyCKqqqmAbM1ciIuqQcnNzkZKSUnPfRtZ600034fDDD0d6ejrmzZuHX/7yl9i+fTseffRR2MTgGgY1+kR4DjsW6NYTesNyOG/9pXZhTBw8p1wONWAkUFUB55vPoOe/G7qhhGR4Jl4ElTMIiIkH8nfB+fIt6PXLgl9v7OnwjDwBiE8GivbB9/6zwPYNrbuTXTLgPe1yqJ4DgMoKOIs+gjP/gwZXV6OOh/eYyUByOlCyH74PX4b+/hvAGwXvJbdDZWQDUdHA/nz4FsyE/mZO625/Z5Q0CCpxABCTBpRug95d/R56E6CyzgxeV3mr15ndcHuJA6FShprnwymD3vc1ULoleJ3oVKjM04NfzyIV0xcqujfgTQaqdsIp+ap2oScJnvjhgDcV0A501Q7o0hUAfKEba3L9KKj4EVDRPdzlFRugy7+3vUfwqEOgVBd58wBUwNGbofWO6uVeeNQgKNXVjIF19FZovamR9ppav7nttT35LVZt6fdcJbAGBteG3HXXXXjooYcaXWfVqlUYMmQIbr311prHRowYgZiYGPz0pz81WbGN4O3H4BqOonw489+F6nOoG0wCeE66BIhPhO+ZXwAJKfD+6DY4hXugV8yv3050LPTOzXDm/Nu0qQaMgOeMn8L30oPAnu1ue8edC9VrEHxv/BHI3wmkdAV89ksWQZRC1IU3w1mzBL7XHzeBNurSX0Dv3we9fEH91UefAO/Rk1D136eAHZuBxBSzb4bjg+/Dl4Bd28wXGrplI+ryO+HbvR061/YXWyfnK4Uu/A4qLssNiDWPl0BveT1gRQ9Uz3OhSzY2EVgPhd79OVC5D/DEAar+P3+VfjRQvgutRTvlJsCpqAwo2YYAnoQx0FV7oYsXACoanoSxQOwg6PJVIdtqan0VPxxKxcAp/BjwxMKTOA5wSqAr6xxQtIgEgQr4HDlALpNwAK9nOBxdDo19JvDKtvkc+XcUDa9nJByUQeu80PvUxPrNba89aAsDknQz17/ttttw5ZVXNrpO//79Qz4+duxYUxbeuHEjBg8eDFsYXMOg1y4x16p7TnBwjYqBGnIUfK9MB8pLzcVZ8ik8w4+DL1RwLdgN/dWHte1KxrpvB1TWAGgJrnGJUEecCt+Me93AKgr3tP4Ods0CumbCmfOWCY7YswPO0rnwjJ4AX93gqhS8E86F7+2/uYFVFBcGz/7euaX+P5P07gCDa/OU5rrXMenBwbWuhF7ul3xJ9edRj4JKGwm9Z54bWIUjgaCO5CFAZYEJ3oiWTKwVVLkHkSbbRHBwhScBuvJb929GV5hMVHm7NPxF2+j6XqjobDjFX8iLAk4VdPkGqJjeloOrZI+BBzWF0MiHUqnQugBKdYfP+cbdBlSZTNOjsuALGQw9Tazf1PKDV0ZGhrkciKVLl8Lj8aB79+5Wt4nBtSXSM6Gk9Lmz+ktQyO2jp4T3/IRkID0Lepf7fJXVH/BVQh061i0L+6qgV38F54s33aDXWpQKvq6+rXrIl3YdXbOgkqR02BfeKVcBHg/0uu/g+/hVoKL2C9t74c1Q/YeZ90fnbYZe7R6gkH0qcSBQsqHhqfdRKVDeeOiYdKj0sW4gLtsGvW8JoCvddbyJUMlDoHe8b67bgy5fDxWdA+0rMNmZisqCrtx0YOt7EqGkVO6rPfDTTiGU95BW3gsPFFLgaDk4ToBSMma0KGCjiwDVu4HnNrV+c9trHx15Ks78+fOxcOFCTJw40YwYlvu33HILLrvsMnTpYveAksG1JaTMKwFFyp/VdHmJ6YdtksdrSsJ6zVdAXvUXQnwiVGwCVJfu8D37K5PJes+7CaqyrOF+XBv27ADyd8Mz4Rw4s980WaZn5HFAbHy9VVV8onvdbyiqnr3P3Paeex28p14C37vP16zne/0xN0DnDILqM9j0R1Mr8CYCcZnQOxo5ePHGmCtl1nP70VW38VBdxkDvdSsTEnR1/jLAab/PSVfthCd+FFTK6SaI6Mrt0BWbD2x9FQWtJbsL+JY2BxKt+5XnUYOhUQINKa1L9uoL2gZtMs6GtsHbxPpNLe8YOvKv4sTGxuK1117Dfffdh/LycvTr188E18B+WFs61qcSaSrLgegYQI4mqwOskoAUkME1GFjPvt4dAPXhi7WPV5SbK+fLt922K8vhLP7EZLG+1gyujg9VbzwO7ymXIOrmPwGF++As+wKewyfUW1XXbOO7QGlRzW0JsPVX1tCb10ANPQqecZPhfPFO6+3DQUolDQAq9gGV+Q2v5Lh99rpgBeCU19yWAGsk9HOrFib7bS/Rpk9Ul62GrthYPRhpOFT84dCli5u/vgms3uo+UR3wddd64xfcgU3x1f2vQgKhJ2gbVKPb0NT6zW2P6pJRwgsW1B9H0hoYXFti7w7A5wOkL9affXbvDeza2nhgPes6wOuF8+Zfgsq9/vJwu9i1Db5X/lBz13PSBSYw1rNnO3Rl87Ib5fUC6T1sbCXVldgfulBGyDaiqhC6OsCGIhktYrpB9Ty/+oEot+rQ8zzorf9BmzB9yl4zotdVaYKmJ/Fo6NIDWN8pdsvknhTAKXB3S/p5fftbMbCmVAdW/7/pkuogmFhbylVJMkihgVaaWr+57bWPjlwWbks8iUQ4JDP1RpnAaI7w/berKkxZ1zN+qju1Jq07PKNPgvPd3MYDa0ysG1jrjgIu2A1n4wp4xp1pBkshMQ2ew0+Cs25p6+9j915uFu7xQg0ZY8rCvs//V3+9qkro7+bDc8wUIC4BiE0wt/UaGWQBoEdvqH7D3Gk4ygM1cCTUYePg/LC89feh01EBmYr/dsA/WRlFLCNuixsZJSyklFiywZ2Go6TSEu3erh4wpfctht7+julvlQuK1gJlee7tNtknBfiKTLYpU3Xcx71QMX0A6U8Npcn1fdCV2+CJk/5j+fcqXS79oCs2tVJgTa0OrIH/ph1ovRMeT7/qLDoeHtUTjq4e1FVPU+s3t732oS39F+mYuYbBM+4MeI49u/b+rc9Ab14N3+uPwPnkZXhOvQLe6/7gzg+Vea4BI4U9590MveV76IXvA9kD4DlktMn8vDc+XrOOs+A9d7ncfu/v8Jw6Dd4b/mRGH+uVC6AXzWz9fZTS7ZgTTVCUAUi+N/5cM+rXe/Gt0Ju/d0vB8rX10cvwTr4CUT/7gxtsv1/qDmgyxyEeeE48D0pGIMvIYTlg+PjVkFN6qHEqdThU6oja+70vgZagt/PjgIFMm2oHJQU+N2MidPlOoDqrlTmtqstRUD2nusG2dIsJqu7CCsAXUI2Q9mQdX6n9fYodBE9c7XQHb+oZ0FW74RTPg1OyCJ64Q6HiDnX/dnx74ZR8EzA4eCy0by90+VoTPJtaX5d+B8SPgCflVLM/Zp6r1ZHCIhYeT09o7cDrGVf72joPjv4ejl4LDwZVL/PPS60d2evxDDejirV2+4qbWr+p5dRxKK2b/uU8Ob+jnHqqoKAgrAm9kaLqkavR2eiyVhxV3A6i756BzsbZ/BI6G53aeb4X/HRS59qnwsJidE0/o9W+x/1x4ud9fonYOnOYm6vcKcPjm6ZHdMxh5kpERAfFaOG2xOBKRETWcECTiwOaiIiILGPmSkRE1sgoHt3Scwt3gsyVwZWIiKxxGj4RZ9ha+vyOgGVhIiIiy5i5EhGRNRzQ5GJwJSIieyz0uaITBFeWhYmIiCxj5kpERNZwQJOLwZWIiKzhVBwXy8JERESWMXMlIiJrWBZ2MbgSEZE18kNruoV13ZY+vyNgcCUiIms4z9XFPlciIiLLmLkSEZE1/D1XF4MrERFZw7Kwi2VhIiIiy5i5EhGRNcxcXQyuRERkuc9Vt7iNSHdQB1dd5kNn49tXhc4kRnW+P1Fn4cPobLSn8/UwRaWehc4kylvY3ptwUOl831xERNRuWBZ2MbgSEZE1PHG/i8GViIiskf5Wp8V9rpEfXTtfRwkREVE7Y+ZKRETWsCzsYnAlIiJr+JNzLpaFiYiILGPmSkRE1vD3XF0MrkREZA3nubpYFiYiooPGb3/7WxxzzDFISEhAWlpayHU2b96MKVOmmHW6d++OX/ziF6iqat7Z75i5EhGRNY6Fea5OK85zraiowAUXXIBx48bhueeeq7fc5/OZwJqZmYl58+Zh+/btuOKKKxAdHY3f/e53Yb8OgysREdk9cb9ueRut5f777zfXM2bMCLn8o48+wsqVK/HJJ5+gR48eGDVqFB588EHceeeduO+++xATExPW67AsTEREHVJhYWHQpby8vNVfc/78+Rg+fLgJrH6TJk0yr79ixYqw22FwJSIi62Vhp4UXkZOTg9TU1JrL9OnTW337d+zYERRYhf++LAsXy8JERGT3DE1oeRsiNzcXKSkpNY/HxsaGXP+uu+7CQw891Gibq1atwpAhQ9BWGFyJiKhDDmhKSUkJCq4Nue2223DllVc2uk7//v3Dem0ZyLRo0aKgx/Ly8mqWhYvBlYiIIlpGRoa52CCjiGW6zs6dO800HPHxxx+bID906NCw22FwJSIiaxxtIXNtxTM0yRzWvXv3mmuZdrN06VLz+MCBA5GUlIRTTz3VBNHLL78cDz/8sOln/fWvf40bbrihwbJ0KAyuRERkjfwWq+7Av+d6zz334MUXX6y5P3r0aHM9a9YsTJgwAV6vF++++y6uu+46k8UmJiZi2rRpeOCBB5r1OgyuRER00JgxY0aDc1z9+vTpg/fff79Fr8PgSkRE1mgLPxmnEfkYXA9Elwx4T7scqucAoLICzqKP4Mz/oMHV1ajj4T1mMpCcDpTsh+/Dl6G//wbwRsF7ye1QGdlAVDSwPx++BTOhv5mDNpfaBdHnToOn/2Dzl+2sW4nK/8wAivfXW9U7/hR4jzwOKisHzqplqHzhsfrrjJ0A78QpUKldTBuVb/4TzoolrboLp59+Ou688xcYPvwwVFZWYu7cz3Hzzbdi69atNeucffZZeOSRh9CzZ08sWfINrrnmJ1izZk2DbTa1fnPba5buo4CMoUB8N6BgI7D2f7XLhlwAJGUBOuBr7NsXgMri0G01tn5UPNB7ApDSC/DGAGUFwNZ5QP4PsE0lDoJK7AdEpwFl2+Ds+dxd4E2Ap8eUOit7q9eZ23B7CQOgkg81z4dTBid/MVBW/XnHZMCTOhqITgF0FXTxBujCZdb3iSLr9IdthcG1uZRC1IU3w1mzBL7XHzeBNurSX0Dv3we9fEH91UefAO/Rk1D136eAHZuBxBQgurpT3PHB9+FLwK5t7pdet2xEXX4nfLu3Q+d+36a7JYFVlD94s9nH6EuvR/Q5V6DypSfrrasL9qHq47fhGTQMKjW93nLv0RPhPeE0VP7zL9BbNwFJKVAx4Q8EOFCpqSl46KFHMGfOHPOTVU888TjeeOM1HHvscWb5oEGD8PLL/8SFF15iTm32q1/9Em+//V8MGzbCDGyoq6n1m9tes1UWAVsXAql9gJik+stzPwfyvgm/vYbWl4BastNdLq+Z1h8YMAVY8TJQthc2aV8JdOEKqLhMKG987QJfCZxt/wpY0wNP1lTokk0NtqUSB0AlDYGz90ugch/giXMDsrsUnq7HQxetgt61yg3eGScBvmLo4nVW94koFJ6hqbm6ZgFdM+HMecsER+zZAWfpXHhGT6i/rlLwTjjXZKomsIriQiB/l3tbRsTt3BKQTVQfraW7w7/bkuraHb5lC4GKcqC8DL6lC6CyeoVc1/nuazjLFwPFRSEaUog67TxUvflPN7CKokLovdX73IpeffU1009SXFyMkpISPPbYnzF27FFmgIK47LJLMWvWbLz33nvmNGoPPvgbM9T+uOPc4FtXU+s3t71m27cOyF8PVJWiVZUXADsWu4FVSMYqQVUyXdvKtrgXp/HT2Kn4XuZvSZfmNrQGVMoIN1OVwCqcMhM83cXRUN5Yk62af1cSVMt2uBkztcnvueoWXiIdM9fmUir4uvq26hEiEHXNgkpKhcrsC++UqwCPB3rdd/B9/CpQUVazmvfCm6H6D4OKiobO2wy9unXLp6H45nwA78ixcFbKsHQF7+hxcFY2IyuqprpnQaWkQfXqh9gfXQ14vPCtXoaqt18Byls5SNRxwgnHm7Oy+LPIESOGY+nS2rKg/ITUypWrzOOzZ8+u9/ym1m9ue9ZlHw30HAeUF7rBcc8qO+tLmTi+K1C6G+1FslJdsrHh3ruoZJP5qph0qC5HmTxBl22DLlhiSsDQFXCK17vt7F8BeBNNtuzkf9XWu3LQYVnYxeDaXHt2APm74ZlwDpzZb5os0zPyOCA2oMRVTcUnutf9hqLq2fvMbe+518F76iXwvft8zXq+1x9zA3TOIKg+g4GqCrQ1Z8P3ppwb+5tnzH29aR0qP3mn+Q0luOVLKRmX/+luczv68hsRNfVSVL3+LNqK+0sW9+OCCy6qeUzmsOXn5wetJ/eTk5NDttHU+s1tz6otXwClewCnCkjJAQacATiVbrbbkvWVBxg4Bdi7Bih2z0rT5qT/NLYHdEEjB3cet5tBxWbC2fmh+1D6sUDaGOh9C819XbIZni5HQaUcBqU8cIrWAGXb22Yf6KDHsnBzOT5UvfE4VI8+iLr5T4ia+v/gLPsCKKlfItVSYpWnfPkuUFpkLnJbDRpVv10phWxeAySmwjNuMtqUUoj5f3eZAFv+y2vMRW7H/L87m99WuZuR+z59xy0bFxeZ296h7lwymy655GLs359vLsuX12aQhx12GD744F3ceONNpi/Ur6ioyJz8O5Dc37+//qCtcNZvbntWFW0HfBVul0LBJmDXt0D6oJatbwLrmW4A3vAx2otkm6bUWxl84BJEV5orR7JSKTE75ea2iuvpLo9Khqfb8XAKlsDZ+jp82/4LFZUKlRri3x512BP3RzJmrgdi1zb4XvlDzV3PSRe4gbGuPduhK5uXhSrpH0wP/kWGVpeQCJWegarPPzKjn0XVFx8h7sQzgMSk0H2rDdC7mr/PB+qVV141l0ASWD/55EPcddev8PLLrwQt+/bb7zBq1Mia+1FRURg69FB8993ykO03tX5z22tVze2jqru+CaxnuAOC1r4dPKq4jamE/m4ptzGV+6Gl/NsQ6Vv1lQD+PlunDLrkB6jkodAF7hl5qHX4w2NLtPT5HQEz1wPRvRcQHWP6E9WQMaYs7Ps8YJqEX1Ul9Hfz4TlmChAnpa4Ec1uvqS539egN1W+YOw1HeaAGjoQ6bBycH9r4y7m4CM6uHYgaf7K7LVHRiDr2FOh9e0IHVo/HXU+uVfXt6kFDqKyEb/GX8Epgjk8w+y23fa08DUfIKcsksP761/dgxozaM7D4vfTSyzjxxImYPHmy+cHj//u/X2H37t2YOzf0VI+m1m9ue82nqke/qtrb8n57Y4HUfoBHjo2VW+btPgLYuzZ0M02t7w+snujqwGphpHNj+2S+dlTA7YCvodgsU/JtbJSwywddvBGe5KFm8JJc5LYu3eIurtgLeOKBuOqxEJ5YqIR+0BXVg5+o1TBzdSkdxrAs+ZFYKXcVFBSE9QsFkaLywcZ/RaEhngnnwjPmRBNUZACS88nr0FvcvivvxbdCb/7eLQWL6Bh4J18BNfhwN9h+v7RmQJPK6gvP6VdAyQhk+RgKdsP5+jM4Sw58MIxvXyNH841QPbIRdfZl8OT0M1+2ztaNqPrfK2bEb9T5V5l1qv79grmOmnSuuQRy1q1CxV9/696JiXXnzA4fI6N8TGCtevvlmpJxc8T/6bWw133++WcxbdoVZqRwoKFDh5ufrhJTp56Nhx/+PXr16mXmpV599bU181LHjx9vysnJybUjShtbP5zloTgLHw5vh3qOg5IBSAF0YS6w7l1g0FQgvnoalBmgtATYHZDtDToH2L8V2L7IHaDU2PrJvaAO/RG0lIMDM9Zti9znh8HJDu/XQlTKcHhShgfvU3kenF2fusul31T7oPfVn9bm6TYBunwn9P6V1Y15odKOdEcWy3NKt9YOaBJxPd3Xikpyl5ftcJc3MVLZz9vrEnQmrf097m9/XMr1iFItm3pXpcsxv/CvER1zGFw7mQMNrh1Vc4JrpAg7uEaQcINrJGFwPbD2x6ZcZyW4Lix8KqJjDvtciYjIGqf6v5Zo6fM7Ava5EhERWcbMlYiIrNFKQ6uWjhaO/AFNDK5ERGSNtjDaV3eC4MqyMBERkWXMXImIyBoZjKQ4oInBlYiI7OEZmlwsCxMREVnGzJWIiKxxlAPVwtHCLAsTEREFYJ+ri8GViIisYXB1sc+ViIjIMmauRERkDUcLuxhciYjIGgc+KPha3EakY1mYiIjIMmauRERkjZwXWLe4LBz55xZmcCUiIms4z9XFsjAREZFlzFyJiMjygCZPi9uIdAyuRERkUcun4kgbkY5lYSIiIssO6sw1+u4Z6Gyi0bnoR19q702gMHjbewOow3C0lHQ9FtqIbAd1cCUiIrt4hiYXgysREVmj4YNuYeYqbUQ69rkSEdFB47e//S2OOeYYJCQkIC0tLeQ6Sql6l9dee61Zr8PMlYiIrHFPAOFYaKN1VFRU4IILLsC4cePw3HPPNbjeCy+8gNNOO63mfkOBuCEMrkREdNCc/vD+++831zNmND6gVYJpZmbmAb8Oy8JERNQhFRYWBl3Ky8vb7LVvuOEGdOvWDUcddRSef/55aN28gM/MlYiIrNFaBjSpFrchcnJygh6/9957cd9996G1PfDAAzjxxBNNv+xHH32E66+/HkVFRbjpppvCboPBlYiIOmSfa25uLlJSUmoej42NDbn+XXfdhYceeqjRNletWoUhQ4aE9fp33313ze3Ro0ejuLgYjzzyCIMrERFFvpSUlKDg2pDbbrsNV155ZaPr9O/f/4C3Y+zYsXjwwQdNWbqhAF8XgysREVme56pa3EZzZGRkmEtrWbp0Kbp06RJ2YBUMrkREZI3WFs7QpFtvKs7mzZuxd+9ec+3z+UzgFAMHDkRSUhLeeecd5OXl4eijj0ZcXBw+/vhj/O53v8Ptt9/erNdhcCUiooPGPffcgxdffDGoT1XMmjULEyZMQHR0NJ588knccsstZoSwBN1HH30U1157bbNeR+kwxhfLEOjU1FQUFBSEVf8mIqKOpbW/x/3td0sZC49qWd7m6CrsLlwY0TGHmSsREXXIqTiRjMGViIgOmjM0tRWeoYmIiMgyZq5ERGR5tLBqcRuRjsGViIgskj7XlrcR6VgWJiIisoyZKxERWeOWdJWFNiIbgysREVnD4OpiWZiIiMgyZq5ERGSN/FycavGJ+yM/c2VwJSIia1gWdrEsTEREZBkzVyIissbGeYE1zy1MRERU97zAjoU2IhuDKxERWWOjv1Szz5WIiIjqYuZKRETWMHN1MbgSEZE1Nuao6k4wz5VlYSIiIsuYuRIRkTUsC7sYXImIyBoGVxfLwkRERJYxcyUiIotsZJ2Rn7kyuBIRkTUsC7tYFiYiIrKMmSsREVnDea4uBlciIrJGawsn7jdtRDYGVyIiskh+Lk61OHeNdOxzJSIisoyZKxERWeOO9FUtbCPyM1cGVyIisqjlwZVlYSIiIqqHmSsREdljoSwMloWJiIhqaQslXc2yMBEREdXFzJWIiCzigCbB4EpERBZpC4N9WRYmIiKiA8lc/RN6CwsLw1mdiIg6GP/3d+ufoEGGI0V+5tkmwXX//v3mOicnp7W3h4iIWpF8n6emplpvNyYmBpmZmdixY4eV9qQtaTNSKR3GYYzjONi2bRuSk5OhVEvPvEFERG1NvuolsGZnZ8PjaZ0ewbKyMlRUVFhpSwJrXFwcOnVwJSIiovBxQBMREZFlDK5ERESWMbgSERFZxuBKRERkGYMrERGRZQyuREREljG4EhERwa7/D2LZ+Kya8v0xAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAGgCAYAAAC0SSBAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAITBJREFUeJzt3Q10VNW99/FfYl4gkcDlRcFLxFtLoTUWQUqptkC0EYr1JQilhIqutndRKcpLrrUpBZKWUhp6WaULKq14wWub3mrbhxZZRcJrtbZceGgrIRAkj6Kg0hQhIQ4mgcyz9o4TQoKbiYY5c06+n7UOmTk5IfufmTm/OXvvMychHA6HBQDAe0h8r28AAGAQFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERRAB9m+fbsSEhLs14j7779f11xzjaftAj4oggKeW7t2rd3BmuX5559v833zKTOZmZn2+5///Oc9aSPQmREUiBtdunRRSUlJm/U7duzQkSNHlJqaKr957LHHVFFR4XUzgA+EoEDcGD9+vJ5++mmdOXPmvPUmPG688Ub17dtXfpOcnOzLgANaIigQN6ZMmaLjx4+rtLS0eV19fb1+/etfKy8vr013lOn7v+uuu9r8P++88466d++u6dOnO3+f6cqaOXOmfvGLX2jQoEH2iMYE0h//+Mc22/71r3/V5z73OWVkZOjyyy/Xrbfeqr/85S8XrelCYxSNjY1avny5rr/+evs7+/Tpo3Hjxmn37t32+6NHj9aQIUMu+P+Zdo4dO/aivxfoSAQF4obZoX7qU5/SL3/5y+Z1f/jDH1RdXa0vfvGLbXbyX/rSl+z333rrrfO+t379etXU1NjvX4zp1po9e7bd9jvf+Y4NKrPTLisra95m3759+sxnPqO///3v+sY3vqH58+fr5Zdf1pgxY7Rz58521/mVr3zF/k4z7vKDH/xA3/zmN21gRILn3nvv1YsvvnheG4xdu3bp4MGDUdUFdChzPQrAS2vWrDHXRAnv2rUrvGLFinC3bt3CoVDIfm/SpEnh7Oxse3vAgAHh22+/vfnnKioq7M89+uij5/1/d955Z/iaa64JNzY2On+v+Vmz7N69u3nd4cOHw126dAnn5uY2r7v77rvDKSkp4crKyuZ1r7/+um3nqFGjmtdt27bN/n/ma8R9991n2x2xdetWu81DDz3Upj2R9p48edK24ZFHHjnv++Zn0tPTw7W1tc66gI7GEQXiyhe+8AWdPn1azzzzjE6dOmW/tu52ivjIRz6iT37yk7brKMIcXZijjKlTp9qjjosxRzCmuyni6quvtt1Zzz77rM6ePWuXTZs26e6779aHPvSh5u369etn22VmaZmjl2j95je/se1auHBhm+9F2mu6zUwbzJFV5Lpiph2/+tWvbDvS09Oj/n1ARyAoEFdMf/1nP/tZO4D929/+1u4gJ06c+J7bT5s2TX/60590+PBhe98Mhjc0NNjum2gMHDjwggEUCoVUVVVlF3PbjA209tGPftSON7z22mtR11dZWamrrrpKPXv2dG5n6nr11Vf13HPP2fubN2/WsWPHoq4L6EgEBeKOeadujgpWrVplB5B79OjxntuasQszsyhyVPHzn/9cw4cPv+CO3U/MgPWVV15p6zHMVzPry4QoEGsEBeJObm6uEhMT7eDue3U7RZh35rfffrsNCnNUYY4u2vOu+6WXXmqzzgwYp6Wl2aMbs5jbFzoX4sCBA7adZlA6Wtdee61ef/31NgPwrV122WW2djPj68SJE1q3bp2dFWbWA7FGUCDumOmnjz76qAoLC3XHHXdcdHsTDOXl5Xr44YftjrT1DCmXP//5z9qzZ0/zfdON9Lvf/U633Xab/b/MYm6bda+88krzdqYbyHSPffrTn7ZTZqN1zz332HGHoqKiNt+LjEe0rMuEhJnmW1tby2wneCbJu18NvLf77rsv6m3NEUWvXr3s+ITpqrriiiui/tmsrCzbzfPQQw/ZE+N+8pOf2PUtd+SLFi2y53aYUJgxY4aSkpL005/+VHV1dSouLm5XXdnZ2TYAfvzjH9ujGTMV14xzmLEI8z1zXkfE0KFDbftMXWY8ZNiwYe36XUBH4YgCvpeSkqLJkyfb2+0d7DUnt/3oRz/Sk08+qQULFtiuLDM+8vGPf7x5m+uuu87uyM1O+/vf/74NkQEDBmjbtm121lV7rVmzRkuXLrXnYpijoMWLF9uZXjfddNMFB7XfT11AR0owc2Q79H8EPDBnzhw9/vjjevPNN+2YQjTMdNSvf/3rWrFiheKVOYPb1Ga6vczUXcALHFHA98xHdphZQab/P9qQ8APzHs6EnznqISTgJcYo4Fv/+Mc/7PkFZmaQ+eiNWbNmKQjefvtt/f73v7ddW3v37rUD6YCXCAr4lpnpZM7ANoPXZnD4hhtuUBCYk/zM1Fhz/si3vvUt3XnnnV43CZ0cYxQAACfGKAAATgQFAKBjxijMyUVmiTAnCZmPITAnOkXzKZ0AgPhhRh3MJzSbD6k0H0XTIUEROdEIABAc5mNr+vfv3zGD2a2PKMxVx8zcbnNmqfn0ziAwqfqxj33MzqYxR0x+F7R6DGryB2qKf+bj+M0nBJw8edJeA6VDjijM5+Bc6CLxJiTMRygE5YlgTtgy9QThiRC0egxq8gdq8o9ohg4YzAYAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMApKdoN6+rq7BJRU1NjvyYmJtolCCJ1UE/8oiZ/oKb41546EsLhcDiaDQsLC1VUVNRmfUlJidLS0trXQgCAp0KhkPLy8lRdXa2MjIyOCYoLHVFkZmbqjTfeUK9evRQEDQ0NKi0tVU5OjpKTk+V3Qasn6DWVlZWpsbFRQXm3mpWVxeMUx+rr67Vo0aKogiLqrqfU1FS7tGaeBEF5IgS1pqDVE9SazM4nCDuglnic4ld7aghGZxsA4JIhKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIA/g+J/yv5HY38+Vlf+8EolfzdZ3Zd0178t/zeNWTtGs/4wS88eetbrJiLAZs2apb59+3rdDFwEj1NsRH2Fu1ia9n+m6ckXnzxvXU1djV1eOfmKdhzeocPVhzX2w2M9ayOCzVwe8tixY143AxfB49RJg2LjoY3nhcSN/W7U2GvH6vKUy1UVqtKeN/boz0f+7GkbAaAzibug2FS5qfn2h3t+WDu/ulOXJV523jbmyGLvsb0KAnOB87lz5+rhhx/WgAEDFARBrAnozOJujOJM45nm2yffOWm7mlrLSM3QzVffrCDYs2ePVq9erVGjRqmyslJBEMSagM4s7oJiWL9hzbf/GfqnPrLiI7rxZzfqa898TY/938d06K1DCpKRI0dq/fr1qqqqsjvWiooK+V0QawI6s7jrevrSx7+klbtWavfru+39xnCjHZcwS8Snr/60VnxuhYb0HaJ4Nn/+fJ04cSKqbbOysrRr1y6NHj1aO3bs0KBBgxSPglgTAJ8FRVJikrZO26rvP/99/ddf/0vH3m47o+H5V59XzpM52jdjn/qk91G8WrNmjY4ePdqunzEzOMrLy+N2pxrEmgD4rOvJ6JbaTYtvXaw38t9Q2QNlevzOx3XfkPvULaVb8zZmBlTrKbTx5siRIwqHwxddamtrNWbMGPszRUVFys3NVbwKYk0AfBgUEQkJCbruiuv05aFf1tq71+rFB15UYsK5Jr90/CX53alTpzRu3Dht375dS5Ys0YIFC+R3fq3JDMKvWrWqzfr9+/dr+fLlnrQJbfE4xV7cdT098bcn9M6ZdzTl+il2dlNL6cnpNijMuIXRo0sP+d2hQ4e0d+9eLVu2THPmzFEQ+LWmefPmaePGjQqFQs3r9u3bp1tuuUWnT5/WhAkTlJmZ6WkbwePkhbgLipdPvqyiHUWa/exsO2h9w5U3qGfXnjp++rh+Xf7r86bPjvvwOPnd0KFD7Y61d+/eCgq/1vTUU09p/Pjxys/PV58+TWNf2dnZ9ryQTZs2sfOJEzxOsRd3QRFhjio2/7/NdrmQfx/27xp9zWgFgd92qEGtqVu3bvad6h133KFt27bZdWfOnNHmzZs1fPhwr5uHd/E4xV7cBcXskbN1/RXXa+vLW7X7jd16s/ZNVb1dpbPhs+qT1kc3XnWjHdie8NEJXjcVAZSenq4NGzborrvusn3hpaWl9ggJ8YXHqZMHhRl3uOdj99gF8ELXrl1tFwbiG49T7MT1rCcAgPcICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAEDHXAq1rq7OLhE1NTX2a0NDg12CIFIH9cSvINeUmBic922RWnic4ld76kgIh8PhaDYsLCxUUVFRm/UlJSVKS0trXwsBAJ4KhULKy8tTdXW1MjIyOuaIoqCgQHPnzj3viCIzM1Pl5eVKSUlRUBI2KytLOTk5Sk5OVhDeAZWWlqqsrEyNjY0KgqA9Ri0fJ2qKbw0Bez3V19dHvW3UQZGammqX1swfLAh/tJbMEzsoT26Dx8gfqMkfGgPyempPDcHobAMAXDIEBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKGLszJkzmjp1qgYPHqyDBw963Ry8h1mzZqlv374KkiDWhNggKGJ86cFJkybZ64xXVFRozJgxOnDggNfNwgWY6wgfO3ZMQRLEmhAbBEWM1NXVacKECVq3bl3zhcxra2ttWOzbt8/r5gHAeyIoYmTy5MnasGGDCgoKlJuba9dt2rRJp0+fVnZ2to4cOeJ1EwHfXOs5Pz9f5eXlXjel00jyugGdxezZs/WJT3xC8+bN0/3332/XjRw5UqWlpVq/fr369+/vdROBuHf27FlNmzbNdt8mJiZq6dKlXjepUyAoYsR0MZmltREjRtgFwMUnguTl5enpp5/WjBkzVFxc7HWTOg2CAoAvmIkgZoyve/fuSkhI0IMPPhj1bK+BAwde8vYFGUEBwBfjEtu3b2+evbVy5cqof3bixIkExQfEYDaAuGfGI7Zs2aKePXuqR48e2rlzp8LhcFTLhbp80T4EBfCuPXv2aNWqVW3W79+/X8uXL5cfBammYcOGaevWrUpKSlJOTo5eeOEFr5vUadD1BLzLzEjbuHGjQqFQ8zpzjsstt9xipzGb82AyMzPlJ0GraciQIdq2bZtuvfVWe4Rx0003ed2kToGgAN711FNPafz48XaOfp8+few6c46LOaPenPPipx1qkGvKysqyYde7d2+vm9Jp0PUEvKtbt2723bfZkVZVVTVPydy8ebM958WPgliTQUjEFkEBtJCenm7PoDd94L169bLdG8OHD5efBbEmxBZdTx5Yu3atXRCfunbtartlgiSINSF2OKIAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAICOuRRqXV2dXSJqamrs18TERLsEQaSOhoYGBUGkjqA8PkF8jFrWEsSaiouL1djYqKA897KysgLzempPHQnhcDgczYaFhYUqKipqs76kpERpaWntayEAwFOhUEh5eXmqrq5WRkZGxxxRFBQUaO7cuecdUWRmZqq8vFwpKSkK0juGnJwcJScnKwjv6kpLS1VWVha4d3VBeYxaPk5BrCmIz72ygNRUX18f9bZRB0VqaqpdWjN/sCD80VoyL9agvGANHiN/CGJNQXzuNQakpvbUEIzONgDAJUNQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCIsbOnDmjqVOnavDgwTp48KDXzUEnMmvWLPXt29frZsCHCIoYX3pw0qRJ9jrjFRUVGjNmjA4cOOB1s9BJmGsjHzt2zOtmwIcIihipq6vThAkTtG7duuYLmdfW1tqw2Ldvn9fNA4D3RFDEyOTJk7VhwwYVFBQoNzfXrtu0aZNOnz6t7OxsHTlyxOsm4iJHgzNnztThw4e9bgoQcwRFjMyePVuLFi3S4sWLm9eNHDlSpaWlmj59uvr37+9p++C2Z88erV69WqNGjVJlZaXXzQFiKim2v67zMl1MZmltxIgRdkF8M6G+fv163XXXXTYstm7dqkGDBnndLCAmCAp0evPnz9eJEyei2jYrK0u7du3S6NGjtWPHDsICnQJBgU5vzZo1Onr0aLt+xsweKi8vJyjQKTBGgU7PTCQIh8MXXSKz1IyioqLmSQlA0BEUQBROnTqlcePGafv27VqyZIkWLFggvwzCr1q1qs36/fv3a/ny5Z60Cf5D1xMQhUOHDmnv3r1atmyZ5syZI7+YN2+eNm7cqFAo1LzOnLdzyy232KnZ5tyezMxMT9uI+EdQAFEYOnSoDYvevXvLT5566imNHz9e+fn56tOnj11nztsx54WY83gICUSDricgSn4LCaNbt272iMKEQ1VVVfPnjW3evNlO+QWiQVAAAZeenm4/FSAnJ0e9evXSli1bNHz4cK+bBR8hKDywdu1aO4sGiJWuXbvarqZ//vOfthsNaA+CAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnJIUpbq6OrtE1NTU2K+JiYl2CYJIHQ0NDQqCSB35+flKTk5WUGoqLS1VcXGxGhsbFZTnXVZWVmCed0F/7uUHpKbjx49r8eLFUW2bEI7y4s2FhYUqKipqs76kpERpaWntbyUAwDOhUEh5eXmqrq5WRkZGxwTFhY4oMjMz9e1vf1spKSkK0ju7nJycQLxjiLwDCko9LWsqKysL3BFFEB8naorvI4p+/fpFFRRRdz2lpqbapTXzYg3KCzbCPAmC8EQIaj0Gzzt/oKb41Z4agjG4AAC4ZAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAngiLGzpw5o6lTp2rw4ME6ePCg180BEAeOH5eKi6XbbpOuukrq0sVcA0jq108aNUp6+GHpueek6C4z1/GivnARPrj6+npNnjxZ69ats/fHjBmjrVu32tAA0Dn97GfS3LnS22+3/d6bbzYtJiR++EPpjTekvn1j30aCIkbMZWTvuecebdiwwV520FxKtra21obFli1bdN1113ndRAAxtnSp9I1vnLufkCBlZ0sjR0qXXy699Zb0t79Jzz8vvfOOd+0kKGLEHEmYkCgoKNDrr7+uJ554Qps2bdLYsWOVnZ2tPXv2qH///l43E0CM7N8vFRScu9+rl/T730s33dR229pa6cknpa5d5QnGKGJk9uzZWrRokRYvXty8buTIkfZi7dOnT/dlSJhrVufn56u8vNzrpuB9dIPOnDlThw8fVlD4raYf/1g6e/bc/VWrLhwShjm6eOABqXt3eYKgiBHTxTRv3rw260eMGKHvfve78puzZ8/q3nvv1bJly7RmzRqvm4N2Mkewq1ev1qhRo1RZWakg8FtNW7acu/0v/yJNmKC4RVDgfc3cmjJlikpKSjRjxgwVm+ka8BVzNLt+/XpVVVXZHWtFRYX8zm81HT167vbAgVJii73xgQNN4xWtl/vv96SpjFGg/SZNmmRnbnXv3l0JCQl68MEHo/q5WbNmaaB5ReCSmj9/vk6cOBHVtllZWdq1a5dGjx6tHTt2aNCgQYpHQaypJRMC8YygQLvHJbZv325vV1dXa+XKlVH/7MSJEwmKGDBdgUdbvl2NwrFjx+xYU7zuVINY07/+q/TSS023zVdzjkQkMK64omlGlLFwoRQKeddOg64ntEtiYqKdztuzZ0/16NFDO3fuVDgcjmox4zS49I4cORLV4xGZnm0UFRUpNzdX8SqINd1667nbZhqsmfEU0bOn9B//0bR4NdOpJYIC7TZs2DB7omBSUpJycnL0wgsveN0ktNOpU6c0btw4e3S4ZMkSLViwQH7nt5pmzpQuu+zc/a99remciXhEUOB9GTJkiLZt26YuXbrYIwz4y6FDh7R37147a+2RRx5REPitpuuuk1pOeDRnYA8fLt1xh1RYKH3ve9JXvyrV1MhzjFHgfTODhvv27VPv3r29bgraaejQoXbHGqTHzo81FRRI6elNZ2fX1TWdV/HMM03LhZiT8rxAUOAD8dOLEsF/7PxY00MPmZmETZ/5tHmzZGb1mgleyclSnz6SGYu/+WbpzjtNGHrTRoLCA2vXrrULABjmU2LN7CazxCPGKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAB0zDWz6+rq7BJRU1NjvyYmJtolCCJ1NDQ0KAgidQSlnpa15OfnK9lcfT4gNZWWlgbycQpiTcXFxWpsbJTf1dfXR71tQjgcDkezYWFhoYqKitqsLykpUVpaWvtaCADwVCgUUl5enqqrq5WRkdExRxQFBQWaO3fueUcUmZmZKi8vV0pKioJyRJGVlaWcnJxAvFuNvFMNSj0GNflDkGsqKyvrdEcUUQdFamqqXVozf7Ag/NFaMk/soDy5g1iPQU3+EMSaGgOyz2tPDcEYXAAAXDIEBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0ERY2fOnNHUqVM1ePBgHTx40OvmwKislC6/XEpIaFpuu01qfeFHcz8n59w26enSSy951WLwWoopgiLGV5SaNGmSvXxsRUWFxowZowMHDnjdLFx7rfSf/3nufmmptHLl+dusWCFt3nzu/g9/KA0cGLs24jy8lmKLoIiRuro6TZgwQevWrWu+Pm1tba19gu/bt8/r5mH6dGn8+HP3H3lEirxLNV/N/Yhx46QHHoh9G2HxWoo9giJGJk+erA0bNthrj+fm5tp1mzZt0unTp5Wdna0jR4543USsXi316tV0OxSSpk0zeyXp3nul06eb1vfsKT3+uKfN7Ox4LcUeQREjs2fP1qJFi7R48eLmdSNHjrQXa58+fbr69+8vP15zNz8/X+Xl5QqEfv2kRx89d3/nTmnECOl///fcOvP9q65SELpuZs6cqcOHD8tvgvhaindJXjegszCHxWZpbcSIEXbxm7Nnz2ratGm2jzgxMVFLly5VIEyaJE2dKv3iF033X3zx3Pfy8qQvfEFBsGfPHq1evVrr16/X1q1bda0Zp/GJoL2W/IAjCryv2SZTpkyxITFjxgwVFxcrUMzAtTm6aOnKK9sOcPuYeQduQqKqqkqjRo2yA8LAe+GIAu1mZpuYgcTu3bsrISFBDz74YFQ/N2vWLA30w0wh08f91lvnrzP3X3lFuuEGxbv58+frxIkTUW2blZWlXbt2afTo0dqxY4cGDRp0ydsH/yEo0O5xie3bt9vb1dXVWtmOd9kTJ06M/6BoaGgavDaD2Bdav3u3lJqqeLZmzRodPXq0XT9z7NgxO9ZEUOBC6HpCu5jxiC1btqhnz57q0aOHdu7cqXA4HNVyoX7luLNwofS3v527//Wvn7tdViZ9+9uKd2bWTzSPR2RKqVFUVNQ8gwhojaBAuw0bNswOgCYlJSknJ0cvvPCCAsHU0XK85ctfbhqv+MpXzq1btkx67jn53alTpzRu3Dh7dLhkyRItWLDA6yYhjhEUeF+GDBmibdu2qUuXLvYIw/feflu67z4znavp/jXXSD/6UdNt8/VDH2q63djYtF1trfzs0KFD2rt3r5YtW6ZHWp5MCFwAYxR438xAqDkTtnfv3vK9/Hyz92y6nZgoPfGE1K1b033zOVD//d/S6NFNQfLyy9KcOdJjj8mvhg4dasMiEI8dLjmOKPCBBGJHs3Gj9NOfnrtvQmDUqPO3ufnm8z/Gw5zFvWGD/CwQjx1igqDwwNq1a+1gIuKE+ewm83hEFvOBfxfyve+dv93tt8e6pWiF11JsEBQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOCUpCjV1dXZJaKmpsZ+TUxMtEsQROpoaGhQEETqCEo9BjX5Q5BrSgzY/i4aCeEoLzhbWFiooqKiNutLSkqUlpbWvhYCADwVCoWUl5en6upqZWRkdMwRRUFBgebOnXveEUVmZqbKy8uVkpKioCRsVlaWysrK1NjYKL8LWj0GNfkDNcW/+vr6qLeNOihSU1Pt0pr5gwXhjxbkmoJWj0FN/kBN8as9NQSjsw0AcMkQFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIATQQEAcCIoAABOBAUAwImgAAA4ERQAACeCAgDgRFAAAJwICgCAE0EBAHAiKAAATgQFAMCJoAAAOBEUAAAnggIA4ERQAACcCAoAgBNBAQBwIigAAE4EBQDAiaAAADglKUp1dXV2iaiurrZfGxoaFBSJiYkKhUKqr69XY2Oj/C5o9RjU5A/UFP8i++5wOHzxjcNRWrhwofnfWFhYWFgUnKWysvKi+/8E88/7OaI4efKkBgwYoFdffVXdu3dXENTU1CgzM1OvvfaaMjIy5HdBq8egJn+gpvhneoWuvvpqnThxQj169OiYrqfU1FS7tGZCIgh/tJZMPUGqKWj1GNTkD9Tkjy61i24Tk5YAAHyLoAAAXJqgMN1QCxcuvGB3lF8Fraag1WNQkz9QU7DqiXowGwDQOdH1BABwIigAAE4EBQDAiaAAADgRFAAAJ4ICAOBEUAAAnAgKAIBc/j+c6Peoa2v/OgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "my_policy = [\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_DOWN,\n", + " A_DOWN, # First row\n", + " A_UP,\n", + " A_DOWN,\n", + " A_DOWN,\n", + " A_LEFT, # Second row\n", + " A_UP,\n", + " A_RIGHT,\n", + " A_DOWN, # Third row\n", + " A_UP,\n", + " A_LEFT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT, # Fourth row\n", + " A_UP,\n", + " A_LEFT,\n", + " A_DOWN,\n", + " A_RIGHT,\n", + " A_UP, # Fifth row\n", + "]\n", + "\n", + "V_my_policy = policy_evaluation(policy=my_policy, P=P, R=R, gamma=gamma)\n", + "\n", + "plot_values(V=V_my_policy, title=\"Value function: my policy\")\n", + "plot_policy(policy=my_policy, title=\"My policy\")" + ] + }, + { + "cell_type": "markdown", + "id": "e61f5ee8-f9cd-4fbc-96c0-0a8d661bd1e5", + "metadata": {}, + "source": [ + "**Exercise 10.** (optional) How can we find an optimal policy?\n", + "(We will discuss this question next week, but you can already start thinking about it!)" + ] + }, + { + "cell_type": "markdown", + "id": "00ae548b", + "metadata": {}, + "source": [ + "To find an optimal policy $π^*$ (a policy that yields the highest possible expected return from every state), we generally use one of two main dynamic programming algorithms:\n", + "\n", + "1. **Policy Iteration**: This method alternates between two steps until convergence:\n", + "\n", + "- *Policy Evaluation*: Calculate the value function Vπ(s) for the current specific policy (as we did in Exercise 8).\n", + "\n", + "- *Policy Improvement*: Update the policy to be greedy with respect to the current values. For every state s, we choose the action a that maximizes the expected next value:\n", + " $$π_{new}​(s) = argmax​_{a} \\sum_{s\\prime} ​P({s \\prime}∣s,a)[R(s)+ \\gamma V_{\\pi}({s\\prime})]$$\n", + "\n", + "1. **Value Iteration**: Instead of evaluating a specific policy until convergence every time, we iteratively update the value function directly using the *Bellman Optimality Equation*:\n", + " $$V_{k+1}​(s) = max_a ​(R(s)+ \\gamma \\sum_{s\\prime} ​P(s\\prime∣s,a)V_k​(s\\prime))$$\n", + "\n", + " Once the values converge to the optimal values $V^{*}$, we simply extract the optimal policy by acting greedily towards those values." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "studies", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/M2/Reinforcement Learning/Lab 2 - Second maze.ipynb b/M2/Reinforcement Learning/Lab 2 - Second maze.ipynb new file mode 100644 index 0000000..b1a8ebc --- /dev/null +++ b/M2/Reinforcement Learning/Lab 2 - Second maze.ipynb @@ -0,0 +1,872 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "44b75d44", + "metadata": {}, + "source": [ + "# Lab 2 - Second maze\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 535, + "id": "100d1e0d", + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "np.set_printoptions(\n", + " precision=3, suppress=True\n", + ") # (not mandatory) This line is for limiting floats to 3 decimal places, avoiding scientific notation (like 1.23e-04) for small numbers.\n", + "\n", + "# For reproducibility\n", + "rng = np.random.default_rng(seed=42) # This line creates a random number generator.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1018deab", + "metadata": {}, + "source": [ + "## 2. Maze definition and MDP formulation\n" + ] + }, + { + "cell_type": "markdown", + "id": "ca4fa301-c14f-44ec-b04f-b01ca42d979a", + "metadata": {}, + "source": [ + "### 2.1 Define the maze " + ] + }, + { + "cell_type": "code", + "execution_count": 536, + "id": "f91cda05", + "metadata": {}, + "outputs": [], + "source": [ + "maze_str = [\n", + " \"############\",\n", + " \"#S#.......X#\",\n", + " \"#.#.###.#.##\",\n", + " \"#.....#X#..#\",\n", + " \"#.###.####.#\",\n", + " \"#...#X#X...#\",\n", + " \"###.######X#\",\n", + " \"#.....X...##\",\n", + " \"#.###.#.#..#\",\n", + " \"#...#...X#.#\",\n", + " \"#X#.X#X##G.#\",\n", + " \"############\",\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 537, + "id": "564cb757-eefe-4be6-9b6f-bb77ace42a97", + "metadata": {}, + "outputs": [], + "source": [ + "n_rows = len(maze_str)\n", + "n_cols = len(maze_str[0])\n", + "\n", + "figsize = (n_cols / 2 if n_cols > n_rows else 8, n_rows / 2 if n_rows > n_cols else 8)" + ] + }, + { + "cell_type": "markdown", + "id": "adc49d58-2730-41d8-96fb-ca7c9cb4fcdf", + "metadata": {}, + "source": [ + "### 2.2 Map each walkable cell (not a wall '#') to a state index\n" + ] + }, + { + "cell_type": "code", + "execution_count": 538, + "id": "7116044b-c134-43de-9f30-01ab62325300", + "metadata": {}, + "outputs": [], + "source": [ + "FREE = {\n", + " \".\",\n", + " \"S\",\n", + " \"G\",\n", + " \"X\",\n", + "} # The vector Free represents cells that the agent is allowed to move into.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c9ad05e-9c6c-4e00-918c-44b858f45298", + "metadata": {}, + "source": [ + "**Dictionaries to convert between grid and state index**" + ] + }, + { + "cell_type": "code", + "execution_count": 539, + "id": "a1258de4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of states (non-wall cells): 62\n", + "Start state: 0 at (1, 1)\n", + "Goal states: [60] at (10, 9)\n", + "Trap states: [8, 18, 27, 28, 33, 39, 54, 56, 58, 59] at (1, 10)\n" + ] + } + ], + "source": [ + "state_to_pos = {} # s -> (i,j)\n", + "pos_to_state = {} # (i,j) -> s\n", + "\n", + "start_state = None # will store the state index of start state\n", + "goal_states = [] # will store the state index of goal state # We use a list in case there are multiple goals\n", + "trap_states = [] # will store the state index of trap state # We use a list in case there are multiple traps\n", + "\n", + "s = 0\n", + "for i in range(n_rows): # i = row index\n", + " for j in range(n_cols): # j = column index\n", + " cell = maze_str[i][j] # cell = the character at that position (S, ., #, etc.)\n", + "\n", + " if (\n", + " cell in FREE\n", + " ): # FREE contains: free cells \".\", start cell \"S\", goal cell \"G\" and trap cell \"X\"\n", + " # Walls # are ignored, they are not MDP states.\n", + " state_to_pos[s] = (i, j)\n", + " pos_to_state[(i, j)] = s\n", + "\n", + " if cell == \"S\":\n", + " start_state = s\n", + " elif cell == \"G\":\n", + " goal_states.append(s)\n", + " elif cell == \"X\":\n", + " trap_states.append(s)\n", + "\n", + " s += 1\n", + "\n", + "n_states = s\n", + "\n", + "print(\"Number of states (non-wall cells):\", n_states)\n", + "print(\"Start state:\", start_state, \"at\", state_to_pos[start_state])\n", + "print(\"Goal states:\", goal_states, \"at\", state_to_pos[goal_states[0]])\n", + "print(\"Trap states:\", trap_states, \"at\", state_to_pos[trap_states[0]])\n" + ] + }, + { + "cell_type": "code", + "execution_count": 540, + "id": "fc61ceef-217c-47f4-8eba-0353369210db", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnwAAAKSCAYAAABIowakAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAaQdJREFUeJzt3QeYFFW6xvFvyEhOCkrOSUFZBCPJgKCzigEUURdF2GsAVtE1oLCXK7oYMCxgWgNgICgOBowIiggGFCWJZEGSIGAgz33eU1TTg8CMzDDVfeb/e56mq8Mwp6a6qt46qVPS09PTDQAAAN7KF3UBAAAAcHgR+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AOSalJQUGzBgQJbfe/3111te1Lp1a3c7nP/n0qVL3d/42WefzdHfAyAxEfiAXKYTrE60un388cd/eF3fdlilShX3+rnnnms+++STT1wA/Pnnn5Pydw8bNozABCApFIi6AEBeVaRIEXvhhRfs1FNPzfD8lClT7IcffrDChQubb37//XcrUKBAhtA1cOBAu+qqq6x06dK5Wpac+N0KfOXLl3f/R05655137HCrVq2a2x4FCxY87L8LQPSo4QMi0qFDBxs7dqzt3Lkzw/MKgc2aNbOKFSuajyE3PvBh/woVKuRuh5NqkLU98ufPf1h/D4DEQOADInLppZfaTz/9ZO+++27sue3bt9u4cePssssu2+/P3H///XbyySdbuXLlrGjRoi4Y6v3xVNsUNhnve4vvP7dt2za7++67rXbt2q42Uc3It9xyi3v+YB555BEXEuKbQh944AH3///jH/+IPbdr1y4rUaKE3XrrrbHn4sug+379+rnlGjVqxMqovmXxJkyYYI0bN3ZlbNSokU2aNMmy4tFHH3XvP+KII6xMmTL2l7/8xYXprPzuZ555xtq2bWtHHnmk+70NGza04cOHZ/j/q1evbnPmzHE1suHPx/eR09+nT58+7u+q/0N/5/vuu8927979p/vbffjhh+7/HzNmjP3f//2fVa5c2YW1du3a2ffff/+Hn3/iiSesVq1a7jNy4okn2kcfffSH9xyoD9/8+fPtkksusQoVKrifr1evnt1xxx0Z3rNy5Urr3r27HXXUUbHt8t///vdPbQMAuYtLbSAiCgwnnXSSvfjii3bOOee459566y3btGmTdenSxQWrfT388MOWmppqXbt2deHwpZdesosvvthef/1169ixo3tPz5497YwzzsjwcwpJo0ePdgFGFDr0/6gP4bXXXmsNGjSwb775xh566CH77rvvXMg6kNNOO839vH427GOoQJEvX74MwWLWrFn2yy+/2Omnn77f/6dTp07ud2n99XvVNCoKGiH9jldeecX+53/+x4VH/U0uvPBCW758uQu9B/Lkk0/ajTfeaBdddJH17t3btm7darNnz7YZM2a4MJ3Z71a4U1DR30g1khMnTnRl0Hpfd9117j1Dhw61G264wYoXLx4LRApA8ttvv1mrVq1cMNL2qFq1qmtCvu222+zHH390P3so7r33Xvd3vvnmm93n5N///rf7LGi9Qk8//bT7nbowUOBcvHixW4+yZcu68Hkw+htp+6qZV58LfUYXLVrk1l9BU9asWWMtW7aMDarR30yf26uvvto2b97sfmdWtgGAXJYOIFc988wz6dr1Pvvss/THHnssvUSJEum//fabe+3iiy9Ob9OmjVuuVq1aeseOHTP8bPi+0Pbt29MbN26c3rZt2wP+voULF6aXKlUq/cwzz0zfuXOne27kyJHp+fLlS//oo48yvHfEiBGubNOmTTvg/7dr1670kiVLpt9yyy3u8e7du9PLlSvnyp4/f/70LVu2uOcffPBB9zs2btwY+1n933fffXfs8ZAhQ9xzS5Ys+cPv0fOFChVK//7772PPff311+75Rx99NP1g/vrXv6Y3atTooO852O/e9+8sZ599dnrNmjUzPKff0apVqz+893//93/TixUrlv7dd99leP6f//yn+xstX778oGXT/xn//06ePNmVtUGDBunbtm2LPf/www+757/55pvY5+HII49Mb9q0aYb3PfHEE+598f+n1lvP6fMYOv30093ncdmyZRnKo20cuvrqq9MrVaqUvn79+gzv6dKli/uchX+7rGwDALmHJl0gQmo6U8d51dBt2bLF3R+s9kNNbKGNGze6Wh7VyHz55Zf7ff+vv/5qF1xwgWtOU21W2F9LfQdVq1e/fn1bv3597KZmTJk8efIBy6AaJtUeTZ061T2eN2+ea5r+5z//6UYYT58+3T2v2j41xWZnMIZqKtU0GTruuOOsZMmSrtbqYPQ7NfDls88+O6TfG/931t9YfxvV2On36nFm9PfVdtHfPf7vq/VRU3f4t/uz/va3v2Xo26ffIeHf4/PPP7e1a9dar169MrxPzfylSpU66P+9bt06Vy411apGMp5q80Tbd/z48Xbeeee55fh1O/vss93fJvwsZncbAMhZNOkCEVJzmEKA+jWpGVBhQE1gB6JAOGjQIPvqq68y9LULT8j76tGjh2uSU3NifBPowoULXVCLbz6Np9BwMAoa6gensKpgV6lSJTvhhBOsSZMm7vGZZ57pmmMVaLNj3+AhClEKuwejfoPvvfee67+mvnNnnXWWC9KnnHJKln7vtGnTXP9GhVdtl3gKNZmFJ/191Xx5qH/frP499LeQ8O+xbNkyd1+nTp0M71MTbc2aNQ/6f4ehUSH9YKFQfRPVR1C3g61bdrcBgJxF4AMippOggtnq1atdX74D1YgpSKkvlvrEaToQhSydyDXAYH8d4dXfT7V6o0aNsqZNm2Z4TX3Rjj32WHvwwQf3+7sy6+ulqWR27NjhApHKFdY06V6P1fFf4SB8/lAdaARp0OJ7YKq9XLBggQvI6r+oWin9ze666y43FcvBKCBrMIRqP/X30d9CtWVvvvmm6++XlUEXeo9CrwbB7E/dunUtN/8eOSVc98svv9yuvPLK/b5HtbDZ3QYAch6BD4iYmlzVyf7TTz+1l19++YDv0wlTIzPffvvtDHP0KfDtS6FLHfvVgV6d+velZtKvv/7aBZsD1Q4ejGptFIL0e3QLR7wqjKqz/vvvvx97fDCH8ruzqlixYta5c2d30wAXDdTQwAMNnNDf8UC/WwMUVHualpaWoUZtf83cB/o/9PfVgJV9B8/kxtx6YQ1j2DwvCudLlixxNbAHEtYAfvvttwd8j2osNXhGNdFZWbfMtgGA3EMfPiBiGuWpUaFqIlXfqIPV7ihg6GQbP7XGviNqNQpUTamqhRsyZMh+/y+9rhGkCmf7UjOt+v4djE7WzZs3dzWIGjEbX8Onn9doWoUe1UJmFggkp79pQ30K4ymcamoV1YQp/Bzsd4e1aPG1ZmrG3V+w1v+xv7Lr76vaT4Xzfen9+869mFM07YlC2YgRI1zACmnqlcz+xvo5BXRNr6JtGi/8W+hvo1HSuvjYXzBUre6f2QYAcg81fEACOFDzWDxNu6Imxvbt27tmYPWV+s9//uP6R6m/WEhTYejEq+ZETduyb3Obbt26dXNzuqlzv2qu1K9KQVJNsXpeQUXh4WAU7jRNiPqzqXlYNO2L5m1TU15Wvn1C8wiKpjXRVDRqolboDcPYoVJ/MU1crfXSVCnqr/jYY4+5v6FqqA72u/WzCidaVs2rauoUjLVuCtP7ll9hXf0qtR30HtWsqcZTNYSatkZ/B71PIVpT32jeRAX1cCqYnKR1UFlUbpVDNWuq2VNYzawPnyio60JB/TE1LYvmKFRZ33jjDddvVLTN9Zlp0aKF64qgELdhwwY3WEN99rSc1W0AIBfl4ohgAPtMy3Iw+5uW5emnn06vU6dOeuHChdPr16/v/i9NcxK/K2vqDT3e3y1+ShRN4XHfffe5qTP0/5UpUya9WbNm6QMHDkzftGlTpuvxxhtvuP/znHPOyfD8Nddc455XWfe1bxnCKUyOOeYYN4VL/DQpWr7uuuv2+3e58sorD1q2xx9/3E0xoulitG61atVK79ev3x/W60C/Oy0tLf24445LL1KkSHr16tXd3+m///3vH6ZxWb16tdtGmspk32lPND3Nbbfdll67dm03vUz58uXTTz755PT777/f/e0PZVqWsWPHZnjf/qZWkWHDhqXXqFHDrftf/vKX9KlTp/7h/zzQz3777bfpF1xwQXrp0qXd+terVy+9f//+Gd6zZs0at22qVKmSXrBgwfSKFSumt2vXzk3/8me3AYDckaJ/cjNgAgAAIHfRhw8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxXI6vcnrlq1yk2WeTi/CgkAAABZo5n1tmzZYkcffbTly5cv+4FPYS+zL1MHAABA7luxYoVVrlw5+4Ev/BocfVl2gQJ8GxsAAEDU9L3c+krDrHxdYZbSW9iMq7Cn72oEAABAYshKdzsGbQAAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4rYHlQWlqa+SQ1NdV84du2AQAknlSPzptZRQ0fAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsCXg7bv2m73fHSPNfxPQyt2TzErObik1X6ktl3w8gX29eqvoy4eAACJY/Jks3z5zFJSzO69d+/zu3aZtWwZPF+tmtnmzVGW0hsEvhzU751+dscHd9i89fPsmBLHWPXS1W3tr2ttwvwJtnDDwqiLBwBA4mjTxqx372D57rvNZs8OlhX+ZswIAt9zz5mVLBlpMX1RIOoC+OTlOS+7+7tOv8sGthnoltPT0+2TFZ/YkcWOjLh0AAAkmMGDzd5+22zePLNu3cyefNJsYHD+tD59zFq3jrqE3qCGLwftTt/t7t9Z/I69/t3rtuaXNZaSkmKnVD3F6pSrE3XxAABILEWKmI0caVagQFDD16qV2Y4dZg0bmt1zT9Sl8wqBLwf9T/P/cfef/vCpnffieVbxgYpW/7H69r9T/te27twadfEAAEg8zZqZ3XlnsLx1q1n+/EEIVBhEjiHw5aABrQfYK5e8YufVPc9KFg76HCz4aYHd9eFd1uv1XlEXDwCAxLRwYcZBG0uXRlkaLxH4ctgFDS6wtEvTbOOtG23mNTPt2COPdc9r4AYAANjH+PFmo0cHyxqVKz17mq1ZE2mxfEPgy0F3fnCnfbX6K7ecLyWfNT+mudUtV9c9LlWkVMSlAwAgwSjU9drTAtahg9n06WblypmtX2/Wo0fUpfMKgS8HPfXlU3b848dbhSEVrNkTzazKQ1Vs/Lzx7rXLGl8WdfHytKlTp1qHDh2sQoUKbiCNbiNGjLBk9MADD1jr1q2tUqVKVrhwYatWrZpdeeWVtnjxYktGQ4cOtSZNmljp0qXd+lSuXNkuvvhimx1O0ZCkLrnkkthnrUuXLpZsBgwYECv/vredO3daMlq3bp3dcMMNbp8pVKiQlS9f3tq1a5d0+87SpUsPuG1007ZLGtdcE4S7smXNnnrKrFIls+HDg9cmTjR7+umoS+gNpmXJQYPaDnKjc2evmW3z18+3nbt3Wr1y9axL4y525+l7OqQiEl9++aW9++67VrNmTVuvg0sSe/TRR2358uVWr149K1q0qC1ZssSef/55e+edd2zBggVWMsnmrJoyZYo7EWvbbN261a3DuHHj7IMPPnDrWaxYMUs2zzzzjI0dO9Z8oFBUq1atDM8pVCQb7fctWrRw+4vCXt26dd20WdOnT7dVq1a5z1+y0IWR1iXezz//7PYd0cVgUlDAe/31YHnYsCDsycUXm3XtGjTz9u1r1q6dWfXqkRbVBwS+HHTNCde4GxJPt27drGfPnrZmzRqrUaOGJbMePXq49alatap73LdvX1dLtnr1anv//fftggsusGTy4osvWpG40Xj9+/e3QYMG2YYNG2z+/PnWTCP4ksiiRYvsxhtvtJNOOslWrFhhP/zwgyWzjh072rPPPmvJ7s4773Rhr1GjRu7iLwxF27dvd8Evmajsn376aYbnrr/+ehf4ypQpY10VlpKldk+3/Rk1Krghx9CkizyhXLlyrjbMB3fccUcs7Mlpp52W4co/2Sjsvfrqq9ayZUtr2LCh3bNn7i01v6sWJpmoqVMn23z58tno0aMtv6aXSHLjx493+45CxrnnnmuzZs2yZKNAN2bMGLdcpUoVO/PMM13NsboSaP2Scb+J99NPP7laZfn73/9uxYsXj7pISEAEPiCJ7dq1y5544gm3rCYp9UdKRqp5nTFjhs2bN892797tamEnT55sJUqUsGQycOBAtx7Dhg1L+ppkUWCtWLGiVa9e3dUgv/HGG67mMtlCn7oMbNy40S1PmjTJNX+qJkz9RC+77DLXhSCZ6fP222+/ueCqPorA/hD4gCT166+/uubbt99+252UJ06cmLQ1Fb169XJBb9myZda5c2fX9Kb7LVu2WLL4/PPPbfDgwXb55ZcnT5PaQSgIrV271hYuXOiCuIKSbNu2zf7zn/9YMokfZNKgQQM3SEM3Lctjjz1mySp+e+izp2MBsD8EPiAJqbalVatWLuSp2XPatGmuOTSZaSCAmqpvv/1293jOnDmuf1+y+Pbbb12Nq2qL1KSmmwadiJoN9XjTpk2WLPS5KquRk3ucffbZrmuEhOuVLNQ9QAM1RM24WtZNy+Go12SlAVuqIdf+c9NNN0VdHCQwAh+QZBSE1N/tiy++cP33NMowmUYY7tv3aOTIka7jfOjNN9/MUIuZbDTSWOXWLRwMoBqm+MfJ4L777ssQ7DTQQdtL1MSbTAoWLGinn366W1Yz7o4dO9wtnPqnTp3k/K5zfZ40TVM4uCassQT2h8B3GLz07Ut2wuMnWNH/K2pl7ytrF425yBZtWBR1sfK0V155xWrXru3mrwvddddd7rlka37r1KmTa/oUNXlqfkEFQN2e0jQHSUTlv+KKK9wcfMcee6yr4bvtttvca+q/p3VNFldddZU7AcffNN+bqHlaj7WeyWL48OEu2GkdVHusGj7RYIc+ffpYstHIb9XqzZ071/Wv1E3L6qcY1ionG9Xwh1Ox9OvXz5KWLuxuuUXJ2+yII8xKlTI77jizIUOUaqMunTcIfDns6S+ftkvHX2qzVs+ySsUr2a70XW7y5ZP/e7Kt/mV11MXLszZv3uymywiDUtiRW8+tXLnSkq3PTuirr75ygwTCW7JNAaIApEmJNQJU2+LHH390oyjVF0nrEwYm5D6FIA0CUk2Y+rtpW+jiSDXLydh9QPPWaW5HXfRpAIdqYs844wzXHaJNmzaWjO6//353f+KJJ8ZqMJPSddcF4e777zX6zExziX7zTRACk7h/ZaJJSc9CG4NOlqVKlbL27du7qvFkl5aWdlj+3+27ttsxDx5j639bbxc2uNDGXTLOVm1ZZfUfq29btm+xG068wR4555Ec/72pqanmi8O1bQAACap2bU1gada+vdlbb5n9/nvwzRtbt5qpX+KeYJuTUj05b+qCTAOq1D84s0n3qeHLQZ+t/MyFPVHgk6NLHG0tK7d0y5O+D0a5AQCAPcK5RDUSvHFjjRgKwp6eZyBKjuGbNnLQis0rYstHFjsytnxU8aPc/fJNyTWyDQCAw07fa757t4Yca1Ra8JxGVasfX5kyUZfOG9Tw5YJkGpkHAECueughs5EjzU45xWzt2iD0adJ1zS/4z39GXTpvEPhyUJWSVWLLa39d+4flqqX2fh0WAAB53m+/6Qu0g9G4F16oSRPNNChI4U/eey/qEnqDwJeDmh/T3MoVDSYm1chc0aCNT38IvuS6fe32kZYPAICEC3zhN6F88UVwr/57YdNusWLRlc0zBL4cVCh/Ibun3T2xwFfz4ZrW4D8N3Ajd8keUt3+eStU0AAAx5cubhVPKjB4dzMWnib01aleuvDLS4vmEwJfDrm12rY26YJQ1rdjU1e6lWIp1atDJPun+iRuxCwAA4kyYEMy5p9G5q1aZ6Zt3WrQwGzXK7H/+J+rSeYNRuodB1+O6uhsAAMiERuLed19ww2FDDR8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCkRdAMBXqampURcBeUBaWpr5xKf9xqdt49N2yauo4QMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4ctgvv/xid999t9WvX9+KFi1qRx99tP3973+3jRs3Rl20PGXq1KnWoUMHq1ChgqWkpLjbiBEjMrxnx44dNnDgQKtZs6YVKlTIKleubH379nXbEMhrsrLPPP7443bqqadasWLFYu+ZP39+ZGXOCzLbLlu2bLE+ffpYs2bNrHz58u68U7duXevfv797DQgR+HLYeeedZ//617/s+++/dzudwoN2zrPOOst27twZdfHyjC+//NLeffddK1u27AHf0717dxswYIAtW7bMhb61a9fa0KFD7dxzz7Xdu3fnanmBZNhn3nrrLZs1a5YLH0iM7fLTTz/Zww8/bHPmzHEXrcWLF7eFCxfaoEGDrHPnzrleXiQuAl8Omjt3rn344YduWTvg119/bV988YV7/Pnnn9uYMWMiLmHe0a1bN9u8ebO9/fbbBzyIjho1KratVEsxfvx493jKlCk2YcKEXC0vkOj7jAwbNsy9RxdKSIztUqRIERsyZIitW7fOvvrqK1uxYoW1bNkyFtBpXUKIwJeD4muF8uXLl+Fe3nvvvUjKlReVK1fONW0ciA6EoQsvvNDdd+zY0R08ZdKkSblQSiB59hlRF5X8+fPnWpmQ+XapWLGi3XzzzVaiRAn3WMew5s2bx84/BQoUyLWyIrER+HJQgwYNrHHjxm75hhtusKZNm9oJJ5wQe33lypURlg7xdBUcOvLII2MHR/WBkeXLl0dWNgA4VOqaErZWdOnSJRYEAQJfDtKVr2qOunbt6oLD4sWL7bTTTrNatWq51wsWLBh1EZGJ9PT0qIsAAIdk0aJFblDNqlWr7JRTTvnDoBvkbdT15jB1mg37hsnWrVtdlbvUq1cvwpIhXpUqVTJcEVeqVMk1yasDtFStWjXC0gHAnzN9+nRLTU219evXu8GDL730kh1xxBFRFwsJhBq+HKbBAOFQ+F27dlm/fv1s06ZN7jEjphJH+/btY8th88cbb7zhAvq+rwNAIhs3bpy1bdvWhT11J9KgM8Ie9kXgy2H//e9/XZ+wY4891tXsPfbYY+55zZN04oknRl28POOVV16x2rVrW+vWrWPP3XXXXe45NblrzqpLL73UPd+7d2/X/zIcvKFm+PPPPz+ysgOJuM/Irbfe6h7rPnT22We75x555JFIyp3Xt4uaby+55BJ3sar5RGfOnGknn3yyG6mrmyohAKFJN4cp1E2ePNn131N/MAULTbx89dVXR120PEXTGKg/SzxNW6Cbmt3lueeeszp16tjzzz/v3qu5xS666CI3f1X86GogL8jKPrNmzZo/vCcc4LRhw4ZcLG3ekdl22b59e6zvsZZnzJjxh58HJCU9C73U9YEpVaqUa+byYeBBWlqa+UT9Nnzh07bxabsgcfm0z/i23/i0bXzaLj7RN0ZpGjF1HStZsuRB30s1BgAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADguQJRFwDZl5aWFnURsB9sl8SVmpoadRGApOLb8Sw1Dx4DqOEDAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAABlNnmyWL59ZSorZvffufX7XLrOWLYPnq1Uz27w5ylIC+BMIfACAjNq0MevdO1i++26z2bODZYW/GTOCwPfcc2YlS0ZaTABZR+ADAPzR4MFmDRqYbd9u1q2b2cyZZgMHBq/16WPWunXUJQTwJxD4AAB/VKSI2ciRZgUKBDV8rVqZ7dhh1rCh2T33RF06AH8SgQ8AsH/NmpndeWewvHWrWf78QQhUGASQVAh8AIADW7gw46CNpUujLA2AQ0TgAwDs3/jxZqNHB8salSs9e5qtWRNpsQD8eQQ+AMAfKdT16hUsd+hgNn26WblyZuvXm/XoEXXpAPxJBL5smDp1qnXo0MEqVKhgKSkp7jZixIgM7+nevbvVqVPHihcvbsWKFbNatWrZjTfeaBs2bLBkW5fQli1b3Hpk9r5EXpfWrVvHXou/nXrqqZZosrptZs+ebRdddJF7X6FCheyYY46xSy65xJJpXZ599tn9bpfw9uGHH0Za/jzlmmuCcFe2rNlTT5lVqmQ2fHjw2sSJZk8/HXUJ84ysHAOWLl1qV111lVWrVs2KFCli9erVs3//+9+2e/duSxQPPPCAO/ZWqlTJChcu7Mp65ZVX2uLFi2Pv2bFjhw0cONBq1qzpjmOVK1e2vn372i+//BJp2X1QIOoCJLMvv/zS3n33XffBXK8D43689tprVqpUKatfv76tW7fOfbAfffRR++6772zSpEmWTOsSuv766zPsoInmz6yL3qODaKhRo0aWjOvz8ccf21lnnWW///67lSxZ0q2HDpD6/CXTumhbtGjRIsNzy5cvtx9//NEtV6xYMdfKmqcp4L3+erA8bFgQ9uTii826dg2aefv2NWvXzqx69UiLmhdktt/o3HLiiSe6e1Uu6Hzz7bff2q233mqrVq2yoUOHWiLQuU/7s8Jo0aJFbcmSJfb888/bO++8YwsWLHDHLlWSjBo1yvLly+cqS3SuUflnzZplH3zwgXseh4a/XDZ069bNNm/ebG+//fYB37Ny5Ur3gf38889t2bJlsRqkadOmWbKti4wZM8btoIlWc3Qo6yL9+/e3Tz/9NHZ7/PHHLdnWJz093Xr06OHCXteuXW316tXu4Lhw4cJMA2+irUvHjh0zbA/dypcv714788wz3YkMuVS7l54e3Dp3zvjaqFHB8/qWDcJeQuw3Y8eOdWFPtM989dVXNnxPbexjjz1mK1assESg45RqIufNm+fOi300n6OZO2a9//77Ltgq7MnDDz9s8+fPt/HqR2pmU6ZMsQkTJkRa/mRH4MuGcuXKuauUg1HVukKFai2qV6/uamIk0ZoOs7IuOmj07NnTmjVrZoMGDbJElZV1CampQE0LunK+9tprbU0CdkbPbH3UlKsDYxj+dPWsWuW2bdu6muRk3TaiWvBvvvnGLffr1+8wlgxIXJntN/HNtmENWHi/a9cum6yvyksAd9xxh1WtWjX2+LTTTost6zj81ltvxR5feOGFsYtAnUclkVrFkhGBLxeopmXmzJmuhk/OOOMMV1OWTHRA0VWm+le88MILVrBgQUt2OoCqn5uaEdW08OSTT9pJJ51kv/76qyUTNYWEtG2OOOIIt6yDvPrL6Io6WQ0ZMsTdN2nSxNXwAfgj9e9TU66ocqFp06bWKxxws6elKdEoiD7xxBNuWRfc7dq1y1ATeeSRR8aCa1jLr+ZgHDoCXy546aWXbPv27a6ZrXHjxvbee+/ZddddZ8lE1euqUtd93bp1Ldk99NBDtnHjRtfPRQeZ2267zT2v4Pfqq69aMtm5c2ds+eqrr3a1fWrSyZ8/v+vHp4EQySjssyM333xz1MUBEpYCk/rBtWnTxgUk9dvTAA4N7pBEu0DXRfUFF1zgmqjVL3fixImuhu9A1HKB7CPw5RLtcLrqUh8GGTlyZMI1tx3M119/7e579+7triTjBzeoH8bJJ59syeT444+PHWB0ULzssstiryXbVaRqKUPNmzd39zVq1IgNRknWGr7777/f3VepUsW6dOkSdXGAhKbWCV0g/fzzz7Z27Vo3+CEMSurmkSjUX69Vq1Yu5KnyQP3ZG+rr+vbs6yGtQ9i69NNPP7nl+OZg/HkEvsPos88+yzCNhGr5VLsXSramw7DMuv3222+x57Zt25bhcaLTgeTBBx9008uEXn755diy+lomE43O0+g20eAgUfeBsBO3RrolG4XusNuDLjIK6PtcARyQ+oermVTUehHWiqs5VM2liWDOnDnWsmVL++KLL1z/venTp7vayVD79u1jy+FgjTfeeMO26mv99nkdfx6BLxteeeUVq127tusnFbrrrrvccxotqQ+3qtjLli3ravc095CuakSP1S8pWdZFzYK6WgxvavoMaTSYmhCTZV0UTm+66Sa3XRo0aOCuGsNBKHrcqVMnSySZrY/6Ig4YMMA9/9RTT7l10GdLB381l2gwSrKsS0jTMKipWoNPEqn8Xps6NZhgWTXDagrUbd/5HhctMrv8clXFqJe90oRZq1aafyqqUucJWdlv1GdP4e64445zc9d98sknrluH5usL+/VGTcfWsC+7LrjV91ABUDcduzQg8NJLL41d6OlYFg7eUEA8//zzIy1/suOyORs0TH6RDoBxVKuim3Y49dfTFYmaQ+fOnet2Pn2ANero9ttvT6j5hDJbl2SS2bqoqVOjxdTnRe/TdCaa7kMHk1tuuSU2IiyZto1GG6uWT0FJg4S0jqmpqTZ48OAM8wwmw7ps2rTJHfxFYa9EiRKRlDXP+fJLs3ffVYewYMLlfal5UANndLGnsKduHVpWUPzoI3W61OiaKEruvazsN5qHU7ViGsSlY5ge6zh3+umnW6JQa1Bo30qCsPbuueeec60Smv5L66zjlyaU10V5Ip0zk1FKehZ6Q+rDpittbZBE6/x5KNLS0qIuAoAIKQz7IseOZ+onpZogTU1Uo0bwnOZyC0d7/vBDULMn995rduutGgpu1rZt8JxaL849N9vFYNsgN6R68jnTzBmarkYXymHXngMhLgMAgu/JPdgcifq2jdq1g+W77zY74QS10Zmpf2X37mbnnJNrRQXw5xH4AACZy58/qNFr1kxtc0ET7s8/m5UpE4Q/vQ4gYRH4AACZ07c5qHn3iy/Uo95MX2Y/dqw6k+kLts342isgoRH4AACZe/99zZERLF95pVmxYmYXXWQW9huKm3IKQOIh8AEAMrdp097lPfM9miaPD+ezVAAEkLAIfAAATfYWDMqIm+vN7roreE5zvbVpE/TXEzXtHnts0HdPEz1o9oY986cBSEwEPgCA5t8KJlbeMzGuo/55em7lymAU77RpQfjT3G8LF5ppjkRN1jxlimaTj7L0ADLBxMsAALOrrgpuB9OggdmoUblVIgA5iBo+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwWiLgCyLzU1NeoiIA9IS0uLugjYD/b/xOXTtmH/T37U8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8h+iBBx6w1q1bW6VKlaxw4cJWrVo1u/LKK23x4sWx9zz++ON26qmnWrFixSwlJcXd5s+fH2m5gShltt9s2bLF+vTpY82aNbPy5ctb0aJFrW7duta/f3/3GgC/z5vdu3e3OnXqWPHixd25s1atWnbjjTfahg0bIi27D1LS09PTM3vT5s2brVSpUta+fXsrWLCgJbu0tLRs/x/Vq1e35cuXW7169Wzbtm22ZMkS93zFihVtwYIFVrJkSTv//PPt3XfftQoVKtiyZcvc6/PmzbP69etbTkpNTc3R/w+Iar/RQb1GjRruZKD9ZOXKlbZ+/Xr3nnPOOcfefPNNywnsM0BinjfLlSvn8kbZsmVt3bp17v1y9tln26RJkyynpHpyDNixY4f7u2zatMn9/Q6GGr5D1KNHD1u6dKkLcLo6Ua2ErF692t5//323PGzYMBeWBwwYEHFpgeTYb4oUKWJDhgxxB/qvvvrKVqxYYS1btnTveeutt2zjxo0RrwGAw3ne1EWeXvv8889dRYlayWTatGmRlt0HBL5DdMcdd1jVqlVjj0877bTYsmon5Oijj7b8+fNHUj4gGfcbXenffPPNVqJECfecAmDz5s3dcr58+axAgQIRlBpAbp03tc+rC0eLFi1cjeDHH3/sng+DHw4dR88csGvXLnviiSfccs2aNa1du3ZRFwnwYr9Zu3atjR8/3i136dIlFgQB+Lv/L1y40GbOnBl7fMYZZ9iYMWMiKadPqOHLpl9//dUuuOACe/vtt13txMSJE2NXKgAOfb9ZtGiRu6pftWqVnXLKKTZixIjIygsg9/b/l156ybZv326zZs2yxo0b23vvvWfXXXddpGX2AYEvG9TvoFWrVu7DqpGE6mPQsGHDqIsFJP1+M336dNd3T1f65513nr3zzjvU7gF56LypAaJNmzZ1/f5k5MiR9t1330VQYn8Q+A7RnDlz3Anpiy++cP0QdIJStTSA7O0348aNs7Zt27rRuTfccINNmDDBjjjiiMjKDCB39v/PPvvMPvzww9hj1fKpdi++ZhCHjj58h6hTp06xqVY0P1iHDh1ir11zzTXuduutt7r+R/Hzh2loua5cNK+QbkBektl+o8eXXHKJabaoQoUKuX48J598cuw9Gvl+wgknRFJ2AId3/9egrL/97W9WpkwZN7hDo/TD+fdU29ekSZPIyu4DAt8h0hxCIU0fEU/zFcqaNWtcP6R44ZxCTCKJvCiz/UZX9OHUoFqeMWNGhvdomiMAfu7/5557rrv/+uuvbe7cuW6WiwYNGljHjh3t9ttvdyP1cegIfIdIcwll5tlnn3U3AFnfb7IwFzwAT/d/zbeJw4O4DAAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADguQJRFwDZl5aWFnURsB+pqalRFwF5APs/gKyghg8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMQjcmTzfLlM0tJMbv33r3P79pl1rJl8Hy1amabN0dZyryJbQN4h8AHIBpt2pj17h0s33232ezZwbICxowZQah47jmzkiUjLWaexLYBvEPgAxCdwYPNGjQw277drFs3s5kzzQYODF7r08esdeuoS5h3sW0ArxD4AESnSBGzkSPNChQIapFatTLbscOsYUOze+6JunR5G9sG8AqBD0C0mjUzu/POYHnrVrP8+YOgocCBaLFtAG8Q+ABEb+HCjAMDli6NsjSIx7YBvEDgAxCt8ePNRo8OljXyU3r2NFuzJtJigW0D+ITAByA6Cg69egXLHTqYTZ9uVq6c2fr1Zj16RF26vI1tA3iFwHeIHnjgAWvdurVVqlTJChcubNWqVbMrr7zSFi9e/If3btmyxWrVqmUpKSnuNmLECEvG9dHr4TrE30499VRLxm0ze/Zsu+iii6xChQpWqFAhO+aYY+ySSy6JrNx50jXXBAGibFmzp54yq1TJbPjw4LWJE82efjrqEuZdSbxtsnIMWLp0qV111VXutSJFili9evXs3//+t+3evdsSzdChQ61JkyZWunRptz6VK1e2iy++2B3D4s8zffv2da/peKZzzsCBA23nzp2WbOsyaNAgO/HEE93r4Xlmq/qQIltS0tPT0zN70+bNm61UqVLWvn17K1iwoCW7tLS0bP8f1atXt+XLl7uDxLZt22zJkiXu+YoVK9qCBQusZNz8VDrQPP/887HHw4cPt17hlXOCyMr66AA6ZcoUq1mzpgtJIe28jz/+uCXTunz88cd21lln2e+//+4ea51++eUX93P6mZyQmppqPsmJ/SYDhYiwpuill8w6d9772uWXB02JJUoEI0SrV8/RX+3Ttsnx7RLxtsmNY4Cea9Soka1bt86KFy/uwtG3335ru3btst69e7tQkkguuOACmzFjhiu/go/WQcG0bNmybj2LFi1qbdu2dcdnnaN1PFu4cKF7T7du3TKcfxJ9XYoVK2ZNmzZ1gVzbZuXKle7ndKxWMM8pqZ4cA3bs2GGTJk2yTZs2Zcgd+0MN3yHq0aOH+0DOmzfPXTX20bxUZrZ69Wp7//33Y+8bM2aM29kSveYoq+sj/fv3t08//TR2S6Swl5V10TWO3qMDSNeuXd3zs2bNcgfI9arRQO7VIOl6U7f4QCGjRgXP65scEjBQeC/Jt01mx4CxY8e6sCc6hn311VfuQlwee+wxW7FihSWSF1980VatWmVffvmlzZ07126//Xb3/IYNG2z+/Pk2YcIEF/bklVdecc+FoXXkyJHu55JlXeT111+3jRs32jX6HCLHEPgO0R133GFVq1aNPT7ttNNiy6qGFh00evbsac2aNXNV1Mm+PiE1G+g5XUVee+21tibBOnBnti5qOggPLAp/qgVQDbaukL/77rtIygwg944B8c22+fQVcnH3quWbrK+WSyCq2Xr11VetZcuW1rBhQ7tnzzyIammpW7euvfXWW+6xavo6qL+lmV144YWxn1cNULKsi6iZV824yFkEvhygA8QTTzzhlhWC2rVrF6tKV3XrCy+8kFRN4ftbn5AOKOrrpp1TzSRPPvmknXTSSfbrr79asqyLmhBC2jZHHHGEW9ZBXs3WqhkA4If9HQMUitRcKC1atHBNiPHdbMJmxESiC2s1harWUueXGjVquGNWiRIlYjWS5cqViwXXo446KvazaipNlnXB4UPgyyYFHfVJePvtt12fhIkTJ7oryIcffthVses+vGpJ5vWRhx56yFWzq6+LDjC33Xabe17BT1dsybIu8Z2Yr776alfbpyad/Pnzu358zz77bKTlBnB4jwEKfu+88461adPGBSQ1MWoAR1irlIgX6AqkCkfLli2zzp07u+Ou7jVYY3+y0D0/adYFOYPAlw3qD9KqVSt3EFGomzZtmquilq+//trdqwOwriTVQTik/iQnn3yyJdP6yPHHHx8LfzowXnbZZQl7BXmwdVENZah58+buXleY4UAUaviA5JfZ8UwtEx988IH9/PPPtnbtWuvevXssJKmbRyLScVdN1WG/tzlz5rg+cVWqVHGP1Qc5bK7WOoXim7cTfV1w+BD4DpE+nOqD8MUXX7j+IdOnT3dXjfu7wtTtt99+iz2nEWLxj5NhfXTwePDBBzNcgb388ssZRsUly7pouH84munzzz9397rSDDtx16lTJ6KSA8it47NG6qu5V9RycfPNN7vl8uXLZ+jGErWffvrJDbzYvn177Lk333wztqzzi2bQEI16DV8br0mz9whfT4Z1weHDtCyHSFeAYQd/9f+IH9igkUX7ji5SrZFqkRJ1WpbM1ueMM85w5S9QoIDVrl3b7Zhhv5EGDRq4EVc5OWT+cG8bNU//4x//cM/Vr1/ffvzxRzesXc0+GtQRP+1MXh/2n6PTfzzwQDCHm/pRbtigeTI0waPZ3Xerg5WZmtP/9rcD/7w60+v92eTTtsmV7SKLFgWPNRpUtUfqb6WWC+1Hf/2rJZKsHAMaN27s+uqpdmzRokXuIlzdOnQhGz/gIWrhuUP9pzV9jI5T4bFXfd6++eYbN8hB/Y8VYnWO1vu0/qrtU0vM6PDbUpJgXTQvomZPUB8/jdxVGBcFdtUKaq7ETp06ZbssqXlwWpYCuVYqz8TP1ab+X/ES5WoqJ9dHAUgj39TvRQdHTWmioHT++efbLbfckjBhL6vbRiONtXNo6gJNx6L10wFg8ODBORL2cACPPqr2f52RNQJIHUDNNEfYO+8EYUN/+xYtMv6M3v/jj8Gygghyf7so3J15ZvC8wpOCnpanTjX76COzWbM0Iacl0zFA83CqFkyDuHT80mMd404//XRLJJqguEuXLjZz5kx37NUJXiFVzdVqDlVAkjfeeMNNmTVu3Dj3PjWXXnHFFXbnnXdasq2LgrhejxdOmq0KKBwaaviAw8SXK8gc3W/+7//MunVTp6Lgcd++mno/WH7lFc3K+sefOe44s2++CQKHAkgO8Gnb5Mp2UV/XPf3E7N57zW69Nahtbds2eE61g+eem/1yALkk1ZNjABMvA0hMd9yxN1RI3PxoruZoX5o/TGFP+vXLhQLmUZltF32tWu3awWM1655wgpma1QoUMOve3eycc3K/zAD+FAIfgGiow/ye+dFcP7H9dZQfMiS4V3OhavgQzXbJnz+o0WvWTO2lQRPuzz+blSkThD+9DiChEfgA5D6NxlPz7dtvB/3y1CS4bw2fQsUHHwTLe0ZQIqLtoqk+NNDsiy8015TZL7+YjR1rppHt119vNmFC1CUHkAkCH4DctXq1WatWQZjQpOTTppnFzY8Wc//9wb36jnXpkuvFzHMOtl30fdpvvBEsX3mlWbFiZhddZBb2GXrvvejKDSBLCHwAcs+cOWYtWwY1ReonNn363mk/4mnE6JgxwbJqlNRXDNFtl02b9i7vmbvSNO1JOC+nAiCAhMZRFEDuUUf/ZcuCZYWFPV/07mjuynD+So0Q1VfglSpldu210ZQ1L8lsu6iZV/31NCeamnYfeSSYlkWTPGjmhksvjazoALKGwAcg98TNj2b7zI9m4fyVqk166qlgWWGPL1SPfruUKxc08Wr6Fs27t3BhEADVBKx53po2zfUiA/hzCHwAck9WvqdYtXpMrpp426VBA7NRo3KjNAAOA/rwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAnitgeVBqamrURcABpKWlRV0EHAD7TWJiuyA3cGxOftTwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAg902ebJYvn1lKitm99+59ftcus5Ytg+erVTPbvDnKUnqDwAcAAHJfmzZmvXsHy3ffbTZ7drCs8DdjRhD4nnvOrGTJSIvpCwIfAACIxuDBZg0amG3fbtatm9nMmWYDBwav9elj1rp11CX0BoEPAABEo0gRs5EjzQoUCGr4WrUy27HDrGFDs3vuibp0XiHwAQCA6DRrZnbnncHy1q1m+fMHIVBhEDmGwAcAAKK1cGHGQRtLl0ZZGi8R+AAAQHTGjzcbPTpY1qhc6dnTbM2aSIvlGwIfAACIhkJdr17BcocOZtOnm5UrZ7Z+vVmPHlGXzisEPnhp6NCh1qRJEytdurQVLlzYKleubBdffLHN3jPs/4cffrBevXrZsccea2XKlLHixYtb48aN7f7777cd6jAMADj8rrkmCHdly5o99ZRZpUpmw4cHr02caPb001GX0BsEPnhpypQptm7dOqtZs6bVqlXLfvzxRxs3bpy1adPGfv31V/v+++/t8ccft6VLl1r16tUtf/78NmfOHOvXr5/1DueFAgAcPgp4r78eLA8bFoQ9ufhis65dg+W+fenPl0MIfPDSiy++aKtWrbIvv/zS5s6da7fffrt7fsOGDTZ//nwrW7asPfnkk7Z+/XqbNWuWC341atRw7xkd9iUBABze2r309ODWuXPG10aNCp7Xt2xUrx5VCb1SIOoCAIdDkSJF7NVXX7X77rvPNm/ebAsWLHDPV6hQwerWrWslSpSw4447LvZ+NeuqSXfJkiWuCRgAAJ8Q+OCtNWvW2Ax9Pc8eqsGbOHGiC3v7UiD84IMP3HIPOgoDADxDky68pUEZu3fvtmXLllnnzp1d7Z3ut2zZkuF9n332mbVq1cr17evUqZMNDL/WBwAATxD44LWUlBSrWrVqrA+fBmaof1/otddes9atW7vawGuvvdbGjBljBfQVPwAAeITAB+/89NNPNnLkSNuuL+Pe480334wtqyZPHn74YVej9/vvv7u+fhq1q9G6AAD4hsAH76jJ9oorrnBz8GmePdXw3Xbbbe419d9TyJs+fbr16dPHNflqDr5XXnnFWrZsGbtpGhcAQA4bOtSsSROz0qXNNECucuVgGpY9c6Ta77+bdeoUjMwtWtSsZEmzBg3M7rgj+J5dHDLaruAdBb0uXbrYzJkzbdGiRW4i5SpVqrh+emrarVatmuvPFx8Q4wd3yLZt2yIoOQB4bsoUs3XrzGrWDAKcZlAYN85Mg+aWLzfTxPeam09fsdaokdnKlWbz55vdc4+ab8xGjIh6DZIWgQ9eBr74fnr7o3576ZrjCQCQe3RsLlJk7+P+/c0GDdIkqUGwO+EEs19+MStUKHh9506zunXNdJE+bVpkxfYBgQ8AAOQOhb1XXzW7775gUuU9c6RahQpBsEtJCcKeJmVWM+8PP5iFXWxOPTXSoic7Ah8AAMg9a9aYxXej0bcc6Xtz4+dI/fZbzZm197G+au2RR3K3nJ5h0AYAAMg9vXqZ7d5ttmxZ8JVqaq7VffwcqZ9+GvTx++gjs6OP1ndemv3v/0ZZ6qRH4AMAALlLTbdVq5rtmSPV5swJ+vfF0yheNeOG37OrgRu//Zb7ZfUEgQ8AABx+GmU7cqRZ3BypFjdHqmmO1PffN/vyy73PaQDH1KnB8q5dTM2SDQQ+AABw+KnJ9oorgjn4jj02qOHbM0eq67+n+ffUhNusmdmRR5o1bRo0537xRfCe884zK1s20lVIZgQ+AABw+CnodeliVqmS2aJFwejbKlXMLr88GMShufdattS8WUGTr5p51ddPEzX/619mY8ZEvQZJjVG6AAAgdwJfJnOkWvv2wQ05jho+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwUsD0pLS4u6CMgD+JwB8EVqamrURUA2UcMHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHwAAACeI/ABQE6YPNksXz6zlBSze+/d+/yuXWYtWwbPV6tmtnlzlKUEkEcR+AAgJ7RpY9a7d7B8991ms2cHywp/M2YEge+558xKloy0mADyJgIfAOSUwYPNGjQw277drFs3s5kzzQYODF7r08esdeuoSwggjyLwAUBOKVLEbORIswIFghq+Vq3Mduwwa9jQ7J57oi4dgDyMwAcAOalZM7M77wyWt241y58/CIEKgwAQEQIfAOS0hQszDtpYujTK0gAAgQ8ActT48WajRwfLGpUrPXuarVkTabEA5G0EPgDIKQp1vXoFyx06mE2fblaunNn69WY9ekRdOgB5GIEvG4YOHWpNmjSx0qVLW+HCha1y5cp28cUX2+xwOoY4P/zwg5UtW9ZSUlLcbdKkSZaM61O9evXYOsTfLr/8ckvGbTN16lRr3769lSlTxooUKeLWr3c4tUaSrMuAAQP2u03C29IEa07Myrb58ssv7fzzz7ejjz7aveeoo46yc845xz766CNLaNdcE4S7smXNnnrKrFIls+HDg9cmTjR7+mlLBpdcckns89OlS5fY8zt27LCBAwdazZo1rVChQm7b9e3b13755RdLtnV5/PHH7dRTT7VixYrFXp8/f74luv2tz5YtW6xPnz7WrFkzK1++vBUtWtTq1q1r/fv3d68BQuDLhilTpti6devcwa9WrVr2448/2rhx46xNmzb266+/xt63e/duu+KKK2zjxo3mw/pIgwYNrEWLFrFb7dq1LdnWZcyYMda2bVt7++23LX/+/NawYUN3EH3zzTctmdZFJ934baGbLi5EYUlhNpnW5+eff7Z27drZa6+95k5WjRo1coFCF0lnnnmm+9mEpID3+uvB8rBhQdiTiy8269o1WO7bN+H78z3zzDM2duzY/b7WvXt3d4GxbNkyt/3Wrl3rAvy5557rjnPJtC5vvfWWzZo1yypUqGDJ4kDr89NPP9nDDz9sc+bMcceD4sWL28KFC23QoEHWuXPnSMqKxEPgy4YXX3zRVq1a5Woj5s6da7fffrt7fsOGDRmuFIcMGWKTJ092V2Y+rI8MGzbMPv3009hNJ4FkWhcFi7///e+2a9cuu+WWW2z16tXuvUuWLHH3ybQu11xzTYZtoc+aAqzoQqNUqVKWTOvz7bffutAnTz31lHvfY4895h5v27bN1iRqXzjV7qWnB7d9T7KjRgXP61s2qle3RLVo0SK78cYb7aSTTnLBIZ62wyith5kLF9pW49VfcU+InzBhgiXLuoTHsM2bNyfcsetQ1ketEzrP6GLoq6++shUrVlhLfbvLnmCb6JUNyB0EvmzQTvbqq6+6HUu1Q/fsmWdLV4yqTg8PkqpWP++881zASPb1CV144YXu/XpegUkHzmRal/fee88FDFGA0AG0XLlylpqamnCB4s9sF3nuuefcgV+1lTfddJMlmszWRzV6Ya2kwqyaqa6//nrXTKVw2Lhx44jXwE87d+60rl27Wr58+Wz06NGxi4aQgkP8/i8dO3Z021MSqZtKZusi6i6wv+cTUWbrU7FiRbv55putRIkS7rG2SfPmzd2yfqaA5oVEnkfgyyaFgxkzZti8efNck0aNGjVcDYt2vN9++80uu+wy16fiv//9ryX7+oS0fMwxx7iaIzUb6Mry7LPPTrgmnYOty4IFC2Lve/755902+v33323ixInWunVr27RpkyXbdhG99uCDD7plXWTUq1fPEtHB1kdhT3311GSoplxdNGlfOvLII61p06ZRF91b6punbaKaL22PfanWKKRtEYYJ7TuyfPlyS5Z1STZ/dn3U1B7Wvqqf377HCeRNBL5s6tWrlzthqU+L+kqoSVD36nt022232XfffedqXMKDYjKvj6ivlZoH1MF+5cqV1k1fH2XmmhI/+eQTS5Z10RVz6F//+pdrRlRfPtF6qQYqmbZLSP3eFMKlX79+lqgOtj5qbr/qqqts8eLFdv/997vQ98ADD8Teq35XyFmff/65DR482A2+Uk3Sn5GupmpP1iUR/dn1UdOvBqOo28Qpp5xiI0aMyJVyIvER+HKAms6qVq0a64ukjrPqp/T111+7xxdccIHrRKtRhiE9d+mll1oyrY/85S9/iTUnqJkgvl9iIl3hZ7YuqqEMhU0fJ554Yuy5RBvZmtl2CSkgiZpLddBPZAdanxdeeMGd5MJBAhpF+be//S0WLt5///1Iy+0jXfCoP6su6HSs0i3cn1VTpMeVwkEoe2qQRKFdAwZE2zJZ1iXRavBzan2mT5/u9n1d9KmG/5133qF2DzEEvkOkg9zIkSNtu74kfY/40Z3hSFCdoLSs21Z9zdIeWlYTYjKtj07ITz/9tOs4L+FBKKQpTZJlXTQ6V81REoaL8F7q1KljyfQ5E9WwhrWs6s+TiLKyPvEn4/1tGwVAHB46LoXHq7DmTrXheqyRuKGwufCNN96IHdc0vVGyrEui1UrmxProWKzj2vr16+2GG25wg2iOOOKIqIuNBJKSnoVPvjrkq7+WduiCBQtasktLS8v2/6EaIPWlUEdyTS2hk1TYx0VXVN98841VC2fZ3+PDDz90U0+EHaAT6QCZlfVRs5vKr6k+NA2LDizhAAcdaDQQQrU2UcvqttF8e4888ogrswYKqAlRfcU0kED9xrSeUfszn7NOnTq5pmhtG/VRDANtIsnK+uhCSPP0KRRqrjf1Q1TXCF1o6Dikfn/xtU2RGjpUc2WYLVtmpgs4TfFx0klm/fubHXdc8J5vvtFkier3oMRrpoE2mp5lT61lotIFXNiM/tJLL7nn1CdZtbD6bGmAjZoPNTffaaed5o5vifiZO9C63HrrrS64qhtBWGOpWkqd4zQaVrdEte/6qPlWA890Otc+c/zxx2d4v/r+nXDCCdn6nRrQhsSj/U8DpnQsLVmy5EHfm5h7ZxLQpLHqDKsTjw56mkusSpUqrp+FOtfuG/Z8WB/NvfePf/zDnYA1kbSuLI899ljXv+T1119PiLD3Z7bNQw89ZPfee68LHgoUmtxXo0E//vjjhAh7f2Zdvv/+e9d/TzQRbqKeeLOyPvXr13fTfPz1r391fV8VXjWCVyc31WAmTNiTKVPMNC9gzZpmtWqZ/fijOrqa6cJOta9z56p93eyVV1RFo6rjIAB27x6ExSSj/sh33XWXC0baftouCkaq6UvUz9yB6GJV6xCGPVFTqZ4LR/AnC10chXU3Wta+FH9LtFkUEA1q+ADgUCnE7ZmWxFHN3qBBwbKaoV9+WRNxagZss1Wrgm/guOMOM01Fo/kRFRCLFo2s+EBWUcOXmKjhA4DcoLCnEd2qxWvYMAhyoqZdNd3GT1UU1oCHNWHqq/jZZxEUGkBexGyMAJAd6sc6Y8bex5onTd+bq9GRnToFTbca6KTm3KOP1rDLve9duTKSIgPIe6jhA4Ds6NUrqMnTwA19pdqSJcG95kg8+WRNjmjWokUQ+jRo44or9v6sB11kACQHAh8AZJeaazUP3Z45BW3OHH1pcLDcsWMwQlcBUDV6Z5+99+cS9JtQAPiHwAcAh0K1dSNHaljk3ufi5hR0o3TDkbwhTUGjKVqkUSMzvhcYQC6hDx8AHArV2Kl5tmfPYEoWDcIIv2827L8X1vBpAtyjjjLT196paVePn3xy70AOADjMqOEDgENRurS+md5M8wIuWhRMsVKlitnllweDOMK5OM87T99DaLZggb4mJAiC06cHEzQDQC6hhg8ADjXw7fNdxvuVlfcAwGFGDR8AAIDnCHwAAACeI/ABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCkRdAGRfampq1EVAHpCWlma+8Gmf8Wm7CNsmMfm0Lr59zrKKGj4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBL4dccskllpKS4m5dunSJPd+9e3erU6eOFS9e3IoVK2a1atWyG2+80TZs2BBpeYGoHWifCW3ZssXtL+F7RowYEUk586IDbZvWrVvHno+/nXrqqZGWNy852H4ze/Zsu+iii6xChQpWqFAhO+aYY9z7k2ldnn322f1+xsLbhx9+GHWxk1aBqAvgg2eeecbGjh2739dee+01K1WqlNWvX9/WrVtnixcvtkcffdS+++47mzRpUq6XFUj0fSZ0/fXXu/0Fibdtatas6UJFqFGjRrlQMhxs23z88cd21lln2e+//24lS5Z02+SXX35x56BkWhd9rlq0aJHhueXLl9uPP/7olitWrJhrZfQNgS+bFi1a5GrsTjrpJFuxYoX98MMPGV5fuXKlFSlSJPb4tNNOczvmtGnTIigtkPj7jIwZM8aef/55VwOgZSTOtpH+/fvbVVddlevly8sOtm3S09OtR48eLux17drVnnzySStatGispjyZ1qVjx47uFu+4445zge/MM890lSc4NDTpZsPOnTvdzpUvXz4bPXq05c+f/w/vUdjTwVFXLNWrV3dhT2gCQV6UlX1GJ4CePXtas2bNbNCgQZGUMy/KyrYJ9e3b1woXLuxq+q699lpbs2ZNrpY1r8ls26gpd/78+bHwV69ePdey1LZtW9ealKyfM1FL2DfffOOW+/Xrl0ul9BOBLxsGDhxoM2bMsGHDhlmNGjUO+L6FCxfazJkzbdmyZe7xGWecQa0F8qTM9pndu3dbt27dbMeOHfbCCy9YwYIFIylnXpTV45lqjtQ3TE1vS5YscbVJqqn59ddfc7W8eUlm22bBggWxZe03RxxxhFuePHmy63e5dOlSS7bPWWjIkCHuvkmTJq6GD4eOwHeIPv/8cxs8eLBdfvnl7mrlYF566SXbvn27zZo1yxo3bmzvvfeeXXfddblWViBZ9pmHH37YpkyZ4u7r1q2b62XMq7J6PHvooYds48aN9u2337qa2Ntuu809r+D36quv5mKJ846sbBvVmoWuvvpqV9v31Vdfudoz9ePTQIhkO2+KzpkffPCBW7755ptzoYR+I/AdIh3wdu3aZePGjXMjcHVTx1IZP368e7xp06bY+1VT0bRpU9fPQkaOHJlwVe1A1PvM1KlT3ePevXu7x/GDAfr06WMnn3xyZOX3WVaPZ8cff7xryhWNmLzsssti/0f4fuT+tlGNa6h58+buXrVn4cCaRKnh+7Pnzfvvv9/dV6lSZb8j+fHnEPiyaevWra4pQzf1nQivtvRYgS5+CLlq+VS7F6IJBHnRwfaZ8HH4+m+//Rb7uW3btmV4jNzdNvr7P/jggxkGAbz88suxZfVRRjTbRrXhGpkb1qKJuhBpZgjR1GDJdgxQEAy7PukCsEABxphmF4HvEGmEmj6Y8bdq1aq51zp37uwez5kzx9q0aWNly5Z1tXuVKlWyiRMnuvfosfokAHlFVvaZCRMmZHhdTYWh4cOHu2YqRLNtFLZvuukmdzxr0KCBVa1aNTaoRo87deoU8Vrk3W2jc8uAAQPcc0899ZTbHjq/qDZN05hoYE2yrEvp0qXd46FDh7oQqMEniVL+ZEfgO4zUX699+/ZupO7cuXPdAVM7ovoiqF+CRikBQDJQ8+Add9zhmnXXrl1r69evd1Nk/POf/3TTTMVPP4Xcp5HTCns67+hCqUSJEm4AlGr84udMTAZq1tW6iMKe1gXZRx1pDtq3n8Rf/vIXe+uttyIrD5DoMutbpGbCsIkH0W4bfVOQavSYKidx9xsN2NAt2ddFtXqbN2+OpDw+o4oJAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPBcgagLgOxLS0uLugjYj9TU1KiLgANgn0lcPm0bn44BPm2XvIoaPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwDgr8mTzfLlM0tJMbv33r3P79pl1rJl8Hy1amabN0dZSuCwI/ABAPzVpo1Z797B8t13m82eHSwr/M2YEQS+554zK1ky0mIChxuBDwDgt8GDzRo0MNu+3axbN7OZM80GDgxe69PHrHXrqEsIHHYEPgCA34oUMRs50qxAgaCGr1Ursx07zBo2NLvnnqhLB+QKAh8AwH/NmpndeWewvHWrWf78QQhUGATyAAIfACBvWLgw46CNpUujLA2Qqwh8AAD/jR9vNnp0sKxRudKzp9maNZEWC8gtBD4AgN8U6nr1CpY7dDCbPt2sXDmz9evNevSIunRAriDw5ZBLLrnEUlJS3K1Lly6x5xctWmSXX365ValSxQoXLmzly5e3Vq1a2WuvvWbJti5Lly61q666yqpVq2ZFihSxevXq2b///W/bvXu3JZIBAwbEyr/vbefOne49O3bssIEDB1rNmjWtUKFCVrlyZevbt6/98ssvURffa1nZNo8//rideuqpVqxYsdhr8+fPt2Rbly1btlifPn2sWbNmbr8vWrSo1a1b1/r37+9eS7bt0r17d6tTp44VL17cbZtatWrZjTfeaBs2bLCEd801QbgrW9bsqafMKlUyGz48eG3iRLOnn466hHlCVj5nrVu33u/rOiYgewpEXQAfPPPMMzZ27Ng/PJ+enm5nnnmmLVmyxIW9Ro0aueWpU6faRx99ZLNmzbImTZpYMqzLunXr7MQTT3T3OuDXr1/fvv32W7v11ltt1apVNnToUEs0OsnqpBRPB47w5DVq1CjLly+fO4ktXrzYrYO2yQcffOCeRzTb5q233nLboUKFCrZs2TJL1nX56aef7OGHH3b7vvaXlStX2sKFC23QoEH2xRdf2JtvvmnJtF10kVqqVCm3LjoOaJ959NFH7bvvvrNJkyZZwlLAe/31YHnYsCDsycUXm3XtGjTz9u1r1q6dWfXqkRY1rzjY5yyki3EdA0I6fyJ7CHzZpBo8XeWedNJJtmLFCvvhhx9ir+kAr4Anqk1SOJo8ebK1bdvWhUG9P5EC38HWRSFQB3n59NNP3c735JNP2rXXXmuPPfaY3XTTTa4WM5F07NjRnn322T88/+WXX7qwJzohX3/99TZx4kRLTU21KVOm2IQJE6xTp04RlDjvONC2kWHDhtlRRx1lI0eOtL/97W+WrOuiWvAhQ4ZYz549rUSJErZ161Zr06aN238Uajdu3GhlypSxZNkuOp5pnUKnnXaaffzxxzZt2jRL+No93fZHx4E9xwIkxucspJpwtSgh51CNkQ2qgu7ataurDRo9erTl1zD/OJUqVbLatWu75bvvvttOOOEEFyQKFCjgapjOOeccS5Z1iW+2DWu/wvtdu3a5IJtoxo8f75rRtB3OPfdcV2skOtmGLrzwwtgBKDyZJXRthScOtG3k6KOP/sPnLxnXpWLFinbzzTe7sCf6fDVv3jy27+g4kEzbReXXSbhFixZWvXp1F/aEpjbk5OcspC42qh1XTZ8qFtYwuCbbCHzZoFq7GTNmuBqJGjVq/OF1nbQUhNSHZ9u2be5D/fPPP7ureoW/RDqpZbYuHTp0cE25ogN+06ZNrVfYCXrP1X8i0d9WJ1ydmFavXm1vvPGGq7nUNlDtZejII4+MnYDVzCDLly+PrNx5wcG2jc/rsnbtWneiE/WNDYNgMq2LmqRnzpwZa2o/44wzbMyYMRGWGskmK58zhcFjjjnGNemqlUytSXrPr7/+GmnZkx2B7xB9/vnnNnjwYDcgQzVj+6NaMYUi9dfp3bu3GxAQNo2qGVFNh8myLrrKeuedd1yTlMKR+u2puj3sd1GwYEFLFJdddpk7uerkNG/evFiNnUL3f/7znwP+nJrZkZjbJtnXRd0lVBOm/eaUU06xESNGWDKuy0svvWTbt293J+fGjRvbe++9Z9ddd12EJUcyycrn7KGHHnLdHdRHXBfnt912m3tewe/VV1+NtPzJjsB3iPRhVFPmuHHjXM2XbmHNkK7i9Vj9wnT1IldeeaUb2XbRRRdZyT1f0q2DZbKsy6ZNm9wVlgY0qJZSO62apcOQpBG7iUIjIctqNN4eZ599tpXTFAx7au/i+xpqPcJwrk72UrVq1Vwvc16R2bbxcV2mT59uLVu2dCe58847z104JVrt3p/ZLrq4Uw1/jz3TmaivpQZuADnxOTv++ONdU66oQkEhMZRsx4hEQ+DLJnXEVjWzbmH4UX84PdbUH/G1aKIDYzglgwJgsqyLHqvPjoKh6ApM/ZNETaHtNMItQdx3330ZDgzvvvtuLMypGaF9+/ax18ImNgVzrb/Ev47c3Ta+rYsuojRIa/369XbDDTe4Wv0jjjjCkm1dPvvsM/vwww9jr6uWL/6ClaY25MTnTBfgDz74YIZpi15++eXYcrIdIxJNSnoW2rE2b97shuPrRJhITXeHKi0t7bD8v/owqm9L586dXdOHPsia8kPhSM2gDRs2dNXSOjjq76i+MLpSTkT7rouoCUd99VRDpiaq3377zfXH0A4ZDn5IlLKHNXkK1ZrDTR9zLetvru2gq8YXX3zRbRdddWp9FNA18lAntpyYlkWjfn2SE/tNVraNRrMriOugH9bAqtZV+4xGkeuWCDJbl9KlS7v5HfWc5npUzUU89ZdVX95kWBfdNGJa/Y+1LdTUFs6/p2OYuq0k1FRGAwaoY/L+X9OFePyAGYULHYcXLw4ea36+uP7J2eHTMSA39n9dDKkPuQY0acCjzpVhn+sGDRq4GRbiR4pnR6on20bnLTWNqxUubD08kATaQ/2jqmpNWaB+cTrwq0lHTTkaAKHpPxI17B3IWWed5T5QCxYscDukHquJN5HCntx+++2uxlE7guYK00TR2gY6KSlQyHPPPWd33XWXO3kp7KlzsIKEavoS6sTlmaxsG43G0zYJw57oJKHnEmmS38zWRbVg4fW0ljUoKv6mC+lkWRdd7OmCXyfbuXPnuos9nYBVy5/Q81ZqIFaLFhlv+8z3ZtdfvzfsIdLPmY7Dd9xxh7s40v6vmnHN+/jPf/7TnUtzKuzlVdTwAYeJL1eQIfYbJI2whu/KK80ONt+bRhh37qyvFwqWhRq+PLH/p3qybajhAwBAfXSLFg2+XePcc83ip8tRU2HPnmbNmpkNGhRlKYFcQeADAPhH85xWrBh8Xdrq1RqZZXbSSUHo00Ty3boF/fleeEFDj6MuLXDYEfgAAH7RVB7qA7pwodm8efr6nOD5bdvMNN/bww+bTZkS3NetG3VpgVxB4AMA+EUhLm6+Nzv7bI2iC5Y1LcjXXwfLvXub6RuEGjXa+94+fcxOPjmXCwwcfgQ+AIBf7rsvCHahd9812zPfm2viDWn+QN1++23vc6oFjH8MeILABwDwi0baKthVq2am6X5Uwyea7F41eBq5qwkqwtuSJRl/9quvIis6cLgQ+AAAfrn9djN9+48GZWiOPQU/fU/4F18EARDIg+KmGwcAwAPXXhvcskq1gZlPSQskNWr4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxWIugDIvtTUVPNFWlqa+cKndRE+Z4nJp+3iG58+Z0h+1PABAAB4jsAHAADgOQIfAACA5wh8AAAAniPwAQAAeI7ABwAA4DkCHwAAgOcIfAAAAJ4j8AEAAHiOwAcAAOA5Ah8AAIDnCHw5ZfJks3z5zFJSzO69d+/zu3aZtWwZPF+tmtnmzVGWEsmOz1niYtsgF23davbQQ2Ynn2xWurRZ4cJmVauanXGG2YMPRl06JCICX05p08asd+9g+e67zWbPDpZ14J8xIzjYP/ecWcmSkRYTSY7PWeJi2yCX/PRTcA3xj3+YTZ9utmOHWd26wfXGlClmN90UdQmRiAh8OWnwYLMGDcy2bzfr1s1s5kyzgQOD1/r0MWvdOuoSwgd8zhIX2wa54Prrzb7+OljWNYYC4DffmC1darZ+vdkzz0RdQiQiAl9OKlLEbORIswIFgqv7Vq2CS6+GDc3uuSfq0sEXfM4SF9sGh9nPP5uNHRssN2kSNN/qYxcqVcrsqqsiKx4SGIEvpzVrZnbnnXs7WeTPH5wA4vdIILv4nCUutg0Oo+++C7qFymmnBc24cv75Qa+B8Pbss5EWEwmIwHc4LFy4d1l7purZgZzG5yxxsW2QC8KwJ/XqBTV+wIEQ+HLa+PFmo0cHyxqRJz17mq1ZE2mx4Bk+Z4mLbYPDSMFOlcbyySd7n7/vPrOXXoqsWEgCBL6cpAN6r17BcocOwfCpcuWCXrQ9ekRdOviCz1niYtvgMFMfvUsuCZY//zwYEB428QIHQ+DLSddcExzYy5Y1e+ops0qVzIYPD16bONHs6aejLmGeMGDAAEtJSdnvbefOnfbss88e8HXdPvzwQ0tofM4SF9sGueDRR82OOy5Y/te/go/b8ccnzyDwdevW2Q033GDVqlWzQoUKWfny5a1du3a2ePFi9/qgQYPsxBNPtMKFC8eOy1vVHxbZUiB7P44YHdxffz1YHjYsONDLxRebde0aNPH07WvWrp1Z9eqRFjWv0EGkVq1aGZ7TgaNChQrWokWLDM8vX77cfvzxR7dcsWJFS1h8zhIX2wa5RJXGn34aBL8xY8wWLDCbP1/HLrOzzza74IJgEEciWr9+vTv+LlmyxIW9unXrWnp6uk2fPt1WrVplNWvWtHHjxtnSpUvdsXrlypVRF9kbBL6cvLLXbX9GjQpuyFUdO3Z0tXn7e163eMcdd5wLfGeeeabVr1/fEhafs8TFtkEuKlrU7JZbglsyufPOO13Ya9Sokb377rtWac+F0fbt213wk9dff92OOeYYGzhwoLshZ9CkC2+NHz/eihYt6g4o5557rs2aNWu/75s0aZJ9o1lLzaxfv365XEoAyBsU6MaoStLMqlSp4i6wixUrZk2aNHHHazXhSuXKlV1rDHIWgQ9eyp8/v2uarV69uq1evdreeOMNO+mkk/Yb+oYMGeLuddDRAQgAcHj67m3cuDF2of3zzz9bmTJlbPbs2XbZZZe5plwcPgQ+eEcHjrVr19rChQtt3rx57sAi27Zts//85z8Z3qsA+MEHH7jlm2++OZLyAkBeoEFzoQYNGrhBGrppWR577LEIS+c/Ah+8o07AZTVsbY+zzz7byqmX857BGfHuv//+WPNCly5dcrmkAJB3aBCGBmqELSpa1k3LooEaOHwIfPDOfffdlyHYqWPwT/p2cdPgyL2jI/WesD9J7969rYC+/xQAcFgULFjQTj/9dLesZtwdO3a4m5alTp06EZfQbwS+QzVgQMYvLoy/xVVbuy9Qv+giXdqY6crmmGP2zpqJw2L48OEu2GmOp4YNG7oaPlHn4D59+sTeN3ToUNfEUKpUKbv22mstaT9nixaZXX65qinN1Om5fHmzVq3MXnst6tL7LSvbRjUW+iZ7feOGvktXX5Pw73+b7d4ddemRZNatM7vhhuCjpFOJdnPN8LNn6jrbsiWY9ady5eB1zUilAa7xp6NEoDn2VKs3d+5cq1GjhrtpWf2ub7/9dveerl27Wu3ate2RRx6J/ZxG9eq5V155JcLSJzeqNLJLe90+c725A758/LHZWWeZ/f67WcmS+sSa/fILJ+LDTAeNsWPH2pw5c1z/EAW/U045xfr372/1dMI1s02bNtlTmjfNzIW9EiVKWFJ+zjSNgQaaLFkShD19xrQ8darZRx+pkyJfsBnVttEZ+sQTg/vixc003c+335rdeqvZqlW64oiqxEgymstbU4dq11aYq1s32PX1RS76KKnh4rzzzKZMUS2aWc2awdc565pE14PPP28JQ3Pwqd+0pmeZOXOmm0nhjDPOcEEwnB9Vc+8tUsHjhJMyb968OZJy+4DAl12az20/c725vVFfpaSwp0lXn3wymDgpvBTDYaMAl1mNnWr1kurAcaDP2Q8/BGcB0eW8wsTkyWZt2wafwRUrCHxRbZuxY4OwJ5olV2FcxwF9NtU5/aabglpZIBN33hns5voIvfvu3jm9t28PdvMJE4KwJ6oAO/fcYFLmG280GznSTA0bJ5xgCUMX4JN1nDqAhP+2oyRFk25OfFG6gpz2QO1l4bQfasrV1OeiPVI1S/oSRJ2Iv/su0iLDo8+ZHteuHSzrSzV1VO/UyUz9Ebt3NzvnnEiLnae3TXyzbb58Ge/15acHOeEBIZ0+9nQ1dtcHqtAvViy4jtNHTxX7b70VvK6Pob7CWS68cO//sWeiAuRxBL7syJ8/+C4b1aevXm32xhtmJ50UHPD1XTehF14wO+KIYFkHeX3hIaORkBOfM72mz1SzZpp3Jnju55/NypQJwp9eRzTbRmdeNeWKmqqaNjXr1Wvvz/KVUcgCVRLvmbrOBbdw91adwmWXmWnqOlXkiyYjCK8pjjpq7/+xz+QEyKMIfIdKe9ratUFHiXnz9l5C6aSrud7ie8pefXVQ2/fVV8EJQv349tcEBPzZz5lqkRQivvhCQ42Dz1bYlHj99UFbD6LZNupI9c47Zm3aBGdhdbbSAI6wj686WwGZiD+VaLo6dWXTbc/Uda53wP7s+ZYyIIbAd6jUazZurjf3jdV75npzl1MajRtq3jy4r1EjGK0r1PAhJz5n778f1CrJlVcGbT0aFa5BQvLeexEUOo/IbNuIavs0sbeqZRQO1cwenon3DCACDiac4EHUjKtl3cKuuTqVhF1BNbgj7Emgj1uoatXcLjUSEYHvUN13X8Z6cvWk3TPXm2ve0ei88KT7+efB/bJleztxM98QcuJztmnT3tfCz5n6iIYDgxQAEc22CUfqq7+eqF0u/DaXcE4NIBOqCN4zdZ1rxt2xI7jtmbrOnUratw+Wt241e/PNYFn9+0Lh68jbCHyHavjw4KCuSZEaNgyu7sMTrIZEqfesxsSLpv9Q/bsuyXTwV5+fRJ33Dcn1OVNzoTr0iJp2jz026LunWiSdKS69NNLi5+ltE24ThbvjjgsmSPvkk6Bbx4gRe/v1ApkYNCio1Zs7N2go0k3L+ihp6rrzzzc79dTgvRqzpdNN+BFUz4NEGqGL6BD4DpX2Ml2h61JLHSp00Nf0K+pLpYO/aBZMhb3GjYMx9ZrrrVu3oCYmbNoFsvM5UxPitGnBcwoU6k+mz5kGDGieBg0UQHTHAM3DqZp+DeLSyGk9VhNv/BBKIBMa86OPjcb7qaJYNXlnnBHs+rrmU/BTzw5Nw6JTi6awUzPuXXfRXRx7paSnZ961U/OVad6y9u3bu69GSXZpaWnmk9TUVPOFb9vGJ3zOEpNP28U3Pn3OfJPqyX6jr6abNGmS+zKBkmE3sgOghg8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzKenp6emZvWnz5s1WqlQpa9++vRUsWDB3SgYAAIAD2rFjh02aNMk2bdpkJUuWPPAbqeEDAADwH4EPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwHIEPAADAcwQ+AAAAzxH4AAAAPEfgAwAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADPEfgAAAA8R+ADAADwXIGsvCk9Pd3d79y583CXBwAAAFkQ5rIwp2U78G3ZssXdv/fee1l5OwAAAHKJclqpUqUO+p6U9CzEwt27d9uqVausRIkSlpKSkpNlBAAAwCFQhFPYO/rooy1fvnzZD3wAAABIXgzaAAAA8ByBDwAAwHMEPgAAAM8R+AAAADxH4AMAAPAcgQ8AAMBzBD4AAADz2/8DXeEo9u3KtT8AAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def plot_maze_with_states():\n", + " \"\"\"Plot the maze with state indices.\"\"\"\n", + " grid = np.ones(\n", + " (n_rows, n_cols)\n", + " ) # Start with a matrix of ones. Here 1 means “free cell”\n", + " for i in range(n_rows):\n", + " for j in range(n_cols):\n", + " if maze_str[i][j] == \"#\":\n", + " grid[i, j] = 0 # We replace walls (#) with 0\n", + "\n", + " fig, ax = plt.subplots(figsize=figsize)\n", + " ax.imshow(grid, cmap=\"gray\", alpha=0.7)\n", + "\n", + " # Plot state indices\n", + " for (\n", + " s,\n", + " (i, j),\n", + " ) in state_to_pos.items(): # Calling .items() returns a list-like sequence of (key, value) pairs in the dictionary.\n", + " cell = maze_str[i][j]\n", + "\n", + " if cell == \"S\":\n", + " label = f\"S\\n{s}\"\n", + " color = \"green\"\n", + " elif cell == \"G\":\n", + " label = f\"G\\n{s}\"\n", + " color = \"blue\"\n", + " elif cell == \"X\":\n", + " label = f\"X\\n{s}\"\n", + " color = \"red\"\n", + " else:\n", + " label = str(s)\n", + " color = \"black\"\n", + "\n", + " ax.text(\n", + " j,\n", + " i,\n", + " label, # Attention : matplotlib, text(x, y, ...) expects (column, row)\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=10,\n", + " fontweight=\"bold\",\n", + " color=color,\n", + " )\n", + "\n", + " ax.set_xticks([]) # remove numeric axes, we don't need.\n", + " ax.set_yticks([])\n", + " ax.set_title(\"Maze with state indices\")\n", + "\n", + " plt.show()\n", + "\n", + "\n", + "plot_maze_with_states()" + ] + }, + { + "cell_type": "markdown", + "id": "db078d86", + "metadata": {}, + "source": [ + "### 2.4 Actions and deterministic movement" + ] + }, + { + "cell_type": "code", + "execution_count": 541, + "id": "f7f0b8e4-1f48-4d03-9e5f-a47e59c3e827", + "metadata": {}, + "outputs": [], + "source": [ + "A_UP, A_RIGHT, A_DOWN, A_LEFT = 0, 1, 2, 3\n", + "ACTIONS = [A_UP, A_RIGHT, A_DOWN, A_LEFT]\n", + "action_names = {A_UP: \"\\u2191\", A_RIGHT: \"\\u2192\", A_DOWN: \"\\u2193\", A_LEFT: \"\\u2190\"}" + ] + }, + { + "cell_type": "code", + "execution_count": 542, + "id": "4b06da5e-bc63-48e5-a336-37bce952443d", + "metadata": {}, + "outputs": [], + "source": [ + "def move_deterministic(i: int, j: int, a: int) -> tuple[int, int]:\n", + " \"\"\"Deterministic movement on the grid. If the movement hits a wall or boundary, the agent stays in place.\n", + "\n", + " Args:\n", + " i (int): current row index\n", + " j (int): current column index\n", + " a (int): action to take (A_UP, A_DOWN, A_LEFT, A_RIGHT)\n", + "\n", + " Returns:\n", + " (tuple[int, int]): new (row, column) position after taking action a\n", + "\n", + " \"\"\"\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j,\n", + " ) # It means “Unless the action succeeds, the robot stays in place.”\n", + "\n", + " # Now each action changes the coordinates of the robot:\n", + " if a == A_UP:\n", + " candidate_i, candidate_j = (\n", + " i - 1,\n", + " j,\n", + " ) # if the action is UP, then row becomes row -1\n", + " elif a == A_DOWN:\n", + " candidate_i, candidate_j = (\n", + " i + 1,\n", + " j,\n", + " ) # if the action is DOWN, then row becomes row +1\n", + " elif a == A_LEFT:\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j - 1,\n", + " ) # if the action is LEFT, then column becomes column -1\n", + " elif a == A_RIGHT:\n", + " candidate_i, candidate_j = (\n", + " i,\n", + " j + 1,\n", + " ) # if the action is RIGHT, then column becomes column +1\n", + "\n", + " # Check boundaries\n", + " if not (0 <= candidate_i < n_rows and 0 <= candidate_j < n_cols):\n", + " # If the robot tries to move outside the maze\n", + " # It will not move and it stays at (i, j).\n", + " return i, j\n", + "\n", + " # Check wall\n", + " if maze_str[candidate_i][candidate_j] == \"#\":\n", + " # If the next cell is a wall, the robot stays in place.\n", + " return i, j\n", + "\n", + " return candidate_i, candidate_j # Otherwise, return the new position\n" + ] + }, + { + "cell_type": "markdown", + "id": "c9e620e6", + "metadata": {}, + "source": [ + "### 2.5 Transition probabilities and reward function" + ] + }, + { + "cell_type": "code", + "execution_count": 543, + "id": "610253e7-f3f7-4a30-be3e-2ec5a1e2ed04", + "metadata": {}, + "outputs": [], + "source": [ + "gamma = 0.95\n", + "p_error = 0.1 # probability of the error to a random other direction\n" + ] + }, + { + "cell_type": "code", + "execution_count": 544, + "id": "7a51f242-fe4e-4e74-8a1f-a8df32b194b8", + "metadata": {}, + "outputs": [], + "source": [ + "# Initialize transition matrices and reward vector\n", + "P = np.zeros((len(ACTIONS), n_states, n_states))\n", + "R = np.zeros(n_states)" + ] + }, + { + "cell_type": "code", + "execution_count": 545, + "id": "49d54d1f-dc29-45b6-ad31-ad0e848f920d", + "metadata": {}, + "outputs": [], + "source": [ + "# Set rewards for each state\n", + "step_penalty = -0.01\n", + "goal_reward = 1.0\n", + "trap_reward = -1.0\n" + ] + }, + { + "cell_type": "code", + "execution_count": 546, + "id": "b9b7495a-c233-425c-99c0-5bddaf6c3225", + "metadata": {}, + "outputs": [], + "source": [ + "for s in range(n_states):\n", + " if s in goal_states:\n", + " R[s] = goal_reward\n", + " elif s in trap_states:\n", + " R[s] = trap_reward\n", + " else:\n", + " R[s] = step_penalty\n" + ] + }, + { + "cell_type": "code", + "execution_count": 547, + "id": "eca4c571-39c7-468b-af86-0bab9489415e", + "metadata": {}, + "outputs": [], + "source": [ + "terminal_states = set(goal_states + trap_states)\n", + "\n", + "\n", + "def is_terminal(s: int) -> bool:\n", + " \"\"\"Check if a state is terminal (goal or trap).\"\"\"\n", + " return s in terminal_states\n" + ] + }, + { + "cell_type": "code", + "execution_count": 548, + "id": "2d03276b-e206-4d1f-9024-f6948ca61523", + "metadata": {}, + "outputs": [], + "source": [ + "for s in range(n_states): # We loop over all states s.\n", + " i, j = state_to_pos[\n", + " s\n", + " ] # We recover the states to their coordinates (i, j) in the maze.\n", + "\n", + " # First, in a goal or trap state,\n", + " # No matter which action you “choose”, you stay in the same state with probability 1.\n", + " # This makes the terminal states as the absorbing states.\n", + " if is_terminal(s):\n", + " # Terminal states: stay forever\n", + " for a in ACTIONS:\n", + " P[a, s, s] = goal_reward\n", + " continue\n", + "\n", + " # If the state is non-terminal, we define the stochastic movement.\n", + " # For a given state s and intended action a,\n", + " # With probability 1 - p_error, the robot will move in direction a;\n", + " # With probability p_error, the robot will move in one of the other 3 directions, each with probability p_error / 3.\n", + " for a in ACTIONS:\n", + " # main action (intended action)\n", + " main_i, main_j = move_deterministic(i, j, a)\n", + " s_main = pos_to_state[\n", + " (main_i, main_j)\n", + " ] # s_main is the state index of that next cell.\n", + " P[a, s, s_main] += (\n", + " 1 - p_error\n", + " ) # We add probability 1 - p_error to P[a, s, s_main].\n", + "\n", + " # error actions\n", + " other_actions = [\n", + " a2 for a2 in ACTIONS if a2 != a\n", + " ] # other_actions = the 3 actions different from a.\n", + " for a2 in other_actions: # for each of the error action,\n", + " error_i, error_j = move_deterministic(i, j, a2)\n", + " s_error = pos_to_state[(error_i, error_j)] # get its state index s_error\n", + " P[a, s, s_error] += p_error / len(\n", + " other_actions\n", + " ) # add p_error / 3 to P[a, s, s_error]\n", + "# So for each (s,a), probabilities over all s_next sum to 1.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 549, + "id": "341fe630-8f87-4773-84ad-92d3516e53e2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Action ↑: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action →: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action ↓: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", + "Action ←: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n", + " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n" + ] + } + ], + "source": [ + "for a in ACTIONS:\n", + " # For each action a:\n", + " # P[a] is a matrix of shape (n_states, n_states).\n", + " # P[a].sum(axis=1) sums over next states s_next, giving for each state s:\n", + " # We print these row sums.\n", + " # If everything is correct, they should be very close to 1.\n", + "\n", + " probs = P[a].sum(axis=1)\n", + " print(f\"Action {action_names[a]}:\", probs)\n" + ] + }, + { + "cell_type": "markdown", + "id": "46d23991", + "metadata": {}, + "source": [ + "## 3. Policy evaluation\n", + "\n", + "### 3.1 Bellman expectation equation" + ] + }, + { + "cell_type": "code", + "execution_count": 550, + "id": "2fffe0b7", + "metadata": {}, + "outputs": [], + "source": [ + "def policy_evaluation( # noqa: PLR0913\n", + " policy: np.ndarray,\n", + " P: np.ndarray,\n", + " R: np.ndarray,\n", + " gamma: float,\n", + " theta: float = 1e-6,\n", + " max_iter: int = 10_000,\n", + ") -> np.ndarray:\n", + " \"\"\"Evaluate a deterministic policy for the given MDP.\n", + "\n", + " Args:\n", + " policy: array of shape (n_states,), with values in {0,1,2,3}\n", + " P: array of shape (n_actions, n_states, n_states)\n", + " R: array of shape (n_states,)\n", + " gamma: discount factor\n", + " theta: convergence threshold\n", + " max_iter: maximum number of iterations\n", + "\n", + " \"\"\"\n", + " n_states = len(R) # get the number of states\n", + " V = np.zeros(n_states) # initialize the value function\n", + "\n", + " for _it in range(max_iter): # Main iterative loop\n", + " V_new = np.zeros_like(\n", + " V\n", + " ) # Create a new value vector and we will compute an updated value for each state.\n", + "\n", + " # Now we update each state using the Bellman expectation equation\n", + " for s in range(n_states):\n", + " a = policy[s] # Extract the action chosen by the policy in state\n", + " V_new[s] = R[s] + gamma * np.sum(P[a, s, :] * V)\n", + "\n", + " delta = np.max(\n", + " np.abs(V_new - V)\n", + " ) # This measures how much the value function changed in this iteration:\n", + " # If delta is small, the values start to converge; otherwise, we need to keep iterating.\n", + " V = V_new # Update V, i.e. Set the new values for the next iteration.\n", + "\n", + " if delta < theta: # Check convergence: When changes are tiny, we stop.\n", + " break\n", + "\n", + " return V # Return the final value function, this is our estimate for V^{pi}(s), s in the state set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 551, + "id": "4c428327", + "metadata": {}, + "outputs": [], + "source": [ + "def plot_values(V: np.ndarray, title=\"Value function\") -> None:\n", + " \"\"\"Plot the value function V on the maze as a heatmap.\"\"\"\n", + " grid_values = np.full(\n", + " (n_rows, n_cols), np.nan\n", + " ) # Initializes a grid the same size as the maze. Every cell starts as NaN.\n", + " for (\n", + " s,\n", + " (i, j),\n", + " ) in (\n", + " state_to_pos.items()\n", + " ): # recall that state_to_pos maps each state index to its maze coordinates (i,j).\n", + " grid_values[i, j] = V[\n", + " s\n", + " ] # For each reachable cell, we write the value V[s] in the grid.\n", + " # Walls # never get values, and they stay as NaN.\n", + "\n", + " fig, ax = plt.subplots(figsize=figsize)\n", + " im = ax.imshow(grid_values, cmap=\"magma\")\n", + " plt.colorbar(im, ax=ax)\n", + "\n", + " # For each state:\n", + " # Place the text label at (column j, row i).\n", + " # Display value to two decimals.\n", + " # Use white text so it’s visible on the heatmap.\n", + " # Center the text inside each cell.\n", + "\n", + " for s, (i, j) in state_to_pos.items():\n", + " ax.text(\n", + " j, i, f\"{V[s]:.2f}\", ha=\"center\", va=\"center\", color=\"white\", fontsize=9\n", + " )\n", + "\n", + " # Remove axis ticks and set title\n", + " ax.set_xticks([])\n", + " ax.set_yticks([])\n", + " ax.set_title(title)\n", + " plt.show()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 552, + "id": "c1ab67f0-bd5e-4ffe-b655-aec030401b78", + "metadata": {}, + "outputs": [], + "source": [ + "def plot_policy(policy: np.ndarray, title=\"Policy\") -> None:\n", + " \"\"\"Plot the given policy on the maze.\"\"\"\n", + " _fig, ax = plt.subplots(figsize=figsize)\n", + " # draw walls as dark cells\n", + " wall_grid = np.zeros((n_rows, n_cols))\n", + " for i in range(n_rows):\n", + " for j in range(n_cols):\n", + " if maze_str[i][j] == \"#\":\n", + " wall_grid[i, j] = 1\n", + " ax.imshow(wall_grid, cmap=\"Greys\", alpha=0.5)\n", + "\n", + " for s, (i, j) in state_to_pos.items():\n", + " cell = maze_str[i][j]\n", + " if cell == \"#\":\n", + " continue\n", + "\n", + " if s in goal_states:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"G\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"blue\",\n", + " )\n", + " elif s in trap_states:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"X\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"red\",\n", + " )\n", + " elif s == start_state:\n", + " ax.text(\n", + " j,\n", + " i,\n", + " \"S\",\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " fontweight=\"bold\",\n", + " color=\"green\",\n", + " )\n", + " else:\n", + " a = policy[s]\n", + " ax.text(\n", + " j,\n", + " i,\n", + " action_names[a],\n", + " ha=\"center\",\n", + " va=\"center\",\n", + " fontsize=14,\n", + " color=\"black\",\n", + " )\n", + "\n", + " ax.set_xticks(np.arange(-0.5, n_cols, 1))\n", + " ax.set_yticks(np.arange(-0.5, n_rows, 1))\n", + " ax.set_xticklabels([])\n", + " ax.set_yticklabels([])\n", + " ax.grid(True)\n", + " ax.set_title(title)\n", + " plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "813121a9", + "metadata": {}, + "source": [ + "### 3.3 Evaluating a random policy" + ] + }, + { + "cell_type": "code", + "execution_count": 553, + "id": "ceb5dfe2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0 3 2 1 1 3 0 2 0 0 2 3 2 3 2 3 2 0 3 1 2 1 0 3 3 2 1 3 2 1 1 0 0 2 3 0 3\n", + " 3 1 2 0 3 2 1 0 3 1 3 2 3 3 0 1 1 1 0 2 0 2 2 3 2]\n" + ] + } + ], + "source": [ + "# Random policy: for each state, pick a random action\n", + "random_policy = rng.integers(low=0, high=len(ACTIONS), size=n_states)\n", + "\n", + "print(random_policy)" + ] + }, + { + "cell_type": "code", + "execution_count": 554, + "id": "8f3e2ac2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Value function under random policy:\n", + "[ -0.2 -0.295 -0.534 -1.301 -1.394 -1.467 -0.827 -1.176 -20.\n", + " -0.2 -0.207 -6.086 -0.548 -0.2 -0.201 -0.204 -0.28 -0.481\n", + " -20. -0.546 -0.566 -0.2 -1.126 -0.588 -0.201 -0.203 -0.209\n", + " -20. -20. -1.769 -1.186 -1.222 -0.229 -20. -1.944 -0.862\n", + " -0.824 -1.381 -18.279 -20. -7.557 -6.924 -0.44 -5.78 -17.248\n", + " -7.364 -0.214 -0.207 -18.427 -17.386 -16.41 -16.347 -17.519 -18.483\n", + " -20. -0.013 -20. -15.666 -20. -20. 20. 5.496]\n" + ] + } + ], + "source": [ + "V_random = policy_evaluation(policy=random_policy, P=P, R=R, gamma=gamma)\n", + "print(\"Value function under random policy:\")\n", + "print(V_random)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 555, + "id": "cf45291e", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAKXCAYAAAASHvsAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQzhJREFUeJzt3Q1wFNed7/2fxgKBMIJIELCDHMcOgdroXgxLtKzt8GJHDksqzkKQWaSAU3ayChguGN0UpWBetGBCEcyaFFzjBAfs3WjvktxadlnlsojXdZaszH14XCDEi8E2MTy+FI6xBJYsCTRPnR4Pg5BjRsBMd5/z/VR1zXSrsc9fp7v1m+7TPRnRaDQqAAAAOCPidwMAAACQXgRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAzho7dqw3xb399tvKyMjQpk2bfG0XAKQaARBAqJhwZkJafOrRo4e+9KUvadasWTp79qzfzQOAUMj0uwEAcCP+5m/+Rl/4whf00Ucf6be//a1eeOEF/eY3v1FdXZ2ys7Nv6L/5+c9/Xs3NzerWrdstby8ABAkBEEAo/cVf/IVGjhzpvf/e976nvLw8rV69Wv/8z/+sqVOn3tB/M35GEQBsxyVgAFZ46KGHvNe33npLly5d0tKlS3XvvfcqKytLd999t370ox+ppaXlU/8bf2wM4NGjR/XYY4+pf//+6tmzp4YMGaIFCxZ4P9u9e7f3b/7pn/6p03+vqqrK+9nvfve7W1orANwsAiAAK5w8edJ7NWcCzRnBRYsWacSIEfrbv/1bjRkzRj/+8Y/1V3/1V13+7x48eFB/9md/pl27dun73/++1qxZo7/8y7/U1q1bvZ+bm0jy8/P1y1/+stO/NctMCP3zP//zW1AhANw6XAIGEEoNDQ167733vDGA//Ef/+GNCTRn54YOHaof/OAHXgj8+c9/7q07c+ZMffazn9WqVau8M3bjxo1L+v8ze/ZsRaNRHThwQHfdddeV5StWrPBezRm+73znO97lZ9OmPn36eMvPnTun7du3XzlTCABBwhlAAKH0ta99zbska86+mTN7t99+u3cZdt++fd7P582b12H98vJy77W6ujrp/4cJcf/+7/+uJ554okP4iwe/uOnTp3uXl3/9619fWfaP//iP3qVoEw4BIGg4AwgglNatW+c9/iUzM1MDBgzwxuVFIhEvBJrXL37xix3WHzhwoPr27atTp04l/f948803vdeCgoJPXc+cdfzKV77iXfJ98sknvWXm/ahRozq1AwCCgAAIIJQKCwuv3AX8Sa4+Q5cO5izgnDlzdPr0ae9s4H/+539q7dq1aW0DACSLS8AArGKe5dfe3q433nijw3LzkOgPPvjA+3my7rnnHu/VPFvwesxl6Ntuu03/8A//4J39M88SnDJlyg1UAACpRwAEYJUJEyZ4r88//3yH5eYmDeMb3/hG0v8tM8Zw9OjR+sUvfqHf//73HX5mbgy5Wr9+/bxnE/793/+9FwDHjx/vLQOAIOISMACrDBs2TI8//rh+9rOfeWf8zCNgXnvtNb388sve41u6cgew8dOf/lQPPvig90iZv/7rv/a+fcQ8L9DcTPL66693ugw8efJk7715DiEABBUBEIB1NmzY4F2+NQ90NjeFmBtAKioqtHjx4hsKlGY838KFC72vmzOPnTGXkc2Doa/1zW9+U5/5zGe8S9CPPvroLaoGAG69jOi11zEAADfEPPblzjvv9ILgSy+95HdzAOCPYgwgANwiW7Zs8Z4daC4FA0CQcQYQAG5SbW2t95VxZtyfufHDfGsIAAQZZwAB4CaZsYEzZszwvm7ulVde8bs5AHBdnAEEAABwDGcAAQAAHEMABAAAcEzSzwE0321ppjjznKv3339feXl5af/OTQAAAHRmRvZduHDBeyRVJBK5+QD44x//WJWVlcmuDgAAAJ+88847GjRo0M3fBHLtGcCGhgbddddd+uEPf+h96XmYmYT8J3/yJ6qvr/fObIadTfVQS3DZVA+1BJdN9VBLcEUsqqetrU0/+clPvK/C7NOnz82fAczKyvKma5nw1717d4W947Ozs706wt7xttVDLcFlUz3UElw21UMtwRWxrB7jesPzuAkEAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAck5nsii0tLd4U19jY6L1GIhFvCrN4+8Neh431UEtw2VQPtQSXTfVQS3BFLKon2RoyotFoNJkVlyxZosrKyk7Lq6qqlJ2d3fUWAgAA4JZqampSSUmJGhoalJOTc/NnACsqKjRv3rwOZwDz8/NVX1+v7t27K+xpuaCgQEVFRerWrZvCrq2tTTU1Naqrq1N7e7vCzKa+ifeLDbXYVg+1BJdN9VBLcLVZ9HeztbU1qfWSDoBZWVnedC3ziwr7LyvObMQ2bMhx9E0w2VSLbfVQS3DZVA+1BFe7BX83k21/+C92AwAAoEsIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAJhi/7Puf+rrf/91DVg1QN2WdlOfFX30hTVf0NhNYzXnf8/Rv534N7+bCMvMmTNHAwcO9LsZQChcunRJpaWlGjp0qI4fP66wC+3+f/KkdPvtUkZGbHrkESka7biOmS8qSqzTq5f0xht+tTj0Mv1ugM2m/9N0/d3Bv+uwrLGl0Zve/uBt7T21V6caTunrX/y6b22EfRoaGnT27Fm/mwEEXmtrq6ZMmaItW7Z482PHjtWuXbu8MBhWod3/771Xeu456Qc/iM3X1Ejr1kmzZiXWWbtW2rEjMb9qlTR4cPrbagkCYIpsO7GtQ/j70zv+VF+/9+u6vfvtOtd0TgfePaDfnf6dr20EAFe1tLTo29/+tqqrq5WTk6PGxkZdvHjRC4E7d+7Ul7/8Zb+b6J6yMulf/kX6zW9i8/Pnx84EfulLkjk7a+bjxo+XZszwrak24BJwimw/uf3K+y/mflG136vVsw8/q4qvVmj111drz3f36NwPz+mH9//Q13a6qr29XeXl5aqvr/e7KejC2ZpZs2bp1KlTCruw1mLTfmPO/JnwV1FRoYkTJ3rLtm/frubmZo0bN06nT5/2u4lu2rBBysuLvW9qkqZPN2ldmjZNam6OLc/NlV56yddm2oAAmCKX2i9def/BRx94l3yvlZOVowfueiDNLcPly5c1bdo0rV69Whs3bvS7OUjSgQMHtGHDBo0ePVonzXihEAtjLbbtN3PnztWyZcu0fPnyK8tGjRqlmpoalZWVadCgQb62z1l33CG98EJivrZWKiyUXnstscz8/M47fWmeTQiAKTLijhFX3r/X9J6+tPZL+tOf/al+8K8/0M//n5/rxPsnfG2fywO+p06dqqqqKs2cOVMrV670u0lIkvnjvHXrVp07d84LTseOHVNYha0WG/cbc6l3wYIFnZYXFhZq6dKlvrQJHysulkpLE/MHDybel5RIjz3mS7NswxjAFPnOf/2O1u1fp//z//0fb7492u6N+zNT3IN3Pai1f7FWwwYO87GlbikuLvYGfPfp00cZGRmaPXt20nfWDWawccosXLhQ58+fT2rdgoIC7d+/X2PGjNHevXs1ZMgQBYlNtcSx3yDtzA0fu3ZJ776bWDZgQOzGENwSBMAUyYxkatf0Xfrxb3+sX/y/v9DZDzvflfXb3/9WRX9XpMMzD6t/r/6+tNMlZvzSnj17rtwpt64LB5LJkyfzhyyFzCXFM2fOdOnfmDsdzVi0oIUmm2ox2G/gCzMG8/33Oy4z82+/Ld13n1+tsgqXgFOod1ZvLX94ud4tf1d1M+r00qMv6fFhj6t3995X1jF3BF/7qBikRiQS8e7uy83NVd++fVVbW6toNJrUZC4XIXXMgPtk+iF+l6ZRWVl5ZfB+kNhUi8F+g7Rra4vd9GFu/khmOW4IATANzCWTL3/2y3pi+BPa9JebdHDGQUUyEr/6N/7AgyzTZcSIEd5zvjIzM1VUVKR9+/bJBuamgvXr13dafuTIEa1Zs0Y2uHDhgsaPH++djVqxYoUWLVqksApbLbbuN7awbv9fvFh6/fXE/FNPJd7X1UnPPONLs2zDJeAUefn1l/XRpY809b9M9e72vVqvbr28AGjGBRp9e/T1qZVuGjZsmHbv3q2HH37YO7Nx//33K+zMYPZt27apyTw24WOHDx/WQw895D3WYtKkScrPz1eYnThxQocOHfLuQn366acVZmGsxcb9xhZW7f/mw8XVNxk98URsPOBHHyUe/bJ6tfToo9JXv+pbM21AAEyRtz54S5V7KzX33+Z6N3vcN+A+5fbM1R+a/6Bf1/+6w2Nixn9xvK9tdZEZfG8OkP369ZMNNm/erAkTJnjPaOvfPzae1DzLzDxvzjzbLDQH/08xfPhwLzjZ0GdhrcW2/cYW1uz/H34oPf64eeZQbP7uu6Xnn4+9N6+7d0tvvmkGpsbWM3cHm6+Pww0hAKaYOQu4480d3vRJvj/i+xpz95i0twuy6o9Y7969vTMA3/zmN72zNPFHd+zYsUMjR46ULWzqs7DWEtZ228ya/b+83Jwej72PRKSXXzbFxeZN0HvlFWnMmFhAfOstyZw9//nPfW1ymDEGMEXmjpqrXxf/WjNHzlTh5wp1V5+71DOzp7rf1l2f6/05PTrkUf2vx/6XfvbNn/ndVFiiV69e3jcbmDFaeXl53mW6UB38Abi7/2/bJr34YmLehLvRozuu88ADHb8OznxrSHV1+tpoGc4ApogZ1/ftP/m2NwHp0rNnT++SD4Dkbdq0yZvCLtT7v/lu32j0+us9+2xswk3jDCAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjslMdsWWlhZvimtsbPReI5GIN4VZvP1tbW2yQbyOsPeLbX0Tr8GGWmyrh1qCy6Z6qCW42iz8u3k9GdFoNJrMikuWLFFlZWWn5VVVVcrOzu56CwEAAHBLNTU1qaSkRA0NDcrJybn5M4AVFRWaN29ehzOA+fn5qq+vV/fu3RX2tFxQUKCioiJ169ZNNnySqampUV1dndrb2xVmNvWNTf1ydd/YUI+Ntdiwz9i239jUNzb1i23HgNbW1qTWSzoAZmVledO1zC8q7L+sOLNDhn2nvBp9E0w29Ytt9dhUi037jEHfBJNN/WJLPcm2P/wXuwEAANAlBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEADT5NKlSyotLdXQoUN1/Phxv5uDq8yZM0cDBw70uxkAcNM4niFZBMA0aG1tVXFxsaqqqnTs2DGNHTtWR48e9btZ+FhDQ4POnj3rdzMA4KZxPEOyCIAp1tLSokmTJmnLli3Kycnxll28eNELgYcPH/a7eQAAwEEEwBSbMmWKqqurVVFRoYkTJ3rLtm/frubmZo0bN06nT5/2u4kAkJT29naVl5ervr7e76YAuEmZN/sfwKebO3euvvKVr2jBggX67ne/6y0bNWqUampqtHXrVg0aNMjvJgLAdV2+fFnTp0/3hrJEIhH95Cc/8btJAG4CATDFzKVeM12rsLDQmwAgDDexlZSU6Fe/+pVmzpyplStX+t0kADeJAAgA+FTmJjYzjrlPnz7KyMjQ7Nmzk74jdfDgwSlvH4CuIwACAD513N+ePXuu3GG6bt26pP/t5MmTCYBAQHETCADgjzLj/Xbu3Knc3Fz17dtXtbW1ikajSU2fNPwFQDAQAOGkAwcOaP369Z2WHzlyRGvWrPGlTUBQjRgxQrt27VJmZqaKioq0b98+v5uEq3A8w43gEjCcZO7K3rZtm5qamq4sM89lfOihh7xH9JhnN+bn5/vaRiBIhg0bpt27d+vhhx/2zgjef//9fjcJH+N4hhtBAISTNm/erAkTJnjPNOvfv7+3zDyX0Xxri3lOIwdLoLOCggIvWPTr18/vpuAqHM9wI7gEDCf17t3b+8RsDpLnzp278qiLHTt2eM9pBPDJCH/Bw/EMN4IACGf16tXL+5YWM6YpLy/Pu6w1cuRIv5sFAF3G8QxdxSXgNNq0aZM3ITh69uzpXSIBgLDjeIau4AwgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI7JTHbFlpYWb4prbGz0XiORiDeFWbz9bW1tskG8jrD3i219Y1O/XF2HDfXYWIsN+4xt+41NfWNTv9h6DLiejGg0Gk1mxSVLlqiysrLT8qqqKmVnZ3e9hQAAALilmpqaVFJSooaGBuXk5Nz8GcCKigrNmzevwxnA/Px81dfXq3v37gp7Wi4oKFBRUZG6desmGz6Z1dTUWFEPtQSXTfXEa6mrq1N7e7vCzNbjGX0TLDbt/7ZtZ62trUmtl3QAzMrK8qZrmV9U2H9ZcWYjtmFDtrEeagkum+rheBZc9E0w2VSLLdtZsu0P/8VuAAAAdAkBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAHFD5syZo4EDB/rdDFjcN5cuXVJpaamGDh2q48eP+90cd508Kd1+u5SREZseeUSKRjuuY+aLihLr9OolvfGGXy2GRWw5ngURARA3pKGhQWfPnvW7GbC0b1pbW1VcXKyqqiodO3ZMY8eO1dGjR/1ulpvuvVd67rnEfE2NtG5dx3XWrpV27EjMr1olDR6cvjbCWjYcz4KKAAggUFpaWjRp0iRt2bJFOTk53rKLFy96IfDw4cN+N89NZWXShAmJ+fnzpfhZWfNq5uPGj5dmzEh/GwF0CQEwxdrb21VeXq76+nq/m4Kr0C/BNWXKFFVXV6uiokITJ070lm3fvl3Nzc0aN26cTp8+7XcT3bRhg5SXF3vf1CRNn27SujRtmtTcHFuemyu99JKvzUTHM+mzZs3SqVOn/G4KAogAmEKXL1/WtGnTtHr1am3cuNHv5uBj9EuwzZ07V8uWLdPy5cuvLBs1apRqampUVlamQYMG+do+Z91xh/TCC4n52lqpsFB67bXEMvPzO+/0pXno7MCBA9qwYYNGjx6tk2YsJ3AVAmAKB7BPnTrVG8M0c+ZMrVy50u8mgX4JBXOpd8GCBZ2WFxYWaunSpb60CR8rLpZKSxPzBw8m3peUSI895kuz8MnMB6etW7fq3LlzXgg042mBuMwr73BLmQHsZgxTnz59lJGRodmzZyd9x9NgBk+nDP0C3CRzw8euXdK77yaWDRjQ+cYQpNTChQt1/vz5pNYtKCjQ/v37NWbMGO3du1dDhgxJefsQfATAFI0v27Nnz5U7mNZ14cA4efJkgkaK0C/ALWDGYL7/fsdlZv7tt6X77vOrVc4xw1fOnDnTpX9j7qY1454JgDC4BJwCkUhEO3fuVG5urvr27ava2lpFo9GkJnP5C6lBvwA3qa0tdtOHufkjmeVIGXMzVDLHrvgd9EZlZeWVG6sAAmCKjBgxQrt27VJmZqaKioq0b98+2TCgeP369Z2WHzlyRGvWrFEY2NgvtvQNQmDxYun11xPzTz2VeF9XJz3zjC/Nwie7cOGCxo8f7135WLFihRYtWqQw4HiWHlwCTqFhw4Zp9+7devjhh70zT/fff7/CzAzM37Ztm5rMIyA+Zp7L9tBDD3mP6DDPbsvPz1fQ2dYvNvUNAsx8WLr6pqknnoiNB/zoo8SjX1avlh59VPrqV31rJhJOnDihQ4cOeU88ePrppxUWHM/SgwCYYmbwrdlw+/Xrp7DbvHmzJkyY4D0/r3///t4y81w286wp85y2MO2QNvWLbX2DAPrwQ+nxx80zlGLzd98tPf987L153b1bevNNM9A2tp65O9h8fRx8NXz4cC8Ehu04x/EsPbgEnAZh2/n+mN69e3ufysyOaB4rEH+syo4dO7zHDYSNLf1iY98gYMrLzemk2PtIRHr5ZbPRxeZN0HvlFem222Lzb70lhehsk+3CeJzjeJYeBEB0Sa9evbxvaTDj5/Ly8rxLqCNHjvS7WaBvkCrbtkkvvpiYN+Fu9OiO6zzwQMevgzPfGlJdnb42wjocz1KPS8Dosp49e3qn4RE8tvXNpk2bvAk+Mt/tG41ef71nn41NwC1i2/EsaDgDCAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOCYzGRXbGlp8aa4xsZG7zUSiXhTmMXb39bWJhvE67ChHmoJLpvqidcQ9mOZzccz+iZYbNr/bd3OricjGo1Gk1lxyZIlqqys7LS8qqpK2dnZXW8hAAAAbqmmpiaVlJSooaFBOTk5N38GsKKiQvPmzetwBjA/P1/19fXq3r27wp6WCwoKVFRUpG7dusmGTzI1NTWqq6tTe3u7bOgbagkem/Yb9pngsqkeG2uxYf+37RjQ2tqa1HpJB8CsrCxvupb5RYX9lxVnNmIbNmQb+4Zagsum/camvrGpFtvqsakWm/Z/W/om2faH/2I3AAAAuoQACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAC6dKlSyotLdXQoUN1/Phxv5sDAFYhAAIInNbWVhUXF6uqqkrHjh3T2LFjdfToUb+bBQDWIAACCJSWlhZNmjRJW7ZsUU5Ojrfs4sWLXgg8fPiw380DACsQAH08wzFr1iydOnXK76YAgTJlyhRVV1eroqJCEydO9JZt375dzc3NGjdunE6fPu13EwGkUHt7u8rLy1VfX+93U6xGAPTJgQMHtGHDBo0ePVonT570uzlAYMydO1fLli3T8uXLrywbNWqUampqVFZWpkGDBvnaPgCpc/nyZU2bNk2rV6/Wxo0b/W6O1TL9boCrzB+0rVu36lvf+pYXAnft2qUhQ4b43SzAd+ZSr5muVVhY6E0A7L3xq6SkRL/61a80c+ZMrVy50u8mWY0AmAILFy7U+fPnk1q3oKBA+/fv15gxY7R3715CIADASebGLzP2t0+fPsrIyNDs2bOT+ndz5szR4MGDU94+2xAAU8Cctj5z5kyX/s3Zs2e98Q4EQACAi+P+9uzZ471vaGjQunXrkv63kydPJgDeAMYApoAZpB6NRq87xe9sNCorK68MeAcAwCWRSEQ7d+5Ubm6u+vbtq9ra2qT+jprpk4aM4PoIgD65cOGCxo8f733iWbFihRYtWuR3kwAA8M2IESO88fCZmZkqKirSvn37/G6S1QiAPjlx4oQOHTrk3ek0f/58v5sDAIDvhg0bpt27d6tHjx7eGUGkDmMAfTJ8+HAvBPbr18/vpgAAEBjm5kjz0Hf+PqYWZwB9xMYNAEBn/H1MPQIgAACAYwiAAAJr06ZN3l1+AIBbiwAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMyk12xpaXFm+IaGxu910gk4k1hFm9/W1ubbBCvI+z9cnUN1BI8Nu037DPBZVM9NtZiw/5v6zHgejKi0Wg0mRWXLFmiysrKTsurqqqUnZ3d9RYCAADglmpqalJJSYkaGhqUk5Nz82cAKyoqNG/evA5nAPPz81VfX6/u3bsr7Gm5oKBARUVF6tatm2z4JFNTU2NFPTbWUldXp/b2doWdTfuNTX1jU7/YegygluBps+gY0NramtR6SQfArKwsb7qW+UWF/ZcVZzZiGzZkG+uxqRab9hmDvgkmm/rFtnqoJbjaLTgGJNv+8F/sBgAAQJcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAMQNmTNnjgYOHOh3M2CLkyel22+XMjJi0yOPSNFox3XMfFFRYp1evaQ33vCrxe6gb+CTS5cuqbS0VEOHDtXx48f9bo51CIC4IQ0NDTp79qzfzYAt7r1Xeu65xHxNjbRuXcd11q6VduxIzK9aJQ0enL42uoq+gQ9aW1tVXFysqqoqHTt2TGPHjtXRo0f9bpZVCIAAgqGsTJowITE/f74U/9RvXs183Pjx0owZ6W+jq+gbpFFLS4smTZqkLVu2KCcnx1t28eJFLwQePnzY7+ZZgwCYYu3t7SovL1d9fb3fTUGSnzpnzZqlU6dO+d0UN23YIOXlxd43NUnTp5u/BtK0aVJzc2x5bq700ku+NtNJjvSNTceAsNYyZcoUVVdXq6KiQhMnTvSWbd++Xc3NzRo3bpxOnz7tdxOtQABMocuXL2vatGlavXq1Nm7c6HdzkIQDBw5ow4YNGj16tE6asU9IrzvukF54ITFfWysVFkqvvZZYZn5+552+NM9pjvSNTceAsNYyd+5cLVu2TMuXL7+ybNSoUaqpqVFZWZkGDRrka/tsQQBM4eDVqVOneuMXZs6cqZUrV/rdJCTBHGS2bt2qc+fOeQdNM/YEaVZcLJWWJuYPHky8LymRHnvMl2bBjb6x6RgQ1lrMpd4FCxZ0Wl5YWKilS5f60iYbZfrdAFuZwatm/EKfPn2UkZGh2bNnJ3137WAGT6fEwoULdf78+aTWLSgo0P79+zVmzBjt3btXQ4YMSXn7cM1NBbt2Se++m1g2YEDnmw+QfiHuG5uOATbVAn8QAFM07m/Pnj1X7pZd14UD4+TJkwmAKWIuw585c6ZL/8bc6WzGb3LATDMzxuf99zsuM/Nvvy3dd59frULI+8amY4BNtcAfXAJOgUgkop07dyo3N1d9+/ZVbW2totFoUpM59Y3UMAOHk+mD+N1mRmVl5ZVByEiTtrbYjQXmBoNkliN9Qt43Nh0DbKoF/iAApsiIESO0a9cuZWZmqqioSPv27VPYmQHF69ev77T8yJEjWrNmjWxw4cIFjR8/3juDu2LFCi1atMjvJrln8WLp9dcT8089lXhfVyc984wvzYIbfWPTMcCmWnDrEQBTaNiwYdq9e7d69OjhnREMOzMod8aMGd5dzXHmmUzm06UZj/LOO+8o7E6cOKFDhw55Nc6/+tlmSA/zQenqG6aeeCI25uzJJxPLzPb36qu+NM9pjvSNTccAm2rBrccYwBQzg29NSOrXr5/CbvPmzZowYYL3XMP+/ft7y8wzmcyzpswzmvLz8xV2w4cP9w6aNvRX6Hz4ofT44+b5SbH5u++Wnn8+9t687t4tvfmmGWQbW8/cgWq+ogyp51Df2HQMsKkW3HqcAUwDW3a+3r17a9u2bV7oM48ViD/uZseOHd7jBmxhS3+FTnm5OWURex+JSC+/bDa62LwJE6+8It12W2z+rbekp5/2r62ucaxvbDoG2FQLbi0CILqkV69e3hPazbjGvLw879L2yJEj/W4Wwm7bNunFFxPzJkCMHt1xnQce6PiVY+abKaqr09dGV9E3gJW4BIwu69mzp3fJF7hlzPfHRqPXX+/ZZ2MT0oe+gY82bdrkTbj1OAMIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMyk12xpaXFm+IaGxu910gk4k1hFm9/W1ubbBCvw4Z6bKwl7PuLjfuNTX1jU7/YegygluBps/AYcD0Z0Wg0msyKS5YsUWVlZaflVVVVys7O7noLAQAAcEs1NTWppKREDQ0NysnJufkzgBUVFZo3b16HM4D5+fmqr69X9+7dFfa0XFBQoLq6OrW3tyvsbKonXktRUZG6deumsH/CrKmpsaJfbN3OqCV4bKrHxlpsODbbdnxubW1Nar2kA2BWVpY3Xcv8osL+y7KxFtvqMQcYGw4ytvWLbfVQS3DZVI9Ntdh0bLalb5Jtf/gvdgMAAKBLCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIp126dEmlpaUaOnSojh8/7ndzAMBNJ09Kt98uZWTEpkcekaLRjuuY+aKixDq9eklvvOFXi0OPAAhntba2qri4WFVVVTp27JjGjh2ro0eP+t0sAHDPvfdKzz2XmK+pkdat67jO2rXSjh2J+VWrpMGD09dGyxAA4aSWlhZNmjRJW7ZsUU5Ojrfs4sWLXgg8fPiw380DAPeUlUkTJiTm58+X4ldmzKuZjxs/XpoxI/1ttAgBEE6aMmWKqqurVVFRoYkTJ3rLtm/frubmZo0bN06nT5/2u4kA4J4NG6S8vNj7piZp+nTziV2aNk1qbo4tz82VXnrJ12bagAAIJ82dO1fLli3T8uXLrywbNWqUampqVFZWpkGDBvnaPgBw0h13SC+8kJivrZUKC6XXXkssMz+/805fmmeTTL8bAPjBXOo107UKCwu9CQDgk+JiqbRU+uUvY/MHDyZ+VlIiPfaYb02zCWcAAQBAsJgbPszZwKsNGND5xhDcMAIgAAAIFjMO+/33Oy4z82+/7VeLrEMABAAAwdHWFrvpw9z8kcxy3BACIAAACI7Fi6XXX0/MP/VU4n1dnfTMM740yzYEQAAAEAz79kkrVybmn3giNh7wyScTy1avll591Zfm2YQACAAA/Pfhh9Ljj0uXL8fm775bev752Hvzes89sfft7bH1Ll70r60WIAACAAD/lZdLJ07E3kci0ssvS717x+bN9wS/8op0222x+bfekp5+2r+2WoAACAAA/LVtm/Tii4l5E+5Gj+64zgMPdPw6OPOtIdXV6WujZQiAcN6mTZsUjUb9bgYAuMt8t685DsenVas+eb1nn+243je+ke6WWoMACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOCYzGRXbGlp8aa4xsZG7zUSiXhTmMXbH/Y6bKwnXkNbW5vCLl6DDf1i63ZGLcFjUz021mLDsdm243OyNWREo9FoMisuWbJElZWVnZZXVVUpOzu76y0EAADALdXU1KSSkhI1NDQoJyfn5s8AVlRUaN68eR3OAObn56u+vl7du3dX2NNyQUGBioqK1K1bN9nwSaampsaKeqgluGyqh1qCy6Z6bKylrq5O7e3tCrvIxznAhnpaW1uTWi/pAJiVleVN1zK/qLD/suLMDhn2ndLWeqgluGyqh1qCy6Z6bKrFpgxgSz3Jtj/8F7sBAADQJQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMARJddunRJpaWlGjp0qI4fP+53c4BgOXlSuv12KSMjNj3yiBSNdlzHzBcVJdbp1Ut64w2FBceA4JozZ44GDhzodzMQAgRAdElra6uKi4tVVVWlY8eOaezYsTp69KjfzQKC4957peeeS8zX1Ejr1nVcZ+1aaceOxPyqVdLgwQoDjgHB1tDQoLNnz/rdDIQAARBJa2lp0aRJk7Rlyxbl5OR4yy5evOj9ATh8+LDfzQOCo6xMmjAhMT9/vhQ/U2ZezXzc+PHSjBkKA44BgD0IgCnW3t6u8vJy1dfXK+ymTJmi6upqVVRUaOLEid6y7du3q7m5WePGjdPp06cVFjb1i431/LEzT7NmzdKpU6cUChs2SHl5sfdNTdL06SZBSdOmSc3NseW5udJLLyksOAYA9iAAptDly5c1bdo0rV69Whs3blTYzZ07V8uWLdPy5cuvLBs1apRqampUVlamQYMGKQxs6xfb6vljDhw4oA0bNmj06NE6acbZBd0dd0gvvJCYr62VCgul115LLDM/v/NOhQXHAMAemX43wFZmkHRJSYl+9atfaebMmVq5cqXCzlzmMdO1CgsLvSkMbOsX2+r5NCZobN26Vd/61re8ELhr1y4NGTJEgVZcLJWWSr/8ZWz+4MHEz0pKpMceU5hwDADsQQBMETNI2oyT6dOnjzIyMjR79uyk7+AaHJLB4GFkW7/YUs/ChQt1/vz5pNYtKCjQ/v37NWbMGO3duzf4IdDc8LFrl/Tuu4llAwZ0vjEEaWHLPgPcLAJgisaW7Nmz58odWeu6cKCfPHkyB5kUsa1fbKrHXIY7c+ZMl/6NudPRjN8KfAA04+Lef7/jMjP/9tvSfff51Son2bTPADeLMYApEIlEtHPnTuXm5qpv376qra1VNBpNavqkyyu4NWzrF5vqMTcPJNPu+B2nRmVl5ZUbEQKrrS1204e5+SOZ5Ugpm/YZ4GYRAFNkxIgR3hilzMxMFRUVad++fX43CRb2i231fJoLFy5o/Pjx3hmcFStWaNGiRQq8xYul119PzD/1VOJ9XZ30zDO+NMtlNu4z5gap9evXd1p+5MgRrVmzxpc2IfgIgCk0bNgw7d69Wz169PA+dSIYbOsX2+r5Y06cOKFDhw55d27Ov/o5ekFlgsXVNxg88URsPOCTTyaWrV4tvfqqL81zmW37zIIFCzRjxgxv34gzz2U0Zy3N+Np33nnH1/YhmBgDmGJmwLrZEfv16+d3U2Bxv9hWzycZPny4FwJDUeOHH0qPP26eNxKbv/tu6fnnY+/N6+7d0ptvmkFpsfXM3cHm6+OQNjbtM5s3b9aECRO85xr279/fW2aey2ienWme05ifn+93ExFAnAFMAxsOMDayrV9sqyfUNZaXm1OWsfeRiPTyy1Lv3rF5E/ReeUW67bbY/FtvSU8/7V9bHRaa7ek6evfurW3btnmh79y5c1ced7Njxw7v8UnAJyEAAsCttG2b9OKLiXkT7kaP7rjOAw90/Do4860h1dXpayOs06tXL+9bWsy4xry8PO/S9siRI/1uFgKMAIgbsmnTJu/OOADXMN/ta/aN+LRq1Sev9+yzHdf7xjcUJhwDgqdnz57eJd/33nvPGzIBfBoCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOyUx2xZaWFm+Ka2xs9F4jkYg3hVm8/W1tbbJBvA4b6qGW4LKpHmoJLpvqsbGWsP/9j4vXYUM9ydaQEY1Go8msuGTJElVWVnZaXlVVpezs7K63EAAAALdUU1OTSkpK1NDQoJycnJs/A1hRUaF58+Z1OAOYn5+v+vp6de/eXWFPywUFBSoqKlK3bt1kwyezmpoa1dXVqb29XTb0DbUEj037jY37jA39YtA3we4XG2qxbTtrbW1Nar2kA2BWVpY3Xcv8osL+y4ozG7ENG7KNfUMtwWXTfmNT39jULwZ9E0w21WLLdpZs+8N/sRsAAABdQgAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQQCBdunRJpaWlGjp0qI4fP+53cwDAKgRAAIHT2tqq4uJiVVVV6dixYxo7dqyOHj3qd7MAwBoEQACB0tLSokmTJmnLli3Kycnxll28eNELgYcPH/a7eQBgBQKgj2c4Zs2apVOnTvndFCBQpkyZourqalVUVGjixInesu3bt6u5uVnjxo3T6dOn/W6is9rb21VeXq76+nq/mwKL/86wnaUHAdAnBw4c0IYNGzR69GidPHnS7+YAgTF37lwtW7ZMy5cvv7Js1KhRqqmpUVlZmQYNGuRr+1x1+fJlTZs2TatXr9bGjRv9bg4s/TvDdpY+BECfmD9oW7du1blz57yd04xzAiDvUu+CBQs6LS8sLNTSpUt9aZPrzA05U6dO9cZkzpw5UytXrvS7SbDw7wzbWXplpvn/54SFCxfq/PnzSa1bUFCg/fv3a8yYMdq7d6+GDBmS8vYBQFeYG3LMmMw+ffooIyNDs2fPTurfzZkzR4MHD055+1xk498ZtrP0IgCmgDltfebMmS79m7Nnz3rjHYK6YwJwkxmPtWfPHu99Q0OD1q1bl/S/nTx5Mn+YU8S2vzNsZ+nHJeAUMIPUo9Hodaf4nY1GZWXllQHvABAUkUhEO3fuVG5urvr27ava2tqkjm9mih/fcOvZ9neG7Sz9CIA+uXDhgsaPH+994lmxYoUWLVrkd5MA4BONGDFCu3btUmZmpoqKirRv3z6/mwQL/86wnaUXAdAnJ06c0KFDh7w7nebPn+93cwDgUw0bNky7d+9Wjx49vDM1CL4w/p1hO0sfxgD6ZPjw4d7O2a9fP7+bAgBJ30xgHsbNcSscwvp3hu0sPTgD6CM2bgBhw3ErXMLaX2Ftd5gQAAEAABxDAAQQWJs2bfLu8gMA3FoEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAck5nsii0tLd4U19jY6L1GIhFvCrN4+9va2mSDeB1h75era6CW4LFpv7Fxn7GhXwz6JpjiNdhQi63b2fVkRKPRaDIrLlmyRJWVlZ2WV1VVKTs7u+stBAAAwC3V1NSkkpISNTQ0KCcn5+bPAFZUVGjevHkdzgDm5+ervr5e3bt3V9jTckFBgYqKitStWzfZ8EmmpqbGinpsrKWurk7t7e0KO5v2Gxu3Mxtqsa0em44BNu3/tvVNa2trUuslHQCzsrK86VrmFxX2X1ac2Yht2JBtrMemWmzaZ2zrG2oJLpvqsekYYFO/2NI3ybY//Be7AQAA0CUEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAEyjOXPmaODAgX43AwiFS5cuqbS0VEOHDtXx48f9bg5scPKkdPvtUkZGbHrkESka7biOmS8qSqzTq5f0xht+tdhpHANSiwCYRg0NDTp79qzfzQACr7W1VcXFxaqqqtKxY8c0duxYHT161O9mIezuvVd67rnEfE2NtG5dx3XWrpV27EjMr1olDR6cvjbCwzEg9QiAAAKlpaVFkyZN0pYtW5STk+Mtu3jxovcH4PDhw343D2FXViZNmJCYnz9fip9dMq9mPm78eGnGjPS30XEcA9KDAIiktbe3q7y8XPX19Qo7m2qxzZQpU1RdXa2KigpNnDjRW7Z9+3Y1Nzdr3LhxOn36tGw4uzFr1iydOnVKNghdPRs2SHl5sfdNTdL06SZ1SNOmSc3NseW5udJLL/naTFe5cAwIAgIgknL58mVNmzZNq1ev1saNGxVmNtVio7lz52rZsmVavnz5lWWjRo1STU2NysrKNGjQIIXdgQMHtGHDBo0ePVonzbi0kAtdPXfcIb3wQmK+tlYqLJReey2xzPz8zjt9aZ7rXDgGBAEBEEkNxJ06dao3FmPmzJlauXKlwsqmWmxlLvMsWLCg0/LCwkItXbpUNjB/zLZu3apz5855ocmMcQqzUNZTXCyVlibmDx5MvC8pkR57zJdmwY1jQBBk+t0ABJ8ZiGvGYvTp00cZGRmaPXt20nc9Dw7Y4GmbakEwLVy4UOfPn09q3YKCAu3fv19jxozR3r17NWTIEAWNbfV0uuFj1y7p3XcTywYM6HxjCGAhAiCuO1Zuz549V+5iXteFA+PkyZMDFZpsqgXBZYYVnDlzpkv/xjwdwIxHDWJgsq2eDsxYsvff77jMzL/9tnTffX61CkgLLgHjU0UiEe3cuVO5ubnq27evamtrFY1Gk5rMafwgsakWBJcZoJ7MNhW/q9GorKy8Mtg9aGyr54q2tthNH+bmj2SWA5YhAKZhcPT69es7LT9y5IjWrFmjMBgxYoR27dqlzMxMFRUVad++fQorm2pBeF24cEHjx4/3zkivWLFCixYtUpiFsp7Fi6XXX0/MP/VU4n1dnfTMM740C0gXAmCKmYGsM2bM8O44jTPPMTKflM3YmnfeeUdhMGzYMO3evVs9evTwzqKFmU21IJxOnDihQ4cOeceF+Vc/dy6kQleP+eB39Q1gTzwRGw/45JOJZeaY/eqrvjQPSAfGAKbY5s2bNWHCBO+Zc/379/eWmecYmedmmeca5efnKyzMAG8TXvv166ews6kWhM/w4cO90GTL9heqej78UHr8cfM8qNj83XdLzz8fe29ed++W3nzTDBqOrWfuDjZfHwdYhjOAKda7d29t27bNC33mEQnxR5Hs2LHDe3RC2ITiAO9gLQgf27a/0NRTXm5OWcbeRyLSyy+bA3Vs3gS9V16RbrstNv/WW9LTT/vXViCFCIBp0KtXL++p5mbMWV5ennfZceTIkX43CwDcsm2b9OKLiXkT7kaP7rjOAw90/Do4860h1dXpayOQJgTANOnZs6d3yfe9997zLpcAuL5NmzZ5d5gCt4T5bl+zPcWnVas+eb1nn+243je+ke6W4mMcA1KHAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgmMxkV2xpafGmuMbGRu81Eol4U5jF29/W1iYbxOuwoR4bawn7/mLjfmPjdmZDLbbVY9MxwKb939a+uZ6MaDQaTWbFJUuWqLKystPyqqoqZWdnd72FAAAAuKWamppUUlKihoYG5eTk3PwZwIqKCs2bN6/DGcD8/HzV19ere/fuCntaLigoUFFRkbp16yYbPsnU1NSorq5O7e3tCjOb+ibeLzbUYrCdBZNN/XJ139hQj43bmQ212LbftLa2JrVe0gEwKyvLm65lflFh/2XFmY3Yhg05jr4JJptqMdjOgsmmfrGtHpu2M5tqsWU7S7b94b/YDQAAgC4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAmAqnDwp3X67lJERmx55RIpGO65j5ouKEuv06iW98YZfLXbWpUuXVFpaqqFDh+r48eMKFbazYKJfkGZ/+IO0cmVsU7vzTqlHDykrS7rjDmn0aOmHP5RefbXzZhgGc+bM0cCBA/1uhpUIgKlw773Sc88l5mtqpHXrOq6zdq20Y0diftUqafDg9LURam1tVXFxsaqqqnTs2DGNHTtWR48eVWiwnQUT/YI0+tnPpM9/Xpo/P7apvfuu1NJijm/S//2/seBnNi8TBM+eVeg0NDTobBgbHgIEwFQpK5MmTEjMm70zfobJvJr5uPHjpRkz0t9Gh7W0tGjSpEnasmWLcnJyvGUXL170QuDhw4cVGmxnwUS/IA1+8pPYpvbhh7F5czL5oYekH/1IWr5c+u//Xfra12JnBIFrEQBTacMGKS8v9r6pSZo+PfbRbNo0qbk5tjw3V3rpJV+b6aIpU6aourpaFRUVmjhxords+/btam5u1rhx43T69GmFBttZMNEvSKEjR6SKisS82dR++1tp507p2WdjPzMB0ZwVPHdO+h//Q+rZ088WI2gIgKlkBmC88EJivrZWKiyUXnstscz83AzaQFrNnTtXy5Yt03LzMfljo0aNUk1NjcrKyjRo0CCFBttZMNEvSKGf/lS6fDkxv369dP/9n7yuGZJqTjL36ZO25iEECICpVlwslZYm5g8eTLwvKZEee8yXZrnOXOpdsGBBp+WFhYVaunSpQoftLJjoF6SIOdMX95nPSJMm+dkahBEBMB3MgG9zNuBqAwZ0HhgO3Ay2s2CiX5ACZ84k3pv7hyJX/TU397LFbzC/evrud31pKgKKAJgOZjzZ++93XGbm337brxbBRmxnwUS/IMVMuAO6igCYam1tsUHfZvB3MsuBG8F2Fkz0C1Lkc59LvDePkLz6GX+f/WzsBhAzZWf70jyEAAEw1RYvll5/PTH/1FOJ93V10jPP+NIsWIbtLJjoF6TIww93PKH8L/+SmDc3l5tHwJgpjHf+HjhwQOvNXS3XOHLkiNasWeNLm2xEAEylfftij2ePe+KJ2HigJ59MLFu9OvakTuBGsZ0FE/2CFJo1S7rttsT8D37Q8bNGmJkb9GbMmKHVZv/4mHk+q7l5b+HChXrnnXd8bZ8tCICpYp7M+fjjifv0775bev752Hvzes89sfft7bH1Ll70r60IL7azYKJfkGJf/rJ09QMLzLd+jBwpffOb0pIlsWcBfu97UmOjQmfz5s168MEHVV5ert/85jfeMvN8VvMAf/O81vz8fL+baAUCYKqUl0snTsTem9uzXn5Z6t078VCmV15JfHx76y3p6af9ayvCi+0smOgXpIF52LO5Imq+99cwnzf+9V+lysrY6ALzjHEz3DQu/lzyoOvdu7e2bdvmhb5z5inWH39v+44dO7znteLWIACmwrZt0osvJubNwd18EePVHnig49dBmW8NqK5OXxsRfmxnwUS/II3+23+LfYYwZ/0efFDq31/KzIyN/bvrLqmoKPazAwc6fkV10PXq1cv7tqaioiLl5eVp586dGmlOceKWybx1/yl0+G7Pq2/J+mPMOXozwVebNm3yptBhOwsm+gVpZh4zae43MpNNevbs6V3yRWpwBhAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAx2Qmu2JLS4s3xTU2NnqvkUjEm8Is3v62tjbZIF5H2PvFtr6J12BDLQbbWTDZ1C9X12FDPTZuZzbUYtt+k2wNGdFoNJrMikuWLFFlZWWn5VVVVcrOzu56CwEAAHBLNTU1qaSkRA0NDcrJybn5M4AVFRWaN29ehzOA+fn5qq+vV/fu3RX2tFxQUKC6ujq1t7cr7Gyqh1qCy6Z6qCW4bKqHWoIrYlE9ra2tSa2XdADMysrypmuZX1TYf1k21mJbPdQSXDbVQy3BZVM91BJc7RbUk2z7w3+xGwAAAF1CAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDGZya7Y0tLiTXENDQ3ea1tbm8IuEomoqalJra2tam9vV9jZVA+1BJdN9VBLcNlUD7UEV8SieuK5LBqNfup6GdHrrfGxJUuWqLKy8ta0DgAAAClz8uRJ3XPPPTcfAK89A/jBBx/o85//vH7/+9+rT58+CrPGxkbl5+frnXfeUU5OjsLOpnqoJbhsqodagsumeqgluBotqsdcob3rrrt0/vx59e3b9+YvAWdlZXnTtUz4C/svK87UYUstttVDLcFlUz3UElw21UMtwZVjUT3msvan/jxtLQEAAEAgEAABAAAcc8MB0FwOXrx48SdeFg4bm2qxrR5qCS6b6qGW4LKpHmoJriyL6km2lqRvAgEAAIAduAQMAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAAMgt/z+e08m5IBNU/wAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_policy(policy=random_policy, title=\"Policy\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 557, + "id": "5a82a3b7", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmgAAAJ8CAYAAABKqF3EAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAm9NJREFUeJzs3Qd4U+XbBvD7JGm6J3SXvctURDYIDgQV3KDgRtyIiFtx4MI9//o5cTAEByhT2Rtl770KpXvPNMn5rvdNmzZtups2ae8f17manPPm5OlpaZ4874iiqqoKIiIiInIamoYOgIiIiIhsMUEjIiIicjJM0IiIiIicDBM0IiIiIifDBI2IiIjIyTBBIyIiInIyTNCIiIiInAwTNCIiIiIno2voAIiIiKhxysvLg8FgaJDn1uv18PDwgKtigkZEREQOSc7atIlEXFxKgzx/WFgYTp065bJJGhM0IiIiqnOiciaSs9Nn5sPPz6tenzsjIwetW90qY2CCRkRERFSKn48H/Hw86/dJzWa4Ok4SICIiInIyTNCIiIiInAy7OImIiMix3Y313eVoZhcnEREREdUxVtCIiIjIcVhBqxFW0IiIiIicDBM0IiIiIifDLk4iIiJyHFW1bPX9nC6OFTQiIiIiJ8MKGhERETmOWW2ASQIqXB0raEREREROhhU0IiIichwus1EjrKARERERORkmaEREREROhl2cRERE5Djs4qwRVtCIiIiInAwraEREROQ4rKDVCCtoRERERE6GCRoRERGRk2EXJxERETmO2gBdnCq7OImIiIiojrGCRkRERA6jqGa51fdzujpW0IiIiIicDCtoRERE5DhcZqNGWEEjIiIicjJM0IiIiIicDLs4iYiIyHHMqmWr7+d0caygERERETkZVtCIiIjIcThJoEZYQSMiIiJyMkzQiIiIiJwMuziJiIjIcdjFWSOsoBERERE5GVbQiIiIyHFUFajvz8ZUucwGEREREdUxJmjkkk6fPg1FUTBr1qwGef7ly5ejV69e8PDwkHGkpaXBGYnYXnnllYYOo9G67LLL5OYsv5dETj0Grb43F8cEjRxu9OjR8PLyQmZmZrltxo8fD71ej+TkZDg7EeOtt94KT09PfP755/jpp5/g7e3dYPEsXbqUSRgRUSPDMWjkcCL5+uuvv/DHH3/gzjvvLHM8JycHixYtwtVXX41mzZrB2f33338y2ZwxYwauuOKKhg5HJmgiUbSXpOXm5kKn43/z+tKqVSt5zd3c3Bo6FCJycaygUb1U0Hx9fTFnzhy7x0Vylp2dLRM5V5CQkCC/BgQEwNmJLlgmaPVHdG+Ka67Vahs6FCLn+yzO+t5cHBM0cjjRFXjjjTdi1apV1uSmJJG4iQROJHIpKSmYNm0aunfvDh8fH/j5+WHkyJHYs2dPtccDFbn77rvRunVrm31msxkfffQRunbtKl9QQ0ND8cADDyA1NbXS57jrrrvk7T59+sgXZHF+QTxH0e2K4lq7dq183Pz58/HGG28gKipKxnD55Zfj+PHjZR6/bds2jBo1CoGBgbIrtUePHvj444+t35uongninEVbRWPQdu3aJa+puLbiGovn3bp1q00bMYZKPHbTpk2YOnUqgoOD5XPfcMMNSExMtGmbnp6Ow4cPy6+VEdfo2muvldfgkksukb8b4mct7gu///67vC+uR+/evWWsRb7//nsZU8l9Rd58802ZFJ0/f77c5xbXQTxexCq6qMX3Lyq2jz/+OPLy8mzaGo1GWSFt164d3N3dZdzPP/888vPzK/z+yhuDVvSc4jqK77lTp0544YUX5LE1a9bIx4gKs73/G+LYli1bKnxeImp8mKBRvRDVMfGiJ5KSkkRCtmLFCvnCL164Tp48iYULF8oX8Q8++ABPPfUU9u3bh6FDhyI2NrbO4hHJmDj3wIEDZbJzzz33YPbs2RgxYgQKCgrKfZx4UZ00aZK8/dprr8nxZ+JcNfH222/LF2WRkD733HMySSpdRfznn38wZMgQHDx4UCYS77//PoYNG4bFixdbv48rr7xS3haxFG3lOXDgAAYPHiwT3qeffhovvfQSTp06JRNIkQiW9thjj8m2L7/8Mh566CHZVf3oo4/atBHfQ5cuXewmGPaIJPT222/Hddddh7feeksmxeK2uP5PPPEEJkyYgFdffRUnTpyQSY1IpoWbb75Z/o6IdqWJfeJ7iIyMrPT5xTlFQiaeWyS+n3zyifVnWmTixImYPn06Lr74Ynz44Yfy90+0HzduHKpr79696Nu3L1avXo37779f/r5df/318loKIu4WLVqU+32JJLF///7Vfl4ip8FJAjXCvg+qF8OHD0d4eLisCJR8gV+wYIFMiIoSE1E9OXr0KDSa4vcOd9xxBzp37oxvv/1WJhS1tXHjRnzzzTfyxU8kCkVE4iPGwYmYSu4vSSRDokrz1VdfySqUqALVlEgSdu/eLSdHCKJCJpKw/fv3o1u3bjCZTDIBE9dNtCvZpaoWrvEjXrg7duwoEzmR2FTmxRdflNdbXIO2bdvKfWJcoKjoiIRt3bp1Nu1Fhenvv/+2VuVEsiQSGlEt8/f3r9H3feTIEWzevNmadERHR8vEWCQvotLUsmVL6/UQ3//69etlEiOqrCKxmTt3Lt555x3r74ioqIkEViTcVdGmTRvZrS488sgjspL2v//9TybKojopEtIffvhBJmlff/21bPfwww8jJCQE7733nqx4id+VqhJJrvh57dy50/q9FSXogri24mcn3pCUvK6iUimufVGljYiaFlbQqF6I7idRfRBdNaIbqIhI2ET3ouhmE0R3UtELr0hQxIxJ0Q0nEgjxAlcXRAImXgRFspWUlGTdRJeaeC7xAlwfRNWuKDkTRGVLEFXEosRDVLemTJlSZrxbyW7MqhLXU7zgiySnKDkTRAIoElKRtGVkZNg8RlSWSj6XiFGc58yZM9Z9optVJCD2unftEQlZyYqQqC4VJfElE5ii/UXXoyiZFJXUkj8jkWiLytpNN91UpecXSVnpBKposkXJr6Jrt6Qnn3xSfl2yZAmqSiRZIsG89957bb43oeR1Fd+X6D799ddfrft++eUXWXWuSuJNRI0PEzSqN0VVsqLJAufOncOGDRtk4lY0qFpUaESXUocOHWSy1rx5czluR3QTVWWMU1UcO3ZMnktURMS5S25ZWVl2x8k5QukXbFExEorGwYkuPkFU0+qCSBbEjFmR7JYmuijFtY+JialWjDVR+pxFFSPRzWdvf8nnEkm1SCiLugNFzKKiNmbMGFlhqwrxu1WS6EIUbwqK3jiI5FPcb9++vU27sLAwmSiXTE4rU5RcVvYzFBViMaaxZDenuN2vX78ycRC5HLUBujdVdnESVZmoUIkXIvGCKgZci6+i8lJy3JUY7C26MUXFQQzSDgoKki+WoopUNBapPKIiUdT1V5Ko+JQkziOSM3tjfgSRqNVEeVUt8fz2ZvWVN9PP3vfQUBwRY3nnrMpziTai2ie6HkW3pJjEICpqtakylfdzq0mVsjZEFU10cYs3LqKaJsYkfvbZZ/UaAxE5DyZoVK9EMiYSMFERE5U0Uc0QlYMiootHjO8R481KEiv1i2paRUR1p2R3WJHSFQ9RMVm5cqWcICC6xuqKeH57nyggnr9kl2JViTgFMSatovXWqppIiMRTLBgsxoCVJsZ+iUS4dBXLGYlERkyWEIPsly1bJr8vMYatOhVUMQ6t5KQFkbQXzfQVa5mJ+6KdqCwWiY+Plz9fcbyqin7u4mdYGVFJFt2q4o1L0VpqY8eOrfJzETkrxWyWW30/p6tjFyfVq6JqmZghJwa+l561KCokpaszYsxYRcsnlExoRKJRchkIMeBbVFlKz+ITVS1RoStNjPmp6cc2iecXVQ+DwWDdJ2Zblu42rCoxg1AkEmI5kNIxlbxGRZ9iUFnc4tpeddVVcoB8yXGAIvEQyfKgQYPkgPnqqs4yG3VBDOQXm5jo8dtvv8nEpjprvRUtS1Lk008/lV/FpA9BzOwUxHUvSQziF6655poqP5dIHsUs3O+++w5nz561OVb691y8AREx/Pzzz7K6KyasVPamhIgaL1bQqF6JhGPAgAHWWXSlEzSxvIZYvkIMoBftxBIb4sWqKhUo0S0qXkRFNeW+++6TY8m+/PJLudZZycHvYskEMTtQLJsgkkSRtIhqhaiYiGRQLIMglnSoLjHrT1QAxQurSALFGDLxYltUCasuUdH64osv5BIU4nM/xTUR469EMiSWyxDLkxR1HQuTJ0+W33vRhAx7Xn/9dTnjUyRjYmaiSGz+7//+T3apiZmRNSGW1xCxiXXKqjpRoC6qaGLWpVDd7k0x8UKsuSd+TmLSivgZiW7Tnj17yuPiq1jrTszUFUmv+H35999/5cxOMcGiOjM4BTHrVVxvkXCLSRfi/4BIkMVkA/H7V/r7Kvrds/cGgoiaDlbQqN4VJWWXXnppmQHQYmyamC0nkg8xHkfM3BQvZFXpehPdUT/++KOs5Iiuoj///FOuCSZeGEsTiZt4ARZJnHhOsQ6ZWKdKvNiLrs+aEMmR6HoTy4SIMXPixV9U0MRCtDUlzilmLIqlNMS5xfclFvwVSVsRsQiwmIkoPsBdLEly2223lXs+kayKiRli0LpIUMV6Y6LLTjxH0axJV/kdEomouC7i96g6xOxIMQHl2Weflb9bYtmX0l3qojonro34WC/xsxS/G+J3ZN68edWOVSR8orIqKmki4RaJtKj8iSSxNPFzFV3lYoKEveNELklUixtic3GK6kwjkomIqkAsiyKqiaKrvKpr44lPEhBJl+gCd9auQ9HFHhERIRO10kkjkasRPRfizUbafx/Bz8ezfp87KxcBfabIN+w1GbrhDNjFSUQuR3yUkhhHKCqGjYn4FA2RQIquTqJGoyFW9je7/iQBJmhE5DJEV6P41ADxGaZiPFjpz1h1VeJjtsTMZjHu7KKLLpLj3oioaWOCRkQuQ0wgER8TJcYJFs2+bAzE2DQxWUFMBin9QetELo8VtBrhGDQiIiJy3Bi0Le81zBi0/tNcegwaZ3ESERERORl2cRIREZHjmFXLVt/P2RQSNPGxJ+Lz7sSHEdf359MRERFRzYhRTJmZmXL5FrH4NTWyBE0kZ67wGX1ERERUlvjIudosml0rnCTguARNVM6KfsCuOtiOiIioKQ7UFwWWotdxamQJWlG3pkjOmKARERG5Fg5Pcj2cJEBEREQOniRQ312cKlwdRwwSERERORlW0IiIiMhxxHr49b0mvsoKGhERERHVMVbQiIiIyHG4zEaNsIJGRERE5GSYoBERERE5GXZxEhERkWMH7Nf3shcqJwkQERERUR1jgkZERESOnyRQ31s1rF+/Htddd538UHnxqQsLFy60OX733XfL/SW3q6++Go7EBI2IiIiatOzsbPTs2ROff/55uW1EQnbhwgXrNnfuXIfGxDFoRERE1KSNHDlSbhVxd3dHWFhYvcXEChoRERE16S7Oqli7di1CQkLQqVMnPPTQQ0hOToYjsYJGREREjVJGRkaZKpjYqkt0b954441o06YNTpw4geeff15W3LZs2QKtVgtHYIJGREREjmNugGU2zJbna9Gihc3ul19+Ga+88kq1Tzdu3Djr7e7du6NHjx5o166drKpdfvnlcAQmaERERNQoxcTEwM/Pz3q/JtUze9q2bYvmzZvj+PHjTNCIiIjIBalmy1bfzwnI5KxkglZXzp07J8eghYeHw1GYoBEREVGTlpWVJathRU6dOoXdu3cjKChIbq+++ipuuukmOYtTjEF7+umn0b59e4wYMcJhMTFBIyIioiZt+/btGDZsmPX+1KlT5de77roLX3zxBfbu3YsffvgBaWlpcjHbq666CjNmzKizLlN7mKARERFRo5wkUFWXXXYZ1Ao+v3PFihWob1wHjYiIiMjJsIJGREREjuOghWMrVN/P5wCsoBERERE5GSZoRERERE6GXZxERETUpCcJOCNW0IiIiIicDCtoRERE5OAKWn1PElDh6lhBIyIiInIyrKARERGR43AMWo2wgkZERETkZJigERERETkZdnESERGRA5kBtb5X9jfD1bGCRkRERORkWEEjIiIix+EkgRphBY2IiIjIyTBBIyIiInIy7OIkIiIix2EXZ42wgkZERETkZBpVBe3AFZPhCrqu/ATObu/lj8MVqKoCZ2d2gRhVF3mz6RLXEs4fo5Bj1MLZXbb5A7gCVTU2dAjOTXwOZ71/FqcZro4VNCIiIiIn06gqaERERORkOAatRlhBIyIiInIyTNCIiIiInAy7OImIiMhx2MVZI6ygERERETkZVtCIiIjIcbjMRo2wgkZERETkZJigERERETkZdnESERGRYz+qpL4/rkTlJAEiIiIiqmOsoBEREZHjcJmNGmEFjYiIiMjJsIJGREREjsMKWo00uQQt7NGb4DegBzTenjDn5iFj3W7Ef70IqtFUpm3I3aPgO7AH3FuGImXhBsR98bvN8fAnxsK7R3voI4MR9+VCpPy+Fk2F4qZFxGM3w/fiTtD6e6MgKR2Jv6xC6vJtZdq6hQSi43fP2ezT6HXI3HYQp1/6Rt5v+fI98O7aBhoPd5gyspGybCsSZv9d6xgjJ98Mn4s7QlcU47zVSLETo9Dug0fhFd0aqqn4d+HwnW/AmJxhvR80qh+Cxw6HW3N/mNKzcf6z35GxeX+tYoyafBN8exfHGC9iXPZvhY/TBfqgy6xnYUhIw5H73yve38wPLaeNhU/PdjBl5CDup7+RvGRrjeMrGWeLx0vFOXc1ku3EqQvwQdQj18OnVztovTyQH5uEC98vR/rmA9Y2Xee9BLdAH6iFf0RVkxl7r32+1jG2nHIT/Hp3kDEaEtMRN28Nkpfav5YaL3e0evIWBPSPhjm/AAl/bMSFH/+xHvdoFYqWj98Ar45RUA1GpG0+gJhPF8q2tRFyw0A0v7oPPNuGI33bYRx/8fty24rnbvnY9fBqF46C9GzEzlqB5BU7rN9vx3cnwbN1KDR6NxiSMhC/YB0S/6r9zzvipgEIG3UJvNuGIWXrYRx47sdy22q93NHhqRvRbGAXeW3O/7YZZ2etksfcQwPQ5+cny/zfT956BAeemVXtuEaNGoVnnnkK3bt3Q0FBAdav34ApU6bi/Pnz1jZjxozGu+/ORGRkJHbu3IWJEyfhyJEj5Z6zsvbVPR9RXWlyCVrKnxsR/81fUPMM0Pp5o8X0e9Fs7OVIspMM5J9PQs5XixA4aoDdc+WfiEXG2l0IufdaNDlaLYwpGTj51OcwXEiGV5dWaP3WgyhITEPWDts/XgUJqThw7dPW+4pOiy7zX0Paml3WfQk/Lkf+uQSoBSaZ0LV5+0EY4lOQtnJ7jUNUtFoUJIsY/wdDrCXGNm8/AENSGrK22/8De+Hrv5D02zq7x4Ku6Y/gmy/D2Rk/IPf4eZkkiYSyNmSMKRk4Pu0La4ztZk5CQWI6MsuJURBJnYhB/A6X1PqlO+R59t84HR5twtDunQeRfy4RWXtO1D7O5Awce7IwzuhWaD9zkkyCSsep8XRHzvFzOP/VXyhIyoB/v2i0nn4Hjjz4IfLOxFvbnZrxE9I31jy5LS/Go1O/RH5sMryjW6HDO/ejICENGduPlmnf8vEbofP1wt5bX4MuwBcdPxC/c6lIXmH5nWv70gRkHTiNY09/Da23B9q/PRHhd16F818vqVWc4prE/rRSJpL64IBy22l9PNBx5kSc/34FDj++Fd6dWqDje5OQH5uCrH2nZFJ79pM/kHsmATCZZULZ+aOHkHsmHll7T9UqRpHsnZm1CoF92sM92L/Ctu2njoGbnxe23vgm9IE+6PHR/ciPS0X88p3Ij0/Dxitfsvm/33/Ri0hcubtGcfn7+2HmzHexbt06qKqKTz/9GPPnz8PAgYPl8Y4dO2L27J8wduztWLlyJZ5//jksWvQ7unbtAVOJN15FKmtf3fMR1aUmNwbNcDZeJmeSokA1m+EeGWy3bfo//yLrv0Mw5eTZPZ7y5wZk7zoK1VC7d9SuSFzD+FnLZHIm5Bw6g+zdx+DdvW2lj/Ub2F1e+/QNe6z78k5dkMmZ5eSWcnh5P5eqMhfFGFscY5aIsVvlMZahURB2z0ic//x3mRgJxtQs6/dfmxjjvl9uG+Ou4/Du3qbcx/gP7AadnxdS/rZNXvURzeDTrS1iv14sz5tz6CxSV+5A0Mi+tYqxKM4LJeM8aInTx06c4pok/LJWJpniZ5m+5QDyYxJlwuRIIsbY75bL5EzIPngGmSLGHmV/3hp3NwQNvwjnv10KU1aeTGITft+I5qOKr5V7RDMk/71DVteN6dlI33QAnm3Dah1n6oZ9SNu4X56zIj5dW8NcYETin1vk/4ds8fPcsA/B1xTGaFaRezJOJmcWlqUMPCKb1zrGpHX7kbzhAArScipsJ65jyOW9cOrrFfI65sYkyQpa2LWX2m3ffEhX+X8pcV3NEvO5c+dh6dKlyM7ORk5ODj766BP07XsptFqtPD5hwnisWbMWS5YsQX5+PmbMeB0hISEYPNiSwJVWWfvqno/KoRZ+kkB9bio/ScAlNR93BTr/9S46//YmPNpFInnh+oYOyeUpbjp4dm6FvJOxlbYNGtkPaat2QC0w2uyPmHwLui15F13mvQqNpx4pK7bVeYxelcQYOuFKdF34Bjr+3zQEXtnHut+9RQjcgvzg1SEKXeZMR5dfXkHUk2NlN1mdx9ilJfJOXLB7XOPtgciHxiDmgwVljnm2jZDVOJE4FhHJpOhKq2uK3hJn7kn7cZbu8vRoFVKmbcupt6LHohno+Pnj8OvbxSExenduidwTZX/e7i1DZFdbzvFY22vVrvhaxf2yFs1GXAJF7wZdkC8CBndH2uaDqDcaBYqi2OwS90vGKHR46z70/vttdP/xGRSkZskkrr54tQyW1zHrWPF1zD4WC+929hPZsGv7IOHvXbLLuC4MHToEhw4dslazevTojt27i9/4GY1GHDx4SO63p7L21T0fUV1qcl2cQtK8lXLTtwxFwOWXwJiS2dAhubyoabfBcD4R6Rv2VthOdF/6XNwJF776s8yx2E8WIPbTX+HZIQp+A7rBlJlbpzG2mDYOhnPlx3jhm8XIOxMnKzG+F3VEq+l3wSTGKW7cJ7vCBDGe7eiD78vbrV66ExEP34Bz782rsxhbPjVWVnPSyokx8oHrkLziX9n97t2tTZmuRVOW7TUT98UYobrWqijO9RX/vEWXVpvpdyJ1zR7kHImx7j/9xmzkHI2R73QDhvRE29fuxtHJn9m0qa3WT41F3vkkpK4vm7BoPfUw5eSXqD4BRnGtPIuvVfq2Q2jzzDhcvOxN+X2IxCd5ad2+aahI1oEz0Hjo5Zg1UUXz7tISgYO7yySspGPPfSuTOd/ubeDbq12tx8hVh9bLznXMzIXOzu+cGI8WeEkHnPzf0jp57l69emHGjFdxyy3jrPt8fHyQlpZm007c9/X1tXuOytpX93xUDk4SqJEmWUEr2d2Zd+I8Ip8e39ChuLTIx2+Be1QITk//ptLVm4Ou7ovc4+fKr2KpKnKPxsCck4+IB8fUXYxTbpFVsFPTvy03xpyDp2HOzpMvNpnbDyN58WYEXHaRPGbKtXSLJ8xZKScxiE3c9uvftc5ijJpys4zx5Evf2Y1RdB+LpCxh7mq7jzfn5suxUiWJ+/IFtA61eMIS54kX7cdpk5y9ejfM+Qacfe8Xm2PZ+05CzS+Q3dqpq3YiffNBBAztWWcxtpx6EzxaBuPEC/ZjFD9PjYcboNXYXqtcy7XS+nii4/sPInHxVuwc8Sx2XfMCzLkGtHmx/v5WiEkeIvlqdsXF6PXHK4h64BokLfsXxgw7XaNmFZl7TkIX6IuwccPqL8YcO9fRxwNGO79zYdf0kZW27OOVV12L3H77bcjMTJPb/v3Flaxu3bph2bLFePTRyXJsWJGsrCz4+9uOmRP3MzPtvwmvrH11z0dUl5pkBa30i4iYhUk1I7olxcD2k9M+tyQ3FVEUBF7dVyY2larDn0vk4zfL7riTT/6v8hhLKvHCnh+TIBMNR4machO8u7TC8Qpi9L24A9zDm6Hbr69Yu0PFGKDuC2fg0L3vIvdkLNya+csuRWOapcri2T5Sju+rKy0K4zw2teJrKZOzV+6SMw1PvvCt3VnSJal1OF6k5ROWGI8+8QVM5cSYfzYBqtEMr3YRyDl6Tu7zah9p7YZ1j2wmr23CbxuslcjEvzajw8xJqE9Z+0/j0COfWu+3e/kOmYiVR9Fp4BFV+zFoVZVzNlFeR5/24cg6Yhmb6dMhAtlibJxNYArCrrkEZ39aU63zz5kzV24lieRs5coVePbZ5zF79hybY3v37kOvXsWJvk6nQ3R0F+zbZ3/MW2Xtq3s+orrUpCpoorsgYERfucSG4N4mHM3Hj0DW9kP2H6DVyBdBRaMBtIq8XfKdongRkvsUBUphW4i2TUTE5JtlRefk0/8r07Vmj0/vTtD5eSNtjWWZgJLdnn6De8qfj7iWYqmL5jcMkVWs2oqcfJMlxqe+qDBG8Tvh27cLFHc32V3kc1EHNLtuoHUig5gIIgbch9x2uayuiPbidm2W2CgS9fhN8OnWBscriTFhwVocvONNHJ74ntzEoP28mER525iWKQfwZ+8/hfCJ18jvw6tzSwRe0bvOuuXEMhviWh6bVnGc4v+ISM5El+vJF78rnvxRyC0kQA7cF8mbaBtwWS8EDOyGtI21HzvVcsqN8loeffLLCmMU3YApa3Yh8r6RsnLmHtkcITcOQtISy7XKO5sgK2bB1w+UMYrvpfm1/ZBTOEGkVsTfCr1O/s2Q48zEbZ1lkHtpXh0i5XUSbZpf21d2YcYvsIyZ9WwfAb9LOspj4pz+/brIalv6f0fqOEZNuTHK5UlW7UHr+0fI6+gZ1RyRNw9E3F+2S5sE9ukAN39vJPxTs9mbRaKjo2Vy9uKL0zFr1g9ljv/882wMHz4MI0eOhF6vxwsvPI+kpCSsX29/nHFl7at7PiqHuUQ3Z71tcHmKKuYqVyIjI0OWddPT0+Hn5wdndeCKyRUeVzz0aPnqRHh0aCGTKVNaJjI27EHCD0tld0vLNx9Ezr4TSJprWQsp4qnxCBxhOwMudcU2xL47W95u/f5j8O7ZweZ4wo/LkPjjsgrj6LryEzi7vZc/XuFxkVR1mfsKzIYCOd2/iFgW4/xH89H6rQdkN1binOJ1pVq+dLdsf27m7DLnavn8HfBoEw4oGhQkpyPtn/+QMHdlpV2mqmo7iNrmvKGBiJ77cpkYU/8RMS5AGxnjCVnRE2u5tXlzEjxahso2hrgUudxGyTXTRAIp1lXzH9QdZoMRGVv2I/Z/C2XXYkXMlcTYbd50OzHuQMyHC9Du7UnI2ncS8bPLVh2DRvRB8M1DbdZBE+uziXXQvHu0hSkzB3E/Vm0dtMr+CuhFnL+UjTNFxPnBArk0SNZeS5xiDbaOHz8qK45F65wJcT+vlMfFchBiORCRFIlziepk3I//yNmelanoWooYe8x/SSYNJWNM/mcHzn7wq1xyI3PvScT9vKp4HbRpJddB24QLPxQvt+PTrTUiH7gWnuL30mxG1v5TOPvpQhgupFQYo4ryYxQi7r4KkfeMsNmXses4jkz5Ah3emSiXyLhQGGPrZ8cicFB3mSiJJT/OfrYIeactS5V4dYpCa9GV2yJELjlhiEtFwsJNVV4HLcdoPykUWt17JVrfd6XNvrSdJ7Dnsf9D9/fuRfreUzj7o6UaJsY4dnz6JgQVroMW+9tmnPne9ve1y2vj5e/Okdfnozou2/yBzf3vvvsGd911p5zBWVJ0dHfExFjGL15//Ri8887biIqKkuuW3Xff/dZ1ywYNGiS7Rn19i5c3qah9VY4Lqlo3kx4coSFfv4ueO+1/D8DPU1+/z51rQMDD/+f0eUtFmlSC5iwaQ4LmLCpK0JxFRUmFs6j8r4BzcIlrWUmC5iwqStCcRekEzVkxQav4udM+m9QwCdqjXzl93lKRptMfR0REROQimKAREREROZkmP4uTiIiIHEeMRS05HrW+ntPVsYJGRERE5GRYQSMiIiLHzkKq75lIKitoRERERFTHWEEjIiIix+FncdYIK2hEREREToYJGhEREZGTYRcnEREROQ67OGuEFTQiIiIiJ8MKGhERETkOK2g1wgoaERERkZNhgkZERETkZNjFSURERI7DLs4aYQWNiIiIyMmwgkZEREQOo6oq1HquaKmN4LM4G1WCptWYGzqERkMD1/jl7rb644YOgerRhoFPw9kN3vQOXME6F7iWqmps6BCIGkyjStCIiIjIyXAMWo1wDBoRERGRk2GCRkRERORk2MVJREREjsMuzhphBY2IiIjIyTBBIyIiIsdX0Op7q4b169fjuuuuQ0REBBRFwcKFC8ss2zF9+nSEh4fD09MTV1xxBY4dOwZHYoJGRERETVp2djZ69uyJzz//3O7xd955B5988gm+/PJLbNu2Dd7e3hgxYgTy8vIcFhPHoBEREVGTNnLkSLnZI6pnH330EV588UWMGTNG7vvxxx8RGhoqK23jxo1zSEysoBEREZHjiFX9G2KrI6dOnUJcXJzs1izi7++Pvn37YsuWLXAUVtCIiIioUcrIyLC57+7uLrfqEMmZICpmJYn7RcccgRU0IiIichjV3DCb0KJFC1ntKtreeustuApW0IiIiKhRiomJgZ+fn/V+datnQlhYmPwaHx8vZ3EWEfd79eoFR2EFjYiIiBrlMht+fn42W00StDZt2sgkbdWqVTZdp2I2Z//+/eEorKARERFRk5aVlYXjx4/bTAzYvXs3goKC0LJlS0yZMgWvv/46OnToIBO2l156Sa6Zdv311zsspiaXoIU+fAt8BvaAxssD5tx8ZK7fhYRvFgJGU5m2ES/dB6/otlA89DBlZiN9+RYkz1lhPa4L8kfY1Nvh1aM9TBnZSJq9HOnLNqMpUNx0CH/sZvhc3Alaf28Yk9KR+MsqpC3fare9R4cWCH/kRni0jYQpPQsJPy5D2j//WY/rmvkh8snb4N2jPYwZ2Uj8eQVSlzpudgw1TuE3DUDoqEvg3TYMKVsP49BzP9pt5xbgjbaPj4Z/rzbQensg73wyznz7D1I2HrS2af/0TfC/qC08o5rh5KeLETt/I5qKiJsGIKzEdTxQznUUtF7u6PDUjWg2sAvM+QU4/9tmnJ21qsrHiZzB9u3bMWzYMOv9qVOnyq933XUXZs2ahaefflqulTZp0iSkpaVh0KBBWL58OTw8PBwWU5NL0FL/Wo+E7xZBzTNA6+eNiBfvQ7Nbr7BJvIok/7QMF84nQC0wQhcciBZvPoyC+BRkrLIkFhHP3w1DbBKO3foc3FuHo8Wbj8BwLgG5+4qz8EZLq4ExJQOnn/ochgtJ8OzSGq3fehDGxDRk7Ths01Tj7YnWbz2A+FnLkLr0E3h2bInWMx+G4UIycvaflG1avGC5lodufgEercPReuZDyD+XiJy9TeBaUp0xJGUgZtYqBPRpD32wf7ntRNKQdfQ8Tv1vqXxM0IDO6PzqeOye+AlyTifINtnHY5G0ag9aPTACTY24JmdmrUJgn/Zwr+A6Cu2njoGbnxe23vgm9IE+6PHR/ciPS0X88p1VOk5NgAt8Fudll10m1zsrj/h0gddee01u9aXJjUEzxMTL5ExSFLlWij4yxG7b/NOxMjmzsKyroo8IlvfcwpvDs2s7JH73pzxf3uEzyFi9HQFXO64/2pmI7zlh1lKZnAm5h04ja/cxeHVvW6atV9c2MBuMSF28Sf6nyRXXauMeBI6yXCt9eHN4dWuLuG//kucVx9NW7UDgyL71/n2Ra0tetx/JGw6gIC2nwnZ5sSk4P3c9DInp8v91yqZDyD2bCN+uLa1tLvy+BWk7jsOcX/Q3oOlIquJ11Li7IeTyXjj19QqYsvKQG5MkK2Rh115apeNEVL4mV0ETgsZeiea3Xw2NpzuM6VlI/GZRuW1DH7sV/lf2g8ZDj4K4ZKT/s03ud28bAWNKOkxpmda2eSfOIfC6wWiKRJenV+dWSF+9o+wxjSLffdjQKPBoE1HiWmbAlFriWh4/h6DRTfNaUv0TXZ6erUOQfcJxaxo1Rl4tg6HR65B1LNa6L/tYLFreMaxKx6lpKLnsRX0+p6trkglayi//yE3fIhR+l/eBMdV2IbuS4j+dj/jPFsCjfQv49O8OU6blHaXGwx3m7FybtuasXGg8Hdcf7cwip92G/POJyNiwp8yxnIOn5Di+oDGDkbJ4k0zk/Ab2hLEwudV6usOUZftO3SSupVf1Z9sQVZei06Lza+ORtHoPsg6fa+hwXIrWSw9TTj5gKn41NGbmQlf4f7ey40RUvibXxVm6uzP/xHmET5tQcUNVRd6xszDn5iFk0g1ylzkvX46tKknjLSYeOO6DU51V+OO3Qh8VgrPTv7b78RqmjBycffErBFx+CTr/+gZC7x+N1BVb5cQKeTw3H9pS11Lr4wmz+MNO5ODkrMsbd8CUV4Bjb//W0OG4HFOOARoPNzkmtYjWxwPGwv+7lR0novI1yQpa6T/Q5Y1BK9NWK9paxqDln4yVszi1AT4wpWXJfR7topB/qriU3xSET74FXl1a4dS0z2DOLj85zTlwCicnf2i93+LFu5FdOAFAXstmpa9lJPKa2LWkBkjOXp8gvx58dhZUOzO5qWI5ZxOhGs3waR+OrCPn5T6fDhHIPhlXpePURKgNMElArefnc4AmVUET3Wz+V/WzVr7cW0eg2e0jkL3jUJm2upBA+A7qJR8jJhN4RrdB4PWXIXu7pW3BhSTkHjyJ4HtGQ3F3g0enVvAb3gdpy7c0reSsW1ucfvpz2b1bEY/2UXKcmqJ3k5MDvHu1R/Jva+UxMdEg58BJhN53nbyWnp1aympb6jL7S3YQlUurgaLXQRFfNYW3ddoyzcTxzjPGy7GlB5/7AWpB2eRMPE4+XoyhLDxvyUpQU7mOqOA6imUzElbtQev7R8jlSjyjmiPy5oGI++vfKh0novI1uQqa3/BLZDelSBbEGKjMjbuR9OMSeSzqjYeQu+8Ekuf9Le8H3nCZXOdMJGhiQkDqonVI/uUf67li3/xeHu+w4G05Nk2sp9YkltgQg6pDAtFszGCYDQXoOOdV6/70lf8h9qP5aPXWg8jZdwKJcyzXq9kNQ+A3qAeg1SL3wCmcevIzGJOLx/7FvPGDXAety29vymsZ99UiLrFB1dbyrsvR6r4rrfcHrXkTaTtPYN9j/4eu792LjL2nEPPjGvh1b43mQ7rBlF+A/ktetraP+Wm1PC50+3AiAi5uJ2/792qLto9eK9dKO/td8d+AxqrVXZejdYnrOKTwOu557P/Q/b17kb73FM4WXqfjHyxEx6dvQr+FL8iELPa3zTZLaFR2nJoAMQSxvgftm+HyFLWihT9KfKSB+JDR9PR0m8+0cjaHr3oUrqDz35/B2e2/fDJcQbdVnzR0CFSPNgx8Gs5u8KZ34ArWucC1HOoi19KZNeTrd9FzJz09Dn7u+vp97nwDmr8zz+nzloo0uQoaERER1R/VrMqtvp/T1TWRARVEREREroMJGhEREZGTYRcnEREROQ4nCdQIK2hEREREToYVNCIiInIcMV6/vsfsq3B5rKARERERORkmaEREREROhl2cRERE5DBcB61mWEEjIiIicjKsoBEREZHjcJmNGmEFjYiIiMjJsIJGREREDqOaLVt9P6erYwWNiIiIyMkwQSMiIiJyMuziJCIiIsfhJIEaaVQJmkZxjXVPDl/1KJydorhGcXX/5ZPh7MxQ4Ox6rPoYrmDI5g/g7FS8A1dgVp3/93LNgGfgCoZtntnQIVAj1KgSNCIiInIunCRQM65RJiEiIiJqQpigERERETkZdnESERGR44jh4fXd5ajC5bGCRkRERORkWEEjIiIih1FVy1bfz+nqWEEjIiIicjKsoBEREZHDcJmNmmEFjYiIiMjJMEEjIiIicjLs4iQiIiLH4Wdx1ggraEREREROhhU0IiIichhOEqgZVtCIiIiInEyTq6AFP3wLfAb0hMbLA2puPjI37ETiNwsBo6lM26h3HodHlzaAqTgVP3XvqzClpMvbze68Fj4DekDfMgxpf65D4pe/1UmMoSLGgT1kjGYR4/pdSCgnxoiX7oNXdFsoHnqYMrORvnwLkuessB7XBfkjbOrt8OrRHqaMbCTNXo70ZZvrJM6wR2+C34Ae0Hh7wpybh4x1uxH/9SKoduIU30v4lFvh27cbVIMBKQs3IHF2cZweHVog7JEb4dEmEqaMLCT8uAzp//xXq/gUNx3CH7sZPhd3gtbfG8akdCT+sgppy7fabS9iCBcxtI2EKd0SQ1qJGHTN/BD55G3w7tEexoxsJP68AqlLt9QyRi0iHrsZvoUxFhTGmLp8W5m2biGB6Pjdczb7NHodMrcdxOmXvrHc93JH5JSx8OvXFWZDAZIXrkfCz3+jsRo1ahSeeeYpdO/eDQUFBVi/fgOmTJmK8+fPW9uMGTMa7747E5GRkdi5cxcmTpyEI0eOlHvOytpX93yuStFq0P7x6xB6VS95P37FLhz/ZDHUEn8Pi3hEBqHj1Ovh17UFzPkFODd/E87OXmc93vWNCfDv3gpaTz0K0nNwYfF/ODNrdb3GqG/uh47TrkdAz9ZyEdO0HSdw9P2FKEjLtrZpNqgL2ky8Cl4tmsOYlYfT369E7MKy/xeJ6kOTS9DS/1qPpG8XQc03QOPnjYgXJyLoliuRMne53fZJ3y1C2h9r7B4riE2UyZ3/yIF1GmPqX+uR8N0iqHkGaGWM96HZrVfYJF5Fkn9ahgvnE6AWGKELDkSLNx9GQXwKMlZZEouI5++GITYJx259Du6tw9HizUdgOJeA3H3Hax1nyp8bEf/NX9Y4W0y/F83GXo6k2WUTgvBHb4LW1xtHb38ZugAftHr3ERgSUmQSJhK8Vm8+gIQfluH00k/g2bElWs18GAUXkpGz/2TNA9RqYEzJwOmnPofhQhI8u7RG67cehDExDVk7Dts0FTG0fusBxM9ahtTCGFrPfBiGEjG0eMFyLQ/d/AI8Woej9cyHkH8uETl7a3EttVoZ40kZYzK8urSSMRbIGG1f9AsSUnHg2qet9xWdFl3mv4a0Nbus+0Syp/P1wqHbXoEu0Adt33kEhvhUm0SzMfH398PMme9i3bp1UFUVn376MebPn4eBAwfL4x07dsTs2T9h7NjbsXLlSjz//HNYtOh3dO3aAyZT2TcSlbWv7vlcWau7L4d/j9b4d/wH8n6P9+9FqzuH4fT3q2wbahT0mHk3EjccwL6nZ8lkrddH9yMvIR0J/+yWTU5/uxI5MYlQC0xwDw1Azw/uRd6FVJlQ1UuM4mc77Xr5dcuNbwMKEP3ybejwxGgcfHmu3B/UtyM6TrsBh16dh7Q9p6Dz9oA+yKdW8ZEFP0mgZppcF6chJl4mZ4KiKIBZhVtkcI3OlbFyG3K2H4Q5J6/uY8yzxAgRo6pCHxlit23+6ViZnFlY/hfoIyzfj1t4c3h2bYfE7/6U58s7fAYZq7cj4Or+dRPnWds4VbMZ7naupeLuBr/LeiPh+8UwZ+fCcD4RKQvXI7AwDq+ubWAuMCJ18Sb588gVcW7Yg4CRtYtTxJYwa6lMzoTcQ6eRtfsYvLq3LdNWxmAoFcPGPQgcZYlBH94cXt3aIu5bS0Iqjqet2oHAkX1rHaNICkVyJuQcOoPs3cfgbSfG0vwGdpfXPX3DHut1DrjsYsR9v8Rync8lygpa0Mh+aKzmzp2HpUuXIjs7Gzk5Ofjoo0/Qt++l0Gq18viECeOxZs1aLFmyBPn5+Zgx43WEhIRg8GBLAldaZe2rez5XFn7tJTjzw2oYkjPlJm6HX9enTDuvlsHwbNlcJmGicpV7NklWyCLGXGptk30yTiZnkqrKZNozqnm9xSh4RgQhYdVemHINMOUYkLBqD7zbhVmPt5l0layYpe06Kf8GGDNzkXMmsdYxEtVUk6ugCYG3Xolmt4+ExtNddmUlfruw3LbNbrsazcaPlFWp1D9WI3Plv/USY9DYK9H89qtljEYR4zeLym0b+tit8L+yHzQeehTEJSP9H0tJ3r1tBIwp6TClZVrb5p04h8Dr6u7FpPm4K9B8/AhoC+M8882fZdq4twiVXXF5x4u7ncTt4NuustxRFIh/JSkaBe5tIuosTnlONx28OrdC+uodZY9pFEvCXpJGgUdhDJZrmQFTaolrefwcgkYPrvMYPTu3QpqdGEsTiZdIEosSdPcWIfI655a4zrknziP49ivRVAwdOgSHDh2yVrN69OiO3bstCaxgNBpx8OAhuX/t2rVlHl9Z++qez1XpfD3hERqArKOx1n1Zx2LhERYIrbcHTNl5Nv93LDdKnEBR4NM+vEwFK2xUb2g99Mi9kIK4pdvrLUYhZt4GhAzvjuTNh+X73pAreyF54yF5TOPhBt9OkXBv7o++86bJx6fvOYVjH/4pEz+qJbMohij1/5wurkkmaKnz/5GbvkUofIf3gSk1w267pO//hOHMBZjzDfDq1QnhL9wHNScfWZuL/0A7Ssov/8hNxOh3eR8Yy4lRiP90PuI/WwCP9i3g0787TJk5cr/Gw11WUkoyZ+VC4+lRZ3EmzVspN33LUARcfgmMKWX/mInE0ZSbD5iLx4WYsnPleCkh99ApOYYuaMxgpCzeJBMU34E9YSyRWNaFyGm3If98oqzOlZZz0DYGkcj5lYhBJKCmLMt1tX4PWcXfQ12JmnabrDCmb9hbYTsxHk2MrbvwVXFCLN9wlL7OWbnQ1nGMzqpXr16YMeNV3HLLOOs+Hx8fpKWl2bQT9319fe2eo7L21T2fqxJjxQRjVvHfDzEmS9B5udskP6LKlBeXijb3X4VTX/8tK2OislX69+7oewtx9P1F8O0UgeaDomWFqr5iFNL3nkbE6EsxeMXL8n7G/rM486Nl+IqbrxcUjQbBQ6Kxe8o3MKbnoOPTNyD65XHYPfnrWsVJVFNNrouzdFdi/snzCJ12h93jeYdOWbovTWbk7DiE9CUb4TP04vqP8cR5hE+bUHFDVUXesbNysH7IpBvkLnNevhxbVZLGW0w8yKv7OM/GI+/EeUQ+Pb7MMXOeARp3N0BT/Osm3qGac/LlbVNGDs6+9BX8h1+CTgveQOjE0UhbsVVOaqgr4Y/fCn1UCM5O/9ru4AQZw4tfySSz869vIPT+0UgtEYNIfLSlrqXWx9P6PdSFyMdvgXtUCE5P/6bSARRBV/dF7vFzyDtZXD0QE0rsXWdTHcbY0G6//TZkZqbJbf/+4kS7W7duWLZsMR59dLIcG1YkKysL/v7+NucQ9zMz7Sf/lbWv7vlclegGFLQ+xW/mxJgswVjq90l0a+575kf4dozAgEUvyKQmbsl2GDNs39BYGqvIPHxenqPdo9fUW4yiZNbz44kySVt/+XS5ids9P5pYeC5L+3MLNiM/Lk2e+9Q3/yDg4rayukbUEJpkBa0kMdBaH2F/fJezjDqUMZYzBq1MW61oaxkHln8yVs7i1Ab4wJSWJfd5tItC/qlYB8ZZdgxavhhTZzTDo10k8o7FWOPIKxFH7oFTOPX4h9b7US/eXbvB9yWET75FDr4/Ne0zmEu9qy4p58ApnJxcHEOLF+9GdmEM8lo2K30tI22+h9qIKIzx5LTPK4xRUhQEXt0XCXOKExEZY0yCvM6e7SKQe+xcncfoDObMmSu3kkRytnLlCjz77POYPXuOzbG9e/ehV6+e1vs6nQ7R0V2wb99+u+evrH11z+eqRHUrLz4Nvh0ikHc+Re7z6RiBPJG82Pn9zDkVjz1TvrXeb/vwSKTtOlXu+TU6rZwpWV8xuvl5wjM8SCZgYpapcO7XzWg54TK4+XvJmaWiCmhPmaEPVG1cB61mmlQFTfFwh99V/axVJX3rCATddjWydxws01a08e7TVQ68FmORPHt1gv81g5C10TIrSdJq5JghURoXVQtxW+yrXYx6+JeI0b11BJrdPgLZOyxjJUrShQTCd1Av+Rjxou0Z3QaB11+G7O2WtgUXkpB78CSC7xktvw+PTq3gN7wP0pbXbmmIom7LgBF9i+NsEy7HomUVPndJan4BMtbtRMjdo2QFTyRxQdcPQeqy4jg82kdZrqXeTQ7M9+7ZHsm/r62b5KxbW5x++nPZvVuRMjH0ao/k3ywxiIkGOQdOIvS+6+S19OzUUlbbUpfZX7KjOiIm3wzvbm1w8un/yS7Jyvj07gSdnzfS1uwoc53T1+5E6D3XWK9z8xuGIGVp7WN0VtHR0TI5e/HF6Zg164cyx3/+eTaGDx+GkSNHQq/X44UXnkdSUhLWr19v93yVta/u+VyZqIK1umu4nMkoNjE78sJf9sfgisH2otIk3qQ1H9oV4ddcgtOzLDMp3cMCEHxZN0uXpKLAr1srRN4yECnbjtZbjCIBy4lJQuRN/eU4TbGJ2yLBE8eE2EXbEHnzALkchzje+p4rkLr9hLVSR1TfFFVMp6lERkaGLOOnp6fDz88PzuroiEcqPK646xHxyiQ5Vktxc5Pji0TClfzTYvniFvn6w8jdfwIp81ZA6++DiNcekmPABKOcJLAGGX8XJxWhT94hk6mS0v/eivj3f6owDrNa/jsykWxFWWPUyRgzN+5G0o9LZIxRbzyE3H0nkDzvb5mgRTxzl2UwvaLICQEZK/+Tx4qqfaLqI9dB695ejk1L+nlZldZBM5krTjRFnC1fnSjXDhNxiokIYmxXwg9LZZwt33wQOftOIGnuP9Z10CKmjIVPv67yeMqiDUj8uXhpk4hpt8NvUA+57ISopsV98Tvyz8RVGmdFv71inFanua/K9cBKrouUvvI/xH40H63essSYOMcSY+RTtjFc+J9tDLrm/pZ10Lq3k9cy4aflVVoHzVxqAkTpGLvMfaVMjGkrt+P8R/Pl0h/Z+05aYxRavnS3bH9u5uwy5xNj4qKeGAvfwuuctGgDEn4quzxLaT1WfQxXoCi2Rf/vvvsGd911p5zBWVJ0dHfExFiqtddfPwbvvPM2oqKi5Lpl9913v3XdskGDBsmuUV/fAOtjK2pfleOqWjSr2rmtGfBM5WuMTbkOoVeWXWOs41OWYRRH3/3DOgMy8oZ+0OjdkHX8Ak58tgTp+85YE7To6eNkEicmFOQnZSB++U6c+XFtrXslqhOjV+sQdHj8Ovh2jpR/L8WEguOfLimeZKBR0O6RUQgb2VveTdt5Asc+WARDiqViXpFhm2fCWTXk63fRc5+9bRz89Pr6fW6DAS3nznP6vKUiTSpBcxYVJWjOorIEzVm4wlo3FSVozsJVEzRn1FgSNKo6JmgVPzcTtJpx/r92RERE5LI4Bq1mXKNMQkRERNSEMEEjIiIicjLs4iQiIiLHfhZnfXdxqnB5rKARERERORlW0IiIiMhhVFWRW30/p6tjBY2IiIjIyTBBIyIiInIy7OIkIiIixzErUM313OVoZhcnEREREdUxVtCIiIjIscts1POyFyqX2SAiIiKiusYKGhERETkMl9moGVbQiIiIiJwMEzQiIiIiJ8MuTiIiInIYtQGW2VAbwTIbjSpBUxTXmLbRecXnDR0C1aNtQ55s6BAaDVU1NnQIjcawzTPh7FYPeLahQyBqMOziJCIiIocvs1HfW3W88sorUBTFZuvcuTMaUqOqoBERERHVRNeuXbFy5UrrfZ2uYVMkJmhERETU5Ol0OoSFhcFZsIuTiIiIHL4OWn1v1XXs2DFERESgbdu2GD9+PM6ePYuGxAoaERERNUoZGRk2993d3eVWWt++fTFr1ix06tQJFy5cwKuvvorBgwdj//798PX1RUNggkZEREQOYzYrcqvv5xRatGiBkl5++WU5IaC0kSNHWm/36NFDJmytWrXC/Pnzcd9996EhMEEjIiKiRikmJgZ+fn7W+/aqZ/YEBASgY8eOOH78OBoKx6ARERFRo1xmw8/Pz2araoKWlZWFEydOIDw8HA2FCRoRERE1adOmTcO6detw+vRpbN68GTfccAO0Wi1uu+22BouJXZxERETUpJ07d04mY8nJyQgODsagQYOwdetWebuhMEEjIiIih6npshe1Ud3nmzdvHpwNuziJiIiInAwraERERNSkK2jOiBU0IiIiIifDBI2IiIjIyTS5Ls7gh26F94Ce0Hh5QM3NR+aGnUj69g/AaLJppwsORKuvXrLZp+jdkP3fAVx45Uub/doAX7T6ejqMCSk4+8hb9fJ9UOMReuNANL+6D7zahiNt22Ece+F7u+30IQHo8ePTNvs0ep18zNHnvpP3PVuFotWUG+DdMRJmgxFpmw7gzKeLYM4vqJfvhag+KVoNOjx+LUKvukh0aiFuxW4c/2QxVJO5TNshK18t838n53QC/r3zY3lf39wPnaaNQUDP1nINrdQdJ3D0/UUoSMuut++nsTKritzq+zldXZNL0NIWr0fSdwuh5hug8fNG+Av3I/CWK5E6d7lNO2NiKk7cMLV4h06LtrPfQtba7WXOGfzIWOQfj4HWz7s+vgVqZAxJGYj9cSX8LukAfXBA+e0S0rD96uet9xWdFhf9/jKSV+2y7ms3fTyy9p/Gkae+htbbA51m3ofIu65EzFdLHf59ENW31ncPh3+P1tg2/gN5v+f796DVncNw+vtVZdquv+Jlm/uX/vg44lfusd4XyZmw+caZgAJEvzwOHZ+4Dgdedr7ZfdQ0NLkuzoKYOJmcCYqiiA/sgj4ipNLH+fTvCWgUZG3abbPfu18PaH28kLH6X4fFTI1b6vp9SN24H8b06r1TDxzcDYpGQcr6fdZ9HhHNkPTPTqhGkzxf6qYD8GzbcCthEzlS+LWX4PQPq2FIzpTb6R/WIOK6Syp9nG+XKHi1DsGFpTus+zwjgpCwai9MuQaYcgzytne7MAd/B02DalYaZHN1TS5BEwJvvQrt/vgAbX95B+5to5D259pKH+N39QBkrv4PaoHRuk90kzafdBMSPp3r4IiJygq+pq8lGTMU/05emLcWzUf0hqLXwS3IF4GDuyNt84EGjZPIEXS+nvAIDUDW0QvWfVnHYuERFgitd8Uf5xNxXR+kbD0KQ1Kmdd/ZeRsRMryHfKzOxwOhV/ZE0sZDDv0eiCrS5Lo4hdT5f8vNrUUY/Ib3gSk1o8L2upAgePXqjLPf/GGzv/nEG5C5cisKYhPh0bWdg6MmKqYPDYR/7w6I+eIvm/1iPFrbZ8eiz/I3ZReoqK4lLmF1lxofradefjVm5Vr3GbPy5FedlztM2fl2H6fxcEPoFT1xcMZ8m/3pe08jYnQfDFlh6QpN338WZ36s/M07Va7kZ2PW53O6uiZZQSvZ3Zl/8hxCn7yzwnZ+V/VH/okYGE6dt+4TCZlHdDukzP+7HiIlshU8qg+yj51Hzoni6oHWxxOdP3gACYu34b+rnsP2US/CnGdAu5fGN2isRI4guiIFUe0qovO23Dbm2E/OBFElM+UZkLz5cPFORUGvjycife8ZrLv8ZbmJ270+us+R3wJRhZp0giYoWi3cIir4rC1Fgd+V/ZCxfLPNbq+LOsMtrLmcOCC6SkMeuhX61hHytjbIz/GBU9OlKAge2QeJi7fZ7PaIbAaNuxvif90gx6CZsnKR8OcWBPTr0mChEjmKMTMXefFp8OkQYd3n0zEceXFp5VbPiro345bttJnp6ebnCc/wQJxbsEnOeBbbuV83w79bS7j5ezn8eyFCU+/iVDzc4Tv4ImRt3gNzdq5MqAJvG4mcHeWPM/C6uDO0/j7IXPufzf6031chY/km632fwRfLcWqxL3wGU1rxuAaiSmk1crkAuWkUOX4MZlUmWfb4X9IROn9vm9mbQu7ZBJhzDQi9YSDi/9wCjd4Nwdf2k5U2osbowpIdaH3XMNk9KbS+cxhi/7L9W12SV8vm8O/eEofeWGCzvyA9BzkxSYi8qT9Of2eZARp1U3+ZAIpjVDtmNMAyG3D9SQJNKkETndK+w/qg+f03QnHTwZSWhaxNu5D802J5OGLGI8jdfxypv6ywPsRvxABkbdgFc45lbEMRcb/kPlNWjlxLzZiUVo/fEDUGkXdegah7RljvX7pyJjJ2Hcehx79Ap3cmInPvKcT+XLxsQPA1lyJl3V6Yskv9TuYacOTZb9HywWsRNXEkVLNZLrlx8k1OYqHGSSynISpcfec8Ke/HrdiFMz+ukbc7PXW9/Hrk3YXW9uHX9kHantPIPZdc5lx7n/lRrqk2cNFzgKKREw7EPqKGoqhq5UPpMjIy4O/vj/T0dPj5OW/33bGrH4Yr6LD8fw0dAtWjbUMsLx7OrO/69xs6BKIyVg94Fq5g+Oa34awa8vW76Ll3Xj4RvjrLpI76kmk04OJV3zh93lKRJj8GjYiIiMjZMEEjIiIicjJNawwaERER1Su1AT6LU20En8XJChoRERGRk2EFjYiIiBxazarvipbKChoRERER1TVW0IiIiMhhxGc2mBvgOV0dK2hEREREToYJGhEREZGTYRcnEREROQwnCdQMK2hEREREToYVNCIiInIYsyo2pd6f09WxgkZERETkZJigERERETkZdnESERGRw3CSQM2wgkZERETkZBpVBU2rcY1RgWeufQDOTnGRNx8t//o/OLt+Gz6Gs1PxPlyBojj/n6zfL3kWrsBd4/xrrSusITSiSQL1/5yujr/9RERERE6GCRoRERGRk3H+/gIiIiJyWZwkUDOsoBERERE5GVbQiIiIyGHMUORW38/p6lhBIyIiInIyrKARERGRw6iqZavv53R1rKARERERORkmaEREREROhl2cRERE5DBmVZFbfT+nq2MFjYiIiMjJsIJGREREDqM2wDIbKpfZICIiIqK6xgSNiIiIyMk0uS7OZg/eCu/+PaHx9oQ5Jx/ZG3ci+bvfAaOpTFt9+5Zo/sAt0LeJhCk9G6mzFyNr9Tbrcffodmg28SboW4TBnJuPzNVbkfrDn3W2AIuid0P4Z9Oh9fNBzLgnyhzX+Psi6P5b4N6tIzReHjBeSETa7L+Q++/eMm3dWkUg/KMXkLt9PxLf+KJO4iuKMexTS4znbisboxDy5lS4d24LtcQ1vvDgdJhS0uVt//Gj4dmvF9xahCFz8VqkfTMfjd2oUaPwzDNPoXv3bigoKMD69RswZcpUnD9/3tpmzJjRePfdmYiMjMTOnbswceIkHDlypNxzVta+uudzBa58HYN6tESvZ66Dd8tmyDqbjD1v/4mUfTF223a8eyg63TOkeIeiQOepx7an5yB2zcFqn6+qAnu0RNenx8C7RTNkn03C/pmLkLY/pkbtgwd2Qrs7hsC3XRjMRhNSdp/GoQ8XIy8ho8bxKVoNujxxDSJG9IKqqohdsQeHP1oC1WSuVluNmxbR00aj2aXtoPf3Rl5iBk79vB7n/tpR49ioGNdBq5kmV0HLWLIeMZNexembp+Lco6/L5Cvg5qvKtBMJXNirjyBzzb84feuTSHjnWzR/aKxMyiwNFIRNfxA5W/fg9NgnETvtXfgMuQS+Vw+ss1hF4mJMTCn3uMbDHYYTMYh78m3EjH1CJmfNn54Itxbhtg0VBc0euwP5h07UWWwlYzRVEGORtFl/4Nytj1u3ouRMMF5IQNqs35C7rWxi2Vj5+/th5sx30aJFa7Rp0x4ZGRmYP3+e9XjHjh0xe/ZPeOKJaQgKCsbq1WuwaNHv0Gq1ds9XWfvqns9VuOp1dPPzRP8P78DJBduwZPgbOLVgG/p/cAfcfDzstj86ax3+GjrDuu14+VcYMnMRt/lojc5X1Rgvef8unFmwBf9cMQNnft2KSz64C7pyzllZezdvD5z4aT1Wj56JtTe8C2N2Hi564zbURrt7hyGwZytsGPcRNt72MYJ6tUK7uy+rdluRvOUnZ+K/R7/DP8Nfxb4Zv6Lz5FFo3rd9reIjqo0ml6AVxMRBzTcU3lNkmu0WEVKmnXuXtkCBEZlLN4j5usg/chrZm3fBb4QlAdN4ecqqUebKrfK4MSEFubsPQ986sk7i1LdrCc/eXZHx6/Jy2xjjk5Dxxz8wJafJ70NUzgrOxUPfuY1NO9/rhqMg5gLy9lv+mNcVt3Yt4XFxV2T8Vn6MVZG9eivydhyAOTcXTcXcufOwdOlSZGdnIycnBx999An69r3U+kI/YcJ4rFmzFkuWLEF+fj5mzHgdISEhGDx4sN3zVda+uudzFa56HSMui5ZVmtMLt8NcYJJf81IyET4sukqPbzWmN86t2AtzvrFOzmdP6FDLOWMWWc4pvookJuyy6Bq1j/17DxI3HYEp1wBTXgFOz9uMgK4tZHJUU1HX9caJ79fK5xGbuC32VbetiOfYVyuRc97yZlNU/ZJ3nERgz9Y1jo3KLrNR35ura3IJmuB/y1Vo/duHaD3vXejbRCH9rzVl2igaReZvpXbKiptgzspBxopN8B0xANBqoAtrDs9enZHz3/7aB6jRIOixO5DyxVybbsFKH+bvK7sJC04Vd+9og4PgN3o4Ur/7rfZxlYpRVOVSv5wLtaDyGP3GjkLknA8Q9tEL8B7Wr25jaQSGDh2CQ4cOwWSyXMsePbpj9+491uNGoxEHDx6S++2prH11z+eqXOU6+nUIRdrRCzb70o/Gwb99aKWP9QjxQ2i/DjizaEednK88vh3CkXE01mZfxtEL8G0fViftgy5qg6zTiXa7I6tC5+sBz9AAm+fMOHYBnuGB0Hm717itoNHrENA1CpnH42oUG1FdaHJj0IT0BX/LTSQzPsMuhSml7BiIvEOnoHi4w+/aochYtgHunVrDe0BPmNIyrW2yN+xA8OMTEHj7NVC0WqT/uQa52w/UOj6/m65CwcmzyD9wDO7dO1btQTotgp+eiJyNO2A4fsa6u9mjE5A2+0+YM7NrHZdNjDdeBcOJwhi7VRxj2g8LURATKyuXHj06o/kzk2DOzUPu1t11GpOr6tWrF2bMeBW33DLOus/HxwdpaWk27cR9X19fu+eorH11z+eKXOk66jzdUZCZZ7OvIDMXOq+yyUJpra67GOnH45B2OLZOzld+jHoUZNme05iVV+45q9Per2M4Oj5wBXY9P7cW8VnOW/L7NhbeFkmXMTu/Rm2Fbs/fgOyYZMStqf3fc7IseVHfy16oXGbD9bs7DSfPIXjqXWWOiYQm7tX/weeyPmg1eyaC7r4Bmf9sgakw0XGLDEXo9IeQ/NWvODVmMs6Mf0aO/Qq65/paxaQLD4bvyCHVq3iJ5Oy5B2DONyD505+su70v6yure9lriic21AURo8/IIUj7vmoxGo6chJqTB5jMyNt1EFnL18Nr8CVoSm6//TZkZqbJbf/+4gpMt27dsGzZYjz66GSsXLnSuj8rKwv+/v425xD3MzOL3yCUVFn76p7PWbnqdYy6uieuW/eS3C7/5TEYc/PLjA8TY7WMObaJQnkJWsnqmVCb8xWJGNETV615WW6D5z4OY66h7DlFMlPOOava3rddKPp8dDcOvPcXkv49XuX4yj6f5bwln1PnY0nESidc1WkrJzm0CsbOp35qHCPNyWU1yQqaDZ0WbpHBdg/lHzyJ2GnvWe+HPHsf8vYdk7f1rSNgSkpF9qZd8r4pNQNZq7Yg4OYRSPl+YY3DcY9uD22AHyL+b4a8r+i0UDzdETX7fSS8+ikMR0+XiT/42UlQdDokzPifzWxUj16d4d6xjXysPJe7HopWQdRP7+DcHU/XOsbwohi1lhgjZ7+PRHsxliJmUDU1c+bMlVtJIqlYuXIFnn32ecyePcfm2N69+9CrV0/rfZ1Oh+joLti3z34XemXtq3s+Z+Wq1/Hc8j1yK9JqdG+0u62/TZuAjuE4PntThecJvrQdPJr5ImZZ8bmEjGPxNTpfSWJWo9iKiPFZbcbZTnry6xiBU3M32n185rELlbYXydmln92HI58vR+zy2lXQRQUsNz4Nvh3DrWPHxPPlxqWVTdCq2Db6qdHw7xqFfx/9tsw5iOpbk6qgiS5Lnyv7yxmaglvrCASOG4ncHZZp6qXp20aJv8ByKQnfEQPh0b0j0heulsfyj5+FNigAXv17ylmSGj8f+Azvi/yTtZvWnrNxO87f/yIuTJ4ht+RPfoSamy9vG0qfW6tB8DOT5PeV8LpIziwDhoukfr0AsQ+9bD2XqFzl7T2KC1PerHWMsZNeRNzkGXJL/tQSY5ydGBVvT3j07gbF3U3OfHXv0Rm+Vw9BzuadNt8H3HRQNBq5idtyXyMWHR0tk4oXX5yOWbN+KHP8559nY/jwYRg5ciT0ej1eeOF5JCUlYf369XbPV1n76p7PVbjqdYxdexCeIf4yURNvwsRXkXiJ/RUR7USb0l2JNT1fReLXHYRHiL9M1MQ5xVf35r6IW3ugRu192oTI5Ozol3/j3OIS//9r4fzinXImpj7IR25t7xqKc39ur1FbkZyJWZ7/PfadtfuT6oZZbZjN1TWtCpqqyi7LZvfdCMVNB1NalqyApc7+Sx4Oe+1R5O0/jrT5llmJ/mOGwat/LznLKO/QSVx47iPr8hDG+GQkzPwWgeOvQcjUu2A2FCB31yEkf7WgdiHmF8CUXzzGxZSRJeOWMzVFFe+Vx5B34DgyFiyDe5d2Mj7RtdlijqVKJqTPXy6Pm7NzALEVMufkQjUUWM9VVzGa021jDH7lMeQXxiiqa/63XQu3FhOt1y312wXI3VT8B1pMiPC5fID1vu91w5C1ajNSPir7gttYTJs2FcHBwfjww/flViQ6ujtiYmJw9OhRTJhwJz7++ANERUXJ9bZGj77BOvh90KBBskvP1zdA3q+sfWXHXZWrXseCjFxsnfozej5zHXo+dS2yziZhy5M/WcdIeYb644r5k7Hy1k+QG59uXcYi4rIu2Pz4j9U+X02Ic26f9iO6PjUaXaeNRnZMErY/+aM1efEI9ceQeVOwftxHyItPr7R92wmDoQ/wQpcp18itSNHja+L4t6vh5u+FIb9Y1mA8v3w3TsxaK293fWaM/Hpg5qJK23qEBaDVzf1gyi/AZYuKexdEla/o8UT1TVGr0N8k1hYS4yzS09Ph5+cHZ3Vy1ENwBVpNzWYt1SfFRcZXtvzr/+DsFMX53wepqm311Vm5wrX8/ZJn4QrcXeHvEFzDyG2165VwpIZ8/S567iWXPg5vXc0nrNREtjEf1/z7sdPnLRVp3P1IRERERC6ICRoRERGRk3H+/gIiIiJyWVwHrWZYQSMiIiJyMqygERERkcM0xLIX5kawzAYraEREREROhhU0IiIichiOQasZVtCIiIiInAwTNCIiIiInwy5OIiIichhOEqgZVtCIiIiInAwraEREROQwZlWRW30/p6tjBY2IiIiavM8//xytW7eGh4cH+vbti3///bdB42GCRkRERE3aL7/8gqlTp+Lll1/Gzp070bNnT4wYMQIJCQkNFhMTNCIiInIYtYG26vjggw9w//3345577kF0dDS+/PJLeHl54bvvvkNDYYJGREREjVJGRobNlp+fX6aNwWDAjh07cMUVV1j3aTQaeX/Lli1oKI1qkkDbpV80dAiNxvkx9zd0CI2GqhobOoRGg9eyaVEU13iJUvFmQ4fg1MSq/vU9aF8t/CSBFi1a2OwXXZivvPKKzb6kpCSYTCaEhoba7Bf3Dx8+jIbiGr/9RERERNUUExMDPz8/6313d3e4CiZoRERE5DDmwq2+n1MQyVnJBM2e5s2bQ6vVIj4+3ma/uB8WFoaGwjFoRERE1GTp9Xr07t0bq1atsu4zm83yfv/+/RssLlbQiIiIqEmbOnUq7rrrLlxyySW49NJL8dFHHyE7O1vO6mwoTNCIiIjIYVRVkVt9P2d1jB07FomJiZg+fTri4uLQq1cvLF++vMzEgfrEBI2IiIiavEcffVRuzoIJGhERETXKSQKujJMEiIiIiJwMEzQiIiIiJ8MuTiIiInIYs2rZ6vs5XR0raEREREROhhU0IiIicujnYhZ9NmZ9PqerYwWNiIiIyMmwgkZEREQOwzFoNcMKGtWO3g2hX76B8Nkf2z2sbR6E8Hmf2mwRv3+JoBcesbZxa9cSzd96GuFzP0Ho/70Jz2EN99lnROS8Ro0ahXXr1iAlJRHx8bFYsOAXREZG2rQZM2Y0jh49hOzsDGzYsA6dOnWq8JyVta/u+YjqChM0qhW/28fAmJBc7nFTUgoujHuseJswBebsHORu+E8eV7w90Wz648hZtw0Xxj+OlPe/RsD946Dv0r4evwsicgX+/n6YOfNdtGjRGm3atEdGRgbmz59nPd6xY0fMnv0TnnhiGoKCgrF69RosWvQ7tFqt3fNV1r665yOqS0zQqMZE5cvjoq7I+n15lR/j2fciKIoGuVt2yvv6zu2gFhQgZ/k6WZMuOHoKuVt3wevKwQ6MnIhc0dy587B06VL5IdY5OTn46KNP0LfvpdaEacKE8VizZi2WLFmC/Px8zJjxOkJCQjB4sP2/J5W1r+75qOJJAvW9uTomaFQzGg0CHrkTaf83B6rRWOWHeV05SFbLUGB5jEjWoNj+R1IUBW6tbbstiIhKGzp0CA4dOgSTySTv9+jRHbt377EeNxqNOHjwkNxvT2Xtq3s+orrEBI1qxOeGESg4GQPDwWNVfow2OAjuPbog+58N1n2GIyegcXeH96hhgFYrK2oe/S6CxsvTQZETUWPQq1cvzJjxqux+LOLj44O0tDSbduK+r6+v3XNU1r6656OKJwnU9+bqmKBRtWnDguF99VCkz1pQrcd5XT4QBafOwnj6nHWfOTMbya9/Cs+hlyJ81nvwu/Mm5KzaBHNmlgMiJyJXcvvttyEzM01u+/cXV7K6deuGZcsW49FHJ2PlypXW/VlZWfD397c5h7ifmZlp9/yVta/u+YjqEpfZoGpzj+4AbYAfQr94Xd5XtFoonh4I++kDJM/4VI4jK0NRZIKW+duyMocMh08g6ZmZ1vuBT01C/v6jjv0miMjpzZkzV24lieRs5coVePbZ5zF79hybY3v37kOvXj2t93U6HaKju2Dfvv12z19Z++qej6gusYJG1Za7cTviHnwBCVNmyC31sx+h5ubJ2wUnz9p9jHuvaGj8fJC7/t8yx9zatBB/+eSSHWJygHu3Tsj6q/hdMRGREB0dLZOzF1+cjlmzfihz/OefZ2P48GEYOXIk9Ho9XnjheSQlJWH9+vV2z1dZ++qej+xjF2fNMEGjalMNBpiTU4u3jEwxTUfehtGEZtMnw+fmUTaP8bpiEPI274Cak1vmfN7XXY7wH95H+I8fwHNgbyS9+D7MKen1+B0RkSuYNm0qgoOD8eGH71u7PsXWokULefzo0aOYMOFOfPzxB0hLS8aVV16B0aNvsE4iGDRokGxfpLL2lR0nciRFVdVK80yx1ozod09PT4efn59DAyLncH7M/XAFkYu+bugQiMhBFMU1RuGoatVnste3hnz9Lnrur3s8Ay+te70+d44pH/fvnenSeQsraEREREROxjXenhAREZFLUhtgTJjKMWhEREREVNeYoBERERE5GXZxEhERkcOYC7f6fk5XxwoaERERkZNhBY2IiIgcRlUVudX3c7o6VtCIiIiInAwTNCIiIiInwy5OIiIichhOEqgZVtCIiIiInAwraEREROQw5gb4JAEzP0mAiIiIiOoaEzQiIiIiJ8MuTiIiInIY0dtY3z2OKlwfK2hERERETqZRVdDOXvcAXIFW4/wTgDVa549RiL/lXjg7rcb538sFz/8JrsD0/UNwdkpYIFyCtyecnaoaGzoEqrNJAvW7sr/Z+f/sVooVNCIiIiIn06gqaERERORcOAatZlhBIyIiInIyTNCIiIiInAy7OImIiMhh+EkCNcMKGhEREZGTYQWNiIiIHEYs2lTfCzeZ4fpYQSMiIiJyMkzQiIiIiJwMuziJiIjIYVTVstX3c7o6VtCIiIiInAwraEREROQwKhSYodT7c7o6VtCIiIiInAwraEREROQwHINWM6ygERERETmZJltBU/RuCPt0OrR+Pjh32xN224S8ORXundtCNZqs+y48OB2mlHTLOTw9EPTIeHj26Q41vwCZS9Yg45eldRek3g2hn7wCja8PLox/vMxhbfMghHz2apnvK2/HPqS88bm879auJfwnjoNb6yiYM7KQMe8v5K7ZUuvQ/B+9F56D+0I1Gq37Ul79AAVHT9h/gFYLv3vGwnNIPzE4ALkbtiLju3mA2bKcoDY0GH73j4e+Y1uo+QZkL1mJ7IXLaxWj3yP3wmNQP5sY02a8X36MRfRuaPb+a9D4+iLx7keLv+cnH4Zbp/ZQ3N2hZmUhd9UGZP++uFYx+jx0H9wH9QNKxJj++nswHrMfY7MfvrDdodPBdP4C0p6eXqXj1TFq1Cg888xT6N69GwoKCrB+/QZMmTIV58+ft7YZM2Y03n13JiIjI7Fz5y5MnDgJR44cKfeclbWv7vnQeQiU9n2BwHDg/CGoq7+27PfwgdLnRiCsPeDmAWQmQd29FIjZX/65PP2gDLwdCG0P5GdD3bMCOLbZcswvGErvMUBIG0CrA1LjoO5YCCScqvxCtrwUiLwI8A0FEo8Bu+YWH+s1FghoCejcAEMucG4ncHJd+edq1g7oeCXg3QzISwcOLweSjlsPK1e/BtVkKC4f5KQCm/9XeYzhvYHQ7oB3MJByEjj0W9k2bl5A70lAfgaw67vyz+UXBbQZDng1A0wFQMI+4HSJ76nPQ4CbtxwlJKlmYMuHlcdI1MQ02QTNf/xomBJTZIJWkbRZfyDzz1V2jwU+MA4aH2+cv/c5aP19EfL6EzAlpCB7zdY6idHv9jEwJiRD72s/RlNSCi6Me6x4h06LsO/fRe6G/+RdxdsTzaY/joy5fyLnhXfh1r41mr8yBaa4RBgOFf9Rr6mcFWssSVYV+Nx8LfRdOiDx8Zfk/aAXp8DnpmuQteAvQKMg8LnHkP/vLqS+9Sm0oc3R7OUnYUpORd6GbbWOMWtWiRfEqsQ69nqYEpNlglZS9oJFMMbGy2RK0zwIgS9MhSkxCXkbavfzzvt7NbJ/qFqMyXc9ZHM/4J3XkL95W5WPV4e/vx9mznwX69atg6qq+PTTjzF//jwMHDhYHu/YsSNmz/4JY8fejpUrV+L555/DokW/o2vXHjCZit/UFKmsfXXPJ+WkQ927Akp4J8A7oHi/zh1qyjlgx5+yDaK6Qhl6N9TF7wHpcXZPJY7LRO6X54GAcChXPQw1IwGIPw7ovaCePwhsngcYsoH2/aFc8RDU316VyVyF8jOBE+ssyZWHn+2x42uA7GRANQEe/sAldwC5qcCFvWXP4xkIXDQO2LPAkugFdwB6jQM2fW55TJGt3wCZ9r/HchkygZhNQEAbQG/7e2/V7iogOx7QeVZwIgWIvhk4tw3Y8xPg7gf0uN2STMbtLm52ZBGQfKx6MZLL4icJ1EyT7OIUVSWPi7si47eaV2gUdzd4D7kE6T8vgpqdC2NsAjL/WgPvqwbWXYwXdUXW71WP0bPvRVAUDXK37JT39Z3bQS0oQM7ydfKTYwuOnkLu1l3wutLyAlufvC4fhKxfF8Ocmi63rF+XwPMKSxy6iDDoIsOQOf9PwGSCKTYeOas2wuvKIfUep65tK+h7dUfOwmVljhnPni+udIkKhdkMbXgoGoquXRtooyKQv25TjY5XZu7ceVi6dCmys7ORk5ODjz76BH37XgqtViuPT5gwHmvWrMWSJUuQn5+PGTNeR0hICAYPtv/7VVn76p5POrsHOLsXyM+y3Z+VDBxYDeSkWSo15/YDItkKbm3/PL7NgZB2UHf8BRgNQNIZ4MR2KB36WY6L+0c3W55H/OxFZU1UfgIjK7+Q8YeAhMNAQU7ZY1kJluRMKhyoI6pj9jRvD2RcABKPWtqKr+nngcheqLXko5aEyV6MQlAHwM0TiK+gAino3KGIdqJqJmLMTwdSTwPeIbWPkaiJaXoVNI0GzR67A6lfzgWUyqfh+o0dBb9x18CUkIzMRaus1TGRUChubjCcjLG2LTgVA/9br66TGAMeuRNp/zdHVpeqyuvKQchZtw0osCQRIlkr/T0qigJdqyq8qFSB59ABcjOlpiF39UZk//WP3ZGZireX7I4V16dIwemz0AU3g+LlKb/fwpYlA4VbqxZ1EGN/uYmkUMSYs+Tv8kePajTwe+BuZH7zc7m/G74TJ8DzsoGym9OUkITctTVLfkpyHzJAbiLG/LUbkFtRjCUfN3wICnbvgzk1rUbHq2vo0CE4dOiQtZrVo0d37N69x3rcaDTi4MFDcv/atWvLPL6y9tU9X7V4+AD+oUBqcfesjcAIIDcdyMu07lJTzkPpPMh++4BwS9dp+gXUWvS1MslStHqoohJ2fpf9dvJ3Uim7z6fUm4TeEwCNFsiMB46uBNLP1S4+rTvQ9nJg/y+W7suKGPOgxu0BQnsC57ZYKmiBrYHjK2zbtR8JdBhlqfyd3QSkVjLsgFyaWbyfredB++ZGMEmgySVofjdeBcOJs8g/cAzu3TpW2Dbth4UoiImVY6I8enRG82cmwZybh9ytu6HxcJe3i8ZQCebsHDkurbZ8bhiBgpMxMBw8Bn0lMRbRBgfBvUcXpM/61brPcOQENO7u8B41DNkr1kPfoTU8+l0Ec3rxi1BNZS9dicwf58OclQ239m0Q+OSD8n9E9uJ/yrRVPNyt16eIWnhbXC/j+TiZ7Pjedj0y5y6ELjxEVtwUr9pdyxwR408L5Hgxt3Zt4D/1IVn1yFlSNkbBa/TVMJ4+i4JDR+EW3cluG5G8ZX47G7o2reDepxfUrEq6tyqRu3wlsn+eL2PUtW8D3ykPQzWryFv6d8UPdNfDfcClyPr8m5odr6ZevXphxoxXccst46z7fHx8kJZmm/yJ+76luoar2r6656syjRbK0HuA07uA5OI3CTZ07pYxYCUZcixJWGl6T8v59v4N5Nb+/xIOLgYOLoHqFw6EdAYK8uy3Sz4BdBphaSOqZ8EdgYAWQMppaxP13++B1LOWNz0t+gB97gQ2fm7pYqypNsOA+H1AXmrlCZqQeAjoOApoNUi+SVRjtwOpJ4uPH/kLyIqzvAlp3gnocgOwdzaQVQfJLlEj0qS6OHXhwfAZOQRp39sZAGuH4chJqDl5gMmMvF0HkbV8PbwGXyKPmfPyobjrS1R/IKtBqkjaakEbFgzvq4cifdaCaj3O6/KBKDh1FsbTxe+WzZnZSH79U3gOvRThs96D3503IWfVJpgzS3UH1YDx5Fk56cDSdXoSWX8sg8egPnbbqnn58qtGVMsKKV5elmPieplMSH37M7i1aYnQb95DwJT7kbN6Y63jNJ46CzUj0xLjsZPI/mOpTFrs0YaFwOuqy5D50/zKT6yqMJ48LWP3vXNsrWI0nToDNTPTcs5jJ5G7aEm5MZbk3q8PkG+AYeeeGh235/bbb0NmZprc9u8vfly3bt2wbNliPProZDk2rEhWVhb8/f1tziHuZ4rvx47K2lf3fFVOzi67T3ZbqpsrGOdnzJeJlw1xv3Sy5OYB5cqHgYQTlkkHdUYFMmItcYgkzB4xVk2MP2s/DBj2NBDVG4jbb9stmXLK0mUqBuef3gxkJVnGqtWUSMjEJqphVeEZBHS9GTi5Etj4DtStnwCezYDWlxW3yTgHmI2WOBMPAinHLYkaETXdCpp7dHtoA/wQ/n8z5H1Fq4Xi6Y7I2e8j8dVPYTha/E7UHjFQuoio+ojEwq1NFApOnJX79G1bwHA6tpYxdpAxhn7xeokYPRD20wdInvGpHEdWhqLIBC3zt7LjpgyHTyDpmZnW+4FPTUL+fjGGpY6VqCSWJqplYkKDSMBM8Ylyn1ubFnIgvppjqVoYY2KR8toH1sf43nEzDAfqOM4Kug3dOneAxt8fzT9+07JDp4Xi4YHgbz9G6lsfw3i8RAWgiFZb92PQqliX9xg+BHlibFk5172y4/bMmTNXbiWJ5GzlyhV49tnnMXv2HJtje/fuQ69ePa33dTodoqO7YN8+++OUKmtf3fNVLTm7V/6c1FVfA+ZyJhoIqbGAp7+lKzTP8sZACYoCUi/YJmdXPQykXYC65ZeaxVSFmMsdgyaIsWxiK9JvUvldolIt+3kCWgMeAUDfwslIihbQugH9Hgd2fAMUlKogi1mgYlJEUuHMW3E8YT8Q1Q84XV43dSPoi6IKqQ3wU1bh+ppUBS1n43bETnoRcZNnyC350x+h5ubL2yXHkhXNgPTo3U1OBhDjwNx7dIbv1UOQs9kyAF8sq5GzYTsCJoyWXXGiW8732mHI/ntjrWLM3bgdcQ++gIQpM+SW+pmIMU/eLjhpSQRLc+8VDY2fD3LX/1vmmEiExFILYukIMTnAvVsnZP1VXAWpKY8Bl1i7c93atYL3jaOQt2VHue1FRczn5mugCfCTm5jBmbNyg/W4rlWUpSKp08Kj78XwHG6ZVFAb7v37WGPUtW0N7+tHIX+b/RjzNv+HpMeeRfJTr8gt44tZUPPy5G3j6TPQNG8G9769Ld21Ynxcx3bwGnUF8nfXMHkopO9nG6PnmGtg2La9wsdow8Og69geeWs21Oh4VUVHR8vk7MUXp2PWrB/KHP/559kYPnwYRo4cCb1ejxdeeB5JSUlYv3693fNV1r6655PEOEux7IX4KsZnidsiyVE0luRMzOYUS2+Iik1FMpOAhJNQLr7OkoA0bwW0vQTqsS22lbP0RKibqjcrWMamKYxRjBmTt7WWWZuh0YBWb4lddFe26muzbEYZfhGF37MeaHeZZeB+bOHsSJ8QQHSTFj2fOJfYV9H5ioO0xFQUY9Ht8/8C2/8P2PmdZTuzAchJtty2N6FAzB7V+wDNCqt2Ir6QbkBWvOW+GJPm16L4/M07WyYgiEkKRNR0K2giqTLlF49xMadbZmSZki37gl95DPkHjiNjwTJZufK/7Vq4tZgojxnjk5H67QLkbrIkaELKl/PkOmiR38+EajAgc8naWi+xIc6jJhuKYxRddGKAZbJlGn2z6ZORf/A4sn4t7l7xumIQ8jbvsFajSvK+7nI5uxNajaWa9uL7MBeu41YbXiMvh/9Dd8kuXnNKGnKWr0H2n8XjpvweuEN+zfi/n+TXrAWL5XpuwZ9YKoO567ci67cl1vYeA/rA+2rxguMG4+kYpM78DMYztRvc7HX1cPg9cJf83mWMK1Yj56/iwcq+91tizPz6J8BggDnFznVPKV6+wOuaK+H30D3yBUwMvM9dtgo5C2vXzeV59eXwmXQ3FK0GppRU5P2zGrmLi2P0nnin/Jr9zY/Wfe7DB8N4+CjMcYUveqVUdryqpk2biuDgYHz44ftyKxId3R0xMTE4evQoJky4Ex9//AGioqLkumWjR99gnUQwaNAg2TXq62tZ/qKy9pUdt0fpOQJKr1HF9+/4EGrcMai7lkJp2QOq0QBl3NvW46oYN7bP8nuqjHkeqrh90pIQq+tnQRlwO5Rxb8nxZ+r2RZYlNoSWPaCEtIEaGAGlVXGVT90yz/r4crUbCkV0Sxa5ajpU0RW593egVX+g2xhLUiQmKJzZBpwskVgPfBQ4ub542Q2xBlpApKU8IMakiTFnojtT0HtbJhyIxE8kpCIp2v4TkFuFSSItB0JpVWK27KCnoaadAfbNAcS6akWMeZbZq2JZjiIXTwRitgCJByyzNg8vBMS5Ol5rqVqmnbJ0eQoi+W13pWXJEHGe3BRL+8za9TyQc+MkgZpR1JL9duXIyMiQY0HS09Ph51dqHR8ncva6B+AKtBrnX6FFo3X+GAWN1vn/F2o1zh9j8HxLIu3sTN/brvPmjJSwQLgE74rWM3MOyuDnGjoEl9eQr99Fz/1qh+fgoa39BLrqyDPl4eVjbzl93lKRJlVBIyIiovrFz+KsmSY1Bo2IiIjIFTBBIyIiInIy7OIkIiIih+FncdYMK2hEREREToYVNCIiInIYLrNRM6ygEREREVVR69atoSiKzfb228XrLdYVVtCIiIiIquG1117D/fffb73v6+uLusYEjYiIiBymMX4Wp6+vL8LCwhz6HOziJCIiokYpIyPDZsvPz6+T84ouzWbNmuGiiy7Cu+++C6Oxks/7rQFW0IiIiKhRThJo0aKFzf6XX34Zr7zySq3OPXnyZFx88cUICgrC5s2b8dxzz+HChQv44IMPUJeYoBEREVGjFBMTY/NZnO7u7nbbPfvss5g5c2aF5zp06BA6d+6MqVOnWvf16NEDer0eDzzwAN56661yz18TTNCIiIjIYVQocqvv5xREclaVD0t/8skncffdd1fYpm3btnb39+3bV3Zxnj59Gp06dUJdYYJGRERETVpwcLDcamL37t3QaDQICQmp05iYoBERERFVwZYtW7Bt2zYMGzZMzuQU95944glMmDABgYGBqEtM0IiIiMhhxHj9+p4koDrovGKM2bx58+REAzEjtE2bNjJBKzkura40qgTNZK7fPu6aavnX1w0dAtWjrEl3wNmpat1PEXcE85cPwdkpV78GV6Bu/xDOznzENf5WajoVL1hKjdvFF1+MrVu31stzNaoEjYiIiJwLP4uzZrhQLREREZGTYYJGRERE5GTYxUlEREQO0xg/i7M+sIJGRERE5GRYQSMiIiKH4SSBmmEFjYiIiMjJsIJGREREDqMW/qvv53R1rKARERERORkmaEREREROhl2cRERE5DCcJFAzrKARERERORlW0IiIiMhhuFBtzbCCRkRERORkmmwFTdG7IfLzl6Dx88HZsVPLHNf4+6LZ/bfAo3sHaLw8UHAhEWmzFyNn215rm2aPjYdHtw5wiwhByje/ImPR6nr+LsjVeX/yte0ONx3MF2KR+9oL9h+g1UJ/63i4XTpAvkcs2LYZhvmzAbNZHlYCAuF++13Qtu8o7xsPH0T+nB+ArEw0Sl2HQunUH2gWAZw9AHXF/1kPKX2uA1r3BALDgP3roG5eUP55/EOg9LsBCG0DaN2A1AtQt/4OxJ0sbtP+Eii9RwE+gUBqHNQN84DEM2gUgnsCzbsCns2A9NPAib+Kj3W6GfAOB1TL75i0fxZQkF32PHpfoOudtvs0OiD9FHD8z+J9zbsBYb0BN1/AmAPErAXSSlzr8gREQ/HvCOiDgOwYqLH/2B737wQlqAeg8wZMeVATtgBZdn5GnmFQoq623afogLQDlscoGihRIwF9IKBoZYxq6j4g/XDlMRLVkSaboAVMuA7GhBTo/XzsHtd4uiP/ZAxSvv8dppR0ePbphpBnJiJ2ytsoiLkg2xhOnkP2+u0IvHNMPUdPjUX25Ptt7ntOfwPG/7aW214/aoxMvnJeeUbe95j8FNxGjkbBkoXyvkjO5Hmfe0K84sBj4kNwH3cH8r/5HxqlnHSoO5dBieoMeAfYHFLTE4Gtf0DpMrDy87h7Qj17AFg3G8jPBjoPgDLyUahzXwLysoGwtlAG3w51ySeWpKzzQCijHoE6dzpgyIPLE8nWhW2AX0vAzc7fxHMbgYRdlZ/HkAns+rz4vqIBekwCUo4U72veHQi9CDixFMhNBHRegMatanGKRCl5FxSvSEsSVpJ/ZyiB3aDGrgbykwGtpyU5tCc3DuqxWcX3tZ5Q2t0ONeOE5b6qQo3fDBjSLJ1l+gAoLa6BKu7nxlUtVrLiJIGaaZJdnPr2LeHVuyvSf11RbhtjXBIyfv8HpuQ0+Z819999KDgXD/fObaxtMpesQ96eI1ANxnqKnBozTeu20IRHwrh5Q7ltdAOHwLBkEdT0dLkZlv4Jt0FDrceV5sEwbt8G5OcD+Xkw/rcNmsgWaLRO7QZO7wHyssoeO7oViDkAFFQhgUo4AxzaaDmPqgKHNlkqRkFR8rAiKnHieRJOFx7fCBTkA216oVFIOw6knQCMuXV73oD2gKIAqccLdyhAZH9LxUwkZ4KooBnSq3a+rNOWipip9M9UgdK8t6X6JZIzwZQLFFSxcuzfwRJDXkLhDhUwpJYdyeTmV7XzEdWBpldB02jQ/LEJSP7fXHm7yg/z94VbizAYTp1zaHjUdIlEy7R/L9R08a7dDi8vaIKawRxT3GUjbmuaNQc8PYHcXBT8sxy63pfCuG+3fNHSXdoPpr1VqHyQraAIwM1DdnVaKJZEoyRFgRIU2SgGI1cqvC8Q0Q8wZADxO4HkQ1V7nOg2FW1Vk+W+RyAUN2+oXqFAqyssFTbRpRqzHjAbah6f3h+KzguqR3MoYYMtP6/sc1ATtwLmgkofrvh1gmqn+1KJHAF4RUDR6KDmJVsSRKo28Z5GbPX9nK6uySVo/jddCcPJGOQdOA6P7pZxOpXSaRHyzH3I3rADhuNnHR0iNUV6d+j69EPed8VjqEpT3D3kVzU3x7pPzcmxHlNzc2E6cRRugy+D94dfyv3mk8dhWFZi7A9VTu8J5Yr7gF3LgdwMuUt0fyojH5RdnbKK1mWwZSya3hON3rlNgEhOzEbAtwXQ9hrAZLBU3CoixqOJLtNzJSrCOsvvsNx/aI7ltjhfi6HAmVLjyapD6y6/iK5P9cwfltvhl0MJ7g81fn3Fj/UMs8SacazMIfW86GVRoHqGAl5iHB57S6j+NKkuTl14MPxGDUHKt79V40FahDw/CWq+AUmf/uzI8KgJE1Uv1WCASVa+7FPzLd06iqeXdZ8iKmdFxxQFnlOegenEMTm2TWzitthHVaT3gHLNY0DcCajbFxfvjz0CddOvUIZMgHLnTCghrYBzh+13rTY22RcsCZno8s04AyTtA4I6Va16lpMI5CYV7zMVVrMu/AsY8yybuB3QtnYxFlbJ1JTdgClfbvK2T8tKH6r4dyqn27SIKsedKWJMW2DP2sXZRJkbaHN1TaqC5tG1PTQBfoj66jV5X9FpoXi6o+Wc9xD/6mfIP3K6bHL23CQoOh3iX/sCMBaW6YnqmNvgoZaxZ4WzMe3KyYE5JRmaFi1hSrSMldG0aAVzSpLs3oSPDzTNg1GwagVgsHQXFaz+G/oR18hjyGoCyUStk7PJlhmc6wurOyUd3gT18CbLbY0Gyu2vQ93XBGduV7XvqFlXIO4/2315KVBFJa6uGdJrdl4xOcG3DdTzKytvK2Z26v2aRpc2OYUmlaBlb9iO3F3FYyfcu7RF8OQ7cP6x12FKLzWYVKtByLP3Q+OhR/wrnwNGO//5RYInxqFoFLn8geKmg2oyV/wiS1SKEhoGTdsOyJtVaskNO4yb18uZnHnHLd0x+lGjUbBhneVgVhbM8XFwG3YlDH9ZunncLrtCJnWNNjkTY5jEWFKxFIK4rdVZEgizqXB/0aYUHivn/6ebB5RRjwFp8VDX2qmUi3MFRgDJ5wF3Lyh9xwCZycDZg2gcxBg70aFSeK3E9RSpiEhgfMKBzHOWa+obBQR3B85UktD4tQJ0nkBKqXFdYiyaGJMW1gfIKRyQL25X1l1aOk6lRJxygJMJyDgOJagn1DxLxU7ctrvERkm+7SwVt5xSY4vdgyyzQMWMTfE74x0F+LWHGlf+BB6iutakEjQ1vwCm/OIB2Ob0LKiqapmpCSD01Ufl2LT0+cvh0aUdvPv3gjnfgJZz37M+Jm3+cnlcCJvxODx7WMaxifXQmt13E1JnL0banBJdI0SVcBt0GczHj0JNiC9zzH383fJr/mzLkgBiBqfe2xder86U9wu2bUJBiTFmuf/7EO63jof3Ox/LFzFTzBnkff4hGiul90gol1xbfP/+T6HGHoX654dQhk6wrJFWdKz7MKhHtkBd86Pl/q0vQRXjzI79J2djKmFtoTaLhNK2eGamrKSJ4xotlGF3Av7BgMkInNoDdZlYuqSR1FMi+kKJKL5W6D0ZamYMcGIJEN4PaBtk2Z+fYRnQn1pivFaH64HM87bVMrHOmWgjukZLEzM4Ww4Hut9rSazE+mcxhW8yKqE0u0jO1rTe73gv1JxYqDFL5AxOJXQglLbjLOfNOgs1oXjJGqX1zXKJDmSesO3eTC+xBIiVBkrzPnLygVSQaTlXicdS1XGZjZpRVJGhVCIjIwP+/v5IT0+Hn5/zTjM+dc2DcAVtllgGcFPTkDXpDjg7n69+giswf/kQnJ3mwS/gCtTtzp+4q77216l0NppOtusZOpOGfP0ueu4Hop6Du6Zwgkg9yTfn4f/OveX0eUtFmlQFjYiIiOpZAyyzgUZQQWtSsziJiIiIXAETNCIiIiInwy5OIiIicpiGWJfMDNfHChoRERGRk2EFjYiIiByGn8VZM6ygERERETkZJmhEREREToZdnEREROQwnCRQM6ygERERETkZVtCIiIjIYcQnSlbhUyXr/DldHStoRERERE6GFTQiIiJyGLNq2er7OV0dK2hEREREToYJGhEREZGTYRcnEREROYzobazvHkcVrq9RJWh5Bje4gv2XT4azM0OBK+ix6mM4O42b8/+pyH1iAlyBLsj5/2RpFeePUVBVI5yd+dychg6BqMG4xl8SIiIickmcJFAzHINGRERE5GSYoBERERE5GXZxEhERkcOwi7NmWEEjIiIicjKsoBEREZGDl9mo58/ihOtjBY2IiIjIybCCRkRERA7DMWg1wwoaERERkZNhgkZERETkZNjFSURERA6jqpatvp/T1bGCRkRERORkWEEjIiIihxFLbJjrfZkNFa6OFTQiIiIiJ9PkKmihj9wM3wE9oPH2gDk3H5nrdyH+60WA0VSmbfBd18BnYA+4twxF6qL1iP/id5vj7X56BbpAX+t8XtVkwtEbnqlVfIqbDuGP3QyfiztB6+8NY1I6En9ZhbTlW+229+jQAuGP3AiPtpEwpWch4cdlSPvnP5s2zW+7EkHXDoDO3wcFyek499ZPyD18ppZxahHx2M3wLYyzoDDO1OXbyrR1CwlEx++es9mn0euQue0gTr/0jbzf9v1H4RXdRl7DIkfueh3G5Aw0Zp4fWL5/K50Oalws8t583m57/R2ToL1kAGAyWvflf/o2zKeOW+9ru18Mt2tvghIcCjU3F8Zlf8C4cXXtg/UPhP6mu6Bp20muAmk6fhAFv84CsjPLttVo4Xb9eGh7D5SDQUw7N6Ng4c+A2SwPu914JzTdekPx9ALycmHa8y8K/poLlPj510hgMLRX3wElsh1QYID5379h3rLMblPtHc9CiWoHmIuf0/j5s0BWGuAXBN1Db9o+QOcG9fhemH75uNphjRo1Cs888xS6d++GgoICrF+/AVOmTMX58+etbcaMGY13352JyMhI7Ny5CxMnTsKRI0fKPWdl7at7PmeneHeE4t0GcAsA8mJhTt5gOaD1gib0mlKNtYVt1pc9URXbK17toPh2ke1hzoM5bQeQV/zzInK0Jpegpf61AQnf/gk1zwCtnzciX7oXzW69AslzVpRpa4hNRMLXixAwakC55zv/5g/I2ry37gLUamBMycDppz6H4UISPLu0Ruu3HoQxMQ1ZOw7bNNV4e6L1Ww8gftYypC79BJ4dW6L1zIdhuJCMnP0nZZvQ+66FV/d2lvPFJslkSbWTjFY/Tq2M86SMMxleXVrJOAtknLYvAgUJqThw7dPW+4pOiy7zX0Paml027eK+/hNJv69DU5I7daLNfY/n34Rxh/1kvIhx/UoU/Paz3WOa6B5wG3s3DD98AfPxw4CHJxQ//zqJVSRnQt5rUwBFgX7CwzLRKvjp8zJtdVeNkYlc3kzLGxb3SU9Bd8VoGP9eaPkeNq6EuvgXwJAPePtAf9dk6IZfC+M/i2oeoKJAN3YKzEd2WpKowGDoxj8FNTMV6n7719S8aoFM4srISIFx5oPF9zVa6J74COYDZd+AVIW/vx9mznwX69atg6qq+PTTjzF//jwMHDhYHu/YsSNmz/4JY8fejpUrV+L555/DokW/o2vXHjDZSVora1/d87kC1ZQDNeMAFI8wKFrP4gOmHJhjF5RoqYEm/HqoOeW8Ca1Ce8W7HRSfzjCnbAIKUgGNhyWJoxrhJIGaaXJdnIaz8TI5kxRF/hT1kcF226b/8y+y/zsIc05evcUnYkuYtVQmZ0LuodPI2n0MXt3blmnr1bUNzAYjUhdvklU8URXL2LgHgaP6y+NaXy80u3kYzr83RyZnRcmSSKzqIk6RGIrkTMg5dAbZu4/B206cpfkN7C6vffqGPbWOozHRtGoLJSwSpq123vVXkdu1N6Ng2R8wHztk+QuVmwM1/kKdxKc0C4Fp9zZLUpWfB9OurdCER9ltq710KAr+WQhkpMmt4J9F0Pa9zHpcTYi1nMdyZhmrEhxWuwCbhQPNwmBet9BSFUuOg3n3emguKn7emlI6Xyx/Z9VDO2r0+Llz52Hp0qXIzs5GTk4OPvroE/Tteym0WsuL/oQJ47FmzVosWbIE+fn5mDHjdYSEhGDwYEsCV1pl7at7PpeQd86ymYt+b+xTPKMsP6vcmCqdtmx7BYpfD0vFTCRngjkPMGXX9jsgqpYmV0ETmo29Es3Hj4DG0x1G0S34Tc3ftYdPGQtMvQ2G84lImr0c2f8erNNYRZenV+dWSF9d9oVB0ShQRJJZkkaBR5sIeVNU31SDEf7DeiPo2oFQjUakr92FhO+X1E0VrVScnp1bIc1OnKUFjeyHtFU7oBYUd9MJIROuQsgdI1AQn4rE39aW6apt7LQDLoP54B6o6WkVttP1HSQ3NSMNxi3rYFy93JKM6d2hadEamoBAuE1/F4qnJ0zHj8Cw4CdLolRLxrXLoO3ZF6aDu+WLmPbi/jAdsK2CSp5e0AQ2g3r+rHWXev4MNEHNZUVPdGnK7+Py66C7cgwUdw+oWZnIXzyvdgEW/V8o+X9CUaCE2k8iBc3g66AZMhpIT4Zp2wqoezfbb9drCMz7tgKmAtSFoUOH4NChQ9ZqVo8e3bF7d/EbFqPRiIMHD8n9a9euLfP4ytpX93yNiah+qTmnRVZVs/Y6X1mhU/RBUAIvlXUMNS8WavpOQLX9m0VVY67yT6Nun9PVNckELfmXf+SmbxkK/+GXwJhiZwxNFcTO/Al5x87K6pXvoJ6Imn4fzkz9GHlHi1+Yaity2m3IP5+IDDvVppyDp6B46BE0ZjBSFm+SiZzfwJ4wplm+H62fF7Q+nnCPCsaxu2bILt1Wb0ySY+8Sfy7bpVsbUdMsSWr6hoq7e0UXqxhfd+GrP232x327GHln4mDOM8Dnoo5o9dLdMOfkI2NTHXYfOzO9O3S9+8Hw45cVNitY+zcMf8wFsrNkxU1/32Py98+4ZjkUL28oGg20PXoj/7O3oWZnQT/uXrjf/RDyP3mr1iGaTx2Ftv8weLzxf5b7Z47DuPKvMu1EwiWoucUVBzU3x3LDvThBM676S25KSAS0vQdAzUivXYDJcUBaEjSX3QDz2j+AoBBoeg62PKe972f1AqhJsXKsmtK6C7Q3PQxTfh7UIzttG/o3g9KmK0yr5qMu9OrVCzNmvIpbbhln3efj44O0NNskWtz39fW1e47K2lf3fI2GGC/mHgo1fVfN22vc5RfFPQzmBMvfSU3QQCCgN9TUmnVxE9VEk+viLN3dmXfyPCKemlCjx+fuPwE1v0BWgjLW7EDW1v3wHdyrzuILf/xW6KNCcHb613Y71E0ZOTj74lcIuPwSdP71DYTePxqpK7bClGF5YRSJmJAwa5lMfET3ZvLv6+DbvxvqUuTjt8A9KgSnp39Tacd/0NV9kXv8HPJOxtrszzl4GuZs0Y1gRtb2w0heshkBwy5CU6G9+FLAYIBpv6hOlU+NOQ1kZcrrbD59Asa/F0Pbu5/lWH6eNYlTU5KB/HwULPkNmg5dZAJYK2LM2UPPyiQt79mJchO33R8sOymmKA7Fw6v44WIygJCfW7Z9QizMsWehv31S7WI0m2Cc/zGU0FbQTfkQuusfhHnPRiAny25z9fwJSzxmE9ST+2HeuRaarn3LtBNJnhp3BoivWpeZcPvttyEzM01u+/cXv7nq1q0bli1bjEcfnSzHhhXJysqCv7/tWEFxPzPT/pvHytpX93yNhaiGyW7JgrSat1ctVVJz5gFLd6o5X95WPCIdFXajJ8ZdNsTm6ppkBa0kRastdwxadal1+Oms4ZNvkQPvT037zJK4lCPnwCmcnPyh9X6LF+9G9l7LjL68E46fcRRRGOfJaZ9XGKekKAi8ui8S5hS/MDXqT7qtBt2Ay2DctsE6y7HK1BLtc3NgTrGMNSyjVE94tXl5QxMUjPz1f8uKk2Da8Dfchl8rB/mLip5NHKnJUCJbQU1OsDx9ZEuYU5Os1bMy4Wm1UJrXcgyakBgL05z3rHc1l98C9WwVZy7a/YOuQNNzEMybllQrjDlz5sqtJJGcrVy5As8++zxmz55jc2zv3n3o1aun9b5Op0N0dBfs27ff7vkra1/d8zUWildbqCKxqk37gkyo7MokJ9CkKmiiO9B/RF85+1Fwbx0ux6JlbT9k/wFajRxbJcZ6QWO5LfYJuuBAeHZvZ93nO+Qi+A7ojsw6mNEpk7NubXH66c9hzrL/glbEo32UJUa9m5wc4N2rPZJ/s4wxKYhLkTM/g+8cAcXdDbpmfgi6fggyN+1DXYiYfDO8u7XByaf/B1MlcQo+vTtB5+eNtDW249TEz8P30mgZoxhDJ7o4m103AOnrm8YkAiUkHJo2HWDcXPnYIO3FfS3juMR1a9kGuquug2l38Vg9sZyG22VXQfEPBNzc4DbyBpiPHJDVtFrJzoI5MQ66QVfI5SbEph10pUzEbJKzQqZ/18PtyjGAr7/c3K4YA9PWwu9P7w7tpUOAwgqbEh4F3ZXXw3ykDrqzQ6IAN72cdal07i2rX6YNtt3pkrsXlPY9AJ3eMk6tdRdoeg+D+dB2m2ZK266Aly/MByqeWVuZ6OhomZy9+OJ0zJr1Q5njP/88G8OHD8PIkSOh1+vxwgvPIykpCevX258wUln76p7PNYh3GZrCr0W3S7yEuYfL7slyZ2+WVm57E9Ts09D4RovBtXITt9Xcc3X63RBVpslV0PyGXYKQSTdA46aTY7UyN+xG4o9L5bEWbzyEnP0nkDzXMu0+fOrtCLiquMsj6PqhSPt7Gy68+7OcYBD2yM3QRwTLtbsM5xJx/vXvkXdIDDatOTFGq9mYwTAbCtBxzqvW/ekr/0PsR/PR6q0HkbPvBBLn/CP3N7thCPwG9ZDLXuQeOIVTT35ms3ZYzJs/InLqONkFKmajpq3cjsRfqlDBqkKczQvj7DznFet+cf7zH82Xy39k7ztpjdM6OWDDnjKVNkWnQeidI9CypWUZB0N8CmK/WIj09RV39zUWugFDYT5xBGpifJljbuPukV8L5n1vaTv0Suhvu1f+vNW0VLnkhnGV5fdXMP79FxRvH7lch2A6ehD5lYxrqyrDtx/A7foJ8HjlEzFDBebzp+U+GecthXEusMQpltOQcTz7jiWOHZtgXFmcKIkJBm6jb7OsLZaVAdOe/2Bc/lutY9REXwpN7+GW88afhWn+J0CC5YVVe9tUqGePwrxpsbx+miFjoNz4kOWBaUkw/T0X6iHbiSmai4ZY9tnpmq2OadOmIjg4GB9++L7cikRHd0dMTAyOHj2KCRPuxMcff4CoqCi5btno0TdYJxEMGjRIdo36+gbI+5W1r+y4K1L8ukHj1916Xxs1Dmp+PMyJqyzHvdtCzT1r7aIsSdP8Mqj5CVAziydxVdReTd8BBPSBJnyMWN8Dau55yyQBqhHRIVLfnSLmRtAJo6hV6KjNyMiQ4xfS09Ph5+cHZ3XoysfgCkzm2vY3OZ651n1i9aPHquovGlrfch6p2RjH+qTo4RJ0Qc7/nlI/3f4adc7GFbrxTOdsu4KdlTbqdjirhnz9LnruMYFPw61w8kV9KTDnY1HqO06ft1TE+f/aERERkcsyN8BncZr5WZxERERETccbb7yBAQMGwMvLCwEBlmEHpZ09exbXXHONbCMWiX7qqafkeoTVwQoaERERURUZDAbccsst6N+/P7799tsyx8VYT5GchYWFYfPmzbhw4QLuvPNOuLm54c03S33GbwWYoBEREZHDiM7Gev8sTjjOq69aJvDNmjXL7vG///4bBw8elGsdhoaGFi5OPQPPPPMMXnnlFTmzuirYxUlERESNUkZGhs0mPpvW0bZs2YLu3bvL5KzIiBEj5PMfOFD1dfqYoBEREZHDJwnU9ya0aNFCziQt2t56q/Yfe1eZuLg4m+RMKLovjlUVEzQiIiJqlGJiYuRSG0Xbc889Z7fds88+C0UsWl3Bdvjw4XqNnWPQiIiIyGHE+LP6XvRCLXxCsQZaVdZBe/LJJ3H33XdX2KZt27ZVem4xOeDff/+12RcfH289VlVM0IiIiKhJCw4OlltdELM7xVIcCQkJcokN4Z9//pGJovjYt6pigkZERERURWKNs5SUFPlVLKmxe7flYwnbt28PHx8fXHXVVTIRu+OOO/DOO+/IcWcvvvgiHnnkEbi7V/0TFZigERERkcM0tk8SmD59On744Qfr/Ysuukh+XbNmDS677DJotVosXrwYDz30kKymeXt746677sJrr71WredhgkZERERURWL9s/LWQCvSqlUrLF26FLXBBI2IiIgcxqw2QAVN5WdxEhEREVEdY4JGRERE5GTYxUlEREQOoxb+q+/ndHWNKkGLXvkFXIGqGuHstg15Eq5AUZz/V9gVft6ucB1d5VqqL1U8eNhZGE1r4ewUX5+GDoGowbjGX2UiIiJySaKWZW6A53R1HINGRERE5GRYQSMiIiKHaWwL1dYXVtCIiIiInAwTNCIiIiInwy5OIiIichhVbYBlNlR2cRIRERFRHWMFjYiIiByGkwRqhhU0IiIiIifDBI2IiIjIybCLk4iIiByGXZw1wwoaERERkZNhBY2IiIgcxlI/q99P41Tr/dM/6x4raEREREROhhU0IiIichiOQauZRl9BGzVqFNatW4OUlETEx8diwYJfEBkZadNmzJjROHr0ELKzM7Bhwzp06tSpwnNW1r6653NFoTcORNevpqDPypno8MY9Fbb1bBWKzh8+iN5LZuCiP15Gm2k3Q+PuJo/pAnzQ7qXxuOjXl3DJsjfQ7ZupCBjYtcZx8eddd3gtmxIFGqUjtJq+0GoGQau5FIoSVuK4FhqlS+GxAVCUVpWcr7L21T0foOhbQ+M9BBq/a6Dx6mN7UOMDjXd/aPyuhsb3KiiePeRzlKuS9hrvAZbn8Rtl3aC4VxojUV1q9Amav78fZs58Fy1atEabNu2RkZGB+fPnWY937NgRs2f/hCeemIagoGCsXr0Gixb9Dq3W/n/uytpX93yuypCUgdgfVyJh8dZK27abPh55ZxOwc8wr2Hf3e/BqH4HIu66Ux7SeemQfO48DD32C7aNexLnvlqP99PEyqasJ/rzrDq9lU6KI/9UwmffAZN4Ik/kwNEo7KAiURzVKB0Bxg8m8FSbzLmiUcChK+f9HK2tf3fMJqjkf5vyjUA1nyz6fV2+opiyYM1bAnLUWisYfinvH8uOrQns17xDMGUutG9T8CuMjqmuNPkGbO3celi5diuzsbOTk5OCjjz5B376XWv/oT5gwHmvWrMWSJUuQn5+PGTNeR0hICAYPHmz3fJW1r+75XFXq+n1I3bgfxvTsStt6RDRD0j87oRpNsn3qpgPwbBsuj+VfSEHcvLUwJKaLD09D2uaDyItJhE/Xyt9R28Ofd93htWxKzDCrpwHkFd7PgIo0KIq/fJlQlBCYzacAGAHkwqyel0mVfZW1r+75ChkvAMY4QDXYeUovqAXn5NBwcVw1xkHR+pZ/ruq2pzrp4qzvzdU1+gSttKFDh+DQoUMwmUzyfo8e3bF79x7rcaPRiIMHD8n99lTWvrrnawouzFuL5iN6Q9Hr4Bbki8DB3ZG2+YDdtqLLU1TPck7E1slz8+ddd3gtmxINFPhBVcUbMC8oinipyCo+rIrb3uU8trL21T1f5dT8E1DcWlhe0hR3KLpwqMb4WrVX3DtA43s1ND5DobhF1Tg2oppqUpMEevXqhRkzXsUtt4yz7vPx8UFaWppNO3Hf19f+u6nK2lf3fE1B2rbDaPvsWPRZ/iYUnRYp6/chccm/ZdqJY+1fmYDkNbuRfUS8u60d/rzrDq9l06JROkFFDlQkis5uqKpIyosrEqqsfJX38qGtpH1lx6tPNSZA49kLit8omfypBRfsdoVWtb057xBgygRgAnTNofG6BKpqtFTwqNrMhf/q+zldXaOroN1++23IzEyT2/79xe/Gu3XrhmXLFuPRRydj5cqV1v1ZWVnw9xdl/GLifmam+M9ZVmXtq3u+xk7r44nOHzyAhMXb8N9Vz8lxZuY8g5wYUDo56zDjLpjzCnDqnQVVPj9/3nWH15KKxocpiifM5v2Fe0yFLxVinJqFIpMpkVTZU1n76p6vMm5ywL9qOANzxhKY0pfJBFDxvLjm7U2phfGogDFRtlXcbCfIEDlao0vQ5syZC1/fALl169bT+gKzcuUKPPfcC5g9e45N+71796FXL0s7QafTITq6C/btK/rjhGq1r+75GjuPyGZyxmb8rxvkGDRTVi4S/tyCgH5dbJOz1+6UX4+9NEu2qyr+vOsOryVZkjM/mMx7CxMpIaew2lWiC1LxAVDe+NPK2lf3fJXQelmqcoZThectgGo4DcUttG7aC6rrj2ci19PoErTSoqOj5QvMiy9Ox6xZP5Q5/vPPszF8+DCMHDkSer0eL7zwPJKSkrB+/Xq756usfXXP57K0GjmmTBFfNYrltq7sbLvcswkw5xoQesNA+RiNpzuCr+0nZ24K4vHtX70TGg89jr7wPdSCqidn9vDnXXd4LZticuYvZ3LaVrPMUNUEaDRtCpei8IRGiYRZvVDOmSprX93zFVFKVN5K3DZliT5LuQyHZb8Wir4VYEq3f5pK2+sAXUjxshva5lDcW0MtqJtxsU2RqqhQFXM9bypcnaKqlb81ENPrRVdDeno6/Pz84KwUpewYhu+++wZ33XWnnIVWUnR0d8TExMjb118/Bu+88zaioqKwc+cu3Hff/Thy5Ig8NmjQINm9I6oKRSpqX5XjciyDk9s25MkKj0fecxWi7hlhsy9j13EcevwLdHpnIjL3nkLsz6vkfp9urdHywWvh2SYMqtmMrP2nceaThXIGp2/Ptoj+9BGY8wugmorHDIjHFj2+Iv02fGxznz/vmuH/nabHaFpb4p47dNr+UFWz7dgwNR5m9WjhumUdoSjNCmd8noeqnrG202i6Q1XToapF47gqbl/5cQslK6P4tnsnaDxs18VTjUkwZ28GtEHQeHQBtH6WapcpBebc/YBq+d3VePWFakqBmn+s8OkraK/oZXtoRVVPhJcrJxWoBZbfeXu0/qPhrBry9bvouS/xnwSdoq/X5zaqBmxP/8rp85YmnaA5o8aQoDmL0gmaM3KFnzf/7zT1BM05lUzQnBkTtIqfu7f//dDWc4JmUg3Ykf610+ctTbqLk4iIiMjVMEEjIiIicjKu0a9BRERELkmsSaZwHbRqYwXt/9u7d9eosjgO4L+bXWMQTRSUHcQ0YmFr42ubYOU/YG9joyCIFja+URBZO0tfnb2dFsLC9nb2pjGNEB+gwZmz3FnirrvGnQxz7pw7+XxCiIFhziUg8+V7XgAAhdGgAQDZrN6O2fSYbadBAwAojAYNAMimV/WiqqxBWy8NGgBAYQQ0AIDCmOIEALJxzMZwNGgAAIXRoAEA2WjQhqNBAwAojIAGAFAYU5wAQDZuEhiOBg0AoDAT1aCl9GXcjzAxDv3+W7RBinY8Z+n839l4fv5pIYo3N+4HYBR60Y0quo2P2XYaNACAwkxUgwYAlCX1v5peg5ai7TRoAACFEdAAAApjihMAyKZX9aKq3CSwXho0AIDCaNAAgMzHbDTbB/UcswEAwKgJaAAAhTHFCQBk1PxdnGGTAAAAo6ZBAwCy6aV6wf7UGMZsNw0aAEBhNGgAQDb1+rPm7+LsRdtp0AAACiOgAQAUxhQnAJBNim6khvug5CYBAABGTYMGAGTT6y/Yb3bR/l9jtpsGDQBgQDdv3oyjR4/Gli1bYvv27d99TVVV//l+8uRJrIcGDQBgQCsrK3HixIk4cuRI3L9/f83XPXz4MI4fP/7197XC3FoENAAgm9T/avoctJTtva9du9b/+ejRox++rg5knU5n6HFMcQIAE+ndu3fffH/+/Lmxsc+cORM7d+6MgwcPxoMHDyKl9YVGDRoAkE1K9TEbVeNj1ubn5+Ofrly5ElevXo3crl+/HseOHeuvU3v27FmcPn06Pnz4EGfPnh34PQQ0AGAiLS4uxuzs7NffN2/e/N3XXbx4MW7fvv3D93r16lXs379/oHEvXbr09d8HDhyIjx8/xp07dwQ0AKAM4zxmY3Z29puAtpbz58/HyZMnf/iavXv3Dv08hw4dihs3bvSnWNcKif8moAEAG9quXbv637m8fPkyduzYMXA4qwloAAADev36dbx9+7b/s9vt9sNXbd++fbF169Z4+vRpLC0txeHDh2NmZiaeP38et27digsXLsR6CGgAQOa7OKuJuYvz8uXL8fjx42/WmNVevHgRCwsLsWnTprh3716cO3euv3OzDm53796NU6dOrWucKg2w77Pemjo3NxfLy8sDzeUCAOM3zs/v1bF/mfs1pqpm+6Be+hJLy3+0Ordo0ACAbFLqNX9QbXIXJwAAIyagAQAUxhQnADCR56C1mQYNAKAwGjQAYCLv4mwzDRoAQGE0aABANqn/1fAxG/G/R7wWT4MGANDGBm31soH6VGAAoB1WP7cHuDSINga09+/f93/Oz8/nfh4AYMTqz/H62qXx3STQ9CaBXmyIgLZ79+5YXFyMbdu2RVU1+0cGAIZTN2d1OKs/x5nAgDY1NRV79uzJ/zQAwEiNqzn7W33MRvNjtp1NAgAAhRHQAAAK4xw0ACDzgn2bBNZLgwYAUBgNGgCQjQZtOBo0AIDCaNAAgGx60Yuq6QYtNGgAAIyYgAYAUBhTnABANjYJDEeDBgBQGA0aAJBNSt0NMeaoadAAAAojoAEAFMYUJwCQTYrUPw2t+THbTYMGAFAYDRoAMFFHXiTHbAAAMGoCGgBAYUxxAgDZmOIcjgYNAKAwGjQAIJvU8BEb4xpz1DRoAACF0aABANlYgzYcDRoAQGEENACAwpjiBACyMcU5HA0aAEBhNGgAQEbjaLN60XYaNACAwghoAACFMcUJAGRjk8BwNGgAAIXRoAEA2biLczgaNACAwmjQAIBsUkqNH3uR+mO2mwYNAKAwAhoAQGFMcQIAGXUjomp4zBRtp0EDACiMBg0AyHxobLMNWrJJAACAURPQAAAKY4oTAMio+SnOsEkAAIBR06ABAPmMYZNA2CQAAMCoadAAgGzSGNaDJWvQAAAYNQENAKAwpjgBgIwcszEMDRoAQGE0aABARmkMhVaKttOgAQAURkADACiMKU4AYOJOQms7AQ0AyKz9galpAhoAMHLT09PR6XTizZs3Yxm/0+n0n6GtqpQm4EZRAKA4nz59ipWVlbGMPT09HTMzM9FWAhoAQGHs4gQAKIyABgBQGAENAKAwAhoAQGEENACAwghoAACFEdAAAKIsfwI9sigExPPe1gAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAKXCAYAAAASHvsAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQI5JREFUeJzt3Q10FOd97/GfZL0AQoKLILwE2dQuMWnUGhwupbHNW7I2VRonEMs0IkCP3XsoNg4vuqmrYoxUY+ISm2NyIEADF+w06i1JemkcpQoSL65Tp5gc6mIhXgwGYhyH4tiWjCVLAu09z2yWRcg1K2B3Zp7n+zlnjnZGA/z/emaHn+ZtM6LRaFQAAABwRqbfBQAAACC9CIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgABwBXbv3q2MjAzva9yf/dmfacSIEb7WBQDJIAACSKstW7Z4wclMP/vZz7p933w6ZVFRkff9P/mTP/GlRgCwHQEQgC969eql6urqbsuff/55nTp1Srm5uQqb73znOzp8+LDfZQDAZREAAfiipKRE3//+93Xu3Lkuy00o/PSnP60hQ4YobLKzs0MZXAG4hwAIwBdf+cpX9Jvf/EZ1dXUXlrW3t+sHP/iBysrKup0WNtfWffGLX+z293zwwQfq16+f5s6d+5H/njmlPH/+fH3ve9/TzTff7B2BNEHzX//1X7ut+x//8R/64z/+YxUUFKhv37767Gc/q3//93+/bE8fdg1gZ2enVq9erd///d/3/s1BgwZp6tSp+sUvfuF9f+LEibrllls+9O8zdd51112X/XcBoKcIgAB8YYLSH/3RH+kf/uEfLiz7l3/5FzU1NelP//RPu4W3r371q97333777S7fe+6559Tc3Ox9/3LM6eWFCxd66/7N3/yNF0BNGGtoaLiwzoEDB3THHXfoP//zP/WXf/mXWrp0qY4fP65JkyZpz549Pe7z/vvv9/5Nc13j3/7t3+qv/uqvvCAYD5SzZs3S/v37u9Rg7N27V0eOHEmqLwDosSgApNHmzZujZtezd+/e6Jo1a6L5+fnRlpYW73ulpaXRyZMne69vuOGG6Oc///kLf+7w4cPen1u3bl2Xv+/uu++OjhgxItrZ2fmR/675s2b6xS9+cWHZyZMno7169YpOmzbtwrIvfelL0ZycnOixY8cuLPvVr37l1TlhwoQLy3bt2uX9feZr3Jw5c7y643bu3Omt87Wvfa1bPfF63333Xa+Ghx9+uMv3zZ/Jy8uLnj179iP7AoArwRFAAL6599571draqh//+Md67733vK+Xnv6N+8QnPqE//MM/9E7hxpmjgeao4MyZM72jhJdjjjia075x119/vXda+ac//anOnz/vTdu3b9eXvvQl3XjjjRfWGzp0qFeXuWvZHG1M1g9/+EOvrmXLlnX7Xrxec/ra1GCOhMZyqrw6/vEf/9GrIy8vL+l/DwCSRQAE4BtzPdznPvc578aPf/qnf/KCzz333PPfrj979mz927/9m06ePOnNm5tIOjo6vNOoyRg5cuSHBsuWlhadOXPGm8xrc+3dpT75yU961/O9/vrrSfd37NgxDRs2TAMGDPjI9Uxfv/zlL/XCCy948/X19Tp9+nTSfQFATxEAAfjKHFkzR/HWr1/v3XjRv3///3Zdc22gudM2fhTw7//+7zV27NgPDWxhYm70GDx4sNePYb6au6BNOAaAVCAAAvDVtGnTlJmZ6d0U8d+d/o0zR9I+//nPewHQHAU0RwN7cpTs1Vdf7bbM3GjRp08f72ikmczrD3uW36FDh7w6zc0cybrpppv0q1/9qtuNK5e67rrrvN7NHdDvvPOOtm3b5t0lbZYDQCoQAAH4yjxmZd26daqsrNQXvvCFy65vAl9jY6O+/vWvewHp0juGP8rPf/5z7du378K8OZ37z//8z7rzzju9v8tM5rVZduLEiQvrmdOx5jT17bff7j0aJllf/vKXvev6qqqqun0vfr3fxX2Z8GceZ3P27Fnu/gWQUlmp/esB4PLmzJmT9LrmCGBhYaF3/Z85Zfyxj30s6T9bXFzsnW792te+5j2w+dvf/ra3/OKAtnz5cu/ZhCbsPfDAA8rKytKGDRvU1tamlStX9qivyZMne8HuW9/6lnf00TxyxlxHaK71M98zzyWMGzNmjFef6ctcb3jrrbf26N8CgJ7gCCCAUMnJydGMGTO81z29ScI8dPnpp5/Wd7/7XT366KPeKWVz/eEf/MEfXFjnU5/6lBfQTBj7xje+4YXDG264Qbt27fLuQu6pzZs365vf/Kb3LEFz1HLFihXenc+f+cxnPvRmkCvpCwB6KsM8C6bHfwoAfLRo0SJt2rRJv/71r71r9pJhHrvy4IMPas2aNQoq84khpjdz+tk8ogYAUoUjgABCxXz0m7lL1lxfl2z4CwPzu7gJteYoJeEPQKpxDSCAUPiv//ov7/l45k5Z8xFuCxYskA3ef/99/ehHP/JOMb/yyiveDSgAkGoEQAChYO78NZ/4YW76MDdVjB49WjYwD582j4Axzz/867/+a919991+lwTAAVwDCAAA4BiuAQQAAHAMARAAAMAxSV8DaB6CaqY48zBT8/FG5oGs5vEKAAAA8Je5su+9997TsGHDvI+vvOoAGH8gKgAAAILNfNTl8OHDr/4mkEuPADY1NXnPqjJPts/OzlaYmYT8e7/3e95dhubIZtjZ1A+9BJdN/dBLcNnUD70EV6ZF/XR0dHifPvTuu++qX79+V38E0HxuppkuZcKf+WimsA+8eaCs6SPsA29bP/QSXDb1Qy/BZVM/9BJcmZb1Y1zu8jxuAgEAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMdkJbtiW1ubN8U1Nzd7XzMzM70pzOL1h70PG/uhl+CyqR96CS6b+qGX4Mq0qJ9ke8iIRqPRZFasrKxUVVVVt+XV1dXq06dPzysEAADANdXS0qKysjI1NTWpoKDg6o8AVlRUaPHixV2OABYVFamxsVE5OTkKe1ouLi5WJBJRdna2wq6jo0N1dXVqaGhQZ2enwsymsYmPiw292NYPvQSXTf3QS3B1WPT/Znt7e1LrJR0Ac3NzvelS5gcV9h9WnNmIbdiQ4xibYLKpF9v6oZfgsqkfegmuTgv+30y2/vCf7AYAAECPEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEwxf5vw//VXX9/lwY/OVjZj2Wr3xP99Durf0eTtkzSgn9ZoJ8e/anfJTpvwYIFGjJkiN9lAIC7jh2T+vaVMjJi0513StFo13XMfCSSWCcvT3r1Vb8qDr0svwuw2ez/N1vf3f/dLsua25q96cS7J/T8yed1sumk7vrdu3yrEVJTU5NOnz7tdxkA4K6bbpKeekr6i7+IzdfVSWvXSvPnJ9ZZs0aqr0/MP/mkNHJk+mu1BAEwRWqP1nYJf58e+mndddNd6pvTV2dazmjfm/v081M/97VGAAACY+5c6Uc/kn7yk9j8ww/HjgR+4hPSkSOx+bipU6V583wr1QacAk6R7ce2X3j9uwN+V3v+fI8e/+zjqrijQqvuWqXdf7ZbZ75+Rl//zNd9rRN26OzsVHl5uRobGxV29BJcNvVDLwG1caNUWBh73dIizZ4ttbVJs2ZJra2x5QMGSJs2+VqmDQiAKXKu89yF1+9+8K53yvdSBbkFuu3629JcGWxz/vx5zZo1S6tWrdLmzZsVZvQSXDb1Qy8BNnSotG5dYn7PHmncOOmllxLLzPeHDfOlPJtwCjhFbh1664XXb7W8pU+s+YRGDxmt/znsf3qngyf/zmTvyCBwNc6dO6eysjJ9//vf1wMPPKCVK1cqrOgluGzqh15CoLRUmjlT+t73YvP79ye+V1Ym3Xuvb6XZhACYIl/9g69q7d61+sWvfuHNd0Y7vev+zBR3+/W3a80fr9EtQ27xsVKEWWlpqbZt26Z+/fopIyNDDz30UNJ3Po8M2MXT9BLMXmzrh16C2Us35oaPnTulN99MLBs8OHZjCK4JAmCKZGVmaefsnfrGz76h//Mf/0en3+9+l+nPfvkzRb4b0YEHDmhQ3iBf6kS4r/vZvXv3hTuZ1/Zgx3jPPfcE6j8AeglmL7b1Qy/B7OVDnTolvf1212Vm/sQJafRov6qyCtcAplB+br5WfHaF3ix/Uw3zGrTp7k2ac8sc5efkX1jH3BF86aNigGRkZmZqx44dGjBggPr37689e/YoGo0mNU2aNElBQi/B7MW2fuglmL1009ERu+nD3PyRzHJcEQJgGphD85/62Kd035j7tOVLW7R/3n5lZiR+9K/+hgdZptu+ffu0fv36bssPHjyo1atXKyxuvfVW7dy5U1lZWYpEInrxxRcVVvQSXDb1Qy8hsGyZ9PLLifkHH0y8bmiQHnnEl7JsQwBMkWdefkYbfrHBe+jzpfKy87oEwP69+qe5OixZskTz5s3z7pyLO3DggPeb8dKlS/X6668rLG655Rbt2rVLvXr18o4IhBm9BJdN/dBLgJkQe/HNLPfdF7se8P77E8vMfvuFF3wpzyZcA5gix989rqrnq7Twpwu9mz1GDx6tAb0H6Detv9EPGn/Q5TExU393qq+1umjr1q0qKSnxnp01aFDs+svJkyervb1d27dvV1FRkcKkuLjYC7ADBw5U2NFLcNnUD70E0PvvS3PmmGfbxOZHjJCefjr22nzdtUt67TVzAWRsPXN3sPn4OFwRAmCKfXDuA9W/Vu9NH+Z/3fq/NHHExLTX5br8/HzV1tbqC1/4gvfbc/yRCvX19Ro7dqzCKPQ7/4vQS3DZ1A+9BEx5uXT0aOx1Zqb0zDNmZx2bN0Hv2WeliRNjAfH4cWnRIuk73/G15DDjFHCKLBy/UD8o/YEeGPuAxn18nK7vd716Z/VWznU5+nj+x3X3zXfrh/f+UH/3hb/zu1Rn5eXlqaamxrt2prCw0Dt9EtbwBwChVlsrbdiQmDfhbsKEruvcdlvXj4MznxpSU5O+Gi3DEcAUMdf1ffn3vuxNCK7evXt7p3wBAD4yn+0bjV5+vccfj024ahwBBAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxWcmu2NbW5k1xzc3N3tfMzExvCrN4/R0dHbJBvI+wj4ttYxPvwYZebOuHXoLLpn7oJbg6LPx/83IyotFoNJkVKysrVVVV1W15dXW1+vTp0/MKAQAAcE21tLSorKxMTU1NKigouPojgBUVFVq8eHGXI4BFRUVqbGxUTk6Owp6Wi4uLFYlElJ2dLRt+k6mrq1NDQ4M6OzsVZjaNjU3jcvHY2NCPjb3Y8J6x7X1j09jYNC627QPa29uTWi/pAJibm+tNlzI/qLD/sOLMGzLsb8qLMTbBZNO42NaPTb3Y9J4xGJtgsmlcbOkn2frDf7IbAAAAPUIABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBMI0WLFigIUOG+F0GLsG4AD1z7tw5zZw5U6NGjdKRI0f8Lge/xbigJwiAadTU1KTTp0/7XQYuwbgAyWtvb1dpaamqq6t1+PBhTZo0SYcOHfK7LOcxLugpAiAAICltbW2aPn26tm3bpoKCAm/Z2bNnvbBx4MABv8tzFuOCK0EABAAkZcaMGaqpqVFFRYWmTZvmLdu+fbtaW1s1efJknTp1yu8SncS44EoQAAEASVm4cKGWL1+uFStWXFg2fvx41dXVae7cuRo+fLiv9bmKccGVyLqiPwUAcI45pWimS40bN86b4A/GBVeCI4AAAACOIQACAAA4hgAIAADgGAJgiu3bt0/r16/vtvzgwYNavXq1LzWBcQEAuI2bQFJsyZIlqq2tVUtLy4Vl5rlMU6ZM8W7RN89uKioq8rVGFzEuAACXEQBTbOvWrSopKVF5ebkGDRrkLTPPZTJPbTfPaSJk+INxAQC4jFPAKZafn+8daTLh4syZMxc+r7G+vt57ThP8wbgAAFxGAEyDvLw87yntkUhEhYWF2rFjh8aOHet3Wc5jXAAAriIApknv3r29U4tvvfWWxowZ43c5+C3GBbgyW7ZsUTQa9bsMXIJxQbIIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4JivZFdva2rwprrm52fuamZnpTWEWr7+jo0M2iPcR9nGxbWxsGpeL+7ChHxt7seE9Y9v7xqaxsWlcbN0HXE5GNBqNJrNiZWWlqqqqui2vrq5Wnz59el4hAAAArqmWlhaVlZWpqalJBQUFV38EsKKiQosXL+5yBLCoqEiNjY3KyclR2NNycXGxIpGIsrOzZcNvZnV1dVb0Qy/BZVM/8V4aGhrU2dmpMLN1f8bYBItN73/btrP29vak1ks6AObm5nrTpcwPKuw/rDizEduwIdvYD70El039sD8LLsYmmGzqxZbtLNn6w3+yGwAAAD1CAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBMA0WrBggYYMGeJ3GQCQvGPHpL59pYyM2HTnnVI02nUdMx+JJNbJy5NefdWvip117tw5zZw5U6NGjdKRI0f8LgcBRwBMo6amJp0+fdrvMgAgeTfdJD31VGK+rk5au7brOmvWSPX1ifknn5RGjkxfjVB7e7tKS0tVXV2tw4cPa9KkSTp06JDfZSHACIAAgI82d65UUpKYf/hhKX6EyXw183FTp0rz5qW/Roe1tbVp+vTp2rZtmwoKCrxlZ8+e9ULggQMH/C4PAUUARNI6OztVXl6uxsZGhZ1NvdjWj029WGXjRqmwMPa6pUWaPdskD2nWLKm1NbZ8wABp0yZfy3TRjBkzVFNTo4qKCk2bNs1btn37drW2tmry5Mk6deqUwoR9QHoQAJGU8+fPa9asWVq1apU2b96sMLOpF9v6sakX6wwdKq1bl5jfs0caN0566aXEMvP9YcN8Kc9lCxcu1PLly7VixYoLy8aPH6+6ujrNnTtXw4cPV1iwD0ifrDT+WwjxhcVlZWX6/ve/rwceeEArV65UWNnUi2392NSLtUpLpZkzpe99Lza/f3/ie2Vl0r33+laay8ypXjNdaty4cd4UFuwD0osAiMsyFxaba0v69eunjIwMPfTQQ0nf9TwyYBeC29SLbf3Y1IvVzA0fO3dKb76ZWDZ4cPcbQ4AeYh+QXgRAXPZajN27d1+4i3ltD3by99xzT6DelDb1Yls/NvViPXM92dtvd11m5k+ckEaP9qsqhBz7gPTjGkB8pMzMTO3YsUMDBgxQ//79tWfPHkWj0aSmDzsl4SeberGtH5t6sVpHR+ymD3PzRzLLgSSxD0g/AmCK7du3T+vXr++2/ODBg1q9erXC4NZbb9XOnTuVlZWlSCSiF198UWFlUy+29WNTL9Zatkx6+eXE/IMPJl43NEiPPOJLWbAD+4D0IgCm2JIlSzRv3jzvjqY481wm8xvL0qVL9frrrysMbrnlFu3atUu9evXyfksLM5t6sa0fm3qxjvnP+OKL8u+7L3Y94P33J5aZ/dwLL/hSHuzAPiB9uAYwxbZu3aqSkhLvmUaDBg3ylpnnMpmntpvnNBUVFSksiouLvfA6cOBAhZ1NvdjWj029WOP996U5c8wzOmLzI0ZITz8de22+7tolvfaauZArtp65O9h8fBxwBdgHpAdHAFMsPz9ftbW1Xug7c+bMhVvd6+vrvec0hY1Nb0iberGtH5t6sUJ5uXT0aOx1Zqb0zDNm5xabN0Hv2Wel666LzR8/Li1a5F+tsAL7gNQjAKZBXl6e95R2c01DYWGhd1h77NixfpcFAJdXWytt2JCYN+FuwoSu69x2W9ePgzOfGlJTk74aAfQYp4DTpHfv3t4pXwAIFfPZvtHo5dd7/PHYBF9t2bLFm4DL4QggAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGOykl2xra3Nm+Kam5u9r5mZmd4UZvH6Ozo6ZIN4Hzb0Qy/BZVM/8R7Cvi+zeX/G2ASLTe9/W7ezy8mIRqPRZFasrKxUVVVVt+XV1dXq06dPzysEAADANdXS0qKysjI1NTWpoKDg6o8AVlRUaPHixV2OABYVFamxsVE5OTkKe1ouLi5WJBJRdna2bPhNpq6uTg0NDers7JQNY0MvwWPT+4b3THDZ1I+Nvdjw/rdtH9De3p7UekkHwNzcXG+6lPlBhf2HFWc2Yhs2ZBvHhl6Cy6b3jU1jY1MvtvVjUy82vf9tGZtk6w//yW4AAAD0CAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAA02jBggUaMmSI32UAoXDu3DnNnDlTo0aN0pEjR/wuB0CasQ9ILQJgGjU1Nen06dN+lwEEXnt7u0pLS1VdXa3Dhw9r0qRJOnTokN9lAUgT9gGpRwAEEChtbW2aPn26tm3bpoKCAm/Z2bNnvf8ADhw44Hd5AFKMfUB6EAABBMqMGTNUU1OjiooKTZs2zVu2fft2tba2avLkyTp16pTfJQJIIfYB6UEABBAoCxcu1PLly7VixYoLy8aPH6+6ujrNnTtXw4cP97U+AKnFPiA9stL07wBAUsxpHjNdaty4cd4EwG7sA9KDI4AAAACOIQACAAA4hgAIAADgGAJgiu3bt0/r16/vtvzgwYNavXq1LzUBAAC3cRNIii1ZskS1tbVqaWm5sMw8x2jKlCneLe3mWUdFRUW+1ggAANxCAEyxrVu3qqSkROXl5Ro0aJC3zDzHyDzl3DzXiPAHAADSjVPAKZafn+8dATSh78yZMxc+37C+vt57rhEAAEC6EQDTIC8vz3uqeSQSUWFhoXbs2KGxY8f6XRYAAHAUATBNevfu7Z3yfeuttzRmzBi/ywFCYcuWLYpGo36XAcAn7ANShwAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGOykl2xra3Nm+Kam5u9r5mZmd4UZvH6Ozo6ZIN4H2Efl4t7oJfgsel9w3smuGzqx8ZebHj/27oPuJyMaDQaTWbFyspKVVVVdVteXV2tPn369LxCAAAAXFMtLS0qKytTU1OTCgoKrv4IYEVFhRYvXtzlCGBRUZEaGxuVk5OjsKfl4uJiRSIRZWdny4bfZOrq6qzox8ZeGhoa1NnZqbCz6X1j09jYNC627gPoJXg6LNoHtLe3J7Ve0gEwNzfXmy5lflBh/2HFmY3Yhg3Zxn5s6sWm94zB2ASTTeNiWz/0ElydFuwDkq0//Ce7AQAA0CMEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAOG8BQsWaMiQIX6X4bZjx6S+faWMjNh0551SNNp1HTMfiSTWycuTXn3Vr4rdwdjAJ+fOndPMmTM1atQoHTlyxO9yrEMAhPOampp0+vRpv8tw2003SU89lZivq5PWru26zpo1Un19Yv7JJ6WRI9NXo6sYG/igvb1dpaWlqq6u1uHDhzVp0iQdOnTI77KsQgAEEAxz50olJYn5hx+W4r/1m69mPm7qVGnevPTX6CrGBmnU1tam6dOna9u2bSooKPCWnT171guBBw4c8Ls8axAAffztZv78+Tp58qTCzqZe4LONG6XCwtjrlhZp9mzzv4E0a5bU2hpbPmCAtGmTr2U6yZGxsWl/FtZeZsyYoZqaGlVUVGjatGnesu3bt6u1tVWTJ0/WqVOn/C7RCgRAn+zbt08bN27UhAkTdMxcYxNiNvUCnw0dKq1bl5jfs0caN0566aXEMvP9YcN8Kc9pjoyNTfuzsPaycOFCLV++XCtWrLiwbPz48aqrq9PcuXM1fPhwX+uzBQHQJ2Zjfu6553TmzBnvzWmucQgrm3pBAJSWSjNnJub370+8LiuT7r3Xl7LgxtjYtD8Lay/mVO+SJUu6LR83bpwee+wxX2qyUZbfBdho6dKleuedd5Jat7i4WHv37tXEiRP1/PPP6+abb1aQ2NQLQsTcVLBzp/Tmm4llgwd3v/kA6RfisbFpf2ZTL/AHATAFNm/erDfeeKNHf8bchdrY2Bi4N6ZNvSBEzDU+b7/ddZmZP3FCGj3ar6oQ8rGxaX9mUy/wB6eAU8BcoBqNRi87xe9qMqqqqi5c7BokNvWCkOjoiN1YYG4wSGY50ifkY2PT/symXuAPAqBP3nvvPU2dOlW7d+/WE088oUcffVRhFcZezMXR69ev77b84MGDWr16tS814beWLZNefjkx/+CDidcNDdIjj/hSFtwYmzDuz1zoBdcep4B9cvToUb3yyitatWqVFi1apDALYy/mAuPa2lq1mMdZ/JZ5vtSUKVO8Rw2YZ1AVFRX5WqOTXnxRWrkyMX/ffbFrzj74IPF4kVWrpLvvlu64w7cyneTI2IRxf+ZCL7j2CIA+GTNmjPfmHDhwoMIujL1s3bpVJSUlKi8v16BBg7xl5vlS5rlZ5nlThD8fvP++NGeOdP58bH7ECOnpp2Ovzdddu6TXXpM6O2PrmTtQzUeUIfUcGpsw7s9c6AXXHqeAfWTTmzJsveTn53tHAE3oM49IiH/uZH19vffoBPigvNwcsoi9zsyUnnnGDFRs3oSJZ5+VrrsuNn/8uMQRjfRxbGzCtj9zpRdcWwRAOCsvL8972nwkElFhYaF27NihsWPH+l2Wm2prpQ0bEvMmQEyY0HWd227r+pFj5pMpamrSV6OrGBvASpwChtN69+7tnfKFz8znx0ajl1/v8cdjE9KHsYGPtmzZ4k249jgCCAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjspJdsa2tzZvimpubva+ZmZneFGbx+js6OmSDeB829GNjL2F/v9j4vrFpbGwaF1v3AfQSPB0W7gMuJyMajUaTWbGyslJVVVXdlldXV6tPnz49rxAAAADXVEtLi8rKytTU1KSCgoKrPwJYUVGhxYsXdzkCWFRUpMbGRuXk5Cjsabm4uFgNDQ3q7OxU2NnUT7yXSCSi7Oxshf03zLq6OivGxdbtjF6Cx6Z+bOzFhn2zbfvn9vb2pNZLOgDm5uZ606XMDyrsPywbe7GtH7ODsWEnY9u42NYPvQSXTf3Y1ItN+2ZbxibZ+sN/shsAAAA9QgAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAOG/BggUaMmSI32UAgLuOHZP69pUyMmLTnXdK0WjXdcx8JJJYJy9PevVVvyoOPQIgnNfU1KTTp0/7XQYAuOumm6SnnkrM19VJa9d2XWfNGqm+PjH/5JPSyJHpq9EyBEAAAOC/uXOlkpLE/MMPS0eOxF6br2Y+bupUad689NdoEQIgAAAIho0bpcLC2OuWFmn2bKmtTZo1S2ptjS0fMEDatMnXMm1AAAQAAMEwdKi0bl1ifs8eadw46aWXEsvM94cN86U8mxAAAQBAcJSWSjNnJub370+8LiuT7r3Xl7JsQwAEAADBYm74MEcDLzZ4cPcbQ3DFCIAAACBYTp2S3n676zIzf+KEXxVZhwAIAACCo6MjdtOHufkjmeW4IgRAOGnfvn1av359t+UHDx7U6tWrfakJACBp2TLp5ZcT8w8+mHjd0CA98ogvZdkmy+8CAD8sWbJEtbW1ajGPGfitAwcOaMqUKWptbdX06dNVVFTka40A4JwXX5RWrkzM33df7HrADz5IPPpl1Srp7rulO+7wrUwbcAQQTtq6datuv/12lZeX6yc/+Ym3bPLkyWpra9P27dsJfwCQbu+/L82ZI50/H5sfMUJ6+unYa/P1xhtjrzs7Y+udPetfrRYgAMJJ+fn53hFAE/rOnDnjLTt37pzq6+s1fvx4v8sDAPeUl0tHj8ZeZ2ZKzzxjdtaxefM5wc8+K113XWz++HFp0SL/arUAARDOysvLU01NjSKRiAoLC7Vjxw6NHTvW77IAwD21tdKGDYl5E+4mTOi6zm23df04OPOpITU16avRMlwDCKf17t3bO+ULAPCR+WzfaPTy6z3+eGzCVeMIIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjspJdsa2tzZvimpubva+ZmZneFGbx+sPeh439xHvo6OhQ2MV7sGFcbN3O6CV4bOrHxl5s2Dfbtn9OtoeMaDQaTWbFyspKVVVVdVteXV2tPn369LxCAAAAXFMtLS0qKytTU1OTCgoKrv4IYEVFhRYvXtzlCGBRUZEaGxuVk5OjsKfl4uJiRSIRZWdny4bfZOrq6qzoh16Cy6Z+6CW4bOrHxl4aGhrU2dmpsMv8bQ6woZ/29vak1ks6AObm5nrTpcwPKuw/rDjzhgz7m9LWfugluGzqh16Cy6Z+bOrFpgxgSz/J1h/+k90AAADoEQIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgisBQsWaMiQIQqVY8ekvn2ljIzYdOedUjTadR0zH4kk1snLk1591a+KYZFQvmfgCwIggMBqamrS6dOnFSo33SQ99VRivq5OWru26zpr1kj19Yn5J5+URo5MX42wVijfM/AFARAArrW5c6WSksT8ww9LR47EXpuvZj5u6lRp3rz01wjAaQRAn7S3t2v+/Pk6efKkwo5egsu2fkJl40apsDD2uqVFmj1bamuTZs2SWltjywcMkDZtUtjZtJ3Z1AvwUQiAPtm3b582btyoCRMm6Ji5ZijE6CW4bOsnVIYOldatS8zv2SONGye99FJimfn+sGEKO5u2M5t6AT4KAdAn48eP13PPPaczZ854O5rDhw8rrOgluGzrJ3RKS6WZMxPz+/cnXpeVSffeKxvYtJ3Z1AvwUbI+8ru4IkuXLtU777yT1LrFxcXau3evJk6cqOeff14333yzgoRegtmLjf1Yy9zwsXOn9OabiWWDB3e/MSSgbNrObOoFuFoEwBTYvHmz3njjjR79GXPXVmNjY+B2MvQSzF5s7Mdap05Jb7/ddZmZP3FCGj1aQWfTdmZTL8DV4hRwCpw6dUrRaPSy09mzZzVp0iTvz1RVVWnatGkKGnoJZi829mOljo7YTR/m5o9klgeQTduZTb0AV4sA6JP33ntPU6dO1e7du/XEE0/o0UcfVVjRS3CFsR9zEf769eu7LT948KBWr16tUFm2THr55cT8gw8mXjc0SI88IhuEcTuzqRer3jNIG04B++To0aN65ZVXtGrVKi1atEhhRi/BFcZ+lixZotraWrWYR6f81oEDBzRlyhS1trZq+vTpKioqUuC9+KK0cmVi/r77YtcDfvBB4tEvq1ZJd98t3XGHwiyM25lNvVjznkFaEQB9MmbMGG9HM3DgQIUdvQRXGPvZunWrSkpKVF5erkGDBnnLJk+e7D2fbfv27eH4j+z996U5c6Tz52PzI0ZITz8de22+7tolvfaa1NkZW8/cHWw+Pi6kwrid2dSLFe8ZpB2ngH0Uph3M5dBLcIWtn/z8fO9ohvkPzDyKwzh37pzq6+u9R3SEQnm5OZQUe52ZKT3zjGksNm+C3rPPStddF5s/flwKyZEmm7Yzm3qx4j2DtCMAAgicvLw81dTUKBKJqLCwUDt27NDYsWMVCrW10oYNiXkT7iZM6LrObbd1/Tg486khNTXpqxHWCfV7Br7gFDCAQOrdu7d3+ip0zGf7RqOXX+/xx2MT4Pp7Br7gCCAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjslKdsW2tjZvimtubva+ZmZmelOYxevv6OiQDeJ92NAPvQSXTf3QS3DZ1I+NvYT9//+4eB829JNsDxnRaDSazIqVlZWqqqrqtry6ulp9+vTpeYUAAAC4plpaWlRWVqampiYVFBRc/RHAiooKLV68uMsRwKKiIjU2NionJ0dhT8vFxcWKRCLKzs6WDb+Z1dXVqaGhQZ2dnbJhbOgleGx639j4nrFhXAzGJtjjYkMvtm1n7e3tSa2XdADMzc31pkuZH1TYf1hxZiO2YUO2cWzoJbhset/YNDY2jYvB2ASTTb3Ysp0lW3/4T3YDAACgRwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEApsm5c+c0c+ZMjRo1SkeOHPG7HCAUFixYoCFDhvhdBi7B/gzpwj4gdQiAadDe3q7S0lJVV1fr8OHDmjRpkg4dOuR3WUDgNTU16fTp036XgYuwP0M6sQ9IHQJgirW1tWn69Onatm2bCgoKvGVnz571dpoHDhzwuzwASBr7M8AeBMAUmzFjhmpqalRRUaFp06Z5y7Zv367W1lZNnjxZp06d8rtEAEgK+7PwHa2dP3++Tp486XcpCCACYIotXLhQy5cv14oVKy4sGz9+vOrq6jR37lwNHz7c1/oAIFnsz8Jl37592rhxoyZMmKBjx475XQ4CJsvvAmxnTo2Y6VLjxo3zJgAIC/Zn4WLC+XPPPacvfvGLXgjcuXOnbr75Zr/LQkAQAAEACJmlS5fqnXfeSWrd4uJi7d27VxMnTtTzzz9PCISHAAgAQMhs3rxZb7zxRo/+jLmbtrGxkQAID9cAAgAQMuaGm2g0etkpfpe2UVVVdeHmHYAACCBwF66vX7++2/KDBw9q9erVvtQEhNF7772nqVOnavfu3XriiSf06KOPKgzYB6QHp4ABBMqSJUtUW1urlpaWC8vMM+amTJniPW7EPIeuqKjI1xqBMDh69KheeeUVrVq1SosWLVJYsA9IDwIggEDZunWrSkpKVF5erkGDBnnLzDPmzDPNzDPn2PEDyRkzZowXAgcOHKgwYR+QHpwCBhAo+fn53m//Zod/5syZC589W19f7z3WAkDywhb+DPYB6UEABBA4eXl53idORCIRFRYWaseOHRo7dqzfZQFIE/YBqUcATKMtW7Z4d2UBuLzevXt7p3veeust71QWgoX9GVKNfUBqEQABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcExWsiu2tbV5U1xzc7P3NTMz05vCLF5/R0eHbBDvI+zjcnEP9BI8Nr1vbHzP2DAuBmMTTPEebOjF1u3scjKi0Wg0mRUrKytVVVXVbXl1dbX69OnT8woBAABwTbW0tKisrExNTU0qKCi4+iOAFRUVWrx4cZcjgEVFRWpsbFROTo7CnpaLi4sViUSUnZ0tG36Tqaurs6IfG3tpaGhQZ2enws6m942N25kNvdjWj037AJve/7aNTXt7e1LrJR0Ac3NzvelS5gcV9h9WnNmIbdiQbezHpl5ses/YNjb0Elw29WPTPsCmcbFlbJKtP/wnuwEAANAjBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwBMk3PnzmnmzJkaNWqUjhw54nc5AODe/uzYMalvXykjIzbdeacUjXZdx8xHIol18vKkV1/1q2LnLViwQEOGDPG7DCsRANOgvb1dpaWlqq6u1uHDhzVp0iQdOnTI77IAwK392U03SU89lZivq5PWru26zpo1Un19Yv7JJ6WRI9NXI7poamrS6dOn/S7DSgTAFGtra9P06dO1bds2FRQUeMvOnj3r7TQPHDjgd3kA4Nb+bO5cqaQkMf/ww1L8KKb5aubjpk6V5s1Lf41AGhAAU2zGjBmqqalRRUWFpk2b5i3bvn27WltbNXnyZJ06dUph0dnZqfLycjU2NirsbOrFpSNP8+fP18mTJxV2Ye3Fmv3Zxo1SYWHsdUuLNHu2SbfSrFlSa2ts+YAB0qZNvpYJpBIBMMUWLlyo5cuXa8WKFReWjR8/XnV1dZo7d66GDx+uMDh//rxmzZqlVatWafPmzQozm3pxyb59+7Rx40ZNmDBBx8y1XCEW1l5s2Z9p6FBp3brE/J490rhx0ksvJZaZ7w8b5kt5QDoQAFPMnBpZsmRJt+Xjxo3TY489prBc8P2Vr3zFu+bngQce0MqVKxVWNvXiGhM0nnvuOZ05c8YLTub6s7AKay827M8uKC2VZs5MzO/fn3hdVibde68vZQHpkpW2fwmhZS74Ntf89OvXTxkZGXrooYeSvntrZMAunrapF5ssXbpU77zzTlLrFhcXa+/evZo4caKef/553XzzzQoSm3qxnrnhY+dO6c03E8sGD+5+YwhgIQIgLnut3O7duy/cjbW2BzvGe+65J1ChyaZebGNOxb/xxhs9+jPmzkBzDWfQQpNNvVjPXLP49ttdl5n5Eyek0aP9qgpIC04B4yNlZmZqx44dGjBggPr37689e/YoGo0mNZnTRUFiUy+2MTcPJDMO8TtOjaqqqgs3IgSJTb1YraMjdtOHufkjmeWAZQiAuKxbb71VO3fuVFZWliKRiF588UWFlU29uOa9997T1KlTvaO4TzzxhB599FGFlU29hNayZdLLLyfmH3ww8bqhQXrkEV/KQuwmqfXr13dbfvDgQa1evdqXmmxEAERSbrnlFu3atUu9evXyjqKFmU29uOTo0aN65ZVXvLu3H774WW0hZFMvoWR+8bv4BrD77otdD3j//Yllq1ZJL7zgS3muMzcazZs3z3t/xJnnTJoj5uYa29dff93X+mzBNYBImrlg3bwJBw4cqLCzqRdXjBkzxgtONoyZTb2EzvvvS3PmmOdBxeZHjJCefjr22nzdtUt67TVz0XBsPXN3sPn4OKTN1q1bVVJS4j2rddCgQd4y85xJ8/xM89zJoqIiv0u0AkcA0SM2/YdlUy+usGnMbOolVMrLzSHY2OvMTOmZZ6T8/Ni8CXrPPitdd11s/vhxadEi/2p1VH5+vmpra73QZx6VFH+EV319vfcIJVwbBEAAgBtqa6UNGxLzJtxNmNB1ndtu6/pxcOZTQ2pq0lcjPHl5ed6nzphrtQsLC73LdcaOHet3WVYhAKbRli1bvLv/ACDsQrk/M5/ta2qOT08++eHrPf541/U+//l0VwpJvXv39k75vvXWW95lE7i2CIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjslKdsW2tjZvimtubva+ZmZmelOYxevv6OiQDeJ92NCPjb2E/f1i4/vGxu3Mhl5s68emfYBN739bx+ZyMqLRaDSZFSsrK1VVVdVteXV1tfr06dPzCgEAAHBNtbS0qKysTE1NTSooKLj6I4AVFRVavHhxlyOARUVFamxsVE5OjsKelouLixWJRJSdnS0bfpOpq6tTQ0ODOjs7FWY2jU18XGzoxWA7CyabxuXisbGhHxu3Mxt6se19097entR6SQfA3Nxcb7qU+UGF/YcVZzZiGzbkOMYmmGzqxWA7CyabxsW2fmzazmzqxZbtLNn6w3+yGwAAAD1CAAQAAHAMARAAAMAxBEAAAADHEAABAAAcQwAEAABwDAEQAADAMQRAAAAAxxAAAQAAHEMABAAAcAwBEAAAwDEEQAAAAMcQAAEAABxDAAQAAHAMARAAAMAxBMBUOHZM6ttXysiITXfeKUWjXdcx85FIYp28POnVV/2q2Fnnzp3TzJkzNWrUKB05ckShwnYWTIwL0uw3v5FWroxtasOGSb16Sbm50tCh0oQJ0te/Lr3wQvfNEG4jAKbCTTdJTz2VmK+rk9au7brOmjVSfX1i/sknpZEj01cj1N7ertLSUlVXV+vw4cOaNGmSDh06pNBgOwsmxgVp9Hd/J91wg/Tww7FN7c03pbY2s3+Tfv3rWPAzm5cJgqdP+10tgoQAmCpz50olJYl58+6MH2EyX8183NSp0rx56a/RYW1tbZo+fbq2bdumgoICb9nZs2e9EHjgwAGFBttZMDEuSINvfjO2qb3/fmzeHEyeMkX667+WVqyQ/vf/lj73udgRQeBSBMBU2rhRKiyMvW5pkWbPjv1qNmuW1NoaWz5ggLRpk69lumjGjBmqqalRRUWFpk2b5i3bvn27WltbNXnyZJ06dUqhwXYWTIwLUujgQamiIjFvNrWf/UzasUN6/PHY90xANEcFz5yRvv1tqXdvhUJnZ6fKy8vV2NjodylWIwCmkrkAY926xPyePdK4cdJLLyWWme+bizaQVgsXLtTy5cu1wvya/Fvjx49XXV2d5s6dq+HDhys02M6CiXFBCn3rW9L584n59eulz3zmw9c1l6Sag8z9+inwzp8/r1mzZmnVqlXavHmz3+VYjQCYaqWl0syZifn9+xOvy8qke+/1pSzXmVO9S5Ys6bZ83LhxeuyxxxQ6bGfBxLggRcyRvrj/8T+k6dNlxU15X/nKV7zrsh944AGtNHe2IGWyUvdXo8sF3zt3xq7OjRs8uPuF4cDVYDsLJsYFKfDGG4nX5v6hzIsO55h72T75ye5/Zs4cacsWBZa5Kc9cl92vXz9lZGTooYceSurPLViwQCO5iarHCIDpYK4ne/vtrsvM/IkT0ujRflUF27CdBRPjghQzN3+Enbnub/fu3d7rpqYmre3BL0j33HMPAfAKcAo41To6Yhd9m4u/k1kOXAm2s2BiXJAiH/944rV5hOTFz/j72MdiN4CYqU8fhUJmZqZ27NihAQMGqH///tqzZ4+i0WhSk7mkBz1HAEy1Zcukl19OzD/4YOJ1Q4P0yCO+lAXLsJ0FE+OCFPnsZ7seUP7RjxLz5uZy8wgYM4Xlzl/j1ltv1c6dO5WVlaVIJKIXX3zR75KsRgBMJbPxXnwR6333xa4Huv/+xLJVq2JP6gSuFNtZMDEuSKH586XrrkvM/8VfdP1dI6xuueUW7dq1S7169fKOCCJ1CICpYp7Maa64jd+nP2KE9PTTsdfm6403xl53dsbWO3vWv1oRXmxnwcS4IMU+9Snp4gcWmE/9GDtW+sIXpMrK2LMA//zPpeZmhU5xcbH3QP6lS5f6XYrVCICpUl4uHT0ae21uz3rmGSk/P/FQpmefTfz6dvy4tGiRf7UivNjOgolxQRqYhz2vXh373F/D/L7x4x9LVVWxqwvMM8bN5aZx8eeSh8HAgQP9LsF6BMBUqK2VNmxIzJudu/kgxovddlvXj4MynxpQU5O+GhF+bGfBxLggjb72tdjvEOao3+23S4MGSVlZsWv/rr9eikRi39u3r+tHVAM8BiYVzGd7XnxL1n/HHKM3E3y1ZcsWbwodtrNgYlzgw4fOmPuNzAQkiyOAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADgmK9kV29ravCmuubnZ+5qZmelNYRavv6OjQzaI9xH2cbFtbOI92NCLwXYWTDaNy8V92NCPjduZDb3Y9r5JtoeMaDQaTWbFyspKVVVVdVteXV2tPn369LxCAAAAXFMtLS0qKytTU1OTCgoKrv4IYEVFhRYvXtzlCGBRUZEaGxuVk5OjsKfl4uJiNTQ0qLOzU2FnUz/0Elw29UMvwWVTP/QSXJkW9dPe3p7UekkHwNzcXG+6lPlBhf2HZWMvtvVDL8FlUz/0Elw29UMvwdVpQT/J1h/+k90AAADoEQIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACAYwiAAAAAjiEAAgAAOIYACAAA4BgCIAAAgGMIgAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOyUp2xba2Nm+Ka2pq8r52dHQo7DIzM9XS0qL29nZ1dnYq7Gzqh16Cy6Z+6CW4bOqHXoIr06J+4rksGo1+5HoZ0cut8VuVlZWqqqq6NtUBAAAgZY4dO6Ybb7zx6gPgpUcA3333Xd1www365S9/qX79+inMmpubVVRUpNdff10FBQUKO5v6oZfgsqkfegkum/qhl+Bqtqgfc4b2+uuv1zvvvKP+/ftf/Sng3Nxcb7qUCX9h/2HFmT5s6cW2fugluGzqh16Cy6Z+6CW4Cizqx5zW/sjvp60SAAAABAIBEAAAwDFXHADN6eBly5Z96GnhsLGpF9v6oZfgsqkfegkum/qhl+DKtaifZHtJ+iYQAAAA2IFTwAAAAI4hAAIAADiGAAgAAOAYAiAAAIBjCIAAAACOIQACAAA4hgAIAADgGAIgAACA3PL/AXUqLoHJA0ZZAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "my_policy = [\n", + " A_DOWN,\n", + " A_DOWN,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_DOWN,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_UP,\n", + " A_DOWN,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_UP,\n", + " A_UP,\n", + " A_LEFT,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_UP,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_UP,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_DOWN,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_RIGHT,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_LEFT,\n", + " A_LEFT,\n", + " A_RIGHT,\n", + " A_RIGHT,\n", + " A_UP,\n", + " A_LEFT,\n", + " A_DOWN,\n", + " A_UP,\n", + " A_UP,\n", + " A_LEFT,\n", + " A_UP,\n", + " A_DOWN,\n", + " A_LEFT,\n", + "]\n", + "\n", + "V_my_policy = policy_evaluation(policy=my_policy, P=P, R=R, gamma=gamma)\n", + "\n", + "plot_values(V=V_my_policy, title=\"Value function: my policy\")\n", + "plot_policy(policy=my_policy, title=\"My policy\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "446c93e4", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "studies", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}