{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 10. Online Continual Learning\n", "\n", "In machine learning, continual learning is a problem setting where a model is\n", "trained on a sequence of tasks and must perform well on all tasks seen so far. A\n", "task is a specific concept or relation the model expects to learn. Online\n", "continual learning adds the constraints that the model considers each example\n", "exactly once, and the model can perform prediction at any time.\n", "\n", "In contrast to continual learning, the typical data stream problem setting\n", "adapts to changes in data distribution by discarding knowledge of the past.\n", "However, in continual learning, the model must retain knowledge of past tasks\n", "while learning new tasks.\n", "\n", "Continual learning is synonymous with overcoming catastrophic forgetting, a\n", "phenomenon in deep learning where a model trained on a sequence of tasks forgets\n", "how to perform well on the initial tasks. Non-deep learning models can be immune\n", "to forgetting.\n", "\n", "---\n", "\n", "In this notebook, we implement “Experience Replay” (ER), a classic online\n", "continual learning strategy that stores a buffer of past examples. By sampling\n", "from the buffer during training, ER avoids catastrophic forgetting." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "# This cell is hidden on capymoa.org. See docs/contributing/docs.rst\n", "from util.nbmock import mock_datasets, is_nb_fast\n", "\n", "if is_nb_fast():\n", " mock_datasets()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 10.1 Reservoir Sampling\n", "\n", "Experience Replay uses [reservoir sampling](https://en.wikipedia.org/wiki/Reservoir_sampling)\n", "to construct a simple random sample incrementally from a data stream of unknown\n", "length. Here, we implement reservoir sampling \"Algorithm R\" (Vitter, 1985).\n", "\n", "- Jeffrey S. Vitter. 1985. Random sampling with a reservoir. ACM Trans. Math.\n", " Softw. 11, 1 (March 1985), 37–57. https://doi.org/10.1145/3147.3165" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import torch\n", "from typing import Tuple\n", "from torch import Tensor\n", "\n", "\n", "class ReservoirSampler:\n", " def __init__(self, item_count: int, feature_count: int):\n", " self.item_count = item_count\n", " self.feature_count = feature_count\n", " self.reservoir_x = torch.zeros((item_count, feature_count))\n", " self.reservoir_y = torch.zeros((item_count,), dtype=torch.long)\n", " self.count = 0\n", "\n", " def update(self, x: Tensor, y: Tensor) -> None:\n", " batch_size = x.shape[0]\n", " assert x.shape == (\n", " batch_size,\n", " self.feature_count,\n", " )\n", " assert y.shape == (batch_size,)\n", "\n", " for i in range(batch_size):\n", " if self.count < self.item_count:\n", " # Fill the reservoir\n", " self.reservoir_x[self.count] = x[i]\n", " self.reservoir_y[self.count] = y[i]\n", " else:\n", " # Reservoir sampling\n", " index = torch.randint(0, self.count + 1, (1,))\n", " if index < self.item_count:\n", " self.reservoir_x[index] = x[i]\n", " self.reservoir_y[index] = y[i]\n", " self.count += 1\n", "\n", " def sample_n(self, n: int) -> Tuple[Tensor, Tensor]:\n", " indices = torch.randint(0, min(self.count, self.item_count), (n,))\n", " return self.reservoir_x[indices], self.reservoir_y[indices]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's check to see if it's samples look uniform." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAHz9JREFUeJzt3XtwVOXBx/HfkpBNwCSQYBKiCUktBbmIKJdGqMKYkaYRQVsrTqQpdLRqFDAOktQGGxU32I6DFwrqjEJbLupUUEFxmABGptwSCIqXADVABg2pRXZJ0BWzz/tHh31dCWjw7JNs/H5mzox7zrPnPHnA8J2z2azLGGMEAABgSbeOngAAAPhhIT4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgVXRHT+CbAoGAPv74Y8XHx8vlcnX0dAAAwHdgjNHx48eVnp6ubt3Ofm+j08XHxx9/rIyMjI6eBgAAOAcNDQ268MILzzqm08VHfHy8pP9NPiEhoYNnAwAAvgufz6eMjIzgv+Nn0+ni49RLLQkJCcQHAAAR5rv8yAQ/cAoAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXtjo+qqipNnDhR6enpcrlcWr169WljPvjgA1133XVKTExUz549NXLkSB06dMiJ+QIAgAjX7vhoaWnRsGHDtHDhwjaP//vf/9bYsWM1cOBAbdq0Se+8847KysoUGxv7vScLAAAin8sYY875yS6XVq1apcmTJwf3TZkyRd27d9ff//73czqnz+dTYmKivF4vHywHAECEaM+/347+zEcgENDatWv1k5/8RBMmTFBKSopGjx7d5kszp/j9fvl8vpANAAB0XdFOnqypqUnNzc2qqKjQww8/rPnz52vdunW64YYbtHHjRl111VWnPcfj8ai8vNzJaQBAxMgqWRu2cx+oyA/buYHvw/E7H5I0adIk3XPPPbr00ktVUlKia6+9VosXL27zOaWlpfJ6vcGtoaHBySkBAIBOxtE7H3369FF0dLQGDRoUsv/iiy/W5s2b23yO2+2W2+12choAAKATc/TOR0xMjEaOHKm6urqQ/Xv37lW/fv2cvBQAAIhQ7b7z0dzcrP379wcf19fXq7a2VklJScrMzNTs2bN100036corr9T48eO1bt06vfbaa9q0aZOT8wYAABGq3fFRXV2t8ePHBx8XFxdLkgoLC7VkyRJdf/31Wrx4sTwej2bMmKEBAwbon//8p8aOHevcrAEAQMRqd3yMGzdO3/arQaZPn67p06ef86QAAEDXxWe7AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq9odH1VVVZo4caLS09Plcrm0evXqM469/fbb5XK5tGDBgu8xRQAA0JW0Oz5aWlo0bNgwLVy48KzjVq1apa1btyo9Pf2cJwcAALqe6PY+IS8vT3l5eWcdc/jwYd1999168803lZ+ff86TAwAAXU+74+PbBAIBTZ06VbNnz9bgwYO/dbzf75ff7w8+9vl8Tk8JAAB0Io7Hx/z58xUdHa0ZM2Z8p/Eej0fl5eVOT+OMskrWhu3cByq4yxPJ+LsR+fgzDBWu9YjEtUDn4ui7XWpqavT4449ryZIlcrlc3+k5paWl8nq9wa2hocHJKQEAgE7G0fh4++231dTUpMzMTEVHRys6OloHDx7Uvffeq6ysrDaf43a7lZCQELIBAICuy9GXXaZOnarc3NyQfRMmTNDUqVM1bdo0Jy8FAAAiVLvjo7m5Wfv37w8+rq+vV21trZKSkpSZmank5OSQ8d27d1daWpoGDBjw/WcLAAAiXrvjo7q6WuPHjw8+Li4uliQVFhZqyZIljk0MAAB0Te2Oj3HjxskY853HHzhwoL2XAAAAXRif7QIAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACrojt6AgAQCbJK1nb0FIAugzsfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAqnbHR1VVlSZOnKj09HS5XC6tXr06eOzkyZOaM2eOhg4dqp49eyo9PV2/+c1v9PHHHzs5ZwAAEMHaHR8tLS0aNmyYFi5ceNqxEydOaOfOnSorK9POnTv18ssvq66uTtddd50jkwUAAJEvur1PyMvLU15eXpvHEhMTtX79+pB9Tz31lEaNGqVDhw4pMzPz3GYJAAC6jHbHR3t5vV65XC716tWrzeN+v19+vz/42OfzhXtKAACgA4U1Pr744gvNmTNHN998sxISEtoc4/F4VF5eHs5pADgHWSVrw3LeAxX5YTkvgMgRtne7nDx5Ur/+9a9ljNGiRYvOOK60tFRerze4NTQ0hGtKAACgEwjLnY9T4XHw4EFt2LDhjHc9JMntdsvtdodjGgAAoBNyPD5Ohce+ffu0ceNGJScnO30JAAAQwdodH83Nzdq/f3/wcX19vWpra5WUlKS+ffvqV7/6lXbu3Kk1a9aotbVVjY2NkqSkpCTFxMQ4N3MAABCR2h0f1dXVGj9+fPBxcXGxJKmwsFB/+tOf9Oqrr0qSLr300pDnbdy4UePGjTv3mQIAgC6h3fExbtw4GWPOePxsxwAAAPhsFwAAYBXxAQAArCI+AACAVcQHAACwivgAAABWER8AAMAq4gMAAFhFfAAAAKuIDwAAYBXxAQAArCI+AACAVcQHAACwivgAAABWER8AAMAq4gMAAFgV3dETAADglKyStWE574GK/LCcF+eGOx8AAMAq4gMAAFhFfAAAAKuIDwAAYBXxAQAArCI+AACAVcQHAACwivgAAABWER8AAMAq4gMAAFhFfAAAAKuIDwAAYBXxAQAArCI+AACAVcQHAACwivgAAABWER8AAMCqdsdHVVWVJk6cqPT0dLlcLq1evTrkuDFGc+fOVd++fRUXF6fc3Fzt27fPqfkCAIAI1+74aGlp0bBhw7Rw4cI2jz/66KN64okntHjxYm3btk09e/bUhAkT9MUXX3zvyQIAgMgX3d4n5OXlKS8vr81jxhgtWLBAf/zjHzVp0iRJ0t/+9jelpqZq9erVmjJlyvebLQAAiHiO/sxHfX29GhsblZubG9yXmJio0aNHa8uWLW0+x+/3y+fzhWwAAKDravedj7NpbGyUJKWmpobsT01NDR77Jo/Ho/LyciengXbIKlkblvMeqMgPy3kB4IciXN+fpY7/Ht3h73YpLS2V1+sNbg0NDR09JQAAEEaOxkdaWpok6ciRIyH7jxw5Ejz2TW63WwkJCSEbAADouhyNj+zsbKWlpamysjK4z+fzadu2bcrJyXHyUgAAIEK1+2c+mpubtX///uDj+vp61dbWKikpSZmZmZo1a5Yefvhh9e/fX9nZ2SorK1N6eromT57s5LwBAECEand8VFdXa/z48cHHxcXFkqTCwkItWbJE9913n1paWnTbbbfp2LFjGjt2rNatW6fY2FjnZg0AACJWu+Nj3LhxMsac8bjL5dKDDz6oBx988HtNDAAAdE0d/m4XAADww0J8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFZFd/QEAACRJatkbUdPod3COecDFflhO3dXxZ0PAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABglePx0draqrKyMmVnZysuLk4XXXSRHnroIRljnL4UAACIQNFOn3D+/PlatGiRli5dqsGDB6u6ulrTpk1TYmKiZsyY4fTlAABAhHE8Pv71r39p0qRJys/PlyRlZWVpxYoV2r59u9OXAgAAEcjxl12uuOIKVVZWau/evZKk3bt3a/PmzcrLy2tzvN/vl8/nC9kAAEDX5fidj5KSEvl8Pg0cOFBRUVFqbW3VvHnzVFBQ0OZ4j8ej8vJyp6fRIbJK1oblvAcq8sNyXgAAOoLjdz5efPFFLVu2TMuXL9fOnTu1dOlS/eUvf9HSpUvbHF9aWiqv1xvcGhoanJ4SAADoRBy/8zF79myVlJRoypQpkqShQ4fq4MGD8ng8KiwsPG282+2W2+12ehoAAKCTcvzOx4kTJ9StW+hpo6KiFAgEnL4UAACIQI7f+Zg4caLmzZunzMxMDR48WLt27dJjjz2m6dOnO30pAAAQgRyPjyeffFJlZWW688471dTUpPT0dP3+97/X3Llznb4UAACIQI7HR3x8vBYsWKAFCxY4fWoAANAF8NkuAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsCq6oyeAb5dVsrajp9BukTjncGI9gK6L/7/bjzsfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAqrDEx+HDh3XLLbcoOTlZcXFxGjp0qKqrq8NxKQAAEGGinT7hZ599pjFjxmj8+PF64403dP7552vfvn3q3bu305cCAAARyPH4mD9/vjIyMvT8888H92VnZzt9GQAAEKEcf9nl1Vdf1YgRI3TjjTcqJSVFw4cP17PPPnvG8X6/Xz6fL2QDAABdl+Px8dFHH2nRokXq37+/3nzzTd1xxx2aMWOGli5d2uZ4j8ejxMTE4JaRkeH0lAAAQCfieHwEAgFddtlleuSRRzR8+HDddtttuvXWW7V48eI2x5eWlsrr9Qa3hoYGp6cEAAA6Ecfjo2/fvho0aFDIvosvvliHDh1qc7zb7VZCQkLIBgAAui7H42PMmDGqq6sL2bd3717169fP6UsBAIAI5Hh83HPPPdq6daseeeQR7d+/X8uXL9czzzyjoqIipy8FAAAikOPxMXLkSK1atUorVqzQkCFD9NBDD2nBggUqKChw+lIAACACOf57PiTp2muv1bXXXhuOUwMAgAjHZ7sAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACrwh4fFRUVcrlcmjVrVrgvBQAAIkBY42PHjh16+umndckll4TzMgAAIIKELT6am5tVUFCgZ599Vr179w7XZQAAQIQJW3wUFRUpPz9fubm5Zx3n9/vl8/lCNgAA0HVFh+OkK1eu1M6dO7Vjx45vHevxeFReXh6OaQDohLJK1nb0FAB0MMfvfDQ0NGjmzJlatmyZYmNjv3V8aWmpvF5vcGtoaHB6SgAAoBNx/M5HTU2NmpqadNlllwX3tba2qqqqSk899ZT8fr+ioqKCx9xut9xut9PTAAAAnZTj8XH11Vfr3XffDdk3bdo0DRw4UHPmzAkJDwAA8MPjeHzEx8dryJAhIft69uyp5OTk0/YDAIAfHn7DKQAAsCos73b5pk2bNtm4DAAAiADc+QAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFYRHwAAwCriAwAAWEV8AAAAq4gPAABgFfEBAACsIj4AAIBVxAcAALCK+AAAAFY5Hh8ej0cjR45UfHy8UlJSNHnyZNXV1Tl9GQAAEKEcj4+33npLRUVF2rp1q9avX6+TJ0/qmmuuUUtLi9OXAgAAESja6ROuW7cu5PGSJUuUkpKimpoaXXnllU5fDgAARBjH4+ObvF6vJCkpKanN436/X36/P/jY5/OFe0oAAKADhfUHTgOBgGbNmqUxY8ZoyJAhbY7xeDxKTEwMbhkZGeGcEgAA6GBhjY+ioiLt2bNHK1euPOOY0tJSeb3e4NbQ0BDOKQEAgA4Wtpdd7rrrLq1Zs0ZVVVW68MILzzjO7XbL7XaHaxoAAKCTcTw+jDG6++67tWrVKm3atEnZ2dlOXwIAAEQwx+OjqKhIy5cv1yuvvKL4+Hg1NjZKkhITExUXF+f05QAAQIRx/Gc+Fi1aJK/Xq3Hjxqlv377B7YUXXnD6UgAAIAKF5WUXAACAM+GzXQAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFXEBwAAsIr4AAAAVhEfAADAKuIDAABYRXwAAACriA8AAGAV8QEAAKwiPgAAgFVhi4+FCxcqKytLsbGxGj16tLZv3x6uSwEAgAgSlvh44YUXVFxcrAceeEA7d+7UsGHDNGHCBDU1NYXjcgAAIIKEJT4ee+wx3XrrrZo2bZoGDRqkxYsXq0ePHnruuefCcTkAABBBop0+4ZdffqmamhqVlpYG93Xr1k25ubnasmXLaeP9fr/8fn/wsdfrlST5fD6npyZJCvhPhOW8AABEinD8G3vqnMaYbx3reHx8+umnam1tVWpqasj+1NRUffjhh6eN93g8Ki8vP21/RkaG01MDAACSEheE79zHjx9XYmLiWcc4Hh/tVVpaquLi4uDjQCCgo0ePKjk5WS6X65zP6/P5lJGRoYaGBiUkJDgxVZwF620X620Pa20X622Xk+ttjNHx48eVnp7+rWMdj48+ffooKipKR44cCdl/5MgRpaWlnTbe7XbL7XaH7OvVq5dj80lISOAvsEWst12stz2stV2st11Orfe33fE4xfEfOI2JidHll1+uysrK4L5AIKDKykrl5OQ4fTkAABBhwvKyS3FxsQoLCzVixAiNGjVKCxYsUEtLi6ZNmxaOywEAgAgSlvi46aab9J///Edz585VY2OjLr30Uq1bt+60H0INJ7fbrQceeOC0l3QQHqy3Xay3Pay1Xay3XR213i7zXd4TAwAA4BA+2wUAAFhFfAAAAKuIDwAAYBXxAQAArOqy8bFw4UJlZWUpNjZWo0eP1vbt2zt6ShHH4/Fo5MiRio+PV0pKiiZPnqy6urqQMV988YWKioqUnJys8847T7/85S9P+wVzhw4dUn5+vnr06KGUlBTNnj1bX331lc0vJeJUVFTI5XJp1qxZwX2stbMOHz6sW265RcnJyYqLi9PQoUNVXV0dPG6M0dy5c9W3b1/FxcUpNzdX+/btCznH0aNHVVBQoISEBPXq1Uu/+93v1NzcbPtL6fRaW1tVVlam7OxsxcXF6aKLLtJDDz0U8hkgrPe5q6qq0sSJE5Weni6Xy6XVq1eHHHdqbd955x397Gc/U2xsrDIyMvToo4+e+6RNF7Ry5UoTExNjnnvuOfPee++ZW2+91fTq1cscOXKko6cWUSZMmGCef/55s2fPHlNbW2t+8YtfmMzMTNPc3Bwcc/vtt5uMjAxTWVlpqqurzU9/+lNzxRVXBI9/9dVXZsiQISY3N9fs2rXLvP7666ZPnz6mtLS0I76kiLB9+3aTlZVlLrnkEjNz5szgftbaOUePHjX9+vUzv/3tb822bdvMRx99ZN58802zf//+4JiKigqTmJhoVq9ebXbv3m2uu+46k52dbT7//PPgmJ///Odm2LBhZuvWrebtt982P/7xj83NN9/cEV9SpzZv3jyTnJxs1qxZY+rr681LL71kzjvvPPP4448Hx7De5+711183999/v3n55ZeNJLNq1aqQ406srdfrNampqaagoMDs2bPHrFixwsTFxZmnn376nObcJeNj1KhRpqioKPi4tbXVpKenG4/H04GzinxNTU1GknnrrbeMMcYcO3bMdO/e3bz00kvBMR988IGRZLZs2WKM+d//FN26dTONjY3BMYsWLTIJCQnG7/fb/QIiwPHjx03//v3N+vXrzVVXXRWMD9baWXPmzDFjx4494/FAIGDS0tLMn//85+C+Y8eOGbfbbVasWGGMMeb99983ksyOHTuCY9544w3jcrnM4cOHwzf5CJSfn2+mT58esu+GG24wBQUFxhjW20nfjA+n1vavf/2r6d27d8j3kjlz5pgBAwac0zy73MsuX375pWpqapSbmxvc161bN+Xm5mrLli0dOLPI5/V6JUlJSUmSpJqaGp08eTJkrQcOHKjMzMzgWm/ZskVDhw4N+QVzEyZMkM/n03vvvWdx9pGhqKhI+fn5IWsqsdZOe/XVVzVixAjdeOONSklJ0fDhw/Xss88Gj9fX16uxsTFkvRMTEzV69OiQ9e7Vq5dGjBgRHJObm6tu3bpp27Zt9r6YCHDFFVeosrJSe/fulSTt3r1bmzdvVl5eniTWO5ycWtstW7boyiuvVExMTHDMhAkTVFdXp88++6zd8+rwT7V12qeffqrW1tbTfptqamqqPvzwww6aVeQLBAKaNWuWxowZoyFDhkiSGhsbFRMTc9oHAaampqqxsTE4pq0/i1PH8P9WrlypnTt3aseOHacdY62d9dFHH2nRokUqLi7WH/7wB+3YsUMzZsxQTEyMCgsLg+vV1np+fb1TUlJCjkdHRyspKYn1/oaSkhL5fD4NHDhQUVFRam1t1bx581RQUCBJrHcYObW2jY2Nys7OPu0cp4717t27XfPqcvGB8CgqKtKePXu0efPmjp5Kl9TQ0KCZM2dq/fr1io2N7ejpdHmBQEAjRozQI488IkkaPny49uzZo8WLF6uwsLCDZ9f1vPjii1q2bJmWL1+uwYMHq7a2VrNmzVJ6ejrr/QPV5V526dOnj6Kiok57F8CRI0eUlpbWQbOKbHfddZfWrFmjjRs36sILLwzuT0tL05dffqljx46FjP/6WqelpbX5Z3HqGP6npqZGTU1NuuyyyxQdHa3o6Gi99dZbeuKJJxQdHa3U1FTW2kF9+/bVoEGDQvZdfPHFOnTokKT/X6+zfR9JS0tTU1NTyPGvvvpKR48eZb2/Yfbs2SopKdGUKVM0dOhQTZ06Vffcc488Ho8k1jucnFpbp7+/dLn4iImJ0eWXX67KysrgvkAgoMrKSuXk5HTgzCKPMUZ33XWXVq1apQ0bNpx2y+3yyy9X9+7dQ9a6rq5Ohw4dCq51Tk6O3n333ZC/2OvXr1dCQsJp3/x/yK6++mq9++67qq2tDW4jRoxQQUFB8L9Za+eMGTPmtLeN7927V/369ZMkZWdnKy0tLWS9fT6ftm3bFrLex44dU01NTXDMhg0bFAgENHr0aAtfReQ4ceKEunUL/ecmKipKgUBAEusdTk6tbU5OjqqqqnTy5MngmPXr12vAgAHtfslFUtd9q63b7TZLliwx77//vrnttttMr169Qt4FgG93xx13mMTERLNp0ybzySefBLcTJ04Ex9x+++0mMzPTbNiwwVRXV5ucnByTk5MTPH7q7Z/XXHONqa2tNevWrTPnn38+b//8Dr7+bhdjWGsnbd++3URHR5t58+aZffv2mWXLlpkePXqYf/zjH8ExFRUVplevXuaVV14x77zzjpk0aVKbb08cPny42bZtm9m8ebPp378/b/1sQ2FhobnggguCb7V9+eWXTZ8+fcx9990XHMN6n7vjx4+bXbt2mV27dhlJ5rHHHjO7du0yBw8eNMY4s7bHjh0zqampZurUqWbPnj1m5cqVpkePHrzV9puefPJJk5mZaWJiYsyoUaPM1q1bO3pKEUdSm9vzzz8fHPP555+bO++80/Tu3dv06NHDXH/99eaTTz4JOc+BAwdMXl6eiYuLM3369DH33nuvOXnypOWvJvJ8Mz5Ya2e99tprZsiQIcbtdpuBAweaZ555JuR4IBAwZWVlJjU11bjdbnP11Veburq6kDH//e9/zc0332zOO+88k5CQYKZNm2aOHz9u88uICD6fz8ycOdNkZmaa2NhY86Mf/cjcf//9IW/bZL3P3caNG9v8Xl1YWGiMcW5td+/ebcaOHWvcbre54IILTEVFxTnP2WXM137FHAAAQJh1uZ/5AAAAnRvxAQAArCI+AACAVcQHAACwivgAAABWER8AAMAq4gMAAFhFfAAAAKuIDwAAYBXxAQAArCI+AACAVcQHAACw6v8AsBzPq4PW6yIAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from matplotlib import pyplot as plt\n", "\n", "x = torch.arange(0, 1_000).reshape(-1, 1).float()\n", "y = torch.zeros(1_000, dtype=torch.long)\n", "sampler = ReservoirSampler(500, 1)\n", "sampler.update(x, y)\n", "x = sampler.sample_n(200)\n", "plt.hist(x[0].numpy(), bins=20)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 10.2 Experience Replay\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from capymoa.base import BatchClassifier\n", "from capymoa.instance import Instance\n", "from capymoa.stream import Schema\n", "from capymoa.type_alias import LabelProbabilities\n", "from torch.nn.functional import cross_entropy\n", "import numpy as np\n", "from torch import nn\n", "\n", "\n", "class ExperienceReplay(BatchClassifier):\n", " def __init__(\n", " self,\n", " schema: Schema,\n", " model: nn.Module,\n", " reservoir_size: int,\n", " batch_size: int,\n", " learning_rate: float,\n", " device: str = \"cpu\",\n", " ):\n", " super().__init__(schema=schema, batch_size=batch_size)\n", " self.reservoir = ReservoirSampler(reservoir_size, schema.get_num_attributes())\n", " self.optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\n", " self.model = model.to(device)\n", " self.device = device\n", " self.batch_size = batch_size\n", "\n", " def batch_train(self, x: np.ndarray, y: np.ndarray):\n", " x: Tensor = torch.from_numpy(x)\n", " y: Tensor = torch.from_numpy(y).long()\n", "\n", " self.reservoir.update(x, y)\n", "\n", " replay_x, replay_y = self.reservoir.sample_n(self.batch_size)\n", " train_x = torch.cat((x, replay_x), dim=0).to(self.device)\n", " train_y = torch.cat((y, replay_y), dim=0).to(self.device)\n", "\n", " self.optimizer.zero_grad()\n", " y_hat = self.model(train_x)\n", " loss = cross_entropy(y_hat, train_y)\n", " loss.backward()\n", " self.optimizer.step()\n", "\n", " @torch.no_grad\n", " def predict_proba(self, instance: Instance) -> LabelProbabilities:\n", " x = torch.from_numpy(instance.x).to(self.device)\n", " y_hat: Tensor = self.model.forward(x)\n", " return y_hat.softmax(dim=0).cpu().numpy()\n", "\n", " def __str__(self) -> str:\n", " return \"ExperienceReplay\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 10.3 Multi Layer Perceptron\n", "\n", "We create a simple multi-layer perceptron (MLP) with a single hidden layer to demonstrate continual learning.\n", "\n", "The output layer of a neural network is often problematic in continual learning because of the extreme and shifting\n", "class imbalance between tasks. Lesort et al. (2021) suggest mitigating this by using a variant of weight normalization\n", "that parameterizes the weights as a magnitude (set to the unit vector) and a direction. \n", "\n", "* Lesort, T., George, T., & Rish, I. (2021). Continual Learning in Deep Networks:\n", " An Analysis of the Last Layer." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "class SimpleMLP(nn.Module):\n", " def __init__(self, schema: Schema, hidden_size: int):\n", " super().__init__()\n", " num_classes = schema.get_num_classes()\n", "\n", " self.fc1 = nn.Linear(schema.get_num_attributes(), hidden_size)\n", " self.fc2 = nn.Linear(hidden_size, num_classes, bias=False)\n", " self.fc2 = nn.utils.parametrizations.weight_norm(self.fc2, name=\"weight\")\n", " weight_g = self.fc2.parametrizations.weight.original0\n", " # Set the magnitude to the unit vector\n", " weight_g.requires_grad_(False).fill_(1.0 / (num_classes**0.5))\n", "\n", " def forward(self, x: Tensor) -> Tensor:\n", " x = torch.relu(self.fc1(x))\n", " x = self.fc2(x)\n", " return x" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e8bfff69ea104021b6e1b51ec4a45896", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Train & Eval: 0%| | 0/560000 [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from matplotlib import pyplot as plt\n", "\n", "fig, ax = plt.subplots(figsize=(8, 4))\n", "\n", "cmap = plt.get_cmap(\"tab10\")\n", "for t in range(5):\n", " ax.scatter(r.task_index, r.accuracy_matrix[:, t], color=cmap(t), label=f\"Task {t}\")\n", " ax.plot(r.anytime_task_index, r.anytime_accuracy_matrix[:, t], color=cmap(t))\n", "\n", "ax.set_xlabel(\"Task\")\n", "ax.set_xticks(range(6))\n", "ax.set_ylabel(\"Accuracy\")\n", "ax.set_title(\"SplitMNIST Per-Task Accuracy Over Tasks\")\n", "ax.legend(frameon=False)\n", "pass" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAGJCAYAAACZ7rtNAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAnuJJREFUeJzs3Xd4U2X7wPFvkrbpXnQzWvbeG0REQEBEcbFlOl8QldefiANEBVQEUVFcLEWWCoKi8CJDQJA9RTaljEKB0r2T8/vjNGlDV9KmTVvuz3XlSs7Jk3OetD3pnefc5340iqIoCCGEEEIIUQFpHd0BIYQQQgghikuCWSGEEEIIUWFJMCuEEEIIISosCWaFEEIIIUSFJcGsEEIIIYSosCSYFUIIIYQQFZYEs0IIIYQQosKSYFYIIYQQQlRYEswKIYQQQogKS4JZIYTZokWL0Gg0REZGmtfdc8893HPPPQ7rkxB3mq1bt6LRaPjxxx8d3RUhKgQJZoWowI4ePcpjjz1GeHg4rq6uVK1alZ49e/Lpp5+W2j6vXLnCW2+9xaFDh/I8N3LkSDQaDd7e3qSmpuZ5/vTp02g0GjQaDR9++KF5vemft0ajYf/+/flu19PT02LdPffcQ5MmTSzWZWRk8PHHH9OyZUu8vb3x9fWlcePGPP3005w4cQLAvJ+iblu3bi3yZ2EwGAgLC0Oj0fD7778X2V5YUhSF7777jrvvvhtfX1/c3d1p2rQpb7/9NsnJyY7unllkZKTVfze5vwgKIcqGk6M7IIQonp07d9KtWzdq1KjBU089RUhICBcvXuTvv//m448/5vnnn7fLfv73v/9ZLF+5coWpU6cSERFBixYt8rR3cnIiJSWFX375hQEDBlg89/333+Pq6kpaWlqB+3vrrbf45ZdfitXXRx99lN9//53Bgwfz1FNPkZmZyYkTJ/j111/p1KkTDRo04LvvvrN4zbfffsvGjRvzrG/YsGGR+9u8eTPR0dFERETw/fff06dPn2L1+05kMBgYMmQIK1eupEuXLrz11lu4u7uzfft2pk6dyg8//MAff/xBcHCwo7tKYGBgnr+PWbNmcenSJT766KM8bYUQZUuCWSEqqGnTpuHj48PevXvx9fW1eC4mJsZu+3FxcbGpvV6vp3PnzixbtixPMLt06VL69u3LTz/9lO9rW7Rowa+//sqBAwdo1aqVTfvdu3cvv/76K9OmTeO1116zeG7u3LnExcUBMGzYMIvn/v77bzZu3JhnvTWWLFlCq1atGDFiBK+99hrJycl4eHjYvJ3SlpWVhdFotPl3WZo++OADVq5cycsvv8zMmTPN659++mkGDBhA//79GTlyZJmPeKekpODu7m6xzsPDI8/fx/Lly7l161ax/m6EEPYlaQZCVFBnz56lcePGeQJZgKCgIItljUbDuHHj+P7776lfvz6urq60bt2abdu2Fbmf3DmzW7dupW3btgCMGjXKfGp10aJFFq8ZMmQIv//+uzmABDXYPH36NEOGDClwX88//zx+fn689dZbRfbrdmfPngWgc+fOeZ7T6XRUqVLF5m0WJjU1ldWrVzNo0CAGDBhAamoqa9asybft77//TteuXfHy8sLb25u2bduydOlSiza7d+/m/vvvx8/PDw8PD5o1a8bHH39sfr6g3OWRI0cSERFhXjadEv/www+ZM2cOtWvXRq/Xc/z4cTIyMpg8eTKtW7fGx8cHDw8PunTpwpYtW/Js12g08vHHH9O0aVNcXV0JDAykd+/e7Nu3D4CuXbvSvHnzfN9v/fr16dWrV6E/u5kzZ1KvXj1mzJiR5/l+/foxYsQI1q9fz99//w3AAw88QK1atfLdXseOHWnTpo3FuiVLltC6dWvc3Nzw9/dn0KBBXLx40aKNKVVl//793H333bi7u+f5ImSLDz/8kE6dOlGlShXc3Nxo3bp1vnmvGzdu5K677sLX1xdPT0/q169f5H7T09N54IEH8PHxYefOnQAkJiby4osvEhERgV6vJygoiJ49e3LgwIFivwchKiIJZoWooMLDw9m/fz/Hjh2zqv2ff/7Jiy++yLBhw3j77be5efMmvXv3tvr1oJ56f/vttwF1BO27774z5zzm9sgjj6DRaFi1apV53dKlS2nQoEGhI67e3t689NJL/PLLLzb/Qw4PDwfUVIasrCybXlsca9euJSkpiUGDBhESEsI999zD999/n6fdokWL6Nu3L7GxsUyaNIn33nuPFi1asH79enObjRs3cvfdd3P8+HFeeOEFZs2aRbdu3fj111+L3b+FCxfy6aef8vTTTzNr1iz8/f1JSEjgm2++4Z577uH999/nrbfe4vr16/Tq1StPDvSYMWN48cUXqV69Ou+//z6vvvoqrq6u5uDyiSee4MiRI3n+fvbu3cupU6cKHbHcsWMHt27dYsiQITg55X+CcPjw4QDmn8HAgQM5f/48e/futWh34cIF/v77bwYNGmReN23aNIYPH07dunWZPXs2L774Ips2beLuu++2+IIFcPPmTfr06UOLFi2YM2cO3bp1K/iHWgRTvvbbb7/N9OnTcXJy4vHHH2fdunXmNv/88w8PPPAA6enpvP3228yaNYsHH3yQv/76q8Dtpqam0q9fP3bu3Mkff/xBp06dAHj22WeZN28ejz76KJ9//jkvv/wybm5u/Pvvv8V+D0JUSIoQokL63//+p+h0OkWn0ykdO3ZUXnnlFWXDhg1KRkZGnraAAij79u0zr7tw4YLi6uqqPPzww+Z1CxcuVADl/Pnz5nVdu3ZVunbtal7eu3evAigLFy7Ms58RI0YoHh4eiqIoymOPPaZ0795dURRFMRgMSkhIiDJ16lTl/PnzCqDMnDnT/LotW7YogPLDDz8ocXFxip+fn/Lggw/mu93c/WrcuLF52Wg0Kl27dlUAJTg4WBk8eLDy2WefKRcuXCj05zh27FilOB+FDzzwgNK5c2fz8ldffaU4OTkpMTEx5nVxcXGKl5eX0r59eyU1NdXi9UajUVEURcnKylJq1qyphIeHK7du3cq3jen95v49mIwYMUIJDw83L5t+vt7e3hZ9Me0rPT3dYt2tW7eU4OBgZfTo0eZ1mzdvVgBl/PjxefZn6lNcXJzi6uqqTJw40eL58ePHKx4eHkpSUlKe15rMmTNHAZTVq1cX2CY2NlYBlEceeURRFEWJj49X9Hq98t///tei3QcffKBoNBrz7zkyMlLR6XTKtGnTLNodPXpUcXJyslhv+nv54osvCuxHQfr27Wvxc1cURUlJSbFYzsjIUJo0aaLce++95nUfffSRAijXr18vcNu5j4fExESla9euSkBAgHLw4EGLdj4+PsrYsWNt7rsQlY2MzApRQfXs2ZNdu3bx4IMPcvjwYT744AN69epF1apVWbt2bZ72HTt2pHXr1ublGjVq8NBDD7FhwwYMBoPd+zdkyBC2bt3K1atX2bx5M1evXi00xcDEx8eHF198kbVr13Lw4EGr96fRaNiwYQPvvvsufn5+LFu2jLFjxxIeHs7AgQPzjMiVxM2bN9mwYQODBw82r3v00UfRaDSsXLnSvG7jxo0kJiaaRzVv7y/AwYMHOX/+PC+++GKelBFTm+J49NFH81yMpNPpzHmzRqOR2NhYsrKyaNOmjcVI+E8//YRGo2HKlCl5tmvqk4+PDw899BDLli1DURRAvahrxYoV9O/fv9Dc4cTERAC8vLwKbGN6LiEhAVBH7fv06cPKlSvN+wNYsWIFHTp0oEaNGgCsWrUKo9HIgAEDuHHjhvkWEhJC3bp186RU6PV6Ro0aVWA/bOHm5mZ+fOvWLeLj4+nSpYvFz9b0O16zZg1Go7HQ7cXHx3Pfffdx4sQJtm7dmueCS19fX3bv3s2VK1fs0n8hKioJZoWowNq2bcuqVau4desWe/bsYdKkSSQmJvLYY49x/Phxi7Z169bN8/p69eqRkpLC9evX7d63+++/Hy8vL1asWMH3339P27ZtqVOnjlWvfeGFF/D19bU5d1av1/P666/z77//cuXKFZYtW0aHDh1YuXIl48aNK8a7yN+KFSvIzMykZcuWnDlzhjNnzhAbG0v79u0tUg1Meby3lxDLzZo2xVGzZs181y9evJhmzZrh6upKlSpVCAwMZN26dcTHx1v0KSwsDH9//0L3MXz4cKKioti+fTsAf/zxB9euXeOJJ54o9HWmQNUU1OYnv4B34MCBXLx4kV27dpn7uX//fgYOHGhuc/r0aRRFoW7dugQGBlrc/v333zwXR1atWtVuF8b9+uuvdOjQAVdXV/z9/QkMDGTevHkWP9uBAwfSuXNnnnzySYKDgxk0aBArV67MN7B98cUX2bt3L3/88QeNGzfO8/wHH3zAsWPHqF69Ou3ateOtt97i3LlzdnkvQlQkEswKUQm4uLjQtm1bpk+fzrx588jMzOSHH35waJ/0ej2PPPIIixcvZvXq1VaNypoUd3Q2t9DQUAYNGsS2bduoW7cuK1eutFsurSlg7dy5M3Xr1jXfduzYwa5du0oloCholLagUfXco4QmS5YsYeTIkdSuXZv58+ezfv16Nm7cyL333lvkKGF+evXqRXBwMEuWLDFvPyQkhB49ehT6OlPZsyNHjhTYxvRco0aNzOv69euHu7u7efR75cqVaLVaHn/8cXMbo9GIRqMxv7fbb19++aXFfvL7ORXH9u3befDBB3F1deXzzz/nt99+Y+PGjQwZMsRiJNnNzY1t27bxxx9/mPOOBw4cSM+ePfP8Lh966CEUReG9997L9/czYMAAzp07x6effkpYWBgzZ86kcePGUvNY3HEkmBWikjFd1R0dHW2x/vTp03nanjp1Cnd3d5tqY9py6nvIkCEcPHiQxMREiwt0rGE67T516lSbXnc7Z2dnmjVrRmZmJjdu3CjRtgDOnz/Pzp07GTduHD/88IPFbcWKFbi4uJgrFdSuXRug0IvsrGkD4Ofnl2+qxIULF6zu+48//kitWrVYtWoVTzzxBL169aJHjx556v7Wrl2bK1euEBsbW+j2dDodQ4YM4ccff+TWrVv8/PPPDB48GJ1OV+jrTFfyL126tMBg/NtvvwXUKgYmHh4ePPDAA/zwww8YjUZWrFhBly5dCAsLs+i7oijUrFmTHj165Ll16NCh0L4V108//YSrqysbNmxg9OjR9OnTp8CgXqvV0r17d2bPns3x48eZNm0amzdvzpMC0b9/fxYsWMDSpUsZO3ZsvtsKDQ3lP//5Dz///DPnz5+nSpUqTJs2ze7vT4jyTIJZISqoLVu2WIz4mPz222+AWh4pt127dlnk7l28eJE1a9Zw3333FRl85GbKhbQmB7Vbt2688847zJ07l5CQEKv3ATmjs2vWrMl3trHbnT59mqioqDzr4+Li2LVrF35+fnYpaG8alX3llVd47LHHLG4DBgyga9eu5jb33XcfXl5ezJgxI0/AaPrdtWrVipo1azJnzpw8P9Pcv9/atWtz4sQJi5SQw4cPF3oV/O1Mv+fc2929e7f5tL3Jo48+iqIo+X6RuP1v7oknnuDWrVs888wzJCUlWVV31d3dnZdffpmTJ0/y+uuv53l+3bp1LFq0iF69euUJPgcOHMiVK1f45ptvOHz4sEWKAaiVNHQ6HVOnTs3TV0VRuHnzZpH9Kw6dTodGo7EIziMjI/n5558t2uX3BcGUC5uenp7nueHDh/PJJ5/wxRdfMHHiRPN6g8Fgkb4Aakm+sLCwfLcjRGUmkyYIUUE9//zzpKSk8PDDD9OgQQMyMjLYuXMnK1asICIiIs9FLU2aNKFXr16MHz8evV7P559/DmDzyGft2rXx9fXliy++wMvLCw8PD9q3b59vjqZWq+WNN94o9nt84YUX+Oijjzh8+HCRkxEcPnyYIUOG0KdPH7p06YK/vz+XL19m8eLFXLlyhTlz5tgUtBfk+++/p0WLFlSvXj3f5x988EGef/5588QPH330EU8++SRt27ZlyJAh+Pn5cfjwYVJSUli8eDFarZZ58+bRr18/WrRowahRowgNDeXEiRP8888/bNiwAYDRo0cze/ZsevXqxZgxY4iJieGLL76gcePG5oukivLAAw+watUqHn74Yfr27cv58+f54osvaNSoEUlJSeZ23bp144knnuCTTz7h9OnT9O7dG6PRyPbt2+nWrZtF/nHLli1p0qQJP/zwAw0bNrR6sotXX32VgwcP8v7777Nr1y4effRR3Nzc2LFjB0uWLKFhw4YsXrw4z+tMudgvv/wyOp2ORx991OL52rVr8+677zJp0iQiIyPp378/Xl5enD9/ntWrV/P000/z8ssvW9VHW/Tt25fZs2fTu3dvhgwZQkxMDJ999hl16tSxSKd4++232bZtG3379iU8PJyYmBg+//xzqlWrxl133ZXvtseNG0dCQgKvv/46Pj4+vPbaayQmJlKtWjUee+wxmjdvjqenJ3/88Qd79+5l1qxZdn9/QpRrjiihIIQoud9//10ZPXq00qBBA8XT01NxcXFR6tSpozz//PPKtWvXLNoCytixY5UlS5YodevWVfR6vdKyZUtly5YtFu2sKc2lKIqyZs0apVGjRoqTk5NFma78SmjdrqjSXLebMmWKAhRZmuvatWvKe++9p3Tt2lUJDQ1VnJycFD8/P+Xee+9VfvzxxwL7Y0tprv379yuA8uabbxbYJjIyUgGUl156ybxu7dq1SqdOnRQ3NzfF29tbadeunbJs2TKL1+3YsUPp2bOn4uXlpXh4eCjNmjVTPv30U4s2S5YsUWrVqqW4uLgoLVq0UDZs2FBgaa7cP18To9GoTJ8+XQkPDzf/Dfz66695tqEoahmvmTNnKg0aNFBcXFyUwMBApU+fPsr+/fvzbPeDDz5QAGX69OmF/fjyMBgMysKFC5XOnTsr3t7eiqurq9K4cWNl6tSphZb2Gjp0qAIoPXr0KLDNTz/9pNx1112Kh4eH4uHhoTRo0EAZO3ascvLkSXOb2/+GbJFfaa758+ebj68GDRooCxcuNP/9mmzatEl56KGHlLCwMMXFxUUJCwtTBg8erJw6dcrcpqDj4ZVXXlEAZe7cuUp6erryf//3f0rz5s3NfzPNmzdXPv/882K9HyEqMo2i5HOeUghRqWg0GsaOHcvcuXMd3RVRCX388ce89NJLREZGmktkCSFEWZGcWSGEEMWmKArz58+na9euEsgKIRxCcmaFEELYLDk5mbVr17JlyxaOHj3KmjVrHN0lIcQdSoJZIYQQNrt+/TpDhgzB19eX1157jQcffNDRXRJC3KEkZ1YIIYQQQlRYkjMrhBBCCCEqLAlmhRBCCCFEhXXH5cwajUauXLmCl5eXTdNyCiGEEEKIsqEoComJiYSFhaHVFj72escFs1euXClw5h4hhBBCCFF+XLx4kWrVqhXa5o4LZr28vAD1h+Pt7e3g3gghhBBCiNslJCRQvXp1c9xWmDsumDWlFnh7e0swK4QQQghRjlmTEioXgAkhhBBCiApLglkhhBBCCFFhSTArhBBCCCEqLAlmhRBCCCFEhSXBrBBCCCGEqLAkmBVCCCGEEBWWBLNCCCGEEKLCcmgwu23bNvr160dYWBgajYaff/65yNds3bqVVq1aodfrqVOnDosWLSr1foo7mNEA57fD0R/Ve6PB0T0SQgghRC4ODWaTk5Np3rw5n332mVXtz58/T9++fenWrRuHDh3ixRdf5Mknn2TDhg2l3FNxRzq+FuY0gcUPwE9j1Ps5TdT1QgghhCgXNIqiKI7uBKgzPKxevZr+/fsX2GbixImsW7eOY8eOmdcNGjSIuLg41q9fb9V+EhIS8PHxIT4+XmYAEwU7vhZWDgduPzyyZyIZ8C00erCseyWEEELcEWyJ1yrUdLa7du2iR48eFut69erFiy++WOBr0tPTSU9PNy8nJCSUVveEidEIxixQDOppefO9Mfs0fTn4/qQoan8s+palPjZkwrqXyL+fCqCB9a9Cg76g1ZVxx4UQQgiRW4UKZq9evUpwcLDFuuDgYBISEkhNTcXNzS3Pa2bMmMHUqVPLqoslk3wDrh2DjGRIT4IM0820nJjrcbK6bMgqpc4oloGoRVBayDrlTsgpVSDhMlzYCTW7OLozQgghxB2tQgWzxTFp0iQmTJhgXk5ISKB69eoO7FEhInfADyMc3YtSpgFNOSmiodGqI6saXfZ99rIhE9KtGMFPulb6fRRCCCFEoSpUMBsSEsK1a5YBxLVr1/D29s53VBZAr9ej1+vLonsl5xEIgQ3BxQP0nuCSfdN7qutcvHI9zn7OyaX0+mMO8m4L9rROedeZls3PafNZpwONpvT6ay/nt6sXexXFM7joNkIIIYQoVRUqmO3YsSO//fabxbqNGzfSsWNHB/XIziI6w9i/Hd0LEd4JvMMgIZr882Y16vPhncq6Z0IIIYS4jUPP9yYlJXHo0CEOHToEqKW3Dh06RFRUFKCmCAwfPtzc/tlnn+XcuXO88sornDhxgs8//5yVK1fy0ksvOaL7orLS6qD3+9kLt48kZy/3fk8u/qoopFawEEJUag4dmd23bx/dunUzL5tyW0eMGMGiRYuIjo42B7YANWvWZN26dbz00kt8/PHHVKtWjW+++YZevXqVed9FJdfoQbX81vqJkHAlZ713mBrISlmuiuH42gJ+h+/L71AIISqJclNntqxInVlhE6MB/nxfvQU1hme3y4hsRSG1goUQosKqtHVmhShzWh3U7KoGs4b08hfIKgqkxUNqLKTcgpSb2Y9jISsNPIPAM0S99woB9yrFew+595N6S73wz3xBYvZFieXp4j6jQR2RlVrBQghR6UkwK0RRPIPU+6QYx/XhViQc+A5ijquBqilgTb1lW21fjU6tmuEVfFuQG6DWNE69lbNd8z5iITWuiP1oQO+Vq/qG6d4LnF1vq4yhzb9ShkarBsXBjSG0BfhUK36AfO5Py9SCPKRWsBBCVBYSzApRFFMwm54AmangnH8ZOLszGuDUBti3AM78QaEzpzl7gLs/uPll3/uDkx6Sr0PiNUi6qk7KoRjUx0lXgcO298nZXd2HYsyZ2ANFvaUnqLfE4r3dPNyrqEFtWEsIa1F4gJuZBpf3QeRfcGEHRFlZFURqBQshRIUnwawQRdF7g06vphkkxYBfeOnuL/GqOgq7fxEkXMpZX6ubelrcIzAnYHWvogaXzq5Fb9eQpQa3SVfV95F4VQ3mkq6pga7eU92Wm39OYGx+7J//fhQleza6pJxZ6kxBrmk5Kz2f2eKM+cweZ4S0OIg+DDH/qikTZzepNxP3gJzANqghXD+hBrCX94Ehw/aftdQKFkKICk+CWSGKotGoQU98VOkFs4oC5/9UR2FPrANj9jTFbv7Qcii0HgVVapdsHzon8A5Vb/ai0ahBsN4TvOy3WTLTIOYfuHIQrhyC6EPZAe4NdZT6zB95X+MZDOGd1XrNNTrCkkfVgF1qBQshRKUmwawQ1vAMUoPZZDvnzabEwuFlahB780zO+uodoM1oaPSQdaOulY2zK1Rtrd5MMtPg2j8QfVANcq+fBL+aavAafpca7OdOQejzQXY1Aw2WAa3UChZCiMpEglkhrGG+CMyOOZY3z8KXXdVT8aBeLNV8oDoKG9LEfvupLJxdoVpr9WYNqRUshBB3BAlmhbBGaVQ0uLhbDWQ9Q+CeV6HpY2pFAGE/jR5U84zXvwp7vlLTCkb8KiOyQghRiTh0OlshKgyPUghmE6PV+zrdoc0oCWRLi1YHte9VH2ckSyArhBCVjASzQlijNNIMErKDWa8Q+21T5M83+6K9Wxcc2w8hhBB2J8GsENYwlXBKvm6/bZpGZr3sWF1A5M+3hnqfFqfOZCaEEKLSkGBWCGuUxsisBLNlR++p1qgFGZ0VQohKRoJZIaxRGheAJV5V7+1Z91UUzFQfOE6CWSGEqEwkmBXCGqYLwDJT1JmtSspoyAlmZWS2bEjerBBCVEoSzAphDb0nOHuoj+2RapB8Q53GVaPNCZRF6ZKRWSGEqJQkmBXCWvZMNUjMLuLvEaROMytKn4zMCiFEpSTBrBDWMgWz9pjSVvJly56MzAohRKUkwawQ1rLnyKxpelXJly07ppHZuChQFMf2RQghhN1IMCuEtUy1Zu2SZmC6+EsmTCgzPtUBjXoRnz3rBQshhHAoCWaFsJaHHWvNmnJmvcJKvi1hHScX8K6qPpa8WSGEqDQkmBXCWna9AExGZh1C8maFEKLSkWBWCGvZ8wKwhOzZv+QCsLJlrmgQ6dBuCCGEsB8JZoWwll1zZmUqW4eQkVkhhKh0JJgVwlqeuXJmS3I1fFY6pMaqjyWYLVtSa1YIISodCWaFsJbpAjBDBqTFF387plFZnR7c/EreL2E9GZkVQohKR4JZIazl7Ap6H/VxSVINck+YoNGUvF/CeqaR2fhLYDQ4ti9CCCHsQoJZIWxhj4vAZMIEx/EKBZ0LGLMg4bKjeyOEEMIOJJgVwhaedqg1ay7LJcFsmdNqsydPQPJmhRCikpBgVghb2KPWbKKMzDqU5M0KIUSlIsGsELbwsEcwmytnVpS9yljRQFHAkOnoXgghhEM4OboDQlQo9hiZTZAasw5V0Udmk29CzHGI+TfX/b+QkQiBDaFaG/VWtQ0E1getztE9FkKIUiXBrBC2ME+cUJKcWQlmHaqijMymJ0LMibyBa2EXH8b8o94OLFaXXbwgrEVOcFutjUyhLISodCSYLQsZyQU/p9GpJZ+saqsFZ7ditk0BCir0rwEX9+K1zUwFxVhwP1w8itk2DZRCSifZ0tbZPacEVla6eiV7cdu6eqv3idFgNKoXFAFkZYCxkNO8Tm5qW0XJqWbg6pP/79DJNWc0rcjt5mpryFRr4BZEpwedUzHaZoEhvZC2LqBztr2t0QBZaQW31TqDk0sx2hohK7Xgtt5h6n3chaLbap3ASa8+VhTITLFP29uP+9gLEHMUrh6Da8fU+1vnC369bw0IbAAB9dX7wPrq32b0Ubh2BC7thysH1dHayO3qzfz+q0JYS6jeHiLugpBm6u9aPiPUxyX9jMjNdNyD9Z8RVrWVzwig9D4jbGpbRp8R5SWOyH1slyMSzJaF6WEFP1f3Phj6Q87yzDoF/4GH3wWj1uUsz2kKKTfzbxvWEp7emrP8WXuIj8q/bWADGLs7Z/nrbnD9RP5tfWrAS0dzlhf2Uf9p5se9CrxyLmd5yWNwYUf+bZ3d4fXonOWVT8Dp/+XfFuCtXJMWrH4ajq8puO1rV3L+sf3yIhxeWnDb/zsLHgHq4w2vwd5v8m937RjEX8w5Zb35bdj5acHb/c/fENQQ0hNyPhi/6Jx/26c2Q9XW6uPd82Dj5IK3O+JXqNlFfbx/Efz2csFth6yEer3Ux0dWwpr/FNz28UXQ+GH18Ylf4IeRBbd96HNoOVR9fHYTLB1QcNv7P4R2T6mPL+yExQ8U3Lbn29D5BfVx9CH4+t6C23Z9FbpNUh/fOAmfdyi4bdvs/SdGw80z8FnbQto+CX1nqY9TbsLM2gW3bT4EHp6nPs5MKfy4r9YWat0D0Ufg6pGc0fr8uFeB5oPVv5+ghrCoL8RFqbfbj5HcnxFGA3xQC9LiLNskXFZvJ35Vl5091BHbKwfVv8/8yGdEDms/IwBeOGL7ZwTA9lnw53sFt5XPCFVpfUZ0eh7ue1d9HH8RPm5WcNvS+oxo9BAM+DZnubzEEbmP7XJEglkhiqs4U9omFBK0iLLh5AounpCRVHgQWZou7VVv1qjaGnpNy7XCyok2tLrC82X13uoIY1o8nP+z6O0ZDerfvGIs4m+/BFM9F0YxwNEf1VSf8E6lsw8hRIWkUZSSTDJf8SQkJODj40N8fDze3t5ls9PycnpATiGW/BSiIRPezx5pefkMeAZmt7XyFOLZLfBdf/X08NNbCmgrpxCB0j2F+NU9am7pkB8gooARcrDfKURDpjqieu0YaJwgqIGayxrSHEKbgV9N0Hvmv93S/IzQ6dVRqqhdELkDLu5RR6JKQuucnafbTh3xDW1WeH54UZ8RJ9bBxjdzqoCAmirS8x2o38fK7UqaASCfEZJmUKHSDGyJ12Rktizk/lB1WFsb/gBtaZv7D92ubV2LblOctk56QF+ytm7+kBoLyddzglknF8Cl6G2aRgK9w6z7/Vm7XVD/AZj+Cdi1rVPOPy17ttXqrP8btqmttui2fuFqMBsfBS73Wbddjcb6Ptzedst0NZB184PndpWsLJu9PyNM6QttRqvLCVcg6m+4uFsNcq8eLfxL6O2MmXlHnr3C1MC2ejs1yA1tnv9xe/tnxPG1sOpp8ny5ToiGn55UT8M2erDoPpX1Z0S+bW04lkurrXxGZLe14jOiOG1L8hlRlHLRtnzmy4IEs0LYzjMoO5iNARrZ9trcwaxwnLKsaHBpP2z7UH3cd3b5ry/sHQZNHlFvoI7GZKaq/3w1GiD7XqPN/3HCZbiYHcxe2qNezJZ4Bf5dq95AHfkKbaYGtXovdZTPyUUdRTQ91rmoOan5niVS1P2tfxUa9JXyY0Lc4SSYFcJWnkHqxS/FqTVrrjEr5ZEcqqxqzWakwOpn1FPcTR7LCRArEhd320Zk/Gupt+YD1eWMZLhySA1sL2YHuMnX4fJ+9VZsiho4X9iZc4GTEOKOJMGsELYy15otRjArNWbLh7Iamd00FW6eVn/f988s3X2VVy4eal6yKTdZUdQvEZf2qXVzs9LU3FNDupr/mZWm5mrGRampGUUpSc1nIYRVDFlZnNi9gdRbl3Hzq0qD9r3QOZWfELL89ESIisI8pW0x/olKMFs+lMXI7Lk/YfcX6uMH54K7f+ntqyLRaMAvQr0V5vz2wssymZi+XIryy2hQR9CTruVUo5DUkArj4IbFhO2aSmNySnhd21iFKx2n0LLXCAf2LIcEs0LYqiRT2pquyJZg1rFMI7OptyAtIWcyDHtJi4efs2t0thkNdXvYd/t3gvBOav5uQjT5581q1OelTFf5dnwtrJ+YM1kMqL+33u9bd/GecKiDGxbTfOd4dSFXVcBA5SaBO8dzEMpFQKt1dAeEqHBMwWxh04rmx2jMCWbL+0VAlZ3eUy3YD6UzOvv7q5BwSS251fMd+2//TqDVqQEPkLe2bvZy7/dkhK88O74WVg63DGRB/YKycrj6vCi3DFlZhO2aSrrixE28iTQG8a+xOhmKFm32IRi6ayqGrELK05URGZkVwlbFHZlNvq5eCKTR5qQqCMfxDVdnvrl1AUKa2m+7//6qziCl0cLDXxRcP1YUrdGDavmtfEf23pORvfLMaFB/b1KNothuXL3IpX/+IjVyL9qUG2irt6N6616EVK9T4m0rRiNRp48Q88+fZMVdRpMWhy49HufMePSZCbgZEvEwJOBDMq7aLNxIIFCjzhB4VfElhDi0GgjhJv/s3kDjzn1L3KeSkGBWCFuZLwCzMWfWlC/rEWR9nUVRevzC4coB+47MJl2HX7Kn1+w0HmoUMmWmsE6jB9WA59v+ELkNWo+Gvh9WjgDIkKVOw3r+TzVHOCNJDdR9qoN3VfCpCt7V1HuPoJxJFcorRYHkG+pZidMb847IWjaWahS5JMTdJOroDhLP7UEfc5iw5H8J4QYBuRvd/BkOvcYlTShX/NqgrXU3EW16ExBSo8jtZ2akc/7YLmKPb0V/ZQ8RKUcJJ4Hwwl6U64SIQdGQgDvJipv69STXc6m3LtvyVkuF/EcVwlamUdWUm+rog7X/VBOlLFe5Yu+KBooCv74IKTcgqDF0e80+2xXqMVa9rRrMajQVN5A1GuH6v+rFgee3wYW/ID3ButdqnbMD3WpqoOsVnD0LV/bkBlrn7NmznNR7rXPOc7rsur1Ortk302N9dl1ffc6yRqP2MyMJ0hNz3RJuW85el3xDDUrjL6nBa2Gze+XnDqxGYQ4sT/6FU/RBgpKOU8N4mSa3tTMqGqJ01bju1RiDqx/+N/dTO/M01YimWuwvEPsL7Ps/LmircdW/Hc6176Zmm974BYaSnBjHuYN/knR6O14xe6mV9i/1NJa/mzTFmbP6BiR7RmDQ+4GbL1o3X5w9/XHx9OdG9EXqHfsQT00qnqThp0nGT5N3xjA3v6ql+NOyjgSzQtjKI0A9hawY1Q9yLyuvppYJE8oXe1c0OLwcTvyqBhGPfJkzbaWwD1P1g1uRjuyFbRQFbp3PCV7Pb1O/7OTm6gMRXaDWPepZn4Qr6pTCCZch/rJ6nxitzqwWd6H0ayNrnQufHrdIGvV9uHrDjVNFN78DqlEkJdzi/MEtJJ/egVfMPmql/0s9Td6pgq9ogrjq0ZCM4BZ41e5AeJOORHj7EZGrTULcTc7t30jaqa0E3NhDraxzhBsvEX7jEtxYBbvhiiaYION1mmpyzdyngXg8OO/WlLTQdvg27ErNpp1o7FpwDWlDVhY3js/GU0kz58jmZlQgRlOFBu17Ff+HYycSzAphK60O3APUC8CSY6wPZmXChPLFniOzcRfh91fUx90m2TcHV6j8aqr3FSWYvbAL1o6Dm2cs1zu7Q42OUKsr1LwbQpoVPdJsyFIDWtMIaPwlNQffkKkGnoYMtY0hI3vZdMsAY/Z6Uw1fc03f7MdZ6VjkteYOZLVOoPdWZ2kz3992c/PLGS32qaZWanFyUc9azWlyR1ajuHE1iqiDm8g4v5OAm/upmXWOpppcPwMNxOHJBbfGpAS2wL1mW6o37kRYUFWKGurw9q1Ci+6DoPsgAOJvXuPsvv+ReWYrQTf3UtN4gTDlGmjgKgFc8m6BoVp7gpp0I7x+K1rorD+roXNy4krHKQTuHI9RwSKgNWa/neiOUwgpB/VmHd6Dzz77jJkzZ3L16lWaN2/Op59+Srt27QpsP2fOHObNm0dUVBQBAQE89thjzJgxA1dXG+beFqKkPIPUQDbpGmBl4GJOM5CR2XLBNNIXd0EdQdPkM/RgDaMR1vxHPeVarR10esFuXRS5mH9fUbal95Q1RYF98+H3iWogqXWGam1zgteqbdRgzxY6J/Ctrt5Ko7+GzJzJKrLS1NQDvVdO2kFxmKpRrByOmmCZO6At39UoDFlZHP3zR9i/iEbJezGiIUPjQgbOZGhcyNS4kKVxIUvjTJbWBYNWj0HrgqLREZRyhmpKtGWuq0Yddb3i1RxD9Y6ENLmH6vVa0NyGwLIgPlWCadXrCej1BAA3r13iysm9BIQ3IjS8PiUdOmnZawQHgbBdUwnOVWc2RlOFaKkzq1qxYgUTJkzgiy++oH379syZM4devXpx8uRJgoLyXu29dOlSXn31VRYsWECnTp04deoUI0eORKPRMHv2bAe8A3HH8gyCa9hW0UByZssXn2qABjJT1HQRz8DibWfPV+rpY2d3tXqBXNxXOrzDck6BJ1wpncCupLLSYd1/4eB36nLjh6HfJ/avY2xPGk12Dq2NAbY1CqpG4RUKfcpfndlrl85y7n9fUDNqFS3ITgfJjrtdyR6xVsh/oDkXo6LhvFMEN/xa4lSzM9WadyOsWu0iR13toUpwNaoEV7PrNlv2GoGh+1D+uW0GsPIwImvi0J7Mnj2bp556ilGjRgHwxRdfsG7dOhYsWMCrr76ap/3OnTvp3LkzQ4YMASAiIoLBgweze/fuMu23EDmzgNkSzEqN2XLFSZ9dlP+yOjpbnGD2xmn4Y4r6+L53oEpt+/ZR5NDqwLcGxJ5VUw3KWzCbcAVWPAGX9wEa6DEFOr9Y/JHNysJUjeL8NvjuYUCBJ/9QKzSUA4asLI5t+xHjvsU0S95FcHY6QByenAjuR1CXUbh7+5OZnkJmehpZGalkpadiyEjFkJGGITMVY0Yaxsw0lKx03ELqUrPlvdT2rUJl+jTQOTk5vPxWYRwWzGZkZLB//34mTZpkXqfVaunRowe7du3K9zWdOnViyZIl7Nmzh3bt2nHu3Dl+++03nnjiiQL3k56eTnp6zhV8CQlWXjkqRGGKU2vWNDIhs3+VH77hajB7KxKqtbH99dtnq6dla98LbcbYvXviNn4ROcFseSrnFPW3ejo96Zp6QddjC6COzPpmptVB7W7q2RDTxW0ODmZjLp/n7IZ51Iz6iea5RmH/cWlKatMnaNJjGB3cPBzaR2E9hwWzN27cwGAwEBxsefFMcHAwJ06cyPc1Q4YM4caNG9x1110oikJWVhbPPvssr71WcAmcGTNmMHXqVLv2XQiba81mpUNqrPpYgtnywy8conYW7wpxowFOb1Af3zVBRuDKQnmsaLBvAfz2ipr+ENQIBn0P/rUc3avyybeGGszeugDVC742pjSdPbKThPXv0Cx5F0EWo7APEHrvczSu38Ih/RIlU84rMFvaunUr06dP5/PPP+fAgQOsWrWKdevW8c47BU8XOWnSJOLj4823ixcvlmGPRaVl65S2pnxZnV69+leUDyWpaHB5v1pr2NVHJkcoK+UpmM1Kh7Xj4deX1EC20UMwZqMEsoUxHW+lXV6sAPGx1/FfNYCWKTvRaRSOuzRlX6v3cZ14ig7PfUm4BLIVlsNGZgMCAtDpdFy7Zjmyde3aNUJC8r9A5s033+SJJ57gySefBKBp06YkJyfz9NNP8/rrr6PNZ3YUvV6PXi/1HoWd2ZpmkDtfVkbwyo+S1Jo9tV69r9NDLUwvSl95CWYTomHlE3BpL6CB7pPhrpfk2C6KvWs72+jEsldpTyIXtNVhwLc0atDKIf0Q9uewkVkXFxdat27Npk2bzOuMRiObNm2iY8eO+b4mJSUlT8Cqyy5toShFXF4ohD2Z0wysDGYlX7Z8KsnI7KnsFIN6ve3XH1G48hDMRu2Gr7qqgayrDwz9AbpImolVfLOnXbXXrHs2OH98L61jVgGQ2G0a4RLIVioOrWYwYcIERowYQZs2bWjXrh1z5swhOTnZXN1g+PDhVK1alRkzZgDQr18/Zs+eTcuWLWnfvj1nzpzhzTffpF+/fuagVogyYapmkBqrFiMvqqyNaWRWgtnyxTRSFH/JttqlcRfh2jF1Jji50KfsmH5fKTfU6VT1XmW7/1MbYPlQNa0gsKGaHysVLKxnTjOIKtPdKkYjyWtexklj5IBHF1p1eahM9y9Kn0OD2YEDB3L9+nUmT57M1atXadGiBevXrzdfFBYVFWUxEvvGG2+g0Wh44403uHz5MoGBgfTr149p06Y56i2IO5Wbnzo7jjFLnYmnqCtzE2VktlzyCi1e7VLThV/V24O7f+n1T1hy9QE3f/VL5K3Isp1pzZAF6yepfysNHlBrCpd1MF3RFffLYwkd2vgdLdMPka44E/LYh2WyT1G2HF7xdty4cYwbNy7f57Zu3Wqx7OTkxJQpU5gyZUoZ9EyIQmi14BGoXtiVHGNFMGsamZUJE8oVrU4NYGPPqXl81gaz5hQDx89Jfsfxi3BMMHvsJ7UsmJs/PPwl6D3Lbt+VRXG/PJZAWkoSwX+/C8DB6sPpULNBqe9TlL0KVc1AiHLFlovAErKrGXjLVLbljq15sxnJcO5P9bHky5Y9/5rqfVnmzRoNsG2m+rjTOAlki0ury555jzJLNTi44h3ClBiuUYVmg2QgrLKSYFaI4rKl1qxMZVt+2XqF9bk/wZCuXswSKKM8Zc4RF4H9sxpungZXX2j7VNnttzIqw4oGVy+eoUXkAgAutp2Eu6dPqe9TOIYEs0IUl7VT2iqKXABWntk6MmsqyVWvt1zB7ghlHcwajfDnB+rjjmPB1bts9ltZlWFFg8srX8ZNk8Fx5ya07iMz9FVmEswKUVzWphmkJ0BmsvpYgtnyxzRSZE1wpChSksvRyjqY/XcN3DgJeh9o/0zZ7LMyK6OKBsd3/U7rxC0YFA36fjPR5FOHXlQe8tsVorisnQXMNCrr6gMu7qXbJ2E73wj13prTntGHIekqOHtAxF2l2i1RAFMwGxel5rKWptyjsh2eU49hUTJlMAuYISsL/R+TANgX8BC1m3UqtX2J8kGCWSGKy9qRWZkwoXwzjcwmRkNmWuFtTaOytbuBk8ws6BDeVdWyeIaMnFz00nLiV4g5Dnpv6PBs6e7rTuFX+iOz+1Z9RG3DeRLwoN6g90ptP6L8kGBWiOKy9gIwyZct39yrqCOtAPEXC2+bO19WOIZWlyvvMrL09qMoOaOy7Z9Ra0uLkjONzCZcBkOm3Tcff/Ma9Y5/DMDx+uPwC5TP3TuBBLNCFJf5ArDrhbeTCRPKN40mV95sIac+E6/BlQPq47r3lX6/RMHKIm/25G9w7Si4eEKH/5Tefu40nkHg5AqKsegvj8VwYtkk/EgkUluDNo+9bPfti/JJglkhisuUZpAeD5mpBbczjcx6SzBbbpnz+CILbnP6f+p9WCvwCi71LolClHYwqyjw5/vq43ZPyyxv9qTR5Iys2znV4Pw/u2l9fTUASfdOw8m5iGnGRaUhwawQxeXqA7rsvMnC8mYlZ7b8s2ZkVlIMyo/SDmZPbVAv9nP2gI75z1ApSsDWcnhWUIxGUtb8H04aIwc87qbJXQ/abdui/JNgVoji0mhyVTQoJNVAcmbLv6KusM5Kh7Nb1Mcyha3jlWYwazEq+yR4VLH/Pu505pFZ+wWzB//3LY0zDpOmOBPy+Id2266oGCSYFaIkzBUNCrkIzDz7lwSz5VZRI7OR29VawV6hENq87Pol8ueXPaVt7Hn7b/vMJjU32skNOj5v/+0Lu1c0SEtJIvTvdwE4VH04YRH17bJdUXFIMCtESRQ1C5jRKDmzFUFRI7PmiRJ6yaxf5YEpGEq5AemJ9tuuosCf2aWc2o4Bz0D7bVvksHOawcHlbxPKda4SQPPBU+2yTVGxSDArREkUVWs2+TooBtBocwJfUf6YgqPUW5CWYPmcoki+bHnj6gNu2Rdl2XNa1HNb4NJe9Wr7TuPtt11hyY5pBvG3btDiwkIALredhJuHV4m3KSoeCWaFKImias2aUgw8gkDnVDZ9ErbTe+UER7f/g71+Qj0d6uQKNbuWfd9E/uydN6sosDU7V7b1KKlYUZpMv7uka4VXgrFC1NG/cNNkcEUTTKs+o0veN1EhSTArREkUNaWtOV82pGz6I4qvoLxZ06hszbtlOuLyxN7B7PltcPFvtUJJ5xfss02RPzc/cMkeQY0rWa3Z5KiDAFx1r49GKyHNnUp+80KURFFpBqZg1jusbPojiq+gvNnc+bKi/LB3MGua7av1CMlvL212rDXrFHMMgPSARiXtlajAJJgVoiQ8iqhmkCAjsxVGfiOzKbFwcbf6uK4Es+WKPYPZyB1wYQfoXKDziyXfniianxUTlVihStJpANyqtyhZf0SFJsGsECXhWcSUtuY0AxmZLffyG5k984c67WZwE/Ct7ph+ifzZM5g11ZVt+QT4VC359kTRTCOzJbiALz0thWoGNU0hpH5be/RKVFASzApREqYLwDKTIT0p7/OSM1tx5Dcya65iIKOy5Y4pmI27oJbAK64Lu9R8Wa0z3PWSXbomrOBb8lqzF08exFljIB4PgqvWslPHREUkwawQJaH3BOfsi4LyuwhMZv+qOHwj1Pu4C+qV7YZMOP2Huk5KcpU/3lVB6wSGjJwvjcWxfZZ633KojL6XJb8iajtbIfbcfgAuudSRi7/ucPLbF6KkCrsIzHwBmASz5Z5vdUADmSmQfAOi/ob0eHAPgKqtHd07cTudE/hkB5/FTTXISIZzW9XHHcfZo1fCWnZIMzBGHwUg0beBPXokKjAJZoUoqYJqzWalQ8pN9bGMzJZ/Tvqc31PchZwUg7r3gVbnuH6JgvlnT2tb3GA2ahcYM8GnBlSpY7duCSuY0gxSY4s9i5tX3L8A6MKa2atXooKSYFaIkvLInvLy9pFZU4qBTq/WVRTlnzlvNlJKclUEJb0I7Nyf6n3Nu2Wa4rLm6p3zuViMvFnFaKR6xjkA/Gu1smfPRAUkwawQJWUemb09mM118Zf8o6wYTKNF57bAzdNqTmbtex3bJ1EwczB7vnivP79Nva8lM7s5RAlSDa5ePI03yWQoOqrXl2D2TifBrBAl5VlArVmZMKHiMY3MHv1RvQ/vrI4gifKpJCOzKbEQfVh9HNHFXj0StihBRYNrp7Iv/tJVx0Xvas9eiQpIglkhSso8pe1ttWZlwoSKx/TPNStNvZcqBuVbSYLZC38BCgTUlws0HaUEFQ1SLx4C4KZXfTt2SFRUEswKUVIFXQAmEyZUPKZ/riaSL1u+mYLZ5Ov513kujCnFoObddu2SsEEJRmb1N48DYAhqbM8eiQpKglkhSsqjgFnAZMKEisc3VzBbpS5Uqe24voiiufrkuojIxtE908Vfki/rOL75TFRipeCUUwB4hre0Z49EBSXBrBAllTtnVlFy1puqGUjObMXhHabOBAUyKltRFCfVIPEq3DgJaNS8aFGqDEaFXWdvsubQZXadvYnBmP05mTvNIPdnZxES42Opqqhnwqo1aGfv7ooKyMnRHRCiwjMFs4Z0SIsHN191OeGKei8jsxWHVgdBDeHqEWjYz9G9Edbwi4ArB20LZk0pBqHNwN2/NHolsq0/Fs3UX44THZ9mXhfq48qUfo3oXS970ov0BEiLs7qE4aUTe2kIXKMKwQHy+SpkZFaIknN2A332Fe+mi8AURaayrageWwBDf4IaHRzdE2GN4ozMnjfVl5UUg9K0/lg0zy05YBHIAlyNT+O5JQdYfyo+J03LhlSDhMiDAES71bVbX0XFJsGsEPZwe3mu9ATITFYfSzBbsQTUhbo9HN0LYS1bg1lFgXOmi78kmC0tBqPC1F+Ok1/ygGnd1F+Oo5hqzdqQ86y5qk5jm+rfsGSdFJWGpBkIYQ8eQXDzTE4waxqVdfUBF3fH9UuIys7Pxiltb0VCfJQ6IUZ4x9LqldVSMrI4djmBI5fiSMkwEOCpJ8DThUAvPQGeegK99Lg6V6zplNOzDGz851qeEdncFCA6Po2boSEEgE0VDXwT1Yu/9NWal6yjotKQYFYIe/C8raKBOV9WRmWFKFXmkdkLYDSCtogTjqZ82WptwcWjVLt2u4wsIyeuJnD4UjxHLsZx5FI8p2MSMRZx7ZOX3okALz2BnnoCvFwI9NTj6+6Cs06Dk06Lk1aDk1aDTqfFWatBp9XgrNNm32vQabU46zS46LQ4O2nVe50WFye1nemmPq8hM0shJTOLlAwDqRkGktOzSMlUH6vrskjOfpyQmsn1pHRuJKab7xPSsqz+mcS6hKrBrJVpBlmZGdTIPA8aCKzbxur9iMpNglkh7OH2WrOSLytE2fCuqo6yGtIh6WrR1UPKMF/2ws1k9kbe4silOA5fiuffKwlkGIx52oV4u9Ksmg/+Hi7cSErnemI6N5IyuJ6YTobBSGJ6FonpWZy/kVzqfbYXnVaTU7WgsHamLyNWjsxePnuUcE0mKYqeqjUblaCHojKRYFYIe/AMVO+TY9T7RBmZFaJM6JzApzrcOq+mEBQWzCpKmUyWkJCWyXu/n2Dp7rwBmo+bM82q+dCiui/NqvnSvJoPQd75T8eqKAoJaVm5Atyc+7iUTAxGhUyDgsFoJNOoYDAoZBkVsozG7OeM5jaZBmP2TSEjS32cYTCSmZW97rYg20mrwc1Fh4eLE+4uOtxcdLi76HDPtezh4oSXq5OaGuGlpkcEZadHeOqd6PLBFq7Gp+WbN6sBQnxciajTCHZjdc7s9TP7CQeinGvRQFex0i9E6ZFgVgh7MI/MmoJZU41ZCWaFKHV+EWowG3sewjsV3C7mX7XiiJMbVCudU9Qb/rnK5DXHuJaQDkCbcD81cK2uBq41/N3RaDRWbUuj0eDj5oyPmzO1Az1Lpb8miqIGwhlZxuwUhJJfHz6lXyOeW3IADeQb0E7p1widf4q6EBelftko4meTefkwAPE+Mo2tyCHBrBD24HFbNQPJmRWi7Fhb0cA0KhveEZz0du1CTGIab639h9+Oql9kawZ4MOORpnSoVcWu+yktGo2aX+uss1+Ro95NQpk3rFWeOrPuLjpmD2hO7yahkJUOaCAzBZJv5JzlKoDHrRPqg5CmduunqPgkmBXCHm6/AMycMysFvYUodVYHs6Z8WfulGCiKwsp9F5m27l8S0rLQaTU8c3ctxnevW+GqEJSG3k1C6dkohD3nY9n07zW+2XGeQC+9GsiC+qXCOwwSLqupBkUEs2FpZwDwrdmqtLsuKhAJZoWwB1OaQXKMekW1OZiVqWyFKHXWBLOGLIjcoT6208VfkTeSmbTqKLvO3QSgWTUf3nukGY3CvO2y/cpCp9XQsXYVmlbzYdHOSC7cTCHyRjIRAdnVJHzD1WD2VmSh6R83rkYRQBxGRUP1Bq3LpvOiQpBJE4SwB4/s0QRjFqTcVK+qBhmZFaIsWBPMRh9WJzPR+0BoyeqTZhmMzNt6ll5ztrHr3E1cnbW80bchq57rJIFsITz1TrSJUKes3Xb6es4T5okTCq9ocOXEXgAu6cJw9/QplT6KikmCWSHswcklZ17xmONqUIsmZ8RWCFF6TMFscgxkFFC+ypRiEHEXaIt/+v/Y5Xge+uwv3l9/gvQsI13qBvC/F7vyZJdaONkx37Sy6lpPTcn682SuYNYvXL0voqJBStQhAK67yzS2wpIceULYiylwjT6cvRyklg0SQpQuN19w9VUfF1R833TxV63ipxj8fPAyD87dwT9XEvBxc+bDx5vz7eh21Kgis/xZq2s99SzWzrM3Sc8yqCt9s4PZIiZOcLr+DwAZgY1LrX+iYpJgVgh7MaUamIJZqWQgRNkpLNUgKx2i/lYfF/Pir+uJ6by55hhGBe5vGsIfE7ryWOtqVpfZEqqGoV4EeulJzTSwL/KWutLKNIOAZHUaW/fqLUqxh6IisjmYjYiI4O233yYqyvp5lIW4I5hHZg+p9xLMClF2/Guq9/kFs5f2QlaqWkIvsEGxNj9t3XES07JoWtWHTwe3ItDLvqW97hQajcY8OvvnqexUA1OaQfxF9QLafKSlJFHdcAmAsAbtSr2fomKxOZh98cUXWbVqFbVq1aJnz54sX76c9PT00uibEBWLqTzXzbPqvUyYIETZKWxk9lyuklzFGEn968wNfj50BY0Gpj3cBJ1WRmNLwhzMmvJmvcKypyTOyLl49jYXTx5Ap1G4hTcBITXKqquigihWMHvo0CH27NlDw4YNef755wkNDWXcuHEcOHCgNPooRMVgCmZNc93IyKwQZaewYLYE+bJpmQbe+PkYAMM7hNOsmm+xuidy3FUnAK0GTl5LJDo+Vb22wLuq+mQBebO3zu0H4JK+NhqtZEgKS8X+i2jVqhWffPIJV65cYcqUKXzzzTe0bduWFi1asGDBAhQlv8nr8vrss8+IiIjA1dWV9u3bs2fPnkLbx8XFMXbsWEJDQ9Hr9dSrV4/ffvutuG9DCPu5vXKBBLNClJ2Cgtn0JLi8T31cjHzZL/88x/kbyQR66flvL5lC1R78PFxoXt0XgG23pxoUUNFAiT4KQLJfo9LunqiAih3MZmZmsnLlSh588EH++9//0qZNG7755hseffRRXnvtNYYOHVrkNlasWMGECROYMmUKBw4coHnz5vTq1YuYmJh822dkZNCzZ08iIyP58ccfOXnyJF9//TVVq1Yt7tsQwn5MU9qaSDArRNkxBbNxFyzzLqN2qaXyfGvktLHS+RvJfLZVnXFq8gON8HZ1tk9fRd682SIuAvOOV6exdQqTaWxFXjbXDTpw4AALFy5k2bJlaLVahg8fzkcffUSDBjlJ9Q8//DBt27YtcluzZ8/mqaeeYtSoUQB88cUXrFu3jgULFvDqq6/mab9gwQJiY2PZuXMnzs7qh0pERIStb0GI0uF5WzArObNClB3vaqDRQVaamnfpnT37nnkKW9tSDBRFYfKaY2Rk15J9oJkcz/Z0d71A5vxxmh2nb5BlMOLkG6E+kU+agdFgoHrGOdBAlToFzxAm7lw2j8y2bduW06dPM2/ePC5fvsyHH35oEcgC1KxZk0GDBhW6nYyMDPbv30+PHj1yOqPV0qNHD3bt2pXva9auXUvHjh0ZO3YswcHBNGnShOnTp2MwGArcT3p6OgkJCRY3IUrF7cGsjMwKUXZ0TuBbXX2cO9XgXPGC2V+ORLP99A1cnLS881ATKcFlZ82r+eLj5kxCWhaHL8UVmmYQfeEUnppUMhQnqtVpVrYdFRWCzcHsuXPnWL9+PY8//rh5dPR2Hh4eLFy4sNDt3LhxA4PBQHCwZZ5hcHAwV6/mfzXjuXPn+PHHHzEYDPz222+8+eabzJo1i3fffbfA/cyYMQMfHx/zrXr16kW8QyGKyT0AyP6Hp9PnzAgmhCgbt+fNpsTCVTXX0pZ82fjUTN759TgA47rVISLAw359FADotBq61A0AsqsamNMM8gaz106rOc9RTuE4u0hJNJGXzcFsTEwMu3fvzrN+9+7d7Nu3zy6dKojRaCQoKIivvvqK1q1bM3DgQF5//XW++OKLAl8zadIk4uPjzbeLFy+Wah/FHUznBB7qhzNeIcUqASSEKIHbg9nI7YCi1pb1sn5q6Vn/O8n1xHRqBXjwTNda9u6lyGaRN2uaBSz+MhiyLNqlXzoEQKyXXIAn8mdzMDt27Nh8A8LLly8zduxYq7cTEBCATqfj2rVrFuuvXbtGSEhIvq8JDQ2lXr166HQ582o3bNiQq1evkpGRke9r9Ho93t7eFjchSo3pIjBTvp4QouzcHsyaSnLZkGJw+GIc3/2tjg6+278JeiddEa8QxWUKZo9cjuemxlc9o6UYIOGSRTvXm+oouTG4SVl3UVQQNgezx48fp1WrVnnWt2zZkuPHj1u9HRcXF1q3bs2mTZvM64xGI5s2baJjx475vqZz586cOXMGY64rVU+dOkVoaCguLi42vAshSokpb9Yr/y9kQohSdHswm3uyBCtkGYy8tvooigIPt6xKpzoBdu+iyBHk7UrDUG8UBXacjc3Jeb6tokFIymkAvMJblnUXRQVhczCr1+vzjKYCREdH4+RkW3GECRMm8PXXX7N48WL+/fdfnnvuOZKTk83VDYYPH86kSZPM7Z977jliY2N54YUXOHXqFOvWrWP69Ok2jQgLUapMtWa9ZGRWiDLnl2tK24QrcPM0aLQQ0dmql3/39wX+uZKAt6sTr93fsPT6KcwsZgMzpRrkqmgQf+s6oajlu6o1lGlsRf5sDmbvu+8+cx6qSVxcHK+99ho9e/a0aVsDBw7kww8/ZPLkybRo0YJDhw6xfv1680VhUVFRREdHm9tXr16dDRs2sHfvXpo1a8b48eN54YUX8i3jJYRD1LsPXH2hbo8imwoh7Mw0Mpt0DU5tUB+HNrfqYsxrCWnM+t8pAF7p3YBAL7nQqCyYgtltp6+jmILZXCOzl/7dC0A0gfj4yUi5yJ/NdWY//PBD7r77bsLDw2nZUh3yP3ToEMHBwXz33Xc2d2DcuHGMGzcu3+e2bt2aZ13Hjh35+++/bd6PEGWiyaPQ+BG5+EsIR3DzVb9MpsXBgW/VdVamGLz963GS0rNoUd2XIe1qlFYPxW1ah/vh4aLjRlIG17RBhIBFRYOkCwcAuOpeFyl2KApi88hs1apVOXLkCB988AGNGjWidevWfPzxxxw9elTKXgkBEsgK4Uim0dkrahBkzcVfW0/GsO5INFoNTHu4CVqtHMNlxcVJa85NPpzko67MlWagufYPAGlVZBpbUTCbR2ZBrSP79NNP27svQgghRMn4RUD0IfWx1hlqdCi0eVqmgclr1IBpVOeaNA7zKd3+iTy61gtk4/FrbI1xpxdYpBn4J54EQF+thUP6JiqGYgWzoFY1iIqKylMS68EHHyxxp4QQQohiMY3MAlRrCy6FT3jw2ZYzRMWmEOLtyks965Vu30S+THmzf0S7MsMFSIyGrHQyjVAj6wJoILiuTGMrCmZzMHvu3Dkefvhhjh49ikajQVEUAPNUf4VNLSuEEEKUqtzBbK3CUwwysows2hkJwOR+jfDUF3t8R5RAdX93agV6cO66QpbOHSdDCsRd5NK1m9TUZJGouBEaLl80RMFszpl94YUXqFmzJjExMbi7u/PPP/+wbds22rRpk+8FW0IIIUSZyR3MFnHx119nb5CYlkWgl57ejaU2tCPdXTcQ0HDDKbu8YdwFbp5V854vutRGq5PJK0TBbA5md+3axdtvv01AQABarRatVstdd93FjBkzGD9+fGn0UQghhLBOYH3Q6MDVB6oWfmp6/dGrAPRqHCwXfTlY1/pqqsGZzCrqirgLZF0+DECij0xjKwpnczBrMBjw8vIC1Clpr1y5AkB4eDgnT560b++EEEIIW3iHwdAf4InV4FTwzJBZBiP/O64Gs32aSNEnR+tQswouTlpOZ2QHs7cu4BF3AgBNaDMH9kxUBDYnCDVp0oTDhw9Ts2ZN2rdvzwcffICLiwtfffUVtWrVKo0+CiGEENar073IJnvOx3IrJRM/d2fa1/Qvg06Jwri56Ghf059L59QyXUrcBaqlnwXAr1YrR3ZNVAA2B7NvvPEGycnJALz99ts88MADdOnShSpVqrBixQq7d1AIIYSwt9+PqaOyPRsF46Sz+SSlKAVd6wWy92wQAMaoPfiRQJaipXp9CWZF4WwOZnv16mV+XKdOHU6cOEFsbCx+fn7migZCCCFEeWU0Kmz4R1IMypt76gey6jc1d1aXeBmAS7qqRLh7OrJbogKw6etoZmYmTk5OHDt2zGK9v7+/BLJCCCEqhANRt4hJTMdL70SnOlUc3R2RrXagJwZvy6mEb3jKxV+iaDYFs87OztSoUUNqyQohhKiwTCkG3RsGoXeSkk/lhUajoVX9COIVd/O6rACZxlYUzeZEoddff53XXnuN2NjY0uiPEEIIUWoURWF9djDbW1IMyp2u9QK5qASZlz3CWzqwN6KisDlndu7cuZw5c4awsDDCw8Px8LCcKvDAgQN265wQQghhT0cvx3M5LhU3Z515GlVRfnSqU4VdBNKESABC67d1bIdEhWBzMNu/f/9S6IYQQojCKIpCfGom1xLSuZaQxtWENGKy768lpJOaYSDM15Xqfu5U93enur8b1f3cCfTSyzUNuZhSDLo1CMTNRVIMyhtvV2cyvKpD8l5u4EtASHVHd0lUADYHs1OmTCmNfgghxB3LaFS4mZxBdHwqV+JSuRKXRnR8KtHxacQkpGcHrGmkZxlt3rbeSUs1Pzc1wPXLCXLDq3gQXsUdD73N/wbKnMGosOd8LDGJaQR5udKupj+6YszYJSkGFYN3WD04DVdc6xLg6M6ICqH8f4oJIUQloCgKB6LiOHE1gei4NDVozQ5Yo+PSyDBYF6j6ujsT4u1KkLcrwV56QnzUx27OOi7fSuXirRQuxqZw6VYq0fGppGcZOXs9mbPXk/PdXoCnnogqanAbUcWd8IDse38PfNyd7fkjKJb1x6KZ+stxouPTzOtCfVyZ0q+RzQHpyWuJnL+RjIuTlnsbBBX9AuEQbfs9zd5lUQR2Hu7orogKwuZgVqvVFnrKSiodCCGEJYNR4e1f/mHxrgsFttFoIMhLT6iPG1V93Qj1cSUk+xbs7UqItyuBXnpcna0/NZ6RZSQ6PpWLsTlB7sVbqUTFphB1M5lbKZncSErnRlI6+y7cyvN6X3dnwqt44KV3QkFBUVBvKIDpMZBrnZuLEzX83ajh704Nf1PKgzverrYHxuuPRfPckgPZW85xNT6N55YcYN6wVjYFtL8fVUdl764bgGcFGJG+U7l7V6HtM/Mc3Q1Rgdh8NK9evdpiOTMzk4MHD7J48WKmTp1qt44JIURlkJphYPzyg2w8fg1QC8NX83MjzNeNMB/1PjQ7YHVxsu9MVC5O2ux0Ao98n49PzSTqZgqRN5O5cDOZCzdTuJC9HJOYTlxKJnEpcXbpi5+7szmwNQW6oT6u6J10OOk0OGk1OGm16LQanHUaNBp4c82xPIEsqAG0Bpj6y3F6NgqxOuVAUgyEqJw0iqLk91lhs6VLl7JixQrWrFljj82VmoSEBHx8fIiPj8fb29vR3RFCVGI3ktIZs3gfhy/G4eKk5aMBLejbrGIEUikZWUTFphB5I4X0rJwzbhqNBg3qSDKABjXwNK2LT83kYmz26G+sOhp8Mzmj1Pq57KkOdKxd9MQH564nce+sP3HSatj/Rs9ykUIhhCiYLfGa3c6zdOjQgaefftpemxNCiArt7PUkRi7cw8XYVHzdnflmeBvaRPg7ultWc3dxokGINw1CSv6lPyk9i4u5gltToHs1Po0so4LBqJBpMGIwKmQZFbIMRtIyDaRmFp1HHJOYVmQbyKli0LF2FQlkhahk7BLMpqam8sknn1C1alV7bE4IISq0vZGxPPXtPuJSMqnh786iUW2pFXjnzi/vqXeiYag3DUOtD4x3nb3J4K//LrJdkJerVdv7/Vg0AH0kxUCISsfmYNbPz8/iAjBFUUhMTMTd3Z0lS5bYtXOVRUpGVoHPaTUaiws6SqttaobBfIHG7TRoLOot2tI2LdOAsZBMFXcXJ4e3dXPWmf9m07MMGIz2aevqpEObnauXkWUky1jwKJItbfVOOnMOoC1tMw1GMgu5It5Fp8VJp7W5bZbBWOiV9s46Lc7FaGswKhanr2/npNWac0htaWs0KqTZqa1OqzFPd6ooCqmZRbdddySaF1ccJNOg0KSqD/OGtqKKp4vF8VpWx31F/oxoUtWbYG891xLSC3xtsLeeJlW9832fubd7JiaRY5cT0ABd6gbkaS+fESr5jLC9bXE+I6xpW14/I8prbWabc2YXLVpkEcxqtVoCAwNp3749fn5+du+gvTkiZzbi1XUFPtetfiALR7UzLzd8c32Bf+Dta/qz4pmO5uVW72wktoBctGbVfFg77i7zcuf3NnM5LjXftnWDPNk4oat5uefsPzkdk5Rv26q+bvz16r3m5Qfn7uDIpfh82/p7uHDgzZ7m5YFf7mL3+fynQXZz1vHvO73Ny6MW7mHLyev5tgWIfK+v+fF/vt/Pb9lXKefn+Nu9zP/Y/rvyMD8duFRg2/1v9KCKpx6AN38+xnd/F3z1+fZXulHdX51DfPpv//LVtnMFtv3fS3dTL9gLgI82nuLjTacLbLtmbGeaV/cF4Ms/zzLj9xMFts2dL/jtrkgmr/mnwLYLRrbh3gbBAPyw7yL/9+ORAtt+NqSVObdz3ZFoxi4teGa/mY814/E2amHzzSeuMXrRvgLbvv1QY4Z3jACKHnmb1KcBz3StDcDhi3E89NlfBbZ9oXtdXupZD4BT1xK576NtBbZ9+u5avHZ/QwAuxqbQ5YMtBbZ9okM47/RvAsDNpHRav/tHgW0fbVWV+iFeTP+t4N+Xyf1NQ/h8aGvzsnxGqAr7jLBV7s+I3nO2ceJqYoFt5TNCJZ8RqtL7jKjGrAHNATWIbDR5Q4Fty+tnRO5ju7SVas7syJEji9svIYSotA5djOOnA5cBGNq+Bt/vjnJwjyo3L1cnEtMKHlXK7Wq8dXm1QoiKyeaR2YULF+Lp6cnjjz9usf6HH34gJSWFESNG2LWD9uaIkdnycnqgPJ1CLMu2cgpRJacQbW9rzWnBlIws/u+HI2w5eR2NBl6/vyGjO0eQVshsXeX1FGJ5/YwwGBX2Rd4iIS2T9Ewjr60+SpZRYe7gltzbMP/JD0zbvRqfRocZmwDY8t97CPbR52krnxEq+Yywva2kGZQeW+I1m4PZevXq8eWXX9KtWzeL9X/++SdPP/00J0+etL3HZUhKcwkh7OVmUjqjF+3l8KV4XJy0zBnYgvubygVGpe2930/wxZ9nCfNxZeOEroVOybt4ZyRT1v5Dyxq+rP5P5zLspRCiJGyJ12yu0B0VFUXNmjXzrA8PDycqSk6rCSHuHFPW/sPhS/H4uTuz7Kn2EsiWkRe616WanxtX4tOY88epQtvmVDEIKYuuCSEcwOZgNigoiCNH8iaFHz58mCpVii5cLYQQlcHpa4msO6oGSt+Obk/r8IpTQ7aic3PR8c5D6kU3C/6K5PiVhHzb3UxKZ0/2BWVSkkuIysvmYHbw4MGMHz+eLVu2YDAYMBgMbN68mRdeeIFBgwaVRh+FEKLc+WTzGRQFejcOoWk1H0d3547TrUEQfZuGYjAqvLb6aL65q/87fg2jopb5MlUVEEJUPjYHs++88w7t27ene/fuuLm54ebmxn333ce9997L9OnTS6OPQghRrpyJSeTXI1cAeL57HQf35s41uV8jvPROHLoYx9I9edPcTLN+yaisEJWbzcGsi4sLK1as4OTJk3z//fesWrWKs2fPsmDBAlxcXEqjj0IIUa7MzR6V7dkomMZhMirrKMHerrzcqz4AH6w/YTG1bXxKJjvP3ACgt+TLClGpFXs627p161K3bl179kUIIcq9c9eTWHtYHZV9obt8BjrasA7h/HTgEkcuxfPOr//y6eCWAPzx7zWyjAr1gj2pfQdPJSzEncDmkdlHH32U999/P8/6Dz74IE/tWSGEqGzmbj6DUYEeDYNoUlVGZR1Np9Uw/eGmaDXwy+ErbDulzhxoSjHoLSkGQlR6Ngez27Zt4/7778+zvk+fPmzbVvD0cEIIUdGdv5HMz4fUWb5e6F7Pwb0RJk2q+jCyk1oy8o2fj3EzKZ1tp9WgVkpyCVH52RzMJiUl5Zsb6+zsTEJC/uVRhBCiMjCNyt7bIEgqGJQzE+6rR6iPK1GxKYxcuJeMLCMRVdxpEOLl6K4JIUqZzcFs06ZNWbFiRZ71y5cvp1GjRnbplBBClDcXbuaMyo6XXNlyx1PvxJR+jQE4ejkeUFMMTFPPCiEqL5svAHvzzTd55JFHOHv2LPfeey8AmzZtYunSpfz4449276AQQpQHczefwWBU6FovkBbVfR3dHZGPXo2D6dEwiD/+jQEkxUCIO4XNwWy/fv34+eefmT59Oj/++CNubm40b96czZs34+8vM+AIISqfi7EprDqYnSvbQ0ZlyyuNRsPUh5pwMGoHYb5uNJNUECHuCBpFUfJOm2KDhIQEli1bxvz589m/fz8Gg8FefSsVCQkJ+Pj4EB8fj7e3t6O7I4SoAF796QjL916kS90AvhvT3tHdEUVIycjCSavFxcnmTDohRDlhS7xW7CN927ZtjBgxgrCwMGbNmsW9997L33//XdzNCSFEuXQxNoUf918C4EUZla0Q3F2cJJAV4g5iU5rB1atXWbRoEfPnzychIYEBAwaQnp7Ozz//LBd/CSEqpc+3niXLqHBXnQBah0sqlRBClDdWf3Xt168f9evX58iRI8yZM4crV67w6aeflmbfhBDCoS7HpfLj/ouAVDAQQojyyuqR2d9//53x48fz3HPPyTS2Qog7wudbzpBpUOhYqwrtasqorBBClEdWj8zu2LGDxMREWrduTfv27Zk7dy43btwozb4JIYTDXIlLZeU+dVRWKhgIIUT5ZXUw26FDB77++muio6N55plnWL58OWFhYRiNRjZu3EhiYmJp9lMIIcrUvK1nyTQotK/pT4daVRzdHSGEEAWw+XJPDw8PRo8ezY4dOzh69Cj//e9/ee+99wgKCuLBBx8sjT4KIUSZuhqfxoq9MiorhBAVQYlql9SvX58PPviAS5cusWzZsmJv57PPPiMiIgJXV1fat2/Pnj17rHrd8uXL0Wg09O/fv9j7FkKI233x51kyDEbaRvjRUUZlhRCiXLNLIT6dTkf//v1Zu3atza9dsWIFEyZMYMqUKRw4cIDmzZvTq1cvYmJiCn1dZGQkL7/8Ml26dClut4UQIo9rCWks3RMFwAvd66HRaBzcIyGEEIVxeFXp2bNn89RTTzFq1CgaNWrEF198gbu7OwsWLCjwNQaDgaFDhzJ16lRq1apVhr0VQlRml26l8Nbaf8jIMtI63I/OdWRUVgghyjubJk2wt4yMDPbv38+kSZPM67RaLT169GDXrl0Fvu7tt98mKCiIMWPGsH379kL3kZ6eTnp6unk5ISGh5B0XQlQYBqPCnvOxxCSmEeTlSrua/ui0OaOtmQYjm0/EsGxPFH+euo5pgu+XesiorBBCVAQODWZv3LiBwWAgODjYYn1wcDAnTpzI9zU7duxg/vz5HDp0yKp9zJgxg6lTp5a0q0KICmj9sWim/nKc6Pg087pQH1em9GtE4zAfVuy9yMp9F4lJzPnC26l2FUZ3rslddQMc0WUhhBA2cmgwa6vExESeeOIJvv76awICrPtHM2nSJCZMmGBeTkhIoHr16qXVRSFEObH+WDTPLTmActv66Pg0nl1yAA2Yn6vi4cJjbaoxqG0NagZ4lHFPhRBClIRDg9mAgAB0Oh3Xrl2zWH/t2jVCQkLytD979iyRkZH069fPvM5oNALg5OTEyZMnqV27tsVr9Ho9er2+FHovRNkyGBUS0zLJyDKSnmUkPcuQfW/MWZdpIMNgJD3TiEFRcHXW4Wa6uWhzll10uDqp93onrfl0usGokJ5lMG8vI9d+cq/TaMBJq8VZp8FJp8VJq8FZp8VJp8El+970vE6rQaPRoAE0GtBm70ujAQ0atBosnrfHqX2DUWHqL8fzBLK5KUDnOlUY0i6cno2CcXFy+CUEQgghisGhwayLiwutW7dm06ZN5vJaRqORTZs2MW7cuDztGzRowNGjRy3WvfHGGyQmJvLxxx/LiKuoFDKyjETeTOb0tSTOxCRxOiaRMzFJnLueTIbBWCr71DtpyTIqGIyFhX+lT6sBP3cX/Dxc8Pdwwd/dBX/P7HsPy5uPmzPxqZlcT0wnJjGNmIR0rmXfn7uRbJFaUJBx3erSsbZc5CWEEBWZw9MMJkyYwIgRI2jTpg3t2rVjzpw5JCcnM2rUKACGDx9O1apVmTFjBq6urjRp0sTi9b6+vgB51gtREZy/kcyRS3EWgWvkzZRCg0qtBvROOvTOWlx0WvTOWvROulyPtbg46dBpIC3TSGqmgbRMQ859hoG0TKNFYJyelTdIzm8/Ljp126BeOJVlMJJpUMgyqvfqupxlWxkVuJmcwc3kDJtfWxwxiUUHvEIIIco3hwezAwcO5Pr160yePJmrV6/SokUL1q9fb74oLCoqCq1WTv+JyufwxTge+uyvfJ/z1DtRJ8iTukGe1A32zH7sRbC3K846jV1OxWcZjKRlGbODWwPOOjUQNgWtTrqSHXeKophHexUFFEz36nPqffbj7PWZBiO3UjKITcogNiWD2OSCb3GpmXi7OhPkpSfYW0+QlytB3nqCvPTcSs5k9h+niuxjkJdrid6jEEIIx9MoiuLY84plLCEhAR8fH+Lj4/H29nZ0d8QdbOaGE3y25SzV/NzoUjfQHLiqQateykKVgMGocNf7m7kan5Zv3qwGCPFxZcfEey3KdAkhhCgfbInXHD4yK8SdaufZmwC80L0uj7eRfG970mk1TOnXiOduq1oAaiALMKVfIwlkhRCiEpDz90I4QGJaJkcuxQPIBUilpHeTUOYNa0WIj2UqQYiPK/OGtaJ3k1AH9UwIIYQ9ycisEA6w53wsBqNCeBV3qvm5O7o7lVbvJqH0bBRS6AxgQgghKjYJZoVwAFOKQafaMstUadNpNTL6LYQQlZikGQjhADnBrARZQgghRElIMCtEGYtNzuDf6AQAOtSSYFYIIYQoCQlmhShju7JHZesHexHoJVMtCyGEECUhwawQZWzn2RuAVDEQQggh7EGCWSHKmGlktnMdufhLCCGEKCkJZoUoQ9HxqZy7kYxWA+1q+ju6O0IIIUSFJ8GsEGXINCrbtKoPPm7ODu6NEEIIUfFJMCtEGTKV5Ooo9WWFEEIIu5BgVogyoigKO8+oF39JfVkhhBDCPiSYFaKMXLiZwpX4NJx1GtpGSL6sEEIIYQ8SzApRRkwpBi1r+OHmonNwb4QQQojKQYJZIcqIqb6spBgIIYQQ9iPBrBBlQFEUcyWDTnLxlxBCCGE3EswKUQZOXkvkZnIGrs5aWlT3dXR3hBBCiEpDglkhysDOM+qobNsIf1yc5LATQggh7EX+qwpRBnbKFLZCCCFEqZBgVohSlmUwsvucKV9WLv4SQggh7EmCWSFK2T9XEkhMz8LL1YnGYT6O7o4QQghRqUgwK0Qp+yu7JFeHWlXQaTUO7o0QQghRuUgwK0QpyynJJSkGQgghhL1JMCtEKUrPMrA3MhaQi7+EEEKI0iDBrBCl6FBUHGmZRgI8Xagb5Ono7gghhBCVjgSzQpQiU0mujrUD0GgkX1YIIYSwNwlmhShFO7Mv/pJ8WSGEEKJ0SDArRClJycjiYFQcIMGsEEIIUVokmBWilOyNvEWWUaGqrxs1/N0d3R0hhBCiUpJgVohSkjvFQPJlhRBCiNIhwawQpcRcX7aOpBgIIYQQpUWCWSFKQXxKJkcvxwPQsZbUlxVCCCFKiwSzQpSCv8/fRFGgVqAHIT6uju6OEEIIUWlJMCtEKZApbIUQQoiyIcGsEKXAdPFX59qSYiCEEEKUJglmhbCz64npnLqWBECHWjIyK4QQQpQmCWaFsDPTqGyjUG/8PFwc3BshhBCicpNgVgg7k3xZIYQQouxIMCuEne2U+rJCCCFEmZFgVgg7uhibQlRsCjqthnY1JZgVQgghSpsEs0LY0V9n1HzZ5tV88NQ7Obg3QgghROUnwawQdrL1ZAzv/HocgC51Ax3cGyGEEOLOIMGsEHbw/e4LjFm8j+QMAx1rVeHJLjUd3aU7zt13383SpUutbr9o0SJ8fX3Ny2+99RYtWrQwL7/66qs8//zzduyhEJXLPffcw4svvujobjjUm2++ydNPP+3obljo0KEDP/30k6O7UaYkmBWiBIxGhem//cvrq49hMCo82qoai0e3w8vV2dFdK1O7du1Cp9PRt29fh+x/7dq1XLt2jUGDBtltmy+//DKLFy/m3LlzdtumEKWhtI+/rVu3otFoiIuLs1i/atUq3nnnnVLZpzVSU1Px9/cnICCA9PT0Mt//1atX+fjjj3n99dfLfN+FeeONN3j11VcxGo2O7kqZkWBWiGJKyzQwdukBvtqmBjsTetbjw8eb4eLkuMPKYFTYdfYmaw5dZtfZmxiMSpnsd/78+Tz//PNs27aNK1eulMk+c/vkk08YNWoUWq39fvYBAQH06tWLefPm2W2bopIzGuD8djj6o3pvNJTJbh11/Pn7++Pl5VVm+7vdTz/9ROPGjWnQoAE///xzme//m2++oVOnToSHh5f5vgvTp08fEhMT+f333x3dlTIjwawQxXA9MZ1BX/3N78eu4qLTMmdgC8Z3r4tGo3FYn9Yfi+au9zcz+Ou/eWH5IQZ//Td3vb+Z9ceiS3W/SUlJrFixgueee46+ffuyaNGiPG1++eUX2rZti6urKwEBATz88MPm59LT05k4cSLVq1dHr9dTp04d5s+fb/X+r1+/zubNm+nXr5/F+tmzZ9O0aVM8PDyoXr06//nPf0hKSrLpvfXr14/ly5fb9Bpxhzq+FuY0gcUPwE9j1Ps5TdT1paio4880qrpp0ybatGmDu7s7nTp14uTJkwBERkai1WrZt2+fxevmzJlDeHg4586do1u3bgD4+fmh0WgYOXIkkDfNICIignfffZfhw4fj6elJeHg4a9eu5fr16zz00EN4enrSrFmzPPvasWMHXbp0wc3NjerVqzN+/HiSk5OLfO/z589n2LBhDBs2LN/PjH/++YcHHngAb29vvLy86NKlC2fPnjU/v2DBAho3boxeryc0NJRx48YVuc/cli9fnudz58cff6Rp06a4ublRpUoVevToYfFevvnmGxo2bIirqysNGjTg888/t3j9xYsXGTBgAL6+vvj7+/PQQw8RGRlpfn7kyJH079+fDz/8kNDQUKpUqcLYsWPJzMw0t9HpdNx///131GeXBLNCFCK/kc4zMYk8/PlfHLoYh6+7M9+NaUf/llUd2s/1x6J5bskBouPTLNZfjU/juSUHSjWgXblyJQ0aNKB+/foMGzaMBQsWoCg5I8Lr1q3j4Ycf5v777+fgwYNs2rSJdu3amZ8fPnw4y5Yt45NPPuHff//lyy+/xNPT0+r979ixA3d3dxo2bGixXqvV8sknn/DPP/+wePFiNm/ezCuvvGLTe2vXrh2XLl2y+GciRB7H18LK4ZBw26hoQrS6vhQD2qKOP5PXX3+dWbNmsW/fPpycnBg9ejSgBqA9evRg4cKFFu0XLlzIyJEjCQ8PN+dfnjx5kujoaD7++OMC+/PRRx/RuXNnDh48SN++fXniiScYPnw4w4YN48CBA9SuXZvhw4eb+3j27Fl69+7No48+ypEjR1ixYgU7duwoMrA8e/Ysu3btYsCAAQwYMIDt27dz4cIF8/OXL1/m7rvvRq/Xs3nzZvbv38/o0aPJysoCYN68eYwdO5ann36ao0ePsnbtWurUqWPFT1wVGxvL8ePHadOmjXlddHQ0gwcPZvTo0fz7779s3bqVRx55xPxev//+eyZPnsy0adP4999/mT59Om+++SaLFy8GIDMzk169euHl5cX27dv566+/8PT0pHfv3mRkZJj3s2XLFs6ePcuWLVtYvHgxixYtyvMlpl27dmzfvt3q91PhKeXA3LlzlfDwcEWv1yvt2rVTdu/eXWDbr776SrnrrrsUX19fxdfXV+nevXuh7W8XHx+vAEp8fLw9ui4qsd+PXlE6TP9DCZ/4q/nW8u0NSsM3f1fCJ/6q3P3BZuVsTKKju6lkGYx5+pn7FjHxV6XD9D+ULIOxVPbfqVMnZc6cOYqiKEpmZqYSEBCgbNmyxfx8x44dlaFDh+b72pMnTyqAsnHjxmLv/6OPPlJq1apVZLsffvhBqVKlinl54cKFio+Pj3l5ypQpSvPmzS1eY/q82Lp1a7H7Jyo5Q5aizGqgKFO8C7j5KMqshmq7UlDU8bdlyxYFUP744w/zunXr1imAkpqaqiiKoqxYsULx8/NT0tLSFEVRlP379ysajUY5f/68xTZu3bplse+uXbsqL7zwgnk5PDxcGTZsmHk5OjpaAZQ333zTvG7Xrl0KoERHRyuKoihjxoxRnn76aYvtbt++XdFqteb+5ee1115T+vfvb15+6KGHlClTppiXJ02apNSsWVPJyMjI9/VhYWHK66+/XuD2i3Lw4EEFUKKioszr9u/frwBKZGRkvq+pXbu2snTpUot177zzjtKxY0dFURTlu+++U+rXr68YjTmf1enp6Yqbm5uyYcMGRVEUZcSIEUp4eLiSlZXz9/T4448rAwcOtNjumjVrFK1WqxgMhmK/R0ezJV5z+MjsihUrmDBhAlOmTOHAgQM0b96cXr16ERMTk2/7rVu3MnjwYLZs2cKuXbuoXr069913H5cvXy7jnovKrKCRztjkTFIyDNQK9GD1fzpTK9D6EcTSsud8bJ5+5qYA0fFp7Dkfa/d9nzx5kj179jB48GAAnJycGDhwoMUpv0OHDtG9e/d8X3/o0CF0Oh1du3Ytdh9SU1NxdXXNs/6PP/6ge/fuVK1aFS8vL5544glu3rxJSkqK1dt2c3MDsOk14g5zYWfeEVkLCiRcVtvZmTXHn0mzZs3Mj0NDQwHM/2f79++PTqdj9erVgFrpo1u3bkRERNjcp9z7CQ4OBqBp06Z51pn2ffjwYRYtWoSnp6f51qtXL4xGI+fPn893HwaDgcWLFzNs2DDzumHDhrFo0SLzRU+HDh2iS5cuODvnvRg3JiaGK1euFPi5ZI3U1FQAi8+e5s2b0717d5o2bcrjjz/O119/za1btwBITk7m7NmzjBkzxuK9vvvuu+bUh8OHD3PmzBm8vLzMz/v7+5OWlmaRHtG4cWN0Op15OTQ0NE/M5ObmhtFodMiFcY7g8Krus2fP5qmnnmLUqFEAfPHFF6xbt44FCxbw6quv5mn//fffWyx/8803/PTTT2zatInhw4eXSZ9F5WYwKkz95TiFXTqVkp6Fj1v5qFgQk1hwIFucdraYP38+WVlZhIWFmdcpioJer2fu3Ln4+PiYA8L8FPactQICAsz/MEwiIyN54IEHeO6555g2bRr+/v7s2LGDMWPGkJGRgbu7u1Xbjo1VvwAEBkrdYFGApGv2bWcDa44/k9xBnSm33xT4ubi4MHz4cBYuXMgjjzzC0qVLC00lKEx++yls30lJSTzzzDOMHz8+z7Zq1KiR7z42bNjA5cuXGThwoMV6g8HApk2b6NmzZ5l87gDcunXL/Pmg0+nYuHEjO3fu5H//+x+ffvopr7/+Ort37zZ/5nz99de0b9/eYlumwDQpKYnWrVvniXPA8jPo9gBdo9HkqVwQGxuLh4eHXd5rReDQYDYjI4P9+/czadIk8zqtVkuPHj3YtWuXVdtISUkhMzMTf3//fJ9PT0+3+GaSkJBQsk6Xov0XYvl08xmMilryyagoGIwKigIGRV1W16sBlzGfvCh7MW1aQcn1GHPuj3nPSs7j3M/l+/qyubDeJvn1OT3LwK2UzIJfBFxNSGfP+Vg61nb8lLVBXnlHJUvSzlpZWVl8++23zJo1i/vuu8/iuf79+7Ns2TKeffZZmjVrxqZNm8xfWHNr2rQpRqORP//8kx49ehSrHy1btuTq1avcunULPz8/APbv34/RaGTWrFnmCgcrV660edvHjh3D2dmZxo0bF6tv4g7gGWzfdlay9viz1pNPPkmTJk34/PPPycrK4pFHHjE/5+LiAqjBor21atWK48eP25SvOn/+fAYNGpSnJNa0adOYP38+PXv2pFmzZixevJjMzMw8wZ+XlxcRERFs2rTJfHGbrWrXro23tzfHjx+nXr165vUajYbOnTvTuXNnJk+eTHh4OKtXr2bChAmEhYVx7tw5hg4dmu82W7VqxYoVKwgKCsLb27tY/TI5duwYLVu2LNE2KhKHBrM3btzAYDCYTzuYBAcHc+LECau2MXHiRMLCwgr8RzhjxgymTp1a4r6WhRtJGWw9ed3R3RBWKo2RzuJoV9OfUB9Xrsan5TuarAFCfFxpVzP/L3zF9euvv3Lr1i3GjBljMQIE8OijjzJ//nyeffZZpkyZQvfu3alduzaDBg0iKyuL3377jYkTJxIREcGIESMYPXo0n3zyCc2bN+fChQvExMQwYMAAABo0aMCMGTMsKiDk1rJlSwICAvjrr7944IEHAKhTpw6ZmZl8+umn9OvXj7/++osvvvjC5ve4fft281XWQuQrvBN4h6kXexV0BHqHqe3syNrjz1oNGzakQ4cOTJw4kdGjR1v8zYeHh6PRaPj111+5//77cXNzs+kizcJMnDiRDh06MG7cOJ588kk8PDw4fvw4GzduZO7cuXnaX79+nV9++YW1a9fSpEkTi+eGDx/Oww8/TGxsLOPGjePTTz9l0KBBTJo0CR8fH/7++2/atWtH/fr1eeutt3j22WcJCgoyl7L666+/zBOlDB8+nKpVqzJjxox8+20aeNuxYwf9+/cHYPfu3WzatIn77ruPoKAgdu/ezfXr180Xp06dOpXx48fj4+ND7969SU9PZ9++fdy6dYsJEyYwdOhQZs6cyUMPPcTbb79NtWrVuHDhAqtWreKVV16hWrVqVv9ct2/fnudLTmXm8DSDknjvvfdYvnw5W7duzTdnDmDSpElMmDDBvJyQkED16tXLqos2aVLVh5mPNUOn1aDVaNBqNWg1oNNo0Gg02evJXq/JXl96/dHkeqDJXtJoctabThdZrst5kemxJnfb0uuuzXL6p7FYPnIpntdWHy3y9fYe6SwunVbDlH6NeG7JATRY/js1/byn9GuETmvfn/78+fPp0aNHnn+koP4z/eCDDzhy5Aj33HMPP/zwA++88w7vvfce3t7e3H333ea28+bN47XXXuM///kPN2/epEaNGrz22mvm50+ePEl8fHyB/dDpdIwaNYrvv//eHMw2b96c2bNn8/777zNp0iTuvvtuZsyYYXMq0vLly3nrrbdseo24w2h10Pt9tWpBQUdg7/fUdnZk7fFnizFjxrBz505zpQOTqlWrMnXqVF599VVGjRrF8OHD8y3BVxzNmjXjzz//5PXXX6dLly4oikLt2rXzpBCYfPvtt3h4eOSb79q9e3fc3NxYsmQJ48ePZ/Pmzfzf//0fXbt2RafT0aJFCzp37gzAiBEjSEtL46OPPuLll18mICCAxx57zLytqKioIutWP/nkkzz11FN88MEHaLVavL292bZtG3PmzCEhIYHw8HBmzZpFnz59zO3d3d2ZOXMm//d//4eHhwdNmzY1lzdzd3dn27ZtTJw4kUceeYTExESqVq1K9+7dbRqpvXz5Mjt37mTJkiVWv6ai0yiK407+mnLXfvzxR/M3G1D/yOLi4lizZk2Br/3www959913+eOPPyxKYxQlISEBHx8f4uPjSzyMLyong1Hhrvc3FznSuWPivXYPEEti/bFopv5y3OJisFAfV6b0a0TvJqEO7Fnpu3r1Ko0bN+bAgQN2K2D++++/89///pcjR47g5FShv/eLsnB8LayfaHkxmHdVNZBt9KDj+mWDd955hx9++MHmIPhOpSgK7du356WXXjJfhFceTJw4kVu3bvHVV185uislYku85tBPaBcXF1q3bs2mTZvMwazRaGTTpk2F1pj74IMPmDZtGhs2bLApkBXCGo4a6Syp3k1C6dkohD3nY4lJTCPIS00tKG/9LA0hISHMnz+fqKgouwWzycnJLFy4UAJZYZ1GD0KDvmrVgqRrao5seCe7j8iWhqSkJCIjI5k7dy7vvvuuo7tTYWg0Gr766iuOHi36TF5ZCgoKsjgjfSdw6MgsqKW5RowYwZdffkm7du2YM2cOK1eu5MSJEwQHB+fJW3n//feZPHkyS5cuNZ8uAMxlLIoiI7PCWnfySKcQ4s4xcuRIli1bRv/+/Vm6dKlF2SchHMWWeM3hwSzA3LlzmTlzJlevXqVFixZ88skn5tIV99xzDxEREeb8nIiICItZPkymTJliVW6bBLPCFgajckeOdAohhBCOVOGC2bIkwawQQgghRPlmS7zm8BnAhBBCCCGEKC4JZoUQQgghRIUlwawQQgghhKiwJJgVQgghhBAVlgSzQgghhBCiwpJgVghRYd1zzz3mqSDvVG+++SZPP/201e0jIyPRaDQcOnQIgK1bt6LRaIiLiwNg/fr1tGjRAqPRWAq9FZWJHH+2H39loUOHDvz000+O7kaZkmBWCFFiu3btQqfT0bdv31LZ/u0Bl8mqVat45513SmWf1khNTcXf35+AgADS09PLfP9Xr17l448/5vXXX7fbNnv37o2zszPff/+93bYpSpccf5Xn+LOHN954g1dfffWO+kIqwawQosTmz5/P888/z7Zt27hy5UrRL7ATf39/vLy8ymx/t/vpp59o3LgxDRo04Oeffy7z/X/zzTd06tTJblP4mowcOZJPPvnErtsUpUeOv8p1/JVUnz59SExM5Pfff3d0V8qMBLNClEOKopCSkeWQm63zqCQlJbFixQqee+45+vbta56tz8Q0qrNp0ybatGmDu7s7nTp14uTJk4B62lur1bJv3z6L182ZM4fw8HDOnTtHt27dAPDz80Oj0TBy5Egg72nOiIgI3n33XYYPH46npyfh4eGsXbuW69ev89BDD+Hp6UmzZs3y7GvHjh106dIFNzc3qlevzvjx40lOTi7yvc+fP59hw4YxbNgw5s+fn+f5f/75hwceeABvb2+8vLzo0qULZ8+eNT+/YMECGjdujF6vJzQ0lHHjxhW5z9yWL19Ov379LNatX7+eu+66C19fX6pUqcIDDzxgsU9r9OvXj3379tn8ukpDUSAj2TE3Of4q9PH3448/0rRpU9zc3KhSpQo9evSweC/ffPMNDRs2xNXVlQYNGvD5559bvP7ixYsMGDAAX19f/P39eeihh4iMjDQ/P3LkSPr378+HH35IaGgoVapUYezYsWRmZprb6HQ67r//fpYvX27T+6nInBzdASFEXqmZBhpN3uCQfR9/uxfuLtZ/NKxcuZIGDRpQv359hg0bxosvvsikSZPQaCyn/X399deZNWsWgYGBPPvss4wePZq//vqLiIgIevTowcKFC2nTpo25/cKFCxk5ciTh4eH89NNPPProo5w8eRJvb2/c3NwK7M9HH33E9OnTefPNN/noo4944okn6NSpE6NHj2bmzJlMnDiR4cOH888//6DRaDh79iy9e/fm3XffZcGCBVy/fp1x48Yxbtw4Fi5cWOB+zp49y65du1i1ahWKovDSSy9x4cIF8yjN5cuXufvuu7nnnnvYvHkz3t7e/PXXX2RlZQEwb948JkyYwHvvvUefPn2Ij4/nr7/+svrnHhsby/Hjxy1+ZgDJyclMmDCBZs2akZSUxOTJk3n44Yc5dOgQWq114xc1atQgODiY7du3U7t2bav7VGlkpsD0MMfs+7Ur4OJhdXM5/srP8RcdHc3gwYP54IMPePjhh0lMTGT79u3mAYLvv/+eyZMnM3fuXFq2bMnBgwd56qmn8PDwYMSIEWRmZtKrVy86duzI9u3bcXJy4t1336V3794cOXIEFxcXALZs2UJoaChbtmzhzJkzDBw4kBYtWvDUU0+Z+9KuXTvee+89q99PRSfBrBCiREyjI6DmW8bHx/Pnn39yzz33WLSbNm0aXbt2BeDVV1+lb9++pKWl4erqypNPPsmzzz7L7Nmz0ev1HDhwgKNHj7JmzRp0Oh3+/v4ABAUF4evrW2h/7r//fp555hkAJk+ezLx582jbti2PP/44ABMnTqRjx45cu3aNkJAQZsyYwdChQ80jTHXr1uWTTz6ha9euzJs3D1dX13z3s2DBAvr06YOfnx8AvXr1YuHChbz11lsAfPbZZ/j4+LB8+XKcnZ0BqFevnvn17777Lv/973954YUXzOvatm1b6HvLLSoqCkVRCAuzDLoeffTRPP0MDAzk+PHjNGnSxOrth4WFceHCBavbC8eQ46/8HH/R0dFkZWXxyCOPmIPqpk2bmp+fMmUKs2bN4pFHHgGgZs2aHD9+nC+//JIRI0awYsUKjEYj33zzjfnLyMKFC/H19WXr1q3cd999gDpCPnfuXHQ6HQ0aNKBv375s2rTJIpgNCwvj4sWLGI1Gq7/EVmQSzApRDrk56zj+di+H7dtaJ0+eZM+ePaxevRoAJycnBg4cyPz58/P8M23WrJn5cWhoKAAxMTHUqFGD/v37M3bsWFavXs2gQYNYtGgR3bp1IyIiwub+595PcHAwYPkPxbQuJiaGkJAQDh8+zJEjRywueFIUBaPRyPnz52nYsGGefRgMBhYvXszHH39sXjds2DBefvllJk+ejFar5dChQ3Tp0sX8jzS3mJgYrly5Qvfu3W1+fyapqakAef7Znz59msmTJ7N7925u3LhhvggkKirKpmDWzc2NlJSUYvevQnN2V0dIHbVvK8nxV76Ov+bNm9O9e3eaNm1Kr169uO+++3jsscfw8/MjOTmZs2fPMmbMGIugMysrCx8fHwAOHz7MmTNn8uQhp6WlWaRHNG7cGJ0u53M6NDSUo0ePWrzGzc0No9FIenp6oSPplYUEs0KUQxqNxqZT/Y4yf/58srKyLEYnFEVBr9czd+5c84c0YPFPxTTqYAq0XFxcGD58OAsXLuSRRx5h6dKlFv+obJHffgrbd1JSEs888wzjx4/Ps60aNWrku48NGzZw+fJlBg4caLHeYDCwadMmevbsWeg/EHv8cwkICADg1q1bBAYGmtf369eP8PBwvv76a8LCwjAajTRp0oSMjAybth8bG2ux3TuKRmPTqX5HkeOvfB1/Op2OjRs3snPnTv73v//x6aef8vrrr7N7927c3dUvKV9//TXt27e32JYpME1KSqJ169b5VhLJfSzeHqBrNJo8lQtiY2Px8PC4IwJZkGBWCFFMWVlZfPvtt8yaNct8+sukf//+LFu2jGeffdbq7T355JM0adKEzz//3HyqzsSUK2YwGOzT+VxatWrF8ePHqVOnjtWvmT9/PoMGDcpTkmfatGnMnz+fnj170qxZMxYvXkxmZmaefz5eXl5ERESwadMm88U1tqpduzbe3t4cP37cfPr05s2bnDx5kq+//pouXboA6sU1tjKNBLVs2bJYfROlT46/8nf8gRpYdu7cmc6dOzN58mTCw8NZvXo1EyZMICwsjHPnzjF06NB8t9mqVStWrFhBUFAQ3t7exeqXybFjx+6o47fyJ1IIIUrFr7/+yq1btxgzZgxNmjSxuD366KP5Xl1cmIYNG9KhQwcmTpzI4MGDLUYUwsPD0Wg0/Prrr1y/fp2kpCS7vY+JEyeyc+dOxo0bx6FDhzh9+jRr1qwp8Mrm69ev88svvzBixIg873v48OH8/PPPxMbGMm7cOBISEhg0aBD79u3j9OnTfPfdd+aryN966y1mzZrFJ598wunTpzlw4ACffvqpeT/Dhw9n0qRJBfZbq9XSo0cPi2DVz8+PKlWq8NVXX3HmzBk2b97MhAkTbP6Z/P333+j1ejp27Gjza0XZkOOv/B1/u3fvZvr06ezbt4+oqChWrVrF9evXzakSU6dOZcaMGXzyySecOnWKo0ePsnDhQmbPng3A0KFDCQgI4KGHHmL79u2cP3+erVu3Mn78eC5dumTTz3X79u15vuRUZhLMCiGKZf78+fTo0cPiVKbJo48+yr59+zhy5IhN2xwzZgwZGRmMHj3aYn3VqlWZOnUqr776KsHBwTaX0ClMs2bN+PPPPzl16hRdunShZcuWTJ48Oc+FVSbffvstHh4e+ebbde/eHTc3N5YsWUKVKlXYvHkzSUlJdO3aldatW/P111+bR4lGjBjBnDlz+Pzzz2ncuDEPPPAAp0+fNm8rKiqK6OjoQvv+5JNPsnz5cvMpRq1Wy/Lly9m/fz9NmjThpZdeYubMmTb/TJYtW8bQoUPNp0ZF+SPHX/k7/ry9vdm2bRv3338/9erV44033mDWrFn06dPH3P6bb75h4cKFNG3alK5du7Jo0SJq1qwJgLu7O9u2baNGjRo88sgjNGzYkDFjxpCWlmbTSO3ly5fZuXMno0aNsvo1FZ1GsbWoZAWXkJCAj48P8fHxJR7GF0LY1zvvvMMPP/xg8z/hO5WiKLRv356XXnqJwYMH22WbN27coH79+uzbt8/8T1bcGeT4s01pHH/2MHHiRG7dusVXX33l6K6UiC3xmozMCiEcLikpiWPHjjF37lyef/55R3enwtBoNHz11Vfm2pn2EBkZyeeffy6B7B1Ejr/iKY3jzx6CgoIcOs2wI8jIrBDC4UaOHMmyZcvo378/S5cutSg7I4QoXXL8ifLIlnhNglkhhBBCCFGuSJqBEEIIIYS4I0gwK4QQQgghKiwJZoUQQgghRIUlwawQQgghhKiwJJgVQgghhBAVlgSzQgghhBCiwpJgVgghSigyMhKNRsOhQ4cc3RWHycjIoE6dOuzcudPq17z11lu0aNHCvDxy5Ej69+9vXh40aBCzZs2yYy9FZSTHX/GOv9J2/PhxqlWrRnJycqnvS4JZIUSJ7dq1C51OR9++fR3dFZ555hl0Oh0//PBDqWz/9oALoHr16kRHR9OkSZNS2ac1li1bhk6nY+zYsQ7Z/xdffEHNmjXp1KmT3bb5xhtvMG3aNOLj4+22zcpIjj85/krj+CupRo0a0aFDB2bPnl3q+5JgVghRYvPnz+f5559n27ZtXLlyxWH9SElJYfny5bzyyissWLCgzPar0+kICQnBycmpzPZ5u/nz5/PKK6+wbNky0tLSynTfiqIwd+5cxowZY9ftNmnShNq1a7NkyRK7breykeNPjr/SOP7sYdSoUcybN6/0p/xV7jDx8fEKoMTHxzu6K0IUKTk9s8BbakaW3dsWR2JiouLp6amcOHFCGThwoDJt2jTzc4MHD1YGDBhg0T4jI0OpUqWKsnjxYkVRFCUhIUEZMmSI4u7uroSEhCizZ89Wunbtqrzwwgs292XRokVKhw4dlLi4OMXd3V2JioqyeH7EiBHKQw89pMycOVMJCQlR/P39lf/85z9KRkaGoiiKMnXqVKVx48Z5ttu8eXPljTfeUKZMmaIAFrctW7Yo58+fVwDl4MGDiqIoypYtWxRAWb9+vdKiRQvF1dVV6datm3Lt2jXlt99+Uxo0aKB4eXkpgwcPVpKTk837MRgMyvTp05WIiAjF1dVVadasmfLDDz8U+b7PnTunuLm5KXFxcUr79u2V77//Pk+b+fPnK40aNVJcXFyUkJAQZezYsebnbt26pTz99NNKUFCQotfrlcaNGyu//PKLVT9zRVGUvXv3KlqtVklISLBY/8orryh169ZV3NzclJo1aypvvPGG+WetKIoyZcoUpXnz5uZl0+8nt6lTpyp33XWX1X2xq/Skgm8ZqTa0TbGubTHI8SfHX37HX3p6ujJ27FglJCRE0ev1So0aNZTp06db7HPMmDFKQECA4uXlpXTr1k05dOiQxXZ//vlnpWXLloper1dq1qypvPXWW0pmZs7/CUD5+uuvlf79+ytubm5KnTp1lDVr1lhsIz09XdHr9coff/xh9fsxsSVec9zXGCFEkRpN3lDgc93qB7JwVDvzcut3/iA105Bv2/Y1/VnxTEfz8l3vbyE2OSNPu8j3bD9NuXLlSho0aED9+vUZNmwYL774IpMmTUKj0TB06FAef/xxkpKS8PT0BGDDhg2kpKTw8MMPAzBhwgT++usv1q5dS3BwMJMnT+bAgQMWuZTWmj9/PsOGDcPHx4c+ffqwaNEi3nzzTYs2W7ZsITQ0lC1btnDmzBkGDhxIixYteOqppxg9ejRTp05l7969tG3bFoCDBw9y5MgRVq1aRVBQEP/++y8JCQksXLgQAH9//wJHw9566y3mzp2Lu7s7AwYMYMCAAej1epYuXUpSUhIPP/wwn376KRMnTgRgxowZLFmyhC+++IK6deuybds2hg0bRmBgIF27di3wfS9cuJC+ffvi4+PDsGHDmD9/PkOGDDE/P2/ePCZMmMB7771Hnz59iI+P56+//gLAaDTSp08fEhMTWbJkCbVr1+b48ePodDqrf+7bt2+nXr16eHl5Waz38vJi0aJFhIWFcfToUZ566im8vLx45ZVXrN52u3btmDZtGunp6ej1eqtfZxfTwwp+ru59MDTXqfSZdSAzJf+24XfBqHU5y3OaQsrNvO3esj2dQo4/Of7yO/4++eQT1q5dy8qVK6lRowYXL17k4sWL5ucff/xx3Nzc+P333/Hx8eHLL7+ke/funDp1Cn9/f7Zv387w4cP55JNP6NKlC2fPnuXpp58GYMqUKebtTJ06lQ8++ICZM2fy6aefMnToUC5cuIC/vz8ALi4utGjRgu3bt9O9e3er35PNbA6VKzgZmRUVSfjEXwu8jVyw26Jtgzd+L7DtgC92WrRt+fb/8m1XHJ06dVLmzJmjKIqiZGZmKgEBAcqWLVsslr/99ltz+8GDBysDBw5UFEUdFXJ2drYY/TCN6tg6MnTq1CnF2dlZuX79uqIoirJ69WqlZs2aitFoNLcZMWKEEh4ermRl5YxUP/744+b+KIqi9OnTR3nuuefMy88//7xyzz33WGzj9tHDgkaGco9GzJgxQwGUs2fPmtc988wzSq9evRRFUZS0tDTF3d1d2bnT8nc1ZswYZfDgwQW+b4PBoFSvXl35+eefFUVRlOvXrysuLi7KuXPnzG3CwsKU119/Pd/Xb9iwQdFqtcrJkycL3EdRXnjhBeXee+8tst3MmTOV1q1bm5etGZk9fPiwAiiRkZHF7l+xTfEu+LbkMcu274YU3HbB/ZZt36+Zf7tikONPjr/8jr/nn39euffeey1+/ibbt29XvL29lbS0NIv1tWvXVr788ktFURSle/fuFiO5iqIo3333nRIaGmpeBpQ33njDvJyUlKQAyu+//27xuocfflgZOXKkze9LRmaFqCSOv92rwOe0Go3F8v43e1jddsfEbiXrWLaTJ0+yZ88eVq9eDYCTkxMDBw5k/vz53HPPPTg5OTFgwAC+//57nnjiCZKTk1mzZg3Lly8H4Ny5c2RmZtKuXc4Is4+PD/Xr17e5LwsWLKBXr14EBAQAcP/99zNmzBg2b95sMSLQuHFji1GP0NBQjh49al42jRDNnj0brVbL0qVL+eijj2zuD0CzZs3Mj4ODg3F3d6dWrVoW6/bs2QPAmTNnSElJoWfPnhbbyMjIoGXLlgXuY+PGjSQnJ3P//fcDEBAQQM+ePVmwYAHvvPMOMTExXLlypcBRkUOHDlGtWjXq1atXrPcIkJqaiqura571K1as4JNPPuHs2bMkJSWRlZWFt7e3Tdt2c3MD1HzMMvdaIfmnmttGzv7vTCFtb7s85cWj+bezkRx/hbuTj7+RI0fSs2dP6tevT+/evXnggQe47777ADh8+DBJSUlUqVIlz3bOnj1rbvPXX38xbdo08/MGg4G0tDRSUlJwd3cHLH/GHh4eeHt7ExMTY7FdNze3Uj9+JZgVohxzd7H+EC2ttoWZP38+WVlZhIXlnI5VFAW9Xs/cuXPx8fFh6NChdO3alZiYGDZu3Iibmxu9e/e2y/5NDAYDixcv5urVqxYXgRgMBhYsWGDxj8TZ2dnitRqNBqPRaF7u168fer2e1atX4+LiQmZmJo899lix+pV7XxqNptB9JyUlAbBu3TqqVq1q0a6w0+vz588nNjbWHPSBeuryyJEjTJ061WJ9fop63hoBAQEWAQmoV9gPHTqUqVOn0qtXL3x8fFi+fLnNpbZiY2MBCAwMLHE/bebi4fi2/9/encdEde1xAP8OKNDqCMrqwqKyKJRFZETUh1hHGStUKFakJCJVE1OwKrGtSwXUplhbKE1F1AQ1NmzVClrAFde6RBapYiNaq44Li0tFmVRUhveHz/s6gSogcLn1+0kmgTNnLt87J4f5zZ1z77wA59+Lvc7zz9PTE1euXMHu3btx4MABTJs2DUqlEtu3b0ddXR369u2Lw4cPN9mWiYkJgGfPx4oVK/Dee+816fP3wvll4wk8m8ODBw9u4961DItZImqTp0+fYuvWrUhMTBTe8T8XFBSEzMxMzJ07F6NGjYK1tTWys7Oxe/duvP/++8I/wEGDBqF79+4oKiqCjY0NAKC2thYXL16Er69vi7MUFBTg4cOHOHPmjM5Rn/LyckRGRuL+/fvCP+mX6datGyIiIrB582YYGBhg+vTpOi84BgYGaGhofm3yq3B2doahoSHUavUL1+f93d27d4UjbS4uLkJ7Q0MDxowZg3379kGlUsHOzg6FhYUYN67pEXk3NzfcuHEDFy9ebPPRoWHDhiE1NRWNjY2Q/e9TgBMnTsDW1hbLli0T+l27dq3V2y4vL8eAAQOEI370DOdf+/q3zT8A6NWrF0JDQxEaGoqpU6dCpVLh3r178PT0FN542NnZNbtNT09PVFRUwN7evk2Z/q68vLzNb0haisUsEbVJXl4e/vzzT8yaNQvGxsY694WEhCAtLQ1z584FAHzwwQdYv349Ll68iEOHDgn95HI5IiIi8Mknn6BPnz6wsLBAXFwc9PT0dP4pL1myBDdv3sTWrVubzZKWlobJkyfD3d1dp93Z2RkLFy5Eenp6q67/OHv2bAwdOhQAhBM1nrOzs8PevXtRUVEBU1PTJvveVnK5HIsWLcLChQuh1WoxZswY4USRXr16ISIiosljfvjhB5iammLatGk6zxfw7GPetLQ0qFQqxMfHY+7cubCwsBBONjl+/DjmzZuHsWPHwtfXFyEhIUhKSoK9vT0uXLgAmUwGlUqFmzdvYvz48di6davOx9F/N27cONTV1eH8+fPCtT4dHBygVquRlZUFhUKB/Px84ePw1jh27FiTYo04/zj//q+5+ZeUlIS+ffti2LBh0NPTw7Zt22BlZQUTExMolUr4+PggKCgIa9asgaOjI27duoX8/HwEBwfDy8sLsbGxCAgIgI2NDaZOnQo9PT38+uuvKC8vxxdffNHi5/Xq1au4efMmlMp/XgbXHnidWSJqk7S0NCiVymZfTEJCQlBcXIyzZ88CAMLDw/Hbb7+hf//+GD16tE7fpKQk+Pj4ICAgAEqlEqNHj8bQoUN1PsqqrKyEWq1uNkd1dTXy8/MREhLS5D49PT0EBwcjLS2tVfvm4OCAUaNGYciQIfD29ta5b86cOXBycoKXlxfMzc2bvNi+ilWrVmH58uVISEjA0KFDoVKpkJ+fj4EDBzbbf9OmTQgODm7yQgo8G4Ndu3bhzp07iIiIQHJyMtatWwcXFxcEBATg0qVLQt+ffvoJCoUCYWFhcHZ2xqeffioc/Xry5AkqKipeuObN1NQUwcHBSE9PF9reffddLFy4ENHR0fDw8MCJEyeanNn+Mo8ePUJubi7mzJnTqse9Djj/OP+ea27+yeVyrFmzBl5eXlAoFLh69SoKCgqENyoFBQXw9fVFZGQkHB0dMX36dFy7dg2WlpYAAH9/f+Tl5WHfvn1QKBQYOXIkvv32W9ja2rbqOc3MzMTEiRNb/bjWkv3vjLTXxoMHD2BsbIza2tpWn4hARB1Po9Ggf//+SExMFO0i4I2NjXBwcMBHH32EmJgYUTJIzdmzZzFhwgRcvnxZuAzUq0pNTUVOTg727dvXLtujl+P8k6aOmH+v6vHjx3BwcEBGRkaTN1Et0Zp6jcsMiEhUZ86cwYULFzBixAjU1tZi5cqVAIApU6aIkuf27dvIyspCVVUVIiMjRckgRW5ubvjqq69w5coVuLq6tss2u3fvju+//75dtkXN4/z7d+iI+feq1Go1li5d2qZCtrVYzBKR6L755htUVFTAwMAAw4cPx7Fjx0Q74cfCwgJmZmbYuHEjevfuLUoGqZo5c2a7bm/27Nntuj1qHuffv0N7z79XZW9v3y4nkLUElxkQERERUZfSmnqNJ4ARERERkWSxmCUiIiIiyWIxS0RERESSxWKWiIiIiCSLxSwRERERSRaLWSIiIiKSLBazRERERCRZXaKYTUlJgZ2dHYyMjODt7Y3Tp0+/sP+2bdswZMgQGBkZwdXVFQUFBZ2UlIiIiIi6EtGL2ezsbMTExCAuLg6lpaVwd3eHv78/ampqmu1/4sQJhIWFYdasWThz5gyCgoIQFBSE8vLyTk5ORERERGIT/RvAvL29oVAosHbtWgCAVquFtbU15s2bh8WLFzfpHxoaCo1Gg7y8PKFt5MiR8PDwwPr161/69/gNYERERERdW2vqtW6dlKlZjx8/RklJCZYsWSK06enpQalU4uTJk80+5uTJk4iJidFp8/f3R25ubrP96+vrUV9fL/xeW1sL4NmTRERERERdz/M6rSXHXEUtZu/cuYOGhgZYWlrqtFtaWuLChQvNPqaqqqrZ/lVVVc32T0hIwIoVK5q0W1tbtzE1EREREXWGhw8fwtjY+IV9RC1mO8OSJUt0juRqtVrcu3cPpqamkMlkHf73Hzx4AGtra1y/fp3LGiSKYyh9HEPp4xhKG8dP+jp7DBsbG/Hw4UP069fvpX1FLWbNzMygr6+P6upqnfbq6mpYWVk1+xgrK6tW9Tc0NIShoaFOm4mJSdtDt1GvXr04gSWOYyh9HEPp4xhKG8dP+jpzDF92RPY5Ua9mYGBggOHDh6OwsFBo02q1KCwshI+PT7OP8fHx0ekPAPv37//H/kRERET07yX6MoOYmBhERETAy8sLI0aMQHJyMjQaDSIjIwEAM2bMQP/+/ZGQkAAAmD9/PsaOHYvExERMnjwZWVlZKC4uxsaNG8XcDSIiIiISgejFbGhoKG7fvo3Y2FhUVVXBw8MDe/bsEU7yUqvV0NP7/wHkUaNGISMjA59//jmWLl0KBwcH5Obm4q233hJrF17I0NAQcXFxTZY6kHRwDKWPYyh9HENp4/hJX1ceQ9GvM0tERERE1FaifwMYEREREVFbsZglIiIiIsliMUtEREREksViloiIiIgki8VsB0tJSYGdnR2MjIzg7e2N06dPix2JWujo0aMIDAxEv379IJPJkJubK3YkaqWEhAQoFArI5XJYWFggKCgIFRUVYseiFkpNTYWbm5twkXYfHx/s3r1b7Fj0ClavXg2ZTIYFCxaIHYVaKD4+HjKZTOc2ZMgQsWPpYDHbgbKzsxETE4O4uDiUlpbC3d0d/v7+qKmpETsatYBGo4G7uztSUlLEjkJtdOTIEURFReHUqVPYv38/njx5gokTJ0Kj0YgdjVpgwIABWL16NUpKSlBcXIy3334bU6ZMwfnz58WORm1QVFSEDRs2wM3NTewo1EouLi6orKwUbr/88ovYkXTw0lwdyNvbGwqFAmvXrgXw7NvNrK2tMW/ePCxevFjkdNQaMpkMOTk5CAoKEjsKvYLbt2/DwsICR44cga+vr9hxqA369OmDr7/+GrNmzRI7CrVCXV0dPD09sW7dOnzxxRfw8PBAcnKy2LGoBeLj45Gbm4uysjKxo/wjHpntII8fP0ZJSQmUSqXQpqenB6VSiZMnT4qYjOj1VVtbC+BZQUTS0tDQgKysLGg0Gn59uQRFRUVh8uTJOq+JJB2XLl1Cv379MGjQIISHh0OtVosdSYfo3wD2b3Xnzh00NDQI32T2nKWlJS5cuCBSKqLXl1arxYIFCzB69Ogu+42B1NS5c+fg4+ODR48eoWfPnsjJyYGzs7PYsagVsrKyUFpaiqKiIrGjUBt4e3tjy5YtcHJyQmVlJVasWIH//Oc/KC8vh1wuFzseABazRPSaiIqKQnl5eZdb60Uv5uTkhLKyMtTW1mL79u2IiIjAkSNHWNBKxPXr1zF//nzs378fRkZGYsehNpg0aZLws5ubG7y9vWFra4sff/yxyyz3YTHbQczMzKCvr4/q6mqd9urqalhZWYmUiuj1FB0djby8PBw9ehQDBgwQOw61goGBAezt7QEAw4cPR1FREb777jts2LBB5GTUEiUlJaipqYGnp6fQ1tDQgKNHj2Lt2rWor6+Hvr6+iAmptUxMTODo6Ijff/9d7CgCrpntIAYGBhg+fDgKCwuFNq1Wi8LCQq73IuokjY2NiI6ORk5ODg4ePIiBAweKHYlekVarRX19vdgxqIXGjx+Pc+fOoaysTLh5eXkhPDwcZWVlLGQlqK6uDpcvX0bfvn3FjiLgkdkOFBMTg4iICHh5eWHEiBFITk6GRqNBZGSk2NGoBerq6nTeeV65cgVlZWXo06cPbGxsRExGLRUVFYWMjAzs3LkTcrkcVVVVAABjY2O88cYbIqejl1myZAkmTZoEGxsbPHz4EBkZGTh8+DD27t0rdjRqIblc3mSNeo8ePWBqasq16xKxaNEiBAYGwtbWFrdu3UJcXBz09fURFhYmdjQBi9kOFBoaitu3byM2NhZVVVXw8PDAnj17mpwURl1TcXExxo0bJ/weExMDAIiIiMCWLVtESkWtkZqaCgDw8/PTad+8eTNmzpzZ+YGoVWpqajBjxgxUVlbC2NgYbm5u2Lt3LyZMmCB2NKLXxo0bNxAWFoa7d+/C3NwcY8aMwalTp2Bubi52NAGvM0tEREREksU1s0REREQkWSxmiYiIiEiyWMwSERERkWSxmCUiIiIiyWIxS0RERESSxWKWiIiIiCSLxSwRERERSRaLWSIiIiKSLBazRESvAT8/PyxYsEDsGERE7Y7FLBFRFyGTyV54i4+PFzsiEVGX003sAERE9ExlZaXwc3Z2NmJjY1FRUSG09ezZU4xYRERdGo/MEhF1EVZWVsLN2NgYMplM+F2j0SA8PByWlpbo2bMnFAoFDhw4oPP4devWwcHBAUZGRrC0tMTUqVP/8W/l5+fD2NgY6enpHb1bREQdisUsEZEE1NXV4Z133kFhYSHOnDkDlUqFwMBAqNVqAEBxcTE+/vhjrFy5EhUVFdizZw98fX2b3VZGRgbCwsKQnp6O8PDwztwNIqJ2x2UGREQS4O7uDnd3d+H3VatWIScnB7t27UJ0dDTUajV69OiBgIAAyOVy2NraYtiwYU22k5KSgmXLluHnn3/G2LFjO3MXiIg6BItZIiIJqKurQ3x8PPLz81FZWYmnT5/ir7/+Eo7MTpgwAba2thg0aBBUKhVUKhWCg4Px5ptvCtvYvn07ampqcPz4cSgUCrF2hYioXXGZARGRBCxatAg5OTn48ssvcezYMZSVlcHV1RWPHz8GAMjlcpSWliIzMxN9+/ZFbGws3N3dcf/+fWEbw4YNg7m5OTZt2oTGxkaR9oSIqH2xmCUikoDjx49j5syZCA4OhqurK6ysrHD16lWdPt26dYNSqcSaNWtw9uxZXL16FQcPHhTuHzx4MA4dOoSdO3di3rx5nbwHREQdg8sMiIgkwMHBATt27EBgYCBkMhmWL18OrVYr3J+Xl4c//vgDvr6+6N27NwoKCqDVauHk5KSzHUdHRxw6dAh+fn7o1q0bkpOTO3lPiIjaF4tZIiIJSEpKwocffohRo0bBzMwMn332GR48eCDcb2Jigh07diA+Ph6PHj2Cg4MDMjMz4eLi0mRbTk5OOHjwIPz8/KCvr4/ExMTO3BUionYla+TCKSIiIiKSKK6ZJSIiIiLJYjFLRERERJLFYpaIiIiIJIvFLBERERFJFotZIiIiIpIsFrNEREREJFksZomIiIhIsljMEhEREZFksZglIiIiIsliMUtEREREksViloiIiIgk678xCbXVqCbxfgAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def hline(ax, y, label, color_id):\n", " ax.hlines(y, 0, 5, linestyles=\"--\", label=label, color=cmap(color_id))\n", "\n", "\n", "fig, ax = plt.subplots(figsize=(8, 4))\n", "# Plot the accuracy on all tasks over the course of tasks\n", "ax.scatter(r.task_index, r.accuracy_all, label=\"Acc. (all)\")\n", "ax.plot(r.anytime_task_index, r.anytime_accuracy_all, label=\"Anytime Acc. (all)\")\n", "hline(ax, r.anytime_accuracy_all_avg, \"Avg. Anytime Acc. (all)\", 0)\n", "\n", "# Plot the accuracy on previously seen tasks over the course of tasks\n", "ax.scatter(r.task_index, r.accuracy_seen, label=\"Anytime Acc. (seen)\")\n", "ax.plot(r.anytime_task_index, r.anytime_accuracy_seen, label=\"Anytime Acc. (seen)\")\n", "hline(ax, r.anytime_accuracy_seen_avg, \"Avg. Anytime Acc. (seen)\", 1)\n", "\n", "ax.legend(ncol=2, frameon=False)\n", "ax.set_xlabel(\"Task\")\n", "ax.set_xticks(range(6))\n", "ax.set_ylabel(\"Accuracy\")\n", "ax.set_title(\"SplitMNIST Accuracy Over Tasks\")\n", "ax.set_ylim(0, 1.05)\n", "pass" ] } ], "metadata": { "kernelspec": { "display_name": "capymoa", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.8" } }, "nbformat": 4, "nbformat_minor": 2 }