Newer
Older
{
"cell_type": "markdown",
"id": "291f0068",
"metadata": {},
"source": [
"# Long Short-Term Memory\n",
"This notebook will show you an example of a long short-term memory neural network with tensorflow. As a dataset we are going to use the Corona_NLP dataset which you can find in the data folder. This dataset contains tweets regarding the corona virus and an estimation of the authors sentiment toward the topic.\n",
"\n",
"The following sentiment classes exist:\n",
">['Extremely Negative', 'Negative', 'Neutral', 'Positive', 'Extremely Positive']\n",
"## Imports\n",
"For this task we require quite a few imports. Some of the more special ones we are going to use here are `re` which will be used in the preprocessing step as well as `keras.models.load_model` and `keras.callbacks.CSVLogger` which will be used to allow saving and loading the model and its training history from disk. For this we are also going to define a `BASE_PATH` which is the folder were we will save the model and logfile to"
]
},
{
"cell_type": "code",
"id": "1f1d51cb-44b6-4d7e-97aa-b1e699e6338e",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "1f1d51cb-44b6-4d7e-97aa-b1e699e6338e",
"outputId": "5181c0bb-a008-4094-964a-9f03e8b86d70"
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import seaborn as sns\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import re\n",
"import tensorflow as tf\n",
"from tensorflow.keras.layers import Dense, Embedding, LSTM, Bidirectional, GlobalAveragePooling1D, Dropout\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.optimizers import Adam\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from keras.models import load_model\n",
"from keras.callbacks import CSVLogger\n",
"BASE_PATH = \"./saved_models/\""
]
},
{
"cell_type": "markdown",
"id": "6Sp0upFjpIL5",
"metadata": {
"id": "6Sp0upFjpIL5"
},
"source": [
"## Loading the dataset and analysis\n",
"We start by doing the usual. Loading our dataset and displaying a few example entries."
]
},
{
"cell_type": "code",
"id": "61c32169-b9a2-4868-aeac-9a44f108e623",
"metadata": {
"id": "61c32169-b9a2-4868-aeac-9a44f108e623"
},
"outputs": [],
"source": [
"train = pd.read_csv(\"../data/Corona_NLP/Corona_NLP_train.csv\", encoding=\"latin1\",engine=\"python\")\n",
"test = pd.read_csv(\"../data/Corona_NLP/Corona_NLP_test.csv\", encoding=\"latin1\",engine=\"python\")"
]
},
{
"cell_type": "code",
"id": "43bdce54-3e08-4af8-ad69-1b0b5b68f184",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
"id": "43bdce54-3e08-4af8-ad69-1b0b5b68f184",
"outputId": "3523d045-620d-4a9f-dd65-35a3b677a176"
},
"outputs": [
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>UserName</th>\n",
" <th>ScreenName</th>\n",
" <th>Location</th>\n",
" <th>TweetAt</th>\n",
" <th>OriginalTweet</th>\n",
" <th>Sentiment</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>3799</td>\n",
" <td>48751</td>\n",
" <td>London</td>\n",
" <td>16-03-2020</td>\n",
" <td>@MeNyrbie @Phil_Gahan @Chrisitv https://t.co/i...</td>\n",
" <td>Neutral</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>3800</td>\n",
" <td>48752</td>\n",
" <td>UK</td>\n",
" <td>16-03-2020</td>\n",
" <td>advice Talk to your neighbours family to excha...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3801</td>\n",
" <td>48753</td>\n",
" <td>Vagabonds</td>\n",
" <td>16-03-2020</td>\n",
" <td>Coronavirus Australia: Woolworths to give elde...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>3802</td>\n",
" <td>48754</td>\n",
" <td>NaN</td>\n",
" <td>16-03-2020</td>\n",
" <td>My food stock is not the only one which is emp...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>3803</td>\n",
" <td>48755</td>\n",
" <td>NaN</td>\n",
" <td>16-03-2020</td>\n",
" <td>Me, ready to go at supermarket during the #COV...</td>\n",
" <td>Extremely Negative</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
"text/plain": [
" UserName ScreenName Location TweetAt \\\n",
"0 3799 48751 London 16-03-2020 \n",
"1 3800 48752 UK 16-03-2020 \n",
"2 3801 48753 Vagabonds 16-03-2020 \n",
"3 3802 48754 NaN 16-03-2020 \n",
"4 3803 48755 NaN 16-03-2020 \n",
"\n",
" OriginalTweet Sentiment \n",
"0 @MeNyrbie @Phil_Gahan @Chrisitv https://t.co/i... Neutral \n",
"1 advice Talk to your neighbours family to excha... Positive \n",
"2 Coronavirus Australia: Woolworths to give elde... Positive \n",
"3 My food stock is not the only one which is emp... Positive \n",
"4 Me, ready to go at supermarket during the #COV... Extremely Negative "
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.head(5)"
]
},
{
"cell_type": "markdown",
"id": "n2SNLzfwqCuG",
"metadata": {
"id": "n2SNLzfwqCuG"
},
"source": [
"As our first preprocessing step we will drop some columns that we are not interested in like the username, screenname, location of the author and the time of the tweet."
]
},
{
"cell_type": "code",
"id": "d0501221-8bee-45b7-8ecf-1d5b483680cb",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
"id": "d0501221-8bee-45b7-8ecf-1d5b483680cb",
"outputId": "2d7a9e7b-21d5-4cf8-9a0b-bb7807ed5225"
},
"outputs": [
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>OriginalTweet</th>\n",
" <th>Sentiment</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>@MeNyrbie @Phil_Gahan @Chrisitv https://t.co/i...</td>\n",
" <td>Neutral</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>advice Talk to your neighbours family to excha...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Coronavirus Australia: Woolworths to give elde...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>My food stock is not the only one which is emp...</td>\n",
" <td>Positive</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Me, ready to go at supermarket during the #COV...</td>\n",
" <td>Extremely Negative</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
"text/plain": [
" OriginalTweet Sentiment\n",
"0 @MeNyrbie @Phil_Gahan @Chrisitv https://t.co/i... Neutral\n",
"1 advice Talk to your neighbours family to excha... Positive\n",
"2 Coronavirus Australia: Woolworths to give elde... Positive\n",
"3 My food stock is not the only one which is emp... Positive\n",
"4 Me, ready to go at supermarket during the #COV... Extremely Negative"
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# drop unnecessary columns\n",
"unnecessary_columns=[\"UserName\", \"ScreenName\", \"TweetAt\", \"Location\"]\n",
"train.drop(unnecessary_columns, axis=1, inplace=True)\n",
"test.drop(unnecessary_columns, axis=1, inplace=True)\n",
"train.head(5)"
]
},
{
"cell_type": "markdown",
"id": "KPw5uQWC0PyB",
"metadata": {
"id": "KPw5uQWC0PyB"
},
"source": [
"Lets check if there are any unknown values in the dataframe that we have to drop or replace"
]
},
{
"cell_type": "code",
"id": "Aqri2Hm60u6Q",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "Aqri2Hm60u6Q",
"outputId": "23fa3815-8968-42ef-891a-beb37ebdb0ee"
},
"outputs": [
"name": "stdout",
"output_type": "stream",
"text": [
"train data null values:\n",
"OriginalTweet 0\n",
"Sentiment 0\n",
"dtype: int64\n",
"\n",
"test data null values:\n",
"OriginalTweet 0\n",
"Sentiment 0\n",
"dtype: int64\n"
]
}
],
"source": [
"print(f\"train data null values:\\n{train.isnull().sum()}\\n\")\n",
"print(f\"test data null values:\\n{test.isnull().sum()}\")"
]
},
{
"cell_type": "markdown",
"id": "DTm-2S531NWq",
"metadata": {
"id": "DTm-2S531NWq"
},
"source": [
"There aren't any missing values so we don't have to do any preprocessing in regards to replacing or dropping missing values.\n",
"Next let's take a look at the classes we are trying to predict. "
]
},
{
"cell_type": "code",
"id": "edef1814-645e-4a5d-b303-ebf2a9dde1c4",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "edef1814-645e-4a5d-b303-ebf2a9dde1c4",
"outputId": "36a0a7ee-40f0-4390-a402-f167c6883eac"
},
"outputs": [
"data": {
"text/plain": [
"array(['Neutral', 'Positive', 'Extremely Negative', 'Negative',\n",
" 'Extremely Positive'], dtype=object)"
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# show variations of sentiment\n",
"train['Sentiment'].unique()"
]
},
{
"cell_type": "markdown",
"id": "3acb201f",
"metadata": {},
"source": [
"We are going to sort a temporary dataframe based on the sentiment before plotting so that our plot shows the most negative sentiment on the left and the positive on the right"
]
},
{
"cell_type": "code",
"id": "32592de7-07f4-411b-a953-d9f9fb6a0c66",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 297
"id": "32592de7-07f4-411b-a953-d9f9fb6a0c66",
"outputId": "eaf06407-90b2-47a1-da1a-cc6903fe28f1"
},
"outputs": [
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAE2CAYAAACgOK8BAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAApDUlEQVR4nO3debhcVZXG4d+XhBmBCGnEJJAgEQQEhTAP0qBMDgFklJaAYFCDgigyCiLQQjfKJIIRItAqg4gSRMXIaAsEAiKjNFGEJAQIBEEZTbL6j7ULissNuTm5VXUr9b3PU0+q9hlqn5uqWmevvc8+igjMzMyq6NfqCpiZWftyEDEzs8ocRMzMrDIHETMzq8xBxMzMKhvQ6go020orrRTDhg1rdTXMzNrGXXfd9UxEDOpuWccFkWHDhjF58uRWV8PMrG1Iemxey5zOMjOzyhxEzMysMgcRMzOrzEHEzMwqcxAxM7PKHETMzKwyBxEzM6vMQcTMzCpzEDEzs8o67op1M3uzwUNX5YlpU1tdjaZ695ChTJ/6eKursUhwEDHrcE9Mm8pe37+11dVoqssP3rzVVVhkOJ1lZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVeYgYmZmlTmImJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVdawICJpvKSnJd1fV/ZOSRMlPVL+HVjKJelsSVMk3Stpg7ptRpf1H5E0uq58Q0n3lW3OlqRGHYuZmXWvkS2Ri4Adu5QdBVwfESOA68trgJ2AEeUxBjgPMugAJwCbABsDJ9QCT1nns3XbdX0vMzNrsIYFkYi4BZjVpXgUcHF5fjGwS135JZFuB1aQtAqwAzAxImZFxHPARGDHsmy5iLg9IgK4pG5fZmbWJM3uE1k5ImaU508CK5fng4GpdetNK2VvVz6tm3IzM2uilnWslxZENOO9JI2RNFnS5JkzZzbjLc3MOkKzg8hTJRVF+ffpUj4dGFq33pBS9nblQ7op71ZEjIuIkRExctCgQQt9EJ1k8NBVkdRRj8FDV231n92sbQxo8vtNAEYDp5Z/r64rP0TSZWQn+vMRMUPSdcB/1nWmbw8cHRGzJL0gaVNgErAfcE4zD6RTPDFtKnt9/9ZWV6OpLj9481ZXwaxtNCyISLoU2AZYSdI0cpTVqcAVkg4EHgP2LKv/CtgZmAK8BBwAUILFScCdZb1vRkSts/4L5AiwpYBfl4eZmTVRw4JIROwzj0XbdbNuAGPnsZ/xwPhuyicD6y5MHc3MbOH4inUzM6vMQcTMzCpzEDEzs8ocRMzMrDIHETMzq8xBxMzMKnMQMTOzyhxEzMysMgcRMzOrzEHEzMwqcxAxM7PKHETMzKwyBxEzM6vMQcTMzCpzEDEzs8ocRMzMrDIHETMzq8xBxMzMKnMQMTOzyhxEzMysMgcRMzOrzEHEzMwqcxAxM7PKHETMzKwyBxEzM6vMQcTMzCpzEDEzs8ocRMzMrLKWBBFJX5b0gKT7JV0qaUlJwyVNkjRF0uWSFi/rLlFeTynLh9Xt5+hS/rCkHVpxLGZmnazpQUTSYOBLwMiIWBfoD+wNnAacERFrAM8BB5ZNDgSeK+VnlPWQtHbZbh1gR+B7kvo381jMzDpdq9JZA4ClJA0AlgZmANsCV5blFwO7lOejymvK8u0kqZRfFhGvRsSjwBRg4+ZU38zMoAVBJCKmA6cDj5PB43ngLuDvETG7rDYNGFyeDwamlm1nl/VXrC/vZps3kTRG0mRJk2fOnNm7B2Rm1sFakc4aSLYihgPvBpYh01ENExHjImJkRIwcNGhQI9/KzKyjtCKd9WHg0YiYGRH/Aq4CtgBWKOktgCHA9PJ8OjAUoCxfHni2vrybbczMrAlaEUQeBzaVtHTp29gOeBC4Edi9rDMauLo8n1BeU5bfEBFRyvcuo7eGAyOAO5p0DGZmRnZwN1VETJJ0JXA3MBv4IzAOuBa4TNLJpezCssmFwP9ImgLMIkdkEREPSLqCDECzgbERMaepB2Nm1uGaHkQAIuIE4IQuxX+lm9FVEfEKsMc89nMKcEqvV9A6W78BZCPZzOanJUHErE+bO5u9vn9rq2vRNJcfvHmrq2BtzNOemJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVeYgYmZmlTmImJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllPQoikrboSZmZmXWWnrZEzulhmZlZ39dvAJI66jF46KoN+VO+7T3WJW0GbA4MknR43aLlgP4NqZGZWaPNnc1e37+11bVoqssP3rwh+33bIAIsDixb1ntHXfkLwO4NqZGZmbWNtw0iEXEzcLOkiyLisSbVyczM2sT8WiI1S0gaBwyr3yYitm1EpczMrD30NIj8FDgfuACY07jqmJlZO+lpEJkdEec1tCZmZtZ2ejrE9xpJX5C0iqR31h4NrZmZmfV5PQ0io4EjgFuBu8pjctU3lbSCpCsl/VnSQ5I2K4FpoqRHyr8Dy7qSdLakKZLulbRB3X5Gl/UfkTS6an3MzKyaHgWRiBjezWP1hXjfs4DfRMRawPrAQ8BRwPURMQK4vrwG2AkYUR5jgPMASkvoBGATYGPghFrgMTOz5uhRn4ik/borj4hLFvQNJS0PbA3sX/bxGvCapFHANmW1i4GbgCOBUcAlERHA7aUVs0pZd2JEzCr7nQjsCFy6oHUyM7NqetqxvlHd8yWB7YC7gQUOIsBwYCbwQ0nrk6mxQ4GVI2JGWedJYOXyfDAwtW77aaVsXuVvIWkM2Yph1VUbc+m/mVkn6lEQiYgv1r+WtAJw2UK85wbAFyNikqSzeCN1VXu/kBQV9/8WETEOGAcwcuTIXtuvmVmnqzoV/Itki6KKacC0iJhUXl9JBpWnSpqK8u/TZfl0YGjd9kNK2bzKzcysSXo6Ffw1kiaUx7XAw8DPq7xhRDwJTJW0ZinaDngQmECOAqP8e3V5PgHYr4zS2hR4vqS9rgO2lzSwdKhvX8rMzKxJetoncnrd89nAYxExbSHe94vAjyUtDvwVOIAMaFdIOhB4DNizrPsrYGdgCvBSWZeImCXpJODOst43a53sZmbWHD3tE7lZ0sq80cH+yMK8aUTcA4zsZtF23awbwNh57Gc8MH5h6mJmZtX1NJ21J3AHsAfZQpgkyVPBm5l1uJ6ms44FNoqIpwEkDQJ+R3aKm5lZh+rp6Kx+tQBSPLsA25qZ2SKqpy2R30i6jjeuBt+L7PA2M7MONr97rK9BXkl+hKTdgC3LotuAHze6cn3N4KGr8sS0qfNf0cysQ8yvJXImcDRARFwFXAUg6f1l2ccbWLc+54lpU9nr+7e2uhpNdfnBm7e6CmbWh82vX2PliLiva2EpG9aQGpmZWduYXxBZ4W2WLdWL9TAzszY0vyAyWdJnuxZKOoicfdfMzDrY/PpEDgN+Lmlf3ggaI4HFgV0bWC8zM2sDbxtEIuIpYHNJ/w6sW4qvjYgbGl4zMzPr83o6d9aNwI0NrouZmbUZX3VuZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVeYgYmZmlTmImJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllDiJmZlZZy4KIpP6S/ijpl+X1cEmTJE2RdLmkxUv5EuX1lLJ8WN0+ji7lD0vaoUWHYmbWsVrZEjkUeKju9WnAGRGxBvAccGApPxB4rpSfUdZD0trA3sA6wI7A9yT1b1LdzcyMFgURSUOAjwIXlNcCtgWuLKtcDOxSno8qrynLtyvrjwIui4hXI+JRYAqwcVMOwMzMgNa1RM4EvgbMLa9XBP4eEbPL62nA4PJ8MDAVoCx/vqz/enk327yJpDGSJkuaPHPmzF48DDOzztb0ICLpY8DTEXFXs94zIsZFxMiIGDlo0KBmva2Z2SJvQAvecwvgE5J2BpYElgPOAlaQNKC0NoYA08v604GhwDRJA4DlgWfrymvqtzEzsyZoekskIo6OiCERMYzsGL8hIvYFbgR2L6uNBq4uzyeU15TlN0RElPK9y+it4cAI4I4mHYaZmdGalsi8HAlcJulk4I/AhaX8QuB/JE0BZpGBh4h4QNIVwIPAbGBsRMxpfrXNzDpXS4NIRNwE3FSe/5VuRldFxCvAHvPY/hTglMbV0MzM3o6vWDczs8ocRMzMrDIHETMzq8xBxMzMKnMQMTOzyhxEzMysMgcRMzOrzEHEzMwqcxAxM7PKHETMzKwyBxEzM6vMQcTMzCpzEDEzs8ocRMzMrDIHETMzq8xBxMzMKnMQMTOzyhxEzMysMgcRMzOrzEHEzMwqcxAxM7PKHETMzKwyBxEzM6vMQcTMzCpzEDEzs8ocRMzMrDIHETMzq6zpQUTSUEk3SnpQ0gOSDi3l75Q0UdIj5d+BpVySzpY0RdK9kjao29fosv4jkkY3+1jMzDpdK1ois4GvRMTawKbAWElrA0cB10fECOD68hpgJ2BEeYwBzoMMOsAJwCbAxsAJtcBjZmbN0fQgEhEzIuLu8vwfwEPAYGAUcHFZ7WJgl/J8FHBJpNuBFSStAuwATIyIWRHxHDAR2LF5R2JmZi3tE5E0DPggMAlYOSJmlEVPAiuX54OBqXWbTStl8yo3M7MmaVkQkbQs8DPgsIh4oX5ZRAQQvfheYyRNljR55syZvbVbM7OO15IgImkxMoD8OCKuKsVPlTQV5d+nS/l0YGjd5kNK2bzK3yIixkXEyIgYOWjQoN47EDOzDteK0VkCLgQeiojv1C2aANRGWI0Grq4r36+M0toUeL6kva4Dtpc0sHSob1/KzMysSQa04D23AD4N3CfpnlJ2DHAqcIWkA4HHgD3Lsl8BOwNTgJeAAwAiYpakk4A7y3rfjIhZTTkCMzMDWhBEIuJ/Ac1j8XbdrB/A2HnsazwwvvdqZ2ZmC8JXrJuZWWUOImZmVpmDiJmZVeYgYmZmlTmImJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVeYgYmZmlTmImJlZZQ4iZmZWmYOImZlV5iBiZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWUOImZmVpmDiJmZVeYgYmZmlbV9EJG0o6SHJU2RdFSr62Nm1knaOohI6g+cC+wErA3sI2nt1tbKzKxztHUQATYGpkTEXyPiNeAyYFSL62Rm1jEUEa2uQ2WSdgd2jIiDyutPA5tExCFd1hsDjCkv1wQebmpFF95KwDOtrkST+Zg7g4+5PawWEYO6WzCg2TVphYgYB4xrdT2qkjQ5Ika2uh7N5GPuDD7m9tfu6azpwNC610NKmZmZNUG7B5E7gRGShktaHNgbmNDiOpmZdYy2TmdFxGxJhwDXAf2B8RHxQIur1Qhtm4pbCD7mzuBjbnNt3bFuZmat1e7pLDMzayEHETMzq8xBxKwNSXpXq+tg7U1Sv/KvFmY/DiJWSe0D2Ncs7BeijVwvaX/oqGPus5+7dlL7vETE3FK0UH9T/4fYApOk2gdQ0iqtrk+NpH5RRoqUedUWKUq14/oGMFbSktEBo2NqwSMi5nZS0GyEuu/IjpJ+AXxd0mZV9+cgYgssIkLSWpJ+Dpwj6UxJq/WBes2VtKykM4EjJX2gxVXqFSV49Is0ByAifgrMAr5WW6eVdWy0upOWg4BrJB0kaVhra9W+JO0MfBk4AVgKOF7SelX25SBiC0zSAPLD90PgQGAH4NCWVgqQtC7wC+AvwAzgMkltP71ECR5zJQ2RNEbS5mXREcD+klbrkNbImcBWwFeBDwNntbRCbaDryYWkPSQNBFYBfgq8B/gYMCEi7q3yHg4iNk9dU0KS9pS0L7A88A9gMeA3wG+Brze/hq/Xa01JK5NnVBOA35GzOd8PPNqqei2Mrrl/SccB1wLvAM6XtFP50l8LnNSCKjZcaYGtVM6aAV4iW14HkNMdXdiyyvVxkpaEN1JXpex9ZMBYsRT9J/AJYKuIOE/SclXS0w4i9haSRkjqX0ud1FkC2JlMo2wHfAXYLSIOjYgXJW3f4Hq9pZ9D0juBs8kzq1WBz5A/LpdExO4R8aykdzSyXo1Ql775ZCl6Cfh3MmCvDHxJ0mDgKGBjSdu1pKK9qLRwX1d+ANcHTi1FOwF/IufH2zoiJkhap7m17PsknQJsWp4PlfSRsugxYASwJHAvcDXZApklaSvgSrJlskAcROxNytnKTsAKkpYqKYSap4Hp5ct9Jvn5eUbS6pKuAEZJWq5B9RoFnFGe9y/vuUREzAKeAj4C3Ao8BxwTEVeVdccDezWiTr1J0tJdXg+TdDbwNUnvBs4B9i3/jgLmAJ+LiBfJtOKeTa5yrymtjSOA9crrj9YtngHcVP4+3waei4izI2KOpE8Bx0ga3vxa9y2S+klaorw8C/jf0qrYAThZ0sci4iXyJORTwGQy9fsNST8lP1c/joj/XeD37oBUqvWApBUj4tm619sAfwQuAKYCl5AtkF9FxLplnYuAucAHgJ9FxCkNruMA3mgN7QpMioizJO0KbBQRx0gaC2xd6rU2GViOiojnG1m3hVFaSp8DvhsRL0vaBNgG+FJEDK5b70Lg2oi4StK3yOC4a0T8qRX1XljlJODV8v+6JDn/3Vzys/Yk8N9k2vTnwKjSqrwWeJZsdQZ5wnBbSw6gj5C0DPBx4Moyn+BS5AnXrPKdGEV+vsaT6dBlgPMj4l8lDTwMuKNqv1pbT8BoC690vK0HbFSCwjrkTXP2AVYnz1oOJD+UBwJ/krRHGR30GWBpoH/tR7qMIpr7ljeqVq9PAn+IiBmSBgHvBy6NiJUlPQxcKulRYLnygAx6lwMfAv4vIu6r7a+vdT7X/lYR8Y/yQ3CPpKfJ9NxvgdGSdoiI68ryR4E9SmvxfWSap1JnaKtJGgpsBFxVfviGkZ+nqeRn7yvAKWQn+v3AaOA7ZOAcCKwTEb9pQdX7DEnrA++NiJ+WdOYukjYAjgXOA06XtHFEXC0pyEEJe5LfqXMAIuIpsiXPPFLY8+V0VoeqjdooP6wvkh1sD5CjrB4EJgKbAe+JiPOBq4DvA2sBs8u2cyPinxHxfGlOqzcCSF29FgNukXQu8K2IuAF4StKXSqfy18hROu8H9pQ0OCJejYhnIuJnEXFf6Zzt19cCCLyp32MJYBA5YOHiEqAfBs4F9ivrvkieof+ZDCBfjohxffG43o6k0eXH75/AJpJ+Kem35MnI7eSxDS+t2tuBn5AtlNllFy9GxNRODyDF8kAt/fRn4KPAf0fET0vr9E7gAEnLRMQE4DQyPbhq6UR/08itKgEEHEQ6UjnjqP/x+Tt52+AngOMjYgZwV3m9D0A5czmXTDfM6rrPElAW6getm47zGeSP6+Aot0AGDiHzuItFxLVkE31ZMs21Qjf1it4KbL2h6xdX0jeBw8mAeBjZ70Fd/nqOpINL2eMRcWJE/EdEtOWoM7I19WfyxGUHYF3gnIi4A5gETAP+AyAizgJ+RKYvVyplbRU0e1v9dyQibgHWlnQCOSrxDGB4GWwCcDrZUf5hSYtHxDNk+nPLiHiht/6WDiIdqHbGIemrkj5BBobNyNbH/mWdKcBtwLtKThVyJMdGEXFzb9eptGJq9fpg6aC/i8zlbly3zi3AzWRHIBFxD/BFslnf5+8lExEhaTFJq5eiq8kUzVDgZ8CLkmrX3DwN/ArYTmXIZjvq5ofvGGAPcrjp98lAQkQ8Rn7mBpZ+LoCLgA0i4vhm1rmvqvuOfLKkePuTo/UGk+m+zSnflzLo5CoyDb1UKZtRtu+1rgwHkQ5QUjqqe721pLvJs5ShwE/KB+5u8kxmw7LqNPLCvYHwpvRLr0wpUl+n8uO6gaTfASeSF0KtHhGXAfdKOqvuzOlgYIzeuEp+dkQ81Vv16k166/UeywDHkf0dy0XEXcAtwBER8S/ybPJQSV8rz/8E7BsRrzS56r2mjKRaRtLHStF9ZKvrKTJorCHpw2XZvcDjwBbl7PlfJW9vgKQtJf2JHFiyDHA98DdyROW/yPTfvpIOlXQ52VI/uOvAkoiYTS/x6KxFXH1nmcoILEk7kKmqJ4HzybOXsyPiW5KOJjs8lyQ7qH8ZdaO2erFeb+qAV97e+ELgsoi4VtL9ZADbj8z9PgxsQabXvgOsGBH393a9elP9MSpHu71ABoWtgd2A6yLil5KWJfuhPl+O/XPASOCMdmhdzY+kL5Jnw38gU3evkWmqRyLiG5KOBIYDU8g01++Bh0tQ7VhdO7rLCcg44AcRcVNd+RrkFCZ/iIifSPoMsD05aOGKsk6vDHjptp4OIou+8iO1PdlpPjoi/lZGcVxAjgS6H7iUvHL1SUlHAc9HxHl1+2jI6CZJnwWeLiNI3kEOFz6bTK2tB/w6chjvWHIY44PkWXulTsBGk7QCsGmt41fSSuT1DcPJY/oIOXz3WHKI6oWRo8+uAd4NbNKbZ4nNVFqW6nJyMIQ8/uMi4pG68vXIgQKjyCG7Xyf/7w+pX6/TlUEXnwBujYjpksaRg1tuJUckrkl+L3Yhv+OnN/vkykFkEdP1x145XfjXyfvQb0Ze53GspD3JYZInlPTVz8kP6t5vt79erOcu5Nn4mmRKbbXIcevfJs9Qz1dO9XEwsHMZabV06XDusyQdQA6lPD8i7pD0IfLv/L3S13ES+SOwKjl8Osgf0SHA7RExvkVVXyhdWl2rkQHxfnImgTPJVuWzwIbkZ/Ei8mr7TSLiI408U24XknYjB7ncFDlX2sfIVveDZKrqZjI99SWyRfsEmRr9PZk12Irst3yt7K8pf1NfJ7KImMdZ4NLk9RK7RsS95Yd7/9IKmQOsL+lw8uz4aPLs5vX9lZFNCx1AummWr0p++I8jr5ydSE7oeBzwMjBS0prkRVA3kFehExEv1fpR+tIoHUnbAo+XwQg3k/XeTtJd5Bd7XUl7A38F1oiIp4EnJM0k0ztDgCMj4rmWHEBFpQ/qU+Qw0yeBlyUdS44wmwR8kByCfQZ5/VHth297cs6vs8hcPmQw7UiStiZP9OaQQ53/LOmrZIv14Ii4sXzGPg/8LSJOLdutT148eH/pN7qyfr9NC8oR4cci9CA7yw8EViiv7yfP5CEnXjuVPEuGHFf+I3L+q9r2/XqxLv3qni9F9m30I89GxwMDyrK1gJlki2RdchqPh4Axrf579uAY30WObptEpt/6k3NcnU/OX7Qu2Qfw/rptPgXsVJ73b/UxVDzuA8lO8d+SI4C+TP4A/gIYUdY5C7iiy3YfJYPO6q0+hr7wIC/o/T3Zz1H7PN1BXvv0BzL9DDmM/RDgZDKNdTY5EGbfVh+D01ltqJyN9yd/iK6pKzsJ2JH8Yq9EzrC7DLB9RHy6rHcy2bF7euQFSG/ab/TCB0LSGpFn5bXXnyP7Y34DvErmyH9T6vpMREQZcXJnRBxU+nBmRxmR1LUl09dIOo3s8P8e8F7yB/XzwDsiU4dXAM8DN5EBZn3g8Ij4fWtqvHAk/RvZ8lgrIv5P0h5ki+JOYEtgbET8vaz7KNkymU4Oy14W+FpETG5F3fuaki3YlzyROjMippbPy3XkIIPdImLPsu6xwMsR8R1J748yG0NZ1rIZGTzEtw2VD8sqwAV6434Z/0ae1Y4kWx9bkU3dm4DBkv5LedHahmX5WnrjHsv96vZbmdK+wBGS3lPKtiVHGm0G/Jq8qG4O+YNzCnmx1BbkWdXHJK0VeRX8KyVdQl8OIMWJ5I/j+eSotmOA1YBXJe1ETl1+I5la/GvktTZtGUAAItNxF5JpUMjreVYkU4/vJYNkzbVkC+Vp4NSI2LaTA4i6DPmO7OO7nbwif+eScn432bK9HlhC0gUlHfoJ8gJc4o3pfGrfkZa1BtwSaSOStiS/jI9GdkKfCKwZEXuXUTA3kGd8/wBOjuzY7U9eiPQp3jhLPhyYGhEX9FK9BgJzIuIF5eSBu5G52/MkfYE8WVmLzJGfGTnXz3Jk62Rz8krzzwJjgKsj4vreqFczleN8b0QcppzH6LvkD8PfgC+UM8wB0aYjr7oqw00fI/tzxpCt4p3KwII9yIsoVyTnahoVeSGh1am1Hkpg+SQ5Yu814LNRJtUsrb49yQsIf9AnTz5anU/zo2cP8sx2KtkxeUop60eesexOngFfQg4ZrW3zPjJ3vWR5vTI52uMOYONeqtd7yD6BC+reZx9yipTVyauSZ1Nyu2X5BmTwg5zWpB+Z770LWKXVf+uKf4d+5MWZ65XX65PprduAd7W6fg065oPIVuV3ydF1tfIdgW+RLbOhra5nqx+Ufi9A5XPyX+RUPvXL3k3eJOrYuu0W62ZftQE0LT+u2sMtkTZRxovvT6YQ1iGnyPgVmbI6kjxb2Qj4JjmVxHAyuPxXRFxY+kwOApaLiG/3ct3+QLZyziY7yC8hUzr/jIhTJd1DXiT1CzKl8xXyQrofS1qeHL2zOHBY5Pw+bUnSZsBZEbFxq+vSDOUzNY2cluQpSUtFxMu1ZdHhPy6SNomISd2U/wz4XWRL/fW/k/KmbrsBv41yP5wu2/XJYdAe4ttHdf3ARN53YRI5wukRMnV1ETl9xCByBNYlkuaQIztWAj4UEU+W7UPS+GhM/8JeZIvox2Sg6A+8QuZzNyBTaZ8FfkC2WsbWvlyRMwB/JdpseGt3IuI2SbMlrRcV71fdTspnandy8r9NagGktqx1NWu90s93uqSDyJb43mSf0Kvk6LQXu9nsbrLPcmB3++yLAQTcJ9IWuuRO9yDH2R9Fnv1vSg77ewHYIupGRZVt+wMLPcNuD+p4EdkReA1574fDyQ78R8irlf+pnKp9eu2YYNH7senrI8kaQdKt5F0WF/nAOT9139UVyYlBFyPTV+PJdPRZ5KzEa0fE2K4ni8r51F5oRd2r8uisPqQ20qKMcupXRlQNLh/K/uXDdjuZQhgbEX8oqanjyDTSKl321y8i5jTph3oseWXyq5FTeB8DvJM8AxsCUBdA+kfRhHo1VacFkGKrTg8gXUdJRc43dzXZN/a+iPgkOcjiLHKE1XslLdu1dVELILWTrHbglkgfsBC504kR8TPlvTVaPlmdpDHA5hGxf3m9NLB8lOmnzRZ1kr5CdhPcFhG3SDqEzBDsU5YfRmYS3kWOuGrWSV7DuCXSYiV3eqakdSStKemE0okOb587fYxyE6ZaAFHrp0K/gLxF53vK65cjJxdsdb3MelV9S6FkDlaWdCWwBtk/+GtJm5IX1f5LOdEo5AWXnydHYw1v9wACDiItU/ch/DN5deqnyKuA1wNOkzScHDO+SW2T2rZlBNO5EXFh/T5bnUopTfP3RsRfyuta074TUzy2iFKX2y2X50uTF/Z+i7xj41/IOd8eJQeb7C5p5ZJefowcXTmoyVVvCAeRJlvUc6eRVzObLXLqvrtzlXenPEXSrso7DK5GDij5OTm0fb2IeJjsWL+BPFlco+xnL7JzfXorjqO3uU+kRToxd2rWjroZQbUCOfPDemSLY9WI2LMMwf92vHEjqOOApSLnT6u/Odz7gBlR5hdrdw4iDdalQ1zkHFfnkqOpLifnFtoOeAY4Hvh9RPygnPUMIfOrW4Vv1GPWUpI2Jic5/Tvwl4g4RtKS5Hf4TPI2BmPJ4fZDyclGj6yNXFtUh3/7YsMG6uaCwSgjlm4iL9D6PG/NnR4saULk/QEeKyO0BpHXW5hZE9S+u+XEb3HyWqwVgfPIPo81Jb0nIv4i6QxymP2HyPvj7AS8EBHXln3V7s2zyAUQcJ9IQzh3atae9MaM1nPLv0HeVXBLYGBE/IJsdTwPbChp8Yj4JZlZOCUi/h4Rl9YFkP6Lehra6axe5NypWXuqTzuX1weQaeZbyLTz+8hpfTaPnCdsLHlr5x9GxB+Vs2i/Em0891tVDiIN4NypWXso6arXbystaQBwBHnbgm+Qk5lOItNZJwJLR8RnlROHXghcCvy8bvs+OUliI7lPZCE5d2rWnupaH6G8v83J5MWAS5AzUm9DfpdvjIi/S/oOcI2kbSPiBuXEoW+6T0qnBRBwn0hlzp2atadyIe/r12pJ+iRwMZkZeI4cVv8TYBh5f5hrJb2rBIxLyPvkUAsgfe1arWZzOmsBOXdq1r7Kyd//AKcBS5EtjSHkxb0rRN5y4XjyPj3fjYjHlLd83gM4uIyatDpuifRQmR+nX93ZywBJR5Mpqf8EPg18nRxd9UuyaQzwI3KenNXL9tMi4hl1udeymTVOrbVQMgcPk/PPnQQ8S7ZCHgQOKKtfDswCLpN0LTkU/7z6ANLprY96/iHrgbq+irmSlpN0NnnnwG5zp+QtaDcpudPnga9ExM+6XDPScblTs2arn2ZI0hKStiXnpHscuDIiJkXeKOp4YIykpcuQ+9PIO4GeERFbRsR19ft16vkNDiJvw7lTs/YkaQS8ZfLPzwF7R8Sp5H1uTikX/1L6Ju8Hvl1ez42IByLid2V/nol6HhxE5qGkm06WtJ6kTSTtTLY2dgROKn0aT5BN33MjYnbJnZ6vnK3z9Ii4oH6fPnsxayxJ20q6AdhN0uLl+/ttSWuRqebnASLiDvIGb6eW7dYkWx/9JS3Wdb8eMTlvHuLbRZfUVS13egPZ33E9cDCZOz2fDCC7kbnTWeT9z0/qmjt18DBrLEmDyTnplgR+EBGXlvInyczBWWTm4Jq6zb4A3CZpdTI1vU9EjGlqxRcBHp1VdLlSfAlgC3L23DHAqRExriz7KNkht2VEvFRaLO8DVqk1fc2sucoUQZdGRL/yenFgF+CmiHha0lDgCnIeul+SFwjeXFooa0fEVXX76rgLBhdGxwcRSSOiywy5kg4F1omIMeXq82uB1SLipbL8EuDFiPh8N/vz1eZmLSDpbmAcOdfcKeTN3o4DXisd6+cAD5Etk12BH3cJHv7uVtCxfSLOnZotcg4GvkdOKbRPRBwREa+WALIU2QqZGBE/Ag6qDyDg725VHdcn4typ2aIpIu6UdCk5ldADtXJJA4H+wCvAYiVd9VxZ5j7LhdRx6SznTs0WXZLeSd6bZ62ImCHpBGBr4JiImNTa2i2aOi6IgHOnZosySScCXyInOZ0OHBcR08oyn/j1sk4NIhuR0zv/lryavL7puxTwQ+DrEfGIpIG1pq+ZtQdJvwDOjogbymsHjwbpuD4RcO7UbFEXEbvUnjuANFZHtkTAuVOzRZ2DR3N0ZEsEICJmSToTeFBSLXc62rlTs0WDv7/N0bEtkRrnTs3Mquv4IFLPAcTMbMF07BXr9dTlVrdmZtYzbomYmVllbomYmVllDiJmZlaZg4iZmVXmIGLWQ5KOlfSApHsl3SNpkwr7+EC51XLt9SckHdW7NX3Le24jafNGvod1ro692NBsQUjaDPgYsEFEvCppJWDxCrv6ADAS+BVAREwAJvRWPedhG+CfwK0Nfh/rQB6dZdYDknYDDoiIj3cp3xD4DrAs8Aywf5lG5yZyks9/B1YADiyvpwBLkTMkfKs8HxkRh0i6CHgZ+CDwb8BngP2AzYBJEbF/ec/tgRPJe9v8pdTrn5L+BlwMfBxYDNiDnAfudmAOMBP4YkT8vlf/ONbRnM4y65nfAkMl/Z+k70n6ULmz5TnA7hGxITCevLVAzYCI2Bg4DDghIl4Djgcuj4gPRMTl3bzPQDJofJlsoZwBrAO8v6TCViJvW/DhiNiAnO788Lrtnynl5wFfjYi/AecDZ5T3dACxXuV0llkPlDP9DYGtyNbF5cDJwLrAREmQM0DPqNusdg+au4BhPXyra8o9be4DnoqI+wAkPVD2MQRYG/hDec/Fgdvm8Z679fwIzapxEDHroXIjspuAm8qP/FjggYjYbB6bvFr+nUPPv2u1bebWPa+9HlD2NTEi9unF9zSrzOkssx6QtKakEXVFHyDvfjmodLojaTFJ68xnV/8A3rEQVbkd2ELSGuU9l5H03ga/p9k8OYiY9cyywMWSHpR0L5lSOh7YHThN0p+Ae4D5DaW9EVi7DBHea0ErEREzgf2BS0s9bgPWms9m1wC7lvfcakHf0+zteHSWmZlV5paImZlV5iBiZmaVOYiYmVllDiJmZlaZg4iZmVXmIGJmZpU5iJiZWWX/D8Y6E5+XottlAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"mapping = {'Extremely Negative':0,'Negative':1,'Neutral':2,'Positive':3,'Extremely Positive': 4} # Assign sentiment class to a value\n",
"tmp = train.sort_values(by='Sentiment', key=lambda col: col.map(mapping)) # sort by the value \n",
"sns.histplot(data=tmp, x=\"Sentiment\")\n",
"plt.xticks(rotation=30)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "UQr4hGqe1hLe",
"metadata": {
"id": "UQr4hGqe1hLe"
},
"source": [
"All in all the data is pretty balanced although the extreme opinions do appear less often than the more moderate ones. If we have trouble predicting those correctly it might be an option to merge the extreme and the moderate columns to form a single column each.\n",
"## Preprocessing\n",
"The 5 classes we have will need to be encoded as numbers so that our model will be able to handle them properly.\n",
"Aside from that we aren't doing a lot of preprocessing. The only things we do is removing URLs and twitter mentions as those are not very helpful when predicting the sentiment. This function will encode the Sentiment values in the dataframe and apply the preprocess_string function on every entry."
]
},
{
"cell_type": "code",
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
"id": "bf84a59f-0e6d-4e95-ab78-3065cb0e34a4",
"metadata": {
"id": "bf84a59f-0e6d-4e95-ab78-3065cb0e34a4"
},
"outputs": [],
"source": [
"def preprocess_dataframe(df: pd.DataFrame):\n",
" def preprocess_string(s):\n",
" '''\n",
" min_word_count describes the minimum amount of words that have to remain after cleaning for the string to not be returned as None\n",
" '''\n",
" # remove urls and twitter mentions\n",
" s = re.sub(r'http\\S+', '', s)\n",
" s = re.sub(\"(@[a-zA-Z_0-9]+)\",\"\", s)\n",
" return s\n",
" \n",
" # encode sentiment\n",
" mapping = {'Extremely Negative':0,'Negative':1,'Neutral':2,'Positive':3,'Extremely Positive': 4}\n",
" df['Sentiment'] = df['Sentiment'].map(mapping, na_action=None)\n",
" # preprocess the tweets\n",
" df['OriginalTweet'] = [preprocess_string(s) for s in df.OriginalTweet]\n",
" return df"
]
},
{
"cell_type": "code",
"id": "0c95f6f2-f9ef-417d-ab2c-b7206495ea82",
"metadata": {
"id": "0c95f6f2-f9ef-417d-ab2c-b7206495ea82"
},
"outputs": [],
"source": [
"train = preprocess_dataframe(train)\n",
"test = preprocess_dataframe(test)"
]
},
{
"cell_type": "code",
"id": "e4665e4f-080b-4ee3-94ad-2c0471943a19",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"id": "e4665e4f-080b-4ee3-94ad-2c0471943a19",
"outputId": "cda0a2a3-ac53-49ad-fb69-1aa49ffac7ed"
},
"outputs": [
{
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>OriginalTweet</th>\n",
" <th>Sentiment</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>and and</td>\n",
" <td>2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>advice Talk to your neighbours family to excha...</td>\n",
" <td>3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Coronavirus Australia: Woolworths to give elde...</td>\n",
" <td>3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>My food stock is not the only one which is emp...</td>\n",
" <td>3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Me, ready to go at supermarket during the #COV...</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
"text/plain": [
" OriginalTweet Sentiment\n",
"0 and and 2\n",
"1 advice Talk to your neighbours family to excha... 3\n",
"2 Coronavirus Australia: Woolworths to give elde... 3\n",
"3 My food stock is not the only one which is emp... 3\n",
"4 Me, ready to go at supermarket during the #COV... 0"
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.head()"
]
},
{
"cell_type": "markdown",
"id": "RQZNBWi83HOD",
"metadata": {
"id": "RQZNBWi83HOD"
},
"source": [
"The data as it is now is not ready to be handed over to the model yet. First we will need to some additional preprocessing. We will start by converting our features and targets into numpy arrays because Keras cannot handle dataframes. We will also create and fit a tokenizer on both datasets so that it can learn all words that occur. After tokenizing the tweets we will apply padding to make all tweets a uniform length."
]
},
{
"cell_type": "code",
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
"id": "d436471f-6349-47c1-82b9-58c842204b6d",
"metadata": {
"id": "d436471f-6349-47c1-82b9-58c842204b6d"
},
"outputs": [],
"source": [
"# LSTM model\n",
"y = np.array(train['Sentiment'])\n",
"x = np.array(train['OriginalTweet'])\n",
"\n",
"y_test = np.array(test['Sentiment'])\n",
"x_test = np.array(test['OriginalTweet'])\n",
"\n",
"# create and fit Tokenizer on the train and test set.\n",
"tokenizer = Tokenizer()\n",
"tokenizer.fit_on_texts(x)\n",
"tokenizer.fit_on_texts(x_test)\n",
"\n",
"sequences_train = tokenizer.texts_to_sequences(x)\n",
"sequences_test = tokenizer.texts_to_sequences(x_test)\n",
"\n",
"# find length of longest tweet in train and test set and set padded_size to the maximum.\n",
"padded_size = max(max([len(tweet) for tweet in sequences_train]), max([len(tweet) for tweet in sequences_test]))\n",
"\n",
"x = pad_sequences(sequences_train, maxlen=padded_size)\n",
"x_test = pad_sequences(sequences_test, maxlen=padded_size)\n",
"\n",
"y = np.reshape(y, (-1,1))\n",
"y_test = np.reshape(y_test, (-1,1))"
]
},
{
"cell_type": "markdown",
"id": "eupqf9UJDs37",
"metadata": {
"id": "eupqf9UJDs37"
},
"source": [
"After padding we can take a look at how our tweets look now. The first tweet in our array was reduced to a simple `and and` by the preprocessing. Each and is now represented by a 4 in the padded and tokenized string. The padding character used is a 0 and it was added in front of the tweet."
]
},
{
"cell_type": "code",
"id": "a4584bb4-09d1-48fe-a190-b5fc44d8f82f",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "a4584bb4-09d1-48fe-a190-b5fc44d8f82f",
"outputId": "9bcca2c1-a930-406d-fdfc-2522ea8d2c3b"
},
"outputs": [
"data": {
"text/plain": [
"array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4])"
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x[0]"
]
},
{
"cell_type": "markdown",
"id": "Bvr1sRMeEpWW",
"metadata": {
"id": "Bvr1sRMeEpWW"
},
"source": [
"## Building and training the model\n",
"Now that the preprocessing is done we can finally start building the model. This process usually requires a bit of experimentation to achieve a good accuracy so feel free to change some variables or add some layers and see if you can get a better result.\n",
"\n",
"As this model takes quite a bit of time to train it is automatically saved after completing the training. When running this notebook a second time you can change the `LOAD_MODEL` variable to `True` to load the pretrained model from memory instead of retraining it."
]
},
{
"cell_type": "code",
"id": "yBxNn2GTa26v",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "yBxNn2GTa26v",
"outputId": "20069693-08ba-43b0-9f46-6ce2db44a6f9"
},
"outputs": [
"name": "stdout",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" bidirectional (Bidirectiona (None, 65, 32) 2816 \n",
" l) \n",
" global_average_pooling1d (G (None, 16) 0 \n",
" lobalAveragePooling1D) \n",
" \n",
"=================================================================\n",
"Total params: 279,402\n",
"Trainable params: 279,402\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"None\n"
]
}
],
"source": [
"# Switch this to True or False to enable/disable loading of the model from disk\n",
"# Make sure to run the entire notebook at least once before setting this to True.\n",
"LOAD_MODEL = False\n",
"\n",
"model = None\n",
"if LOAD_MODEL:\n",
" model = load_model(BASE_PATH+'long_short_term_memory_model.h5')\n",
"else:\n",
" dropout_rate = 0.4\n",
"\n",
" num_classes = train['Sentiment'].unique()\n",
" num_words = len(tokenizer.word_index)+1\n",
"\n",
" model = Sequential([\n",
" Embedding(input_dim=num_words,output_dim=5, input_length=padded_size),\n",
" Bidirectional(LSTM(16, return_sequences=True)),\n",
" Dropout(dropout_rate),\n",
" LSTM(16, return_sequences=True),\n",
" GlobalAveragePooling1D(),\n",
" Dense(len(num_classes), activation='softmax'),\n",
" ])\n",
" model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
" optimizer=Adam(learning_rate=0.01, decay=0.001),\n",
" metrics=['accuracy'])\n",
" print(model.summary())"
]
},
{
"cell_type": "code",
"id": "2965e0d3-7649-4e04-b44f-950f4b3a2373",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
"id": "2965e0d3-7649-4e04-b44f-950f4b3a2373",
"outputId": "f0c59874-855d-4e08-af51-e684f6531a0b"
},
"outputs": [
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"1287/1287 [==============================] - 43s 31ms/step - loss: 1.1219 - accuracy: 0.5346 - val_loss: 0.8971 - val_accuracy: 0.6651\n",
"1287/1287 [==============================] - 39s 30ms/step - loss: 0.6193 - accuracy: 0.7790 - val_loss: 0.7436 - val_accuracy: 0.7351\n",
"1287/1287 [==============================] - 37s 29ms/step - loss: 0.4338 - accuracy: 0.8569 - val_loss: 0.7290 - val_accuracy: 0.7615\n",
"1287/1287 [==============================] - 37s 29ms/step - loss: 0.3273 - accuracy: 0.8970 - val_loss: 0.7934 - val_accuracy: 0.7551\n",
"1287/1287 [==============================] - 38s 30ms/step - loss: 0.2597 - accuracy: 0.9215 - val_loss: 0.8643 - val_accuracy: 0.7370\n"
],
"source": [
"history = None\n",
"if LOAD_MODEL:\n",
" history = pd.read_csv(BASE_PATH+'lstm_training.log', sep=',', engine='python')\n",
" csv_logger = CSVLogger(BASE_PATH+'lstm_training.log', separator=',', append=False)\n",
" history=model.fit(x,y, epochs = epochs, verbose = 'auto', validation_data=(x_test, y_test), callbacks=[csv_logger]).history\n",
" model.save(BASE_PATH+'long_short_term_memory_model.h5')"
]
},
{
"cell_type": "markdown",
"id": "nUjXgdOMFwD1",
"metadata": {
"id": "nUjXgdOMFwD1"
},
"source": [
"## Evaluation\n",
"After completing the training we can plot the accuracy and loss curves using pyplot."
]
},
{
"cell_type": "code",
"id": "26128add-7843-4533-8829-0fc6f03fe1a4",
"metadata": {
"base_uri": "https://localhost:8080/",
"height": 281
"id": "26128add-7843-4533-8829-0fc6f03fe1a4",
"outputId": "770d716c-44d8-40dd-9901-75179a54c0dc"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABPDUlEQVR4nO3deVxVZf7A8c+XRVBEZHEFFRR3EVBEEyptMUtzrdS0pKZMpzKtqWmaSqfylzM5LZbWWNliJtnmaFlOpuaCGyq4pYmAijsgCCKyPb8/zhVRWS7rvZf7vF+v+/Jy7jn3+YLnfu85z3nO9xGlFJqmaZp9cLB0AJqmaVrd0Ulf0zTNjuikr2maZkd00tc0TbMjOulrmqbZEZ30NU3T7IhO+lUkIj+JyMSaXlfTbJX+TNgGsadx+iKSXeLHRsAloND082NKqcV1H1X1iUgAcBj4j1JqiqXj0WxHfftMiMgA4AullJ+FQ7FadnWkr5RqfPkBHAXuLrGseOcWESfLRVklDwLngDEi4lKXDYuIY122p9WsevyZ0MpgV0m/LCIyQERSROSvInIK+EREPEXkBxE5KyLnTM/9SmyzTkQeMT2PEpGNIjLHtG6SiNxZxXUDRGS9iGSJyGoRmSciX5QTu2Ak/ReBfODua14fLiJxInJeRA6LyGDTci8R+URETpjiWFYyvmveQ4lIoOn5pyLyvoisFJELwEARGSIiu0xtHBORmddsHykiMSKSYXo9SkT6iMjpkl8aIjJKROLN+T/TapctfybK+Z26mtrNEJF9IjKsxGt3ich+UxvHReQvpuU+pt8zQ0TSRWSDiNh03rTp4GtYS8ALaAdMwvjbfGL6uS1wEXivnO37AgcBH+BfwMemhFzZdb8EtgHewEzggQrijgT8gGhgKVDcTyoi4cDnwLNAU+AmINn08iKM0/nuQHPgrQraKel+YBbgDmwELmB88TQFhgBTRGSEKYZ2wE/Au0AzIASIU0ptB9KAQSXe9wFTvJp1sNXPxHVExBlYAfwPY39/ElgsIp1Nq3yM0Z3lDvQA1piWPwOkYOy7LYAXANvuE1dK2eUDI/ndZno+AMgDXMtZPwQ4V+LndcAjpudRQEKJ1xph7BgtK7MuxgepAGhU4vUvMPooy4rrI2CZ6fkNGEf7zU0//wd4q5RtWgFFgGcpr0UBG69ZpoBA0/NPgc8r+Nu+fbld4G/A92Ws91dgsem5F5ADtLL0vmGvj/rwmTDFnVLK8huBU4BDiWVLgJmm50eBx4Am12z3CvDfy/t/fXjoI/0rziqlci//ICKNROQ/InJERM4D64GmUnYf9qnLT5RSOaanjSu5bmsgvcQygGNlBSwiDYF7gcWm99qMsfPeb1qlDcYF3mu1MbVzrqz3rsBVMYlIXxFZazrtzwQmYxyxlRcDGB/eu0XEDbgP2KCUOlnFmLSaZ3OfiXK0Bo4ppYpKLDsC+JqejwbuAo6IyG8icoNp+RtAAvA/EUkUkeer0LZV0Un/imtP2Z4BOgN9lVJNMLpGAMo6Pa0JJwEvEWlUYlmbctYfCTQB5ovIKVPfqy9XuniOAR1K2e6YqZ2mpbx2AeNICwARaVnKOtf+rb4ElgNtlFIewAdc+TuVFQNKqePAZmAUxin7otLW0yzGFj8TZTkBtLmmP74tcBxAKbVdKTUco+tnGUZXKUqpLKXUM0qp9sAw4GkRubUK7VsNnfTL5o7RZ5khIl7AjNpuUCl1BIgFZopIA9PRxt3lbDIRWAgEYZxqhwARQLCIBGH0Uz4kIreKiIOI+IpIF9PR9E8YXxaeIuIsIpc/wPFAdxEJERFXjD7UirhjHI3lmq4j3F/itcXAbSJyn4g4iYi3iISUeP1z4DnT7/CdGW1plmMLnwkARMS15APjmkAO8Jxpfx9gep9o0/uOFxEPpVQ+cB6j+xMRGSoigabrC5kYw1mLSmvTVuikX7a3gYZAKrAF+LmO2h2P0TefBrwGfIUxdvoqIuIL3Aq8rZQ6VeKxwxTrRKXUNuAhjIu0mcBvGBfhwDiyzgcOAGeAaQBKqT8w+jFXA4cwLtRW5M/AKyKSBbyM6SjJ9H5HMU6bnwHSgTgguMS235ti+v6aU3jN+ryNFX8mSvDF+HIq+WiDkeTvxIh/PvCgUuqAaZsHgGRTt9VkU5sAHTE+C9kYZ6XzlVJra+w3swC7ujnLFonIV8ABpVStH1VZiogcxhg5sdrSsWjWzx4+E7VJH+lbGTHGr3cwdccMBoZj9DHWSyIyGqPveE1F62r2yd4+E7VN32VnfVpi9G17Y4wPnqKU2mXZkGqHiKwDugEPXDOqQtNKspvPRF3Q3Tuapml2RHfvaJqm2RGr697x8fFR/v7+lg5Dq8d27NiRqpRqVtft6n1bq03m7tdWl/T9/f2JjY21dBhaPSYiRyzRrt63tdpk7n6tu3c0TdPsiE76mqZpdkQnfU3TNDtidX36mqbVnfz8fFJSUsjNza14Zc0quLq64ufnh7Ozc5W210lf0+xYSkoK7u7u+Pv7U/b8Jpq1UEqRlpZGSkoKAQEBVXoP3b2jaXYsNzcXb29vnfBthIjg7e1drTMznfQ1zc7phG9bqvv/pZO+VqeKihRrD57h3V8PWTqUOpd2Ic/SIWiaTvpa3UjLvsT76w5z85y1PPTJdhZtOcKFSwWWDqtOnTmfi651dbW0tDRCQkIICQmhZcuW+Pr6Fv+cl1f+l2RsbCxTp06tsI3+/fvXSKzr1q1j6NChNfJelqQv5Gq1RinFjiPn+GLLEVbuOUVeYRF9A7x47o4u3NG9JQ2c7OuYo6BIcfB0Fl1aNrF0KFbD29ubuLg4AGbOnEnjxo35y1/+Uvx6QUEBTk6lp6mwsDDCwsIqbCMmJqZGYq0v7OtTp9WJ7EsFLNpyhDvf2cA9H2zm19/PcH/ftvwy/Sa+euwG7g5ubXcJ/7JNCWmWDsHqRUVFMXnyZPr27ctzzz3Htm3buOGGGwgNDaV///4cPHgQuPrIe+bMmTz88MMMGDCA9u3bM3fu3OL3a9y4cfH6AwYM4J577qFLly6MHz+++Mxr5cqVdOnShd69ezN16tRKHdEvWbKEoKAgevTowV//+lcACgsLiYqKokePHgQFBfHWW28BMHfuXLp160bPnj0ZO3Zs9f9YVaCP9LUa8/vJ83yx5QjLdh3nQl4h3Vs34fVRQQwPaU2jBnpXa+DkwKaEVP4UWbWhdrXtHyv2sf/E+Rp9z26tmzDj7u6V3i4lJYWYmBgcHR05f/48GzZswMnJidWrV/PCCy/w7bffXrfNgQMHWLt2LVlZWXTu3JkpU6ZcN5Z9165d7Nu3j9atWxMREcGmTZsICwvjscceY/369QQEBDBu3Diz4zxx4gR//etf2bFjB56engwaNIhly5bRpk0bjh8/zt69ewHIyMgAYPbs2SQlJeHi4lK8rK7pT6JWLZcKCvlpzym+2HKE2CPncHFyYGjP1kzo15aQNk31yJASGrs4sTUxjfzCIpwd7fNMx1z33nsvjo6OAGRmZjJx4kQOHTqEiJCfn1/qNkOGDMHFxQUXFxeaN2/O6dOn8fPzu2qd8PDw4mUhISEkJyfTuHFj2rdvXzzufdy4cSxYsMCsOLdv386AAQNo1swobjl+/HjWr1/PSy+9RGJiIk8++SRDhgxh0KBBAPTs2ZPx48czYsQIRowYUem/S03QSV+rkqNpOSzedoSvY1NIv5BHgI8bLw7pyj29/WjaqIGlw7NK7i5OXMgrJP5YBmH+XpYO5zpVOSKvLW5ubsXPX3rpJQYOHMj3339PcnIyAwYMKHUbFxeX4ueOjo4UFFw/UMCcdWqCp6cn8fHxrFq1ig8++IClS5eycOFCfvzxR9avX8+KFSuYNWsWe/bsKfOaRW3RSV8zW2GRYs2BM3yx5QjrD53FQYTbujZnQr92RHTwwcFBH9WXx83FiXyBjQmpVpn0rVVmZia+vr4AfPrppzX+/p07dyYxMZHk5GT8/f356quvzN42PDycqVOnkpqaiqenJ0uWLOHJJ58kNTWVBg0aMHr0aDp37syECRMoKiri2LFjDBw4kMjISKKjo8nOzqZp06Y1/juVRyd9rUJnsnJZuv0YX249yonMXFo0cWHqLR0ZF96Wlh6ulg7PZjg6CEG+HsQkpDHtNktHYzuee+45Jk6cyGuvvcaQIUNq/P0bNmzI/PnzGTx4MG5ubvTp06fMdX/99deruoy+/vprZs+ezcCBA1FKMWTIEIYPH058fDwPPfQQRUXG1M+vv/46hYWFTJgwgczMTJRSTJ06tc4TPpg5R65pBvp3AEfgI6XU7GtebwcsBJoB6cAEpVSK6bWJwIumVV9TSn1WXlthYWFKTzRheUoptiSm88XWI6zae4qCIkVkoA8T+rXl1q4tbLpPWkR2KKUqHutXw8LCwtS9r33Bh+sTiZ8xCDcXyx9z/f7773Tt2tXSYVhcdnY2jRs3RinF448/TseOHZk+fbqlwypTaf9v5u7XFe51IuIIzANux5iJfruILFdK7S+x2hzgc6XUZyJyC/A68ICIeAEzgDBAATtM254z83fT6ljmxXy+25nC4q1HSTiTjUdDZyb292d837a0b9bY0uHZvIgOPry/7jDbktIZ2KW5pcPRTD788EM+++wz8vLyCA0N5bHHHrN0SLXGnEONcCBBKZUIICLRwHCgZNLvBjxter4WWGZ6fgfwi1Iq3bTtL8BgYEm1I9dq1N7jmXyx5Qj/jTvBxfxCgts05Y17enJ3cGtcnR0tHV69EebvWTx0Uyd96zF9+nSrPrKvSeYkfV/gWImfU4C+16wTD4zC6AIaCbiLiHcZ2/pe24CITAImAbRt29bc2LVqys0vZEX8Cb7YepT4Yxm4OjswIsSX8X3bEeTnYenw6iVXZ0fC2nmyMSHV0qFodqqmOhX/ArwnIlHAeuA4UGjuxkqpBcACMPo9aygmrRw/7z3Fi8v2kpp9iQ7N3JhxdzdG9fLDo2HVJmbQzBcR6MMbqw6Smn0Jn8YuFW+gaTXInKR/HGhT4mc/07JiSqkTGEf6iEhjYLRSKkNEjgMDrtl2XTXi1aopIyePGcv38d+4E3Rv3YS540K4ob2up16XIk1JP+ZwGsOCW1s6HM3OmDMEYzvQUUQCRKQBMBZYXnIFEfERkcvv9TeMkTwAq4BBIuIpIp7AINMyzQJW7z/N7W+t58fdJ5l+WyeWPR5B/w4+OuHXsR6+HjRxdWLTId3Fo9W9CpO+UqoAeAIjWf8OLFVK7RORV0RkmGm1AcBBEfkDaAHMMm2bDryK8cWxHXjl8kVdre5kXsznmaXxPPJ5LN5uDVj2eARP3dbRpodd2jJHB+GGDt5sTEi1+1LLAwcOZNWqq48D3377baZMmVLmNgMGDODysO677rqr1Bo2M2fOZM6cOeW2vWzZMvbvvzIe5eWXX2b16tWViL501l6C2aw+faXUSmDlNcteLvH8G+CbMrZdyJUjf62OrTt4hue/3cPZ7Es8eUsgT97S0W4rXFqTyEAfVu07zdH0HNp5u1W8QT01btw4oqOjueOOO4qXRUdH869//cus7VeuXFnxSmVYtmwZQ4cOpVu3bgC88sorVX4vW6I//fVUVm4+z3+7m6hPtuPu6sR3U/rzzKDOOuFbif6BPgB2P4rnnnvu4ccffyyeMCU5OZkTJ05w4403MmXKFMLCwujevTszZswodXt/f39SU42/4axZs+jUqRORkZHF5ZfBGIPfp08fgoODGT16NDk5OcTExLB8+XKeffZZQkJCOHz4MFFRUXzzjXHs+uuvvxIaGkpQUBAPP/wwly5dKm5vxowZ9OrVi6CgIA4cOGD272otJZgtf0ugVuM2HkrluW/iOXU+l8k3d2DabR31WHsr097HjVYersQkpDG+bztLh2P46Xk4tadm37NlENw5u8yXvby8CA8P56effmL48OFER0dz3333ISLMmjULLy8vCgsLufXWW9m9ezc9e/Ys9X127NhBdHQ0cXFxFBQU0KtXL3r37g3AqFGjePTRRwF48cUX+fjjj3nyyScZNmwYQ4cO5Z577rnqvXJzc4mKiuLXX3+lU6dOPPjgg7z//vtMmzYNAB8fH3bu3Mn8+fOZM2cOH330UYV/BmsqwawP++qRC5cKeHHZHiZ8vBXXBo58M6U/z9/ZRSd8KyQi9O/gw6bDqRQV2Xe//uUuHjC6di7Xs1+6dCm9evUiNDSUffv2XdX/fq0NGzYwcuRIGjVqRJMmTRg2bFjxa3v37uXGG28kKCiIxYsXs2/fvnLjOXjwIAEBAXTq1AmAiRMnsn79+uLXR40aBUDv3r1JTk4263csWYLZycmpuARz+/bti0sw//zzzzRpYsyqdrkE8xdffFHjVTj1kX49sflwGs9+E8/xjIs8EhnAX+7orJO9lYvs6M23O1PYf/I8PXyt4Ga4co7Ia9Pw4cOZPn06O3fuJCcnh969e5OUlMScOXPYvn07np6eREVFkZubW6X3j4qKYtmyZQQHB/Ppp5+ybt26asV7uTxzTZRmtkQJZn2kb+Ny8gqYuXwf4z7cgqODsPSxG3hxaDed8G1ARAejX3+TnffrN27cmIEDB/Lwww8XH+WfP38eNzc3PDw8OH36ND/99FO573HTTTexbNkyLl68SFZWFitWrCh+LSsri1atWpGfn8/ixYuLl7u7u5OVlXXde3Xu3Jnk5GQSEhIAWLRoETfffHO1fsfw8HB+++03UlNTKSwsZMmSJdx8882kpqZSVFTE6NGjee2119i5c+dVJZj/+c9/kpmZSXZ2drXaL0kf6duw7cnpPPt1PMlpOUT19+e5wZ31tIQ2pHkTVzo2b8zGhFQeu7mDpcOxqHHjxjFy5Mjibp7g4GBCQ0Pp0qULbdq0ISIiotzte/XqxZgxYwgODqZ58+ZXlUd+9dVX6du3L82aNaNv377FiX7s2LE8+uijzJ07t/gCLoCrqyuffPIJ9957LwUFBfTp04fJkydX6vex5hLMZpVWrku6tHLFcvMLmbPqIB9vSsK3aUPeuCeYGzp4Wzosm2HJ0srX7tszl+8jevtR4mcMwsWp7s/OdGll21Sd0sq6e8fG7Dp6jrvmbuCjjUncH96WVdNu0gnfhkUG+pCbX8TOIxmWDkWzEzrp24jc/EJm/3SA0e/HkJtXyKI/hTNrZJBVTMRR34jIQhE5IyJ7y3hdRGSuiCSIyG4R6VXVtvq298LRQey+X1+rOzrp24DdKRnc/e5GPvjtMPeFtWHV9Ju4sWMzS4dVn32KMe9DWe4EOpoek4D3q9qQu6szwX4ebDpsuaRvbV28Wvmq+/+lk74Vyyso4t//O8jI+TGcz83nk4f6MHt0T9xddfnj2qSUWo8x7WdZhmPMFKeUUluApiLSqqrtRQb6EH8sg/O5+VV9iypzdXUlLS1NJ34boZQiLS0NV9eqz02t+was1L4TmTyzNJ4Dp7IY3cuPl4d2w6ORTvZWoqzJgU5eu6I5EwT1D/Rh7poEthxOY1D3ljUfbTn8/PxISUnh7NmzddquVnWurq5XjQyqLJ30rUx+YRHz1x7m3TWH8HRrwIcPhnF7txaWDkurInMmCApt25SGzo7EWCDpOzs7ExAQUKdtapalk74VOXgqi2e+jmPv8fMMD2nNzLu74+nWwNJhadercGKhynBxcqRPgJfdF1/T6obu07cCBYVFzFubwNB3N3AyI5cPJvTinbGhOuFbr+XAg6ZRPP2ATKXUdV07lREZ6E3CmWxOn69aqQFNM5c+0rew4xkX+fPincQfy+CuoJa8OrwH3nreVIsSkSUYEwP5iEgKMANwBlBKfYAxt8RdQAKQAzxU3TYjAq+UZBjVq+r9tZpWEZ30LSg3v5BJn8dyNC2Hd8eFcreeL9UqKKXGVfC6Ah6vyTa7tmyCl1sDNuqkr9Uys7p3RGSwiBw03YzyfCmvtxWRtSKyy3Szyl2m5f4iclFE4kyPD2r6F7Blr/ywn30nzvPWmBCd8O2cg2kKxZgEPXxSq10VJn0RcQTmYdyQ0g0YJyLdrlntRYy5c0MxJk6fX+K1w0qpENOjclWL6rHvdqbw5dajTBnQgdv06BwNY7z+qfO5HD57wdKhaPWYOUf64UCCUipRKZUHRGPcnFKSApqYnnsAJ2ouxPrnj9NZ/P37vfQN8OKZ2ztZOhzNSuhSy1pdMCfpl3UjSkkzgQmmi14rgSdLvBZg6vb5TURuLK0BEZkkIrEiElvfbxLJvlTA5C924ObixLvjQnFy1AOoNENb70a08Wqok75Wq2oq44wDPlVK+WGMalgkIg4Ydyi2NXX7PA18KSJNrt1YKbVAKRWmlApr1qz+1pRRSvG37/aQnHqBd8eF0rxJ1W+l1uqniA4+bE5Mo6CwyNKhaPWUOUnfnBtR/gQsBVBKbQZcAR+l1CWlVJpp+Q7gMGC3/RmLthxhRfwJnhnUWZdD1koVEehDVm4Be0+ct3QoWj1lTtLfDnQUkQARaYBxoXb5NescBW4FEJGuGEn/rIg0M10IRkTaY1QlTKyp4G1J3LEMXv1hP7d0ac4UO58lSStbf9PBgO7i0WpLhUlfKVUAPAGsAn7HGKWzT0ReEZHLU84/AzwqIvHAEiDKNJb5JmC3iMQB3wCTlVLlVS+sl85dyOPxxTtp7u7Km/cF4+Aglg5Js1LejV3o2qoJGw/ppK/VDrNuzlJKrcS4QFty2cslnu8HrpvEUin1LfBtNWO0aUVFiqeXxnEmK5dvJvenaSNdWkErX2SgN5/FHOFiXiENG+gJ7rWapYeO1LL3fzvM2oNneXloN4LbNLV0OJoNiAj0Ia+wiNgjdndSrNUBnfRrUczhVP79v4PcHdyaCf3aWToczUaEB3jh7Ci66qZWK3TSryVnzucydUkcAT5uvD4qCBHdj6+Zp1EDJ0LbehKTkGbpULR6SCf9WlBQWMQTS3Zx4VIB70/oTWM9eblWSZGBPuw9kUlGTp6lQ9HqGZ30a8Gc//3BtqR0/m9UDzq1cLd0OJoNigj0RinYfFgf7Ws1Syf9GvbL/tN88Nth7u/blpGhukSuVjU9/ZrS2MVJ9+trNU4n/Rp0LD2HZ5bG0cO3CS8PvbYQqaaZz9nRgb4BXvomLa3G6aRfQ3LzC5myeAcKmH9/b1yd9fhq7RpFhZVaPSLQh+S0HFLO5dRSQJo90km/hrz6w372Hj/Pm/eF0Na7kaXD0axR+mHIv2j26pEdjVLLehSPVpN00q8By3YdZ/HWozx2c3tu1xOiaGXJy4HvJkGReRU0OzZvTDN3F92vr9UonfSr6dDpLP723R7C/b14dlBnS4ejWTOP1vD7clj9csXrAiJCRAdvYg6n6ikUtRqjk341XLhUwJTFO3FzceTd+/WEKFoF3JpDn0ch5l3Y/pFZm0QE+pCancfB01m1HJxmL3SWqqLLE6Ikns1m7rhQWugJUTRzDJ4NnQbDymfhj/9VuHpEoNGvr6tuajVFJ/0q+mLrUZbHn+Dp2zvR3zS3qaZVyNEJRn8MLXrA11FwMr7c1Vs3bUh7Hzdi9E1aWg3RSb8Kdqdk8OqK/Qzo3Iw/Dwi0dDiarXFpDPcvhYae8OUYyLx2Irqr9Q/0ZmtiGvl6CkWtBuikX0kZOXlM+WInzdxdeOu+ED0hilY1TVrB+KVwKRu+vA9yy54eMTLQhwt5hcQfy6i7+LTKKyyAnYvg2HZLR1Ius5K+iAwWkYMikiAiz5fyelsRWSsiu0Rkt4jcVeK1v5m2Oygid9Rk8HWtqEjxzNJ4zmTlMm98Lzzd9IQoWjW06A5jPoezB4yunsL8Ule7ob0PIuihm9ZKKfh9BbzfH5Y/AfFfWjqiclWY9E1z3M4D7gS6AeNE5NoaAy9iTKMYijGH7nzTtt1MP3cHBgPzL8+Za4s+WH+YXw+c4cUh3QjRE6JoNaHDLTD0LTj8K/z4jJFAruHRyJkgXw9dksEaJW+Ej2+HryaAKoL7FsGQNy0dVbnMqfkbDiQopRIBRCQaGA7sL7GOApqYnnsAJ0zPhwPRSqlLQJKIJJjeb3MNxF6nNh9OY86qgwzp2YoHb9ATomg1qNeDcC4ZNvwbvAIgcvp1q0QE+vDh+kQuXCrATZfqtryTu+HXf0DCanBvDXfPhZDxxoV6K2dO944vcKzEzymmZSXNBCaISArGXLpPVmJbRGSSiMSKSOzZs2fNDL3unDmfy5NLduHv48Y/R/fUE6JoNW/gi9BjNKyeCXu/u+7lyEAfCooU25L0FIoWlZ4I3/wJ/nMjpMTC7a/A1J3Qe6JNJHyouQu544BPlVJ+wF3AIhEx+72VUguUUmFKqbBmzZrVUEg1o6CwiCeX7CL7Uj7vj9cTomi1xMEBhs+HtjfA95Ph6JarXu7dzpMGTg66X99Sss/Aj3+B9/rAgR8h8ml4Kh4ingLnhpaOrlLMyWDHgTYlfvYzLSvpTxh99iilNouIK+Bj5rZW7c1f/mBrUjr/vjeYzi31hChaLXJ2hbFfwke3wZJx8Mhq8O4AgKuzI338PXW/fl3LPQ8xc2HzfCjINbribv6rMfrKRplzNL4d6CgiASLSAOPC7PJr1jkK3AogIl0BV+Csab2xIuIiIgFAR2BbTQVf2379/TTz1x1mXHgbRvfWE6JodaCRF4z/2ni++F7IudKd07+DDwdOZZGafclCwdmR/FzYPA/eCYb1b0CnQfDEdrj7bZtO+GBG0ldKFQBPAKuA3zFG6ewTkVdEZJhptWeAR0UkHlgCRCnDPmApxkXfn4HHlVKVKypuIcfSc3h6aTzdWzdhxt3dLR2OZk+8O8C4JZCZAtH3GwkIo18f0Hfn1qaiQti1GN4Lg1UvQKtgmLQO7v20+KzL1pnVQa2UWolxgbbkspdLPN8PRJSx7SxgVjVirHOXCgp5/MudFCnF/PG99IQoWt1r2w9GfgDfPAT//TOM+ogevh40cXVi06FUhgW3tnSE9YtScHAl/PqKcd9E61AY/h60H2DpyGqcvipZitd++J3dKZn854HetPN2s3Q4mr3qMQoyjhgjejz9cbz1ZW7o4M3GBKPUsh5FVkOOxBh/42NbwTsQ7v0Mug2Hevr31Un/Gv+NO86iLUeYdFN77uje0tLhaPYuYhqkJxlj+D39iQy8iVX7TnM0PUcfkFTXqb3Gkf2hVeDeCoa+DaETwNHZ0pHVKl17p4SEM8aEKH38PXn2Dj0himYFRGDIv6HDrbBiGre5GPdE6qGb1XAu2ZjB7INIOLYFbpsJT+6EsIfqfcIHnfSLXbhUwOQvdtLQ2ZF3x/XCWU+IYteqU2+qxjk6GxcSm3Wh5arHiHQ/rYduVkX2WVj5HLwbBvv/a4yxfyreuAO6gf3Ma60zm8k/VuzjsGlClJYeekIUe1adelO1xrUJjF+KNHDjXV7nUMIhior0FIpmyT0Pa1+HuSHGjGUh98PUXXD7P4zy1nZGJ30gN7+QZXEnuD+8bfFMRZpdK643pZTKAy7XmyqprHpTtcfDD+7/CveiLN4sfJ0DR0/WepM2reASbHnfSPa/zYbAW+HxrTBsLjSx39FPOukD8ccyyCsoYkDn5pYORbMO1ak3dZUaryvVKpjsuz+imxzBbcUkY1y5drWiQoiPNsba//y8UcL60TVw3+fg09HS0VmcTvrAtqR0RKCPv/2d6mlVZla9qdqoK9U0ZCjzGj5Gu7QN8NNfSy3HbJcK8yHuS5gXDt8/ZnTdPPA9PLgcfHtbOjqroYdsAluT0uncwp2mjfSkKBpQvXpTZ+oiwPRuD/JxbAp/2v6hUY75hsfrolnrVHAJ4hbDxrcg4yi0CDLG2ncdZhSy065i93+R/MIidhw5R98AL0uHolmP6tSbqhORgT68ljeWtLaDYdXfjZmb7E1eDmz5AN4JgR+mg1szGPcVTN4A3UfohF8Guz/S33M8k4v5hfRt723pUDQroZQqEJHL9aYcgYWX600BsUqp5Rj1pj4UkekYF3WjlKq7fpa+7b1wcHBkUcsXmFaUBt8+ClE/gp8ddGNcyoLYhRDzLlw4C+0iYMQ8aD+w3t5FW5PsPulfnpSij78+0teuqE69qbrg7upMsJ8H65KymTZxCXx0KywZY5Rj9vS3VFi162IGbFsAW+bDxXPGVJM3/gX8LfbfYJPs/vxnW1I67Zu50czdxdKhaFqlRAb6sDslg/NOTWH8N8aFzMX3GQmxPrmQBr++Cm8HwdpZ0KYfPPKrcZFWJ/xKs+sj/cIixfakdIZaqmLhxQzY+RkkrTf6I91bGeOH3VsZNbub+BrLHXSVT+16EYE+zF2TwJbDaQzq3gnGLobPR8BXD8CE78DJxgcmZJ0yunBiF0L+Reg2zDiyb9XT0pHZNLtO+r+fPE/WpYK6v4ibdhi2fmDU7c6/AM26wJkDkH0KigquXlccwb3llS8C99ZXvhBKfknY0W3kmiG0rScNnR3ZlJDKoO4twT8Shs+D7yfBiqkw4n3b7OPOTIFN78COz6AoH4LuNaYnbN7F0pHVC3ad9Lea+vPD6yLpKwXJG43+yIM/gYOTsTP3m3LlyKWoyLgwlXUCzp80/Vvi+dk/IPE3uHT++vd39SjxhdD6yvOS/zby1iMa6pEGTg6EB3ixqeSkKsFjjHLMa2cZffsDrisbZL3SE41hl3FLAAXB44y6OPVk8hJrYddJf1tSGm28GtK6aS1ObFyQB/u+M6ZeO7UbGnrBTX+BPo8YR/AlOTiAewvj0Tq07Pe8lA1ZJ+H88RJfDidNy07A6f1w4Qyoomve3/nK2cHlswW/Psbt6S56/l9bFBHozf+tPMCpzNwrNaNuetaoJLnudWjaDkLGWTTGCp09CBvehD1fGwdDvaOMYmhN21S4qVZ5ZiV9ERkMvIMxfO0jpdTsa15/Cxho+rER0Fwp1dT0WiGwx/TaUaXUMKyAUoptSenc0qVF7TRwIQ12LIRtHxndNs26wN3vQM8x4FzNLxmXxuDSsfxbygsLIPv0lS+Cy/9efn5yNxz8GTa/B44NIOAm6HwndLoTPK6tOKBZq4jiKRRTGdXLNI+ziFEbPvMYLH/SqNkTcKPlgizLqT2wfo5R8dK5oXHWe8MTNj8HrbWrMOmXqDh4O0YNku0istw0ZA0ApdT0Eus/CZQ8TL2olAqpsYhryKEz2ZzLyadv+xru2jl70OjCiY+GglyjDvqIeca/ddm/6uhkJO/yEnhhAaRsM6aJO7ASfnzGeLQKhs53GV8CLXvaZr+wnejasglebg3YmFAi6YNxEfe+RbDwDmOe3YCbjDO7Jq2NL4Emra90A9b1Bd+UHbBhjrHfNXCHG5+Gfn8GN13ssC6Yc6RfXHEQQEQuVxzcX8b644AZNRNe7bncn18jF3GVgsS1sHk+JPwCji5G32q/P0PzrtV//9ri6ATt+huPQa9B6iHjg3jwJ1g32+geaOJrJP/Od4L/jeCkh7ZaEwcH4YYO3mwqbQrFhk1h/Nfw0/OQlmCMEivtepBbc9OXgOkg4fLzkv/WxP/7kc2w/l9weA24NoUBL0DfSXZZ3tiSzEn6pVUc7FvaiiLSDggA1pRY7CoisUABMFsptayU7SYBkwDatm1rVuDVtTUxjZZNXGnrVY1RL/m5sGepUb71zH7jwzPw7xD2sG0etfh0BJ+njP7UC6nwxyrjSyDuS6MOeYPGRv9/57ug4yBopG9oswaRgT78uPskh89eILB546tfbNoWxn155efc86ZuvuMl/jU9P5dkDDa4lHl9I418rj9LaFLyeevSuy2VgsR1RjfOkY3GEOTb/gF9/qSvI1lITV/IHQt8o5QqWe+1nVLquIi0B9aIyB6l1OGSGymlFgALAMLCwmr9VvbL/fn92ntXbXLp7DNGEtz+MeSkQosexvC4HqPrz5Gwmw+Ejjce+bnGUeLls4D9/wVxgLY3XOkG0iMsLCbS1K+/KSH1+qR/LdcmxqO84Y+XsoyBASW/EIq/GI4YE4nnZly/XSPvq88O3FsZBw7HY43ng2dDr4l6eLGFmZP0zak4eNlY4Kpyf0qp46Z/E0VkHUZ//+HrN607R9JyOJN1qfJDNU/vM7pw9iyFwjzoNNjowgm4qX73ezu7QqdBxmPIm3Ayzkj+B3+C//3dePh0NnUD3QV+YfqGsjrUxqsRbbwasjEhlYn9/av/hi7u0MwdmnUqe528C6YvhpQrXwqZpi+GzONwbBtcTAePtsY+Ezqh/hwQ2Thzkn5xxUGMZD8WuP/alUSkC+AJbC6xzBPIUUpdEhEfjFol/6qJwKtja5IxrrmfORdxi4qMfvrN8yDpN3BuBL0ehL5TwCewliO1Qg4O4NvLeNzyd6OU7cGf4eCPxkigTW8bXQGdBhtfAh0GQgM3S0dd70UG+vDD7pMUFBbhVBfzOzdwM/b/8j4D+ReN61v63hCrUmHSN7PiIBhfBtHXVBrsCvxHRIow6vzMLjnqx1K2JqXj7daADs3KORXOuwDxS4zSrWmHjFEOt800Tk91X/YVTdsaF+P6ToLcTEhYbZwBHFgBcV8YH/r2A0zDQQfr4Xi1JCLQhyXbjrHneCahba3kwmh1hyZrtcKsPv2KKg6afp5ZynYxQFA14qsVWxPTCQ/wKr0///wJ2PahUe8jN8O4SWrUR0Z9bkfnug7Vtrh6GNc1eow2in8d3Wz6AvgRDq0y1mnd68p1gBbd63e3WB26wVQaPOZwmvUkfc0q2d0duSnncjiecZFHbgy4+oXT+41bwPd9Z9zJ2mUI9Hsc2vbTiakqHJ2Nax0BN8Ed/wdnD1y5ELz2Ndj4JjyXZFwv0KrNu7EL3Vo1YeOhVB4faIfdjprZ7C7pb08upd7Oyd2wcLAxIiV8EvR9rP7WJLcEEeN+heZd4cZnIOs0nNmnE34Niwj05rOYI1zMK6RhA30hXSud3V1h2ZqYThNXJ7q0bGIsOH8Svhxj3MjyxHYY/LpO+LXNvYUxAYZWoyICfcgrLCL2SLqlQ9GsmN0l/W1J6fTx98LRQYyLtUvGGHcp3v+Vvsio2bTwAC+cHYWNCamWDkWzYnaV9M+czyUx9YJRb6eoCL6bZBR9umchtLS6682aVimNGjgR2taTTTrpa+Wwq6S/rbg/3xtWz4ADP8Adr0OnOywcmabVjMhAH/adOM+5C3mWDkWzUvaV9JPSadTAkaBT30PMXOjzqHHRVtPqiYhAH5SCzYlpFa+s2SW7SvpbE9N5sEUSjj/9BQJvM2qB6OGYWj0S7OdBYxcn3cWjlcluhmyeu5BHwZkDTG/0Gvh0gns+MUoLa1o94uToQL/2Xjrpa2WymyP9XQcTWOj8Bg7OLsZIHdcmlg5J02pF/w4+JKflkHIux9KhaFbIPpJ+fi6Bax6jhZxDjVti1IvRtHoqsqNpCsUE3a+vXa/+J32lYPkTtM3ezftez9KgXanzv2havdGxeWOaubvo8fpaqep/0v/tn7Dna+YU3IfqNtLS0WharRMRIjp4E3PYmEJR00qq30l/99ew7nVOBozkvYLhNTMfrqbZgIhAH1Kz8zh4OsvSoWhWpv4m/aNb4L9/hnaRLPJ5BicHB3rpkrOanYgwTaG48ZDu4tGuVj+TfnoSRN8PHm1gzCK2HDlPTz8PXXlQsxutmzakvY8bMYf1xVztamYlfREZLCIHRSRBRJ4v5fW3RCTO9PhDRDJKvDZRRA6ZHhNrMPbSXcyAL++DokIY/zUXnTzYnZJJX9MkE5pmLyICfdiSmEZ+YZGlQ9GsSIVJX0QcgXnAnUA3YJyIdCu5jlJqulIqRCkVArwLfGfa1guYAfQFwoEZpnlza0dhPnw90TjSH7sYvDuw8+g5CopU5SdB1zQbFxHoTU5eIXHHMiwdimZFzDnSDwcSlFKJSqk8IBoYXs7644Alpud3AL8opdKVUueAX4DB1Qm4TErBj89A4jq4+x3wjwSM+XAdBMLa6f58zb7c0N4HEfTdudpVzEn6vsCxEj+nmJZdR0TaAQHAmspsKyKTRCRWRGLPnj1rTtzX2/we7PzMmJkpdHzx4q2JaXRv7YG7q57fVrMvHo2c6enroZO+dpWavpA7FvhGKVVYmY2UUguUUmFKqbBmzZpVvtXff4D/vQTdhsPAF4sXXyooZNexDN21o9mt/oE+7DqawYVLBZYORbMS5iT940CbEj/7mZaVZixXunYqu23VnIiD7x4F314w8j/gcOVX2p2SSV5BkR6fr9mtmzo2o6BIsWjLEUuHolkJc5L+dqCjiASISAOMxL782pVEpAvgCWwusXgVMEhEPE0XcAeZltWMzOOwZCw08oaxS8C54VUvbzXVFO/jr5O+Zp/6tfdicPeW/Pt/B9l7PNPS4WhWoMKkr5QqAJ7ASNa/A0uVUvtE5BURGVZi1bFAtCpx37dSKh14FeOLYzvwimlZ9V3KNs1vm21UzXRvcd0qW5PS6dzCHU+3BjXSpGY/KhqmbFrnPhHZLyL7ROTLuo7RHCLC7NFBeLu5MHXJLnLydDePvTOroLxSaiWw8pplL1/z88wytl0ILKxifKUrKoRvH4HT++D+pdCi+3Wr5BcWsePIOe7p7VejTWv1X4lhyrdjDD7YLiLLlVL7S6zTEfgbEKGUOicizS0TbcWaNmrAm2OCGf/RVl79YT+vj+pp6ZA0C7LNO3J/eRn++Anu/Bd0vL3UVfadOE9OXqG+iKtVhTnDlB8F5pmGIqOUOlPHMVZK/w4+TL65A0u2HePnvSctHY5mQbaX9Ld/bAzP7DsZwh8tc7VtSUZ/vk76WhWYM9S4E9BJRDaJyBYRKfX+kxoZjlxDpt/WiZ5+Hvz12z2czLxo0Vg0y7GtpJ/wK6x8FjoOgjv+r9xVtyam097HjeburnUUnGZnnICOwACMGxI/FJGm165U7eHINaiBkwPvjA0lv7CIp7+Kp7BIl122R7aT9M/8Dl9HQbMucM9CcCi7eFphkWJbcro+yteqypyhxinAcqVUvlIqCfgD40vAqgX4uDHz7u5sTkxjwfpES4ejWYBtJP3ss0YRNeeGxkgdF/dyVz9w6jxZuQX0ba+TvlYl5gxTXoZxlI+I+GB099hEFr03zI+7goxhnLtTMiwdjlbHrD/p51+E6HFG4h+3BJq2qXCTbUnGqNDwAF1ZU6s8M4cprwLSRGQ/sBZ4VillE3WMRYTXR/akubsLT0XH6bt17Yx1J/2iIlj2Z0jZDqMWgG9vszbblpSOn2dDfJs2rHhlTSuFUmqlUqqTUqqDUmqWadnLSqnlpudKKfW0UqqbUipIKRVt2Ygrx6ORM2+OCSE57QL/WLHP0uFodci6k/6612Hfd3DbTOg2rMLVAZRSbEvS/fmaVpF+7b3584AOLI1NYeUePYzTXlhv0o+PhvX/gtAJEDHN7M0On80m7UKerrejaWaYdlsngts05flvd3MiQw/jtAfWmfSPxMDyJ8H/RhjyFoiYvemWRKM/v6/uz9e0Cjk7OvDOmBAKixTTv4rTwzjtgPUl/YJLED0emraFMYvAqXJ1c7YlpdPc3YV23o1qKUBNq1/8fdz4x/AebE1K54PfDls6HK2WWV/ST08ElFFTp2HlZru63J/ft703UomzA02zd6N7+TK0Zyve+uUPPb1iPWd9Sb8wD8Z+Cd4dKr3p0fQcTp3P1RdxNa2SRIRZI4No0cSVp6J3ka2HcdZb1pf0W3SHdv2rtOnWpMv9+Trpa1pleTR05q0xIRxLz2Hmcj2Ms76yvqTvYFa151JtTUzHy60BHZs3rsGANM1+hAd48cTAQL7ZkcKK+BOWDkerBdaX9KthW3Iaffw9dX++plXD1Fs7Etq2KS98v4eUczmWDkerYfUm6Z/IuMix9It6qKamVZOTowPvjAlFKXQ1znrIrKRfnanjRKRQROJMj+vm1q0pV+rt6P58Tauutt6NeGV4d7YlpzN/bYKlw9FqUIUd6DUwddxFpVRIzYZ9va1J6bi7OtG1VZPabkrT7MLIUF/WHTzL278eIqKjD73aVm4ItWadzDnSt4mp47YmpdHH3wtHB92fr2k1QUR4bWQPWnm4Mi06jqzcfEuHpNUAc5J+daeOczVNF7dFREaU1kB1p5Q7m3WJxLMXdNeOptWwJq7OvD0mhJRzOcz4rx7GWR/U1IXc8qaOa6eUCgPuB94WkevuuqrulHLbk/X4fE2rLWH+Xjx5S0e+23Wc/8ZdO4GYZmvMSfrVmjpOKXXc9G8isA4IrWbM19mamEZDZ0d6+HrU9FtrmgY8eUsgvdt58uL3ezmWrodx2jJzkn6Vp44TEU8RcSmxPALYTw3bmpRO73aeODvWmxGommZVnBwdeHtMCADTv4qjoLDIsgFpVVZhlqzm1HFdgVgRiTctn11y1E9NyMjJ4+DpLN21o2m1rI1XI14b2YPYI+d4Tw/jtFlm1TxQSq0EVl6z7OUSzxXwtOlRcp0YIKj6YZZte/I5lNLj8zWtLgwPMYZxzv31EDd29KF3O/25szU23x+yLSmNBk4OBLdpaulQNM0uvDK8O76eDXkqOo7zehinzbH5pL81KZ2QNk1xdXa0dCiaZhfcXZ15e0woJzNzeXnZXkuHo1WSTSf97EsF7D2eqfvzNa2O9W7nydRbOrIs7gTf70qxdDhaJdh00o9NTqdI6flwNc0SHh/YgT7+nry0bB9H0/QwTlth00l/W1I6Tg5Cr3ZNLR2KptkdJ0cH3hoTgghM+2qXHsZpI2w+6Qf5edCoQdUnXtE0rer8PBsxa2QQO49mMHeNHsZpC2w26V/MKyQ+JUMP1dQ0CxsW3JpRvXx5b82h4pIomvWy2aS/69g58guVvoiraVbgleE98PNsxLToODIv6mGc1sxmk/7WxHREjGJQmqZZVmMXJ94ZG8Kp87m8uGwvxv2amjWy2aS/LSmdbq2a0MTV2dKhaJoGhLb1ZPptHVkRf4LvdupqnNbKJpN+XkERO4+e00M1Nc3KTBkQSHiAFy//dy9H0i5YOhytFDaZ9HenZHCpoEhfxNU0K+PoILw1JgRHB+Gp6Djy9TBOq2OTSX+rngRd06yWb9OG/N+oIOKOZfDKiv0UFun+fWtikwPctyal06lFY7zcGlg6FE3TSjG0Z2t2Hc3g441JHE3PYe7YUDwa6etv1sDmjvQLCovYkZyuj/I1zcq9NLQbs0b2IOZwKsPmbeSP01mWDknDBpP+/pPnuZBXqC/iapoNGN+3HUse7UdOXiEj5m3i570nLR2S3TMr6YvIYBE5KCIJIvJ8GevcJyL7RWSfiHxZYvlEETlkekysbsBbE/Uk6JpmS8L8vVjxRCSdWrgz+YudzFl1kCLdz28xFSZ9EXEE5gF3At2AcSLS7Zp1OgJ/AyKUUt2BaablXsAMoC8QDswQEc/qBLw1KZ0AHzeaN3GtzttomlaHWnq48tVj/RgT1ob31ibwyOex+s5dCzHnSD8cSFBKJSql8oBoYPg16zwKzFNKnQNQSp0xLb8D+EUplW567RdgcFWDLSpSbE9OJ1zfhavVMnPObk3rjRYRJSJhdRmfLXJxcmT26CBeHdGD9X+cZcS8TSSc0f38dc2cpO8LHCvxc4ppWUmdgE4isklEtojI4Epsi4hMEpFYEYk9e/ZsmYEcPJ1F5sV8fRFXq1XmnN2a1nMHngK21m2EtktEeKBfO758tB9ZufmMmBfDqn2nLB2WXampC7lOQEdgADAO+FBEmpq7sVJqgVIqTCkV1qxZszLX22Yan9+3vU76Wq0y5+wW4FXgn0BuXQZXH4QHeLHiyUg6NHPjsUU7ePOXP3Q/fx0xJ+kfB9qU+NnPtKykFGC5UipfKZUE/IHxJWDOtmbbmpSGb9OG+Hk2qupbaJo5KjxDFZFeQBul1I/lvZG5Z7H2qJVHQ7567Abu6e3H3F8PMWlRrJ5ovQ6Yk/S3Ax1FJEBEGgBjgeXXrLMM4ygfEfHB6O5JBFYBg0TE03QBd5BpWaUppdiWlK5H7WgWJyIOwJvAMxWta+5ZrL1ydXbkjXt68o9h3Vl78HI/f7alw6rXKkz6SqkC4AmMZP07sFQptU9EXhGRYabVVgFpIrIfWAs8q5RKU0qlY5wCbzc9XjEtq7TDZy+Qmp2n+/O1ulDRGao70ANYJyLJQD9gub6YWzUiwsT+/ix+pC+ZOfmMmLeJ1ftPWzqsesusPn2l1EqlVCelVAel1CzTspeVUstNz5VS6mmlVDelVJBSKrrEtguVUoGmxydVDXSbrrej1Z1yz26VUplKKR+llL9Syh/YAgxTSsVaJtz6oV97b5Y/GUmAjxuPfB7LO6sP6X7+WmAztXe2JaXRzN2FAB+3617Lz88nJSWF3Fx9PU27wtXVFT8/P5ydK1fzRSlVICKXz24dgYWXz26B2MsHO1rN823akK8n38AL3+3hrdV/sO9EJv++Lxh3PW9GjbGJpK+UYmuSUW9HRK57PSUlBXd3d/z9/Ut9XbM/SinS0tJISUkhICCgKtuvBFZes+zlMtYdUKUgtVK5Ojvy7/uC6eHrwayVvzNyfgwLHuhN+2aNLR1avWATtXdSzl3kZGYu/cro2snNzcXb21snfK2YiODt7a3P/myUiPBwZACL/hROWvYlhs/bxJoDup+/JthE0t+SmAZAeDlF1nTC166l9wnb17+DD8ufiKSNZyP+9Fks7605pOffrSabSPrbktJp2siZjs316Z2m2Zs2Xo34dkp/hgW3Zs7//uDPi3dy4VKBpcOyWbaR9E31dhwcrPPILS0tjZCQEEJCQmjZsiW+vr7FP+fl5ZW7bWxsLFOnTq2wjf79+9dUuABMmzYNX19fior0dHaa9WvYwJG3x4Tw4pCurNp3ipHzN5GcqufgrQqrT/qnMnM5kpZj1UM1vb29iYuLIy4ujsmTJzN9+vTinxs0aEBBQdlHJWFhYcydO7fCNmJiYmos3qKiIr7//nvatGnDb7/9VmPve63yfm9NqywR4ZEb2/P5w305k3WJYe9tZN3BMxVvqF3F6kfvbE0y+vP7tTdv0pR/rNjH/hPnazSGbq2bMOPu7pXaJioqCldXV3bt2kVERARjx47lqaeeIjc3l4YNG/LJJ5/QuXNn1q1bx5w5c/jhhx+YOXMmR48eJTExkaNHjzJt2rTis4DGjRuTnZ3NunXrmDlzJj4+Puzdu5fevXvzxRdfICKsXLmSp59+Gjc3NyIiIkhMTOSHH364LrZ169bRvXt3xowZw5IlSxg4cCAAp0+fZvLkySQmJgLw/vvv079/fz7//HPmzJmDiNCzZ08WLVpEVFQUQ4cO5Z577rkuvpdeeglPT08OHDjAH3/8wYgRIzh27Bi5ubk89dRTTJo0CYCff/6ZF154gcLCQnx8fPjll1/o3LkzMTExNGvWjKKiIjp16sTmzZvRd7Nql0V29GHFE5FMWrSDhz7dzrN3dGbKzR30NRwz2UDST6exixNdWzWxdCiVlpKSQkxMDI6Ojpw/f54NGzbg5OTE6tWreeGFF/j222+v2+bAgQOsXbuWrKwsOnfuzJQpU64bZ75r1y727dtH69atiYiIYNOmTYSFhfHYY4+xfv16AgICGDduXJlxLVmyhHHjxjF8+HBeeOEF8vPzcXZ2ZurUqdx88818//33FBYWkp2dzb59+3jttdeIiYnBx8eH9PSKb6jeuXMne/fuLR4quXDhQry8vLh48SJ9+vRh9OjRFBUV8eijjxbHm56ejoODAxMmTGDx4sVMmzaN1atXExwcrBO+dh2jn/8GnvtmN//6+SD7jp/nX/f0xM3F6lOaxVn9X2hbUjph/p44mtmfX9kj8tp077334ujoCEBmZiYTJ07k0KFDiAj5+aUXlhoyZAguLi64uLjQvHlzTp8+jZ+f31XrhIeHFy8LCQkhOTmZxo0b0759++JEO27cOBYsWHDd++fl5bFy5UrefPNN3N3d6du3L6tWrWLo0KGsWbOGzz//HABHR0c8PDz4/PPPuffee/Hx8QHAy6vibrbw8PCrxsbPnTuX77//HoBjx45x6NAhzp49y0033VS83uX3ffjhhxk+fDjTpk1j4cKFPPTQQxW2p9mnRg2ceHdcKEG+Hvzz5wMcPpvNggfCaOutCzKWx6r79FOzL5FwJttm58N1c7ty9/BLL73EwIED2bt3LytWrChz/LiLi0vxc0dHx1L7xc1ZpyyrVq0iIyODoKAg/P392bhxI0uWLDF7+8ucnJyKLwIXFRVddcG65O+9bt06Vq9ezebNm4mPjyc0NLTcsfNt2rShRYsWrFmzhm3btnHnnXdWOjbNfogIj93cgU8fCudkZi53v7eR9X/oaqblseqkv70e1dvJzMzE19eozvvpp5/W+Pt37tyZxMREkpOTAfjqq69KXW/JkiV89NFHJCcnk5ycTFJSEr/88gs5OTnceuutvP/++wAUFhaSmZnJLbfcwtdff01amnFt5XL3jr+/Pzt27ABg+fLlZZ65ZGZm4unpSaNGjThw4ABbtmwBoF+/fqxfv56kpKSr3hfgkUceYcKECVedKWlaeW7q1IzlT0TQysOVqE+28f66wxQU6pFppbHqpL81KZ2Gzo4E+XpYOpRqe+655/jb3/5GaGhorYxqadiwIfPnz2fw4MH07t0bd3d3PDyu/rvl5OTw888/M2TIkOJlbm5uREZGsmLFCt555x3Wrl1LUFAQvXv3Zv/+/XTv3p2///3v3HzzzQQHB/P0008D8Oijj/Lbb78RHBzM5s2brzq6L2nw4MEUFBTQtWtXnn/+efr16wdAs2bNWLBgAaNGjSI4OJgxY8YUbzNs2DCys7N1145WKe283fh2Sn/u7NGKf/58gNvfWs+yXccp1EXbriLWdndbWFiYio01ihXe+c4GvNycWfxIv3K3+f333+natWtdhGfVsrOzady4MUopHn/8cTp27Mj06dMtHValxcbGMn36dDZs2FDt9ypt3xCRHUqpOi+DXHLf1mqPUor/7T/NW7/8wYFTWQQ2b8y02zpyV49WVnuvT00wd7+22iP9zJx8Dpw6T7i/bfbnW8KHH35ISEgI3bt3JzMzk8cee8zSIVXa7NmzGT16NK+//rqlQ9FslIhwR/eWrJx6I/PH90KAJ77cxV1zN/Dz3lN2X8bBao/0f/39NH/6LJboSf0qHKOvj/S1sugjfa2wSPHD7hO8s/oQiakX6N66CU/f3olbujSvV2P7bf5If2tSOg0cHQhp09TSoWiaZsMcHYThIb78b/pN/PveYLJyC/jTZ7GMmB/Db3+ctbsjf7OSvogMFpGDIpIgIs+X8nqUiJwVkTjT45ESrxWWWG725BNbk9IJadMUV2c9ekPTtOpzcnRgdG8/fn3mZv45OojUrEtMXLiNez/YTExCqt0k/wpvzhIRR2AecDuQAmwXkeVKqf3XrPqVUuqJUt7iolIqpDJBZV8qYO/xTKbc3KEym2maplXI2dGBMX3aMjLUj6Wxx3hvTQL3f7SVvgFePDOoc70YIl4ec470w4EEpVSiUioPiAaG12ZQO4+co7BI1fs/vqZpltPAyYEJ/dqx7tkBzLy7G4mpF7jvP5uZ8NFWdhw5Z+nwao05Sd8XOFbi5xTTsmuNFpHdIvKNiLQpsdxVRGJFZIuIjCitARGZZFon9uzZs2xLSsfRQejdztPsX8SSBg4cyKpVq65a9vbbbzNlypQytxkwYACXL+rdddddZGRkXLfOzJkzmTNnTrltL1u2jP37r5x0vfzyy6xevboS0ZdPl2DW6jtXZ0eiIgLY8NxAXhzSld9Pnmf0+zFEfbKN+GMZlg6vxtXUhdwVgL9SqifwC/BZidfama4o3w+8LSLX9dkopRYopcKUUmHNmjVja1IaPXw9bKZ40rhx44iOjr5qWXR0dLlFz0pauXIlTZs2rVLb1yb9V155hdtuu61K73UtXYJZsyeuzo48cmN71j83kL8O7kLcsQyGz9vEI5/Fsu9EpqXDqzHmJP3jQMkjdz/TsmJKqTSl1CXTjx8BvUu8dtz0byKwDggtrzGlIP5YZpnz4Vbop+fhkyE1+/jpumvXV7nnnnv48ccfi+vPJCcnc+LECW688UamTJlCWFgY3bt3Z8aMGaVu7+/vT2pqKgCzZs2iU6dOREZGcvDgweJ1PvzwQ/r06UNwcDCjR48mJyeHmJgYli9fzrPPPktISAiHDx8mKiqKb775BoBff/2V0NBQgoKCePjhh7l06VJxezNmzKBXr14EBQVx4MCBUuO6XIJ5ypQpV9XnOX36NCNHjiQ4OJjg4ODiWv+ff/45PXv2JDg4mAceeADgqnjAKMF8+b1vvPFGhg0bRrdu3QAYMWIEvXv3pnv37lcVi/v555/p1asXwcHB3HrrrRQVFdGxY0fOnjVqrBQVFREYGFj8s6ZVh5uLE1MGdGDDcwN55vZObEtKY8jcjUz5YgcHT2VZOrxqMyfpbwc6ikiAiDQAxgJXjcIRkVYlfhwG/G5a7ikiLqbnPkAEcO0F4Kvk5BWQV1hkU/35Xl5ehIeH89NPPwHGUf59992HiDBr1ixiY2PZvXs3v/32G7t37y7zfXbs2EF0dDRxcXGsXLmS7du3F782atQotm/fTnx8PF27duXjjz+mf//+DBs2jDfeeIO4uDg6dLhyEpWbm0tUVBRfffUVe/bsoaCgoLiuDoCPjw87d+5kypQpZXYhXS7BPHLkSH788cfi+jqXSzDHx8ezc+dOunfvXlyCec2aNcTHx/POO+9U+HfbuXMn77zzDn/88QdglGDesWMHsbGxzJ07l7S0NM6ePcujjz7Kt99+S3x8PF9//fVVJZgBXYJZqxXurs48eWtHNvz1Fqbe2pENh1IZ/M56nvhyJwlnsi0dXpVV2H+ilCoQkSeAVYAjsFAptU9EXgFilVLLgakiMgwoANKBKNPmXYH/iEgRxhfM7FJG/VzlwqVCHAXC/KuY9O+cXbXtqulyF8/w4cOJjo7m448/BmDp0qUsWLCAgoICTp48yf79++nZs2ep77FhwwZGjhxJo0ZGadhhw4YVv7Z3715efPFFMjIyyM7O5o477ig3noMHDxIQEECnTp0AmDhxIvPmzWPatGmA8SUC0Lt3b7777rvrttclmDXN4NHQmadv78TDEf4sWJ/IpzHJrNxzkhEhvky9tSP+PqXXnbJWZnWaK6VWAiuvWfZyied/A/5WynYxQFBlArqQV0Dflk3waOhc8cpWZPjw4UyfPp2dO3eSk5ND7969SUpKYs6cOWzfvh1PT0+ioqLKLStcnqioKJYtW0ZwcDCffvop69atq1a8l8szl1WauWQJZjCKtTVs2JChQ4dWqp2qlGBu1KgRAwYMqFQJ5stH/ZpWW5o2asBzg7vwp8gA/rM+kc83J/Pf+BOM7uXLk7d0pI2XbdTxt7o7ci/kFdhU185ljRs3ZuDAgTz88MPFF3DPnz+Pm5sbHh4enD59urj7pyw33XQTy5Yt4+LFi2RlZbFixYri17KysmjVqhX5+flXJTh3d3eysq7vZ+zcuTPJyckkJCQAsGjRIm6++Wazfx9dglnTSufd2IUX7urK+ucG8uAN7VgWd4KBc9bxt+/2cCLjoqXDq5DVJX2loF9720v6YHTxxMfHFyf94OBgQkND6dKlC/fffz8RERHlbt+rVy/GjBlDcHAwd955J3369Cl+7dVXX6Vv375ERETQpUuX4uVjx47ljTfeIDQ0lMOHDxcvd3V15ZNPPuHee+8lKCgIBwcHJk+ebNbvoUswa1rFmru7MuPu7qx/diDjwtvyzY5jDHhjHfPXJVg6tHJZXcE1n4Cu6sCeOHwau1S8sokuuGafzCnBrAuuaXXleMZF3luTQL/2XgwPKe1Wptpl7n5tdQPh/b3dKpXwNfs0e/Zs3n//fd2Xr1kN36YNeX1UpS5hWoTVde9omjmef/55jhw5QmRkpKVD0TSbUm+SvrV1U2mWp/cJTbtevUj6rq6upKWl6Q+5VkwpRVpaGq6urpYORdOsitX16VeFn58fKSkp+jZ87Squrq74+flZOgxNsyr1Iuk7OztfdWenpmmaVrp60b2jaZqmmUcnfU3TNDuik76maZodsbo7ckUkCzhY4Yq1wwdItaN2Ldm2JX/nzkop97puVO/bdtGuJds2a7+2xgu5By1xizyAiMRaom1LtWvJti39O1uiXfS+Xe/btWTb5u7XuntH0zTNjuikr2maZkesMekvqHiVete2/p3to239O9f/di3ZtlntWt2FXE3TNK32WOORvqZpmlZLdNLXNE2zI1aT9EVkoYicEZG9ddxuGxFZKyL7RWSfiDxVh227isg2EYk3tf2Pumrb1L6jiOwSkR/quN1kEdkjInF1OXxSRJqKyDcickBEfheRG+qoXb1v28G+ban92tS22fu21fTpi8hNQDbwuVKqRx222wpopZTaKSLuwA5ghFJqfx20LYCbUipbRJyBjcBTSqkttd22qf2ngTCgiVJqaF20aWo3GQhTStXpDSwi8hmwQSn1kYg0ABoppTLqoF29b9vBvm2p/drUttn7ttUc6Sul1gPpFmj3pFJqp+l5FvA7UCcTXCpDtulHZ9OjTr6FRcQPGAJ8VBftWZqIeAA3AR8DKKXy6iLhm9rS+7bet2tNZfdtq0n61kBE/IFQYGsdtukoInHAGeAXpVRdtf028BxQVEftlaSA/4nIDhGZVEdtBgBngU9Mp/0fiYhbHbVtcXrfrhOW2K+hkvu2TvomItIY+BaYppQ6X1ftKqUKlVIhgB8QLiK1fvovIkOBM0qpHbXdVhkilVK9gDuBx03dH7XNCegFvK+UCgUuAM/XQbsWp/ftOmOJ/RoquW/rpA+Y+hy/BRYrpb6zRAym07G1wOA6aC4CGGbqg4wGbhGRL+qgXQCUUsdN/54BvgfC66DZFCClxNHmNxgflHpN79t1t29baL+GSu7bdp/0TRecPgZ+V0q9WcdtNxORpqbnDYHbgQO13a5S6m9KKT+llD8wFlijlJpQ2+0CiIib6aIiplPQQUCtj2pRSp0CjolIZ9OiW4Fav6BpSXrfrrt921L7NVR+37aaKpsisgQYAPiISAowQyn1cR00HQE8AOwx9T8CvKCUWlkHbbcCPhMRR4wv4KVKqTodPmkBLYDvjXyEE/ClUurnOmr7SWCxaXRDIvBQXTSq92272LctuV9DJfZtqxmyqWmaptU+u+/e0TRNsyc66WuaptkRnfQ1TdPsiE76mqZpdkQnfU3TNDuik76maZod0Ulf0zTNjvw/GAts879CMJ8AAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
],
"source": [
"# plot accuracy and val_accuracy\n",
"acc = history['accuracy']\n",
"val_acc = history['val_accuracy']\n",
"\n",
"loss = history['loss']\n",
"val_loss = history['val_loss']\n",
"\n",
"epochs_range = range(1, epochs+1)\n",
"\n",
"plt.subplot(1,2,1)\n",
"plt.plot(epochs_range, acc, label='Training Accuracy')\n",
"plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
"plt.legend(loc='best')\n",
"plt.xlim(1,len(val_acc)+1)\n",
"plt.title('Training Accuracy')\n",
"\n",
"plt.subplot(1,2,2)\n",
"plt.plot(epochs_range, loss, label='Training Loss')\n",
"plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
"plt.legend(loc='best')\n",
"plt.xlim(1,len(val_acc)+1)\n",
"plt.title('Training Loss')\n",
"plt.show()"
]
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
{
"cell_type": "code",
"execution_count": 16,
"id": "bbbc20bf",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Text(32.99999999999999, 0.5, 'predicted')"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAccAAAFfCAYAAAA7yQecAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABOY0lEQVR4nO3dd3gU5drH8e8vIUjvRZp0RbAgomKXInaxgoq9cOwFUWkHez/q0ddyxIodbEfsehS7IlVQERtdBJXek3C/f8wEljUkG7KbyS73h2uu7Dw7O3Mvgb33eeYpMjOcc845t1FW1AE455xz5Y0nR+eccy6OJ0fnnHMujidH55xzLo4nR+eccy6OJ0fnnHMuToWoA3DROeuFqRk3jueSLs2jDiEl2jetEXUISbc+Q4eRZUlRh5ASlSpQqjdWebeLE/6Fr550f+R/iZ4cnXPOpZ7Sq6HSk6NzzrnUy8qOOoIS8eTonHMu9dKsudmTo3POudTzZlXnnHMujtccnXPOuThec3TOOefieM3ROeeci+O9VZ1zzrk43qzqnHPOxfFmVeeccy6O1xydc865OJ4cnXPOuTjZ3iHHOeec25Tfc3TOOefieLOqc845F8drjgFJ+cDUmKIXzOy2Io4fbGa3pCqezVzzIGCAmR2Z4PFnAo8DHc1sSlj2LXCkmc1MYlwdgcZm9la4fzTQvqi/v6hJcG3PNixelcu9n86ie9u6HLx9XRpW34ZLXvmeFevyNzm+ZZ3KDOnRmv98MZvxc5dFFHXR/nPXDUwa+xk1atXmzuEjN5S/89pI3h/9IsrKYre99qPvuZeSl5fH8HtuYubPP5Cfn8/+PQ7nmJPOijD6kvl9/nyGDLqaRX/9BRInnNibvqedEXVYpTZzxq9cM6D/hv15c+dwwcWXpv17GzZ0EJ98/BF16tTlldfeiDqcxHjNcYPVZtaxBMcPBv6WHCUJkJmtT1ZgpTQXGAL0SeE1OgKdgbcAzGw0MDqF1yu1g7evx/xla6lUIfgP8NMfK5n82zIGdmv1t2MlOHHXbfnu9xVlHWaJHNjzSA45ujcP3nnthrLvJo9nwhcfc9tDz5FTsSJLlywCYOwn/yMvdx13PPwCa9esYUC/3ux70CHU37ZxVOGXSHaFbAZcPZAd23dg5coVnHTi8XTZe19at2kTdWil0qJlK0a+/F8A8vPzOaTbgXTt3iPaoJKg1zHHcfIppzJk0DVRh5K4JNYcJe0AjIwpagUMA54Ky1sAM4HeZrY4zCP3AocDq4AzzWxiUdco01Quqaak6eEbQ9Lzks6TdBtQWdJkSc9KahEe9xTwLdBM0lWSxkmaIun68PUtJP0g6UlJP4av7SHpc0k/SdozPK6qpMclfS1pkqRecXFlhcfXj9n/uWA/zhtAh4L3EHeenpK+lDRR0ouSqoXlh4dxTpB0n6Q3wvI9w+MnSfpC0g6SKgI3AH3Cv48+ks6UdH/49zdLCr6Che9rjqQcSa0lvRNe41NJ7ZLyS0tA7coV2LVxdT75ZdGGstlL1vDXytxCj+/Rti7j5yxl2dq8sgpxi+y4cyeqVa+xSdn7b7zM0X3OIKdiRQBq1qoTPCGxds1q8vPzWLduDRUq5FC5StWyDnmL1a/fgB3bdwCgatVqtGrVioULF0QcVXJ9/dWXNG3WjMaNm0QdSqnt3nkPatSsGXUYJZOVnfhWDDObbmYdwwrY7gQJ71VgIPCBmbUFPgj3AQ4D2oZbP+ChYsPdkveYoIJkV7D1MbOlwMXAk5JOAmqb2SNmNpCwpmlmfcPXtwUeNLMOwA7h/p4EtardJR0QHtcGuAtoF26nAPsBAwhqoxDU9D40sz2BrsCdkjZ8coW10meAgmv3AL4xsz8KeV/rgTtizg2ApHrAUKCHmXUCxgP9JVUCHgYOM7PdgdiE+wOwv5ntRvCt5xYzWxc+Hhn+fWz4dhT+/U0GDgyLjgTeNbNcYDhwSXiNAcCDhcSeEid3asyoyfNJpGpfq3IFOjWtwZifFxV/cDn0+7xZ/PDtZIZeeibXD+jHL9O/A2Cv/buzTaXKXHDyYVxy6lEceUJfqtVIsw+v0Lx5c/lh2jR23mXXqENJqnfffotDDz8i6jC2XspKfCuZ7sAvZjYL6AWMCMtHAMeEj3sBT1ngK6CWpEZFnbTMm1XN7H1JJwIPAEX975sVvgmAnuE2KdyvRpAsZwMzzGwqgKTvCL41mKSpBFXrgtcfLWlAuF8J2C7ueo8DrwH/Bs4GnigitueAIZJaxpR1AdoDnwc1eCoCXxIk7F/NbEZ43PME31wAagIjJLUFDMgp4poFRhI06Y4BTgIeDGuo+wAvamPTxTYJnKvUdm1cneVr8pi1eA07NCi+pnTKbo158ZvfsTKILRXy8/NZsXwZN977BL9M/557bx7MvSP+yy/TvyMrK4sHn3ublSuWcf2V57HTbnvSsFHTqEMukVUrV3Ll5Zdy1cDBVKtWLepwkiY3dx0ff/Qhl1zev/iDXWqUoFlVUj82fk4CDDez4Zs5/CSCz1WAhmY2P3z8O9AwfNwEmBPzmrlh2Xw2o8x7q4ZNgjsSVINrEwRZmJWxLwNuNbOH487VAlgbU7Q+Zn89G9+fgOPNbHrc6wv+4jCzOZIWSOpGUEPty2aYWZ6ku4DYBn8B75vZyXHX6Li58wA3AmPM7NjwvXxUxLEFRgO3SKpD0JzwIVAVWJLIPd7Yf3R7nzuMHbqfkMAlN69tvSp0bFKDXRpXJydLVMrJpl+Xpgz/qvBfa4s6lblgn+B7SbWK2ezSqDr5BpPmlc9OOfHq1GvAnvt2RRJt2nVAWWL50iV8PuYddu28DxUqVKBmrTps335Xfv1xWlolx9zcXPpffimHH3EUPQ7uGXU4SfXZp5/Sbsf21K1XL+pQtl4lqBGGiXBzyXDjKYPbUEcDgwo5h0na4u/hUXQfugKYRtD8+YSkgtpSbszjeO8CZ8fcw2siqUEJrvkucEl4UxZJu23muEcJmldfNLP8zRxT4EmC5teCZtKvgH0ltQmvUVXS9sB0oFWY/GDTjjw1gXnh4zNjypcD1Qu7qJmtAMYR3Fx+w8zyzWwZMCOskaNAobVyMxtuZp3NrHNpEyPAS1MWcOXoH7jq9ek89OUcpi1YsdnECHD1G9O56vVgGz93GU+Pn5c2iRGg8z4H8f034wGYP3cWebm5VK9Zi3r1t+W7yeMAWLNmNT//8C2Nm7WIMNKSMTOuGzaEVq1acfqZ6dPLNlHvvPWmN6lGLTXNqocBE82s4Ab5goLm0vDnwrB8HtAs5nVN2fjZW6iyvOd4W9iJ5VzgSjP7FPiE4D4dBN8Spkh6Nv5EZvYeQVPml2Fz6UtsJnlsxo0ETZZTwqbXGzdz3GiCJtuimlQLYloH3Ac0CPf/IEhwz0uaQtikamargQuBdyRNIEh8S8PT3AHcKmkSm9bixwDtC+7VFnL5kcCpbNpbqy9wjqRvgO8I2tgj06NtXe46uh21K+dww2FtOWuP9OsEcd+tQxh2xdnMnzuLi/oewZh3XqPrIUez4Pd5XNWvD/fdOoQLrroOSfQ8+kTWrFnNgPN6M/SSMziw51E0b9U26reQsEkTJ/DG6Nf4+uuv6H1cL3of14tPP/k46rCSYvWqVYz98nO69cic2vA1A/pz+iknMWvmDA7udgCvvPxi1CEVL4kdcmKczMYmVQg+wwvG6ZxBcKusoPz0sOLQBVga0/xaKJml692f5JPUGbjHzPZP8nmrmdmKsOb6APCTmd2TzGtsibNemJpxv/xLujSPOoSUaN+0RvEHpZn1GfrZk5Vmg90TVakCpXpjlY8ZnvAvfPV/+xV7rbBT5WygVdhZEUl1gVEEfUpmEQzlWBR+9t4PHEpwS+8sMxtf1Pl9hpyQpIHABRRxr7EUzpN0BkEnnUkEvVedc27rkeRJAMxsJVA3ruwvgt6r8ccacFFJzu/JMRTOPpOSGWjCWmLkNUXnnItMmtWoPTk655xLOXlydM455zblydE555yLoyxPjs4559wmvObonHPOxfHk6JxzzsXx5Oicc87FS6/c6MnROedc6nnN0TnnnIuTlRXFOhdbzpOjc865lPOao3POORcvvXKjJ0fnnHOp5zVH55xzLo4nR+eccy6OTx/nnHPOxfGao0sbNx6yQ9QhJF3bbv2jDiElfhlzd9QhJN3qdflRh5ASjWtXjjqEcsmTo3POORfHk6NzzjkXJ92SY3pNWeCccy49qQRbIqeTakl6SdIPkqZJ2ltSHUnvS/op/Fk7PFaS7pP0s6QpkjoVd35Pjs4551IuKysr4S1B9wLvmFk7YFdgGjAQ+MDM2gIfhPsAhwFtw60f8FCx8Zbs7TnnnHMlJynhLYFz1QQOAB4DMLN1ZrYE6AWMCA8bARwTPu4FPGWBr4BakhoVdQ1Pjs4551KvBM2qkvpJGh+z9Ys7W0vgD+AJSZMkPSqpKtDQzOaHx/wONAwfNwHmxLx+bli2Wd4hxznnXMqVpEOOmQ0HhhdxSAWgE3CJmY2VdC8bm1ALzmGSbEtiBa85OuecKwPJbFYlqPnNNbOx4f5LBMlyQUFzafhzYfj8PKBZzOubhmWb5cnROedcyiUzOZrZ78AcSQUzmXQHvgdGA2eEZWcAr4WPRwOnh71WuwBLY5pfC+XNqs4551IuBXOrXgI8K6ki8CtwFkGFb5Skc4BZQO/w2LeAw4GfgVXhsUXy5Oiccy7lkj0JgJlNBjoX8lT3Qo414KKSnN+To3POuZRLtxlyPDk655xLuTTLjZ4cnXPOpZ7XHJ1zzrk4Wb7YsXPOObepNKs4enIsqXDGhbvN7MpwfwBQzcyuS/J1BpvZLTH7X5jZPsm8RirMmTWDG4detWF//ry5nNnvInbttAf/vv1G1q1bS3Z2NpddNZR2HXaOMNKitW3egKdvP3vDfssmdbnxoTfZa5eWtG0RzEhVq3pllixfTZeTbuOkwzpz+Rk9Nhy/c9vG7H3y7Uz5schxxpGaPWsGNwzZ9Hd1Vr+LqNegAU8+8hCzZ/7KQ088zw47dogwysTcc+u1fP3FJ9SqXYeHnnoZgOXLlnLrtVez8PffaLBtYwbdcCfVq9dg+fJl/PvWa5k/by4Vt6nI5QOvp0WrNhG/g5JZtmwZN1w7lJ9//hEhrrvxFnbtuFvUYRUp3WqOCnq4ukRJWgPMB/Ywsz9TmBxXmFm1ZJ4z3tzF61L6y8/Pz6fPUd154LHnuOvW6zj+pNPYa5/9GfvFJ4x8+gnufuiJpF+zbbf+ST9nVpb45d2bOfD0O5k9f/GG8tv6H8vSFau5dfg7mxzfoU1jRt19Hh2Ovj5pMfwy5u6knasw+fn5nHhkdx58/DnWrlmDssTdt93ABZcOSFlyXL0uP2nnmjp5ApUrV+Gum4duSI6PPXgP1WvUpPepZzPqmcdZsXwZZ19wOY89cDeVqlSh71nnM2fWDB68+1ZuvbeomcpKpnHtykk71+YMHXwNnTp15rgTTiQ3dx2rV6+hRo0aKb1m5ZxEF5MqXIch7yX8efPdzT0jz6Q+Q07J5RHM+XdF/BOS6kt6WdK4cNs3pvx9Sd+FE+TOklQvfO6/kiaEz/ULy24DKkuaLOnZsGxF+PMFSUfEXPNJSSdIypZ0Z3jdKZL+kfK/iWJMGj+Wxk2a0bBRYySxauVKAFauWEHd+vUjji5xXffcgRlz/9gkMQIcf3AnRr0z4W/H9z50d158d2JZhZcUE8eNpXHTZmzbqDHNW7Ziu+Ytow6pRHbuuDvV45LDV599RI9DjwKgx6FH8eWnYwCYPfNXdu20JwDNmrdkwe+/sXjRX2UbcCksX76ciRPGcezxJwCQk1Mx5YkxGZI8fVzKeXLcMg8AfcNlU2LdC9xjZnsAxwOPhuXXAh+aWQeCOQC3i3nN2Wa2O8Fg1ksl1TWzgcBqM+toZn3jrjGScNaHcGaI7sCbwDkEUyLtAewBnCcp0k+4Me+/TbeehwFw4eXXMPz+uzjp6B785//u4twLLo8ytBI58ZDd/5YE9+3UmgWLlvPL7D/+dvwJPTsx6p3xZRVeUnz4/tt0D39XmWLJ4r+oUy/4Ela7bj2WLA4SYMs22/PFxx8AMP37qSxcMJ8//1gQWZwlNW/eXGrXrsOwoYPoc8IxXD9sCKtXrYo6rGJJiW/lgSfHLWBmy4CngEvjnuoB3C9pMsFcfjUkVQP2A14IX/sOEFsFuVTSN8BXBBPjti3m8m8DXSVtQ7CA5ydmthroSTB34GRgLFA3gXOlTG5uLl98+hEHdOsJwOuvjOSCy67mhdH/48LLruJfNw+LKrQSyamQzREH7swr70/apLz3oZ15sZAEuMdOzVm1Jpfvfyly2sZypeB3dWD4u8pEklDYKtj71LNZsWI5F5/Vm9Evv0DrtjuUZIHdyOXn5fHDtO/p3edkRr70XypVrszjjyWvWThVUrDYcUqVjyjS078JamtVY8qygC5hja+jmTUxsxWbO4GkgwgS6t5mtiswCahU1EXNbA3wEXAI0IegJgnBSmiXxFy7pZm9V8g1N6yT9uyTj8Y/nTRff/kpbXfYkTp16wHw3luj2b9r0GHlwO6H8MP336bs2sl0yH7tmfzDHBYuWr6hLDs7i17dduWlQppOg1pmetUax37xKdvH/K4yRa3adVn0Z1CzX/TnH9SsXQeAKlWr0X/wDdz/xCgGDL2JpUsW06hx0yhDLZGG225Lg4bbsvMuuwJwcM9Dmfb99xFHVTyvOW4lzGwRMIogQRZ4j2AyXAAkdQwffs7GptCeQO2wvCaw2MxWSWoHdIk5V66knM1cfiTBxLn7AwW9Qd4FLih4jaTtw8U/4+Mebmadzaxz3zPPTfTtltiH721sUgWoW68+30wMksak8WNp0my7zb20XOl9aOe/Nal222sHfpy5gHkLl2xSLonje3bixXf/fh+yPIv/XWWKLvseyP/eeR2A/73zOl32OwiAFcuXkZubC8C7r7/CTrvuTpWqKe37llT16tVn2223ZeaMXwEY+9WXtGrdOuKoipdu9xx9KEfp3AVcHLN/KfCApCkEf7efAOcD1wPPSzoN+JJgherlBIntfEnTgOkETasFhgNTJE0s5L7je8DTwGtmti4sexRoAUxU8K/rD+CYJL3PElm9ehUTvv6SKwZubDrtP+g6HrjnNvLz86lYcRv6D7o2itBKpEqlinTbqx0X3/T8JuWF3YME2K9TG+b+vpiZ89Knc0fB76r/oI2/q08/+oD7/nULS5csZtAVF9J6+3bced/DEUZZvNuvG8iUSeNZtnQJpx3Xk1PPvoATTz2bW4ddzXtvvkqDho0ZdMMdQDDc6K6b/4kkmrdszWUDr4s2+C1wzeB/MviaAeTm5tKkWTNuuPHWqEMqVjnJeQnzoRxlILw/mG9meZL2Bh4ys44Rh5XyoRxRSMVQjvIg1UM5opDMoRzlSVkM5YhCaYdy7H7jmIQ/byb8s2vkqdRrjmVjO4I1xrKAdcB5EcfjnHNlKt1qjp4cy4CZ/QSU7+krnHMuhdJthhxPjs4551KuvHS0SZQnR+eccymXZrnRk6NzzrnUS7eao49zdM45l3LJngRA0kxJU8M5qMeHZXXCeax/Cn/WDssl6T5JP4dzT3cq7vyeHJ1zzqVcVpYS3kqgazgjWOdwfyDwgZm1BT4I9yGYarNtuPUDHio23pJE4Zxzzm2JMpohpxcwInw8go0TofQCnrLAV0AtSY2KOpEnR+eccylXkuQYOwd0uPUr5JQGvBcu+VfwfEMzK5j1/3egYfi4CTAn5rVzw7LN8g45zjnnUq4kFUIzG04whWZR9jOzeZIaAO9L+iHuHCZpi2cB85qjc865lEt2s6qZzQt/LgReBfYEFhQ0l4Y/F4aHzyNYErBA07Bsszw5OuecS7lk9laVVFVS9YLHBOvZfkuwju4Z4WFnAK+Fj0cTrHcrSV0IFoYvctFVb1Z1zjmXckmePq4h8GpYy6wAPGdm70gaRzCP9TnALMKlAoG3gMOBn4FVBEv+FcmTo3POuZTLSuIkAGb2K7BrIeV/Ad0LKTfgopJcw5Ojc865lEuzCXI8OTrnnEu9dJs+zpOjc865lEuzFas8OTrnnEs9X8/RpY161StGHULS/frR3VGHkBJnPDMx6hCS7sm+xc79nJbSrPWwzIj0+ospMjlK6l/U82aWmZ9EzjnnkirNKo7F1hyrhz93APYgGEgJcBTwdaqCcs45l1kyqkOOmV0PIOkToJOZLQ/3rwPeTHl0zjnnMkKa5caE7zk2BNbF7K9j42znzjnnXJGSOQlAWUg0OT4FfC3p1XD/GDaumeWcc84VKSN7q5rZzZLeBvYPi84ys0mpC8s551wmSbOKY4mGclQBlpnZE5LqS2ppZjNSFZhzzrnMkZHNqpKuBToT9Fp9AsgBngH2TV1ozjnnMkV6pcbEa47HArsBEwHM7LeCtbScc8654mTUUI4Y68zMJBlsWFzSOeecS0ia9cdJODmOkvQwUEvSecDZwKOpC8s551wmydTeqv+SdDCwjOC+4zAzez+lkTnnnMsYGdmsKul2M7sGeL+QMuecc65IaVZxJCvB4w4upOywZAbinHMuc0lKeCsPikyOki6QNBVoJ2lKzDYDmFo2ITrnnEt3KsGW8DmlbEmTJL0R7reUNFbSz5JGSqoYlm8T7v8cPt+iuHMXV3N8jmAFjtfCnwXb7mbWtwTvwTnn3FYsO0sJbyVwGTAtZv924B4zawMsBs4Jy88BFofl94THFam4VTmWAksl3QssilmVo4akvcxsbEnexdYgHO5yt5ldGe4PAKqZ2XVbcK5awClm9uAWvHYm0NnM/izpa5Nl7dq1nHV6X3LXrSMvP5+Dex7ChRdfGlU4pbJ8+TLuvPk6ZvzyE5K4ZugNVKxUibtvu5F1a9eSnZ3NFdcMZccOO0cdakKyBPefuDN/rlzHsDenc9ex7alSMRuAWpVzmL5gBde9/SNVKmYzsEcb6levSHaWeGnSfN774Y+Ioy/a7FkzuGHIVRv258+by1n9LqLn4Udzw9AB/P7bb2zbuDHX3vwvqteoGWGkW+7zTz/h9ttuZn3+eo49/kTOOa9f1CEVK9nNpZKaAkcANwP9FVygG3BKeMgI4DrgIaBX+BjgJeB+STIz29z5Ex3K8RAQu2z3ikLKXGAtcJykW5OQmGoBFwJ/S46SKphZXinPn1IVK1bk0cdHUKVqVXJzcznztFPYb/8D2GXXjlGHVmL333U7e3bZlxtuu5vc3FzWrFnN9YMHcOa557PXPvvz1eef8J//u5t7//NE1KEm5NhdtmX24tUbEuKVr36/4bl/HtqWL2csBuDonRsya/Fqhr01nZqVKvBY3458+OOf5K3f7GdK5LZr3pJHn3kJgPz8fE48sjv7HdSd5556jE6d9+KUM87luRGP8txTj/GPi4tcz71cys/P55abb+DhR56gYcOGnNLnBA7q2o3WbdpEHVqRSpIbJfUDYjP+cDMbHnfYv4Gr2bjucF1gSczn4lygSfi4CTAHwMzyJC0Nj9/sZ3SiHXI2ybBmtp6Szcu6NckDhgNXxD8Rzkn7sqRx4bZvWH5dWMMsOO7bsE38NqC1pMmS7pR0kKRPJY0Gvg+P/a+kCZK+C/9BlRuSqFI1mC8iLy+PvLy89Jt9GFixYjnfTJrAEb2OAyAnJ4fq1WsgxMqVKwFYuWIF9erVjzLMhNWrWpE9W9Tmne8X/u25KjnZdGxSky9+DZIjBlVygo+JyjnZLF+bR345TozxJo4bS+Omzdi2UWO++GQMhxzRC4BDjujF5x+PiTi6LfPt1Ck0a9acps2akVOxIocefgQfjfkg6rCKlSUlvJnZcDPrHLNtkhglHQksNLMJqYo30QT3q6RLCWqLENRmfk1NSBnhAWCKpDviyu8laA//TNJ2wLvAjkWcZyCwk5l1BJB0EEFtfaeYSd/PNrNFkioD4yS9bGZ/Je+tlE5+fj4nn3gcs2fPps/Jp7DLLrtGHVKJzf9tHrVq1+a2G4byy08/sn279lxy5TVc3P8arrr0Hzx0778wM+5/9OmoQ03IBfs159EvZlM5J/tvz+3TqjaT5y5lVW4+AK9N/Z3rD9+B58/sRJWK2dz87k+kT2qED99/m+49g471ixb9Rd3wC0yduvVYtKjc/DcpkYULFrBto2037Ddo2JCpU6ZEGFFikvy9eF/gaEmHA5WAGgSfr7ViWtWaAvPC4+cBzYC5kioANYEi/wEkWnM8H9gnvMBcYC82rfK6GGa2jGANzPgbbD0I2ronA6OBGpKqlfD0X8ethnKppG+Arwh++W23LOrUyM7OZtQrr/Hehx/z7dQp/PTTj1GHVGL5efn8OH0avY7vw6PPvEjlypV5bsRjvPbySC664mpefON/XHT5Vdxx07CoQy3WXs1rsWR1Lj/9sbLQ57u2rceYnzZ+ZnTerha//rmKk5+cyAUjp3DxAS2oUkhSLY9yc3P54tOPOLBbz789FwwZKPuYtmbJHMphZoPMrKmZtQBOAj4MO4mOAU4IDzuDoDMpBJ+3Z4SPTwiPL/J7XkLJ0cwWmtlJZtbAzBqa2Slm9vc2GRfr3wQ9pGLnoc0CuphZx3BrYmYrCJpiY38XlYo474ZPtbAm2QPY28x2BSYV81ok9ZM0XtL4xx6Jb8JPnRo1arDHnnvxxWefltk1k6V+g4bUb9CQ9jvtAsCB3Q7mp+nTePfN0RzQtQcAB/U4hB++/zbKMBPSoVF1urSszVOn7cbgQ9rQsUkNrunRGoAalSqwQ8OqjJ21eMPxPdvV57NfFwHw29K1/L5sLc1qF/lPrNwY+8WnbL/DjtSpWw+AOnXq8tefQWeiv/78g9q160YZ3hZr0LAhv8//fcP+wgULaNiwYYQRJSZbSngrhWsIOuf8THBP8bGw/DGgbljen6BVrkjFjXO8Ovz5f5Lui99K8w4ynZktAkaxsSsxwHvAJQU7kjqGD2cSdm6S1AloGZYvZ+PN5sLUJOievEpSO6BLAnFtaMtPdQ+3RYsWsWzZMgDWrFnDV19+QYuWrVJ6zVSoW68eDRpsy+xZQYV9wrixNG/Zmrr16zN54ngguLfVtNl2UYaZkMe/mkPfEZM4/elJ3PLuz0yet4zb//cLAPu3rsPYmUvIzd/4hXrhinXs1jTo0Vmrcg5Na1Vm/rK1kcReUh++9zbdem6cq2Sf/Q/i3TeDisS7b77GPgd0jSq0Uumw087Mnj2TuXPnkLtuHe+89SYHdu0WdVjFylLiW0mY2UdmdmT4+Fcz29PM2pjZiWa2NixfE+63CZ8v9rZgcfccC8aPjC9ZuC50F3BxzP6lwAOSphD83X9C0GT9MnC6pO+AscCPAGb2l6TPJX0LvA28GXf+d4DzJU0DphM0rZYbf/6xkKGDB7J+fT7r1xs9DzmUAw9Kzw+kS68axE3/HEheXi6NGjdl4LAb2feArtx/923k5+VTcZttuHLQtVGHWSoHta3HyInzNil7dtxcruremodP2gUBj305m2VrynUnaQBWr17FhK+/pP+gjU3dJ59xDtcPHsBbo1+lYaNGXHvzXRFGuOUqVKjAoCHDuKDfuaxfn88xxx5Pmzbl6m5KodJt+jgV0+zqMtiavLTqW5GQxSvXRR1CSpzxzMSoQ0i6J/tm5kiwOtUqRh1CSlSqULr1iq98fXrCnzd3HbVD5Km0yJqjpNdh8x+gZnZ00iNyzjmXcdKt5lhcs+q/wp/HAdsCz4T7JwMLUhWUc865zJJuvYOLmz7uYwBJd5lZ55inXpfk9yGdc84lpEKaZcdExzlWlbShm6Gklmw6RME555zbLCnxrTxIdIacK4CPJP1KsKJIc+AfKYvKOedcRskqL1kvQQklRzN7R1JboF1Y9EPB+BHnnHOuOGmWGxNLjpKqEMwq0NzMzpPUVtIOZvZGasNzzjmXCdKtt2qi9xyfANYBe4f784CbUhKRc865jJOixY5TJtHk2NrM7gByAcxsFZRuQKhzzrmtR6qmj0uVRDvkrAuXRDIASa0JFvV1zjnniqU0q08lmhyvJZjHs5mkZwnW0jozVUE555zLLOWlRpioYpOjpCygNsEsOV0ImlMvM7M/Uxybc865DJFxydHM1ku62sxG8fdVIZxzzrliJbKIcXmSaLPq/yQNAEYSs9huuGahc845V6TsRLt/lhOJJsc+BJ1xLowrT7+Va51zzpW5jJwhB2hPkBj3I0iSnwL/SVVQzjnnMkvG3XMMjQCWAfeF+6eEZb1TEZRzzrnMkmYVx4ST405m1j5mf4yk71MRkCs7efkJL8ydNtbmro86hJR4+rTdow4h6Vocc0fUIaTE3NeviTqElKhUIdF0UbisJI5zlFQJ+ATYhiCPvWRm14YrRr0A1AUmAKeZ2TpJ2wBPAbsDfwF9zGxm0fEmZqKkLjGB7QX4eo7OOecSkuQlq9YC3cxsV6AjcGiYo24H7jGzNsBi4Jzw+HOAxWH5PeFxRUo0Oe4OfCFppqSZwJfAHpKmSpqS4Dmcc85tpSpkKeGtOBZYEe7mhJsB3YCXwvIRwDHh417hPuHz3VXM2JJE68mHJnicc8459zcluecoqR/QL6ZouJkNjzsmm6DptA3wAPALsMTM8sJD5gJNwsdNgDkAZpYnaSlB0+tmJ7NJdD3HWYkc55xzzhWmJEM5wkQ4vJhj8oGOkmoBr7JxveGkSLNhmc4559JRku85bmBmS4AxBEsq1pJUUOlrSrC8IuHPZkEcqgDUJOiYs1meHJ1zzqVcVgm24kiqH9YYCVeMOhiYRpAkTwgPOwN4LXw8OtwnfP5DMyuyu37p+uY655xzCUjyDDmNgBHhfccsYJSZvREOMXxB0k3AJOCx8PjHgKcl/QwsAk4q7gKeHJ1zzqVcMpOjmU0Bdiuk/Fdgz0LK1wAnluQanhydc86lXJpNkOPJ0TnnXOpl6vRxzjnn3BbL1PUcnXPOuS2WbkMjPDk655xLuUxdz9E555zbYt6s6pxzzsXxZlXnnHMujtcct1KS8oGpBH+n04AzzGxVCV7fGLjPzE6Q1BFobGZvhc8dDbQ3s9uSH3lyXT9sMJ9+/BF16tRl1KuvAzD9h2nccuN1rFu3luzsbAYOuZaddt4l4khL5tVRz/L26JcxMw47+niO63MqAK+9+ByjXxlJVlYWe+1zAOdedEXEkZbM8uXLuOOma5nxy88gGPjPG9mueUuuG3wl8+f/RqNGjbn+1ruoXqNm1KEWqW2zOjz9z+M27LdsVIsbn/yYxvWqc/jebVmXm8+M+Uvod/vrLF25lgrZWTw04Ag6tt2WCtlZPPveVP71/BcRvoPE5Ofnc/apvalfvyH/uu9Bbrp2MJMmjKdatWoADLn+ZrbfYceIoyxceqVGT47JtNrMOgJIehY4H7g70Reb2W9snBOwI9AZeCt8bjTB3IDl3lFHH0vvk/py7ZCBG8ruvedO+p1/EfvufwCfffox991zJ8MffzrCKEtm5q8/8fbol7nv0WfJqZDD4CsvZK99D+CPBb/zxWcf8dCIF6lYsSJLFhc5j3G5dN9dt7HX3vty4+33kJuby5o1q3nmiUfotEcXTj3zXJ558lGeGfEYF1zSP+pQi/TTnEV06fcoAFlZ4pdRlzL6s+m0bVaXfz4yhvz1xk3ndeWqU/Zh6CNjOP7AHdkmJ5s9zn2EyttUYNIT/2DUh98xe8HSiN9J0UY9/zQtWrZi5YqVG8ouuvxKuvU4JMKoEpNmFce0awZOF58CbSTVkfRfSVMkfSVpFwBJB0qaHG6TJFWX1ELSt5IqAjcAfcLn+0g6U9L9kmpKmiUpKzxPVUlzJOVIai3pHUkTJH0qKanLtySqU+c9qFlz01qGJFauDNYlXbF8OfXqN4gitC02e+YM2nXYmUqVKpNdoQK7dNydzz/+gDf++yJ9Tj2bihUrAlCrdt2IIy2ZFSuW882kCRzR63gAcnJyqF69Bp99PIZDj+wFwKFH9uKzjz6MMswS69qpBTN+W8zsBcv4YPwM8tcH80t/Pe03mtSvAYBhVKlckewsUXmbHNbl5rN81doowy7WwgW/88Wnn3DUMcdHHcoWyZYS3soDT45JFi6HchhBE+v1wCQz2wUYDDwVHjYAuCisae4PrC54vZmtA4YBI82so5mNjHluKTAZODAsOhJ418xyCdY+u8TMdg/P/2Cq3mNJDbh6MP+++04OP/gg/n33HVxyWfmuhcRr0aoN334zkWVLl7BmzWrGffkZfyz4nXmzZ/HtNxO59Ly+DLjobKZP+zbqUEtk/rx51KpVm1uvH8o5fU/g9puGsXr1KhYv+ot69eoDULduPRYvSq8a8YldOzDqw+//Vn76Ybvy7te/APDKxz+wavU6Zrx0GT8+fzH/HjWWxcvXlHWoJfLvf93GRZddSVbWph/bwx+4j9N6H8u9/7qNdevWRRRd8VSCP+WBJ8fkqSxpMjAemE0wC/x+wNMAZvYhUFdSDeBz4G5JlwK1YlauTsRIoE/4+CRgpKRqwD7Ai2EMDxPMWl8uvDjqea68aiBvvf8R/a8axA3XDo06pBLZrkUrevc9i0FXnM+Q/hfSqu0OZGVlk5+fx/JlS7l3+DOce9EV3PzPqyhmFZxyJT8/j5+mT+OYE/rw2LMvUalSZZ598rFNjtGWLLAXoZwKWRyxT1te+XjaJuVX992X/Pz1vPC/4AvMHu0ak7/eaHXifezY9wEu670XLRrViiDixHz+yUfUrlOHdu07bFJ+/sVX8Pwrb/DYMyNZtmwpzzz5aEQRFi9V6zmmiifH5Fkd1vQ6mtklYQ2wUGHHmnOBysDnJWwCHQ0cKqkOsDvwIcHvcUnM9TuaWaF35SX1kzRe0vjHHy1yoe2keWP0f+nWoycAB/c8lO++nVIm102mQ486jgcef4G7HnyCatVr0HS75tRr0JB9D+yOJNq135ksZbF0yeKoQ01Y/QbbUr9BQ9rvFHSOOqh7T36c/j2169Tlzz//AODPP/+gdu06UYZZIofs2YbJP/3OwsUb78mdesguHN6lDWfe/N8NZb27d+C9cb+Ql7+eP5as4stv57L79uXm++TfTPlmEp99/BHHHXEwwwYNYML4sVw35Brq1a+PJCpWrMgRRx/L99+W39aLLJTwVh54ckytT4G+AJIOAv40s2WSWpvZVDO7HRgHxCfH5UD1wk5oZivC19wLvGFm+Wa2DJgh6cTwWpK062ZeP9zMOptZ57PP7Vf6d5iA+vUbMGH81wCMG/sVzbZrXibXTaaCzjYLf5/P5x9/QNeDD2Of/bvyzcRxAMydPZPcvFxq1qodZZglUrdePRo03JbZM2cAMGHcV7Ro2Zp9DziId94I1oh9543X2O/ArlGGWSK9u7Vn1Iffbdg/eI9W9O/ThROGvsjqtRsbaOYuXMZBu7UAoEqlHPbcsTHT55Tf5uMLLrmC1975kFfefJ8bbv0Xu3fei+tuvp0//wi+xJgZn4z5gFZt2kQc6ealW83Re6um1nXA45KmAKvYuBL15ZK6AuuB74C32bQZdAwwMGwivbWQ844EXgQOiinrCzwkaSiQA7wAfJOsN5KowVf3Z/z4cSxZspjDehzIPy68hKHX3si/br+Z/Px8KlbchqHX3lDWYZXaDYOvZPmypWRXqMDFVw6mWvUaHHLksdx9yzD6nXocOTk5XDX0xrQby3XZgMHcOOwacnNzadykGYOG3cj69ca1g67kzdGvsO22wVCOdFClUg7ddm/Jxfe8vaHsnksPYZucCrxx5ykAfP39PC7999v857/jGX7NUUx4vB8Cnn53Ct/+ujCiyLfcdUOuZsmSxZgZbbdvx9VDhkUd0mal2/RxSqd7JC65VqzNvF/+n8vLd4/DLVW5YnbUISRdi2PuiDqElJj7+jVRh5ASdatWKFV2++CHPxP+vOnerl7kmdRrjs4551KuvPRCTZQnR+eccymXZq2q3iHHOedc6iVznKOkZpLGSPpe0neSLgvL60h6X9JP4c/aYbkk3Sfp53BSlk7FXcOTo3POuZTLUuJbAvKAK82sPdAFuEhSe2Ag8IGZtQU+CPchmJilbbj1Ax4qNt4Sv0PnnHOuhLKkhLfimNl8M5sYPl5OsNhDE6AXMCI8bARwTPi4F/CUBb4CakkqcmCrJ0fnnHMpp5JsMZOVhNtmB2VLagHsBowFGprZ/PCp34GG4eMmwJyYl80NyzbLO+Q455xLuZKMczSz4QTzRRcpnDrzZeDycIKV2HOYpC0eruY1R+eccylXkppjQueTcggS47Nm9kpYvKCguTT8WTCzwzygWczLm4Zlm+XJ0TnnXOolMTsqqCI+Bkwzs9h1c0ezcSayM4DXYspPD3utdgGWxjS/FsqbVZ1zzqVckqeP2xc4DZgaTrMJwbKAtwGjJJ0DzAJ6h8+9BRwO/EwwledZxV3Ak6NzzrmUS2ZqNLPPijhl90KON+CiklzDk6NzzrnUS7MZcjw5OuecSzmfW9U555yLk25zq3pydM45l3Jplhs9OTrnnEu9dFsI3JOjc865lEuz3OjJ0TnnXOqlWW5EwfAPtzVak0fG/fLz8jPuLQGQlYFzWU2btzzqEFLi4XFzij8oDQ0/sUOp8ts3c5Yn/J9z12bVI8+lXnN0zjmXcj6UwznnnIvj9xydc865OJ4cnXPOuTjerOqcc87F8Zqjc845FyfNcqMnR+ecc2UgzbKjJ0fnnHMpl+TFjlPOk6NzzrmUS6/U6MnROedcWUiz7OjJ0TnnXMql21CODJyx0TnnXHkjJb4Vfy49LmmhpG9jyupIel/ST+HP2mG5JN0n6WdJUyR1SiReT47OOedSTiXYEvAkcGhc2UDgAzNrC3wQ7gMcBrQNt37AQ4lcwJOjc865lJOU8FYcM/sEWBRX3AsYET4eARwTU/6UBb4CaklqVNw1PDk655xLuZI0q0rqJ2l8zNYvgUs0NLP54ePfgYbh4yZA7Dpic8OyInmHHOeccylXku44ZjYcGL6l1zIzk1SqxV0jSY6S8oGpMUUvmNltRRw/2MxuSX1km1zzIGCAmR2Z4PFnAncC84CKwD1m9kgJr3k+sMrMngrP956Z/RY+9yhwt5l9X5JzRun3+fMZMuhqFv31F0iccGJv+p52RtRhbZHrhw3m048/ok6duox69fUN5S889zSjXniO7Oxs9tv/QC7rf1WEUZbOM089yasvv4Qk2rRty/U33co222wTdVgJefiuG5g09jNq1KrNHcNHAnDfzYOYP3cWACtXrqBq1Wrc+tBzLF+2hHtvHMgvP37PAQcfyVkXXx1l6MUSMKRHK5aszuP+z2dTt0oO/bo0peo22cxavIbHx84j34webeuyX6tarF8Py9fmMWL8byxalRt1+BulvrPqAkmNzGx+2Gy6MCyfBzSLOa5pWFakqGqOq82sYwmOHwz8LTkqaJyWma1PVmClNNLMLpbUAPhO0mgzW5Doi83sPzG7ZwLfAr+Fz52b1EjLQHaFbAZcPZAd23dg5coVnHTi8XTZe19at2kTdWgldtTRx9L7pL5cO2TghrJxX3/Fx2M+5IWXXqNixYrBl4A0tXDBAp5/9mlefu1NKlWqxNVXXs67b7/J0cccF3VoCTmg55H0PLo3D9157YayS4fcuuHxMw/fQ5Wq1QDIqbgNJ5xxPnNn/sKcmb+Ueawl1b1tXeYvX0vlCtkAHL9LQ/7301+Mm7OMvp0asV/LWnz862LmLFnNLf9bxLp848BWtTl+l4Y88tXciKPfqAyGcowGzgBuC3++FlN+saQXgL2ApTHNr5tVbu45SqopabqkHcL95yWdJ+k2oLKkyZKeldQiPO4pguTRTNJVksaF3XSvD1/fQtIPkp6U9GP42h6SPg+7+u4ZHlc17Bb8taRJknrFxZUVHl8/Zv/ngv3CmNlC4BeguaTu4XmnhtfZJjzPbZK+D2P+V1h2naQBkk4AOgPPhu+7sqSPJHWWdL6kO2PiO1PS/eHjU8P3MVnSw5Kyk/Tr2SL16zdgx/YdAKhatRqtWrVi4cKEvyuUK50670HNmjU3KXtp1Aucec55VKxYEYA6detGEVrS5Ofls3btGvLy8lizejX16zeIOqSE7bhzJ6pVr1Hoc2bGV5/8j727HgJApUqVabdTR3LC31t5VqtyBXZuVI3Pfl2yoaxdg6pMmLsMgC9nLqFjk+B9T/9jFevyg5bEXxetpnblnDKPtyhJHsrxPPAlsIOkuZLOIUiKB0v6CegR7gO8BfwK/Aw8AlyYSLxRJceCZFew9TGzpcDFwJOSTgJqm9kjZjaQsKZpZn3D17cFHjSzDsAO4f6eQEdgd0kHhMe1Ae4C2oXbKcB+wACC2ijAEOBDM9sT6ArcKalqQaBhrfQZoODaPYBvzOyPzb05Sa2AVgQ3fp8E+pjZzgQ19Qsk1QWOBTqY2S7ATbGvN7OXgPFA3/B9r455+uXwtQX6AC9I2jF8vG9YK8+PiTly8+bN5Ydp09h5l12jDiVpZs+ayaQJ4zn9lN6cd9apfPft1OJfVE41aNiQ0888m8N6dOPgrvtTrXp19t53v6jDSoofvp1Ezdp1adRku6hDKbE+Hbfl5SkLMIKkV61iNqty81kf3k1bvDqXWpX/3gC4X8tafPv78rIMtVhZSnwrjpmdbGaNzCzHzJqa2WNm9peZdTeztmbWw8wWhceamV1kZq3NbGczG59QvKV7u1usINkVbCMBzOx9gnuRDwBFNSPOCrvkAvQMt0nARIIk2DZ8boaZTQ0T3HcEY2AsvEaLmNcPlDQZ+AioBMT/L3ocOD18fDbwxGbi6hOe53ngH0D9MIYfw+dHAAcAS4E1wGOSjgNWFfFeNxEm5V8ldQmTbDvgc6A7sDswLoyhO0GCjtyqlSu58vJLuWrgYKpVqxZ1OEmTn5fPsmVLGfHsSC7rfzUDB1xO8M8r/SxbupSPxnzAG+/+j/c+/ITVq1fz5uujow4rKb4Y8x77HNQz6jBKbOdG1Vi+Jp/ZS9aU6HV7bVeT5rUr89708tbMn+SRjilWbppVIWiyBHYkSBa1izh0ZezLgFtjEm0bM3ssfG5tzHHrY/bXs/F+q4DjY16/nZlNi72Ymc0huNnbjaCG+vZm4hoZnmMvM3t1c8GbWV54npeAI4F3inivhXkB6A0cD7waJnwBI2Lexw5mdl38C2O7SD/2yBZ3BktYbm4u/S+/lMOPOIoeB6ffB1RRGjRsSNfuByOJnXbeBWVlsWTx4qjD2iJjv/qSxk2aUqdOHXJycujW/WC+mTwp6rBKLT8/j3Gfj6HLgQdHHUqJtalbhV0bV+eWw9tyXpemtGtQlT4dt6VKTvaG2lXtyjksWZ234TU7NqjK4TvW54HPZ5O3vnx9UUtms2pZKFfJEbgCmEbQ/PmEpIJG89yYx/HeBc6WVA1AUpOwQ0yi3gUuCTv3IGm3zRz3KEHz6otmlp/guacDLSQV9EA5Dfg4jLWmmb1F8J4La2tcDlTfzHlfJRjYejJBooRgRogTCt67gqmUmse/0MyGm1lnM+t8znmJDB3acmbGdcOG0KpVK04/86yUXisKB3XrwfhxXwMwa+YM8nJzqVW7qO905de2jRoxdco3rF69GjPj67Ff0rJVuWh4KJVvJ35N42bNqVu/YfEHlzOvfruQa978kcFv/cQjX83lh4UreezreUxfuJLdmwb3GfduUYvJvwX3H5vVqsSpuzfmgc9ns3xtoh9RZSe96o3R9VatHDb9FXiHoKnyXGBPM1su6RNgKHAtwXiXKZImEtwj3MDM3gvvt30Z5rcVwKkE99wScSPw7/D8WcAMgtpcvNFhjJtrUv0bM1sj6SzgRUkVgHHAf4A6wGuSKhH8W+hfyMufBP4jaTWwd9x5F0uaBrQ3s6/Dsu8lDQXeC99HLnARMCvReJNt0sQJvDH6Ndpuvz29jwv6OV1yeX/2P+DAqELaYoOv7s/48eNYsmQxh/U4kH9ceAm9jj2O64cNofexR1EhJ4frbrotodk9yqOdd9mVHgf35JTex5GdXYF27Xbk+BP7RB1Wwv7v1iFMmzKB5UuXcHHfIzj+tH50PbQXX378HvscdMjfjr/09KNZvXIleXm5TPjyYwbe8n80bZ4eXwZenrqA87o0pddODZizeA2fz1gCwAm7NGSbCln8Y++mACxalcsDn88p4kxlK93+ayhd75GUNUmdCcYu7h91LMmyJo+M++Xn5WfcWwIgq7y18STBtHnlq8NIsjw8rvwkpGQafmKHUqW335flJvyfc9saOZGnUp8hJwGSBgIXUI56fzrnXDqJPNuVUAZ+H00+M7vNzJqb2WdRx+Kcc+ko3TrkeM3ROedcyqXbYseeHJ1zzqVeeuVGT47OOedSL81yoydH55xzqZdVXm4mJsiTo3POuZRLs9zovVWdc865eF5zdM45l3LpVnP05Oiccy7lfCiHc845F8drjs4551wcT47OOedcHG9Wdc455+KkW83Rh3I455xLuWQvdizpUEnTJf0crpyUVJ4cnXPOpV4Ss6OkbOAB4DCgPXCypPbJDNebVZ1zzqVckqeP2xP42cx+BZD0AtAL+D5ZF/DkuBWrVKFs7pBL6mdmw8viWlQouxsbZfq+ylBZva/dW9RI9SU2KMvf1fAWHcriMkB6/RssyeeNpH5Av5ii4XHvswkwJ2Z/LrBX6SLclDerurLQr/hD0pK/r/SRie8JMvR9mdlwM+scs5X5FwBPjs4559LNPKBZzH7TsCxpPDk655xLN+OAtpJaSqoInASMTuYF/J6jKwtpcU9kC/j7Sh+Z+J4gc99XkcwsT9LFwLtANvC4mX2XzGvIzJJ5Pueccy7tebOqc845F8eTo3POORfHk6NzzjkXx5OjSzpJ20v6QNK34f4ukoZGHZdz6U6BUyUNC/e3k7Rn1HFlIk+OLhUeAQYBuQBmNoWgq3VGkNRcUo/wcWVJ1aOOaUtJqlPUFnV8pZGhX9IeBPYGTg73lxPMMeqSzIdyuFSoYmZfa9O5FPOiCiaZJJ1HMCtJHaA1weDj/wDdo4yrFCYARuHTPRvQqmzDSapHgKuAhyH4kibpOeCmSKMqnb3MrJOkSQBmtjgc5+eSzJOjS4U/JbUm+HBF0gnA/GhDSpqLCCY9HgtgZj9JahBtSFvOzFpGHUMKZeKXtNxwRYqC/1v1gfXRhpSZPDm6VLiIYHByO0nzgBlA32hDSpq1Zrau4ANXUgXCD6p0J6k20BaoVFBmZp9EF1GpZeKXtPuAV4EGkm4GTgDSvam4XPJJAFzSSco2s3xJVYEsM1sedUzJIukOYAlwOnAJcCHwvZkNiTKu0pJ0LnAZQTPxZKAL8KWZdYsyrtKQ1IrgS9o+wGLCL2lmNivSwEpJUjuCZnwBH5jZtIhDykieHF3SSZoNvAOMBD60DPpHJikLOAfoSfDh9C7waLq/R0lTgT2Ar8ysY/gBfIuZHRdxaFssE7+kSboPeMHMvog6lkznvVVdKrQD/kfQvDpD0v2S9os4pmQ5BnjKzE40sxPM7JF0T4yhNWa2BkDSNmb2A7BDxDGV1gxJwwlqwSuiDiZJJgBDJf0i6V+SOkcdUKby5OiSzsxWmdmosNaxG1AD+DjisJLlKOBHSU9LOjK855gJ5kqqBfwXeF/Sa0BaNz+SgV/SzGyEmR1OUMufDtwu6aeIw8pI3qzqUkLSgUAf4FBgPDDSzF6ONqrkkJQDHEbw/vYD3jezc6ONKnnC311N4B0zWxd1PMkQdja6l+CeY3bU8ZRWOPC/D9ALmGZmR0UcUsbx5OiSTtJMYBIwChhtZiujjSj5wgR5KHAWcICZ1Ys4pC0WDg34zszaRR1LsmXal7SwQ9ixwC8E9/RfNbMlkQaVoTKlSciVL7uY2bKog0gFSQU1xoOAj4BHgd4RhlRqYaeV6ZK2M7PZUceTLHFf0q7KkC9pvwB7m9mfUQeS6bzm6JJG0tVmdoek/6OQsX9mdmkEYSWVpOcJvrG/bWZro44nWSR9QnB/+GtgQxIxs6MjC6qUJNXIlC9pktqZ2Q+SOhX2vJlNLOuYMp3XHF0yFYy3Gh9pFClkZicXf1Ra+mfUASRLwZc04GZJmfIlrT/BtIV3FfKcAWk7HrW88uToksbMXg8frjKzF2Ofk3RiBCEljaTPzGw/ScvZtFYswMysRkShJcvhZnZNbIGk20nPXsYZ9yXNzPqFDw8rGHJTQFKlQl7iSsmbVV3SSZpoZp2KK3Plx2Z+Z1PMbJeoYiotSScW9iUtviyd+P+tsuM1R5c0YWeVw4Em4UweBWqQ/hM+AyDpaTM7rbiydCHpAoIp8FpLmhLzVHUg3WdhGQTEJ8LCyso9SdsCTYDKknZj4yoqNYAqkQWWwTw5umT6jaAp62iCmTwKLAeuiCSi5OsQuxNOArB7RLEkw3PA28CtwMCY8uVmtiiakEonQ7+kHQKcSTD37d0x5cuBwVEElOm8WdUlnaQcM8uNOo5kkjSI4EOoMrCqoBhYBww3s0FRxZYMkrYrrDwdh3ZI2hXoCNwADIt5ajkwxswWRxFXMkg6Pp3HaaYTT44u6SS1JaiJtGfT5Y/SeeFcACTdmu6JsDDhxOMFix5XAloC082sQ5EvLMckVTCzdK0pbkLSqWb2jKQrKXyY1N2FvMyVgjerulR4ArgWuAfoSjCLTEbM42tmgzJw3UPMbOfY/XA83YURhVMqkkaZWW9gUtxQjoKexenYyahq+LNapFFsRbzm6JJO0gQz213S1IIP3YKyqGMrrUxc93BzYn9/6URSIzObL6l5Yc+n+3qOrmx4zdGlwtpw3cOfJF0MzCNzvvFexsZ1D7sWrHsYcUylJql/zG4W0Imgg1XaMbP54cM/gdVmtl7S9gSrdLwdXWSlF86tehOwmmDN1F2AK8zsmUgDy0AZ0dTlyp3LCLqXX0rQk/M04IxII0qeTFz3EIKhGwXbNsCbBCs+pLNPgEqSmgDvEfw7fDLSiEqvZzgl3pHATKANcFWkEWUorzm6pDOzceHDFQT3GzNJ/LqHi0n/dQ8xs+sBJFUxs1XFHZ8mZGarJJ0DPBjO+zs56qBKqeAz+wjgRTNbKqmo490W8uTokk7S6/y9R91SgjGQD8dPf5VOzOzY8OF1ksYQrnsYYUhJIWlv4DGC5u/twuEQ/zCztOyUE1L4vvoC54Rl6b6W4xuSfiBoVr1AUn0gbf8/lWfeIcclnaR7gfrA82FRH2AZQcKska6zyQBIqlNI8fJ0H9cpaSxwAsH6m7uFZd+a2U7RRrblwrUcrwQ+N7PbJbUCLk/Ticc3CP8NLg2XGqtC8H/q96jjyjSeHF3SSRpnZnsUVibpuzQfOzcTaAYsJhgaUAv4HVgAnGdmEzb74nJM0lgz20vSpJjk+I2Z7Rp1bKUlqRqAma2IOpbSChfZvgA4ICz6GPhPun85K4+8Q45LhWqxM66Ejwt6q66LJqSkeZ9gBYt6ZlYXOAx4g2BM4IORRlY6cyTtA5ikHEkD2Li6RVqStLOkScB3wPeSJkhK2y9moYcIOrk9GG6dwjKXZF5zdEkn6XDgPwSrlotgtpULgY8Ialf/jiy4Uips7F/B6hWSJptZx4hCKxVJ9YB7gR4Ev7P3gMvM7K9IAysFSV8AQ8xsTLh/EHCLme0TZVylUVhtPlNq+OWNd8hxSWdmb4VTyLULi6bHdML5dzRRJc18SdcAL4T7fYAFkrKB9dGFVTpm9idBx5VMUrUgMQKY2UeSqhb1gjSQL6m1mf0CEN5HzY84pozkydElXdhJoD/Q3MzOk9RW0g5m9kbUsSXBKQRT4/2XoIPR52FZNtA7urC2jKRhRTxtZnZjmQWTfL9K+ifwdLh/KvBrhPEkw1XAGEm/EtTwm5N5w6XKBW9WdUknaSTBklWnm9lOYbL8Il2bHAsjqaqZrYw6jtIKJ7KOV5Vg6ENdM0vbmY3COXCvB/Yj+CLzKXB9uq7KEQ7baA7MBRqExdPNbG10UWUuT44u6SSNN7POGdrzcR/gUaCamWXKeEAAJFUnmN3oHGAUcJeZLYw2qpKTVAk4n2D2mKnA4+nemzOc0/cWgvv4LYF+ZjY62qgym/dWdamwTlJlwokAJLUGMuXb7T0EC8/+BWBm37CxW31aklRH0k3AFIJbLZ3M7Jp0TIyhEUBngsR4GHBntOEkxeVABzPbG9gHyLhl08obv+foUuFaglljmkl6FtiXYBXzjGBmc+Km7ErbDhGS7gSOA4YDO2fCWECgfcxqMI8BX0ccTzKsM7M/AMzsV0nbRB1QpvPk6JLOzN6XNJFgOScRDAn4M+KwkmWT8YAEzZDpPB7wSoJa/VBgSEzSL1j7sEZUgZXChiZUM8vLkLlHm0q6b3P76T7rT3nk9xxd0sQO/C+Mmc0uq1hSJRPHA2YaSflAQWcpAZWBVaRxwpdU5Ko2ZjairGLZWnhydEkjaSrBfcbYr+pGMM9qAzNL90mfnXNbCW9WdUlTyMwxLYBrCGpZab0gcIaPB3TOxfHeqi7pwkH/TxKsuj6BoIPE/0UbVamtLGSDYNjDNVEF5ZxLDW9WdUkjaSdgCNABuAN43szStifn5mTKeECXfiTV9fvbZcOTo0uasCPEHOBNChnekO496sJ19PoTzEE6Arg3XWdbcelJ0k/AZOAJ4G3zD/CU8XuOLpnOjjqAVMnQ8YAu/WxPcA//bOA+SaOAJ83sx2jDyjxec3QuAZLWE4wHzCOc+afgKdJ0eIBLb5K6As8QzIX7DTDQzL6MNqrM4cnROefShKS6BKuLnAYsAB4DRgMdgRfNrGV00WUWb1Z1zrn08SXBElzHmNncmPLxkv4TUUwZyWuOLum8R51zqSFJ3gmnbHhydEnnPeqcSy5Jr7Ppve5NmNnRZRjOVsGTo0s6BTM9F/So24NgLKD3qHNuC0k6sKjnzezjsopla+HJ0aWU96hzLrnCtVK3M7PpUceSyXz6OJd0kupKukzSeGAAcAlQj2B5pOciDc65NCbpKIJbFu+E+x0ljY40qAzlvVVdKniPOudS4zpgT+AjADObLMmHb6SAJ0eXCjtsrhOOmd1e1sE4l0FyzWxp3ALOfm8sBTw5uqSJ7VFX2Orr3qPOuVL7TtIpQLaktsClwBcRx5SRvEOOSxrvUedcakmqQrDyTU+CqQvfBW40szWRBpaBPDm6lPAedc65dOa9VV3SeY8651JDUmdJr0iaKGlKwRZ1XJnIa44u6SRNALoBH5nZbmHZVDPbOdrInEtvkqYDVwFTgfUF5WY2K7KgMpR3yHGp4D3qnEuNP8zMW2HKgCdHlwreo8651LhW0qPABwTriwJgZq9EF1Jm8mZVl3Teo8651JD0DNAO+I6NzapmZmdHF1Vm8uTonHNpQtJ0M9sh6ji2Bt6s6pJOUmdgMNCCmH9jZrZLVDE5lyG+kNTezL6POpBM5zVHl3Teo8651JA0DWgNzCC45yiCZlX/4plkXnN0qeA96pxLjUOjDmBr4TVHl3SSugMn4z3qnEs6SfsBbc3sCUn1gWpmNiPquDKN1xxdKpxF0KMuh5gedYAnR+dKQdK1QGdgB+AJgv9jzwD7RhlXJvLk6FJhD+9R51xKHAvsBkwEMLPfJFWPNqTM5HOrulT4QlL7qINwLgOtC9dKLVgarmrE8WQsrzm6VOgCTJbkPeqcS65Rkh4Gakk6DzgbeCTimDKSd8hxSSepeWHlPpTDuS2nYLLipgT38zfMPmVm70caWIby5OhSwnvUOZd8vrpN2fF7ji7pwh511wCDwqKCHnXOudKZKGmPqIPYGvg9R5cK3qPOudTYC+graRawEr+fnzKeHF0qrDMzk+Q96pxLrkOiDmBr4c2qLhXie9T9D+9R51wy3GRms2I34Kaog8pEXnN0SRX2qBtJ0KNuGcFMHsO8R51zSdEhdkdSNrB7RLFkNE+OLqnC5tS3wh51nhCdSwJJgwiWgassaVlBMbAOb5VJCR/K4ZJO0gjgfjMbF3UszmUSSbea2aDij3Sl5fccXSrsBXwp6RdJUyRNlTQl6qCcywA/x+5Iyg6HTrkk82ZVlwreo8651Ogu6XjgHKAuwcocH0cbUmbyZlWXdJKeNrPTiitzzpWcpD7AAwTjHE8xs88jDikjebOqSwXvUedcCkhqC1wGvAzMAk6TVCXaqDKTJ0eXNJIGSVoO7CJpWbgtBxYCoyMOz7lM8DrwTzP7B3Ag8BPgHd9SwJtVXdJ5jzrnUkNSDTNbFle2vZn9GFVMmcprji4VvEedc0kk6WoAM1sm6cS4p88s+4gynydHlwrdJb0lqZGknYCvAJ943Lktd1LM4/hWmUPLMpCthQ/lcElnZqeEPeqm4j3qnEsGbeZxYfsuCbzm6JLOe9Q5l3S2mceF7bsk8A45Lukk/QBcZGYfhBOR9wfONrMOxbzUOVcISflsXL+xMrCq4CmgkpnlRBVbpvLk6JLOe9Q559KdN6u6pPEedc65TOHJ0SWT96hzzmUET44umbxHnXMuI3hydMnkPeqccxnBO+S4pPEedc65TOHJ0TnnnIvjzarOOedcHE+OzjnnXBxPjs4551wcT47ObSUk1ZJ0YdRxOJcOPDk6t/WoBfwtOUry1Xmci+P/KZzbetwGtJY0GcgF1gCLgXaSegJvmNlOAJIGANXM7DpJrYEHgPoEw3POM7MfongDzpUVT47ObT0GAjuZWUdJBwFvhvszJLUo4nXDgfPN7CdJewEPAt1SHaxzUfLk6NzW62szm1HUAZKqAfsALwarjwGwTaoDcy5qnhyd23qtjHmcx6Z9ECqFP7OAJWbWsayCcq488A45zm09lgPVN/PcAqCBpLqStgGOhGD5MWBGwRJkCuxaJtE6FyGvOTq3lTCzvyR9LulbYDVBQix4LlfSDcDXwDwgtsNNX+AhSUOBHOAF4Juyi9y5sudzqzrnnHNxvFnVOeeci+PJ0TnnnIvjydE555yL48nROeeci+PJ0TnnnIvjydE555yL48nROeeci+PJ0TnnnIvz/6KNTPvuVHA6AAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"labels=['Extremely Negative','Negative','Neutral','Positive','Extremely Positive']\n",
"prediction = model.predict(np.array(x_test))\n",
"prediction = [np.argmax(p) for p in prediction]\n",
"mat = confusion_matrix(y_true=y_test, y_pred=prediction)\n",
"sns.heatmap(mat,annot=True, cmap='Blues',fmt='d', xticklabels=labels, yticklabels=labels)\n",
"plt.xlabel('true')\n",
"plt.ylabel('predicted')"
]
},
{
"cell_type": "markdown",
"id": "9058966c",
"metadata": {},
"source": [
"Although the graph shows quite a high accuracy this is not what we are interested in. What's more important than accuracy is the validation accuracy which drops off after the 3rd epoch. Continuing the training after that will not improve the model any more but rather lead to overfitting.\n",
"## User input\n",
"The next part is an experiment. We are going to use the model that we just trained to try and predict the sentiment of user inputted strings"
]
},
{
"cell_type": "code",
"id": "124900d0",
"metadata": {},
"outputs": [],
"source": [
"user_input = ''"
]
},
{
"cell_type": "code",
"id": "e9fcdd36",
"metadata": {},
"outputs": [],
"source": [
"def predict_input(input: str)->str:\n",
" def format_string(string: str)->np.array:\n",
" tmp_df = pd.DataFrame({'OriginalTweet':[string],'Sentiment':['Neutral']}) # Creating a dataframe so we can reuse the previous preprocessing function\n",
" tmp_df = preprocess_dataframe(tmp_df)\n",
" x = np.array(tmp_df['OriginalTweet'])\n",
" tokenizer.fit_on_texts(x) # reusing tokenizer from training the model\n",
" seq = tokenizer.texts_to_sequences(x)\n",
" return pad_sequences(seq, maxlen=padded_size) # reusing padded_size from training the model\n",
" input = format_string(input)\n",
" pred = model.predict(input)\n",
" {'Extremely Negative':0,'Negative':1,'Neutral':2,'Positive':3,'Extremely Positive': 4}\n",
" classes = {0: 'Extremely Negative', 1: 'Negative', 2:'Neutral', 3:'Positive',4:'Extremely Positive'}\n",
" return classes[np.argmax(pred)]"
]
},
{
"cell_type": "code",
"id": "3c576520",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Neutral\n"
]
}
],
"source": [
"print(predict_input(user_input))"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "long_short_term_memory.ipynb",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
}
},
"nbformat": 4,
"nbformat_minor": 5