diff --git a/README.md b/README.md
index f21ff65ad580c1356112fef0b087827bbda72479..b20f7216bf925dd5e628bbfeb10a2d7594e79824 100644
--- a/README.md
+++ b/README.md
@@ -1,92 +1 @@
-# Explainable Authorship Attribution using BERT
-
-
-
-## Getting started
-
-To make it easy for you to get started with GitLab, here's a list of recommended next steps.
-
-Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
-
-## Add your files
-
-- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
-- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
-
-```
-cd existing_repo
-git remote add origin https://gitlab.liu.se/henah490/explainable-authorship-attribution-using-bert.git
-git branch -M main
-git push -uf origin main
-```
-
-## Integrate with your tools
-
-- [ ] [Set up project integrations](https://gitlab.liu.se/henah490/explainable-authorship-attribution-using-bert/-/settings/integrations)
-
-## Collaborate with your team
-
-- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
-- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
-- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
-- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
-- [ ] [Automatically merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
-
-## Test and Deploy
-
-Use the built-in continuous integration in GitLab.
-
-- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
-- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
-- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
-- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
-- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
-
-***
-
-# Editing this README
-
-When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.
-
-## Suggestions for a good README
-Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
-
-## Name
-Choose a self-explaining name for your project.
-
-## Description
-Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
-
-## Badges
-On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
-
-## Visuals
-Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
-
-## Installation
-Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
-
-## Usage
-Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
-
-## Support
-Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
-
-## Roadmap
-If you have ideas for releases in the future, it is a good idea to list them in the README.
-
-## Contributing
-State if you are open to contributions and what your requirements are for accepting them.
-
-For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
-
-You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
-
-## Authors and acknowledgment
-Show your appreciation to those who have contributed to the project.
-
-## License
-For open source projects, say how it is licensed.
-
-## Project status
-If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
+Authorship analysis is a long-standing research problem, but the recent advent of BERT has given new life to the field. This study applies BERT to classify authors in the dataset from the Spooky Author Identification competition on Kaggle. The results show that BERT outperforms a Naive Bayes model, a Logistic Regression model on the task. Further, BERT is examined in a qualitative way by extracting important features for correct predictions made using integrated gradients. Two different methods of extracting important feature were implemented and the results compared, the first using inverse document frequency and the second only the document frequency. The two methods were shown to extract different types of features, with the first method extracting more stylistic features and the second extracting features that were easier to interpret. The features provide a picture of what BERT considers characteristic for each author in the dataset.
\ No newline at end of file
diff --git a/simple-feature-engg-notebook-spooky-author.ipynb b/simple-feature-engg-notebook-spooky-author.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..39ec8eddae97ab57cbfcac2758be69eee9108f68
--- /dev/null
+++ b/simple-feature-engg-notebook-spooky-author.ipynb
@@ -0,0 +1 @@
+{"metadata":{"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Objective of the competition:**\n\nThe competition dataset contains text from works of fiction written by spooky authors of the public domain: \n 1. Edgar Allan Poe (EAP)\n 2. HP Lovecraft (HPL)\n 3. Mary Wollstonecraft Shelley (MWS)\n \nThe objective is to accurately identify the author of the sentences in the test set.\n\n**Objective of the notebook:**\n\nIn this notebook, let us try to create different features that will help us in identifying the spooky authors. \n\nAs a first step, we will do some basic data visualization and cleaning before we delve deep into the feature engineering part.","metadata":{"_uuid":"22754d0cc0847be93bf947c7998d7fb65a2817d7","_cell_guid":"02adde30-2633-41f1-a672-af8ff83a1b02"}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport nltk\nfrom nltk.corpus import stopwords\nimport string\nimport xgboost as xgb\nfrom sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer\nfrom sklearn.decomposition import TruncatedSVD\nfrom sklearn import ensemble, metrics, model_selection, naive_bayes\nfrom sklearn.model_selection import train_test_split\n\ncolor = sns.color_palette()\n\n%matplotlib inline\n\neng_stopwords = set(stopwords.words(\"english\"))\npd.options.mode.chained_assignment = None","metadata":{"_uuid":"0fcdf81ce439d2215892af58f839edfc0ca80a91","_cell_guid":"b31f62fb-bde8-410e-972c-3c092f22d497","execution":{"iopub.status.busy":"2023-01-12T09:33:55.080858Z","iopub.execute_input":"2023-01-12T09:33:55.081973Z","iopub.status.idle":"2023-01-12T09:33:57.258339Z","shell.execute_reply.started":"2023-01-12T09:33:55.081859Z","shell.execute_reply":"2023-01-12T09:33:57.257138Z"},"trusted":true},"execution_count":1,"outputs":[]},{"cell_type":"code","source":"!cd /\n!ls /kaggle/input","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:33:57.260496Z","iopub.execute_input":"2023-01-12T09:33:57.260898Z","iopub.status.idle":"2023-01-12T09:33:59.577735Z","shell.execute_reply.started":"2023-01-12T09:33:57.260861Z","shell.execute_reply":"2023-01-12T09:33:59.576080Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stdout","text":"bertpredictions spooky-author-identification\n","output_type":"stream"}]},{"cell_type":"code","source":"## Read the train and test dataset and check the top few lines ##\ntrain_df = pd.read_csv('/kaggle/input/spooky-author-identification/train.zip', compression='zip')\ntest_df = pd.read_csv(\"/kaggle/input/spooky-author-identification/test.zip\", compression=\"zip\")\nprint(\"Number of rows in train dataset : \",train_df.shape[0])\nprint(\"Number of rows in test dataset : \",test_df.shape[0])","metadata":{"collapsed":false,"_uuid":"ae86f9515b3be4956e223db4c2472c2c0c40d9fb","_cell_guid":"add1b71c-e802-408f-8f62-7ea71ed155cf","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2023-01-12T09:33:59.579923Z","iopub.execute_input":"2023-01-12T09:33:59.580722Z","iopub.status.idle":"2023-01-12T09:33:59.766629Z","shell.execute_reply.started":"2023-01-12T09:33:59.580679Z","shell.execute_reply":"2023-01-12T09:33:59.765325Z"},"trusted":true},"execution_count":3,"outputs":[{"name":"stdout","text":"Number of rows in train dataset : 19579\nNumber of rows in test dataset : 8392\n","output_type":"stream"}]},{"cell_type":"code","source":"train_df.head()","metadata":{"collapsed":false,"_uuid":"83a9dfc6af30753f82f07ef6162d3b9ba155b06d","_cell_guid":"35e22f81-77f9-438e-ba50-803aea15ad14","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2023-01-12T09:33:59.769819Z","iopub.execute_input":"2023-01-12T09:33:59.770971Z","iopub.status.idle":"2023-01-12T09:33:59.797066Z","shell.execute_reply.started":"2023-01-12T09:33:59.770906Z","shell.execute_reply":"2023-01-12T09:33:59.796122Z"},"trusted":true},"execution_count":4,"outputs":[{"execution_count":4,"output_type":"execute_result","data":{"text/plain":" id text author\n0 id26305 This process, however, afforded me no means of... EAP\n1 id17569 It never once occurred to me that the fumbling... HPL\n2 id11008 In his left hand was a gold snuff box, from wh... EAP\n3 id27763 How lovely is spring As we looked from Windsor... MWS\n4 id12958 Finding nothing else, not even gold, the Super... HPL","text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>id</th>\n <th>text</th>\n <th>author</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>id26305</td>\n <td>This process, however, afforded me no means of...</td>\n <td>EAP</td>\n </tr>\n <tr>\n <th>1</th>\n <td>id17569</td>\n <td>It never once occurred to me that the fumbling...</td>\n <td>HPL</td>\n </tr>\n <tr>\n <th>2</th>\n <td>id11008</td>\n <td>In his left hand was a gold snuff box, from wh...</td>\n <td>EAP</td>\n </tr>\n <tr>\n <th>3</th>\n <td>id27763</td>\n <td>How lovely is spring As we looked from Windsor...</td>\n <td>MWS</td>\n </tr>\n <tr>\n <th>4</th>\n <td>id12958</td>\n <td>Finding nothing else, not even gold, the Super...</td>\n <td>HPL</td>\n </tr>\n </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"We can check the number of occurrence of each of the author to see if the classes are balanced. ","metadata":{"_uuid":"d7682e0812990e3409db6076e7570ce984e466a1","_cell_guid":"5bee9e9a-5fbf-4ee6-9e92-0c77821fe77d"}},{"cell_type":"code","source":"cnt_srs = train_df['author'].value_counts()\n\nplt.figure(figsize=(8,4))\nsns.barplot(cnt_srs.index, cnt_srs.values, alpha=0.8)\nplt.ylabel('Number of Occurrences', fontsize=12)\nplt.xlabel('Author Name', fontsize=12)\nplt.show()","metadata":{"collapsed":false,"_uuid":"045fe5712b26a72855dad61f05c947ab829a295b","_cell_guid":"95529569-3380-4794-8248-caf5e8c67f6a","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2023-01-12T09:33:59.798207Z","iopub.execute_input":"2023-01-12T09:33:59.798725Z","iopub.status.idle":"2023-01-12T09:34:00.104362Z","shell.execute_reply.started":"2023-01-12T09:33:59.798692Z","shell.execute_reply":"2023-01-12T09:34:00.103052Z"},"trusted":true},"execution_count":5,"outputs":[{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n FutureWarning\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 576x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAf0AAAEJCAYAAACez/6HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgIklEQVR4nO3dfbRdVXnv8e9PEEGhghKBBjBYEMXegjQFar1WRRHQKw6vIrZKRBRtqW/Xq6JVUZRWWytqrd6iRIMvvFwVoUrFXMSBoojBF1SQEhEkkZdAAF9AIfDcP9Y8sDnk5KyQvU842d/PGGvsteaaa61nZ5yRZ6+55pozVYUkSdrwPWB9ByBJkmaGSV+SpDFh0pckaUyY9CVJGhMmfUmSxsTG6zuAUdt6661r3rx56zsMSZJmzIUXXnh9Vc2ZXL7BJ/158+axZMmS9R2GJEkzJsmVqyu3eV+SpDFh0pckaUyY9CVJGhMzlvSTvC7JT5L8OMlJSTZNslOS7yRZmuSUJJu0ug9q20vb/nkD53lzK780yTNmKn5Jkma7GUn6SeYCrwbmV9UfAxsBhwDvBY6rqp2BG4HD2yGHAze28uNaPZLs1o57HLA/8JEkG83Ed5Akababyeb9jYHNkmwMPBi4Gngq8Lm2fxHwnLZ+UNum7d83SVr5yVX1+6r6ObAU2GtmwpckaXabkaRfVcuB9wG/oEv2NwMXAjdV1apWbRkwt63PBa5qx65q9R8+WL6aY+6S5IgkS5IsWbFixfC/kCRJs9BMNe9vRXeXvhPwh8BD6JrnR6Kqjq+q+VU1f86ce41NIEnSWJqp5v2nAT+vqhVVdTvwBeAvgC1bcz/A9sDytr4c2AGg7X8ocMNg+WqOkSRJazBTI/L9AtgnyYOBW4F9gSXAOcDzgJOBBcDprf4Zbfvbbf/XqqqSnAF8Nsn76VoMdgEuGFXQL/rgl0d1aq1nn37NM9d3CJI042Yk6VfVd5J8DvgesAr4PnA88GXg5CTvbmUntENOAD6VZCmwkq7HPlX1kySnAhe38xxZVXfMxHeQJGm2m7Gx96vqaODoScWXs5re91X1O+D5U5znWODYoQcoSdIGzhH5JEkaEyZ9SZLGhElfkqQxYdKXJGlMmPQlSRoTJn1JksaESV+SpDFh0pckaUyY9CVJGhMmfUmSxoRJX5KkMWHSlyRpTJj0JUkaEyZ9SZLGhElfkqQxYdKXJGlMmPQlSRoTM5L0k+ya5AcDy6+SvDbJw5IsTnJZ+9yq1U+SDyVZmuSiJHsOnGtBq39ZkgUzEb8kSRuCGUn6VXVpVe1RVXsAfwrcApwGHAWcXVW7AGe3bYADgF3acgTwUYAkDwOOBvYG9gKOnvihIEmS1mx9NO/vC/ysqq4EDgIWtfJFwHPa+kHAidU5H9gyyXbAM4DFVbWyqm4EFgP7z2j0kiTNUusj6R8CnNTWt6mqq9v6NcA2bX0ucNXAMcta2VTl95DkiCRLkixZsWLFMGOXJGnWmtGkn2QT4NnA/528r6oKqGFcp6qOr6r5VTV/zpw5wzilJEmz3kzf6R8AfK+qrm3b17Zme9rnda18ObDDwHHbt7KpyiVJ0jRmOum/kLub9gHOACZ64C8ATh8oP7T14t8HuLk9BjgL2C/JVq0D336tTJIkTWPjmbpQkocATwdeMVD8HuDUJIcDVwIHt/IzgQOBpXQ9/Q8DqKqVSd4FfLfVO6aqVs5A+JIkzXozlvSr6rfAwyeV3UDXm39y3QKOnOI8C4GFo4hRkqQNWa+kn2Q34IaqujbJ5sAbgDuBf66qW0YZoCRJGo6+z/RPArZs6+8DngTsA/z7CGKSJEkj0Ld5f15VXZokwHOB3YBbgZ+PLDJJkjRUfZP+75JsQZfsf1FV1yfZGNh0dKFJkqRh6pv0Pwt8DdgC+HAr2xPv9CVJmjV6Jf2qel2S/YDbq+qcVnwn8LqRRSZJkoaq9yt7VfXVJDsk2aeqzq+qJaMMTJIkDVev3vtJdkxyHvBT4P+1sucl+fgog5MkScPT95W9fwe+TPdM//ZWtphuhD1JkjQL9G3e3wt4ZlXdmaQAqurmJA8dXWiSJGmY+t7pXwvsPFjQRun7xdAjkiRJI9E36b8P+FKSw4CNk7wQOAV478gikyRJQ9X3lb2FSW6gmyHvKuBQ4G1V9cURxiZJkoZobV7ZO52757uXJEmzTN9X9j6U5AmTyp6Q5AMjiUqSJA1d32f6LwQmD8ZzIfBXww1HkiSNSt+kX6upu9FaHC9Jktazvkn7G8C7kzwAoH2+o5X3kmTLJJ9L8tMklyT58yQPS7I4yWXtc6tWN+2RwtIkFyXZc+A8C1r9y5Is6P1NJUkac32T/muApwFXJ7kA+CXdaHyvWotrfRD4SlU9BtgduAQ4Cji7qnYBzm7bAAcAu7TlCOCjAEkeBhwN7E03YNDREz8UJEnSmvV9ZW9Zu9veG9ie7rW9C6rqzj7Ht5H7ngS8pJ3vNuC2JAcBT27VFgFfB94EHAScWFUFnN9aCbZrdRdX1cp23sXA/sBJfeKQ1rdrPvaC9R2CRmTbl5+yvkOQprU2r+zdCXx7ookfumb+nol/J2AF8Ikku9N1AnwNsE1VXd3qXANs09bn0v2wmLCslU1VLkmSptH3lb09k3w7yW/pJty5HVjF3ZPvTGdjYE/go1X1eOC33N2UD0C7q6++gU8T7xFJliRZsmLFimGcUpKkWa/vM/1FwDnAfOBRbdmpffaxDFhWVd9p25+j+xFwbWu2p31e1/YvB3YYOH77VjZV+T1U1fFVNb+q5s+ZM6dniJIkbdj6Jv1HAn9fVZdU1ZWDS5+Dq+oa4Koku7aifYGLgTOAiR74C7h7xL8zgENbL/59gJvbY4CzgP2SbNU68O3XyiRJ0jT6PtM/jXVPsK8CPpNkE+By4DC6Hx2nJjkcuBI4uNU9EzgQWArc0upSVSuTvAv4bqt3zESnPkmStGZ9k/6mwGlJvknX4e4uVXVonxNU1Q/oHg9Mtu9q6hZw5BTnWQgs7HNNSZJ0t75J/+K2SJKkWarve/rvHHUgkiRptHqPnZ/k6UlOSPIfbXt+kqeOLjRJkjRMfd/TfxXdULiX0Y2sB3Ar8O4RxSVJkoas7zP91wL7VtUVSd7Uyn4K7Dr1IZKkUXvpKS9d3yFoRBa+YPh91vs272/B3cPfToya90DgtqFHJEmSRqJv0j+XScPmAq+mG6VPkiTNAn2b918F/EeSlwNbJLkU+DXwrJFFJkmShmrapN9m1Xss8N+B/0Y3JO9aTa0rSZLWv2mTflXdmeT0qtoCuKAtkiRplun9TL9NfCNJkmapvs/0rwT+M8npdE37d817X1VvH0VgkiRpuPom/c2AL7b17QfK695VJUnS/VGfjnwb0d3dH1tVvx99SJIkaRSmfaZfVXcAfwPcPvpwJEnSqPTtyPcp4JWjDESSJI1W32f6ewGvSvJG7t2R70lTHiVJku43+ib9j7VFkiTNUr2SflUtWtcLJbmCbujeO4BVVTU/ycOAU4B5wBXAwVV1Y5IAHwQOBG4BXlJV32vnWQC8tZ323cOITZKkcdAr6SeZcu7Gqlqbuf+eUlXXD2wfBZxdVe9JclTbfhNwALBLW/YGPgrs3X4kHA3Mp3vEcGGSM6rqxrWIQZKksdS3ef/Fk7a3Bf4IOA9Ylwl/DwKe3NYXAV+nS/oHASdWVQHnJ9kyyXat7uKqWgmQZDGwP3DSOsQgSdJY6Nu8/5TJZe3u/7Frca0CvpqkgH+vquOBbarq6rb/GmCbtj6XrsPghGWtbKryybEdARwBsOOOO65FiJIkbbj63umvzieB64E39Kz/xKpanuQRwOIkPx3cWVXVfhCss/aD4niA+fPnO2qgJEn0fE8/yQMmLZvT3Unf1PdCVbW8fV4HnEb3GuC1rdme9nldq74c2GHg8O1b2VTlkiRpGn0H51lFNyLfxHIz8Bbgb/scnOQhSbaYWAf2A34MnAEsaNUWAKe39TOAQ9PZB7i5PQY4C9gvyVZJtmrnOavnd5Akaaz1bd7fadL2byf1wp/ONsBp3Zt4bAx8tqq+kuS7wKlJDqebye/gVv9Mutf1ltK9sncYQFWtTPIu4Lut3jETnfokSdKa9U36q4BbBl+Na3fam1XVL6c7uKouB3ZfTfkNwL6rKS/gyCnOtZB1e2NAkqSx1Ld5/4vcc0pd2vZpQ41GkiSNTN+kv2tV/WiwoG0/ZvghSZKkUeib9K9LsvNgQdu+YfghSZKkUeib9BcCn0/yrCS7JfkfwOeAj48uNEmSNEx9O/K9h+5VvffRvSf/C+AE4P0jikuSJA1Z32F47wT+uS2SJGkW6jsi31FJ/mxS2V5J3jiasCRJ0rD1fab/GuDiSWUXA68dajSSJGlk+ib9Teie6Q+6Ddh0uOFIkqRR6Zv0L+Te4+y/EvjecMORJEmj0rf3/uvopsN9MfAz4I+AbYGnjyowSZI0XH177/8kyaOBZ9G9svcF4EtV9ZtRBidJkoan750+wHZ0M+FdWFWXjSgeSZI0ItM+00/y3CRXAJcC5wE/TXJFkueNOjhJkjQ8a0z6SZ4JfAL4CPAoYDO65/kfBT6e5Fkjj1CSJA3FdM37bwNeUVUnD5RdAbw3yS/a/i+NKDZJkjRE0zXvPw44bYp9XwB2G244kiRpVKZL+r8H/mCKfVvSDdDTW5KNknw/yZfa9k5JvpNkaZJTkmzSyh/Utpe2/fMGzvHmVn5pkmeszfUlSRpn0yX9rwD/OMW+fwDOWsvrvQa4ZGD7vcBxVbUzcCNweCs/HLixlR/X6pFkN+AQuhaI/YGPJNloLWOQJGksTZf03wTsm+SiJEcnOaJ9/hDYF+g94U6S7YFnAh9v2wGeCnyuVVkEPKetH9S2afv3bfUPAk6uqt9X1c+BpcBefWOQJGmcrTHpV9VyYE/gdLo76ze0zzOAPatq2Vpc6wN0PxLubNsPB26qqlVtexkwt63PBa5qMawCbm717ypfzTF3aT9OliRZsmLFirUIUZKkDde0g/NU1Y10vfTfdl8v0l7tu66qLkzy5Pt6nr6q6njgeID58+fXqK8nSdJssDYj8q2LvwCeneRAupn5/gD4ILBlko3b3fz2wPJWfzndcL/LkmwMPBS4YaB8wuAxkiRpDfrOsrdOqurNVbV9Vc2j64j3tar6a+AcYGJkvwV0jxGge3ywoK0/r9WvVn5I692/E7ALcMFMfAdJkma7mbrTn8qbgJOTvBv4PnBCKz8B+FSSpcBKuh8KExP/nApcDKwCjqyqO2Y+bEmSZp8pk36S86tqn7Z+dFW9cxgXrKqvA19v65ezmt73VfU74PlTHH8scOwwYpEkaZysqXn/0Uk2beuvn4lgJEnS6Kypef904L/aDHubJTl3dZWq6kmjCEySJA3XlEm/qg5L8kRgHvBn3P28XZIkzUJr7MhXVd8Evplkk6patKa6kiTp/q1X7/2qWtgG1TmUbgS85cCnquqc0YUmSZKGqdd7+kleBpwKXEM3pe7VwElJXj7C2CRJ0hD1fU//jcDTq+qHEwVJTgE+D3xsFIFJkqTh6jsi38PpBsQZdCnwsOGGI0mSRqVv0v8m8P4kDwZI8hDgn4FvjSowSZI0XH2T/iuB3YGbk1wL3NS2XzGiuCRJ0pD17b1/NfCkJNsDfwj8sqqWjTQySZI0VGs14U5L9CZ7SZJmoRmZWleSJK1/Jn1JksbEtEk/yQOSPDXJJjMRkCRJGo1pk35V3QmcXlW3zUA8kiRpRPo275+bZJ+RRiJJkkaqb+/9K4H/THI6cBVQEzuq6u3THZxkU+Bc4EHtmp+rqqOT7AScTDfi34XAi6vqtiQPAk4E/hS4AXhBVV3RzvVm4HDgDuDVVXVWz+8gSdJY63unvxnwRbpkvz2ww8DSx++Bp1bV7sAewP6t5eC9wHFVtTNwI10yp33e2MqPa/VIshtwCPA4YH/gI0k26hmDJEljre/gPIety0WqqoDftM0HtqWApwJ/1coXAe8APgoc1NYBPgd8OEla+clV9Xvg50mWAnsB316X+CRJGge9X9lL8pgkb0vy4ba9a5I/WYvjN0ryA+A6YDHwM+CmqlrVqiwD5rb1uXSPEWj7b6Z7BHBX+WqOGbzWEUmWJFmyYsWKviFKkrRB65X0kzwf+AZdgj20FW8BvL/vharqjqrag+7xwF7AY9Yq0rVQVcdX1fyqmj9nzpxRXUaSpFml753+McDTquqVdB3oAH5IN+nOWqmqm4BzgD8Htkwy8Yhhe2B5W19O6y/Q9j+UrkPfXeWrOUaSJK1B36T/COCitl4Dn7X66veUZE6SLdv6ZsDTgUvokv/zWrUFwOlt/Yy2Tdv/tdYv4AzgkCQPaj3/dwEu6PkdJEkaa31f2bsQeDHda3QTDqF/wt0OWNR62j8AOLWqvpTkYuDkJO8Gvg+c0OqfAHyqddRb2a5FVf0kyanAxcAq4MiqugNJkjStvkn/1cBXkxwOPCTJWcCjgf36HFxVFwGPX0355XTP9yeX/w54/hTnOhY4tmfckiSp6fvK3k+TPAZ4FvAluh70X6qq36z5SEmSdH/R906fqrolyXnAz4FfmvAlSZpd+r6yt2OSbwBXAF8GrkjyjSSPHGVwkiRpePr23l9E15lvy6p6BLAVsKSVS5KkWaBv8/6fAvtV1e0AVfWbJG+ie3dekiTNAn3v9M/n3r3s5+OY95IkzRpT3uknOWZg82fAmUm+TNdzfwfgQOCzow1PkiQNy5qa9ydPm/uF9vkIuqlyTwM2HUVQkiRp+KZM+us6na4kSbp/6f2efpIHAzsDmw+WV9W3hh2UJEkavl5JP8mhwIeB24BbB3YVsOMI4pIkSUPW907/n4D/WVWLRxmMJEkanb6v7N0GfH2EcUiSpBHrm/TfBrw/ydajDEaSJI1O36T/X8CzgWuT3NGWO5M4l70kSbNE32f6nwJOBE7hnh35JEnSLNE36T8ceHtV1SiDkSRJo9O3ef8TwIvv60WS7JDknCQXJ/lJkte08oclWZzksva5VStPkg8lWZrkoiR7DpxrQat/WZIF9zUmSZLGTd+kvxfw8SSXJjl3cOl5/Crg9VW1G7APcGSS3YCjgLOrahfg7LYNcACwS1uOAD4K3Y8E4Ghg7xbT0RM/FCRJ0pr1bd7/WFvuk6q6Gri6rf86ySXAXOAg4Mmt2iK61wLf1MpPbI8Tzk+yZZLtWt3FVbUSIMliYH/gpPsamyRJ46JX0q+qRcO6YJJ5wOOB7wDbtB8EANcA27T1uXSz+U1Y1sqmKp98jSPoWgjYcUcHDJQkCfoPw/vSqfZV1cK+F0uyOfB54LVV9askg+epJEPpKFhVxwPHA8yfP9/Oh5Ik0b95f3Invm2BPwLOA3ol/SQPpEv4n6mqiWl6r02yXVVd3Zrvr2vly7nn1L7bt7Ll3P04YKL86z2/gyRJY61XR76qesqk5bHAK4ElfY5Pd0t/AnBJVb1/YNcZwEQP/AXA6QPlh7Ze/PsAN7fHAGcB+yXZqnXg26+VSZKkafSeWnc1PglcD7yhR92/oGst+FGSH7SytwDvAU5NcjhwJXBw23cmcCCwFLgFOAygqlYmeRfw3VbvmIlOfZIkac36PtOf3CLwYOBFwE19jq+qbwKZYve+q6lfwJFTnGshPR8pSJKku/W9018FTO4Qtxx4+XDDkSRJo9I36e80afu3VXX9sIORJEmj0/c9/StHHYgkSRqtNSb9JOdw72b9QVVV93omL0mS7n+mu9P/9BTlc4FX03XokyRJs8Aak35VnTC4neThwJvpOvCdAhwzutAkSdIw9RqcJ8kftPfjl9KNj79nVR1RVctGGp0kSRqaNSb9JJsleTNwOfBY4IlV9eKq+tmMRCdJkoZmumf6V9D9MPgnuiF3t0myzWCFqvraaEKTJEnDNF3Sv5Wu9/7fTLG/gEcNNSJJkjQS03XkmzdDcUiSpBHr1ZFPkiTNfiZ9SZLGhElfkqQxYdKXJGlMmPQlSRoTJn1JksbEjCT9JAuTXJfkxwNlD0uyOMll7XOrVp4kH0qyNMlFSfYcOGZBq39ZkgUzEbskSRuKmbrT/ySw/6Syo4Czq2oX4Oy2DXAAsEtbjgA+Ct2PBOBoYG9gL+DoiR8KkiRpejOS9KvqXGDlpOKDgEVtfRHwnIHyE6tzPrBlku2AZwCLq2plVd0ILObePyQkSdIU1ucz/W2q6uq2fg3d7H0Ac4GrBuota2VTlUuSpB7uFx35qqroxvEfiiRHJFmSZMmKFSuGdVpJkma19Zn0r23N9rTP61r5cmCHgXrbt7Kpyu+lqo6vqvlVNX/OnDlDD1ySpNlofSb9M4CJHvgLgNMHyg9tvfj3AW5ujwHOAvZLslXrwLdfK5MkST1MN7XuUCQ5CXgysHWSZXS98N8DnJrkcOBK4OBW/UzgQGApcAtwGEBVrUzyLuC7rd4xVTW5c6AkSZrCjCT9qnrhFLv2XU3dAo6c4jwLgYVDDE2SpLFxv+jIJ0mSRs+kL0nSmDDpS5I0Jkz6kiSNCZO+JEljwqQvSdKYMOlLkjQmTPqSJI0Jk74kSWPCpC9J0pgw6UuSNCZM+pIkjQmTviRJY8KkL0nSmDDpS5I0Jkz6kiSNCZO+JEljwqQvSdKYmJVJP8n+SS5NsjTJUes7HkmSZoNZl/STbAT8G3AAsBvwwiS7rd+oJEm6/5t1SR/YC1haVZdX1W3AycBB6zkmSZLu91JV6zuGtZLkecD+VfWytv1iYO+q+ruBOkcAR7TNXYFLZzzQ2Wlr4Pr1HYQ2KP5NaZj8e+rvkVU1Z3LhxusjklGrquOB49d3HLNNkiVVNX99x6ENh39TGib/ntbdbGzeXw7sMLC9fSuTJElrMBuT/neBXZLslGQT4BDgjPUckyRJ93uzrnm/qlYl+TvgLGAjYGFV/WQ9h7Wh8JGIhs2/KQ2Tf0/raNZ15JMkSffNbGzelyRJ94FJX5KkMWHSHxNJ7kjyg4HlqIF9Wye5PckrJx1zRZIfJbkoyVeTbDvzkev+JEkl+fTA9sZJViT5UjrXJ9mq7duu1X/iQP0VSR6eZNckX29/i5ck8VmtSPKbSdsvSfLhtv6OJMvb38yPkzx7oPx/r494ZyOT/vi4tar2GFjeM7Dv+cD5wAtXc9xTqupPgCXAW2YiUN2v/Rb44ySbte2n016Zra6D0PnAn7d9TwC+3z5JsitwQ1XdAHwIOK79LT4W+NeZ+wqaxY6rqj3o/s9amMQctpb8BxN0yf71wNwk209R51xg55kLSfdjZwLPbOsvBE4a2PctWpJvn8dxzx8B57X17YBlEwdV1Y9GFaw2PFV1CbCKboQ+rQWT/vjYbFLz/gsAkuwAbFdVFwCnAi+Y4vhnAf7HLOjmuzgkyabAnwDfGdh3Hncn/b2A07h7MK0n0P0ogO7HwNeS/GeS1yXZcuRRaza4x/9TwDGrq5Rkb+BOYMVMBrchmHXv6es+u7U1i032ArpkD91/5guBfxnYf06SO4CLgLeONELNClV1UZJ5dHf5Z07a/V3g8UkeAjywqn6T5PIkO9Ml/X9p5/hEkrOA/ekmzHpFkt2r6vcz9kV0f3SP/6eSvAQYHHb3dUleBPwaeEFVVZKZjXCWM+nrhcC2Sf66bf9hkl2q6rK2/ZSqcoILTXYG8D7gycDDJwqr6pYklwEvBb7Xis8HDgQewcDkV1X1S7ofmQuT/Bj4Y+DCmQhes9ZxVfW+9R3EbGbz/hhL8mhg86qaW1Xzqmoe8I+svkOfNGgh8M4pnsV/C3gt8O22/W3gNcD5rbMfSfZP8sC2vi3dDwfn0JBGzKQ/PiY/038PXXI/bVK9z2PS1zSqallVfWiK3ecBj+LupP89uomxvjVQZz/gx0l+SDek9huq6ppRxasN3luTLJtY1ncw92cOwytJ0pjwTl+SpDFh0pckaUyY9CVJGhMmfUmSxoRJX5KkMWHSl3QPSZ7sa0/ShsmkL20g2lS1NyZ50FoeV22Y3BnRpkKtJAcPlG3cyubNVBzSODLpSxuAliz/O1DAs9dvNHdLMtVQ3yuBdybZaCbjkcadSV/aMBxKN8b9J4EFgztaC8DLBrZfkuSbbf3cVvzDJL+ZmH2x7Xt9kuuSXJ3ksIHyhyY5McmKJFcmeevEvObt3OclOS7JDcA7poj3K8BtwItWtzPJM5N8P8mvklyV5B0D++a1VoHD2r4bk7wyyZ8luSjJTUk+POl8L01ySat7VpJHrvFfU9pAmfSlDcOhwGfa8owk2/Q5qKqe1FZ3r6rNq+qUtr0t8FBgLnA48G9Jtmr7/rXtexTwl+3ah919VvYGLge2AY6d6tLA24CjJ8bgn+S37bxbAs8E/ibJcybV2RvYhW6myA8Afw88DXgccHCSvwRIchDwFuC5wBzgG8BJU8QlbdBM+tIsl+SJwCOBU6vqQuBnwF+t42lvB46pqtur6kzgN8CurTn+EODNVfXrqrqCbrrcFw8c+8uq+teqWlVVt051gao6g24+9JetZt/Xq+pHVXVnVV1El6T/clK1d1XV76rqq3Q/Ek6qquuqajldYn98q/dK4B+r6pKqWgX8A7CHd/saRyZ9afZbAHx1YArkzzKpif8+uKElyAm3AJsDWwMPBK4c2HclXYvAhKvW4jpvpbtD33SwMMneSc5pjxBupkvcW0869tqB9VtXs715W38k8MHW7H8TXX+CTIpZGgtTdbKRNAsk2Qw4GNgoycQsdQ8Ctkyye1X9kO4u+MEDh227Dpe8nq4V4JHAxa1sR+45LW7vWbyqanGSpcDfTtr1WeDDwAFV9bskH+DeSb+vq4Bjq+oz9/F4aYPhnb40uz0HuAPYDdijLY+la94+tNX5AfDcJA9ur+YdPukc19I9n59WVd0BnAocm2SL1kT+v4BPr8N3+HvgjZPKtgBWtoS/F+v2uOL/AG9O8ji4qyPi89fhfNKsZdKXZrcFwCeq6hdVdc3EQneX/Nftlbnj6HrKXwssouvsN+gdwKLW/H0w03sVXevB5cA36e7KF97XL1BV5wEXTCr+W+CYJL8G3k73Q+O+nv804L3AyUl+BfwYOOC+nk+azVLVuyVOkiTNYt7pS5I0Jkz6kiSNCZO+JEljwqQvSdKYMOlLkjQmTPqSJI0Jk74kSWPCpC9J0pj4/zUxXkizY01NAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"This looks good. There is not much class imbalance. Let us print some lines of each of the authors to try and understand their writing style if possible.","metadata":{"_uuid":"7b1ddc8bf782ae87ed5c24fdb6edbff986255d60","_cell_guid":"bff5de3a-b92f-4192-b142-60d0367e1bb2"}},{"cell_type":"code","source":"grouped_df = train_df.groupby('author')\nfor name, group in grouped_df:\n print(\"Author name : \", name)\n cnt = 0\n for ind, row in group.iterrows():\n print(row[\"text\"])\n cnt += 1\n if cnt == 5:\n break\n print(\"\\n\")","metadata":{"collapsed":false,"_uuid":"cc6b841404940a0f84481c76f5c2328b373416e1","_cell_guid":"6c85f84f-d0ee-444e-8b5f-2bf62452624a","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2023-01-12T09:34:00.105942Z","iopub.execute_input":"2023-01-12T09:34:00.106305Z","iopub.status.idle":"2023-01-12T09:34:00.123675Z","shell.execute_reply.started":"2023-01-12T09:34:00.106272Z","shell.execute_reply":"2023-01-12T09:34:00.121933Z"},"trusted":true},"execution_count":6,"outputs":[{"name":"stdout","text":"Author name : EAP\nThis process, however, afforded me no means of ascertaining the dimensions of my dungeon; as I might make its circuit, and return to the point whence I set out, without being aware of the fact; so perfectly uniform seemed the wall.\nIn his left hand was a gold snuff box, from which, as he capered down the hill, cutting all manner of fantastic steps, he took snuff incessantly with an air of the greatest possible self satisfaction.\nThe astronomer, perhaps, at this point, took refuge in the suggestion of non luminosity; and here analogy was suddenly let fall.\nThe surcingle hung in ribands from my body.\nI knew that you could not say to yourself 'stereotomy' without being brought to think of atomies, and thus of the theories of Epicurus; and since, when we discussed this subject not very long ago, I mentioned to you how singularly, yet with how little notice, the vague guesses of that noble Greek had met with confirmation in the late nebular cosmogony, I felt that you could not avoid casting your eyes upward to the great nebula in Orion, and I certainly expected that you would do so.\n\n\nAuthor name : HPL\nIt never once occurred to me that the fumbling might be a mere mistake.\nFinding nothing else, not even gold, the Superintendent abandoned his attempts; but a perplexed look occasionally steals over his countenance as he sits thinking at his desk.\nHerbert West needed fresh bodies because his life work was the reanimation of the dead.\nThe farm like grounds extended back very deeply up the hill, almost to Wheaton Street.\nHis facial aspect, too, was remarkable for its maturity; for though he shared his mother's and grandfather's chinlessness, his firm and precociously shaped nose united with the expression of his large, dark, almost Latin eyes to give him an air of quasi adulthood and well nigh preternatural intelligence.\n\n\nAuthor name : MWS\nHow lovely is spring As we looked from Windsor Terrace on the sixteen fertile counties spread beneath, speckled by happy cottages and wealthier towns, all looked as in former years, heart cheering and fair.\nA youth passed in solitude, my best years spent under your gentle and feminine fosterage, has so refined the groundwork of my character that I cannot overcome an intense distaste to the usual brutality exercised on board ship: I have never believed it to be necessary, and when I heard of a mariner equally noted for his kindliness of heart and the respect and obedience paid to him by his crew, I felt myself peculiarly fortunate in being able to secure his services.\nI confess that neither the structure of languages, nor the code of governments, nor the politics of various states possessed attractions for me.\nHe shall find that I can feel my injuries; he shall learn to dread my revenge\" A few days after he arrived.\nHe had escaped me, and I must commence a destructive and almost endless journey across the mountainous ices of the ocean, amidst cold that few of the inhabitants could long endure and which I, the native of a genial and sunny climate, could not hope to survive.\n\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"Only thing I can see is that there are quite a few special characters present in the text data. So count of these special characters might be a good feature. Probably we can create them later.\n\nApart from that, I do not have much clue.. In case if you find any interesting styles (features which we can create), please add them in the comments. \n\n**Feature Engineering:**\n\nNow let us come try to do some feature engineering. This consists of two main parts.\n\n 1. Meta features - features that are extracted from the text like number of words, number of stop words, number of punctuations etc\n 2. Text based features - features directly based on the text / words like frequency, svd, word2vec etc.\n\n**Meta Features:**\n\nWe will start with creating meta featues and see how good are they at predicting the spooky authors. The feature list is as follows:\n1. Number of words in the text\n2. Number of unique words in the text\n3. Number of characters in the text\n4. Number of stopwords \n5. Number of punctuations\n6. Number of upper case words\n7. Number of title case words\n8. Average length of the words\n","metadata":{"_uuid":"d5739c4a5967d2bcb9326d39529f84791c09dfb8","_cell_guid":"84398b75-325f-4259-90ea-bb6293cc5235"}},{"cell_type":"code","source":"## Number of words in the text ##\ntrain_df[\"num_words\"] = train_df[\"text\"].apply(lambda x: len(str(x).split()))\ntest_df[\"num_words\"] = test_df[\"text\"].apply(lambda x: len(str(x).split()))\n\n## Number of unique words in the text ##\ntrain_df[\"num_unique_words\"] = train_df[\"text\"].apply(lambda x: len(set(str(x).split())))\ntest_df[\"num_unique_words\"] = test_df[\"text\"].apply(lambda x: len(set(str(x).split())))\n\n## Number of characters in the text ##\ntrain_df[\"num_chars\"] = train_df[\"text\"].apply(lambda x: len(str(x)))\ntest_df[\"num_chars\"] = test_df[\"text\"].apply(lambda x: len(str(x)))\n\n## Number of stopwords in the text ##\ntrain_df[\"num_stopwords\"] = train_df[\"text\"].apply(lambda x: len([w for w in str(x).lower().split() if w in eng_stopwords]))\ntest_df[\"num_stopwords\"] = test_df[\"text\"].apply(lambda x: len([w for w in str(x).lower().split() if w in eng_stopwords]))\n\n## Number of punctuations in the text ##\ntrain_df[\"num_punctuations\"] =train_df['text'].apply(lambda x: len([c for c in str(x) if c in string.punctuation]) )\ntest_df[\"num_punctuations\"] =test_df['text'].apply(lambda x: len([c for c in str(x) if c in string.punctuation]) )\n\n## Number of title case words in the text ##\ntrain_df[\"num_words_upper\"] = train_df[\"text\"].apply(lambda x: len([w for w in str(x).split() if w.isupper()]))\ntest_df[\"num_words_upper\"] = test_df[\"text\"].apply(lambda x: len([w for w in str(x).split() if w.isupper()]))\n\n## Number of title case words in the text ##\ntrain_df[\"num_words_title\"] = train_df[\"text\"].apply(lambda x: len([w for w in str(x).split() if w.istitle()]))\ntest_df[\"num_words_title\"] = test_df[\"text\"].apply(lambda x: len([w for w in str(x).split() if w.istitle()]))\n\n## Average length of the words in the text ##\ntrain_df[\"mean_word_len\"] = train_df[\"text\"].apply(lambda x: np.mean([len(w) for w in str(x).split()]))\ntest_df[\"mean_word_len\"] = test_df[\"text\"].apply(lambda x: np.mean([len(w) for w in str(x).split()]))","metadata":{"_uuid":"ec015064f318cfd7d0a405bd28cd002f72724bbe","_cell_guid":"5758c1ff-ca4d-4c66-8c85-ac1725bd631b","execution":{"iopub.status.busy":"2023-01-12T09:34:00.125704Z","iopub.execute_input":"2023-01-12T09:34:00.126168Z","iopub.status.idle":"2023-01-12T09:34:01.777322Z","shell.execute_reply.started":"2023-01-12T09:34:00.126133Z","shell.execute_reply":"2023-01-12T09:34:01.776248Z"},"trusted":true},"execution_count":7,"outputs":[]},{"cell_type":"markdown","source":"Let us now plot some of our new variables to see of they will be helpful in predictions.","metadata":{"_uuid":"d79f96166a7fe609fd8431b341b2d5ad189ca07a","_cell_guid":"164b0d54-3d9e-4b58-ae7c-20c447efdc69"}},{"cell_type":"code","source":"train_df['num_words'].loc[train_df['num_words']>80] = 80 #truncation for better visuals\nplt.figure(figsize=(12,8))\nsns.violinplot(x='author', y='num_words', data=train_df)\nplt.xlabel('Author Name', fontsize=12)\nplt.ylabel('Number of words in text', fontsize=12)\nplt.title(\"Number of words by author\", fontsize=15)\nplt.show()","metadata":{"_uuid":"51962b52791977deefab8b7991d59a51211c8a5e","_cell_guid":"fe9a91d9-3df7-4e1b-9e69-f701cc714a15","execution":{"iopub.status.busy":"2023-01-12T09:34:01.778626Z","iopub.execute_input":"2023-01-12T09:34:01.779647Z","iopub.status.idle":"2023-01-12T09:34:02.103428Z","shell.execute_reply.started":"2023-01-12T09:34:01.779610Z","shell.execute_reply":"2023-01-12T09:34:02.102524Z"},"trusted":true},"execution_count":8,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 864x576 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAtAAAAH1CAYAAADf3smEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAACg2UlEQVR4nOzdd3yb1dn/8c+RZMvydmzHIzsBslhhQ4G2tPBrn+4+pX3ooi0ttE+fQidd7FVGGGHvvTchgZAQAmTvvacTx7Hjva1x3+f3hyTHcexYtiXdGtf79dIrtnxL+sa2rEvnvs45SmuNEEIIIYQQIjQ2qwMIIYQQQggRT6SAFkIIIYQQoh+kgBZCCCGEEKIfpIAWQgghhBCiH6SAFkIIIYQQoh+kgBZCCCGEEKIfpIAWQoSNUuoGpZRWSn3Uw9feVEp9GsUsXwpkOT5aj9kfSqmJSqn5SqnWQM7RVmfqiVIqM5DvF4O4j5j+WQQppY4L/A7ndrv+F4H8mRZFE0LEGCmghRCRcJFS6nSrQ8S4u4Bc4NvA2cABS9MIgOOA6/H/XIQQoldSQAshwq0OWA/82+ogkaSUShvkXUwA5mit52qtl2it3eHINRDKb7D/HzEA8r0XIj5JAS2ECDcN3Ap8Wyl1Qm8HBU6V1/RwvVZK/V+Xz/copaYqpf6hlDqglGpUSt0dKDz+Sym1USnVrJR6VymV18NDlSqlZgRaJfYqpX7bw2Oep5T6TCnVppSqVUo9oZTK6vL14Cn8M5RSnyql2oG/HeX/drJSam7g/uqVUi8ppYoCXxutlNLAOOBPgfv9tJf7eU4pNbvL5+MDx7/d5bpTA9cd2+W6/1NKbVdKuZVSO5RSf+p2vzcopWqUUucqpZYDHcDFga/9t1Jqm1KqXSn1Of5Cv3uubyulVga+p/VKqaVKqS/29v3ootefReBnaSqlxnR7rDGB67/T250qpf6ilFoe+N2oUkq9r5Q6ptsxe5RSU7td19maoZT6EvB+4Eu7A9fv6fZQY5RScwL5tyilvt9DlgF/74UQ8UMKaCFEJLwBbCd8o9D/A5wB/BK4E/gzcA9wM3At8Fvgi8B/erjtU8A64PvAB8AjSqlvBr+olPoC8DFQCfwA+CPwX8AzPdzXK/iLrP8CZvQUVClVCHwKpAM/Bv4QyDZHKZWKv1Xj7MDjvRz4+H97+X/PB85WStkDn5+Pv+A6t8sx5wNVWuvtgcf/DfAAMB34Fv6fxd1KqX90u+904DngSeBrwDKl1CnAa8Ba/N+v94HXu/3/xgFvAp8E7v8nge/FkF7+D10d7WfxEVABXNrtNr8ADgIzj3K/w4EHge8AvwHswCKlVE4ImYJWAX8NfPx9/D+X73U75mX839fv4f/9flUpNTz4xcF87/uRUwgRC7TWcpGLXOQSlgtwA1AT+PgXgAEcF/j8TeDTno7tdh8a+L8un+8BdgD2LtctA3zAmC7X3Ym/kAx+/qXAfT3e7f7nAEu6fD4fmNftmAsCtz2+y/9FA1eF8D24HWgAsrtcd2bg9pd0+39N7eO+xgdud1rg8+eBRwEvMCFw3dvAG4GPbcB+4Jlu9/Mw0Aikdfnea+A73Y57HdgEqC7X/Ttw7C8Cn/8AqO3n70WoP4tbgN3BxwdUKN+nbvdpB1xAM/Dzo32/u/xcMwOffzPw+ehejvtVl+vyA7+Dvw3H914ucpFLfF1kBFoIESkvAnuBf4bhvj7VWhtdPt8B7NFa7+52XWFglLerd7p9/jZwqlLKrpRKxz/S+LpSyhG8AAvwF6mndrvt0UZBg84AZmutm4JXaK2X4i/gzu3tRj3RWm/FP/p6XuCq84EP8Y+WBq87F/+bAPCPxJbiH/ns6jUgG+jaUqMD99U9+3Stte5y3dvdjlkP5ATaSy5SSmX047/U688i8PnTwCj8BTfAlwOf93Q2oJNS6qxAa0Ut/qK2DcjEPykwnDrbabTWtfh/NsER6MF+74UQcUQKaCFERGitffhHhX+qlBo1yLtr6Pa5p5frFNC9gD7Yw+cOoADIwz9i+TD+gjl4cQMpwIhut60KIWtJL8dVEVqbQ3fzgfOUUiOAkfiL++B1E4FCDhXQJb3kDH7e9fHrtdaebscV0/P3q1OgqP8OMBZ/G0aNUurlQOtKX472s0BrvQt/+8svA1//JbBMa72xtztUSo3EX9gq4ArgC8DpgfsO9+S8hm6fe7o8xmC/90KIOCIFtBAikp7GX8j8vYevddCt2FU9TwIcrKE9fO4DavAXRBr/0mWn93B5utttNX070MNjAhThX6Gkv+bjH2U+H9gUGPmcj38E+nygCX/PcvCx6eHxiwL/dn38nv4vlT3c9oj/i9Z6ptb6PPxtDJcBX8Xf+9uXo/0sgp4E/lspNQx/L/JRR5/x9xCn42+JeFNrvQhYw5FvVo74fcP/BipcBvu9F0LEESmghRARo/1Ls00FfsWhEbqgciArUCgFXRSBGN0ngn0PWKm1NrTWrcASYLzWekUPl4oBPN5S4P91W8XjdGA0/tHj/voc/yjz5YGPwV9Aj8I/SXGR1toMXF+OfyJe91Udfoi/0F7fx2Mtx796iupy3RErTQRprRu11i/jb82Y1Pd/pfefRZfr3sY/svsq/teoV/u4Txdg4i/Eg36If2S7q3JgYrfruv++BUeFBzJyPdjvvRAijnT/AyOEEOH2GPAv4Bzgsy7XzwLagaeVUncDY/CvphFuX1dK3Rp47O8DF+JvQQi6GpirlDLxT3Rsxt8q8Q3g31rrbf18vHuA3wEfKaXuwN+Lezv+AuqtAeRfi78AOx94BEBrXaeU2hS4rnOlE621qZS6AXgs0A88B/8KIL8D/qW17ujjse7A/wbgdaXUU8Dx+EeYOymlrsDfNz4Lf8F4LP6i8fkQ/i99/SzQWncopV4Cfg+8orVu6OM+P8HfhvNMIPNk/KtpdL/dO8ADSql/4X+j8N+BY7vaGvj3CqXUq0Cb1jqkwjcM33shRByREWghRERprduAe3u4vgZ/ETMceBf4Kf4R1XD7NXBK4DG+Cfxeaz29S44F+AvRQuAF/Eu3XQ3sI7Se58NoravxT37rwL/s3UP4R4wvHEjfa2B0eVHg08+7fCnY97yg2/FPAFfhH92dAVwC/EVrfXsIj7UC/5KBU/B/v74L/KjbYevwf6/uwd97fA3wBD236XR31J9FF+8G/u3eQtNT5vX4V8k4E///98f4C/rGboc+DtwHXIl/tRE3/lU/ut5XGf7i+/vAQg6tCx2SwXzvhRDxJbhUkBBCCBETlFJ34m99GNulPUUIIWKGtHAIIYSICUqp8fh7qX8H3CjFsxAiVskItBBCiJgQ2NL8TPw7+f1MlnoTQsQqKaCFEEIIIYToB5lEKIQQQgghRD9IAS2EEEIIIUQ/xN0kwoKCAj169GirYwghhBBCiAS2cuXKGq11YU9fi7sCevTo0axYscLqGEIIIYQQIoEppcp6+5q0cAghhBBCCNEPUkALIYQQQgjRD1JACyGEEEII0Q9SQAshhBBCCNEPUkALIYQQQgjRD1JACyGEEEII0Q9SQAshhBBCCNEPUkALIYQQQgjRD1JACyGEEEII0Q9SQAshhBBCCNEPUkALIYQQQgjRD1JACyGEEEII0Q9SQAshhBBCCNEPUkALIYQQQgjRD1JACyGEEEII0Q9SQAshhBBCCNEPUkALIYQQQgjRDw6rA4jksHLlShYuXIhSiq9//escc8wxVkcSQgghIsbn8/HOO+/Q3t7O+PHjOfPMM62OJMJICmgRFQ8//Ajbd+wANPX19Vx33XVWRxJCCCEiZtWqVTzwwAMAZGZlMnPGTJRSFqcS4SItHCLifD4fu/fsxlM8GV/OCLZu2251JCGEECKidu3aBYA50aSluYXa2lqLE4lwkgJaRNy+ffvweb2Y6UMw04ewv3wfbrfb6lhCJI2Kigp27NiB1trqKEIkjd27d2Nz2dBD/c+7PXv2WBtIhJUU0CLitm/3jzib6fn+i2nKHxIhoqStrY3/+Z//4Ve/+hUrV660Oo4QSWPXrl0YWQZk+z/fvXu3tYFEWEkBLSJuw4YNKHsKOi0XM7Og8zohROQdPHiw8+Py8nILkwiRPLxeL7t270LnaHCCLc3Gtm3brI4lwkgKaBFxy5avwJtVDDYb2pkFadmsWLHC6lhCJIXq6uoePxZCRM6WLVvwerzoAg0KjHyD1WtWWx1LhJEU0CKiqqqqqNhfjpFd2nmdN7uUlatW4fP5LEwmRHKoqanp/FgmMQkRHWvXrvV/UOj/RxdqDlYdpKqqyrpQIqykgBYRFey5NHKGdV5nZA+jo72dzZs3WxVLiKQRbOEYmWVQWXnA4jRCJIe1a9eichQ4/Z/rQt15vUgMUkCLiFqwYAHKmYF25XVeZ+SUgFIsWrTIwmRCJIe9e/dS4IKRGT72lZVZHUeIhOd2u1mzZg1GgXHoyhxQTsXy5cutCybCSgpoETGtra0sWboUT+5o6Lp4vCMNI7uUj+d+IstqCRFhZWV7KHF5Kckwqa6to62tzepIQiS0VatW4Xa70aVdXt8UGMUGCxctlPbFBCEFtIiYRYsW4fN68eWPOeJrviFjqao8ILOShYggrTV7y/ZSkmFQku4fDdu3b5/FqYRIbAsWLEClqM7+5yBdqmlpbpFVqBKEFNAiYj75ZB7KmYGZWXTE13x5o0Ap5s2bZ0EyIZJDZWUlHW43pRkGwzL9BXRwdzQhRPiZpsn8BfMxigywd/tiESibYsGCBZZkE+ElBbSIiObmZpYsXYInb/Th7RtBKWkY2cOY8/HHmKYZ9XxCJIPgJkajswxK0k1S7YeuE0KE38aNG2mob4BhPXwxBcyhJvM+nSftiwlACmgREZ999hmGz4cv/5hej/EWjKP64EE5nSVEhGzfvh2lYESmgU3ByEyD7dI2JUTEzJs3D2VX6JKeC2Q9QlN9sFpWoUoAUkCLiPjoo4/AlYuZUdDrMUbeaJQ9hdmzZ0cxmRDJY/v27ZRmaJyBU8mjsrxs375dzvoIEQGmaTJ33lzMYhNSej5Gl2qUTdoXE4EU0CLsqqqqWLt2LZ78cT23bwTZU/DmjmTu3E/weDzRCyhEkti2dQujMg89t0ZnGbS1t3PggKwHLUS4bdiwgfraevTwo7RnpIJZZDL3k7nSxhHnpIAWYffZZ58B4Msf2+exvvxxtLa2dG64IoQIj/r6empq6xiddWgt2uDH0gctRPh9/vnnR23fCNLDNTXVNWzZsiVKyUQkSAEtwu6zzz6HjCHotJw+jzVyhqEcqcyfPz8KyYRIHl0nEAYNzzSwK2T5SCHCTGvN5/M/xyzsvX2j89gSDQrZTCzOSQEtwqq+vp4NG9bjyR0V2g1sdrzZw/n88/kYhtH38UKIkASL5FFdCugUGwzLNKWAFiLMysrKqDxQefjmKb1xAgUwf4EMHMUzKaBFWC1cuBCtNUbe6JBv4xsymqamRlmNQ4gwKisrY4hLkZFy+Av68AwvZXt2W5RKiMQUXNs5pAIaMEtMdu3cRWVlZSRjiQiSAlqE1apVq/ybp6QPCfk2Rs5wUEr6oIUIo3379lKU5j3i+iKXycHqGtxutwWphEhMK1asQOUqcIV2fLBPetWqVRFMJSIpagW0UupPSqmNSqkNSqlXlFJpSqkxSqmlSqkdSqnXlFKp0cojImPDxo140wuPvvpGd45UcOXKhAohwqh83z6K049siyrJMNBaU1FRYUEqIRKPaZps3rIZY0g/2hCzQKUqWQ86jkWlgFZKDQOuBE7TWh+Pf4PL/wHuAO7VWh8D1AOXRSOPiIzGxkYqDxzAzCzs9229GYVs3LRJlvURIgyam5tpam6hyHXkC3qRy78GdHl5ebRjCZGQ9u3bR3tbO4R+4hUUmLkmGzZK62K8imYLhwNwKaUcQDpwALgAeDPw9eeA70YxjwizrVu3AmBk9L+ANjMKaG5qkvVphQiDxsZGAHJSj3xDGrwueIwQYnCCo8h6SP8GgPQQze5du6WdKk5FpYDWWu8HpgJ78RfOjcBKoEFr7QscVk7Pu8eLOLFv3z4AzPTcft/WdOUBsH///nBGEiIptba2ApCecuQLenqKedgxQojB2bNnj7+ayu7nDXP97R9yNig+RauFIw/4DjAGKAUygK/14/aXK6VWKKVWVFdXRyilGKyWlhb/B3Znv2+rHc7D70MIMWDB51G648gCOs0OCnmuCREujY2N2NJs/idWP2in//nZ0NAQ/lAi4qLVwvFVYLfWulpr7QXeBr4A5AZaOgCGAz0OP2qtH9dan6a1Pq2wsP/tASI6mpubUfYUsNn7f2O7FNBChEtbWxsAafYjC2ibAqdD0d7eHu1YQiSkhoYGdA/tUn1yHrq9iD/RKqD3AmcppdKVUgr4CrAJmAf8IHDMpcB7UcojIqClpQVS+j/6DKAd/gVYmpubwxlJiKSUkuLfCs3o5TXdZ2ocDkfPXxRC9EtDQwNmqtn/GwZeLmU+QnyKVg/0UvyTBVcB6wOP+zjwd+DPSqkdQD7wVDTyiMgwTZN+n8MKCix7578PIcRgpKWlAeA2jnw+Gib4zEPHCCEGp72j3b+2WH8F3sPKJML4FLUhCK319cD13a7eBZwRrQwisrKzs8HbMaDbKq//D0hOTk44IwmRlJxO/9CWp4cC2mMefowQYnAyMzKhaQA3DOxzlJ6eHtY8IjpkJ0IRNjk5OWjDC6av74O7UT5/4Z2d3d9pzEKI7oIvyG2+Iwvo4HXyoi1EeGRkZGDzDaCcCrxUynMxPkkBLcImOHqsfAM4HRUooGUEWojBC062rnMfWUDXu22HHSOEGJyMjAxUD29W++Q9dHsRf6SAFmFTUFAAgHL3fyUNW+A2wfsQQgxceno6Geku6jqO/BNfG7hu6NCh0Y4lRELKyclBd2jo70IcHYduL+KPFNAibEaPHg2Arb2+37e1tdeTkpJKSUlJmFMJkZyGDh3aWSx3JQW0EOE1cuRItFd3FsShUs2q8/Yi/kgBLcKmuLgYpzMNW1v/C2jVXs/IUSOx2wcylVkI0V1xSSk17iPnidd02HClOcnKyrIglRCJp7MA7u8qrE2Qk5sjz8U4JQW0CBubzcboMaOxtdf1+7YpHY0cM25cBFIJkZyGDRtGVbsd3e20clWbnWHDhqHUAJecFEIcJlhAq6b+PadszTZGjxodgUQiGqSAFmF1zLhxONrrOeJV+2i8HWh3C2PHjo1cMCGSzLBhw3D7NA2ew1/UqzpSGD5CThkLES4FBQWkZ6RDf/ZD0f4WjjFjxkQsl4gsKaBFWI0fPx7t7UB5Qp9IaG+tAeC4446LVCwhks6IESMA/4hzkGFCdRsMHz7cqlhCJBylFBMnTMRe348WxGbQHs2ECRMiF0xElBTQIqyCRbAtUBSHwiYFtBBhF5yQe7D90J/5OrcNQ0NpaalVsYRISBMnTkQ3ajBCO17V+88MSQEdv6SAFmE1btw4bHZ7vwvo4pISmUghRBgVFRWhlKKmy0oc1YFiuri42KpYQiSkCRMmgAk0hHiDOnCmORk1alQEU4lIkgJahJXT6WTUyFHY2kKfSJjSUc+E8eMjmEqI5JOamkr+kNzOohnoLKalgBYivCZOnAiAqgttIqGt3sZxxx0nK0/FMSmgRdiNGzeWlI6G0A42vOj2RplAKEQEFBeXHjYCXdNhQykla0ALEWaFhYXkDcmDUMaOTFANismTJkc8l4gcKaBF2I0dOxbd0Qw+T5/H2tobOm8jhAivgsJCGr0pnZ83uG3kZGWSmppqYSohEtPkSZOxN4QwotwI2tCdo9YiPkkBLcIuWAyHsiNh8BhZykeI8MvNzaW5yzJ2zR5FTm6udYGESGATJkxAN2noY+wo2OYhEwjjmxTQIuyCy2fZOvpeFFN1NGKz22VVACEiICcnhxaPxgwsy97stZGbl2dtKCESVGdB3NDHgfWQmZUpcxHinBTQIuyC/ZXK09rnsTZ3K/n5BTKRQogIyMnJQQMtXv+IV7PPTk5OrqWZhEhUwbOvqvHoEwltjTaOGXeM7AYa56SAFmHndDrJys4JqYBWnhaKi4uikEqI5ONyuQDwGP4Xardp67xOCBFe+fn5ZGZlHn1HQu3f8nvcuHFRyyUiQwpoERFFRUNR7r53I3T42iiSFQGEiAin0wmAx/R/7jUPXSeECC+lFMeMOwZb01FKq1bQPi0T5xOAFNAiIgoLCrAbHX0f6GlnyJAhkQ8kRBIKrrbhNf0j0F4DWYFDiAgaPXo0qvkorRnN/n9kA5X4JwW0iIiMjAxshvfoB2kTbXjJyMiITighkkxKin8JO2/nCLTuvE4IEX4lJSVoT+8rcag2f3EtE+fjnxTQIiLS09OhrwI68HUpoIWILJmqJER0lJSU+D9o6+WAVnCkOOTMawKQAlpEREZGRp8bqahAAZ2enh6NSEIkHa3969cFJ/vblOq8TggRfp1L0/U2h77Vv1KVzSblV7yTn6CICKfTiTZ9cLQXa9MHSE+mEJFimv7ejeAItOpynRAi/AoLCwFQ7T2f97F12CgaKitPJQIpoEVEhLa+pb+4ljWghYiMYLFsCzwdlQLDMCxMJERiy8rK8n/QSwejzWsjJycneoFExEgBLSLi0Ompo4xA6+7HCiHCyePxt1Gl2PxPtlQbeL19zE0QQgxYamoqzjRn79t5e7oU2SKuSeUiIqKzKD5qv2WwP1OmOAkRCW63G/AXzgApdt15nRAiMjIzM3suoDVot5YCOkFIAS2EEAkqWCyn2A+NQAdHpYUQkZGeno7y9jAwpEGbWibOJwgpoEVEdPZZqqP8igVGnqUnU4jIODQCHSygTTo6QtjgSAgxYA6Ho+fuRbPL10XckwJaRMShAvpo7Rm2w48VQoRVe3s7AM7APF2nzey8TggRGQ6Ho7NYPkygqJaJ84lBCmgREaZpHn30GTqLa1lWS4jIaG9vx+lQnatwpNk17W29LVArhAgHGYFODlJAi4jw+XyoPgtoW+exQojwa29vJ63La3WaQ9Pe3tsWaUKIcOhrsyLZzCgxSAEtIsLn80Ffy9MpaeEQIpLa2tpIsx96sU6za1pbpYAWIpI8Hg/01KURuE6WkkwMUkCLiPD5fH22cGgZgRYiotra2nDZD7VIuRxaeqCFiDCPx4O29TDKbDv0dRH/pIAWEdGfEWgpoIWIjNbWFtJsh87wuOyatvYOmXcgRAS5Pe6eR6BtgJICOlFIAS0iwt+WIQW0EFZqa23F5Tg0Ehb8WEahhYicjvaOngtoQKUoef4lCCmgRUQYhtHHEnbIKhxCRFhLS0uPBXRrq6zEIUQkaK1pbWmF1J6/rlIVzc3N0Q0lIkIKaBERsoydENZrbW0lXQpoIaKmvb3d/5qW0vPXdYqmpaUluqFEREgBLSIipBFoZCdCISJFa01rW9thI9DBYlpewIWIjM7R5V5GoE2HKSPQCUIKaBERpmmiCaGFQykpoIWIALfbjWGYh41Ap8sItBAR1djYCIBO7WWt51Soa6iLYiIRKVJAi4jwt3D0NQIN9FVkCyEGJDjKLAW0ENFTX1/v/yCt56/rNE19XX30AomIkQJaRIR/p6UQimOlpAdaiAgIFtAZ0sIhRNT0VUCTBq0trbL6VAKQAlpERKgj0AopoIWIhGCR3FMPtPRgChEZnQW0s5cDAtc3NDREI46IICmgRURoremlA+xwMgItRER0jkCnHHomptjAYZMCWohIqa2tRTkUOHr+uk7TnceJ+CYFtIgIf1EcWguHv91DCBFOTU1NwOE90EpBRqqSFg4hIqSurg7lUr2//KUdOk7ENymgRUSE3AMtLRxCRERwlLnrCDT4e6JlBFqIyKipqcFwHmVlKZf/HxmBjn9SQIuI8C9jFwIZgRYiInqaRAiQYfdJAS1EhFTXVHe2afQoMAItBXT8kwJaRIRhGOhQlrGTdaCFiIjm5macDoWj21/5jBSTpsYGSzIJkejq6up6X4EDwAY2p42ampqoZRKRIQW0iAj/Ej0h/HopmxTQQkRAY2MjWT1s5pDp0DQ1NVqQSIjE1tHRQXtb+9ELaEC7tPRAJwApoEVEeH0+UKEV0LIephDh19zcTKbjyPkFmSma5maZRChEuHUWxa6jH2emmtTUygh0vJMCWkSE4ZMWDiGs1NjYQIbjyOdWRoqmrb0Dr9drQSohElewr/moPdD4R6ClhSP+SQEtIsLj8YDN3udxWtn9xwohwqqpoYHMlB5aOALXNTZKG4cQ4dQ5At3bJipBTmiob5AJ9HFOCmgRER1uN9h6WUm+C22z43a7o5BIiOTS0NhIVsqRLRxZqf7rgutECyHCo3N3wT56oEkDr9dLe3t7pCOJCJICWkSE292BDmEE2lRSQAsRbj6fj5bWNrJ6GIHOkhFoISKis4AOYQT6sONFXJICWkSE2+0JaQQa5aC9oyPygYRIIi0tLWitpYAWIorq6+uxpdr6rKy0U3ceL+KXFNAi7LTWuDs60KG0cNgdtLW1RSGVEMkjOLKVmdp7C4eMfgkRXg0NDX2PPoOMQCcIKaBF2Hk8HgzDB46+/5JoeyotLa1RSCVE8gi+MGf3sA50cARaXryFCK+mpibMHuYdHCH10PEifkkBLcIuuIWwtqf0fbAjldZWWZNWiHDqLKB7aOFw2CAjRUkBLUSYNTQ2oHt403qEwNiStFHFNymgRdgdKqBT+zxW21PpaG+XtaCFCKNgcZzVQwsH+Eempf9SiPBqbGzs7G8+KgdgkxHoeCcFtAi75uZm/wehtHAEjmltlTYOIcKlrq4ORc8j0ABZKT4poIUIs+bm5s72jKNSYEu1SQEd56SAFmEXPC2lHX0thnnoGDmVJUT41NfXk+lU2Hv5C5+TalIvWwkLETY+n4+O9g4IoXMRgNQug00iLkkBLcIuePpYp/RdQBMooKUfU4jwqaurI/sok5myU03qZARaiLAJti6GNAINmA5TCug4JwW0CLt+jUCnSAEtRLjV19WRk+Lr9es5qZrmllZ8vt6PEUKErrMYDrGA1qmapmZp4YhnUkCLsKuvr0fZHaFt5S0j0EKEXW1tDbnO3kegcwOTC+vq6qIVSYiEFiygdS/zDrrTKVp6oOOcFNAi7Orq6iA1HZTq81id4jp0GyHEoGmtqa2tI/coy2nlBlYKqKmRPmghwqG/LRzSAx3/pIAWYVdbW4fhcB12XWrZYlLLFh95sM2OSkmjtrY2SumESGwtLS14vN6jj0AHvibPOyHCo78tHKRCW2sbphnCxisiJkkBLcKuuqYGs1v/s621Fltrzy/WOsUlL+RChEnwuXS0AjpPCmghwqqzgA51FY4U/9mitra2iGUSkSUFtAi7utpadEp6yMcbDhe1tdLCIUQ4HDx4EIAhR9nQISdVY1dQXV0drVhCJLTOfuZ+jEAfdjsRd6SAFmHldrtpbW1Bp4ZeQOsUF9XSiylEWAQL6Py03kegbQryXKrzWCHE4DQ1NaEcCuyhHR/csVAK6PglBbQIq+BkwP6MQJup6dTX16F1aLOXhRC9q66uRnGoTaM3eak+qqqqohNKiATX2NiIcvY9cb6TjEDHPSmgRVgFeyr7NwKdjs/rPTSLWQgxYFVVVeSkKRx9/HXPdxpUVR6ITighElxTUxP6KCvfHCFQQMsuvPFLCmgRVodGoF19HHlIcLRaJjQJMXiVlZUUOvveIKXQZVBdXYNhGFFIJURiq6+vx0ztx4oaTv8/sgdC/JICWoRV5wh0P1o4dKrrsNsKIQbuwP5yCtL6LooL0kx8hiFrQQsRBrX1tZ19zSFJBZQU0PFMCmgRVnV1daBU5xbdoZARaCHCw+fzUVVdQ6Gr7wJ6qMs/WnbggLRxCDFYjQ2NEPrLHiiwpdmkgI5jUkCLsKqtrUWlukCF/qslBbQQ4VFdXY1pmhS6+j6VXCgFtBBh0d7ejsft6WzL6EqtUag1PU8u1E4tu/DGMYfVAURiqaurQztC738GwJ6CsjvkD4kQg7R//34AikIooAvSTGzq0G2EEAPT+drVUwHd4C+eNUe2d5hOk9o6GTiKVzICLcKqpqYWX38LaKUgNV0KaCEGqbOATu+7hcNhg4J0KaCFGKzOuT+u/i3FqtO0zEGIY1JAi7Cqravr1wocQYY9TQpoIQapvLycFDvkhTiZqcjppXzf3ginEiKxdb529acHOnB8Q32D7IEQp6SAFmGjtaahvr5fK3AEmSkuqmvkVJYQg7Fv3z6K0jW2EPdzKE432LevXF7AhRiEzvk7AyigfT6fbKYSp6SAFmHT3NyMYfgGNAKtU1zUywi0EIOyr2wPJS5vyMeXpJu0tbfL2R8hBqGmpsZfTfXQA31Uri63F3EnagW0UipXKfWmUmqLUmqzUupspdQQpdQcpdT2wL950cojwm8gm6gE6RQXLS3N+Hx9bwAhhDiSz+ejorKSkhD6n4NKMvzH7tu3L1KxhEh41dXV2NJt0I+dvOFQz3R1dXUEUolIi+YI9DRgltZ6AnASsBn4BzBXa30sMDfwuYhT9fX1wMALaK21rIkpxABVVFRgGCbF6aHvhhY8du9e6YMWYqCqq6sxnf3YhTBIRqDjWlQKaKVUDnA+8BSA1tqjtW4AvgM8FzjsOeC70cgjImNwI9Dph92HEKJ/gkVwaUboI9D5aSYpdimghRiMqoNVmCEsHXmEwEvlwYMHwxtIREW0RqDHANXAM0qp1UqpJ5VSGUCR1jq4in8lUBSlPCICBtvCAYdGsYUQ/VNWVgb0XEC/sNXFC1uPfF7alL8POnhbIUT/aK39LRj9f9kDG9hcNhmBjlPRKqAdwCnAI1rrKUAr3do1tH8aeI9TwZVSlyulViilVkivUOyqra3170Do6O9MCtmNUIjB2rt3L3lpivQetscqa7ZT1mzv8Xal6T727tkd4XRCJKbm5ma8Hi/0f/EpAEyXKSPQcSpaBXQ5UK61Xhr4/E38BXWVUqoEIPBvj79FWuvHtdanaa1PKywsjEpg0X91dXUoZ4Z/Y5R+0qmuzvsQQvRfWT9X4AgqzTCoPFiN2+2OQCohEluw+NXpA1sKUrs0ByoP9H2giDlRKaC11pXAPqXU+MBVXwE2AdOBSwPXXQq8F408IjLq6uow+7sLYZDNgXI4ZQRaiAHQWlO2Zw+lGf1fxaY0w0BrLStxCDEAnaPHA3zp0y4tq3DEqR5O9kXMH4CXlFKpwC7gl/gL+NeVUpcBZcAPo5hHhNnBg9UYjv6uJH+ITk2XAlqIAaitraW1rZ3SEf2fyDQs0DNdVlbGMcccE+5oQiS0qqoq/wcDbOEgHTraO2hpaSEzMzNsuUTkRa2A1lqvAU7r4UtfiVYGEVnVNTXo9OEDvr3hcMk7cSEGIDgJcFg/VuAIKk43UQr27NkT5lRCJL7KykqUTfV/F8KAYOtHVVWVFNBxRnYiFGHhdrtpbWlGpw70bTiYqekclAJaiH4LFr/9WcIuKMUGRenIShxCDMDBgwdRGarfm6h0CrxkVlZWhi2TiA4poEVYBFsvdGrGgO9Dp6RTV1uHaQ5gPU0hktiePXvISFXkpg5sItOwdA+7d+0IcyohEt+BygMYrv6/ce0UeMmUAjr+SAEtwqJzJnLKwEegdWoGhuGjsbExXLGESAq7d+9iWLp3IAvgAP7Wj/L9FXi9/V/FQ4hkVnGgYsArcADgBGVXUkDHISmgRVgEC2jTOfAeLp3qv23npAwhRJ+01uzetWtA/c9BwzMNDMOUlTiE6IeOjg4a6hpgMK3LCsiEioqKMKUS0SIFtAiLYNE7qBYOZ8Zh9yWE6FttbS3NLa0MH0wBneFvm9q9WzZUESJUnaPGA3/ZA8BMN9lfsX/wgURUSQEtwuLgwYOolDSwpwz4PkwZgRai33bt2gXAyKyBF9ClGQZ2dei+hBB9C44a64xBtHAEbl9RUYF/Q2YRL6SAFmFx4MCBzgJ4wBxOlD1FCmgh+mHnzp0AjMgceAHtsEFppmbHDplIKESoysvL/R8MdvW5TP9a0A0NDYONJKJICmgRFvsrKjAGW0ArhXZmceCAbGsqRKh27drFEJciM2Vwo1cjMjzs3LE9TKmESHx79+7F5rSBc3D3o7N05/2J+BFSAa2Uur+X6+8LaxoRl0zTpKqqCjMta9D35UvNZP9+mUwhRKi2b9vKiHTPoO9nZKbBweoampqawpBKiMS3d+9ezMwwLLsaeOmUSbzxJdQR6F/0cv3PwpRDxLHa2lp8Xi/aOfgCWjuz/MsCSS+YEH1yu93sKdvL6GzfoO9rdKCHetu2bYO+LyGSwZ69ezCzwlBAp/uXspMR6Phy1K28lVK/Ch7X5eOgsUBNRFKJuLJ/v3/2sHZmD/q+zLQs3FX+XrC8vLxB358QiWznzp2YptlZ/A7GqC4F9GmnnTbo+xMikTU3N/uXsBsWhjtTQJasghNvjlpAc2iEOZXDR5s1UAVcGolQIr4EJ1KYaYMvoHVaTud9SgEtxNFt3boVgDFhGIHOStUUph+6TyFE74KTd3VueM6WGtkGO3bKJN54ctQCWmv9ZQCl1C1a62uiE0nEm/LycrDZ0IPYRCUoWITv27ePE044YdD3J0Qi27JlC9lORb4zPC/iYzI9bNm0MSz3JUQiCxbQ5ITpDnOhdm8tTU1NZGcPfjBKRF6oPdCze7pSKXVJGLOIOFVeXg5p2aAGv6iLdmaBsh1aHkgI0auNG9YzLss94C28uzsmx8eBqoPU1dWF5w6FSFC7du3yr8CRFp770zm6835FfAi14nlTKXWHUioFQCmVq5R6DbgxctFEvNhTthdfapjeMSsbuLJlMoUQfWhqamLvvnKOyRl8/3PQMTn+VpCNG2UUWoij2bJ1C0aO4e9fDodc/z8yiTd+hFpAnxy4LFdKXQasBxqAKRFJJeKGz+djf3k5pis3fPfpzGHX7j1huz8hEtHmzZsBODZn8P3PQaOzDOw2KaCFOBqPx8OunbvQeWFcLSoNbBk2tmzZEr77FBEVUgGtta4Avhs4/nHgQ631FVrr1ghmE3HgwIEDGIYPHcYC2kzLpaJiPz5f+AoDIRLNunXrsCkYG4YJhEGpdn8RvX7d2rDdpxCJZufOnRiGgR4S3uVWjVyDTZs3hfU+ReSEupHKycByYBfwHeACpdTLSqncyEUT8WDPnj0AYR2B1q5cTMPoXB5PCHGkNatXMSbbIK2vtZT6aWKul82bt9DR0RHeOxYiQXSOEod5oSidp6nYX0Fzc3N471hERKgtHHOB+7TW39VazwBOAtrxt3KIJNZZQKflhu0+zXT/XyVZE1OInnV0dLB58xYm5nrDft8T8rz4DEPaOIToxaZNm7Cl2SA9vPcbHNEOtmeJ2BZqAX261vrJ4Cda61at9WXA7yMTS8SLXbt2odKywJEatvsMFuMyG1mInm3cuBGfYTA+L/wF9HG5PmwK1q6VNg4herJm7RqM/DBOIAzKBxSsXy9jk/Eg1B7oXUqpCUqpa5VSDwEopcYDeyIZTsS+7Tt24g3j6DMAdge4cqSAFqIXq1atwqZgfG745wmkO2B0tsnKlSvCft9CxLva2lqqKqvQ+eHtfwb8O3Pk+uc3iNgXag/0xcDn+DetDO5ImAXcE6FcIg54vV7Ky/dhpg8J+3370vLYvmNn2O9XiESwYsVyxuUYpIe5/zlocp6bTZs20dbWFpkHECJOBUeHdUEECmjAzDfZtGmTTKKPA6G2cNwEXKi1/i0QXHR0Lf5eaJGk9uzZg2kYESmgzfQhVB6okBdwIbppbm5m65atTM7zROwxJg/xYRimtHEI0c3atWtRDhX2CYSdCsDtdst60HEg1AJ6KBA8p6C7/BuZt2AiLuzYsQMAMz0/7PdtZuSjtZY2DiG6WbNmDabWTB4SuRGq43J8pNhhxQpp4xCiq9VrVmMOMUOvnvpJF/rLKnnzGvtC/RVYyaHWjaD/AZaFN46IJ9u2bUPZU9BpYdqFsItgUb59+/aw37cQ8Wz58uU4HSqsG6h0l2qH8Tleli9bGrHHECLeNDU1sXvX7s4iNyLSQGUrVq9eHbnHEGERagfdlcDswC6EGUqpj4DjgIsilkzEvO3bd2CkD/Fvvx1mOjUDlZImBbQQ3SxftpRJuW4cERoBCzoh38sr2/dSXV1NYWFhZB9MiDiwdu1atNaRLaABo8Bgzdo1+Hw+HI4ITXQQgxbqKhxbgAnAQ8A1wDPACVprqW6SlGmabN22FcMV/v5nAJTCl57Plq1bI3P/QsShiooK9lcc4IQItm8EBR9D2jiE8Fu9erW//zlCL3udhkJHe4f0Qce4UFfhuF9r3aa1fl1rfZfW+lWtdYtS6r4I5xMxat++fbg7OjAzIzcyZaQXsHvXLtxud8QeQ4h4snz5cgCOzw//+s/djcg0yHHCsmXSqScEwPIVyzHzTbBH9nGCI9yrVq2K7AOJQQn1JOAverm+e1+0SBJbAyPDRkZBxB7DzCjAMAyZSChEwIoVKxjigpJ0M+KPpZR/ObuVK5ZjmpF/PCFiWV1dHWV7ytBDo7B2QhqoXMXKlSsj/1hiwI7aXKOU+lXwuC4fB40FaiKSSsS8LVu2+CcQunIj9hhmpr8437p1KxMnTozY4wgRDwzDYNXKFUzJdaPCvQNaL44f4mPRpiZ27tzJscceG50HFSIGBSf1RaWABoxCg3Xr1uF2u3E6nVF5TNE/fY1A/yxwSe3y8c+AnwLjgEsjmk7ErE2bNkVsAmGQTs1EpbrYvHlzxB5DiHixbds2mltao9K+EXT8EP9jSR+0SHYrVqxApSrIjc7j6aEar9fLhg0bovOAot+OWv1orb+stf4ycHvw48DlAq31JVrrJVHKKWKI1+tl2/bt+DKHRvaBlMKbXsCGDRsj+zhCxIHg6dzJedHboWxImqY0U7NSCmiRxLTWLF22FLMwcus/H6EQsMmb11gW6ioc10Q6iIgfO3bswOf1YmZEuIAGzMyh7Nu3l+bm5og/lhCxbNWqlQzP1OQ4o7t/1aRcN+vWrZOthUXSKi8vp6a6Bl0UxedeCjAEli2XSbyxKlrvpUQC2bRpE0BEV+AIMgKj3Fu2bIn4YwkRq7xeLxvWr2dSXvRXpJmU56PD7ZZWKpG0gqvfRLWABswikx3bd9DY2BjVxxWhkQJa9NvGjRtRzgx0akbEH8vMKOx8TCGS1ebNm+lwe5gYxfaNoOBjypJaIlktW7YMlakg8i95h9FFGq21tHHEKCmgRb+tXbcOb8ZQorIUgCMVMvJZt3595B9LiBi1evVqFFhSQGelakZmmaxZsybqjy2E1bxeLytXrcQoMiBKq990GgLKqTpHwEVsCXmPSKVUDjAeyOx6vdb6k3CHErGrurqa6oMHMUaeFbXH9GYUsnHDRgzDwG6P8Ar2QsSgNWvWMCLLJDMluqeQgybkevh8w3rZWlgknQ0bNuDucEe9fQMA5V/ObvGSxWitUdFav1KEJNSdCH8BVADvA091uTwZsWQiJgWX1DGziqL2mEZWEe3tbezZsydqjylErPD5fGzcsJ4JuR7LMkzI9dHh9shcBJF0li5d6q+UIj9nvmfFUF9XLxuKxaBQWzhuBX6gtS7SWo/pchkbyXAi9qxbtw5ld2CmD4naY5qZxQCslzYOkYS2bNlCh9vDhFzrVsEYH2gdkTYOkWwWL1kM+fhXxbCALvaPfC9ZIqsGx5pQC2gHMDuSQUR8WLtuHb6MQrBFr5VCOzNRzgzWrVsXtccUIlYEi9bxFvQ/B+WkaoZlaimgRVI5ePAgu3ftxiyxcCt7l39b78WLF1uXQfQo1AL6DuAapSK47ZyIea2trezcsQMjqzi6D6wU3owiVq9ZG93HFSIGrFmzhmGZmpxUa/qfgybkulm/bq2sBy2SRnDUV5dY+9wzig02bNgg+yHEmFAL4j8B1wDNSqm9XS8RzCZizMaNG9FaY0Sx/znIyCqitqaaysrKqD+2EFbx+XysX7eWibnRX/+5u4l5Pto73Gzbts3qKEJExeIli1EZCrKszaFLNKZpsmyZbKoSS0KdTv3TiKYQcWHt2rWgFGakt/DugRkY9V63bh3FxVEeARfCItu2baO9w80EC9s3goI92GvWrGHSpEkWpxEistxuN8uXL8cYbsHydd0FlrNbsmQJX/nKVywOI4JC3cr7s94ukQ4oYsfqNWvQGYVgT436Y5vpQ1ApTunBFEkluIHCpBgooHOdmuGZmhUrZE1akfjWrFmDx+2xvH0DABsYRQYLFy3EMAyr04iAXkeglVL/1lrfGvj4pt6O01pfF4lgIra4A1v5egsnWhMg0Ae9avUaax5fCAssX7aM0dkm2Rb3Pwcdn+dm7tq1uN1unE6n1XGEiJiFCxeiHMq65eu6K4WWvS1s2rSJE044weo0gqOPQA/v8vGIo1xEEti8eTOGz9fZSmEFI7uYiv3l1NbWWpZBiGhpa2tj48aNHJ9n3frP3Z2Q78Xr9cmKOCKhaa1ZsHAB5lATYmTvLl2kweYv7EVs6LWA1lr/rsvHv+ztEp2YwmrB1omor8DRRbB4X7tWVuMQiW/VqlX4DIPj871WR+k0Ps+HwxbYXEKIBLVr1y5qqmvQpbFx5geAVKAAFixcYHUSESDL0omQrF69Bp2RDw7rTtuaGQUoR4r0QYuksHDhQlwpytINVLpLs8PEPC8LF8xH6xgqLoQIowUL/EVqTPQ/d2GWmOwt20tFRYXVUQRSQIsQeDweNmzcgM/C0WcAlA1fRhGrVq22NocQEWYYBosWzOekIW4cMfZX+tRCD/srDlBWVmZ1FCEiYsHCBf7dB9OsTnK44Ii4tHHEhhj70yxi0ZYtW/B6PBhZJVZHwcguYe/eMurr662OIkTEbNq0ifrGJk4pjJ3+56BTCvwtJcFROiESSW1tLVu3bLV298HeZILKUVJAxwgpoEWfVq/2j/ha2f8cFCzipY1DJLL58+djt8FJ+bHTvhE0JE0zNsfks0/nWR1FiLCLld0He2MUG6xZu4bW1laroyS9kApopdSXlVJjAh+XKKWeU0o9o5SyvqISEbdq1Wp//3OK9eezzIwClD2ls6gXItGYpsncj+dw4hAvGSmx+SJ+5tAOtm7bTnl5udVRhAirxYsXo9IV5FidpGe6RGMaJsuXy3rsVgt1BPphILh6991ACmACj0cilIgdHo+HDRs24IuB9g0AbDZ8mUWsXLXK6iRCRMSGDRuorqnlrKLYa98IOjOQ7ZNPPrE4iRDh4/F4WLpsKUZxDOw+2Jt8UKmKxYsXW50k6YVaQA/TWu9VSjmA/wdcDvwOOCdiyURM2LRpE16vByM7Rgpo/H3Q+/bulfWgRUL6+OOPSbUrTo3B/ueggjTNcbkGH8+ZbXUUIcJm7dq1uDvcMdu+ARzalXDxQkwzBvu0k0ioBXSTUqoI+CKwSWvdErg+JTKxRKxYuXIlKBUT/c9BRnYp4F8nV4hE4vV6mffJXKYUuEnrdZ/Y2HBOcQd7yvayfft2q6MIERZLly5F2WNo98HelEBTQ5M89ywWagH9ALAceAl4KHDdF4AtkQglYsfy5SvQGYWWrv/cnZmRj0pJ8xf3QiSQJUuW0NjUzHklbquj9OnMoV4cNpg1a5bVUYQIi0WLF2EWmBDjb151kX+EPDjhUVgjpAJaa30H8FXgC1rrVwNX7wd+Halgwnqtra1s2bIZbwy1bwCgbHgzi1m6bLls5iASyocffkBuGpwwJPZW3+guK1VzSoGH2R/NwuuNnd0ShRiIAwcOUL6vHF0cB68pacAQWLJUCmgrhbyMndZ6m9Z6Z7fP10cmlogFa9aswTRNjOxhVkc5gpEzjNqaalkFQCSMhoYGFi9ezDlFHdjjZIHR80o8NDY1y0iYiHvLli0DiI8CGjCLTDZt3ERzc7PVUZJWr3+mlVL7lFJ7+7pEM6yIrmXLlqHsKZhZRVZHOYKR4y/qg3/0hIh3s2fPxjDMuGjfCDoh30uO0z9yLkQ8W7ZsGSpDQZbVSUKjizVaa2lltNDRxjl+CvwscLkfaARuxt+2cTNQD0yLdEBhncVLluDNKgab3eooR9Bp2eDKYenSpVZHEWLQtNbMeH8643IMRmTGz8x6hw3OLe5g0aLFsiqOiFs+n48VK1dgDI3h5eu6GwIqRbFixQqrkyStXgtorfVnwQvwC+BrWusntNaztdZPAN8AfhmlnCLKysvLqTxwACNnhNVReuXNHsaq1atxu+NnxE6InmzevJk9ZXv5YkmH1VH67YulbkzTZPZsWdJOxKetW7fS3tYOsXeytXc2MAtMli6XQSSrhNppVwq0dLuuBYi95lgRFsGRXSN3uMVJemfkjsDjdrNu3TqrowgxKDNnziTVrjirOHbXfu5NaYbJcbkGM96fLpN6RVwKjuLqofH1+6uLNFUHqqioqLA6SlIKtYCeDkxXSl2olJqolLoIeCdwvUhACxcuAleuv1UiRhlZJSibg0WLFlkdRYgBa2tr4+M5szljaAfpMb58Vm++WNrBvvL98mZWxKXly5ej8hTEzmqtIQkuZydtHNYItYD+LbAYeBRYBTwCLA1cLxJMW1sbq9esxhvD7RsA2B14s0tYsGChjHyJuPXJJ5/Q3uHmgmHx24p0ZpEHV4ri/ffftzqKEP3S3t7Oxo0b/f3P8SYLbOk2KaAt0mcBrZSyA48B12utx2mtXYF//6G1bo98RBFty5cvx/D58OXFeAENGLkjqaqqZM+ePVZHEWJA3p/+HsMyNcfmxOELeECaHc4paufTefNkWS0RV9atW4dhGJ2juXFFga/QPwFStvWOvj4LaK21AVwEyE8nSSxevBiV4sTMjJ3tu3tj5I0EYOHChRYnEaL/duzYweYtW/lyaTsqXmb/9+KCYR48Xi8fffSR1VGECNmKFSv823cXWJ1kgIqgpblFtvW2QKgtHPcCNyqlUiMZRljPMAzmL1iAN3s42GJ/NwedmoHOLGTBggVWRxGi395++21S7YpzS+Jv8mB3o7IMxuUYvPPWWzIaJuLGsuXL0PkaYm+11pAEJz5KG0f0hVoh/QH4G9DUfYOVCGYTFtiwYQPNTU348kZZHSVk3tyRbN68WdahFXGlqamJObM/4gvFHWSmxOHp4x5cNLyDffv3y4u5iAs1NTXs3rUbsyiO3/C5QOUqli6T5eyiLdQC+qfAV4H/x+EbrPwsQrmERebPnw82e0wvX9edL28kWmtZjUPElZkzZ+L2eLloRPyt/dybM4s85DjhrTfftDqKEH3qXL4uTrbv7o0x1GD9uvW0t8u0tGgKqYDuuqlK90ukA4ro0Vrz+fz5GNklYI+fbh3tGgJp2Xw+f77VUYQIic/n4+233mRCni+udh7si8MGF5S2s2TpEvbulROUIrYtW7YMW5oNcqxOMji6WGMYBmvWrLE6SlIJqYBWSqUopW5USu1SSnUE/pWe6ASze/duKg8cwJcbP+0bACiFN3ckK1eupK2tzeo0QvRp1qxZVB2s5pujEm/E6Ksj3KTYFM8//7zVUYTolc/nY/GSxfiKfPGzfXdvCkA5lJyFjbJQWzjuxN/C8VvgpMC/FwB3RCiXsEBwIl5wZYt44ssbhc/rZdmyZVZHEeKofD4fLzz/HGNzTE7K91kdJ+xyUjVfGdbOx3PmUF5ebnUcIXq0YcMGWlta0aXx3b4BgB3MIpP5C+bLnghRFGoBfTHwba31bK31Vq31bOB7wA8jF01E2+fz56Mzh6JTM6yO0m9mVhEqJU1W4xAxb86cORyorOK7o9vifum63nxjVAd2GzIKLWLWwoULUTYFRVYnCQ9doqmrrWPbtm1WR0kaoRbQvf2ZT9A//8mnpqaGbVu34s2Nv9FnAJQNT84IFi5chM+XeKN6IjF0dHTwzNNPMTrbZEqB1+o4EZPr1FxQ2s7s2bPZtWuX1XGEOIzWmvkL5mMONSHF6jThoUv8I8+yJ0L0hFpAvwG8r5T6f0qpiUqprwHvAq9HLJmIqmDvlC8O2zeCjLyRtLa2sGHDBqujCNGjF198kcqqg/zk2NaEHX0O+u6YDjIcmnvvuVtOK4uYsnPnTir2VyRG+0ZQGlAIn8z7xOokSSPUAvpq4GPgIWAl8AAwD/h7hHKJKFu0aBGkZaFdeVZHGTAjZxjYbDKRQsSkffv28crLL/GFYjcT8xL/LElWquaH41pYu249s2fPtjqOEJ3mzp0LCvTwBCqgAXO4yd6yvXLWJ0pCXcbOo7W+Tmt9jNY6XWt9rNb6Wq21O9IBReS53W6Wr1iBN2cEcT0sZk/FyCphvvRBixijtea+e+/FoUwuOTbxVt7ozRdLPRyTY/Dwgw/Q3NxsdRwh0Foz5+M56CINTqvThJcerkEF3iCIiAt1Gbt3lFJXKaVOinQgEX2rVq3C6/FgxGv/cxe+3JHsLy9n3759VkcRotN7773H8hUruHhsK7nOxBr1Ohqbgl+Mb6WxqYm7754qrRzCcps3b+Zg1UH0iAT8XUwDhuJ/gyDPtYgLtYXjfeAU4D2lVJ1SarpS6i9KqdMjmE1EyeLFi1H2FIzsYqujDJqROwKApUtlW1MRG7Zt28YDD9zPSflevjo8+U7ajc42+MG4Nj75ZB7Tp0+3Oo5IcjNnzkQ5FHpYYhaY5iiTygOVrF271uooCS/UFo6ntdaXaq1HA1OA9cB1wJIIZhNRoLVm8eIleLOKweawOs6g6bRscOWwZIn8agrrtba2cv1115Jl9/Hbya3Y4rhDajC+OcrNifk+7r9/Gtu3b7c6jkhSbW1tzJ4zG2O4kTCrb3Snh2lUquL999+3OkrCC7WFY6JS6gql1MvAfOBC4DHg2/15MKWUXSm1Wik1I/D5GKXUUqXUDqXUa7KzYfTt27ePqqrKzpHbRODNGc7q1Wtwu5NvtE/EDq01d911JwcOHOB/JzeTlZqYI16hsCn47eQWMu0+rr/uWlpaWqyOJJLQvHnzcHe40WMS+LnoAGOEwaeffirzDiIs1BaOjcBf8a/EMUlrfYbW+mqt9cx+Pt5VwOYun98B3Ku1PgaoBy7r5/2JQQqO1CZSAW3kjMDr9bB69Wqro4gk9vTTT/PJJ/P4wbg2JiTBqht9yU7V/H5yMwcqKrj2mmvwehN3HWwRm6a/Px2VrSDf6iSRpcdovF6vrH4TYaEW0D8DPsFfRK9QSj2ulPqJUirkqkspNRz4BvBk4HOFfzvwNwOHPAd8N9T7E+GxeMkSSM9FO7OsjhI2RnYxyu6QPmhhmRkzZvDcc8/xxVI33xolZ0KCJuT5uGxiKytXreKOO+6QiU4iajZt2sTmTZsxxhqJvwVcHjAE3njzDUzTtDpNwgq1B/olrfUVWutJwJeAauBhYE8/Hus+/OtJB3+a+UCD1jo4NFMODOvH/YlBam9vZ82aNXizh1sdJbxsDrxZJSxavNjqJCIJLVmyhKlTp3JCvo9fTkjc7boH6vxSD/891r9L4ZNPPml1HJEkXn/9dVSqQo9Ojjdt5rEmFfsrZD5QBIXaAz1FKfUnpdS7+Ns5LgVmAL8L8fbfBA5qrVcOJKRS6nKl1Aql1Irq6uqB3IXowerVqzF8PnwJ1L4RZOQM50BFhSxnJ6Jq8+bNXHfttYzM9HHlCc04Qj3Hl2S+O6aDL5W6eeGFF3j33XetjiMSXFVVFZ9++inG6MSdPNidHq5R6YrXXnvN6igJK9Q/7+8AJwHTgTO01sO11j/RWj8e4u2/AHxbKbUHeBV/68Y0IFcpFVz6YTiwv6cba60f11qfprU+rbCwMMSHFH1ZsmQJyp6CmRX/y9d1F+zplnffIlq2b9/OX//yZ7Lsbv5yUhOu+F/UJmKUgl9OaGNKgZd7772HDz/80OpIIoG9/fbbmNpEH5Mco88A2MAYZ7B69WpZ+SZCQm3hGK21/kVgObud/X0QrfU/A0X3aOB/gE+01j/Bvx34DwKHXQq819/7FgOjtWbBgoV4s0vBZrc6TtjptGxIz2WhbOstomD37t38+U9/JNVo5V9TGslLos1SBspugz+c0MLkPB933H677J4mIqKpqYm333kbc7gJGVaniS49VqNSFC+88ILVURKS1ScY/w78WSm1A39P9FMW50ka27dvp6amGiNvlNVRIsaTM5I1a9bIUj4iovbt28ef/ngVytPMP6c0UuiSSTuhSrXDH09q5thcHzfffDPz58+3OpJIMG+++aZ/6bqJSfimNtU/Cv3ZZ59RVlZmdZqEE/UCWmv9qdb6m4GPdwWWxDtGa32x1lqmq0fJwoULARKy/znIyBuJaRgsW7bM6igiQVVUVPDHq67E19bAP09upDhdiuf+SrPDX09qYmyWl+uvu47FMvlXhElrayuvv/E6ulRDjtVprKGP02CHF1980eooCcfqEWhhkc/nz8fMKoIUl9VRIsbMHIpKdcmoloiIyspKrvrD/9HeVMffT25iWKYUzwPlcsDfTm5meIaXa675t7zpFWHxzjvv0NbahjkpiZ+bTjDGGsyZM4f9+3ucZiYGqNcCWim1pMvH10cnjoiGffv2sXPHDrx5o62OElnKhid3FAsXLqKjo8PqNCKBVFVVcdWVf6CloZa/n9zIqCzD6khxLyNF848pTZSkefjXP//JypUDWrRJCACam5t58aUX0SXavy5yEtPHabRN8/TTT1sdJaEcbQT6OKVUWuDjv0QjjIiO4GQdI3+sxUkiz5c/Fre7o7NlRYjBqqmp4Y9XXUlD7UGuPrmRMdlSPIdLZormn1OaGJrm4R9/v5o1a9ZYHUnEqddee80/+nx8Eo8+B7n8vdAff/wxu3fvtjpNwjhaAf0esE0p9TngUkp93tMlSjlFmGitmT1nDmZ2CTo18ackm1nFKGcGH38sM/zF4DU0NPDHq66k9mAlV5/UyLgcKZ7DLStV88+TG8lP9XD13/7Gpk2brI4k4kx9fT2vvf6af+WNXKvTxAY9XkMKsnlRGPVaQGutfwn8GHgc8OFfIaOni4gj27dvp3zfPrxDEn/0GfC3ceSNYcnSJTQ1NVmdRsSxtrY2rv7bXzlQsZ+/nNTEsblSPEdKjlPzzymNZNvd/O2vf2HPnj1WRxJx5MUXX8TtdqMnJ+HKG71xgnGMwfz58+VNaZgcdRKh1nqB1vpF4Pda6+d6ukQppwiTGTNmoGwOfPljrI4SNb6CYzF8Pj766COro4g45fF4+Pe//8W2bdv4v+ObmZjnszpSwstzav5+ciM2byt/+fOfqKqqsjqSiAP79+/3b5wy2oRsq9PEFn2cRqUpHnr4IbSWNxeDFepGKk8rpb6klHpaKfVR4N8vRzqcCK+2tjZmffQRniFjwJHW9w0ShJmRj84cyrvvvSd/NES/GYbBzTffzMqVq/j1xFZOLfRaHSlpFKWbXH1yE62Ndfz5T3+koaHB6kgixj3++OOYypTR556kgDHJYP269SxYsMDqNHEvpAJaKfVr4HWgEngbOAC8opT6TQSziTD75JNP6Ghvxzd0vNVRos5TOJ59e/eybt06q6OIOPPII4/w2Wef8eNj2zi/1GN1nKQzKsvgzyc2UXmggn/+4x94PPIzED3buHEj8+bNwzjOgMRdoXVQ9BiNylY88sgj+HxyJm0wQl0H+mrgQq31v7TWj2mt/w1cFLhexAGtNe+8+y6k52FmFlkdJ+p8+WNRjlTee092ixeh+/DDD3n99de5cHgH/zVK9nmyyoQ8H7+d1MzGTZuYOnWqnEkSR9Ba88CDD6Bcyj9hTvTMBr4TfJSXlzN9+nSr08S1UAvofKB71/lWYEh444hIWbt2Ldu3bcM9dCIoZXWc6LOn4Ck4jk/mzePgwYNWpxFxYP369Uy9604mD/Hx0+ParY6T9M4s8vK9Me3MmjWLN954w+o4IsZ8/PHHbNq4CWOyAQ6r08S4EmAoPPnUkzK5fhBCLaAXAPcopdIBlFIZwF3AokgFE+H16quvolJc+AqOszqKZbzFkzFNk7feesvqKCLGVVVVcc2//8WQVB9/OKEFu+zZGhO+N7aD0ws9PPzQQ7JboejU3t7OQw8/hBqi0KNl9LlPCoyTDVpaWnjqKVlMbaBCfVn4LXAS0KiUqgIaAp9fEaFcIozKyspYtGgR7qETwJ68b821MwvfkDG8++57tLa2Wh1HxCifz8eNN1xPR0sTfz6xicwUeUGOFTYFV0xuZXimwY03XC9nkwTgX7aurrYO30k+SMITrAOSA+ZYk3fffZedO3danSYuhboKxwGt9fnAGOBbwBit9Re11hURTSfC4tVXX0XZHHiLJlkdxXLe4hNob2+T3i/Rq2effZYNGzfxywnNDMuUXcxiTZoD/nB8M56ONm65+SYMQ9bjTmYVFRW88uormCNNKLA6TXzRkzWkwn3T7pN5BQPQrxOTWutyrfUyrXV5pAKJ8KqqqmLWrFl4Co6FFJmWbGYWYuSU8sqrr+J2y6QwcbhVq1bxwgvPc36Jm3OKZbm6WFWSYXLpcS2sWbuOF1980eo4wiJaa+69714MDPSJUgD2mxOMyQZr16xl7lzZrbe/pLMvwb388ssYpsZbeqLVUWKGp/RkGurrmTFjhtVRRAxpaGjg5ptupDhd8/MJbVbHEX04r8TDOcVunnn6aVmeMkktWLCApUuWYkySZesGSo/VMATuf+B+WlparI4TV6SATmA1NTW8P2MG3oJj0M4sq+PEDDOrBDOrmBdfeknWlBWd7r33Xhoa6vn95GbS7FanCZ8Xtrooa7ZT1mznlhWZvLA1MSoNpeAXE9oocGluu+Vm2trkTU8yaW9v59777kXlKvQxMvo8YAqMKQYN9Q0yobCf+iyglVI2pdQFSqnUaAQS4fPqq6/i8/nwlp5sdZTYohSe0pOpralh1qxZVqcRMWDu3LnMmzeP749pZ3R2YvXUljXbaTdstBs2tjSkUNacOO8O0h1w+cRmDlRW8cgjj1gdR0TRs88+S011Db4pPhkKHKwhYI4zefvtt9m6davVaeJGn792WmsTeE9rLUN1caS2tpZ33n0Xb/4x6LRsq+PEHCNnGDpzKM89/zxer/S6JrPa2lruuXsq43JMvjmqw+o4op8m5Pn4fyM7eO+991i+fLnVcUQUbN++nddeew1ztEwcDBd9vIY0uOPOO2SHwhCF+r7tc6XUWRFNIsLqlVdewev14h12stVRYpNSuIdNofrgQRmFTmJaa6ZOvYuO9jaumNQs6z3HqR+Oa6ckQ3P7f26TPs4EZxgGd9x5BzpVo0+S1o2wSQXfST52bN8heyWEKNSXizLgQ6XUs0qpm5VSNwUvkQwnBqaurq7L6HOO1XFilpEzHJ05lGefk1HoZDVr1iwWLlzExWNbKc2QJeviVaodrpjUTG1tLQ888IDVcUQEvfXWW2zbug3jJANivLFUrVH+XTMawPapzf95LBsOukTzxJNPUFEhqxT3JdQC2gW8C2hgODCiy0XEmFdffVVGn0PROQpdxUcffWR1GhFlVVVV3D/tPsbnGXxtpCxpGO+OyTH45qh2PvzwQxYtkk1yE9GBAwd44skn0CUaPSL2R59Vg0J5A5dqhWqI8QJagXmKidf0MnXqVFkbug+hbqTyy94ukQ4o+qehoYF33nkX35CxMvocAv8odCHPPf+C9H0lEa01d95xBz53B5dPbMEW469rIjTfG9vBiCyTO++4ncbGRqvjiDDSWnPnnXfiMTyYU0zZcTBS0sE43mDFihXS3tiHkDv+lFITlFLXKqUeDHw+XikliwvHmDfeeAO3241HVt4IjVK4S06mqvKALCSfRN577z2Wr1jBJce2UJQurRuJIsUGv53UQmNDA/fee4+MoCWQDz74gJUrV2KcYECG1WkSmx6noQCm3T+Nmpoaq+PErJAKaKXUxcB8YBjw88DVWcA9EcolBqC5uZk333wL35DR6PQ8q+PEDSNvJDojn+eef162BU4Ce/bs4aEHH+CEfB9fGSaLCyWaUVkG3x/bziefzJPWrARRXV3N/Q/cD4WBjT9EZCkwTjNo72hn6t3SytGbUEegbwK+qrX+LRCsMNYCJ0UklRiQt99+m/b2tphb9zm1bDG2tlpsbbWkbZpBatliqyMdTincJSdRvm8fn3/+udVpRAS53W5uvOEGnMrLFZNaUHIaOCF9a3QHE/J83HvP3ZSXl1sdRwyC1pqpd0+lw9OBcZohrRvRkuXf5nvRwkVydrYXoRbQQ4HgXqm6y7/ytiRGuN1uXn/jDYzcEZgZ+VbHOYyttRZleFGGF3tzJbbWWqsjHcEYMhpcubz44kvybjuBPf744+zctYvfTGgh1yk/50RlU/C7yS3YDA833XiDzG+IY7NmzWLxosUYkw3ItDpNctHHaciHe+69h9ra2HvdtlqoBfRK4GfdrvsfYFl444iB+vDDD2luasJTIm3pA6JsuIuPZ/v2baxatcrqNCICFi1axBtvvMGFwzuYUijLFia6/DTNZROa2bJ1G0888YTVccQAHDx4kPum3edv3ThW3vBGnQLjdIPWtlZZlaMHoRbQVwK3KKU+AzKUUh8BNwN/ilgyETKfz8dLL7+CzhyKmVVsdZy45Ss4BpWazksvvWx1FBFmFRUV3HLzTYzKMrnk2Har44goOaPIywXD3LzyyivMnz/f6jiiH7TW3HHHHdK6YbUs/6ocCxcuZPbs2VaniSmhLmO3BZgAPARcAzwDnKC13h7BbCJE8+fPp6ryAO6SE5GmzkGwOXAPncSKFcvZvl1+tROFx+PhuuuuxfS0c9UJzaTarU4koumnx7UxJtvktltvkc0h4siMGTNYvny5f9UNad2wlD7WvyrHvffdy8GDB62OEzNCXsZOa90GLAQ+BeZrrWW/1Bjx2muvgSsbI2+k1VHinrdoIsqewhtvvGF1FBEm999/P9u2beeKSc0MlSXrkk6qHa48oRl8HVx7zb9xu2XTnFhXUVHhX3VjaGBJNWGtQCtHu7ud/9z+H2nlCAh1GbuRSqn5wB5gJrBHKTVfKTUqkuFE3zZt2sSmTZtwD50MKuT3Q6I3Diee/GOY8/HHMmkiAXz44YdMnz6db47q4FTpe05ahS6T305qYvuOndx9991SAMQwwzC45dZb8JgejNOldSNmZIJxosHKFSt59913rU4TE0KtuJ7DP5EwV2s9FMgDVgSuFxZ6/fXXUQ4nvsLjrI6SMLzFx2P4fLz33ntWRxGDsGnTJqZOvYvJQ3xcPE76npPdlAIf3xvTzqxZs3j77betjiN68frrr7Nh/QaMkw1ItzqN6EqP1VAMDz70IPv27bM6juVCLaBPBf6mtW4FCLRv/D1wvbDIwYMH+fTTT/EUHAf2FKvjJAztysHIHcHbb78jp3vjVG1tLdf8+1/kpvj4v+NbsMvJGYF/q+9TCrw88MADrF692uo4opudO3fy+BOPo4dp9Eg5SxBzAhus+JSPm26+KemXhwz1ZWUJcEa3604DYmxHjOTyzjvvYGqNt3iS1VESjqf4eJqaGvnkk0+sjiL6yev1cu2119DUUMcfT2giK1VeiIWfTcFvj2+hON3gumuvobKy0upIIsDtdnPDjTdgppiYp5rSuhGrXGBMMdi6ZSvPPZfcTQi9FtBKqZuCF2An8IFS6mWl1B1KqZeBD4Ad0QoqDud2u3n3ven4ckeinVlWx0k4ZnYppOfx+htvSL9knJk2bRobNmzk8oktjMqSrdnF4dId8McTmvC0t/Dvf/2Tjo4OqyMJ4JFHHqFsTxm+03zgtDqNOBo9QmOOMnn++edZv3691XEsc7QR6BFdLmnA24Ab/66EbuCdwPXCAnPmzKG1pRlv8fFWR0lMSuEumszOHTtYt25d38eLmPDee+8xffp0vjW6nbOKZdKg6FlphsnvJzexY8dObr/9dnmTbLHFixfz9ttvYx5rgmxlEBf0FA0ZcONNN9LSkpyLsjl6+4LW+pfRDCJCp7XmtddfR2fky8YpEeTLP4a08hW88cYbnHTSSVbHEX1Yu3Yt9913LycVeLl4nIwqiqM7ucDHxce08fonn3Dcccfx4x//2OpISamuro7b/nMbKlehT5A3MnEjBXxn+Dg47yD33HMP1157LSrJ9qEIeWqNUipdKXWiUuqcrpdIhhM9W758OWV79uApmiwbp0SS3YG7cDzz58+XDRhiXFVVFdde82+Gphn8/vgWbPK0ECH41ig3ZxV5eOyxR1myZInVcZKOaZrcfMvNNDU34TvDB7LJUXzJB3OSyccff8ysWbOsThN1oa4D/XOgEvgEeK3L5dXIRRO9efXV11Cp6fjyx1kdJeH5iiahlZKNVWKY2+3mmn//C3dbM386sYn0Xs+rCXE4peA3k1oZmWly0403sH//fqsjJZVXXnmFlStW+pesy7E6jRgIPVHDULj7nrspKyuzOk5UhToCfSfw31rrAq31iC4X2fouynbu3MmKFctxD50ENnm7Hmk6NQPvkHHMmDGT5uZmq+OIHkybNo2t27bz24nNlGbIToOif5x2uOrEZrS3nWv+/W+ZVBglGzdu5IknnsAcbqLHSOtG3FJgnGHgVV6uu/66pFr6NdQC2oN/C29hsddeew1lT8FbNMHqKEnDV3w8bneHbKwSg2bMmMGMGTP49uh2Th0qkwbFwAx1mfzvpCZ27drF1KlTZVJhhDU3N3P9DdejXRp9qpYl6+KdC3yn+di9azcPPfSQ1WmiJtQC+lrgHqVUQSTDiKM7cOAAs+fM8W+c4pAFUKLFzMjHyBnOq6+9LqNTMWTr1q3ce+89HJ/v4wcyaVAM0kkFPr4/tp3Zs2fLVsURpLXmtttu42D1QXxn+iDV6kQiLErAPM7k3XffZe7cuVaniYpQC+htwLeBKqWUEbiYSilZZDWKXnnlFf/GKSUnWh0l6XiGnUxTYwMzZsywOooA2trauPGG68ly+Pj9ZJk0KMLjO2M6OCnfy4MPPsDOnTutjpOQ3njjDRYuXIh5ggn5VqcR4aRP0JAPd9x5R1Js9R1qAf0C8DxwEnBc4HJs4F8RBTU1Nbw/YwbeguPQzgyr4yQdM6sYM7uEF196CY/HY3WcpDdt2jT276/gd5OaZadBETY2BZdPbiXd5uPGG65Pqn7OaNi4cSMPP/Kwf6vuY+V5m3BsYJxl4DbdXHvdtQn//Am1gM4HrtNab9Ba7+x6iWQ4ccjLL7+MYRgy+mwhd+nJ1NXW8sEHH1gdJanNnTuXDz/8kG+PaWdins/qOCLB5KRqfjupmT1le5OqnzPSGhsbufa6a9EujXmabNWdsNLBd7qPXTt3cf/991udJqJCLaCfAX4WySCid5WVlbzz7rt4C45Fp2VbHSdpmdmlmFnFPPPMs9ILbZGDBw8y9a47OTbX4Ptj5GcgIuOEfB//NaqDd999l0WLFlkdJ+6Zpsmtt95KbV2t9D0ngxIwx5u8//77zJ492+o0ERNqAX0G8KRSaqtS6vOul0iGE35PP/00pqnxDjvF6ijJTSncI06jvr6ON9980+o0SUdrzd13343P08HvJrVgD3kbKCH674fj2hmRZTL1rjtpbW21Ok5ce/XVV1myZAnGiQYMsTqNiAZ9vIZCuGvqXQm7PnSoL0FPAL8BbgOe6nYREbRr1y4++ugj3EWT0M5Mq+MkPTOrGCN3JC+8+BJNTU1Wx0kq8+bNY/Hixfz3mFaGpst6zyKyHDb49YQWamvreOyxx6yOE7fWrVvH448/7l/veZz0PScNGxhnGnjwcM211yTkWduQCmit9XO9XSIdMNk99thjYE/BW3KS1VFEgHvEabS3tfH8889bHSVpNDU1Me3eexiTbfL/RiT2xBQRO8blGFw0ooP33nuX9evXWx0n7jQ0NHDd9dehMzT6NFnvOem4/P3QZXvKmDZtmtVpwi7Urbx/1dsl0gGT2dKlS1m8eDHukpMgRdZ9jhU6fQjewmN586232Lt3r9VxksJjjz1GY1MTv54orRsiui4e105+Gtx15x34fDJpNVRaa26/43bq6+v9fc8pVicSligGc4LJzJkzE2596FBfin7W7fI34FFkYmHEeL1e7pt2P7hy8RYfb3Uc0Y1nxGloZef++++XXcsibNeuXcycMYOvDu9gVJYsPS+iK80BPzm2hT1le5k5c6bVceLGO++8w6KFizBOMCDP6jTCSnqyf33oO++6kwMHDlgdJ2xCbeH4crfLROC3wIrIxkteb731FvvL99Ex8kyw2a2OI7pLSaejdArLli1j8eLFVqdJaI888jAuB3xPVt0QFjmt0Mv4PIOnn3yCtrY2q+PEvJ07d/LgQw+iS2S9Z0FnP3SHr4Mbb7wxYc7kDOZk6LPAZWHKIbqoqanh6WeewcgdgZE7wuo4ohe+okmQnst90+5P+AXjrbJ8+XKWLl3Gd0a3yoYpwjJKwY+PaaW+sYmXX37Z6jgxzePxcMONN2A4DFnvWRySAcYpBps2bUqY+UOh9kDbul0ygcuBhoimS1LTpk3D7fHiHnWW1VHE0djstI88m8oDFQnzByGWaK157NFHKEyHC2XioLDYuByDs4s8vPbqK9TV1VkdJ2Y9++yzlO0pw3eqD2TqjuhCj9CYI02ef+F5tm/fbnWcQQt1BNoHeLtcGoF/Ab+LUK6ktWjRIj777DPcJSej03KsjiP6YOYMw1twDC+9/DK7d++2Ok5CWbJkCdu27+C7o1pJkYmDIgZ8f2w7Xq+X1157zeooMWnz5s289NJLmKNNKLE6jYhFeopGp2puufUWvF6v1XEGJdSXpTHA2C6XIq31SK31RxFLloTa2tqYevc9kJ6Ht+QEq+OIEHlGnom2pXDHnXdimrI+cThorXnu2WcpcMEXSjxWxxECgJIMkzOHenjn7bdpbGy0Ok5M8Xg83Paf28AF+iRptxK9SAXfKT5279rNCy+8YHWaQQl1EmFZt0tNpIMlo6eeeoqa6oO0jz5XJg7GkxQXHSPOYNPGjUyfPt3qNAlh1apVbNq8mW+NasUho8+D0u5TuFwufvCDH+ByuWj3SVPqYHxnTDsdbrfsRtrNiy++6G/dOEW26hZ9KKWzlWPXrl1Wpxmwo740KaXmKaU+OcolsRb1s9D69et548038Q6diJlVZHUc0U++gmMxckp56OGHqaqqsjpO3HvppRfJTYPzZPR50Np8im984xtceeWVfOMb36BNCuhBGZ5pcnqhh7fefIP29nar48SEiooKXnzpRcyR0rohQqNP1miH5t777o3bpWD7Gtt5EXiph8unwInA2ZEMlyzcbje3/ed2cGbiGXm61XHEQCiFe/R5eLwGd951V9z+QYgFu3fvZsWKlVw4rJ1UOREzaOkOzcyZM7n//vuZOXMm6Q753Rysr4/qoKW1jY8+ki5GgAceeABDG+gT5XdLhMgJxmSDtWvWMm/ePKvTDMhRC2it9VNdL8C7wETgL8DbwHGRj5j4nn32Wf+az6O+AHY59xWvdFoWHcNPY/myZcyaNcvqOHHrzTffJMWuuGCYrLwRDi6Hpr29nTfffJP29nZcUkAP2rE5BmOyTd584/Wkn/ewbNkyFi5ciDHRAJfVaUQ80WM1Kk/xwIMPxOXZnFCXsctWSt0M7ACKgFO01pdrrcsjmi4JbNmyhVdeeQVv4XEYucOtjiMGyVc0CTOrmGn3309NjUwV6K+mpiZmfzSLc4o6ZN1nEbOUgv83op29+8pZsSJ59xPz+XzcN+0+VJaSDVNE/ynwneyjtqaWV155xeo0/dZXD7RLKfVPYBf+kedztdY/01rvjEq6BOfxeLj11tvQKS48I8+0Oo4IB6XoGHse7e1uaeUYgFmzZuH2eLlI1n0WMe7MIg/ZTnj33XetjmKZOXPmUL6vHN/xPpB2KzEQBaCHa1597VUaGhqsTtMvfY1A7wH+DNwJPAwUKaUu6HqJdMBE9uyzz1JWtse/6obDaXUcESY6LYeO4aeyZPFi6ZHsB601H8ycwdgck1FZhtVxhDiqFBucW9zB4sWLqK+vtzpO1Hm9Xp5+5mnIA4ZZnUbEM3OySUd7R9yNQvdVQLcDLfg3THmqh8uTEU2XwIILzvtbN2S77kTjK56MmVXEfdOmUV1dbXWcuLBt2zZ27d7D+SXx1wsnktMXS9wYhsns2bOtjhJ1H3zwAVWVVRiTDdmuWwxOtn9ZuzffepPa2lqr04Ssr0mEo7XWY45yGRutoIkk2LpBarq0biQqZaNj7Pm0d7i5805p5QjFBx98QIpdcXZRfO9OJZLHsEyTcTkGH8yckVTPca/Xy7PPPQsFQLHVaUQi0JM0Xq83rkahZYsCCzz33HPs3VsmrRsJTqfl0DHsNJYuXSKtHH0wTZNP533ClHw3GSnJU4iI+HdusZvde8ooKyuzOkrUfPzxx9TW1GJMkNFnESZZYI4weW/6ezQ3N1udJiRSQEfZli1bpHUjifhbOYq5b9o0WZXjKDZv3kx9QyOnDZWNU0R8OaXQ/zu7cOFCi5NEh9aal195GZWjZPRZhJUer3F3uHnvvfesjhISKaCjyOPxcOttsupGUpFVOUKyYMEC7ApOyvdZHUWIfslP04zJNlkwf77VUaJi2bJllO0pwzhORp9FmOUCRfD6G6/j8cT+YIoU0FH0/PPPU7ZHVt1INl1X5ZgzZ47VcWLSwvmfMz7PJ+0bIi6dUuBm0+bNSbEax6uvvopKV+iR8lwV4WeMN2iob2Du3LlWR+mTFNBRsmPHDl586SW8BcdK60YSCq7Kce9906irq7M6Tkxpbm5mz959TM6TyYMiPk0e4kVrzcaNG62OElF79+5l5cqVGGMNqR5C4QWXy8UPfvADXC4XyJ+4vg0FlaN46+23rE7SJ3kKRIHP5+O2//wHbXfiGSWtG0lJ2egYcy5tbW3cd999VqeJKdu2bQNgTLa0b4j4NDLLQKlDv8uJ6r333kPZFHqMjD6HxAvf+MY3uPLKK/nGN74hBXQoFBhjDbZt3caWLVusTnNUUkBHwauvvsqO7dtpH3U2ONKsjiMsol15uEtP5tNPP+Xzzz+3Ok7M6CygZfMUEafS7FCaodm6davVUSKmo6ODmR/MxBhmgLyMhSYFZs6cyf3338/MmTMhxepA8UGP0iiHivldPqWAjrB9+/bx9DPP4BsyGmPIGKvjCIt5S05CZ+Qz9e6742apnkjbvn07+S7ISpVRLRG/xmR62LY1tkfMBmPevHm0tbahx8nzNGQp0N7ezptvvkl7e7sU0KFKAWOkwZw5c2L6dVIK6AjSWnPnXXdhYMMz6hyr44hYYLPRMeY8GhoaeOyxx6xOExPq6+sZkiqjzyK+5aWZNDY2JexKOzNnzkRlK//mKUJEmB7j31hl3rx5VkfplRTQEfThhx+yds0aOoafjk5NtzqOiBFmRgHeoslMnz6ddevWWR3Hcq0tzbgcptUxhBgUl0PjM4y4WH6rvyoqKli3bh3GSFm6TkRJnn8y4QcffGB1kl5JAR0h9fX1PPDgg5jZxfgKx1sdR8QYz/BTIS2LO+68MyFfcPujtaUFlyMxR+1E8nDZ/f+2tLRYGyQCZs2aBfh7U4WICgXGKINNmzaxd+9eq9P0SAroCHnwwQdpbWujY/QXQMlbdtGNPYWOUeewb+9eXnnlFavTWKq9ox2nTV6YRXxz2v2/w+3t7RYnCS+tNR/O+hCKADmRKqJIj9Sg4KOPPrI6So+kgI6ANWvWMGfOHDzFJ6JdeVbHETHKyB2Bb8gYnn/+BSoqKqyOY5mcnByavfImU8S3lsDvcE5OjsVJwmvbtm1UVVZhjpA2KxFlLqDQP4E1FucWSAEdZj6fj7vvvgfSsvCWnmx1nNhgeA5fTN5I7paFrjwjz8Jnau5/4AGro1imoLCIeo/D6hhCDEpdh400ZyqZmZlWRwmrzz//HBTo0tgrYETiM4eZlJeXU1ZWZnWUI0gBHWZvvvkmZWV76Bh5FtilKABQPs9hi8krnxTQQdqZgbt0CosWLmTRokVWx7FEYWEh9W671TGEGJQ6t42C/HxUgrXszft0HhQCTquTiGSkh/nfuH322WcWJzlSVApopdQIpdQ8pdQmpdRGpdRVgeuHKKXmKKW2B/6N636Hmpoannr6aYzcERi5I62OEzO0I/WwxeS1I9XqSDHFWzwZ0vO49977cLvdVseJuuLiYhrduvMUuBDxqLLdQVFJqdUxwmrfvn2U7yvHHCbtG8IiLiAfPvs8SQtowAf8RWs9CTgL+L1SahLwD2Cu1vpYYG7g87j11FNP4fZ4cI86SyYOdmVPPXwxebsU0Iex2WkfeRZVVZW8/fbbVqeJuilTpgCwsU7O2Ij41OBW7G22ceqpp1odJaxWr14NgC6S9g1hHbPIZOeOnTG3wk1UCmit9QGt9arAx83AZmAY8B3gucBhzwHfjUaeSNixYwcffPAB3qGT0GmJNYlERJ6ZMwwjdwTPPvscDQ0NVseJqokTJ5KR7mJdrWzTJeLT+sDv7umnn25xkvBau3YtNpcNEqutW8QZXajRWrN+/Xqroxwm6j3QSqnRwBRgKVCktT4Q+FIl/oVyerrN5UqpFUqpFdXV1dEJ2g9aax544AFwOPEMm2J1HBGn3CPPoL2jnaefftrqKFHlcDg47fQzWFfnJAYnWgvRp3W1KeTlZHPsscdaHSWsVq1ZhS/fJ5unCGsNAWz+N3SxJKoFtFIqE3gL+KPWuqnr17R/jZIeXz611o9rrU/TWp9WWFgYhaT9s2TJElavXk1H6RRwyEwLMTDalYd36ESmT58ekzOOI+kLX/gC9R3SxiHiT6tXsarWydlfOBebLXHm5R88eJDa6lr/BEIhrOQA8mDd+tjauTdqz3alVAr+4vklrXWw0bNKKVUS+HoJcDBaecJFa80TTz4Jadn4hk60Oo6Ic55hU9DKzjPPPGN1lKj68pe/zJC8XGbudVkdRYh++bjcidunufjii62OElbl5eUA6Gw5LSSsZ2aZlO8vtzrGYaK1CocCngI2a63v6fKl6cClgY8vBd6LRp5wmj9/Pju2b8ddejIk0OiDsEiKC3fRJD6ZN49du3ZZnSZqnE4nP7j4h6yvdVDWLEvaifjgMWB2uYszTj+dcePGWR0nrCorK/0fyO6DIhZkQENdQ0ytVBWtiu8LwM+AC5RSawKX/wJuBy5USm0Hvhr4PG6YpukffXbl4is4xuo4IkF4S05A2VOSrhf6O9/5Dq40JzPLpA1KxIcFB1JpdMMlP/6x1VHCrrKy0t/7LAW0iAUZ/n8OHoydRoVorcKxQGuttNYnaq1PDlw+0FrXaq2/orU+Vmv9Va11XTTyhMtnn31G2Z49/t5nJaPPIkwcabiLJvP555+zY8cOq9NETVZWFt//7x+wqNLJlnrphRaxrdmjeHN3BpMnTeSUU06xOk7YVVVV+VfgkJc2EQN0ur+VqPPMSAyQp8YAaa156eWXwZWDkT/G6jgiwXiLj0fZU3jllVesjhJVP//5zykuGspTWzLxGFanEaJ3L2930eqz8de/XZ1wuw+C/wyrVAgiZgSeYjqGlmqSp8cArV27lm1bt+IuOl5Gn0X4OZx4Cscz95NPqKqqsjpN1LhcLv76t6s50KqYvifN6jhC9GhDrYP5B5z8+Mc/Sbje5yC73Q6yAaGIFYG62W6PnTkyUvkN0Msvv4xKceErTKx1P0Xs8BZPxjRN3nzzTaujRNUZZ5zBhRdeyPtlLvY0xc4fSyEA2nzw9NYshg8r5ec//7nVcSLGbrf3srCsEBYIvJmTAjrO7d27lyVLluAeOhFs0qspIkM7s/ANGct706fT1tZmdZyo+sMf/sCQIfnctz6bJk/inR4X8cnU8PCGTGrdNv7xz3/hdCbuhNeUlBS0IRW0iBGBAjolJXZ2rJUCegCmT58Oyoa3aILVUUSC8xZNpqO9nTlz5lgdJapyc3O59bb/0ORL4f71mfjkVLKIAW/uTGNNTQpXXnkVJ554otVxIqq0tBTt0dBhdRIhQDX7B1JKS0stTnKIFND95Ha7mfnBB/jyRkGKrO8jIsvMLERn5PPOu+/F1OSJaJgwYQJX//3vbKl38NI22WBloEZlGbjsJi67yYRcL6OyZHbmQCypSmH6Hhff/OY3+e53v2t1nIgbO3as/4Omox8nRFQ0QnZONnl5eVYn6SQFdD/NmzeP1pYWvLLroIgGpfAUTmDXzh1s3rzZ6jRRd9FFF/GjH/2IOeVpfLQ3cU+XR9LPxrczKstgVJbBNae18LPx7VZHijvbGuw8sTmL4ydP4k9/+lNCrrrR3Zgx/tWlVGPi/19F7LM12WJuwq4U0P303vTp4MrFzC6xOopIEr6CcSh7ir91KAldccUVnHvuubywLZ1PylOtjiOSzK4mO3etzaGwqISbb7k1pnowI2nIkCFkZWdBvdVJRNIzQDUpxo4Za3WSw0gB3Q/l5eVs3LABT8GxkAQjECJG2FPxDBnDJ/Pm0dGRfA2JDoeDG264gbPOOpNntmQwv0KKaBEdZc127liTQ27+UO6bdj/5+flWR4oapRRnnnEm9kpZzk5Y7CBon+aMM86wOslhpIDuh9mzZwPgy5dtu0V0+QqOoaO9nQULFlgdxRKpqancfPMtnHLqKTyxOYMllckxCiiss7/Fxu2rs8nIGcJ90+5n6NChVkeKui996Utot4Zqq5OIZKbKFa50F6eeeqrVUQ4jBXSItNZ8+OEsjJxhaGeG1XFEkjGzSiAti1mzZlkdxTJOp5PbbvsPJ5xwAg9vzORzGYkWEbK7yc6tq3NIzczj3vvup6QkOVv2zjjjDFKdqaj9csZVWMQE+wE7537hXFJTY+tvvhTQIVq/fj1VVZUy+iysoRSeIeNYvnw5NTU1VqexjMvl4o4772LKKafw+KYMZuyRiYUivDbWObhtVQ6u7AKm3f8AI0aMsDqSZdLS0jj7rLOx77eDLN4irFAF2q354he/aHWSI0gBHaK5c+ei7A58Q0ZbHUUkKV/+OLTWfPbZZ1ZHsVR6ejp33HEnF1zwZV7dkc7L212YybXCn4iQZVUp3LUmi+LhI3n40ccYOXKk1ZEs953vfAfdoVF7ZRRaRJ99m50h+UM4++yzrY5yBCmgQ+Dz+Zj7yTy8OSPALr2Xwho6PQ8y8pnz8cdWR7Fcamoq1113Pd/73vf4oCyNxzely2YrYlA+Lk/lgfWZTJx8PA8+9DCFhYVWR4oJp556KmPHjcW+Tbb2FlFWDxyEH/3wRzG5+o0U0CFYs2YNTY0N+PJjawkVkXw8eWPYtHEjlZWVVkexnM1m449//COXXXYZCw44uXNNFi1eGSUT/WNqeGmbi2e3ZHD2OWdzzz33kpWVZXWsmKGU4seX/BjdpEH+7IgoUlsVaa40vvWtb1kdpUdSQIdg7ty5KEcqRm7y9sKJ2BB8Ezdv3jyLk8QGpRSXXnop//rXv9je5OSGFTkcaJU/ayI0bT64Z20mH+5N4/vf/z633HIrTqf01Xd3wQUXUFBYgH2zjEKLKGkGW7mNb3/r22RmZlqdpkfyStMHn8/HZ5/P97dv2BxWxxFJTqdlozMK+PTT5O6D7u5rX/sa9943jXZ7NjesyGVDnTxXxdFVt9u4aWUu6+qc/PnPf+aPf/wjDof83vTE4XDwq1/+CmqB/VanEcnAtt6G0+nkkksusTpKr6SA7sO6detoaW6SyYMiZnjzRrF58yaqq2Vx1q5OPPFEHnv8CQpLR3Dn6izmyq6FohfbGuxcvyKHBjODqVOn8t3vftfqSDHv61//OqNGj8KxwSEbq4jIqgG1X/HTn/w0pjcvkgK6D59//jnK7sDIGW51FCEA8A0ZA/h/N8XhSktLeeTRxzjjTP+uhU9tTscrL/aii7nlqdy6Kpus/BIefexxTjvtNKsjxQW73c7v//f36GaN2ilzDUSEaLCvs5OXn8ePfvQjq9MclRTQR6G15tPPPsObPUxW3xAxQ7tyIT2Pzz+fb3WUmJSRkcF//nM7P/nJT5i338ltq7Kpd8sLfrLzmvDU5nSe2ZLBqaedzuNPPCnL1PXTmWeeySmnnOLvhXZbnSb26VyNTglcCjU6VxrI+6L2KqiFy399OWlpaVbHOSopoI9i165d1NXWYuTKH1kRWzw5I1i3bi1tbW1WR4lJdrudK664ghtvvJF97S6uXZ7Ljka71bGERerdittWZTNvv5Of/OQn3HHHnbLSxgAopbjqqqtQPoVaJ29K+6JP1pAL5IL5JdP/ueidxz/6PH7CeL72ta9ZnaZPUkAfxdKlSwEwcqV9Q8QWI2c4hmGwatUqq6PEtC9/+cs88uhjuHKLuGVlNvP2S190stnRaOfa5bnsa3dxww03cMUVV2C3y5upgRozZgyX/M8l2PbYQKZhiDBS6xW44eq/XR0Xz1EpoI9iydKlkDEEnZphdRQhDmNmFaHsKZ1v8kTvxo0bx+NPPMnJp5zKU5ulLzpZaO3vd755ZTau3CIeefQxLrjgAqtjJYRLL72UouIiHKtkQqEIk1qw7bJx8cUXc+yxx1qdJiRSQPeio6OD9evW4ckeZnUUIY5ks+PNKmHJ0mVWJ4kLOTk53HXXVH76058yb7+TW1ZmU9shp6ATlceAJ7v0Oz/x5FOMGzfO6lgJIy0tjb/8+S/oJo3aLM8jMUgmOFY6yC/I51e/+pXVaUImBXQvduzYgWEYmJnFVkcRokdmVhFVlQdobGy0OkpcsNvtXH755dx8881UuNO5dnkum2S96IRT06G4eWUOn1U4ufTSS7njjjvJzs62OlbCOeuss/jqhV/FtsUGDVanEfFMbVLoRs3Vf7ua9PR0q+OETAroXmzduhUAM7PA4iRC9MzIKAQO/a6K0Hzxi1/kscefIHfocG5fncWHZU60zO1JCBvqHFy7LJcqbwa33XYbl112WVz0Usarq668ipzsHBwrpJVDDFA92LbY+NrXvsbZZ59tdZp+kQK6F1u2bEE509Ep8fNuSCQXM8O/wLwU0P03evRoHnv8Cb5w7nm8tD2dhzdk0GFYnUoMlNYwY4+TO1dnkV88gieefJJzzz3X6lgJLycnh6v/djW6XqO2SCuH6CcTHCsc5OXl8Yc//MHqNP0mBXQvtm7dhteVD0r+KIgY5XCiXDls377d6iRxKSMjg1tuuYXLL7+cJQdTuWlFDlVt8icx3nT44MENGby6I53zzv8ijz7+BCNGjLA6VtI477zz+MpXvuJfG7re6jQinqiNCt2g+fvVf4/LZSXl1aIXdfV1svqGiHk+h4u6ennVGiilFD/96U+5666p1Ossrluew9oa6YuOF5VtNm5Ykcvyg05+97vfcdNNN8VVD2Wi+NOf/kReXh6OZQ7wWZ1GxIVqf+vGf/3Xf3HOOedYnWZApIDugdaalpYWtN1pdRQhjko7nDKJMAzOOOMMnnjyKYpHjGXqmize250mfdExbk2Ng+uW59Kksph6991ccsklKDljaIns7GyuveZa/zbfssGK6IsHHMsdlJSWcOWVV1qdZsCkgO5Be3s7pmGAQwpoEdu0w0lTU5PVMRJCaWkpjzz6KF+98Ku8sdPF/esz6JDRtJijNby3O42712QxbNRYnnjyKU477TSrYyW9U0891b/Byk4bVFidRsQsDWqVQnUobrj+hrg+YyQFdA9aW1sB0PYUi5MI0Qd7Km2tsp13uKSlpXHNNdfy+9//npXVTm5YkSt90TGkwwf3r8/gjZ0uvnrhV3no4UcoKSmxOpYIuOyyyxg7bqx/VQ75syR6oHYrbPtsXPary5g4caLVcQZFXhl6kJ2djVIK5W23OkpCMDPy0fYUtD0FI6u4c/UIMXjK205uXp7VMRKKUoof/ehHTL37bhrJ5LrlOayrlb5oq1W12bhhZS4rq538/ve/55prriUtLc3qWKKL1NRUbr7pZtLsadgX20FWthFd1YF9jZ1TTzuVH//4x1anGTQpoHvgdDrJG5KPcjdbHSUheEadjZmej5meT8ekb+IZFV9rPcYym7uZ4cNKrY6RkE477bRAX/QY7lqdxUxZL9oyG2odXLcil0adydS77+ZHP/qR9DvHqBEjRvDvf/0b6kCtkZ+RCHCDY4mD/Px8rr/u+oRYn10K6F4MHzYMu1t6S0Vss3uaKS2VAjpSSktLefiRR/nil77EK9vTeWxjOh4ZVYsareGjvU7uXJPF0NKR0u8cJ84//3x+/OMfY9tlQ+2RIjrpabAvtWNz27j15lvJzc21OlFYSAHdi+HDh2HvaARTtlcSsUl52tCedoYNG2Z1lITmcrm48cYbueyyy1hQ6eTWVdnUu6UoiDSvCU9uTueFbemcc84XeOTRx+TNYhz59a9/zZQpU7CvskON1WmEldRaBVXw5z/9Oe77nruSAroX5513Htrbgb1hr9VRhOiRo8a/gYrsuBZ5SikuvfRSbrnlFvZ3pHPd8lx2Nsb/KchY1ehW/GdVNp9VOPn5z3/OLbfeGtez9ZORw+HgxhtvpLSkFMciB8gJ3aSktils2218//vf55vf/KbVccJKCuhenHXWWRQUFJJycLPVUYQ4kjZJrd7CySdPYeTIkVanSRrnn38+jzz6KM6codyyKofFlbJST7iVNdu5bkUuZW1p3HDDDfz617/GZpOXqniUm5vL3VPvJtuVjWOBA2ReflJR+xS2tTbOP/98/vCHPyTcvAX5q9QLu93Ot7/9LeyN+1Ed8tZZxBZ7437oaOa73/2O1VGSzrhx43j8iSeZOOl4HtqQyVs70zBlcmFYrDyYwk0rc1DpQ3jwoYe54IILrI4kBqm0tJSpd00l1Uj1F9FeqxOJqKgG+3I7x59wPNdee21CTBrsTgroo/jGN76BzWYjZf8qq6MIcYg2Sa1YQ3ZOLuedd57VaZJSbm4u99x7L1//+td5Z7eLB9dn4JbJhQOmNby/x8l96zIZM+5YHn/iScaPH291LBEm48eP59ZbbkU1K+wL7FJEJ7oacCxyMHzYcG7/z+04nYm5KZ0U0EdRWFjIz372M1JqduCo3mZ1HCEASNm3EltzFVf+4f9ISZEWAqukpqbyj3/8g9/97ncsr07llpU51HUk1inKaPCa8NimdF7bkc6XL/gyDzz4EAUFBVbHEmF2xhlncMP1N2Crt2H/3A4eqxOJiDgIjvkOSoeWcu8995KdnW11ooiRAroPv/jFLzjp5JNJK1uEaqu3Oo5IcvaGfaQeWMs3v/lNLrroIqvjJD2lFJdccgn/+c/tVHpcXL8il91NiXeqMlKaPP7JggsOOPnVr37F9dffkLCjVQK+9KUvcdutt2FvsuP4zAEdVicSYXUAHAscjBw+kocefIihQ4danSiipIDug91u5/rrriM7K5P0nZ+AIeeehDWUuxXXrs8YM2YsV111ldVxRBfnnHMODz/yKKnZhdy8ModlVXJmoC/7WmxcvyKXPa3+yYK/+MUvEm6SkTjSOeecw5133ImjzeEvomViYWIo97dtjBs7jgcfeJAhQ4ZYnSjipIAOQUFBATdcfz2qvYG07R+DT849iehS7hbSt80i1Q4333yTjNLFoODkwmPHT+T+9Zm8tztNdi7sxdoaBzetyMVMy+OBBx+UyYJJ5vTTT+eeu+8h1ZOK4xMHyMnd+KX9S9XZl9iZOGEi0+6bRk5OjtWpokIK6BCdeuqp/OMf/yClpZKMLTNQ7harI4kkYWutIWPTdFzaze3/+Y8sWxfD8vLyuG/aNC688ELe2OniUdm58DBaw6y9Tu5em8Xw0WN57PEnEmpjBRG6k046iYcefIj8jHwc8xyovXL2Ie4YoJYHlqo773zuuecesrKyrE4VNVJA98PXv/517p46FZfZQcam6dhaZXslEVn2+r2kb55Jfk4mjz76CKeeeqrVkUQfnE4n11xzDb/+9a9ZWOnktlXZNMrOhfhMeHpLOi9uS+cL557Hgw89nPA9kuLojjvuOJ568imOn3Q8tqU21DoFctYmPrSD/VM7tjIbl112GTfddFPSbXYkBXQ/nXrqqTz66CPk52SSvnkm9nrZqVBEhqNyE2nb53DM2DE8/tijjBkzxupIIkRKKX7+859z0003Ud7h8m8M0py8kwubPYo7Vmcxb7+Tn/3sZ9x88824XC6rY4kYkJeXx3333ce3v/1tbFtt/mXupEsyttWCY64DZ5uTW2+9lUsvvTQpNztKvv9xGIwZM4YnHn+MY8aNIW3bHFL3LJK+aBE2ytOKc/vHOMsWcc7ZZ/Pggw/Isl5x6ktf+hIPPPgQKn0IN63MYeXB5JtcWNFq44YVOexoSeOaa67hN7/5TVK+2IrepaSk8Ne//pW//OUv2KptOD52QLXVqcQRNKjNCvundopyi3j8sceTei8C+Ss2QPn5+Tz4wAP8939/n9SDm8nc8Bb2uj1WxxLxTJs4KjeRsf4tXM0VXH755dx6660yUhfnxo8fz+NPPMnYY47jvnWZTN/jTJrJhetqHdywIhdPSg7Tpt0vSy+Ko/rOd77DQw8+RFF2EfZP7f6WDplDEBtaAi0bG2x8+Ytf5qknn0r6s6JSQA+Cy+Xiqquu4pFHHmHMsGLStn+Mc9scmWAo+k211ZK+aQbOskVMOekEnnvuWX76058m5PanyaigoID7H3iQL19wAa/vSOeRBJ9cGJwsOHVNFiUjRvPYE09y/PHHWx1LxIHJkyfz7DPP+ncC3mrzr9LRaHWqJKZB7VY4PnbganNxzTXXcMMNNyTVZMHeSAEdBpMmTeLJJ5/gd7/7HeltlWRseAtH5QbQptXRRKwzfKTsXUb6hvfItnVwzTXXcO899zB8+HCrk4kwczqdXH/99fz6179mUaWTW1flUJ+Akwu9Jjy5+dBkwYcefoTi4mKrY4k4kp6ezt///nduu+02snQWjrkO1DaZYBh1brAtsmFbYePEySfy/HPPc9FFF8l67QFSQIeJw+Hgkksu4YXnn+f0U0/BWbaEjPVvY6/ZIYW0OJLpw1G5gcx1r5N6YB1f/9r/4+WXXpI/TgkuOLnw1ltvZX+Hi+uW57IrgXYubAzsLPhZhZNLL72Um2++Oelm5ovwOffcc3n+uec584wzsa21Yf/ULqPR0aBB7VE4PnLgOOjgf//3f5l23zSKioqsThZTlI6zZrzTTjtNr1ixwuoYR6W1Zv78+Tz51FPs2b0bXLl0lJ6MkT8WVHK+Z0nbNAOAjknftDiJxUwfjoNbSDuwDu1p48STTuI3v/41J510ktXJRJTt3LmTf/z9aupqa/jNxGbOKY7cLqe3rMgE4JrTItdeVtZs59512TQbKfzzX/+WzVFE2GitmTVrFg88+AAtrS2Yx5roSRocVifrm+1T/2u++aU4GUhrAvsqO1TDpMmTuPpvVzN27FirU1lGKbVSa31aT1+Lg1+/+KOU4vzzz+fcc8/l888/55lnn2X3zk+hYg0dpSdh5I9L2kI6aZk+HFVbSKsMFM4nn8yvfvlLpkyZYnUyYZFx48bxxJNPcc2//83D69ezr6Wdi8d1YIvDExDLqlJ4bFMWWbl5PHj7HYwfP97qSCKBKKX4+te/ztlnn82jjz7KBx98gCpX+Kb4oMTqdAnCCKywsdVOeno6//u3//X3ocuKOb2SEegoME2T+fPn8/Qzz7B71y5w5eIuOQlfQfIU0kk7Am0ERpwDhfPJJ0/hV7/6JSeffLLVyUSM8Hq93HvvvcyYMYMpBV5+d3wL6WEe2ojUCLSp4Z1dabyz28WkiRO59bbbyM/PD+tjCNHdmjVruGvqXezbuw89TGNOMSFGFyuKixHoKnCsdqCbNRdeeCH/93//R15entWpYsLRRqClgI4i0zRZsGABTz/9DLt27QRnJp6hE/EWjoeUNKvjRVSyFdDK3YKjahPOmm1obwdTpkzhl7+Uwln0TGvNO++8w/33309pusGfT2xiaHr4XnAjUUB3GPDYxgyWH0zla1/7Gn/9619JTU0N2/0LcTRer5dXX32VZ599Fp/2YUww0MdpiLEpBTFdQLeAbZ0NtV9ROqyUv/31b7LbbTdSQMcY0zRZtGgRb7zxBqtXr0bZHHjyx+EtnoROT8zRm6QooLXG1lxJSuVGHA1lKOC8887nhz+8mBNPPNHqdCIOrFy5kuuuvQbtaeOqE5qYmOcLy/2Gu4Cu7VDcsy6bfc12/vf3v+fiiy+Wya/CEhUVFTz00EPMnz8flanwneiDUiBGfh1jsoD2Bdo1tttJTUnl5z/7OT/84Q9xOp1WJ4s5UkDHsF27dvHWW28x66OP8Ho8mNkleIomYeSNSqj2joQuoE0fjpqdpB7chGqtJSMzi+98+1t873vfk1nLot/279/P36/+G/v3l/OrCa18sXTwu5yGs4De2Wjn3vXZeJSLG2+6iTPPPHPQ9ynEYK1cuZL7pt1H2Z4yGArGyQbkWJ0qxgpoDWqvwr7ejm7XXHTRRVxxxRUUFhZanSxmSQEdB5qampg5cyZvvvU21QerIC0LT+GEhGnvSMQC2t+msRlnzVa0t4MxY8dy8Q9+wIUXXijv5MWgNDc3c/3117FixUq+MaqDHx3TPqjJheEqoJcEJgsWFA7ljjvvYvTo0YO6PyHCyefzMX36dJ548glaW1sxx5royRos/HMcMwV0LdjX2qEWjht/3P9v786jo67vf48/3zOTPYEgAcImu4AkBjAi+sOihyoooFwVkVoXrEfcoViqVbTWHQRtgapVf7jU36/V3iLivRa8R7H+Cm6ICyg7giFITIBAIHvmc/+YUUcKmElmyfJ6nJPjzHd9R893fOUzn4Xp06ZrcaN6UIBuRurq6oLdO/43n3zyMebxUtOuJzUdB+DPyIZm+jVpiwnQzo+3tICEbzbg3b8z2E3jDC6++GLy8vL0NbZETG1tLfPnz2fJkiUMzarhhpyDJDdwcGFjA7RzsOTLZP6+LYXc3BweeOBBMjMzG1aMSJTt37+fZ599liVLluASHHUD63B9XFxWvoh7gK4AW2t4dnjIbJfJ9dddz+jRozW7Rj0pQDdTW7duZenSpSxbtpyKinJIzaQqqz+1Wf2aXat0cw/QVnUQX/HGwKDAqkNktmvH+HHjGD9+vFZZk6havHgx8+fPp2dGLbfmHaBtYvif2Y0J0LV+WLQ+lXe+TtJgQWlWtm3bxoKFC/ho9UdYm2D/6BhPexe3AF0HtjEwLZ0XL5MmTeLyyy/XwkZhUoBu5ioqKnj77bdZsuRV1q//AjxeaptZq3SzDNDOj3dfAQnFG/CWFmBmnHLKKZx//vmcfvrp+HyaRl1i49133+Xuu2aR6avm14MP0CnMGToaGqAr62DB2nQ+LUngqquuYsqUKfqWRZoV5xyrVq1iwcIF7CrcBdlQl1cHbWJz/5gHaAdWYHjXeXGHHD8Z+RNuuP4GunTpEpv7tzAK0C3I1q1bee211/jHsuVUlB8KzCnd4QRqs05o0q3SzSlAW1UZvuJN37U2t2t3HOPHj2Ps2LF07qxZ+yU+Pv/8c34981dYzSFm5h2gV5u6ep/bkABdVm3M/TSDLw/4+OWMGVxwwQVh1yzSVNTU1LB48WIWPbuIiooK/H2C/aOj/GVKTAP03mA/5xLo3ac306dN19SpjaQA3QJVVlayYsUKXl26lC8+/7zJt0o3+QAd2rf529bmYcO44PzzOe2009TaLE3Cjh07uHXGLzmwbw/TTzpAznH1m+Yu3ABdUuFh9idtKKlO4J57fscZZ5zR4JpFmpLS0lKeeeYZXnvtNUiEutw6XE8XtWnvYhKgq8DWGZ5tHtpmtmXqtVM599xz8Xqb2KTYzZACdAu3bds2XnvtNV7/x7JAq3Rqu0Bf6Q59wdc0WqWbaoC2qkMhfZsPftfaPG7cOPVtliappKSEX906g4KvdvCrvAMMqkeIDidA76k07l+TSTkpPDx7Dnl5eY2uWaSp2bp1K/Mence6teugPdQNrYPMyN8nqgHagX0Z6K5hNcbFF1/MlClTSEtLi/y9WikF6FaisrKSt956iyVLXmXDhvWBBVqO60VtxwH40zvGtVW6SQVo58e7vxDfN+vxlRaAc+Tnn8KECReob7M0C6Wlpdxy8018vbOAmYMPMOBHFlz588bAOseX96845nH7qowH1rTlgD+FRx/7PSeeeGLEahZpapxzLFu2jD8+/kcOHDgQ6NaR4yAhcveIWoDeB96PA9PS5eTmcOuMW+nTp09k7yEK0K3R5s2bWbp0KcvfeIPKigpIPY6qDsEZPHyxH0HfFAK0VZfjK95IYskmqCyjTdtMxo8by/jx4zXAQpqdvXv3cvNNN1K8exe3D9lP37b17xN9JPurA+F5b00y8x59lNzc3AhVKtK0lZWV8fTTT/Pqq69CcrBbx/GR6dYR8QBdDfa54dnqoU3bNtx0w02MHj1ag3ujRAG6FSsvL+fNN99kyZJX2bx5E+ZLoLr9CdRkD8Ilx2gYMvEN0J5DJfh2ryNh7zbw+xkyZAgTJkxgxIgRJCREsKlBJMaKi4u5+aYbKS0p4s6h++mR0bAQXV4L93/UlqKqJB6ZO08Dj6RV2rhxI3PnzWXjho24zg7/yX5Iadw1Ixqgi8C32gcVMGHCBK655hoyMjIaf105KgVoAWDDhg387W9/48233sJfV0dtux7UZOfiz+gU9e4dMQ/QzuEt/YrE3evwHPiapKRkxo0by4UXXkj37t1jU4NIDBQVFXHD9dfhDu3hvlNKyQhznmi/g8c+TeezfUnMnj2HYcOGRalSkabP7/ezePFinnjiCWo9tdQOqYVG/C8jIgG6NrgYyhYP3bp3Y9ads9S9KkYUoOUHSkpKeOWVV1j8yhIOHSzDpWdR1SmHuuN6Q5RWJ4pZgK6rCUxB980XULGfrKwOTJx4MePGjdNf6tJiffHFF9x8040MaFvFzMFlYS37/eqXyfxtawrTpk3joosuil6RIs3Ijh07uO/++9i0cRP+4/24IQ2b8q7RAXoP+D704cocEydO5NprryUpKY5rk7cyCtByRJWVlSxfvpy/vvQShTt3YklpVHU8kZqOA8AX2Qc02gHaqg7hK/qcpJKNuJoqBgwYyKRJlzBy5EgNCpRWYenSpcydO5cLelUwsU9lvc75bI+PRz7OYNRPf8pdd92lfpQiIWpra3nxxRd57vnncEmO2vxa6BTeNRocoP1gXxieDR6ysrK48447Ofnkk8O7hjSaArQck9/v5/333+ell15izZo1mC+RquxcarJzwBuZPsJRC9A1FSTu+oTEbzaA8zNy5EguueQScnJyInsfkSbOOcfs2bN5/fXXmTm4jLysY8/MsbfSuOODdnTs2oMnnvwTKSmN7Owp0kJt2LCB++6/j4KvCvDn+nH96z/AsEEBuhq873rhGxgzZgy33HIL6enpDahcGksBWupty5YtPPOf/8mqlSuxxFQqO+dR23EAeBo3IXvEA3RtNQm715JUtA7qajnvvPO44oortFKgtGpVVVVc84urqSwp4OHh+0g4Ro+sJ9al8sGeNJ577nmNCxD5EVVVVTz40IOseGsF/p5+3MkO6tHjMewAXQa+VT485R5uv+12Ro8e3YiqpbGOFaCj0+FVmq2+ffvy8EMP8fjjj5M7sB9JO94lbe3f8RVvBheDpUh/jL8W39drSf/sZRILP+Yn/3E6L7zwArfddpvCs7R6SUlJ3DJtOkXlsPyro3fD2lzqZeXuJC69dLLCs0g9JCUl8du7f8tVV12FZ7sH7zteqIrwTb4B3wof6aTzh9//QeG5iVPnUDminJwcFsyfz4cffsgTTz7J1i3/JGn3Wiq7nUxd5vFhz9rhT2vfuIKcH1/xZpJ2fQxVBxl6cj5Tp17LgAEDGnddkRbmlFNO4fTTT2fJB+8yonM1mUk//JbR7+DPm9Npf1w7LrvssjhVKdL8eDwerr76arp168ZDDz+ErTBq/6MWjjE+3WXW71t++9LwrPHQtWtX5j4yV2sTNAPqwiE/yu/3s2LFCp5+5hl2FRZS264HVb1GQEJs+kxa5X6St72Dp6yI/gMGcN3UqRpMIXIMBQUFXHnFFfwku5yrB5b/YN+q3Qk8vi6dO++8Uy1cIg20du1afnPHbyirLqP2rFpoxOrZ9qXhWe3h5PyTufd392rGqCZEXTikUTweD6NGjeLFP/+Z66+/nqSyQtLXLca776vo3tg5fEXrSVu3hHT/QWbNmsVTf/qTwrPIj+jevTvnjB7NyqJkKg4bS/hWYTJdu3Tm7LPPjk9xIi1Abm4uCxcsJMWTgu9fvoZ35/gaPB95yM/PZ87sOQrPzYgCtNSbz+dj8uTJPPP00/Tq1oXkTW+QuO1/oK464vey6nKSN71B0vaVDB2cxwvPP88555yjabZE6mns2LFU1To+KPp+8trd5R427PMxdtx4PFGa812ktejZsycPP/QwnnIP3lVeCHch0L3ge89Hnz59uP/++7UybjOjT1AJW58+fXjqqT9x2WWXkViyibR1S/CU7Y7Y9b17viRt3WKSy4uYNm0a8+bNpWPHjhG7vkhrkJOTw/Hdu/HPr5O/2/bOrkQ8ZowZMyaOlYm0HIMHD+buu+7G9hie9z1Q316xB8G30kfWcVk8MucRUlNTo1qnRF7cA7SZjTGzjWa2xcxuj3c9Uj+JiYlMnTqVhQsX0ikzjZT1/xfvnq2Nvm5C4Sckb3mTfr178OyiRVx00UVqKRNpADPjvLHj2FTq5ZsKD87BqqIUhp06jKysrHiXJ9JinHXWWdx4441YoWGb6/EtqQPv+15Sfak8Ou9RPY/NVFyTiZl5gT8C5wInApPNTAu8NyO5ubk89+wicnNySd76Nt6ShofohMKPSdy5mrPPPpsnn3iC448/PoKVirQ+w4cPB2DjPh97Kj2UVMDw4afFuSqRlmfixIkMHz4c7xdeKD/2sbbNYC/cOuNWevToEZsCJeLi3bQ3DNjinNvmnKsG/gpcEOeaJExpaWk88sgcTjrpJJK3vY23ZHPY10jYuYbEnR8xevRo7rjjDi2/LRIBPXv2JC01hU37fWzaH1gMKTc3N85VibQ8Zsb06dPx4cPz6TGiVSV413kZOnQoo0aNil2BEnHxDtBdgYKQ9zuD26SZSU1N5ZE5cxicN5jkrf8MLLxSH86RsPMjEgvXMGbMGG6//Xa83sateigiAR6Ph0E5uWw+kMjmUh8pyUn06tUr3mWJtEhdunThyiuvxHYaHGVYkH1mePweZsyYoUHxzVy8A3S9mNm1ZrbazFYXFxfHuxw5ipSUFObMmc3QoUNJ2vZPPGVFP3qOr2QLiYUfM3bsWIVnkSjIycmhsMxYX5rIwBNP1Lc7IlF06aWX0rVbV3xrff8+oHA/eHZ4+Nnkn6mLYgsQ7wBdCISuI9stuO0HnHNPOefynXP5HTp0iFlxEr7k5GQefPBBsjp0IGX7v8B/jHl9aspJLniPQTk5zJw5U4MFRaKgW7duOGDnQQ/dumnZbpFoSkxMZNIlk3ClDvb9cJ99aXi9XiZOnBif4iSi4p1YPgT6mVkvM0sELgWWxrkmaaTU1FR+PXMmlO8jYdcnRz0uafu7eFwdt992m8KzSJR06tTpu9fZ2dlxrESkdRg1ahQJCQnY9pAuGn7wFngZMWIEmZmZcatNIieuqcU5VwvcBCwH1gMvO+c+j2dNEhnDhw/nnHPOIXHXp1j5nn/b7927Hd/eL7l6yhSNQhaJotDQHBqmRSQ6MjIyOPPMM/EWhCyusgtcpWPcuHFxrU0iJ+7Nfs65151zJzjn+jjnHoh3PRI5N998M20yMkj66sMf7nCO5IIP6NW7N5MnT45PcSKtRPv27WnTJrA8cN++feNcjUjrcN555+Gq3XeDCT0FHtq1b0d+fn58C5OIiXuAlparbdu2XHjh/8K7fydWdfC77Z4DX0PlAS7/+c81oEkkyjweD4sXv8KyZcs0A4dIjOTm5uLz+bASCyycssdL/tB8DZRvQRSgJaq+XTLYFzI3dELxJlJS0zjjjDPiVZZIq5KYmKilgkViKDExkRP6n4BnjwfKwV/hJycnJ95lSQQpQEtUdenShby8PBL3bAHnoLaahNLtnP3TUSQlJcW7PBERkajIzcnF9hlWHBhMOGjQoDhXJJGkAC1Rd+6550LFfjzle/CWfoWrq/2uZVpERKQlGjRoEM7vsO1GQmICvXv3jndJEkHqgCpRN2TIEAA8B4vxVJaSkJjIgAED4lyViIhI9HTp0gUAKzayj8/WmJ8WRv81Jeqys7NJS8+g5lAx3sr99OvXTx8kIiLSooVOIdk5u3McK5FoUBcOiTozY+DAAfjKS/CW72WgWp9FRKSFy8jIICk5MNZHc7C3PArQEhP9TzgBO7QXV1dDv3794l2OiIhIVJkZ7dq1A6Bjx45xrkYiTQFaYiL0r2/9JS4iIq2BxwIxKyMjI86VSKQpQEtMtG/f/oivRUREWiqzwBR26enpca5EIk0BWmJCAVpERFqrtLS0eJcgEaYALTER2m1Df4mLiEhrMHjwYAA6dOgQ30Ik4sw5F+8awpKfn+9Wr14d7zKkATZu3EhycjI9evSIdykiIiJRV11dTVFREd27d493KdIAZvaRcy7/SPs0Ga/ETP/+/eNdgoiISMwkJiYqPLdQ6sIhIiIiIhIGBWgRERERkTAoQIuIiIiIhEEBWkREREQkDArQIiIiIiJhUIAWEREREQmDArSIiIiISBgUoEVEREREwqAALSIiIiISBgVoEREREZEwKECLiIiIiIRBAVpEREREJAwK0CIiIiIiYVCAFhEREREJgwK0iIiIiEgYFKBFRERERMKgAC0iIiIiEgZzzsW7hrCYWTGwI951SINkASXxLkKkldLzJxIfevaarx7OuQ5H2tHsArQ0X2a22jmXH+86RFojPX8i8aFnr2VSFw4RERERkTAoQIuIiIiIhEEBWmLpqXgXINKK6fkTiQ89ey2Q+kCLiIiIiIRBLdAiIiIiImFQgJaIMbM6M/sk5Of2kH1ZZlZjZtcdds52M1trZp+Z2Rtmlh37ykWaNzM7eNj7q8xsYfD1PWZWGHwm15nZ+SHbfxWPekWaMzNzZvZiyHufmRWb2f+xgBIzaxfc1zl4/IiQ44vNrL2Z9Tezt4PP5nozU1ePZkQBWiKpwjk3OOTn4ZB9E4H3gMlHOO8s59xJwGrgjlgUKtLKPOacG0zgOVxkZvrsF2m4Q0COmaUE358NFAK4QL/Y94DTgvtOBz4O/hMz6w/scc7tAeYTfDadcwOBBbH7FaSx9CEqsTIZuBXoambdjnLMO0Df2JUk0ro459YDtQQWdhCRhnsdGBt8PRn4S8i+VQQDc/Cfj/HDQL0y+LozsPPbk5xza6NVrESeArREUsphXTgmAZhZd6Czc+4D4GVg0lHOHwfoA0QkfD949oB7j3SQmZ0K+IHiWBYn0gL9FbjUzJKBk4D3Q/at5PsAPQx4BegefH86gYANgWD9lpn9w8x+aWaZUa9aIsYX7wKkRakIfk18uEkEgjMEPnQWAfNC9q8wszrgM2BWVCsUaZl+8OyZ2VVA6MpnvzSznwNlwCTnnDOz2FYo0oI45z4zs54EWp9fP2z3h8AQM0sDEpxzB81sm5n1JRCg5wWv8ayZLQfGABcAU80szzlXFbNfRBpMAVpiYTKQbWaXBd93MbN+zrnNwfdnOedK4lSbSGvwmHNubryLEGlhlgJzgTOB9t9udM6Vm9lm4GpgTXDze8B5QEdgY8ixuwg0Ki0ys3VADvBRLIqXxlEXDokqMzsBSHfOdXXO9XTO9QQe4siDCUVERJqLRcDvjtJ3eRUwHXg3+P5dYBrwXnCgIWY2xswSgq+zCYTwwmgXLZGhAC2RdHgf6IcJBOVXDjvu7yhAizQFs8xs57c/8S5GpDlxzu10zs0/yu6VQG++D9BrgG583/8Z4BxgnZl9CiwHZjrndkerXoksrUQoIiIiIhIGtUCLiIiIiIRBAVpEREREJAwK0CIiIiIiYVCAFhEREREJgwK0iIiIiEgYFKBFRJooMztT08uJiDQ9CtAiIhFmZm+b2T4zSwrzPBdc7jcmzOye4D0vCdnmC27rGas6RESaGwVoEZEICgbPMwAHnB/far5nZr6j7NoL/M7MvLGsR0SkOVOAFhGJrCuA94DngCtDdwRbpq8JeX+Vmf0r+Pqd4OZPzeygmU0KOe5WM/vGzL42sykh29ua2QtmVmxmO8xslpl5Qq690sweM7M9wD1HqXcZUA38/Eg7zWysmX1sZgfMrMDM7gnZ1zPYWj0luG+fmV1nZqeY2WdmVmpmCw+73tVmtj547HIz63HMf5siIk2QArSISGRdAfxX8Ge0mXWqz0nOuZ8EX+Y559Kdcy8F32cDbYGuwC+AP5pZu+C+BcF9vYGRwXtP+f6qnApsAzoBDxzt1sBdwG/NLOEI+w8Fr5sJjAWuN7MJhx1zKtAPmAT8HrgT+CkwCLjEzEYCmNkFwB3AhUAH4H+AvxylLhGRJksBWkQkQsxsBNADeNk59xGwFfhZIy9bA9zrnKtxzr0OHAT6B7tcXAr8xjlX5pzbDswDLg85d5dzboFzrtY5V3G0GzjnlgLFwDVH2Pe2c26tc87vnPuMQOAdedhh9znnKp1zbxAI3H9xzn3jnCskEJKHBI+7DnjIObfeOVcLPAgMViu0iDQ3CtAiIpFzJfCGc64k+P6/OawbRwPsCYbNb5UD6UAWkADsCNm3g0BL9bcKwrjPLAItx8mhG83sVDNbEewmsp9ACM467NyikNcVR3ifHnzdA/hDsGtHKYH+13ZYzSIiTd7RBpWIiEgYzCwFuATwmtnu4OYkINPM8pxznxJonU0NOS27EbcsIdA63QP4IrjteKAw5BhX34s55/6fmW0Bbjhs138DC4FznXOVZvZ7/j1A11cB8IBz7r8aeL6ISJOgFmgRkciYANQBJwKDgz8DCXRhuCJ4zCfAhWaWGpyu7heHXaOIQH/mH+WcqwNeBh4ws4xgN4gZwIuN+B3uBH592LYMYG8wPA+jcV1SngR+Y2aD4LtBkBMbcT0RkbhQgBYRiYwrgWedc18553Z/+0Og9fay4DRyjxGY8aIIeJ7AQMNQ9wDPB7s4XMKPu5lAq/Y24F8EWosXNfQXcM6tBD44bPMNwL1mVgbcTSC0N/T6rwCzgb+a2QFgHXBuQ68nIhIv5ly9v+ETEREREWn11AItIiIiIhIGBWgRERERkTAoQIuIiIiIhEEBWkREREQkDArQIiIiIiJhUIAWEREREQmDArSIiIiISBgUoEVEREREwqAALSIiIiIShv8PUrVeUVF29hAAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"EAP seems slightly lesser number of words than MWS and HPL. ","metadata":{"_uuid":"abe5dcadd42cd45169e44e3e299c303d5197b8ab","_cell_guid":"66833874-08f2-48b8-9561-b54de997071d"}},{"cell_type":"code","source":"train_df['num_punctuations'].loc[train_df['num_punctuations']>10] = 10 #truncation for better visuals\nplt.figure(figsize=(12,8))\nsns.violinplot(x='author', y='num_punctuations', data=train_df)\nplt.xlabel('Author Name', fontsize=12)\nplt.ylabel('Number of puntuations in text', fontsize=12)\nplt.title(\"Number of punctuations by author\", fontsize=15)\nplt.show()","metadata":{"_uuid":"1c4c0950d33e29d270fbc021aa8c5ff0e7ab205b","_cell_guid":"91514e74-d734-4642-a9dd-470cb8a65733","execution":{"iopub.status.busy":"2023-01-12T09:34:02.105481Z","iopub.execute_input":"2023-01-12T09:34:02.106585Z","iopub.status.idle":"2023-01-12T09:34:02.415461Z","shell.execute_reply.started":"2023-01-12T09:34:02.106537Z","shell.execute_reply":"2023-01-12T09:34:02.414252Z"},"trusted":true},"execution_count":9,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 864x576 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAtAAAAH1CAYAAADf3smEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAACgJ0lEQVR4nOzdd5wb1bn/8c+jstre3da9AsbGgMFgSigJkAAJJLnJTSMhvSf3JjftpgdSfwk396ZRQoBQg+lgTDcugHvvuLdde3tflZnz+0PSer1u2t3Rjsrzfr30WkkrzXxXK808OnPmHDHGoJRSSimllEqMx+0ASimllFJKpRMtoJVSSimllOoDLaCVUkoppZTqAy2glVJKKaWU6gMtoJVSSimllOoDLaCVUkoppZTqAy2glcpwIvIzETEi8uJxfveYiLw+iFkuj2WZNljr7AsROUNEFolIeyznOLczJUJEcmL/57OTuI5ZIvKz49z/MxGpS9Z6+0pEXheRx9zOcSoi8gURufE49+8Wkd+7EEkp1QdaQCuVPa4WkfPdDpHi/h9QCrwPmA1Uu5omcTnAT4Gzk7iOWbF19PZ34JokrjdTfQG40e0QSqn+8bkdQCk1KBqAA8APyeCdtojkGmO6BrCI04FnjDGvOpUp0xlj9gP73c6hTk1E8owxnW7nUCoTaAu0UtnBAL8E3ici00/0oBMdjo91Z/haj9u7ReT3IvJ9EakWkWYR+YNEXSsiG0WkVUSeEpGy46yqSkSei3WV2CsiXzrOOi8VkQUi0iEi9SJyl4gU9fj9zbFcs2KH7TuB75zkbztbRF6NLa9RRB4UkWGx340TEQNMBP4zttzXT7IsIyLfEpH/FZEGEWkSkT+JSM4AXsv/FJH9sWyPiEhpr+dViMgdsde7S0S2ish/xH7dGvt5T2z5JvY3HbfLTO9uDiIyW0SeiS27XUTWiMjHe77WwJ965O9+fY73d4rI+Nj/viX2PnhWRCYd53X4poj8SkRqReSwiPxFRAI9HlMqIn8XkYOxv3mviNx1ov9Lr+V/IfbadorIXBEZ2eN3y0Tk3uM8514RWX2SZZ4e+9/si72PNorIf4iIp8dj4u/Lwl7P7e6aEXvtZgKf6vF63tzr8ad6PyT6Gn9LRP4oIrXA+lO+cEqphGgLtFLZYw7wC6Kt0B9xYHkfAZYBnyZaDNxK9Ev5O4AfA3nAn4FfA70L5LuB+4kWZe8H/iYi+40xzwGIyMXAK8BTwL8BFcBvgLLY7Z4eBv4K/BxoOl5QERkCvA5sBj4GFMaW97KInEe0q8Zs4EngtViullP8/d8GlgAfB84k+gWli5MU8SfxYWAd0cP6o4DbgF8BX4nlz4vlH0r079wCTIpdAK6M5b4VmBu7rxoYl+D6xwJvALfH/oaLiRbjtjHm4dgy/0D0b54de85xX59YAfwqEAY+D0RimReIyHRjTEOPh387lvsTwFlE3yt7gN/Ffn8bcBHwn0ANMJro++tUZgOnAd8CcoHfEn0vxbsw3Q38QUS+Zoxpi+UuJPre+sFJljsS2Ao8SPRLy9mxvy0vlj1RXwEeB3YCt8Tu29Hj96d6P/TlNf4OsBC4CW00U8o5xhi96EUvGXwBfgbUxa7fDFjAlNjtx4DXj/fYXsswwNd63N4NbAe8Pe5bRnRHPr7Hfb8DDvW4fXlsWXf2Wv7LwJIetxcB83s95srYc6f1+FsM8M0EXoPfEC2ui3vcd0Hs+R/t9Xf9PoHlGaJFrKfHfT8EOoDyfryWOwBfj/v+CNT0uP1FwAbOPkGewthyb+51f/z1ntbr/teBx06wLCHauHIH8FqP+78W3WWc+P0Vu/2l2PtgQo/7RgEh4Ae9XoeFvZb1VK/3wQbg6318v79OtLAc0+O+i2Pre3fsdjHQDny6x2M+AwSBigTXE3+d/hvY2eP++PuysNfjj3pvASuAe4+z3ETeD315jVf15fXTi170kthFv40qlV0eAPZy8la2RL1ujLF63N4O7DbG7Op135CeXRtinux1+wlgpoh4RSSfaAvioyLii1+AxUQLo5m9njuXU5sFvGSM6W41NcYsJVqsXJLA84/naWOM3etvyAP6M8LIfGNMpMftTcBQEfHHbl8JrDbGrOlX0lMQkTIR+T8R2UP0NQ4Tbf2c0o/FzSJatO2M32Gi/aTf4NjX+qVetzcRLQTj1gDfEZGviEhfsqwyxuztsf43gMOxbMTeB48RLXbjbiba/73+RAsVkVwR+bmIbCdabIeJHnkYH3uPOuVU74e+vMbPO5hLKRWjBbRSWSS2U/4d8AkRGTvAxTX1uh06wX1CdJSIng4f57YPqCTaTcNLtFtGuMclCPiJHsbv6VACWUec4HGHgPIEnn88x/sb4uvqq6Zet+OvW7w/cAXJHRHkXuDfiY5CcjXRrg7/INr9oa/68lo39bod6rXOrxFtlf4JsFVE3haRRLof9f7fxO/r+b+5G7hURCaIyETgUqJ/88n8Fvgv4E7gWqKv062x3/XntTqRpl63e78f+vIaJ/L5UEr1kfaBVir7/AP4EfC94/yui17Frhz/JMCBGnqc2xGgjmghYoh2DThe69nBXrdNAuurPs46AYYBKxN4/vEc72+IrwucfS3rOdLfuS/iI5L0/gJTRvS1RkRygeuBrxpjbo8/oOeJcX1UTbRPeG/DiI4GkzBjTBPwDeAbInIW8F3gQRFZZ4zZdJKnHu9/PZQeX0KMMQtF5G2iLc9C9H3Vu0W8tw8BfzLGxPtoIyLX9XrMyV5zp/TlNU7k86GU6iNtgVYqyxhjgsDvifb57N1auh8o6jliAdEWSae9/zi3VxpjLGNMO9GT804zxqw4zqV3AZ2IpcA1cvQoHucTPclucT//hht6FZkfADqJ9tsFZ1/LV4FzYkXk8YRiP3u3gsaHlzsjfoeIjCY6XF9cgOi+INjjMUVEx8I+Zh2xgvtklhLtjjO+x/JGEj0ZsL+vNcaYdURPiPP0yn8854rImB7rv5hoAb2s1+P+AXwK+CTwz15dko4nj6NfJy/HnpB7vNf8AqL9rnvq3dreF0l5jZVSidMWaKWy0x1ET366CFjQ4/4XiBaB/xCRPwDjOXYEDSe8R0R+GVv3B4CrgBt6/P67wKsiYhPtq9oKjAGuA35ojNnWx/XdBnwZeFFEfsuRUTjWEx0NoT+KgDmxYdXOJDryyF/MkREQnHwt/wl8FXhJorMBbo0tb4ox5vvGmJCI7AI+LCIbiLaCrjPG7BeRFcAtItJBtPj8b3q0UhpjmkVkOfATEWkherLi94Fmji76tsR+flNEXgNajDFbj5P1XqJHN+aJyE+InrT6U6It3nf05Y8WkcVE+8tvINqS+nmiJ//1LoR7qwXmishPOTIKxypjzAu9Hncf0S4YPuCeBCK9DHw11ge6gej/JNDrMcuIjrn+fyLyY6JdKr7LsaOWbCH6pe4aokcYdp2s/3Uv9+LQa6yU6h9tgVYqCxljOoD/Oc79dcAHiZ7I9RTR4cU+loQInwPOja0j3n3gmR45FhMdrmwI0eHuniVahOyjH306jTG1wBVEC8uHgb8QHenjKmNM6GTPPYk/ED2U/jDRPrp3Ey1O4+t07LU00clhriT6OvwCmEf09ejZGv8lon3IXwGWA1Wx+z9K9MTRB4gOhfYLogV4Tx8jOqTaP4H/Jfql4p+9HrOIaB/pbxJtAT1uoRY7wvEuogXi3USL1L3A5ebo4dUS8RbRLhaPAY/G/r73xE6YO5k3if6P/xjLsIHjTCBkjKmJ/S1vJPil7OtEX4e/EG293kCv4eti76f3E/0i8hjRofq+DDT2WtatRIdVfJTo/+u9Caw/vg4nX2OlVD+IMdo9Siml+kKik6583RjzZ7ezqP4TkXKircVfM8bc7XYepVT60C4cSimlskqsj/dUoq3prUSPIiilVMK0gFZKKZVtZgLzic56+MlYlyallEqYduFQSimllFKqD/QkQqWUUkoppfpAC2illFJKKaX6IO36QFdWVppx48a5HUMppZRSSmWwlStX1hljhhzvd2lXQI8bN44VK1a4HUMppZRSSmUwEdlzot9pFw6llFJKKaX6QAtopZRSSiml+kALaKWUUkoppfpAC2illFJKKaX6QAtopZRSSiml+kALaKWUUkoppfpAC2illFJKKaX6QAtopZRSSiml+kALaKWUUkoppfpAC2illFJKKaX6QAtopZRSSiml+kALaKWUUkoppfpAC2illFJKKaX6QAtopZRSSiml+kALaKWUUkoppfpAC2illFJKKaX6QAtopZRSSiml+sDndgCV+bZu3cqLL77Yfdvj8fDBD36QESNGuJhKKaWUUqp/tIBWSff3v/+dpcuWId4cAEwkhDGGr3/96y4nU0oppZLj4MGDrF27FgC/38873vEOcnJyXE6lnKIFtEqqcDjM6jVrCA89g9C4iwDI3TKPZcuXu5xMKaWUSp7bbruNZcuWdd/+4Q9/yDXXXONiIuUk7QOtkmrTpk2EgkGs4qru+6ziKvbs3k19fb2LyZRSSqnkMMawafMm7DE21rUW4hO2bNnidizlIC2gVVKtXLkSRLCKj/R3jhfTK1eudCuWUkoplTSHDx+mrbUNKoECMKWGLVu1gM4kWkCrpHp9wQLswqHgC3TfZxdUIDl5LFq0yMVkSimlVHJs3boVAFNmALBLbd5++20sy3IzlnKQFtAqaXbs2MHuXbsIl088+hfiIVQ2njfefJP29nZ3wimllFJJsnHjRsQjUBK7oxxCwRC7du1yNZdyjhbQKmlefvllECFSMf6Y30UqJhEJh1m4cKELyZRSSqnkWbFyBabCgDd62wyJtkRr18XMoQW0Sgrbtnnp5VewSkaCP+/Y3xcOgbxiXnzpJRfSKZU96urq+Mynb+bjH/3IUSMCKKWSo7m5me1vb8ceah+5Mx+kWFixYoV7wZSjtIBWSbF06VLqag8Trpx8/AeIECqfxKpVq9i3b9/ghlMqi2zatIntO3ay78BBlixZ4nYcpTLe6tWrMcZghpmj7reGWKxZu4ZwOOxSMuUkLaBVUsyZ8xgSKMAqG3fCx0SGng4iPPHEE4MXTKksU1NTA0BpwHDo0CGX0yiV+ZYsWYLkCJQdfb8ZZgh2BbsnV1HpTQto5bhdu3axYsVygkPOAI/3hI8zOfmEyyfw3Ny5tLW1DWJCpbJHTU0NuT5hbGGE6oMH3I6jVEaLRCIsXLQQa7h1bIU1DMQnLFiwwJVsyllaQCvHPfbYY4jHR3jo6ad8bGT4NIJdXcydO3cQkimVfaqrq6nMsxmaZ1FdXY0x5tRPUkr1y+rVq2lrbcOMOs7nzAf2cJvXF7yuw9llAC2glaPq6uqY98ILhCongT/3lI+3Cyqxi0fw8CP/IhQKDUJCpbLL7l07qcoLM6LApr2jU2cAVSqJFixYgPgEhh//92aUobmpmfXr1w9uMOU4LaCVox555BEikQjhEWcl/Jxg1Qwa6uuYN29eEpMplX06Ozs5WF3DqEKLUQXRFq+dO3e6nEqpzBSJRJj/+nysEVb38HW9meEG8Qqvvfba4IZTjtMCWjmmsbGRJ596inDFJExuccLPs4tHYgqHcv8DDxCJRJKYUKnssmfPHowxjC60GF2oBbRSybR06VJaW1oxY07STcoPVpXFK6++oqNxpDktoJVj/vWvfxEOhQhXzejbE0UIVp3N4UOHePHFF5MTTqkstH37dgBGF1oU5RhKc4UdO3a4nEqpzPTiiy8iuSfuvhFnxhraWtt0WMk0pwW0ckRTUxOPP/4EkYoJmLzSPj/fKh2NKajkvn/er63QSjlky5Yt5PuFoXnRCR3GFwbZvGmjy6mUyjxtbW0sfmMx1qjjjL7R2zCQXOElnUgsrWkBrRwxZ84cgsEgoapz+reAWCt0TfVBXn31VWfDKZWlNm3cwISiEB6J3p5UYrF3335aW1vdDaZUhlmwYAGRcOTk3TfiPGCNsnjjjTf0s5jGtIBWA9ba2sqcxx4jUj4Ok1926iecgFU2FlNQwb333qdD/Cg1QF1dXezctZuJxUeO6EyIXd+6datbsZTKSC++9CJSJFCe2OPNWEMkEtExodOYFtBqwObMmUNXZyehkWcPbEEiBEeczYED+5k/f74j2ZTKVtu2bcO2bSaWHPkyOqE4en3jRu3GoZRTDh06xNo1a7HGWCAJPqkMpFh48SU97yddaQGtBiQYDPLY408QKRuDya8Y8PKs8nGQX8ZDDz+sEz4oNQDxInliyZEW6AK/YWShYdOmTW7FUirjvPrqqxhjEuu+ESdgjbZYu2Ythw4dSl44lTRaQKsBefnll2lrbSE8fJozCxQhOHQq299+WweaV2oANm3axNB8KMk5eqc+sSjEpo0b9AuqUg55bf5r0a4bhX17Xrzg1m4c6UkLaNVvxhjmPPYYpqACu2iEY8uNVE5C/AEee+wxx5apVLbZuGE9k4qDx9w/qSRCc0srBw8edCGVUpmltraWbVu3YVfZfX9yIUiJsPiNxc4HU0mnBbTqt7Vr17Jr505CQ6eCJNrxKwFeP8HKKSxcuFAPbSnVD/X19dTVN3T3ee4p3id6y5Ytgx1LqYzz5ptvAmCq+ndExxphsW7tOh2NIw1pAa367aWXXkJ8OUQqJzq+7MjQM7BtW6c7Vaof4pOljCk8toCuKrDwis5IqJQTFr+xGCkUSHzy3aOYKoNt27z11lvOBlNJpwW06hfLsli4aDHhklHg8Tm+fJNbjCmoYOGiRY4vW6lMFy+gRx+ngPZ7YESh6Z6lUCnVP5ZlsWbNGqxhfRh9o7dykICwevVqR7Op5NMCWvXLpk2baGluIlI2NmnrCJeOZdPGjTQ0NCRtHUploh07dlCeJxTlHP+w8piCEDve3jbIqZTKLPv27SPYFUx47OfjErBLbTZv2exYLjU4tIBW/bJo0SLweLBKRidtHVbZWIwx3X3MlFKJOXBgP8NzQyf8/Yh8m8N19YTD4UFMpVRmiZ9HYMoHNqKNKTfs3rWbYPDYk35V6hqUAlpE/iEih0VkQ4/7ykXkZRF5O/az/1PYqUG3YcMG7IIh4MtJ2jrs/HIkJ58NGzac+sFKqW71tbWUBk48KkD8d3p0R6n+27JlC+ITKBrYckxZtB+0dqtKL4PVAn0v8O5e930feNUYMxl4NXZbpYk9e/dh5ZYmdyUiWIFi9u7bl9z1KJVBjDE0NDZRGjhxq1hprGtHfX39YMVSKuPs378fU2z63/85LnYC4j7d16WVQSmgjTELgd5NHTcA98Wu3wfcOBhZ1MC1trbS2tKMye3nacd9YOUW60ZFqT5oa2sjFA5TmnPqFmgtoJXqv8amRswJzjPok0D0R0tLy8CXpQaNm32ghxljqmPXa4BhLmZRfbB//34A7NySpK/L5JbQ3NREe3t70telVCYIhaJ9nwPeE+/Yc2K/iz9WKdV3Tc1NzhTQfkCgubl54MtSgyYlTiI00TllT/guFJEviMgKEVlRW1s7iMnU8cS/JRt/XtLXFV+HfjNXKjEeT3SzbpsTH1eOz+Idf6xSqu9aW1q7W48HRMAT8Oh+Ls24ufU8JCIjAGI/D5/ogcaYO40x5xljzhsyZMigBVTHl5MTO3HQPnaMWcfF1tG9TqXUSUlsVlBzkoaxeHEtTs4gqlSWiUQizlVRHnRUnDTjZgH9DPCp2PVPAU+7mEX1QbyYFaMFtFKpxuv1AmCdtIA++rFKqb7Lz88Hh2peEzYUFhY6szA1KAZrGLuHgbeA00Rkv4h8FvgNcJWIvA28K3ZbpYHBbIGOF+laQCuVmIKCArxeD63hE7cux39XUpL88xiUylQFBQXOFNAmWkAXFBQ4sDA1WJyfg/k4jDEfPcGv3jkY61fOiu90JdSR9HVJqJ1AIFcLaKUS5PF4qCgroyHYecLHNHRF204qKysHK5ZSGaeoqAhpFsyJT+FKTKwI1wI6vegZJKrPhgwZQkFhEZ6O5A+B5e1oYOKkidpXU6k+GDJ0GI3BE2/eG4JaQCs1UBXlFXg6HSijYt91y8sHMie4GmxaQKs+ExEmT56EtzPJs5gZg7ezkSmTJyd3PUplmKHDhlEf9J/w9/VdHkqLi/TIjlIDMG7cOGgFTjzkekKkRY4sT6UNLaBVv0yeNAlvZyOYAW45TkJCbZhIkIkTJyZtHUplolGjRnG4A8In+Hge7PAyeuzYwQ2lVIYZN24cxjYw0GkKWqINU6NHj3YklxocWkCrfpk6dSrGiuBpr0vaOrwtNQCcccYZSVuHUplo3Lhx2AZqOo7dxBsDBzr8jBs33oVkSmWO7hbjAQ7fLM3C8KrhBAJODCqtBosW0KpfzjvvPEQEb1Pyptn2Nu2jpLSUSZMmJW0dSmWi+I79QPuxw9S1hIT2kNHDxUoN0Pjx4/F4PEjjwM7R8TZ7mTJpikOp1GDRAlr1S0lJCWeccQb+5gPJWYGx8bce5KLZs3W2NKX6aMyYMXhEONB2bAEdL6rHahcOpQYkNzeXCRMnIPUDKKC7wG6zmTZtmnPB1KDQykT12+zZs5G2wxA+8XBZ/eVpO4wJd3HBBRc4vmylMl0gEGDEiGHHbYHeH7tv/HjtwqHUQE2fNh1Pg6f/JxLGBrM688wzHcukBocW0KrfZs+eDYCvca/jy/Y27sHj9XL++ec7vmylssG48RPZ33HsSBwH2rwU5OfpEHZKOWDatGmYiIHm/j1f6gSfz8eUKdqFI91oAa36bfLkyQwdNgxf4y5nF2wMOY17mHnuuRQVFTm7bKWyxPjx4znULkR6tYwdaPcyfvwEHVtdKQfEu170txuHp97DlClTdEjJNKQFtOo3EeHKK67A23IQIkHHluvpaICuFq644grHlqlUthkzZgyWgcO9Jnqo7vQzRvs/K+WI4cOHU1ZeBv0ZkMoCaRRmzJjheC6VfFpAqwG57LLLwLbxNjnXjcPbsAsR4ZJLLnFsmUplm+HDhwPRSVPiQhY0Bw0jRoxwK5ZSGUVEmHHWDHwNvr4/uRGMbfQEwjSlBbQakDPOOIOy8gp8DXscW2ZO817OmjGD0tJSx5apVLaJF9B1PQroeDE9bNgwVzIplYnOOuss7HYbOvr2PKmLdvuYPn16ElKpZNMCWg2Ix+Ph0ksuxt96AGxrwMuTrhZob+BSbX1WakAqKyvxeDzU9ejCES+m48W1Umrgpk6dGr3S0LfnSYMwomqENhalKS2g1YBdfPHFmEgYb0v1gJcV7wpy8cUXD3hZSmUzn89HaUkRzaEjm/n49YqKCrdiKZVxJk6ciMfb9wlVvE1ezjhdZ9pNV1pAqwE799xzyQkE8DYOvBuHr3EvY8aOY+TIkQ4kUyq7FeQX0GUd2al3xQ4S5efnu5RIqcwTCAQYN3Yc0tSHAjoIdrutw9elMS2g1YAFAgHOmzkTf+vBgS3IiuBtq+Hii2Y7E0ypLJdXUEBnpEcBHbuuBbRSzjr99NPxNnnBJPiEpuiP0047LVmRVJJpAa0ccc4550BnMxJq7/cyPG2HwbY5++yznQumVBbL79UC3WkJIkJubq6LqZTKPBMmTMDusiGU2OOlJfq51BlB05cW0MoR8aLXM4B+0N7WakSEs846y6FUSmU3n8+HzZEC2jbg9Xh0EhWlHDZq1KjolbYEn9AGuXm5lJWVJS2TSi4toJUjJk2aRF5ePt7Wmn4vw9dSzeTJUygoKHAwmVLZKxKJ4OHIVIQeAcu2T/IMpVR/xAtoaUvsy6m0CqNGjdIvs2lMC2jlCK/Xy7RpZ+Jr7890TIAxeDvqmT5dB5RXyimWZeHtsX/2ChhjsLWIVspRw4cPjxbDrYk93tvhZczoMckNpZJKC2jlmEmTJiGdjWD6vnOWYAvGCjNp0qQkJFMqO1lWBE+vAhqiLdNKKefk5ORQWlYKnQk82IDpMDqhUZrTAlo5ZuLEiWBbSGdzn5/r6Wg4sgyllCO6OjsJeI8MCxC/HgwG3YqkVMYqLy9Hggl0yYiAsYz2f05zWkArx0yYMAE4Ugz3haejARFh3LhxDqdSKnt1tLeT16OAzo1d7+jo45zDSqlTqiivSKyA7or+0AI6vWkBrRwzevRoADxd/WiB7mphyNBhOryWUg7q6Owkz3ekgI5f1wJaKeeVlZXhCSZQVgWPPF6lLy2glWMCgQAlpWVIKNFxfI7whNqoGjE8CamUyl4dHZ3k+o5tgW5v7/947Uqp4ysuLsaEEphJJXTk8Sp9aQGtHDVixHA8wb4X0N5wO8OHawGtlFNCoRARyyLXe+Q+bYFWKnny8/MxYXPK2QhFZwTNCFpAK0eNGD4cX7iPrVu2jQm26xnJSjkoXiQf3Qf66N8ppZzTXRBbp3hguNfjVVrSAlo5asiQIRDq285ZIp1gDJWVlUlKpVT2iRfJgeN04ejsTGSsLaVUX3QXxOFTPDA2imReXl5S86jk0gJaOaq0tBRjhcFOfJxZCXd1P1cp5YyurujnKrfnMHbahUOppOk+Cf5ULdCx3wcCgaTmUcmlBbRyVElJCQAS7sM4s5Guo56rlBq4cDjaDObrMapW/LplnWoPr5Tqq5ycnOiVBApoj9eDz+dLeiaVPFpAK0d1F9CxojgR8cdqC7RSzokXyV7PkRbo+HUtoJVyXncBfarJeC3w+/1Jz6OSSwto5ajuYXkiR7dA5+x5i5w9bx33ORJ7bFFRUVKzKZVNugtoncpbqUHRXRT3+H4qawRZ02tyFbtHsa3Slh4/UI6Kn0QhVuio+z3t9Sd8jljRQ80FBQXJC6ZUlrHtaDNYz1239PqdUso5x2uBlqbop870HNtOW6AzgrZAK0fFi+B4UZwQK4TH49ETKpRyUHwHHe6x347Eduza+qWU87r7NJ/q+6mN9n/OAFpAK0d1tyL3aoE+GbHC5OblIyKnfrBSKiHxIjliH/lchWPXtfVLKed1f64SKKD1M5j+tIBWjoqPayl9GMYOK3xk+B+llCO6W6B77Mwj5ujfKaWck2gLtNiiLdAZQAto5agjfcD6MA60bZGbq903lHJS/Etp0DrSAh2/rl9YlXJevCgW+xRHU41+ic0EWkArR4kIfn9Onwpo7AiBgO7QlXJSYWEhAJ2RIzvzjtj1+O+UUs7xeGIllTn54zDg9XiTnkcllxbQynH+nBzETnycWW2BVsp5eXl5eERo71lAh7WAVipZvN5YUZxIAe3TAjrdaQGtHOf3+8D0YZgsY5Gjh7OUcpTH4yE/P6+7aAZtgVYqmRIuoG1tgc4EWkArx3k8HjCn2oIc5zlKKUcVFhRoFw6lUo0OOJURtGpRjhPxcOqv4D0ej9ECWqkkKCouProLR+y6zvqplPO6JyhKoEDWyYzSn1YtynEej/StBdpoAa1UMhQWFdMeOfLZag8LHo+ne7hJpZRzTKL7PQG7L90cVUrSqkU5zrYN9HFSlIQ3PEqphBUVFdFhHelr2RERCvLzdNIipZLAsmInz5/q4yVgW1pApzstoJXjwuEwSOJvLSNeQqHEZy5USiUmLy+PkH30ONDa+qxUcsT3Y8Z7igYhDwRDwUFIpJJJC2jluHA4DH04w9h4vHQFdWOilNNycnK6p++G6FTegfhkR0opR3V2dkavnGL3Z3yGjo6O5AdSSaUFtHJcJBLGSB+G6PF4CYXCyQukVJby+/2EewzJHraj47QrpZzX1dUVvXKqWbp9PR6r0pYW0MpR4XCYSDgMnlNtQXrw+PTbuFJJcGwBLTqFsFJJ0t0CrQV0VtACWjmqtbUVAONLfGpu4wvQ2tqSrEhKZS3Lsug5wI1HzJETnZRSjmpubo5eOdVBnhwIBUMEtetiWtMCWjmqpSVaCBt/4lNzG18unR0dRCKRZMVSKiuFw2H8Pbbyfg+E9YRdpZKiqakpeuVUu7/cXo9XaUkLaOWoeAGNty8FdPSxbW1tyYikVNYKh8P4emzlfR5DKKznGyiVDI2NjYhfTtmFwwSio3Q0NDQMQiqVLFpAK0fFNwjGn/hQWcYf/Tre2NiYlExKZatgMIjfc2RIrRwPBIPa91KpZGhsbERyExhjPXDk8Sp9aQGtHFVbWwuAnVOQ8HNMTiEAhw8fTkompbJVY2Mjxb4jfZ6LcmyaW1p1GmGlkqDmUA1WbgLnGORHf+g+L71pAa0cVVtbi3h84OtDF45YsR0vvpVSzmhsqKc458gOvSTHYFm2dpdSKgkOVh/EFCQwq24uiEeorq5OfiiVNFpAK0fV1tZCoKBPU3kbf/TruH4bV8pZjQ0NlOQc2aEX50RbnrXvpVLOCofDNNY3drcun5SAFAg1NTVJz6WSRwto5ajq6hoi/kS2ID14PEigkEOHDiUnlFJZKBwO09zaRknOke4apbFiur6+3q1YSmWkQ4cOYYxJrIAGrDxLW6DTnBbQylEHDh7EDhT1+XlWTgEHDx5MQiKlstPhw4cxxjAk70gBXRm7ri1fSjkrvv8yhQl04Yg9bv+B/cmMpJJMC2jlmK6uLpqbGjH9KKDtnCL2H9ACWimnxFu3KnOPFNDlARsRtOVLKYcdOHAgeqUwwScUQltrW/fkYyr9aAGtHBNv1epPC7SdW0RDfR1hHaNWKUfEP49De7RA+zxQkact0Eo57cCBA4hPuidJOZV4S7UeeU1fWkArx8RbtfrTAm0CRRhj9ERCpRxSXV2NR6AscPSQdZU5Eap1p62Uow4cOAAFQKLnz8daqvfv124c6UoLaOWY+EmAJpDoMawj7NhY0NoyppQzqqurqcwDb6+t/JA8i+rqA+6EUipDHTh4ALugD+Orx6ZK0O5U6UsLaOWYmpoa8Hi7h6Xri3jRrSNxKOWMmupqKgPHdokakmdT39BEKBRyIZVSmccYQ3V1NSY/sRMIAfCBJ9ejjUZpTAto5ZiamhokUNinMaDj4pOp6MZEKWdUVx886gTCuMpcG2OMfllVyiEtLS0Eu4LdrcqJsvNt3eelMS2glWNqa2uJ+PL692SPFwnkU1dX52wopbKQbds0NjZRdpwCujzWJ1rHglbKGfEiOKFZCHuw820OHNTuVOlKC2jlmLr6hn5134gzvjydIU0pB7S0tGAbQ7H/2B16UWwylaampkFOpVRm6t5vJTgCR7dcnRU0nbleQIvIf4rIRhHZICIPi0hf34IqRTQ1NmL8/WyBBiK+XOp1Y6LUgMWL46KcY1ug49N5Nzc3D2YkpTJW99GcfhTQnR2dBINBxzOp5HO1gBaRkcA3gPOMMdMAL/ARNzOp/gkGg3R2dgyogDb+POrrtYBWaqDiBfRxW6Bj9zU2Ng5mJKUyVvdnqR8FNGgrdLpyvQUa8AF5IuIjOou8DlCahuKzKRlfoP8L8eXS1tbmUCKlsld8hI0c77EFtM8DXkFH4VDKIY2NjYhfok2AfWAC+mU2nblaQBtjDgC/B/YC1UCzMeal3o8TkS+IyAoRWVFbWzvYMVUCOjs7ATBeX7+XYTw+gl2dGNO3EzGUUkezLAuIFsrH4/EceYxSamA6OjqQnL6PPoX/yPNV+nG7C0cZcAMwHqgCCkTkE70fZ4y50xhznjHmvCFDhgx2TJWAeAGNx9//hXj9GGO0ZUypAYoXx54TFdAiWkAr5ZDOzs7osfS+8vV4vko7bnfheBewyxhTa4wJA08AF7mcSfXDkRbo/hfQJlZ867dxpQYmfhTnRG1iHokOdaeUGriOjg5sbz8+T1pAp7WECmgRWX2C+1cMcP17gQtFJF9EBHgnsHmAy1QuCIdjM55JHzuB9eTxHL0spVS/5OTkABA+wT49ZBkCgQGcr6CU6tbZ2Yk5zvkGp6QFdFpLtAV6Uu87YgXvhIGs3BizFHgMWAWsj+W5cyDLVO7obs3qxyyE3cRz9LKUUv2Snx8dj73LOvbzGLbBso88Rik1MOFIuH/H82PPiUQijuZRg+OkvXZE5J+xqzk9rseNAzYONIAx5qfATwe6HOWu7v6UAymgY7SAVmpg8vKiw0l2Ro79PHbF7os/Rik1MJFI5MT9pU4mVkDr+Qjp6VTd3nec4LoB3gDmOJ5IpaUjI2f0v4A2seJbNyZKDUy8dbnzOC3Q8fu0BVopZ0Qikf61QMc+nrrPS08nLaCNMT8HEJElxpgXByeSSkfdBfSAWqAH3nqtlIKioiIA2sPHfqbi98Ufo5QamH63QEuP56u0k+h3pvNjfZ67xU78uyMJmZRSSg1AYWEhHo+HtuMU0PH7SkpKBjuWUhnJsiyMpx8nEWoXjrSWaAH9buANEZkAICIXAeuA4mQFU+ml+/vVgCZBMUcvSynVLyJCcWEBbeFjN/GtsQK6uFg330o5wbKsAbVAawGdnhItoN8BzAWWi8j9wFPAj40xH01WMKWUUv1XVFxM+3FOImyPFdXahUMpZ/S7DzQgHtEuHGkqoX+5McYGHgdqgX8DXgeeTl4slW4cGcYu9nVcp/JWauDy8vIIHqdhK36fjsKhlDMiVj/7QAN4dO6DdJXoRCpfIzrqxh3AKKLH2teKyIVJzKbSSPwQlHFgHGj9Nq7UwOXm5RM8zigc8ftyc3MHO5JSGamrs6t/U3kD4hedSCVNJfov/yzwDmNMfNznfxeRm4BngSFJSabSypE+XAOYHV6HsVPKMXl5ebTbx34eQ7YQyPHj8Qzgs6qUAqJHX4NdQfD3cwE+nYkwXSVaQM8yxhx1jMEYc7+IvO58JJWOuluNZSAFtLZAK+UUr9eLZY5tgbYMWjwr5ZDu4refLdDGZ+jo6HAukBo0iW5FIyLyeRF5TUTWAYjIO4DZyYum0kkwGIxe8Xj7vQzjiW6Burq6nIikVFbr6uoix3PsrJ4BjyEYCuu5Bko5oK2tLXqlny3Qts+mtbXVuUBq0CRaQP+CaDeOO4Exsfv2A99LRiiVfuIbEePL6f9CvNHntre3OxFJqazW1dlB4Dhj0+Z4o4ed9cQlpQaurq4OAJPXvy+kJs9wuPawk5HUIEm0gL4ZuN4Y8wjxwXphFzAhGaFU+mlra4v2Yfb0tyPYkeJbC2ilBq6rq4uA99idevw+PdKj1MDV1tZGr/R3UJs8qK+r1yNCaSjRAtoLxI5TdBfQhT3uU1muvb0d8QUGNIydibVA6+EspQauvr6eopxjd8pF/uh9DQ0Ngx1JqYxz+HCs9Ti/nwvIi57309zc7FgmNTgSLaCfB24TkQBAbFrvW4iOwqEUjY2N4AsMbCGxArypqcmRTEplq5aWFppbWhmef+yINvH79u3bN9ixlMo4NTU1iE/63Qfa5Ee/0B48eNDBVGowJFpAfwsYATQDJURbnscC309SLpVmDh06TMRfMLCFiAfJKThySEwp1S/79+8HYHj+sScRxu/TAlqpgdu1axem2PR/IpXi6I89e/Y4lkkNjoQGXjHGtADvF5GhRAvnfcaYmqQmU2nl0OHD2DmlA16O5c/n0KFDAw+kVBbbu3cvACOO0wJd4DeUBKT7MUqp/tuxawd20bFfVBNWCOIVdu3a5VwoNSgSnYlwNYAx5rAxZnm8eBaRFckMp9KDZVk0NtRjcvrbCewI259PjRbQSg3Ipk2byPUJw/KOv2MfUxhi08YNg5xKqczS2tpKU0NT9Lh8fwlQBDt37nQolRosiXbhmNT7jlg/aB2FQ3H48GFs28YEiga8LBMo4vCh6PKUUv2zbu0aJhWH8J5gC396aYTde/bqiUtKDcCOHTsAol04BsAqtnh7+9tORFKD6KRdOETkn7GrOT2ux40DNqKyXry/pZ1bPOBl2bnFhMMhamtrGTZs2ICXp1S2aW1tZdfuPXxg/Iln9DytNPq79evXc8kllwxWNKUyypYtW6JXyga4oDJo3NtIXV0dlZWVA86lBsepWqB3xC49r+8AtgMPAjckL5pKF/EC2uQO5DhWlB1bRnyZSqm+WbduHcaY7iL5eCYUR/B5YO3atYOYTKnMsnXrVjwFHsgd2HJMuelenkofJ22BNsb8HEBElhhjXhycSCrd7Nu3D/H6Mf6B94E2sVbs/fv3M3PmzAEvT6lss2LFCnK8wqSSExfQOV6YUhJhxfLlg5hMqcyyafMmrNJjT9Tts1JAogX0xRdfPPDlqUGRUB9oLZ7Vyezfvz/afWMAk6jEmZwCxOvTIbaU6qfly5ZyWmmIHO/JHzetPMyOnTt1QhWl+qG1tZXqg9WYMgdmEPSBFMuRLiEqLSR6EqFSJ7R7z16swMD7PwMggskt0QJaqX44dOgQe/ftZ3p5+JSPnV4RfcyKFTqYklJ9tW3bNgBnCmjAKrXYvGWzTumdRrSAVgMSDoc5fKimu++yEyKBYnbv0TFqleqrVatWATCt/MTdN+LGFlkU5QgrV65MdiylMk68gB7wCYRxZdDc1ExdXZ1DC1TJpgW0GpCDBw9Gh7BzsIC2c0s4VFNNJHLqIkApdcSGDRso8AujCk/dL9MjMKk4yIb16wYhmVKZpfsEwoAzy4u3ZOuJhOkjoZkIAUTkauBsoLDn/caYnzicSaWRgwcPAs4MYRdncouxbZvDhw9TVVXl2HKVynQb1q9jYnEIT4KnI0wujbB6+wGam5spKXHuS7BSmW7b29uwShw4gTCuNPpjx44dOrRkmkh0JsI/Aw8AM4HRPS6jkhdNpYP4tNsmUHiKRybOzokuq6ZGZ4tXKlFtbW3s3rOXyScZfaO3ybECYNOmTcmKpVTGiUQi0RMIBziBylF84CnwsHevdl9MF4m2QH8MmGGM0TO71FFqampAPI4MYRcXL8YP6ZTeSiVsx44dGGOYUJx4AT0+9tht27Yxe/bsZEVTKqNUV1djWRYMfPLdo1gFFnv27nF2oSppEu0DXQc0JTGHSlM1NTVIbpEjQ9jFmZwCILqRUkolpra2FoDKPDvh5+R6oTgg3c9VSp1afJQoU+TsiBmmyLB3714diSNNJFpA/wF4UERmi8iEnpdkhlOpr76+nohvgNMw9ebxIjl5Oj6tUn0QL4LLAokX0NHHW1pAK9UH8XN/cK7nYvfyujq7aG5udnjBKhkS7cLxt9jP63vdb4BTDNevMllTczPG6QIaML5cWlpaHF+uUpmqtraWPL+Qf5yt+v1b8wC46bTOY35XnhOh9rB2l1IqUY2NjSBAjsMLju1Km5qaKC0tdXjhymkJFdDGGB3uTh1XU1Mzxj/U8eVa3hyampocX65SmaqtrY0C//F/t6f1xO0cBT5DdWtrklIplXmamprw5HqwxMFROACTa7qXr1KfFsaq34wxtLW2YnwODYTZc9m+XBp1I6JUwnJycgj3Y38esoWcgNNNaUplrsbGRkwgCf2UA0eWr1LfCVugReQFY8y7Y9cXEe2ucQxjzDuSlE2luHA4jGVFwOv8ztd4c2hv135gSiUqJyeHUN+6PwMQtiEnx/kvwUplqtbWVmx/Pz5spxLblWr3xfRwsi4c/+xx/e/JDqLSTzAYBMB4Ep6PJ3EeL6HY8pVSpxYIBAhZfW8VC9tCINf58xiUylTBYDA5x+9jPa3C4XASFq6cdsLKxxjzUI/r9w1OHJVO4gU0niScR+rxEQqFnF+uUhmquLgYy4b2sFDgT7yQbgl7GVmssxAqlahgKJic4RNiRbnu+9KD9oFW/db9IU9CC7TxeHUjolQfjBw5EoDDnYlv1o2BQ53e7ucqpU4tFAphPEnoAx376GoLdHrQAlr1W7zANZ4kvI3Ei21Z2HYS+pkplYHiRfChjsQ/j80hIRgxWkAr1QfhcDg51ZMHEG2BThdaQKt+i0RiUwZLEo5liefodSilTqqqqgqItignKt5aHX+uUurUbNuOjgOdDII2HKUJLaBVvx0poJ1/G8VbtbWAVioxeXl5VFaUUd2HFujq9mixPWbMmGTFUirjJLOAFhEtoNNEQltaEfmWiJwdu36hiOwVkV0iMjup6VRK6+6nlYQCOr5M7QumVOLGjh1PdUfi5yQc7PDi93kZPnx4ElMplVks20pqC7RlOTtBi0qORCuf/wR2xa7/GrgNuBX4YxIyqTSR3GHsfEetQyl1amPGjuVghw+T4PlNB9s9jBo1Cq83GUMKKJWZIuFI0o7fi1f0yGuaSLTyKTHGNItIETADeJcxxhKRPyQxm0pxnZ2d0SveZIzC4T96HUqpUxozZgydYUNzSChNYKa0ms4cppw5LvnBlMogwa5g4tVTX/l0v5cuEn0L7BORi4AzgYWx4rkY0OMMWSz+ITdev/MLjy2zo6PD+WUrlaEqKioAaA55KA2cevPcHBIqKyuTHUupjBEOh6MtxEnY7QHg0/1euki0gP4O8BgQAj4Yu+96YFkyQqn00P0t2eP8lsTEWrV1Q6JU4oqLi4HoZCqnYtnQETYUFRUlO5ZSGaN7v5ekFmjba2sLdJpI6C1gjHke6D3O0ZzYRWWpxsZGAIwv4PiyjS8PgKamJseXrVSmKimJzijYlkAB3R6Ro56jlDq17n1STnKWb3IMDY0NyVm4clTC36FEpAQ4DSjs9avXHE2k0kZtbS2Sk5+UqbxNTn73OpRSicnPj35uOiKnLqA7Y4+JP0cpdWrV1dUAmIIkzEQYW258HSq1JVRAi8jNwF+ANqDnMXUDTHA+lkoHhw8fxvYnaefrzUG8furq6pKzfKUyUGtrKwAF/lPv3OOPiT9HKXVq3cVtQZJWUADtbe20tbVRWNi7vVKlkkQHYvkl8G/GmGHGmPE9Llo8Z7FDhw9jJauAFsEECjh8+HBylq9UBop3qyrJOfVEDAU+g9dz5DlKqVM7ePAg4hXITc7y4y3b2gqd+hItoH3AS8kMotJLJBLhwP4DmNzi5K0jp4jde/YkbflKZZqGhmjfydKcU7dAi0BJQLqfo5Q6tZ07d0Y7siZrIpWiHutRKS3RAvq3wI9EkjHlnEpHBw4cIBwOYeeXJ20ddn45e/fu1dkIlUrQ/v37Y4VxYlMBl+dE2L9/X5JTKZUZbNtm/Yb1WOVJHMG3GMQvbNy4MXnrUI7oy0yEPwJaY9N4d1+SmE2lsB07dgAkvYC2LYu9e/VtplQiVixfxqQSi0CC5/WeURZi06ZNtLe3JzeYUhlg7969dHZ0QkUSVyJgl9usW78uiStRTkh0FI5PJDWFSjs7duwAEey80qStw84r717XxIkTk7YepTJBS0sLW7Zu48ZxoYSfM708wrO7bVavXs0ll1ySxHRKpb8NGzYAYCqTMwJHnCk37Nqyi46ODh0lJ4Ul1AJtjFlwokuyA6rUtGnTJsgvB0+y5jMFk1eCeP3RdSmlTmrlypUYYzirIvEuT1NKIwR8wrJlOieWUqeycuVKPLmeYwfzdZgZYjDGsHbt2uSuSA1IQgW0iPhF5OcislNEumI/fy4iSRpKXKWySCTCho0bCRcMTe6KxEOkYAhr1uqhLKVO5bXXXqUoByYUJ94/0+eBaWVBFi54PTo9sVLquCzLYsnSJUSGRZJ3AmFcJYhPeOutt5K8IjUQifaB/h3wLuBLwIzYzyuJnlyosszOnTsJdnVhFQ1L+rqsomHs2rlDp/RW6iTq6+tZvHgxl47owtvHU70vrwrR0NjEG2+8kZxwSmWATZs20d7WDiMGYWVesIfavPnWmxiT3O4iqv8S3dR+CHifMeYlY8xWY8xLwPuBDycvmkpV69evB8AuGp70ddmFwzDG6BnJSp3E888/j2XZXDky2OfnzqgMU5EHzzz9dBKSKZUZlixZAgJm2OAUtGaE4fChw+zRoVxTVqIF9IkOWCT7QIZKQevWrUMChZhA8mdJsgqHggjr1mk3DqWOx7Ztnn3maaaWRxien9jwdT15BC4f0cnyFSs4ePBgEhIqlf7eePMNqAQGqeOqGR4t1LUbR+pKtICeAzwrIteIyBki8m7gKeDRpCVTKckYw+o1awkXJr/7BgC+HExBJWv0ZAqljmvZsmXUHDrMlSO7+r2My6qCeASeeeYZB5MplRkOHTrEzh07sUf0/Qtqv+WDlAmL31g8eOtUfZJoAf1d4BXgL8BK4M/AfOB7ScqlUtTBgwdpamzAGoTuG3GRwmFs2rhJJ1RR6jiefPJJSgJw3pD+fz7Kcw3nVoaY+9yzBIN97waiVCaLtwKbEYPbH9kabrFh/QZaWloGdb0qMYkOYxcyxvzEGDPJGJMf+/ljY4xuabNMfFidwSygraLhhMMhtmzZMmjrVCodVFdXs2TJEi6v6sQ3wHli3zUqSHNLKwsW6OikSvX0xhtvIEXSPc32YDFV0eHslixZMrgrVgk54SZXRN7R4/qVJ7oMTkyVKjZv3oz4cjBJnEClN7twaPe6lVJHPPfccwimXycP9ja1PMLwAsPTTz018GBKZYhIJMLqNauxhlqDf9ZXGUhAWLVq1SCvWCXiZLNg/BWYFrt+9wkeY4AJjiZSKW3zli1E8itABm9LYnLykUAB27ZtG7R1KpUOFi14nTPKIlTkDvzQskfgkmFdPL5xI42NjZSVlTmQUKn0tm3bNkLBEGaIC8PJCdgVNqtWawGdik7YAm2Mmdbj+vgTXLR4ziKRSISdO3diFVQO+rrDeeVs3qxdOJSKq66uZvfefZxTmfjU3adydmUYYwxLly51bJlKpbPu2QCHuLN+M8RQU11DXV2dOwHUCSU6E+FxBwgVkScGGkBESkXkMRHZIiKbRWT2QJepkmP37t1EwmFsFwpou6CS/fv36YQqSsXE+0XO6MPU3acytsiiNFeHzlIqbt26ddH+z7nurN9URlu+dVrv1JPoaSdXnOD+yx3I8L/AC8aY04nOcqgdXVPUvn37ADC5pYO+bjuvDGMM1dXVg75upVLR8uXLGZpPv8Z+PhERmFEeZNnSJToDmlLAtu3bsEot9wKUAhKdAVillpP1gUZEfhG7mtPjetwEYEBT5IhICfAO4GaIjvYBOHc8UjmqpqYGADswyKciQ/ekLdXV1UycOHHQ169Uqqk+eICR+SHHT0cYWWCx4GAn7e3tFBYmf7IkpVJVOBym7nAdnO5iCA94Cj06yVEKOlUL9OjYxdPj+mhgFLCP6BTfAzEeqAXuEZHVIvJ3ESkY4DJVklRXVyP+XPAN0lRMPcSL9ngRr1S2a2pqosjvfCtxYWyZTU1Nji9bqXRSU1MTPRLj8vdIK99i//797oZQxzhpC7Qx5tMAIvKmMeauJK3/XODrxpilIvK/wPeBH/d8kIh8AfgCwJgxY5IQQyWipqYGO8el7ze+XMTr1y4cShGdEbSlpZXikc4X0MU50S4hzc3NjBo1yvHlK5Uu4q2+psDd7kymwLD/gBbQqSbRiVTuAhCRIhEZLyIT4pcBrn8/sN8YEz/l+zGiBXXv9d9pjDnPGHPekCEunQqraGpuxvK5dCaFCPhzdUYmpYBgMEg4EiHP5/yOPT+2zNbWVseXrVQ6aW5ujl5xabfXLRfa29qx7UGcSlydUqKjcJwhIquBZmB77PJ27NJvxpgaYJ+InBa7653ApoEsUyVPe3sHePyurd94/ToKh1JAIBAgLzdAc8j58dibgtHdgo4DrbJdKBQ7Jcvrbo74+sNh50bcUQOX6CgcfwPmA+VAC1AG3AF8yoEMXwceFJF1wNnArxxYpkqCjo4OjNe9AtoWH52dna6tX6lUISKMGDGC2s4Bzt99HLVd0WVWVVU5vmyl0kkwGJvhM0UK6O48KiWctA90DzOAq4wxYRERY0yziHwH2AA8MJAAxpg1wHkDWYYaHJ2dnVDoXhca4/HT1t7u2vqVSiUjqkayZ90ux5db2+mhID+PoqLBH21HqVTS3QLt/PfUvtECOiUl+rboAuJNj3UiMib23IqkpFIpyYpEQNz7Km48HiLhiGvrVyqVjBw5kkMdQsThbpEHO3yM1JMHlUodsVMdxOkxK9WAJFpALwI+HLv+GDAPWAC8loxQKjUZDMbVz69g0MkdlAKYNm0aYQt2tzr3pTZiw45mP9Onn+XYMpVKVzk5sSFb3T53L7Z+v9+9LpTqWAl14TDGfLjHzf8GNhIdGfGfyQilUpOxDeBiBS2CbWsBrRTAjBkzANjS6GNSiTMzpe1u9RK0TPeylcpmPl+sRNICWh1Hon2guxljbOD+JGRRKS76r3f3EJJtuzilqlIppKysjNGjRrK1aRfX40zfyK1N0V3CWWdpC7RSKdMCHdvtdedRKSGhAlpE7ofjHzs3xnzS0UQqZVmWhePzBveFeKIZlFIAnDXjbBa+fBBj2h35aG5v9lE1Yjjl5eUDX5hSac7rjXWPcruAjlVf3XlUSki0D/R2YEePSzvwHqAhSblUirEsKzqlqcfFD7B4dBxMpXqYPHkyrSFDQ9CZL7Z72nI47fQzHFmWUunO43F7+I0YEz2BUE8iTC2J9oH+ee/7RORu4KeOJ1IpKRKJjX4h7m1QjHiwLB2FQ6m4yZMnA7C31UdF7sC+XHZE4HAH3DBpkhPRlEp73QVrCpx6o8Vz6hlINbQGuMyhHCrFxQto43IXjrAOY6dUt4kTJyIijozEsbc12p4SL8qVynbdLdBuF9BGC+hUlGgf6Ct73ZUPfASddjtrtLa2Rq94A65lML4cOpraMcboxkQpID8/n8L8PFpCA5+hMz4t+JAh7k2WpFQq6Z751u2ux95oN8pQKKQnEqaQREfhuLvX7XaiLdAfdTSNSlkNDdHu7saf61oG488jHA7R0dFBQUGBazmUSiXhSAS/Az2r4svQ8wyUimpqaopeca/d6Kj1Nzc36xfcFJJoH+jxyQ6iUltjYyMQLWJd44uuu6GhQQtopWJC4Qg+z8CPMftjy9ACWqmopqYmxCv9GPDXWSZguvNoAZ06Emq3EJHVJ7h/hbNxVKpKhQI6vu54a7hS2S4SiWDbtqMt0F1dXQNfmFIZoLGxEckVt6c/6G6Bju+HVWpIdLN7zGnZEu2EOsHZOCpV7d69G/H4MDnutfzaucUA7N2717UMSqWStWvXAjCyYODjo4/It/AIrFu3bsDLUioTbN+xHcuBz9aAFUV/7Ny5090c6ignPTAhIvGpunN6XI8bR3RKb5UFtm/fjp1f5u4wdoEixJfD9u3bXcugVCp57bXXCPiEGZUD73ZREjCcXhrhtVde4TOf+YyeqKuyWjAYZNfOXZgpbg/BAeSCFAibN292O4nq4VTVUHzilJ7XdxCdWOVB4IbkRVOpwhjD29u3E8lzeXYyEay8Mra9/ba7OZRKAZFIhNfnv8a5FUECDo0ScMGwIPsOHNAvqSrrbdu2Ddu2MRUpUEADVpnFho0b3I6hejhpC3R8AhURWWKMeXFwIqlUU1tbS2tLC/bYM92OgpVfwfbt27EsS6c1VVlt8eLFtLa1c+HEkGPLnDU0zH1b4fnnn+eb3/ymY8tVKt1s3Bg7wF7mbo5uFVC7tpa6ujoqKyvdTqNIsA+0MeZFETlNRD4sIp/peUl2QOW+NWvWAGAXDXM3CGAVDiXY1cW2bdvcjqKUazo6Ovjzn/6PkYWGGRXOjZpRlGO4rCrIk08+ydt6pEdlscWLFyMlAi4OPNWTGRptCX/zzTddTqLiEh2F47+BtcC3gZt6XD6RvGgqVSxfvhzJycPOr3A7ClbJSABWrNABYFT2+sc//sHh2jo+e3orPodPS/jIpE4K/Tb/73e/xbJS4AQqpQZZfX0969evxxqZQu//EpAiYf7r891OomIS3fT+BzDLGHOBMeaKHpfeMxSqDGOMYdmy5YQLR0AqnFTkz8MUVLBs+XK3kyjliq1bt/LYnDlcOTLIlFLnd/AFfsMnJrexZes2nnrqKceXr1SqW7RoEcYYzKjU6P8MgIA10mL1qtU0Nze7nUaReAHdCWxJZhCVmnbv3k1jY0N3y28qCBdXsWH9ejo6OtyOotSg6urq4te/+iXFAcO/Txr49N0nMntYmOkVEe64/XYdNlJlndfmv4YUCxS7neRoZpTBtm0WLVrkdhRF4gX0j4E/icgIEfH0vCQznHJfvL+VVTLK5SRHWCWjsSyLlStXuh1FqUFjjOG2225j167dfP70Vgr8yWsdE4HPndGG3wT50Q//m87O5BXrSqWSmpoa1q5ZizXKcn8Cld5KQYqFefPmuZ1EkXgBfS/weWA/EI5dIrGfKoMtWrwYUzgEE0idqbPtouGIL8Abb7zhdhSlBs3cuXN54YUXuGF8JzMqI0lfX0Wu4StntrJnz15+//vfY0wKHc5WKklefPHFaPeNcSn4fhewxlqsX7+e/fv3u50m6yVaQI+PXSb0uMRvqwzV2NjI5k2bCJeMdjvK0TwewiWjWLz4DT3JSWWFt99+m//5n9uYVh7hAxMGb6rt6RURPjChk5dffplnn3120NarlBuMMcx9fi4MBVKnzegoZqwBiRb6yl2JDmO350SXZAdU7nnrrbcwxmCVjXU7yjEiZWNoaWnWmZlUxuvq6uJnP/0Jhd4IX57WhmeQDyvfML6L6RUR/vd//8ju3bsHd+VKDaINGzZQU12DPdZ2O8qJ5YEZZpg7by62ncI5s0Ciw9jdLyL/PN4l2QGVe5YtW4YECrDzXZ6B8DisklEgwrJly9yOolRS3XXXXezbf4AvntFKSc7gH1b2CHxxahu5EuGXt95CJJL87iNKueG1115DvIIZmYLdN3owYwx1h+u0AclliXbh2M7RU3m3A+8BGpKUS7nMsiyWLltOqKgqNYav680XwC4cwtKlWkCrzLV69WrmzJnDVaO6mFbhXuFaGjDcfFobW7e9zYMPPuhaDqWSxbZtXpv/GvZwG/xupzk5U2UQjzB/vo4J7aZEu3D8vNfly0QL6InJjafcsm3bNtrbWlNq+LreIsUj2bJlM62trW5HUcpxHR0d/PpXv2RYvuHfJ7s/CsYFw8JcOCzEvffeo7MUqoyzfv16GhsaU2vs5xPxgz0sVvBrNw7XDGQYujXAZQ7lUCkmPtNfKhfQVskojDE6nJ3KSM8//zw1hw7z+TPayPW6nSbqU6d1kO+1uffee92OopSjFi5cGO2+MSINCmiiY0LX1daxdetWt6NkrUT7QF/Z63I90aHtNiU1nXLN5s2bIb8U/HluRzkhu6ASPF7tB6YyjjGGuc89y4Rim9PLUqfPcVGO4dLhXbz55hs0Nja6HUcpx6xZuwa7IvW7b8SZYdFCf/369S4nyV6JtkDf3evym9j9H01GKOW+zVu2EsmrcDvGyXm8mLwytm3b5nYSpRy1bds2duzcxTuq3O+60ds7qoJYls1LL73kdhSlHBEMBtmxYwemPD1anwHIA0+Bh40bN7qdJGsl2gd6fK/LNGPMJ4wxu5IdUA2+pqYm6utqsQqcKaBz9ryFp6MeT0c9uZueI2fPW44sFyCSX8GWrdt0kgeVUebOnYvfK8welnpzVY0qtJlYYjH3OR0XWmWGbdu2YVs2psLZ/YisEWgCmsDzuid620GRsggbNm5wdJkqcQn3gRaRUhH5uIh8J/azLJnBlHviJwjZ+ZWOLM/TXo9YYcQK422twdNe78hyAeyCCtrbWjl8+LBjy1TKbTu2b2diUSip03UPxFnlIXbv2asTGamM0N0N0OERW6VJkHDsUitIk8MjWlVA7eFa7U7lkoT7QAO7gW8A5wNfB3aJyDuTF025paamBgCTW+xyklOzYxmrq6tdTqKUc8TjGfQJU/oiPrKlxzOQ89CVSg21tbWITyDX7SR9YwqiX7C1AckdvgQf92fgC8aYR+N3iMiHgL8ApycjmHJPbW0tAMaf73KSUzP+6Hyr9fXOtWor5TaPx4Odmo3PAMR7TEkqjhGvVB81NTUhgTR8LweiP5qbm93NkaUSbT6oAh7vdd+TwHBn46hUUF9fjwTyIQ1al0xOtMiPF/1KZQIRwTapu0O3jRbPKnM0NTVhAin8jfVEYgV0U1OTqzGyVaIV0v3AV3vd92VAp/LOQHV1ddi+1B2+7ijeHMTrp66uzu0kSjlm5MiR7Gv3E07RORJ2tvqoGqHtJyozNDc3Y/tT9MN2MtoC7apEC+hzgD+IyH4RWSoi+4E/AOeIyML4JXkx1WBqaWnB9gbcjpEYEfAHaGtrczuJUo659NJL6YwYNjUk2stu8LSHhU0Nft5x2eVuR1HKEZZtDWxaObfEDgLpbITuSHTrfFfsorJAe3sHxpsmo8kDxuOno6PD7RhKOebcc88lLzfAitogMypTZyIVgDV1fiwTLfKVygQe8UAa9uBAz0VwVUIFtDHmvmQHUamjvaMD4y1yO0bCjMenBbTKKIFAgAtnX8TKN+dzs92BN4Vax5Yf9lNRXsrUqVPdjqKUI8Qj6VlAx+hoOO7QV10do7OzEzzp0wJte/y0t2sBrTLLu971LlqCsK4+dT6LrSFhdX0OV1z5Lt1pq4zh8WgLtOo73QKqoxhj6Ohox/hy3I6SMOP109rW6nYMpRw1e/ZsykqKWXAwdT6Lb9TkYNlw3XXXuR1FKccI6V2AagHtDi2g1VE6OzuxLQvS5SRCwPgCtLRoAa0yi8/n45r3XMvquhyag+7vII2BBdW5nH7aFCZOnOh2HKWUctUJC2gRWdLj+k8HJ45yW2trtBA1vvQqoNt1FA6Vga699losA8sOu98KfbDdw75WD++5VluflVLqZC3QU0QkPrHltwcjjHJffDzJdOrCgTdAOBwiGAy6nUQpR40dO5b8vFyqO9w/WHig3QvAmWee6XISpZxlm/QeBk6HsXPHyUbheBrYJiK7gbwTjfNsjHlHMoIpdxw8eBAAE0ifUTjsQCEABw4cYMKECS6nUco5IkJVVRWHm1vcjsLhzmgRX1VV5XISpZxVV1+HyU3DswhzAIHGxka3k2SlExbQxphPi8glwDjgfODuwQql3LNnzx4A7NwSl5MkzuSVAdHsWkCrTFM1chQ7Du0E2l3NUdvppbiokMLCQldzKOUk27ZpqGuASW4n6QcBT76Hw4cPu50kK510HGhjzGJgsYjk6FjQ2WHPnj1IbhGk0UQqdl602I8X/0plkkAgQEvY/ZMIW8NCIBDAGKNn/auM0djYiGVZkO92kv6xc20toF2SUMc6Y8w/RORyEfmHiLwY+3lFssOpwbdz5y7CgfRpfQbA40Pyitm1a5fbSZRyVFdXF28sXsSM8i63ozCjIkxtXT0bNmxwO4pSjunutpifhl04ADvfZu/+vRiTnvnTWUIFtIh8DngUqAGeAKqBh0Xk80nMpgZZQ0MDO3fuwC4a5naUPgvnD2X5ipXRlgSlMsTChQvp6OziHSNCbkdh1rAQAa8wb948t6Mo5Zjly5eDABVuJ+mnSqivrWf//v1uJ8k6iZ7a/V3gKmPMfxtj7jDG/BC4Ona/yhBLlkRHLrRKx7icpO8iZaNpa21h06ZNbkdRyjHPz53LkHw4vSzidhTyfDBraBevvfpKdLZSpTLAm2+9GS2e02fk1qOYEdGW57feesvlJNkn0QK6AuhdmWwFyp2No9z05ptvIoFC7Pz0+7daJaNARDciKmM88MADrFq9miuqOvCkSJfjK0YG6ejs4uc//xnhcNjtOEoNSH19Pdu2bsMensbDwBWAFOu+zw2JFtCLgdtEJB9ARAqA/we8maxganB1dXWxdNkyQrFCNO34AlhFw1mwcKH2BVNp75FHHuHOO+/kouFBrh+bOuObTym1uPn0dt588y1+9rOfEYm43zKuVH8tWLAAONKKm66sERZr1qyhoaHB7ShZJdEC+kvADKBZRA4BTbHbX0xSLjXIXnnlFYJdXUQq0neK3kjFRPbt3cu6devcjqJUvz322GP89a9/ZdbQEF+cmjqtz3HvGhXiE1M6WLRoEbfccosW0SotRSIRHvnXI9Hj6Gl23nxvZrzBsiwef/xxt6NklURH4aiOTZgyHngvMN4Yc5kx5mBS06lBYYzhscefgIJy7KLhbsfpt0jFRMQX4Mknn3Q7ilJ9ZozhX//6F//3f//HzCEhvjKtHa/7ExAe17vHBPno5A7mz5/PrbfeSleX+6OEKNUXCxcupKa6Bus0K3oSYTorAjPS8PgTj9PR0eF2mqzRp82zMWa/MWaZMUZP98wgGzZsYOeO7QSHTk3P7htxXj+hysm8vmABdXV1bqdRKmENDQ18/3vf4y9/+Qszh4T5+vR2fClaPMddNzbIv0/q4LXXXuNzn/0MW7dudTuSUgkxxvDAgw8gRQIj3U7jDPt0m472Dp555hm3o2SNFN9Eq8Hw+OOPI75AWnffiAsPOwPbsnQjotLGG2+8wc2f+iQrli3hpikdfPOstpQvnuPeOy7I989ppbV2P1/60hd54IEHdChJlfKWLFnC9re3Y03JgNbnuHJgKDz08EM6Ss4gSZPNtEqWAwcOMH/+fIJDpqTV7IMnYnJLiJSNYc5jeihLpbbOzk5+//vf84Mf/IBiu4lbZjVzzZhgyvV5PpVpFRF+fUETMyu6uPPOO/nmN75BTU2N27GUOq5IJMKf//JnpEgw49L75MHerDMtmhqbeOSRR9yOkhVOWUCLiEdErhSRnMEIpAbXQw89BOIhMny621EcEx5xNu1trTz77LNuR1HquDZu3MjnPvMZnn3mGa4d28XPz29mVGH6DqVV6Dd8fXo7X5zaztubN3Dzpz7FvHnzdEQclXLmzp3Lvr37iEyPZF4TYiWYUYYHH3pQuzEOglO+fYwxNvC0Mcb9qbCUo2pra3l+3jxClZMxOflux3GMXTQUq3gEDz30MKGQvm1V6ggGg/ztb3/jq1/5Ch0NB/j+ua18bHIn/gzYkYvApVUhfnlBE6MCbfz617/m+9/7nu7IVcpob2/n73f/HYYAVW6nSQ57uk04HObuu+92O0rGS3SzvVBELkxqEjXoHn30USzLIjziLLejOC5UdTaNjQ28+OKLbkdRCoi2On/2M5/m4Ycf5rKqLn59QSNnlmfeEHBD82x+NLOFT0zpYOXypXzypk9oa7RKCf/6179obmrGOiuD+j73VgjWRIvnn3+e3bt3u50moyVaQO8B5onIvSJyi4j8In5JZjiVPB0dHTzz7HNEysZjcovdjuM4u7gKU1DBo4/O0R23clXPVuf2uv1875xWPntGB/k+t5Mlj0eiQ9396oImqvyt2hqtXNfU1MTDjzyMGWkyfg5lc4YBH9oKnWSJFtB5wFOAAUYBo3tcVBp64YUX6OxoJzz8TLejJIcIoWFnsmfPblatWuV2GpWlampq+PznPntUq/P0isxrdT6R4fnHtkavXLnS7VgqCz300EMEu4LY09L3XIOEBcCabLFgwQK2bdvmdpqMlehEKp8+0SXZAZXzbNtmzpzHMIVDsAuHuh0naSIVE5CcPObMmeN2FJWF3n77bb70xS9Qe3Af382CVucT6dkaXepp5zv/9V+89NJLbsdSWaSuro7HH38ce6wNmXfA9bjMFIMEhLv+fpfbUTJWwqeuiMjpIvJjEflz7PZpIpJ5nWezwMqVKzlwYD/BYWem98Qpp+LxERxyOm+99RbV1dVup1FZZPny5Xztq19Bupr48cwmzsqiVucTGZ5v8+Nzm5lSHOTWW2/lgQce0O5ValA89thjhCNhzNQser/5wZpisXTJUnbs2OF2moyUUAEtIh8CFhGds+eTsbuLgNuSlEsl0YIFCxCvH6t8nNtRki5SORljDIsWLXI7isoSL7zwAt/97ncY4u/kZ+c1pfXwdE4r8Bu+c04rs4eFuPPOO/mf//kfnXhFJZVlWbzw4guY4QYK3U4zuMx4Ax70ZPokSbQF+hfAu4wxXwLiW7u1wAwnQoiIV0RWi8hzTixPnZht2yxa/Abh4pHgyfzjySa3GArKWbR4sdtRVBZYuXIlv/rVrzi9JMSPZjZTFsiiFq8E+T3w5WntXD+2i6eeeor77rvP7Ugqg61Zs4aG+gbM2Cz8LAbADDe8+NKL+kU1CRItoIcC62LXTY+fTr0jvwlsdmhZ6iS2bdtGY0M9VtlYt6MMmlDJaNavW0dLS4vbUVSGe/TRRykOwLdntGZlf+dEeQQ+MrmTcyvDPPnE4wSDQbcjqQz1wgsvIDmCqcrCAhqwx9o0NjTqybtJkGgBvRK4qdd9HwGWDTSAiIwCrgP+PtBlqVN78803QYRIafYMoGKVjcW2bZYtG/DbVakTOnjwIEuWvMWVVZ3keN1Oc8T9W/PY0+plT6uXW1cUcv/WPLcjdbt6TBfNLa3Mnz/f7SgqA8W771lVFqTQZ3JQjQDxC4v1KKzjEi2gvwHcKiILgAIReRG4BfhPBzL8EfguoB0FB8HOnTshrwT8uW5HGTR2QSWIh127drkdRWWwp556CgGuHJlaral7Wr10Wh46LQ9bmvzsaU2dSuLMsghVBYYnHn/M7SgqA9XX19PR0QFlbidxkRdMsdH9XxIkOozdFuB04C/Aj4B7gOnGmLcHsnIRuR44bIw56bEFEfmCiKwQkRW1tbUDWWXW27t3H5GcIrdjDC7xILlF7N+/3+0kKoMteetNppSEKc/NzkPF/SECFwztYsvWbTQ1NbkdR2WYPXv2ANECMpvZRTa7dmsB7bSEh7EzxnQAbwCvA4uMMW0OrP9i4H0isht4BLhSRB44zrrvNMacZ4w5b8iQIQ6sNjvZts2BAwcwuSVuRxl04UAxe/bsdTuGymDTz5rB7rYcgnquTp9sbfYzemQVJSXZt11SyRUvoMmyNqNjFENLc4ueB+SwRIexGyMii4DdwFxgt4gsEpEBnYlmjPmBMWaUMWYc0T7VrxljPjGQZaoTq6+vJxwOYWfg1N2nYnJLOHDggNsxVAa78sor6YoY1tb53Y6SNhqDwuYGH++86mokk8ekV66oqalBvALZ02PxuExBtAW+pqbG5SSZJdEW6PuInkhYaowZSrRH0YrY/SpNtLe3A2B8AZeTDD7jzSEY7NKJG1TSnH322ZSVlvBWTY7bUdLG8kM5GKJfPpRymjEGhOglm2X7358kiRbQM4HvGGPaAWLdN74Xu98RxpjXjTHXO7U8dazucSAl4Z47mSP2N0ciOiOcSg6v18vV17yblXU5rK7TMexOpa7Tw1N78pkyeTLjxo1zO47KQB6Px7nBdjOAx5OF+/4kSvTVXALM6nXfecBbzsZRydRdPGZxAR0Oh10OojLZZz7zGSZOnMhfNxSzry0LP2cJ6ozAH9YVYXnz+PFPfuJ2HJWhPB6PHnWE7i8R2k3KWSfcwovIL+IXYAfwvIg8JCK/FZGHgOeB7YMVVA1cNhfQJvY362xMKpny8vL49W9+S15RCbetK6ElpDus3mwDf91QyIF2H7+45VbGjs2eSZ3U4PJ6vdEBcrO8hhY7uh3yelNnCMtMcLJKanSPSy7wBBAkOithEHiSrO+ar5RSRxs6dCi/+vVvaA77+eO6IiI6wv1RHt2ex+o6P1//+tc5//zz3Y6jMlhBQUH0Sra3m8TazrpfD+WIE3bUM8Z8ejCDqOTr7v+UlYe0olWM9gFTg2Hq1Kn84L//m5///Ofcvy2PT5/e6XaklPBWjZ/n9uTyvve9jw984ANux1EZLj8/P3olzEmqnSwQ67moBbSzEn5LiUg+MAko7Hm/MeZNp0Op5DhSPGZfAS3aB0wNsne+851s27aNhx9+mAlFFpeNDLkdyVV7W73ctbmI6dOm8s1vflM/iyrpugvGMJA6M9gPvnB035eXl80vgvMSKqBF5JPAn4EQ0LMpxQBjkpBLJUH3DisrW6Cjf7PutNVg+vznP8+2bVu5Z/UqRhVaTCzJzmPJbWHhj+uLKSop5Re33Irfr2Nlq+QLBGJDtmbnx+4IC3x+n+7/HJbo8ezfAR80xlQaY0b3uGjxnEY6OjoAMN7s23kZT/Rv7uzUQ+lq8Ph8Pn72s59TUTmE/11fTGeWjqJ416YCGoJebrn1l1RUVLgdR2WJ7u19NnffAPBBOBTGtvWEDCclWkCHiE7hrdJYa2tr9Eo2TqQS+5t1KlM12EpKSvj2f32Hhi7Y3px9e/KgBatq/Xzowx9m2rRpbsdRWSTeaET2tRkdLbbZ0QYkZyVaQP8YuE1EKpMZRiVXc3MzkJ0zEaIFtHLR1KlTAdjTmn3DSO1v82KAM8880+0oKsvEZ9/N+hbo2BeI7tdDOSLRAnob8D7gkIhYsYstItnesyitNDU1AWB82Tf6oPFH/+bGxkaXk6hsVFxczJDKcva2ZV8BHf+bJ02a5HISlW0OHTqE+ASy72N3FBOIngNUU1PjcpLMkmgBfT/wT2AGMCV2mRz7qdLEli1bkNxiyMI+0HZuCYiHbdu2uR1FZanhI0ZyoCP7PnsH2rx4PB7t+6wG3ZKlS7CH2JDt584NAQRWrFjhdpKMkmgBXQH8xBizwRizo+clmeGUc4wxrFm7jlDhULejuMPrxxRUsmbNWreTqCw0b9481q9fz7Sy7BvK7szyCLZtc9ttt+m0ymrQHDhwgJrqGswwfc+RA5RHv1Ao5yRaQN8D3JTMICq59u/fT0tzE3bRcLejuCZSOJTNWzYTCmVfEaPcs2nTJn7/+//H1PIIH5qYfSfxnDMkzI3jO5k3bx5PPPGE23FUlli+fDkAZrgW0AD2MJutW7bqeUAOSrSAngX8XUS2isjCnpdkhlPOWbVqFQBW0TCXk7jHKhpOJBxm48aNbkdRWaKuro4f/vcPKPVH+Pq0NnxZOhHmByZ0cW5lmD/96U+sXLnS7Tgqw9m2zdPPPI0US6+p37KXqTIYY3juuefcjpIxEt2c3wV8HvgVcHevi0oDzz77HOSXYXJL3Y7iGqtkJOLL0Q2ISjpjDPPnz+erX/kybS1NfOusZopysrclzCPwpWltjMi3+O8ffJ8HHniAYDDodiyVoebPn8+O7TuwTre0/3NcWbQ1/v4H7j8ypK0akIQKaGPMfSe6JDugGrgtW7awbdtWgkNOh2yeicjrJ1Qxidfmz+8ekUQpp61fv54vf+lL/PSnP8XXXs13z25mdKFOYJDvg++e3cwZhW3ceeedfPxjH+GFF17QyR2UoyKRCHf9/S6kRDBjsvdL6/HY02za29p55JFH3I6SERKdyvszJ/qdMeYfzsVRyfDUU08hXj+RysluR3FdeOgZ+A9tYt68eXz0ox91O47KIPv27eP2229n0aJFlOXC589o59KqEJ4s/s7aW0Wu4Vtnt7G50cfDb9v86le/Ys6j/+IrX/0aM2fOdDueygDPP/88Bw8cxLpYW5+PUQb2aJt/PfovPvCBD+jIOAOU6PDivU8gHA5MBN4AtIBOYU1NTbz8yiuEyieAL8ftOK4z+WXYxcN54okn+dCHPoTPl+0j7KuBamxs5L777uPpp5/CJ4YPTujkPWO7yM3ysWdP5oyyCD87v5klh/w8umMH//mf/8kFF8ziS1/6MhMnTnQ7nkpTtbW1/O32v0ElMMLtNKnJnGkIHwhz2223ceuttyLZfFR6gBKqHowxV/S+L9YqfYbjiZSjHnroIcLhMOER092OkjJCw6dzaNvLvPDCC1x//fVux1FpyBjD5s2beeKJJ5j/2qtYlsXlVUE+MKGT0oAeNk6ER+Ci4WHOG9LIS/sCPLNqOZ/+9DLOPeccPvDBD3LRRRfpF1yVMGMMv/nNb+jo6sC6RFufT6gIrDMtFi1axAsvvMB73vMetxOlrYFsne4F6oDvOBNFOa2+vp7Hn3iCcMVETF6p23FShlU6BlM4lHvuvZdrrrkGvz/7JrdQ/RMMBnnttdd4/PHH2LbtbXJ9wmXDO7l6dJCqAu3L2x85Xrh+XJDLqkK8diDAa1tW86MfrWbokEred8ONvPe976WsrMztmCrFPfXUUyxfvhz7HBuK3E6T2sxpBmrgf/74P5xzzjkMH569w9sORKJ9oHufbJgPfAJocjqQcs6DDz4YbX0eeY7bUVKLCMGR51K79QXmzp3LjTfe6HYileIOHjzI008/zXPPPkNrWztVhYZPndbJxSOC5GsjqSOKcgw3jO/i+rFdrK7z88r+CH//+9+59557uOLKK3n/+9/PmWeeqYec1TH27dvHn//yZxgOZqIeATolAet8i+DLQX75q1/yv3/8XzyeLB1jcwAS3fRHgN7vygNEh7ZTKaiuro6nnn6acOVkTG6J23FSjlUyErtoGPfedx/XXnstOTnaP1wdzRjD8uXLefzxx1myZAmCYeaQEFdNCXJGWSSrB7RJJq8Hzhsa5ryhYQ62e3hlf4BFr7/Cyy+/zORJE/nAB/+Nd73rXQQCAbejqhQQiUT4xS2/ICIRrPO060bCCsCaYbF2xVrmzJnDv//7v7udKO0kWkCP73W73RhT53QY5ZyHH36YSCRCuEpbn48r1grdsGUe8+bN44YbbnA7kUoRwWCQl19+mUf/9Qi79+ylJAA3jOvkipFBKnK1dWswVRXYfPK0Tj48sZM3anJ4ef/b/Pa3v+WO2//Gje//ADfeeCPl5eVux1QuevDBB9m6ZSv2hTbkuZ0mvZhxBnPQcMeddzBr1izGj+9d6qmTSfQkwj3JDqKc09jYGG19rpiEydXOYCdiF1dhiobyz/sf4LrrrtMTlrJcQ0MDTz31FE8+8TjNLa2MLbL50pmdXDgslLUzCKaKXB+8c1SIK0eG2NzoY97eMPfeey8PPvgAV111NR/+8IeZMGGC2zHVINu6dSv33HsP9mgbM1q/3PaZgD3TxnrZ4he3/II777hTzwnqg5NWDCIyn2O7bvRkjDHvdDaSGqh//etf0b7PVWe7HSW1iRAccTa1217ipZde4tprr3U7kXLBjh07ePTRR3nl5ZeIRCzOrgzznnO7tJtGChKBqeURppa3cbDdw4v7Arzy4jyef/55zjtvJv/+7x9h1qxZ2k86CwSDQX5xyy8wAYM5V4vnfsuFyLkRdry5g/vuu4/Pfe5zbidKG6dqcnvgBPePBL5B9GRClUI6Ojp4/IkniJRPwORp3+dTsUpHYworuf+BB3nPe96jO94ssnXrVu6443ZWrFhJjhcuG9HFNaODjNDRNNJCVYHNp0/v5N8mdDH/QICXNqzkOytWMnbMaG7+9Gd45zu1bSeTPfvss+zbuw/rUgv0FJaBGQn2GJuHHnqI973vfQwdOtTtRGnhpAcmjTF397wATxEd+/nbwBPAlORHVH2xZMkSgl1dhIee7naU9CBCaMjpHNi/jx07dridRg0C247uKL70pS/y9vqVfHhiJ/93SRM3n96pxXMaKsoxvG98F3+8qJEvndmOadzDz3/+c2655Rba29vdjqeSIBgM8s/7/wlDiU7rpgbMTDNE7AgPPHCidlPVW0I9+0SkWERuAbYDw4BzjTFfMMbsT2o61Wfz589HcvKxi4a5HSVtRMrGggjz5893O4pKsrq6Or79rW9x++23c25FF7+7sIn3je+i0K+HgNOdzwOXjAhxy/nNfHBCJ6++8jKf/fTNbNq0ye1oymFPP/00TY1NWFMtt6NkjgKwx9k8+9yzHDp0yO00aeGkBbSI5InID4CdRFueLzHG3GSM0aa6FNTZ2clbS5YQKh0LxwzdrU7In4dVPILXXpuPMVpIZao33niDmz/1STasXc1nz2jnG9PbKdDCOeN4PfD+CV38aGYLoeZDfOUrX+H+++/HsrTYygRdXV3c/8D90dbnIW6nySzmDINt29oKnaBTVVm7gW8BvwP+CgwTkSt7XpIdUCVu5cqVhIJBIuU6FE1fRcrGc+DAfvbt2+d2FJUE//znP/nBD35AGS3cMquJK0aG9ATBDDel1OKXs5qYNaSLu+66i+999zvYtnbRSXfr1q2juakZa4p+IXJcPlijLF597VVtTErAqQroTqAN+DJw93Euf09qOtUn1dXVANgFKTYuqhUiLy+Pf/u3fyMvLw+skNuJjmHnR1+z+GuoMsuzzzzF1LIwPzu/Oeum3O6MyFGfv85I9nxzKPAbvjqtnfeN62TZ8hUcPnzY7UhqgLZu3Rq9UuFujoSEOXrfF3Y7UAIqoa21TbtxJOCko3AYY8YNUg7lgIaGhmjXDW9qzdAlkRDXve86vvGNbwDw6DMvuJzoWMYfHVCmoaHB5STKacYYGhqaOH+khT8LezZ1RITrrj/y+Vvw3L9cTjS4RGBKaQSA+vp6hg/Xs87S2datW/EUebBy0qAFOgzXXXfkszdn3hyXA52aKYu2PG/ZskU/K6egM0dkkIaGBiRQQKodmza+HObOnQvA3LlzMb7Umy7K+KOZ6uvrXU6inNbW1kY4EqEkJ7tanuPyfeaoz99QX/Ydmi3Jif7N+gU5/W3esplI7AtRyvNz1GeP1GrbOr4SwBP9onL55Ze7nSalZWF7TOZqbGzETrHWZwC8OXR2dvLYY4/R2dkJ3hQctNPrQ7x+mpqa3E6iHNbY2AiAlX11IwB5PnPU5y8vCwtoO/YnawGd/pqbmyHX7RQJ8nP0vi8dJvnzgifgib7O6qS0gM4gxcXFeKyg2zHSkxXBWGGKi4vdTqIcVlVVxZQpk3l0Rz4LD6bglzeVVLtavPxhXTHFRYXMnDnT7ThqgMrLy6HL7RQZzAa7y6aiIh06mbtLC+gMUlVVhelqBTsN+oalGAm2AtHXUGUWn8/H//3fnzjn3HO5c1MBz+0OoCeYZ4cN9T5+uaqEvJIh/PVvtzNq1Ci3I6kBGlI5BOlKrW6KGSUIGKisrHQ7ScrTAjqDxIs/Cba5nCT9eIItgBbQmSo/P5/f/e7/ceWVV/LI9nweejuv+7C+ykxLDvn5f2uLqBo9lr/dfgdjxoxxO5JyQGVlJd6g1+0Ymasz+kNboE9NTyLMICNHjgTA09WMlVficpr0Il3R/l5aQGcuv9/PT37yE8rKynj88cfZ2+bjk6e1MzLLhrXLdG1h4bEduby6P5fp06fz69/8hqKiIrdjKYeMHj0a+3U7OiRcOvQpTjPSFG3dHz16tMtJUp+2QGeQSZMm4fX58LTqWMZ95W2pYfiIEZSWlrodRSWRx+PhG9/4Bt/+9rfZEyzmv5eU8NDbeXSmyUn96sRsA/MP5PCdt0p57UAeH/jgB/nDbbdp8ZxhzjvvPDCADlOcFFIjVFRW6BGbBGgLdAbJy8tj2rRprHl7X1qM154ybBt/Ww2z3vFut5OoQSAi3HDDDbzjHe/gzjvvZO7cubx5KJePTmznouE6Q2E62t7s5Z9bC9nZ4mHGWdP5j//8FhMnTnQ7lkqCadOmkZuXS8ehDswo7YflKBs8tR4uvOpCRDeEp6Qt0Blm1vnnI+31EO50O0ra8LTXYiKhaMuGyhplZWV873vf4/bbb2fYmMn8bWMBt64qZk+r9q9MF80h4a5N+fxseTHNvgp+/OMf839/+rMWzxnM5/Mxc+ZMvIe80ZZo5ZxGMCHDrFmz3E6SFrSAzjDxYZq8LQddTpI+vM0HEBHOPfdct6MoF0ydOpU77ryL73znO1RHSvjJsmLm7gnoSYYpbl29jx8sLWXxoTw++tGP8uBDD3PVVVdpy1kWOP+88zHtBtrdTpJZ5JDovrAPtIDOMFOmTCEvvwBvsxbQifK1HGTS5Mk6BnQW83g8vPe97+Whhx/h4ksv5eG38/n9miKaQ1qMpZqIDQ+/ncfvVhdRMWIsd9/9D7785S+Tn5/vdjQ1SOIFnhzWz6eTPLUexk8YT0mJDkKQCC2gM4zP5+Pcc87G31bjdpT0YEXwtB/mPJ1gQRGdjOiWW27lW9/6Fptbcvnh0lI2NOipIqnicIeHW1aWMHdPLu973/u4866/M2HCBLdjqUE2duxYSkpLoNbtJBnEAqkXzpupXRkTpQV0Bpo5cyZ0NndPDqJOzNtaA7atM5SpbiLCjTfeyJ133kXx0FH8dlURc7bn6uQrLlt+2M8Pl5dSEy7gF7/4Bf/1X/9FIBBwO5ZygYgw89yZ+Gp92g/aKfVgLKPdN/pAC+gMdM455wDRodnUyXlaa/B4vUyfPt3tKCrFTJw4kbv+fjdXXX01T+/OY3OjtkS7JWTBXzcUMnr8JP5xz71cfvnlbkdSLps0aRJ2pw068a4jpC3aHUZPwE2cFtAZaOzYsYgIEptdT52Yp6uZYcOGk5eX53YUlYLy8vL4xje+AcD2Fh2dwy17Wr2EbfjUp25mxIgRbsdRKaC7z7uO4e6M2Ouo5xIkTgvoDOTz+Sgrr9ApvRPgCbVTNWK42zFUCisuLmbUyBHsbNYWaLfsaIm+9meccYbLSVSq6G700ALaGbHXURuTEqcFdIaqGjEcj/aBPiVvuI3hw7WAVid3+hlnsrNN+9u6ZWeLl4ryMiorK92OolKEtkA7LBJtfPP5tKEgUVpAZ6hhw4bhi+hkKidl25hgB0OGDHE7iUph8+bNY/GihWCMnkjoovqGRv785z8TDus8qwoOHYrN5a3fa50RgEgkQn19vdtJ0oYW0BkqFAphPNpn86REQDy6Q1bH1dnZya9+9St+/etfMz6/g5+f36TTfLvk81M7uHp0F48++ihf++pXqK6udjuSctmqVauQIgHtceAIMyTaOrBmzRp3g6QRLaAzVGNjI5ZHv5qflAiSk0dTU5PbSVSK2bVrF1/8/Od48YUXeP/4Tn5wbitlAW1+dovfA588rZNvTG9jz45tfPYzn2bRokVux1IusSyL1WtWY1XqEByOKQXxC6tXr3Y7SdrQzi4ZqrGpCePPdTtGyjO+XJqbm92OoVLEoUOHmDdvHg8+cD+5EuZ757YyrVw7WaaKWcPCjCtq4s8bLX74wx9y7bXXcuONN3LaaafpFN5ZZOvWrXR2dMJQt5NkEA/YlTbLVyzHGKOfpwRoAZ2BjDE0NDRiCse4HSXlWd4AtbV1bsdQLurq6mLhwoXMm/c8q1atwhg4pzLMZ89op1RbnVPO0HybH89sZs72PF5+cR7PP/8848eN5T3XXsdVV11FRUWF2xFVkj366KOITzDD9PPpJDPSUL2imrfeeouLLrrI7TgpTwvoDLRr1y46O9qxh+kZ66diFVSyfftGurq6yM3VFvtsYYxhw4YNzJs3j9defYWOzi6G5MMHxndyyYgQQ/JstyOqk/B74GNTOrlhfBdLD/lZWL2Tv/71r9x+++1ceOGFvOc97+Giiy7C7/e7HVU5bOfOncyfPx/rNEtPIHSYGWuQLcJdf7+L2bNnayv0KWgBnYHifZis4iqXk6Q+u3gEVvU61q9fz/nnn+92HJVkhw4d4qWXXmLe83PZf+AgAZ8wa0gX75ga4rTSCB7dX6SVAr/hylEhrhwV4mC7h4UHAyxe9SZvvvkmxUWFXHX1NbznPe9h8uTJWgxkiHvuuQd8YKZo67PjPGCdYbFj+Q4WL17MpZde6nailKYFdAZatWoV5BZhcovcjpLyrKLhIB5Wr16tBXSGOnToEIsXL2bRwoWsXrMGYwynl0X4wtQgs4aGyNWtYEaoKrD5yOROPjSxkw0NPhZVh3jmycd5/PHHGT9uLJddfgWXXnopkyZN0mI6TW3bto0FCxZgn2Fr63OSmDFHWqEvuugivF4dzetEdNeRYSKRCKtWrSZcqNPdJsTrxy4cwrLly/nCF77gdhrlAGMMe/bsYeHChSxauICt294GoKrQcOO4Li4ZEWJYvnbRyFReD8yojDCjMkJ7uIO3Dvl5s2YH9927h3vvvZfhw4Zy6Tsu49JLL2X69OlaIKQJYwz/88f/QQKirc/J5IHImRF2L9nNs88+y4033uh2opSlBXSGWbt2Le3tbVhVegJhoiIlo9m2dQWHDh1i2LBhbsdR/WDbNps2bWLRokUsWriA/QcOAjCxxOLfJwWZOSRMVYEWzdmmwG9416gQ7xoVojkkrKr1s7L2AE8+Poc5c+ZQUlzExZdcyqWXXsp5551HIKDNmqnqpZdeYuOGjdgzbchxO02GGwUMhTvuvIPLL7+c0tJStxOlJC2gM8yCBQsQrx+rZLTbUdJGpHw8OftXsHDhQj70oQ+5HUclKBwOs3r1ahYuXMjiRQtpaGzCK3BGWZh3nR7i3Mow5bnaUqWiSnIMV4wMccXIEJ0RWFfvZ8XhEPNfjo7kkRsIcMGFF3LppZcye/Zsioq0C1yqaGtr489/+TOUgxmvn+mkE7DOsWh/uZ077riD733ve24nSklaQGcQy7KY//rrhEtGgVf/tYkyeSVQUM7811/XAjoNbNu2jblz5/LySy/S1t5BwCfMKA8y88wwZ1eGKfDrDladXJ4PLhgW5oJhYSJ2O5safaw8HGTl0gUsWLAAn9fLxZdczLXXXsesWbO0m4fL/vnPf9Lc1Iz1Tgu0+/rgKAZ7ks3zzz/fPda6OppWWRlk06ZNNDc1EZl0jttR0k6odBwbN6ymoaGB8vJyt+OoXlpbW3n55ZeZ+9yzvL19B34PnDckxOxJIaaVh8nR+kb1k88DZ1VEOKsiwqdMBztbvCypyeGNtxayYMFCKivKec+113HddddRVaUjGw225uZmnnjyCeyxNuimeVCZqQZ2w/3338+tt97qdpyUowV0Blm2bBmIYJWMcjtK2rHKxmAOrGLFihVcffXVbsdRRPs1r169mueee46FCxcQDkcYW2TzydO6uGh4iEJtaVYO8whMKrGYVNLJRyZ3sqrWz4KDYR584H7uv/9+zjnnbK677nouu+wy7S89SJ566ilCwRDmNP28Dzo/WBMsFi1axL59+xg9WruG9qQFdAZZumwZduFQ8OmGva/s/ArEn8vy5cu1gHZZbW0tzz//PM/PfY7qmkMU+IXLh3VyWVWIccWW2/FUlvB5olOHzxoWpr5LWHQwwIKta7h19Rr+57Y/cNXV13D99dczZcoUt6NmrK6uLh6d8yhmhIESt9NkJzPZwNvw8MMP893vftftOCnF4+bKRWS0iMwXkU0islFEvulmnnTW0tLC1i1biOjkKf0jQrioimXLlmOMtnS4Ze3atdz0iY9z9913Ux46wFemtfGnSxr41OmdWjwr11TkGm6c0MUfZjfy3+e2MqO4ibnPPs3nPvc55syZ43a8jPXSSy/R2tKKfZqOoOOaXLDGWrzwwgs0NTW5nSaluN0CHQG+bYxZJSJFwEoRedkYs8nlXGln7dq1GGOwSka6HSVtWSUjady1k7179zJ27Fi342SdlStX8oPvf48yX5BfXNTCcB2rWaUYj8DU8ghTyyO0hzv5+6Z8/vSnPxEKhfj4xz/udryMs2nTJjx5HqxK/fLsJjPaENkZYceOHcycOdPtOCnD1RZoY0y1MWZV7HorsBnQCrAf9u/fD4CdV+ZykvRl50dfu/hrqQbPkiVL+O53v8OQnC5+dG6zFs8q5RX4DV+b3s7sYSHuuOMO7rnnHj165bB9+/ZhF9g68obbCqM/dN94NLdboLuJyDjgHGCpy1HSUk1NDeIPaP/nAbAD0a1ETU2Ny0myy+LFi/npT37MyPww3zu7haIcLUJUevB64MvT2vF7DPfccw/hcFhnNHXQ3n17sUv1y7Tr8kC8wr59+9xOklJcbYGOE5FC4HHgP4wxLcf5/RdEZIWIrKitrR38gGmgpqYGk6MD/w+ILw/x+rSAHmT33nMPkYjFl6e2avGs0o5H4NNndFCRa/Pggw/S1tbmdqSM0NnZSXNTc3frp3KRAIVw8OBBt5OkFNcLaBHxEy2eHzTGPHG8xxhj7jTGnGeMOW/IkCGDGzBN1NQcIuLPdztGehOBQCGHDx92O0lW+c9vfYvc3AD/u76Y5pAeq1XpxbLhbxsKqO/y8B//8R8UFmrF5wS/309uXi50uJ1EYYBOKCvTLqI9uT0KhwB3A5uNMbe5mSXtSeoWHnZBBcbrx3j9WEXDsQsq3I6kUsiZZ57Jb3/3/6gP5/Db1cW0hVP3vZyOxhZZ5Hlt8rw2p5eGGVukJ2Q5xTZw16Z8lh3O4Stf+Qrvf//73Y6UMXw+HzPOmoG3Pn1nSTKlBuOPXYYYTGmaHmFrBhMyzJgxw+0kKcXtFuiLgZuAK0VkTexyrcuZ0lJubgCxU3PHGBo7Gzu/Aju/gq6p1xMaO9vtSCdmWzpBggvOPvtsfvmrX1Pd6ed3q4vZ05q+O81Uc9NpnYwtshhbZPGj89q46bROtyNlhIYu4a5N+SyuCfDZz36Wj3zkI25HyjgzZszANBvocjtJ/5izDZQCpWBfbkdvpyGpjTZqaAF9NFdPIjTGLEbPr3VEbiAAJjUL6LSiBbRrZs2axS9uuZWf/uQn/HCphzFFNu8Y0cXs4SFKtG+0SgEhC1bU+llUHWBjgx/bwE033cQnP/lJt6NlpLPPPhsAOSSYsboNcIvUCMOGD2PYsGFuR0kpKTMKhxqYQCCAx464HSP9WWEtoF108cUX8/gTT/DKK6/wwrzneWDb2zz8dj5nVYS5dESQc4aE8bt93ExlFWPg7WYvCw8GWFqbS2fYMHRIJZ+46VquueYand44iU4//XRGjR7Fgc0HiIyOuH/MPBvVRQvoaz+tnQN60wI6Q1RWVuKJrHM7RnqLhDBWmIoK7aPtppKSEj74wQ/ywQ9+kF27dvHCCy/w0osv8H/rGynMEWYP7eSSESEmFFup3PVfpbm6Tg+LqnNYfCiXQ+1CbiDAZVdezrvf/W7OOeccPB6t5pLN5/Px9a99ne9973vIDolOK60GjwHvOi9l5WXaRek4tIDOEMOGDcMEO8C2wKP9R/tDQtHhp/QwVeoYP348X/7yl/n85z/PypUrmTdvHgsWLeLl/WGqCg0XDOnirIowE4otvFrPqAEwBqo7PKyr97OyNofNjdHd4znnnM1n3v0eLrvsMvLzdaSjwXbhhRdy7rnnsnrjaiJjI5DjdqLsIfsE6uGL3/8ieXl5bsdJOVpAZ4ihQ4cC0SLQ5Ja4nCY9eYJaQKcqn8/HBRdcwAUXXEBbWxvz58/nhRfm8fSGjTy5y1DgF84sC3JWRZjpFWEqcrWlSp1aRwQ2NvhZV+9nfUOAutj5lWNGj+KzH7iGq6++mhEjRrgbMsuJCF/72tf47Gc/i2wUzDn62R4UYfBu8DJ+4niuueYat9OkJC2gM0R3AR1s1wK6nyTUDhx5LVVqKiws5L3vfS/vfe97aWlpYcWKFSxbtoxlS5ewbHMDACMLDWeVRwvq00oj5OhBGUV02LndrV7W1flZ1+Bne7MP20B+Xi7nzjyPCy64gFmzZmnRnGImTZrE+9//fp544gmskRboJjrpZK1AB3z7W9/G69UN6PFoAZ0hSktLAZBImo73kwLir138tVSpr7i4mCuvvJIrr7wSYwy7du1i2bJlLF26lFfWrmHeXgu/F84oDXe3Tlfl29p3Oos0BYX19dFW5g1NAVqD0RbMKVMm8/H3XsisWbM488wz8fl0d5jKvvSlL7F02VIOrjhI5F3alSOpDoJnl4ePffxjTJ8+3e00KUu3GBmipCTa6iyRoMtJ0pdEggRyc/H7/W5HUf0gIkyYMIEJEybwkY98hK6uLtasWcPSpUtZtnQJD2w7AEBlHpxVHmRGZZgzy8Lk6lYwo1g2bG/xsqbOz7qGAHtaop3jS0uKuejy2Zx//vmcf/75OqtamsnNzeXHP/oxX/nKV5A1gpmlXTmSIgi+VT7GTRjHpz/9abfTpDTddWSIoqIiQAvoAYl0UVRU7HYK5ZDc3FwuvPBCLrzwQuCbVFdXd7dOv7ViOa8dCOLzwOmlYc6uDDOjIsxwbZ1OS80hYV2dn7WxvsztYYPH42HatDP5woWzmTVrFpMmTdKRM9Lc1KlT+eQnP8m9996LNcqCKrcTZR5ZLXjCHn7y45+Qk6PN/CejBXSGCAQC+Px+QpYW0P0lkRCFZQVux1BJMmLECG644QZuuOEGwuEw69atY8mSJSx5600e2LaPB4Bh+TCjoosZFWHOKNO+06nKNrAr1sq8tiHArmYPBigrLeGyqy7mwgsvZObMmd0NCypzfPKTn2T+6/PZu2YvkaERrWKcdAg8+zzc/NmbmThxottpUp6+9TJEMBgkEg6DVycB6S/jy6GtrdntGGoQ+P1+Zs6cycyZM/nqV7/KwYMHWbp0KUuWvMWClSt5aV+YHK8wtSzEjIpoC/WQPNvt2FmtLSysr/exts7PusYALcFot52pZ5zBZ/5tNhdeeCGTJ0/WVuYM5/P5+Pa3vs03vvENZItgpmlXDkdY4FvtY3jVcB3zOUFaQGeIxsZGAIxfx2rsL+PPo6luD8YYRI/jZ5Wqqire//738/73v59gMMiaNWtYsmQJb735BvdtreG+rTCu2OaKqk5mDw+Rr1vOQWHZsKbez/wDAdbW+zEGiosKueAd0YL5/PPP15N+s9DZZ5/NVVddxSuvvRIdG1oPNAyYbBNMq+FbP/mWzsabIN0NZIiGhujwXVpA95/x5xEJh2lra9NDv1ksEAh0jzn9zW9+k3379vHWW28x7/m53LNlFw+9XciFw7q4YmSQiTobYlLUdnp4/WAOC6vzaOyCivIyPvaxa7nkkks4/fTTdVgtxVe+8hUWv7GYjvUd2Bfp0aEBCYJ3i5dLLr2ECy64wO00aUML6AxRU1MDgMnRmbL6y/ij/Z8PHTqkBbTqNnr0aEaPHs2HPvQhNm/ezLPPPsurr7zCgoNBxhTZXFHVxUXDQxT49VDyQERsWF3n57UDATbU+0GECy6YxXvf+z5mz56tw8ypo1RUVHDjDTfy8L8ehiCgjab9JvsFEzHcfPPNbkdJK7pFyhDLli1DfAHsvHK3o6Qtu2g4AEuXLmXSpEkup1GpRkSYOnUqU6dO5Wtf+xqvvPIKzz7zNPdt3c7D2wuYNbSLK0cGmVyirdJ9cbjDw/xYa3NzEIZUVvCpm9/Lddddp7OCqpO6+uqrefjhh5F9gpmkX2D7y7PXw9hxY3W/10daQGcAy7JYvPgNwiWjQE+g6TcTKMAUVLJo8WI+/vGPux1HpbCCgoLuET22bt3Ks88+y8svvcji6iCXVQW5+fQO/PpRPClj4NUDOdy/tQCDMPui2bz3ve9j1qxZ2tqsEjJx4kTGjR/H7r27sSZZbsdJT+1AHVzzgWv03J8+0k18Bti4cSMtLc1Eysa4HSXthUvHsHnTpu4+5UqdymmnncZ//dd/8eRTT3PTTTex4GCAX64spjGoO6MTCdtw9+Z87t1SwPkXXMCjc+bw61//hosuukiLZ9Un11x9DdQDnW4nSU+yP7qdete73uVykvSjBXQGeO2118DjwSoZ7XaUtGeVjcUYw+uvv+52FJVm8vPz+fznP88vfvEL9nfl8+PlpWxv1pPdemsMCr9aVczrBwPcdNNN/PrXv2Ho0KFux1JpqrvbQYe7OdJWOxQUFjB8+HC3k6QdLaDTXHNzM8/NnUu4fAL4dNaggbLzyzGFQ3nkX49iWXpIUPXd5Zdfzt9uv5280mHcurKYhQf1cxm3s8XLT5aXsq8zj5///Od8/vOf1xE11ICUlJREr+gcYv0TguISnYG3P7SATnNPPfUUoWCQ8Iiz3I6SGUQIjphOTfVBFi5c6HYalaYmTpzInXf9nWlnzeCuzQW0hrQ7B8DfNxfhL6rkr3+7nSuuuMLtOCoDxAto0S5T/SJBobxUBx/oDy2g01gwGOTROXOwSkdj8vUD4BSrbCzklfDAgw9ijJ7ZrfqnpKSEj370YxgDhzp1U2sbqG4X3vmuq/Rsf+WY7iFHQ+7mSFeesIfiYm2B7g/dqqexefPm0drSQkhbn50lHoLDp/P2tm2sWrXK7TQqjcX7FdZqAU1zSAjbaF9L5aj29vboFb+7OdKV7bNpa2tzO0Za0q16mrIsi4ceehhTOLR7/GLlnEjlJCQnn4ceesjtKCqNxccxru3STW1d7EuEFtDKSfX19QCYXD1a2B8m11BbV+t2jLSkW/U0tXDhQmpqqgmOmI7O2pAEHh/BoVNZvnw527dvdzuNSlNNTU0A2LZ+RuOTLTc3N7uaQ2WWeAFNnrs50lYuNDQ0aHfFftACOg0ZY3jggQchryTaX1clRXjYGYjPz8MPP+x2FJWm5syZg1fg0iodImByiUVVoeFfjzysO2vlmO4x+3PdzZG28iAUDNHRoeMA9pUW0Glo8+bNvP32NoLDpoHovzBpfAFClafx6quvdrckKpWo5uZm5j73LBcND1Khh5fxCFw3poMdO3exfPlyt+OoDNG9bQ64GiN9xV43PTLUd1p9paEVK1YAECkf73KSzBcpH49t26xZs8btKCrNPPXUU3QFQ1w7tsvtKCnjouEhSnPh4Yf13ALljKamJjw5Hq1m+skEol/utZGo7/Qtl4ZWrVoFBeXg12NWyWYXDEG8flavXu12FJVGIpEITz35BNMrwowutE/9hCzh98DVIztZuXIVu3fvdjuOygDNzc3a+jwQ2gLdb1pAp5lQKMT69RsIF45wO0p28HiIFA5jxUodzk4l7s0336S+oZF3jdK+z71dNjKI1wPPPPOM21FUBmhtbcX26ZfUfosN/9fa2upujjSkBXSa2bVrF+FwCKtoqNtRsoZVNIx9e/foSRYqYU899SQVeXBOZdjtKCmnJMcwa0iIF+Y9T1eXdm9RA1NZWYlHh4nsv87oj4qKCndzpCF916WZ7mlLLd0xDxaJBPHn5JCbq11m1KlVV1ezYsVKLh/RiUdHrzuud44K0tbeweuvv+52FJXmRo8ejd1pg+4S+0Vaoxup0aNHu5wk/WgBnWaGDh2Kz+9HulrcjpI1JNhKVVUVHo9+XNSptbREP5ujCi2Xk6Su+Guj/S7VQHUXftoDoX9aISeQQ2VlpdtJ0o5WBGnG4/EwfPgIPFpADxpfqJUx+u1cJWjEiOj5CTp994nFX5v4a6VUf40dG50LQer0cE+fGfDUexg9erQ2EPWDvmJpaNzYMfg6aiGNunHYBRXYBenXx0qCbUhnM6NGjXI7ikoTRUVFFOTnpVQBPbbIYmxR6rSIH469NlVVVS4nUelu9OjRTJ8+He82L0TcTnMsU2owpSk6DnwN0AA33nCj20nSUups4VXCPvzhD0OwHf+B9BkZIjR2NqGxs92O0TfGENjzJn6/jxtuuMHtNCpNiAhVVVVsb/ZhpcjgADed1slNp3W6HaPbtiYfoC3QauBEhC9+8YuYToNsT71WaHO2wZydggW0Ad8GH8NHDOe6665zO01a0gI6DZ199tlcf/315NRswNNe53acjOVt2IW3cS+f++xntaVM9ckHPvj/27vz6Cjre4/j7+9kkkwSNmUXWUQWvSJIi4BSBTyoVKi2t1Xg1oKKC1qRRaBFSKXKYgUUFEW0otcLapVbezxeBL23pVQEFdk3RRAkxABRtpB1Zn73jxlrTEGYZJInk3xe58xhnv2bHJ4nn/nN7/k9v+Dz435e+jQNPbX6u1bnJLN8X4Brr72WjIwMr8uRGqBz58706Nkj0gpd7HU1icH2Ge6I4/bht+P3+70uJyEpQCeou+++mwYNGhD4/D1w1aSZqyYJFpH2xRrat+/AL37xC6+rkQQzYMAAhgwZwv9lBVj2hZ7y8I1PjyTx7LY6XHxxJ8aPH+91OVKD3HnHnbgih+9jH+hD6/fLg6RNSbQ5rw39+vXzupqEpQCdoOrWrcvYMWOwE7mk7F4JYYXouAkWk7bzf7FgIb/5zQR9Opdyueuuu7jyyit4eWc6Hx9M9roczx3I9/H4pno0adac6dNnkJKS4nVJUoO0b9+eX//611iWYR+bQvSpFID/H34y/BlMeXCKbh6sAP3mEljv3r0ZPnw4ybmfEfh0OYT03VVFWfEJMna8RfKJg0yaNIkOHTp4XZIkKJ/Px+TJmXTs2IF5W+uydG8q4Vr6R31jrp+p6+pDSgaPzpz1z/HsReJp0KBBDBs2DN/nPmyjQvS/KIqE55RgCrNnzaZt27ZeV5TQFKATmJkxbNgwJkyYQPLxL0nf8TaU6Gl55WUFh0nf/hapoXxmzpzJ1Vdf7XVJkuACgQCPzpxF956X8fLOdKavq8fB/Npz2S0MwsLt6czcUJf6TVoyZ+4TemCDVKrbbruNn//85/h2+rBt1e+mQs+UgP89P/58P4/+4VEuvPBCrytKeOYS7A6Xbt26ubVr13pdRrWzevVqJmdmUuILkN/xWlxALTyx8B3PIX3nu9TLSGf2rJlqeZa4cs6xbNky5s6ZQ6ikkF+2y6Nvi2KsBv9933HYz4LtdcktgEGDBjN8+HBSU9UfXCpfOBzmkUceYdmyZYTbhiOjYCR5XZWHjoN/tR/LM2ZMn8FllyXYiFgeMrOPnXPdTrpMAbrm2LZtG+MnTCAvv5DCVj0JNmpPjf4LHQ8uTHL2RlKy13NO8+Y8Nnu2RtyQSnPgwAFmTJ/OuvXr6dwwyPAL82gYSKxr8OkUh2DJrjTe/iJA82ZNmThpMl26dPG6LKllQqEQzz77LK+88go0hFDPEKR7XZUHssH/kZ+M1Ax+P+X3XHrppV5XlFAUoGuR7Oxspk2bzubNmwg1aEXReT/CpdTGq8bpWcFh0navxPIO0adPH+6//371zZRKFw6HeeONN3hm/tMQDvKTVvlc17qQlARvIXMO1hxI5k+76pBbANdffz333HMP6em6/oh3VqxYwbTp0yihhJIeJdDE64qqiAPbavi2+2jfoT3Tpk6jWbNmXleVcBSga5lwOMySJUtYsOBZgvgoaHUZoYZt1Rr9DRfGn7OFQNbHZGRkMO7+sVx11VVeVyW1THZ2NvOffpq/r1xJwzQYfH4ePZuWJORpuutoEot2ZrDzSBLtzm/LfaNGc8kll3hdlggAe/bs4YFJD5CVlUX44jCug4MEPM/OWBEkfZgEOXDdddcxZswYdZ8qJwXoWmrv3r1MmzadHTu2EzyrDUXn9YLkNK/L8pQVHiWweyW+4we4vFcvxo8bR8OGifeIcak5NmzYwBNz5/DZrt20bxDil+1P0K5+9Xns9vf5qtB47bM0VuWkclaD+tx51wj69+9PUlKCN6dLjZOfn8/06dNZuXIlNINQtxDUxD+HOeD/2I+v2MeY0WP4yU9+giXip/JqQgG6FgsGg7z66qs8//zzhH3JFJ7bjWDjjrWvNTocJPnLzaRmbyQQSGXM6FFce+21urBItRAKhVi2bBnPLniGw0eO0qtZEYPaFXB2Ne0fXRSC/9kb4K296ThfEoMGDebmm29Wdw2p1pxz/OUvf2HeU/MIWpBQ1xCuZfU8x2IWBNtk+Hb5aNW6FZmTM+nYsaPXVSU8BWjh888/Z9as2WzevAlXpwmFbS4nnNHI67KqRNKRLAJfrIaCo/Tp04eRI0fSuHFjr8sS+Rf5+fksWrSIP/3pVXwuxA1tTvDjVkUkV5OR75yDjw4m8/JnkX7Offr0YcSIEbrxVhLKvn37eHjqw+zYvoNwyzDuBw4S+bk+X0VuFCQPbrzxRu644w512YgTBWgBIp++33nnHZ6c9xTHjh6hpMmFFLfsBv6aeaJZUR4pe9fgP7yHc1q0YOyYMXTv3t3rskROKzs7m3lPPsl7q1bRLMNxc/s8LmkU9LSm/Xk+Xvo0g61f+2l7XhtGjR5D165dPa1JpLyCwSCLFy/mhRdfwKU6gj8MQqLdYxcG22b4dvho1LgRmZMydU7GmQK0fMfx48dZuHAhf/7znyE5QOG5l9asIe/CIZJztpCavQF/kjFs6FAGDx6sRwdLwvnggw94Yu4c9mXtp2ujEm7ukE/T9HCV1pAfhD/vTuPdfQHS0tMZfvsd3HDDDXrEvdQIn3zyCQ89/BD7vthH+PwwrrODRPivfSx6o+Bh6N+/P/fddx916tTxuqoaRwFaTurTTz9l9uzH2L59G+F6zShs3QuXfpbXZVWI79iXpO19H/IPc3mvXoy67z6aN2/udVki5VZSUsKSJUt48YWFlBQXMbB1AT89rxB/FXTrWJ2TzKKddThWDAMGDOTOO++kQYMGlX9gkSpUVFTEc889x2uvvYbVNYKXBqG63lvuwHYaSVuSqJNRh99M+A1XXnml11XVWArQckrhcJilS5fy1NPzOXHiBMXNOlHSoiskJXtdWmxKCkj54kOSc3fSuElTxo4ZTa9evbyuSiRucnNzmT9/Pu+++y5t64e556LjNKuk1uj8ILy0I533clK58IKOjBl7PxdccEGlHEukuli3bh1Tp00lNzeX8AVh3L85qCb3HwCQD0kfJcFBuPzyy5kwYQJnn32211XVaArQclpHjhxh/vz5vP322xCoS2GrnoTOau11WafnHP5DnxDI+ggLB/mPIUMYOnQogUDA68pEKsWKFSuY+egfKCnMZ2iHPK5oHt9Hgu86msRTW+uRW2AMHTaMoUOHqruG1Bp5eXnMnTuX5cuXY2dHW6PreV0V2F4jaUMSKb4URt03igEDBmgUqSqgAC1nbOPGjcyaNZu9e/cQPKs1xa0vw6VWz35VvhNfEdj7Pnb8AJ27dGHc/ffTpk0br8sSqXQHDhxg2tSH2bBxEz2bFnPbhSdIr2DGDTt4a0+A/96dRqNGjch8cAqdO3eOT8EiCWbFihU8OvNRThScIPTDEK6VR1kpBLbO8O3x0eniTkyeNFmj3lQhBWiJSTAY5PXXX+f5hQspCYUpbNmTYOMO1ecmQxcmOXsjKfvXU69uXUaOvJdrrrlGn8alVgmFQixevJiFCxfSMBBmXJejtMgoX5eOgiDM2VSXrV/76du3L+PGjaNu3bpxrlgkseTm5pL5u0y2btlKuF0Y16WKu3TkgX+NH3fYMWzYMG655RY9pKiKKUBLueTk5DBt+nQ2bthA8KxWFJ13hedPMrSCo6Tt/juWd5B+/foxevRo6tWrBt+viXhk69atPDDxt7jCozzQ9SjnxBiiC4Pw6IZ6fHYsmXHjxumrYZFSgsEg8+fP5/XXX4eGEOoZgqp4XlB2ZGzn9JR0fpf5Oy677LIqOKiUpQAt5RYOh1myZAnPLFhAyPwUtP4RobM96BvtHP6D2wns+5D0tADjx43jqquuqvo6RKqhPXv2MGrkvbiiY0zqepTmZxiiC0Mwa0M9Pj2azIMPPkjfvn0ruVKRxPTXv/6VGY/MoJhigj2C0KSSDuTAtkTGdm7Xvh1TH56qLhse+r4AXZ3uL5VqyOfzcdNNN/H8H/9I21YtCOx8l5Rdf4dgcZXVYMUnCHyyjNQ979Pthz/gv156SeFZpJQ2bdow54knIbUe09fXJyf/9Jf2whDM3lCXT4/4yczMVHgW+R5XXXUVzz37HOc2ORf/P/yQVQkHCYN9GAnPAwYMYP7T8xWeqzG1QMsZKykp4cUXX2TRokW4tPrkt78GF6jc7hO+vIOk73yXZAsz8t57uf766/X1ssgp7N69m1H3jSQ/7zgB//efJyVhR3HImJyZSb9+/aqoQpHElpeXx7jx49i2bRvhS8O41nHKUCHwfeDD9ht33HEHv/rVr+KzX6kQdeGQuFq/fj0PTJpEflGI/Pb9CNdtWinHSfpqN2m7V9KkSWNmPvoHjbAhcgb27NnDm2++SSgUOu26PXv2VN9KkRjl5+czceJE1q9fT/gHYdz5FcxRIUh6PwlyYOTIkdx4443xKVQqTAFa4m7fvn2MnzCBL7/MoeC8Kwk1Oj9+O3cuMspG1lou6tSJGdOn6+lnIiJSbRQVFZGZmcmaNWsIdwnjOpQzSwUhaVUSdsgYP348AwcOjG+hUiHqAy1x17JlSxY88wwXd7qIwK6/kZy1DuLxYSwcImX3SlKy1tKvXz/mPP64wrOIiFQrqampTJs2jd69e+Pb6Ctfn2gHvo982CFj8uTJCs8JRgFayq1+/fo89thj9O/fn5T960jO+rhiO3SO1F0rSM7dya233kpmZiapqanxKVZERCSOkpOTyczMpOMFHfGv9cPx2La3nYZlGSNGjODqq6+unCKl0ihAS4WkpKQwceJEBg4cSEr2BvwHd5R/X198iP/rz7n77ru59dZbdbOgiIhUaykpKUx9eCoZgQz8q/0QPMMND4Fvk48rrryCwYMHV2qNUjkUoKXCzIyxY8dyaffupO5ZRdKRfTHvw5+zleSczfzsZz/TxURERBJG06ZNmfLgFDgGvjU+bJed9uX/wM8555zDxN9OVGNRgtJNhBI3+fn5/Pree9n9+V4K2vbBneFTC30FX5P6+Xv06tWLqVOn6lGlIiKScBYvXsyCBQvOaN30jHSemvcU558fxxvwJe40CodUmdzcXO68awS5hw7GtF3HCy7gySeeIBAIVFJlIiIilevYsWOUlJScdr2MjAz9vUsA3xeg/VVdTFlm1h+YCyQBf3TOPeJxSVIBjRo14oWFz7Njx5n3hTYzOnfurIuJiIgktHr1KvfhYlJ9eBqgzSwJeAq4msggMB+Z2ZvOuW1e1iUVU79+fXr06OF1GSIiIiKVwuubCLsDnznndjvnioFXgRs8rklERERE5JS8DtAtgNJDNmRF54mIiIiIVEteB+gzYmZ3mtlaM1t76NAhr8sRERERkVrM6wC9H2hZavrc6LzvcM4965zr5pzr1rhx4yorTkRERESkLK8D9EdAezM7z8xSgMHAmx7XJCIiIiJySp6OwuGcC5rZvcByIsPYLXTObfWyJhERERGR7+P5ONDOuaXAUq/rEBERERE5E1534RARERERSSgK0CIiIiIiMVCAFhERERGJgQK0iIiIiEgMFKBFRERERGKgAC0iIiIiEgMFaBERERGRGChAi4iIiIjEQAFaRERERCQGCtAiIiIiIjEw55zXNcTEzA4Be72uQ8qlEZDrdREitZTOPxFv6NxLXK2dc41PtiDhArQkLjNb65zr5nUdIrWRzj8Rb+jcq5nUhUNEREREJAYK0CIiIiIiMVCAlqr0rNcFiNRiOv9EvKFzrwZSH2gRERERkRioBVpEREREJAYK0BI3ZhYysw2lXr8ttayRmZWY2Ygy2+wxs81mtsnM3jGzZlVfuUhiM7O8MtO3mNm86PspZrY/ek5uMbPrS80f50W9IonMzJyZLSo17TezQ2b2lkXkmtlZ0WXNo+v/qNT6h8ysoZl1NLMV0XNzu5mpq0cCUYCWeCpwzl1S6vVIqWU3AmuAISfZrq9zrjOwFnigKgoVqWUed85dQuQ8XGhmuvaLlN8JoJOZpUWnrwb2A7hIv9g1wGXRZZcD66P/YmYdga+cc18BTxA9N51zFwJPVt2PIBWli6hUlSHA/UALMzv3FOusBNpVXUkitYtzbjsQJPJgBxEpv6XAgOj7IcArpZa9TzQwR/99nO8G6lXR982BrG82cs5trqxiJf4UoCWe0sp04RgEYGYtgebOuQ+B14BBp9h+IKALiEjsvnPuAQ+dbCUz6wGEgUNVWZxIDfQqMNjMAkBn4INSy1bxbYDuDrwBtIxOX04kYEMkWP/VzN42szFm1qDSq5a48XtdgNQoBdGvicsaRCQ4Q+SisxCYXWr538wsBGwCJldqhSI103fOPTO7BSj95LMxZnYzcBwY5JxzZla1FYrUIM65TWbWhkjr89Iyiz8CuppZBpDsnMszs91m1o5IgJ4d3ccLZrYc6A/cANxlZl2cc0VV9oNIuSlAS1UYAjQzs19Gp88xs/bOuZ3R6b7OuVyPahOpDR53zs3yugiRGuZNYBbQB2j4zUznXL6Z7QRuA9ZFZ68BrgOaAJ+UWjebSKPSQjPbAnQCPq6K4qVi1IVDKpWZdQDqOOdaOOfaOOfaADM4+c2EIiIiiWIh8PtT9F1+HxgNrI5OrwZGAWuiNxpiZv3NLDn6vhmREL6/souW+FCAlngq2wf6ESJB+Y0y6/03CtAi1cFkM8v65uV1MSKJxDmX5Zx74hSLVwFt+TZArwPO5dv+zwDXAFvMbCOwHBjvnMuprHolvvQkQhERERGRGKgFWkREREQkBgrQIiIiIiIxUIAWEREREYmBArSIiIiISAwUoEVEREREYqAALSJSTZlZHw0vJyJS/ShAi4jEmZmtMLPDZpYa43Yu+rjfKmFmU6LHvKnUPH90XpuqqkNEJNEoQIuIxFE0eF4BOOB6b6v5lpn5T7Hoa+D3ZpZUlfWIiCQyBWgRkfgaCqwBXgSGlV4QbZm+vdT0LWb2XvT9yujsjWaWZ2aDSq13v5kdNLMvzezWUvPrm9lLZnbIzPaa2WQz85Xa9yoze9zMvgKmnKLeZUAxcPPJFprZADNbb2bHzGyfmU0ptaxNtLX61uiyw2Y2wswuNbNNZnbEzOaV2d9tZrY9uu5yM2v9vb9NEZFqSAFaRCS+hgKLo69rzazpmWzknLsy+raLc66Oc+5P0elmQH2gBTAceMrMzoouezK6rC3QO3rsW7/dKz2A3UBTYNqpDg1kAg+aWfJJlp+I7rcBMAC428x+WmadHkB7YBAwB5gE9AMuAm4ys94AZnYD8ADw70Bj4B/AK6eoS0Sk2lKAFhGJEzP7EdAaeM059zGwC/iPCu62BHjIOVfinFsK5AEdo10uBgMTnXPHnXN7gNnAr0ptm+2ce9I5F3TOFZzqAM65N4FDwO0nWbbCObfZORd2zm0iEnh7l1ntYedcoXPuHSKB+xXn3EHn3H4iIblrdL0RwAzn3HbnXBCYDlyiVmgRSTQK0CIi8TMMeMc5lxudfpky3TjK4ato2PxGPlAHaAQkA3tLLdtLpKX6G/tiOM5kIi3HgdIzzayHmf0t2k3kKJEQ3KjMtgdKvS84yXSd6PvWwNxo144jRPpfW5maRUSqvVPdVCIiIjEwszTgJiDJzHKis1OBBmbWxTm3kUjrbHqpzZpV4JC5RFqnWwPbovNaAftLrePOdGfOuXfN7DPgnjKLXgbmAT92zhWa2Rz+NUCfqX3ANOfc4nJuLyJSLagFWkQkPn4KhIB/Ay6Jvi4k0oVhaHSdDcC/m1l6dLi64WX2cYBIf+bTcs6FgNeAaWZWN9oNYiywqAI/wyRgQpl5dYGvo+G5OxXrkvIMMNHMLoJ/3gR5YwX2JyLiCQVoEZH4GAa84Jz7wjmX882LSOvtL6PDyD1OZMSLA8B/ErnRsLQpwH9GuzjcxOmNJNKqvRt4j0hr8cLy/gDOuVXAh2Vm3wM8ZGbHgd8RCe3l3f8bwB+AV83sGLAF+HF59yci4hVz7oy/4RMRERERqfXUAi0iIiIiEgMFaBERERGRGChAi4iIiIjEQAFaRERERCQGCtAiIiIiIjFQgBYRERERiYECtIiIiIhIDBSgRURERERioAAtIiIiIhKD/wd5sjtem9KLQAAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"This also seems to be somewhat useful. Now let us focus on creating some text based features. \n\nLet us first build a basic model to see how these meta features are helping. ","metadata":{"_uuid":"240a9ac16cf5d342e6d01cb020f1d8cea670ec84","_cell_guid":"3a622bfc-da8b-4f13-b9d8-a63dd79a5218"}},{"cell_type":"code","source":"## Prepare the data for modeling ###\nauthor_mapping_dict = {'EAP':0, 'HPL':1, 'MWS':2}\ntrain_y = train_df['author'].map(author_mapping_dict)\ntrain_id = train_df['id'].values\ntest_id = test_df['id'].values\n\n### recompute the trauncated variables again ###\ntrain_df[\"num_words\"] = train_df[\"text\"].apply(lambda x: len(str(x).split()))\ntest_df[\"num_words\"] = test_df[\"text\"].apply(lambda x: len(str(x).split()))\ntrain_df[\"mean_word_len\"] = train_df[\"text\"].apply(lambda x: np.mean([len(w) for w in str(x).split()]))\ntest_df[\"mean_word_len\"] = test_df[\"text\"].apply(lambda x: np.mean([len(w) for w in str(x).split()]))\n\ncols_to_drop = ['id', 'text']\ntrain_X = train_df.drop(cols_to_drop+['author'], axis=1)\ntest_X = test_df.drop(cols_to_drop, axis=1)","metadata":{"_uuid":"c27a85339ef5cf5ac3c87527249f878b615d3996","_cell_guid":"d97fc321-32a9-429e-9135-ae52300332ec","execution":{"iopub.status.busy":"2023-01-12T09:34:02.419548Z","iopub.execute_input":"2023-01-12T09:34:02.420047Z","iopub.status.idle":"2023-01-12T09:34:03.001617Z","shell.execute_reply.started":"2023-01-12T09:34:02.420003Z","shell.execute_reply":"2023-01-12T09:34:03.000260Z"},"trusted":true},"execution_count":10,"outputs":[]},{"cell_type":"markdown","source":"We can train a simple XGBoost model with these meta features alone.","metadata":{"_uuid":"1fddaea1621cba593f86d78851a26e1645f4954e","_cell_guid":"c4198d3f-5fad-41b1-9bcf-3b2faef2287f"}},{"cell_type":"code","source":"def runXGB(train_X, train_y, test_X, test_y=None, test_X2=None, seed_val=0, child=1, colsample=0.3):\n param = {}\n param['objective'] = 'multi:softprob'\n param['eta'] = 0.1\n param['max_depth'] = 3\n param['silent'] = 1\n param['num_class'] = 3\n param['eval_metric'] = \"mlogloss\"\n param['min_child_weight'] = child\n param['subsample'] = 0.8\n param['colsample_bytree'] = colsample\n param['seed'] = seed_val\n num_rounds = 2000\n\n plst = list(param.items())\n xgtrain = xgb.DMatrix(train_X, label=train_y)\n\n if test_y is not None:\n xgtest = xgb.DMatrix(test_X, label=test_y)\n watchlist = [ (xgtrain,'train'), (xgtest, 'test') ]\n model = xgb.train(plst, xgtrain, num_rounds, watchlist, early_stopping_rounds=50, verbose_eval=20)\n else:\n xgtest = xgb.DMatrix(test_X)\n model = xgb.train(plst, xgtrain, num_rounds)\n\n pred_test_y = model.predict(xgtest, ntree_limit = model.best_ntree_limit)\n if test_X2 is not None:\n xgtest2 = xgb.DMatrix(test_X2)\n pred_test_y2 = model.predict(xgtest2, ntree_limit = model.best_ntree_limit)\n return pred_test_y, pred_test_y2, model","metadata":{"_uuid":"7e736a33d7aefcc49edb0e629e57462f92e86c99","_cell_guid":"1534f55b-1334-4f1e-a597-d51e4d190fb3","execution":{"iopub.status.busy":"2023-01-12T09:34:03.003082Z","iopub.execute_input":"2023-01-12T09:34:03.003455Z","iopub.status.idle":"2023-01-12T09:34:03.014138Z","shell.execute_reply.started":"2023-01-12T09:34:03.003422Z","shell.execute_reply":"2023-01-12T09:34:03.013218Z"},"trusted":true},"execution_count":11,"outputs":[]},{"cell_type":"markdown","source":"For the sake of kernel run time, we can just check the first fold in the k-fold cross validation for the scores. Please remove the 'break' line while running in local.","metadata":{"_uuid":"607c32af563879d1f7e011a438216ab8bea74ae3","_cell_guid":"1effdaf3-faca-455d-a7a2-ae4d81b5c209"}},{"cell_type":"code","source":"kf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\ncv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_X.loc[dev_index], train_X.loc[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runXGB(dev_X, dev_y, val_X, val_y, test_X, seed_val=0)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\n break\nprint(\"cv scores : \", cv_scores)","metadata":{"_uuid":"441491131b9a7272863714494dd1303022c9d630","_cell_guid":"97965af9-da5a-4ccc-94c5-bc541ed00d49","execution":{"iopub.status.busy":"2023-01-12T09:34:03.015221Z","iopub.execute_input":"2023-01-12T09:34:03.016486Z","iopub.status.idle":"2023-01-12T09:34:13.808122Z","shell.execute_reply.started":"2023-01-12T09:34:03.016423Z","shell.execute_reply":"2023-01-12T09:34:13.807185Z"},"trusted":true},"execution_count":12,"outputs":[{"name":"stdout","text":"[09:34:03] WARNING: ../src/learner.cc:627: \nParameters: { \"silent\" } might not be used.\n\n This could be a false alarm, with some parameters getting used by language bindings but\n then being mistakenly passed down to XGBoost core, or some parameter actually being used\n but getting flagged wrongly here. Please open an issue if you find any such cases.\n\n\n[0]\ttrain-mlogloss:1.09242\ttest-mlogloss:1.09321\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/xgboost/core.py:571: FutureWarning: Pass `evals` as keyword args. Passing these as positional arguments will be considered as error in future releases.\n format(\", \".join(args_msg)), FutureWarning\n","output_type":"stream"},{"name":"stdout","text":"[20]\ttrain-mlogloss:1.04163\ttest-mlogloss:1.05206\n[40]\ttrain-mlogloss:1.02159\ttest-mlogloss:1.03705\n[60]\ttrain-mlogloss:1.00633\ttest-mlogloss:1.02497\n[80]\ttrain-mlogloss:0.99540\ttest-mlogloss:1.01691\n[100]\ttrain-mlogloss:0.98804\ttest-mlogloss:1.01233\n[120]\ttrain-mlogloss:0.98099\ttest-mlogloss:1.00796\n[140]\ttrain-mlogloss:0.97523\ttest-mlogloss:1.00485\n[160]\ttrain-mlogloss:0.97026\ttest-mlogloss:1.00214\n[180]\ttrain-mlogloss:0.96549\ttest-mlogloss:0.99993\n[200]\ttrain-mlogloss:0.96126\ttest-mlogloss:0.99812\n[220]\ttrain-mlogloss:0.95772\ttest-mlogloss:0.99691\n[240]\ttrain-mlogloss:0.95440\ttest-mlogloss:0.99581\n[260]\ttrain-mlogloss:0.95070\ttest-mlogloss:0.99442\n[280]\ttrain-mlogloss:0.94753\ttest-mlogloss:0.99339\n[300]\ttrain-mlogloss:0.94429\ttest-mlogloss:0.99274\n[320]\ttrain-mlogloss:0.94176\ttest-mlogloss:0.99259\n[340]\ttrain-mlogloss:0.93856\ttest-mlogloss:0.99157\n[360]\ttrain-mlogloss:0.93625\ttest-mlogloss:0.99123\n[380]\ttrain-mlogloss:0.93373\ttest-mlogloss:0.99055\n[400]\ttrain-mlogloss:0.93154\ttest-mlogloss:0.98993\n[420]\ttrain-mlogloss:0.92961\ttest-mlogloss:0.98986\n[440]\ttrain-mlogloss:0.92704\ttest-mlogloss:0.98921\n[460]\ttrain-mlogloss:0.92475\ttest-mlogloss:0.98910\n[480]\ttrain-mlogloss:0.92281\ttest-mlogloss:0.98925\n[500]\ttrain-mlogloss:0.92069\ttest-mlogloss:0.98871\n[520]\ttrain-mlogloss:0.91871\ttest-mlogloss:0.98873\n[540]\ttrain-mlogloss:0.91685\ttest-mlogloss:0.98893\n[560]\ttrain-mlogloss:0.91492\ttest-mlogloss:0.98877\n[573]\ttrain-mlogloss:0.91395\ttest-mlogloss:0.98877\ncv scores : [0.9886236644066505]\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/xgboost/core.py:94: UserWarning: ntree_limit is deprecated, use `iteration_range` or model slicing instead.\n UserWarning\n","output_type":"stream"}]},{"cell_type":"markdown","source":"We are getting a mlogloss of '0.987' using just the meta features. Not a bad score. Now let us see which of these features are important.","metadata":{"_uuid":"6ebaf270a27c8b07455498111bd0a8b2c11b6b16","_cell_guid":"ef5d6aa2-9d59-48e8-87ad-7ab254e0dc8a"}},{"cell_type":"code","source":"### Plot the important variables ###\nfig, ax = plt.subplots(figsize=(12,12))\nxgb.plot_importance(model, max_num_features=50, height=0.8, ax=ax)\nplt.show()","metadata":{"_uuid":"506938cebfedaad3e70d531e4cad554abe2a3c55","_cell_guid":"2856aa57-cce7-4f18-af56-a77ebd52eef7","execution":{"iopub.status.busy":"2023-01-12T09:34:13.812503Z","iopub.execute_input":"2023-01-12T09:34:13.813215Z","iopub.status.idle":"2023-01-12T09:34:14.103156Z","shell.execute_reply.started":"2023-01-12T09:34:13.813167Z","shell.execute_reply":"2023-01-12T09:34:14.101811Z"},"trusted":true},"execution_count":13,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 864x864 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAyEAAALJCAYAAACjjiyoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABXBUlEQVR4nO3de5hWZb3/8fdXMA+QGCGKDoYEeQLEQyaZNDu3WlrYwUqzFLHcWB4qLN0/D6l7l2RalrG3m7JELTsrtVXUzPF8CBRFLcI2U0qoqaCCSgLf3x/PYnoYZoaBh1kzw7xf1zXXrHWvte51ry8P8HzmXuuZyEwkSZIkqSybdPYAJEmSJPUshhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJKnHi4j/FxHf7+xxSFJPEf6eEElSLSKiEdgWWFHV/LbM/FuNfX46M39b2+i6n4g4FxiWmZ/s7LFIUkdxJkSStCF8IDP7Vn2tdwDZECKid2eef31113FL0royhEiSOkRE9IuIyyNiYUQsiIj/jIhexba3RsTvIuL5iHguIn4UEVsX264CdgR+ExFLIuLLEVEfEU81678xIv61WD43In4REVdHxEvA+LbO38JYz42Iq4vlIRGREXFcRDwZEYsiYmJEvD0iHomIxRHx3apjx0fE3RHx3Yh4MSL+GBEHVm3fPiJ+HREvRMQTEfGZZuetHvdE4P8BHy+u/eFiv+Mi4g8R8XJE/F9E/FtVH/UR8VRETIqIZ4vrPa5q+xYRcXFE/KUY310RsUWxbb+IuKe4pocjon49/qglaZ0ZQiRJHeUKYDkwDNgTOBj4dLEtgAuA7YFdgcHAuQCZ+Sngr/xzduXCdp7vcOAXwNbAj9Zy/vZ4BzAc+DhwCXAm8K/A7sDHIuLdzfb9MzAA+Arwq4joX2z7CfBUca1HAF+LiPe0Mu7Lga8BPy2ufY9in2eB9wNbAccB34qIvar62A7oB+wAHA9MiYg3FdsuAvYG3gn0B74MrIyIHYDrgf8s2k8DfhkR26xDjSRpvRhCJEkbwnXFT9MXR8R1EbEtcCjw+cxcmpnPAt8CjgTIzCcy85bMXJaZfwe+Cby79e7b5d7MvC4zV1J5s97q+dvpPzLztcy8GVgKXJOZz2bmAuBOKsFmlWeBSzLz9cz8KTAXOCwiBgP7A6cXfc0Gvg8c09K4M/PVlgaSmddn5p+z4nbgZuCAql1eB84vzn8DsATYOSI2ASYAp2bmgsxckZn3ZOYy4JPADZl5Q3HuW4CZRd0kqUN576kkaUP4YPVD5BGxL7ApsDAiVjVvAjxZbN8W+DaVN9JvLLYtqnEMT1Ytv6Wt87fTM1XLr7aw3rdqfUGu/kkvf6Ey87E98EJmvtxs2z6tjLtFEfE+KjMsb6NyHVsCc6p2eT4zl1etv1KMbwCwOZVZmubeAnw0Ij5Q1bYpcNvaxiNJtTKESJI6wpPAMmBAszfHq3wNSGBkZr4QER8Evlu1vflHNy6l8sYbgOLZjua3DVUfs7bzb2g7RERUBZEdgV8DfwP6R8Qbq4LIjsCCqmObX+tq6xGxGfBLKrMn0zPz9Yi4jsotbWvzHPAa8Fbg4WbbngSuyszPrHGUJHUwb8eSJG1wmbmQyi1DF0fEVhGxSfEw+qpbrt5I5ZahF4tnE77UrItngKFV638CNo+IwyJiU+AsYLMazr+hDQROiYhNI+KjVJ5zuSEznwTuAS6IiM0jYhSVZzaubqOvZ4Ahxa1UAG+gcq1/B5YXsyIHt2dQxa1pPwC+WTwg3ysixhTB5mrgAxFxSNG+efGQe926X74krRtDiCSpoxxD5Q3041RutfoFMKjYdh6wF/AilYejf9Xs2AuAs4pnTE7LzBeBz1J5nmIBlZmRp2hbW+ff0O6n8hD7c8BXgSMy8/li21HAECqzItcCX1nL7z/5efH9+Yh4sJhBOQX4GZXr+ASVWZb2Oo3KrVu/B14Avg5sUgSkw6l8GtffqcyMfAnfG0gqgb+sUJKkGkTEeCq/WPFdnT0WSeou/GmHJEmSpFIZQiRJkiSVytuxJEmSJJXKmRBJkiRJpfL3hPRAW2+9dQ4bNqyzh9FtLV26lD59+nT2MLot61cb61cb61cb61cb61cb61ebzqjfrFmznsvM5r/TCTCE9EjbbrstM2fO7OxhdFsNDQ3U19d39jC6LetXG+tXG+tXG+tXG+tXG+tXm86oX0T8pbVt3o4lSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpYrM7OwxqGQ7Dh2Wm3zs2509jG5r0sjlXDynd2cPo9uyfrWxfrWxfrWxfrWxfrXZGOrXOPmwTjt3Q0MD9fX1pZ4zImZl5j4tbXMmRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSSrJhAkTGDhwICNGjGhqe/jhhxkzZgwjR47kAx/4AC+99FLTtgsuuIBhw4ax8847c9NNNwHw2muvse+++7LHHnuw++6785WvfKXFcy1btoyPf/zjDBs2jBNPPJHGxsYOvbZ1YQiRJEmSSjJ+/HhmzJixWtunP/1pJk+ezJw5c/jQhz7EN77xDQAef/xxfvKTn/DYY48xY8YMPvvZz7JixQo222wzfve73/Hwww8ze/ZsZsyYwX333bfGuS6//HLe9KY38cQTT/DRj36U008/vZRrbA9DSCeKiCER8Wgb2+sj4n/LHJMkSZI6ztixY+nfv/9qbX/6058YO3YsAAcddBC//OUvAZg+fTpHHnkkm222GTvttBPDhg3jgQceICLo27cvAK+//jqvv/46EbHGuaZPn86xxx4LwLvf/W5uvfVWMrMjL6/dDCEliojenT0GSZIkdS27774706dPB+DnP/85Tz75JAALFixg8ODBTfvV1dWxYMECAFasWMHo0aMZOHAgBx10EO94xzvW6Lf6+F69etGvXz+ef/75jr6cdukWb4ojYggwA7gPeCfwe+CHwHnAQOBo4DHgUmAEsClwbmZOL469CuhTdHdSZt4TEfXAucBzxTGzgE9mC/EwIt4O/HtmfjgiDgd+AvSjEuIez8yhETEauAzYEvgzMCEzF0VEAzAbeBdwTbH+g6Lrm9ehBn1aub7xwLjivG8Frs3ML7dw/AnACQADBmzDOSOXt/fUambbLWCS9Vtv1q821q821q821q821q82G0P9GhoaAHj66adZunRp0/rEiRP56le/ype//GX2339/NtlkExoaGliwYAF/+MMfmvZbuHAhjz32GAMGDADgkksuYcmSJZx99tnssssu7LTTTqudb+nSpdx7771ss802LFmyhNdee427776bfv36lXXJreoWIaQwDPgoMIFKCPkElTf244D/BzwO/C4zJ0TE1sADEfFb4FngoMx8LSKGA9cA+xR97gnsDvwNuBvYH7irhXM/BIwulg8AHgXeTqV+9xftVwInZ+btEXE+8BXg88W2N2TmPgAR8QiVIHRHRHxjHa7/zFauj2JsewLLgLkRcWlmPll9cGZOBaYC7Dh0WF48pzv90Xctk0Yux/qtP+tXG+tXG+tXG+tXG+tXm42hfo1H11e+NzbSp08f6uvrm7Ydc8wxQOXWrMcee4z6+nruvfdegKb9LrjgAg4++GDGjBmzWr8PPvggzz//PMcdd9xq7W9729uoq6tjzJgx3HrrrSxbtoxx48a1eOtW2brT7VjzM3NOZq6kMutxazFrMQcYAhwMnBERs4EGYHNgRyqzBt+LiDnAz4Hdqvp8IDOfKvqcXfSzhsxcDvw5InYF9gW+CYylEkjujIh+wNaZeXtxyLRi+yo/BSjCw9aZeUfRftU6XH9r10dRixcz8zUqYewt69CvJEmSOtGzzz4LwMqVK/nP//xPJk6cCMC4ceP4yU9+wrJly5g/fz7z5s1j33335e9//zuLFy8G4NVXX+WWW25hl112WaPfcePGMW3aNABuv/123vOe93SJAALdayZkWdXyyqr1lVSuYwXwkcycW31QRJwLPAPsQSV0vdZKnytoux53AO8DXgd+C1wB9AK+1I6xL23HPmsTtHx972DdrkOSJEmd5KijjqKhoYHnnnuOuro6zjvvPJYsWcKUKVMA+PCHP9w0o7H77rvzsY99jN12243evXszZcoUevXqxcKFCzn22GNZsWIFK1eu5GMf+xjvf//7ATjnnHPYZ599GDduHMcffzyf+tSnGDZsGJtuuinXX399p113cxvTm9WbgJMj4uTMzIjYMzMfovLsxlOZuTIijqUSHNbHnVRuuboyM/8eEW8GtgUeLc63KCIOyMw7gU8BtzfvIDMXR8TiiHhXZt5F5VmWWq9PkiRJ3cQ111zTYvupp57aYvuZZ57JmWeeuVrbqFGjeOihlt8Gnn/++U3Lm2++OT//+c+ByvMoQ4cOXZ8hd4judDvW2vwHlVuvHomIx4p1gP8Cjo2Ih4FdWP9ZifuphI5Vt1I9AsypepD9WOAbxTMfo4Hz1+ih4jhgSnFb1brMh7V2fZIkSVK30i1mQjKzkcqnQq1aH9/Ktn9r4dh5wKiqptOL9gYqz1as2u+ktYzhVWCzqvUTmm2fDezXwnH1zdZnUbk1bJU1Psmqat+mMRbnb+n6rqBya9iq9fe3ehGSJElSF7AxzYRIkiRJ6ga6xUxImSLiWmCnZs2nZ+ZNHXjOQ4CvN2uen5kf6qhzSpIkSZ3FENJMZ7zxLwJOh4UcSZIkqSvxdixJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUql6d/YAVL4tNu3F3MmHdfYwuq2GhgYaj67v7GF0W9avNtavNtavNtavNtavNtZv4+JMiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKlVkZmePQSXbceiw3ORj3+7sYXRbk0Yu5+I5vTt7GN2W9auN9auN9auN9auN9atNR9SvcfJhG7S/rqyhoYH6+vpSzxkRszJzn5a2ORMiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZKkHmvChAkMHDiQESNGNLXNnj2b/fbbj9GjR7PPPvvwwAMPAPDHP/6RMWPGsNlmm3HRRRet1s/ixYs54ogj2GWXXdh1112599571zhXZnLKKacwbNgwRo0axYMPPtixF9eFGUIkSZLUY40fP54ZM2as1vblL3+Zr3zlK8yePZvzzz+fL3/5ywD079+f73znO5x22mlr9HPqqafy3ve+lz/+8Y88/PDD7Lrrrmvsc+ONNzJv3jzmzZvH1KlTOfHEEzvmoroBQ0gni4j6iPjfzh6HJElSTzR27Fj69++/WltE8NJLLwHw4osvsv322wMwcOBA3v72t7Ppppuutv+LL77IHXfcwfHHHw/AG97wBrbeeus1zjV9+nSOOeYYIoL99tuPxYsXs3Dhwg64qq6vd2cPQLWJiN6ZubyzxyFJkrSxuOSSSzjkkEM47bTTWLlyJffcc0+b+8+fP59tttmG4447jocffpi9996bb3/72/Tp02e1/RYsWMDgwYOb1uvq6liwYAGDBg3qkOvoynp8CImIIcCNwF3AO4EFwOFF22mZOTMiBgAzM3NIRIwHPgj0AYYDFwFvAD4FLAMOzcwXWjnXMOAyYBtgBfDRYlPfiPgFMAKYBXwyMzMizgE+AGwB3AP8W9HeAMwG3gVcExF/Bb5S9PliZo5t4dwnACcADBiwDeeMNLesr223gEnWb71Zv9pYv9pYv9pYv9pYv9p0RP0aGhoAePrpp1m6dGnT+ne+8x2OP/543v3ud3Pbbbfx4Q9/mIsvvrjpuMbGRrbYYoum/efOncusWbMYP34848eP59JLL+XEE09kwoQJq53v+eef56GHHmL58sp1LFq0iFmzZrFkyZINel0tWbJkSdN4u4IeH0IKw4GjMvMzEfEz4CNr2X8EsCewOfAEcHpm7hkR3wKOAS5p5bgfAZMz89qI2JzK7XCDi752B/4G3A3sTyUUfTczzweIiKuA9wO/Kfp6Q2buU2ybAxySmQsiYuuWTpyZU4GpADsOHZYXz/GPfn1NGrkc67f+rF9trF9trF9trF9trF9tOqJ+jUfXV743NtKnTx/q6yvrhx9+OL/85S+JCN797nfzrW99q2kbVMJL3759m9p22WUXLrjgAj772c8C0KtXLyZPnrzaMQCjRo1iwIABTe1Lly5l3LhxpcyENDQ0rDGezuQzIRXzM3N2sTwLGLKW/W/LzJcz8+/Ai/wzGMxp7diIeCOwQ2ZeC5CZr2XmK8XmBzLzqcxcSWWGY1Uf/xIR9xch4z1UgsoqP61avhu4IiI+A/Ray9glSZLUhu23357bb78dgN/97ncMHz68zf232247Bg8ezNy5cwG49dZb2W233dbYb9y4cVx55ZVkJvfddx/9+vXrkbdigTMhqyyrWl5B5fan5fwzpG3exv4rq9ZXsn41bX7+3sVMyX8B+2TmkxFxbrNxLF21kJkTI+IdwGHArIjYOzOfX49xSJIk9ShHHXUUDQ0NPPfcc9TV1XHeeefxve99j1NPPZXly5ez+eabM3XqVKBy29Y+++zDSy+9xCabbMIll1zC448/zlZbbcWll17K0UcfzT/+8Q+GDh3KD3/4QwAuu+wyACZOnMihhx7KDTfcwLBhw9hyyy2b9umJDCGtawT2Bh4Ajqi1s8x8OSKeiogPZuZ1EbEZbc9arAocz0VE32IMv2hpx4h4a2beD9wfEe+jcouXIUSSJGktrrnmmhbbZ82atUbbdtttx1NPPdXi/qNHj2bmzJlrtE+cOLFpOSKYMmXKeo504+LtWK27CDgxIh4CBmygPj8FnBIRj1B50Hy71nbMzMXA94BHgZuA37fR7zciYk5EPFr0+/AGGq8kSZK0wfX4mZDMbKTyoPmq9epffzmqavmsYvsVwBVV+w+pWl5tWwvnmkfl2Y5q/wc0VO1zUtXyWavO26yf+mbrH27tnJIkSVJX40yIJEmSpFL1+JmQjhARU6h8zG61b2dmz336SJIkSSoYQjpAZn6us8cgSZIkdVXejiVJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKlXvzh6AyrfFpr2YO/mwzh5Gt9XQ0EDj0fWdPYxuy/rVxvrVxvrVxvrVxvrVxvptXJwJkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSpV784egMr36usrGHLG9Z09jG5r0sjljLd+68361cb61cb61cb61aat+jVOPqzk0Uidy5kQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZKkLmLChAkMHDiQESNGNLWde+657LDDDowePZrRo0dzww03AHDLLbew9957M3LkSPbee29+97vfrdHfuHHjVuurWmZyyimnMGzYMEaNGsWDDz7YMRcltcAQIkmS1EWMHz+eGTNmrNH+hS98gdmzZzN79mwOPfRQAAYMGMBvfvMb5syZw7Rp0/jUpz612jG/+tWv6Nu3b6vnuvHGG5k3bx7z5s1j6tSpnHjiiRv2YqQ2GELWIiLqI+KdnT2O5iLiiog4orPHIUmSNpyxY8fSv3//du275557sv322wOw++678+qrr7Js2TIAlixZwje/+U3OOuusVo+fPn06xxxzDBHBfvvtx+LFi1m4cGHtFyG1gyFk7eqBTg0hEdG7M88vSZI613e/+11GjRrFhAkTWLRo0Rrbf/nLX7LXXnux2WabAXD22WczadIkttxyy1b7XLBgAYMHD25ar6urY8GCBRt+8FILuvyb24gYAtwI3EUlDCwADi/aTsvMmRExAJiZmUMiYjzwQaAPMBy4CHgD8ClgGXBoZr7QyrlOASYCy4HHgTOK9RUR8UngZOBJ4AfAAODvwHGZ+deIuAJ4DdgH2Ar4Ymb+b0RcD/x7Zj4SEQ8B12bm+RFxftHX94ELgfcBCfxnZv40IuqB/wAWAbtExM7ApcBBxXH/qBr3ZGBcMe6bM/O0Fq7tBOAEgAEDtuGckcvXWnu1bNstYJL1W2/WrzbWrzbWrzbWrzZt1a+hoaFp+emnn2bp0qVNbaNGjeLyyy8nIvjBD37AJz7xCU4//fSm/efPn89ZZ53FhRdeSENDA0888QQPPPAAhx9+OPfdd99qfVV7/vnneeihh1i+vDKmRYsWMWvWLJYsWbLBrnlDWrJkSYvXofbpavXr8iGkMBw4KjM/ExE/Az6ylv1HAHsCmwNPAKdn5p4R8S3gGOCSVo47A9gpM5dFxNaZuTgiLgOWZOZFABHxG2BaZk6LiAnAd6iEHoAhwL7AW4HbImIYcCdwQET8hUpI2L/Y9wAqAefDwGhgDyrB5vcRcUexz17AiMycHxEfBnYGdgO2pRKSfhARbwY+BOySmRkRW7d0YZk5FZgKsOPQYXnxnO7yR9/1TBq5HOu3/qxfbaxfbaxfbaxfbdqqX+PR9f9cbmykT58+1NfXr7Hf0KFDef/739+07amnnuKEE07gZz/7GfvvX3mL8Yc//IH58+czfvx4li9fzrPPPsu55567xhvQUaNGMWDAgKa+li5dyrhx4xg0aFCtl9ohGhoaWqyJ2qer1a+73I41PzNnF8uzqLzZb8ttmflyZv4deBH4TdE+Zy3HPgL8qJj1aO1HPWOAHxfLVwHvqtr2s8xcmZnzgP8DdqESQsZSCR/XA30jYksqYWducfw1mbkiM58BbgfeXvT3QGbOL5bHVu33N2DVR2C8SGUG5vIiqLzSxvVJkqRupvo5jWuvvbbp064WL17MYYcdxuTJk5sCCMCJJ57I3/72NxobG7nrrrt429ve1uJPwMeNG8eVV15JZnLffffRr1+/LhtAtPHpLj/OWFa1vALYgkpIWBWiNm9j/5VV6ytp+5oPo/Jm/wPAmRExch3HmS2s/57KLVr/B9xCZbbjM1TC1NosXesJM5dHxL7AgcARwEnAe9ZhzJIkqYs46qijaGho4LnnnqOuro7zzjuPhoYGZs+eTUQwZMgQ/ud//geoPCfyxBNPcP7553P++ecDcPPNNzNw4MBW+7/ssssAmDhxIoceeig33HADw4YNY8stt+SHP/xhx1+gVOguIaQljcDewANU3nzXJCI2AQZn5m0RcRdwJNAXeJnKMx6r3FNsuwo4mspMxyofjYhpwE7AUGBuZv4jIp4EPgqcD2xD5TmVi4pj7gT+rTiuP5UQ9CUqsyjV7qjabyDwL8CPI6IvsGVm3hARd1MJO5IkqRu65ppr1mg7/vjjW9z3rLPOavPTrwCGDBnCo48+2rQ+ceLEpuWIYMqUKes5Uqk23TmEXAT8rHjg+voN0F8v4OqI6AcE8J3imZDfAL+IiMOpPJh+MvDDiPgSxYPpVX38lUoo2gqYmJmvFe13Agdm5qsRcSdQxz/Dy7VUbvF6mMrMyZcz8+mIaB5CrqUyw/F4cZ57i/Y3AtMjYvNi3F/cALWQJEmSOkyXDyGZ2UjlQfNV6xdVbR5VtXxWsf0K4Iqq/YdULa+2rdl5Xmf15ztWtf+p2Xmg9dudfpuZE5s3ZubZwNnF8t+ohIVV25LKzMeXmh3TADQ02++kVs67byvtkiRJUpfTXR5MlyRJkrSR6PIzIR0hIqbwz4/KXeXbmbneT2Rl5viaBiVJkiT1ED0yhGTm5zp7DJIkSVJP5e1YkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRS9e7sAah8W2zai7mTD+vsYXRbDQ0NNB5d39nD6LasX22sX22sX22sX22sn/RPzoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRS9e7sAah8r76+giFnXN/Zw+i2Jo1cznjrt96sX22sX22sX22s39o1Tj6ss4cgdQvOhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJG1gEyZMYODAgYwYMWKNbRdffDERwXPPPQfAiy++yAc+8AH22GMPdt99d374wx827Ttt2jSGDx/O8OHDmTZtWovneuGFFzjooIMYPnw4Bx10EIsWLeqYi5I2IEOIJEnSBjZ+/HhmzJixRvuTTz7JzTffzI477tjUNmXKFHbbbTcefvhhGhoamDRpEv/4xz944YUXOO+887j//vt54IEHOO+881oMGJMnT+bAAw9k3rx5HHjggUyePLlDr03aEAwh3VhELOnsMUiSpDWNHTuW/v37r9H+hS98gQsvvJCIaGqLCF5++WUykyVLltC/f3969+7NTTfdxEEHHUT//v1505vexEEHHdRisJk+fTrHHnssAMceeyzXXXddh12XtKH07uwBqH0iondmLu/scUiSpPVz1113scMOO7DHHnus1n7SSScxbtw4tt9+e15++WV++tOfsskmm7BgwQIGDx7ctF9dXR0LFixYo99nnnmGQYMGAbDddtvxzDPPdOyFSBtAjw0hETEEuBG4C3gnsAA4vGg7LTNnRsQAYGZmDomI8cAHgT7AcOAi4A3Ap4BlwKGZ+UIL5xkI3JiZe0fEHsBs4C2Z+deI+DMwEhgI/AAYAPwdOK7YfgXwGrAncHdEXAr8GOgLTK86xyDgp8BWVP5MT8zMO5uN4wTgBIABA7bhnJHmmfW17RYwyfqtN+tXG+tXG+tXG+u3dg0NDU3LTz/9NEuXLqWhoYHXXnuNq666iosvvrhp/e6776Zfv37cfvvtDBgwgB//+Mf87W9/49Of/jTf//73+fOf/8w//vGPpj7nz5/PZptttto5AJYvX75a24oVK9bYZ2OwZMmSjfK6ytLV6tdjQ0hhOHBUZn4mIn4GfGQt+4+gEgg2B54ATs/MPSPiW8AxwCXND8jMZyNi84jYCjgAmAkcEBF3Ac9m5itFuJiWmdMiYgLwHSqBB6AOeGdmroiIXwP/nZlXRsTnqk7zCeCmzPxqRPQCtmxhHFOBqQA7Dh2WF8/p6X/062/SyOVYv/Vn/Wpj/Wpj/Wpj/dau8ej6fy43NtKnTx/q6+uZM2cOzz77LCeddBIAzz33HCeffDIPPPAA3/jGNzjjjDM44IADALj88svZZpttGDt2LA0NDdTXV/q85pprGDt2bNP6KjvssAM777wzgwYNYuHChWy//fZr7LMxqK6F1l1Xq19PfyZkfmbOLpZnAUPWsv9tmflyZv4deBH4TdE+Zy3H3gPsD4wFvlZ8PwBYNVsxhsoMB8BVwLuqjv15Zq4olvcHrqnab5XfA8dFxLnAyMx8eS3XIUmSSjRy5EiuvfZaGhsbaWxspK6ujgcffJDtttuOHXfckVtvvRWo3Fo1d+5chg4dyiGHHMLNN9/MokWLWLRoETfffDOHHHLIGn2PGzeu6ZOzpk2bxuGHH17qtUnro6eHkGVVyyuozAwt55912byN/VdWra+k7VmlO6iEjrdQuY1qDypB4842jlllabP1bL5DZt5BJdgsAK6IiGPa0a8kSeogRx11FGPGjGHu3LnU1dVx+eWXt7rv2WefzT333MPIkSM58MAD+frXv86AAQPo378/Z599Nm9/+9t5+9vfzjnnnNP0sPunP/1pZs6cCcAZZ5zBLbfcwvDhw/ntb3/LGWecUco1SrVwTnVNjcDewAPAERuozzuBrwJ3ZObKiHgBOBT492L7PcCRVGY3jqb1cHJ3sd/VxX4ARMRbgKcy83sRsRmwF3DlBhq7JElaR9dcc80abdX34zc2NjYtb7/99tx8880t9jNhwgQmTJiwRvv3v//9puU3v/nNTTMpUnfR02dCWnIRcGJEPETlQfGaZWYjEFRmRKDyMPzizFz1Yd8nU7md6hEqD7qf2kpXpwKfi4g5wA5V7fXAw8WYPw58e0OMW5IkSeoIPXYmpAgGI6rWL6raPKpq+axi+xXAFVX7D6laXm1bK+cbXLX8NSrPhqxa/wvwnhaOGd9sfT6V50eaj20a0PKvUZUkSZK6GGdCJEmSJJWqx86EdISImELlE6yqfTszf9gZ45EkSZK6IkPIBpSZn1v7XpIkSVLP5u1YkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKp2hVCIuKtEbFZsVwfEadExNYdOjJJkiRJG6X2zoT8ElgREcOAqcBg4McdNipJkiRJG632hpCVmbkc+BBwaWZ+CRjUccOSJEmStLFqbwh5PSKOAo4F/rdo27RjhiRJkiRpY9beEHIcMAb4ambOj4idgKs6bliSJEmSNla927NTZj4eEacDOxbr84Gvd+TAJEmSJG2c2vvpWB8AZgMzivXREfHrDhyXJEmSpI1Ue2/HOhfYF1gMkJmzgaEdMiJJkiRJG7V2P5iemS82a1u5oQcjSZIkaePXrmdCgMci4hNAr4gYDpwC3NNxw5IkSZK0sWrvTMjJwO7AMiq/pPBF4PMdNCZJkiRJG7G1zoRERC/g+sz8F+DMjh+SOtoWm/Zi7uTDOnsY3VZDQwONR9d39jC6LetXG+tXG+tXG+snaUNZ60xIZq4AVkZEvxLGI0mSJGkj195nQpYAcyLiFmDpqsbMPKVDRiVJkiRpo9XeEPKr4kuSJEmSatLe35g+raMHIkmSJKlnaFcIiYj5QDZvz0x/YaEkSZKkddLe27H2qVreHPgo0H/DD0eSJEnSxq5dvyckM5+v+lqQmZcAfsarJEmSpHXW3tux9qpa3YTKzEh7Z1EkSZIkqUl7g8TFVcvLgfnAxzb8cCRJkiRt7NobQo7PzP+rboiInTpgPJIkSZI2cu16JgT4RTvbJEmSJKlNbc6ERMQuwO5Av4j4cNWmrah8SpYkSZIkrZO13Y61M/B+YGvgA1XtLwOf6aAxSZIkSdqItRlCMnM6MD0ixmTmvSWNSZIkSdJGrL0Ppj8UEZ+jcmtW021YmTmhQ0alDvXq6ysYcsb1nT2MbmvSyOWMt37rzfrVxvrVxvrVpqfXr3GyvyJN2lDa+2D6VcB2wCHA7UAdlVuyJEmSJGmdtDeEDMvMs4GlmTmNym9Lf0fHDUuSJEnSxqq9IeT14vviiBgB9AMGdsyQJEmSJG3M2vtMyNSIeBNwNvBroC9wToeNSpIkSdJGq10hJDO/XyzeDgztuOFIkiRJ2ti163asiNg2Ii6PiBuL9d0i4viOHZokSZKkjVF7nwm5ArgJ2L5Y/xPw+Q4YjyRJkqSNXHtDyIDM/BmwEiAzlwMrOmxUkiRJkjZa7Q0hSyPizUACRMR+wIsdNipJkiRJG632fjrWF6l8KtZbI+JuYBvgiA4blSRJkqSNVpshJCJ2zMy/ZuaDEfFuYGcggLmZ+Xpbx0qSJElSS9Z2O9Z1Vcs/zczHMvNRA4gkSZKk9bW2EBJVy/5+EEmSJEk1W1sIyVaWJUmSJGm9rO3B9D0i4iUqMyJbFMsU65mZW3Xo6CRJkiRtdNoMIZnZq6yBSJIkSeoZ2vt7QiRJkiRpgzCESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSpHaaMGECAwcOZMSIEWtsu/jii4kInnvuOQAaGhro168fo0ePZvTo0UybNq1d/aySmZxyyikMGzaMUaNG8eCDD274C5I6iSFEkiSpncaPH8+MGTPWaH/yySe5+eab2XHHHVdrP+CAA5g9ezazZ8/m2GOPXWs/1W688UbmzZvHvHnzmDp1KieeeOKGuQipC+ixISQiboiIrTt7HOsrIs6NiNM6exySJPUkY8eOpX///mu0f+ELX+DCCy8kImrqp9r06dM55phjiAj2228/Fi9ezMKFC9dr3FJX02NDSGYempmLO3sc7REVPfbPSpKkrmz69OnssMMO7LHHHmtsu/fee9ljjz143/vex/z589ep3wULFjB48OCm9bq6OhYsWFDzeKWuoHdHdRwRQ4AbgbuAdwILgMOLttMyc2ZEDABmZuaQiBgPfBDoAwwHLgLeAHwKWAYcmpkvtHKuhjb6HAdsCbwVuDYzv1wc0wjsk5nPRcSZwLHAs8CTwKzMvKiNfnsBk4F6YDNgSmb+TytjmwLclJm/johrgUWZOSEiJgBvzcwzI+KLwITikO9n5iVF/W4C7gf2Bg6NiE82H2dxjlOAicBy4PHMPLKFcZwAnAAwYMA2nDNyeUvDVTtsuwVMsn7rzfrVxvrVxvrVpqfXr6GhAYCnn36apUuX0tDQwGuvvcYZZ5zBN77xjab1u+++m379+rF06VKuvvpqtthiC+677z7OPPNMdtppp6b+qvtpyfPPP89DDz3E8uWVmi9atIhZs2axZMmSjr7ULmnJkiWt1kpr19Xq12EhpDAcOCozPxMRPwM+spb9RwB7ApsDTwCnZ+aeEfEt4BjgkvUYw+iiz2XA3Ii4NDOfXLUxIvYGjiz26w08SPHmvg3HAy9m5tsjYjPg7oi4OTNb+hHHncABwK+BHYBBRfsBwE+K8x8HvAMI4P6IuB1YRKV+x2bmfWsZ5xnATpm5rLVbzDJzKjAVYMehw/LiOR39R7/xmjRyOdZv/Vm/2li/2li/2vT0+jUeXV/53thInz59qK+vZ86cOTz//POcdNJJADz33HOcfPLJPPDAA2y33XZNx9bX13PJJZcwYsQIBgwYsEY/LRk1ahQDBgxo2r506VLGjRvHoEGDWtx/Y9fQ0NBqrbR2Xa1+HX2Lz/zMnF0szwKGrGX/2zLz5cz8O/Ai8JuifU47jm3NrZn5Yma+BjwOvKXZ9gOozJC8kpkvUQkLa3MwcExEzKYyU/FmKoGhJXcCB0TEbsX5n4mIQcAY4B7gXcX5l2bmEuBXxZgA/pKZ97VjnI8APypmSnruj6gkSSrZyJEjefbZZ2lsbKSxsZG6ujoefPBBtttuO55++mkyE4AHHniAzOTNb35zu/seN24cV155JZnJfffdR79+/XpsANHGp6NDyLKq5RVUfoK/vOq8m7ex/8qq9ZW0PWvT3j5XjaG9Wus3gJMzc3TxtVNm3txSB5m5ANgaeC9wB5VQ8jFgSWa+vJbzL23nOA8DpgB7Ab+PiJ77YypJkjrQUUcdxZgxY5g7dy51dXVcfvnlre77i1/8ghEjRrDHHntwyimncPbZZzc9uN5aP5dddhmXXXYZAIceeihDhw5l2LBhfOYzn+G//uu/Ov4CpZJ0xpvVRirPODwAHNEF+rwDuCIiLqBSjw8Aq57vaK3fm4ATI+J3mfl6RLwNWJCZrYWG+4DPA++hMmvyi+ILKqHkioiYTCXcfIjKczDtGmfxwPrgzLwtIu6icstWX2DxuhRBkiSt3TXXXNPm9sbGxqblk046qek2LWC1+/Fb62fixIlNyxHBlClT1m+gUhfXGZ+4dBGVN/APAQM6u8/MfBD4KfAwlYfmf9+Ofr9P5daqByPiUSqhpa1AdyfQOzOfoPIsR/+ibdX5r6ASdO6n8mD6Q+swzl7A1RExB3gI+E53+dQvSZIk9UwdNhOSmY1UHjRftX5R1eZRVctnFduvoPJmfNX+Q6qWV9vWwrn+2M4+399K/18FvgqV37/Rjn5XAv+v+FqrzLwcuLxYfp3KJ4BVb/8m8M1mbY1U1a/5OJt5V3vGIUmSJHUF/u4JSZIkSaXqVg8wF79zY/9mzd/OzB9uqHNk5rnrc1xEjASuata8LDPfUfOgJEmSpI1Itwohmfm5zh5DazJzDpXf4SFJkiSpDd6OJUmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqVe/OHoDKt8WmvZg7+bDOHka31dDQQOPR9Z09jG7L+tXG+tXG+tXG+knaUJwJkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJperd2QNQ+V59fQVDzri+s4fRbU0auZzx1m+9Wb/aWL/aWL/10zj5sM4egqSNjDMhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiSpXb7+9a8zcOBARowY0dT2pS99iV122YVRo0bxoQ99iMWLF692zF//+lf69u3LRRdd1NS2ePFijjjiCHbZZRd23XVX7r333jXOlZmccsopDBs2jFGjRvHggw922HVJKp8hRJIktct73/teZsyYsVrbQQcdxKOPPsojjzzC2972Ni644ILVtn/xi1/kfe9732ptp556Ku9973v54x//yMMPP8yuu+66xrluvPFG5s2bx7x585g6dSonnnjihr8gSZ3GENJBImLJehzz/5qt31N8HxIRn6hqr4+I/619lJIktd8ee+xB//79V2s7+OCD6d27NwD77bcfTz31VNO26667jp122ondd9+9qe3FF1/kjjvu4PjjjwfgDW94A1tvvfUa55o+fTrHHHMMEcF+++3H4sWLWbhwYQdclaTOYAjZACKi9wbqarUQkpnvLBaHAJ9YY29JkrqQH/zgB02zHkuWLOHrX/86X/nKV1bbZ/78+WyzzTYcd9xx7Lnnnnz6059m6dKla/S1YMECBg8e3LReV1fHggULOvYCJJVmQ7153qAiYghwI3AX8E5gAXB40XZaZs6MiAHAzMwcEhHjgQ8CfYDhwEXAG4BPAcuAQzPzhRbOMxC4MTP3jog9gNnAWzLzrxHxZ2AkMBD4ATAA+DtwXLH9CuA1YE/g7oi4FPgx0BeYXnWOQcBPga2o1PvEzLyzhbFMBraIiNnAY5l5dEQsycy+wGRg12LbNOChquP6AJcCI4BNgXMzc3oL/Z8AnAAwYMA2nDNyeYu119ptuwVMsn7rzfrVxvrVxvqtn4aGBqASLO677z6WLl3a1LbK1VdfzeLFi9lhhx1oaGjgv//7vzn44IOZOXMmjY2NbLHFFjQ0NDB37lxmzZrF+PHjGT9+PJdeeiknnngiEyZMWK2/559/noceeojlyyt/XosWLWLWrFksWbLONxp0GUuWLFmjbmo/61ebrla/LhlCCsOBozLzMxHxM+Aja9l/BJVAsDnwBHB6Zu4ZEd8CjgEuaX5AZj4bEZtHxFbAAcBM4ICIuAt4NjNfKcLFtMycFhETgO9QCTwAdcA7M3NFRPwa+O/MvDIiPld1mk8AN2XmVyOiF7BlS4PPzDMi4qTMHN3C5jOohK/3Q+V2rKptZwK/y8wJEbE18EBE/DYzV/uxUmZOBaYC7Dh0WF48pyv/0Xdtk0Yux/qtP+tXG+tXG+u3fhqPrgcqYWTEiBH06dOH+vr6pu1XXHEFjz32GLfeeitbbln5b+7ss8/m/vvvZ9q0aSxevJhNNtmE3XffnSOOOIILLriAz372swD06tWLyZMnr9YfwKhRoxgwYEBT+9KlSxk3bhyDBg3q6MvtMA0NDWtcp9rP+tWmq9WvK/9LPD8zZxfLs6jcktSW2zLzZeDliHgR+E3RPgcY1cZx9wD7A2OBrwHvBQJYNVsxBvhwsXwVcGHVsT/PzBXF8v78MyhdBXy9WP498IOI2BS4ruqaNpSDgXERcVqxvjmwI/CHDXweSZLWMGPGDC688EJuv/32pgACcOed/5z0P/fcc+nbty8nnXQSAIMHD2bu3LnsvPPO3Hrrrey2225r9Dtu3Di++93vcuSRR3L//ffTr1+/bh1AJK2uK4eQZVXLK4AtgOX88zmWzdvYf2XV+kravs47qMyCvIXKbVSnAwlc344xNr+JNZvvkJl3RMRY4DDgioj4ZmZe2Y6+2yuAj2Tm3A3YpyRJa/iP//gPHn/8cZ577jnq6uo477zzuOCCC1i2bBkHHXQQUHk4/bLLLmuzn0svvZSjjz6af/zjHwwdOpQf/vCHAE3HTZw4kUMPPZQbbriBYcOGseWWWzbtI2nj0JVDSEsagb2BB4AjNlCfdwJfBe7IzJUR8QJwKPDvxfZ7gCOpzG4czT9nSJq7u9jv6mI/ACLiLcBTmfm9iNgM2AtoLYS8HhGbZubrzdpfBt7YyjE3ASdHxMmZmRGxZ2Y+1Mq+kiStt7PPPnuN2zlWfcpVW84999zV1kePHs3MmTPX2G/ixIlNyxHBlClT1muckrq+7vbpWBcBJ0bEQ1QeFK9ZZjZSmU24o2i6C1icmYuK9ZOB4yLiESoPup/aSlenAp+LiDnADlXt9cDDxZg/Dny7jeFMBR6JiB81a38EWBERD0fEF5pt+w8qD6Q/EhGPFeuSJElSl9UlZ0KKYDCiav2iqs3Vz3ecVWy/Ariiav8hVcurbWvlfIOrlr9G5dmQVet/Ad7TwjHjm63Pp/L8SPOxTaPyiVZrlZmnU7kdbNV63+L76y2MoaHY9irwb+3pX5IkSeoKuttMiCRJkqRurkvOhHSEiJhC5ROsqn07M0t/0i0i7gc2a9b8qcycU/ZYJEmSpLL1mBCSmZ9b+17lyMx3dPYYJEmSpM7i7ViSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFL17uwBqHxbbNqLuZMP6+xhdFsNDQ00Hl3f2cPotqxfbaxfbayfJHUNzoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklap3Zw9A5Xv19RUMOeP6zh5GtzVp5HLGW7/1Zv1qY/1qszHVr3HyYZ09BElab86ESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJLUjU2YMIGBAwcyYsSIpraf//zn7L777myyySbMnDlztf0feeQRxowZw+67787IkSN57bXXADjzzDMZPHgwffv2bfN8F1xwAcOGDWPnnXfmpptu2vAXJKlHMISsh4gYEhGfqLGPz0fEllXrN0TE1jUPTpLUo4wfP54ZM2as1jZixAh+9atfMXbs2NXaly9fzic/+Ukuu+wyHnvsMRoaGth0000B+MAHPsADDzzQ5rkaGxv5yU9+wmOPPcaMGTP47Gc/y4oVKzbsBUnqEQwh62cIUFMIAT4PNIWQzDw0MxfX2KckqYcZO3Ys/fv3X61t1113Zeedd15j35tvvplRo0axxx57APDmN7+ZXr16AbDffvsxaNCgNs919913c+SRR7LZZpux0047MWzYsLUGF0lqSWkhpJg9+ENEfC8iHouImyNii4hoiIh9in0GRERjsTw+Iq6LiFsiojEiToqIL0bEQxFxX0T0b+NcDRHx7YiYHRGPRsS+Rfu5EXFa1X6PFuNqcWzFPsMi4rcR8XBEPBgRbwUmAwcU/X+hGOt3q/r934ioL5b/OyJmFv2eV7SdAmwP3BYRtxVtjRExoFj+YjG2RyPi823Vb1V/EfF4RDwSET/ZEH9ekqSNz5/+9CcigkMOOYS99tqLCy+8cJ2Of+655xg8eHDTel1dHQsWLNjQw5TUA/Qu+XzDgaMy8zMR8TPgI2vZfwSwJ7A58ARwembuGRHfAo4BLmnj2C0zc3REjAV+UPS1rmO7GvgRMDkzr42IzakEtzOA0zLz/VAJTG30e2ZmvhARvYBbI2JUZn4nIr4I/EtmPle9c0TsDRwHvAMI4P6IuB1Y1MYYzwB2ysxlrd3SFREnACcADBiwDeeMXL6Wcqg1224Bk6zferN+tbF+tdmY6tfQ0NC0/PTTT7N06dLV2gAWL17MrFmzWLJkCQBz587lt7/9LZdddhmbbbYZkyZNolevXuy9995Nx6xYsWKNflZ5/fXX+cMf/tC0feHChTz22GMMGDBgQ17aRmvJkiWt1lZrZ/1q09XqV3YImZ+Zs4vlWVRua2rLbZn5MvByRLwI/KZonwOMWsux1wBk5h0RsVU7nrdYY2wR8UZgh8y8tujrNYCIWEtXq/lYEQB6A4OA3YBH2tj/XcC1mbm0ONevgAOAX7c0xmL5EeBHEXEdcF1LnWbmVGAqwI5Dh+XFc8r+o994TBq5HOu3/qxfbaxfbTam+jUeXf/P5cZG+vTpQ319/Wr7bL311uy9997ss88+QCWsvPLKKxx++OEA/P73v2flypWrHderV681+lnlRz/6EX379m3afsEFF3DwwQczZsyYDXVZG7WGhoZWa6u1s3616Wr1K/uZkGVVyyuovDFfXjWOzdvYf2XV+krWHqCyhfXqczU/X0tja68W+42InYDTgAMzcxRwPWte47pobYyHAVOAvYDfR8TG8T+sJGmDOuSQQ5gzZw6vvPIKy5cv5/bbb2e33XZr9/HvfOc7+clPfsKyZcuYP38+8+bNY9999+3AEUvaWHWFB9MbgVXzwEdswH4/DhAR7wJezMwXi3PtVbTvBezUVgfFLMxTEfHB4pjNik+0ehl4Y9WujcDoiNgkIgYDq/5F3gpYCrwYEdsC76s6pnkfq9wJfDAitoyIPsCHirYWRcQmwODMvA04HegHtP35ipKkjcZRRx3FmDFjmDt3LnV1dVx++eVce+211NXVce+993LYYYdxyCGHAPCmN72JL37xi7z97W9n9OjR7LXXXhx22GEAfPnLX6auro5XXnmFuro6zj33XAB+/etfc8455wCw00478bGPfYzddtuN9773vUyZMqXpwXZJWhdd4SfmFwE/K25Zun4D9vtaRDwEbApMKNp+CRwTEY8B9wN/akc/nwL+JyLOB14HPkrl9qcVEfEwcAWVZ1PmA48DfwAeBMjMh4sx/BF4Eri7qt+pwIyI+Ftm/suqxsx8MCKuAFZ93Mj3M/OhiBjSyvh6AVdHRD8qz5B8x0/ZkqSe45prrmmx/UMf+lCL7Z/85Cf55Cc/uUb7hRde2OKD6uPGjWPcuHFN62eeeSZnnnnmeo5WkipKCyGZ2UjVw+GZeVHV5urnO84qtl9B5Q3+qv2HVC2vtq0VV2fm55uN4VXg4Fb2b3FsmTkPeE8L+zdvO7qlTjNzfCvtlwKXVq0PqVr+JvDNZvs3tjZGKs+RSJIkSd1CV7gdS5IkSVIP0hVux1pvETEF2L9Z87czs74ThiNJkiSpHbp1CMnMz3X2GCRJkiStG2/HkiRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVS9O3sAKt8Wm/Zi7uTDOnsY3VZDQwONR9d39jC6LetXG+tXG+snSV2DMyGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUvTt7ACrfq6+vYMgZ13f2MLqtSSOXM976rTfrVxvrV5tJI5dT39mDkCQ5EyJJkiSpXIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJ6nEmTJjAwIEDGTFiRFPbCy+8wEEHHcTw4cM56KCDWLRoEQCZySmnnMKwYcMYNWoUDz744Gp9vfTSS9TV1XHSSSe1eK7W+pWknswQIknqccaPH8+MGTNWa5s8eTIHHngg8+bN48ADD2Ty5MkA3HjjjcybN4958+YxdepUTjzxxNWOO/vssxk7dmyr52qtX0nqyQwh6ygilnT2GCRJtRk7diz9+/dfrW369Okce+yxABx77LFcd911Te3HHHMMEcF+++3H4sWLWbhwIQCzZs3imWee4eCDD271XK31K0k9mSGkDRHRu7PHsD4ioldnj0GSuptnnnmGQYMGAbDddtvxzDPPALBgwQIGDx7ctF9dXR0LFixg5cqVTJo0iYsuumi9+pWknqzUN9kRMQS4EbgLeCewADi8aDstM2dGxABgZmYOiYjxwAeBPsBw4CLgDcCngGXAoZn5QgvnGQjcmJl7R8QewGzgLZn514j4MzASGAj8ABgA/B04rth+BfAasCdwd0RcCvwY6AtMrzrHIOCnwFZU6nhiZt7ZynUvycy+xfIRwPszc3zVufYp+vliZv5vcd0fAvoBOwBXZ+Z5xfGfBE4p6nA/8NnMXFHM0PwP8K/A54oaV4/hBOAEgAEDtuGckctbGqraYdstYJL1W2/WrzbWrzbbbgENDQ0APP300yxdurRpffny5U3LACtWrKChoYHnn3+ehx56iOXLK3VftGgRs2bN4qqrrmLnnXfmiSee4I9//CMLFixY7fhVWuu3O1qyZEm3HXtXYP1qY/1q09Xq1xk/6R8OHJWZn4mInwEfWcv+I6gEgs2BJ4DTM3PPiPgWcAxwSfMDMvPZiNg8IrYCDgBmAgdExF3As5n5ShEupmXmtIiYAHyHSuABqAPeWby5/zXw35l5ZUR8ruo0nwBuysyvFjMPW65PMYAhwL7AW4HbImJY0b5vce2vAL+PiOuBpcDHgf0z8/WI+C/gaOBKKkHt/syc1NJJMnMqMBVgx6HD8uI53XKSp0uYNHI51m/9Wb/aWL/aTBq5nI/V1wPQ2NhInz59qC/Wd9hhB3beeWcGDRrEwoUL2X777amvr2fUqFEMGDCgab+lS5cybtw47rjjDu68805uuukmlixZwj/+8Q923nnnNZ75aK3f7qihoaHbjr0rsH61sX616Wr164zbseZn5uxieRaVN+FtuS0zX87MvwMvAr8p2ues5dh7gP2BscDXiu8HAKtmK8ZQmeEAuAp4V9WxP8/MFcXy/sA1Vfut8nvguIg4FxiZmS+v5Tpa87PMXJmZ84D/A3Yp2m/JzOcz81XgV8X4DgT2phJKZhfrQ4v9VwC/XM8xSFKPN27cOKZNmwbAtGnTOPzww5var7zySjKT++67j379+jFo0CB+9KMf8de//pXGxkYuuugijjnmmBYfOm+tX0nqyTojhCyrWl5BZTZmedVYNm9j/5VV6ytpeybnDiqh4y1UbqPag8ob+RZvmWpmabP1bL5DZt5BJdgsAK6IiGPa6K/6+ObX17zvbKM9qMzejC6+ds7Mc4vtr1UFJ0lSG4466ijGjBnD3Llzqaur4/LLL+eMM87glltuYfjw4fz2t7/ljDPOAODQQw9l6NChDBs2jM985jP813/911r7//SnP83MmTMBWu1XknqyrjKn30jlJ/wPAEdsoD7vBL4K3JGZKyPiBeBQ4N+L7fcAR1KZ3Tia1sPJ3cV+Vxf7ARARbwGeyszvRcRmwF5UbotqyTMRsSswl8qzHtWzJh+NiGnATlRmNeZSuf3soIjoD7xK5TaxCVRuzZoeEd8qbjnrD7wxM//SzppIkoBrrrmmxfZbb711jbaIYMqUKW32N378eMaPH9+0/v3vf79p+c1vfnOL/UpST9ZVPh3rIuDEiHiIyoPiNcvMRiozB3cUTXcBizNz1W+JOpnK7VSPUHnQ/dRWujoV+FxEzKHykPgq9cDDxZg/Dny7jeGcAfwvleCzsNm2v1IJXzcCEzPztaL9ASq3Vz0C/DIzZ2bm48BZwM3FuG8BBrVxXkmSJKnLKXUmpAgGI6rWqz/XcFTV8lnF9iuAK6r2H1K1vNq2Vs43uGr5a1SeDVm1/hfgPS0cM77Z+nwqz480H9s0YFpb56/q4xfAL1rZ/NvMnNhC+1OZ+cEW+voplU/lat7etz1jkSRJkjpbV5kJkSRJktRDdJVnQtZbREyh8glW1b6dmT/shLHcD2zWrPlTmTmnpf2bz7pUtV/BWmZ5JEmSpO6q24eQzPzc2vcqR2a+o7PHIEmSJHV13o4lSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSpV784egMq3xaa9mDv5sM4eRrfV0NBA49H1nT2Mbsv61cb61aahoaGzhyBJwpkQSZIkSSUzhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpDCGSJEmSSmUIkSRJklQqQ4gkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJKZQiRJEmSVCpDiCRJkqRSGUIkSZIklcoQIkmSJKlUhhBJkiRJpTKESJIkSSqVIUSSJElSqQwhkiRJkkplCJEkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEmlMoRIkiRJKpUhRJIkSVKpIjM7ewwqWUS8DMzt7HF0YwOA5zp7EN2Y9auN9auN9auN9auN9auN9atNZ9TvLZm5TUsbepc8EHUNczNzn84eRHcVETOt3/qzfrWxfrWxfrWxfrWxfrWxfrXpavXzdixJkiRJpTKESJIkSSqVIaRnmtrZA+jmrF9trF9trF9trF9trF9trF9trF9tulT9fDBdkiRJUqmcCZEkSZJUKkOIJEmSpFIZQnqQiHhvRMyNiCci4ozOHk9XFBGDI+K2iHg8Ih6LiFOL9nMjYkFEzC6+Dq065t+Lms6NiEM6b/RdQ0Q0RsScok4zi7b+EXFLRMwrvr+paI+I+E5Rv0ciYq/OHX3nioidq15jsyPipYj4vK+/1kXEDyLi2Yh4tKptnV9vEXFssf+8iDi2M66lM7RSv29ExB+LGl0bEVsX7UMi4tWq1+FlVcfsXfy9f6KocXTC5ZSulfqt89/Xnvr/cyv1+2lV7RojYnbR7uuvmTbes3SPfwMz068e8AX0Av4MDAXeADwM7NbZ4+pqX8AgYK9i+Y3An4DdgHOB01rYf7eilpsBOxU17tXZ19HJNWwEBjRruxA4o1g+A/h6sXwocCMQwH7A/Z09/q7yVfydfRp4i6+/Nus0FtgLeLSqbZ1eb0B/4P+K728qlt/U2dfWifU7GOhdLH+9qn5Dqvdr1s8DRU2jqPH7OvvaOrF+6/T3tSf//9xS/Zptvxg4p1j29bfmdbf2nqVb/BvoTEjPsS/wRGb+X2b+A/gJcHgnj6nLycyFmflgsfwy8AdghzYOORz4SWYuy8z5wBNUaq3VHQ5MK5anAR+sar8yK+4Dto6IQZ0wvq7oQODPmfmXNvbp8a+/zLwDeKFZ87q+3g4BbsnMFzJzEXAL8N4OH3wX0FL9MvPmzFxerN4H1LXVR1HDrTLzvqy8o7mSf9Z8o9bK6681rf197bH/P7dVv2I242PANW310cNff629Z+kW/wYaQnqOHYAnq9afou031z1eRAwB9gTuL5pOKqYvf7BqahPr2pIEbo6IWRFxQtG2bWYuLJafBrYtlq1f645k9f98ff2137q+3qxj6yZQ+cnpKjtFxEMRcXtEHFC07UClZqtYv3X7++rrr2UHAM9k5ryqNl9/rWj2nqVb/BtoCJFaEBF9gV8Cn8/Ml4D/Bt4KjAYWUpkiVsvelZl7Ae8DPhcRY6s3Fj+p8rPB2xARbwDGAT8vmnz9rSdfb+svIs4ElgM/KpoWAjtm5p7AF4EfR8RWnTW+Lsy/rxvGUaz+gxhff61o4T1Lk678b6AhpOdYAAyuWq8r2tRMRGxK5S/zjzLzVwCZ+UxmrsjMlcD3+OctL9a1mcxcUHx/FriWSq2eWXWbVfH92WJ369ey9wEPZuYz4OtvPazr6806NhMR44H3A0cXb2IobiN6vlieReU5hrdRqVX1LVs9un7r8ffV118zEdEb+DDw01Vtvv5a1tJ7FrrJv4GGkJ7j98DwiNip+CnrkcCvO3lMXU5xD+rlwB8y85tV7dXPKXwIWPVJHr8GjoyIzSJiJ2A4lQfkeqSI6BMRb1y1TOUB10ep1GnVp20cC0wvln8NHFN8Ysd+wItVU8g92Wo/AfT1t87W9fV2E3BwRLypuHXm4KKtR4qI9wJfBsZl5itV7dtERK9ieSiV19v/FTV8KSL2K/4NPYZ/1rzHWY+/r/7/vKZ/Bf6YmU23Wfn6W1Nr71noLv8GdvST7351nS8qn4rwJyo/PTizs8fTFb+Ad1GZtnwEmF18HQpcBcwp2n8NDKo65syipnPpIZ/I0Ub9hlL5ZJeHgcdWvc6ANwO3AvOA3wL9i/YAphT1mwPs09nX0NlfQB/geaBfVZuvv9brdQ2V2zRep3If8/Hr83qj8uzDE8XXcZ19XZ1cvyeo3B++6t/Ay4p9P1L8vZ4NPAh8oKqffai82f4z8F0gOvvaOrF+6/z3taf+/9xS/Yr2K4CJzfb19bdm/Vp7z9It/g2M4sSSJEmSVApvx5IkSZJUKkOIJEmSpFIZQiRJkiSVyhAiSZIkqVSGEEmSJEml6t3ZA5AkqaNFxAoqH0m5ygczs7GThiNJPZ4f0StJ2uhFxJLM7Fvi+Xpn5vKyzidJ3Y23Y0mSeryIGBQRd0TE7Ih4NCIOKNrfGxEPRsTDEXFr0dY/Iq6LiEci4r6IGFW0nxsRV0XE3cBVxW94/mVE/L742r8TL1GSuhRvx5Ik9QRbRMTsYnl+Zn6o2fZPADdl5lcjohewZURsA3wPGJuZ8yOif7HvecBDmfnBiHgPcCUwuti2G/CuzHw1In4MfCsz74qIHYGbgF077AolqRsxhEiSeoJXM3N0G9t/D/wgIjYFrsvM2RFRD9yRmfMBMvOFYt93AR8p2n4XEW+OiK2Kbb/OzFeL5X8FdouIVefYKiL6ZuaSDXVRktRdGUIkST1eZt4REWOBw4ArIuKbwKL16Gpp1fImwH6Z+dqGGKMkbUx8JkSS1ONFxFuAZzLze8D3gb2A+4CxEbFTsc+q27HuBI4u2uqB5zLzpRa6vRk4ueocozto+JLU7TgTIkkS1ANfiojXgSXAMZn594g4AfhVRGwCPAscBJxL5datR4BXgGNb6fMUYEqxX2/gDmBih16FJHUTfkSvJEmSpFJ5O5YkSZKkUhlCJEmSJJXKECJJkiSpVIYQSZIkSaUyhEiSJEkqlSFEkiRJUqkMIZIkSZJK9f8BUOGAZ829pvgAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"Number of characters, mean word length and number of unique words turn out to be the top 3 variables. Now let us focus on creating some text based features. \n\n**Text Based Features :**\n\nOne of the basic features which we could create is tf-idf values of the words present in the text. So we can start with that one.\n","metadata":{"_uuid":"25225b8d3a5ed46ec90308478b3788760eb5ea1d","_cell_guid":"739ce74e-af56-4fd6-a7b3-fe809f33bc54"}},{"cell_type":"code","source":"### Fit transform the tfidf vectorizer ###\ntfidf_vec = TfidfVectorizer(stop_words='english', ngram_range=(1,3))\nfull_tfidf = tfidf_vec.fit_transform(train_df['text'].values.tolist() + test_df['text'].values.tolist())\ntrain_tfidf = tfidf_vec.transform(train_df['text'].values.tolist())\ntest_tfidf = tfidf_vec.transform(test_df['text'].values.tolist())","metadata":{"_uuid":"41b4430c7e9699bd7f430c471009082bf7449928","_cell_guid":"dd89dcab-b7a2-4b11-9564-ac8558f938e0","execution":{"iopub.status.busy":"2023-01-12T09:34:14.104785Z","iopub.execute_input":"2023-01-12T09:34:14.105953Z","iopub.status.idle":"2023-01-12T09:34:19.797495Z","shell.execute_reply.started":"2023-01-12T09:34:14.105905Z","shell.execute_reply":"2023-01-12T09:34:19.796416Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"markdown","source":"Now that we have got the tfidf vector, here is the tricky part. The tfidf output is a sparse matrix and so if we have to use it with other dense features, we have couple of choices. \n1. We can choose to get the top 'n' features (depending on the system config) from the tfidf vectorizer, convert it into dense format and concat with other features. \n2. Build a model using just the sparse features and then use the predictions as one of the features along with other dense features.\n\nBased on the dataset, one might perform better than the other. Here we can use the second approach since there are some very [good scoring kernels](https://www.kaggle.com/the1owl/python-tell-tale-tutorial) using all the features of tfidf.\n\nAlso it seems that, [Naive Bayes is performing better](https://www.kaggle.com/thomasnelson/spooky-simple-naive-bayes-scores-0-399) in this dataset. So we could build a naive bayes model using tfidf features as it is faster to train.","metadata":{"_uuid":"9c51efb2c3ff436baf8811a298b699b430b812bf","_cell_guid":"fb46183a-43b6-4785-823a-f017a0821b78"}},{"cell_type":"code","source":"def runMNB(train_X, train_y, test_X, test_y, test_X2):\n model = naive_bayes.MultinomialNB()\n model.fit(train_X, train_y)\n pred_test_y = model.predict_proba(test_X)\n pred_test_y2 = model.predict_proba(test_X2)\n return pred_test_y, pred_test_y2, model","metadata":{"_uuid":"924a649ddf02867e7bd1e68d8cab33d3c36bb83e","_cell_guid":"7ce26604-5935-41cd-85ac-33e499a186f1","execution":{"iopub.status.busy":"2023-01-12T09:34:19.798909Z","iopub.execute_input":"2023-01-12T09:34:19.799245Z","iopub.status.idle":"2023-01-12T09:34:19.805701Z","shell.execute_reply.started":"2023-01-12T09:34:19.799215Z","shell.execute_reply":"2023-01-12T09:34:19.804580Z"},"trusted":true},"execution_count":15,"outputs":[]},{"cell_type":"markdown","source":"**Naive Bayes on Word Tfidf Vectorizer:**","metadata":{"_uuid":"f3ef5dd66afde39552eaa8f59929b263b831b190","_cell_guid":"a409c693-6d31-485d-b05f-d98008a1c185"}},{"cell_type":"code","source":"cv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nkf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_tfidf[dev_index], train_tfidf[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runMNB(dev_X, dev_y, val_X, val_y, test_tfidf)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\nprint(\"Mean cv score : \", np.mean(cv_scores))\npred_full_test = pred_full_test / 5.","metadata":{"_uuid":"c0ed8394d5fbcfb833efa2836509583a2ec79de1","_cell_guid":"61063815-507a-45f7-8866-5298da11e62d","execution":{"iopub.status.busy":"2023-01-12T09:34:19.807426Z","iopub.execute_input":"2023-01-12T09:34:19.807867Z","iopub.status.idle":"2023-01-12T09:34:20.219712Z","shell.execute_reply.started":"2023-01-12T09:34:19.807832Z","shell.execute_reply":"2023-01-12T09:34:20.218200Z"},"trusted":true},"execution_count":16,"outputs":[{"name":"stdout","text":"Mean cv score : 0.8422161983612855\n","output_type":"stream"}]},{"cell_type":"markdown","source":"We are getting a mlogloss of 0.844 using just tfidf vectorizer. Much better than the meta features. Let us look at the confusion matrix.","metadata":{"_uuid":"1f6da22535edd4c6faf256836f925bb18d18aba0","_cell_guid":"e40f18ae-9db6-43dd-a010-27a34b1aa33f"}},{"cell_type":"code","source":"### Function to create confusion matrix ###\nimport itertools\nfrom sklearn.metrics import confusion_matrix\n\n### From http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py #\ndef plot_confusion_matrix(cm, classes,\n normalize=False,\n title='Confusion matrix',\n cmap=plt.cm.Blues):\n \"\"\"\n This function prints and plots the confusion matrix.\n Normalization can be applied by setting `normalize=True`.\n \"\"\"\n if normalize:\n cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n #print(\"Normalized confusion matrix\")\n #else:\n # print('Confusion matrix, without normalization')\n\n #print(cm)\n\n plt.imshow(cm, interpolation='nearest', cmap=cmap)\n plt.title(title)\n plt.colorbar()\n tick_marks = np.arange(len(classes))\n plt.xticks(tick_marks, classes, rotation=45)\n plt.yticks(tick_marks, classes)\n\n fmt = '.2f' if normalize else 'd'\n thresh = cm.max() / 2.\n for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n plt.text(j, i, format(cm[i, j], fmt),\n horizontalalignment=\"center\",\n color=\"white\" if cm[i, j] > thresh else \"black\")\n\n plt.tight_layout()\n plt.ylabel('True label')\n plt.xlabel('Predicted label')","metadata":{"_uuid":"2bd4a5be4f492f1eb75836a7fbaed8c4374701cf","_cell_guid":"586f6708-8cc1-4422-90dd-6e4bef3cf2c0","_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-12T09:34:20.222129Z","iopub.execute_input":"2023-01-12T09:34:20.222594Z","iopub.status.idle":"2023-01-12T09:34:20.236390Z","shell.execute_reply.started":"2023-01-12T09:34:20.222548Z","shell.execute_reply":"2023-01-12T09:34:20.235017Z"},"trusted":true},"execution_count":17,"outputs":[]},{"cell_type":"code","source":"cnf_matrix = confusion_matrix(val_y, np.argmax(pred_val_y,axis=1))\nnp.set_printoptions(precision=2)\n\n# Plot non-normalized confusion matrix\nplt.figure(figsize=(8,8))\nplot_confusion_matrix(cnf_matrix, classes=['EAP', 'HPL', 'MWS'],\n title='Confusion matrix, without normalization')\nplt.show()\n","metadata":{"_uuid":"b0792565a816242c8e29a242264c5e77ec90a7eb","_cell_guid":"800547fa-c070-4bdf-ad1a-1f97e7db3fdf","_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-12T09:34:20.238719Z","iopub.execute_input":"2023-01-12T09:34:20.239653Z","iopub.status.idle":"2023-01-12T09:34:20.579139Z","shell.execute_reply.started":"2023-01-12T09:34:20.239606Z","shell.execute_reply":"2023-01-12T09:34:20.577838Z"},"trusted":true},"execution_count":18,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 576x576 with 2 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAjQAAAI4CAYAAAB9SXN4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAA8hElEQVR4nO3dd5xcVf3/8dcnoUakhBAITRAwgAgKCEgvSkfQL9I7iCgqUlRAFATlhxVUEER6753Qi/SSIB3pvSchtNASPr8/7t1kCMlms2R35uy+njzmkZlz78w9OzthPnmfc8+NzESSJKlkfZrdAUmSpM/KgkaSJBXPgkaSJBXPgkaSJBXPgkaSJBVvmmZ3QJIkdU7fmb+QOea9bjtevvf6VZm5brcdcApY0EiSVKgc8x7TD96s2473/r1HDei2g00hh5wkSVLxTGgkSSpWQJhNgAmNJEnqAUxoJEkqVQARze5FSzChkSRJxTOhkSSpZM6hAUxoJElSD2BCI0lSyZxDA5jQSJKkHsCERpKkYrkOTRvfBUmSVDwTGkmSSuYcGsCERpIk9QAWNJIkqXgOOUmSVKrAScE13wVJklQ8ExpJkooVTgqumdBIkqTimdBIklQy59AAJjSSJKkHMKGRJKlkzqEBTGgkSVIPYEIjSVKxvDhlG98FSZJUPBMaSZJKFTiHpmZCI0mSimdBI0mSiueQkyRJJXNSMGBCI0mSegATGkmSiuVp2218FyRJUvFMaCRJKlkfT9sGExpJktQDmNBIklSqwDk0Nd8FSZJUPBMaSZJK5qUPABMaSZLUA5jQSJJULNehaeO7IEmSimdBI0mSiueQkyRJJXNSMGBCI0mSegATGkmSSuakYMCERpIkTSURcUJEvBYRD05k294RkRExoH4cEfH3iHgiIu6PiKUb9t0+Ih6vb9t35NgWNJIklSqie2+TdxKw7qe7GfMBawPPNTSvByxS33YFjq737Q8cCCwPLAccGBGzTe7AFjSSJGmqyMybgJET2XQ48AsgG9o2Bk7Jyh3ArBExCFgHuCYzR2bmG8A1TKRImpBzaCRJKln3zqEZEBFDGx4fm5nHtveEiNgYeDEz74tPpjzzAM83PH6hbptUe7ssaCRJUkcNz8xlO7pzRPQD9qcabupSDjlJklSy1ppDM6GFgAWB+yLiGWBe4J6ImAt4EZivYd9567ZJtbfLgkaSJHWJzHwgMwdm5gKZuQDV8NHSmfkKcAmwXX220wrAm5n5MnAVsHZEzFZPBl67bmuXQ06SJBWrtS5OGRFnAqtTzbV5ATgwM4+fxO5DgPWBJ4DRwI4AmTkyIg4B7q73OzgzJzbR+BNa512QahExY0RcGhFvRsS5n+F1to6Iq6dm35olIlaJiEdb5XgRsUC9noT/KGow4fsSEVd0dA2NKTzOQxGx+tR+XemzyswtM3NQZk6bmfNOWMzUSc3w+n5m5u6ZuVBmfiUzhzbsd0JmLlzfTuzIsS1o1GkRsVVEDI2IdyLi5fp/3itPhZfeFJgTmD0zv9fZF8nM0zOzyyeifVb1F+DC7e2TmTdn5uDu6tOEx4uIZyLim91x7Ig4KSJ+1x3H6mqZuV5mnvxZXmNi70dmfjkzb/xMnZN6GAsadUpE7AUcARxKVXzMD/yTal2Bz+oLwGOZOWYqvFbxTEG6ju+teoTWnhTcbSxoNMUiYhbgYGD3zLwgM9/NzI8y89LM/Hm9z/QRcUREvFTfjoiI6ettq0fEC/Uy2K/V6c6O9bbfAr8BNq+Tn50j4qCIOK3h+BPG+jtExFMR8XZEPB0RWze039LwvBUj4u56KOvuiFixYduNEXFIRNxav87VbctzT+Tnb+v/Lxr6v0lErB8Rj0XEyIjYv2H/5SLi9ogYVe97ZERMV2+7qd7tvvrn3bzh9X8ZEa8AJ7a11c9ZqD7G0vXjuSPi9Y4MQUTEyRGxd31/nvp93H2C1+0zwfFOpSpYL637+IuGl9w6Ip6LiOER8auG47T3+//E76Vuy4hYOCJ2BbYGflEf69JJ/BwZEbtFtSz6qIg4KqL6v23d/wMi4tn693NK/Zlt/OzsHBHPAdfX/bk1Ig6vX+up+rOyQ0Q8X7/G9g3H3iAi/hsRb9XbD2rn/b4xInap77f9jttu2fY7i4hzI+KV+rN5U0R8uW6f6PsRDYnZZN7rSf5dk3oaCxp1xjeAGYAL29nnV8AKwFeBpaiWrz6gYftcwCxUiyXtDBwVEbNl5oFUqc/ZmTlTO5PJAIiIzwF/B9bLzM8DKwL3TmS//sDl9b6zA38FLo+I2Rt224pqUtpAYDpgn3YOPRfVezAPVQH2b2AbYBlgFeDXEbFgve9YYE9gANV7txbwI4DMXLXeZ6n65z274fX7U6VVuzYeODOfBH4JnBbVGg8nAid3cAjiP1QT9gBWA54CVm14fHNmfjzB8balWq58o7qPf2zYvDIwuP6ZfhMRi9Xtk/v9T1S9QNfpwB/rY23Uzu4bAl8HlgQ2o1pdFGCH+rYG8EVgJuDICZ67GrBYw3OWB+6n+mycAZxVv/bCVL/XIyNipnrfd4HtgFmBDYAfRsQmHfjZ2n7HMwF7AY8C99Sbr6Ba/n1g3Xb6FLwfnfq7Nrn+qhBBNSm4u24trLV7p1Y1O9XiSu0NCW1NNTP9tcx8HfgtsG3D9o/q7R9l5hDgHaovxs74GFgiImbMzJcz86GJ7LMB8HhmnpqZYzLzTOB/QOMXxImZ+VhmvgecQ/UFMSkfAb/PzI+ovvwGAH/LzLfr4z9M9eVCZg7LzDvq4z4D/IvqC3VyP9OBmflB3Z9PyMx/U50ZcCcwiOpLrSP+A6wcEX2oCpk/AivV21art0+J32bme5l5H3Af9c/M5H//U8NhmTkqM58DbmD872tr4K+Z+VRmvgPsB2wRnxxeOqhOFtve26cz88TMHAucTbUGxsH1+3818CFVcUNm3lifivpxZt4PnMnkf5/jRDXP7HfAtzPzrfo1T6g/Ox8ABwFLtaVKHdCdf9eklmVBo84YQXVKXnvzD+YGnm14/GzdNu41JiiIRlP9S3qKZOa7wObAbsDLEXF5RCzagf609alxOe1XpqA/I+ovP4C2L8VXG7a/1/b8iPhSRFxWDym8RZVATXQ4q8Hrmfn+ZPb5N7AE8I/6i3Cy6nTnXaov/1WAy4CXImIwnStoJvWeTe73PzVMybGnoZrr1aZxWXX49O+OzJzU73P5iLihHuZ7k+qzN7nfJ/Vz56MqlrfPzMfqtr4RcVhEPFl/Pp6pd+/Qa9JNf9fUqsKEptbavVOruh34ANiknX1eohouaTN/3dYZ7wL9Gh7P1bgxM6/KzG9RJRX/o/qin1x/2vo02dUnp4Kjqfq1SGbOTLUM+ORm12V7G+vhjyOA44GD6iG1jvoP1Zlk02Xmi/Xj7YHZmMhwXUf6MxHt/f4/8fuMasXQz3Ksjhx7DJ8sWj7LMc6gWhBsvsycBTiGyf8+iYgZgYuAIzLzioZNW1FNpv8m1dDQAm1P6WBfp+bfNalYFjSaYpn5JtW8kaOimgzbLyKmjYj1IqJtfsWZwAERMUdUk2t/A5w2qdecjHuBVSNi/jqG369tQ0TMGREb13NpPqCK0z+eyGsMAb4U1anm00TE5sDiVAlFV/s88BbwTp0e/XCC7a9SzfWYEn8DhmbmLlRzg45p2xDVJOob23nuf4AfA20Tkm+sH9/SkDpNaEr72N7v/z7gyxHx1YiYgWqI5bMca2LH3jMiFqwLv7Y5WVPrrLnPAyMz8/2IWI6qIOmIE4D/TTAHqe31PqBKPvvV/W00ufdjav5dU4k8ywmwoFEnZeZfqCY2HgC8ThXh/5jqX6BQzREYSjXR8gGqiY6dWlskM6+hmtdwPzCMTxYhfep+vER1yfrV+HTBQGaOoJpEujfVF8cvgA2zXuCpi+1D9aX3NlV6dPYE2w8CTq7PsNlsci8W1ZVr12X8z7kXsHTUZ3dRzf+4tZ2X+A/Vl2hbQXML1RfpTZN8Bvw/qi/NURHR3mTpNpP8/ddDLQcD1wKP18dvdDyweH2sizpwrAmdAJxK9fM8DbwP/KQTrzMpPwIOjoi3qYqHczr4vC2A70xwptMqwClUw0QvUs29umOC503u/Zhqf9ekkkXmZ013JbWSiLgXWKsu4iT1YH1m/UJOv9r+k99xKnn/kt2G5RRcbbs7uaiU1MNk5leb3QdJ6m4WNJIklazF57Z0F+fQSJKk4lnQSJKk4jnkJElSqSJafsG77tIjC5qYZsaM6T7f7G6oAF9dbP5md0FSD/Tss88wYvhwJ7d0o55Z0Ez3eaYfPNnlPCRuveMfze6CCjJmrMtcqGNWW2m57juYk4IB59BIkqQeoEcmNJIk9RZhQgOY0EiSpB7AhEaSpEIFJjRtTGgkSVLxTGgkSSpV1DeZ0EiSpPKZ0EiSVKxwDk3NhEaSJBXPgkaSJBXPISdJkgrmkFPFhEaSJBXPhEaSpIKZ0FRMaCRJUvFMaCRJKpgJTcWERpIkFc+ERpKkUnnpg3FMaCRJUvFMaCRJKlR46YNxTGgkSVLxTGgkSSqYCU3FhEaSJBXPgkaSJBXPISdJkgrmkFPFhEaSJBXPhEaSpIKZ0FRMaCRJUvFMaCRJKpWXPhjHhEaSJBXPhEaSpII5h6ZiQiNJkopnQiNJUqG8OOV4JjSSJKl4JjSSJBXMhKZiQiNJkopnQSNJkornkJMkSSVzxAkwoZEkST2ACY0kSaUKJwW3MaGRJEnFM6GRJKlgJjQVExpJklQ8ExpJkgpmQlMxoZEkScUzoZEkqVBenHI8ExpJklQ8ExpJkkpmQAOY0EiSpKkkIk6IiNci4sGGtj9FxP8i4v6IuDAiZm3Ytl9EPBERj0bEOg3t69ZtT0TEvh05tgWNJEmaWk4C1p2g7RpgicxcEngM2A8gIhYHtgC+XD/nnxHRNyL6AkcB6wGLA1vW+7bLISdJkkrVYpc+yMybImKBCdqubnh4B7BpfX9j4KzM/AB4OiKeAJartz2RmU8BRMRZ9b4Pt3dsExpJktRRAyJiaMNt1yl8/k7AFfX9eYDnG7a9ULdNqr1dJjSSJBWsmxOa4Zm5bGeeGBG/AsYAp0/dLlUsaCRJUpeKiB2ADYG1MjPr5heB+Rp2m7duo532SbKgkSSpYK00h2ZiImJd4BfAapk5umHTJcAZEfFXYG5gEeAuqhPRF4mIBakKmS2ArSZ3HAsaSZI0VUTEmcDqVHNtXgAOpDqraXrgmrr4uiMzd8vMhyLiHKrJvmOA3TNzbP06PwauAvoCJ2TmQ5M7tgWNJEkla6GAJjO3nEjz8e3s/3vg9xNpHwIMmZJje5aTJEkqngmNJEkFa/U5NN3FhEaSJBXPhEaSpEJFhAlNzYKmYMccuDXrrboEr498m2W/dygAv/rB+uz03RV5/Y13ADjwyEu46paHWfbLX+DIX1dztSLg98cM4ZIb7p/k66j3+MH3d+LKIZczxxwDGXrvAwBsu9UWPPbYowC8+eYoZpllVu4c+t9mdlMtZOzYsay20nIMmntuzr3gUtZZazXeeedtAF5/7TWWWfbrnHnuhU3upXobC5qCnXrpHRxz9n847pDtPtH+j9Nu4IhTr/tE20NPvsRKW/+RsWM/Zq4BM3Pn2ftx+U0PMnbsx5N8HfUO2263A7v96Md8f8ftx7WdesZZ4+7v+4u9mXnmWZrRNbWoo4/8O18avChvv/0WAFdd959x27bZYlPW3+jbzeqaejHn0BTs1nueZOSboye/I/De+x8xduzHAEw/3bSMX6hxyl5HPc/Kq6xK/9n6T3RbZnL+eeey2eYTOxNTvdGLL7zAVVcOYfsdd/7Utrfeeoub/nMDG260Sfd3rBdrG3bqjlsrM6HpgXbbYlW22nA57nn4Ofb96wWMevs9AL6+xBc45qBtmH9Qf3Y+4ORxBY40KbfecjMDB87Jwoss0uyuqEXs+/M9Ofj3h40bYmp02aUXsdrqazLzzDM3oWfq7bo9oYmIsRFxb8Nt34ZtAyLio4jYbYLnPBMRD0TE/RFxdUTM1d39LsW/z72ZxTc6iOW3OIxXhr/FYXt9d9y2ux98lmU2/T0rb/NHfr7T2kw/nfWs2nfO2Wey2eZbNLsbahFXDLmMAQMH8rWll5no9vPOOYtNN/Pz0t1MaCrNGHJ6LzO/2nA7rGHb94A7gInl22tk5pLAUGD/7uhoiV4b+TYff5xkJidccCvLLvGFT+3z6NOv8s7oD/jywnM3oYcqxZgxY7jkogv5v+9t3uyuqEXcefttXHHZpSwx+IvsuN1W3HTjDeyy47YAjBg+nGFD72ad9TZoci/VW7XaHJotgb2BeSJi3knscxOwcPd1qSxzDRgf9W685lI8/OTLAHxh7tnp27f6dc8/aDYGLzgXz740oil9VBmuv+5avjR4Ueadd1J/FdXbHHTIofzvyed48NGnOPGUM1h19TU47sRTAbjowvNYd70NmGGGGZrcy14ouvHWwpox5jBjRNzb8Pj/ZebZETEfMCgz76ovVrU58JeJPH9D4IEJGyNiV2BXAKadaap3uhWd/P92YJVlFmHArDPxxJWHcMgxQ1h1mUVYcvC8ZCbPvjySn/zuTABW/NoX2WfHtflozFg+/jjZ49CzGTHq3Um+zskX3d7MH03daPtttuKmm25kxPDhLLzgfBzwm4PYYcedOe+cs/mew03qoPPPPYc99/lFs7uhXiwaz3bplgNGvJOZn6o4ImIfYLbM/FVELEl1dc1l623PAG8DY4H7gZ9m5qhJHaNPv4E5/eDNuqL76mFG3vWPZndBBRkztnv/f6lyrbbSctwzbGiXZxrTz7lIzrP137r6MOM8ffgGw9q+m1tNK80K3RKYKyK2rh/PHRGLZObj9eM1MnN4k/omSZJaWEsUNBHxJWCmzJynoe23VEXOwU3rmCRJrSy8OGWbVphDcyXwHjDhOtnnA2djQSNJkiaj2wuazOzbwf3uBxar7y/QlX2SJKlEQXV9PrXeaduSJElTrCXm0EiSpM5o/RV8u4sJjSRJKp4FjSRJKp5DTpIkFcwRp4oJjSRJKp4JjSRJBXNScMWERpIkFc+ERpKkUoVzaNqY0EiSpOKZ0EiSVKgA+vQxogETGkmS1AOY0EiSVDDn0FRMaCRJUvFMaCRJKpjr0FRMaCRJUvEsaCRJUvEccpIkqVQurDeOCY0kSSqeCY0kSYUKnBTcxoRGkiQVz4RGkqRihQlNzYRGkiQVz4RGkqSCGdBUTGgkSVLxTGgkSSqYc2gqJjSSJKl4JjSSJJXKlYLHMaGRJEnFs6CRJEnFc8hJkqRCeemD8UxoJElS8UxoJEkqmAFNxYRGkiQVz4RGkqSCOYemYkIjSZKKZ0IjSVLBDGgqJjSSJKl4JjSSJJUqnEPTxoRGkiQVz4RGkqRCVSsFN7sXrcGERpIkFc+CRpIkFc8hJ0mSihVOCq6Z0EiSpOKZ0EiSVDADmooJjSRJKp4FjSRJBYuIbrt1oC8nRMRrEfFgQ1v/iLgmIh6v/5ytbo+I+HtEPBER90fE0g3P2b7e//GI2L4j74MFjSRJmlpOAtadoG1f4LrMXAS4rn4MsB6wSH3bFTgaqgIIOBBYHlgOOLCtCGqPBY0kSaWKag5Nd90mJzNvAkZO0LwxcHJ9/2Rgk4b2U7JyBzBrRAwC1gGuycyRmfkGcA2fLpI+xUnBkiSpowZExNCGx8dm5rGTec6cmflyff8VYM76/jzA8w37vVC3Taq9XRY0kiQVqrr0Qbee5jQ8M5ft7JMzMyMip2aH2jjkJEmSutKr9VAS9Z+v1e0vAvM17Ddv3Tap9nZZ0EiSVLBWOstpEi4B2s5U2h64uKF9u/pspxWAN+uhqauAtSNitnoy8Np1W7sccpIkSVNFRJwJrE411+YFqrOVDgPOiYidgWeBzerdhwDrA08Ao4EdATJzZEQcAtxd73dwZk440fhTLGgkSdJUkZlbTmLTWhPZN4HdJ/E6JwAnTMmxLWgkSSqYlz6oOIdGkiQVz4RGkqSCdfNp2y3LhEaSJBXPhEaSpFJ18JIEvYEJjSRJKp4JjSRJhQo+04J3PUqPLGgWWGAuDj5u38nvqF7vqkdeaXYXVJBvDp5z8jtJaooeWdBIktRbGNBUnEMjSZKKZ0IjSVLB+hjRACY0kiSpB7CgkSRJxXPISZKkgjniVDGhkSRJxTOhkSSpUBFenLKNCY0kSSqeCY0kSQXrY0ADmNBIkqQewIRGkqSCOYemYkIjSZKKZ0IjSVLBDGgqJjSSJKl4JjSSJBUqgMCIBkxoJElSD2BBI0mSiueQkyRJBXNhvYoJjSRJKp4JjSRJpYpwYb2aCY0kSSqeCY0kSQUzoKmY0EiSpOKZ0EiSVKgA+hjRACY0kiSpBzChkSSpYAY0FRMaSZJUPBMaSZIK5jo0FRMaSZJUPAsaSZJUPIecJEkqVISTgtuY0EiSpOKZ0EiSVDAX1quY0EiSpOKZ0EiSVDDzmYoJjSRJKp4JjSRJBXNhvYoJjSRJKp4JjSRJhQqgjwENYEIjSZJ6ABMaSZJKFeEcmpoJjSRJKp4FjSRJKp5DTpIkFcwRp4oJjSRJKp4JjSRJBXNScMWERpIkFW+SCU1E/APISW3PzJ92SY8kSVKHuLDeeO0NOQ3ttl5IkiR9BpMsaDLz5MbHEdEvM0d3fZckSVJHOYemMtk5NBHxjYh4GPhf/XipiPhnl/dMkiSpgzoyKfgIYB1gBEBm3ges2oV9kiRJHRTdeGtlHTrLKTOfn6BpbBf0RZIkqVM6sg7N8xGxIpARMS2wB/BI13ZLkiRNTgT0cQ4N0LGEZjdgd2Ae4CXgq/VjSZKkljDZhCYzhwNbd0NfJElS4SJiT2AXqrXsHgB2BAYBZwGzA8OAbTPzw4iYHjgFWIZqru7mmflMZ47bkbOcvhgRl0bE6xHxWkRcHBFf7MzBJEnS1BXRfbfJ9yXmAX4KLJuZSwB9gS2APwCHZ+bCwBvAzvVTdgbeqNsPr/frlI4MOZ0BnENVXc0NnAuc2dkDSpKkHm0aYMaImAboB7wMrAmcV28/Gdikvr9x/Zh6+1rRyYV1OlLQ9MvMUzNzTH07DZihMweTJElTV0R02w0YEBFDG267NvYlM18E/gw8R1XIvEk1xDQqM8fUu71ANS+X+s/n6+eOqfefvTPvQ3vXcupf370iIvalGvtKYHNgSGcOJkmSijY8M5ed1MaImI0qdVkQGEU1qrNud3SsvUnBw6gKmLbo5wcN2xLYr6s6JUmSOqbFztr+JvB0Zr4OEBEXACsBs0bENHUKMy/wYr3/i8B8wAv1ENUs1Av5Tqn2ruW0YGdeUM2z50bfYIZ+n6NP37707duXg08dwgX/+is3XnQGn5+tSvC+96Nf8tWV1xz3nOGvvMi+31uT7+y6Jxtsu1uzuq5utut6yzFjv5no07cPfftOw5/PvJK333yDv/xiN1576QUGzj0v+/zpX8w086xkJsf/4dcMu+V6pp9hRn5yyOEstNiSzf4R1ERjx45l1RWXY9Dcc3PehZeOa//5Xntw6skn8sqIt5rYOzXZc8AKEdEPeA9Yi+pi1zcAm1KN9mwPXFzvf0n9+PZ6+/WZmZ05cEcW1iMilgAWp2HuTGae0pkDqmvt/69z+Pys/T/Rts5Wu0yyWDnjrwez5IprdEfX1GIOOe5cZp5t/FD1BSccyVeWW5n/2/knnH/8P7jg+CPZbs8DuOeW63npuaf556W38tgD9/Cv3+3HH0+/vIk9V7P988i/M3jworz19vjC5Z5hQxn1xhtN7FXvFERLLayXmXdGxHnAPcAY4L/AscDlwFkR8bu67fj6KccDp0bEE8BIqjOiOqUjp20fCPyjvq0B/BH4dmcPqNYx9MYrmWOe+Zj3i19qdlfUAu664SrW+PZmAKzx7c2484Yrx7dvtCkRweAll+Hdt99k5OuvNrOraqIXX3iBq64YwvY77jyubezYsRyw3y855NBOn3GrHiQzD8zMRTNziczcNjM/yMynMnO5zFw4M7+XmR/U+75fP1643v5UZ4/bkbOcNqWKjF7JzB2BpajGuNRqIvjD7lvz623W5/oLTh/XfO05J7P/Ft/i37/dm3ffGgXA+6Pf5fKTj+Y739+zSZ1VMwXBb3fbkr23WIerzzsNgFEjh9N/jjkBmG3AQEaNHA7AiNdeYfY55x733NnnnJuRr73S/Z1WS/jlz/fkkEMPo0+f8V8f/zr6KNbfcCPmGjSoiT3rpbpxDZoWCoImqiNDTu9l5scRMSYiZgZeo5rA0ykR8U5mztTweAeqBXh+HBEHAd8HXq/7tn9mXlK3v5OZf+7scXuDXx93Pv0HDuLNkcP5w+5bMfcCC7HWptuyyS57QATnH/0nzjj8EL5/4F+44Ni/su5WuzBDv881u9tqgkNPuojZ5xzEqBHD+e1uWzDPggt/YntEEC1/bV11tyuGXMYccwzka0svw83/uRGAl196iQvPP48rrrm+uZ1Tr9eRgmZoRMwK/JvqzKd3qCbvdJXDM/PPEbEYcHNEDOzCY/Uo/QdW/zqapf8All19XZ586F4WXXqFcdtX/85W/OVnOwDw5IP/5e7rhnDW3w9l9NtvEX2C6aabgW9tvkMTeq7uNvuc1Wdl1tkHsPya6/L4g/9l1v4DGPn6q/SfY05Gvv4qs/Sv5tfMPnAuRrz60rjnjnj1JfoPnKsp/VZz3XHbbQy5/FKuvvIK3v/gfd5+6y2WW/orTDf99Cy1eDV0PXr0aJZa/Evc9/BjTe5t79HJdeh6nMkOOWXmjzJzVGYeA3wL2L4eeupSmfkI1YSiAV19rJ7g/fdG896774y7/8CdNzHfQoMZNXz8XIehN1zJvAsNBuDXx13A4ZfezuGX3s46W+7MRjv+2GKml3h/dMNnZfRo7r39P8y/8KJ8ffW1ueGScwC44ZJzWG6NdQCq9kvPIzN59P5h9Jtp5nFDU+pdfvu7Q3n0yed46LGnOOmUM1h19TV4/pURPPnsSzz02FM89NhT9OvXz2JGTdHewnpLt7ctM+/p5DFnjIh7Gx73pzpta8JjLA98TDX8pMl4a8TrHPHz7wPw8dixfGOdjVlyxTU45td78OxjDxERDBg0Lzv96rAm91TNNmrk6/xhz2pC59gxY1hl/e+w9EprsPCXl+LPP9+N6y46izkGzcM+f/oXAMusshbDbrmOH264YnXa9sGHN7P7kjRRManTvSPihnael5m5ZjvbJ33Ajs+heZtqDs3NHZlDUy+/vCvA7HPNs8wRl93Rme6pl5lp2r7N7oIK8s3BJlPqmFVXXI57hg3t8rGggQsvkZv/6dyuPsw4R3538WHtrRTcTO0trNesxUkO78zk38w8lupcd764+JKdWpRHkiSVqUML60mSpNYTOCm4TUfWoWkVB0TEC223ZndGkiS1jm5PaBrnz9SPTwJOqu8fNInnHARMdJskSb1ZHwMaoGOXPoiI2CYiflM/nj8iluv6rkmSJHVMRxKaf1KdPr0mcDDV2UfnA1/vwn5JkqQOMKGpdKSgWT4zl46I/wJk5hsRMV0X90uSJKnDOlLQfBQRfYEEiIg5qBIbSZLURNVFI41ooGNnOf0duBAYGBG/B24BDu3SXkmSJE2BySY0mXl6RAwD1qI65X2T+jpLkiSpyZxDU5lsQRMR8wOjgUsb2zLzua7smCRJUkd1ZA7N5VTzZwKYAVgQeBT4chf2S5IkqcM6MuT0lcbH9VW4f9RlPZIkSR3mnODKFF/6IDPvAZbvgr5IkiR1Skfm0OzV8LAPsDTwUpf1SJIkdUgAfYxogI7Nofl8w/0xVHNqzu+a7kiSJE25dguaekG9z2fmPt3UH0mSNAWmeO5IDzXJ9yEipsnMscBK3dgfSZKkKdZeQnMX1XyZeyPiEuBc4N22jZl5QRf3TZIkTYZTaCodmUMzAzCC6mrbbevRJGBBI0mSWkJ7Bc3A+gynBxlfyLTJLu2VJEmarIjwLKdaewVNX2AmPlnItLGgkSRJLaO9gublzDy423oiSZKmmAFNpb2zvXyLJElSEdoraNbqtl5IkiR9BpMccsrMkd3ZEUmSNOX6OJ4CuMCgJEnqATqyDo0kSWpBXpxyPBMaSZJUPBMaSZIKZkBTMaGRJEnFM6GRJKlU4VlObUxoJElS8UxoJEkqWLiwP2BCI0mSegATGkmSClWtQ9PsXrQGExpJklQ8CxpJklQ8h5wkSSqYQ04VExpJklQ8ExpJkgoWXvsAMKGRJEk9gAmNJEmF8rTt8UxoJElS8UxoJEkqVYBTaComNJIkqXgmNJIkFayPEQ1gQiNJknoAExpJkgrlWU7jmdBIkqTiWdBIkqTiOeQkSVLBnBNcMaGRJEnFM6GRJKlYQR+MaMCERpIk9QAmNJIkFSpwDk0bExpJklQ8ExpJkkoVLqzXxoRGkiRNNRExa0ScFxH/i4hHIuIbEdE/Iq6JiMfrP2er942I+HtEPBER90fE0p09rgWNJEkF6xPRbbcO+htwZWYuCiwFPALsC1yXmYsA19WPAdYDFqlvuwJHd/p96OwTJUmSGkXELMCqwPEAmflhZo4CNgZOrnc7Gdikvr8xcEpW7gBmjYhBnTm2BY0kSYVqO8upu27AgIgY2nDbdYIuLQi8DpwYEf+NiOMi4nPAnJn5cr3PK8Cc9f15gOcbnv9C3TbFnBQsSZI6anhmLtvO9mmApYGfZOadEfE3xg8vAZCZGRE5tTtmQiNJkqaWF4AXMvPO+vF5VAXOq21DSfWfr9XbXwTma3j+vHXbFLOgkSSpYK00KTgzXwGej4jBddNawMPAJcD2ddv2wMX1/UuA7eqznVYA3mwYmpoiDjlJkqSp6SfA6RExHfAUsCNVgHJOROwMPAtsVu87BFgfeAIYXe/bKRY0kiQVrNUufZCZ9wITm2ez1kT2TWD3qXFch5wkSVLxemRC0yeCmafrkT+aprK1F5ur2V1QQQ666tFmd0GFePmt97vlOIHJRBvfB0mSVDxjDEmSShUQrTaJpklMaCRJUvFMaCRJKpj5TMWERpIkFc+ERpKkQgV0aAXf3sCERpIkFc+CRpIkFc8hJ0mSCuaAU8WERpIkFc+ERpKkgjknuGJCI0mSimdCI0lSscJLH9RMaCRJUvFMaCRJKlRgMtHG90GSJBXPhEaSpII5h6ZiQiNJkopnQiNJUsHMZyomNJIkqXgWNJIkqXgOOUmSVKpwUnAbExpJklQ8ExpJkgrlwnrj+T5IkqTimdBIklQw59BUTGgkSVLxTGgkSSqY+UzFhEaSJBXPhEaSpII5haZiQiNJkopnQiNJUqGqdWiMaMCERpIk9QAWNJIkqXgOOUmSVDAnBVdMaCRJUvFMaCRJKlYQTgoGTGgkSVIPYEIjSVLBnENTMaGRJEnFM6GRJKlQLqw3ngmNJEkqngmNJEmlCufQtDGhkSRJxTOhkSSpYCY0FRMaSZJUPAsaSZJUPIecJEkqmJc+qJjQSJKk4pnQSJJUqAD6GNAAJjSSJKkHMKGRJKlgzqGpmNBIkqTimdBIklQwF9armNBIkqTimdBIklQw59BUTGgkSVLxTGgkSSqU69CMZ0IjSZKKZ0EjSZKK55CTJEnFCicF10xoJElS8UxoepixY8ey95brMvvAufj1kady3x03c9JfDyYzmaFfP/Y45G8Mmn9Bjvvjb3jw7tsA+OD993hz5HDOuPXRJvdezfCDXXbiiiGXMcfAgQy790EAzj/vXH5/yEH875FHuPm2u1hm2WWb3Es1y4gXnuKSP+w17vGoV55n5W1+yjsjXuWJu26g7zTTMutc87P+zw5lhplmHrffW6+9xHE/2pCVttqd5b+7czO63jtE6y2sFxF9gaHAi5m5YUQsCJwFzA4MA7bNzA8jYnrgFGAZYASweWY+09njmtD0MJed/m/mW3CRcY+P+f2+7HXYURxx7rWsut53OefYIwDY5RcHc8S513LEudeywZY7scJa6zepx2q2bbffgYsvu/ITbV/+8hKcdc4FrLzKqk3qlVrF7PN+kR3/cRE7/uMitj/ifKadfka+9I1vssBXV2Tnoy5lpyMvof88C3DHucd+4nnXHXcYX1xmlSb1Wk22B/BIw+M/AIdn5sLAG0Bbhbsz8Ebdfni9X6dZ0PQgw195iaE3Xce3vrtVQ2sw+p13ABj9zlv0n2POTz3vpisuYtX1NumeTqrlrLzKqvTv3/8TbYsuthhfGjy4ST1Sq3r2vtuZddB8zDJwHhZcemX69K1C/rkHL8Xbw18Zt99jt1/LrHPNy4D5F25WV3uV6MbbZPsSMS+wAXBc/TiANYHz6l1OBjap729cP6bevla9f6c45NSDHPfH37D9Xgfw3rvvjmv78UF/5pDdt2G66Wdgxplm4k+nXf6J57z20vO89uJzfGW5lbu7u5IK88hNQ1hs1Q0+1X7/Neez2KpVyvvhe+9y53n/ZvPfncBdF5zQ3V1U1xsQEUMbHh+bmY3x3BHAL4DP149nB0Zl5pj68QvAPPX9eYDnATJzTES8We8/vDMd67KEJiIyIk5reDxNRLweEZdFZXhEzFZvG1Tvv3LD/q9HxOwRMTgiboyIeyPikYg4dmLH6+3u/s81zNp/AAsvvtQn2i857Vh+fdRpnHDtPay18RYc/6eDPrH95isvZsVvbUjfvn27sbeSSjP2ow954q7rWXTldT/RftvZx9Cn7zQsvvpGANxyxpEsu8kOTDfj55rRzV6nWlgvuu0GDM/MZRtu476TI2JD4LXMHNaM96IrE5p3gSUiYsbMfA/4FvAiQGZmRNwBfAMYAqwI/Lf+85aIGAyMyMwREXEG1djbxQAR8ZUu7HOxHrn3Lu668WqG3XIdH37wAaPffZuDd9+GF59+gsFLLg3AKut+m4N+uNUnnnfzlRfzg/0PbUaXJRXkqWE3M+dCi/O52QaMa3vg2gt48q4b2OL3J9E2UvDyo/fz6K1XceOJf+KDd98mog/TTDs9y2y0TbO6ru6zEvDtiFgfmAGYGfgbMGtETFOnNPNS1wL1n/MBL0TENMAsVJODO6Wrh5yGUI2lnQdsCZwJtM0Su42qgGkraA4HvltvWxG4tb4/iCqiAiAzH+jiPhdpuz1+xXZ7/AqAB+6+jYtOPpr9jziR7ddckhefeZJ5FliIe2+/6RMThl94+nHefWsUiy7lGSyS2vfwfy7/xHDTU8Nu5s7zj2erw05l2hlmHNe+9R9PH3f/ltP/wbQz9rOY6WKtcpJTZu4H7AcQEasD+2Tm1hFxLrAp1ZlO2wMX10+5pH58e739+szMzh6/qycFnwVsEREzAEsCdzZsu5WqcAFYDriQqlKjbr+tvn84cH1EXBERe0bErBM7UETsGhFDI2LoW290usDrUfpOMw27H/hn/rDXLuyx6VrceNl57LD3b8Ztv/mKi1l53U3G/ctKvdN222zJ6qt8g8cefZSFFpiXk044nosvupCFFpiXO++4ne9uvAEbrb9Os7upJvrw/dE8c++tDF5x7XFt1xxzCB++9y5nH7ATJ/5kE6468sAm9lAt7pfAXhHxBNUcmePr9uOB2ev2vYB9P8tB4jMUQ+2/cMQ7mTlTPXnoKGAR4Gqqim3DiOhHFTfNC1yXmStExDnA/lTV2/9l5v/q15obWJdqRvRgYKnM/GBSx174y0vlX8+6qkt+LvUsay82V7O7oIIcdJVrNaljTv7Z//Hy4w92+b8WF/vK1/LEi27o6sOM842FZxuWmS0Z63fHaduXAH+mGm4aJzNHA48DOwH31M13AOsDA4FHG/Z9KTNPyMyNgTHAEt3Qb0mSVIjuKGhOAH47ibkvtwE/oxo/o/5zD+COtnG0iFg3Iqat789FFVe9+OmXkiRJvVWXFzSZ+UJm/n0Sm28Fvsj4guYeqiGo2xr2WRt4MCLuA64Cfp6ZryBJkurLU3bPf62sy85yysyZJtJ2I3Bjw+NzaZigXc+LmX6C5+xFNVlIkiRpolwpWJKkgnmiasVrOUmSpOKZ0EiSVDADmooJjSRJKp4JjSRJJTOiAUxoJElSD2BCI0lSoQJafn2Y7mJCI0mSimdCI0lSqcJ1aNqY0EiSpOJZ0EiSpOI55CRJUsEccaqY0EiSpOKZ0EiSVDIjGsCERpIk9QAmNJIkFStcWK9mQiNJkopnQiNJUsFcWK9iQiNJkopnQiNJUqECT3JqY0IjSZKKZ0IjSVLJjGgAExpJktQDWNBIkqTiOeQkSVLBXFivYkIjSZKKZ0IjSVLBXFivYkIjSZKKZ0IjSVLBDGgqJjSSJKl4JjSSJJXKax+MY0IjSZKKZ0IjSVLBXIemYkIjSZKKZ0IjSVKhAtehaWNCI0mSimdBI0mSiueQkyRJBXPEqWJCI0mSimdCI0lSyYxoABMaSZLUA5jQSJJUMBfWq5jQSJKk4pnQSJJUMBfWq5jQSJKk4pnQSJJUMAOaigmNJEkqngmNJEklM6IBTGgkSVIPYEEjSZKK55CTJEmFClxYr40JjSRJKp4JjSRJpQoX1mtjQiNJkopnQiNJUsEMaComNJIkqXgmNJIklcyIBjChkSRJPYAJjSRJxQrXoamZ0EiSpOJZ0EiSVLCI7rtNvi8xX0TcEBEPR8RDEbFH3d4/Iq6JiMfrP2er2yMi/h4RT0TE/RGxdGffBwsaSZI0tYwB9s7MxYEVgN0jYnFgX+C6zFwEuK5+DLAesEh92xU4urMHtqCRJElTRWa+nJn31PffBh4B5gE2Bk6udzsZ2KS+vzFwSlbuAGaNiEGdObaTgiVJKlTQ7WdtD4iIoQ2Pj83MYye2Y0QsAHwNuBOYMzNfrje9AsxZ358HeL7haS/UbS8zhXpkQfPkw/cP33jJQc82ux8taAAwvNmdUBH8rKij/KxM3Bea3YEuMjwzl53cThExE3A+8LPMfCsaJuBkZkZETu2O9ciCJjPnaHYfWlFEDO3IB1Hys6KO8rPSAlrsrO2ImJaqmDk9My+om1+NiEGZ+XI9pPRa3f4iMF/D0+et26aYc2gkSdJUEVUUczzwSGb+tWHTJcD29f3tgYsb2rerz3ZaAXizYWhqivTIhEaSpN6ixRbWWwnYFnggIu6t2/YHDgPOiYidgWeBzeptQ4D1gSeA0cCOnT2wBU3vMtGJW9JE+FlRR/lZ0TiZeQuTHgRbayL7J7D71Di2BU0vMqmZ6NKE/Kyoo/ysNF9HFrzrDZxDI0mSimdCI0lSwQxoKiY0kqSpLsKBEHUvE5peJiLmBkYCH2fmh83uj6Sepb5uz0uZOarZfekVOnjRyN7AhKYXiYh1gUuBfwH/iIhZmtwltTj/la0pEREbUF2nZ+WI8PtF3coPXC9RFzMHAz+nKmjGAD/zC0sTiojBEbEejFui3M+IJisivgUcAuyRmZdl5scN2/wMdanoxlvrcsiph6v/RzIjcARwQ2ZeX7cvBixSrwEgARAR0wHfBb4QER9n5lWNRc2E95vZV7WcbwJHZ+ZtETEzsCDVuiPXA/cDfl7UpUxoerj6kuyjgW2ANSJit3rTYkC/5vVMrSYiBgDTAn8EHgfWaUxqGP/Ps+ksZtQmIpas775HVQh/BTgaOAjYnGoIap3m9E69iQVNDxYR07fdz8yhVMtR7xMR1wMLA3vX+7V2jqguV899uAK4hWpo8njgZWDtehuZ+XFE7AFcERF9/dz0bg2//9Mi4jDgb8DXgROBd4C/Z+bywJHADyLCEYEuEFSTgrvr1sr8gPVQEbEOsEtEXJaZJwNk5t0R8T3gdODCzPwoIqbJzDFN7ayaqv6s/InqgnEvAjcAb1N9Qe0BrBkRLwNLAj8CtsrMsU3qrlpEQ0r3f1SJzGJU1+SZPTNfa5gU/BHwBq0+AUPFs6DpuWYFVgMG1//CPg+4PTP/GxHbA6dERL/M/EMzO6nmioiZgJ2B24CHMnN0ffG4HTPzvYg4karQORRYFlgzM+9vXo/VCiLim1TDk0Mz8/GIuAzYCHg4M1+rd5smIjYHfgzskJkfNam7PZ6VYsUhp57rFuAc4HvAlVRDTNfUcyL+C3wb+L+ImK15XVQzRcRCVGe7/Q14k2pYYCaqz8aHEdEnM1+lmgNxOfANixnVdgFOAg6MiG9Qpb7LA6sAREQ/4Pv1fjtm5oNN6qd6EQuaHiQilomIFQAy80XgfeAvmXkCcCOwENX/ZK4Fvkr1BfVGc3qrZqoL2Z8CvwbuBi4AFgAuAb6emT+s58z0rYuaozLz8aZ1WK3m11RFzDDgcGApqhT4mIhYpD4R4Qxg08x8oHnd7B2cQ1OxoOkh6uTlWGB0W1tm7gM8ExG/A04DNqA66+BXwL3Og+h9GiZyjqJK7gL4JXAH1WfkZeDKOqmh7TPSuKaIeqeI+GZE7FinL08Ab9WbNqY61f8jqs/VzyJilsx8IzNfb05v1RtZ0PQA9aJ5BwD7Zeb9ETFbRCxUT8p7BfghsE1mXg2Mzcxb/dd2r9W3/jMy8wrgPmATqqLmAeAoYCDV2XBzN6WHajkRMQPVRPGfAb8B1gYOAzajWmtmL+BJqsm/GzSnl71XdON/rcyCpnAR0R8YAvwpM6+u50VcAsxX/6v631T/6p4N/Jd2b1avM/NERAysh5Pmphp2uhOYGfgFMJTq8zMj1ZClern6LLjbqdLftaj+f/Jj4EBgN2BVYP7MvD4zVwZWycw3m9Vf9V4WNIXLzJFUZxf8pl7g6hjgosy8sWFS59HAco3r0qj3yczhwE+A6yNiCeBU4IzM/BHV8NMsVGvQ3A4cVH+2pC8Bi1PNlflaZv4N+AHVWZRHAvNSJTYAZObzzehkr+aVDwBP2+4RMvPyiBgL3Avsn5l/qSdzjo2INYHPAf/MzA+a2lE1XWZeGhEfUS1Fv39mHlVvuhmYnupLaua6+JEAzgS+CDwP/CgiZsvMc4AVI2JvYCtgv4g4oZ4MLDWFCU0PkZlXUi0vvkNEzFoXMzsC/w84u05qpAk/K7PUbWMz8yrgdxYzioglGy5pMBL4kCqlORrYpl5fhsz8C9WJBstZzDSPAU3FgqYHycxrgD2BmyPih8BOwE6Z+Wxze6ZW0/BZuaueh9XW7pdSLxcRs1OlvZdHxKbAMlRnRn5A9Z12OrBVRGwNkJlP1MtESE3lkFMPk5lXRERfqnVFvpaZDzW7T2pN9WdlOuDaiFiW+lqmze6XmiszR9QrAV9LdbmLxaiK3xeBOTLztIiYEdgoIi4B3vFzo1ZgQdMDZeZl9bCT/9pWuzLz4oi4zrPf1Cgzr4+IbwEnAEsDm1LNlRkUEecA5wPnZ+bbTeymKGPBu+5iQdNDWcyoozLznWb3Qa0nM6+rh65vpFpV/F8RsWBmfkg1p0ZqKRY0kqSJyswh9eLSd0fESpn5NFQrTjvM1DpafcG77mJBI0mapLqomRbnWqnFWdBIktrlXKsWZ0ADeNq2JKkDnGulVmdCI0lSwQxoKiY0kiSpeCY0kiQVzHVoKiY0UguIiLERcW9EPBgR50ZEv8/wWifVS9YTEcdFxOLt7Lt6RKzYiWM8ExEDOto+wT5TNBcjIg6KiH2mtI+SehcLGqk1vJeZX83MJagWLdutcWNEdCpNzcxdMvPhdnZZHZjigkZSq4hu/a+VWdBIredmYOE6Pbm5vl7OwxHRNyL+FBF3R8T9EfEDqBY5i4gjI+LRiLgWGNj2QhFxY712CBGxbkTcExH3RcR1EbEAVeG0Z50OrRIRc0TE+fUx7o6Ilernzh4RV0fEQxFxHB2YhxgRF0XEsPo5u06w7fC6/bqImKNuWygirqyfc3NELDpV3k1JvYJzaKQWUicx6wFX1k1LA0tk5tN1UfBmZn49IqYHbo2Iq4GvAYOBxYE5gYeprsHT+LpzAP8GVq1fq39mjoyIY6guLvjner8zgMMz85aImB+4iurihAcCt2TmwRGxAbBzB36cnepjzEi10uz5mTkC+BwwNDP3jIjf1K/9Y+BYYLfMfDwilgf+CazZibdRUi9kQSO1hhkj4t76/s3A8VRDQXe1LTcPrA0s2TY/BpgFWARYFTgzM8cCL0XE9RN5/RWAm9peKzNHTqIf3wQWj/GzDGeOiJnqY3y3fu7lEfFGB36mn0bEd+r789V9HQF8DJxdt58GXFAfY0Xg3IZjT9+BY0i9WuCk4DYWNFJreC8zv9rYUH+xv9vYBPwkM6+aYL/1p2I/+gArZOb7E+lLh0XE6lTF0Tcyc3RE3AjMMIndsz7uqAnfA0nqKOfQSOW4CvhhfV0dIuJLEfE54CZg83qOzSBgjYk89w5g1YhYsH5u/7r9beDzDftdDfyk7UFEfLW+exOwVd22HjDbZPo6C/BGXcwsSpUQtekDtKVMW1ENZb0FPB0R36uPERGx1GSOIUnjWNBI5TiOan7MPRHxIPAvqpT1QuDxetspwO0TPjEzXwd2pRreuY/xQz6XAt9pmxQM/BRYtp50/DDjz7b6LVVB9BDV0NNzk+nrlcA0EfEIcBhVQdXmXWC5+mdYEzi4bt8a2Lnu30PAxh14TyQJgPCiqZIklelrSy+bN9x6Z7cdb7Z+0wzLzGW77YBTwIRGkiQVz0nBkiQVrNUXvOsuJjSSJKl4JjSSJJUqXIemjQmNJEkqngmNJEmFCjpwYbVewoRGkiQVz4JGkiQVzyEnSZJK5pgTYEIjSZJ6ABMaSZIK5sJ6FRMaSZJUPBMaSZIK5sJ6FRMaSZJUPBMaSZIKZkBTMaGRJEnFM6GRJKlkRjSACY0kSeoBTGgkSSqY69BUTGgkSVLxLGgkSVLxHHKSJKlQgQvrtTGhkSRJxYvMbHYfJElSJ0TElcCAbjzk8MxctxuP12EWNJIkqXgOOUmSpOJZ0EiSpOJZ0EiSpOJZ0EiSpOJZ0EiSpOL9f8ZvqNiVr7JLAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"So many instances are predicted as EAP and is highly biased towards that class. \n\n**SVD on word TFIDF:**\n\nSince the tfidf vectors are sparse, one another way of compressing the information and representing it in a much compact way is through SVD. Also generally SVD features have performed well for me in the past text based competitions. So we could created svd features on word tfidf and add them to our feature set.","metadata":{"_uuid":"e16c6d867d6bf1bb4e2661cc28841c672091fe0a","_cell_guid":"404695a0-c33b-4cba-9dd1-bf2a5e8c12fc"}},{"cell_type":"code","source":"n_comp = 20\nsvd_obj = TruncatedSVD(n_components=n_comp, algorithm='arpack')\nsvd_obj.fit(full_tfidf)\ntrain_svd = pd.DataFrame(svd_obj.transform(train_tfidf))\ntest_svd = pd.DataFrame(svd_obj.transform(test_tfidf))\n \ntrain_svd.columns = ['svd_word_'+str(i) for i in range(n_comp)]\ntest_svd.columns = ['svd_word_'+str(i) for i in range(n_comp)]\ntrain_df = pd.concat([train_df, train_svd], axis=1)\ntest_df = pd.concat([test_df, test_svd], axis=1)\ndel full_tfidf, train_tfidf, test_tfidf, train_svd, test_svd","metadata":{"_uuid":"95a7ae438276387fa9f2c4a2b25ee29f9b1f7d2d","_cell_guid":"94d49082-2834-4633-8420-10f12f1b16eb","execution":{"iopub.status.busy":"2023-01-12T09:34:20.580575Z","iopub.execute_input":"2023-01-12T09:34:20.581233Z","iopub.status.idle":"2023-01-12T09:34:26.221299Z","shell.execute_reply.started":"2023-01-12T09:34:20.581197Z","shell.execute_reply":"2023-01-12T09:34:26.219978Z"},"trusted":true},"execution_count":19,"outputs":[]},{"cell_type":"markdown","source":"**Naive Bayes on Word Count Vectorizer:**","metadata":{"_uuid":"468ef624b3f6d752737b9c112ae6f0592a929da2","_cell_guid":"77309c44-f947-46f4-814c-7d7a01768293"}},{"cell_type":"code","source":"### Fit transform the count vectorizer ###\ntfidf_vec = CountVectorizer(stop_words='english', ngram_range=(1,3))\ntfidf_vec.fit(train_df['text'].values.tolist() + test_df['text'].values.tolist())\ntrain_tfidf = tfidf_vec.transform(train_df['text'].values.tolist())\ntest_tfidf = tfidf_vec.transform(test_df['text'].values.tolist())","metadata":{"_uuid":"9cd43c20764c81a3216b8371c477ae5552e24c7d","_cell_guid":"92ab9f7c-8c2c-4233-a6dc-eaa12ca7d144","execution":{"iopub.status.busy":"2023-01-12T09:34:26.222867Z","iopub.execute_input":"2023-01-12T09:34:26.223322Z","iopub.status.idle":"2023-01-12T09:34:31.351322Z","shell.execute_reply.started":"2023-01-12T09:34:26.223290Z","shell.execute_reply":"2023-01-12T09:34:31.350327Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"markdown","source":"Now let us build Multinomial NB model using count vectorizer based features..","metadata":{"_uuid":"2e05fb42bcd850e74c767961f58400b0dac463ec","_cell_guid":"48d68a84-6c6d-40b3-9069-d4c324355fc6"}},{"cell_type":"code","source":"cv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nkf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_tfidf[dev_index], train_tfidf[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runMNB(dev_X, dev_y, val_X, val_y, test_tfidf)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\nprint(\"Mean cv score : \", np.mean(cv_scores))\npred_full_test = pred_full_test / 5.\n\n# add the predictions as new features #\ntrain_df[\"nb_cvec_eap\"] = pred_train[:,0]\ntrain_df[\"nb_cvec_hpl\"] = pred_train[:,1]\ntrain_df[\"nb_cvec_mws\"] = pred_train[:,2]\ntest_df[\"nb_cvec_eap\"] = pred_full_test[:,0]\ntest_df[\"nb_cvec_hpl\"] = pred_full_test[:,1]\ntest_df[\"nb_cvec_mws\"] = pred_full_test[:,2]","metadata":{"_uuid":"4f9354473820d59076bf991b5110fc4dd309c204","_cell_guid":"b7927348-b6b1-4960-b76c-ebf14fae37f0","execution":{"iopub.status.busy":"2023-01-12T09:34:31.354112Z","iopub.execute_input":"2023-01-12T09:34:31.354614Z","iopub.status.idle":"2023-01-12T09:34:31.704570Z","shell.execute_reply.started":"2023-01-12T09:34:31.354566Z","shell.execute_reply":"2023-01-12T09:34:31.703454Z"},"trusted":true},"execution_count":21,"outputs":[{"name":"stdout","text":"Mean cv score : 0.45091841616567435\n","output_type":"stream"}]},{"cell_type":"code","source":"cnf_matrix = confusion_matrix(val_y, np.argmax(pred_val_y,axis=1))\nnp.set_printoptions(precision=2)\n\n# Plot non-normalized confusion matrix\nplt.figure(figsize=(8,8))\nplot_confusion_matrix(cnf_matrix, classes=['EAP', 'HPL', 'MWS'],\n title='Confusion matrix of NB on word count, without normalization')\nplt.show()","metadata":{"_uuid":"75658bb392011311ffefbfc562483ac5e11a0cab","_cell_guid":"5d412e2f-8988-4895-9434-33df93c263db","_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-01-12T09:34:31.706070Z","iopub.execute_input":"2023-01-12T09:34:31.706436Z","iopub.status.idle":"2023-01-12T09:34:32.244587Z","shell.execute_reply.started":"2023-01-12T09:34:31.706404Z","shell.execute_reply":"2023-01-12T09:34:32.243803Z"},"trusted":true},"execution_count":22,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 576x576 with 2 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAjQAAAI4CAYAAAB9SXN4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABA20lEQVR4nO3dd5xcVf3/8dcnCYRESgKhhiqEAFICBAIoCASRDiodleaPLwIiCqKgVAsqKkXaF6UpRbr03hJ6AgRUavzSCSWBRAghEPj8/rh3l0nMbjZLdmfu7uvJYx7MnHtn7pm7NzNn3ufccyMzkSRJqrIe9a6AJEnSZ2WDRpIkVZ4NGkmSVHk2aCRJUuXZoJEkSZXXq94VkCRJ7dNz/mUyp03ptO3llLduycwtOm2Ds8EGjSRJFZXTptB78M6dtr0Pxpw+oNM2NpvscpIkSZVnQiNJUmUFhNkEmNBIkqQuwIRGkqSqCiCi3rVoCCY0kiSp8kxoJEmqMsfQACY0kiSpCzChkSSpyhxDA5jQSJKkLsCERpKkynIemibuBUmSVHkmNJIkVZljaAATGkmS1AXYoJEkSZVnl5MkSVUVOCi45F6QJEmVZ0IjSVJlhYOCSyY0kiSp8kxoJEmqMsfQACY0kiSpCzChkSSpyhxDA5jQSJKkLsCERpKkyvLilE3cC5IkqfJMaCRJqqrAMTQlExpJklR5NmgkSVLl2eUkSVKVOSgYMKGRJEldgAmNJEmV5WnbTdwLkiSp8kxoJEmqsh6etg0mNJIkqQswoZEkqaoCx9CU3AuSJKnyTGgkSaoyL30AmNBIkqQuwIRGkqTKch6aJu4FSZJUeTZoJElS5dnlJElSlTkoGDChkSRJXYAJjSRJVeagYMCERpIkdQEmNJIkVVWEY2hKJjSSJKnyTGgkSaoyx9AAJjSSJKkLMKGRJKnKHEMDmNBIkqQuwIRGkqTK8uKUTdwLDSoi+kTEdRExKSIu/wyvs0dE3Don61YvEbFhRDzTAa87R/Z1VxcR50fEL+pdj44WEUtHxHsR0bOVdTIiVujMelVB7X6JiLMi4qgO2MZNEbHnnH5dVZ8Nms8oInaPiNHlB+C48h/bl+bAS+8ILAoslJk7tfdFMvOizNx8DtSnQ7XlCyIzR2bm4A7YfKv7OiKOLeu3c01Zr7Js2fLx+RHxYXkcvBsRj0TElzugripFxF4Rce+cft3MfCkz583Mj8vt3B0R35nT25mZ8li7sDO21dEyc//M/PlneY2Z7Y/M3DIzL/hstVNHiYhzI+LNiPhnTdmJEfF0RDwREVdHRL+aZUdExNiIeCYivlpTvkVZNjYiftKWbdug+Qwi4ofAycCvKL4QlwbOALafAy+/DPBsZk6bA69VeRHRkd2jbdnXbwPHtfarHfhtZs4LzA+cCVw1i/UbVlXrrbbr4H9T6kxNk+t1xm3Wzge2mKHsNmDVzFwdeBY4oqh2rALsCnyhfM4ZEdGz/Pw5HdgSWAXYrVy3VTZo2ikiFgCOBw7MzKsyc3JmfpSZ12Xmj8p1ekfEyRHxWnk7OSJ6l8s2johXIuLQsjU7LiL2LpcdBxwN7FL+4t93xl8qEbFsmRD0Kh/vFRH/V6YDz0fEHjXl99Y8b4OIGFV2r4yKiA1qlt0dET+PiPvK17k1Iga08P6b6n94Tf13iIitIuLZiHg7Io6sWX/diHggIiaW654WEXOXy0aUqz1evt9dal7/xxHxOnBeU1n5nOXLbaxVPl4iIt6KiI1bqO/K5fubGBH/iojtWtrXLfzJbwY+BL7ZwvJmmZnAxcCCFA3dmdWnXcfGTF5nk4j4R83j2yJiVM3jkRGxQ2v7oFx2fkScGRE3RsRkYJOIWDMiHi2PhUuBeVp73xHx/yLiqXL9J2v+Nq1td7r0YybHa0bE/hHxXPn806OwMnAWsH75d5vYWt3K1zouIv5Y3p8rIiZHxInl4z4R8UFELBg1/7Yi4pfAhsBp5XZOq3nJzWasV/laPSLiZxHxYvn3+0sUnxfNf9sZ6vVCRGwWEVsAR/Lpsfh4C+/jhYg4LIpfu5Mi4tKImKdm+f+L4lft2xFxbUQsMcP+PDAingOeizn473gm9Tw/yi7KKLp036u5fRIRe5XLTomIlyPiP1EkmxuW5TPdH7XHzCz2ddPfcc+IeCkixkfET1s9SPSZZeYIih+AtWW31vxgfBBYsry/PfC3zJyamc8DY4F1y9vYzPy/zPwQ+BttCAps0LTf+hQf8Fe3ss5PgfWAIcAaFH+kn9UsXwxYABgI7AucHhH9M/MYitTn0jL6Pqe1ikTE54BTgS0zcz5gA2DMTNZbELihXHch4A/ADRGxUM1quwN7A4sAcwOHtbLpxSj2wUCKRsGfKL7w16b4EjgqIpYr1/0Y+AEwgGLfDQcOAMjMjcp11ijf76U1r78gRYKyX+2GM/PfwI+BCyOiL3AecEFm3j2T9z0XcB1wa/m+vgdcFBGDZ2NfJ3AUcEz5ei2K4tfFt4HngTdaWK1dx8ZMXudBYFBEDCjrtTqwRETMFxF9gKHAyNb2Qc1r7Q78EpgPeBj4O/BXir/B5cA3WnnPOwHHlu97fmA7YEIbtzsr2wDrlO9tZ+CrmfkUsD/wQPl369eG17kH2Li8vw7wOtB07K0PPJOZM34Q/xQYCRxUbueg1upVlu9V3jYBPg/MC9Q2hGYqM29m+mNxjVZW35niF+1y5fb3AoiITYETyuWLAy9SfBnU2gEYRvHLF+bQv+NZvLdty/c0L7ATxb6/o1w8iuLfwYIUPwQuj4h52rg/9mLW+/pLwOCyrkeXjeGuIygGBXfWDQZEMcyi6bbfLGo4o32Am8r7A4GXa5a9Upa1VN4qGzTttxAwfhbdFHsAx2fmm5n5FnAc8K2a5R+Vyz/KzBuB9yj+4bXHJ8CqEdEnM8dl5r9mss7WwHOZ+dfMnJaZlwBPA9vWrHNeZj6bmVOAyyg+aFryEfDLzPyI4kNzAHBKZr5bbv9Jii9rMvORzHyw3O4LwP8Csxpj8glwTNl6nzLjwsz8E0WL/iGKD++Wfn2tR/FB9+vM/DAz7wSuB3abxfZn3N61wFtAS+MpDiuTgvcouiKPahqHMRNz5Ngo98soii/mtYHHgfuAL1K87+cycwJt2wfXZOZ9mfkJxd99LuDksg5XlNtpyXcoutxGZWFsZr7Yxu3Oyq8zc2JmvgTcRevHZGseoGj8LUSxv84BBkbEvBTH4j2z+Xot1WsP4A/lr8v3KOL1XWPOdvGcmpmvlQ2w62bY9rmZ+WhmTi23vX6UY71KJ2Tm2zX/pjr633GziFgRuADYOTNfLl/zwsycUL7m74HetP1zsC37+rjMnJKZj1P8+2itoahZG5+ZQ2tuZ7f1iWVCNg24qCMqZoOm/SZQtFRb+5BaguIXUpMXy7Lm15ihQfQ+xYf/bMnMycAuFL9Yx0XEDRGxUhvq01Sn2pbv67NRnwk1X9hNH461icSUpudHxIoRcX1EvB4R/6H45TXT7qwab2XmB7NY50/AqsAfyw/wmVkCeLn8om4y4/tuq59RNJxm1v3yuzIp6EuRjJwYEVu2Uqc5dWw0JQ8blffvpviSqf2Sbss+qP1FtATwatl9Vrt+S5YC/j2T8jmx72fnmGxR+QU+mmK/NO2r+ykaf+1p0LRUr5n9bXvRQvdjO7Vp2+WX/ARa/jtDx/87pnzuAsA1wM8ys7Zb8bAouionlT8IFmjra9K2fT1Hjp/GFZ2d0LSvlkUX4zbAHjWfK69SfHY0WbIsa6m8VTZo2u8BYCpFfNuS1yi6S5osXZa1x2SKL8omi9UuzMxbMvMrFEnF0xRf9LOqT1OdZnmgzAFnUtRrUGbOT9E3PqsRZtnawvKX9ckUv7SPLbvUZuY1YKmI6f41tut9Z+ZtFKlQizF7mVD8kyIp2bqVOs2pY2PGBs09/HeDpi37oHZ/j6NIL2KG9VvyMrD8TMpntd1Wj+tZaPX4aME9wKbAmhSJ0z0UXUXrAiNaeM7sbmdmf9tpFI2E6d5v2T258GfYVqvbLrujF6Llv/Psas+/Y8q//8XAXbW/6MvxModTdJH1L38QTKp5zVnVtbV9rQZRjoc6HNguM9+vWXQtRaLWu+zWHETR3T2KIk1drhyjtWu5bqts0LRTZk6i6G8+vRxE1zeKgYZbRsRvy9UuAX4WEQtHMbj2aKC9p2SOATaKYo6MBShHiQNExKIRsX354TWVonvik5m8xo3AilGcat4rInah6Ee/vp11mh3zAf8B3ivTo+/OsPwNij7w2XEKMDozv0MxNuisFtZ7iOKX2eHl32hjim62GccWtNVPKf5xtqh8j18CZtb1B3P22LifIqJfF3i47CZYhmKcRNOX9OzugwcovhgOLtf/evn6LfkzRZfb2lFYISKWacN2xwBfL//9rEAxXqit3gCWjJpBqVEMKn6hlefcQzHO58ksBhveTdFd9nzZ9dfSdmbn2LwE+EH5YTwvn44DmUZxhsc8EbF1Ob7oZxRdLLXbWnaGBuDsuATYOyKGRDHI/FfAQ2X30Jwwq3/HLfkl8Dng+zN5vWkUXbm9IuJoijFYTWa1P1rb191HA53lFBGXUHx+DI5iwPm+FOOa5gNui4gxEXEWQPlZdRlFt+bNFCfZfFz+/Q4CbgGeAi5rYRjFdGzQfAZlf+8PKT6U3qL4lXoQxWBKgF9QRNxPAP8AHi3L2rOt24BLy9d6hOkbIT3KerxGMbr8y8zkg6YcS7ENcChFDH04sE1mjm9PnWbTYRSDTt+lSI8unWH5scAFUZw9sTOzEBHbUwyKbHqfPwTWivLsrlrlF9e2FKcAjqc4tf7bmfl0e95IZt5H8StiRodHcTbGZIpBsOdRjDGYmTl5bEwun/+v8r1C8YHyYma+Wa4zW/ugXP/rFAMu36bo0ryqlTpcTvGldTHF3/jvwIJt2O5JFGePvUExtmJ2+tbvpGgwvh4RTcfwUhTJWEvuB/rwaUPvSeADWk5noGg47xgR70TEqW2o17kUg6lHUAwM/4BiMHTTD6EDKBqAr1IkNrVnPTVN7DghIh5tw7amk5m3Uwxev5IiZVue4tftnDKrf8ct2Y1iPNU78emZTntQfGHdTNHQe5FiX9V2ic1qf7S4r1UfmblbZi6emXNl5pKZeU5mrpCZS2XmkPK2f836v8zM5TNzcGbeVFN+Y2auWC77ZVu2HdN3kUtSdUUxK/b3szgLSuryevRbJnt/+chZrziHfHDt/o9k5tBO2+BscGIlSV1GVmBWbEkdwwaNJElV1rYZfLs8x9BIkqTKs0EjSZIqzy4nSZKqKuIzTXjXlXTJBk306pMx93z1roYqYI2VW5srTpreJ594Vqja5uWXXuTtCeMd3NKJumaDZu756D14llOZSNxzX1umFZEK737QveZrU/tttcn6nbcxBwUDjqGRJEldQJdMaCRJ6i7ChAYwoZEkSV2ACY0kSRUVmNA0MaGRJEmVZ0IjSVJVRXmTCY0kSao+ExpJkiorHENTMqGRJEmVZ4NGkiRVnl1OkiRVmF1OBRMaSZJUeSY0kiRVmAlNwYRGkiRVngmNJEkVZkJTMKGRJEmVZ0IjSVJVeemDZiY0kiSp8kxoJEmqqPDSB81MaCRJUuWZ0EiSVGEmNAUTGkmSVHk2aCRJUuXZ5SRJUoXZ5VQwoZEkSZVnQiNJUoWZ0BRMaCRJUuWZ0EiSVFVe+qCZCY0kSao8ExpJkirMMTQFExpJklR5JjSSJFWUF6f8lAmNJEmqPBMaSZIqzISmYEIjSZIqzwaNJEmqPLucJEmqMnucABMaSZLUBZjQSJJUVeGg4CYmNJIkqfJMaCRJqjATmoIJjSRJqjwTGkmSKsyEpmBCI0mSKs+ERpKkivLilJ8yoZEkSZVnQiNJUpUZ0AAmNJIkqQuwQSNJkirPLidJkqrKSx80M6GRJEmVZ0IjSVKFmdAUTGgkSVLlmdBIklRhJjQFExpJklR5JjSSJFWZAQ1gQiNJkroAExpJkirMMTQFExpJklR5JjSSJFVURJjQlExoKuysY/bgxTtOYPTlRzaXHX3A1jx86RE8+LefcN0ZB7L4wgsAsM3GqzWX33vR4Www5PPNz7nmtAMYN+K3XHnK/p3+HlR/Z552KuutvTrD1lqNM/54CgA/O+Jwhq6xChusM4Q9dv46EydOrG8lVTeHHrQfawxakuHrr9lc9t199mDzDddh8w3XYb3VV2TzDdcBYMRdt7PlxusxfIO12HLj9bhvxF31qra6IRs0FfbX6x5k+wNPn67spAvuYN1dTmC9XX/NTSP/yRH7bQnAXQ8901y+/7EXcsbRu3/6nL/czr4/+0un1l2N4cl//ZMLzvszd458kPsefoybb7qBf/97LJsM34wHH3mC+0eNYflBK/KHE39d76qqTnba7VtceMV105Wdee5F3DpyFLeOHMVW2+3AltvuAMCCCw3gvEuu4o77H+WkM87h4P33qUON1V3ZoKmw+x79N29Pen+6sncnf9B8v2+f3mQmAJOnfNhc/rk+vSmLAbj74Wd5d/LUjq2sGtIzTz/F2uusS9++fenVqxdf2nAjrvv71QzfbHN69Sp6pNdZdxivvfpKnWuqelnvixvSr3//mS7LTK67+kq2/8bOAKy6+hAWW3wJAAavvAofTJnC1Kl+tnS0pm6nzrg1MsfQdEHHHrgte2yzLpPem8IW+53aXL7dJqtz/Pe2Y+EF5+PrB59VxxqqUazyhVX5+bFH8faECczTpw+33nwTa6619nTrXPiX8/j6jjvXqYZqZA/dfy8LL7IIn19+0H8tu+Haq1ltjSH07t27DjVTd9TpCU1EfBwRY2puP6lZNiAiPoqI/Wd4zgsR8Y+IeCIibo2IxTq73lVy7OnXMWjLo/jbTaPZf5eNmsuvvesJhnz9F+z8w7M5+oCt61hDNYrBK63MIYf+iB223YJvbLcVq62xBj179mxefuJvfkWvnr3Yedc96lhLNaprrry0OZ2p9cxTT3LCsUfy65NOn8mzNKeZ0BTq0eU0JTOH1NxqO+d3Ah4EdpvJ8zbJzNWB0cCRM1muGVx64yh2GD7kv8rve/TfLDdwAAv1+1znV0oN59t77cuI+0dx0+13069ff5YftCIAF/31fG658Qb+dP6FDf9Bps43bdo0brr+Grb92k7Tlb/26it851s7cfKZ57LscsvXqXbqjhptDM1uwKHAwIhYsoV1RgArdF6VqmX5pRduvr/Nxqvz7AtvAPD5pQY0lw9ZaUl6z92LCRMnd3r91HjeevNNAF5+6SWuu+ZqdtplN26/9WZO+cPv+NsVf6dv3751rqEa0ci772D5QYNZYuCnH9WTJk1kz1124Ihjfsk6621Qx9p1M9GJtwZWjzE0fSJiTM3jEzLz0ohYClg8Mx+OiMuAXYDfz+T52wD/mLEwIvYD9gNgrnnneKUb0QUn7MWGaw9iQL95GXvzz/n5WTeyxZe+wKBlFuGTT5KXxr3Nwb/8GwBfGz6E3bcZxkfTPuaDqR/xrR+f2/w6t59zCCsutyjz9unN2Jt/zv7HXcztDzxVr7elTvat3Xbi7bcnMNdcc/G7k/9Iv379OOwHB/Ph1KnssM1XARi67jBO/uOZda6p6uHAfb/FA/eN4O0J4xn6hc9z6E+OYrdv7c21V13ODjN0N53/pzN54fl/c/Jvf8nJv/0lABdfdQMDFl6kHlVXNxNZe7pLZ2ww4r3M/K8WR0QcBvTPzJ9GxOrAuZk5tFz2AvAu8DHwBHBwZk5saRs9+i6SvQc7iFGz9sYDp856Jan07gfT6l0FVcRWm6zP44890uGZRu9FB+XAPU7p6M00e/6krR9p+m5uNI10ltNuwGIR0TT6cImIGJSZz5WPN8nM8XWqmyRJamANMYYmIlYE5s3MgZm5bGYuC5zAzAcHS5IkgGiss5wi4tyIeDMi/llTtmBE3BYRz5X/71+WR0ScGhFjy7OY16p5zp7l+s9FxJ5t2RX1aND0meG07V9TNFyunmG9K7FBI0lSlZwPbDFD2U+AOzJzEHBH+RhgS2BQedsPOBOKBhBwDDAMWBc4pqkR1JpO73LKzJ6zXgsy8wlg5fL+sh1ZJ0mSqiiARppVITNHRMSyMxRvD2xc3r8AuBv4cVn+lywG8z4YEf0iYvFy3dsy822AiLiNopF0SWvbbqQxNJIkqbENiIjRNY/PzsyzZ/GcRTNzXHn/dWDR8v5A4OWa9V4py1oqb5UNGkmSKqvTZ/Ad/1nOcsrMjIgOOb26IQYFS5KkLuuNsiuJ8v9vluWvAkvVrLdkWdZSeats0EiSpI50LdB0ptKewDU15d8uz3ZaD5hUdk3dAmweEf3LwcCbl2WtsstJkqQKa6RBwRFxCcWg3gER8QrF2Uq/Bi6LiH2BF4GmmW9vBLYCxgLvA3sDZObbEfFzYFS53vFNA4RbY4NGkiTNEZnZ0nQrw2eybgIHtvA65wLnzmxZS2zQSJJUYZ08KLhhOYZGkiRVngmNJElVFY01hqaeTGgkSVLlmdBIklRRAfToYUQDJjSSJKkLMKGRJKnCHENTMKGRJEmVZ0IjSVKFOQ9NwYRGkiRVng0aSZJUeXY5SZJUVU6s18yERpIkVZ4JjSRJFRU4KLiJCY0kSao8ExpJkiorTGhKJjSSJKnyTGgkSaowA5qCCY0kSao8ExpJkirMMTQFExpJklR5JjSSJFWVMwU3M6GRJEmVZ4NGkiRVnl1OkiRVlJc++JQJjSRJqjwTGkmSKsyApmBCI0mSKs+ERpKkCnMMTcGERpIkVZ4JjSRJFWZAUzChkSRJlWdCI0lSVYVjaJqY0EiSpMozoZEkqaKKmYLrXYvGYEIjSZIqzwaNJEmqPLucJEmqrHBQcMmERpIkVZ4JjSRJFWZAUzChkSRJlWdCI0lShTmGpmBCI0mSKs+ERpKkqgrH0DQxoZEkSZVnQiNJUkUVlz4wogETGkmS1AWY0EiSVGEmNAUTGkmSVHk2aCRJUuXZ5SRJUoXZ41QwoZEkSZVnQiNJUoU5KLhgQiNJkirPhEaSpKry0gfNTGgkSVLlmdBIklRRQTiGptQlGzSrr7QUd4w4ud7VUAV89ZSR9a6CKuS2QzasdxVUET172MjobF2yQSNJUndhQFNwDI0kSao8ExpJkiqshxENYEIjSZK6ABs0kiSp8uxykiSpwuxxKpjQSJKkyjOhkSSpoiK8OGUTExpJklR5JjSSJFWYkxIXTGgkSVLlmdBIklRhjqEpmNBIkqTKM6GRJKnCDGgKJjSSJKnyTGgkSaqoAAIjGjChkSRJXYANGkmSVHl2OUmSVGFOrFcwoZEkSZVng0aSpKqKIDrx1rYqxQ8i4l8R8c+IuCQi5omI5SLioYgYGxGXRsTc5bq9y8djy+XLtndX2KCRJElzREQMBA4GhmbmqkBPYFfgN8BJmbkC8A6wb/mUfYF3yvKTyvXaxQaNJEkVFtF5tzbqBfSJiF5AX2AcsClwRbn8AmCH8v725WPK5cOjnddysEEjSZLaakBEjK657Ve7MDNfBX4HvETRkJkEPAJMzMxp5WqvAAPL+wOBl8vnTivXX6g9FfMsJ0mSKiqAHp177YPxmTm0pYUR0Z8idVkOmAhcDmzRGRUzoZEkSXPKZsDzmflWZn4EXAV8EehXdkEBLAm8Wt5/FVgKoFy+ADChPRu2QSNJUoU12Bial4D1IqJvORZmOPAkcBewY7nOnsA15f1ry8eUy+/MzGzPfrBBI0mS5ojMfIhicO+jwD8o2hlnAz8GfhgRYynGyJxTPuUcYKGy/IfAT9q7bcfQSJJUYe08KajDZOYxwDEzFP8fsO5M1v0A2GlObNeERpIkVZ4NGkmSVHl2OUmSVFGzOeFdl2ZCI0mSKs+ERpKkCuvkifUalgmNJEmqPBMaSZIqzHymYEIjSZIqz4RGkqQKa7SJ9erFhEaSJFWeCY0kSRUVQA8DGsCERpIkdQEmNJIkVVWEY2hKJjSSJKnybNBIkqTKs8tJkqQKs8epYEIjSZIqz4RGkqQKc1BwwYRGkiRVXosJTUT8EciWlmfmwR1SI0mS1CZOrPep1rqcRndaLSRJkj6DFhs0mXlB7eOI6JuZ73d8lSRJUls5hqYwyzE0EbF+RDwJPF0+XiMizujwmkmSJLVRWwYFnwx8FZgAkJmPAxt1YJ0kSVIbRSfeGlmbznLKzJdnKPq4A+oiSZLULm2Zh+bliNgAyIiYC/g+8FTHVkuSJM1KBPRwDA3QtoRmf+BAYCDwGjCkfCxJktQQZpnQZOZ4YI9OqIskSVK7tOUsp89HxHUR8VZEvBkR10TE5zujcpIkqXURnXdrZG3pcroYuAxYHFgCuBy4pCMrJUmSNDva0qDpm5l/zcxp5e1CYJ6OrpgkSZq1iOi0WyNr7VpOC5Z3b4qInwB/o7i20y7AjZ1QN0mSpDZpbVDwIxQNmKYm2f/ULEvgiI6qlCRJapsGD046TYtdTpm5XGZ+vvz/jDcHBTeYg7/7HVZabgm+tO6Q5rJjfvpj1ltrVTZab02+vduOTJo4cbrnvPLySyyzWD9OO+UPnVtZ1d2uQwdy0b5DuXCfoRy37crM3TNYe+l+nL/nWly4z1CO2mowPcsPyc/N3ZMTv7Eqf9l7bS7adyhbr7ZofSuvunn22WfYYN21mm9LLNyP0/94CgBnnXEaa62+CuusuRo/O/LHda6puqM2zRQcEatGxM4R8e2mW0dXTLNn1z325NKrr5+ubONNN+Peh8cw4sHHWH6FQZz8+99Mt/yoI37E8K9s0ZnVVANYeN652WntgexzwaN889zR9OwBm6+yKEdtPZijrn2Kb547mtf/M5WtVlsMgB3XGsjz4yfz7fMe4cCLH+fgTZanVw9/EnZHK644mPsffpT7H36UkQ+Mok/fvmy73Q6MuPsubrjuWh4Y9RijHvsH3z/k0HpXtdsIgh7RebdG1pbTto8B/ljeNgF+C2zXwfXSbNrgSxvSv/+C05VtMvwr9OpV9CoOXWcYr732SvOyG6+7hqWXWZbBK6/SqfVUY+jZI+jdqwc9A+bp1ZMpH33MRx8nL78zBYCHX3iHjVccAECS9J27JwB95u7Jfz6YxsefZN3qrsZw9513sNxyy7P0Msvw5z+dxQ8PO5zevXsDsPAii9S5duqO2pLQ7AgMB17PzL2BNYAFOrRWmuMu+uv5zWnMe++9x6knnciPjjiqzrVSPbz13odc/PArXP3d9bjuoPV5b+o07nj6LXr2CFZabF4ANhk8gEXnL76crnj0NZZd6HNcd+B6XLjPUE66fSw2Z3TF5Zey0y67AjD2uee4/7572WTD9dlis014ZPSoOteuG+nEOWgaPKBpU4NmSmZ+AkyLiPmBN4Gl2rvBiHhvhsd7RcRp5f1jI+LViBgTEf+MiO1qyg9r7za7uz+ceAK9evVip112B+C3vzqe/Q/6PvPOO2+da6Z6mK93LzYctBDfOOshtj39QeaZqydfXWURjr72Kb6/6Qqc8601ef/Dj/n4k2L9Ycv157k332Pb0x9kz/NGc+hXVmhObNQ9ffjhh9x4w3V87es7AjBt2jTeeedt7hxxP7844TfsuceuZNrsVedqy8UpR0dEP+BPFGc+vQc80IF1OikzfxcRKwMjI8Ls8jO45MILuPWmG7jq+lub5xB4dPTDXHfNVRx31BFMmjSRHj16MM88vfnO/3iJru5gnWX7MW7SB0yc8hEA9zw7ntUGzs8tT77Jdy8eA8C6y/Zn6QX7ALD1aovx1wdfBuCViR/w2qQPWHahvjw57t261F/1d+stNzFkyJossmgxQHzgwIFst/3XiAiGrrMuPXr0YPz48Sy88MJ1rmn30Ojzw3SWtlzL6YDy7lkRcTMwf2Y+0bHVgsx8KiKmAQM6eltd1R233cIfT/491950B3379m0uv/7Wu5vv/+ZXx/O5z81rY6Ybef0/U/nCEvPTu1cPpk77hKHL9OOp19+lf9+5eOf9j5irZ/CtYUtx/gMvAfDGf6YydJl+PP7KJPr3nYtlFuzLqxOn1PldqJ6uuOxv7Ljzrs2Pt9lue0bcczcbbbwJzz33LB9++CEDBvjRrc7V2sR6a7W2LDMfbec2+0TEmJrHCwLXzmQbw4BPgLfauZ1u5f/t/U3uG3kPb08Yz2qDl+XHRx7NKX/4LVOnTmXH7YuxM2uvM4zfn3JGnWuqenty3Lvc9cxbXLDX2kz7JHn2jfe45vFx/M+Gy/HFFRYkCK4e8xqPvDQRgPPuf5GfbTWYC/dZGwhOv/v/mDRlWl3fg+pn8uTJ3HnH7Zxy2lnNZd/acx8O2G9f1l1rdeaee27+98/nmRqo00VL/ZwRcVcrz8vM3LRdG4x4LzPnrXm8FzA0Mw+KiGOB/0fRiHkXODIzR5bl72Xm71p53f2A/QCWXGrptcc8+e/2VE/dzDan31fvKqhCbjtkw3pXQRWx0Qbr8ugjozu8VbfICqvmLide3tGbaXba11d5JDOHdtoGZ0OLCU1mbtKZFalxUmsNl5Zk5tnA2QBD1lrb0WiSJHUjbRkULEmSGlDgoOAmbZopuEH8LCJeabrVuzKSJKlxdHpCUzt+pnx8PnB+ef/YFp5zLDDTZZIkdWdeiaTQlksfRER8MyKOLh8vHRHrdnzVJEmS2qYtCc0ZFKdPbwocT3H20ZXAOh1YL0mS1AYmNIW2NGiGZeZaEfEYQGa+ExFzd3C9JEmS2qwtDZqPIqInFNeji4iFKRIbSZJUR8VFI41ooG1nOZ0KXA0sEhG/BO4FftWhtZIkSZoNbbmW00UR8QgwnOKU9x0y86kOr5kkSZolx9AUZtmgiYilgfeB62rLMvOljqyYJElSW7VlDM0NFONnApgHWA54BvhCB9ZLkiSpzdrS5bRa7ePyKtwHdFiNJElSmzkmuDDblz7IzEeBYR1QF0mSpHZpyxiaH9Y87AGsBbzWYTWSJEltEkAPIxqgbWNo5qu5P41iTM2VHVMdSZKk2ddqg6acUG++zDysk+ojSZJmw2yPHemiWtwPEdErMz8GvtiJ9ZEkSZptrSU0D1OMlxkTEdcClwOTmxZm5lUdXDdJkjQLDqEptGUMzTzABIqrbTfNR5OADRpJktQQWmvQLFKe4fRPPm3INMkOrZUkSZqliPAsp1JrDZqewLxM35BpYoNGkiQ1jNYaNOMy8/hOq4kkSZptBjSF1s72chdJkqRKaK1BM7zTaiFJkvQZtNjllJlvd2ZFJEnS7OthfwrgBIOSJKkLaMs8NJIkqQF5ccpPmdBIkqTKM6GRJKnCDGgKJjSSJKnyTGgkSaqq8CynJiY0kiSp8kxoJEmqsHBif8CERpIkdQE2aCRJqqhiHprOu7WpThH9IuKKiHg6Ip6KiPUjYsGIuC0iniv/379cNyLi1IgYGxFPRMRa7d0XNmgkSdKcdApwc2auBKwBPAX8BLgjMwcBd5SPAbYEBpW3/YAz27tRGzSSJGmOiIgFgI2AcwAy88PMnAhsD1xQrnYBsEN5f3vgL1l4EOgXEYu3Z9sOCpYkqcIa7LTt5YC3gPMiYg3gEeD7wKKZOa5c53Vg0fL+QODlmue/UpaNYzaZ0EiSpLYaEBGja277zbC8F7AWcGZmrglM5tPuJQAyM4Gc0xUzoZEkqcKic699MD4zh7ay/BXglcx8qHx8BUWD5o2IWDwzx5VdSm+Wy18Flqp5/pJl2WwzoZEkSXNEZr4OvBwRg8ui4cCTwLXAnmXZnsA15f1rgW+XZzutB0yq6ZqaLSY0kiRVVNNp2w3me8BFETE38H/A3hQBymURsS/wIrBzue6NwFbAWOD9ct12sUEjSZLmmMwcA8ysW2r4TNZN4MA5sV0bNJIkVVVA5w6haVyOoZEkSZVnQiNJUoX1MKIBTGgkSVIXYEIjSVJFNehZTnVhQiNJkirPBo0kSao8u5wkSaowxwQXTGgkSVLlmdBIklRZQQ+MaMCERpIkdQEmNJIkVVTgGJomJjSSJKnyTGgkSaqqcGK9JiY0kiSp8kxoJEmqMC9OWTChkSRJlWdCI0lSRXmW06dMaCRJUuXZoJEkSZVnl5MkSRXmoOCCCY0kSao8ExpJkirMgKZgQiNJkiqvyyY0We8KqBLuOvTL9a6CKmTQIdfUuwqqiDdentgp2wlMJpq4HyRJUuV12YRGkqQuLyAcRAOY0EiSpC7AhEaSpAoznymY0EiSpMozoZEkqaICZwpuYkIjSZIqzwaNJEmqPLucJEmqMDucCiY0kiSp8kxoJEmqMMcEF0xoJElS5ZnQSJJUWeGlD0omNJIkqfJMaCRJqqjAZKKJ+0GSJFWeCY0kSRXmGJqCCY0kSao8ExpJkirMfKZgQiNJkirPBo0kSao8u5wkSaqqcFBwExMaSZJUeSY0kiRVlBPrfcr9IEmSKs+ERpKkCnMMTcGERpIkVZ4JjSRJFWY+UzChkSRJlWdCI0lShTmEpmBCI0mSKs+ERpKkiirmoTGiARMaSZLUBdigkSRJlWeXkyRJFeag4IIJjSRJqjwTGkmSKisIBwUDJjSSJKkLMKGRJKnCHENTMKGRJEmVZ0IjSVJFObHep0xoJElS5ZnQSJJUVeEYmiYmNJIkqfJMaCRJqjATmoIJjSRJqjwbNJIkqfLscpIkqcK89EHBhEaSJFWeCY0kSRUVQA8DGsCERpIkdQEmNJIkVZhjaAomNJIkqfJs0EiSVGERnXdrW32iZ0Q8FhHXl4+Xi4iHImJsRFwaEXOX5b3Lx2PL5ct+lv1gg0aSJM1J3weeqnn8G+CkzFwBeAfYtyzfF3inLD+pXK/dbNBIklRh0Yn/zbIuEUsCWwN/Lh8HsClwRbnKBcAO5f3ty8eUy4eX67eLDRpJktRWAyJidM1tvxmWnwwcDnxSPl4ImJiZ08rHrwADy/sDgZcByuWTyvXbxbOcJEmqqDrMQzM+M4fOtC4R2wBvZuYjEbFxp9YKGzSSJGnO+CKwXURsBcwDzA+cAvSLiF5lCrMk8Gq5/qvAUsArEdELWACY0N6N2+UkSZI+s8w8IjOXzMxlgV2BOzNzD+AuYMdytT2Ba8r715aPKZffmZnZ3u2b0EiSVFltG6xbZz8G/hYRvwAeA84py88B/hoRY4G3KRpB7WaDRpIkzVGZeTdwd3n//4B1Z7LOB8BOc2qbdjl1EQd/9zusvNwSbLjukOayY3/6Y9Zfa1W+vN6a7LnbjkyaOBGADz/8kO/tvy8bDRvCxuuvxX0j76lPpdUwBq+wLEOHrMawtYfwxWHFeL/jjjmKddZcnWFrD2GbLTfntddeq3Mt1Vl+t8cQHjthC24/cpPmsn595+Kig9ZnxNHDueig9Vmgz1zTPWeNpfvx/CnbstWQxZvLdhy2FCOOHs6Io4ez47ClOq3+3UonTqrX/hOqO4cNmi5i1z325G9XXz9d2Zc33YyRD4/hngcfY/kVBnHK74s5i/56/p8BGPHQGC6/9maOPvJHfPLJJ//1mupebr79Lh56ZAz3PTQagB8c+iNGPfYEDz0yhi232oYTfnF8nWuoznL5gy/zrdMfmK7sgK8M4r5nxrPR8Xdw3zPjOWDzQc3LegQcsf0qjHj6reayfn3n4pAtB7Pd70aw7YkjOGTLwf/VCJLmJBs0XcQGX9qQ/v0XnK5sk+FfoVevoldx7XWG8dprrwDwzNNPseGXi19eCy+8CAss0I8xj47u3Aqr4c0///zN999/fzKfYb4rVcxD/57AxPc/nK5s89UX54qHXgLgiode4qurf5rE7P3lz3PT4+OY8O7U5rIvr7wII59+i4nvf8SkKR8x8um32HiVRTrnDXQz0Ym3RmaDppu4+K/nM/wrWwCw6qqrc/ON1zNt2jRefOF5Hh/zKK+++kqda6h6igi23XJzNlh3bc7509nN5ccc9VNWWG4p/nbJRRx1rAlNdzZgvt68+Z+iwfLmf6YyYL7eACy2wDxsscbi/GXk89Otv1i/eRj3zpTmx69PnMJi/ebpvAqr2+mwBk1EZERcWPO4V0S8FRHXR2F8RPQvly1erv+lmvXfioiFImJwRNwdEWMi4qmIOHtm21PL/nDiCfTq1Ysdd9kdgN2/vTdLDBzIZhsN42c/PpR1hq1Pzx4961xL1dMdd9/LA6Me5e/X38T/nnk6944cAcBxP/8lY59/mV1324OzzjitzrVUI0mKs2uP+caq/OqaJ2n/ybb6LIqJ9aLTbo2sI89ymgysGhF9MnMK8BXKyXQyMyPiQWB94EZgA4pTuTYA7o2IwcCEzJwQERdTXNTqGoCIWK0D69zlXHLhBdx20w1cef2tzV0GvXr14he//n3zOlsN35DlBw1q6SXUDQwcWMxEvsgii7DdDl9j1KiH+dKGGzUv32W3Pfjadltx1DHH1auKqrPx705lkfmLlGaR+Xsz4d2iS2r1pftx+t7FQPIF552bTb6wKB9/krw+8QPWGzSg+fmL9evDg8+Nr0vd1T10dJfTjRQXqQLYDbikZtn9FA0Yyv+fRNHAaXp8X3l/cYprPwCQmf/oqMp2NXfcdgunnfx7/nrp1fTt27e5/P3332fy5MkA3H3n7fTs1YvBK61Sr2qqziZPnsy7777bfP/2227lC19YlbHPPde8zvXXXsOKg1eqVxXVAG77xzh2HLY0ADsOW5pbnxgHwBePvZ0NjrmNDY65jRsfe42fXvo4tzzxOvc89SYbrbQwC/SZiwX6zMVGKy3MPU+9Wc+30GU5hqbQ0fPQ/A04OiKuB1YHzgU2LJfdBxxT3l+3vP/98vEGFA0eKBo6d0bE/cCtwHmZOXHGDZUXyNoPYMmllp7jb6TR7bf3N7lv5D28PWE8qw9elsOPPJpT/vBbPpw6lR23L8bODF1nGL875QzGv/UmO++wNT169GDxJZbgjD+dX9/Kq67efOMNdtnxawBM+3gau+y6O5t/dQt23fkbPPfsM/SIHiy9zDKcevpZda6pOstpe63NeoMGsOC8c/Pwzzfn9zc+zem3PceZ+6zDrusvzStvT+GAc0e1+hoT3/+IU29+lusPL5K+U25+lonvf9QZ1Vc3FZ9hluHWXzjivcycNyJGA6cDgygaJIdl5jYR0ZeiC2pJ4I7MXC8iLgOOpJgW+RuZ+XT5WksAW1BcanwwsEZmTv3vrRaGrLV23j7ioQ55X+pa5p3HuSXVdoMOuWbWK0nAG5cdyodvju3wUGPl1dbM8/5+V0dvptn6K/R/pKWLU9ZbZ5zldC3wO6bvbiIz3weeA/YBHi2LHwS2AhYBnqlZ97XMPDcztwemAat2Qr0lSVJFdEaD5lzguBbGvtwPHAI0zeD0AEW304NNF6iKiC0iYq7y/mLAQnx6pU5JkqSOb9Bk5iuZeWoLi+8DPs+nDZpHKbqg7q9ZZ3PgnxHxOHAL8KPMfL2j6itJUpVEJ/7XyDpsAEFmzjuTsrspL1ZVPr6cmoHT5biY3jM854fADzuqnpIkqfocESlJUoU1+Hx3ncZLH0iSpMozoZEkqcIMaAomNJIkqfJMaCRJqjIjGsCERpIkdQEmNJIkVVRx0UgjGjChkSRJXYAJjSRJVRXOQ9PEhEaSJFWeDRpJklR5djlJklRh9jgVTGgkSVLlmdBIklRlRjSACY0kSeoCTGgkSaqscGK9kgmNJEmqPBMaSZIqzIn1CiY0kiSp8kxoJEmqqMCTnJqY0EiSpMozoZEkqcqMaAATGkmS1AXYoJEkSZVnl5MkSRXmxHoFExpJklR5JjSSJFWYE+sVTGgkSVLlmdBIklRhBjQFExpJklR5JjSSJFWV1z5oZkIjSZIqz4RGkqQKcx6aggmNJEmqPBMaSZIqKnAemiYmNJIkqfJs0EiSpMqzy0mSpAqzx6lgQiNJkirPhEaSpCozogFMaCRJUhdgQiNJUoU5sV7BhEaSJFWeCY0kSRXmxHoFExpJklR5JjSSJFWYAU3BhEaSJFWeCY0kSVVmRAOY0EiSpC7ABo0kSao8u5wkSaqowIn1mpjQSJKkyjOhkSSpqsKJ9ZqY0EiSpMozoZEkqcIMaAomNJIkqfJMaCRJqjIjGsCERpIkdQEmNJIkVVY4D03JhEaSJFWeCY0kSRXmPDQFExpJklR5NmgkSVLl2eUkSVJFBZ613aRLNmgef+zR8QvPN9eL9a5HAxoAjK93JVQJHitqK4+VmVum3hXobrpkgyYzF653HRpRRIzOzKH1rocan8eK2spjpQE0UEQTEUsBfwEWBRI4OzNPiYgFgUuBZYEXgJ0z852ICOAUYCvgfWCvzHy0Pdt2DI0kSZpTpgGHZuYqwHrAgRGxCvAT4I7MHATcUT4G2BIYVN72A85s74Zt0EiSVGHRif/NSmaOa0pYMvNd4ClgILA9cEG52gXADuX97YG/ZOFBoF9ELN6e/WCDpns5u94VUGV4rKitPFa6lwERMbrmtl9LK0bEssCawEPAopk5rlz0OkWXFBSNnZdrnvZKWTbbuuQYGs1cZvrBozbxWFFbeazUXydPrDe+LWOmImJe4ErgkMz8T9RUMjMzInJOV8yERpIkzTERMRdFY+aizLyqLH6jqSup/P+bZfmrwFI1T1+yLJttNmgkSaqw6MTbLOtSRDHnAE9l5h9qFl0L7Fne3xO4pqb821FYD5hU0zU1W+xykiTNcRERmTnHuxXU8L4IfAv4R0SMKcuOBH4NXBYR+wIvAjuXy26kOGV7LMVp23u3d8M2aLqZiFgCeBv4JDM/rHd9JHUt5Sm6r2XmxHrXpVuIxro4ZWbeS8thzvCZrJ/AgXNi23Y5dSMRsQVwHfC/wB8jYoE6V0kNLqKRPirV6CJia4pTcr8UEX6/qFN5wHUTZWPmeOBHFA2aacAhfmFpRhExOCK2hOazETxGNEsR8RXg58D3M/P6zPykZpnHUIdqpFE09WOXUxdXfpD0AU4G7srMO8vylYFB9nGrVkTMDXwdWCYiPsnMW2obNTPer2dd1XA2A87MzPsjYn5gOYouhjuBJyimwZc6jAlNF1fOvvg+8E1gk4jYv1y0MtC3fjVTo4mIAcBcwG+B54Cv1iY1fPrzbG4bM2oSEauXd6dQNIRXo5i+/lhgF4ouqK/Wp3bqTmzQdGER0bvpfmaOphh5flhE3AmsABxartfYOaI6XDn24SbgXoquyXOAccDm5TIy85OI+D5wU0T09Ljp3mr+/hdGxK8pLjC4DnAe8B5wamYOA04D/ici7BHoAEExKLizbo3MA6yLioivAt+JiOsz8wKAzBwVETsBFwFXZ+ZHEdErM6fVtbKqq/JYOZFibohXgbuAdym+oL4PbBoR44DVgQOA3TPz4zpVVw2iJqX7BkUiszLF6bcLZeabNYOCPwLeodEHYKjybNB0Xf2ALwODy1/YVwAPZOZjEbEn8JeI6JuZv6lnJVVf5fTk+wL3A//KzPfLeSL2zswpEXEeRUPnV8BQYNPMfKJ+NVYjiIjNKLonR2fmcxFxPbAt8GRmNs0A2ysidgEOAvbKzI/qVN0uz5ZiwS6nrute4DJgJ+Bmii6m28oxEY8B2wHfiIj+9aui6ikilqc42+0UYBJFt8C8FMfGhxHRIzPfoBgDcQOwvo0Zlb4DnA8cExHrU6S+w4ANASKiL/D/yvX2zsx/1qme6kZs0HQhEbF2OXU0mfkq8AHw+8w8F7gbWJ7iQ+Z2YAjFF9Q79amt6qlsyB4MHAWMAq4ClqWYhnydzPxuOWamZ9moOT0zn6tbhdVojqJoxDwCnASsQZECnxURg8oTES4GdszMf9Svmt2DY2gKNmi6iDJ5OZti6mgAMvMw4IWI+AVwIbA1xVkHPwXGOA6i+6kZyDmRIrkL4MfAgxTHyDjg5jKpoekYqZ1TRN1TRGwWEXuX6ctY4D/lou0pTvX/iOK4OiQiFsjMdzLzrfrUVt2RDZouoJw072fAEZn5RET0j4jly0F5rwPfBb6ZmbcCH2fmff7a7rZ6lv+PzLwJeBzYgaJR8w/gdGARirPhlqhLDdVwImIeioHihwBHA5tTXJtnZ4q5Zn4I/Jti8O/W9all9xWd+F8js0FTcRGxIMXFvU7MzFvLcRHXAkuVv6r/RPGruz/4S7s7K+eZGRsRi5TdSUtQdDs9BMwPHA6Mpjh++lB0WaqbK8+Ce4Ai/R1O8XlyEHAMsD+wEbB0Zt6ZmV8CNszMSfWqr7ovGzQVl5lvU5xdcHQ5wdVZwN8z8+6aQZ1nAuvWzkuj7iczxwPfA+6MiFWBvwIXZ+YBFN1PC1DMQfMAcGx5bEkrAqtQjJVZMzNPAf6H4izK04AlKRIbADLz5XpUslvzygeAp213CZl5Q0R8DIwBjszM35eDOT+OiE2BzwFnZObUulZUdZeZ10XERxRT0R+ZmaeXi0YCvSm+pOYvGz8SwCXA54GXgQMion9mXgZsEBGHArsDR0TEueVgYKkuTGi6iMy8mWJ68b0iol/ZmNkbOAG4tExqpBmPlQXKso8z8xbgFzZmFBGr11zS4G3gQ4qU5kzgm+X8MmTm7ylONFjXxkz9GNAUbNB0IZl5G/ADYGREfBfYB9gnM1+sb83UaGqOlYfLcVhN5X4pdXMRsRBF2ntDROwIrE1xZuRUiu+0i4DdI2IPgMwcW04TIdWVXU5dTGbeFBE9KeYVWTMz/1XvOqkxlcfK3MDtETGU8lqm9a6X6iszJ5QzAd9OcbmLlSkav68CC2fmhRHRB9g2Iq4F3vO4USOwQdMFZeb1ZbeTv7bVqsy8JiLu8Ow31crMOyPiK8C5wFrAjhRjZRaPiMuAK4ErM/PdOlZTVGPCu85ig6aLsjGjtsrM9+pdBzWezLyj7Lq+m2JW8f+NiOUy80OKMTVSQ7FBI0maqcy8sZxcelREfDEzn4dixmm7mRpHo09411ls0EiSWlQ2aubCsVZqcDZoJEmtcqxVgzOgATxtW5LUBo61UqMzoZEkqcIMaAomNJIkqfJMaCRJqjDnoSmY0EgNICI+jogxEfHPiLg8Ivp+htc6v5yynoj4c0Ss0sq6G0fEBu3YxgsRMaCt5TOsM1tjMSLi2Ig4bHbrKKl7sUEjNYYpmTkkM1elmLRs/9qFEdGuNDUzv5OZT7ayysbAbDdoJDWK6NT/GpkNGqnxjARWKNOTkeX1cp6MiJ4RcWJEjIqIJyLif6CY5CwiTouIZyLidmCRpheKiLvLuUOIiC0i4tGIeDwi7oiIZSkaTj8o06ENI2LhiLiy3MaoiPhi+dyFIuLWiPhXRPyZNoxDjIi/R8Qj5XP2m2HZSWX5HRGxcFm2fETcXD5nZESsNEf2pqRuwTE0UgMpk5gtgZvLorWAVTPz+bJRMCkz14mI3sB9EXErsCYwGFgFWBR4kuIaPLWvuzDwJ2Cj8rUWzMy3I+IsiosL/q5c72LgpMy8NyKWBm6huDjhMcC9mXl8RGwN7NuGt7NPuY0+FDPNXpmZE4DPAaMz8wcRcXT52gcBZwP7Z+ZzETEMOAPYtB27UVI3ZINGagx9ImJMeX8kcA5FV9DDTdPNA5sDqzeNjwEWAAYBGwGXZObHwGsRcedMXn89YETTa2Xm2y3UYzNglfh0lOH8ETFvuY2vl8+9ISLeacN7OjgivlbeX6qs6wTgE+DSsvxC4KpyGxsAl9dsu3cbtiF1a4GDgpvYoJEaw5TMHFJbUH6xT64tAr6XmbfMsN5Wc7AePYD1MvODmdSlzSJiY4rG0fqZ+X5E3A3M08LqWW534oz7QJLayjE0UnXcAny3vK4OEbFiRHwOGAHsUo6xWRzYZCbPfRDYKCKWK5+7YFn+LjBfzXq3At9rehARQ8q7I4Ddy7Itgf6zqOsCwDtlY2YlioSoSQ+gKWXanaIr6z/A8xGxU7mNiIg1ZrENSWpmg0aqjj9TjI95NCL+CfwvRcp6NfBcuewvwAMzPjEz3wL2o+jeeZxPu3yuA77WNCgYOBgYWg46fpJPz7Y6jqJB9C+KrqeXZlHXm4FeEfEU8GuKBlWTycC65XvYFDi+LN8D2Les37+A7duwTyQJgPCiqZIkVdOaaw3Nu+57qNO2179vr0cyc2inbXA2mNBIkqTKc1CwJEkV1ugT3nUWExpJklR5JjSSJFVVOA9NExMaSZJUeSY0kiRVVNCGC6t1EyY0kiSp8mzQSJKkyrPLSZKkKrPPCTChkSRJXYAJjSRJFebEegUTGkmSVHkmNJIkVZgT6xVMaCRJUuWZ0EiSVGEGNAUTGkmSVHkmNJIkVZkRDWBCI0mSugATGkmSKsx5aAomNJIkqfJs0EiSpMqzy0mSpIoKnFiviQmNJEmqvMjMetdBkiS1Q0TcDAzoxE2Oz8wtOnF7bWaDRpIkVZ5dTpIkqfJs0EiSpMqzQSNJkirPBo0kSao8GzSSJKny/j9hk0qtU3CEhgAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"Wow. We got a cross validation mlogloss of 0.451 using count vectorizer inplace of tfidf vectorizer. LB score using this model is 0.468. Also the confusion matrix looks much better than the previous one.\n\n** Naive Bayes on Character Count Vectorizer:**\n\nOne idea from the \"data eyeballing\" is that counting the special charaters might help. Instead of just counting the special characters, we can use the count vectorizer at character level to get some features. Again we can run Multinomial NB on top of it.","metadata":{"_uuid":"fb099e48fde5f3960bc2409d323749e855b2e269","_cell_guid":"5a5d31ea-278c-432a-9840-ca9209410ff7"}},{"cell_type":"code","source":"### Fit transform the tfidf vectorizer ###\ntfidf_vec = CountVectorizer(ngram_range=(1,7), analyzer='char')\ntfidf_vec.fit(train_df['text'].values.tolist() + test_df['text'].values.tolist())\ntrain_tfidf = tfidf_vec.transform(train_df['text'].values.tolist())\ntest_tfidf = tfidf_vec.transform(test_df['text'].values.tolist())\n\ncv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nkf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_tfidf[dev_index], train_tfidf[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runMNB(dev_X, dev_y, val_X, val_y, test_tfidf)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\nprint(\"Mean cv score : \", np.mean(cv_scores))\npred_full_test = pred_full_test / 5.\n\n# add the predictions as new features #\ntrain_df[\"nb_cvec_char_eap\"] = pred_train[:,0]\ntrain_df[\"nb_cvec_char_hpl\"] = pred_train[:,1]\ntrain_df[\"nb_cvec_char_mws\"] = pred_train[:,2]\ntest_df[\"nb_cvec_char_eap\"] = pred_full_test[:,0]\ntest_df[\"nb_cvec_char_hpl\"] = pred_full_test[:,1]\ntest_df[\"nb_cvec_char_mws\"] = pred_full_test[:,2]","metadata":{"_uuid":"d26ba4563285c10418479465e10de172f527f2a5","_cell_guid":"aed10880-e717-4a04-85b9-a77f8f58aee9","execution":{"iopub.status.busy":"2023-01-12T09:34:32.246458Z","iopub.execute_input":"2023-01-12T09:34:32.247289Z","iopub.status.idle":"2023-01-12T09:35:30.397960Z","shell.execute_reply.started":"2023-01-12T09:34:32.247243Z","shell.execute_reply":"2023-01-12T09:35:30.396802Z"},"trusted":true},"execution_count":23,"outputs":[{"name":"stdout","text":"Mean cv score : 3.7507639226818825\n","output_type":"stream"}]},{"cell_type":"markdown","source":"The cross val score is very high and is 3.75. But this might add some different information than word level features and so let us use this for the final model as well.\n\n**Naive Bayes on Character Tfidf Vectorizer:**\n\nLet us also get the naive bayes predictions on the character tfidf vectorizer.","metadata":{"_uuid":"227e565a4a8a6aa04dcfb648688bf0deb0853910","_cell_guid":"55e95b9a-807b-4e99-bf95-069a11e07a64"}},{"cell_type":"code","source":"### Fit transform the tfidf vectorizer ###\ntfidf_vec = TfidfVectorizer(ngram_range=(1,5), analyzer='char')\nfull_tfidf = tfidf_vec.fit_transform(train_df['text'].values.tolist() + test_df['text'].values.tolist())\ntrain_tfidf = tfidf_vec.transform(train_df['text'].values.tolist())\ntest_tfidf = tfidf_vec.transform(test_df['text'].values.tolist())\n\ncv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nkf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_tfidf[dev_index], train_tfidf[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runMNB(dev_X, dev_y, val_X, val_y, test_tfidf)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\nprint(\"Mean cv score : \", np.mean(cv_scores))\npred_full_test = pred_full_test / 5.\n\n# add the predictions as new features #\ntrain_df[\"nb_tfidf_char_eap\"] = pred_train[:,0]\ntrain_df[\"nb_tfidf_char_hpl\"] = pred_train[:,1]\ntrain_df[\"nb_tfidf_char_mws\"] = pred_train[:,2]\ntest_df[\"nb_tfidf_char_eap\"] = pred_full_test[:,0]\ntest_df[\"nb_tfidf_char_hpl\"] = pred_full_test[:,1]\ntest_df[\"nb_tfidf_char_mws\"] = pred_full_test[:,2]","metadata":{"_uuid":"bfeaa62ae81e2af3fdb595422f08f560506c1f42","_cell_guid":"a15097b8-f9c7-4b3b-815e-a8f4a5f2558e","execution":{"iopub.status.busy":"2023-01-12T09:35:30.399228Z","iopub.execute_input":"2023-01-12T09:35:30.399534Z","iopub.status.idle":"2023-01-12T09:36:02.526391Z","shell.execute_reply.started":"2023-01-12T09:35:30.399507Z","shell.execute_reply":"2023-01-12T09:36:02.525193Z"},"trusted":true},"execution_count":24,"outputs":[{"name":"stdout","text":"Mean cv score : 0.790415258947421\n","output_type":"stream"}]},{"cell_type":"code","source":"tfidf_vec = TfidfVectorizer(ngram_range=(1,5), analyzer='char')\nfull_tfidf = tfidf_vec.fit_transform(train_df['text'].values.tolist() + test_df['text'].values.tolist())\ntrain_tfidf = tfidf_vec.transform(train_df['text'].values.tolist())\ntest_tfidf = tfidf_vec.transform(test_df['text'].values.tolist())","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:02.527892Z","iopub.execute_input":"2023-01-12T09:36:02.528324Z","iopub.status.idle":"2023-01-12T09:36:34.123004Z","shell.execute_reply.started":"2023-01-12T09:36:02.528281Z","shell.execute_reply":"2023-01-12T09:36:34.121555Z"},"trusted":true},"execution_count":25,"outputs":[]},{"cell_type":"markdown","source":"**SVD on Character TFIDF:**\n\nWe could also create svd features on character tfidf features and used them for modeling.","metadata":{"_uuid":"af6e2880a16f96304bd6556ed67d9eac5491a3bc","_cell_guid":"c807a434-c6d8-4a6c-a592-76c70bd043fe"}},{"cell_type":"code","source":"n_comp = 20\nsvd_obj = TruncatedSVD(n_components=n_comp, algorithm='arpack')\nsvd_obj.fit(full_tfidf)\ntrain_svd = pd.DataFrame(svd_obj.transform(train_tfidf))\ntest_svd = pd.DataFrame(svd_obj.transform(test_tfidf))\n \ntrain_svd.columns = ['svd_char_'+str(i) for i in range(n_comp)]\ntest_svd.columns = ['svd_char_'+str(i) for i in range(n_comp)]\ntrain_df = pd.concat([train_df, train_svd], axis=1)\ntest_df = pd.concat([test_df, test_svd], axis=1)\ndel full_tfidf, train_tfidf, test_tfidf, train_svd, test_svd","metadata":{"_uuid":"bd34ef340648d0f9f000a916d8cbcb258dc22838","_cell_guid":"a764f24e-b48d-4460-9f1b-d242599049bd","execution":{"iopub.status.busy":"2023-01-12T09:36:34.124340Z","iopub.execute_input":"2023-01-12T09:36:34.124679Z","iopub.status.idle":"2023-01-12T09:36:53.139506Z","shell.execute_reply.started":"2023-01-12T09:36:34.124650Z","shell.execute_reply":"2023-01-12T09:36:53.138253Z"},"trusted":true},"execution_count":26,"outputs":[]},{"cell_type":"markdown","source":"**Add BERT Embeddings**","metadata":{}},{"cell_type":"code","source":"import torch\nbert_prediction_df = torch.load(\"/kaggle/input/bertpredictions/predictions.th\")\nbert_test_df = torch.load(\"/kaggle/input/bertpredictions/bert-test-pred\")\nbert_prediction_df[\"id\"] = bert_prediction_df[\"id\"].astype(object)\n\nbert_prediction_df.info()\n#bert_prediction_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.140877Z","iopub.execute_input":"2023-01-12T09:36:53.141211Z","iopub.status.idle":"2023-01-12T09:36:53.624003Z","shell.execute_reply.started":"2023-01-12T09:36:53.141181Z","shell.execute_reply":"2023-01-12T09:36:53.622791Z"},"trusted":true},"execution_count":27,"outputs":[{"name":"stdout","text":"<class 'pandas.core.frame.DataFrame'>\nInt64Index: 19427 entries, 0 to 19426\nData columns (total 5 columns):\n # Column Non-Null Count Dtype \n--- ------ -------------- ----- \n 0 id 19427 non-null object \n 1 pred 19427 non-null object \n 2 eap 19427 non-null float64\n 3 hpl 19427 non-null float64\n 4 mws 19427 non-null float64\ndtypes: float64(3), object(2)\nmemory usage: 910.6+ KB\n","output_type":"stream"}]},{"cell_type":"code","source":"train_df = train_df[train_df['text'].str.len() < 510]","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.631120Z","iopub.execute_input":"2023-01-12T09:36:53.631772Z","iopub.status.idle":"2023-01-12T09:36:53.659599Z","shell.execute_reply.started":"2023-01-12T09:36:53.631719Z","shell.execute_reply":"2023-01-12T09:36:53.658356Z"},"trusted":true},"execution_count":28,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nencoder = LabelEncoder()\n\ntrain_df[\"bert_pred\"] = encoder.fit_transform(bert_prediction_df[\"pred\"])\ntrain_df[\"bert_eap\"] = bert_prediction_df[\"eap\"]\ntrain_df[\"bert_hpl\"] = bert_prediction_df[\"hpl\"]\ntrain_df[\"bert_mws\"] = bert_prediction_df[\"mws\"]\n\ntest_df[\"bert_pred\"] = encoder.fit_transform(bert_test_df[\"pred\"])\ntest_df[\"bert_eap\"] = bert_test_df[\"eap\"]\ntest_df[\"bert_hpl\"] = bert_test_df[\"hpl\"]\ntest_df[\"bert_mws\"] = bert_test_df[\"mws\"]","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.661144Z","iopub.execute_input":"2023-01-12T09:36:53.661613Z","iopub.status.idle":"2023-01-12T09:36:53.685439Z","shell.execute_reply.started":"2023-01-12T09:36:53.661579Z","shell.execute_reply":"2023-01-12T09:36:53.684304Z"},"trusted":true},"execution_count":29,"outputs":[]},{"cell_type":"code","source":"valid_ratio=0.2\nred_train_df, red_val_df = train_test_split(\n train_df,\n test_size=valid_ratio,\n stratify=train_df[\"author\"],\n random_state=22)\nred_train_df.reset_index(inplace=True, drop=True)\nred_val_df.reset_index(inplace=True, drop=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.687418Z","iopub.execute_input":"2023-01-12T09:36:53.687893Z","iopub.status.idle":"2023-01-12T09:36:53.729945Z","shell.execute_reply.started":"2023-01-12T09:36:53.687849Z","shell.execute_reply":"2023-01-12T09:36:53.728731Z"},"trusted":true},"execution_count":30,"outputs":[]},{"cell_type":"code","source":"red_train_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.732505Z","iopub.execute_input":"2023-01-12T09:36:53.733412Z","iopub.status.idle":"2023-01-12T09:36:53.758095Z","shell.execute_reply.started":"2023-01-12T09:36:53.733362Z","shell.execute_reply":"2023-01-12T09:36:53.756836Z"},"trusted":true},"execution_count":31,"outputs":[{"execution_count":31,"output_type":"execute_result","data":{"text/plain":" id text author \\\n0 id06487 A single step cannot be taken without M. Beauv... EAP \n1 id21846 She urged the old story of decorum that bug be... EAP \n\n num_words num_unique_words num_chars num_stopwords num_punctuations \\\n0 19 18 102 6 8 \n1 25 23 133 11 1 \n\n num_words_upper num_words_title ... svd_char_14 svd_char_15 \\\n0 2 3 ... -0.009168 -0.045902 \n1 0 1 ... -0.044453 -0.032089 \n\n svd_char_16 svd_char_17 svd_char_18 svd_char_19 bert_pred bert_eap \\\n0 0.011645 0.011462 -0.019926 0.007369 0 -2.294242 \n1 -0.006036 0.044267 0.025082 0.004226 0 -3.501722 \n\n bert_hpl bert_mws \n0 4.403566 -2.858669 \n1 6.084036 -3.688547 \n\n[2 rows x 64 columns]","text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>id</th>\n <th>text</th>\n <th>author</th>\n <th>num_words</th>\n <th>num_unique_words</th>\n <th>num_chars</th>\n <th>num_stopwords</th>\n <th>num_punctuations</th>\n <th>num_words_upper</th>\n <th>num_words_title</th>\n <th>...</th>\n <th>svd_char_14</th>\n <th>svd_char_15</th>\n <th>svd_char_16</th>\n <th>svd_char_17</th>\n <th>svd_char_18</th>\n <th>svd_char_19</th>\n <th>bert_pred</th>\n <th>bert_eap</th>\n <th>bert_hpl</th>\n <th>bert_mws</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>id06487</td>\n <td>A single step cannot be taken without M. Beauv...</td>\n <td>EAP</td>\n <td>19</td>\n <td>18</td>\n <td>102</td>\n <td>6</td>\n <td>8</td>\n <td>2</td>\n <td>3</td>\n <td>...</td>\n <td>-0.009168</td>\n <td>-0.045902</td>\n <td>0.011645</td>\n <td>0.011462</td>\n <td>-0.019926</td>\n <td>0.007369</td>\n <td>0</td>\n <td>-2.294242</td>\n <td>4.403566</td>\n <td>-2.858669</td>\n </tr>\n <tr>\n <th>1</th>\n <td>id21846</td>\n <td>She urged the old story of decorum that bug be...</td>\n <td>EAP</td>\n <td>25</td>\n <td>23</td>\n <td>133</td>\n <td>11</td>\n <td>1</td>\n <td>0</td>\n <td>1</td>\n <td>...</td>\n <td>-0.044453</td>\n <td>-0.032089</td>\n <td>-0.006036</td>\n <td>0.044267</td>\n <td>0.025082</td>\n <td>0.004226</td>\n <td>0</td>\n <td>-3.501722</td>\n <td>6.084036</td>\n <td>-3.688547</td>\n </tr>\n </tbody>\n</table>\n<p>2 rows × 64 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"**XGBoost model:**\n\nNow with these new variables, we can re-run the xgboost model and evaluate the results.","metadata":{"_uuid":"0577ac7e16cd158944d1ee3c27ca92c588e2d10a","_cell_guid":"365abbc9-d630-4075-a2aa-c025d146d6aa"}},{"cell_type":"code","source":"cols_to_drop = ['id', 'text', \"bert_pred\", \"bert_eap\", \"bert_hpl\", \"bert_mws\"]\ntrain_X = red_train_df.drop(cols_to_drop+['author'], axis=1)\ntrain_y = red_train_df['author'].map(author_mapping_dict)\nval_X = red_val_df.drop(cols_to_drop+['author'], axis=1)\nval_y = red_val_df['author'].map(author_mapping_dict)\ntest_X = test_df.drop(cols_to_drop, axis=1)\n\n\npred_val_y, pred_test_y, model = runXGB(train_X, train_y, val_X, val_y, test_X, seed_val=0, colsample=0.7)\nprint(f\"Log loss: {metrics.log_loss(val_y, pred_val_y):.5f}\")\n\nout_df = pd.DataFrame(pred_test_y)\nout_df.columns = ['EAP', 'HPL', 'MWS']\nout_df.insert(0, 'id', test_id)\nout_df.to_csv(\"sub_fe.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:36:53.759963Z","iopub.execute_input":"2023-01-12T09:36:53.760416Z","iopub.status.idle":"2023-01-12T09:37:31.132523Z","shell.execute_reply.started":"2023-01-12T09:36:53.760371Z","shell.execute_reply":"2023-01-12T09:37:31.131301Z"},"trusted":true},"execution_count":32,"outputs":[{"name":"stdout","text":"[09:36:53] WARNING: ../src/learner.cc:627: \nParameters: { \"silent\" } might not be used.\n\n This could be a false alarm, with some parameters getting used by language bindings but\n then being mistakenly passed down to XGBoost core, or some parameter actually being used\n but getting flagged wrongly here. Please open an issue if you find any such cases.\n\n\n[0]\ttrain-mlogloss:1.00102\ttest-mlogloss:1.00240\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/xgboost/core.py:571: FutureWarning: Pass `evals` as keyword args. Passing these as positional arguments will be considered as error in future releases.\n format(\", \".join(args_msg)), FutureWarning\n","output_type":"stream"},{"name":"stdout","text":"[20]\ttrain-mlogloss:0.41136\ttest-mlogloss:0.42684\n[40]\ttrain-mlogloss:0.33982\ttest-mlogloss:0.36394\n[60]\ttrain-mlogloss:0.31403\ttest-mlogloss:0.34373\n[80]\ttrain-mlogloss:0.29652\ttest-mlogloss:0.33378\n[100]\ttrain-mlogloss:0.28356\ttest-mlogloss:0.32858\n[120]\ttrain-mlogloss:0.27255\ttest-mlogloss:0.32375\n[140]\ttrain-mlogloss:0.26260\ttest-mlogloss:0.32082\n[160]\ttrain-mlogloss:0.25404\ttest-mlogloss:0.31885\n[180]\ttrain-mlogloss:0.24616\ttest-mlogloss:0.31781\n[200]\ttrain-mlogloss:0.23871\ttest-mlogloss:0.31673\n[220]\ttrain-mlogloss:0.23167\ttest-mlogloss:0.31584\n[240]\ttrain-mlogloss:0.22519\ttest-mlogloss:0.31409\n[260]\ttrain-mlogloss:0.21900\ttest-mlogloss:0.31324\n[280]\ttrain-mlogloss:0.21305\ttest-mlogloss:0.31240\n[300]\ttrain-mlogloss:0.20705\ttest-mlogloss:0.31243\n[320]\ttrain-mlogloss:0.20150\ttest-mlogloss:0.31272\n[340]\ttrain-mlogloss:0.19594\ttest-mlogloss:0.31188\n[360]\ttrain-mlogloss:0.19098\ttest-mlogloss:0.31145\n[380]\ttrain-mlogloss:0.18627\ttest-mlogloss:0.31167\n[400]\ttrain-mlogloss:0.18156\ttest-mlogloss:0.31177\n[419]\ttrain-mlogloss:0.17691\ttest-mlogloss:0.31204\nLog loss: 0.31118\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/xgboost/core.py:94: UserWarning: ntree_limit is deprecated, use `iteration_range` or model slicing instead.\n UserWarning\n","output_type":"stream"}]},{"cell_type":"code","source":"cols_to_drop = ['id', 'text']#, \"bert_pred\", \"bert_eap\", \"bert_hpl\", \"bert_mws\"]\ntrain_X = train_df.drop(cols_to_drop+['author'], axis=1)\ntest_X = test_df.drop(cols_to_drop, axis=1)\n\nkf = model_selection.KFold(n_splits=5, shuffle=True, random_state=2017)\ncv_scores = []\npred_full_test = 0\npred_train = np.zeros([train_df.shape[0], 3])\nfor dev_index, val_index in kf.split(train_X):\n dev_X, val_X = train_X.loc[dev_index], train_X.loc[val_index]\n dev_y, val_y = train_y[dev_index], train_y[val_index]\n pred_val_y, pred_test_y, model = runXGB(dev_X, dev_y, val_X, val_y, test_X, seed_val=0, colsample=0.7)\n print(pred_test_y)\n pred_full_test = pred_full_test + pred_test_y\n pred_train[val_index,:] = pred_val_y\n print(val_y)\n print(pred_val_y)\n cv_scores.append(metrics.log_loss(val_y, pred_val_y))\n break\nprint(\"cv scores : \", cv_scores)\n\nout_df = pd.DataFrame(pred_full_test)\nout_df.columns = ['EAP', 'HPL', 'MWS']\nout_df.insert(0, 'id', test_id)\nout_df.to_csv(\"sub_fe.csv\", index=False)","metadata":{"_uuid":"633eb97dc8b15a7959b962c20222697590071af7","_cell_guid":"7091aede-e589-45d9-9a85-1560af442725","execution":{"iopub.status.busy":"2023-01-12T09:37:31.134294Z","iopub.execute_input":"2023-01-12T09:37:31.134640Z","iopub.status.idle":"2023-01-12T09:37:31.361678Z","shell.execute_reply.started":"2023-01-12T09:37:31.134608Z","shell.execute_reply":"2023-01-12T09:37:31.359697Z"},"trusted":true},"execution_count":33,"outputs":[{"traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_27/3568347866.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0mpred_train\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtrain_df\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mdev_index\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_index\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mkf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrain_X\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 10\u001b[0;31m \u001b[0mdev_X\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_X\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_X\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mdev_index\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_X\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mval_index\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 11\u001b[0m \u001b[0mdev_y\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_y\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_y\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mdev_index\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_y\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mval_index\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0mpred_val_y\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpred_test_y\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrunXGB\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdev_X\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdev_y\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_X\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_y\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_X\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mseed_val\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcolsample\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 929\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 930\u001b[0m \u001b[0mmaybe_callable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_if_callable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 931\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmaybe_callable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 932\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 933\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_is_scalar_access\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_axis\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m 1151\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Cannot index with multidimensional key\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1152\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1153\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_iterable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1154\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1155\u001b[0m \u001b[0;31m# nested tuple slicing\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_iterable\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m 1091\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1092\u001b[0m \u001b[0;31m# A collection of keys\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1093\u001b[0;31m \u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_listlike_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1094\u001b[0m return self.obj._reindex_with_indexers(\n\u001b[1;32m 1095\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mallow_dups\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_get_listlike_indexer\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m 1312\u001b[0m \u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_indexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reindex_non_unique\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkeyarr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1313\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1314\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_read_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1315\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1316\u001b[0m if needs_i8_conversion(ax.dtype) or isinstance(\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_validate_read_indexer\u001b[0;34m(self, key, indexer, axis)\u001b[0m\n\u001b[1;32m 1375\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1376\u001b[0m \u001b[0mnot_found\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mensure_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mmissing_mask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnonzero\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munique\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1377\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"{not_found} not in index\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1378\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1379\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mKeyError\u001b[0m: '[26, 133, 153, 496, 503, 698, 788, 818, 1222, 1320, 1456, 1682, 1835, 1991, 2036, 2248, 2285, 2543, 2706, 3089, 3176, 3311, 3373, 3450, 3701, 3717, 3956, 4111, 4181, 4184, 4249, 4336, 4656, 4660, 4664, 4690, 4926, 5111, 5335, 5347, 5553, 5574, 5765, 5788, 6055, 6373, 6869, 6922, 6927, 6970, 6993, 7056, 7121, 7301, 7732, 7903, 7929, 8026, 8270, 8328, 8500, 9348, 9406, 9761, 10183, 10268, 10269, 10594, 10814, 10927, 10939, 11008, 11166, 11279, 11525, 11616, 11916, 12010, 12222, 12285, 12288, 12346, 12417, 12527, 12700, 12722, 12757, 12854, 12952, 12992, 13170, 13225, 13452, 13733, 14054, 14079, 14197, 14464, 14532, 14582, 14692, 14699, 14914, 15077, 15090, 15175, 15207, 15298, 15450, 15600, 16000, 16881, 17010, 17423, 17578, 18003, 18107, 18387, 18954, 19064, 19219, 19254, 19255] not in index'"],"ename":"KeyError","evalue":"'[26, 133, 153, 496, 503, 698, 788, 818, 1222, 1320, 1456, 1682, 1835, 1991, 2036, 2248, 2285, 2543, 2706, 3089, 3176, 3311, 3373, 3450, 3701, 3717, 3956, 4111, 4181, 4184, 4249, 4336, 4656, 4660, 4664, 4690, 4926, 5111, 5335, 5347, 5553, 5574, 5765, 5788, 6055, 6373, 6869, 6922, 6927, 6970, 6993, 7056, 7121, 7301, 7732, 7903, 7929, 8026, 8270, 8328, 8500, 9348, 9406, 9761, 10183, 10268, 10269, 10594, 10814, 10927, 10939, 11008, 11166, 11279, 11525, 11616, 11916, 12010, 12222, 12285, 12288, 12346, 12417, 12527, 12700, 12722, 12757, 12854, 12952, 12992, 13170, 13225, 13452, 13733, 14054, 14079, 14197, 14464, 14532, 14582, 14692, 14699, 14914, 15077, 15090, 15175, 15207, 15298, 15450, 15600, 16000, 16881, 17010, 17423, 17578, 18003, 18107, 18387, 18954, 19064, 19219, 19254, 19255] not in index'","output_type":"error"}]},{"cell_type":"markdown","source":"**This has a val score of 0.3055 and LB score of 0.32xx** Running it on all the folds might give a better score. Now let us check the important variables again.","metadata":{"_uuid":"4512493a6527cc244b9c47eb1ed9ff0dbce5eb5a","_cell_guid":"7d93a2d6-435a-46bb-a522-c4d6b03d84f0"}},{"cell_type":"code","source":"### Plot the important variables ###\nfig, ax = plt.subplots(figsize=(12,12))\nxgb.plot_importance(model, max_num_features=50, height=0.8, ax=ax)\nplt.show()","metadata":{"_uuid":"ddefa6111b588449df9f2abbc541c0ca051174ab","_cell_guid":"ed36de2b-e0bb-42e6-bed8-bbbdeb513ba5","execution":{"iopub.status.busy":"2023-01-12T09:37:31.362875Z","iopub.status.idle":"2023-01-12T09:37:31.363285Z","shell.execute_reply.started":"2023-01-12T09:37:31.363086Z","shell.execute_reply":"2023-01-12T09:37:31.363104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Naive bayes features are the top features as expected. Now let us get the confusion matrix to see the misclassification errors.","metadata":{"_uuid":"c818e95a21559bdb4def3aed522b6616eb1a9de8","_cell_guid":"c4b258fe-ac27-4d5b-908c-eda43670fbf7"}},{"cell_type":"code","source":"cnf_matrix = confusion_matrix(val_y, np.argmax(pred_val_y,axis=1))\nnp.set_printoptions(precision=2)\n\n# Plot non-normalized confusion matrix\nplt.figure(figsize=(8,8))\nplot_confusion_matrix(cnf_matrix, classes=['EAP', 'HPL', 'MWS'],\n title='Confusion matrix of XGB, without normalization')\nplt.show()","metadata":{"_uuid":"c195596cd7dff2de98bddec1453e30d8d5511169","_cell_guid":"2f3aaa7b-396c-426b-89b3-1e66385946fd","execution":{"iopub.status.busy":"2023-01-12T09:37:31.364641Z","iopub.status.idle":"2023-01-12T09:37:31.365052Z","shell.execute_reply.started":"2023-01-12T09:37:31.364860Z","shell.execute_reply":"2023-01-12T09:37:31.364879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import classification_report, accuracy_score\n\nprint(accuracy_score(val_y, np.argmax(pred_val_y, axis=1)))\nprint(classification_report(val_y, np.argmax(pred_val_y,axis=1)))","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:37:32.189523Z","iopub.status.idle":"2023-01-12T09:37:32.190332Z","shell.execute_reply.started":"2023-01-12T09:37:32.190083Z","shell.execute_reply":"2023-01-12T09:37:32.190106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"EAP and MWS seem to be misclassified more often than others. We could potentially create features which improves the predictions for this pair.\n\n**Next steps in this FE notebook:**\n* Using word embedding based features \n* Other meta features if any\n* Sentiment on the sentences","metadata":{"_uuid":"5e40512bdaf05ffbd7a1914735c46784d7c33329","_cell_guid":"855a4e2b-d377-4a91-bac9-4202119c8666"}},{"cell_type":"markdown","source":"**Ideas for further improvements:**\n* Parameter tuning for tfidf and count vectorizer\n* Parameter tuning for naive bayes and XGB models\n* Ensembling / Stacking with other models","metadata":{"_uuid":"117205aae679d7b6481df1c446cd42f41cf924de","_cell_guid":"7a43b172-b6b6-4fd2-89d6-7300672cc462"}},{"cell_type":"code","source":"out_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:37:31.368596Z","iopub.status.idle":"2023-01-12T09:37:31.369030Z","shell.execute_reply.started":"2023-01-12T09:37:31.368817Z","shell.execute_reply":"2023-01-12T09:37:31.368836Z"},"trusted":true},"execution_count":null,"outputs":[]}]}
\ No newline at end of file
diff --git a/spooky-bert.ipynb b/spooky-bert.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..905dd100df3986a111ee16c7f4aece86e58c3082
--- /dev/null
+++ b/spooky-bert.ipynb
@@ -0,0 +1 @@
+{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"#!pip install transformers\n#!pip install sentence_transformers\n!pip install captum\n#!pip install GPUtil","metadata":{"execution":{"iopub.status.busy":"2023-01-13T11:57:51.342024Z","iopub.status.idle":"2023-01-13T11:57:51.342387Z","shell.execute_reply.started":"2023-01-13T11:57:51.342214Z","shell.execute_reply":"2023-01-13T11:57:51.342231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nfrom torch import nn\nimport torch.utils as tutils\nimport pandas as pd\nimport numpy as np\nfrom tqdm import tqdm\nfrom transformers import AutoModel, AutoModelForMultipleChoice, AutoTokenizer, pipeline\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.dummy import DummyClassifier\nfrom sklearn.metrics import classification_report\nfrom torch.utils.data import TensorDataset, DataLoader\nfrom torch.utils.data.distributed import DistributedSampler\nfrom torch.multiprocessing import Process\n\nimport matplotlib.pyplot as plt\n\nimport logging\nimport os\nlogging.basicConfig(level='ERROR')\nos.environ[\"WANDB_DISABLED\"] = \"true\"","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:23:58.362824Z","iopub.execute_input":"2023-01-15T15:23:58.363418Z","iopub.status.idle":"2023-01-15T15:24:15.552973Z","shell.execute_reply.started":"2023-01-15T15:23:58.363373Z","shell.execute_reply":"2023-01-15T15:24:15.552005Z"},"trusted":true},"execution_count":2,"outputs":[]},{"cell_type":"code","source":"device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:24:15.554946Z","iopub.execute_input":"2023-01-15T15:24:15.555561Z","iopub.status.idle":"2023-01-15T15:24:15.672597Z","shell.execute_reply.started":"2023-01-15T15:24:15.555524Z","shell.execute_reply":"2023-01-15T15:24:15.671483Z"},"trusted":true},"execution_count":3,"outputs":[]},{"cell_type":"markdown","source":"# Create the datasets","metadata":{}},{"cell_type":"code","source":"all_labeled = pd.read_csv('/kaggle/input/spooky-author-identification/train.zip', compression='zip')\nall_labeled = all_labeled[all_labeled['text'].str.len() <= 510]\n\ntrain, valid = train_test_split(all_labeled, test_size=0.1, stratify=all_labeled[\"author\"], random_state=22)\ntrain.reset_index(inplace=True)\nvalid.reset_index(inplace=True)\ntest = pd.read_csv('/kaggle/input/spooky-author-identification/test.zip', compression='zip')\ntrain.head(5)","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:24:15.674335Z","iopub.execute_input":"2023-01-15T15:24:15.675106Z","iopub.status.idle":"2023-01-15T15:24:15.923859Z","shell.execute_reply.started":"2023-01-15T15:24:15.675058Z","shell.execute_reply":"2023-01-15T15:24:15.922673Z"},"trusted":true},"execution_count":4,"outputs":[{"execution_count":4,"output_type":"execute_result","data":{"text/plain":" index id text author\n0 4764 id21166 Beasts balk at the critters hosses wuss'n mule... HPL\n1 2801 id25058 All around were horror, and thick gloom, and a... EAP\n2 1928 id14261 And of such passion I could form no estimate, ... EAP\n3 6460 id20486 So you are busy delving into the shady past of... HPL\n4 17983 id04931 In the electric light the colour seemed to be ... HPL","text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>index</th>\n <th>id</th>\n <th>text</th>\n <th>author</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>4764</td>\n <td>id21166</td>\n <td>Beasts balk at the critters hosses wuss'n mule...</td>\n <td>HPL</td>\n </tr>\n <tr>\n <th>1</th>\n <td>2801</td>\n <td>id25058</td>\n <td>All around were horror, and thick gloom, and a...</td>\n <td>EAP</td>\n </tr>\n <tr>\n <th>2</th>\n <td>1928</td>\n <td>id14261</td>\n <td>And of such passion I could form no estimate, ...</td>\n <td>EAP</td>\n </tr>\n <tr>\n <th>3</th>\n <td>6460</td>\n <td>id20486</td>\n <td>So you are busy delving into the shady past of...</td>\n <td>HPL</td>\n </tr>\n <tr>\n <th>4</th>\n <td>17983</td>\n <td>id04931</td>\n <td>In the electric light the colour seemed to be ...</td>\n <td>HPL</td>\n </tr>\n </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"all_labeled[\"author\"].sort_values().hist()\n#plt.show()\nplt.title(\"Excerpt count for each author\")\nplt.savefig('class_distribution.pdf', format='pdf')","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:24:15.926176Z","iopub.execute_input":"2023-01-15T15:24:15.926644Z","iopub.status.idle":"2023-01-15T15:24:16.552269Z","shell.execute_reply.started":"2023-01-15T15:24:15.926608Z","shell.execute_reply":"2023-01-15T15:24:16.551162Z"},"trusted":true},"execution_count":5,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcnklEQVR4nO3de5QV5Z3u8e8T8EIwEdRkxwAjzEBMNEZjetTcZrUhQSSZ4Dor3oYoGBIyZ3HmaBYZB7NcQ8bLObpWGKImMcNEAiaMhDHjgaNODAftyUkMeA+o6KFjcADFCyDamsu08zt/1NtQtL3p3Ze92+73+ay1V1e99VbVW3tXP7X3W7VrKyIwM7M8vGWgG2BmZo3j0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD36yHJFUk/UzSK5IWDnR7qpEUkiYO0LqXSrpqINZtB+bQH6IkbZH0W0ltpce3BrpdZQMZStVImiXp591UmwO8CLw9IuY1oFlvajU+Z/YmMXygG2B19ecR8X8GYsWShkXE6wOx7gY4Bng8evHNRknDI6K9Dm0akiQJUET850C3ZajwO/0MSbpR0o9L49dKWqvCMElfk/Tr1H3xoKRxqd57Ja2RtEvSk5LOKS1jaVrunZJeBU5PZd9N87wi6d8kHZPq/yzN+qv0KeTcKm39kqRNaf7HJZ2cyt8nqUXSS5Iek/TZ0jwtkr5YGt/vnWj6hPGXkjan+b+dtv19wHeBD6c2vdRFe5YCM4FLU51PSjpE0jclPZMe35R0SKrfLGmbpL+RtAP4fpXt/ELazt2S7up4ntK06yRtlfRyej0+XppW9fVKPtl5O6us/xRJv0z1npX0LUkHp2nj03M2vFS/RdIXu3nORku6I7VrvaQ/Kc3/EUn3S9qT/n6k07KvlvQL4DXgj7tqs/VSRPgxBB/AFuCTVaa9Ffh/wCzg4xRdFWPTtL8GNgLHAgJOBI4ERgJbgYsoPiF+MM13XJpvKbAH+CjFm4lDU9krwJ8BhwDXAT8vtSOAiQfYhrOB7cCfprZMpHiXfRDQCnwNOBj4RFrPsWm+FuCLpeXM6mK9twOjgD8CXgCmdlW3SruWAleVxq8A1gHvBN4B3AtcmaY1A+3Atek5GNHF8qan7Xlfem4vB+4tTf98eg2GA/OAHcChB3q9utvOLtrwIeC0tI7xwCbgkjRtfFrW8FL9vc9xV89Zeo52AqekZS4HVqRpRwC7gQvStPPT+JGlZf87cHyaftBA/z8NpceAN8CPOr2wRei3AS+VHl8qTT8V2AU8DZxfKn8SmN7F8s4F/m+nsn8AFqThpcDNnaYv7fhHT+OHAa8D49J4d6F/F3BxF+UfT8H3llLZLcDX0/DeQErj+4VSWu/HSuMrgfld1a3SrqXsH/q/BqaVxs8AtqThZuAPpJCusrx/BWaXxt9C8Q73mCr1dwMnHuj16m47a9h/LgFuS8Pj6V3of680Pg14Ig1fANzXqf4vgVmlZV/RyP+XnB7u3hnazoqIUaXHP3ZMiIj1wFMU7w5XluYZRxFinR0DnJo+/r+UPsbPAN5VqrO1i/n2lkVEG8WB5t01tr9aW94NbI39+3mfBsbUuFwoDhodXqM4IPXWu9P6y20pb+MLEfG7A8x/DHBd6XndRfG6jAGQ9NXU9bMnTT8cOCrNW+056lDTdkp6j6TbJe2Q9DLwP0rr6K1q6+78fMEbX7+u9iXrBw79TEmaS9Hd8AxwaWnSVuBPuphlK/BvnQ4ih0XEfy3V6erE5t7+ZUmHUXy0f6bGZlZryzPAOEnl/fePKLqCAF6l6MLqUD4wdac3t519hiK4y20pb2N3y9wKfLnTczsiIu5N/feXAucAoyNiFEU3mkrzdvUc9dSNwBPApIh4O0XXWcc6Xk1/qz2nPX3OOj9fsP/r15tlWo0c+hmS9B7gKoq+4gsoTkqelCZ/D7hS0qR0cvMDko6k6Bt+j6QLJB2UHn+aTuQdyDRJH0snBa8E1kVEx7u45zjwSbrvAV+V9KHUlonpBOd6ineOl6Z2NAN/DqxI8z0C/BdJb1VxSejsGp+ajjaN7TiJWaNbgMslvUPSUcDfAj/swfzfBS6TdDyApMMlnZ2mvY3inMALwHBJfwu8vTRvtderp94GvAy0SXovsPdgHhEvUATy59OJ4y+w/4Gmp8/ZnRT70l9IGq7iJP5xFPuY1ZlDf2j739r/Ov3b0hUYPwSujYhfRcRmind1P0hXnPw9RXfPTylC4CaKk4+vAFOA8yjeqe1g38nJA/knYAFFl8WHKA40Hb4OLEvdGud0njEi/hm4Oi3jFeB/AUdExB8oQv5MipPJ3wEujIgn0qyLKPrRnwOWUZxErNXdwGPADkkv1jjPVcADwAaKk6oPpbKaRMRtFM/litS18ijFtkFxXuMnFCfenwZ+x/5dH12+XrWuu+SrwF9QPM//CPyo0/QvUZw03klxgvXe0rQePWcRsRP4DMVJ6Z0Un2Q+ExG1Pt/WB0onTsz6nYrLG7dFxOUD3RYzK/idvplZRhz6ZmYZcfeOmVlG/E7fzCwjb+obrh111FExfvz4Xs//6quvMnLkyP5rkFmJ9y+rp77sXw8++OCLEfGOrqa9qUN//PjxPPDAA72ev6Wlhebm5v5rkFmJ9y+rp77sX5I6f+N5L3fvmJllpKbQl/QVFbevfVTSLZIOlTQh3S61VdKPSrdhPSSNt6bp40vLuSyVPynpjDptk5mZVdFt6EsaA/x3oCki3g8Mo/hW5rXAooiYSHHXv46vus8GdqfyRakeko5L8x0PTAW+I2lY/26OmZkdSK3dO8OBEekr/G8FnqW4h/mtafoy4Kw0PD2Nk6ZPlqRUviIifh8Rv6G4f/gpfd4CMzOrWbcnciNiu6RvUPyowW8p7vHxIPBS7PvZt23suy3qGNK9QSKiXdIeih+AGEPxQxN0Mc9ekuZQ/AYplUqFlpaWnm9V0tbW1qf5zQ7E+5fVU732r25DX9JoinfpEyh+iOOfKbpn6iIiFgOLAZqamqIvV0f46gqrJ+9fVk/12r9q6d75JPCbiHghIv4D+BeKn8QbpX2/mTmWfffC3k66h3qafjjFnfT2lncxj5mZNUAtof/vwGnp3uQCJgOPA/cAn0t1ZgKr0vDqNE6afncU93pYDZyXru6ZAEwC7uufzTAzs1rU0qe/XtKtFPcIbwcepuh+uYPi/t9XpbKb0iw3UdybvZXiHurnpeU8JmklxQGjHZgbEa/38/aYmdkB1PSN3IhYQPFDGGVP0cXVN+m3QM/uXJ6mXU3xoxgNsXH7HmbNv6NRq9tryzWfbvg6zcxq4W/kmpllxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llpNvQl3SspEdKj5clXSLpCElrJG1Of0en+pJ0vaRWSRsknVxa1sxUf7OkmdXXamZm9dBt6EfEkxFxUkScBHwIeA24DZgPrI2IScDaNA5wJsWPnk8C5gA3Akg6guInF0+l+JnFBR0HCjMza4yedu9MBn4dEU8D04FlqXwZcFYang7cHIV1wChJRwNnAGsiYldE7AbWAFP7ugFmZla7mn4YveQ84JY0XImIZ9PwDqCShscAW0vzbEtl1cr3I2kOxScEKpUKLS0tPWziPpURMO+E9l7P31t9abMNHm1tbX6trW7qtX/VHPqSDgY+C1zWeVpEhKTojwZFxGJgMUBTU1M0Nzf3elk3LF/Fwo09Pa713ZYZzQ1fpzVeS0sLfdk/zQ6kXvtXT7p3zgQeiojn0vhzqduG9Pf5VL4dGFeab2wqq1ZuZmYN0pPQP599XTsAq4GOK3BmAqtK5Remq3hOA/akbqC7gCmSRqcTuFNSmZmZNUhNfR+SRgKfAr5cKr4GWClpNvA0cE4qvxOYBrRSXOlzEUBE7JJ0JXB/qndFROzq8xaYmVnNagr9iHgVOLJT2U6Kq3k61w1gbpXlLAGW9LyZZmbWH/yNXDOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwyUlPoSxol6VZJT0jaJOnDko6QtEbS5vR3dKorSddLapW0QdLJpeXMTPU3S5pZfY1mZlYPtb7Tvw74SUS8FzgR2ATMB9ZGxCRgbRoHOBOYlB5zgBsBJB0BLABOBU4BFnQcKMzMrDG6DX1JhwN/BtwEEBF/iIiXgOnAslRtGXBWGp4O3ByFdcAoSUcDZwBrImJXROwG1gBT+3FbzMysG8NrqDMBeAH4vqQTgQeBi4FKRDyb6uwAKml4DLC1NP+2VFatfD+S5lB8QqBSqdDS0lLrtrxBZQTMO6G91/P3Vl/abINHW1ubX+shbuP2PQO27gmHD6vL/lVL6A8HTgb+KiLWS7qOfV05AERESIr+aFBELAYWAzQ1NUVzc3Ovl3XD8lUs3FjLJvavLTOaG75Oa7yWlhb6sn/am9+s+XcM2LqXTh1Zl/2rlj79bcC2iFifxm+lOAg8l7ptSH+fT9O3A+NK849NZdXKzcysQboN/YjYAWyVdGwqmgw8DqwGOq7AmQmsSsOrgQvTVTynAXtSN9BdwBRJo9MJ3CmpzMzMGqTWvo+/ApZLOhh4CriI4oCxUtJs4GngnFT3TmAa0Aq8luoSEbskXQncn+pdERG7+mUrzMysJjWFfkQ8AjR1MWlyF3UDmFtlOUuAJT1on5mZ9SN/I9fMLCMOfTOzjDj0zcwy4tA3M8tI47+5ZDZEbNy+Z0C+vLPlmk83fJ02dPidvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlpGaQl/SFkkbJT0i6YFUdoSkNZI2p7+jU7kkXS+pVdIGSSeXljMz1d8saWa19ZmZWX305J3+6RFxUkR0/GzifGBtREwC1qZxgDOBSekxB7gRioMEsAA4FTgFWNBxoDAzs8boS/fOdGBZGl4GnFUqvzkK64BRko4GzgDWRMSuiNgNrAGm9mH9ZmbWQ7XeTz+An0oK4B8iYjFQiYhn0/QdQCUNjwG2lubdlsqqle9H0hyKTwhUKhVaWlpqbOIbVUbAvBPaez1/b/WlzTZ4eP8a+gbi9e3Q1tZWl9e61tD/WERsl/ROYI2kJ8oTIyLSAaHP0gFlMUBTU1M0Nzf3elk3LF/Fwo2N/52YLTOaG75OazzvX0PfQPxIToelU0fSl/yrpqbunYjYnv4+D9xG0Sf/XOq2If19PlXfDowrzT42lVUrNzOzBuk29CWNlPS2jmFgCvAosBrouAJnJrAqDa8GLkxX8ZwG7EndQHcBUySNTidwp6QyMzNrkFo+m1aA2yR11P+niPiJpPuBlZJmA08D56T6dwLTgFbgNeAigIjYJelK4P5U74qI2NVvW2JmZt3qNvQj4ingxC7KdwKTuygPYG6VZS0BlvS8mWZm1h/8jVwzs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMlJz6EsaJulhSben8QmS1ktqlfQjSQen8kPSeGuaPr60jMtS+ZOSzuj3rTEzswPqyTv9i4FNpfFrgUURMRHYDcxO5bOB3al8UaqHpOOA84DjganAdyQN61vzzcysJ2oKfUljgU8D30vjAj4B3JqqLAPOSsPT0zhp+uRUfzqwIiJ+HxG/AVqBU/phG8zMrEbDa6z3TeBS4G1p/EjgpYhoT+PbgDFpeAywFSAi2iXtSfXHAOtKyyzPs5ekOcAcgEqlQktLS41NfKPKCJh3Qnv3FftZX9psg4f3r6FvIF7fDm1tbXV5rbsNfUmfAZ6PiAclNfd7CzqJiMXAYoCmpqZobu79Km9YvoqFG2s9rvWfLTOaG75OazzvX0PfrPl3DNi6l04dSV/yr5pa9tiPAp+VNA04FHg7cB0wStLw9G5/LLA91d8OjAO2SRoOHA7sLJV3KM9jZmYN0G2ffkRcFhFjI2I8xYnYuyNiBnAP8LlUbSawKg2vTuOk6XdHRKTy89LVPROAScB9/bYlZmbWrb58Nv0bYIWkq4CHgZtS+U3ADyS1ArsoDhRExGOSVgKPA+3A3Ih4vQ/rNzOzHupR6EdEC9CShp+ii6tvIuJ3wNlV5r8auLqnjTQzs/7hb+SamWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWWk29CXdKik+yT9StJjkv4ulU+QtF5Sq6QfSTo4lR+SxlvT9PGlZV2Wyp+UdEbdtsrMzLpUyzv93wOfiIgTgZOAqZJOA64FFkXERGA3MDvVnw3sTuWLUj0kHUfxe7nHA1OB70ga1o/bYmZm3eg29KPQlkYPSo8APgHcmsqXAWel4elpnDR9siSl8hUR8fuI+A3QShe/sWtmZvVT0w+jp3fkDwITgW8DvwZeioj2VGUbMCYNjwG2AkREu6Q9wJGpfF1pseV5yuuaA8wBqFQqtLS09GyLSiojYN4J7d1X7Gd9abMNHt6/hr6BeH07tLW11eW1rin0I+J14CRJo4DbgPf2e0v2rWsxsBigqakpmpube72sG5avYuHGmjaxX22Z0dzwdVrjef8a+mbNv2PA1r106kj6kn/V9OjqnYh4CbgH+DAwSlLHHj8W2J6GtwPjANL0w4Gd5fIu5jEzswao5eqdd6R3+EgaAXwK2EQR/p9L1WYCq9Lw6jROmn53REQqPy9d3TMBmATc10/bYWZmNajls+nRwLLUr/8WYGVE3C7pcWCFpKuAh4GbUv2bgB9IagV2UVyxQ0Q8Jmkl8DjQDsxN3UZmZtYg3YZ+RGwAPthF+VN0cfVNRPwOOLvKsq4Gru55M83MrD/4G7lmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZaSWH0YfJ+keSY9LekzSxan8CElrJG1Of0enckm6XlKrpA2STi4ta2aqv1nSzGrrNDOz+qjlnX47MC8ijgNOA+ZKOg6YD6yNiEnA2jQOcCYwKT3mADdCcZAAFgCnUvy27oKOA4WZmTVGt6EfEc9GxENp+BVgEzAGmA4sS9WWAWel4enAzVFYB4ySdDRwBrAmInZFxG5gDTC1PzfGzMwObHhPKksaD3wQWA9UIuLZNGkHUEnDY4Ctpdm2pbJq5Z3XMYfiEwKVSoWWlpaeNHE/lREw74T2Xs/fW31psw0e3r+GvoF4fTu0tbXV5bWuOfQlHQb8GLgkIl6WtHdaRISk6I8GRcRiYDFAU1NTNDc393pZNyxfxcKNPTqu9YstM5obvk5rPO9fQ9+s+XcM2LqXTh1JX/Kvmpqu3pF0EEXgL4+If0nFz6VuG9Lf51P5dmBcafaxqaxauZmZNUgtV+8IuAnYFBF/X5q0Gui4AmcmsKpUfmG6iuc0YE/qBroLmCJpdDqBOyWVmZlZg9Ty2fSjwAXARkmPpLKvAdcAKyXNBp4GzknT7gSmAa3Aa8BFABGxS9KVwP2p3hURsas/NsLMzGrTbehHxM8BVZk8uYv6AcytsqwlwJKeNNDMzPqPv5FrZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpaRWn4jd4mk5yU9Wio7QtIaSZvT39GpXJKul9QqaYOkk0vzzEz1N0ua2dW6zMysvmp5p78UmNqpbD6wNiImAWvTOMCZwKT0mAPcCMVBAlgAnAqcAizoOFCYmVnjdBv6EfEzoPMPmE8HlqXhZcBZpfKbo7AOGCXpaOAMYE1E7IqI3cAa3nggMTOzOuv2h9GrqETEs2l4B1BJw2OAraV621JZtfI3kDSH4lMClUqFlpaWXjYRKiNg3gntvZ6/t/rSZhs8vH8NfQPx+nZoa2ury2vd29DfKyJCUvRHY9LyFgOLAZqamqK5ubnXy7ph+SoWbuzzJvbYlhnNDV+nNZ73r6Fv1vw7BmzdS6eOpC/5V01vr955LnXbkP4+n8q3A+NK9camsmrlZmbWQL0N/dVAxxU4M4FVpfIL01U8pwF7UjfQXcAUSaPTCdwpqczMzBqo28+mkm4BmoGjJG2juArnGmClpNnA08A5qfqdwDSgFXgNuAggInZJuhK4P9W7IiI6nxw2M7M66zb0I+L8KpMmd1E3gLlVlrMEWNKj1pmZWb/yN3LNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy0jDQ1/SVElPSmqVNL/R6zczy1lDQ1/SMODbwJnAccD5ko5rZBvMzHLW6Hf6pwCtEfFURPwBWAFMb3AbzMyyNbzB6xsDbC2NbwNOLVeQNAeYk0bbJD3Zh/UdBbzYh/l7Rdc2eo02QLx/Wd2cfm2f9q9jqk1odOh3KyIWA4v7Y1mSHoiIpv5Yllln3r+snuq1fzW6e2c7MK40PjaVmZlZAzQ69O8HJkmaIOlg4DxgdYPbYGaWrYZ270REu6T/BtwFDAOWRMRjdVxlv3QTmVXh/cvqqS77lyKiHss1M7M3IX8j18wsIw59M7OMDNrQl/S6pEdKj/mlaUdJ+g9Jf9lpni2SNkraIOmnkt7V+JbbYCCprdP4LEnfSsNfl7Q97XePSvpsqfyrA9FeGxwkhaQflsaHS3pB0u0qvChpdJp2dKr/sVL9FyQdKelYSS1pH9wkqeb+/0Eb+sBvI+Kk0uOa0rSzgXXA+V3Md3pEfAB4APhaIxpqQ9KiiDiJYl9bImkw/y9Z47wKvF/SiDT+KdJl61GcYF0HfDhN+wjwcPqLpGOBnRGxE7ietA9GxPuAG2ptwFDdUc8H5gFjJI2tUudnwMTGNcmGoojYBLRTfDvXrBZ3Ap9Ow+cDt5Sm3UsK+fR3EfsfBH6Rho+muKMBABGxsdaVD+bQH9Gpe+dcAEnjgKMj4j5gJXBulfk/A9T8RFl29tu/gCu6qiTpVOA/gRca2Tgb1FYA50k6FPgAsL407RfsC/1TgNvY94XWj1AcFKA4GNwt6V8lfUXSqFpX/qa7DUMP/DZ9vO7sXIqwh+LJXQIsLE2/R9LrwAbg8rq20Aaz/fYvSbOA8lfivyLp88ArwLkREZIa20IblCJig6TxFO/y7+w0+X7gg5JGAgdFRJukpyRNpAj9hWkZ35d0FzCV4qaVX5Z0YkT8vrv1D+bQr+Z84F2SZqTxd0uaFBGb0/jpEdHwm2TZkLMoIr4x0I2wQWs18A2gGTiyozAiXpO0GfgC8FAqXgdMA94JPFmq+wzFm9olkh4F3g882N2KB3P3zhtIeg9wWESMiYjxETEe+J90fULXzGygLAH+rkpf/L3AJcAv0/gvgYuBdelkb8ePUR2Uht9FceCo6T5mgzn0O/fpX0MR7rd1qvdjHPrWOJdL2tbxGOjG2JtTRGyLiOurTP4F8MfsC/2HKG5OeW+pzhTgUUm/oritzV9HxI5a1u3bMJiZZWQwv9M3M7MecuibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlpH/D8l42oJ2teodAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"code","source":"train[\"author\"].sort_values().hist()\nplt.show()\nvalid[\"author\"].sort_values().hist()","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:18.593466Z","iopub.execute_input":"2023-01-12T09:11:18.596108Z","iopub.status.idle":"2023-01-12T09:11:19.059121Z","shell.execute_reply.started":"2023-01-12T09:11:18.596063Z","shell.execute_reply":"2023-01-12T09:11:19.057872Z"},"trusted":true},"execution_count":20,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUf0lEQVR4nO3df6xf9X3f8eerOGmQ2wUD7Q2z0cwUNx0tDWFXQNpqugmKY0gV54+GgNhiMiR3EquSim1xqkqspNGIVMYCWyNZw8XpWChKi2wFVmqRXFVLCgESiiEU2aVG2ONHggnthTads/f+uJ8bvhhf7vf++jrXn+dD+up7zvt8zjmf4+/x65zv+Z7v96aqkCT14ceOdwckSaNj6EtSRwx9SeqIoS9JHTH0Jakjq453B97I6aefXuvXr1/w/C+//DKrV69eug5JA9y/tJwWs3899NBD362qnzrWtB/p0F+/fj0PPvjgguefnJxkYmJi6TokDXD/0nJazP6V5KnZpnl5R5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0JekjswZ+knekeThgcffJPlEklOT7Emyrz2vae2T5KYk+5M8kuS8gWVtae33JdmynBsmSXq9OUO/qp6oqnOr6lzgnwOvAHcC24B7q2oDcG8bB7gY2NAeW4HPAyQ5FbgWuAA4H7h25kAhSRqN+X4j9yLgr6rqqSSbgYlW3wlMAp8ENgNfqOm/znJfklOSnNHa7qmqwwBJ9gCbgC8udiNms/fQS1y57a7lWvysDlz/gZGvU5KGMd/Qv4xXQ3qsqp5pw88CY214LfD0wDwHW222+msk2cr0OwTGxsaYnJycZxdfNXYyXHPOkQXPv1CL6bNWjqmpKV9rLZvl2r+GDv0kbwY+CHzq6GlVVUmW5O8uVtV2YDvA+Ph4Lea3TW6+bRc37B39zwsduGJi5OvU6PnbO1pOy7V/zefunYuBb1bVc238uXbZhvb8fKsfAs4cmG9dq81WlySNyHxC/3Jee/19NzBzB84WYNdA/aPtLp4LgZfaZaB7gI1J1rQPcDe2miRpRIa69pFkNfA+4NcGytcDdyS5CngKuLTV7wYuAfYzfafPxwCq6nCSTwMPtHbXzXyoK0kajaFCv6peBk47qvYC03fzHN22gKtnWc4OYMf8uylJWgp+I1eSOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0ZKvSTnJLkS0n+MsnjSd6d5NQke5Lsa89rWtskuSnJ/iSPJDlvYDlbWvt9SbYs10ZJko5t2DP9zwF/UlU/C7wTeBzYBtxbVRuAe9s4wMXAhvbYCnweIMmpwLXABcD5wLUzBwpJ0mjMGfpJ3gr8C+AWgKr6h6r6HrAZ2Nma7QQ+1IY3A1+oafcBpyQ5A3g/sKeqDlfVi8AeYNMSboskaQ6rhmhzFvAd4PeTvBN4CPg4MFZVz7Q2zwJjbXgt8PTA/Adbbbb6ayTZyvQ7BMbGxpicnBx2W15n7GS45pwjC55/oRbTZ60cU1NTvtZaNsu1fw0T+quA84Bfr6r7k3yOVy/lAFBVlaSWokNVtR3YDjA+Pl4TExMLXtbNt+3ihr3DbOLSOnDFxMjXqdGbnJxkMfun9EaWa/8a5pr+QeBgVd3fxr/E9EHguXbZhvb8fJt+CDhzYP51rTZbXZI0InOGflU9Czyd5B2tdBHwbWA3MHMHzhZgVxveDXy03cVzIfBSuwx0D7AxyZr2Ae7GVpMkjciw1z5+HbgtyZuBJ4GPMX3AuCPJVcBTwKWt7d3AJcB+4JXWlqo6nOTTwAOt3XVVdXhJtkKSNJShQr+qHgbGjzHpomO0LeDqWZazA9gxj/5JkpaQ38iVpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkdG/xdGJGmFWL/truO27ls3rV6W5XqmL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSR4YK/SQHkuxN8nCSB1vt1CR7kuxrz2taPUluSrI/ySNJzhtYzpbWfl+SLcuzSZKk2cznTP89VXVuVY238W3AvVW1Abi3jQNcDGxoj63A52H6IAFcC1wAnA9cO3OgkCSNxmK+kbsZmGjDO4FJ4JOt/oWqKuC+JKckOaO13VNVhwGS7AE2AV9cRB+k42bvoZe48jh8Y/PA9R8Y+Tp14hj2TL+AP03yUJKtrTZWVc+04WeBsTa8Fnh6YN6DrTZbXZI0IsOe6f9yVR1K8tPAniR/OTixqipJLUWH2kFlK8DY2BiTk5MLXtbYyXDNOUeWolvzspg+a+Vw/zrxHY/Xd8bU1NSyvNZDhX5VHWrPzye5k+lr8s8lOaOqnmmXb55vzQ8BZw7Mvq7VDvHq5aCZ+uQx1rUd2A4wPj5eExMTRzcZ2s237eKGvaP/TbkDV0yMfJ0aPfevE9/xuHw349ZNq1lM/s1mzss7SVYn+cmZYWAj8CiwG5i5A2cLsKsN7wY+2u7iuRB4qV0GugfYmGRN+wB3Y6tJkkZkmNOUMeDOJDPt/2dV/UmSB4A7klwFPAVc2trfDVwC7AdeAT4GUFWHk3waeKC1u27mQ11J0mjMGfpV9STwzmPUXwAuOka9gKtnWdYOYMf8uylJWgp+I1eSOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI0OHfpKTknwryZfb+FlJ7k+yP8kfJnlzq/94G9/fpq8fWManWv2JJO9f8q2RJL2h+Zzpfxx4fGD8s8CNVfV24EXgqla/Cnix1W9s7UhyNnAZ8HPAJuD3kpy0uO5LkuZjqNBPsg74APDf23iA9wJfak12Ah9qw5vbOG36Ra39ZuD2qvp+Vf01sB84fwm2QZI0pFVDtvsvwH8AfrKNnwZ8r6qOtPGDwNo2vBZ4GqCqjiR5qbVfC9w3sMzBeX4oyVZgK8DY2BiTk5NDdvH1xk6Ga845MnfDJbaYPmvlcP868R2P13fG1NTUsrzWc4Z+kl8Bnq+qh5JMLHkPjlJV24HtAOPj4zUxsfBV3nzbLm7YO+xxbekcuGJi5OvU6Ll/nfiu3HbXcVv3rZtWs5j8m80we+wvAR9McgnwFuAfAZ8DTkmyqp3trwMOtfaHgDOBg0lWAW8FXhiozxicR5I0AnNe06+qT1XVuqpaz/QHsV+pqiuArwK/2pptAXa14d1tnDb9K1VVrX5Zu7vnLGAD8I0l2xJJ0pwW8970k8DtSX4H+BZwS6vfAvxBkv3AYaYPFFTVY0nuAL4NHAGurqofLGL9kqR5mlfoV9UkMNmGn+QYd99U1d8DH55l/s8An5lvJyVJS8Nv5EpSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSNzhn6StyT5RpK/SPJYkt9u9bOS3J9kf5I/TPLmVv/xNr6/TV8/sKxPtfoTSd6/bFslSTqmYc70vw+8t6reCZwLbEpyIfBZ4MaqejvwInBVa38V8GKr39jakeRs4DLg54BNwO8lOWkJt0WSNIc5Q7+mTbXRN7VHAe8FvtTqO4EPteHNbZw2/aIkafXbq+r7VfXXwH7g/KXYCEnScIa6pp/kpCQPA88De4C/Ar5XVUdak4PA2ja8FngaoE1/CThtsH6MeSRJI7BqmEZV9QPg3CSnAHcCP7tcHUqyFdgKMDY2xuTk5IKXNXYyXHPOkbkbLrHF9Fkrh/vXie94vL4zpqamluW1Hir0Z1TV95J8FXg3cEqSVe1sfh1wqDU7BJwJHEyyCngr8MJAfcbgPIPr2A5sBxgfH6+JiYl5bdCgm2/bxQ1757WJS+LAFRMjX6dGz/3rxHfltruO27pv3bSaxeTfbIa5e+en2hk+SU4G3gc8DnwV+NXWbAuwqw3vbuO06V+pqmr1y9rdPWcBG4BvLNF2SJKGMMxpyhnAznanzY8Bd1TVl5N8G7g9ye8A3wJuae1vAf4gyX7gMNN37FBVjyW5A/g2cAS4ul02kiSNyJyhX1WPAO86Rv1JjnH3TVX9PfDhWZb1GeAz8++mJGkp+I1cSeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI7MGfpJzkzy1STfTvJYko+3+qlJ9iTZ157XtHqS3JRkf5JHkpw3sKwtrf2+JFuWb7MkSccyzJn+EeCaqjobuBC4OsnZwDbg3qraANzbxgEuBja0x1bg8zB9kACuBS4AzgeunTlQSJJGY87Qr6pnquqbbfhvgceBtcBmYGdrthP4UBveDHyhpt0HnJLkDOD9wJ6qOlxVLwJ7gE1LuTGSpDe2aj6Nk6wH3gXcD4xV1TNt0rPAWBteCzw9MNvBVputfvQ6tjL9DoGxsTEmJyfn08XXGDsZrjnnyILnX6jF9Fkrh/vXie94vL4zpqamluW1Hjr0k/wE8EfAJ6rqb5L8cFpVVZJaig5V1XZgO8D4+HhNTEwseFk337aLG/bO67i2JA5cMTHydWr03L9OfFduu+u4rfvWTatZTP7NZqi7d5K8ienAv62q/riVn2uXbWjPz7f6IeDMgdnXtdpsdUnSiAxz906AW4DHq+o/D0zaDczcgbMF2DVQ/2i7i+dC4KV2GegeYGOSNe0D3I2tJkkakWHem/4S8K+AvUkebrXfBK4H7khyFfAUcGmbdjdwCbAfeAX4GEBVHU7yaeCB1u66qjq8FBshSRrOnKFfVf8byCyTLzpG+wKunmVZO4Ad8+mgJGnp+I1cSeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1ZM7QT7IjyfNJHh2onZpkT5J97XlNqyfJTUn2J3kkyXkD82xp7fcl2bI8myNJeiPDnOnfCmw6qrYNuLeqNgD3tnGAi4EN7bEV+DxMHySAa4ELgPOBa2cOFJKk0Zkz9Kvqz4DDR5U3Azvb8E7gQwP1L9S0+4BTkpwBvB/YU1WHq+pFYA+vP5BIkpbZqgXON1ZVz7ThZ4GxNrwWeHqg3cFWm63+Okm2Mv0ugbGxMSYnJxfYRRg7Ga4558iC51+oxfRZK4f714nveLy+M6amppbltV5o6P9QVVWSWorOtOVtB7YDjI+P18TExIKXdfNtu7hh76I3cd4OXDEx8nVq9Ny/TnxXbrvruK371k2rWUz+zWahd+881y7b0J6fb/VDwJkD7da12mx1SdIILTT0dwMzd+BsAXYN1D/a7uK5EHipXQa6B9iYZE37AHdjq0mSRmjO96ZJvghMAKcnOcj0XTjXA3ckuQp4Cri0Nb8buATYD7wCfAygqg4n+TTwQGt3XVUd/eGwJGmZzRn6VXX5LJMuOkbbAq6eZTk7gB3z6p0kaUn5jVxJ6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjow89JNsSvJEkv1Jto16/ZLUs5GGfpKTgP8GXAycDVye5OxR9kGSejbqM/3zgf1V9WRV/QNwO7B5xH2QpG6tGvH61gJPD4wfBC4YbJBkK7C1jU4leWIR6zsd+O4i5l+QfHbUa9Rx4v6lZfOezy5q//ons00YdejPqaq2A9uXYllJHqyq8aVYlnQ09y8tp+Xav0Z9eecQcObA+LpWkySNwKhD/wFgQ5KzkrwZuAzYPeI+SFK3Rnp5p6qOJPm3wD3AScCOqnpsGVe5JJeJpFm4f2k5Lcv+lapajuVKkn4E+Y1cSeqIoS9JHVmxoZ/kB0keHnhsG5h2epL/m+TfHDXPgSR7kzyS5E+TvG30PddKkGTqqPErk/zXNvwfkxxq+92jST44UP93x6O/WhmSVJL/MTC+Ksl3knw5076bZE2bdkZr/8sD7b+T5LQk70gy2fbBx5MMff1/xYY+8HdVde7A4/qBaR8G7gMuP8Z876mqXwAeBH5zFB3VCenGqjqX6X1tR5KV/H9Jo/My8PNJTm7j76Pdtl7TH7DeB7y7TftF4FvtmSTvAF6oqheAm2j7YFX9M+DmYTtwou6olwPXAGuTrJulzZ8Bbx9dl3QiqqrHgSNMfztXGsbdwAfa8OXAFwemfZ0W8u35Rl57EPhaGz6D6V80AKCq9g678pUc+icfdXnnIwBJzgTOqKpvAHcAH5ll/l8Bhv6HUndes38B1x2rUZILgP8HfGeUndOKdjtwWZK3AL8A3D8w7Wu8GvrnA3fy6hdaf5HpgwJMHwy+kuR/JfmNJKcMu/IfuZ9hmIe/a2+vj/YRpsMepv9xdwA3DEz/apIfAI8Av7WsPdRK9pr9K8mVwOBX4n8jyb8E/hb4SFVVktH2UCtSVT2SZD3TZ/l3HzX5AeBdSVYDb6qqqSRPJnk706F/Q1vG7ye5B9jE9I9W/lqSd1bV9+da/0oO/dlcDrwtyRVt/B8n2VBV+9r4e6pq5D+SpRPOjVX1u8e7E1qxdgO/C0wAp80Uq+qVJPuAfw18s5XvAy4Bfhp4YqDt/2H6pHZHkkeBnwcemmvFK/nyzusk+RngJ6pqbVWtr6r1wH/i2B/oStLxsgP47VmuxX8d+ATw5238z4GPA/e1D3tn/hjVm9rw25g+cAz1O2YrOfSPvqZ/PdPhfudR7f4IQ1+j81tJDs48jndn9KOpqg5W1U2zTP4a8E95NfS/yfSPU359oM1G4NEkf8H0z9r8+6p6dph1+zMMktSRlXymL0maJ0Nfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdeT/Ax1i2tQpzoviAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}},{"execution_count":20,"output_type":"execute_result","data":{"text/plain":"<AxesSubplot:>"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAT3UlEQVR4nO3df6zd9X3f8eerOCSu22F+tCfMtmYmvHRRKIRdUdJU001YKkOqmD8SAmLDMEvuJNblB9vqVpWyTpMGUhgLrEK1BsVsLAmjRbYCa4OAq2pJoQFCMQmNuGWm2DM/C24vJG2dvffH+RgOju177rn33Mv9+vmQjs7n+/l8vuf7OT4fv+73fu75npOqQpLULT+21AOQJC08w12SOshwl6QOMtwlqYMMd0nqoBVLPQCA0047rdavXz/Svq+//jqrVq1a2AFJjfNL4zafOfboo4++XFU/daS2d0S4r1+/nkceeWSkfaemppicnFzYAUmN80vjNp85luTZo7W5LCNJHTRUuCf5XJLvJHkyyZeTvCfJGUkeTjKd5KtJTmx93922p1v7+rE+A0nSj5g13JOsAf4VMFFVHwBOAC4FrgNuqKozgVeBLW2XLcCrrf6G1k+StIiGXZZZAaxMsgL4cWA/8FHgrta+A7i4lTe1bVr7BUmyIKOVJA1l1j+oVtW+JF8E/hz4PvB14FHgtao62LrtBda08hrgubbvwSQHgFOBlwcfN8lWYCtAr9djampqpCcwMzMz8r7SbJxfGrdxzbFZwz3JyfTPxs8AXgP+J7Bxvgeuqu3AdoCJiYka9a/FvptB4+T80riNa44NsyzzT4D/U1UvVdXfAr8HfBhY3ZZpANYC+1p5H7AOoLWfBLyyoKOWJB3TMOH+58D5SX68rZ1fAHwXeBD4ZOuzGdjZyrvaNq39gfJzhSVpUc0a7lX1MP0/jD4G7G77bAd+Ffh8kmn6a+q3tF1uAU5t9Z8Hto1h3JKkYxjqCtWq+gLwhcOqnwHOO0LfHwCfmv/QhrN73wGu3HbPYh3ubfZc+/ElOa4kzcYrVCWpgwx3Seogw12SOshwl6QOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA4y3CWpgwx3Seogw12SOshwl6QOMtwlqYNmDfck70vy+MDtL5N8NskpSe5L8nS7P7n1T5Ibk0wneSLJueN/GpKkQcN8h+r3quqcqjoH+EfAG8Dd9L8b9f6q2gDcz1vflXohsKHdtgI3j2HckqRjmOuyzAXAn1XVs8AmYEer3wFc3MqbgNur7yFgdZLTF2KwkqThDPUF2QMuBb7cyr2q2t/KzwO9Vl4DPDewz95Wt3+gjiRb6Z/Z0+v1mJqamuNQ2iBWwjVnHRxp3/kadcxaPmZmZnydNVbjmmNDh3uSE4FPAL92eFtVVZKay4GrajuwHWBiYqImJyfnsvubbrpjJ9fvnuvPqIWx5/LJJTmuFs/U1BSjzk1pGOOaY3NZlrkQeKyqXmjbLxxabmn3L7b6fcC6gf3WtjpJ0iKZS7hfxltLMgC7gM2tvBnYOVB/RXvXzPnAgYHlG0nSIhhqPSPJKuBjwC8PVF8L3JlkC/AscEmrvxe4CJim/86aqxZstJKkoQwV7lX1OnDqYXWv0H/3zOF9C7h6QUYnSRqJV6hKUgcZ7pLUQYa7JHWQ4S5JHWS4S1IHGe6S1EGGuyR1kOEuSR1kuEtSBxnuktRBhrskdZDhLkkdZLhLUgcZ7pLUQYa7JHWQ4S5JHWS4S1IHGe6S1EFDhXuS1UnuSvKnSZ5K8qEkpyS5L8nT7f7k1jdJbkwyneSJJOeO9ylIkg437Jn7l4Dfr6qfAc4GngK2AfdX1Qbg/rYNcCGwod22Ajcv6IglSbOaNdyTnAT8Y+AWgKr6m6p6DdgE7GjddgAXt/Im4PbqewhYneT0BR63JOkYVgzR5wzgJeB3kpwNPAp8BuhV1f7W53mg18prgOcG9t/b6vYP1JFkK/0ze3q9HlNTUyM9gd5KuOasgyPtO1+jjlnLx8zMjK/zcWD3vgNLduwzTjphLHNsmHBfAZwL/EpVPZzkS7y1BANAVVWSmsuBq2o7sB1gYmKiJicn57L7m266YyfX7x7maSy8PZdPLslxtXimpqYYdW5q+bhy2z1LduzbNq4ayxwbZs19L7C3qh5u23fRD/sXDi23tPsXW/s+YN3A/mtbnSRpkcwa7lX1PPBckve1qguA7wK7gM2tbjOws5V3AVe0d82cDxwYWL6RJC2CYdczfgW4I8mJwDPAVfR/MNyZZAvwLHBJ63svcBEwDbzR+kqSFtFQ4V5VjwMTR2i64Ah9C7h6fsOSJM2HV6hKUgcZ7pLUQYa7JHWQ4S5JHbQ0V/9Iy8TufQeW7AKXPdd+fEmOq27wzF2SOshwl6QOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA4y3CWpgwx3SeqgocI9yZ4ku5M8nuSRVndKkvuSPN3uT271SXJjkukkTyQ5d5xPQJL0o+Zy5v6Rqjqnqg593d424P6q2gDc37YBLgQ2tNtW4OaFGqwkaTjzWZbZBOxo5R3AxQP1t1ffQ8DqJKfP4ziSpDka9vPcC/h6kgJ+u6q2A72q2t/anwd6rbwGeG5g372tbv9AHUm20j+zp9frMTU1NdIT6K2Ea846ONK+8zXqmLV8OL+OD0v1GgPMzMyM5bUeNtx/oar2Jflp4L4kfzrYWFXVgn9o7QfEdoCJiYmanJycy+5vuumOnVy/e2m+c2TP5ZNLclwtHufX8WGpvpAF4LaNqxg1/45lqGWZqtrX7l8E7gbOA144tNzS7l9s3fcB6wZ2X9vqJEmLZNZwT7IqyU8eKgO/CDwJ7AI2t26bgZ2tvAu4or1r5nzgwMDyjSRpEQzz+2YPuDvJof7/o6p+P8m3gDuTbAGeBS5p/e8FLgKmgTeAqxZ81JKkY5o13KvqGeDsI9S/AlxwhPoCrl6Q0UmSRuIVqpLUQYa7JHWQ4S5JHWS4S1IHGe6S1EGGuyR1kOEuSR1kuEtSBxnuktRBhrskdZDhLkkdZLhLUgcZ7pLUQYa7JHWQ4S5JHWS4S1IHGe6S1EGGuyR10NDhnuSEJN9O8rW2fUaSh5NMJ/lqkhNb/bvb9nRrXz+msUuSjmIuZ+6fAZ4a2L4OuKGqzgReBba0+i3Aq63+htZPkrSIhgr3JGuBjwP/tW0H+ChwV+uyA7i4lTe1bVr7Ba2/JGmRrBiy338G/i3wk237VOC1qjrYtvcCa1p5DfAcQFUdTHKg9X958AGTbAW2AvR6PaampkZ6Ar2VcM1ZB2fvOAajjlnLh/Pr+LBUrzHAzMzMWF7rWcM9yS8BL1bVo0kmF+rAVbUd2A4wMTFRk5OjPfRNd+zk+t3D/oxaWHsun1yS42rxOL+OD1duu2fJjn3bxlWMmn/HMsys/TDwiSQXAe8B/g7wJWB1khXt7H0tsK/13wesA/YmWQGcBLyy4COXJB3VrGvuVfVrVbW2qtYDlwIPVNXlwIPAJ1u3zcDOVt7VtmntD1RVLeioJUnHNJ/3uf8q8Pkk0/TX1G9p9bcAp7b6zwPb5jdESdJczWkxsaqmgKlWfgY47wh9fgB8agHGJkkakVeoSlIHGe6S1EGGuyR1kOEuSR1kuEtSBxnuktRBhrskdZDhLkkdZLhLUgcZ7pLUQYa7JHWQ4S5JHWS4S1IHGe6S1EGGuyR1kOEuSR1kuEtSB80a7knek+SPk/xJku8k+c1Wf0aSh5NMJ/lqkhNb/bvb9nRrXz/m5yBJOswwZ+5/DXy0qs4GzgE2JjkfuA64oarOBF4FtrT+W4BXW/0NrZ8kaRHNGu7VN9M239VuBXwUuKvV7wAubuVNbZvWfkGSLNSAJUmzG+oLspOcADwKnAn8FvBnwGtVdbB12QusaeU1wHMAVXUwyQHgVODlwx5zK7AVoNfrMTU1NdIT6K2Ea846OHvHMRh1zFo+nF/Hh6V6jQFmZmbG8loPFe5V9UPgnCSrgbuBn5nvgatqO7AdYGJioiYnJ0d6nJvu2Mn1u4d6Ggtuz+WTS3JcLR7n1/Hhym33LNmxb9u4ilHz71jm9G6ZqnoNeBD4ELA6yaFZvxbY18r7gHUArf0k4JWFGKwkaTjDvFvmp9oZO0lWAh8DnqIf8p9s3TYDO1t5V9umtT9QVbWAY5YkzWKY3zdPB3a0dfcfA+6sqq8l+S7wlST/Afg2cEvrfwvw35JMA38BXDqGcUuSjmHWcK+qJ4APHqH+GeC8I9T/APjUgoxOkjQSr1CVpA4y3CWpgwx3Seogw12SOshwl6QOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA4y3CWpgwx3Seogw12SOshwl6QOMtwlqYOG+YLsdUkeTPLdJN9J8plWf0qS+5I83e5PbvVJcmOS6SRPJDl33E9CkvR2w5y5HwSuqar3A+cDVyd5P7ANuL+qNgD3t22AC4EN7bYVuHnBRy1JOqZZw72q9lfVY638V8BTwBpgE7CjddsBXNzKm4Dbq+8hYHWS0xd64JKko1sxl85J1gMfBB4GelW1vzU9D/RaeQ3w3MBue1vd/oE6kmylf2ZPr9djampqjkPv662Ea846ONK+8zXqmLV8OL+OD0v1GgPMzMyM5bUeOtyT/ATwu8Bnq+ovk7zZVlWVpOZy4KraDmwHmJiYqMnJybns/qab7tjJ9bvn9DNqwey5fHJJjqvF4/w6Ply57Z4lO/ZtG1cxav4dy1DvlknyLvrBfkdV/V6rfuHQcku7f7HV7wPWDey+ttVJkhbJMO+WCXAL8FRV/aeBpl3A5lbeDOwcqL+ivWvmfODAwPKNJGkRDPP75oeBfwbsTvJ4q/t14FrgziRbgGeBS1rbvcBFwDTwBnDVQg5YkjS7WcO9qv43kKM0X3CE/gVcPc9xSZLmwStUJamDDHdJ6iDDXZI6yHCXpA4y3CWpgwx3Seogw12SOshwl6QOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA4y3CWpg4b5DtVbk7yY5MmBulOS3Jfk6XZ/cqtPkhuTTCd5Ism54xy8JOnIhjlzvw3YeFjdNuD+qtoA3N+2AS4ENrTbVuDmhRmmJGkuZg33qvpD4C8Oq94E7GjlHcDFA/W3V99DwOokpy/QWCVJQ5r1C7KPoldV+1v5eaDXymuA5wb67W11+zlMkq30z+7p9XpMTU2NNpCVcM1ZB0fad75GHbOWD+fX8WGpXmOAmZmZsbzWo4b7m6qqktQI+20HtgNMTEzU5OTkSMe/6Y6dXL973k9jJHsun1yS42rxOL+OD1duu2fJjn3bxlWMmn/HMuq7ZV44tNzS7l9s9fuAdQP91rY6SdIiGjXcdwGbW3kzsHOg/or2rpnzgQMDyzeSpEUy6++bSb4MTAKnJdkLfAG4FrgzyRbgWeCS1v1e4CJgGngDuGoMY5YkzWLWcK+qy47SdMER+hZw9XwHJUmaH69QlaQOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA4y3CWpgwx3Seogw12SOshwl6QOMtwlqYMMd0nqIMNdkjrIcJekDjLcJamDxhLuSTYm+V6S6STbxnEMSdLRLXi4JzkB+C3gQuD9wGVJ3r/Qx5EkHd04ztzPA6ar6pmq+hvgK8CmMRxHknQUK8bwmGuA5wa29wI/d3inJFuBrW1zJsn3RjzeacDLI+47L7luKY6qReb80lh95Lp5zbG/d7SGcYT7UKpqO7B9vo+T5JGqmliAIUk/wvmlcRvXHBvHssw+YN3A9tpWJ0laJOMI928BG5KckeRE4FJg1xiOI0k6igVflqmqg0n+JfAHwAnArVX1nYU+zoB5L+1Ix+D80riNZY6lqsbxuJKkJeQVqpLUQYa7JHXQOz7ck/wwyeMDt20Dbacl+dsk/+KwffYk2Z3kiSRfT/LexR+5loMkM4dtX5nkv7Tyv0uyr827J5N8YqD+Xy/FePXOl6SS/PeB7RVJXkrytfS9nOTk1nZ66/8LA/1fSnJqkvclmWrz76kkc1qbf8eHO/D9qjpn4HbtQNungIeAy46w30eq6meBR4BfX4yBqpNuqKpz6M+1W5Msh/8zWlqvAx9IsrJtf4z2dvDq/5HzIeBDre3ngW+3e5K8D3ilql4BbqTNv6r6h8BNcxnEcp+olwHXAGuSrD1Knz8Ezly8IamLquop4CD9K1al2dwLfLyVLwO+PND2TVqYt/sbeHvYf6OVT6d/hT8AVbV7LgNYDuG+8rBlmU8DJFkHnF5VfwzcCXz6KPv/EjCnfxQdV942v4B/f6ROSX4O+H/AS4s5OC1bXwEuTfIe4GeBhwfavsFb4X4ecDdvXfj58/TDH/qh/0CS/5Xkc0lWz2UAS/bxA3Pw/fZr8eE+TT/Uof8PeStw/UD7g0l+CDwB/MZYR6jl7G3zK8mVwOCl4J9L8k+BvwI+XVWVZHFHqGWnqp5Isp7+Wfu9hzV/C/hgklXAu6pqJskzSc6kH+7Xt8f4nSR/AGyk/+GLv5zk7Kr662HGsBzC/WguA96b5PK2/XeTbKiqp9v2R6pqST7wSZ1yQ1V9cakHoWVpF/BFYBI49VBlVb2R5GngnwOPteqHgIuAnwa+N9D3/9I/cb01yZPAB4BHhzn4cliW+RFJ/gHwE1W1pqrWV9V64D9y5D+sStJSuBX4zaOslX8T+CzwR237j4DPAA+1P7oe+tKjd7Xye+n/gBj6c7qWQ7gfvuZ+Lf0Qv/uwfr+L4a7F8xtJ9h66LfVg9M5TVXur6sajNH8D+Pu8Fe6P0f+QxW8O9PlF4Mkkf0L/41z+TVU9P+zx/fgBSeqg5XDmLkmaI8NdkjrIcJekDjLcJamDDHdJ6iDDXZI6yHCXpA76/xCldcdI0q62AAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"# Baselines","metadata":{}},{"cell_type":"markdown","source":"## Stratified Dummy Classifier","metadata":{}},{"cell_type":"code","source":"dummy_clf = DummyClassifier(strategy='stratified')\ndummy_clf.fit(train['text'], train['author'])\n\npred = dummy_clf.predict(valid['text'])\nprint(classification_report(valid['author'], pred))","metadata":{"execution":{"iopub.status.busy":"2023-01-05T14:29:48.700259Z","iopub.execute_input":"2023-01-05T14:29:48.700766Z","iopub.status.idle":"2023-01-05T14:29:48.798033Z","shell.execute_reply.started":"2023-01-05T14:29:48.700723Z","shell.execute_reply":"2023-01-05T14:29:48.796981Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":5,"outputs":[{"name":"stdout","text":" precision recall f1-score support\n\n EAP 0.40 0.42 0.41 782\n HPL 0.28 0.27 0.28 561\n MWS 0.31 0.30 0.31 600\n\n accuracy 0.34 1943\n macro avg 0.33 0.33 0.33 1943\nweighted avg 0.34 0.34 0.34 1943\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## Preprocessing","metadata":{}},{"cell_type":"code","source":"import nltk\nfrom nltk.tokenize import word_tokenize, sent_tokenize\nfrom nltk.corpus import stopwords\nfrom nltk.stem import WordNetLemmatizer\nimport string\n\nstopwords = set(stopwords.words('english'))\nnltk.download('wordnet')\nnltk.download('omw-1.4')","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:24.097670Z","iopub.execute_input":"2023-01-12T09:11:24.098246Z","iopub.status.idle":"2023-01-12T09:11:24.171214Z","shell.execute_reply.started":"2023-01-12T09:11:24.098201Z","shell.execute_reply":"2023-01-12T09:11:24.170147Z"},"trusted":true},"execution_count":21,"outputs":[{"name":"stderr","text":"[nltk_data] Downloading package wordnet to /usr/share/nltk_data...\n[nltk_data] Package wordnet is already up-to-date!\n[nltk_data] Downloading package omw-1.4 to /usr/share/nltk_data...\n[nltk_data] Package omw-1.4 is already up-to-date!\n","output_type":"stream"},{"execution_count":21,"output_type":"execute_result","data":{"text/plain":"True"},"metadata":{}}]},{"cell_type":"code","source":"print(train['text'][0])\nprint(\"-----------\")\n' '.join(word_tokenize(train['text'][0]))","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:24.272064Z","iopub.execute_input":"2023-01-12T09:11:24.274630Z","iopub.status.idle":"2023-01-12T09:11:24.299021Z","shell.execute_reply.started":"2023-01-12T09:11:24.274588Z","shell.execute_reply":"2023-01-12T09:11:24.298180Z"},"trusted":true},"execution_count":22,"outputs":[{"name":"stdout","text":"Beasts balk at the critters hosses wuss'n mules but when they got autos that was all right.\n-----------\n","output_type":"stream"},{"execution_count":22,"output_type":"execute_result","data":{"text/plain":"\"Beasts balk at the critters hosses wuss'n mules but when they got autos that was all right .\""},"metadata":{}}]},{"cell_type":"code","source":"lemmatizer = WordNetLemmatizer()\ndef preprocess_nltk(sentence):\n token_words = word_tokenize(sentence)\n \n # Punctuation might be important! Don't discard.\n \n return ' '.join([lemmatizer.lemmatize(word)\n for word in token_words \n if (word not in stopwords and\n word not in string.punctuation)])\n\npreprocess_nltk(train['text'][0])","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:24.433821Z","iopub.execute_input":"2023-01-12T09:11:24.434277Z","iopub.status.idle":"2023-01-12T09:11:27.013156Z","shell.execute_reply.started":"2023-01-12T09:11:24.434224Z","shell.execute_reply":"2023-01-12T09:11:27.012194Z"},"trusted":true},"execution_count":23,"outputs":[{"execution_count":23,"output_type":"execute_result","data":{"text/plain":"\"Beasts balk critter ho wuss'n mule got auto right\""},"metadata":{}}]},{"cell_type":"markdown","source":"## TF-IDF -> Logistic Regression.","metadata":{}},{"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\nfrom sklearn.naive_bayes import MultinomialNB\nfrom sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer\nfrom sklearn.pipeline import Pipeline","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:08:19.633582Z","iopub.execute_input":"2023-01-12T09:08:19.633956Z","iopub.status.idle":"2023-01-12T09:08:19.644783Z","shell.execute_reply.started":"2023-01-12T09:08:19.633923Z","shell.execute_reply":"2023-01-12T09:08:19.643508Z"},"trusted":true},"execution_count":6,"outputs":[]},{"cell_type":"code","source":"lr_pipeline = Pipeline(\n [\n ('vectorizer', TfidfVectorizer(preprocessor=preprocess_nltk)),\n ('model', LogisticRegression(n_jobs=2, verbose=1))\n ])","metadata":{"execution":{"iopub.status.busy":"2023-01-10T18:50:39.381048Z","iopub.execute_input":"2023-01-10T18:50:39.381760Z","iopub.status.idle":"2023-01-10T18:50:39.387031Z","shell.execute_reply.started":"2023-01-10T18:50:39.381708Z","shell.execute_reply":"2023-01-10T18:50:39.385710Z"},"trusted":true},"execution_count":11,"outputs":[]},{"cell_type":"code","source":"lr_pipeline.fit(train['text'], train['author'])\nres = lr_pipeline.predict(valid['text'])\n\ncorrect = [res[i] == valid['author'][i] for i in range(len(res))]\nprint(f\"Accuracy: {sum(correct) / len(res)*100:.2f}%\")\nprint(classification_report(valid['author'], res))","metadata":{"execution":{"iopub.status.busy":"2023-01-10T18:50:39.965206Z","iopub.execute_input":"2023-01-10T18:50:39.966056Z","iopub.status.idle":"2023-01-10T18:50:48.739868Z","shell.execute_reply.started":"2023-01-10T18:50:39.966022Z","shell.execute_reply":"2023-01-10T18:50:48.738603Z"},"trusted":true},"execution_count":12,"outputs":[{"name":"stderr","text":"[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.\n","output_type":"stream"},{"name":"stdout","text":"RUNNING THE L-BFGS-B CODE\n\n * * *\n\nMachine precision = 2.220D-16\n N = 69327 M = 10\n\nAt X0 0 variables are exactly at the bounds\n\nAt iterate 0 f= 1.92081D+04 |proj g|= 1.20800D+03\n","output_type":"stream"},{"name":"stderr","text":" This problem is unconstrained.\n","output_type":"stream"},{"name":"stdout","text":"\nAt iterate 50 f= 9.87699D+03 |proj g|= 1.60612D+01\n\nAt iterate 100 f= 9.86150D+03 |proj g|= 4.74839D-01\n\n * * *\n\nTit = total number of iterations\nTnf = total number of function evaluations\nTnint = total number of segments explored during Cauchy searches\nSkip = number of BFGS updates skipped\nNact = number of active bounds at final generalized Cauchy point\nProjg = norm of the final projected gradient\nF = final function value\n\n * * *\n\n N Tit Tnf Tnint Skip Nact Projg F\n69327 100 110 1 0 0 4.748D-01 9.862D+03\n F = 9861.5029459684520 \n\nSTOP: TOTAL NO. of ITERATIONS REACHED LIMIT \n","output_type":"stream"},{"name":"stderr","text":"[Parallel(n_jobs=2)]: Done 1 out of 1 | elapsed: 2.6s finished\n","output_type":"stream"},{"name":"stdout","text":"Accuracy: 80.49%\n precision recall f1-score support\n\n EAP 0.77 0.85 0.81 782\n HPL 0.82 0.75 0.79 561\n MWS 0.85 0.80 0.82 600\n\n accuracy 0.80 1943\n macro avg 0.81 0.80 0.80 1943\nweighted avg 0.81 0.80 0.80 1943\n\n","output_type":"stream"}]},{"cell_type":"code","source":"nb_pipeline = Pipeline(\n [\n ('vectorizer', CountVectorizer(preprocessor=preprocess_nltk, ngram_range=(1,2))),\n ('model', MultinomialNB())\n ])","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:29.865722Z","iopub.execute_input":"2023-01-12T09:11:29.866091Z","iopub.status.idle":"2023-01-12T09:11:29.871837Z","shell.execute_reply.started":"2023-01-12T09:11:29.866059Z","shell.execute_reply":"2023-01-12T09:11:29.870693Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"code","source":"nb_pipeline.fit(train['text'], train['author'])\nres = nb_pipeline.predict(valid['text'])\n\ncorrect = [res[i] == valid['author'][i] for i in range(len(res))]\nprint(f\"Accuracy: {sum(correct) / len(res)*100:.2f}%\")\nprint(classification_report(valid['author'], res))","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:11:30.734346Z","iopub.execute_input":"2023-01-12T09:11:30.734719Z","iopub.status.idle":"2023-01-12T09:11:37.364422Z","shell.execute_reply.started":"2023-01-12T09:11:30.734687Z","shell.execute_reply":"2023-01-12T09:11:37.363439Z"},"trusted":true},"execution_count":25,"outputs":[{"name":"stdout","text":"Accuracy: 84.92%\n precision recall f1-score support\n\n EAP 0.84 0.87 0.85 782\n HPL 0.87 0.82 0.84 561\n MWS 0.85 0.85 0.85 600\n\n accuracy 0.85 1943\n macro avg 0.85 0.85 0.85 1943\nweighted avg 0.85 0.85 0.85 1943\n\n","output_type":"stream"}]},{"cell_type":"code","source":"nb_pipeline.named_steps","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:13:49.078425Z","iopub.execute_input":"2023-01-12T09:13:49.078804Z","iopub.status.idle":"2023-01-12T09:13:49.087384Z","shell.execute_reply.started":"2023-01-12T09:13:49.078773Z","shell.execute_reply":"2023-01-12T09:13:49.086299Z"},"trusted":true},"execution_count":31,"outputs":[{"execution_count":31,"output_type":"execute_result","data":{"text/plain":"{'vectorizer': CountVectorizer(ngram_range=(1, 2),\n preprocessor=<function preprocess_nltk at 0x7fd3371b7950>),\n 'model': MultinomialNB()}"},"metadata":{}}]},{"cell_type":"code","source":"eap_class_prob_sorted = nb_pipeline.named_steps.model.feature_log_prob_[0, :].argsort()[::-1]\nhpl_class_prob_sorted = nb_pipeline.named_steps.model.feature_log_prob_[1, :].argsort()[::-1]\nmws_class_prob_sorted = nb_pipeline.named_steps.model.feature_log_prob_[2, :].argsort()[::-1]\n\n\nprint(np.take(nb_pipeline.named_steps.vectorizer.get_feature_names(), eap_class_prob_sorted[:10]))\nprint(np.take(nb_pipeline.named_steps.vectorizer.get_feature_names(), hpl_class_prob_sorted[:10]))\nprint(np.take(nb_pipeline.named_steps.vectorizer.get_feature_names(), mws_class_prob_sorted[:10]))","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:15:45.568010Z","iopub.execute_input":"2023-01-12T09:15:45.568384Z","iopub.status.idle":"2023-01-12T09:15:46.387769Z","shell.execute_reply.started":"2023-01-12T09:15:45.568351Z","shell.execute_reply":"2023-01-12T09:15:46.386700Z"},"trusted":true},"execution_count":37,"outputs":[{"name":"stdout","text":"['The' 'upon' 'one' 'It' 'could' 'would' 'In' 'said' 'But' 'time']\n['The' 'could' 'one' 'thing' 'would' 'old' 'It' 'He' 'time' 'seemed']\n['The' 'would' 'one' 'could' 'life' 'He' 'heart' 'day' 'time' 'Raymond']\n","output_type":"stream"}]},{"cell_type":"code","source":"from sklearn.model_selection import GridSearchCV\n\nparam_grid = {\n 'vectorizer__ngram_range':[(1,2), (1, 1), (2,3), (1, 3), (1, 6)],\n 'vectorizer__binary':[False, True],\n}\n\ngrid = GridSearchCV(nb_pipeline, param_grid, verbose=2)\ngrid.fit(train['text'], train['author'])\n\nbest_model = grid.best_estimator_\nprint('Best params:', grid.best_params_)\nres = best_model.predict(valid['text'])\n\ncorrect = [res[i] == valid['author'][i] for i in range(len(res))]\nprint(f\"Accuracy: {sum(correct) / len(res)*100:.2f}%\")\nprint(classification_report(valid['author'], res))","metadata":{"execution":{"iopub.status.busy":"2023-01-10T19:09:01.141081Z","iopub.execute_input":"2023-01-10T19:09:01.142222Z","iopub.status.idle":"2023-01-10T19:14:14.622137Z","shell.execute_reply.started":"2023-01-10T19:09:01.142149Z","shell.execute_reply":"2023-01-10T19:14:14.621004Z"},"trusted":true},"execution_count":22,"outputs":[{"name":"stdout","text":"Fitting 5 folds for each of 10 candidates, totalling 50 fits\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 2); total time= 5.4s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 2); total time= 5.4s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 2); total time= 6.0s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 2); total time= 5.6s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 2); total time= 5.3s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 1); total time= 4.5s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 1); total time= 4.9s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 1); total time= 4.6s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 1); total time= 5.5s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 1); total time= 4.6s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(2, 3); total time= 6.1s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(2, 3); total time= 5.6s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(2, 3); total time= 5.9s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(2, 3); total time= 6.3s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(2, 3); total time= 6.0s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 3); total time= 6.2s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 3); total time= 5.9s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 3); total time= 6.2s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 3); total time= 6.2s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 3); total time= 6.6s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 6); total time= 8.2s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 6); total time= 7.7s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 6); total time= 8.2s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 6); total time= 8.7s\n[CV] END vectorizer__binary=False, vectorizer__ngram_range=(1, 6); total time= 8.1s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 2); total time= 5.3s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 2); total time= 5.5s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 2); total time= 5.7s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 2); total time= 5.5s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 2); total time= 5.3s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 1); total time= 4.5s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 1); total time= 5.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 1); total time= 4.6s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 1); total time= 4.9s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 1); total time= 5.1s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(2, 3); total time= 6.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(2, 3); total time= 5.6s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(2, 3); total time= 6.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(2, 3); total time= 5.7s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(2, 3); total time= 6.5s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 3); total time= 6.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 3); total time= 6.2s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 3); total time= 6.4s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 3); total time= 6.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 3); total time= 6.7s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 6); total time= 8.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 6); total time= 7.8s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 6); total time= 8.0s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 6); total time= 8.4s\n[CV] END vectorizer__binary=True, vectorizer__ngram_range=(1, 6); total time= 7.8s\nBest params: {'vectorizer__binary': False, 'vectorizer__ngram_range': (1, 2)}\nAccuracy: 84.92%\n precision recall f1-score support\n\n EAP 0.84 0.87 0.85 782\n HPL 0.87 0.82 0.84 561\n MWS 0.85 0.85 0.85 600\n\n accuracy 0.85 1943\n macro avg 0.85 0.85 0.85 1943\nweighted avg 0.85 0.85 0.85 1943\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Tokenizers / Embeddings","metadata":{}},{"cell_type":"markdown","source":"## BERT","metadata":{}},{"cell_type":"code","source":"from transformers import AutoModelForSequenceClassification\nberttokenizer = AutoTokenizer.from_pretrained('bert-base-cased')\n\nclass SpookyDataset(torch.utils.data.Dataset):\n def __init__(self, encodings, labels):\n self.encodings = encodings\n self.labels = labels\n \n def __len__(self):\n return len(self.labels)\n \n def __getitem__(self, idx):\n item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}\n item['labels'] = torch.tensor(self.labels[idx])\n return item\n\ndef encode(texts):\n return berttokenizer(\n [text for text in texts],\n padding=\"max_length\",\n max_length=510,\n truncation=True,\n add_special_tokens=True)\n\nencoder = LabelEncoder()\ndef label(authors):\n return encoder.fit_transform(authors)\n\ndef unlabel(labels):\n return encoder.inverse_transform(labels)\n\ntrain_dataset = SpookyDataset(encode(train[\"text\"]), label(train[\"author\"]))\nvalid_dataset = SpookyDataset(encode(valid[\"text\"]), label(valid[\"author\"]))\n\ntrain_dataloader = torch.utils.data.DataLoader(\n train_dataset,\n batch_size=8,\n shuffle=True,\n num_workers=2)\n\nvalid_dataloader = torch.utils.data.DataLoader(\n valid_dataset,\n batch_size=8,\n shuffle=True,\n num_workers=2)","metadata":{"execution":{"iopub.status.busy":"2023-01-14T07:31:42.113648Z","iopub.execute_input":"2023-01-14T07:31:42.113998Z","iopub.status.idle":"2023-01-14T07:31:48.037780Z","shell.execute_reply.started":"2023-01-14T07:31:42.113968Z","shell.execute_reply":"2023-01-14T07:31:48.036772Z"},"trusted":true},"execution_count":26,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/29.0 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3faa0d7f6e1f497d8d7fdb2ec91449e2"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/570 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ba9ed8fe6ca943a3bb57be92671d31d7"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/208k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0e3ae6c881244e61b92704437316565c"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/426k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"7ed25bba28dd46398e8d0853bf45e559"}},"metadata":{}}]},{"cell_type":"code","source":"berttokenizer.convert_ids_to_tokens(encode(train[\"text\"][:10])[\"input_ids\"][0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluation","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport datasets\nfrom datasets import load_metric\n\nmetric = load_metric(\"accuracy\")\n\ndef compute_metrics(eval_pred):\n logits, labels = eval_pred\n predictions = np.argmax(logits, axis=-1)\n return metric.compute(predictions=predictions, references=labels)","metadata":{"execution":{"iopub.status.busy":"2023-01-04T14:03:06.807561Z","iopub.execute_input":"2023-01-04T14:03:06.807954Z","iopub.status.idle":"2023-01-04T14:03:07.719132Z","shell.execute_reply.started":"2023-01-04T14:03:06.807921Z","shell.execute_reply":"2023-01-04T14:03:07.718198Z"},"trusted":true},"execution_count":17,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading builder script: 0%| | 0.00/1.41k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"756ab470a80e4517a9507633788f499b"}},"metadata":{}}]},{"cell_type":"markdown","source":"## Training loop","metadata":{}},{"cell_type":"code","source":"from transformers import TrainingArguments\nfrom transformers import Trainer\n\nbertmodel = AutoModelForSequenceClassification.from_pretrained(\n'bert-base-cased', num_labels=3)\n\ntraining_args = TrainingArguments(\n '/working',\n overwrite_output_dir=True,\n num_train_epochs=5, \n evaluation_strategy='epoch',\n logging_dir='./logs')\n\ntrainer = Trainer(\n model=bertmodel,\n args=training_args,\n compute_metrics=compute_metrics,\n train_dataset=train_dataset,\n eval_dataset=valid_dataset)","metadata":{"execution":{"iopub.status.busy":"2023-01-04T16:56:07.250522Z","iopub.execute_input":"2023-01-04T16:56:07.250927Z","iopub.status.idle":"2023-01-04T16:56:11.467475Z","shell.execute_reply.started":"2023-01-04T16:56:07.250891Z","shell.execute_reply":"2023-01-04T16:56:11.465980Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":113,"outputs":[{"name":"stderr","text":"loading configuration file https://huggingface.co/bert-base-cased/resolve/main/config.json from cache at /root/.cache/huggingface/transformers/a803e0468a8fe090683bdc453f4fac622804f49de86d7cecaee92365d4a0f829.a64a22196690e0e82ead56f388a3ef3a50de93335926ccfa20610217db589307\nModel config BertConfig {\n \"_name_or_path\": \"bert-base-cased\",\n \"architectures\": [\n \"BertForMaskedLM\"\n ],\n \"attention_probs_dropout_prob\": 0.1,\n \"classifier_dropout\": null,\n \"gradient_checkpointing\": false,\n \"hidden_act\": \"gelu\",\n \"hidden_dropout_prob\": 0.1,\n \"hidden_size\": 768,\n \"id2label\": {\n \"0\": \"LABEL_0\",\n \"1\": \"LABEL_1\",\n \"2\": \"LABEL_2\"\n },\n \"initializer_range\": 0.02,\n \"intermediate_size\": 3072,\n \"label2id\": {\n \"LABEL_0\": 0,\n \"LABEL_1\": 1,\n \"LABEL_2\": 2\n },\n \"layer_norm_eps\": 1e-12,\n \"max_position_embeddings\": 512,\n \"model_type\": \"bert\",\n \"num_attention_heads\": 12,\n \"num_hidden_layers\": 12,\n \"pad_token_id\": 0,\n \"position_embedding_type\": \"absolute\",\n \"transformers_version\": \"4.20.1\",\n \"type_vocab_size\": 2,\n \"use_cache\": true,\n \"vocab_size\": 28996\n}\n\nloading weights file https://huggingface.co/bert-base-cased/resolve/main/pytorch_model.bin from cache at /root/.cache/huggingface/transformers/092cc582560fc3833e556b3f833695c26343cb54b7e88cd02d40821462a74999.1f48cab6c959fc6c360d22bea39d06959e90f5b002e77e836d2da45464875cda\nSome weights of the model checkpoint at bert-base-cased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias']\n- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\nSome weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifier.weight', 'classifier.bias']\nYou should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\nPyTorch: setting up devices\nThe default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).\nUsing the `WAND_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).\n","output_type":"stream"}]},{"cell_type":"code","source":"!ls /working\n!mv /working/bert-base-uncased-5-epochs /kaggle/working/\n!ls /kaggle/working","metadata":{"execution":{"iopub.status.busy":"2023-01-04T18:35:00.910780Z","iopub.execute_input":"2023-01-04T18:35:00.911793Z","iopub.status.idle":"2023-01-04T18:35:05.035460Z","shell.execute_reply.started":"2023-01-04T18:35:00.911752Z","shell.execute_reply":"2023-01-04T18:35:05.034095Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":122,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nbert-base-uncased-5-epochs checkpoint-2000 checkpoint-4000 checkpoint-5000\ncheckpoint-1000\t\t checkpoint-2500 checkpoint-4500\ncheckpoint-1500\t\t checkpoint-3500 checkpoint-500\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n__notebook_source__.ipynb bert-base-uncased-5-epochs freqs.json\nbert-5.zip\t\t checkpoint-3000\t logs\n","output_type":"stream"}]},{"cell_type":"code","source":"!tar -czvf bert-3.zip -C /kaggle/working/checkpoint-3000 .\nfrom IPython.display import FileLink\nFileLink(r'bert-3.zip')","metadata":{"execution":{"iopub.status.busy":"2023-01-04T18:36:47.283378Z","iopub.execute_input":"2023-01-04T18:36:47.283787Z","iopub.status.idle":"2023-01-04T18:37:53.583296Z","shell.execute_reply.started":"2023-01-04T18:36:47.283754Z","shell.execute_reply":"2023-01-04T18:37:53.581943Z"},"trusted":true},"execution_count":128,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n./\n./optimizer.pt\n./scheduler.pt\n./config.json\n./trainer_state.json\n./rng_state.pth\n./pytorch_model.bin\n./training_args.bin\n","output_type":"stream"},{"execution_count":128,"output_type":"execute_result","data":{"text/plain":"/kaggle/working/bert-3.zip","text/html":"<a href='bert-3.zip' target='_blank'>bert-3.zip</a><br>"},"metadata":{}}]},{"cell_type":"code","source":"trainer.train()\ntrainer.save_model('/working/bert-base-uncased-5-epochs')","metadata":{"execution":{"iopub.status.busy":"2023-01-04T14:04:44.605998Z","iopub.execute_input":"2023-01-04T14:04:44.607121Z","iopub.status.idle":"2023-01-04T15:24:50.729458Z","shell.execute_reply.started":"2023-01-04T14:04:44.607077Z","shell.execute_reply":"2023-01-04T15:24:50.728159Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":21,"outputs":[{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/transformers/optimization.py:310: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n FutureWarning,\n***** Running training *****\n Num examples = 17484\n Num Epochs = 5\n Instantaneous batch size per device = 8\n Total train batch size (w. parallel, distributed & accumulation) = 16\n Gradient Accumulation steps = 1\n Total optimization steps = 5465\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"\n <div>\n \n <progress value='5465' max='5465' style='width:300px; height:20px; vertical-align: middle;'></progress>\n [5465/5465 1:19:58, Epoch 5/5]\n </div>\n <table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: left;\">\n <th>Epoch</th>\n <th>Training Loss</th>\n <th>Validation Loss</th>\n <th>Accuracy</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>1</td>\n <td>0.416000</td>\n <td>0.322981</td>\n <td>0.881112</td>\n </tr>\n <tr>\n <td>2</td>\n <td>0.180900</td>\n <td>0.384800</td>\n <td>0.876994</td>\n </tr>\n <tr>\n <td>3</td>\n <td>0.077500</td>\n <td>0.632483</td>\n <td>0.883170</td>\n </tr>\n <tr>\n <td>4</td>\n <td>0.028800</td>\n <td>0.706606</td>\n <td>0.891405</td>\n </tr>\n <tr>\n <td>5</td>\n <td>0.007000</td>\n <td>0.752961</td>\n <td>0.892949</td>\n </tr>\n </tbody>\n</table><p>"},"metadata":{}},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:818: ConvergenceWarning: lbfgs failed to converge (status=1):\nSTOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n\nIncrease the number of iterations (max_iter) or scale the data as shown in:\n https://scikit-learn.org/stable/modules/preprocessing.html\nPlease also refer to the documentation for alternative solver options:\n https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG,\nSaving model checkpoint to /working/checkpoint-500\nConfiguration saved in /working/checkpoint-500/config.json\nModel weights saved in /working/checkpoint-500/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-1000\nConfiguration saved in /working/checkpoint-1000/config.json\nModel weights saved in /working/checkpoint-1000/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n***** Running Evaluation *****\n Num examples = 1943\n Batch size = 16\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-1500\nConfiguration saved in /working/checkpoint-1500/config.json\nModel weights saved in /working/checkpoint-1500/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-2000\nConfiguration saved in /working/checkpoint-2000/config.json\nModel weights saved in /working/checkpoint-2000/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n***** Running Evaluation *****\n Num examples = 1943\n Batch size = 16\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-2500\nConfiguration saved in /working/checkpoint-2500/config.json\nModel weights saved in /working/checkpoint-2500/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-3000\nConfiguration saved in /working/checkpoint-3000/config.json\nModel weights saved in /working/checkpoint-3000/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n***** Running Evaluation *****\n Num examples = 1943\n Batch size = 16\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-3500\nConfiguration saved in /working/checkpoint-3500/config.json\nModel weights saved in /working/checkpoint-3500/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-4000\nConfiguration saved in /working/checkpoint-4000/config.json\nModel weights saved in /working/checkpoint-4000/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n***** Running Evaluation *****\n Num examples = 1943\n Batch size = 16\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-4500\nConfiguration saved in /working/checkpoint-4500/config.json\nModel weights saved in /working/checkpoint-4500/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\nSaving model checkpoint to /working/checkpoint-5000\nConfiguration saved in /working/checkpoint-5000/config.json\nModel weights saved in /working/checkpoint-5000/pytorch_model.bin\n/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.\n warnings.warn('Was asked to gather along dimension 0, but all '\n***** Running Evaluation *****\n Num examples = 1943\n Batch size = 16\n\n\nTraining completed. Do not forget to share your model on huggingface.co/models =)\n\n\nSaving model checkpoint to /working/bert-base-uncased-5-epochs\nConfiguration saved in /working/bert-base-uncased-5-epochs/config.json\nModel weights saved in /working/bert-base-uncased-5-epochs/pytorch_model.bin\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## SentenceTransformer (distilroberta)","metadata":{}},{"cell_type":"code","source":"from sentence_transformers import SentenceTransformer\n\nencoder = SentenceTransformer('all-distilroberta-v1')\n\nlabelencoder = LabelEncoder()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_encoded = encoder.encode(train['text'])\nvalid_encoded = encoder.encode(valid['text'])\n\ntrain_y = labelencoder.fit_transform(train['author'])\nvalid_y = labelencoder.fit_transform(valid['author'])","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:40:23.159373Z","iopub.execute_input":"2022-12-28T12:40:23.159772Z","iopub.status.idle":"2022-12-28T12:40:55.163916Z","shell.execute_reply.started":"2022-12-28T12:40:23.159739Z","shell.execute_reply":"2022-12-28T12:40:55.162723Z"},"trusted":true},"execution_count":18,"outputs":[{"output_type":"display_data","data":{"text/plain":"Batches: 0%| | 0/551 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6e42352e7b7f4bd9a02439691cdd36af"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Batches: 0%| | 0/62 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9627a0bfefc2439b880c0a4815a21ebf"}},"metadata":{}}]},{"cell_type":"markdown","source":"# Making the model head","metadata":{}},{"cell_type":"code","source":"class LinearClassifier(nn.Module):\n def __init__(self):\n super().__init__()\n self.fc = nn.Linear(768, 3)\n \n def forward(self, x):\n return nn.functional.softmax(self.fc(x), dim=1)","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:40:55.167576Z","iopub.execute_input":"2022-12-28T12:40:55.167883Z","iopub.status.idle":"2022-12-28T12:40:55.174044Z","shell.execute_reply.started":"2022-12-28T12:40:55.167839Z","shell.execute_reply":"2022-12-28T12:40:55.172734Z"},"trusted":true},"execution_count":19,"outputs":[]},{"cell_type":"markdown","source":"# Combat overfitting","metadata":{}},{"cell_type":"code","source":"class EarlyStopper:\n def __init__(self, patience=1, min_delta=0):\n self.patience = patience\n self.min_delta = min_delta\n self.counter = 0\n self.min_validation_loss = np.inf\n self.previous_loss = np.inf\n\n def early_stop(self, validation_loss):\n if abs(self.previous_loss - validation_loss) < self.min_delta:\n print('in here')\n self.counter += 1\n if self.counter >= self.patience:\n return True\n self.previous_loss = validation_loss\n return False","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:40:55.175782Z","iopub.execute_input":"2022-12-28T12:40:55.176230Z","iopub.status.idle":"2022-12-28T12:40:55.185512Z","shell.execute_reply.started":"2022-12-28T12:40:55.176188Z","shell.execute_reply":"2022-12-28T12:40:55.184196Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"markdown","source":"# Training the model","metadata":{}},{"cell_type":"code","source":"def train_loop(modelhead, batch_size, epochs):\n print_ = False\n \n early_stopper = EarlyStopper(patience=3, min_delta=1e-4)\n optimizer = torch.optim.Adam(modelhead.parameters())\n loss_fn = torch.nn.CrossEntropyLoss()\n\n losses = {'train':[], 'valid':[]}\n accs = {'train':[], 'valid':[]}\n for i in tqdm(range(epochs)):\n # Train loop\n modelhead.train()\n trainlen = train_encoded.shape[0]\n train_loss = 0\n train_correct = 0\n num_ex = 0\n for i in range(0, trainlen, batch_size):\n x = torch.tensor(train_encoded[i:i+batch_size], device=device)\n y = torch.tensor(train_y[i:i+batch_size], device=device)\n\n logits = modelhead(x)\n prediction = torch.argmax(logits, dim=1)\n loss = loss_fn(logits, y)\n\n train_correct += sum(prediction == y)\n train_loss += loss.item()*x.size(0)\n\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n if print_:\n print(f\"Average training loss: {train_loss/trainlen:.6f}\")\n print(f\"Average training acc: {(train_correct / trainlen) *100:.2f} %\")\n\n losses['train'].append(train_loss/trainlen)\n accs['train'].append(train_correct / trainlen*100)\n\n # Validation\n modelhead.eval()\n with torch.no_grad():\n valid_loss = 0\n validlen = valid_encoded.shape[0]\n valid_correct = 0\n for i in range(0, validlen, batch_size):\n x = torch.tensor(valid_encoded[i:i+batch_size], device=device)\n y = torch.tensor(valid_y[i:i+batch_size], device=device)\n\n logits = modelhead(x)\n prediction = torch.argmax(logits, dim=1)\n loss = loss_fn(logits, y)\n\n valid_correct += sum(prediction == y)\n valid_loss += loss.item()*x.size(0) \n\n if print_:\n print(f\"Average Loss: {valid_loss / validlen:.6f}\")\n print(f\"Average acc: {(valid_correct / validlen) *100:.2f} %\")\n\n losses['valid'].append(valid_loss/validlen)\n accs['valid'].append(valid_correct / validlen*100)\n \n \"\"\"\n if early_stopper.early_stop(valid_loss/validlen):\n print(f'stopping after {i} epochs')\n break\n \"\"\"\n\n print(f\"Final valid: loss: {losses['valid'][-1]}, accuracy: {accs['valid'][-1]}\")\n \n return losses, accs","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:40:55.188388Z","iopub.execute_input":"2022-12-28T12:40:55.189092Z","iopub.status.idle":"2022-12-28T12:40:55.204665Z","shell.execute_reply.started":"2022-12-28T12:40:55.189056Z","shell.execute_reply":"2022-12-28T12:40:55.203821Z"},"trusted":true},"execution_count":21,"outputs":[]},{"cell_type":"code","source":"Multiclass_head = LinearClassifier()\n\nlosses, accs = train_loop(Multiclass_head.to(device), 128, 400)","metadata":{"_kg_hide-output":true,"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(losses['valid'][-10:])\nprint(losses['valid'][-2] - losses['valid'][-1])","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:43:13.702676Z","iopub.execute_input":"2022-12-28T12:43:13.703590Z","iopub.status.idle":"2022-12-28T12:43:13.710441Z","shell.execute_reply.started":"2022-12-28T12:43:13.703552Z","shell.execute_reply":"2022-12-28T12:43:13.709170Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":23,"outputs":[{"name":"stdout","text":"[0.7920745982578266, 0.7920642092108605, 0.7920538728278552, 0.7920436533405784, 0.7920334891352395, 0.7920234216124094, 0.7920134464493811, 0.7920035187752424, 0.7919936866268318, 0.7919839012367124]\n9.785390119421145e-06\n","output_type":"stream"}]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nprint(f\"Final acc: {accs['valid'][-1]:.2f} %\")\nplt.plot(list(range(len(losses['train']))), losses['train'])\nplt.plot(list(range(len(losses['valid']))), losses['valid'])\nplt.show()\n\n#accs['train'] = accs['train'].cpu()\n#accs['train'] = [t.cpu().item() for t in accs['train']]\n#accs['valid'] = [t for t in accs['valid']]\n\nplt.figure()\nplt.plot(list(range(len(accs['train']))), accs['train'])\nplt.plot(list(range(len(accs['valid']))), accs['valid'])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-28T12:43:13.712163Z","iopub.execute_input":"2022-12-28T12:43:13.712927Z","iopub.status.idle":"2022-12-28T12:43:14.198678Z","shell.execute_reply.started":"2022-12-28T12:43:13.712890Z","shell.execute_reply":"2022-12-28T12:43:14.197205Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":24,"outputs":[{"name":"stdout","text":"Final acc: 75.33 %\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlRUlEQVR4nO3deXQd9X338ff37tolS/KCNxnbLGYJEAMhhCWkEEMbTNP2CSQNpM1TmqehfbqdFE77NCltTpfTJk2ahJQklJA2IZSGhDSkQAgkYbcNtsEY4w2wZGzLkrUvd/s9f8xIurposy3dkUef1zlzZuY3c3W/Gsuf39zfzL3XnHOIiEh4RYIuQEREZpaCXkQk5BT0IiIhp6AXEQk5Bb2ISMjFgi6gWENDg2tqagq6DBGRE8qmTZsOO+cax9o264K+qamJjRs3Bl2GiMgJxczeGG+bhm5EREJOQS8iEnIKehGRkFPQi4iEnIJeRCTkFPQiIiGnoBcRCbnQBH3vYJbPPbKDzfs6gi5FRGRWCU3QD2RyfPGnu9iioBcRGSU0QR+Ler9KJpcPuBIRkdklNEEfjxoA2by+MUtEpFBogj4W8X6VrM7oRURGCU3QD53RZ3I6oxcRKRSaoDczohEjm9cZvYhIoUmD3szuMrNDZvbyONvNzL5oZrvMbKuZnVewLWdmm/3pweksfCyxiJHVGb2IyChTOaO/G1g3wfargdX+dDNwR8G2fufcOf507TFXOUXxaERDNyIiRSYNeufcz4H2CXZZD9zjPM8CtWa2aLoKPBqxqIZuRESKTccY/WJgX8F6s98GkDKzjWb2rJldN94PMLOb/f02tra2HnMhsYjO6EVEis30xdjlzrm1wIeBfzazlWPt5Jy70zm31jm3trFxzK88nJJ41HR7pYhIkekI+hZgacH6Er8N59zQfA/wBHDuNDzfuLyhG53Ri4gUmo6gfxC40b/75l1Ap3PuLTOrM7MkgJk1ABcDr0zD840rHonoIxBERIrEJtvBzL4DXA40mFkz8GkgDuCc+yrwEHANsAvoA37Lf+jpwL+aWR6vQ/k759yMBn0sqtsrRUSKTRr0zrkbJtnugE+O0f40cNaxl3b0YpGI7roRESkSmnfGgncxVnfdiIiMFqqgj0V1Ri8iUixcQR/RGb2ISLFQBX08GtF99CIiRUIV9LqPXkTk7cIT9P1H+LODn2Jt/9NBVyIiMquEJ+id4/SBF6nPHgq6EhGRWSU8QR9NABBxmYALERGZXcIX9HkFvYhIoRAFfdybKehFREYJT9CbkbWYhm5ERIqEJ+iBnMWJumzQZYiIzCqhCvq8xYjqjF5EZJRQBX0uojN6EZFioQr6vMWJ6YxeRGSUcAV9JEaMLHl9DIKIyLBwBb3FiZMlo48qFhEZFq6gj8RJkNXXCYqIFAhV0LuId0avoBcRGRGuoI8mNHQjIlIkVEGfj8SJW05n9CIiBUIV9Phj9Bl9y5SIyLBQBb2L+mP0ur1SRGRYyII+QYycvjdWRKTApEFvZneZ2SEze3mc7WZmXzSzXWa21czOK9h2k5nt9KebprPwMWvxz+gzGqMXERk2lTP6u4F1E2y/GljtTzcDdwCY2Tzg08CFwAXAp82s7niKnVQ0QYIsaZ3Ri4gMmzTonXM/B9on2GU9cI/zPAvUmtki4P3Ao865dufcEeBRJu4wjptFE8QtSzqroBcRGTIdY/SLgX0F681+23jtb2NmN5vZRjPb2NraesyFWMy7j15BLyIyYlZcjHXO3emcW+ucW9vY2HjMP8diSeJkGczmprE6EZET23QEfQuwtGB9id82XvuMicT8MXqd0YuIDJuOoH8QuNG/++ZdQKdz7i3gYeAqM6vzL8Je5bfNmEjcH7rRxVgRkWGxyXYws+8AlwMNZtaMdydNHMA591XgIeAaYBfQB/yWv63dzP4a2OD/qNudcxNd1D1u0ViCqDnSGX35iIjIkEmD3jl3wyTbHfDJcbbdBdx1bKUdvWgsCUA2PViqpxQRmfVmxcXY6RKNe0GfySjoRUSGhDLoc+l0wJWIiMweoQr6WMIPep3Ri4gMC1XQR2IJAPJZBb2IyJBQBT1RL+izOqMXERkWsqCPA+B0Ri8iMixcQR9LAeAyAwEXIiIye4Qy6E1BLyIyLFxBHy/z5lkFvYjIkHAFvX9Gr6AXERkRrqD3z+gjOQW9iMiQcAW9f0YfyfYHXIiIyOwRrqAfPqPX7ZUiIkPCFfRDZ/QauhERGRauoPfP6GN5ndGLiAwJV9BHomSJEVXQi4gMC1fQA5lIknheQzciIkNCF/TZSJJYXp9HLyIyJKRBrzN6EZEhoQv6XDRJPD+I91W2IiISuqDPR1MkyZDO5YMuRURkVghd0LtoihRp+tO5oEsREZkVwhf0sRQpS9OnoBcRAcIY9PEyUijoRUSGTCnozWydme0ws11mdusY25eb2WNmttXMnjCzJQXbcma22Z8enM7ix6w1pqEbEZFCscl2MLMo8GXgSqAZ2GBmDzrnXinY7R+Be5xz3zSzK4C/BT7qb+t3zp0zvWVPUG+ijKRlOJzOluopRURmtamc0V8A7HLO7XHOpYF7gfVF+6wBfuovPz7G9pKJ+EM3/Rmd0YuIwNSCfjGwr2C92W8rtAX4oL/8q0CVmdX76ykz22hmz5rZdcdT7FREkuWUMaihGxER33RdjP1T4DIzexG4DGgBhpJ2uXNuLfBh4J/NbGXxg83sZr8z2Nja2npchUSTVVTYIH2DmeP6OSIiYTGVoG8BlhasL/Hbhjnn9jvnPuicOxf4c7+tw5+3+PM9wBPAucVP4Jy70zm31jm3trGx8Rh+jRHR8hoAMgPdx/VzRETCYipBvwFYbWYrzCwBXA+MunvGzBrMbOhn3Qbc5bfXmVlyaB/gYqDwIu60i5dVAZDr65zJpxEROWFMGvTOuSxwC/AwsB24zzm3zcxuN7Nr/d0uB3aY2WvAAuCzfvvpwEYz24J3kfbviu7WmXYJ/4w+N9Azk08jInLCmPT2SgDn3EPAQ0Vtf1mwfD9w/xiPexo46zhrPCqRVLX33IM6oxcRgRC+M5ZkJQBOZ/QiIkAYgz7hBT1pXYwVEYEwBn3SuxgbSeuMXkQEFPQiIqEXvqD3h24iGQW9iAiEMehjSbLEiGR6g65ERGRWCF/QmzEYLSeuM3oRESCMQQ9kouXEc31BlyEiMiuEM+hjlaTyvWT1BeEiIuEM+lyimmrro2dQXz4iIhLOoE/WUksP3QMKehGRUAa9K6uj1nro7Ndn0ouIhDLorXwedfTQ1Z8OuhQRkcCFMuijFfUkLUNfrz7vRkQklEGfqGoAYKDrcMCViIgEL5RBn6z2vpc829MWcCUiIsELZdCnqr3vnc1064xeRCSUQR+t8M/oe9sDrkREJHihDHrK5wHg+hT0IiLhDPoyL+ij/RqjFxEJZ9DHEvREqkkNaoxeRCScQQ/0JuqpzOiMXkQktEE/kGykLt+uT7AUkTkvtEGfLZ/PfOvgSJ8+70ZE5rbQBr2rWkAjHbT3DAZdiohIoKYU9Ga2zsx2mNkuM7t1jO3LzewxM9tqZk+Y2ZKCbTeZ2U5/umk6i59IrGohSctypO1QqZ5SRGRWmjTozSwKfBm4GlgD3GBma4p2+0fgHufc2cDtwN/6j50HfBq4ELgA+LSZ1U1f+eOrqF8MQNfh5lI8nYjIrDWVM/oLgF3OuT3OuTRwL7C+aJ81wE/95ccLtr8feNQ51+6cOwI8Cqw7/rInV71gOQADh98sxdOJiMxaUwn6xcC+gvVmv63QFuCD/vKvAlVmVj/Fx2JmN5vZRjPb2NraOtXaJ5RsaAIg36GgF5G5bbouxv4pcJmZvQhcBrQAuak+2Dl3p3NurXNubWNj4/RUVLWILFHi3S3T8/NERE5QsSns0wIsLVhf4rcNc87txz+jN7NK4Neccx1m1gJcXvTYJ46j3qmLRGmPzaeiX0EvInPbVM7oNwCrzWyFmSWA64EHC3cwswYzG/pZtwF3+csPA1eZWZ1/EfYqv60kulKLqEsfKNXTiYjMSpMGvXMuC9yCF9Dbgfucc9vM7HYzu9bf7XJgh5m9BiwAPus/th34a7zOYgNwu99WEoMVi1ngDjGQmfIokohI6Exl6Abn3EPAQ0Vtf1mwfD9w/ziPvYuRM/ySitQ1sfDgD9nd2sbKk+YHUYKISOBC+85YgOTCUwE4/Mb2gCsREQlOqIO+bpn3vq6+/Qp6EZm7Qh30tUtOA8Ad3hVwJSIiwQl10FuigoPWSFnXnqBLEREJTKiDHqCtrIn6/r1BlyEiEpjQB31v3eksz73JwMBA0KWIiAQi9EEfPekskpalZdeWoEsREQlE6IO+bsV5AHTsfSHgSkREghH6oD9p1VkMuDhu/+agSxERCUTogz6ZSPJa/FTq2l4MuhQRkUCEPugBDs87j+XpnbjB7qBLEREpuTkR9JHlFxEjz8FXngq6FBGRkpsTQb/wjEvJO6Njx8+DLkVEpOTmRNCvWnoSO1hGvOX5oEsRESm5ORH0sWiEveVnc1LPS5DLBF2OiEhJzYmgB+g56WLK3ACZPU8GXYqISEnNmaCvO+sqBlyctk0PBF2KiEhJzZmgv/C0ZTzpzqJsz8PgXNDliIiUzJwJ+upUnNdqL6UmfQAOvBR0OSIiJTNngh4gdto15J3Rv/X7QZciIlIycyrozz/zFJ7OryG/5V7I54MuR0SkJOZU0J+9pJYfxX6Jir4W2PuzoMsRESmJORX00YgRO+MDdLoKspvuCbocEZGSmFNBD3D1uSv4Xu492Ks/hJ7WoMsREZlxUwp6M1tnZjvMbJeZ3TrG9mVm9riZvWhmW83sGr+9ycz6zWyzP311un+Bo3Xhinp+lPplLJ+F5wIvR0Rkxk0a9GYWBb4MXA2sAW4wszVFu/0FcJ9z7lzgeuArBdt2O+fO8adPTFPdxywaMc5+x/k8kj8f9/ydMNAVdEkiIjNqKmf0FwC7nHN7nHNp4F5gfdE+Dqj2l2uA/dNX4vT7jbVL+FLmWmywCzZ8PehyRERm1FSCfjGwr2C92W8r9BngN82sGXgI+P2CbSv8IZ2fmdklYz2Bmd1sZhvNbGNr68yPm5++qJryprU8E3kn7snPQ2/bjD+niEhQputi7A3A3c65JcA1wLfMLAK8BSzzh3T+GPi2mVUXP9g5d6dzbq1zbm1jY+M0lTSxmy5q4v/1fwiX7oWf/X1JnlNEJAhTCfoWYGnB+hK/rdDHgfsAnHPPACmgwTk36Jxr89s3AbuBU4636Olw1RkL6KlaxU/K1sHGb8DBV4IuSURkRkwl6DcAq81shZkl8C62Pli0z5vA+wDM7HS8oG81s0b/Yi5mdjKwGtgzXcUfj3g0wu9edjJ/1v4BMvFq+MEnIZcNuiwRkWk3adA757LALcDDwHa8u2u2mdntZnatv9ufAL9jZluA7wAfc8454FJgq5ltBu4HPuGca5+B3+OY3HDBMuJVjXyl7GbY/wI886WgSxIRmXbmZtlH9q5du9Zt3LixZM9391N7+cwPt/H8qm8yf//j8NsPw5J3luz5RUSmg5ltcs6tHWvbnHtnbLEPX7ickxsq+fiRm3BVC+E/PwZ9s+ZFh4jIcZvzQZ+IRfh/H1jDS23GA6s+Cz0H4Ts3QGYg6NJERKbFnA96gPeeOp8r1yzgtucSHHzf52Hfs/D9T+ijjEUkFBT0vr+57kxS8Si/t2UF+ff9FWx7AH78KYW9iJzwFPS+BdUp/uraM9j0xhE+37cOLroFNnwNfvRHCnsROaHFgi5gNrnu3MU8vfsw//L4bs658fd5XzQBT37OG6+/9l8glgi6RBGRo6Yz+iK3rz+TM06q5g/v28Lr7/gTeO9fwNZ74VvX6TNxROSEpKAvkopH+epvvpNoxLjp7g20nvcH8GvfgOaN8PUrYP+LQZcoInJUFPRjWDqvnG/cdD6Huga56a7n6Vq9Hj72I8hl4OtXwjNfgVn2RjMRkfEo6MfxzuV13PGb5/HawW5++9820NV4DnziSVh9JTx8G3zzA9C2O+gyRUQmpaCfwOWnzueLN5zL5n0dfPhrz9KWr4Drvw0f+AK8tRXueDf84p8gmw66VBGRcSnoJ3HNWYv42o1r2Xmwh9/412d4o70P3vkxuOV5WH0VPHY7fOVCeOUHGs4RkVlJQT8F7z1tPvf89gW09aRZ/+WneHr3YahaCB/6Fnzkfogm4b4b4RtXwu7HFfgiMqso6KfowpPrefCWi2moTPLRbzzPv/5sN/m888bsP/Gkd599Z7N3G+bX3uud4edzQZctIqKPKT5a3QMZPnX/Vn788gEuXlXP5/7XOSyoTnkbMwPePfdPfQHa90D9Kjj/f8M7roeyumALF5FQm+hjihX0x8A5x30b9/GZB18hFY/wmWvP4Np3nISZeTvkc7D9QXj6S9CyEWIpWHOdN7a/9EKI6IWUiEwvBf0M2d3awx9/dzNbmju5ZHUDf73+TJoaKkbv9NZW2HQ3bL0P0t1QvQTOuA7O/CCcdB4MdQ4iIsdBQT+DcnnHfzz3Bv/wPztI5/L8ziUr+N3LVlKdio/eMd0L2/8btn0Pdj0G+QzUNcFpvwKnvB+WvkufpSMix0xBXwIHuwb424e28/3N+6krj/PJ967ioxctJxmLvn3n/iN+6D8Ae3/uhX6iClZeDquuhBWXep2AzvZFZIoU9CX0cksnf/8/r/KLnYdZXFvGJy47md9Yu5RUfIzABxjsgb0/g52PwM5HoavFa69eDMsvhqaLYfl7oH6lgl9ExqWgD8BTuw7zj4/s4MU3O6ivSPBbFzfx0Xc1UVMeH/9BzkHrq/D6k/DGU/D6U9B7yNtWXu+N6S8+b2ReOb80v4yIzHoK+oA453h+bzt3/Gw3T+xopTwRZf05i/nIhcs4c3HNVH4AHN4JbzwJzZtg/wteR+D8L0KpWQoLz4YFa2D+GlhwBsxbCVF9zYDIXKOgnwW2v9XFXU/u5Ydb9zOQyfOOpbV85MJl/MrZiyhPHEUwD/bAga3Q8gK0bIKDL0PbrpHwjyag4VQv/BtWe8FfvwrmnQzJypn55UQkcAr6WaSzP8MDLzTz78+9ya5DPZQnorz/jIWsP+ck3rOqgVj0GO6xzwzA4dfg0CtwcBsc2u4tD433D6lc6I3116/0OoB5J0PtUqhZBuXzdA1A5ASmoJ+FhoZ1HnixhYdeeouugSz1FQl++exFrDtzIRc0zTu20C+U7vXeodu22zvrL1zuOzx633i5NxRUu3T0vGap97k+VQshXnZ89YjIjDnuoDezdcAXgCjwdefc3xVtXwZ8E6j197nVOfeQv+024ONADvgD59zDEz3XXAn6QoPZHE/saOUHm1v4yfZDpLN5asvjXHHqfK46YwGXntJ4dMM7UzHQCe17oXMfdOzz52+OrPe3v/0xqRqoWuQH/yKoXFCwvtC7OFzeAMkqvToQKbHjCnoziwKvAVcCzcAG4Abn3CsF+9wJvOicu8PM1gAPOeea/OXvABcAJwE/AU5xzo37aV9zMegL9Q5m+cXOVh7ZdpDHXj1EZ3+GRCzCRSfXc8nqBi5Z3cgpCypHPm5hpqR7/Q6gGXoOQLc/FS53H/DeA1AsmvACv6LenzeMs94AZfO8DkQXkEWOy0RBP5X/XRcAu5xze/wfdi+wHnilYB8HVPvLNcB+f3k9cK9zbhDYa2a7/J/3zFH/FnNERTLGujMXse7MRWRyeTa83s6jrxzk56+18jc/2g5sZ35VkktWN3LJ6gbevbKe+UMfqjadEhUw/zRvGk8+7735q/stL/R7W72p77D3Rep9h6H3MBzZ662nuyd4viooq/VCP1XrL082r/FePcRSegUhMoGpBP1iYF/BejNwYdE+nwEeMbPfByqAXyp47LNFj11c/ARmdjNwM8CyZcumUvecEI9GePfKBt69sgGAlo5+ntzZyi92HuaxVw/yXy80A7C8vpzzm+ZxQdM8zl8xj6b68pk/4wfvw9kq6r1p4ZmT758ZgL6CDqCvDfravWGkgQ7o7xiZt+8ZWc/0TfxzLeoF/tCUqPSX/XmiauL1eAUkyr3rFPFyfRSFhM50vV6+AbjbOfdPZnYR8C0zm8L/fI9z7k7gTvCGbqapptBZXFvGh85fxofOX0Yu79i2v5Pn9rTz/OvtPLb9IPdv8oK/oTLJ+U11nLesjrOX1HDm4hoqkrNgaCSegprF3nQ0smkv8Ac6R3cGAx2Q7oHBbu+208FuGOzy2gY6vGGnwe6RfZjin1Yk5oV/vMzvAIY6grIpLA91Finv942lIJaEWJk/L2iPxPRKREpiKv/7W4ClBetL/LZCHwfWATjnnjGzFNAwxcfKMYhGjLOX1HL2klp+59KTyecdew738PzeI2x4vZ3n97bz45cPABAxWD2/irOX1HD20lresaSG0xZWk4idIB+XHEt4F3qP553A+Txkekc6hHS33zF0Q7rPe9WQ6fOXeyHT712nyPSNLKd7vVcixe3jX3KamEXG7gCG1mMF6/GyovYkROPe9ZDiKVa4Hve+AW1oeaLH6TpJaE3lYmwM72Ls+/BCegPwYefctoJ9fgx81zl3t5mdDjyGN0SzBvg2IxdjHwNW62JsabR2D/JSSweb93WytbmDrc2dtPd6X2SeiEZYNb+S0xZVsWZRNactrOb0RVXUVyYDrvoE4xzk0gWdhD9lByE74A1XZQf89f4x2gunQa/zGNqnsH34Mf7Pyc3AF9JbZIwOIj66I4jEvbZIbGSKxiESnWDbGMvjbvN/VjTuL8f8542NrEei3mTRkXWLFLQVziMF+xS0hfCV1HFdjHXOZc3sFuBhvFsn73LObTOz24GNzrkHgT8BvmZmf4T3+vhjzutBtpnZfXgXbrPAJycKeZlejVVJrjhtAVectgDw7t1vPtLP1uZOtrZ0sP2tbp7ceZjvvdAy6jGnL6rm9IVVnL6omlXzKzm5sWL6b+8MCzP/TDtZ2m8Rcw7yWS/4c2nIZSA36M/T3pRNjyyPmjJFjyvani3aNzfoteWz3l1W+Szk/OcuXB/aXrhcvG3oHdxBs0hRpxDzrjmN2VEUrEdik3QqQ/sUdCjDncvQfOjx9vb2uiZ49y3T/+vqDVPS1jPIqwe62f5WF9vf8ua7DvWQzo38p1xcW8bK+ZWsaqxk1fxKVjZWsGp+JfMqEqW58CvhkM/7oT+FTuFtHUjG+/a2fNYbLhteznvLQ23D28ZoG7Vvdoy2KT6+sIYxH5/1OmOXL9juL7vcyLbi9kXnwI3fP6ZDe7y3V0rI1VcmuXhVkotXNQy3ZXJ59rT2sru1h12HeobnG/a2058ZeVFWWx5nZWMlTfUVLK8v96cKmurLqS3X3StSJBKBSALQ30YpKehlTPFohFMXVnHqwqpR7fm8Y39nP7tbe4c7gN2Henh692H+64WBUftWp2I0NVSwbN5IB7B8XjlL55WzoDpFNKJXAiKloKCXoxKJGEvqyllSV85lpzSO2jaQybGvvY/X2/p4o62XN9r6eL2tl5daOvnxywfI5UeGCWMRY2FNisW1ZSyuK2OJP19cW87iujJOqk2N/e1cInLUFPQybVLxKKsXVLF6QdXbtmVyefZ39PN6Wx/NR/poOdJPS0c/LUf6eWZ3Gwe6Bii+XNRYlRzuCE6qSbGgOsXCmhQLq73lBdWpE+cWUZEAKeilJOLRiDd0U18x5vZMLs+BzgGaCzqAlo4+Wjr62dbSyU9eOchg9u13bNRXJIY7gAXVXiewsCY5qlOoKYvrgrHMaQp6mRXi0QhL/fH7sTjn6OzPcKBrgAOdAxzsGuBA5yAHuoaWB9iyr4O23rffX56IRWisTNJQlaSxMkljVWLUeuG8IhFVpyCho6CXE4KZUVueoLY8wWkLq8fdbzCb41DXoBf+fgdwqHuQw92DtPYM0nykj837jtDWm37bUBFAWTxKw1BHUJmkscqbN1QmmFeRpK4iTr0/n1eeOP7vDBApAQW9hEoyFp3wlcGQbC5Pe1+aw91pWntGOoLhec8gr7f1svGNI8PvJh5LdSpGfWWSuvI48yqSzKvw5vUVCeoqEm+b6xWDBEFBL3NSLBphflWK+VWTf8RzJpfnSF+a9t6R6UhvmrbCeV+a5iN9vNTibc/kxn4jYiIWoa48Tm1ZgpryOLVlcWrK4tSWx6ktTwwv15R5+9SWx6kpj1OVjKmDkGOmoBeZRPwoOgXwrif0DGZHdQyFU0dfho5+b/5mex+d/Rk6+jKj3ohWLBoxavxOYbhjKPM6h+qykQ6juixOVSpGVSpGdSpOdSpOZSqm9yzMcQp6kWlmZlSl4lSl4uPeZTSWgUyOrv4MHX7wex1Aergj6OhP09mfpcN/dbGntZeOvjTdg9kxrzcUqkyOhH9VKjbcIQytV6XiVJf589TIfGi/sriGnE5kCnqRWSIVj5KKR4/6G8NyeUf3gNcZdA9k6R7I0DWQoWsgS/dAlq5+r71rIONt689yqHuAXYe8fbsHsmTzE/cUsYgNdwiVyRiVyRgVySiVqTiVyai/HivYFqMy5S8nvE5maLve+1B6CnqRE1w0MnJH0rFwztGfyQ13Cl3DncLbO4nugSy9g14H0tozyN7DvfQM5ugdzE449FQoEY34ncToTqAiGaMqOdZylPJEjPJElPJElIqk9wqjIum1JWMRvdqYhIJeZI4zMz9IYyw4ju8fzuby9Ka90O8ZmvyOYWi9dzBLtz/vHcwNdxxtPWnebOsb3q8vPfVPM48YVCRilCVGwt+bYsPzimTU217cFi98zOjOJEwdiIJeRKZFLBqhpixCTVn8uH9WLu/oS4+Eft9gjt50lv60N/fasvSmcyNtgzn6Ml57XzpHR1+alo7cqMekx3h39XgiBuV+B1Ke8DqFVLxg2Z8XbhvaNxX32oa2D+1bvB4v0fswFPQiMutEIyMXtKdTJpenr6Bz6PdfgfSlc/QNdxhZv8PI+e3esFR/Ojc87+z37pIaSHudS386N+ZHdEwmFjGvA0h401mLa/jSh8+b1t8ZFPQiMofEp/FVR7F83jGQzQ13JAOZkY6hz+8U+gva+ovXMzmW1JVNe12goBcRmRaRyMi1jtlG9zmJiIScgl5EJOQU9CIiIaegFxEJOQW9iEjIKehFREJOQS8iEnIKehGRkDM32QdZl5iZtQJvHMePaAAOT1M500l1HR3VdXRma10we2sLW13LnXONY22YdUF/vMxso3NubdB1FFNdR0d1HZ3ZWhfM3trmUl0auhERCTkFvYhIyIUx6O8MuoBxqK6jo7qOzmytC2ZvbXOmrtCN0YuIyGhhPKMXEZECCnoRkZALTdCb2Toz22Fmu8zs1oBred3MXjKzzWa20W+bZ2aPmtlOf15XolruMrNDZvZyQduYtZjni/4x3Gpm0/+dZhPX9Rkza/GP22Yzu6Zg221+XTvM7P0zWNdSM3vczF4xs21m9n/99kCP2QR1BXrMzCxlZs+b2Ra/rr/y21eY2XP+83/XzBJ+e9Jf3+VvbypxXXeb2d6C43WO316yv33/+aJm9qKZ/be/PrPHyzl3wk9AFNgNnAwkgC3AmgDreR1oKGr7B+BWf/lW4O9LVMulwHnAy5PVAlwD/Bgw4F3AcyWu6zPAn46x7xr/3zQJrPD/raMzVNci4Dx/uQp4zX/+QI/ZBHUFesz837vSX44Dz/nH4T7ger/9q8D/8Zd/D/iqv3w98N0ZOl7j1XU38Otj7F+yv33/+f4Y+Dbw3/76jB6vsJzRXwDscs7tcc6lgXuB9QHXVGw98E1/+ZvAdaV4Uufcz4H2KdayHrjHeZ4Fas1sUQnrGs964F7n3KBzbi+wC+/ffCbqess594K/3A1sBxYT8DGboK7xlOSY+b93j78a9ycHXAHc77cXH6+h43g/8D4zsxLWNZ6S/e2b2RLgl4Gv++vGDB+vsAT9YmBfwXozE/8nmGkOeMTMNpnZzX7bAufcW/7yAWBBMKVNWMtsOI63+C+d7yoY3gqkLv9l8rl4Z4Oz5pgV1QUBHzN/GGIzcAh4FO/VQ4dzLjvGcw/X5W/vBOpLUZdzbuh4fdY/Xp83s2RxXWPUPN3+GfgUkPfX65nh4xWWoJ9t3uOcOw+4GvikmV1auNF5r8NmxX2ts6kW4A5gJXAO8BbwT0EVYmaVwH8Bf+ic6yrcFuQxG6OuwI+Zcy7nnDsHWIL3quG0UtcwluK6zOxM4Da8+s4H5gF/VsqazOxXgEPOuU2lfN6wBH0LsLRgfYnfFgjnXIs/PwQ8gPfHf3DopaA/PxRUfRPUEuhxdM4d9P9z5oGvMTLUUNK6zCyOF6b/4Zz7nt8c+DEbq67Zcsz8WjqAx4GL8IY+YmM893Bd/vYaoK1Eda3zh8Ccc24Q+DdKf7wuBq41s9fxhpivAL7ADB+vsAT9BmC1f+U6gXfR4sEgCjGzCjOrGloGrgJe9uu5yd/tJuAHQdTnG6+WB4Eb/TsQ3gV0FgxXzLiiMdFfxTtuQ3Vd79+BsAJYDTw/QzUY8A1gu3PucwWbAj1m49UV9DEzs0Yzq/WXy4Ar8a4fPA78ur9b8fEaOo6/DvzUf4VUirpeLeisDW8cvPB4zfi/o3PuNufcEudcE15O/dQ59xFm+nhN55XkICe8q+av4Y0P/nmAdZyMd7fDFmDbUC1442qPATuBnwDzSlTPd/Be0mfwxv4+Pl4teHccfNk/hi8Ba0tc17f8593q/4EvKtj/z/26dgBXz2Bd78EbltkKbPana4I+ZhPUFegxA84GXvSf/2XgLwv+HzyPdxH4P4Gk357y13f5208ucV0/9Y/Xy8C/M3JnTsn+9gtqvJyRu25m9HjpIxBEREIuLEM3IiIyDgW9iEjIKehFREJOQS8iEnIKehGRkFPQi4iEnIJeRCTk/j+H1q3RcTs0bQAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}},{"traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_23/2111547878.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maccs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'train'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'train'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 14\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maccs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'valid'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'valid'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 15\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/matplotlib/pyplot.py\u001b[0m in \u001b[0;36mplot\u001b[0;34m(scalex, scaley, data, *args, **kwargs)\u001b[0m\n\u001b[1;32m 2767\u001b[0m return gca().plot(\n\u001b[1;32m 2768\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscalex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mscalex\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscaley\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mscaley\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2769\u001b[0;31m **({\"data\": data} if data is not None else {}), **kwargs)\n\u001b[0m\u001b[1;32m 2770\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2771\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/matplotlib/axes/_axes.py\u001b[0m in \u001b[0;36mplot\u001b[0;34m(self, scalex, scaley, data, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1633\u001b[0m \"\"\"\n\u001b[1;32m 1634\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcbook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnormalize_kwargs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmlines\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mLine2D\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1635\u001b[0;31m \u001b[0mlines\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_lines\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1636\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mline\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mlines\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1637\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_line\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mline\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/matplotlib/axes/_base.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, data, *args, **kwargs)\u001b[0m\n\u001b[1;32m 310\u001b[0m \u001b[0mthis\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 311\u001b[0m \u001b[0margs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 312\u001b[0;31m \u001b[0;32myield\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_plot_args\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mthis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 313\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 314\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_next_color\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/matplotlib/axes/_base.py\u001b[0m in \u001b[0;36m_plot_args\u001b[0;34m(self, tup, kwargs, return_kwargs)\u001b[0m\n\u001b[1;32m 486\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 487\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_check_1d\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 488\u001b[0;31m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_check_1d\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 489\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 490\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mindex_of\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/matplotlib/cbook/__init__.py\u001b[0m in \u001b[0;36m_check_1d\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 1309\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'ndim'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1310\u001b[0m len(x.shape) < 1):\n\u001b[0;32m-> 1311\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0matleast_1d\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1312\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1313\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m<__array_function__ internals>\u001b[0m in \u001b[0;36matleast_1d\u001b[0;34m(*args, **kwargs)\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/numpy/core/shape_base.py\u001b[0m in \u001b[0;36matleast_1d\u001b[0;34m(*arys)\u001b[0m\n\u001b[1;32m 63\u001b[0m \u001b[0mres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 64\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mary\u001b[0m \u001b[0;32min\u001b[0m \u001b[0marys\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 65\u001b[0;31m \u001b[0mary\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0masanyarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mary\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 66\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mary\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndim\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 67\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mary\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/torch/_tensor.py\u001b[0m in \u001b[0;36m__array__\u001b[0;34m(self, dtype)\u001b[0m\n\u001b[1;32m 730\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mhandle_torch_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mTensor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__array__\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 731\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mdtype\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 732\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 733\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 734\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mastype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mTypeError\u001b[0m: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first."],"ename":"TypeError","evalue":"can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.","output_type":"error"},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAANT0lEQVR4nO3cYYjkd33H8ffHO1NpjKb0VpC706T00njYQtIlTRFqirZc8uDugUXuIFgleGAbKVWEFEuU+MiGWhCu1ZOKVdAYfSALntwDjQTEC7chNXgXItvTeheFrDHNk6Ax7bcPZtKdrneZf3Zndy/7fb/gYP7/+e3Mlx97752d2ZlUFZKk7e8VWz2AJGlzGHxJasLgS1ITBl+SmjD4ktSEwZekJqYGP8lnkzyZ5PuXuD5JPplkKcmjSW6c/ZiSpPUa8gj/c8CBF7n+VmDf+N9R4F/WP5YkadamBr+qHgR+/iJLDgGfr5FTwNVJXj+rASVJs7FzBrexGzg/cXxhfO6nqxcmOcrotwCuvPLKP7z++utncPeS1MfDDz/8s6qaW8vXziL4g1XVceA4wPz8fC0uLm7m3UvSy16S/1zr187ir3SeAPZOHO8Zn5MkXUZmEfwF4F3jv9a5GXimqn7t6RxJ0taa+pROki8BtwC7klwAPgK8EqCqPgWcAG4DloBngfds1LCSpLWbGvyqOjLl+gL+emYTSZI2hO+0laQmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqYlBwU9yIMnjSZaS3HWR69+Q5IEkjyR5NMltsx9VkrQeU4OfZAdwDLgV2A8cSbJ/1bK/B+6vqhuAw8A/z3pQSdL6DHmEfxOwVFXnquo54D7g0Ko1BbxmfPm1wE9mN6IkaRaGBH83cH7i+ML43KSPArcnuQCcAN5/sRtKcjTJYpLF5eXlNYwrSVqrWb1oewT4XFXtAW4DvpDk1267qo5X1XxVzc/Nzc3oriVJQwwJ/hPA3onjPeNzk+4A7geoqu8CrwJ2zWJASdJsDAn+aWBfkmuTXMHoRdmFVWt+DLwNIMmbGAXf52wk6TIyNfhV9TxwJ3ASeIzRX+OcSXJPkoPjZR8E3pvke8CXgHdXVW3U0JKkl27nkEVVdYLRi7GT5+6euHwWeMtsR5MkzZLvtJWkJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNTEo+EkOJHk8yVKSuy6x5p1JziY5k+SLsx1TkrReO6ctSLIDOAb8GXABOJ1koarOTqzZB/wd8JaqejrJ6zZqYEnS2gx5hH8TsFRV56rqOeA+4NCqNe8FjlXV0wBV9eRsx5QkrdeQ4O8Gzk8cXxifm3QdcF2S7yQ5leTAxW4oydEki0kWl5eX1zaxJGlNZvWi7U5gH3ALcAT4TJKrVy+qquNVNV9V83NzczO6a0nSEEOC/wSwd+J4z/jcpAvAQlX9qqp+CPyA0Q8ASdJlYkjwTwP7klyb5ArgMLCwas3XGD26J8kuRk/xnJvdmJKk9Zoa/Kp6HrgTOAk8BtxfVWeS3JPk4HjZSeCpJGeBB4APVdVTGzW0JOmlS1VtyR3Pz8/X4uLilty3JL1cJXm4qubX8rW+01aSmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmBgU/yYEkjydZSnLXi6x7R5JKMj+7ESVJszA1+El2AMeAW4H9wJEk+y+y7irgb4CHZj2kJGn9hjzCvwlYqqpzVfUccB9w6CLrPgZ8HPjFDOeTJM3IkODvBs5PHF8Yn/s/SW4E9lbV11/shpIcTbKYZHF5efklDytJWrt1v2ib5BXAJ4APTltbVcerar6q5ufm5tZ715Kkl2BI8J8A9k4c7xmfe8FVwJuBbyf5EXAzsOALt5J0eRkS/NPAviTXJrkCOAwsvHBlVT1TVbuq6pqqugY4BRysqsUNmViStCZTg19VzwN3AieBx4D7q+pMknuSHNzoASVJs7FzyKKqOgGcWHXu7kusvWX9Y0mSZs132kpSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmhgU/CQHkjyeZCnJXRe5/gNJziZ5NMk3k7xx9qNKktZjavCT7ACOAbcC+4EjSfavWvYIMF9VfwB8FfiHWQ8qSVqfIY/wbwKWqupcVT0H3AccmlxQVQ9U1bPjw1PAntmOKUlaryHB3w2cnzi+MD53KXcA37jYFUmOJllMsri8vDx8SknSus30RdsktwPzwL0Xu76qjlfVfFXNz83NzfKuJUlT7Byw5glg78TxnvG5/yfJ24EPA2+tql/OZjxJ0qwMeYR/GtiX5NokVwCHgYXJBUluAD4NHKyqJ2c/piRpvaYGv6qeB+4ETgKPAfdX1Zkk9yQ5OF52L/Bq4CtJ/j3JwiVuTpK0RYY8pUNVnQBOrDp398Tlt894LknSjPlOW0lqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpoYFPwkB5I8nmQpyV0Xuf43knx5fP1DSa6Z+aSSpHWZGvwkO4BjwK3AfuBIkv2rlt0BPF1Vvwv8E/DxWQ8qSVqfIY/wbwKWqupcVT0H3AccWrXmEPBv48tfBd6WJLMbU5K0XjsHrNkNnJ84vgD80aXWVNXzSZ4Bfhv42eSiJEeBo+PDXyb5/lqG3oZ2sWqvGnMvVrgXK9yLFb+31i8cEvyZqarjwHGAJItVNb+Z93+5ci9WuBcr3IsV7sWKJItr/dohT+k8AeydON4zPnfRNUl2Aq8FnlrrUJKk2RsS/NPAviTXJrkCOAwsrFqzAPzl+PJfAN+qqprdmJKk9Zr6lM74Ofk7gZPADuCzVXUmyT3AYlUtAP8KfCHJEvBzRj8Upjm+jrm3G/dihXuxwr1Y4V6sWPNexAfiktSD77SVpCYMviQ1seHB92MZVgzYiw8kOZvk0STfTPLGrZhzM0zbi4l170hSSbbtn+QN2Ysk7xx/b5xJ8sXNnnGzDPg/8oYkDyR5ZPz/5LatmHOjJflskicv9V6ljHxyvE+PJrlx0A1X1Yb9Y/Qi738AvwNcAXwP2L9qzV8BnxpfPgx8eSNn2qp/A/fiT4HfHF9+X+e9GK+7CngQOAXMb/XcW/h9sQ94BPit8fHrtnruLdyL48D7xpf3Az/a6rk3aC/+BLgR+P4lrr8N+AYQ4GbgoSG3u9GP8P1YhhVT96KqHqiqZ8eHpxi952E7GvJ9AfAxRp/L9IvNHG6TDdmL9wLHquppgKp6cpNn3CxD9qKA14wvvxb4ySbOt2mq6kFGf/F4KYeAz9fIKeDqJK+fdrsbHfyLfSzD7kutqarngRc+lmG7GbIXk+5g9BN8O5q6F+NfUfdW1dc3c7AtMOT74jrguiTfSXIqyYFNm25zDdmLjwK3J7kAnADevzmjXXZeak+ATf5oBQ2T5HZgHnjrVs+yFZK8AvgE8O4tHuVysZPR0zq3MPqt78Ekv19V/7WVQ22RI8Dnquofk/wxo/f/vLmq/merB3s52OhH+H4sw4ohe0GStwMfBg5W1S83abbNNm0vrgLeDHw7yY8YPUe5sE1fuB3yfXEBWKiqX1XVD4EfMPoBsN0M2Ys7gPsBquq7wKsYfbBaN4N6stpGB9+PZVgxdS+S3AB8mlHst+vztDBlL6rqmaraVVXXVNU1jF7POFhVa/7QqMvYkP8jX2P06J4kuxg9xXNuE2fcLEP24sfA2wCSvIlR8Jc3dcrLwwLwrvFf69wMPFNVP532RRv6lE5t3McyvOwM3It7gVcDXxm/bv3jqjq4ZUNvkIF70cLAvTgJ/HmSs8B/Ax+qqm33W/DAvfgg8Jkkf8voBdx3b8cHiEm+xOiH/K7x6xUfAV4JUFWfYvT6xW3AEvAs8J5Bt7sN90qSdBG+01aSmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElq4n8BzPZculjwdYoAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"code","source":"!pip install gputil","metadata":{"execution":{"iopub.status.busy":"2023-01-05T15:22:15.408763Z","iopub.execute_input":"2023-01-05T15:22:15.409165Z","iopub.status.idle":"2023-01-05T15:22:30.439799Z","shell.execute_reply.started":"2023-01-05T15:22:15.409129Z","shell.execute_reply":"2023-01-05T15:22:30.438582Z"},"trusted":true},"execution_count":8,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nCollecting gputil\n Downloading GPUtil-1.4.0.tar.gz (5.5 kB)\n Preparing metadata (setup.py) ... \u001b[?25ldone\n\u001b[?25hBuilding wheels for collected packages: gputil\n Building wheel for gputil (setup.py) ... \u001b[?25ldone\n\u001b[?25h Created wheel for gputil: filename=GPUtil-1.4.0-py3-none-any.whl size=7411 sha256=fe852456bd1693d3610fbeb2607b6205f7711ff915fa394a5b517bf1e86469aa\n Stored in directory: /root/.cache/pip/wheels/6e/f8/83/534c52482d6da64622ddbf72cd93c35d2ef2881b78fd08ff0c\nSuccessfully built gputil\nInstalling collected packages: gputil\nSuccessfully installed gputil-1.4.0\n\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n\u001b[0m","output_type":"stream"}]},{"cell_type":"code","source":"from GPUtil import showUtilization as gpu_usage\nimport gc\n\ndef free_gpu_cache():\n print(\"Initial GPU Usage\")\n gpu_usage() \n\n gc.collect()\n torch.cuda.empty_cache()\n print(\"GPU Usage after emptying the cache\")\n gpu_usage()\nfree_gpu_cache()","metadata":{"execution":{"iopub.status.busy":"2023-01-05T15:28:21.040498Z","iopub.execute_input":"2023-01-05T15:28:21.041125Z","iopub.status.idle":"2023-01-05T15:28:22.434229Z","shell.execute_reply.started":"2023-01-05T15:28:21.041088Z","shell.execute_reply":"2023-01-05T15:28:22.432748Z"},"trusted":true},"execution_count":30,"outputs":[{"name":"stdout","text":"Initial GPU Usage\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n| ID | GPU | MEM |\n-------------------\n| 0 | 0% | 100% |\n| 1 | 0% | 0% |\nGPU Usage after emptying the cache\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n| ID | GPU | MEM |\n------------------\n| 0 | 91% | 32% |\n| 1 | 0% | 0% |\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Evaluation\n","metadata":{}},{"cell_type":"code","source":"vizmodel = AutoModelForSequenceClassification.from_pretrained('/kaggle/input/bert3-seed-22')\nvizmodel.to(device)\nvizmodel.eval()\nvizmodel.zero_grad()\ntokenizer = berttokenizer","metadata":{"execution":{"iopub.status.busy":"2023-01-06T09:25:06.988597Z","iopub.execute_input":"2023-01-06T09:25:06.988980Z","iopub.status.idle":"2023-01-06T09:25:14.519298Z","shell.execute_reply.started":"2023-01-06T09:25:06.988947Z","shell.execute_reply":"2023-01-06T09:25:14.517991Z"},"trusted":true},"execution_count":11,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## Produce prediction for all data","metadata":{}},{"cell_type":"code","source":"class SpookyDataset2(torch.utils.data.Dataset):\n def __init__(self, ids, encodings, labels):\n self.ids = ids\n self.encodings = encodings\n self.labels = labels\n \n def __len__(self):\n return len(self.ids)\n \n def __getitem__(self, idx):\n item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}\n #item['labels'] = torch.tensor(self.labels[idx])\n item['id'] = self.ids.iloc[idx]\n return item\n \nall_dataset = SpookyDataset2(all_labeled[\"id\"], encode(all_labeled[\"text\"]), label(all_labeled[\"author\"]))\n\nall_dataloader = torch.utils.data.DataLoader(\n all_dataset,\n batch_size=8,\n num_workers=2)\n\ntest_dataset = SpookyDataset2(test[\"id\"], encode(test[\"text\"]), label(all_labeled[\"author\"]))\n\ntest_dataloader = torch.utils.data.DataLoader(\n test_dataset,\n batch_size=8,\n num_workers=2)","metadata":{"execution":{"iopub.status.busy":"2023-01-06T09:29:13.374242Z","iopub.execute_input":"2023-01-06T09:29:13.374634Z","iopub.status.idle":"2023-01-06T09:29:19.894466Z","shell.execute_reply.started":"2023-01-06T09:29:13.374602Z","shell.execute_reply":"2023-01-06T09:29:19.893375Z"},"trusted":true},"execution_count":29,"outputs":[]},{"cell_type":"code","source":"test_result_df = pd.DataFrame(columns=(\"id\", \"pred\", \"eap\", \"hpl\", \"mws\"))\n\nwith torch.no_grad():\n for data in tqdm(test_dataloader):\n ids = data.pop(\"id\")\n x = {key:val.to(device) for key, val in data.items()}\n logits = vizmodel(**x).logits.cpu()\n preds = unlabel(torch.argmax(logits, dim=1))\n for id, pred, logitss in zip(ids, preds, logits):\n test_result_df.loc[len(test_result_df)] = [id, pred, *[t.item() for t in logitss]]\n\ntest_result_df.info()\nprint(test_result_df.head(10))","metadata":{"execution":{"iopub.status.busy":"2023-01-06T09:29:47.540733Z","iopub.execute_input":"2023-01-06T09:29:47.541080Z","iopub.status.idle":"2023-01-06T09:34:44.677081Z","shell.execute_reply.started":"2023-01-06T09:29:47.541049Z","shell.execute_reply":"2023-01-06T09:34:44.675621Z"},"trusted":true},"execution_count":34,"outputs":[{"name":"stderr","text":" 0%| | 0/1049 [00:00<?, ?it/s]","output_type":"stream"},{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"},{"name":"stderr","text":"100%|██████████| 1049/1049 [04:57<00:00, 3.53it/s]","output_type":"stream"},{"name":"stdout","text":"<class 'pandas.core.frame.DataFrame'>\nInt64Index: 8392 entries, 0 to 8391\nData columns (total 5 columns):\n # Column Non-Null Count Dtype \n--- ------ -------------- ----- \n 0 id 8392 non-null object \n 1 pred 8392 non-null object \n 2 eap 8392 non-null float64\n 3 hpl 8392 non-null float64\n 4 mws 8392 non-null float64\ndtypes: float64(3), object(2)\nmemory usage: 393.4+ KB\n id pred eap hpl mws\n0 id02310 MWS -3.141249 -2.306986 5.610037\n1 id24541 EAP 6.279230 -2.883798 -2.380444\n2 id00134 HPL -3.699961 6.078186 -3.682606\n3 id27757 EAP 5.298801 -2.495149 -1.756756\n4 id04081 EAP 1.576924 0.450743 -2.041980\n5 id27337 EAP 5.862383 -2.564406 -2.283409\n6 id24265 EAP 5.345832 -2.396763 -1.945789\n7 id25917 HPL -3.256679 3.503856 -0.790709\n8 id04951 EAP 6.074453 -2.863337 -2.187590\n9 id14549 EAP 5.720592 -2.551389 -2.104292\n","output_type":"stream"},{"name":"stderr","text":"\n","output_type":"stream"}]},{"cell_type":"code","source":"torch.save(test_result_df, \"/kaggle/working/bert-test-pred\")","metadata":{"execution":{"iopub.status.busy":"2023-01-06T09:41:17.883239Z","iopub.execute_input":"2023-01-06T09:41:17.883713Z","iopub.status.idle":"2023-01-06T09:41:17.909122Z","shell.execute_reply.started":"2023-01-06T09:41:17.883663Z","shell.execute_reply":"2023-01-06T09:41:17.908201Z"},"trusted":true},"execution_count":35,"outputs":[]},{"cell_type":"code","source":"result_df = pd.DataFrame(columns=(\"id\", \"pred\", \"eap\", \"hpl\", \"mws\"))\n\nwith torch.no_grad():\n for data in tqdm(all_dataloader):\n ids = data.pop(\"id\")\n x = {key:val.to(device) for key, val in data.items()}\n logits = vizmodel(**x).logits.cpu()\n preds = unlabel(torch.argmax(logits, dim=1))\n for id, pred, logitss in zip(ids, preds, logits):\n result_df.loc[len(result_df)] = [id, pred, *[t.item() for t in logitss]]\n\nresult_df.info()\nprint(result_df.head(10))","metadata":{"execution":{"iopub.status.busy":"2023-01-05T15:45:12.084929Z","iopub.execute_input":"2023-01-05T15:45:12.086113Z","iopub.status.idle":"2023-01-05T15:56:34.021905Z","shell.execute_reply.started":"2023-01-05T15:45:12.086051Z","shell.execute_reply":"2023-01-05T15:56:34.020565Z"},"trusted":true},"execution_count":73,"outputs":[{"name":"stderr","text":" 0%| | 0/2429 [00:00<?, ?it/s]","output_type":"stream"},{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"},{"name":"stderr","text":"100%|██████████| 2429/2429 [11:21<00:00, 3.56it/s]\n","output_type":"stream"},{"name":"stdout","text":"<class 'pandas.core.frame.DataFrame'>\nInt64Index: 19427 entries, 0 to 19426\nData columns (total 5 columns):\n # Column Non-Null Count Dtype \n--- ------ -------------- ----- \n 0 id 19427 non-null object \n 1 pred 19427 non-null object \n 2 eap 19427 non-null float64\n 3 hpl 19427 non-null float64\n 4 mws 19427 non-null float64\ndtypes: float64(3), object(2)\nmemory usage: 910.6+ KB\n id pred eap hpl mws\n0 id26305 EAP 5.960782 -2.923888 -1.926633\n1 id17569 HPL -3.508775 5.984394 -3.617640\n2 id11008 EAP 5.437940 -2.526213 -2.039323\n3 id27763 MWS -2.795347 -3.060079 6.400214\n4 id12958 HPL -2.591162 5.359665 -3.349132\n5 id22965 MWS -2.316027 -3.305767 6.174081\n6 id09674 EAP 5.858667 -2.805764 -2.002376\n7 id13515 EAP 6.511631 -3.291044 -2.052387\n8 id19322 EAP 5.727561 -2.704725 -2.006320\n9 id00912 MWS -2.021483 -2.794908 4.934676\n","output_type":"stream"}]},{"cell_type":"code","source":"torch.save(result_df, \"/kaggle/working/predictions.th\")\ntorch.load(\"/kaggle/working/predictions.th\")","metadata":{"execution":{"iopub.status.busy":"2023-01-05T16:32:24.820084Z","iopub.execute_input":"2023-01-05T16:32:24.820498Z","iopub.status.idle":"2023-01-05T16:32:24.892167Z","shell.execute_reply.started":"2023-01-05T16:32:24.820467Z","shell.execute_reply":"2023-01-05T16:32:24.890756Z"},"trusted":true},"execution_count":77,"outputs":[{"execution_count":77,"output_type":"execute_result","data":{"text/plain":" id pred eap hpl mws\n0 id26305 EAP 5.960782 -2.923888 -1.926633\n1 id17569 HPL -3.508775 5.984394 -3.617640\n2 id11008 EAP 5.437940 -2.526213 -2.039323\n3 id27763 MWS -2.795347 -3.060079 6.400214\n4 id12958 HPL -2.591162 5.359665 -3.349132\n... ... ... ... ... ...\n19422 id17718 EAP 5.243217 -2.563747 -1.894591\n19423 id08973 EAP 5.879582 -2.807469 -2.037973\n19424 id05267 EAP 6.315331 -3.148599 -2.093434\n19425 id17513 EAP 5.084399 -2.418648 -1.871007\n19426 id00393 HPL -2.993505 5.676175 -3.483623\n\n[19427 rows x 5 columns]","text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>id</th>\n <th>pred</th>\n <th>eap</th>\n <th>hpl</th>\n <th>mws</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>id26305</td>\n <td>EAP</td>\n <td>5.960782</td>\n <td>-2.923888</td>\n <td>-1.926633</td>\n </tr>\n <tr>\n <th>1</th>\n <td>id17569</td>\n <td>HPL</td>\n <td>-3.508775</td>\n <td>5.984394</td>\n <td>-3.617640</td>\n </tr>\n <tr>\n <th>2</th>\n <td>id11008</td>\n <td>EAP</td>\n <td>5.437940</td>\n <td>-2.526213</td>\n <td>-2.039323</td>\n </tr>\n <tr>\n <th>3</th>\n <td>id27763</td>\n <td>MWS</td>\n <td>-2.795347</td>\n <td>-3.060079</td>\n <td>6.400214</td>\n </tr>\n <tr>\n <th>4</th>\n <td>id12958</td>\n <td>HPL</td>\n <td>-2.591162</td>\n <td>5.359665</td>\n <td>-3.349132</td>\n </tr>\n <tr>\n <th>...</th>\n <td>...</td>\n <td>...</td>\n <td>...</td>\n <td>...</td>\n <td>...</td>\n </tr>\n <tr>\n <th>19422</th>\n <td>id17718</td>\n <td>EAP</td>\n <td>5.243217</td>\n <td>-2.563747</td>\n <td>-1.894591</td>\n </tr>\n <tr>\n <th>19423</th>\n <td>id08973</td>\n <td>EAP</td>\n <td>5.879582</td>\n <td>-2.807469</td>\n <td>-2.037973</td>\n </tr>\n <tr>\n <th>19424</th>\n <td>id05267</td>\n <td>EAP</td>\n <td>6.315331</td>\n <td>-3.148599</td>\n <td>-2.093434</td>\n </tr>\n <tr>\n <th>19425</th>\n <td>id17513</td>\n <td>EAP</td>\n <td>5.084399</td>\n <td>-2.418648</td>\n <td>-1.871007</td>\n </tr>\n <tr>\n <th>19426</th>\n <td>id00393</td>\n <td>HPL</td>\n <td>-2.993505</td>\n <td>5.676175</td>\n <td>-3.483623</td>\n </tr>\n </tbody>\n</table>\n<p>19427 rows × 5 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"# Integrated Gradients","metadata":{}},{"cell_type":"code","source":"def fetch_output(text_ids):\n return vizmodel(text_ids)[0]\n \ndef summarize_attributions(attributions):\n attributions = attributions.sum(dim=-1).squeeze(0)\n attributions = attributions / torch.norm(attributions)\n \n return attributions\n\n \ndef construct_input_and_baseline(text):\n #free_gpu_cache()\n max_length = 510\n baseline_token_id = tokenizer.pad_token_id \n sep_token_id = tokenizer.sep_token_id\n cls_token_id = tokenizer.cls_token_id\n\n if isinstance(text, str):\n text_ids = tokenizer.encode(text, \n max_length=max_length,\n truncation=True,\n add_special_tokens=False)\n input_ids = [cls_token_id] + text_ids + [sep_token_id]\n else:\n # Already encoded\n text_ids = text[1:-1]\n input_ids = text\n \n token_list = tokenizer.convert_ids_to_tokens(input_ids)\n \n\n baseline_input_ids = [cls_token_id] + [baseline_token_id] * len(text_ids) + [sep_token_id]\n return torch.tensor([input_ids], device=device), torch.tensor([baseline_input_ids], device=device), token_list\n\nfrom captum.attr import LayerIntegratedGradients\nlig = LayerIntegratedGradients(fetch_output, vizmodel.bert.embeddings)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T11:19:22.943290Z","iopub.execute_input":"2023-01-10T11:19:22.943653Z","iopub.status.idle":"2023-01-10T11:19:23.118269Z","shell.execute_reply.started":"2023-01-10T11:19:22.943622Z","shell.execute_reply":"2023-01-10T11:19:23.116838Z"},"trusted":true},"execution_count":13,"outputs":[{"traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_23/1611953692.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 34\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 35\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mcaptum\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mattr\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mLayerIntegratedGradients\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 36\u001b[0;31m \u001b[0mlig\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mLayerIntegratedGradients\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfetch_output\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvizmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbert\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0membeddings\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m","\u001b[0;31mNameError\u001b[0m: name 'vizmodel' is not defined"],"ename":"NameError","evalue":"name 'vizmodel' is not defined","output_type":"error"}]},{"cell_type":"markdown","source":"## Compute attributions","metadata":{}},{"cell_type":"code","source":"from captum.attr import visualization as viz\n\ndef interpret_text(text, true_class: int, visualise=True):\n input_ids, baseline_input_ids, all_tokens = construct_input_and_baseline(text)\n attributions, delta = lig.attribute(inputs = input_ids,\n baselines = baseline_input_ids,\n target = true_class,\n return_convergence_delta=True,\n internal_batch_size=2\n )\n attributions_sum = summarize_attributions(attributions)\n\n prediction = fetch_output(input_ids)\n score_vis = viz.VisualizationDataRecord(\n word_attributions = attributions_sum,\n pred_prob = torch.max(prediction),\n pred_class = torch.argmax(prediction),\n true_class = true_class,\n attr_class = text,\n attr_score = attributions_sum.sum(), \n raw_input_ids = all_tokens,\n convergence_score = delta)\n if visualise:\n viz.visualize_text([score_vis])\n \n return score_vis","metadata":{"execution":{"iopub.status.busy":"2023-01-05T08:45:52.331649Z","iopub.execute_input":"2023-01-05T08:45:52.332202Z","iopub.status.idle":"2023-01-05T08:45:52.349265Z","shell.execute_reply.started":"2023-01-05T08:45:52.332159Z","shell.execute_reply":"2023-01-05T08:45:52.348315Z"},"trusted":true},"execution_count":21,"outputs":[]},{"cell_type":"code","source":"labels = [int(l) for l in label(valid['author'])]","metadata":{"execution":{"iopub.status.busy":"2023-01-05T08:44:54.208243Z","iopub.execute_input":"2023-01-05T08:44:54.208600Z","iopub.status.idle":"2023-01-05T08:44:54.214949Z","shell.execute_reply.started":"2023-01-05T08:44:54.208570Z","shell.execute_reply":"2023-01-05T08:44:54.213800Z"},"trusted":true},"execution_count":13,"outputs":[]},{"cell_type":"code","source":"res = interpret_text(valid[\"text\"][0], labels[0], visualise=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-05T08:46:02.205016Z","iopub.execute_input":"2023-01-05T08:46:02.205765Z","iopub.status.idle":"2023-01-05T08:46:05.152553Z","shell.execute_reply.started":"2023-01-05T08:46:02.205727Z","shell.execute_reply":"2023-01-05T08:46:05.151568Z"},"trusted":true},"execution_count":22,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<table width: 100%><div style=\"border-top: 1px solid; margin-top: 5px; padding-top: 5px; display: inline-block\"><b>Legend: </b><span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 60%)\"></span> Negative <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 100%)\"></span> Neutral <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(120, 75%, 50%)\"></span> Positive </div><tr><th>True Label</th><th>Predicted Label</th><th>Attribution Label</th><th>Attribution Score</th><th>Word Importance</th><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>0 (5.31)</b></text></td><td><text style=\"padding-right:2em\"><b>At length, one escaping through a sewer, gave freedom to all the rest.</b></text></td><td><text style=\"padding-right:2em\"><b>2.28</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> At </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> length </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> one </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> escaping </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> through </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 60%); opacity:1.0; line-height:1.75\"><font color=\"black\"> sewer </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> gave </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> freedom </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> all </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> rest </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr></table>"},"metadata":{}},{"execution_count":22,"output_type":"execute_result","data":{"text/plain":"<captum.attr._utils.visualization.VisualizationDataRecord at 0x7f55c7f13d00>"},"metadata":{}}]},{"cell_type":"code","source":"attributions = []\n\nfor i in tqdm(range(len(valid[\"text\"]))):\n res = interpret_text(valid[\"text\"][i], labels[i], visualise=False)\n \n attributions.append(\n {\n \"attr_score\": float(res.attr_score),\n \"conv_score\": float(res.convergence_score),\n \"pred_class\": int(res.pred_class),\n \"true_class\": int(res.true_class),\n \"word_attr\": res.word_attributions.cpu(),\n \"words\": res.raw_input_ids\n })","metadata":{"execution":{"iopub.status.busy":"2023-01-05T09:29:09.224693Z","iopub.execute_input":"2023-01-05T09:29:09.225275Z","iopub.status.idle":"2023-01-05T09:54:33.395273Z","shell.execute_reply.started":"2023-01-05T09:29:09.225230Z","shell.execute_reply":"2023-01-05T09:54:33.394351Z"},"trusted":true},"execution_count":68,"outputs":[{"name":"stderr","text":"100%|██████████| 1943/1943 [25:24<00:00, 1.27it/s]\n","output_type":"stream"}]},{"cell_type":"code","source":"correct_count = sum([a[\"pred_class\"] == a[\"true_class\"] for a in attributions])\nprint(f\"{(correct_count / len(attributions) )*100:.2f} %\")","metadata":{"execution":{"iopub.status.busy":"2023-01-10T12:11:39.012574Z","iopub.execute_input":"2023-01-10T12:11:39.013118Z","iopub.status.idle":"2023-01-10T12:11:39.021979Z","shell.execute_reply.started":"2023-01-10T12:11:39.013086Z","shell.execute_reply":"2023-01-10T12:11:39.020850Z"},"trusted":true},"execution_count":109,"outputs":[{"name":"stdout","text":"88.99 %\n","output_type":"stream"}]},{"cell_type":"code","source":"score_vis = []\nfor a in attributions:\n if a[\"pred_class\"] != a[\"true_class\"]:\n q = a\n score_vis.append(viz.VisualizationDataRecord(\n word_attributions = q[\"word_attr\"],\n attr_class = q[\"words\"],\n pred_prob = 0.5,\n pred_class = q[\"pred_class\"],\n true_class = q[\"true_class\"],\n attr_score = q[\"attr_score\"], \n raw_input_ids = q[\"words\"],\n convergence_score = q[\"conv_score\"]))\nviz.visualize_text(score_vis[:20])\n\nprint(len(score_vis))\n\nlen(attributions)","metadata":{"execution":{"iopub.status.busy":"2023-01-05T10:09:20.980598Z","iopub.execute_input":"2023-01-05T10:09:20.980963Z","iopub.status.idle":"2023-01-05T10:09:21.012711Z","shell.execute_reply.started":"2023-01-05T10:09:20.980931Z","shell.execute_reply":"2023-01-05T10:09:21.011820Z"},"trusted":true},"execution_count":103,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<table width: 100%><div style=\"border-top: 1px solid; margin-top: 5px; padding-top: 5px; display: inline-block\"><b>Legend: </b><span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 60%)\"></span> Negative <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 100%)\"></span> Neutral <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(120, 75%, 50%)\"></span> Positive </div><tr><th>True Label</th><th>Predicted Label</th><th>Attribution Label</th><th>Attribution Score</th><th>Word Importance</th><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'Let', 'us', 'pursue', 'our', 'fan', '##cies', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.50</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Let </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> us </font></mark><mark style=\"background-color: hsl(0, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> pursue </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> our </font></mark><mark style=\"background-color: hsl(0, 75%, 67%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fan </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##cies </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'Now', 'the', 'net', 'work', 'was', 'not', 'permanently', 'fastened', 'to', 'the', 'ho', '##op', ',', 'but', 'attached', 'by', 'a', 'series', 'of', 'running', 'loops', 'or', 'no', '##ose', '##s', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.13</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Now </font></mark><mark style=\"background-color: hsl(0, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> net </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> work </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> not </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> permanently </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fastened </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ho </font></mark><mark style=\"background-color: hsl(0, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##op </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> but </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> attached </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> by </font></mark><mark style=\"background-color: hsl(0, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> series </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> running </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> loops </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> or </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> no </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ose </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##s </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>1</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'Nor', 'has', 'he', 'yet', 'had', 'any', 'difficulty', 'in', 'obtaining', 'tenants', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-1.34</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Nor </font></mark><mark style=\"background-color: hsl(0, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> has </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> he </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> yet </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> had </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> any </font></mark><mark style=\"background-color: hsl(0, 75%, 73%); opacity:1.0; line-height:1.75\"><font color=\"black\"> difficulty </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> obtaining </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> tenants </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'I', 'will', 'be', 'cool', ',', 'per', '##se', '##vering', ',', 'and', 'p', '##rud', '##ent', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>1.18</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> will </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> be </font></mark><mark style=\"background-color: hsl(120, 75%, 59%); opacity:1.0; line-height:1.75\"><font color=\"black\"> cool </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> per </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##se </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##vering </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> p </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##rud </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ent </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'I', 'found', 'myself', 'sitting', 'in', 'the', 'library', ',', 'and', 'again', 'sitting', 'there', 'alone', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>0.31</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> found </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> myself </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> sitting </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 75%); opacity:1.0; line-height:1.75\"><font color=\"black\"> library </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> again </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> sitting </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> there </font></mark><mark style=\"background-color: hsl(120, 75%, 78%); opacity:1.0; line-height:1.75\"><font color=\"black\"> alone </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'The', 'fear', 'of', 'immediate', 'death', 'vanished', 'with', 'the', 'heats', 'of', 'September', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>1.12</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> The </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fear </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> immediate </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> death </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> vanished </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> with </font></mark><mark style=\"background-color: hsl(120, 75%, 65%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> heats </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(0, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> September </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'His', 'reckless', 'courage', 'and', 'comprehensive', 'genius', 'brought', 'him', 'into', 'notice', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>0.67</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> His </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> reckless </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> courage </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(0, 75%, 69%); opacity:1.0; line-height:1.75\"><font color=\"black\"> comprehensive </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> genius </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> brought </font></mark><mark style=\"background-color: hsl(120, 75%, 82%); opacity:1.0; line-height:1.75\"><font color=\"black\"> him </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> into </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> notice </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'I', 'do', 'not', 'think', 'that', 'the', 'pursuit', 'of', 'knowledge', 'is', 'an', 'exception', 'to', 'this', 'rule', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>2.02</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> do </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> not </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> think </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> that </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> pursuit </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> knowledge </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> is </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> an </font></mark><mark style=\"background-color: hsl(120, 75%, 78%); opacity:1.0; line-height:1.75\"><font color=\"black\"> exception </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> this </font></mark><mark style=\"background-color: hsl(120, 75%, 66%); opacity:1.0; line-height:1.75\"><font color=\"black\"> rule </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>1</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'But', 'it', 'was', 'so', 'silent', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.83</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 81%); opacity:1.0; line-height:1.75\"><font color=\"black\"> But </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> it </font></mark><mark style=\"background-color: hsl(120, 75%, 77%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> so </font></mark><mark style=\"background-color: hsl(0, 75%, 73%); opacity:1.0; line-height:1.75\"><font color=\"black\"> silent </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'I', 'took', 'thee', ',', 'just', 'now', ',', 'from', 'a', 'rose', 'wood', 'coffin', 'in', '##laid', 'with', 'ivory', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.10</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(0, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> took </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> thee </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> just </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> now </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> from </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> rose </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> wood </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> coffin </font></mark><mark style=\"background-color: hsl(0, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##laid </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> with </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ivory </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'A', 'few', 'months', 'before', 'my', 'arrival', 'they', 'had', 'lived', 'in', 'a', 'large', 'and', 'luxurious', 'city', 'called', 'Paris', ',', 'surrounded', 'by', 'friends', 'and', 'possessed', 'of', 'every', 'enjoyment', 'which', 'virtue', ',', 're', '##fine', '##ment', 'of', 'in', '##tel', '##lect', ',', 'or', 'taste', ',', 'accompanied', 'by', 'a', 'moderate', 'fortune', ',', 'could', 'afford', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>1.83</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> A </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> few </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> months </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> before </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> my </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> arrival </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> they </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> had </font></mark><mark style=\"background-color: hsl(0, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> lived </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> large </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> luxurious </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> city </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> called </font></mark><mark style=\"background-color: hsl(120, 75%, 79%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Paris </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> surrounded </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> by </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> friends </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(0, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> possessed </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> every </font></mark><mark style=\"background-color: hsl(120, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> enjoyment </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> which </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> virtue </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> re </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##fine </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ment </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##tel </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##lect </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> or </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> taste </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> accompanied </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> by </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> moderate </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fortune </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> could </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> afford </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'Show', '##ed', 'that', ',', 'I', 'should', 'say', ',', 'to', 'some', 'purpose', ',', 'in', 'the', 'late', 'tremendous', 'swamp', 'fight', 'away', 'down', 'South', ',', 'with', 'the', 'B', '##uga', '##boo', 'and', 'Kick', '##ap', '##oo', 'Indians', '.', '\"', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.76</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Show </font></mark><mark style=\"background-color: hsl(0, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ed </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> that </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> should </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> say </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> some </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> purpose </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> late </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> tremendous </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> swamp </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fight </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> away </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> down </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> South </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> with </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> B </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##uga </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##boo </font></mark><mark style=\"background-color: hsl(0, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Kick </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ap </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##oo </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Indians </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> \" </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', '\"', 'That', 'evidence', ',', '\"', 'he', 'observed', ',', '\"', 'was', 'hardly', 'required', 'in', 'so', 'glaring', 'a', 'case', ',', 'but', 'I', 'am', 'glad', 'of', 'it', ',', 'and', ',', 'indeed', ',', 'none', 'of', 'our', 'judges', 'like', 'to', 'con', '##de', '##m', '##n', 'a', 'criminal', 'upon', 'c', '##ir', '##cum', '##stant', '##ial', 'evidence', ',', 'be', 'it', 'ever', 'so', 'decisive', '.', '\"', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>3.78</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> \" </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> That </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> evidence </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> \" </font></mark><mark style=\"background-color: hsl(120, 75%, 75%); opacity:1.0; line-height:1.75\"><font color=\"black\"> he </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> observed </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> \" </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> hardly </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> required </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> so </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> glaring </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> case </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> but </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> am </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> glad </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> it </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> indeed </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> none </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> our </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> judges </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> like </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> con </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##de </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##m </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##n </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> criminal </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> upon </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> c </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ir </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##cum </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##stant </font></mark><mark style=\"background-color: hsl(120, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ial </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> evidence </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> be </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> it </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ever </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> so </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> decisive </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> \" </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'The', 'w', '##ret', '##ched', '##ness', 'of', 'earth', 'is', 'multi', '##form', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-0.08</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> The </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> w </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ret </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ched </font></mark><mark style=\"background-color: hsl(120, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ness </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(0, 75%, 79%); opacity:1.0; line-height:1.75\"><font color=\"black\"> earth </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> is </font></mark><mark style=\"background-color: hsl(0, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> multi </font></mark><mark style=\"background-color: hsl(120, 75%, 77%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##form </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'As', 'time', 'passed', 'and', 'no', 'discovery', 'ensued', ',', 'a', 'thousand', 'con', '##tra', '##dict', '##ory', 'rumors', 'were', 'circulated', ',', 'and', 'journalists', 'bus', '##ied', 'themselves', 'in', 'suggestions', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>0.76</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> As </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> time </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> passed </font></mark><mark style=\"background-color: hsl(0, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> no </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> discovery </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ensued </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> thousand </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> con </font></mark><mark style=\"background-color: hsl(0, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##tra </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##dict </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ory </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> rumors </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> were </font></mark><mark style=\"background-color: hsl(120, 75%, 78%); opacity:1.0; line-height:1.75\"><font color=\"black\"> circulated </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> journalists </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> bus </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ied </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> themselves </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> suggestions </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'There', 'was', 'a', 'curious', 'feeling', 'of', 'un', '##real', '##ity', 'attached', 'by', 'him', 'to', 'his', 'foreign', 'life', 'in', 'comparison', 'with', 'the', 'years', 'of', 'his', 'youth', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>2.62</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> There </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> curious </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> feeling </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> un </font></mark><mark style=\"background-color: hsl(120, 75%, 70%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##real </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ity </font></mark><mark style=\"background-color: hsl(0, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> attached </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> by </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> him </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> his </font></mark><mark style=\"background-color: hsl(120, 75%, 82%); opacity:1.0; line-height:1.75\"><font color=\"black\"> foreign </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> life </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> comparison </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> with </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> years </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> his </font></mark><mark style=\"background-color: hsl(120, 75%, 81%); opacity:1.0; line-height:1.75\"><font color=\"black\"> youth </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>1</b></text></td><td><text style=\"padding-right:2em\"><b>2 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'It', 'is', 'sufficient', 'for', 'me', 'to', 'relate', 'events', 'without', 'anal', '##ys', '##ing', 'causes', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-2.48</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> It </font></mark><mark style=\"background-color: hsl(0, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> is </font></mark><mark style=\"background-color: hsl(0, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> sufficient </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> for </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> me </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> relate </font></mark><mark style=\"background-color: hsl(0, 75%, 76%); opacity:1.0; line-height:1.75\"><font color=\"black\"> events </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> without </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> anal </font></mark><mark style=\"background-color: hsl(0, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ys </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ing </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> causes </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>0 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'It', 'was', 'not', 'it', 'is', 'not', 'a', 'common', 'in', '##fi', '##delity', 'at', 'which', 'I', 're', '##pine', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>3.13</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 88%); opacity:1.0; line-height:1.75\"><font color=\"black\"> It </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> not </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> it </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> is </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> not </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> a </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> common </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##fi </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##delity </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> at </font></mark><mark style=\"background-color: hsl(120, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> which </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> re </font></mark><mark style=\"background-color: hsl(120, 75%, 75%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##pine </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'His', 'room', 'was', 'as', 'black', 'as', 'pitch', 'with', 'the', 'thick', 'darkness', ',', 'for', 'the', 'shut', '##ters', 'were', 'close', 'fastened', ',', 'through', 'fear', 'of', 'r', '##ob', '##bers', ',', 'and', 'so', 'I', 'knew', 'that', 'he', 'could', 'not', 'see', 'the', 'opening', 'of', 'the', 'door', ',', 'and', 'I', 'kept', 'pushing', 'it', 'on', 'steadily', ',', 'steadily', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>-3.03</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> His </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> room </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> as </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> black </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> as </font></mark><mark style=\"background-color: hsl(120, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> pitch </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> with </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> thick </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> darkness </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> for </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> shut </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ters </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> were </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> close </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fastened </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> through </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> fear </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> r </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ob </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##bers </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> so </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> knew </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> that </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> he </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> could </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> not </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> see </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> opening </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(0, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> door </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(0, 75%, 94%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> kept </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> pushing </font></mark><mark style=\"background-color: hsl(0, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> it </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> on </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> steadily </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> steadily </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr><tr><td><text style=\"padding-right:2em\"><b>2</b></text></td><td><text style=\"padding-right:2em\"><b>1 (0.50)</b></text></td><td><text style=\"padding-right:2em\"><b>['[CLS]', 'The', 'debate', 'had', 'ended', 'at', 'three', 'in', 'the', 'morning', '.', '[SEP]']</b></text></td><td><text style=\"padding-right:2em\"><b>1.87</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> The </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> debate </font></mark><mark style=\"background-color: hsl(120, 75%, 72%); opacity:1.0; line-height:1.75\"><font color=\"black\"> had </font></mark><mark style=\"background-color: hsl(120, 75%, 83%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ended </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> at </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> three </font></mark><mark style=\"background-color: hsl(120, 75%, 90%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(120, 75%, 80%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> morning </font></mark><mark style=\"background-color: hsl(120, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr></table>"},"metadata":{}},{"name":"stdout","text":"214\n","output_type":"stream"},{"execution_count":103,"output_type":"execute_result","data":{"text/plain":"1943"},"metadata":{}}]},{"cell_type":"code","source":"torch.save(attributions, \"/kaggle/working/attr.json\")","metadata":{"execution":{"iopub.status.busy":"2023-01-05T10:52:56.759520Z","iopub.execute_input":"2023-01-05T10:52:56.760220Z","iopub.status.idle":"2023-01-05T10:52:56.954154Z","shell.execute_reply.started":"2023-01-05T10:52:56.760177Z","shell.execute_reply":"2023-01-05T10:52:56.953253Z"},"trusted":true},"execution_count":125,"outputs":[]},{"cell_type":"code","source":"attributions = torch.load(\"/kaggle/input/word-frequncies-attribution/attr.json\")\nattributions[0]\n\nattributions_t = []\nfor a in attributions:\n if a not in attributions_t:\n attributions_t.append(a)\n#print(len(attributions_t), len(attributions))","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:44:11.125239Z","iopub.execute_input":"2023-01-15T15:44:11.125707Z","iopub.status.idle":"2023-01-15T15:44:11.471696Z","shell.execute_reply.started":"2023-01-15T15:44:11.125668Z","shell.execute_reply":"2023-01-15T15:44:11.470565Z"},"trusted":true},"execution_count":6,"outputs":[]},{"cell_type":"markdown","source":"## Confusion matrix","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import confusion_matrix\nimport seaborn as sns\n\ntrue_labels = [a[\"true_class\"] for a in attributions]\npred_labels = [a[\"pred_class\"] for a in attributions]\nconf_matrix = confusion_matrix(true_labels, pred_labels)\n\nax = sns.heatmap(conf_matrix, annot=True, fmt='g')\nax.set_xlabel('Predicted labels')\nax.set_ylabel('True labels') \nax.set_title('Confusion Matrix')\nax.xaxis.set_ticklabels(['EAP', 'HPL', 'MWS']); ax.yaxis.set_ticklabels([\"EAP\", \"HPL\", \"MWS\"]);\n\nprint(classification_report(true_labels, pred_labels, target_names=['EAP', 'HPL', 'MWS']))\n\neap_wrong = 92\nhpl_wrong = 39\nmws_wrong = 52\n\neap_right = 690\nhpl_right = 522\nmws_right = 517","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:44:11.898141Z","iopub.execute_input":"2023-01-15T15:44:11.899157Z","iopub.status.idle":"2023-01-15T15:44:12.316561Z","shell.execute_reply.started":"2023-01-15T15:44:11.899116Z","shell.execute_reply":"2023-01-15T15:44:12.314995Z"},"trusted":true},"execution_count":7,"outputs":[{"name":"stdout","text":" precision recall f1-score support\n\n EAP 0.89 0.88 0.89 782\n HPL 0.87 0.93 0.90 561\n MWS 0.91 0.86 0.88 600\n\n accuracy 0.89 1943\n macro avg 0.89 0.89 0.89 1943\nweighted avg 0.89 0.89 0.89 1943\n\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 2 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAW4AAAEWCAYAAABG030jAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqiklEQVR4nO3dd5wV5dnG8d/FLlIVkd6sGI3RYHsVMCqW2GtiN4rGVzQxJiExtiQSNcUeo7G8KBqwYLAFrBhBjRgxKrEThWChSFUUUITdvd8/ZhYP65bDsmfPznJ985nPTp/7nOC9z97zzDOKCMzMLDtaFDsAMzNbM07cZmYZ48RtZpYxTtxmZhnjxG1mljFO3GZmGePEbWtNUhtJD0n6RNK9a3GeEyU90ZCxFYOkxyQNLnYc1nw5ca9DJJ0g6SVJSyV9mCaYbzXAqY8CugGdIuLo+p4kIu6KiP0aIJ7VSBokKSQ9WGV9v3T903me5zeS7qxrv4g4MCJG1jNcszo5ca8jJP0MuBb4PUmS3Ri4ETi8AU6/CfBORJQ1wLkKZQEwQFKnnHWDgXca6gJK+L8pKzj/I1sHSOoAXAKcFREPRMSyiFgZEQ9FxC/SfVpJulbSnHS6VlKrdNsgSbMk/VzS/LS1fmq67WLgIuDYtCV/WtWWqaRN05Ztabp8iqQZkpZIelfSiTnrJ+UcN1DSi2kJ5kVJA3O2PS3pUknPped5QlLnWr6GFcDfgOPS40uAY4G7qnxXf5I0U9Knkl6WtHu6/gDgwpzP+WpOHL+T9BzwGbB5uu5/0+03Sbo/5/yXS5ogSfn+/2dWlRP3umEA0Bp4sJZ9fgn0B7YH+gG7AL/K2d4d6AD0Ak4DbpDUMSKGkbTi/xoR7SNiRG2BSGoHXAccGBHrAwOBV6rZbyPgkXTfTsA1wCNVWswnAKcCXYH1gHNquzYwCjg5nd8feAOYU2WfF0m+g42Au4F7JbWOiMerfM5+OcecBAwB1gfer3K+nwPbpb+Udif57gaHx5qwteDEvW7oBCyso5RxInBJRMyPiAXAxSQJqdLKdPvKiHgUWApsVc94KoBtJbWJiA8j4s1q9jkYmBYRd0REWUSMBv4DHJqzz+0R8U5EfA6MIUm4NYqIfwIbSdqKJIGPqmafOyNiUXrNq4FW1P05/xIRb6bHrKxyvs9IvsdrgDuBsyNiVh3nM6uVE/e6YRHQubJUUYOerN5afD9dt+ocVRL/Z0D7NQ0kIpaRlCjOBD6U9IikrfOIpzKmXjnLc+sRzx3Aj4C9qOYvEEnnSJqalmcWk/yVUVsJBmBmbRsj4gVgBiCSXzBma8WJe93wPPAFcEQt+8whuclYaWO+WkbI1zKgbc5y99yNETE+Ir4N9CBpRd+SRzyVMc2uZ0yV7gB+CDyatoZXSUsZ5wLHAB0jYkPgE5KEC1BTeaPWsoeks0ha7nPS85utFSfudUBEfEJyA/EGSUdIaiuppaQDJV2R7jYa+JWkLulNvotI/rSvj1eAPSRtnN4YvaByg6Rukg5Pa91fkJRcKqo5x6PA19IujKWSjgW2AR6uZ0wARMS7wJ4kNf2q1gfKSHqglEq6CNggZ/s8YNM16Tki6WvAb4HvkZRMzpW0ff2iN0s4ca8j0nrtz0huOC4g+fP+RyQ9LSBJLi8BrwGvA1PSdfW51t+Bv6bnepnVk22LNI45wEckSfQH1ZxjEXAIyc29RSQt1UMiYmF9Yqpy7kkRUd1fE+OBx0m6CL4PLGf1Mkjlw0WLJE2p6zppaepO4PKIeDUippH0TLmjsseOWX3IN7fNzLLFLW4zs4xx4jYzyxgnbjOzjHHiNjPLmNoeyCiqlQtn+K5pgfXpe3CxQ1gnlFWUFzuEZm/hp++s9dgva5JzWnbevKhjzTTZxG1m1qgy9AvWidvMDCCqew6saXLiNjMDqHDiNjPLlHCL28wsY8qb8gucVufEbWYGvjlpZpY5LpWYmWWMb06amWWLb06amWWNW9xmZhlTvrLufZoIJ24zM/DNSTOzzHGpxMwsY9ziNjPLGLe4zcyyJSp8c9LMLFvc4jYzyxjXuM3MMsaDTJmZZYxb3GZmGeMat5lZxvhFCmZmGeMWt5lZtkT45qSZWbZkqMXdotgBmJk1CVGR/1QHSRtKuk/SfyRNlTRA0kaS/i5pWvqzY7qvJF0nabqk1yTtWNf5nbjNzCBpcec71e1PwOMRsTXQD5gKnA9MiIgtgQnpMsCBwJbpNAS4qa6TO3GbmUHSqyTfqRaSOgB7ACMAImJFRCwGDgdGpruNBI5I5w8HRkViMrChpB61XcOJ28wM1qhUImmIpJdypiE5Z9oMWADcLunfkm6V1A7oFhEfpvvMBbql872AmTnHz0rX1aggNycldQUuBPoCrwN/iIhPC3EtM7MGsQY3JyNiODC8hs2lwI7A2RHxgqQ/8WVZpPL4kBT1DbVQLe5RwDLgeqA9cF2BrmNm1jAarsY9C5gVES+ky/eRJPJ5lSWQ9Of8dPtsoE/O8b3TdTUqVHfAHhHxy3R+vKQpBbpOo/p0yVKGXXYt02e8DxKXXjiU1q1acemV1/PZ58vp2aMrlw87l/bt2gFwy6i/8sDD4ylp0YILhv6A3XbdqcifIHtefO1Jli5ZRnlFOeVl5ey/19FsuGEH/u/2a+izcS9mfjCbIacM5ZNP/Afd2mjRogVPPvMAcz+cxwnHnMHNt17F9jtsy8qVZUx5+TV+/pOLKCvLzpOF9dJAY5VExFxJMyVtFRFvA/sAb6XTYOCy9OfY9JBxwI8k3QPsCnySU1KpVsFq3JI6pt1fNgJKqixn0mXX3sxuu+7MQ6Nv4YGRN7D5Jn0Ydtm1/PQHp/LgHTexzx4Duf2u+wH477vv89iEZxh7583cfM1vufSqP1Nenp0O/k3Jdw8dzL67f4f99zoagLOHns6zzzzPwJ0O4NlnnufsoacXOcLsO+MHg5n2zn9XLd835iH673QAu/c/hDZtWnPS4KOLGF0jaaCbk6mzgbskvQZsD/yeJGF/W9I0YN90GeBRYAYwHbgF+GFdJy9U4u4AvJwzbQBMSedfKtA1C2rJ0mW8/OobfPfQ/QFo2bIlG6zfnvdnzmbn7bcDYMD/7Mjfn5kEwMRnJ3PgPnuy3nrr0btndzbu3ZPXp75TtPibk/0P2psxo5PGypjRYzng4H2KHFG29ejZjW/vP4g7R967at2TTzyzan7Ky6/Ro2f3YoTWuBqwO2BEvBIRO0fENyPiiIj4OCIWRcQ+EbFlROwbER+l+0ZEnBURW0TEdhFRZ44sSOKOiE0jYvOI2KyaafNCXLPQZs+ZS8cNO/Cr313DUaecxUV/uJbPPl/OFpttwsRnnwfgiaeeZe68hQDMX7CI7t26rDq+W9fOzF+wsCixZ1lEcM+DIxj/9H18L231denaifnzFgAwf94CunTtVMwQM+93l/2Siy+6gopqElJpaSnHHHs4E598tgiRNbIGfACn0BqtO6CkLST9WtKbjXXNhlRWXs7Ud6Zz7JEHc99fbqBNm9aMuGMMl144lHseeJhjvn82yz77nJYtPYpAQzrsgBPZb8/vcuJRQzj19BPoP3Dnr+wTUe+b8+u8/Q4YxMKFi3j1ler/s7zymt/wz3++yOTnM/mH8ppp2AdwCqqgiVtST0lDJb0IvJle77ha9l/VN/LWUaMLGdoa6961M926dOab39gagP0GfYu33pnO5pv04ZZrf8+Y267noH33pE+vpN981y6dmJu2CgHmzV9I1y6dixJ7ls39MLnxvnDhRzz28JPssON2LJi/iK7pXzNdu3Vh4YKPihlipu2y604ccOA+THl9IsNv/yPf2qM/N91yJQC/OP9HdOq8Eb++4A9FjrKRrOuJO03ATwFPA52A04API+LiiHi9puMiYnhaF9r5f08+vhCh1VvnThvRvWsX3n1/FgCTX36FLTbdmEUfLwagoqKC/xt5D8cccRAAe32rP49NeIYVK1Ywa85cPpg1h+2+/rVihZ9Jbdu2oV37tqvm99xrN/4zdRpPPDaRY44/HIBjjj+c8Y9OLGaYmfbbi6/mm1/fgx2325shpw5l0j8m84PTf8H3Tj6avfb5FkO+P3Td+YsmIv+pyAr1d/2fgeeBEyoL7WvT2bypuHDoDzjv4itYWbaSPj17cOmFQxn3+ATueeBhAPbdcyBHHrwfAH0334T9996dw048g9KSEn75sx9SUlJSzPAzp3OXTtx+1/UAlJaU8sB9D/PUhEm8MuUNhv/lGk446ShmzZzDkFOGFjnS5ueqay9m5sw5PPbkGAAeeegJrrr8hiJHVWAZ6u6oQvw2ldQJOBo4HugOjAFOiYg+tR6YY+XCGZlP9E1dn74HFzuEdUJZhl5Cm1ULP31Ha3uOz+/8Zd45p833frfW11sbhepVsigibo6IPUk6ny8meWpoqqTfF+KaZmZrZV2vceeKiFkRcXVE7EwyCtbyQl/TzGyNZajGXaibk+fmzK965Coi3gFaF+KaZmZrxS3u1br8XVBl2wEFuqaZWf1lKHEXqleJapivbtnMrOgiQ2MJFSpxRw3z1S2bmRVfE2hJ56tQibufpE9JWtdt0nnSZde4zazpaQJjkOSrIIk7IvykiZllS0V2igEeEcnMDFwqMTPLHN+cNDPLGLe4zcwyxjVuM7OMWdd7lZiZZY5b3GZm2RKucZuZZYx7lZiZZUyGSiWN9pZ3M7MmrQFHB5T0nqTXJb0iqfL1jRtJ+rukaenPjul6SbpO0nRJr0nasa7zO3GbmUHS4s53ys9eEbF9+hIZgPOBCRGxJTAhXQY4ENgynYYAN9V1YiduMzNIugPmO9XP4cDIdH4kcETO+lGRmAxsKKlHbSdy4jYzgzVqcUsaIumlnGlIlbMF8ISkl3O2dYuID9P5uUC3dL4XMDPn2Fnpuhr55qSZGRBl+fcqiYjhwPBadvlWRMyW1BX4u6T/VDk+JNX7bqhb3GZm0KA17oiYnf6cDzwI7ALMqyyBpD/np7vPBvrkHN47XVcjJ24zM2iwGrekdpLWr5wH9gPeAMYBg9PdBgNj0/lxwMlp75L+wCc5JZVquVRiZgYN2Y+7G/CgJEhy7N0R8bikF4Exkk4D3geOSfd/FDgImA58Bpxa1wWcuM3MgGigxB0RM4B+1axfBOxTzfoAzlqTazhxm5kBrMHNyWJz4jYzg0w98u7EbWYGTtxmZlmTlJqzwYnbzAzc4jYzyxwn7rXXceOv9JqxBjZ/2KBih7BO2ODXTxQ7BMtDlPkNOGZm2ZKdvO3EbWYGDfcATmNw4jYzA9e4zcwyx6USM7NscanEzCxjosyJ28wsW1wqMTPLlvq/A7jxOXGbmYFb3GZmWdNsW9ySOgJ9IuK1AsVjZlYUUVbsCPJXZ+KW9DRwWLrvy8B8Sc9FxM8KHJuZWaPJUos7n7e8d4iIT4HvAKMiYldg38KGZWbWuBroJe+NIp/EXSqpB8kbiR8ucDxmZsURyn8qsnxq3JcA44FJEfGipM2BaYUNy8yscTWFlnS+6kzcEXEvcG/O8gzgu4UMysyssUVF8VvS+aoxcUu6HqjxGdCI+HFBIjIzK4KK8oZN3JJKgJeA2RFxiKTNgHuATiQdPU6KiBWSWgGjgJ2ARcCxEfFebeeurcX9UkMEb2aWBQUolfwEmApskC5fDvwxIu6RdDNwGnBT+vPjiOgr6bh0v2NrO3GNiTsiRuYuS2obEZ/V/zOYmTVdDVkqkdQbOBj4HfAzSQL2Bk5IdxkJ/IYkcR+ezgPcB/xZkqKW187X2atE0gBJbwH/SZf7SbqxXp/GzKyJish/kjRE0ks505Aqp7sWOJcvH6TvBCyOWPWYzyygVzrfC5iZxBBlwCfp/jXKp1fJtcD+wLj0xK9K2iOP48zMMmNNWtwRMRwYXt02SYcA8yPiZUmDGiS4KvJ65D0iZiYt/VXKCxGMmVmxNODNyd2AwyQdBLQmqXH/CdhQUmnaqu4NzE73nw30AWZJKgU6kNykrFE+D+DMlDQQCEktJZ1DUnA3M2s2okJ5T7WeJ+KCiOgdEZsCxwETI+JE4CngqHS3wcDYdH5cuky6fWJt9W3IL3GfCZxFUoeZA2yfLpuZNRsRynuqp/NIblROJ6lhj0jXjwA6pet/Bpxf14nyeQBnIXBifSM1M8uCQjw5GRFPA0+n8zOAXarZZzlw9JqcN59eJZtLekjSAknzJY1NH3s3M2s2KkJ5T8WWT6nkbmAM0APoSfL4++hCBmVm1tgaoVTSYPJJ3G0j4o6IKEunO0nulJqZNRsV5cp7KrbaxirZKJ19TNL5JM/YB8mjmI/W94KSPoiIjet7vJlZITSLQaZIBkEJoPLTnJGzLYAL6nnN7Hw7ZrbOaAq163zVNlbJZgW6Zq39E83MiqEp1K7zldeTk5K2BbYhp7YdEaNq2b+m91EKaL8mATZFvXr14JZbr6Zr185EBLffNpobb/wL2273df503W9p364t738wm9NO/SlLliwtdriZ0vrMK2HFcqioICrK+WLUJbQcdAwlfbeH8jIqFs9nxaMj4IvPabHpNqy359FQUgrlZax4agwVH/jZsDVxy/CrOfigfZm/YCHb77APAB07bsjou25ik0368P77MznuhDNZvPiTIkdaeLU/8tK05NMdcBhwfTrtBVxB8vLg2qxfw9Se5NHPTCsrL+OCC37Hzjvtx16DvsPpZ5zM1lv35YYb/8CwX1/BrrscyEPjxvPToVXHnbF8LB99Ocv/MowvRl0CQPl7b7J8xK9YfvtFxEfzaNn/EADis6V8cf+fWH7br/nikVtZ75DTixl2Jo0aNYaDD1n9MY3zzj2LiU9N4uvf+BYTn5rEeeeuG8/bZak7YD4t7qOAfsC/I+JUSd2AO2s7ICIuBpDUOX2Ap1mZN3cB8+YuAGDp0mW8/fZ0evTsTt++mzFp0gsATJwwibHjRnLpJdcUM9RmoeK9N7+cn/NfSrbaGYCY/8Gq9bFwNiptuar1bfl5dtILbLJJ79XWHXro/uyzb/Jk9qg77mXCk/dxwYW/L0Z4jaoiQzcn8+kO+HlEVABlkjYA5pMMiFIjSYdIWgC8JmlWOtZJs7Txxr3o128bXnrxFaZOncYhh34bgCO/cxC9evcocnQZFEHrY86h9eBhlPTb8yubS7+5O+UzXv/K+pKtdqZi3vtO2g2gW9fOzJ07H4C5c+fTrWvnIkfUOLLU4s4ncb8kaUPgFpKeJlOA5+s45vfA7hHRk+T9lH/IJ5jcMW5Xli3J55CiateuLXeNvonzzr2UJUuW8sMzz+X000/i2efGsf767VixYmWxQ8ycL+76PctH/obl915Dyx33pkXvr63aVjrgEKKinPK3Vv/np849abnn0awYP7Lq6awB1DHeUbORpQdw8hmr5Ifp7M2SHgc2iIjX6jisLCL+kx7/gqT18wkmd4zb9m03a9L/WkpLS7nr7pv46z1jGTd2PADvvDODww87GYC+fTdj/wP2LmaImRRLFyczny2h/J0ptOi5ORWz3qFk290o2aIfX9xz5Wr7a/2OtDrybFY8cguxeEHjB9wMzZu/kO7duzJ37ny6d+/K/AW1jjDabDSFlnS+ansAZ8fatkXElFrO27VKz5LVliMi84XfG2+6nLffns6frx+xal2XLp1YsGARkjj3vB8x4ta7ihhhBrVcD9Qi6VXScj1abLYtK58bS4vNtqXlrgey/O7LoWzFl/u3akOro37Kymfuo2L29OLF3cw8/NATnHzS0Vxx5Q2cfNLRPPTQ+GKH1CiadEuxitpa3FfXsi1I3p9Wk1tIepHUtJxpAwbszAknfoc3Xv8P/5z8CAC/GXYlfbfYlNPPSFrc48Y+zh2j7i1mmJmjth1o9Z0fJQstSih7azIV775B6yGXQUlLWh97DgDlc/7LyidGUbrjvmjDbrQceBgtByYdnZaPuQo+a/pltqbizjtuYM89BtC580a8N+MlLr7kKi6/8gbuuftmTj3leD74YBbHnXBmscNsFOUV+VSOmwY11fpVUy+VNAfzhw0qdgjrhA1+/USxQ2j2ylbMXus6x7Pdj8o75+w+976i1lXyegBnTUm6rrbtEfHjQlzXzKy+IkOjcRQkcZP0Pql0MTCsQNcxM2sQFRn6G78giTsiVvXLkvTT3GUzs6aoIkMt7nweeZek70m6KF3eWNJXXr9Tiwz9HjOzdVWgvKdiy+c26o3AAOD4dHkJcEPBIjIzK4JylPdUbPmUSnaNiB0l/RsgIj6WtF5tB0hawpct7baSPq3clJwiNqh3xGZmBVCAdwUXTD6Je6WkEtJELKkLdXzGiGg2fbbNbN2QpcSdT6nkOuBBkqcffwdMIhmLxMys2chSjTufsUrukvQysA9JqeOIiPBo9WbWrDTUqK6SWgP/AFqR5Nj7ImKYpM1I3t3biaTL9EkRsUJSK2AUsBOwCDg2It6r7Rr59CrZGPgMeAgYByxL15mZNRsVKO+pDl8Ae0dEP2B74ABJ/YHLgT9GRF/gY+C0dP/TgI/T9X9M96tVPqWSR4CH058TgBnAY3kcZ2aWGeVrMNUmEpXvLGyZTpXjO92Xrh8JHJHOH54uk27fR1Ktvx3yKZVsl7ucjhr4wxp2NzPLpIrac+VqJA0Bct9NODwdlrpyewlJOaQvSffp/wKLI6LyTR+zgF7pfC9gJkBElEn6hKScUuPbw9b4ycmImCJp1zU9zsysKVuTJwVz3x1Qw/ZyYPv0JTQPAluvXXSrqzNxVxlXuwWwIzCnIYMwMyu2QnQHjIjFkp4ieYhxQ0mlaau7NzA73W02yesgZ0kqBTqQ3KSsUT417ty3tLciqXUfXq9PYWbWRFUo/6k2krqkLW0ktQG+DUwFniJ5+TrAYGBsOj8uXSbdPjHqGG+71hZ3WqdZPyLOqT1UM7Nsa8BH2XsAI9P82QIYExEPS3oLuEfSb4F/A5WvzxoB3CFpOvARcFxdF6jt1WWlaaF8t7X9FGZmTV1D9eNO38m7QzXrZwBfGaAvIpYDR6/JNWprcf+LpJ79iqRxwL3AspyLPbAmFzIza8qy9Mh7Pr1KWpMUyvcmufGq9KcTt5k1G1kaf7q2xF35ZvY3+DJhV8rSZzQzq1NDlUoaQ22JuwRoD9VW7J24zaxZaS6lkg8j4pJGi8TMrIjKm0mLO0Mfw8xs7TSXFvc+jRaFmVmRNYvEHREfNWYgZmbFlKUbd2s8yJSZWXPUXHqVmJmtM5pFqcTMbF1S1wsSmhInbjMzXCoxM8scl0rMzDLGvUoaQIdWbYsdQrPX5eKnih3COuGTYX4kIgsqMpS6m2ziNjNrTL45aWaWMa5xm5lljHuVmJlljGvcZmYZk5207cRtZga4xm1mljnlGWpzO3GbmZGtFneLYgdgZtYUVBB5T7WR1EfSU5LekvSmpJ+k6zeS9HdJ09KfHdP1knSdpOmSXpO0Y12xOnGbmZHcnMx3qkMZ8POI2AboD5wlaRvgfGBCRGwJTEiXAQ4EtkynIcBNdV3AidvMjKRUku9Um4j4MCKmpPNLgKlAL+BwYGS620jgiHT+cGBUJCYDG0rqUds1nLjNzEhuTuY7SRoi6aWcaUh155S0KbAD8ALQLSI+TDfNBbql872AmTmHzUrX1cg3J83MWLMHcCJiODC8tn0ktQfuB34aEZ9KXz6aGREhqd7dWNziNjOjQWvcSGpJkrTviogH0tXzKksg6c/56frZQJ+cw3un62rkxG1mRoP2KhEwApgaEdfkbBoHDE7nBwNjc9afnPYu6Q98klNSqZZLJWZmNGg/7t2Ak4DXJb2SrrsQuAwYI+k04H3gmHTbo8BBwHTgM+DUui7gxG1mBkQDPTkZEZOAmsYa/MpbNSIigLPW5BpO3GZm+JF3M7PMydIj707cZmZARbjFbWaWKdlJ207cZmaA34BjZpY5DdWrpDE4cZuZAWVO3GZm2eIWt5lZxrg7YBXpgCvbArMjYn5d+5uZNbbIUHfAggwyJelmSd9I5zsArwKjgH9LOr4Q1zQzWxsNNchUYyjU6IC7R8Sb6fypwDsRsR2wE3Buga5pZlZva/IihWIrVKlkRc78t4F7ASJibu5g4mZmTUVTaEnnq1CJe7GkQ0gGA98NOA1AUinQpkDXNDOrtyzVuAuVuM8ArgO6k7y2Z266fh/gkQJds1FNfvUJli5dRkV5BWVlZRy097HcNOIqtthyMwA26LA+n36yhP32+G6RI82mXr16cMut19C1a2cigttvG82NN94OwJlnDmbIGSdTXl7O+Mcn8qtfXVbkaLOnzVlXw4rlRFRARQXLbxtGydb/w3p7HIk692T57RdT8eG7AJR8YwAtBxy06tgWXfuwfMRFVMz7oFjhF4R7lcCCiDig6sqIGA+ML9A1G93Rh57Kxx8tXrX8g9POWTV/0aW/4NNPlxYhquahvLyMCy/4La+88ibt27dj0nMPMXHis3Tt2oVDDvk2/Xc9kBUrVtClS6dih5pZn9/5B/j8y3+jFQtms/y+62h10Orj+Je/+Tzlbz4PgLr0pvXRP2l2SRvcjxvgbUkLgeeAfwLPRcQ7BbpWk3TokftzzGHfL3YYmTV37gLmzl0AwNKly3j77f/Ss2d3Tjn1OK6++iZWrEhuoyxYsKiYYTYrsWhOnfuUfqM/ZW+90AjRNL4s1bgL0qskIroCR5Ak7gHAA5LmSRorqVn0KokIRj9wC489NYYTBx+92rZdB+7EgvmLeHdG82uVFMPGG/emX79tePHFV9hyy80ZuNsuPP3M33h8/F/ZcadvFju8zGp9wrm0/v7FlO4wKO9jSrfZlbK09d3clEdF3lOxFewBnLSF/Q7wF0lbkLxT7SfAfsAVhbpuYznywJOY++F8OnXeiHsevJXp02bwwj9fBuCI7x7E2PsfLXKEzUO7dm25e/RNnHvuJSxZspTSkhI6duzAoD2PYKed+3HHHTfwjW12L3aYmbN81G+JJR9D2/VpfcJ5VCz8kIqZb9d6TIuem8PKFcSCWl9AnllZKpUU6gGcgZLOkXS/pH8BvwNKgO8BHWo5boiklyS9tOyLjwsRWoOZ+2HyAOiihR/x2MNPsv2O2wFQUlLCgYfsy7gHHy9meM1CaWkpd999M3+952+MG5vcGpk9Z+6q+ZdfepWKigo6d96omGFmUixJ//v6bAnlb7+cJOU6lG7Tn7I3Jxc4suKpiMh7KrZCPYAzCTgOeAAYFBHHRcS1ETE5IlbUdFBEDI+InSNi53atOhYotLXXpm0b2rVvu2p+z70H8vbU6QDsPmgA06e9y4dz5hUzxGbhppsu5+23p3P99SNWrXvooSfYY8/+APTtuxnrrdeShQs/KlaI2dRyPViv9ar5ks23JRbMquMgUbLNLpS91XwTd6zBVGyFKpX0BAam0xlp/+0pwPPA8xExo0DXbRRdunRixJ3XAUkL+2/3P8LTEyYBcPh3DnSZpAEMGLAzJ5z4Xd54fSrPT06+z98Mu4JRI8dw881X8OKL41mxciVDTv95kSPNHrXrQKujfpLMt2hB2ZvPUz7jdUq22on19jsJtV2f1sf8jPJ5H/DFPVcC0GLjrYhPPyIWLyhm6AWVpZuTaoxO55LaAt8HfgpsFhEldR3Tq+M3svMtZtTiL5YVO4R1wvxf7lHsEJq9dr8ctdaPZA/otVfeOef52U8V9RHwgrS404GlBvBlq3sHYBrwEElPEzOzJqUhe4tIug04BJgfEdum6zYC/gpsCrwHHBMRHysZB+RPJB04PgNOiYgptZ2/UDXu6cAPgc+BS4DeEdE/IoZGxH0FuqaZWb3FGvwvD38Bqj6EeD4wISK2BCakywAHAlum0xDgprpOXpAWd0R0KcR5zcwKpSHLxhHxD0mbVll9ODAonR8JPA2cl64fFUkAkyVtKKlHRHxY0/kLVSoZV9v2iDisENc1M6uvNbk5KWkISeu40vCIGF7HYd1ykvFcoFs63wuYmbPfrHRd4yZukvr2TGA08ALgsVzNrElbkxZ3mqTrStS1HR+S6t3EL1Ti7k4yDvfxwAkkIwKOznm5gplZk1Je+PEB51WWQCT1ACpf4zgb6JOzX+90XY0KNVZJeUQ8HhGDgf4kNyuflvSjQlzPzGxtNcKTk+OAwen8YGBszvqTlegPfFJbfRsKOFaJpFbAwSSt7k1Jxud+sFDXMzNbGw05Vomk0SQ3IjtLmgUMAy4Dxkg6DXgfOCbd/VGSroDTSboDnvqVE1ZRqJuTo0je6v4ocHFEvFGI65iZNZSGHIMkImp6Kfo+1ewbwFlrcv5Ctbi/BywjGQ3wxznvmRRJnBsU6LpmZvWSpdEBC9WPu1AP9piZFURTGPUvXwWrcZuZZUlTeEFCvpy4zcxwqcTMLHPCLW4zs2zJ0njcTtxmZjTsIFOF5sRtZoZb3GZmmVNe4Rq3mVmmuFeJmVnGuMZtZpYxrnGbmWWMW9xmZhnjm5NmZhnjUomZWca4VGJmljEe1tXMLGPcj9vMLGPc4jYzy5gKD+tqZpYtvjlpZpYxTtxmZhmTnbQNytJvmaZO0pCIGF7sOJozf8eF5++46WtR7ACamSHFDmAd4O+48PwdN3FO3GZmGePEbWaWMU7cDct1wcLzd1x4/o6bON+cNDPLGLe4zcwyxonbzCxjnLjzJKlc0is50/k52zpLWinpzCrHvCfpdUmvSXpCUvfGjzxbJC2tsnyKpD+n87+RNDv9/t+QdFjO+nOKEW9WSApJd+Ysl0paIOlhJRZK6phu65Hu/62c/RdI6iRpK0lPp/8fTJXkengROHHn7/OI2D5nuixn29HAZOD4ao7bKyK+CbwEXNgYgTZzf4yI7Um+89sk+d9wfpYB20pqky5/G5gNEMmNrsnAgHTbQODf6U8kbQUsiohFwHWk/x9ExNeB6xvvI1gl/6NvGMcDPwd6Sepdwz7/APo2XkjNW0RMBcqAzsWOJUMeBQ5O548HRuds+ydpok5//pHVE/lz6XwPYFblQRHxeqGCtZo5ceevTZVSybEAkvoAPSLiX8AY4Ngajj8E8D/yuq32PQOXVLeTpF2BCmBBYwaXcfcAx0lqDXwTeCFn23N8mbh3AR4E+qTLA0kSOyQJfaKkxyQNlbRhwaO2r/AgU/n7PP0TvapjSRI2JP9h3AZcnbP9KUnlwGvArwoaYfOw2vcs6RRg55ztQyV9D1gCHBsRIalxI8yoiHhN0qYkre1Hq2x+EdhBUjugZUQslTRDUl+SxH11eo7bJY0HDgAOB86Q1C8ivmi0D2JO3A3geKC7pBPT5Z6StoyIaenyXhGxsEixNUd/jIirih1Eho0DrgIGAZ0qV0bEZ5KmAd8HpqSrJwMHAV2Bt3P2nUPSQLlN0hvAtsDLjRG8JVwqWQuSvga0j4heEbFpRGwK/IHqb1KaNQW3ARfXUJv+J/BT4Pl0+XngJ8Dk9AYmkg6Q1DKd706S/GcXOmhbnRN3/qrWuC8jSdAPVtnvfpy4i+FXkmZVTsUOpqmKiFkRcV0Nm58DNufLxD0F6M2X9W2A/YA3JL0KjAd+ERFzCxWvVc+PvJuZZYxb3GZmGePEbWaWMU7cZmYZ48RtZpYxTtxmZhnjxG1fkTMS4huS7pXUdi3O9RdJR6Xzt0rappZ9B0kaWNP2Wo57T9JXxiypaX2VfZbWtr2a/T0SoRWdE7dVp3IkxG2BFUDV4Wrr9cRtRPxvRLxVyy6D+HK8DDOrgRO31eVZoG/aGn5W0jjgLUklkq6U9GI63vgZAOnYzn+W9LakJ0kelybd9rSkndP5AyRNkfSqpAnpGBpnkoxF8oqk3SV1kXR/eo0XJe2WHttJyfjmb0q6FahzsBJJf5P0cnrMkCrb/piunyCpS7puC0mPp8c8K2nras75Y0lvpZ//nnp+v2ZrzGOVWI3SlvWBwOPpqh2BbSPi3TT5fRIR/yOpFfCcpCeAHYCtgG2AbsBbJI9Z5563C3ALsEd6ro0i4iNJNwNLK8cikXQ3ydgkkyRtTPKk3teBYcCkiLhE0sHAaXl8nO+n12gDvCjp/nR86XbASxExVNJF6bl/RPLC3DMjYlo6EuGNwN5Vznk+sFlEfOFR8qwxOXFbddqkQ6pC0uIeQVLC+FdEvJuu3w/4ZmX9GugAbAnsAYyOiHJgjqSJ1Zy/P/CPynNFxEc1xLEvsE3O6H8bSGqfXuM76bGPSPo4j8/0Y0lHpvN90lgXkQwN+9d0/Z3AA+k1BgL35ly7VTXnfA24S9LfgL/lEYNZg3Ditup8ZQjbNIEty10FnB0R46vsd1ADxtEC6B8Ry6uJJW+SBpH8EhiQjoL3NNC6ht0jve7iGobxzXUwyS+RQ4FfStouIsrWKDizenCN2+prPPCDnJHivpaO5fwP4Ni0Bt4D2KuaYycDe0jaLD12o3T9EmD9nP2eAM6uXJC0fTr7D+CEdN2BQMc6Yu0AfJwm7a1JWvyVWgCVfzWcQFKC+RR4V9LR6TUkqV/uCZW8Mq1PRDwFnJdeo30dcZg1CCduq69bSerXU5SMyfx/JH/BPQhMS7eN4suR5laJiAXAEJKyxKt8Wap4CDiy8uYk8GNg5/Tm31t82bvlYpLE/yZJyeSDOmJ9HCiVNBW4jOQXR6VlwC7pZ9ibL9+4cyJwWhrfmyQvDchVAtwp6XWS9zNeFxGL64jDrEF4dEAzs4xxi9vMLGOcuM3MMsaJ28wsY5y4zcwyxonbzCxjnLjNzDLGidvMLGP+H5GfFOHej4erAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"code","source":"from nltk import ngrams\n\ndef extract_ngram(text, i, n):\n textpart = text[max(i-(n-1), 0):i+n]\n return (\"_\".join(w) for w in ngrams(textpart, n))\n\n\nlist(extract_ngram(\"This is a test sentence\".split(), 3, 3))","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:44:15.001965Z","iopub.execute_input":"2023-01-15T15:44:15.002425Z","iopub.status.idle":"2023-01-15T15:44:15.985074Z","shell.execute_reply.started":"2023-01-15T15:44:15.002385Z","shell.execute_reply":"2023-01-15T15:44:15.984139Z"},"trusted":true},"execution_count":8,"outputs":[{"execution_count":8,"output_type":"execute_result","data":{"text/plain":"['is_a_test', 'a_test_sentence']"},"metadata":{}}]},{"cell_type":"code","source":"from collections import defaultdict\nnegative_contribs = {0:defaultdict(int), 1:defaultdict(int), 2:defaultdict(int)}\nneg_text = [\"\", \"\", \"\"]\n\npos_contribs = {0:defaultdict(int), 1:defaultdict(int), 2:defaultdict(int)}\nsum_contribs = {0:defaultdict(int), 1:defaultdict(int), 2:defaultdict(int)}\npos_text = [\"\", \"\", \"\"]\nk=4\nn=1\n\nfor a in attributions:\n if a[\"pred_class\"] != a[\"true_class\"]:\n topk_neg = torch.topk(a[\"word_attr\"], k=k, largest=False).indices\n for i in topk_neg:\n for ngram in extract_ngram(a['words'], i, n):\n negative_contribs[a[\"true_class\"]][ngram] += 1\n\n else:\n topk_pos = torch.topk(a[\"word_attr\"], k=k, largest=True).indices\n for i in topk_pos:\n for ngram in extract_ngram(a[\"words\"], i, n):\n if ngram == \"fever_which\":\n print(a)\n pos_contribs[a[\"true_class\"]][ngram] += 1\n #sum_contribs[a[\"true_class\"]][ngram] += a['word_attr']\n \neap_neg, hpl_neg, mws_neg = negative_contribs.values()\neap_pos, hpl_pos, mws_pos = pos_contribs.values()","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:53:31.367133Z","iopub.execute_input":"2023-01-15T15:53:31.367831Z","iopub.status.idle":"2023-01-15T15:53:31.600849Z","shell.execute_reply.started":"2023-01-15T15:53:31.367795Z","shell.execute_reply":"2023-01-15T15:53:31.599575Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"code","source":"len(set(eap_pos))/2/eap_right","metadata":{"execution":{"iopub.status.busy":"2023-01-14T07:29:29.591438Z","iopub.execute_input":"2023-01-14T07:29:29.594000Z","iopub.status.idle":"2023-01-14T07:29:29.602901Z","shell.execute_reply.started":"2023-01-14T07:29:29.593961Z","shell.execute_reply":"2023-01-14T07:29:29.601815Z"},"trusted":true},"execution_count":20,"outputs":[{"execution_count":20,"output_type":"execute_result","data":{"text/plain":"0.8934782608695652"},"metadata":{}}]},{"cell_type":"code","source":"from sklearn.feature_extraction.text import TfidfTransformer\nfrom collections import defaultdict\nfrom transformers import AutoModelForSequenceClassification\nberttokenizer = AutoTokenizer.from_pretrained('bert-base-cased')\n\ndef encode_text(author):\n auth_text = valid[valid['author'] == author]['text']\n encodings = berttokenizer(\n [text for text in auth_text],\n add_special_tokens=True)\n for sent in encodings['input_ids']:\n yield berttokenizer.convert_ids_to_tokens(sent)\n\ndef document_freq(encodings):\n df = defaultdict(int)\n nr_docs = 0\n for document in encodings:\n nr_docs += 1\n for token in document:\n df[token] += 1\n \n return df, nr_docs\n\ndef get_tfidf_most_attributed(d, author, n):\n sentences_tokenized = encode_text(author)\n print(author)\n dfs, document_count = document_freq(sentences_tokenized)\n \n get_idf = lambda tok : np.log( (1+document_count) / (1+dfs[tok]) + 1)\n #print(get_idf('the'))\n \n return {tok:freq*get_idf(tok) for tok, freq in d.items()}\n\nid2auth = {0:'EAP', 1:\"HPL\", 2:\"MWS\"}\nauth2id= {val:key for key, val in id2auth.items()}\nauthor=\"EAP\"\n\nsorted(get_tfidf_most_attributed(pos_contribs[auth2id[author]], author, 1).items(), key=lambda v:v[1], reverse=True)[:10]","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:44:38.763947Z","iopub.execute_input":"2023-01-15T15:44:38.764320Z","iopub.status.idle":"2023-01-15T15:44:41.517062Z","shell.execute_reply.started":"2023-01-15T15:44:38.764288Z","shell.execute_reply":"2023-01-15T15:44:41.515868Z"},"trusted":true},"execution_count":11,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/29.0 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ba06a47442e44528b6c2c1890db53dcd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/570 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"78b9fe3cd9ad4487835160ae19227301"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/208k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b80db7fac45748cd97abd5226953f499"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading: 0%| | 0.00/426k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6d96ddc379a54972af5ceca4426af072"}},"metadata":{}},{"name":"stdout","text":"EAP\n","output_type":"stream"},{"execution_count":11,"output_type":"execute_result","data":{"text/plain":"[('._[SEP]', 373.20690513962285),\n ('[CLS]_I', 373.20690513962285),\n ('upon_the', 179.93904354946102),\n ('of_the', 159.94581648840978),\n ('[CLS]_\"', 126.62377138665775),\n ('and_,', 119.95936236630733),\n (',_in', 99.96613530525612),\n ('at_length', 93.30172628490571),\n ('[CLS]_This', 86.63731726455529),\n ('Mr_.', 79.97290824420489)]"},"metadata":{}}]},{"cell_type":"code","source":"from wordcloud import WordCloud\nimport nltk\nfrom collections import defaultdict\nfrom sklearn.feature_extraction.text import TfidfVectorizer\n\n\nfrom transformers import AutoModelForSequenceClassification\nberttokenizer = AutoTokenizer.from_pretrained('bert-base-cased')\n\ndef get_ngram_freq(author, n):\n sent_grams = (set(nltk.ngrams(sent, n)) for sent in encode_text(author))\n grams = defaultdict(int)\n for sent in sent_grams:\n for gram in sent:\n grams[\"_\".join(gram)] += 1\n \n return grams\n\ndef encode_text(author):\n auth_text = valid[valid['author'] == author]['text']\n encodings = berttokenizer(\n [text for text in auth_text],\n add_special_tokens=True)\n for sent in encodings['input_ids']:\n yield berttokenizer.convert_ids_to_tokens(sent)\n \ndef normalize_ngrams(d: dict, author, n):\n all_bigrams = get_ngram_freq(author, n)\n return {gram:freq / all_bigrams[gram] for gram, freq in d.items()}\n\ndef remove_idf(d: dict, author, n):\n tfidf = TfidfVectorizer()\n texts = encode_text(valid['text'][valid['author'] == author])\n tfidf.fit(texts)\n \n get_idf = lambda word: tfidf.idf_[tfidf.vocabulary_[word]]\n \n return {word:freq*get_idf(word) for word, freq in d.items()}\n\n\ndef make_hist(d, cutoff_ones=False, cutoff_count=10):\n value_counts = pd.Series(d).sort_values(ascending=False)\n \n if cutoff_ones:\n value_counts = value_counts[value_counts>1]\n \n if cutoff_count is not None:\n value_counts = value_counts[:cutoff_count]\n \n value_counts.sort_values(ascending=True, inplace=True)\n \n #ax = value_counts.plot(kind='bar')\n ax = plt.barh(value_counts.index, value_counts)\n rects = ax.patches\n labels = [value_counts.index[i] for i in range(len(rects))]\n \n #plt.bar_label(ax, labels, label_type=\"center\")\n #plt.yticks(color=\"w\")\n \"\"\"\n for rect, label in zip(rects, labels):\n height = rect.get_height()\n ax.text(\n rect.get_x() + rect.get_width() / 2, height, label, ha=\"center\", va=\"bottom\"\n )\n \"\"\"\n \n#get_ngram_freq('EAP', n)","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:44:48.058284Z","iopub.execute_input":"2023-01-15T15:44:48.059191Z","iopub.status.idle":"2023-01-15T15:44:49.644929Z","shell.execute_reply.started":"2023-01-15T15:44:48.059155Z","shell.execute_reply":"2023-01-15T15:44:49.643976Z"},"trusted":true},"execution_count":12,"outputs":[]},{"cell_type":"code","source":"!ls ..","metadata":{"execution":{"iopub.status.busy":"2023-01-15T16:08:49.600352Z","iopub.execute_input":"2023-01-15T16:08:49.600993Z","iopub.status.idle":"2023-01-15T16:08:50.679269Z","shell.execute_reply.started":"2023-01-15T16:08:49.600929Z","shell.execute_reply":"2023-01-15T16:08:50.677734Z"},"trusted":true},"execution_count":37,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nfigs.zip input lib working\n","output_type":"stream"}]},{"cell_type":"code","source":"import shutil\nshutil.make_archive(\"../figs\", 'zip', \"/kaggle/working\")\n!mv ../figs.zip .\n\n#!rm /kaggle/working/figs.zip","metadata":{"execution":{"iopub.status.busy":"2023-01-15T16:09:03.281734Z","iopub.execute_input":"2023-01-15T16:09:03.282099Z","iopub.status.idle":"2023-01-15T16:09:04.377474Z","shell.execute_reply.started":"2023-01-15T16:09:03.282061Z","shell.execute_reply":"2023-01-15T16:09:04.376188Z"},"trusted":true},"execution_count":38,"outputs":[{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"}]},{"cell_type":"code","source":"author = \"MWS\"\nauth_dict = {\"EAP\":0, \"HPL\":1, \"MWS\":2}\ncutoff = 4\n\nprint('Positive contributions to correct predictions of', author)\n#make_wordcloud(eap_pos)\n\nall_bigrams = get_ngram_freq(author, n)\ncontribs = pos_contribs[auth_dict[author]]\nnorms = normalize_ngrams(contribs, author, n)\nidfs = get_tfidf_most_attributed(contribs, author, n)\nfreq_occuring = {k:v for k,v in norms.items() if all_bigrams[k] > cutoff}\n\nplt.figure(figsize=(7, 4))\nmake_hist(idfs)\nplt.title(\"Frequency of maximum attribution\")\nplt.xlabel(\"Frequency * idf(token)\")\nplt.savefig(f\"{author}_{n}_freq.pdf\", format=\"pdf\", bbox_inches='tight')\nplt.tight_layout()\nplt.show()\n\nplt.figure(figsize=(7, 4))\nmake_hist(freq_occuring)\nplt.title(f\"Relative importance of ngrams appearing at least {cutoff} times.\")\nplt.xlabel(\"frequency / df(token)\")\nplt.savefig(f\"{author}_{n}_norm.pdf\", format=\"pdf\", bbox_inches='tight')\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-01-15T15:53:44.641939Z","iopub.execute_input":"2023-01-15T15:53:44.642282Z","iopub.status.idle":"2023-01-15T15:53:45.798726Z","shell.execute_reply.started":"2023-01-15T15:53:44.642250Z","shell.execute_reply":"2023-01-15T15:53:45.797808Z"},"trusted":true},"execution_count":27,"outputs":[{"name":"stdout","text":"Positive contributions to correct predictions of MWS\nMWS\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 504x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAfAAAAEYCAYAAACju6QJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAju0lEQVR4nO3deZgdVZnH8e/PBLJA7ChkMASxBQIRCAQIqxDZRARRRBQRhkUkio4LIzBBRkTRAQFX3CayBAQhwIgiURaBCAgCnZCVXWmWKJAAaZZAgM47f9RpKNrupDvpvnVP9+/zPPfpuqfqnnpP3dv93nNOdZUiAjMzM8vLW6oOwMzMzLrPCdzMzCxDTuBmZmYZcgI3MzPLkBO4mZlZhpzAzczMMuQEbtYHSNpE0ixJz0v6UoVxHCLpuqr2XwVJL0jaIC1PkfTtHqz7F5K+3lP1Wd8i/x+41TtJzcA6QGupeOOI+Ec1EdUfSecCz0XEsVXHkiNJpwAbRcShpbLpwEURcU436pkCPB4R/70SMRwBfCYidu7ua61/cg/ccrFfRKxZerwpeUsaWFVgdeJdwPyqg+hP/JmzqjmBW7YkhaQvSHoQeDCVfSgNJS+WdJukLUrbbyVpZhpmnirp0rbhTklHSLq1g/o3SsuDJJ0l6VFJT6ahzSFp3a6SHpf0VUlPSfqnpCNL9QyR9D1Jj0hqkXRrKpsm6Yvt9jlH0kc7ae+HJc1PbZsu6T2p/EZgN+AnaTh34w5eO13St9MxeUHS7yWtJeliSc9JuktSY2n7H0l6LK2bIWmX0ro/SPpe6fmlks7r6DimY/h5SQ+m436qpA1THM9JukzS6l18D6ZI+pmkP6Y2/EXSOyT9UNKzku6TtFVHx255bZK0N/A14KBU72xJ3wF2KR3Tn5Tiaf+Zez3GZG1J16f2/lnSu9J2jWnbgaWYpkv6THovfwHsmPa3uNTmb5e2P1rSQ5KekXSVpHXbHavPpWO9WNJPJamz42F9QET44UddP4BmYM8OygO4Hng7MATYCngK2B4YAByeXjsIWB14BDgWWA04EHgV+Haq6wjg1g7q3ygt/wC4Ku1rGPB74LS0blfgNeBbqe59gCXA29L6nwLTgVEprp1STJ8A7ijtb0vgaWD1Dtq6MfAi8P60jxOAh9q2TfV/ZjnHcHrafkOgAbgHeADYExgIXAicX9r+UGCttO6rwBPA4LTuHek47w4cAvwdGNbRcUzH8HfAW4HNgKXADcAGpTgO7+J7MAVYBGwDDAZuBB4GDkvH9dvATcs5Bstr0ykUw+Xtj9lnOojn9c9cJzE+D0xI7/GP2toENKZtB3a0j07aP4U3PqO7p/Zvneo+G7i5XWxXA8OB9YGFwN5V//760XsP98AtF79NvYrFkn5bKj8tIp6JiJeAicD/RsQdEdEaERdQJIwd0mM14IcR8WpEXAHc1ZUdp17MRODYtK/ngf8BPlna7FXgW6nuPwAvAJtIegvwaeDLEbEgxXVbRCyl+EKwsaTRqY5/B6ZGxCsdhHEQMC0iro+IV4GzKL607NSVNiTnR8TfIqIF+CPwt4j4U0S8BlxO8QUIgIi4KCKejojXIuJ7FAljk7TuCeAY4AKKBHVYOiadOSMinouI+cA84LqI+Hspjk57zR24MiJmRMTLwJXAyxFxYUS0AlOXV9fy2tRN5c9cR6ZFxM3pPT6Jolf9zpXYT3uHAOdFxMxU94mp7sbSNqdHxOKIeBS4CRjXA/u1OuUEbrnYPyKGp8f+pfLHSsvvAr5aSvSLgXcC66bHgogon7X5SBf3PQIYCswo1XtNKm/zdEqEbZYAawJrU/QW/9a+0pSEpgKHpkR/MPCrTmJYtxxvRCyjaPuoLrYB4MnS8ksdPF+z7Ymk4yTdm4b8F1P0ltcubf97il7v/RHxpmHvVdlvF6x0XV1oU1c91tX1EfEC8AzF+7eq2n8GXqAYsSl/Bp4oLbd9Bq2PcgK33JUT8mPAd0qJfnhEDI2IS4B/AqPazQmuX1p+kSJJAyDpHaV1iyiSw2alehsioit/HBcBL1MMXXfkAoqe1R7Akoi4vZPt/kHxBaUtPlF8OVnQhRi6Jc0Nn0AxxP+2iBgOtADlY/cd4F5gpKSDe2jXy3sPVkkX2tTRv+N09i86K/rXndd725LWpBhu/wdF+6DURorpiK7W2/4zsAbFlECPfwYsD07g1pf8EvicpO1VWEPSvpKGAbdTzFN/SdJqkg4Atiu9djawmaRxkgZTzIkCr/d2fwn8QNK/AUgaJekDKwoovfY84PuS1pU0QNKOkgal9bcDy4Dv0XnvG+AyYF9Je0hajWIOdylwW5eOTPcMozhWC4GBkk6mmMMGQNIE4EiKuefDgbMldWckoDOdvgc9YLltoujJN6aRkHLZBiuxr30k7ZxOzjsV+GtEPBYRCymS7aHpc/Bp3vzF7klgvbaT+jpwCXBkOj6DKKZx7oiI5pWI0foAJ3DrMyKiCTga+AnwLMVJW0ekda8AB6Tnz1DMKf+m9NoHKE5C+xPF2cXth4X/K9X3V0nPpe26On96HDCXYs79GeC7vPl370JgLHDRctp2P8VJWGdT9Or3o/jXuo7my1fVtRRTBA9QDNm+TBoWlvTWFO9/pDn9W4BzgfNX9YznLrwHq6LTNiWXp59PS5qZln8EHJjOcP9xN/b1a+AbFO/1NhTvW5ujgeMphr43481fwG6k+FfAJyQtal9pRPwJ+DrwfxQjShvy5vMwrJ/xhVys39IqXHSjh+M4DJgYvoCHmXWDe+BmFZI0FPg8MLnqWMwsL07gZhVJc+gLKeY+f11xOGaWGQ+hm5mZZcg9cDMzswz1y4vxr7322tHY2Fh1GGZmZq+bMWPGoogYseItC/0ygTc2NtLU1FR1GGZmZq+T1NWrQwIeQjczM8uSE7iZmVmGnMDNzMwy5ARuZmaWISdwMzOzDDmBm5mZZcgJ3MzMLENO4GZmZhnqlxdymbughcZJ06oOw8zMMtZ8+r6V7t89cDMzsww5gZuZmWXICdzMzCxDTuBmZmYZWmECl9QqaZakeZJ+L2l4DeJaaZKaJa1ddRxmZma9qSs98JciYlxEbA48A3yhl2MyMzOzFejuEPrtwCgASdtJul3S3ZJuk7RJKr9Z0ri2F0i6VdKWkk6RdIGkWyQ9IukASWdImivpGkmrpe33SHXOlXSepEGpvFnSNyXNTOvGpPK1JF0nab6kcwCt+mExMzOrb11O4JIGAHsAV6Wi+4BdImIr4GTgf1L5ucAR6TUbA4MjYnZatyGwO/Bh4CLgpogYC7wE7CtpMDAFOCiVDwSOKYWxKCK2Bn4OHJfKvgHcGhGbAVcC63e1TWZmZrnqSgIfImkW8ASwDnB9Km8ALpc0D/gBsFkqvxz4UOpRf5oiIbf5Y0S8CswFBgDXpPK5QCOwCfBwRDyQyi8AJpRe/5v0c0banrT+IoCImAY821EjJE2U1CSpqXVJSxeabWZmVr+6PAcOvItieLptDvxUih705sB+wGCAiFhCkeQ/AnwCuLhU19K0zTLg1YiIVL6Mrl0Vbmn62drF7V8XEZMjYnxEjB8wtKE7LzUzM6s7XR5CT4n5S8BXJQ2k6IEvSKuPaLf5OcCPgbsiosMecSfuBxolbZSe/zvw5xW85mbgUwCSPgi8rRv7MzMzy1K3TmKLiLuBOcDBwBnAaZLupl1vOCJmAM8B53ez/peBIymG5udS9Mx/sYKXfROYIGk+cADwaHf2aWZmliO9MYrdg5VK6wLTgTFpuLyuDBo5OkYe/sOqwzAzs4z19M1MJM2IiPFd3b7Hr8Qm6TDgDuCkekzeZmZmfUGP3040Ii4ELuzpes3MzOwNvha6mZlZhnq8B56DsaMaaKr4RuxmZmarwj1wMzOzDDmBm5mZZcgJ3MzMLEP9cg587oIWGidNqzoMMzNbjp7+P+u+xj1wMzOzDDmBm5mZZcgJ3MzMLENO4GZmZhlyAjczM8tQn0zgkgZUHYOZmVlvqjyBS/qWpK+Unn9H0pclnSlpnqS5kg5K63aVdHVp259IOiItN0v6rqSZwMdr3AwzM7OaqjyBA+cBhwFIegvwSeBxYBywJbAncKakkV2o6+mI2DoiLm2/QtJESU2SmlqXtPRY8GZmZlWoPIFHRDPwtKStgL2Au4GdgUsiojUingT+DGzbheqmLmc/kyNifESMHzC0oQciNzMzq069XIntHOAI4B0UPfL3d7Lda7z5S8fgdutf7PHIzMzM6lDlPfDkSmBvil72tcAtwEGSBkgaAUwA7gQeATaVNEjScGCPiuI1MzOrVF30wCPiFUk3AYsjolXSlcCOwGwggBMi4gkASZcB84CHKYbbzczM+p26SODp5LUdSGePR0QAx6fHm0TECcAJHZQ39m6UZmZm9aPyIXRJmwIPATdExINVx2NmZpaDynvgEXEPsEHVcZiZmeWk8gRehbGjGmjyfWbNzCxjlQ+hm5mZWfc5gZuZmWXICdzMzCxD/XIOfO6CFhonTas6DDOzyjT7PKDsuQduZmaWISdwMzOzDDmBm5mZZcgJ3MzMLEM1T+CSTpO0m6T9JZ3Ybt0mki6Q9BZJty+njumSxqflP0ganh6f7+34zczM6kEVPfDtgb8C7wNubrdul1Q2luKOYysUEftExGJgOOAEbmZm/ULN/o1M0pnAB4B3A7cDGwJ7SLoCuAk4G1gfeBIYBiyT1BQR4yUNAc4HtgTuA4aU6m0GxgOnAxtKmgVcHxH/ciczMzOzvqJmCTwijk/38j4M+E9gekS8t7TJuDRsvhNwHnBWRMxP644BlkTEeyRtAczsYBeTgM0jYlxH+5c0EZgIMOCtI3qiSWZmZpWp9RD61sBsYAxwb3mFpKHA0nQv8NHA/aXVE4CLACJiDjCnuzuOiMkRMT4ixg8Y2rCS4ZuZmdWHmvTAJY0DpgDrAYuAoUWxZgE7AlMpkvpwSXOARqBJ0mkRMbUWMZqZmeWkJgk8ImZRDJHfBuxMMUR+RroXOMCHJR0P/B14GtgnIk4oVXEz8CngRkmbA1t0sJvnKebOzczM+ryaDaFLGgE8GxHLgDGl5N1mAnArxZnof2637ufAmpLuBb4FzGhff0Q8DfxF0rx0wpyZmVmfVcuT2BYC+6blHTpYv19aPLWDdS8Bn+yk3sbS8qd6IlYzM7N65yuxmZmZZcgJ3MzMLEP98n7gY0c10OR74ZqZWcbcAzczM8uQE7iZmVmGnMDNzMwy1C/nwOcuaKFx0rSqwzAz6xHNPqenX3IP3MzMLENO4GZmZhlyAjczM8uQE7iZmVmG6iaBS3qh6hjMzMxyUTcJ3MzMzLqu7hK4Cmem24LOlXRQKr9U0r6l7aZIOlDSgLT9XZLmSPpsddGbmZnVRt0lcOAAYBywJbAncKakkcBU4BMAklYH9gCmAUcBLRGxLbAtcLSkd1cQt5mZWc3UYwLfGbgkIloj4kngzxSJ+Y/AbpIGAR8Ebk73Cd8LOEzSLOAOYC1gdPtKJU2U1CSpqXVJS42aYmZm1juyuRJbRLwsaTrwAeAg4NK0SsAXI+LaFbx+MjAZYNDI0dGLoZqZmfW6euyB3wIclOa2RwATgDvTuqnAkcAuwDWp7FrgGEmrAUjaWNIaNY7ZzMyspuqxB34lsCMwGwjghIh4Iq27DvgV8LuIeCWVnQM0AjMlCVgI7F/LgM3MzGqtbhJ4RKyZfgZwfHq03+ZV4O3typYBX0sPMzOzfqEeh9DNzMxsBZzAzczMMuQEbmZmlqG6mQOvpbGjGmg6fd8Vb2hmZlan3AM3MzPLkBO4mZlZhpzAzczMMtQv58DnLmihcdK0qsMwsz6s2efZWC9zD9zMzCxDTuBmZmYZcgI3MzPLkBO4mZlZhuoygUs6SdJ8SXMkzZK0vaRmSWtXHZuZmVk9qLuz0CXtCHwI2DoilqakvXrFYZmZmdWVeuyBjwQWRcRSgIhYFBH/SOu+KGmmpLmSxgBIWkPSeZLulHS3pI9UFbiZmVmt1GMCvw54p6QHJP1M0vtK6xZFxNbAz4HjUtlJwI0RsR2wG3CmpDXaVyppoqQmSU2tS1p6uw1mZma9qu4SeES8AGwDTAQWAlMlHZFW/yb9nAE0puW9gEmSZgHTgcHA+h3UOzkixkfE+AFDG3orfDMzs5qouzlwgIhopUjG0yXNBQ5Pq5amn628EbuAj0XE/TUN0szMrEJ11wOXtImk0aWiccAjy3nJtRRz40qv36oXwzMzM6sLdZfAgTWBCyTdI2kOsClwynK2PxVYDZgjaX56bmZm1qfV3RB6RMwAdupgVWNpmyZg17T8EvDZWsRmZmZWL+qxB25mZmYr4ARuZmaWobobQq+FsaMaaPK9es3MLGPugZuZmWXICdzMzCxDTuBmZmYZ6pdz4HMXtNA4aVrVYZhZH9Psc2ushtwDNzMzy5ATuJmZWYacwM3MzDLkBG5mZpYhJ3AzM7MMOYGbmZllqO4TuKRGSfdJmiLpAUkXS9pT0l8kPShpu/RzRNr+LZIeantuZmbWF9V9Ak82Ar4HjEmPTwE7A8cBXwMuAg5J2+4JzI6IheUKJE2U1CSpqXVJS80CNzMz6w25JPCHI2JuRCwD5gM3REQAcynuE34ecFja9tPA+e0riIjJETE+IsYPGNpQo7DNzMx6Ry4JfGlpeVnp+TJgYEQ8BjwpaXdgO+CPNY7PzMyspnJJ4F1xDsVQ+uUR0Vp1MGZmZr2pLyXwq4A16WD43MzMrK+p+5uZREQzsHnp+RGdrNuS4uS1+2oYnpmZWSXqPoF3haRJwDG8cSa6mZlZn9YnhtAj4vSIeFdE3Fp1LGZmZrXQJ3rg3TV2VANNvm+vmZllrE/0wM3MzPobJ3AzM7MMOYGbmZllqF/Ogc9d0ELjpGlVh2FmfUCzz6exirgHbmZmliEncDMzsww5gZuZmWXICdzMzCxDdZvAJTVKmld1HGZmZvWobhP4qpDUL8+uNzOz/qPeE/gASb+UNF/SdZKGSNpQ0jWSZki6RdIYAElTJP1C0h3AGRXHbWZm1qvqvac6Gjg4Io6WdBnwMeBI4HMR8aCk7YGfAbun7dcDdoqI1mrCNTMzq416T+APR8SstDwDaAR2Ai6X1LbNoNL2l3eWvCVNBCYCDHjriN6I1czMrGbqPYEvLS23AusAiyNiXCfbv9hZRRExGZgMMGjk6OipAM3MzKpQ73Pg7T0HPCzp4wAqbFlxTGZmZjWXWwIHOAQ4StJsYD7wkYrjMTMzq7m6HUKPiGZg89Lzs0qr9+5g+yN6PyozM7P6kGMP3MzMrN9zAjczM8uQE7iZmVmG6nYOvDeNHdVA0+n7Vh2GmZnZSnMP3MzMLENO4GZmZhlyAjczM8tQv5wDn7ughcZJ06oOw8wy1OzzZ6xOuAduZmaWISdwMzOzDDmBm5mZZcgJ3MzMLEN1lcAlnSZpN0n7Szqx3bpNJF0g6S2Sbq8qRjMzs3pQVwkc2B74K/A+4OZ263ZJZWOBeTWOy8zMrK7Uxb+RSToT+ADwbuB2YENgD0lXADcBZwPrA08Cw4BlkpoiYrykRuBXwBqpuv+IiNtq3AQzM7OaqosEHhHHS7oMOAz4T2B6RLy3tMm4NGy+E3AecFZEzE/rngLeHxEvSxoNXAKMb78PSROBiQAD3jqi9xpjZmZWA3WRwJOtgdnAGODe8gpJQ4GlEREpSd9fWr0a8BNJ44BWYOOOKo+IycBkgEEjR0ePR29mZlZDlSfwlHinAOsBi4ChRbFmATsCUymS+nBJc4BGoEnSaRExFTiWYmh9S4o5/Zdr2wIzM7PaqzyBR8QsiiHy24CdKYbIz4iIe9ImH5Z0PPB34Glgn4g4oVRFA/B4RCyTdDgwoHbRm5mZVaMuzkKXNAJ4NiKWAWNKybvNBOBWijPR/9xu3c+AwyW1Db+/2NvxmpmZVa3yHjhARCwE9k3LO3Swfr+0eGoH6x4EtigV/VdvxGhmZlZP6qIHbmZmZt3jBG5mZpahuhhCr7Wxoxpo8j19zcwsY+6Bm5mZZcgJ3MzMLENO4GZmZhnql3Pgcxe00DhpWtVhmFmGmn3+jNUJ98DNzMwy5ARuZmaWISdwMzOzDDmBm5mZZajmCVxSo6R5vVDvOEn79HS9ZmZm9ahP9MAlDQTGAU7gZmbWL1T1b2QDJP0S2AlYAHwEWBf4KTACWAIcHRH3SdoP+G9gdYr7gR8SEU9KOgXYENgAeBR4LzBE0s7AaRExtcZtMjMzq5mqeuCjgZ9GxGbAYuBjwGTgixGxDXAcxX2+obgP+A4RsRVwKXBCqZ5NgT0j4mDgZGBqRIzrKHlLmiipSVJT65KW3mqXmZlZTVTVA384Imal5RlAI0Vv/HJJbdsMSj/XA6ZKGknRC3+4VM9VEfFSV3YYEZMpviQwaOToWJXgzczMqlZVAl9aWm4F1gEWR8S4DrY9G/h+RFwlaVfglNK6F3spPjMzs7pWLyexPQc8LOnjACpsmdY1UMyTAxy+nDqeB4b1XohmZmb1o14SOMAhwFGSZgPzKU5sg6LHfbmkGcCi5bz+JmBTSbMkHdSrkZqZmVWs5kPoEdEMbF56flZp9d4dbP874HcdlJ/S7vkzwLY9FaeZmVk9q6ceuJmZmXWRE7iZmVmG+uX9wMeOaqDJ9/Q1M7OMuQduZmaWISdwMzOzDDmBm5mZZcgJ3MzMLEP98iS2uQtaaJw0reowzKzGmn3yqvUh7oGbmZllyAnczMwsQ07gZmZmGcoigUtqlDSvg/JvSdqzipjMzMyqlPVJbBFxctUxmJmZVSGLHngyQNIvJc2XdJ2kIZKmSDoQQNLpku6RNEfSWSuqzMzMLGc59cBHAwdHxNGSLgM+1rZC0lrAR4ExERGShlcUo5mZWU3k1AN/OCJmpeUZQGNpXQvwMnCupAOAJe1fLGmipCZJTa1LWno7VjMzs16VUwJfWlpupTR6EBGvAdsBVwAfAq5p/+KImBwR4yNi/IChDb0dq5mZWa/KaQi9U5LWBIZGxB8k/QX4e9UxmZmZ9aY+kcCBYcDvJA0GBPxnxfGYmZn1qiwSeEQ0A5uXnnd0lvl2NQvIzMysYjnNgZuZmVniBG5mZpYhJ3AzM7MMZTEH3tPGjmqgyfcFNjOzjLkHbmZmliEncDMzsww5gZuZmWXICdzMzCxDTuBmZmYZcgI3MzPLkBO4mZlZhpzAzczMMuQEbmZmliFFRNUx1Jyk54H7q46jh60NLKo6iB7W19rU19oDblMu3Kb6tzawRkSM6OoL+uWlVIH7I2J81UH0JElNblN962vtAbcpF25T/UvtaezOazyEbmZmliEncDMzswz11wQ+ueoAeoHbVP/6WnvAbcqF21T/ut2efnkSm5mZWe76aw/czMwsa07gZmZmGep3CVzS3pLul/SQpElVx7MyJJ0n6SlJ80plb5d0vaQH08+3VRljd0h6p6SbJN0jab6kL6fynNs0WNKdkmanNn0zlb9b0h3p8zdV0upVx9odkgZIulvS1el57u1pljRX0ixJTaks288dgKThkq6QdJ+keyXtmHObJG2S3p+2x3OSvpJzmwAkHZv+NsyTdEn6m9Gt36d+lcAlDQB+CnwQ2BQ4WNKm1Ua1UqYAe7crmwTcEBGjgRvS81y8Bnw1IjYFdgC+kN6XnNu0FNg9IrYExgF7S9oB+C7wg4jYCHgWOKq6EFfKl4F7S89zbw/AbhExrvQ/xTl/7gB+BFwTEWOALSner2zbFBH3p/dnHLANsAS4kozbJGkU8CVgfERsDgwAPkl3f58iot88gB2Ba0vPTwROrDqulWxLIzCv9Px+YGRaHklxsZrK41zJtv0OeH9faRMwFJgJbE9x5aiBqfxNn8d6fwDrUfyh3B24GlDO7UkxNwNrtyvL9nMHNAAPk05Q7gttateOvYC/5N4mYBTwGPB2iguqXQ18oLu/T/2qB84bB63N46msL1gnIv6Zlp8A1qkymJUlqRHYCriDzNuUhptnAU8B1wN/AxZHxGtpk9w+fz8ETgCWpedrkXd7AAK4TtIMSRNTWc6fu3cDC4Hz01THOZLWIO82lX0SuCQtZ9umiFgAnAU8CvwTaAFm0M3fp/6WwPuFKL6+Zff/gZLWBP4P+EpEPFdel2ObIqI1imG/9YDtgDHVRrTyJH0IeCoiZlQdSw/bOSK2pphW+4KkCeWVGX7uBgJbAz+PiK2AF2k3tJxhmwBI88EfBi5vvy63NqX5+o9QfOFaF1iDf50WXaH+lsAXAO8sPV8vlfUFT0oaCZB+PlVxPN0iaTWK5H1xRPwmFWfdpjYRsRi4iWJIbLiktnsQ5PT5ey/wYUnNwKUUw+g/It/2AK/3hIiIpyjmVbcj78/d48DjEXFHen4FRULPuU1tPgjMjIgn0/Oc27Qn8HBELIyIV4HfUPyOdev3qb8l8LuA0elMv9UphmOuqjimnnIVcHhaPpxiHjkLkgScC9wbEd8vrcq5TSMkDU/LQyjm9O+lSOQHps2yaVNEnBgR60Vxs4VPAjdGxCFk2h4ASWtIGta2TDG/Oo+MP3cR8QTwmKRNUtEewD1k3KaSg3lj+BzybtOjwA6Shqa/f23vU/d+n6qezK/g5IF9gAco5iNPqjqelWzDJRTzJq9SfOM+imI+8gbgQeBPwNurjrMb7dmZYvhrDjArPfbJvE1bAHenNs0DTk7lGwB3Ag9RDAUOqjrWlWjbrsDVubcnxT47Pea3/T3I+XOX4h8HNKXP3m+Bt/WBNq0BPA00lMpyb9M3gfvS34dfAYO6+/vkS6mamZllqL8NoZuZmfUJTuBmZmYZcgI3MzPLkBO4mZlZhpzAzczMMuQEblYDklrb3VGpseqYekL6H1YknVJ+Xlq/rqQrOnntdEnj0/LH052zbkrPt5J0blreVdJOXYhliqQDV7RdF+o5S9Luq1qPWW9zAjerjZci3VEpPZrbVqiQ6+/idyR9BFhL0o8p7n71uoj4R0R0JakeBRwdEbul518DfpyWdwVWmMB70NlkdGcr679y/aNhljVJjSruS38hxYUc3inpeEl3SZqjdP/wtO1Jkh6QdGu6b/Bxqbzcg107Xea07SYqZ5bq+mwq3zW9pu1e0ReXetDbSrpNxf3L75Q0TNLNksaV4rhVUvsE/TWKazgfCvw0ImZ10M55aXmIpEtTT/tKYEgqP5niYj7npriHAVtExOw0UvE54Ng0crFLqvPG1LYbJK3fwfE9NfXIB3R0XFMd90r6pYp7Ml+XrphHRDxC8YXkHd1/Z81qxwncrDaGlIbPr0xlo4GfRcRmwCbp+XYUV9LaRtIESdtQXLp0HMXV6bbtwr6OAloiYtu0/dGS3p3WbQV8BdiU4qpP702XFZ4KfDmK+5fvCbxEcXnbIwAkbQwMjojZ5R1J+jZwDXARxc1A3pTg2zkGWBIR7wG+QXFvZyLiWxRXDjskIo4HxlN8qSGNVPyC4h7J4yLiFooe8gURsQVwMW/01NtiOhMYARxJcYnKfzmuadPRFF86NgMWAx8rVTOT4trUZnVr4Io3MbMe8FIUdyYDXr9t6iMR8ddUtFd63J2er0mRYIYBV0bEkvS6rly7fy9gi9J8cEOq6xXgzoh4PNU1i+K+8i3APyPiLoBId4KTdDnwdUnHA58GpnSwr69HREjaKiJOaT8H3s4EUrKNiDmS5nSy3UiKW2J2ZkfggLT8K+CMcjzAHRExMbWhs+P6KMXNJGal8hkUx6LNUxR3iTKrW07gZtV5sbQs4LSI+N/yBpK+spzXv8Ybo2iD29X1xYi4tl1duwJLS0WtLOdvQEQskXQ9xW0PP0HqMbfbJtLPU8rPV9FLvLk93XEXRS/77RHxDJ0f10b+9VgMKT0fnOIwq1seQjerD9cCn1ZxT3QkjZL0b8DNwP5p/ngYsF/pNc28kVQPbFfXMSpu0YqkjVXcbasz9wMjJW2bth+mN25peA5Fr/muiHh2lVpYtOVTaR+bU9zwpSP3AhuVnj9PMRLR5jaKaQWAQ4BbSuuuAU4HpqXj1dlxXZGNScP4ZvXKPXCzOhAR10l6D3B7GoV+ATg0ImZKmkpxx6ynKHqYbc4CLpM0EZhWKj+HYjh4ZhrSXgjsv5x9vyLpIODsdCLXSxTz4C9ExAxJzwHn90Azfw6cL+leiiQ9o5N47pPUIGlYRDwP/B64QsXZ7l9Mj/PT0P5Cirnu8usvT8n7KorzBn5Nu+NK0ePuUPrisxHFvLxZ3fLdyMwyouL/rV+IiLNqtL91genAmIhYVot9pv0eCzwfEefUap+lfX8U2Doivl7rfZt1h4fQzaxDkg4D7qC4T3bNknfyc948R11LA4HvVbRvsy5zD9zMzCxD7oGbmZllyAnczMwsQ07gZmZmGXICNzMzy5ATuJmZWYb+H9kFC/kr7EAcAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}},{"output_type":"display_data","data":{"text/plain":"<Figure size 504x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAfAAAAEYCAYAAACju6QJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAApdUlEQVR4nO3deZhcVZnH8e+PAFlI6KhBDWFphUBYEgIEBIQQBBdwARVZZXNhBAZGHRBcRoOMsrivMBGRXVAUhzGCIBACSIQOCVkIUYQgBkGC0CwhCMk7f5zTcimqu6uT7qq63b/P89TTt+45de97Ty1vnXNP11VEYGZmZuWyVqMDMDMzs55zAjczMyshJ3AzM7MScgI3MzMrISdwMzOzEnICNzMzKyEn8H5E0gxJH1vNx24i6VlJg3o5psMlXd+b2xxIlPxE0pOS7mx0PAb5ffLmRsdRSdLRkm5rdBxrqlnbtxk5gTcZSUskPZ9fxI9KulDS8D7azz4d9yPiLxExPCJW9uZ+IuKyiHhHb25zdZX0A2534O3ARhGxc6ODMcjvkwfqvV9JIWnzeu+3IoaaOwmSjswxd1q/2vYa1b5l5ATenN4bEcOBicD2wGcbG075SVq70TGspk2BJRHxXF/vqMRtVBdun9pJeg3wOWBho2Pp1yLCtya6AUuAfQr3zwGmF+7vAvweeAq4B5hSKJsBfCwvbwbcBDwBLAMuA0bmskuAVcDzwLPAZ4BWIIC1gYOBtoq4PgVck5cHA18H/gI8BpwHDO3keI4GbivcD+B44E/AM8AZOdbfA08DPwPWzXWnAH8lfRAsy21zeGFbLcDFwOPAQ8AXgLUK+70d+FZug18AK4CV+ZifyvXeDczJ+34YmFrYfkebHJWPdRnw+UL5oBzbn/OxzAY2zmXjgBuAfwCLgYO6eM43BK7Jde8HPp7Xf7Qi5tM7a9/8fDwJPAjsWyh/EzAzx/c74AfApRXH99F8fDPz+p8DjwLt+bHbFLZ3IfBD4Noc0+3AG4Fv5/3fB2xfqH8qsDTvfzGwdydtUMvzcCzwCPA34ORC+VTgKuDKvJ+7ge0q2vcX+XXyIHBSoWxn4A7S++lvwPfJr7/C6/UE0uv1wcK6zQvt8QNget73H4DNCo9/Rz7u9txut5Dfo1XaoNNY8vMQwHO53Q+u4b3W6Wuwm/YeAlxKet88BdwFvAH4Cum1uCLH8P0uXtPnkd7nM7o43qrbq9K+PXm9dfdct+Vjfgz4Zr0+1/vq1vAAfKt4QgoJHNgImA98J98fk99U+5FGT96e72+Qy//1ZgE2z+WDgQ3yB8C3q+0n32/l5QQ+jPRhNLZQfhdwSF7+FinhvBYYAfwfcGYnx1P5oRLA/wLrA9sALwA3Am8mJeR7gaNy3SnAS8A383HsSfoA2zKXX5y3NSLH/0fgo4X9vgScmI9paGUshX2Mz+05Ib+xD6hokx/lx2+X490ql5+Sn58tAeXy1wHrkT4Uj8n73p6U/LfupI1mkj6khpBGXR4H3lat/Tpp3xeBj5O+UBxHSnLK5XeQkvu6pOH4p3l1Ar84xzw0r/9IbtPBpA/KuYX9XZiPZccc702kD8oj8/7/G7g5190yt8OGhf1t1slx1PI8/DTHOT63Ucf7ZGpugwOBdYCTc0zr5O3NBr6Y2+DNwAPAO/NjdyR9KV4772cR8MmK1+sNpNf60MK6YoJ5gpQc1iZ9Ub4il43K7f2BXPYfOc7OElotsWzezWvhtrzc5Wuwm/b+N9J7elh+TncE1q/8jOkijo5EuVZ39auVV2nfWl9v3T3XdwBH5OXhwC6N+pzvrVvDA/Ct4glJifVZUgINUnIbmctOBS6pqP9bXk54nb5ZgAOAORX7qZrA8/1LgS/m5bE5nmGkRPUcr+xl7ErunVTZ778+VPL9AN5auD8bOLVw/xvkLxq8nMDXK5T/DPiv/Ob9J4WkmD94ZhT2+5euYukk3m8D36pok40K5Xfy8heZxcD+VbZxMHBrxbr/Ab5Upe7GpF7IiMK6M4ELa4k5l99fuD8sx/xGYJPcfsMK5Zfy6gT+5i62PzLXacn3LwR+VCg/EVhUuD+el0c3Ngf+DuwDrNPD90G152Fcofwc4Md5eSowq1C2FqkHuwfwliqvg88CP+lkv58Erq54vb6tok5lgjm/ULYfcF9ePhK4o1AmUlLtMgF2E0utCbzm12CV9v4IaURsQpV6M7qKn/S+bCMnxxrqv6q8SvvW+nrr8rkmfVE+HRjVk9diM998Drw5HRARI0gJbBzpmzyk86EfkvRUx43UqxpduQFJb5B0haSlkp4mfXCPqqzXhcuBQ/PyYcCvImI5qTc/DJhdiOG6vL5WjxWWn69yvzhp78l45fnfh0jDZKNIPayHKsrGFO4/3F0gkt4i6WZJj0tqBz7Bq9vp0cLy8kJ8G5OGzyttCryl4nk6nJRUK20I/CMinuniOLrzr/jyc0SOsWPbywt1q7XJv9ZJGiTpLEl/zq+bJbmo2CY1PX8RcT8pCU0F/p5fjxtWO4Aan4di7B2vg1eVRcQq0qmXDUnPxYYVz8XnSEPCSNpC0q/zhNGnga92s99qOnt9bFgRV+S4qqoxllp1+Rrspr0vIXUMrpD0iKRzJK1T436PB+ZFxKzVjLuaWj8vunyuSaeKtgDuk3SXpPf0YowN4QTexCLiFtI30K/nVQ+TeuAjC7f1IuKsKg//Kumb7PiIWB/4MKkH8K/Nd7P7G4ANJE0kJfLL8/plpDfNNoUYWiJNuusLr5G0XuH+JqQh4mWk4chNK8qWFu5XHmO1Y76cdDpg44hoIZ27U5V61TxMOn9fbf0tFc/T8Ig4rkrdR4DXShrRxXGsrr/lbQ8rrNu4Sr1iuxwG7E/qNbeQer9Qe5u8csMRl0fE7qTnKYCzO6lay/NQjL3jdfCqMklrkU4/PUJ6Lh6seC5GRMR+ufq5pPOoY/P75HNV9tvde6Uzf8txdMSl4v0qaomlVt29Bjtt74h4MSJOj4itgd2A95BGE6D7ttgbeH/+EvJofvw3JH2/k/qr27bVdPlcR8SfIuJQ4PWk1+FVFZ8tpeME3vy+Dbxd0nakXvR7Jb0z95SGSJoiqdqHwgjSUHy7pDGk87VFj5HOEVUVES+SJjN9jXT+74a8fhXpnPC3JL0eQNIYSe9ck4PsxumS1pW0B+nD5OeR/t3tZ8BXJI2QtCnwaVIbdeYxYCNJ6xbWjSD1UldI2pmUwGp1PnCGpLH5/7UnSHod8GtgC0lHSFon33aStFXlBiLiYdJw5Zn5+ZxA6il0dRw1iYiHSMOZU3P77Qq8t5uHjSCd53+CNNLy1dXdv6QtJb1N0mDSRKXnSZMnO9tvd8/Df0kaJmkb0rndKwtlO0r6QJ4p/sl8DLNIpzyekXSqpKH5fbOtpJ0K+30aeFbSONIcgt4yHRgv6YAc1wlUH4Xp0F0sXb5nK3T3Guy0vSXtJWm80m9CPE36otzxvHUXw9HAVqS5HBNJr7/Tgc93Ur8nx9SdLp9rSR+WtEH+DHsqP6az12MpOIE3uYh4nDTJ6Iv5w35/0jfzx0nfOE+h+vN4OrADafbrdOCXFeVnAl/IQ00nd7L7y0k9sZ9HxEuF9aeSZkvPykN9vyNNWOoLj5Jmmz5CmiD0iYi4L5edSDof/wBpJvblwAVdbOsm0r+1PCppWV53PPBlSc+QJr/8rAexfTPXv570Qfdj0kSnZ0izjw/JcT9K+sY/uJPtHErq6T4CXE06T/m7HsTRlcNJcxSeIE34uZKU3DpzMWl4eilpQuGaDIUOBs4ijZY8Sur5dPYvkbU8D7eQXnc3Al+PiOIPBP0v6bzvk8ARwAdyT3Il6UvfRNLkp2WkL14t+XEnk5LXM6QvpsUvBWskIpYBHyKdr38C2JqU0Dpr/+5imQpclN+zB3Wz7+5eg1219xtJs/qfJk2ku4U0rA7wHeBApR8W+m6V/T4VEY923EjzVJ6OiPZOQu1yez1Rw3P9LmChpGfzfg+JiOfhXz8es8ea7L8ROmaqmjUdSVNIE666Gna0HpB0JWmS1ZcaHUutJLWSZ5VXfJHsKJ9KmvT04TqH1iN5aP+vpH+FvLnR8Vj5uQdu1o/lYdPNJK0l6V2kEZxfNTisASOf7hqZTyN0nNPuzQleNoD5l4XM+rc3kk6fvI7U+zsuIuY0NqQBZVfSqZ11SackDugYtjVbUx5CNzMzKyEPoZuZmZXQgBxCHzVqVLS2tjY6DDMzs27Nnj17WUS86seyBmQCb21tpa2trdFhmJmZdUvSQ9XWewjdzMyshJzAzczMSsgJ3MzMrIScwM3MzErICdzMzKyEnMDNzMxKyAnczMyshJzAzczMSmhA/pDL/KXttJ42vdFhmJlZP7PkrHfXbV/ugZuZmZWQE7iZmVkJOYGbmZmVkBO4mZlZCfV5Apd0gKSQNK6T8hmSJnVSdr6krfs2QjMzs/KpRw/8UOC2/LdmkgZFxMci4t6+CcvMzKy8+jSBSxoO7A58FDgkrxsq6QpJiyRdDQwt1H9W0jck3QPsWuydSzpXUpukhZJOLzxmiaTTJd0taX5nPX0zM7P+pK//D3x/4LqI+KOkJyTtCOwJLI+IrSRNAO4u1F8P+ENE/CeApOK2Ph8R/5A0CLhR0oSImJfLlkXEDpKOB04GPtbHx2VmZtZQfT2EfihwRV6+It+fDFwKkBPwvEL9lcAvOtnWQZLuBuYA2wDFc+O/zH9nA63VHizp2NyDb1u5vL3nR2JmZtZE+qwHLum1wNuA8ZICGAQEKQF3ZkVErKyyrTeRetY7RcSTki4EhhSqvJD/rqSTY4qIacA0gMGjx0bPjsbMzKy59GUP/EDgkojYNCJaI2Jj4EFSL/kwAEnbAhNq2Nb6wHNAu6Q3APv2UcxmZmal0JfnwA8Fzq5Y9wtge2CopEXAIlJC71JE3CNpDnAf8DBwey/HamZmVip9lsAjYq8q677bzWOGV9yfUlg+upPHtBaW24Ap1eqZmZn1J/4lNjMzsxJyAjczMyshJ3AzM7MS6usfcmlK48e00FbHi66bmZn1NvfAzczMSsgJ3MzMrIScwM3MzEpoQJ4Dn7+0ndbTpjc6DDMz62eW1HF+lXvgZmZmJeQEbmZmVkJO4GZmZiXkBG5mZlZCfZrAJT3bTXmrpAV9GYOZmVl/1C974JIG5Ox6MzMbOOqSwCUNl3SjpLslzZe0f6F4bUmXSVok6SpJw/Jj9pY0J9e/QNLgvH6JpFF5eZKkGXl5qqRLJN0OXFKP4zIzM2uUevXAVwDvj4gdgL2Ab0hSLtsS+GFEbAU8DRwvaQhwIXBwRIwn/b/6cTXsZ2tgn4g4tLJA0rGS2iS1rVzevuZHZGZm1kD1SuACvippHvA7YAzwhlz2cETcnpcvBXYnJfUHI+KPef1FwOQa9nNNRDxfrSAipkXEpIiYNGhYy+oeh5mZWVOo17niw4ENgB0j4kVJS4AhuSwq6lber/QSL3/xGFJR9tyaBGlmZlYW9eqBtwB/z8l7L2DTQtkmknbNy4cBtwGLgVZJm+f1RwC35OUlwI55+YN9GrWZmVmTqlcCvwyYJGk+cCRwX6FsMXCCpEXAa4BzI2IFcAzw8/yYVcB5uf7pwHcktQEr6xS/mZlZU+nTIfSIGJ7/LgN27aTauE4eeyOwfZX1twJbVFk/dbUDNTMzK5l++X/gZmZm/Z0TuJmZWQkNyF8sGz+mhbY6XrPVzMyst7kHbmZmVkJO4GZmZiXkBG5mZlZCA/Ic+Pyl7bSeNr3RYZiZWT+zpI7zq9wDNzMzKyEncDMzsxJyAjczMyshJ3AzM7MSalgCl/RGSVdI+rOk2ZJ+I2kLSQsaFZOZmVlZNGQWuiQBVwMXRcQhed12wBt6uJ21I+KlPgjRzMysqTWqB74X8GJEdFwilIi4B3i4476kVkm3Sro733bL66fk9dcA9+Z1v8q9+IWSjq3zsZiZmdVdo/4PfFtgdjd1/g68PSJWSBoL/BSYlMt2ALaNiAfz/Y9ExD8kDQXukvSLiHiiuLGc2I8FGLT+Br11HGZmZg3RzD/ksg7wfUkTgZW88hrgdxaSN8BJkt6flzcGxgKvSOARMQ2YBjB49Njoq6DNzMzqoVEJfCFwYDd1PgU8BmxHGupfUSh7rmNB0hRgH2DXiFguaQYwpBdjNTMzazqNOgd+EzC4eL5a0gRS77lDC/C3iFgFHAEM6mRbLcCTOXmPA3bpo5jNzMyaRkMSeEQE8H5gn/xvZAuBM4FHC9V+CBwl6R5gHIVed4XrgLUlLQLOAmb1XeRmZmbNoWHnwCPiEeCgKkXb5vI/ARMK60/N62cAMwrbeQHYt6/iNDMza0b+JTYzM7MScgI3MzMroWb+N7I+M35MC211vGarmZlZb3MP3MzMrIScwM3MzErICdzMzKyEBuQ58PlL22k9bXqjwzAzs35mSR3nV7kHbmZmVkJO4GZmZiXkBG5mZlZCTuBmZmYl1NQJXNKGkq5qdBxmZmbNpqlnoecLnnR33XAzM7MBp1d64JJaJd0n6TJJiyRdJWmYpC9KukvSAknTJCnXP0nSvZLmSboir9tT0tx8myNpRN7uglw+RNJPJM3P5Xvl9UdL+qWk6yT9SdI5vXFMZmZmzaw3h9C3BH4YEVsBTwPHA9+PiJ0iYltgKPCeXPc0YPuImAB8Iq87GTghIiYCewDPV2z/BNKlxMcDhwIXSRqSyyYCBwPjgYMlbdyLx2VmZtZ0ejOBPxwRt+flS4Hdgb0k/UHSfOBtwDa5fB5wmaQPAy/ldbcD35R0EjAyIl7ilXbP2yUi7gMeArbIZTdGRHtErADuBTatDE7SsZLaJLWtXN7eG8drZmbWML2ZwKPK/R8CB+Ze84+Ajh7zu4EfADsAd0laOyLOAj5G6qnfLmlcD/b9QmF5JVXO7UfEtIiYFBGTBg1r6cGmzczMmk9vJvBNJO2alw8DbsvLyyQNJ09Gk7QWsHFE3AycCrQAwyVtFhHzI+Js4C6gMoHfChyet7EFsAmwuBfjNzMzK43enIW+GDhB0gWkYexzgdcAC4BHSUkZYBBwqaQWQMB3I+IpSWfkiWmrgIXAtcDowvZ/CJybh+NfAo6OiBfyvDgzM7MBRRGVI9+rsRGpFfh1nqzW9AaPHhujj/p2o8MwM7N+pi8uZiJpdkRMqlzf1D/kYmZmZtX1yhB6RCwBStH7NjMz6w/cAzczMyuhpv4p1b4yfkwLbXW86LqZmVlvcw/czMyshJzAzczMSsgJ3MzMrIQG5Dnw+UvbaT1teqPDMDOzfqYv/g+8M+6Bm5mZlZATuJmZWQk5gZuZmZWQE7iZmVkJNTyBS1opaW7h1tpF3QsldVyW9HxJW9ctUDMzsybSDLPQn4+IiT19UER8rA9iMTMzK4WG98CrkbSjpFskzZb0W0mjq9SZIWmSpE0l/UnSKElrSbpV0jsaEbeZmVm9NEMPfKikuXn5QeAg4HvA/hHxuKSDga8AH6n24Ih4SNLZwLnAncC9EXF9ZT1JxwLHAgxaf4NePwgzM7N6aoYE/oohdEnbki5NeoMkgEHA37raQEScL+lDwCeAiZ3UmQZMAxg8emz0RuBmZmaN0gwJvJKAhRGxa80PkIYBG+W7w4Fn+iIwMzOzZtGM58AXAxtI2hVA0jqStunmMWcDlwFfBH7Ux/GZmZk1XNMl8Ij4J3AgcLake4C5wG6d1Ze0J7ATcHZEXAb8U9Ix9YjVzMysURo+hB4Rw6usmwtMrrL+6MLylELRLoX1H+jVAM3MzJpQ0/XAzczMrHtO4GZmZiXU8CH0Rhg/poW2Ol6z1czMrLe5B25mZlZCTuBmZmYl5ARuZmZWQgPyHPj8pe20nja90WGYmVmTWVKi+VHugZuZmZWQE7iZmVkJOYGbmZmVkBO4mZlZCXWbwCWtlDRX0gJJ/ydpZB3iWm2Slkga1eg4zMzM+lItPfDnI2JiRGwL/AM4oY9jMjMzs270dAj9DmAMgKSdJd0haY6k30vaMq+fKWlixwMk3SZpO0lTJV0k6VZJD0n6gKRzJM2XdJ2kdXL9vfM250u6QNLgvH6JpNMl3Z3LxuX1r5N0vaSFks4HtObNYmZm1txqTuCSBgF7A9fkVfcBe0TE9sAXga/m9T8Gjs6P2QIYEhH35LLNgLcB7wMuBW6OiPHA88C7JQ0BLgQOzuvXBo4rhLEsInYAzgVOzuu+BNwWEdsAVwObdBL/sZLaJLWtXN5e62GbmZk1pVoS+FBJc4FHgTcAN+T1LcDPJS0AvgVsk9f/HHhP7lF/hJSQO1wbES8C84FBwHV5/XygFdgSeDAi/pjXX8Qrrwv+y/x3dq5PLr8UICKmA09WO4iImBYRkyJi0qBhLTUctpmZWfOq+Rw4sClpeLrjHPgZpB70tsB7gSEAEbGclOT3Bw4CLits64VcZxXwYkREXr+K2n4V7oX8d2WN9c3MzPqlmofQc2I+CfhPSWuTeuBLc/HRFdXPB74L3BURVXvEnVgMtEraPN8/Arilm8fMBA4DkLQv8Joe7M/MzKyUejSJLSLmAPOAQ4FzgDMlzaGiNxwRs4GngZ/0cPsrgGNIQ/PzST3z87p52OnAZEkLgQ8Af+nJPs3MzMpIL49i9+JGpQ2BGcC4PFzeVAaPHhujj/p2o8MwM7Mm04wXM5E0OyImVa7v9V9ik3Qk8Afg882YvM3MzPqDXp8IFhEXAxf39nbNzMzsZQNyJvf4MS20NeEwiZmZWa18MRMzM7MScgI3MzMrISdwMzOzEhqQ58DnL22n9bTpjQ7DzMx6oBn/xauR3AM3MzMrISdwMzOzEnICNzMzKyEncDMzsxKqewKXdKakvSQdIOmzFWVbSrpI0lqS7uhiGzMkTcrLv5E0Mt+O7+v4zczMmkEjeuBvAWYBe5IuBVq0R143HlhQy8YiYr+IeAoYCTiBm5nZgFC3fyOT9DXgncCbgDuAzYC9JV0F3Ax8D9gEeAwYAayS1BYRkyQNJV2adDvgPmBoYbtLgEnAWcBmkuYCN0TEKXU6NDMzs7qrWwKPiFMk/Qw4Evg0MCMi3lqoMjEPm+8GXAB8PSIW5rLjgOURsZWkCcDdVXZxGrBtREzss4MwMzNrEvUeQt8BuAcYBywqFkgaBrwQ6QLlY4HFheLJwKUAETEPmNfTHUs6VlKbpLaVy9tXM3wzM7PmUJceuKSJwIXARsAyYFharbnArsCVpKQ+UtI8oBVok3RmRFzZGzFExDRgGsDg0WOjN7ZpZmbWKHVJ4BExlzRE/ntgd9IQ+TkRcW+u8j5JpwAPAE8A+0XEZwqbmAkcBtwkaVtgQpXdPEM6d25mZtbv1W0IXdIGwJMRsQoYV0jeHSYDt5Fmot9SUXYuMFzSIuDLwOzK7UfEE8DtkhbkCXNmZmb9Vj0nsT0OvDsv71Kl/L158YwqZc8Dh3Sy3dbC8mG9EauZmVmz8y+xmZmZlZATuJmZWQk5gZuZmZVQ3c6BN5PxY1po84XhzcysxNwDNzMzKyEncDMzsxJyAjczMyuhAXkOfP7SdlpPm97oMMzMrAeWeO7SK7gHbmZmVkJO4GZmZiXkBG5mZlZCTuBmZmYl1BQJXNKZkvaSdICkz1aUbSnpIklrSbqjsH6SpO/WP1ozM7PGa4oEDrwFmAXsSbr2d9Eeed14YEHHyohoi4iT6hahmZlZE2nov5Hl63a/E3gTcAewGbC3pKuAm4HvAZsAjwEjgFWS2iJikqQpwMkR8R5JewLfyZsNYHJEPFPXgzEzM6ujhibwiDhF0s+AI4FPAzMi4q2FKhPzsPluwAXA1yNiYZVNnQycEBG3SxoOrKisIOlY4FiAQetv0MtHYmZmVl/NMIS+A3APMA5YVCyQNAx4ISICGAss7mQbtwPflHQSMDIiXqqsEBHTImJSREwaNKylVw/AzMys3hrWA5c0EbgQ2AhYBgxLqzUX2BW4kpTUR0qaB7QCbZLOjIgri9uKiLMkTQf2A26X9M6IuK9Oh2JmZlZ3DUvgETGXNET+e2B30hD5ORFxb67yPkmnAA8ATwD7RcRnqm1L0mYRMR+YL2knUuJ3Ajczs36roUPokjYAnoyIVcC4QvLuMBm4jTQT/ZYuNvVJSQtyT/1F4No+CdjMzKxJNHoS2+PAu/PyLlXK35sXz6hSNgOYkZdP7LMgzczMmlAzTGIzMzOzHnICNzMzK6EBeT3w8WNaaPN1Zc3MrMTcAzczMyshJ3AzM7MScgI3MzMroQF5Dnz+0nZaT5ve6DDMzAxY4jlJq8U9cDMzsxJyAjczMyshJ3AzM7MScgI3MzMroaZI4JKe7aXttEpa0BvbMjMza2ZNkcBXl6QBOYvezMysaROgpInAecAw4M/ARyLiSUkzgLmka4j/NN+/ID/s+roHamZm1gDN3AO/GDg1IiYA84EvFcrWjYhJEfEN4CfAiRGxXVcbk3SspDZJbSuXt/dd1GZmZnXQlAlcUgswMiJuyasuAiYXqlyZ643M9Wbm9Zd0ts2ImJaT/qRBw1r6IGozM7P6acoEXoPnGh2AmZlZIzVlAo+IduBJSXvkVUcAt1Sp9xTwlKTd86rD6xOhmZlZYzXLJLZhkv5auP9N4CjgPEnDgAeAYzp57DHABZICT2IzM7MBoikSeER0NhKwS5W6UyruzwaKE9g+03uRmZmZNaemHEI3MzOzrjmBm5mZlVBTDKHX2/gxLbT5+rNmZlZi7oGbmZmVkBO4mZlZCTmBm5mZlZATuJmZWQkNyEls85e203ra9EaHYWZmwBJPKl4t7oGbmZmVkBO4mZlZCTmBm5mZlVDdErikkyQtknRZJ+VTJP06Lx8t6fv1is3MzKxs6jmJ7Xhgn4j4a7c1zczMrEt16YFLOg94M3CtpM9LukDSnZLmSNq/m8e2SrpJ0jxJN0raRNIgSQ8qGSlppaTJuf5MSWPrcVxmZmaNUpcEHhGfAB4B9gLWA26KiJ3z/a9JWq+Lh38PuCgiJgCXAd+NiJXAYmBrYHfgbmAPSYOBjSPiT313NGZmZo3XiEls7wBOkzQXmAEMATbpov6uwOV5+RJSwga4FZicb2fm9TsBd1XbiKRjJbVJalu5vH0ND8HMzKyxGpHABXwwIibm2yYRsWg1tjMT2APYGfgNMBKYQkrsrxIR0yJiUkRMGjSsZfUiNzMzaxKNSOC/BU6UJABJ23dT//fAIXn5cF5O0HcCuwGrImIFMBf4N1JiNzMz69cakcDPANYB5klamO935UTgGEnzgCOA/wCIiBeAh4FZud6twAhgfl8EbWZm1kwUEY2Ooe4Gjx4bo4/6dqPDMDMz/Fvo3ZE0OyImVa73L7GZmZmVkBO4mZlZCTmBm5mZldCAvB74+DEttPmci5mZlZh74GZmZiXkBG5mZlZCTuBmZmYl5ARuZmZWQk7gZmZmJeQEbmZmVkJO4GZmZiXkBG5mZlZCTuBmZmYlNCCvRibpGWBxo+PoB0YByxodRMm5DXuH27F3uB3XXF+04aYRsUHlygH5U6rA4mqXZrOekdTmdlwzbsPe4XbsHW7HNVfPNvQQupmZWQk5gZuZmZXQQE3g0xodQD/hdlxzbsPe4XbsHW7HNVe3NhyQk9jMzMzKbqD2wM3MzErNCdzMzKyE+nUCl/QuSYsl3S/ptCrlgyVdmcv/IKm1AWE2tRra8NOS7pU0T9KNkjZtRJzNrrt2LNT7oKSQ5H/lqaKWdpR0UH5NLpR0eb1jbHY1vKc3kXSzpDn5fb1fI+JsZpIukPR3SQs6KZek7+Y2nidphz4JJCL65Q0YBPwZeDOwLnAPsHVFneOB8/LyIcCVjY67mW41tuFewLC8fJzbcPXaMdcbAcwEZgGTGh13s91qfD2OBeYAr8n3X9/ouJvpVmMbTgOOy8tbA0saHXez3YDJwA7Agk7K9wOuBQTsAvyhL+Lozz3wnYH7I+KBiPgncAWwf0Wd/YGL8vJVwN6SVMcYm123bRgRN0fE8nx3FrBRnWMsg1peiwBnAGcDK+oZXInU0o4fB34QEU8CRMTf6xxjs6ulDQNYPy+3AI/UMb5SiIiZwD+6qLI/cHEks4CRkkb3dhz9OYGPAR4u3P9rXle1TkS8BLQDr6tLdOVQSxsWfZT0rdNeqdt2zENsG0fE9HoGVjK1vB63ALaQdLukWZLeVbfoyqGWNpwKfFjSX4HfACfWJ7R+paefnatloP6UqvUySR8GJgF7NjqWspG0FvBN4OgGh9IfrE0aRp9CGg2aKWl8RDzVyKBK5lDgwoj4hqRdgUskbRsRqxodmL1Sf+6BLwU2LtzfKK+rWkfS2qThoifqEl051NKGSNoH+Dzwvoh4oU6xlUl37TgC2BaYIWkJ6ZzZNZ7I9iq1vB7/ClwTES9GxIPAH0kJ3ZJa2vCjwM8AIuIOYAjpAh1Wu5o+O9dUf07gdwFjJb1J0rqkSWrXVNS5BjgqLx8I3BR5BoIBNbShpO2B/yElb59vrK7LdoyI9ogYFRGtEdFKmkvwvohoa0y4TauW9/SvSL1vJI0iDak/UMcYm10tbfgXYG8ASVuREvjjdY2y/K4Bjsyz0XcB2iPib729k347hB4RL0n6d+C3pJmXF0TEQklfBtoi4hrgx6ThoftJExIOaVzEzafGNvwaMBz4eZ7/95eIeF/Dgm5CNbajdaPGdvwt8A5J9wIrgVMiwqNqWY1t+J/AjyR9ijSh7Wh3bF5J0k9JXxRH5bkCXwLWAYiI80hzB/YD7geWA8f0SRx+XszMzMqnPw+hm5mZ9VtO4GZmZiXkBG5mZlZCTuBmZmYl5ARuZmZWQk7gZg0k6SRJiyRd1uhYepOkXST9qJs6F0o6MC/vka8eNlfSUEmjJf06l02s5YpYkqZKOrkXYv93SR9Z0+2Y9TUncLPGOh54e0QcXlyZfxmwzPYFrutB/cOBMyNiYkQ8D3wa6PgCMJH0P7X1cgH+/W8rASdwswaRdB7pso7XSvpU7kFeIul20g8MbSDpF5Luyre35se9TtL1ucd6vqSHJI2S1Fq8PrGkkyVNzcubSbpO0mxJt0oal9dfmK9b/HtJD3T0iHPZqZLmS7pH0ll5G3cXyscW71fYG/hdxfFK0veVrkX9O+D1ef3HgIOAMwojER8Ersu/FvZl4ODcOz9Y0msl/UrpOsuzJE2o0rYfl3Rt7s1/WNKd+fH/I2lQrvOspK/k45sl6Q0A+ep6SyTtXNszadYYTuBmDRIRnyBdqnGviPhWXr01sE9EHAp8B/hWROxESmjn5zpfAm6LiG2Aq4FNatjdNODEiNgROBn4YaFsNLA78B7gLABJ+5IuifiWiNgOOCci/gy0S5qYH3cM8JPKHeWfMH0xItorit4PbJmP8Uhgt9wO55N+evKUiDhc0puAJyPihXzJyy+SrjM/MSKuBE4H5kTEBOBzwMUV+//3fCwHAK3AwcBbI2Ii6dfZOkY71gNm5eObSboUaYc2YI+qLWnWJMo+TGfW31yTh5AB9gG21suXqF9f0nBgMvABgIiYLunJrjaYH7MbL//cLcDgQpVf5StN3dvRC837/knHtd4jouPax+cDx0j6NCkxVuulvgO4vsr6ycBPI2Il8IikmzoJeTRd//b27qQvNETETXlEouP61UeSLuN4QES8KGlvYEfgrnzsQ4GO3+z/J/DrvDwbeHthH38HxnURg1nDOYGbNZfnCstrAbtExIpihUISrvQSrxxVG1LYzlO5B1pN8QpynW48+wVpBOAmYHYnvzO+L+nyqKvreV6Ovafmk86ZbwQ8SDqeiyLis1Xqvlj4je+VvPLzcEiOw6xpeQjdrHldT2EyVWHoeiZwWF63L/CavP4x4PW5RzqYNIxMRDwNPCjpQ/kxkrRdN/u+gdTTHpYf89q8rRWkC2GcS/XhcwETgLlVtjmTdC57kKTRwF6d7PuPpKHvDs+QLrna4VbyMLikKcCyfIwAc4B/I12OdUPgRuBASR3n218radMujrvDFsCCbmuZNZATuFnzOgmYlCdr3Qt8Iq8/HZgsaSFpKP0vABHxImnC152kBHxfYVuHAx+VdA+wkHR+u1MRcR3pvHSbpLmk8+YdLgNWUX2YfEfS+elqV0m6GvgTcC/pvPUdnez7OeDPkjbPq24mnUqYK+lgYCqwo6R5pHP2R1U8/rYc73TSUPgXgOtz/RtIQ/TdeWuua9a0fDUys5KTtASYFBHL6rS/k4GWiPivKmVfAO6PiCvWcB/vB3aMiC+syXZWc9/bA5+OiCPqvW+znvA5cDOrmaSrgc2At1Urj4j/7o39RMTVkl7XG9taDaOAV305MWs27oGbmZmVkM+Bm5mZlZATuJmZWQk5gZuZmZWQE7iZmVkJOYGbmZmV0P8DRACNAko5GxEAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"code","source":"viz.visualize_text([res])\nprint(res.attr_class)\nprint(res.word_attributions.size())\nprint(len(res.raw_input_ids))\n\ntorch.topk(res.word_attributions, k=5).indices\n\nfor i in torch.topk(res.word_attributions, k=5, largest=False).indices:\n print(res.raw_input_ids[i])\n\nprint(res.true_class)\n ","metadata":{"execution":{"iopub.status.busy":"2023-01-04T16:00:06.908884Z","iopub.execute_input":"2023-01-04T16:00:06.909401Z","iopub.status.idle":"2023-01-04T16:00:06.931767Z","shell.execute_reply.started":"2023-01-04T16:00:06.909356Z","shell.execute_reply":"2023-01-04T16:00:06.930649Z"},"trusted":true},"execution_count":71,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<table width: 100%><div style=\"border-top: 1px solid; margin-top: 5px; padding-top: 5px; display: inline-block\"><b>Legend: </b><span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 60%)\"></span> Negative <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(0, 75%, 100%)\"></span> Neutral <span style=\"display: inline-block; width: 10px; height: 10px; border: 1px solid; background-color: hsl(120, 75%, 50%)\"></span> Positive </div><tr><th>True Label</th><th>Predicted Label</th><th>Attribution Label</th><th>Attribution Score</th><th>Word Importance</th><tr><td><text style=\"padding-right:2em\"><b>0</b></text></td><td><text style=\"padding-right:2em\"><b>0 (6.89)</b></text></td><td><text style=\"padding-right:2em\"><b>For several days ensuing, her name was unmentioned by either Usher or myself: and during this period I was busied in earnest endeavors to alleviate the melancholy of my friend.</b></text></td><td><text style=\"padding-right:2em\"><b>1.78</b></text></td><td><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [CLS] </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> For </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> several </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> days </font></mark><mark style=\"background-color: hsl(120, 75%, 87%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ensuing </font></mark><mark style=\"background-color: hsl(0, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> , </font></mark><mark style=\"background-color: hsl(120, 75%, 85%); opacity:1.0; line-height:1.75\"><font color=\"black\"> her </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> name </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> un </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ment </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ioned </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> by </font></mark><mark style=\"background-color: hsl(120, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> either </font></mark><mark style=\"background-color: hsl(120, 75%, 89%); opacity:1.0; line-height:1.75\"><font color=\"black\"> Us </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##her </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> or </font></mark><mark style=\"background-color: hsl(120, 75%, 86%); opacity:1.0; line-height:1.75\"><font color=\"black\"> myself </font></mark><mark style=\"background-color: hsl(0, 75%, 92%); opacity:1.0; line-height:1.75\"><font color=\"black\"> : </font></mark><mark style=\"background-color: hsl(0, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> and </font></mark><mark style=\"background-color: hsl(0, 75%, 97%); opacity:1.0; line-height:1.75\"><font color=\"black\"> during </font></mark><mark style=\"background-color: hsl(120, 75%, 78%); opacity:1.0; line-height:1.75\"><font color=\"black\"> this </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> period </font></mark><mark style=\"background-color: hsl(120, 75%, 84%); opacity:1.0; line-height:1.75\"><font color=\"black\"> I </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> was </font></mark><mark style=\"background-color: hsl(0, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> bus </font></mark><mark style=\"background-color: hsl(120, 75%, 96%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##ied </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> in </font></mark><mark style=\"background-color: hsl(0, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> earnest </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> end </font></mark><mark style=\"background-color: hsl(120, 75%, 91%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##eavor </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> ##s </font></mark><mark style=\"background-color: hsl(120, 75%, 99%); opacity:1.0; line-height:1.75\"><font color=\"black\"> to </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> alleviate </font></mark><mark style=\"background-color: hsl(120, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> the </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> melancholy </font></mark><mark style=\"background-color: hsl(120, 75%, 98%); opacity:1.0; line-height:1.75\"><font color=\"black\"> of </font></mark><mark style=\"background-color: hsl(120, 75%, 93%); opacity:1.0; line-height:1.75\"><font color=\"black\"> my </font></mark><mark style=\"background-color: hsl(0, 75%, 95%); opacity:1.0; line-height:1.75\"><font color=\"black\"> friend </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> . </font></mark><mark style=\"background-color: hsl(0, 75%, 100%); opacity:1.0; line-height:1.75\"><font color=\"black\"> [SEP] </font></mark></td><tr></table>"},"metadata":{}},{"name":"stdout","text":"For several days ensuing, her name was unmentioned by either Usher or myself: and during this period I was busied in earnest endeavors to alleviate the melancholy of my friend.\ntorch.Size([41])\n41\n:\nun\nearnest\nalleviate\nfriend\n0\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Ensembling","metadata":{}},{"cell_type":"code","source":"bert_vld_pred = torch.LongTensor()\nbert_vld_y = torch.LongTensor()\n\nprint_i = 200\nfor i, data in enumerate(tqdm(valid_dataloader)):\n y = data.pop(\"labels\")\n x = {key:val.to(device) for key, val in data.items()}\n batch_pred = torch.argmax(vizmodel(**x).logits, dim=1).to('cpu')\n \n bert_vld_pred = torch.cat((batch_pred, bert_vld_pred))\n bert_vld_y = torch.cat((y, bert_vld_y))\nprint(torch.sum(bert_vld_pred == bert_vld_y) / (bert_vld_y.size()[0]))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T15:51:46.392037Z","iopub.execute_input":"2023-01-04T15:51:46.392596Z","iopub.status.idle":"2023-01-04T15:52:46.670193Z","shell.execute_reply.started":"2023-01-04T15:51:46.392549Z","shell.execute_reply":"2023-01-04T15:52:46.668688Z"},"trusted":true},"execution_count":54,"outputs":[{"name":"stderr","text":" 0%| | 0/243 [00:00<?, ?it/s]","output_type":"stream"},{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"},{"name":"stderr","text":"100%|██████████| 243/243 [01:00<00:00, 4.04it/s]","output_type":"stream"},{"name":"stdout","text":"To disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n","output_type":"stream"},{"name":"stderr","text":"100%|██████████| 243/243 [01:00<00:00, 4.03it/s]","output_type":"stream"},{"name":"stdout","text":"tensor(0.9897)\n","output_type":"stream"},{"name":"stderr","text":"\n","output_type":"stream"}]},{"cell_type":"code","source":"print(torch.sum(bert_vld_pred == bert_vld_y) / (bert_vld_y.size()[0]))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T15:53:19.215373Z","iopub.execute_input":"2023-01-04T15:53:19.215774Z","iopub.status.idle":"2023-01-04T15:53:19.225125Z","shell.execute_reply.started":"2023-01-04T15:53:19.215736Z","shell.execute_reply":"2023-01-04T15:53:19.223921Z"},"trusted":true},"execution_count":55,"outputs":[{"name":"stdout","text":"tensor(0.9897)\n","output_type":"stream"}]},{"cell_type":"code","source":"print(torch.sum(bert_vld_pred == bert_vld_y) / (bert_vld_y.size()[0]))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bert_train_pred = torch.LongTensor()\nrectrect = torch.LongTensor()\n\nprint_i = 200\nfor i, data in enumerate(tqdm(train_dataloader)):\n y = data.pop(\"labels\")\n x = {key:val.to(device) for key, val in data.items()}\n #print(x.keys())\n batch_pred = torch.argmax(vizmodel(**x).logits, dim=1).to('cpu')\n break\n \n bert_train_pred = torch.cat((batch_pred, bert_train_pred))\n bert_train_y = torch.cat((y, bert_train_y))\n \n if i > 0 and i%print_i == 0:\n print(torch.sum(bert_train_pred == bert_train_y) / bert_train_y.size()[0])\n #print(tokenizer.convert_ids_to_tokens(x[\"input_ids\"][0]))","metadata":{"execution":{"iopub.status.busy":"2023-01-05T15:03:49.047399Z","iopub.execute_input":"2023-01-05T15:03:49.047786Z","iopub.status.idle":"2023-01-05T15:03:49.746802Z","shell.execute_reply.started":"2023-01-05T15:03:49.047749Z","shell.execute_reply":"2023-01-05T15:03:49.745604Z"},"trusted":true},"execution_count":68,"outputs":[{"name":"stderr","text":" 0%| | 0/2186 [00:00<?, ?it/s]","output_type":"stream"},{"name":"stdout","text":"huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\nhuggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\nTo disable this warning, you can either:\n\t- Avoid using `tokenizers` before the fork if possible\n\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\ndict_keys(['input_ids', 'token_type_ids', 'attention_mask'])\n","output_type":"stream"},{"name":"stderr","text":" 0%| | 0/2186 [00:00<?, ?it/s]\n","output_type":"stream"}]},{"cell_type":"code","source":"from xgboost import XGBClassifier\n\nxgbclf = XGBClassifier(verbosity=1, n_jobs=2)\ntfidftrain = label(pipeline.predict(train[\"text\"]))\ntfidfvalid = label(pipeline.predict(valid['text']))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T13:49:25.061016Z","iopub.execute_input":"2023-01-04T13:49:25.062306Z","iopub.status.idle":"2023-01-04T13:49:32.368695Z","shell.execute_reply.started":"2023-01-04T13:49:25.062256Z","shell.execute_reply":"2023-01-04T13:49:32.367574Z"},"trusted":true},"execution_count":78,"outputs":[]},{"cell_type":"code","source":"x_train = []\nfor i in range(len(tfidftrain)):\n x_train.append([tfidftrain[i], int(bert_train_pred[i])])\n\nxgbclf.fit(x_train, label(train[\"author\"]))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T13:49:32.371031Z","iopub.execute_input":"2023-01-04T13:49:32.371824Z","iopub.status.idle":"2023-01-04T13:49:33.106922Z","shell.execute_reply.started":"2023-01-04T13:49:32.371777Z","shell.execute_reply":"2023-01-04T13:49:33.105215Z"},"trusted":true},"execution_count":79,"outputs":[{"execution_count":79,"output_type":"execute_result","data":{"text/plain":"XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,\n colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1,\n early_stopping_rounds=None, enable_categorical=False,\n eval_metric=None, gamma=0, gpu_id=-1, grow_policy='depthwise',\n importance_type=None, interaction_constraints='',\n learning_rate=0.300000012, max_bin=256, max_cat_to_onehot=4,\n max_delta_step=0, max_depth=6, max_leaves=0, min_child_weight=1,\n missing=nan, monotone_constraints='()', n_estimators=100,\n n_jobs=2, num_parallel_tree=1, objective='multi:softprob',\n predictor='auto', random_state=0, reg_alpha=0, ...)"},"metadata":{}}]},{"cell_type":"code","source":"x_valid = []\nfor i in range(len(tfidfvalid)):\n x_valid.append([tfidfvalid[i], int(predictions[i])])\n\nres = xgbclf.predict(x_valid)\n\nprint(classification_report(valid[\"author\"], unlabel(res)))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T13:49:41.175225Z","iopub.execute_input":"2023-01-04T13:49:41.175625Z","iopub.status.idle":"2023-01-04T13:49:41.236779Z","shell.execute_reply.started":"2023-01-04T13:49:41.175591Z","shell.execute_reply":"2023-01-04T13:49:41.235146Z"},"trusted":true},"execution_count":81,"outputs":[{"name":"stdout","text":" precision recall f1-score support\n\n EAP 0.78 0.86 0.82 782\n HPL 0.84 0.80 0.82 561\n MWS 0.86 0.78 0.82 600\n\n accuracy 0.82 1943\n macro avg 0.83 0.81 0.82 1943\nweighted avg 0.82 0.82 0.82 1943\n\n","output_type":"stream"}]},{"cell_type":"code","source":"print(classification_report(valid[\"author\"], unlabel(predictions)))","metadata":{"execution":{"iopub.status.busy":"2023-01-04T13:50:07.464222Z","iopub.execute_input":"2023-01-04T13:50:07.464633Z","iopub.status.idle":"2023-01-04T13:50:07.511290Z","shell.execute_reply.started":"2023-01-04T13:50:07.464595Z","shell.execute_reply":"2023-01-04T13:50:07.510238Z"},"trusted":true},"execution_count":82,"outputs":[{"name":"stdout","text":" precision recall f1-score support\n\n EAP 0.42 0.42 0.42 782\n HPL 0.30 0.30 0.30 561\n MWS 0.33 0.33 0.33 600\n\n accuracy 0.36 1943\n macro avg 0.35 0.35 0.35 1943\nweighted avg 0.36 0.36 0.36 1943\n\n","output_type":"stream"}]},{"cell_type":"code","source":"import torch\ntest_logit_df = torch.load(\"/kaggle/input/bertpredictions/bert-test-pred\")","metadata":{"execution":{"iopub.status.busy":"2023-01-06T15:44:16.266124Z","iopub.execute_input":"2023-01-06T15:44:16.266507Z","iopub.status.idle":"2023-01-06T15:44:16.302636Z","shell.execute_reply.started":"2023-01-06T15:44:16.266478Z","shell.execute_reply":"2023-01-06T15:44:16.301626Z"},"trusted":true},"execution_count":4,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\ntest_logit_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2023-01-06T15:47:25.666294Z","iopub.execute_input":"2023-01-06T15:47:25.666690Z","iopub.status.idle":"2023-01-06T15:47:25.696953Z","shell.execute_reply.started":"2023-01-06T15:47:25.666658Z","shell.execute_reply":"2023-01-06T15:47:25.696024Z"},"trusted":true},"execution_count":11,"outputs":[{"execution_count":11,"output_type":"execute_result","data":{"text/plain":" id pred eap hpl mws\n0 id02310 MWS -3.141249 -2.306986 5.610037\n1 id24541 EAP 6.279230 -2.883798 -2.380444","text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>id</th>\n <th>pred</th>\n <th>eap</th>\n <th>hpl</th>\n <th>mws</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>id02310</td>\n <td>MWS</td>\n <td>-3.141249</td>\n <td>-2.306986</td>\n <td>5.610037</td>\n </tr>\n <tr>\n <th>1</th>\n <td>id24541</td>\n <td>EAP</td>\n <td>6.279230</td>\n <td>-2.883798</td>\n <td>-2.380444</td>\n </tr>\n </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"logits = torch.tensor(test_logit_df[[\"eap\", \"hpl\", \"mws\"]].values)\nprobs = torch.softmax(logits, dim=1)\nprobs\n\nout_df = pd.DataFrame()\nout_df[\"id\"] = test_logit_df[\"id\"]\nout_df[\"EAP\"] = probs[:, 0]\nout_df[\"HPL\"] = probs[:, 1]\nout_df[\"MWS\"] = probs[:, 2]\nout_df.to_csv('pred.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-06T15:49:02.323535Z","iopub.execute_input":"2023-01-06T15:49:02.324205Z","iopub.status.idle":"2023-01-06T15:49:02.424713Z","shell.execute_reply.started":"2023-01-06T15:49:02.324158Z","shell.execute_reply":"2023-01-06T15:49:02.423377Z"},"trusted":true},"execution_count":15,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nfrom sklearn.metrics import log_loss\n\npreds = torch.load(\"/kaggle/input/bertpredictions/predictions.th\")\npreds.drop(\"pred\", axis=1, inplace=True)\n\nvalid_preds = preds[preds[\"id\"].isin(valid[\"id\"])]\nvalid_preds = torch.softmax(torch.tensor(valid_preds.drop(\"id\", axis=1).values), dim=1)\nlog_loss(LabelEncoder().fit_transform(valid[\"author\"]), valid_preds)","metadata":{"execution":{"iopub.status.busy":"2023-01-06T16:02:20.693179Z","iopub.execute_input":"2023-01-06T16:02:20.693536Z","iopub.status.idle":"2023-01-06T16:02:20.734050Z","shell.execute_reply.started":"2023-01-06T16:02:20.693508Z","shell.execute_reply":"2023-01-06T16:02:20.732964Z"},"trusted":true},"execution_count":36,"outputs":[{"name":"stdout","text":"tensor([[5.7383e-05, 3.9990e-04, 9.9954e-01],\n [7.5776e-03, 9.9188e-01, 5.4202e-04],\n [1.1260e-04, 9.9981e-01, 8.1612e-05],\n ...,\n [9.9910e-01, 4.0064e-04, 4.9754e-04],\n [9.9407e-01, 4.9792e-03, 9.4700e-04],\n [5.7982e-05, 9.9988e-01, 6.1903e-05]], dtype=torch.float64)\n","output_type":"stream"},{"execution_count":36,"output_type":"execute_result","data":{"text/plain":"5.182596702548633"},"metadata":{}}]},{"cell_type":"code","source":"lens = pd.DataFrame([len(text) for text in test[\"text\"] if len(text)<=510])\nlens.hist()\nsum([\"text\"].str.len() > 510)","metadata":{"execution":{"iopub.status.busy":"2023-01-06T16:26:23.647128Z","iopub.execute_input":"2023-01-06T16:26:23.647504Z","iopub.status.idle":"2023-01-06T16:26:24.521319Z","shell.execute_reply.started":"2023-01-06T16:26:23.647472Z","shell.execute_reply":"2023-01-06T16:26:24.519874Z"},"trusted":true},"execution_count":46,"outputs":[{"traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_23/343683295.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mlens\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mtext\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtest\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"text\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m<=\u001b[0m\u001b[0;36m510\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mlens\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"text\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m510\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m","\u001b[0;31mAttributeError\u001b[0m: 'list' object has no attribute 'str'"],"ename":"AttributeError","evalue":"'list' object has no attribute 'str'","output_type":"error"},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQ70lEQVR4nO3db4xldX3H8fen4B+KWkB0sgHSwXTbBrsVyQaw+mDUFBdoik2MkRJZLGabFhJNNmmWNimtxmT7AG1JLOm2bsRGoTRK2AApblc3pg9QQIEFEVntEtgAGwXRxUQ79tsH9zeb23WXnZm9c+/M/N6v5Oae8z1nzv19Z+587plzzz2TqkKS1IdfmfQAJEnjY+hLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj60gIlOS3J7UleSvJkkj+e9Jik+Tpx0gOQVqBPAz8HpoBzgbuSPFRVj050VNI8xE/kSvOX5GTgBeB3quq7rfavwP6q2jLRwUnz4OEdaWF+E5idC/zmIeDNExqPtCCGvrQwrwF+fFjtReC1ExiLtGCGvrQwB4HXHVZ7HfCTCYxFWjBDX1qY7wInJlk7VHsL4Ju4WhF8I1daoCS3AgV8mMHZO3cDv+fZO1oJ3NOXFu7PgZOAA8AtwJ8Z+Fop3NOXpI64py9JHTH0Jakjhr4kdcTQl6SOLOsLrp1++uk1PT0NwEsvvcTJJ5882QFNgH33pde+od/el6LvBx544AdV9YYjLVvWoT89Pc39998PwO7du5mZmZnsgCbAvvvSa9/Qb+9L0XeSJ4+2zMM7ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUkWX9idyVanrLXSPb1uZ1s1y1gO3t23rpyB5b0urjnr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhwz9JOcleSrSb6d5NEkH2n105LsTPJEuz+11ZPkxiR7kzyc5LyhbW1s6z+RZOPStSVJOpL57OnPApur6hzgQuCaJOcAW4BdVbUW2NXmAS4G1rbbJuAmGLxIANcDFwDnA9fPvVBIksbjmKFfVc9U1Tfb9E+Ax4AzgMuAm9tqNwPvbdOXAZ+rgXuBU5KsAd4D7Kyq56vqBWAnsGGUzUiSXt6CrrKZZBp4K/B1YKqqnmmLngWm2vQZwFNDX/Z0qx2tfvhjbGLwFwJTU1Ps3r0bgIMHDx6aXu42r5sd2bamTlrY9lbK9+hYVtLPe5R67Rv67X3cfc879JO8Bvgi8NGq+nGSQ8uqqpLUKAZUVduAbQDr16+vmZkZYBBmc9PL3UIuhXwsm9fNcsOe+b8277tiZmSPPUkr6ec9Sr32Df32Pu6+53X2TpJXMAj8z1fVl1r5uXbYhnZ/oNX3A2cNffmZrXa0uiRpTOZz9k6AzwCPVdUnhxbtAObOwNkI3DFUv7KdxXMh8GI7DHQPcFGSU9sbuBe1miRpTOZz3ODtwAeBPUkebLW/BLYCtyW5GngSeH9bdjdwCbAX+CnwIYCqej7Jx4H72nofq6rnR9GEJGl+jhn6VfVfQI6y+N1HWL+Aa46yre3A9oUMUJI0On4iV5I6YuhLUkcMfUnqyII+nKXlb3qEnxFYiH1bL53I40paGPf0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdeTESQ9gKU1vuWvSQ5CkZcU9fUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHjhn6SbYnOZDkkaHa3yTZn+TBdrtkaNl1SfYmeTzJe4bqG1ptb5Ito29FknQs89nT/yyw4Qj1T1XVue12N0CSc4APAG9uX/OPSU5IcgLwaeBi4Bzg8rauJGmMjnkZhqr6WpLpeW7vMuDWqvoZ8N9J9gLnt2V7q+r7AElubet+e+FDliQt1vFce+faJFcC9wObq+oF4Azg3qF1nm41gKcOq19wpI0m2QRsApiammL37t0AHDx48ND0fG1eN7ug9ZejqZNWRh8L/dkcy2J+3qtBr31Dv72Pu+/Fhv5NwMeBavc3AH8yigFV1TZgG8D69etrZmYGGITK3PR8XbUKLri2ed0sN+xZ/tfF23fFzEi3t5if92rQa9/Qb+/j7ntRaVJVz81NJ/ln4M42ux84a2jVM1uNl6lLksZkUadsJlkzNPtHwNyZPTuADyR5VZKzgbXAN4D7gLVJzk7ySgZv9u5Y/LAlSYtxzD39JLcAM8DpSZ4GrgdmkpzL4PDOPuBPAarq0SS3MXiDdha4pqp+0bZzLXAPcAKwvaoeHXUzkqSXN5+zdy4/QvkzL7P+J4BPHKF+N3D3gkYnSRopP5ErSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1JETJz0ArQ7TW+4a6fY2r5vlqnlsc9/WS0f6uNJq556+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdeSYoZ9ke5IDSR4Zqp2WZGeSJ9r9qa2eJDcm2Zvk4STnDX3Nxrb+E0k2Lk07kqSXM589/c8CGw6rbQF2VdVaYFebB7gYWNtum4CbYPAiAVwPXACcD1w/90IhSRqfY4Z+VX0NeP6w8mXAzW36ZuC9Q/XP1cC9wClJ1gDvAXZW1fNV9QKwk19+IZEkLbHF/mP0qap6pk0/C0y16TOAp4bWe7rVjlb/JUk2MfgrgampKXbv3g3AwYMHD03P1+Z1swtafzmaOml19LFQ8+17oc+J5W4xz/PVotfex933YkP/kKqqJDWKwbTtbQO2Aaxfv75mZmaAwS/33PR8XbXlrlENa2I2r5vlhj3H/WNacebb974rZpZ+MGO0mOf5atFr7+Pue7Fn7zzXDtvQ7g+0+n7grKH1zmy1o9UlSWO02NDfAcydgbMRuGOofmU7i+dC4MV2GOge4KIkp7Y3cC9qNUnSGB3z7+cktwAzwOlJnmZwFs5W4LYkVwNPAu9vq98NXALsBX4KfAigqp5P8nHgvrbex6rq8DeHJUlL7JihX1WXH2XRu4+wbgHXHGU724HtCxqdJGmk/ESuJHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjpw46QFIx2N6y10Te+x9Wy+d2GNLi+WeviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6clyhn2Rfkj1JHkxyf6udlmRnkifa/amtniQ3Jtmb5OEk542iAUnS/I1iT/+dVXVuVa1v81uAXVW1FtjV5gEuBta22ybgphE8tiRpAZbi8M5lwM1t+mbgvUP1z9XAvcApSdYsweNLko7ieEO/gC8neSDJplabqqpn2vSzwFSbPgN4auhrn241SdKYHO8/UXlHVe1P8kZgZ5LvDC+sqkpSC9lge/HYBDA1NcXu3bsBOHjw4KHp+dq8bnZB6y9HUyetjj4WaiX0vdDn43ws5nm+WvTa+7j7Pq7Qr6r97f5AktuB84Hnkqypqmfa4ZsDbfX9wFlDX35mqx2+zW3ANoD169fXzMwMMPgFm5uer6sm+F+VRmXzullu2NPfPzhbCX3vu2Jm5NtczPN8tei193H3vejDO0lOTvLauWngIuARYAewsa22EbijTe8Armxn8VwIvDh0GEiSNAbHsys1BdyeZG47X6iq/0hyH3BbkquBJ4H3t/XvBi4B9gI/BT50HI8tSVqERYd+VX0feMsR6j8E3n2EegHXLPbxJEnHz0/kSlJHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUkeX9OXdpGZtegst8bF43e8zLh+zbeunIH1f9cE9fkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXESytLK8xSXNJ5vrys88rnnr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRr70jad6W8ro/m9fNctVRtu81f0bHPX1J6oihL0kd8fCOpGXPy0mPjnv6ktQRQ1+SOjL2wztJNgD/AJwA/EtVbR33GCRpvpb60NLRzlpaqsNKY93TT3IC8GngYuAc4PIk54xzDJLUs3Ef3jkf2FtV36+qnwO3ApeNeQyS1K1U1fgeLHkfsKGqPtzmPwhcUFXXDq2zCdjUZn8LeLxNnw78YGyDXT7suy+99g399r4Uff96Vb3hSAuW3SmbVbUN2HZ4Pcn9VbV+AkOaKPvuS699Q7+9j7vvcR/e2Q+cNTR/ZqtJksZg3KF/H7A2ydlJXgl8ANgx5jFIUrfGeninqmaTXAvcw+CUze1V9eg8v/yXDvl0wr770mvf0G/vY+17rG/kSpImy0/kSlJHDH1J6siyD/0kG5I8nmRvki2THs+oJdme5ECSR4ZqpyXZmeSJdn9qqyfJje178XCS8yY38sVLclaSryb5dpJHk3yk1Vd13wBJXp3kG0kear3/baufneTrrcd/ayc6kORVbX5vWz490QaOU5ITknwryZ1tftX3nWRfkj1JHkxyf6tN7Lm+rEO/k8s2fBbYcFhtC7CrqtYCu9o8DL4Pa9ttE3DTmMY4arPA5qo6B7gQuKb9XFd73wA/A95VVW8BzgU2JLkQ+DvgU1X1G8ALwNVt/auBF1r9U229lewjwGND8730/c6qOnfofPzJPderatnegLcB9wzNXwdcN+lxLUGf08AjQ/OPA2va9Brg8Tb9T8DlR1pvJd+AO4Df77DvXwW+CVzA4BOZJ7b6oec9gzPd3tamT2zrZdJjX2S/ZzIIuHcBdwLppO99wOmH1Sb2XF/We/rAGcBTQ/NPt9pqN1VVz7TpZ4GpNr3qvh/tz/a3Al+nk77bIY4HgQPATuB7wI+qaratMtzfod7b8heB1491wKPz98BfAP/b5l9PH30X8OUkD7TLzMAEn+vL7jIM+v+qqpKsyvNqk7wG+CLw0ar6cZJDy1Zz31X1C+DcJKcAtwO/PdkRLb0kfwAcqKoHksxMeDjj9o6q2p/kjcDOJN8ZXjju5/py39Pv9bINzyVZA9DuD7T6qvl+JHkFg8D/fFV9qZVXfd/DqupHwFcZHNY4JcncTthwf4d6b8t/DfjheEc6Em8H/jDJPgZX130Xg/+rsdr7pqr2t/sDDF7kz2eCz/XlHvq9XrZhB7CxTW9kcMx7rn5le4f/QuDFoT8RV4wMduk/AzxWVZ8cWrSq+wZI8oa2h0+Skxi8l/EYg/B/X1vt8N7nvifvA75S7WDvSlJV11XVmVU1zeD3+CtVdQWrvO8kJyd57dw0cBHwCJN8rk/6TY55vAlyCfBdBsc9/2rS41mC/m4BngH+h8Hxu6sZHLvcBTwB/CdwWls3DM5m+h6wB1g/6fEvsud3MDjO+TDwYLtdstr7br38LvCt1vsjwF+3+puAbwB7gX8HXtXqr27ze9vyN026hxF8D2aAO3vou/X3ULs9Opdhk3yuexkGSerIcj+8I0kaIUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdeT/ACO7ZWPmEO+OAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"}}]}]}
\ No newline at end of file