Finetuning_language_model_BERtimbau_LeNER_Br.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Finetuning of the language model BERTimbau on LeNER-Br text files"
      ],
      "metadata": {
        "id": "9s6mFOorYEo6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- **Credit**: this notebook is copied/pasted with small changes from [PyTorch Examples](https://huggingface.co/docs/transformers/notebooks#pytorch-examples) of Hugging Face (notebook [language_modeling.ipynb](https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/language_modeling.ipynb)).\n",
        "- **Author**: [Pierre GUILLOU](https://www.linkedin.com/in/pierreguillou/)\n",
        "- **Date**: 12/20/2021\n",
        "- **Blog post**: [NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro](https://medium.com/@pierre_guillou/nlp-modelos-e-web-app-para-reconhecimento-de-entidade-nomeada-ner-no-dom%C3%ADnio-jur%C3%ADdico-b658db55edfb)"
      ],
      "metadata": {
        "id": "AJ328rUQYXxK"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a3KD3WXU3l-O"
      },
      "source": [
        "## Overview"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAscNNUD3l-P"
      },
      "source": [
        "In this notebook, we'll see how to fine-tune one of the [🤗 Transformers](https://github.com/huggingface/transformers) model on a masked language modeling tasks. \n",
        "\n",
        "Note: a Masked language modeling is a model that has to predict some tokens that are masked in the input. It still has access to the whole sentence, so it can use the tokens before and after the tokens masked to predict their value.\n",
        "\n",
        "![Widget inference representing the masked language modeling task](https://github.com/huggingface/notebooks/blob/master/examples/images/masked_language_modeling.png?raw=1)\n",
        "\n",
        "We will see how to easily load and preprocess the dataset for each one of those tasks, and how to use the `Trainer` API to fine-tune a model on it.\n",
        "\n",
        "A script version of this notebook you can directly run on a distributed environment or on TPU is available in our [examples folder](https://github.com/huggingface/transformers/tree/master/examples)."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Configuration"
      ],
      "metadata": {
        "id": "zH_MY4GuZSw0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_checkpoint = \"neuralmind/bert-base-portuguese-cased\""
      ],
      "metadata": {
        "id": "0FROJ7_qQSqn"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you're opening this Notebook on colab, you will need to connect to your Google Drive and to install 🤗 Transformers and 🤗 Datasets."
      ],
      "metadata": {
        "id": "f36m8O2hYVsd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive \n",
        "drive.mount('/content/drive', force_remount=True)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k4zV5HnuP7wS",
        "outputId": "825c8a0d-abe4-468f-eef3-525c8d77eadf"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "MOsHUjgdIrIW"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "! pip install datasets transformers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Q_Qyc0YQGxE"
      },
      "source": [
        "If you're opening this notebook locally, make sure your environment has an install from the last version of those libraries.\n",
        "\n",
        "To be able to share your model with the community and generate results like the one shown in the picture below via the inference API, there are a few more steps to follow.\n",
        "\n",
        "First you have to store your authentication token from the Hugging Face website (sign up [here](https://huggingface.co/join) if you haven't already!) then execute the following cell and input your username and password:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "ID_bdI5EQGxF",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 387,
          "referenced_widgets": [
            "346b2c7413e24b6e9a4fec62df9c77bf",
            "b053aa0217fe481f86e1f4ff04609318",
            "25f71c960336451cb9607fcb083b30c6",
            "ac5823324cb04c46a53b8ac4067b7657",
            "20eb45883cca47d1bcca7240e4ca4d7f",
            "3e464cd6e07142f3b6fd40bb5270b80f",
            "f742ecc1f85f426bbe93dc27ba6e8254",
            "cd937c4062514f11b69b2b7035165a09",
            "1fb8b901801f4a9fba8924b004f7d947",
            "9ec6f0dc51eb46c399682c408ab17272",
            "58f7098a299648e988d634226e10bf13",
            "3a2a15911c51435db77c7cc9abcdb2c1",
            "d98cea9e5e414deca52167ae351c0016",
            "28fac573b08e4ec681458d9c67336989",
            "9febad88718d4e90a63721ecd6f4eb09",
            "dcb00bab415247e9895ac8521241935f",
            "b39e6c23eb2246c19fb6a5786814040a"
          ]
        },
        "outputId": "f5311537-3714-45c3-c773-aae8835061df"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Login successful\n",
            "Your token has been saved to /root/.huggingface/token\n",
            "\u001b[1m\u001b[31mAuthenticated through git-credential store but this isn't the helper defined on your machine.\n",
            "You might have to re-authenticate when pushing to the Hugging Face Hub. Run the following command in your terminal in case you want to set this credential helper as the default\n",
            "\n",
            "git config --global credential.helper store\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "from huggingface_hub import notebook_login\n",
        "\n",
        "notebook_login()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a-CTNRqpQGxG"
      },
      "source": [
        "Then you need to install Git-LFS. Uncomment the following instructions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "gLleehJiQGxG"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!apt install git-lfs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t2MKwGLYQGxH"
      },
      "source": [
        "Make sure your version of Transformers is at least 4.11.0 since the functionality was introduced in that version:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "p6eZ9_L6QGxI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3487ab5c-7b61-4414-965c-ec15d962af59"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "4.15.0\n"
          ]
        }
      ],
      "source": [
        "import transformers\n",
        "\n",
        "print(transformers.__version__)\n",
        "# 4.14.1"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import datasets\n",
        "\n",
        "print(datasets.__version__)\n",
        "# 1.17.0"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IlsM5bJLAFrK",
        "outputId": "2045f791-ee5a-40f9-c88b-9416a8ff4734"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1.17.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pathlib\n",
        "from pathlib import Path\n",
        "\n",
        "import pandas as pd"
      ],
      "metadata": {
        "id": "XPVQ3eFVZ_Zq"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from datasets import Dataset, DatasetDict"
      ],
      "metadata": {
        "id": "jEQGmKlybp0Y"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1r_n9OWV3l-Q"
      },
      "source": [
        "## Preparing the dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1. Load LeNER-Br text files"
      ],
      "metadata": {
        "id": "fMoO8X5MaHDX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "path_to_text_files = \"https://cic.unb.br/~teodecampos/LeNER-Br/LeNER-Br.zip\""
      ],
      "metadata": {
        "id": "aUw_WEN4RF-A"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!wget {path_to_text_files}"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1jfvIodRTncT",
        "outputId": "0ec9ef08-f816-467a-d35a-9178176688fb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2021-12-22 08:43:02--  https://cic.unb.br/~teodecampos/LeNER-Br/LeNER-Br.zip\n",
            "Resolving cic.unb.br (cic.unb.br)... 164.41.110.66\n",
            "Connecting to cic.unb.br (cic.unb.br)|164.41.110.66|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 93637203 (89M) [application/zip]\n",
            "Saving to: ‘LeNER-Br.zip’\n",
            "\n",
            "LeNER-Br.zip        100%[===================>]  89.30M  4.12MB/s    in 18s     \n",
            "\n",
            "2021-12-22 08:43:22 (4.86 MB/s) - ‘LeNER-Br.zip’ saved [93637203/93637203]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!ls -al"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OZomr0EFTxTb",
        "outputId": "7e04c30d-2205-4c89-9368-c1813ce012ca"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "total 91460\n",
            "drwxr-xr-x 1 root root     4096 Dec 22 08:43 .\n",
            "drwxr-xr-x 1 root root     4096 Dec 22 08:26 ..\n",
            "drwxr-xr-x 4 root root     4096 Dec  3 14:33 .config\n",
            "-rw-r--r-- 1 root root 93637203 Aug 31  2018 LeNER-Br.zip\n",
            "drwxr-xr-x 1 root root     4096 Dec  3 14:33 sample_data\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!unzip LeNER-Br.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cb-9TmviT1LG",
        "outputId": "45939bad-76dd-43f1-c6ca-a1a9dff3c0e8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Archive:  LeNER-Br.zip\n",
            "   creating: LeNER-Br/\n",
            "  inflating: LeNER-Br/index.html     \n",
            "  inflating: LeNER-Br/luz_etal_propor2018.pdf  \n",
            "  inflating: LeNER-Br/README.md      \n",
            "   creating: LeNER-Br/model/\n",
            "  inflating: LeNER-Br/model/evaluate.py  \n",
            " extracting: LeNER-Br/model/requirements.txt  \n",
            "  inflating: LeNER-Br/model/build_data.py  \n",
            "  inflating: LeNER-Br/model/train.py  \n",
            "  inflating: LeNER-Br/model/evaluateText.py  \n",
            "  inflating: LeNER-Br/model/evaluateSentence.py  \n",
            "  inflating: LeNER-Br/model/classScores.py  \n",
            "  inflating: LeNER-Br/model/LICENSE.txt  \n",
            "   creating: LeNER-Br/leNER-Br/\n",
            "   creating: LeNER-Br/model/results/\n",
            "   creating: LeNER-Br/model/model/\n",
            "  inflating: LeNER-Br/model/model/config.pyc  \n",
            "  inflating: LeNER-Br/model/model/data_utils.pyc  \n",
            "  inflating: LeNER-Br/model/model/ner_model.py  \n",
            "  inflating: LeNER-Br/model/model/data_utils.py  \n",
            "  inflating: LeNER-Br/model/model/ner_model.pyc  \n",
            "  inflating: LeNER-Br/model/model/base_model.pyc  \n",
            "  inflating: LeNER-Br/model/model/config.py  \n",
            "  inflating: LeNER-Br/model/model/__init__.pyc  \n",
            " extracting: LeNER-Br/model/model/__init__.py  \n",
            "  inflating: LeNER-Br/model/model/base_model.py  \n",
            "  inflating: LeNER-Br/model/model/general_utils.pyc  \n",
            "  inflating: LeNER-Br/model/model/general_utils.py  \n",
            "   creating: LeNER-Br/model/data/\n",
            "  inflating: LeNER-Br/model/data/train.txt  \n",
            "  inflating: LeNER-Br/model/data/words.txt  \n",
            "  inflating: LeNER-Br/model/data/glove.6B.300d.trimmed.npz  \n",
            "  inflating: LeNER-Br/model/data/chars.txt  \n",
            "  inflating: LeNER-Br/model/data/test.txt  \n",
            "  inflating: LeNER-Br/model/data/dev.txt  \n",
            "  inflating: LeNER-Br/model/data/tags.txt  \n",
            "   creating: LeNER-Br/leNER-Br/train/\n",
            "  inflating: LeNER-Br/leNER-Br/train/AgRgSTJ2.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/adi3767.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AIRR15708820115050222.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Ag10000170733596001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Port77DF.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/train.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/airr801422012.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/ERR731004520105130003.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/EDRR1TST.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AgRgTSE3.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/DespSEPLAGDF.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/ADI2TJDFT.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/ACORDAOTCU25052016.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/lei11340.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AP00001415620157010201.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/RR942006420095040028.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/EDEDARR208420135040232.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/HC151914AgRES.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AC10024133855890001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/ED1STM.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AgRgTSE1.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Pet128TSE5.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/TSTRR16037920105200001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/LoaDF2018.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AC1TJAC.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AgAIRR11889820145030011.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AIAgRAgI6193ARAGUARIMG.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Rcl3495STJ.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/EEDRR9715120105020002.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/HC340624SP.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Ag10105170208398001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/TCU4687.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/REsp1583083RS.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/HC70000845920187000000.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AP00000794920137060006.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/CP32320177080008PA.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AC1TJMG.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AIRR3999520145020086.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AgRgSTJ1.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/HC110260SP.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/ED1TJAC.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AP771420167080008PA.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/RR474820145230056.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/Lei11788.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AgCr10582160008758001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/RR2574407120025020372.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AC1TCU.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/HC418951PR.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/EDAgRgTSE2.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/REE5908TSE4.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/train/AP00001441420167030203.conll  \n",
            "   creating: LeNER-Br/leNER-Br/test/\n",
            "  inflating: LeNER-Br/leNER-Br/test/AIRR285001420095060020.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/AgAIRR617420125150072.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/AIRR581406820065030079.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/test.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/AIRR3731820145060141.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/AIRR10006691020135020322.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/HC04798525420128130000.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/ACORDAOTCU11602016.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/RR14976020105020085.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/HC10000150589281000.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/test/RR-578030-46.1999.5.04.0018.conll  \n",
            "   creating: LeNER-Br/leNER-Br/scripts/\n",
            "  inflating: LeNER-Br/leNER-Br/scripts/abbrev_list.pkl  \n",
            "  inflating: LeNER-Br/leNER-Br/scripts/textToConll.py  \n",
            "   creating: LeNER-Br/leNER-Br/raw_text/\n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/REE5908TSE4.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC04798525420128130000.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Ag10000170733596001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/RR-578030-46.1999.5.04.0018.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/LoaDF2018.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/adi3767.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/RR474820145230056.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/RR942006420095040028.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/DespSEPLAGDF.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AC10024133855890001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ACO2821STF.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ADI1TJDFT.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC110260SP.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AP00000794920137060006.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgAIRR11889820145030011.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AC1TCU.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ERR731004520105130003.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ED1STM.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Port77DF.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AP00001415620157010201.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIAgRAgI6193ARAGUARIMG.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/TSTRR16037920105200001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/EEDRR9715120105020002.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgRgTSE3.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/TCU4687.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC70000692720177000000.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Agr10540170008341001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/CP32320177080008PA.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR15708820115050222.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC10000170833503000.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgCr10582160008758001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/EDAgRgTSE2.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgRgSTJ2.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC70000845920187000000.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ADI2TJDFT.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ACORDAOTCU11602016.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Lei11788.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Rcl3495STJ.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/20150110436469APC.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR3999520145020086.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Pet128TSE5.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/lei11340.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/REsp1583083RS.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ED1TJAC.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgRg1STM.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/airr801422012.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/EDEDARR208420135040232.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR285001420095060020.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/RR2574407120025020372.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AC1TJMG.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC340624SP.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/RR14976020105020085.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC151914AgRES.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgRgSTJ1.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR581406820065030079.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/EDRR1TST.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR3731820145060141.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AC20150110436469APC.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgRgTSE1.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/INSTRUCAOON06043378120186000000.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AP771420167080008PA.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AC1TJAC.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/Ag10105170208398001.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC418951PR.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AIRR10006691020135020322.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AP00001441420167030203.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/AgAIRR617420125150072.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/APO1TJDFT.txt  \n",
            " extracting: LeNER-Br/leNER-Br/raw_text/metadadosAgRgTSE1_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/ACORDAOTCU25052016.txt  \n",
            "  inflating: LeNER-Br/leNER-Br/raw_text/HC10000150589281000.txt  \n",
            "   creating: LeNER-Br/leNER-Br/metadata/\n",
            "  inflating: LeNER-Br/leNER-Br/metadata/20150110436469APC_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/APO1TJDFT_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC10000150589281000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/RR2574407120025020372_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Ag10000170733596001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC340624SP_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ACO2821STF_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/TCU4687_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AC20150110436469APC_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AC1TJAC_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ED1TJAC_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ERR731004520105130003_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC110260SP_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/EDRR1TST_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC151914AgRES_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/RR942006420095040028_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/EDAgRgTSE2_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgAIRR617420125150072_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/RR-578030-46.1999.5.04.0018_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/CP32320177080008PA_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC70000692720177000000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR15708820115050222_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AP00000794920137060006_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Port77DF_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ADI2TJDFT_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Rcl3495STJ_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AC10024133855890001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC418951PR_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgRgSTJ1_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Ag10105170208398001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR581406820065030079_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AC1TJMG_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR3731820145060141_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIAgRAgI6193ARAGUARIMG_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ED1STM_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/adi3767_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/REsp1583083RS_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/EDEDARR208420135040232_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Lei11788_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AC1TCU_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ACORDAOTCU11602016_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC10000170833503000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgRgSTJ2_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR285001420095060020_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR10006691020135020322_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgCr10582160008758001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ADI1TJDFT_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Pet128TSE5_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgRgTSE1_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/DespSEPLAGDF_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC70000845920187000000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AP00001415620157010201_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/INSTRUCAOON06043378120186000000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/RR14976020105020085_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/EEDRR9715120105020002_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/Agr10540170008341001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/lei11340_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/LoaDF2018_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AIRR3999520145020086_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/RR474820145230056_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgRg1STM_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/airr801422012_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/airr8014220212_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AP00001441420167030203_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AP771420167080008PA_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/HC04798525420128130000_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/TSTRR16037920105200001_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgAIRR11889820145030011_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/REE5908TSE4_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/ACORDAOTCU25052016_meta.json  \n",
            "  inflating: LeNER-Br/leNER-Br/metadata/AgRgTSE3_meta.json  \n",
            "   creating: LeNER-Br/leNER-Br/dev/\n",
            "  inflating: LeNER-Br/leNER-Br/dev/AC20150110436469APC.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/INSTRUCAOON06043378120186000000.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/ACO2821STF.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/HC10000170833503000.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/Agr10540170008341001.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/AgRg1STM.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/ADI1TJDFT.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/20150110436469APC.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/APO1TJDFT.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/dev.conll  \n",
            "  inflating: LeNER-Br/leNER-Br/dev/HC70000692720177000000.conll  \n",
            "   creating: LeNER-Br/model/results/prototype_revised/\n",
            "  inflating: LeNER-Br/model/results/prototype_revised/events.out.tfevents.1527043006.pedro-Lenovo-ideapad-320-15IKB  \n",
            "  inflating: LeNER-Br/model/results/prototype_revised/log.txt  \n",
            "   creating: LeNER-Br/model/model/__pycache__/\n",
            "  inflating: LeNER-Br/model/model/__pycache__/__init__.cpython-36.pyc  \n",
            "  inflating: LeNER-Br/model/model/__pycache__/ner_model.cpython-36.pyc  \n",
            "  inflating: LeNER-Br/model/model/__pycache__/general_utils.cpython-36.pyc  \n",
            "  inflating: LeNER-Br/model/model/__pycache__/config.cpython-36.pyc  \n",
            "  inflating: LeNER-Br/model/model/__pycache__/data_utils.cpython-36.pyc  \n",
            "  inflating: LeNER-Br/model/model/__pycache__/base_model.cpython-36.pyc  \n",
            "   creating: LeNER-Br/model/results/prototype_revised/model.weights/\n",
            "  inflating: LeNER-Br/model/results/prototype_revised/model.weights/checkpoint  \n",
            "  inflating: LeNER-Br/model/results/prototype_revised/model.weights/_meta  \n",
            "  inflating: LeNER-Br/model/results/prototype_revised/model.weights/_index  \n",
            "  inflating: LeNER-Br/model/results/prototype_revised/model.weights/_data-00000-of-00001  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "path_to_text_files = '/content/LeNER-Br/leNER-Br/raw_text'\n",
        "\n",
        "p = Path(path_to_text_files).glob('**/*')\n",
        "files = [x for x in p if x.is_file() and x.suffix == '.txt']\n",
        "files"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7vpPzC7rRb2n",
        "outputId": "2db60058-767a-4128-87ab-1e59ca294786"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgAIRR11889820145030011.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgRgTSE3.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR10006691020135020322.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ED1STM.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Pet128TSE5.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC04798525420128130000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Rcl3495STJ.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/20150110436469APC.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ADI1TJDFT.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/RR-578030-46.1999.5.04.0018.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/REE5908TSE4.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AC1TCU.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC418951PR.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AP00001441420167030203.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AC10024133855890001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/TCU4687.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR3731820145060141.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgRgTSE1.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ACORDAOTCU25052016.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/RR942006420095040028.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ERR731004520105130003.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ACO2821STF.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Ag10105170208398001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ADI2TJDFT.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AP771420167080008PA.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgCr10582160008758001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR581406820065030079.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIAgRAgI6193ARAGUARIMG.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR15708820115050222.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC70000845920187000000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Ag10000170733596001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC10000150589281000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/LoaDF2018.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/TSTRR16037920105200001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/EDEDARR208420135040232.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/RR14976020105020085.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AP00001415620157010201.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Lei11788.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/REsp1583083RS.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgRg1STM.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Port77DF.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/lei11340.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgRgSTJ2.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/EDRR1TST.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC110260SP.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AC1TJMG.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/RR474820145230056.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC10000170833503000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/airr801422012.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC70000692720177000000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/adi3767.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/RR2574407120025020372.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgAIRR617420125150072.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AC1TJAC.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/Agr10540170008341001.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/EDAgRgTSE2.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AP00000794920137060006.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/DespSEPLAGDF.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AgRgSTJ1.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AC20150110436469APC.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ACORDAOTCU11602016.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC340624SP.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/EEDRR9715120105020002.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/ED1TJAC.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/CP32320177080008PA.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/APO1TJDFT.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/INSTRUCAOON06043378120186000000.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR3999520145020086.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/AIRR285001420095060020.txt'),\n",
              " PosixPath('/content/LeNER-Br/leNER-Br/raw_text/HC151914AgRES.txt')]"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(files)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "soaLT3m7WbQM",
        "outputId": "b2c3a578-8ea6-47f3-c922-e6c1f1432b81"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "70"
            ]
          },
          "metadata": {},
          "execution_count": 112
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "paragraphs_list = list()\n",
        "\n",
        "for file in files:\n",
        "  paragraphs_by_file_list = list()\n",
        "  with open(file, 'r') as f:\n",
        "    data = f.read()\n",
        "    paragraphs = data.split(\"\\n\\n\")\n",
        "    num = 0\n",
        "    for paragraph in paragraphs:\n",
        "      p = paragraph.strip()\n",
        "      if p != '':\n",
        "        paragraphs_by_file_list.append(p.replace('\\n', ' '))\n",
        "        num += 1\n",
        "  paragraphs_list.extend(paragraphs_by_file_list)"
      ],
      "metadata": {
        "id": "ySs_pWImRQ0r"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "len(paragraphs_list)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C4pHFsBPhOAP",
        "outputId": "f8ea7a48-144b-496e-9c3f-6085fb691e7c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3324"
            ]
          },
          "metadata": {},
          "execution_count": 116
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.DataFrame(paragraphs_list)\n",
        "df.rename(columns={0: 'text'}, inplace=True)\n",
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "i5ur-5tHXUnA",
        "outputId": "338a55ea-ebe6-43bb-86b0-8edca6f88043"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-81ae2c98-9422-4242-a8ca-a3b52baccc61\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A C Ó R D Ã O</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>(7ª Turma)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>GMDAR/NB/LPLM</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>AGRAVO. AGRAVO DE INSTRUMENTO EM RECURSO DE RE...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Vistos, relatados e discutidos estes autos de ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-81ae2c98-9422-4242-a8ca-a3b52baccc61')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-81ae2c98-9422-4242-a8ca-a3b52baccc61 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-81ae2c98-9422-4242-a8ca-a3b52baccc61');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                                text\n",
              "0                                      A C Ó R D Ã O\n",
              "1                                         (7ª Turma)\n",
              "2                                      GMDAR/NB/LPLM\n",
              "3  AGRAVO. AGRAVO DE INSTRUMENTO EM RECURSO DE RE...\n",
              "4  Vistos, relatados e discutidos estes autos de ..."
            ]
          },
          "metadata": {},
          "execution_count": 118
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df['text'].str.split().apply(len).value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1hCFbrJo4bL_",
        "outputId": "68d07ab2-71f4-4dba-bf11-8a2930787758"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1       178\n",
              "2       158\n",
              "3       133\n",
              "4       101\n",
              "6       100\n",
              "       ... \n",
              "980       1\n",
              "934       1\n",
              "2957      1\n",
              "860       1\n",
              "1968      1\n",
              "Name: text, Length: 260, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 119
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "There are files that have not been broken into small paragraphs, but the code that will be applied to the tokenized datasets (train and validation) will concatenate them all and group them into chunks. Therefore, we don't need to deal with this problem."
      ],
      "metadata": {
        "id": "ryeu7moc6-6R"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2. Create train and validation datasets"
      ],
      "metadata": {
        "id": "6t01zjoxacPa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "train, validation = train_test_split(df, test_size=0.2)\n",
        "train.reset_index(drop=True, inplace=True)\n",
        "validation.reset_index(drop=True, inplace=True)"
      ],
      "metadata": {
        "id": "TvJOBsJMagzO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "YReJiEy-aj1p",
        "outputId": "7c3dfde6-a4ab-42cc-e77a-e0a33a63f5d8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-e0fec7e6-ce00-4a43-a616-b7b51b9b221b\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Os honorários periciais inserem-se em um conte...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Com efeito, nos termos do art. 61, §1º, da Lei...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Tendo, com suporte nas razões já demonstradas,...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Neste sentido é o entendimento do Superior Tri...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>E continua: \"por isso é que se diz que a decis...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e0fec7e6-ce00-4a43-a616-b7b51b9b221b')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e0fec7e6-ce00-4a43-a616-b7b51b9b221b button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e0fec7e6-ce00-4a43-a616-b7b51b9b221b');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                                text\n",
              "0  Os honorários periciais inserem-se em um conte...\n",
              "1  Com efeito, nos termos do art. 61, §1º, da Lei...\n",
              "2  Tendo, com suporte nas razões já demonstradas,...\n",
              "3  Neste sentido é o entendimento do Superior Tri...\n",
              "4  E continua: \"por isso é que se diz que a decis..."
            ]
          },
          "metadata": {},
          "execution_count": 121
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset = Dataset.from_pandas(train)\n",
        "validation_dataset = Dataset.from_pandas(validation)"
      ],
      "metadata": {
        "id": "OaXaiMvobk_3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "datasets = DatasetDict()\n",
        "datasets['train'] = train_dataset\n",
        "datasets['validation'] = validation_dataset\n",
        "\n",
        "datasets"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vG3F86I1b1Yc",
        "outputId": "1ff6bf58-63c3-46b4-dfb6-84c0cdcb36c5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['text'],\n",
              "        num_rows: 15252\n",
              "    })\n",
              "    validation: Dataset({\n",
              "        features: ['text'],\n",
              "        num_rows: 3813\n",
              "    })\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 122
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3. Push to the HF dataset hub"
      ],
      "metadata": {
        "id": "IMZEIPoP8x5x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "repo_id = \"pierreguillou/lener_br_finetuning_language_model\"\n",
        "datasets.push_to_hub(repo_id)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134,
          "referenced_widgets": [
            "f439a958357a42599a95731edafb3a81",
            "041bab5a79324ff984011a2339b16c65",
            "83e2859f8b614172a6debb807dc0682b",
            "acbeee60d3f24faf8e57c92aeca7e139",
            "cdf1978d3f41436abe9785efa62d68ab",
            "5cd7d5720e6e409fb029174df66ca1a8",
            "761b4bb246614bf08c06669f2299659f",
            "16891c44a29a40338e0877211e821ffb",
            "1deb78cace124341b24611f739048656",
            "afad5d2ab5ee458ca9d468ac02640430",
            "02962bf0547d44d183d29c058a2b6f99",
            "580521d9157c4c408605ba804b1eec51",
            "d826cf6dcde94a66bc5af8e4f3074402",
            "a04c3c5849974cb49f71274170c6c430",
            "3a094acaa16649f4aa9319c176ede264",
            "7e841dfe9b1c4286ba81714cdf019720",
            "f2190613dadf4e38a0224cf58f04a5f4",
            "9d990bfc0d9b4935bdd827256baca46c",
            "babea81cdcb94fd5af051439283b9b03",
            "604362f48cae4a9f84950ac0a7381609",
            "ffee0f297ba042f28ac1c9dc0c776d13",
            "0a64bf6e8159403db8e2c07a0547e4b6"
          ]
        },
        "id": "rP7656_i82j6",
        "outputId": "40d29c73-4bee-4ebf-ddbb-b83fbd2ed805"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Pushing split train to the Hub.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f439a958357a42599a95731edafb3a81",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Pushing dataset shards to the dataset hub:   0%|          | 0/1 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Pushing split validation to the Hub.\n",
            "The repository already exists: the `private` keyword argument will be ignored.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "580521d9157c4c408605ba804b1eec51",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Pushing dataset shards to the dataset hub:   0%|          | 0/1 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4. Load dataset from dataset HF"
      ],
      "metadata": {
        "id": "_cdm0JLASZkz"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kswRMhPc3l-Q"
      },
      "source": [
        "For each of the following tasks, we will use this dataset that we pushed to the HF library. You can load it very easily with the 🤗 Datasets library."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "n2ZRs1cL3l-R",
        "outputId": "70f58d06-21e3-4cac-b43a-f62c6162e018",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281,
          "referenced_widgets": [
            "d1e641482a464922a778ed8115ac4f1b",
            "b0b1fcf6851745cf8487c5a654823fc4",
            "a139915979c1407685297e764b5fa5f1",
            "6be15030dfaf4a2a9ffc250c6562c48a",
            "fcd52074285f45b0872e5b6d097682df",
            "70185f713c2848f7b066bf5f93c7e985",
            "ec5595ffdf764d4da0f5fe729cc165f2",
            "816e525357b54ecf9ac0a9fad459d7e1",
            "30374b889a7c422a8e32560a55981f62",
            "3ead20d74bdc4e4d8c109e969da3d90f",
            "d7d25b6ca08f493087d5d2579b24c2f0",
            "443bb06723b6412db643ebf0bc6d0cb5",
            "92fcb3cffed04c8ba9d3b8d340a4f2b8",
            "1cbfa06185e544129e095fd4bae8151b",
            "e8009ffcf1fe41d3bdf9b2f04c92730a",
            "62bd8e4d113147d4b3b1418c73e6fd0f",
            "6883b1b37763401b8e614797c3ed1e3a",
            "2bbbc239d15441d29df32eaecc8212c0",
            "ce8b12b58beb443ab8899f2fed5987a9",
            "b3097f92e8e743f6967314f90b4bd41e",
            "f909938b793e4fd1bd7d8442d683e851",
            "e5574543d50a46c2afdd7a1f5ef451eb",
            "499e4a474c654bd6bd3b7aaa2d7ce295",
            "7ab71e613b7c42eba3e3a1307e9f404e",
            "ee8fc115179e4909a9a0e95de6ac520d",
            "16160687fc5f4b17bad0a1136538f6b1",
            "dfcab1a404db4a43a9423a1ca2d667a4",
            "ab819d5c5c6e4ce2b7c9a7386e36671e",
            "410a7d57fe854f8e88a454583a8752f6",
            "a2aba3ccbcee4766a21d4d2ad55155a4",
            "6a6b0f4302064fe9ab669cd9172f597f",
            "212b10a9dd724e66b0c74044e6b3e015",
            "a6e7f6b2ac3a40b4afe0f94974fee4aa",
            "d087107576c54ba68e062f256eac5631",
            "1b078ab25caa4bf0a0a44890bcae784a",
            "35bfee597f4a48df9d25836a56edee61",
            "d946534b5db443988c4a6e250a99cc68",
            "49b077926f5b4145a65343075d53d2c1",
            "3e7a1ed826f547ec9fb76a57264d6e41",
            "2d4195e80ff1463a97678decb138df78",
            "dae471ebb0be46dbbff8fb69099fefd4",
            "632b7e8397a94ce5818e651112884f55",
            "504a9cf2dd4343e3ba9635ff27b2b1f7",
            "b3d21a855ee84a26bb7858cfb54cec3e",
            "4d014648da7e475588f51c0147b6a01c",
            "77f36c11561e4287895d34b6d1e709c7",
            "a8d642040b8740b18bbe1c2c3c89714f",
            "36984dbb1df045a897f6b9401b5a35b4",
            "2222c6fc1f1d4a52a4e77abe35811175",
            "29f047c7197a4cbfbc174ab526e050ec",
            "8e350b8c117d4326b52f1e6e4d1761d8",
            "419f16e436714af380d8b47b1706e2af",
            "06f87e207c504946a20ae61213441ceb",
            "a8d3a03b6dc8455e998c61875aa4bf5e",
            "229b74f4acd24fda886fef7d20f427c7",
            "17abfba35d7d4d9da747d93420860468",
            "f2f0ea5852c1406ab35cee2499486d4b",
            "dbe41b751bd243ffb87b9fb2bc0a03e2",
            "272c7f8e8ae64026b3b1532bf2b41044",
            "187d2ecf1ff34d2bb4a3c81c59ad312d",
            "8a16286a769b48fca716536e407413df",
            "bb5f5d80eb554f23900354a6b8676cb1",
            "c229215247104322bd8a53e03e03c7be",
            "2cd6cdc631ed423c823739254bd8041a",
            "62afdb943c14427d9430279124120a3d",
            "a877227e8a35483281d53922e8217ed2"
          ]
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d1e641482a464922a778ed8115ac4f1b",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/736 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Using custom data configuration pierreguillou--lener_br_finetuning_language_model-d5d35d543fec7e31\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading and preparing dataset None/None (download: 1.11 MiB, generated: 1.79 MiB, post-processed: Unknown size, total: 2.90 MiB) to /root/.cache/huggingface/datasets/parquet/pierreguillou--lener_br_finetuning_language_model-d5d35d543fec7e31/0.0.0/1638526fd0e8d960534e2155dc54fdff8dce73851f21f031d2fb9c2cf757c121...\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "443bb06723b6412db643ebf0bc6d0cb5",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "  0%|          | 0/2 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "499e4a474c654bd6bd3b7aaa2d7ce295",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/243k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d087107576c54ba68e062f256eac5631",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/925k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4d014648da7e475588f51c0147b6a01c",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "  0%|          | 0/2 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Dataset parquet downloaded and prepared to /root/.cache/huggingface/datasets/parquet/pierreguillou--lener_br_finetuning_language_model-d5d35d543fec7e31/0.0.0/1638526fd0e8d960534e2155dc54fdff8dce73851f21f031d2fb9c2cf757c121. Subsequent calls will reuse this data.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "17abfba35d7d4d9da747d93420860468",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "  0%|          | 0/2 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "repo_id = \"pierreguillou/lener_br_finetuning_language_model\"\n",
        "datasets = load_dataset(repo_id)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "datasets"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AjrC2Fu1-lDl",
        "outputId": "6f5a8fe3-0bb3-4005-9fcd-8b02f0cde4ca"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    validation: Dataset({\n",
              "        features: ['text'],\n",
              "        num_rows: 3813\n",
              "    })\n",
              "    train: Dataset({\n",
              "        features: ['text'],\n",
              "        num_rows: 15252\n",
              "    })\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f1-9jepM3l-W"
      },
      "source": [
        "You can replace the dataset above with any dataset hosted on [the hub](https://huggingface.co/datasets) or use your own files. Just uncomment the following cell and replace the paths with values that will lead to your files:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uxSaGa_l3l-W"
      },
      "outputs": [],
      "source": [
        "# datasets = load_dataset(\"text\", data_files={\"train\": path_to_train.txt, \"validation\": path_to_validation.txt}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jY1SwIrY3l-a"
      },
      "source": [
        "You can also load datasets from a csv or a JSON file, see the [full documentation](https://huggingface.co/docs/datasets/loading_datasets.html#from-local-files) for more information."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u3EtYfeHIrIz"
      },
      "source": [
        "To access an actual element, you need to select a split first, then give an index:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "X6HrpprwIrIz",
        "outputId": "dab02e6d-e35b-4dce-b9aa-433e6385bbea",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'text': 'Branco-AC - Mod. 500258 - Autos n.º 1002199-81.2017.8.01.0000/50000'}"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "datasets[\"train\"][10]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHUmphG3IrI3"
      },
      "source": [
        "To get a sense of what the data looks like, the following function will show some examples picked randomly in the dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "ur5sNUcZ3l-g"
      },
      "outputs": [],
      "source": [
        "from datasets import ClassLabel\n",
        "import random\n",
        "import pandas as pd\n",
        "from IPython.display import display, HTML\n",
        "\n",
        "def show_random_elements(dataset, num_examples=10):\n",
        "    assert num_examples <= len(dataset), \"Can't pick more elements than there are in the dataset.\"\n",
        "    picks = []\n",
        "    for _ in range(num_examples):\n",
        "        pick = random.randint(0, len(dataset)-1)\n",
        "        while pick in picks:\n",
        "            pick = random.randint(0, len(dataset)-1)\n",
        "        picks.append(pick)\n",
        "    \n",
        "    df = pd.DataFrame(dataset[picks])\n",
        "    for column, typ in dataset.features.items():\n",
        "        if isinstance(typ, ClassLabel):\n",
        "            df[column] = df[column].transform(lambda i: typ.names[i])\n",
        "    display(HTML(df.to_html()))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "1Uk8NROQ3l-k",
        "outputId": "29a4998d-1928-424a-e043-3d9c8f43c078",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 362
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>documento pode ser acessado no endereço eletrônico http://www.stf.jus.br/portal/autenticacao/ sob o número 3373814.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>por ele constituída, \"(...) no caso em tela, esta Impetrante NÃO FOI INTIMADA, E DECLARA QUE NÃO</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>\"eventual questão atinente à existência de coligação exige instrução, o que não</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>BELMIRO CONSELHO ESPECIAL, Data de Julgamento:</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Documento assinado digitalmente conforme MP n° 2.200-2/2001 de 24/08/2001, que institui a Infraestrutura de Chaves Públicas Brasileira - ICP-Brasil. O</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>R$ 1.243.372,38 (9,09%) para R$ 161.340,49 (1,18%) o total</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>Endereço: Rua Tribunal de Justiça, s/n, Via Verde, CEP 69.915-631, Tel. 68 3302-0444/0445, Rio BrancoAC</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>[ACÓRDÃO]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>Não procede o inconformismo, uma vez que não se pode fazer distinção, para aplicação do direito, entre servidor público estatutário e servidor com contrato regido pela CLT.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>Social teriam extrapolado a previsão legal, criando novos</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "show_random_elements(datasets[\"train\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CKerdF353l-o"
      },
      "source": [
        "As we can see, some of the texts are a full paragraph of a Wikipedia article while others are just titles or empty lines."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q-EIELH43l_T"
      },
      "source": [
        "## Masked language modeling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LWk97-Ny3l_T"
      },
      "source": [
        "For masked language modeling (MLM) we are going to use the same preprocessing as before for our dataset with one additional step: we will randomly mask some tokens (by replacing them by `[MASK]`) and the labels will be adjusted to only include the masked tokens (we don't have to predict the non-masked tokens).\n",
        "\n",
        "We will use the [`neuralmind/bert-base-portuguese-cased`](https://huggingface.co/neuralmind/bert-base-portuguese-cased) model for this example. You can pick any of the checkpoints listed [here](https://huggingface.co/models?filter=masked-lm) instead:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tSLSNNRwBaiC"
      },
      "source": [
        "To tokenize all our texts with the same vocabulary that was used when training the model, we have to download a pretrained tokenizer. This is all done by the `AutoTokenizer` class:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 177,
          "referenced_widgets": [
            "3fecdd00e01844bb9c5b10898f2c6989",
            "77d63edeef4d41ce9827d8d365bf38e9",
            "f55186d7194e445989bc46f78419e6c7",
            "9c023102720d40308d71f5c9b4ef441e",
            "60c23f9c82bd48ef85f7e229043add93",
            "866816c453eb43dab279f4006a067e61",
            "05133cbc9b304d9d89d3d8230539f8ff",
            "ab23ccf5fc484b259a51ea72018e4e54",
            "407f4f3825c64539ace6bedd40983136",
            "20a79c7e998a429e8c9cd06117fe3b3e",
            "b426338603734e2e9168145dc6235ec2",
            "91a92d0239c9490d85c817a2817e94fb",
            "503ba250c37048f89c87de4999306a56",
            "ca7a05a0817340269aa07a3318604c25",
            "d6f19b238b26419baec5e9ea8074ed28",
            "0304976c59dc4884931f9c8fd554e4ef",
            "97fd5f372d6c4541bc223d177d3c06fa",
            "4697ddb9202f415393f09e4310b55d30",
            "9ad98a1c73b5429cab6b9defc137034e",
            "facdcc2ed3fb4bd9a60336c99d49bff3",
            "c143e4e99a114192b85a87c7979986c8",
            "948e7729a94947b3882dfe851a6d543a",
            "ec4cc75cfa0648328db7ddb52c58d16a",
            "62b3c7c55c5d4ee993475a3d1bb6e4b8",
            "5120bc6a2514400b87a772bab21a2ecb",
            "ebe9376735af4e8db56aeada7d24cce5",
            "965e3b33476f48dd90695f3cab8ca347",
            "90793a87cddd46b39f689736e718ea72",
            "f1620195897445b7b51c737fd91d01db",
            "3a35c5f878014cbda3ba66ba492c53fd",
            "bd4151195be04fe4b7a2f2fdcae37ac9",
            "833fa044713d4aa3963da622346ffacc",
            "3f04008d0d854f99b08fc70342701352",
            "7af5044f6d3c443486a5707036aa3fd8",
            "33af9bb7755b478f893c5cf4c81752b5",
            "49af0e707aa0490cb6fd758bfade4274",
            "1cb25572218b4eebb63030a137e0eaab",
            "d61c86c609f344eeba179528e93022dd",
            "513e0af2a83d4c1f808d6b8369e81654",
            "4d9ebb35fb824b6b87d18267954b9b6c",
            "81ca8599c53f4e79bf1f09d22ef5d0ee",
            "59c1eeb382464e97bb31ef369a725238",
            "e8448a95012a43f2a8c5479357552507",
            "ea5651ec85d34c8b95b63eb4b4638d0a",
            "2f4800d141e147278c2ce6e6ba279133",
            "98d6339848784f6989bd3bf9a281c4f6",
            "3752e596c8324f789c0eb1e7a94bdc9e",
            "45a4729c2804406d933daaf52da6aaca",
            "6f1db82111de481aa4e770953e66b328",
            "13ed5df100654b6593707c042a8d5bc0",
            "49524c3a50ab4bafa2aba29921b45dbb",
            "f67a133e1cf14b949b8eb49589edefa5",
            "3d6734c99e6a48da9f452f0d6bc43d50",
            "77de2bf6f6754ade895f1657d7541bdf",
            "76c9c66e5f74422ca36fc4990d3e947e"
          ]
        },
        "id": "Y4Kaa_3ZBaiC",
        "outputId": "407b7ef1-8684-49fb-fc3b-256323f9299f"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3fecdd00e01844bb9c5b10898f2c6989",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/43.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "91a92d0239c9490d85c817a2817e94fb",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/647 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ec4cc75cfa0648328db7ddb52c58d16a",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/205k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "7af5044f6d3c443486a5707036aa3fd8",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/2.00 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2f4800d141e147278c2ce6e6ba279133",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/112 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "from transformers import AutoTokenizer\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_checkpoint, use_fast=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IpRjQ7rlBaiD"
      },
      "source": [
        "We can now call the tokenizer on all our texts. This is very simple, using the [`map`](https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.map) method from the Datasets library. First we define a function that call the tokenizer on our texts:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "8K98HDIfBaiD"
      },
      "outputs": [],
      "source": [
        "def tokenize_function(examples):\n",
        "    return tokenizer(examples[\"text\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m2DNlIboBaiD"
      },
      "source": [
        "Then we apply it to all the splits in our `datasets` object, using `batched=True` and 4 processes to speed up the preprocessing. We won't need the `text` column afterward, so we discard it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "3IQkPf6qBaiD"
      },
      "outputs": [],
      "source": [
        "tokenized_datasets = datasets.map(tokenize_function, batched=True, num_proc=4, remove_columns=[\"text\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GE6vS3MZBaiE"
      },
      "source": [
        "If we now look at an element of our datasets, we will see the text have been replaced by the `input_ids` the model will need:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "outputId": "6dbc56b0-14e7-4c4c-8f5e-bbd59a4f47f1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QK6xfCWrBaiE"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],\n",
              " 'input_ids': [101,\n",
              "  3305,\n",
              "  293,\n",
              "  5576,\n",
              "  179,\n",
              "  117,\n",
              "  5776,\n",
              "  285,\n",
              "  123,\n",
              "  20165,\n",
              "  182,\n",
              "  125,\n",
              "  102],\n",
              " 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "tokenized_datasets[\"train\"][1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y03VOOXcBaiF"
      },
      "source": [
        "Now for the harder part: we need to concatenate all our texts together then split the result in small chunks of a certain `block_size`. To do this, we will use the `map` method again, with the option `batched=True`. This option actually lets us change the number of examples in the datasets by returning a different number of examples than we got. This way, we can create our new samples from a batch of examples.\n",
        "\n",
        "First, we grab the maximum length our model was pretrained with. This might be a big too big to fit in your GPU RAM, so here we take a bit less at just 128."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "F0-ApCdUBaiF"
      },
      "outputs": [],
      "source": [
        "# block_size = tokenizer.model_max_length\n",
        "block_size = 128"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zPE9-auaBaiG"
      },
      "source": [
        "Then we write the preprocessing function that will group our texts:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "LqP-xQFFBaiG"
      },
      "outputs": [],
      "source": [
        "def group_texts(examples):\n",
        "    # Concatenate all texts.\n",
        "    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}\n",
        "    total_length = len(concatenated_examples[list(examples.keys())[0]])\n",
        "    # We drop the small remainder, we could add padding if the model supported it instead of this drop, you can\n",
        "        # customize this part to your needs.\n",
        "    total_length = (total_length // block_size) * block_size\n",
        "    # Split by chunks of max_len.\n",
        "    result = {\n",
        "        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]\n",
        "        for k, t in concatenated_examples.items()\n",
        "    }\n",
        "    result[\"labels\"] = result[\"input_ids\"].copy()\n",
        "    return result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kbGygrDdBaiH"
      },
      "source": [
        "First note that we duplicate the inputs for our labels. This is because the model of the 🤗 Transformers library apply the shifting to the right, so we don't need to do it manually.\n",
        "\n",
        "Also note that by default, the `map` method will send a batch of 1,000 examples to be treated by the preprocessing function. So here, we will drop the remainder to make the concatenated tokenized texts a multiple of `block_size` every 1,000 examples. You can adjust this behavior by passing a higher batch size (which will also be processed slower). You can also speed-up the preprocessing by using multiprocessing:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "J6FKBm5EBaiH"
      },
      "outputs": [],
      "source": [
        "lm_datasets = tokenized_datasets.map(\n",
        "    group_texts,\n",
        "    batched=True,\n",
        "    batch_size=1000,\n",
        "    num_proc=4,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WBp_jkbGBaiI"
      },
      "source": [
        "And we can check our datasets have changed: now the samples contain chunks of `block_size` contiguous tokens, potentially spanning over several of our original texts."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "outputId": "26d99a30-d315-4000-861b-fc590b212c3b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        },
        "id": "Yk0NjF54BaiI"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'o eminente Relator, pedindo respeitosas vênias à divergência. [SEP] [CLS] 5. Instruído o feito, a Unidade Técnica apresentou proposta final de encaminhamento acorde, que, nos termos do inciso I, § [UNK] do art. [UNK] da Lei [UNK] 8. 443 / 92 transcrevo ( Peças 15 / 16 ) : [SEP] [CLS] qualquer outro cadastro de inadimplentes pelo mesmo motivo [SEP] [CLS] Presidência da República [SEP] [CLS] Branco - AC - Mod. 500258 - Autos n. [UNK] 1002199 - 81. 2017. 8. 01. 0000 / 50000 [SEP] [CLS]'"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "tokenizer.decode(lm_datasets[\"train\"][1][\"input_ids\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ohaK7GcBaiJ"
      },
      "source": [
        "Now that the data has been cleaned, we're ready to instantiate our `Trainer`. irst we use a model suitable for masked LM:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "PM10A9Za3l_Z",
        "outputId": "23437dff-d9d6-4066-867d-92e0cccd3e60",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121,
          "referenced_widgets": [
            "e37a7eba1c22461a852708824efd855f",
            "731079479e9d40b28f3688dd5a3cd9d7",
            "e34fa275bc6844ceb27cc46aa49f6cf7",
            "b6d585d3bb7345509bf636f385fa482b",
            "1ab4971a73974734a5af17766716a01e",
            "5df5740537c841d3af8b5766eac12d5e",
            "f7b11fe83d264494a9fd4de982d9ba46",
            "f9560e17863747408a6191efd55f997f",
            "363529e3db0b4bb39fe75fae19fa0877",
            "2cc6826828ed49acbe2be54230be6127",
            "00c4fc42223340b089f69c4da85baaf5"
          ]
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e37a7eba1c22461a852708824efd855f",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/418M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the model checkpoint at neuralmind/bert-base-portuguese-cased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias']\n",
            "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
          ]
        }
      ],
      "source": [
        "from transformers import AutoModelForMaskedLM\n",
        "model = AutoModelForMaskedLM.from_pretrained(model_checkpoint)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ijhRRM4yHvpM"
      },
      "source": [
        "And some `TrainingArguments`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "SQM3-rvoHvpN"
      },
      "outputs": [],
      "source": [
        "from transformers import Trainer, TrainingArguments"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# hyperparameters, which are passed into the training job\n",
        "\n",
        "per_device_batch_size = 8\n",
        "gradient_accumulation_steps = 1\n",
        "\n",
        "learning_rate = 2e-5 # (AdamW) we started with 3e-4, then 1e-4, then 5e-5 but the model overfits fastly\n",
        "num_train_epochs = 5 # we started with 10 epochs but the model overfits fastly\n",
        "weight_decay = 0.01\n",
        "\n",
        "save_total_limit = 2\n",
        "logging_steps = 100 # melhor evaluate frequently (5000 seems too high)\n",
        "eval_steps = logging_steps\n",
        "evaluation_strategy = 'steps'\n",
        "logging_strategy = 'steps'\n",
        "save_strategy = 'steps'\n",
        "save_steps = logging_steps\n",
        "load_best_model_at_end = True\n",
        "\n",
        "fp16 = True\n",
        "\n",
        "# folders\n",
        "model_name = model_checkpoint.split(\"/\")[-1]\n",
        "folder_model = 'e' + str(num_train_epochs) + '_lr' + str(learning_rate)\n",
        "output_dir = '/content/drive/MyDrive/' + 'lm-lenerbr-' + str(model_name) + '/checkpoints/' + folder_model\n",
        "logging_dir = '/content/drive/MyDrive/' + 'lm-lenerbr-' + str(model_name) + '/logs/' + folder_model\n",
        "\n",
        "# get best model through a metric\n",
        "metric_for_best_model = 'eval_loss'\n",
        "if metric_for_best_model == 'eval_f1':\n",
        "    greater_is_better = True\n",
        "elif metric_for_best_model == 'eval_loss':\n",
        "    greater_is_better = False  \n",
        "\n",
        "training_args = TrainingArguments(\n",
        "    output_dir=output_dir,\n",
        "    learning_rate=learning_rate,\n",
        "    per_device_train_batch_size=per_device_batch_size,\n",
        "    per_device_eval_batch_size=per_device_batch_size*2,\n",
        "    gradient_accumulation_steps=gradient_accumulation_steps,\n",
        "    num_train_epochs=num_train_epochs,\n",
        "    weight_decay=weight_decay,\n",
        "    save_total_limit=save_total_limit,\n",
        "    logging_steps = logging_steps,\n",
        "    eval_steps = logging_steps,\n",
        "    load_best_model_at_end = load_best_model_at_end,\n",
        "    metric_for_best_model = metric_for_best_model,\n",
        "    greater_is_better = greater_is_better,\n",
        "    gradient_checkpointing = False,\n",
        "    do_train = True,\n",
        "    do_eval = True,\n",
        "    do_predict = True,\n",
        "    evaluation_strategy = evaluation_strategy,\n",
        "    logging_dir=logging_dir, \n",
        "    logging_strategy = logging_strategy,\n",
        "    save_strategy = save_strategy,\n",
        "    save_steps = save_steps,\n",
        "    fp16 = fp16,\n",
        "    push_to_hub=False,\n",
        ")"
      ],
      "metadata": {
        "id": "R5fbf2beG2XO"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z6uuUnvz3l_b"
      },
      "source": [
        "Finally, we use a special `data_collator`. The `data_collator` is a function that is responsible of taking the samples and batching them in tensors. In the previous example, we had nothing special to do, so we just used the default for this argument. Here we want to do the random-masking. We could do it as a pre-processing step (like the tokenization) but then the tokens would always be masked the same way at each epoch. By doing this step inside the `data_collator`, we ensure this random masking is done in a new way each time we go over the data.\n",
        "\n",
        "To do this masking for us, the library provides a `DataCollatorForLanguageModeling`. We can adjust the probability of the masking:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "nRZ-5v_P3l_b"
      },
      "outputs": [],
      "source": [
        "from transformers import DataCollatorForLanguageModeling\n",
        "data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm_probability=0.15)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqHnWcYC3l_d"
      },
      "source": [
        "Then we just have to pass everything to `Trainer` and begin training:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "V-Y3gNqV3l_d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "53c1c248-8a4a-4dbe-a071-a6e70b7c64f0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Using amp half precision backend\n"
          ]
        }
      ],
      "source": [
        "from transformers.trainer_callback import EarlyStoppingCallback\n",
        "\n",
        "# wait early_stopping_patience x eval_steps before to stp the training in order to get a better model\n",
        "early_stopping_patience = save_total_limit\n",
        "\n",
        "trainer = Trainer(\n",
        "    model=model,\n",
        "    args=training_args,\n",
        "    train_dataset=lm_datasets[\"train\"],\n",
        "    eval_dataset=lm_datasets[\"validation\"],\n",
        "    data_collator=data_collator,\n",
        "    callbacks=[EarlyStoppingCallback(early_stopping_patience=early_stopping_patience)],\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "Y9TFqDG_3l_e",
        "outputId": "fe05b164-4a93-4b4a-bb90-e3dda21d3730",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running training *****\n",
            "  Num examples = 3227\n",
            "  Num Epochs = 5\n",
            "  Instantaneous batch size per device = 8\n",
            "  Total train batch size (w. parallel, distributed & accumulation) = 8\n",
            "  Gradient Accumulation steps = 1\n",
            "  Total optimization steps = 2020\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='1100' max='2020' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [1100/2020 25:01 < 20:58, 0.73 it/s, Epoch 2/5]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>1.988700</td>\n",
              "      <td>1.616412</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>200</td>\n",
              "      <td>1.724900</td>\n",
              "      <td>1.561100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>300</td>\n",
              "      <td>1.713400</td>\n",
              "      <td>1.499991</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>400</td>\n",
              "      <td>1.687400</td>\n",
              "      <td>1.451414</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>1.579700</td>\n",
              "      <td>1.433665</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>600</td>\n",
              "      <td>1.556900</td>\n",
              "      <td>1.407338</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>700</td>\n",
              "      <td>1.591400</td>\n",
              "      <td>1.421942</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>800</td>\n",
              "      <td>1.546000</td>\n",
              "      <td>1.406395</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>900</td>\n",
              "      <td>1.510100</td>\n",
              "      <td>1.352389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1000</td>\n",
              "      <td>1.507100</td>\n",
              "      <td>1.394799</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1100</td>\n",
              "      <td>1.462200</td>\n",
              "      <td>1.368093</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-100\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-100/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-100/pytorch_model.bin\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-200\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-200/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-200/pytorch_model.bin\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-300\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-300/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-300/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-100] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-400\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-400/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-400/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-200] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-500\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-500/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-500/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-300] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-600\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-600/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-600/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-400] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-700\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-700/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-700/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-500] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-800\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-800/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-800/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-600] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-900\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-900/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-900/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-700] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1000\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1000/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1000/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-800] due to args.save_total_limit\n",
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n",
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1100\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1100/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1100/pytorch_model.bin\n",
            "Deleting older checkpoint [/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-1000] due to args.save_total_limit\n",
            "\n",
            "\n",
            "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
            "\n",
            "\n",
            "Loading best model from /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/checkpoints/e5_lr2e-05/checkpoint-900 (score: 1.3523892164230347).\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=1100, training_loss=1.6243395579944957, metrics={'train_runtime': 1502.8827, 'train_samples_per_second': 10.736, 'train_steps_per_second': 1.344, 'total_flos': 578387661603840.0, 'train_loss': 1.6243395579944957, 'epoch': 2.72})"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KDBi0reX3l_g"
      },
      "source": [
        "Like before, we can evaluate our model on the validation set. The perplexity is much lower than for the CLM objective because for the MLM objective, we only have to make predictions for the masked tokens (which represent 15% of the total here) while having access to the rest of the tokens. It's thus an easier task for the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "4hSaANqj3l_g",
        "outputId": "1573d08f-baf5-4c85-a31d-1fd3755f871f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 106
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "***** Running Evaluation *****\n",
            "  Num examples = 826\n",
            "  Batch size = 16\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='52' max='52' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [52/52 00:30]\n",
              "    </div>\n",
              "    "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Perplexity: 4.11\n"
          ]
        }
      ],
      "source": [
        "import math\n",
        "eval_results = trainer.evaluate()\n",
        "print(f\"Perplexity: {math.exp(eval_results['eval_loss']):.2f}\")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# save best model\n",
        "model_dir = '/content/drive/MyDrive/' + 'lm-lenerbr-' + str(model_name) + '/model/'\n",
        "trainer.save_model(model_dir)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r0L1AqpvSt7m",
        "outputId": "33e98246-0651-4c9e-f573-56ff53a1a476"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Saving model checkpoint to /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/\n",
            "Configuration saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/config.json\n",
            "Model weights saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/pytorch_model.bin\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# save tokenizer\n",
        "tokenizer.save_pretrained(model_dir)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CDxm3Sq0vmWc",
        "outputId": "fe972e41-146b-4832-ca18-24a322a8e495"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "tokenizer config file saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/tokenizer_config.json\n",
            "Special tokens file saved in /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/special_tokens_map.json\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/tokenizer_config.json',\n",
              " '/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/special_tokens_map.json',\n",
              " '/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/vocab.txt',\n",
              " '/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/added_tokens.json',\n",
              " '/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/tokenizer.json')"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Push model to HF model hub"
      ],
      "metadata": {
        "id": "S2kjtrGpwi0i"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "o3so0BOsQGxd"
      },
      "outputs": [],
      "source": [
        "# trainer.push_to_hub()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# load best model\n",
        "from transformers import AutoModelForMaskedLM\n",
        "model = AutoModelForMaskedLM.from_pretrained(model_dir)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zg8ZFavBwBel",
        "outputId": "2585a65e-c22c-43c8-943a-687eadee491e"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "loading configuration file /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/config.json\n",
            "Model config BertConfig {\n",
            "  \"_name_or_path\": \"/content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/\",\n",
            "  \"architectures\": [\n",
            "    \"BertForMaskedLM\"\n",
            "  ],\n",
            "  \"attention_probs_dropout_prob\": 0.1,\n",
            "  \"classifier_dropout\": null,\n",
            "  \"directionality\": \"bidi\",\n",
            "  \"hidden_act\": \"gelu\",\n",
            "  \"hidden_dropout_prob\": 0.1,\n",
            "  \"hidden_size\": 768,\n",
            "  \"initializer_range\": 0.02,\n",
            "  \"intermediate_size\": 3072,\n",
            "  \"layer_norm_eps\": 1e-12,\n",
            "  \"max_position_embeddings\": 512,\n",
            "  \"model_type\": \"bert\",\n",
            "  \"num_attention_heads\": 12,\n",
            "  \"num_hidden_layers\": 12,\n",
            "  \"output_past\": true,\n",
            "  \"pad_token_id\": 0,\n",
            "  \"pooler_fc_size\": 768,\n",
            "  \"pooler_num_attention_heads\": 12,\n",
            "  \"pooler_num_fc_layers\": 3,\n",
            "  \"pooler_size_per_head\": 128,\n",
            "  \"pooler_type\": \"first_token_transform\",\n",
            "  \"position_embedding_type\": \"absolute\",\n",
            "  \"torch_dtype\": \"float32\",\n",
            "  \"transformers_version\": \"4.15.0\",\n",
            "  \"type_vocab_size\": 2,\n",
            "  \"use_cache\": true,\n",
            "  \"vocab_size\": 29794\n",
            "}\n",
            "\n",
            "loading weights file /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/pytorch_model.bin\n",
            "All model checkpoint weights were used when initializing BertForMaskedLM.\n",
            "\n",
            "All the weights of BertForMaskedLM were initialized from the model checkpoint at /content/drive/MyDrive/lm-lenerbr-bert-base-portuguese-cased/model/.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForMaskedLM for predictions without further training.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# push model and tokenizer to HF model hub\n",
        "if model_checkpoint == \"neuralmind/bert-base-portuguese-cased\":\n",
        "  model.push_to_hub('pierreguillou/bert-base-cased-pt-lenerbr')\n",
        "  tokenizer.push_to_hub('pierreguillou/bert-base-cased-pt-lenerbr')\n",
        "else:\n",
        "  model.push_to_hub('pierreguillou/bert-large-cased-pt-lenerbr')\n",
        "  tokenizer.push_to_hub('pierreguillou/bert-large-cased-pt-lenerbr')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 274,
          "referenced_widgets": [
            "06f4a1e636254773b57d1aada679ccf9",
            "fec32d21df2b47d38dda6c294a7cda50",
            "543472090b00438c87b31b31b21b55b6",
            "50fcc449f3e74bd4a15d5ce19eef364a",
            "002e2503a16b4e87bc3d5e6bcbe01a2f",
            "36fea090b6704ce0adaf00a882c298e0",
            "95fa2f2d92074b22a9592d1fa4e2446e",
            "11cef1fda8e24986a164a41de13d21d8",
            "d360f32f045047b08640c54fdc394f3e",
            "c921d6b324a748999682acf00812f660",
            "6343a86b3da94c78a2f4a4fc270a7676"
          ]
        },
        "id": "PQsFYv93u7o9",
        "outputId": "2866e195-2ac7-4854-89a5-4072c101fb44"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/huggingface_hub/hf_api.py:726: FutureWarning: `create_repo` now takes `token` as an optional positional argument. Be sure to adapt your code!\n",
            "  FutureWarning,\n",
            "Cloning https://huggingface.co/pierreguillou/bert-base-cased-pt-lenerbr into local empty directory.\n",
            "Configuration saved in pierreguillou/bert-base-cased-pt-lenerbr/config.json\n",
            "Model weights saved in pierreguillou/bert-base-cased-pt-lenerbr/pytorch_model.bin\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "06f4a1e636254773b57d1aada679ccf9",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "Upload file pytorch_model.bin:   0%|          | 3.37k/416M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "To https://huggingface.co/pierreguillou/bert-base-cased-pt-lenerbr\n",
            "   58f9e9a..d906a41  main -> main\n",
            "\n",
            "tokenizer config file saved in pierreguillou/bert-base-cased-pt-lenerbr/tokenizer_config.json\n",
            "Special tokens file saved in pierreguillou/bert-base-cased-pt-lenerbr/special_tokens_map.json\n",
            "To https://huggingface.co/pierreguillou/bert-base-cased-pt-lenerbr\n",
            "   d906a41..786c3cf  main -> main\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CRABNSNEQGxe"
      },
      "source": [
        "You can now share this model with all your friends, family, favorite pets: they can all load it with the identifier `\"your-username/the-name-you-picked\"` so for instance:\n",
        "\n",
        "```python\n",
        "from transformers import AutoModelForMaskedLM\n",
        "\n",
        "model = AutoModelForMaskedLM.from_pretrained(\"sgugger/my-awesome-model\")\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# END"
      ],
      "metadata": {
        "id": "NsKsltmUIVRV"
      }
    }
  ],
  "metadata": {
    "colab": {
      "name": "Finetuning_language_model_BERtimbau_LeNER_Br",
      "provenance": [],
      "collapsed_sections": []
    },
    "accelerator": "GPU",
    "language_info": {
      "name": "python"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "f439a958357a42599a95731edafb3a81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_041bab5a79324ff984011a2339b16c65",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_83e2859f8b614172a6debb807dc0682b",
              "IPY_MODEL_acbeee60d3f24faf8e57c92aeca7e139",
              "IPY_MODEL_cdf1978d3f41436abe9785efa62d68ab"
            ]
          }
        },
        "041bab5a79324ff984011a2339b16c65": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "83e2859f8b614172a6debb807dc0682b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_5cd7d5720e6e409fb029174df66ca1a8",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Pushing dataset shards to the dataset hub: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_761b4bb246614bf08c06669f2299659f"
          }
        },
        "acbeee60d3f24faf8e57c92aeca7e139": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_16891c44a29a40338e0877211e821ffb",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_1deb78cace124341b24611f739048656"
          }
        },
        "cdf1978d3f41436abe9785efa62d68ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_afad5d2ab5ee458ca9d468ac02640430",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [00:03&lt;00:00,  3.24s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_02962bf0547d44d183d29c058a2b6f99"
          }
        },
        "5cd7d5720e6e409fb029174df66ca1a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "761b4bb246614bf08c06669f2299659f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "16891c44a29a40338e0877211e821ffb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "1deb78cace124341b24611f739048656": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "afad5d2ab5ee458ca9d468ac02640430": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "02962bf0547d44d183d29c058a2b6f99": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "580521d9157c4c408605ba804b1eec51": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_d826cf6dcde94a66bc5af8e4f3074402",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_a04c3c5849974cb49f71274170c6c430",
              "IPY_MODEL_3a094acaa16649f4aa9319c176ede264",
              "IPY_MODEL_7e841dfe9b1c4286ba81714cdf019720"
            ]
          }
        },
        "d826cf6dcde94a66bc5af8e4f3074402": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a04c3c5849974cb49f71274170c6c430": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_f2190613dadf4e38a0224cf58f04a5f4",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Pushing dataset shards to the dataset hub: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_9d990bfc0d9b4935bdd827256baca46c"
          }
        },
        "3a094acaa16649f4aa9319c176ede264": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_babea81cdcb94fd5af051439283b9b03",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_604362f48cae4a9f84950ac0a7381609"
          }
        },
        "7e841dfe9b1c4286ba81714cdf019720": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_ffee0f297ba042f28ac1c9dc0c776d13",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [00:02&lt;00:00,  2.91s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_0a64bf6e8159403db8e2c07a0547e4b6"
          }
        },
        "f2190613dadf4e38a0224cf58f04a5f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "9d990bfc0d9b4935bdd827256baca46c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "babea81cdcb94fd5af051439283b9b03": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "604362f48cae4a9f84950ac0a7381609": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ffee0f297ba042f28ac1c9dc0c776d13": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "0a64bf6e8159403db8e2c07a0547e4b6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "346b2c7413e24b6e9a4fec62df9c77bf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "VBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "VBoxView",
            "_dom_classes": [],
            "_model_name": "VBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_b053aa0217fe481f86e1f4ff04609318",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_25f71c960336451cb9607fcb083b30c6",
              "IPY_MODEL_ac5823324cb04c46a53b8ac4067b7657",
              "IPY_MODEL_20eb45883cca47d1bcca7240e4ca4d7f",
              "IPY_MODEL_3e464cd6e07142f3b6fd40bb5270b80f",
              "IPY_MODEL_f742ecc1f85f426bbe93dc27ba6e8254"
            ]
          }
        },
        "b053aa0217fe481f86e1f4ff04609318": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": "column",
            "width": "50%",
            "min_width": null,
            "border": null,
            "align_items": "center",
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": "flex",
            "left": null
          }
        },
        "25f71c960336451cb9607fcb083b30c6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_cd937c4062514f11b69b2b7035165a09",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "<center>\n<img src=https://huggingface.co/front/assets/huggingface_logo-noborder.svg alt='Hugging Face'>\n<br>\nCopy a token from <a href=\"https://huggingface.co/settings/token\" target=\"_blank\">your Hugging Face tokens page</a> and paste it below.\n<br>\nImmediately click login after copying your token or it might be stored in plain text in this notebook file.\n</center>",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_1fb8b901801f4a9fba8924b004f7d947"
          }
        },
        "ac5823324cb04c46a53b8ac4067b7657": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "PasswordModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "PasswordView",
            "style": "IPY_MODEL_9ec6f0dc51eb46c399682c408ab17272",
            "_dom_classes": [],
            "description": "Token:",
            "_model_name": "PasswordModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "",
            "_view_count": null,
            "disabled": false,
            "_view_module_version": "1.5.0",
            "continuous_update": true,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_58f7098a299648e988d634226e10bf13"
          }
        },
        "20eb45883cca47d1bcca7240e4ca4d7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ButtonModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ButtonView",
            "style": "IPY_MODEL_3a2a15911c51435db77c7cc9abcdb2c1",
            "_dom_classes": [],
            "description": "Login",
            "_model_name": "ButtonModel",
            "button_style": "",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "tooltip": "",
            "_view_count": null,
            "disabled": false,
            "_view_module_version": "1.5.0",
            "layout": "IPY_MODEL_d98cea9e5e414deca52167ae351c0016",
            "_model_module": "@jupyter-widgets/controls",
            "icon": ""
          }
        },
        "3e464cd6e07142f3b6fd40bb5270b80f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_28fac573b08e4ec681458d9c67336989",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "\n<b>Pro Tip:</b> If you don't already have one, you can create a dedicated 'notebooks' token with 'write' access, that you can then easily reuse for all notebooks.\n<br>\n<i>Logging in with your username and password is deprecated and won't be possible anymore in the near future. You can still use them for now by clicking below.</i>\n</center>",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_9febad88718d4e90a63721ecd6f4eb09"
          }
        },
        "f742ecc1f85f426bbe93dc27ba6e8254": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ButtonModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ButtonView",
            "style": "IPY_MODEL_dcb00bab415247e9895ac8521241935f",
            "_dom_classes": [],
            "description": "Use password",
            "_model_name": "ButtonModel",
            "button_style": "",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "tooltip": "",
            "_view_count": null,
            "disabled": false,
            "_view_module_version": "1.5.0",
            "layout": "IPY_MODEL_b39e6c23eb2246c19fb6a5786814040a",
            "_model_module": "@jupyter-widgets/controls",
            "icon": ""
          }
        },
        "cd937c4062514f11b69b2b7035165a09": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "1fb8b901801f4a9fba8924b004f7d947": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "9ec6f0dc51eb46c399682c408ab17272": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "58f7098a299648e988d634226e10bf13": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3a2a15911c51435db77c7cc9abcdb2c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ButtonStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ButtonStyleModel",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "button_color": null,
            "font_weight": "",
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "d98cea9e5e414deca52167ae351c0016": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "28fac573b08e4ec681458d9c67336989": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "9febad88718d4e90a63721ecd6f4eb09": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "dcb00bab415247e9895ac8521241935f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ButtonStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ButtonStyleModel",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "button_color": null,
            "font_weight": "",
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "b39e6c23eb2246c19fb6a5786814040a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "d1e641482a464922a778ed8115ac4f1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_b0b1fcf6851745cf8487c5a654823fc4",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_a139915979c1407685297e764b5fa5f1",
              "IPY_MODEL_6be15030dfaf4a2a9ffc250c6562c48a",
              "IPY_MODEL_fcd52074285f45b0872e5b6d097682df"
            ]
          }
        },
        "b0b1fcf6851745cf8487c5a654823fc4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a139915979c1407685297e764b5fa5f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_70185f713c2848f7b066bf5f93c7e985",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_ec5595ffdf764d4da0f5fe729cc165f2"
          }
        },
        "6be15030dfaf4a2a9ffc250c6562c48a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_816e525357b54ecf9ac0a9fad459d7e1",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 736,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 736,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_30374b889a7c422a8e32560a55981f62"
          }
        },
        "fcd52074285f45b0872e5b6d097682df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_3ead20d74bdc4e4d8c109e969da3d90f",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 736/736 [00:00&lt;00:00, 14.9kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_d7d25b6ca08f493087d5d2579b24c2f0"
          }
        },
        "70185f713c2848f7b066bf5f93c7e985": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "ec5595ffdf764d4da0f5fe729cc165f2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "816e525357b54ecf9ac0a9fad459d7e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "30374b889a7c422a8e32560a55981f62": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3ead20d74bdc4e4d8c109e969da3d90f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "d7d25b6ca08f493087d5d2579b24c2f0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "443bb06723b6412db643ebf0bc6d0cb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_92fcb3cffed04c8ba9d3b8d340a4f2b8",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_1cbfa06185e544129e095fd4bae8151b",
              "IPY_MODEL_e8009ffcf1fe41d3bdf9b2f04c92730a",
              "IPY_MODEL_62bd8e4d113147d4b3b1418c73e6fd0f"
            ]
          }
        },
        "92fcb3cffed04c8ba9d3b8d340a4f2b8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "1cbfa06185e544129e095fd4bae8151b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_6883b1b37763401b8e614797c3ed1e3a",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_2bbbc239d15441d29df32eaecc8212c0"
          }
        },
        "e8009ffcf1fe41d3bdf9b2f04c92730a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_ce8b12b58beb443ab8899f2fed5987a9",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 2,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 2,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_b3097f92e8e743f6967314f90b4bd41e"
          }
        },
        "62bd8e4d113147d4b3b1418c73e6fd0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_f909938b793e4fd1bd7d8442d683e851",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 2/2 [00:08&lt;00:00,  3.98s/it]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_e5574543d50a46c2afdd7a1f5ef451eb"
          }
        },
        "6883b1b37763401b8e614797c3ed1e3a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "2bbbc239d15441d29df32eaecc8212c0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ce8b12b58beb443ab8899f2fed5987a9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "b3097f92e8e743f6967314f90b4bd41e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f909938b793e4fd1bd7d8442d683e851": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "e5574543d50a46c2afdd7a1f5ef451eb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "499e4a474c654bd6bd3b7aaa2d7ce295": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_7ab71e613b7c42eba3e3a1307e9f404e",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_ee8fc115179e4909a9a0e95de6ac520d",
              "IPY_MODEL_16160687fc5f4b17bad0a1136538f6b1",
              "IPY_MODEL_dfcab1a404db4a43a9423a1ca2d667a4"
            ]
          }
        },
        "7ab71e613b7c42eba3e3a1307e9f404e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ee8fc115179e4909a9a0e95de6ac520d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_ab819d5c5c6e4ce2b7c9a7386e36671e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_410a7d57fe854f8e88a454583a8752f6"
          }
        },
        "16160687fc5f4b17bad0a1136538f6b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_a2aba3ccbcee4766a21d4d2ad55155a4",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 243169,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 243169,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_6a6b0f4302064fe9ab669cd9172f597f"
          }
        },
        "dfcab1a404db4a43a9423a1ca2d667a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_212b10a9dd724e66b0c74044e6b3e015",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 243k/243k [00:00&lt;00:00, 335kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_a6e7f6b2ac3a40b4afe0f94974fee4aa"
          }
        },
        "ab819d5c5c6e4ce2b7c9a7386e36671e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "410a7d57fe854f8e88a454583a8752f6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a2aba3ccbcee4766a21d4d2ad55155a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "6a6b0f4302064fe9ab669cd9172f597f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "212b10a9dd724e66b0c74044e6b3e015": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "a6e7f6b2ac3a40b4afe0f94974fee4aa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "d087107576c54ba68e062f256eac5631": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_1b078ab25caa4bf0a0a44890bcae784a",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_35bfee597f4a48df9d25836a56edee61",
              "IPY_MODEL_d946534b5db443988c4a6e250a99cc68",
              "IPY_MODEL_49b077926f5b4145a65343075d53d2c1"
            ]
          }
        },
        "1b078ab25caa4bf0a0a44890bcae784a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "35bfee597f4a48df9d25836a56edee61": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_3e7a1ed826f547ec9fb76a57264d6e41",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_2d4195e80ff1463a97678decb138df78"
          }
        },
        "d946534b5db443988c4a6e250a99cc68": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_dae471ebb0be46dbbff8fb69099fefd4",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 925144,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 925144,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_632b7e8397a94ce5818e651112884f55"
          }
        },
        "49b077926f5b4145a65343075d53d2c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_504a9cf2dd4343e3ba9635ff27b2b1f7",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 925k/925k [00:01&lt;00:00, 1.36MB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_b3d21a855ee84a26bb7858cfb54cec3e"
          }
        },
        "3e7a1ed826f547ec9fb76a57264d6e41": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "2d4195e80ff1463a97678decb138df78": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "dae471ebb0be46dbbff8fb69099fefd4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "632b7e8397a94ce5818e651112884f55": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "504a9cf2dd4343e3ba9635ff27b2b1f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "b3d21a855ee84a26bb7858cfb54cec3e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "4d014648da7e475588f51c0147b6a01c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_77f36c11561e4287895d34b6d1e709c7",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_a8d642040b8740b18bbe1c2c3c89714f",
              "IPY_MODEL_36984dbb1df045a897f6b9401b5a35b4",
              "IPY_MODEL_2222c6fc1f1d4a52a4e77abe35811175"
            ]
          }
        },
        "77f36c11561e4287895d34b6d1e709c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a8d642040b8740b18bbe1c2c3c89714f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_29f047c7197a4cbfbc174ab526e050ec",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8e350b8c117d4326b52f1e6e4d1761d8"
          }
        },
        "36984dbb1df045a897f6b9401b5a35b4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_419f16e436714af380d8b47b1706e2af",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 2,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 2,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_06f87e207c504946a20ae61213441ceb"
          }
        },
        "2222c6fc1f1d4a52a4e77abe35811175": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_a8d3a03b6dc8455e998c61875aa4bf5e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 2/2 [00:00&lt;00:00, 28.14it/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_229b74f4acd24fda886fef7d20f427c7"
          }
        },
        "29f047c7197a4cbfbc174ab526e050ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "8e350b8c117d4326b52f1e6e4d1761d8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "419f16e436714af380d8b47b1706e2af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "06f87e207c504946a20ae61213441ceb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "a8d3a03b6dc8455e998c61875aa4bf5e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "229b74f4acd24fda886fef7d20f427c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "17abfba35d7d4d9da747d93420860468": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_f2f0ea5852c1406ab35cee2499486d4b",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_dbe41b751bd243ffb87b9fb2bc0a03e2",
              "IPY_MODEL_272c7f8e8ae64026b3b1532bf2b41044",
              "IPY_MODEL_187d2ecf1ff34d2bb4a3c81c59ad312d"
            ]
          }
        },
        "f2f0ea5852c1406ab35cee2499486d4b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "dbe41b751bd243ffb87b9fb2bc0a03e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_8a16286a769b48fca716536e407413df",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_bb5f5d80eb554f23900354a6b8676cb1"
          }
        },
        "272c7f8e8ae64026b3b1532bf2b41044": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_c229215247104322bd8a53e03e03c7be",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 2,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 2,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_2cd6cdc631ed423c823739254bd8041a"
          }
        },
        "187d2ecf1ff34d2bb4a3c81c59ad312d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_62afdb943c14427d9430279124120a3d",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 2/2 [00:00&lt;00:00, 36.32it/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_a877227e8a35483281d53922e8217ed2"
          }
        },
        "8a16286a769b48fca716536e407413df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "bb5f5d80eb554f23900354a6b8676cb1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "c229215247104322bd8a53e03e03c7be": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "2cd6cdc631ed423c823739254bd8041a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "62afdb943c14427d9430279124120a3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "a877227e8a35483281d53922e8217ed2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3fecdd00e01844bb9c5b10898f2c6989": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_77d63edeef4d41ce9827d8d365bf38e9",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_f55186d7194e445989bc46f78419e6c7",
              "IPY_MODEL_9c023102720d40308d71f5c9b4ef441e",
              "IPY_MODEL_60c23f9c82bd48ef85f7e229043add93"
            ]
          }
        },
        "77d63edeef4d41ce9827d8d365bf38e9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f55186d7194e445989bc46f78419e6c7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_866816c453eb43dab279f4006a067e61",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_05133cbc9b304d9d89d3d8230539f8ff"
          }
        },
        "9c023102720d40308d71f5c9b4ef441e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_ab23ccf5fc484b259a51ea72018e4e54",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 43,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 43,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_407f4f3825c64539ace6bedd40983136"
          }
        },
        "60c23f9c82bd48ef85f7e229043add93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_20a79c7e998a429e8c9cd06117fe3b3e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 43.0/43.0 [00:00&lt;00:00, 1.10kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_b426338603734e2e9168145dc6235ec2"
          }
        },
        "866816c453eb43dab279f4006a067e61": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "05133cbc9b304d9d89d3d8230539f8ff": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ab23ccf5fc484b259a51ea72018e4e54": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "407f4f3825c64539ace6bedd40983136": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "20a79c7e998a429e8c9cd06117fe3b3e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "b426338603734e2e9168145dc6235ec2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "91a92d0239c9490d85c817a2817e94fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_503ba250c37048f89c87de4999306a56",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_ca7a05a0817340269aa07a3318604c25",
              "IPY_MODEL_d6f19b238b26419baec5e9ea8074ed28",
              "IPY_MODEL_0304976c59dc4884931f9c8fd554e4ef"
            ]
          }
        },
        "503ba250c37048f89c87de4999306a56": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ca7a05a0817340269aa07a3318604c25": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_97fd5f372d6c4541bc223d177d3c06fa",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_4697ddb9202f415393f09e4310b55d30"
          }
        },
        "d6f19b238b26419baec5e9ea8074ed28": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_9ad98a1c73b5429cab6b9defc137034e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 647,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 647,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_facdcc2ed3fb4bd9a60336c99d49bff3"
          }
        },
        "0304976c59dc4884931f9c8fd554e4ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_c143e4e99a114192b85a87c7979986c8",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 647/647 [00:00&lt;00:00, 10.3kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_948e7729a94947b3882dfe851a6d543a"
          }
        },
        "97fd5f372d6c4541bc223d177d3c06fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "4697ddb9202f415393f09e4310b55d30": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "9ad98a1c73b5429cab6b9defc137034e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "facdcc2ed3fb4bd9a60336c99d49bff3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "c143e4e99a114192b85a87c7979986c8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "948e7729a94947b3882dfe851a6d543a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ec4cc75cfa0648328db7ddb52c58d16a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_62b3c7c55c5d4ee993475a3d1bb6e4b8",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_5120bc6a2514400b87a772bab21a2ecb",
              "IPY_MODEL_ebe9376735af4e8db56aeada7d24cce5",
              "IPY_MODEL_965e3b33476f48dd90695f3cab8ca347"
            ]
          }
        },
        "62b3c7c55c5d4ee993475a3d1bb6e4b8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "5120bc6a2514400b87a772bab21a2ecb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_90793a87cddd46b39f689736e718ea72",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_f1620195897445b7b51c737fd91d01db"
          }
        },
        "ebe9376735af4e8db56aeada7d24cce5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_3a35c5f878014cbda3ba66ba492c53fd",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 209528,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 209528,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_bd4151195be04fe4b7a2f2fdcae37ac9"
          }
        },
        "965e3b33476f48dd90695f3cab8ca347": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_833fa044713d4aa3963da622346ffacc",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 205k/205k [00:00&lt;00:00, 307kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_3f04008d0d854f99b08fc70342701352"
          }
        },
        "90793a87cddd46b39f689736e718ea72": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "f1620195897445b7b51c737fd91d01db": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3a35c5f878014cbda3ba66ba492c53fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "bd4151195be04fe4b7a2f2fdcae37ac9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "833fa044713d4aa3963da622346ffacc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "3f04008d0d854f99b08fc70342701352": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "7af5044f6d3c443486a5707036aa3fd8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_33af9bb7755b478f893c5cf4c81752b5",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_49af0e707aa0490cb6fd758bfade4274",
              "IPY_MODEL_1cb25572218b4eebb63030a137e0eaab",
              "IPY_MODEL_d61c86c609f344eeba179528e93022dd"
            ]
          }
        },
        "33af9bb7755b478f893c5cf4c81752b5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "49af0e707aa0490cb6fd758bfade4274": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_513e0af2a83d4c1f808d6b8369e81654",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_4d9ebb35fb824b6b87d18267954b9b6c"
          }
        },
        "1cb25572218b4eebb63030a137e0eaab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_81ca8599c53f4e79bf1f09d22ef5d0ee",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 2,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 2,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_59c1eeb382464e97bb31ef369a725238"
          }
        },
        "d61c86c609f344eeba179528e93022dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_e8448a95012a43f2a8c5479357552507",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 2.00/2.00 [00:00&lt;00:00, 37.4B/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_ea5651ec85d34c8b95b63eb4b4638d0a"
          }
        },
        "513e0af2a83d4c1f808d6b8369e81654": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "4d9ebb35fb824b6b87d18267954b9b6c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "81ca8599c53f4e79bf1f09d22ef5d0ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "59c1eeb382464e97bb31ef369a725238": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "e8448a95012a43f2a8c5479357552507": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "ea5651ec85d34c8b95b63eb4b4638d0a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "2f4800d141e147278c2ce6e6ba279133": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_98d6339848784f6989bd3bf9a281c4f6",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_3752e596c8324f789c0eb1e7a94bdc9e",
              "IPY_MODEL_45a4729c2804406d933daaf52da6aaca",
              "IPY_MODEL_6f1db82111de481aa4e770953e66b328"
            ]
          }
        },
        "98d6339848784f6989bd3bf9a281c4f6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3752e596c8324f789c0eb1e7a94bdc9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_13ed5df100654b6593707c042a8d5bc0",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_49524c3a50ab4bafa2aba29921b45dbb"
          }
        },
        "45a4729c2804406d933daaf52da6aaca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_f67a133e1cf14b949b8eb49589edefa5",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 112,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 112,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_3d6734c99e6a48da9f452f0d6bc43d50"
          }
        },
        "6f1db82111de481aa4e770953e66b328": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_77de2bf6f6754ade895f1657d7541bdf",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 112/112 [00:00&lt;00:00, 2.86kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_76c9c66e5f74422ca36fc4990d3e947e"
          }
        },
        "13ed5df100654b6593707c042a8d5bc0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "49524c3a50ab4bafa2aba29921b45dbb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f67a133e1cf14b949b8eb49589edefa5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "3d6734c99e6a48da9f452f0d6bc43d50": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "77de2bf6f6754ade895f1657d7541bdf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "76c9c66e5f74422ca36fc4990d3e947e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "e37a7eba1c22461a852708824efd855f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_731079479e9d40b28f3688dd5a3cd9d7",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_e34fa275bc6844ceb27cc46aa49f6cf7",
              "IPY_MODEL_b6d585d3bb7345509bf636f385fa482b",
              "IPY_MODEL_1ab4971a73974734a5af17766716a01e"
            ]
          }
        },
        "731079479e9d40b28f3688dd5a3cd9d7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "e34fa275bc6844ceb27cc46aa49f6cf7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_5df5740537c841d3af8b5766eac12d5e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Downloading: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_f7b11fe83d264494a9fd4de982d9ba46"
          }
        },
        "b6d585d3bb7345509bf636f385fa482b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_f9560e17863747408a6191efd55f997f",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 438235074,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 438235074,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_363529e3db0b4bb39fe75fae19fa0877"
          }
        },
        "1ab4971a73974734a5af17766716a01e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_2cc6826828ed49acbe2be54230be6127",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 418M/418M [00:15&lt;00:00, 33.2MB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_00c4fc42223340b089f69c4da85baaf5"
          }
        },
        "5df5740537c841d3af8b5766eac12d5e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "f7b11fe83d264494a9fd4de982d9ba46": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "f9560e17863747408a6191efd55f997f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "363529e3db0b4bb39fe75fae19fa0877": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "2cc6826828ed49acbe2be54230be6127": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "00c4fc42223340b089f69c4da85baaf5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "06f4a1e636254773b57d1aada679ccf9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_fec32d21df2b47d38dda6c294a7cda50",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_543472090b00438c87b31b31b21b55b6",
              "IPY_MODEL_50fcc449f3e74bd4a15d5ce19eef364a",
              "IPY_MODEL_002e2503a16b4e87bc3d5e6bcbe01a2f"
            ]
          }
        },
        "fec32d21df2b47d38dda6c294a7cda50": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "543472090b00438c87b31b31b21b55b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_36fea090b6704ce0adaf00a882c298e0",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": "Upload file pytorch_model.bin: 100%",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_95fa2f2d92074b22a9592d1fa4e2446e"
          }
        },
        "50fcc449f3e74bd4a15d5ce19eef364a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_11cef1fda8e24986a164a41de13d21d8",
            "_dom_classes": [],
            "description": "",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 435899819,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 435899819,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_d360f32f045047b08640c54fdc394f3e"
          }
        },
        "002e2503a16b4e87bc3d5e6bcbe01a2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_c921d6b324a748999682acf00812f660",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 416M/416M [06:09&lt;00:00, 795kB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_6343a86b3da94c78a2f4a4fc270a7676"
          }
        },
        "36fea090b6704ce0adaf00a882c298e0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "95fa2f2d92074b22a9592d1fa4e2446e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "11cef1fda8e24986a164a41de13d21d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "d360f32f045047b08640c54fdc394f3e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "c921d6b324a748999682acf00812f660": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "6343a86b3da94c78a2f4a4fc270a7676": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}