focused_coding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Focused Coding\n",
    "___\n",
    "\n",
    "## Table of Content\n",
    "\n",
    "1. [Libraries](#libraries)\n",
    "2. [Data Preprocessing](#preprocessing-data-and-grouping)\n",
    "3. [Focused Coding Helper](#focused-coding----sample-helper)\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries\n",
    "\n",
    "All libraries which are needed to execute the code are listed here. Install the packages by using the `requirements.txt` file. \n",
    "\n",
    "The documentation can be found in the [README.md](README.md) file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import packages\n",
    "import pandas as pd \n",
    "import os\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "from nltk.tokenize import TweetTokenizer\n",
    "import nltk\n",
    "from preprocessing_functions import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing Data and Grouping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>video_id</th>\n",
       "      <th>published_at</th>\n",
       "      <th>like_count</th>\n",
       "      <th>text</th>\n",
       "      <th>author</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-19T21:22:45Z</td>\n",
       "      <td>1</td>\n",
       "      <td>The answer is if China and India don't help, i...</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-19T00:43:40Z</td>\n",
       "      <td>2</td>\n",
       "      <td>and that guy is an expert, we're screwed</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-18T22:57:38Z</td>\n",
       "      <td>4</td>\n",
       "      <td>Kennedy is a gem.</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      video_id          published_at  like_count  \\\n",
       "0  uW6fi2tCnAc  2023-02-19T21:22:45Z           1   \n",
       "1  uW6fi2tCnAc  2023-02-19T00:43:40Z           2   \n",
       "2  uW6fi2tCnAc  2023-02-18T22:57:38Z           4   \n",
       "\n",
       "                                                text  author  \n",
       "0  The answer is if China and India don't help, i...     0.0  \n",
       "1           and that guy is an expert, we're screwed     1.0  \n",
       "2                                  Kennedy is a gem.     2.0  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load data\n",
    "df = pd.read_csv('data/comments_final.csv')\n",
    "df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\\begin{tabular}{rrr}\n",
      "\\toprule\n",
      "author & count & unique_video_id_count \\\\\n",
      "\\midrule\n",
      "17605 & 206 & 206 \\\\\n",
      "3743 & 188 & 124 \\\\\n",
      "18119 & 166 & 161 \\\\\n",
      "17604 & 155 & 138 \\\\\n",
      "18380 & 154 & 154 \\\\\n",
      "17977 & 142 & 138 \\\\\n",
      "17906 & 139 & 115 \\\\\n",
      "17676 & 135 & 103 \\\\\n",
      "17755 & 134 & 134 \\\\\n",
      "14017 & 129 & 126 \\\\\n",
      "14057 & 128 & 123 \\\\\n",
      "2610 & 125 & 125 \\\\\n",
      "25732 & 116 & 88 \\\\\n",
      "1645 & 107 & 62 \\\\\n",
      "17581 & 106 & 104 \\\\\n",
      "6165 & 103 & 52 \\\\\n",
      "6 & 99 & 98 \\\\\n",
      "1106 & 97 & 75 \\\\\n",
      "17608 & 92 & 80 \\\\\n",
      "17590 & 87 & 79 \\\\\n",
      "\\bottomrule\n",
      "\\end{tabular}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# group data by author and see distribution of comments \n",
    "df['author'] = pd.to_numeric(df['author'], errors='coerce').astype('Int64')\n",
    "summary = df.groupby('author').agg(\n",
    "    count=('author', 'size'),\n",
    "    unique_video_id_count=('video_id', 'nunique')\n",
    ").reset_index()\n",
    "summary.sort_values(by='count', ascending=False, inplace=True)\n",
    "\n",
    "# print top 20 authors\n",
    "latex_table = summary.head(20).to_latex(index=False)\n",
    "print(latex_table)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# extract original text for seeing in comparising capitalized words etc.\n",
    "extracted_col = df[\"text\"]\n",
    "\n",
    "# process data with using functions from functions.py\n",
    "processed_df = (\n",
    "    df.pipe(remove_users, 'text')\n",
    "      .pipe(lowercase_text, 'text')\n",
    "      .pipe(remove_whitespace, 'text')\n",
    "      .pipe(remove_punctuation, 'text')\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>video_id</th>\n",
       "      <th>published_at</th>\n",
       "      <th>like_count</th>\n",
       "      <th>text</th>\n",
       "      <th>author</th>\n",
       "      <th>og_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-19T21:22:45Z</td>\n",
       "      <td>1</td>\n",
       "      <td>the answer is if china and india dont help it ...</td>\n",
       "      <td>0</td>\n",
       "      <td>The answer is if China and India don't help, i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-19T00:43:40Z</td>\n",
       "      <td>2</td>\n",
       "      <td>and that guy is an expert were screwed</td>\n",
       "      <td>1</td>\n",
       "      <td>and that guy is an expert, we're screwed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>uW6fi2tCnAc</td>\n",
       "      <td>2023-02-18T22:57:38Z</td>\n",
       "      <td>4</td>\n",
       "      <td>kennedy is a gem</td>\n",
       "      <td>2</td>\n",
       "      <td>Kennedy is a gem.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      video_id          published_at  like_count  \\\n",
       "0  uW6fi2tCnAc  2023-02-19T21:22:45Z           1   \n",
       "1  uW6fi2tCnAc  2023-02-19T00:43:40Z           2   \n",
       "2  uW6fi2tCnAc  2023-02-18T22:57:38Z           4   \n",
       "\n",
       "                                                text  author  \\\n",
       "0  the answer is if china and india dont help it ...       0   \n",
       "1             and that guy is an expert were screwed       1   \n",
       "2                                   kennedy is a gem       2   \n",
       "\n",
       "                                             og_text  \n",
       "0  The answer is if China and India don't help, i...  \n",
       "1           and that guy is an expert, we're screwed  \n",
       "2                                  Kennedy is a gem.  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add the extracted column to the second DataFrame\n",
    "processed_df = pd.concat([processed_df, extracted_col.rename(\"og_text\")], axis=1)\n",
    "processed_df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use lemmatization to reduce words to their root form\n",
    "processed_df['text'] = processed_df['text'].astype('str')\n",
    "processed_df = lemmatize_words(processed_df, 'text')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_df.lemmatized_text = processed_df.lemmatized_text.apply(lambda x: '' if str(x) == 'nan' else x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Focused Coding -- Sample helper\n",
    "\n",
    "this helper is written to find comments with important keywords we could firstly find through exploring the comments or through topic modeling and Word2Vec.\n",
    "\n",
    "How does it work?\n",
    "\n",
    "- Input: Insert a lemmatized keyword of interest or multiple in `substrings` or `string`\n",
    "- Ouput: Get comments as output which are talking about those keywords to get a variety of comments based on that topic or word of interest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "substrings = ['god plan', 'greenhouse gas', 'natural cycle', 'hoax']\n",
    "string = ['climate model']\n",
    "pattern = '|'.join(string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "202\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>video_id</th>\n",
       "      <th>published_at</th>\n",
       "      <th>like_count</th>\n",
       "      <th>text</th>\n",
       "      <th>author</th>\n",
       "      <th>og_text</th>\n",
       "      <th>lemmatized_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>15320</th>\n",
       "      <td>7LVSrTZDopM</td>\n",
       "      <td>2023-02-02T16:18:31Z</td>\n",
       "      <td>2</td>\n",
       "      <td>would have been interesting to hear what dr lindzen thinks about copernicus satellite data with the amount of different measurements and level of detail to calibrate any digital model the uncertainty margins that are included in current digital models and have been decreasing over the years based on these data the fact that currently there is more mass created by men than biological mass bacteria plants animals on earth if buffers to absorb max and min temperatures and humidity like in forests especially rainforest would exist research papers that say some species are close to extinction why the amount of flooding events and droughtfires seems so much higher over the last decade to me it is also clear that politics has oversimplified the climate model by just focusing on one of the many greenhouse gasses and ignoring buffers or other disturbing processes but there is certainly effect of what we western humans have been doing during the last century e i think rich people will survive by climate engineering but many animals plants and other parts of earths beauty will be destroyed not quite something to be proud of id say</td>\n",
       "      <td>12701</td>\n",
       "      <td>Would have been interesting to hear what Dr. Lindzen thinks about:\\n-Copernicus satellite data with the amount of different measurements and level of detail to calibrate any digital model\\n-The uncertainty margins that are included in current digital models and have been decreasing over the years, based on these data\\n-The fact that currently there is more mass created by men than biological mass (bacteria, plants, animals) on Earth\\n-If buffers to absorb max. and min. temperatures and humidity, like in forests (especially Rainforest) would exist\\n-Research papers that say some species are close to extinction\\n-Why the amount of flooding events and drought/fires seems so much higher over the last decade\\n\\nTo me it is also clear that politics has oversimplified the climate model by just focusing on one of the many greenhouse gasses and ignoring buffers or other disturbing processes, but there is certainly effect of what we, (Western) humans, have been doing during the last century. E \\nI think rich people will survive by climate engineering, but many animals, plants and other parts of Earth's beauty will be destroyed. Not quite something to be proud of, I'd say.</td>\n",
       "      <td>would have be interest to hear what dr lindzen think about copernicus satellite data with the amount of different measurement and level of detail to calibrate any digital model the uncertainty margin that be include in current digital model and have be decrease over the year base on these data the fact that currently there be more mass create by men than biological mass bacteria plant animal on earth if buffer to absorb max and min temperature and humidity like in forest especially rainforest would exist research paper that say some specie be close to extinction why the amount of flood event and droughtfires seem so much high over the last decade to me it be also clear that politics have oversimplify the climate model by just focus on one of the many greenhouse gas and ignore buffer or other disturbing process but there be certainly effect of what we western human have be do during the last century e i think rich people will survive by climate engineering but many animal plant and other part of earths beauty will be destroy not quite something to be proud of id say</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25960</th>\n",
       "      <td>niZdc2BEuqY</td>\n",
       "      <td>2024-05-11T00:12:35Z</td>\n",
       "      <td>0</td>\n",
       "      <td>all the climate models are bogus none can replicate the historic climate record because they all think co2 drives the climate</td>\n",
       "      <td>18017</td>\n",
       "      <td>All the climate models are bogus. None can replicate the historic climate record (because they all think CO2 drives the climate!)</td>\n",
       "      <td>all the climate model be bogus none can replicate the historic climate record because they all think co2 drive the climate</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62574</th>\n",
       "      <td>U0Xqu6BJHkk</td>\n",
       "      <td>2023-11-15T17:22:43Z</td>\n",
       "      <td>2</td>\n",
       "      <td>being agnostic is not prudent here sure no one can predict the climate but all rational persond should instead focus on what we absolutely know namely that the climate models are not and can not be good evidence for alarm this is the issue after the gretards have no clothes</td>\n",
       "      <td>14036</td>\n",
       "      <td>Being agnostic is not prudent here.  Sure.. no one can predict the climate but all rational persond should instead focus on what we absolutely know. Namely that the climate models are not, and can not be, good evidence for alarm. This is the issue after. The Gretards have no clothes.</td>\n",
       "      <td>be agnostic be not prudent here sure no one can predict the climate but all rational persond should instead focus on what we absolutely know namely that the climate model be not and can not be good evidence for alarm this be the issue after the gretards have no clothes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16689</th>\n",
       "      <td>7LVSrTZDopM</td>\n",
       "      <td>2023-01-11T21:21:15Z</td>\n",
       "      <td>0</td>\n",
       "      <td>co2 is a much smaller contributor to warming than water vapor climate models by the ipcc are terminally flawed the warming is primarily driven by the solar flux changes combined with earths orbital changes and axial processions earths magnetic field is slowly weakening as it prepares to swap solar winds now deposit more energy into the troposphere cloud formation is natural feedback increasing aldebo keeping temperatures lower co2 is far weaker than all of that and it lags temperature increases since the oceans will give up the gas as they warm humans contribute less than 5 of all co2 volcanos contribute as much as humans and ocean thermal vents and under ocean volcanos contribute more than that while the land and oceans contribute 90  most warming is not anthropogenic they best studies with back testing show this say 4 more co2 which is logarithmic in its ir absorption cant possibly contribute enough</td>\n",
       "      <td>8940</td>\n",
       "      <td>CO2 is a much smaller contributor to warming than water vapor. Climate models by the IPCC are terminally flawed. The warming is primarily driven by the solar flux changes combined with earth's orbital changes and axial processions. Earth's magnetic field is slowly weakening as it prepares to swap, solar winds now deposit more energy into the troposphere. Cloud formation is natural feedback increasing aldebo keeping temperatures lower. CO2 is far weaker than all of that and it lags temperature increases since the oceans will give up the gas as they warm. Humans contribute less than 5% of all CO2. Volcanos contribute as much as humans and ocean thermal vents and under ocean volcanos contribute more than that while the land and oceans contribute 90+ %. Most warming is NOT anthropogenic. They best studies with back testing show this. Say, 4% more CO2 which is logarithmic in its IR absorption can't possibly contribute enough.</td>\n",
       "      <td>co2 be a much small contributor to warm than water vapor climate model by the ipcc be terminally flaw the warming be primarily drive by the solar flux change combine with earths orbital change and axial procession earth magnetic field be slowly weaken a it prepare to swap solar wind now deposit more energy into the troposphere cloud formation be natural feedback increase aldebo keep temperature low co2 be far weak than all of that and it lag temperature increase since the ocean will give up the gas a they warm human contribute less than 5 of all co2 volcano contribute as much a human and ocean thermal vent and under ocean volcano contribute more than that while the land and ocean contribute 90 most warming be not anthropogenic they best study with back test show this say 4 more co2 which be logarithmic in it ir absorption cant possibly contribute enough</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50565</th>\n",
       "      <td>Wq5YUhpmnPc</td>\n",
       "      <td>2024-01-09T03:07:08Z</td>\n",
       "      <td>95</td>\n",
       "      <td>this is how i feel about the worthless climate models noaa uses “it doesnt matter how beautiful your theory is it doesnt matter how smart you are if it doesnt agree with experiment its wrong” ― richard p feynman</td>\n",
       "      <td>3743</td>\n",
       "      <td>This is how I feel about the worthless climate models NOAA uses:\\n\\n“It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong.”\\n\\r\\n― Richard P. Feynman</td>\n",
       "      <td>this be how i feel about the worthless climate model noaa use “ it doesnt matter how beautiful your theory be it doesnt matter how smart you be if it doesnt agree with experiment it wrong ” ― richard p feynman</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36285</th>\n",
       "      <td>fBZIM4rNq78</td>\n",
       "      <td>2023-03-26T18:18:41Z</td>\n",
       "      <td>6</td>\n",
       "      <td>i spent 8 years as a meteorologist then another 12 years as a senior simulation analyst do you know what a climate model is it is a vast simplification of the real world you feed your model your simplistic assumptions and then it regurgitates a solution that seems “credible” to the model developer note the word credible credible means believable credible does not mean the truth climate models don’t create truth or valid climate predictions they just deliver credible believable to the climatologist results the same holds true with virus propagation modeling no truth just credible results based on somebodys spurious opinion</td>\n",
       "      <td>18738</td>\n",
       "      <td>I spent 8 years as a meteorologist, then another 12 years as a senior simulation analyst. Do you know what a climate model is? It is a vast simplification of the real world. You feed your model your simplistic assumptions and then it regurgitates a solution that seems “credible” to the model developer. Note the word credible. Credible means believable. Credible does not mean the truth. Climate models don’t create truth or valid climate predictions; they just deliver credible (believable to the climatologist) results. \\r\\nThe same holds true with virus propagation modeling. No truth, just credible results based on somebody's spurious opinion.</td>\n",
       "      <td>i spend 8 year a a meteorologist then another 12 year a a senior simulation analyst do you know what a climate model be it be a vast simplification of the real world you fee your model your simplistic assumption and then it regurgitate a solution that seem “ credible ” to the model developer note the word credible credible mean believable credible do not mean the truth climate model don ’ t create truth or valid climate prediction they just deliver credible believable to the climatologist result the same hold true with virus propagation model no truth just credible result base on somebody spurious opinion</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51911</th>\n",
       "      <td>NlNm4OBuEWo</td>\n",
       "      <td>2024-01-30T14:02:15Z</td>\n",
       "      <td>9</td>\n",
       "      <td>i ran across her a while ago when she discussed some random thing i thought was interesting but she made herself impossible to take serioulsy when she addressed the issue of climate being a chaotic system that can not be modeled which creates a problem for climate alarmists because climate models can not make predictions about climate change because they all share a major technical glitch which is that hey dont exist so presents this pseudomodel which creates a threateninglooking whirly sort of readout which she asserts allows us to infer that theres going to be temperature rise which will cost boatloads of money as if the people who cant make computer models of climate can make economic predictions any more than they can make climate predictions</td>\n",
       "      <td>25362</td>\n",
       "      <td>I ran across her a while ago when she discussed some random thing I thought was interesting, but she made herself impossible to take serioulsy when she addressed the issue of climate being a chaotic system that can not be modeled, which creates a problem for climate alarmists because climate models can not make predictions about climate change, because they all share a major technical glitch, which is that hey don't exist. So presents this pseudo-model which creates a threatening-looking whirly sort of readout, which she asserts allows us to infer that there's going to be temperature rise, which will cost boatloads of money, as if the people who can't make computer models of climate, can make economic predictions any more than they can make climate predictions.</td>\n",
       "      <td>i run across her a while ago when she discuss some random thing i think be interest but she make herself impossible to take serioulsy when she address the issue of climate be a chaotic system that can not be model which create a problem for climate alarmist because climate model can not make prediction about climate change because they all share a major technical glitch which be that hey dont exist so present this pseudomodel which create a threateninglooking whirly sort of readout which she assert allow u to infer that theres go to be temperature rise which will cost boatload of money a if the people who cant make computer model of climate can make economic prediction any more than they can make climate prediction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56365</th>\n",
       "      <td>BmdjUYYNeSI</td>\n",
       "      <td>2023-08-29T13:01:31Z</td>\n",
       "      <td>1</td>\n",
       "      <td>i wonder if you have seen sabine hossenfelders explanation of anthropogenic climate change she mentions stratospheric cooling being a prediction of climate models and a nobel prize for the prediction i didnt understand the significance of this and have watched it more than once</td>\n",
       "      <td>22102</td>\n",
       "      <td>I wonder if you have seen Sabine Hossenfelder's explanation of anthropogenic climate change, she mention's Stratospheric cooling being a prediction of climate models (and a Nobel prize for the prediction) i didn't understand the significance of this and have watched it more than once?</td>\n",
       "      <td>i wonder if you have see sabine hossenfelders explanation of anthropogenic climate change she mention stratospheric cool be a prediction of climate model and a nobel prize for the prediction i didnt understand the significance of this and have watch it more than once</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57847</th>\n",
       "      <td>sMn5i8FX0xU</td>\n",
       "      <td>2023-09-27T07:14:51Z</td>\n",
       "      <td>6</td>\n",
       "      <td>2938 if you add up all the error bars in all the climate models the error is 8 degrees c to 8 degrees c out to 2100</td>\n",
       "      <td>25760</td>\n",
       "      <td>29:38. If you add up all the error bars in all the climate models the error is +8 degrees C to -8 degrees C out to 2100</td>\n",
       "      <td>2938 if you add up all the error bar in all the climate model the error be 8 degree c to 8 degree c out to 2100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36106</th>\n",
       "      <td>cI1utwowtoY</td>\n",
       "      <td>2023-09-26T06:54:21Z</td>\n",
       "      <td>0</td>\n",
       "      <td>theres a channel which features space weather on youtube its called smasho mash and ive been following it for a few years the owner is a solar scientist and despite many claiming that the current solar cycle 25 is the minimum hes been observing that the last cycle solar cycle 24 was actually the minimum and the data bears this out as we are already seeing far more sunspots than should be seen if this was a minimum what this means is that the 20 year pause in temperatures was actually down to the advent of the solar minimum between december 2008 and the end in december 2019 decreased solar activity also meant that there was less solar wind and coronal mass ejections and this allowed the cosmic rays to increase the amount of cloud cover which cooled the planet through cloud condensation nuclei allowing more clouds to form its really very easy to see that carbon dioxide is not the only thing that changed since the little ice age despite claims to the contrary the increase of solar activity since the minimums over the last 600 years accounts for all the global warming that weve seen carbon dioxide plays a minuscule part in the climate when compared to water vapor and solar activity just recently there was a video on channel american thought leaders called nobel laureate john clauser climate models miss key variable and he backs up that the missing piece of the climate models is clouds who would have thought that william wordsworth was a climate scientist</td>\n",
       "      <td>12011</td>\n",
       "      <td>There's a channel which features Space Weather on YouTube. It's called smAsho mAsh and I've been following it for a few years. The owner is a Solar Scientist and despite many claiming that the current Solar Cycle 25 is the minimum, he's been observing that the last cycle Solar Cycle 24 was actually the minimum and the data bears this out as we are already seeing far more sunspots than should be seen if this was a minimum.\\n\\nWhat this means is that the 20 year pause in temperatures was actually down to the advent of the Solar Minimum between December 2008 and the end in December 2019. Decreased solar activity also meant that there was less solar wind and Coronal Mass Ejections and this allowed the Cosmic Rays to increase the amount of cloud cover which cooled the planet through Cloud Condensation Nuclei allowing more clouds to form.\\n\\nIt's really very easy to see that Carbon Dioxide is not the only thing that changed since the Little Ice Age. Despite claims to the contrary, the increase of solar activity since the minimums over the last 600 years accounts for all the Global Warming that we've seen. Carbon Dioxide plays a minuscule part in the Climate when compared to water vapor and solar activity. \\n\\nJust recently there was a video on channel American Thought Leaders called \"Nobel Laureate John Clauser: Climate Models Miss Key Variable\" and he backs up that the missing piece of the Climate Models is CLOUDS.\\n\\nWho would have thought that William Wordsworth was a Climate Scientist?!</td>\n",
       "      <td>there a channel which feature space weather on youtube it call smasho mash and ive be follow it for a few year the owner be a solar scientist and despite many claim that the current solar cycle 25 be the minimum he be observe that the last cycle solar cycle 24 be actually the minimum and the data bear this out a we be already see far more sunspot than should be see if this be a minimum what this mean be that the 20 year pause in temperature be actually down to the advent of the solar minimum between december 2008 and the end in december 2019 decrease solar activity also mean that there be less solar wind and coronal mass ejection and this allow the cosmic ray to increase the amount of cloud cover which cool the planet through cloud condensation nucleus allow more cloud to form it really very easy to see that carbon dioxide be not the only thing that change since the little ice age despite claim to the contrary the increase of solar activity since the minimum over the last 600 year account for all the global warming that weve see carbon dioxide play a minuscule part in the climate when compare to water vapor and solar activity just recently there be a video on channel american thought leader call nobel laureate john clauser climate model miss key variable and he back up that the miss piece of the climate model be cloud who would have think that william wordsworth be a climate scientist</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          video_id          published_at  like_count  \\\n",
       "15320  7LVSrTZDopM  2023-02-02T16:18:31Z           2   \n",
       "25960  niZdc2BEuqY  2024-05-11T00:12:35Z           0   \n",
       "62574  U0Xqu6BJHkk  2023-11-15T17:22:43Z           2   \n",
       "16689  7LVSrTZDopM  2023-01-11T21:21:15Z           0   \n",
       "50565  Wq5YUhpmnPc  2024-01-09T03:07:08Z          95   \n",
       "36285  fBZIM4rNq78  2023-03-26T18:18:41Z           6   \n",
       "51911  NlNm4OBuEWo  2024-01-30T14:02:15Z           9   \n",
       "56365  BmdjUYYNeSI  2023-08-29T13:01:31Z           1   \n",
       "57847  sMn5i8FX0xU  2023-09-27T07:14:51Z           6   \n",
       "36106  cI1utwowtoY  2023-09-26T06:54:21Z           0   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     text  \\\n",
       "15320                                                                                                                                                                                                                                                                                                                                                   would have been interesting to hear what dr lindzen thinks about copernicus satellite data with the amount of different measurements and level of detail to calibrate any digital model the uncertainty margins that are included in current digital models and have been decreasing over the years based on these data the fact that currently there is more mass created by men than biological mass bacteria plants animals on earth if buffers to absorb max and min temperatures and humidity like in forests especially rainforest would exist research papers that say some species are close to extinction why the amount of flooding events and droughtfires seems so much higher over the last decade to me it is also clear that politics has oversimplified the climate model by just focusing on one of the many greenhouse gasses and ignoring buffers or other disturbing processes but there is certainly effect of what we western humans have been doing during the last century e i think rich people will survive by climate engineering but many animals plants and other parts of earths beauty will be destroyed not quite something to be proud of id say   \n",
       "25960                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       all the climate models are bogus none can replicate the historic climate record because they all think co2 drives the climate   \n",
       "62574                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  being agnostic is not prudent here sure no one can predict the climate but all rational persond should instead focus on what we absolutely know namely that the climate models are not and can not be good evidence for alarm this is the issue after the gretards have no clothes   \n",
       "16689                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  co2 is a much smaller contributor to warming than water vapor climate models by the ipcc are terminally flawed the warming is primarily driven by the solar flux changes combined with earths orbital changes and axial processions earths magnetic field is slowly weakening as it prepares to swap solar winds now deposit more energy into the troposphere cloud formation is natural feedback increasing aldebo keeping temperatures lower co2 is far weaker than all of that and it lags temperature increases since the oceans will give up the gas as they warm humans contribute less than 5 of all co2 volcanos contribute as much as humans and ocean thermal vents and under ocean volcanos contribute more than that while the land and oceans contribute 90  most warming is not anthropogenic they best studies with back testing show this say 4 more co2 which is logarithmic in its ir absorption cant possibly contribute enough   \n",
       "50565                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 this is how i feel about the worthless climate models noaa uses “it doesnt matter how beautiful your theory is it doesnt matter how smart you are if it doesnt agree with experiment its wrong” ― richard p feynman   \n",
       "36285                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               i spent 8 years as a meteorologist then another 12 years as a senior simulation analyst do you know what a climate model is it is a vast simplification of the real world you feed your model your simplistic assumptions and then it regurgitates a solution that seems “credible” to the model developer note the word credible credible means believable credible does not mean the truth climate models don’t create truth or valid climate predictions they just deliver credible believable to the climatologist results the same holds true with virus propagation modeling no truth just credible results based on somebodys spurious opinion   \n",
       "51911                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                i ran across her a while ago when she discussed some random thing i thought was interesting but she made herself impossible to take serioulsy when she addressed the issue of climate being a chaotic system that can not be modeled which creates a problem for climate alarmists because climate models can not make predictions about climate change because they all share a major technical glitch which is that hey dont exist so presents this pseudomodel which creates a threateninglooking whirly sort of readout which she asserts allows us to infer that theres going to be temperature rise which will cost boatloads of money as if the people who cant make computer models of climate can make economic predictions any more than they can make climate predictions   \n",
       "56365                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              i wonder if you have seen sabine hossenfelders explanation of anthropogenic climate change she mentions stratospheric cooling being a prediction of climate models and a nobel prize for the prediction i didnt understand the significance of this and have watched it more than once   \n",
       "57847                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 2938 if you add up all the error bars in all the climate models the error is 8 degrees c to 8 degrees c out to 2100   \n",
       "36106  theres a channel which features space weather on youtube its called smasho mash and ive been following it for a few years the owner is a solar scientist and despite many claiming that the current solar cycle 25 is the minimum hes been observing that the last cycle solar cycle 24 was actually the minimum and the data bears this out as we are already seeing far more sunspots than should be seen if this was a minimum what this means is that the 20 year pause in temperatures was actually down to the advent of the solar minimum between december 2008 and the end in december 2019 decreased solar activity also meant that there was less solar wind and coronal mass ejections and this allowed the cosmic rays to increase the amount of cloud cover which cooled the planet through cloud condensation nuclei allowing more clouds to form its really very easy to see that carbon dioxide is not the only thing that changed since the little ice age despite claims to the contrary the increase of solar activity since the minimums over the last 600 years accounts for all the global warming that weve seen carbon dioxide plays a minuscule part in the climate when compared to water vapor and solar activity just recently there was a video on channel american thought leaders called nobel laureate john clauser climate models miss key variable and he backs up that the missing piece of the climate models is clouds who would have thought that william wordsworth was a climate scientist   \n",
       "\n",
       "       author  \\\n",
       "15320   12701   \n",
       "25960   18017   \n",
       "62574   14036   \n",
       "16689    8940   \n",
       "50565    3743   \n",
       "36285   18738   \n",
       "51911   25362   \n",
       "56365   22102   \n",
       "57847   25760   \n",
       "36106   12011   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     og_text  \\\n",
       "15320                                                                                                                                                                                                                                                                                                                                           Would have been interesting to hear what Dr. Lindzen thinks about:\\n-Copernicus satellite data with the amount of different measurements and level of detail to calibrate any digital model\\n-The uncertainty margins that are included in current digital models and have been decreasing over the years, based on these data\\n-The fact that currently there is more mass created by men than biological mass (bacteria, plants, animals) on Earth\\n-If buffers to absorb max. and min. temperatures and humidity, like in forests (especially Rainforest) would exist\\n-Research papers that say some species are close to extinction\\n-Why the amount of flooding events and drought/fires seems so much higher over the last decade\\n\\nTo me it is also clear that politics has oversimplified the climate model by just focusing on one of the many greenhouse gasses and ignoring buffers or other disturbing processes, but there is certainly effect of what we, (Western) humans, have been doing during the last century. E \\nI think rich people will survive by climate engineering, but many animals, plants and other parts of Earth's beauty will be destroyed. Not quite something to be proud of, I'd say.   \n",
       "25960                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      All the climate models are bogus. None can replicate the historic climate record (because they all think CO2 drives the climate!)   \n",
       "62574                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           Being agnostic is not prudent here.  Sure.. no one can predict the climate but all rational persond should instead focus on what we absolutely know. Namely that the climate models are not, and can not be, good evidence for alarm. This is the issue after. The Gretards have no clothes.   \n",
       "16689                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 CO2 is a much smaller contributor to warming than water vapor. Climate models by the IPCC are terminally flawed. The warming is primarily driven by the solar flux changes combined with earth's orbital changes and axial processions. Earth's magnetic field is slowly weakening as it prepares to swap, solar winds now deposit more energy into the troposphere. Cloud formation is natural feedback increasing aldebo keeping temperatures lower. CO2 is far weaker than all of that and it lags temperature increases since the oceans will give up the gas as they warm. Humans contribute less than 5% of all CO2. Volcanos contribute as much as humans and ocean thermal vents and under ocean volcanos contribute more than that while the land and oceans contribute 90+ %. Most warming is NOT anthropogenic. They best studies with back testing show this. Say, 4% more CO2 which is logarithmic in its IR absorption can't possibly contribute enough.   \n",
       "50565                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  This is how I feel about the worthless climate models NOAA uses:\\n\\n“It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong.”\\n\\r\\n― Richard P. Feynman   \n",
       "36285                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              I spent 8 years as a meteorologist, then another 12 years as a senior simulation analyst. Do you know what a climate model is? It is a vast simplification of the real world. You feed your model your simplistic assumptions and then it regurgitates a solution that seems “credible” to the model developer. Note the word credible. Credible means believable. Credible does not mean the truth. Climate models don’t create truth or valid climate predictions; they just deliver credible (believable to the climatologist) results. \\r\\nThe same holds true with virus propagation modeling. No truth, just credible results based on somebody's spurious opinion.   \n",
       "51911                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    I ran across her a while ago when she discussed some random thing I thought was interesting, but she made herself impossible to take serioulsy when she addressed the issue of climate being a chaotic system that can not be modeled, which creates a problem for climate alarmists because climate models can not make predictions about climate change, because they all share a major technical glitch, which is that hey don't exist. So presents this pseudo-model which creates a threatening-looking whirly sort of readout, which she asserts allows us to infer that there's going to be temperature rise, which will cost boatloads of money, as if the people who can't make computer models of climate, can make economic predictions any more than they can make climate predictions.   \n",
       "56365                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          I wonder if you have seen Sabine Hossenfelder's explanation of anthropogenic climate change, she mention's Stratospheric cooling being a prediction of climate models (and a Nobel prize for the prediction) i didn't understand the significance of this and have watched it more than once?   \n",
       "57847                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                29:38. If you add up all the error bars in all the climate models the error is +8 degrees C to -8 degrees C out to 2100   \n",
       "36106  There's a channel which features Space Weather on YouTube. It's called smAsho mAsh and I've been following it for a few years. The owner is a Solar Scientist and despite many claiming that the current Solar Cycle 25 is the minimum, he's been observing that the last cycle Solar Cycle 24 was actually the minimum and the data bears this out as we are already seeing far more sunspots than should be seen if this was a minimum.\\n\\nWhat this means is that the 20 year pause in temperatures was actually down to the advent of the Solar Minimum between December 2008 and the end in December 2019. Decreased solar activity also meant that there was less solar wind and Coronal Mass Ejections and this allowed the Cosmic Rays to increase the amount of cloud cover which cooled the planet through Cloud Condensation Nuclei allowing more clouds to form.\\n\\nIt's really very easy to see that Carbon Dioxide is not the only thing that changed since the Little Ice Age. Despite claims to the contrary, the increase of solar activity since the minimums over the last 600 years accounts for all the Global Warming that we've seen. Carbon Dioxide plays a minuscule part in the Climate when compared to water vapor and solar activity. \\n\\nJust recently there was a video on channel American Thought Leaders called \"Nobel Laureate John Clauser: Climate Models Miss Key Variable\" and he backs up that the missing piece of the Climate Models is CLOUDS.\\n\\nWho would have thought that William Wordsworth was a Climate Scientist?!   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       lemmatized_text  \n",
       "15320                                                                                                                                                                                                                                                                                                                                        would have be interest to hear what dr lindzen think about copernicus satellite data with the amount of different measurement and level of detail to calibrate any digital model the uncertainty margin that be include in current digital model and have be decrease over the year base on these data the fact that currently there be more mass create by men than biological mass bacteria plant animal on earth if buffer to absorb max and min temperature and humidity like in forest especially rainforest would exist research paper that say some specie be close to extinction why the amount of flood event and droughtfires seem so much high over the last decade to me it be also clear that politics have oversimplify the climate model by just focus on one of the many greenhouse gas and ignore buffer or other disturbing process but there be certainly effect of what we western human have be do during the last century e i think rich people will survive by climate engineering but many animal plant and other part of earths beauty will be destroy not quite something to be proud of id say  \n",
       "25960                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       all the climate model be bogus none can replicate the historic climate record because they all think co2 drive the climate  \n",
       "62574                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    be agnostic be not prudent here sure no one can predict the climate but all rational persond should instead focus on what we absolutely know namely that the climate model be not and can not be good evidence for alarm this be the issue after the gretards have no clothes  \n",
       "16689                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                co2 be a much small contributor to warm than water vapor climate model by the ipcc be terminally flaw the warming be primarily drive by the solar flux change combine with earths orbital change and axial procession earth magnetic field be slowly weaken a it prepare to swap solar wind now deposit more energy into the troposphere cloud formation be natural feedback increase aldebo keep temperature low co2 be far weak than all of that and it lag temperature increase since the ocean will give up the gas a they warm human contribute less than 5 of all co2 volcano contribute as much a human and ocean thermal vent and under ocean volcano contribute more than that while the land and ocean contribute 90 most warming be not anthropogenic they best study with back test show this say 4 more co2 which be logarithmic in it ir absorption cant possibly contribute enough  \n",
       "50565                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                this be how i feel about the worthless climate model noaa use “ it doesnt matter how beautiful your theory be it doesnt matter how smart you be if it doesnt agree with experiment it wrong ” ― richard p feynman  \n",
       "36285                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             i spend 8 year a a meteorologist then another 12 year a a senior simulation analyst do you know what a climate model be it be a vast simplification of the real world you fee your model your simplistic assumption and then it regurgitate a solution that seem “ credible ” to the model developer note the word credible credible mean believable credible do not mean the truth climate model don ’ t create truth or valid climate prediction they just deliver credible believable to the climatologist result the same hold true with virus propagation model no truth just credible result base on somebody spurious opinion  \n",
       "51911                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             i run across her a while ago when she discuss some random thing i think be interest but she make herself impossible to take serioulsy when she address the issue of climate be a chaotic system that can not be model which create a problem for climate alarmist because climate model can not make prediction about climate change because they all share a major technical glitch which be that hey dont exist so present this pseudomodel which create a threateninglooking whirly sort of readout which she assert allow u to infer that theres go to be temperature rise which will cost boatload of money a if the people who cant make computer model of climate can make economic prediction any more than they can make climate prediction  \n",
       "56365                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      i wonder if you have see sabine hossenfelders explanation of anthropogenic climate change she mention stratospheric cool be a prediction of climate model and a nobel prize for the prediction i didnt understand the significance of this and have watch it more than once  \n",
       "57847                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  2938 if you add up all the error bar in all the climate model the error be 8 degree c to 8 degree c out to 2100  \n",
       "36106  there a channel which feature space weather on youtube it call smasho mash and ive be follow it for a few year the owner be a solar scientist and despite many claim that the current solar cycle 25 be the minimum he be observe that the last cycle solar cycle 24 be actually the minimum and the data bear this out a we be already see far more sunspot than should be see if this be a minimum what this mean be that the 20 year pause in temperature be actually down to the advent of the solar minimum between december 2008 and the end in december 2019 decrease solar activity also mean that there be less solar wind and coronal mass ejection and this allow the cosmic ray to increase the amount of cloud cover which cool the planet through cloud condensation nucleus allow more cloud to form it really very easy to see that carbon dioxide be not the only thing that change since the little ice age despite claim to the contrary the increase of solar activity since the minimum over the last 600 year account for all the global warming that weve see carbon dioxide play a minuscule part in the climate when compare to water vapor and solar activity just recently there be a video on channel american thought leader call nobel laureate john clauser climate model miss key variable and he back up that the miss piece of the climate model be cloud who would have think that william wordsworth be a climate scientist  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filtered_df = processed_df[processed_df['lemmatized_text'].str.contains(pattern, case=False, na=False)]\n",
    "pd.set_option('display.max_colwidth', None)\n",
    "print(len(filtered_df))\n",
    "filtered_df.sample(n=10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}