rag_book_qa_prompt.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interact with your book 📖❓🙋🏻‍♀️\n",
    "\n",
    "A simple demonstration of how you can implement retrieval augmented generation for a book.\n",
    "\n",
    "## How retrieval augmented generation works\n",
    "\n",
    "Following are the high level steps needed for the implementation for retrieval augmented generation.\n",
    "\n",
    "1. Extract text from source. If the source is unstructured, like PDF, the extraction can be a challenge.\n",
    "2. Index the extracted text, often as vector embeddings and store.\n",
    "3. Let the user ask questions related to the source.\n",
    "4. Perform a similarity search in the index and retrieve relevant text chunks.\n",
    "5. Insert these text chunks in the prompt along with the question.\n",
    "6. Request an LLM (e.g. chatgpt) to produce an answer *only* based on the context\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1 & 2\n",
    "Execute the python file extract_text_and_save_index.py for extracting text and saving index. You should run the python file again if you change the source PDF file or want to change how the text is extracted.\n",
    "\n",
    "Rest of the steps are provided below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import necessary packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import glob\n",
    "import time\n",
    "\n",
    "from langchain.schema import Document\n",
    "from langchain.text_splitter import NLTKTextSplitter\n",
    "from langchain.embeddings import HuggingFaceInstructEmbeddings\n",
    "from langchain import PromptTemplate\n",
    "from langchain.vectorstores import FAISS\n",
    "from langchain.text_splitter import NLTKTextSplitter\n",
    "from tqdm import tqdm\n",
    "import pandas as pd\n",
    "import pdfplumber"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load already saved vector embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load INSTRUCTOR_Transformer\n",
      "max_seq_length  512\n",
      "CPU times: user 238 ms, sys: 103 ms, total: 341 ms\n",
      "Wall time: 346 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "### download embeddings model\n",
    "embeddings = HuggingFaceInstructEmbeddings(\n",
    "    model_name = 'sentence-transformers/all-MiniLM-L6-v2',\n",
    "    model_kwargs = {\"device\": \"cpu\"}\n",
    ")\n",
    "\n",
    "### load vector DB embeddings\n",
    "vectordb : FAISS = FAISS.load_local(\n",
    "    \"faiss_index_hp\",\n",
    "    embeddings\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify that similarity search is working"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='2 Data transmission\\n(VIN). Check digits are used to identify errors in data entry caused by mis-typing\\nor mis-scanning a barcode. They can usually detect the following types of error:\\n» an incorrect digit entered, for example 5327 entered instead of 5307\\n» transposition errors where two numbers have changed order, for example 5037\\ninstead of 5307\\n» omitted or extra digits, for example 537 instead of 5307 or 53107 instead\\nof 5307\\n» phonetic errors, for example 13 (thirteen), instead of 30 (thirty).\\nThere are a number of different methods used to generate a check digit. Two\\ncommon methods will be considered here:\\n» ISBN 13\\n» Modulo-11\\nExample 1: ISBN 13\\nThe check digit in ISBN 13 is the thirteenth digit in the number. We will now consider\\ntwo different calculations. The first calculation is the generation of the check digit.\\nThe second calculation is a verification of the check digit (that is, a recalculation).\\nCalculation 1 – Generation of the check digit from the other 12 digits in a number\\nThe following algorithm generates the check digit from the 12 other digits:\\n1 add all the odd numbered digits together\\n2 add all the even numbered digits together and multiply the result by 3\\n3 add the results from 1 and 2 together and divide by 10\\n4 take the remainder, if it is zero then use this value, otherwise subtract the\\nremainder from 10 to find the check digit.\\nUsing the ISBN 9 7 8 0 3 4 0 9 8 3 8 2 (note this is the same ISBN as in Figure 2.15):\\n9 7 8 0 3 4 0 9 8 3 8 2\\n1 9 + 8 + 3 + 0 + 8 + 8 = 36\\n2\\n3 ( + )/10 = 111/10 = 11 remainder 1\\n4 10 – 1 = the check digit\\n\\n', metadata={'source': 'Cambridge IGCSE and O Level Computer Science.pdf', 'page': 60}),\n",
       " Document(page_content='2.2 Methods of error detection\\nSo we end up with the following thirteen-digit number (which matches the number\\nshown in Figure 2.15):\\n9 7 8 0 3 4 0 9 8 3 8 2\\nCalculation 2 – Re-calculation of the check digit from the thirteen-digit number\\nAdvice (which now includes the check digit)\\nYou will not need To check that an ISBN 13-digit code is correct, including its check digit, a similar\\nto remember the process is followed:\\nsteps shown in these 1 add all the odd numbered digits together, including the check digit\\n2 add all the even number of digits together and multiply the result by 3\\nalgorithms; the steps\\n3 add the results from 1 and 2 together and divide by 10\\nwill be given to you,\\n4 the number is correct if the remainder is zero.\\nbut it is important that\\nyou understand how Using the ISBN 9 7 8 0 3 4 0 9 8 3 8 2 9 (including its check digit) from Figure 2.17:\\nto use an algorithm 1 9 + 8 + 3 + 0 + 8 + 8 + 9 = 45\\nto calculate or verify 2\\ncheck digits. 3 (45 + 75)/10 = 120/10 = 12 remainder\\n4 remainder is 0, therefore number is correct.\\nExample 2: Modulo-11\\nThe modulo-11 method can have varying lengths of number which makes it suitable\\nfor many applications, such as product codes or VINs. The first calculation is the\\ngeneration of the check digit. The second calculation is a verification of the check digit\\n(that is, a recalculation).\\nCalculation 1 – Generation of the check digit from the other digits in a number\\n(In this example, we will assume the original number contained only 7 digits.)\\nThe following algorithm generates the check digit from the other 7 digits:\\n1 each digit in the number is given a weighting of 8, 7, 6, 5, 4, 3 or 2 starting from the\\nleft (weightings start from 8 since the number will become eight-digit when the\\ncheck digit is added)\\n2 the digit is multiplied by its weighting and then each value is added to make a total\\n3 the total is divided by 11\\n4 the remainder is then subtracted from 11 to find the check digit (note if the\\nremainder is 10 then the check digit ‘X’ is used).\\nThe example to be used has the following seven-digit number:\\n1 7-digit number: 4 1 5 6 7 1 0\\nweighting values: 8 7 6 5 4 3 2\\n2 sum: ( × 4) + ( × ) + ( × ) + ( × ) + ( × ) + ( × ) + ( × )\\n= 32 + 7 + 30 + 30 + 28 + 3 + 0\\ntotal = 130\\n3 divide total by 11: 130/11 = 11 remainder 9\\n4 subtract remainder from 11: 11 – 9 = 2 (check digit)\\nSo we end up with the following eight-digit: 4 1 5 6 7 1 0 2\\n\\n', metadata={'source': 'Cambridge IGCSE and O Level Computer Science.pdf', 'page': 61}),\n",
       " Document(page_content='7 Algorithm design And problem solving\\nActivity 7.7\\n1 State, with reasons, which validation checks you could use for the following\\ninputs.\\nYou may decide that more than one validation check is required.\\n– Entering a telephone number\\n– Entering a pupil’s name\\n– Entering a part number in the form XXX999, when X must be a letter and 9\\nmust be a digit.\\n2 Write an algorithm using pseudocode to check the age and height of a child\\nwho wants to go on a fairground ride. The age must be over 7 and under 12, the\\nheight must be over 110 centimetres and under 150 centimetres.\\n3 Write an algorithm using pseudocode to check that the length of a password is\\nbetween 8 and 12 characters inclusive.\\n7.5.2 Verification\\nVerification is checking that data has been accurately copied from one source to\\nanother – for instance, input into a computer or transferred from one part of a\\ncomputer system to another.\\nVerification methods for input data include:\\n» Double entry\\n» Screen/visual check.\\nFor double entry the data is entered twice, sometimes by different operators.\\nThe computer system compares both entries and if they are different outputs an\\nerror message requesting that the data is entered again.\\nLink\\nParity checks and\\nchecksums are\\nused when data is\\ntransferred from one A screen/visual check is a manual check completed by the user who is entering\\npart of a computer\\nthe data. When the data entry is complete the data is displayed on the screen\\nsystem to another,\\nand the user is asked to confirm that it is correct before continuing. The user\\nor across a network,\\neither checks the data on the screen against a paper document that is being\\nand are discussed in\\nused as an input form or, confirms whether it is correct from their own\\nChapter 2.\\nknowledge.\\n\\n', metadata={'source': 'Cambridge IGCSE and O Level Computer Science.pdf', 'page': 280}),\n",
       " Document(page_content='7.5 Validation and verification\\nFormat check and check digit\\nA format check checks that the characters entered conform to a pre-defined\\npattern, for example, in Chapter 9 the cub number must be in the form CUB9999.\\nThe pseudocode for this example will be given in the string handling section of\\nChapter 9.\\nA check digit is the final digit included in a code; it is calculated from all the\\nother digits in the code. Check digits are used for barcodes, product codes,\\nLink\\nInternational Standard Book Numbers (ISBN) and Vehicle Identification Numbers\\n(VIN).\\nISBN and modulo-11\\ncheck digit Check digits are used to identify errors in data entry caused by mis-typing or\\ncalculations are mis-scanning a barcode. They can usually detect the following types of error:\\ncovered in Chapter 2.\\n» an incorrect digit entered, for example, 5327 entered instead of 5307\\n» transposition errors where two numbers have changed order for example 5037\\ninstead of 5307\\n» omitted or extra digits, for example, 537 instead of 5307 or 53107 instead of\\n5307\\n» phonetic errors, for example, 13, thirteen, instead of 30, thirty.\\nFind out more\\n\\n\\n\\n<table border=\"1\" class=\"dataframe\">\\n  <tbody>\\n    <tr>\\n      <td></td>\\n      <td>Find out more</td>\\n    </tr>\\n    <tr>\\n      <td>1 Find an ISBN, then show that its check digit is correct.\\\\nWorking in pairs find two ISBNs each, copy one down with a transposition error and\\\\nthe other one correctly.\\\\nSwap your ISBNs and see if you can find the one with the error.\\\\nLook at a correct ISBN, can you think of an error that this system will not identify\\\\nand explain with an example why this is the case?\\\\n2 Find out how limit checks and consistency checks are used.</td>\\n      <td>None</td>\\n    </tr>\\n  </tbody>\\n</table>', metadata={'source': 'Cambridge IGCSE and O Level Computer Science.pdf', 'page': 279})]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### test if vector DB was loaded correctly\n",
    "results = vectordb.similarity_search('check digit')\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a prompt template requiring the LLM to generate an answer only based on the provided context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_template = \"\"\"\n",
    "Don't try to make up an answer, if you don't know just say that you don't know.\n",
    "Answer in the same language the question was asked.\n",
    "Use only the following pieces of context to answer the question at the end.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "Answer:\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure that we will use top 3 results from similarity search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory.vectorstore import VectorStoreRetriever\n",
    "\n",
    "retriever : VectorStoreRetriever = vectordb.as_retriever(search_kwargs = {\"k\": 3, \"search_type\" : \"similarity\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Provide a question for which answer is required.\n",
    "- The final prompt including context will be copied to the clipboard.\n",
    "- You can paste the prompt on an LLM interface (e.g. chat.openai.com) and get your answer!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Page Number: 105\n",
      "3.2 Input and output devices\n",
      "Laser printers\n",
      "Laser printers use dry powder ink rather than liquid ink and make use of the\n",
      "properties of static electricity to produce the text and images. Unlike inkjet\n",
      "printers, laser printers print the whole page in one go. Colour laser printers\n",
      "use 4 toner cartridges – blue, cyan, magenta and black. Although the actual\n",
      "technology is different to monochrome printers, the printing method is similar\n",
      "but coloured dots are used to build up the text and images.\n",
      "The following table describes briefly the stages that occur when a document is\n",
      "printed using a laser printer:\n",
      "Applications of inkjet and laser printers\n",
      "The choice of whether to use an inkjet printer or a laser printer depends\n",
      "on which features make it the most appropriate output device for the given\n",
      "application.\n",
      "Inkjet printer – inkjet printers are often used for printing one-off photos\n",
      "or where only a few pages of good quality, colour printing is needed; the\n",
      "small ink cartridges or small paper trays would not be an issue with such\n",
      "applications.\n",
      "Laser printer – these devices produce high quality printouts and are very fast\n",
      "when making multiple copies of a document; any application that needs high\n",
      "volume printing (in colour or monochrome) would choose the laser printer\n",
      "(for example, producing a large number of high-quality flyers or posters\n",
      "for advertising). Laser printers have two advantages: they have large toner\n",
      "cartridges and large paper trays (often holding more than a ream of paper).\n",
      "\n",
      "\n",
      "\n",
      "<table border=\"1\" class=\"dataframe\">\n",
      "  <tbody>\n",
      "    <tr>\n",
      "      <td>Stage in\\nprocess</td>\n",
      "      <td>Description of what happens</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>1</td>\n",
      "      <td>the data from the document is sent to a printer driver</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>2</td>\n",
      "      <td>the printer driver ensures that the data is in a format that the chosen printer can understand</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>3</td>\n",
      "      <td>a check is made by the printer driver to ensure that the chosen printer is available to print (e.g. is it busy, is it\\noff-line, is it out of ink, and so on)</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>4</td>\n",
      "      <td>the data is then sent to the printer and it is stored in a temporary memory known as a printer buffer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>5</td>\n",
      "      <td>the start of the printing process involves a printing drum being given a positive charge; as this drum rotates, a\\nlaser beam is scanned across it removing the positive charge in certain areas; this leaves negatively charged\\nareas that exactly match the text/images of the page to be printed</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>6</td>\n",
      "      <td>the drum is then coated with positively charged toner (powdered ink); since the toner is positively charged, it\\nonly sticks to the negatively charged parts of the drum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>7</td>\n",
      "      <td>a negatively charged sheet of paper is then rolled over the drum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>8</td>\n",
      "      <td>the toner on the drum now sticks to the paper to produce an exact copy of the page sent to the printer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>9</td>\n",
      "      <td>to prevent the paper sticking to the drum, the electric charge on the paper is removed after one rotation of the\\ndrum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>10</td>\n",
      "      <td>the paper finally goes through a fuser which is a set of heated rollers; the heat melts the ink so that it fixes\\npermanently to the paper</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>11</td>\n",
      "      <td>at the very end, a discharge lamp removes all the electric charge from the drum making it ready to print the\\nnext page</td>\n",
      "    </tr>\n",
      "  </tbody>\n",
      "</table>\n",
      "\n",
      "\n",
      "Page Number: 110\n",
      "3 Hardware\n",
      "» When an electric current flows through the coil of wire that is wrapped around\n",
      "an iron core, the core becomes a temporary electromagnet; a permanent\n",
      "magnet is also positioned very close to this electromagnet.\n",
      "» As the electric current through the coil of wire varies, the induced magnetic\n",
      "field in the iron core also varies. This causes the iron core to be attracted\n",
      "towards the permanent magnet and as the current varies this will cause the\n",
      "iron core to vibrate.\n",
      "» Since the iron core is attached to a cone (made of paper or thin synthetic\n",
      "material), this causes the cone to vibrate, producing sound waves.\n",
      "Activity 3.4\n",
      "1 a Explain the main differences in operation of a laser printer compared to an\n",
      "inkjet printer.\n",
      "b i Name one application of a laser printer and one application of an inkjet printer.\n",
      "ii For each of your named applications in b i, give a reason why the chosen\n",
      "2 The nine stages in printing a page using an inkjet printer are shown below. The\n",
      "nine stages are NOT in the correct order.\n",
      "By writing the letters A to I, put each of the stages into the correct order.\n",
      "– the data is then sent to the printer and it is stored in a temporary memory\n",
      "known as a printer buffer\n",
      "B – as the sheet of paper is fed through the printer, the print head moves from\n",
      "side to side across the paper printing the text or image; the four ink colours are\n",
      "sprayed in their exact amounts to produce the desired final colour\n",
      "– the data from the document is sent to a printer driver\n",
      "– once the printer buffer is empty, the printer sends an interrupt to the CPU\n",
      "in the computer; this is a request for more data to be sent to the printer; the\n",
      "whole process continues until the whole of the document has been printed\n",
      "E – the printer driver ensures that the data is in a format that the chosen\n",
      "printer can understand\n",
      "– at the end of each full pass of the print head, the paper is advanced very\n",
      "slightly to allow the next line to be printed; this continues until the whole page\n",
      "has been printed\n",
      "G – a check is made by the printer driver to ensure that the chosen printer is\n",
      "available to print (e.g. is it busy, is it off-line, is it out of ink, and so on)\n",
      "H – if there is more data in the printer buffer, then the whole process from\n",
      "stage 5 is repeated until the buffer is finally empty\n",
      "I – a sheet of paper is then fed into the main body of the printer; a sensor\n",
      "detects whether paper is available in the paper feed tray – if it is out of paper\n",
      "(or the paper is jammed) then an error message is sent back to the computer\n",
      "3 a Explain the difference between LED screens and LCD-LED backlit screens.\n",
      "b Modern LCD screens use blue-white LEDs as backlighting. Cold cathode ray\n",
      "(fluorescent) tubes were used. Give three advantages of using LEDs.\n",
      "4 Filipe has music stored on his computer’s backing store. He wishes to listen\n",
      "to his music through a pair of loudspeakers. Describe how the music, which is\n",
      "digitally stored can be played through his two analogue loudspeakers.\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Page Number: 104\n",
      "3 Hardware\n",
      "Inkjet and laser printers\n",
      "Inkjet printers\n",
      "Inkjet printers are essentially made up of:\n",
      "» a print head, which consists of nozzles that spray droplets of ink onto the\n",
      "paper to form characters\n",
      "» an ink cartridge or cartridges; either one cartridge for each colour (blue,\n",
      "yellow and magenta) and a black cartridge or one single cartridge containing\n",
      "all three colours + black (Note: some systems use six colours)\n",
      "» a stepper motor and belt, which moves the print head assembly across the\n",
      "page from side to side\n",
      "» a paper feed, which automatically feeds the printer with pages as they are\n",
      "required.\n",
      "The ink droplets are produced currently using two different technologies:\n",
      "Thermal bubble – tiny resistors create localised heat which makes the ink\n",
      "vaporise. This causes the ink to form a tiny bubble; as the bubble expands,\n",
      "some of the ink is ejected from the print head onto the paper. When the bubble\n",
      "collapses, a small vacuum is created which allows fresh ink to be drawn into the\n",
      "print head. This continues until the printing cycle is completed.\n",
      "Piezoelectric – a crystal is located at the back of the ink reservoir for each\n",
      "nozzle. The crystal is given a tiny electric charge which makes it vibrate. This\n",
      "vibration forces ink to be ejected onto the paper; at the same time more ink is\n",
      "drawn in for further printing.\n",
      "When a user wishes to print a document using an inkjet printer, the following\n",
      "sequence of events takes place. Whatever technology is used, the basic steps in\n",
      "the printing process are the same.\n",
      "\n",
      "\n",
      "\n",
      "<table border=\"1\" class=\"dataframe\">\n",
      "  <tbody>\n",
      "    <tr>\n",
      "      <td>Stage in\\nprocess</td>\n",
      "      <td>Description of what happens</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>1</td>\n",
      "      <td>the data from the document is sent to a printer driver</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>2</td>\n",
      "      <td>the printer driver ensures that the data is in a format that the chosen printer can understand</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>3</td>\n",
      "      <td>a check is made by the printer driver to ensure that the chosen printer is available to print (e.g. is it busy, is it off-\\nline, is it out of ink, and so on)</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>4</td>\n",
      "      <td>the data is then sent to the printer and it is stored in a temporary memory known as a printer buffer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>5</td>\n",
      "      <td>a sheet of paper is then fed into the main body of the printer; a sensor detects whether paper is available in the\\npaper feed tray – if it is out of paper (or the paper is jammed) then an error message is sent back to the computer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>6</td>\n",
      "      <td>as the sheet of paper is fed through the printer, the print head moves from side to side across the paper printing\\nthe text or image; the four ink colours are sprayed in their exact amounts to produce the desired final colour</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>7</td>\n",
      "      <td>at the end of each full pass of the print head, the paper is advanced very slightly to allow the next line to be\\nprinted; this continues until the whole page has been printed</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>8</td>\n",
      "      <td>if there is more data in the printer buffer, then the whole process from stage 5 is repeated until the buffer is\\nfinally empty</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>9</td>\n",
      "      <td>once the printer buffer is empty, the printer sends an interrupt to the CPU in the computer; this is a request\\nfor more data to be sent to the printer; the whole process continues until the whole of the document has been\\nprinted</td>\n",
      "    </tr>\n",
      "  </tbody>\n",
      "</table>\n",
      "\n",
      "*\n",
      "*\n",
      "*\n",
      "\n",
      "FINAL PROMPT\n",
      "\n",
      "Don't try to make up an answer, if you don't know just say that you don't know.\n",
      "Answer in the same language the question was asked.\n",
      "Use only the following pieces of context to answer the question at the end.\n",
      "\n",
      " 3.2 Input and output devices\n",
      "Laser printers\n",
      "Laser printers use dry powder ink rather than liquid ink and make use of the\n",
      "properties of static electricity to produce the text and images. Unlike inkjet\n",
      "printers, laser printers print the whole page in one go. Colour laser printers\n",
      "use 4 toner cartridges – blue, cyan, magenta and black. Although the actual\n",
      "technology is different to monochrome printers, the printing method is similar\n",
      "but coloured dots are used to build up the text and images.\n",
      "The following table describes briefly the stages that occur when a document is\n",
      "printed using a laser printer:\n",
      "Applications of inkjet and laser printers\n",
      "The choice of whether to use an inkjet printer or a laser printer depends\n",
      "on which features make it the most appropriate output device for the given\n",
      "application.\n",
      "Inkjet printer – inkjet printers are often used for printing one-off photos\n",
      "or where only a few pages of good quality, colour printing is needed; the\n",
      "small ink cartridges or small paper trays would not be an issue with such\n",
      "applications.\n",
      "Laser printer – these devices produce high quality printouts and are very fast\n",
      "when making multiple copies of a document; any application that needs high\n",
      "volume printing (in colour or monochrome) would choose the laser printer\n",
      "(for example, producing a large number of high-quality flyers or posters\n",
      "for advertising). Laser printers have two advantages: they have large toner\n",
      "cartridges and large paper trays (often holding more than a ream of paper).\n",
      "\n",
      "\n",
      "\n",
      "<table border=\"1\" class=\"dataframe\">\n",
      "  <tbody>\n",
      "    <tr>\n",
      "      <td>Stage in\\nprocess</td>\n",
      "      <td>Description of what happens</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>1</td>\n",
      "      <td>the data from the document is sent to a printer driver</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>2</td>\n",
      "      <td>the printer driver ensures that the data is in a format that the chosen printer can understand</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>3</td>\n",
      "      <td>a check is made by the printer driver to ensure that the chosen printer is available to print (e.g. is it busy, is it\\noff-line, is it out of ink, and so on)</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>4</td>\n",
      "      <td>the data is then sent to the printer and it is stored in a temporary memory known as a printer buffer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>5</td>\n",
      "      <td>the start of the printing process involves a printing drum being given a positive charge; as this drum rotates, a\\nlaser beam is scanned across it removing the positive charge in certain areas; this leaves negatively charged\\nareas that exactly match the text/images of the page to be printed</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>6</td>\n",
      "      <td>the drum is then coated with positively charged toner (powdered ink); since the toner is positively charged, it\\nonly sticks to the negatively charged parts of the drum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>7</td>\n",
      "      <td>a negatively charged sheet of paper is then rolled over the drum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>8</td>\n",
      "      <td>the toner on the drum now sticks to the paper to produce an exact copy of the page sent to the printer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>9</td>\n",
      "      <td>to prevent the paper sticking to the drum, the electric charge on the paper is removed after one rotation of the\\ndrum</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>10</td>\n",
      "      <td>the paper finally goes through a fuser which is a set of heated rollers; the heat melts the ink so that it fixes\\npermanently to the paper</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>11</td>\n",
      "      <td>at the very end, a discharge lamp removes all the electric charge from the drum making it ready to print the\\nnext page</td>\n",
      "    </tr>\n",
      "  </tbody>\n",
      "</table> 3 Hardware\n",
      "» When an electric current flows through the coil of wire that is wrapped around\n",
      "an iron core, the core becomes a temporary electromagnet; a permanent\n",
      "magnet is also positioned very close to this electromagnet.\n",
      "» As the electric current through the coil of wire varies, the induced magnetic\n",
      "field in the iron core also varies. This causes the iron core to be attracted\n",
      "towards the permanent magnet and as the current varies this will cause the\n",
      "iron core to vibrate.\n",
      "» Since the iron core is attached to a cone (made of paper or thin synthetic\n",
      "material), this causes the cone to vibrate, producing sound waves.\n",
      "Activity 3.4\n",
      "1 a Explain the main differences in operation of a laser printer compared to an\n",
      "inkjet printer.\n",
      "b i Name one application of a laser printer and one application of an inkjet printer.\n",
      "ii For each of your named applications in b i, give a reason why the chosen\n",
      "2 The nine stages in printing a page using an inkjet printer are shown below. The\n",
      "nine stages are NOT in the correct order.\n",
      "By writing the letters A to I, put each of the stages into the correct order.\n",
      "– the data is then sent to the printer and it is stored in a temporary memory\n",
      "known as a printer buffer\n",
      "B – as the sheet of paper is fed through the printer, the print head moves from\n",
      "side to side across the paper printing the text or image; the four ink colours are\n",
      "sprayed in their exact amounts to produce the desired final colour\n",
      "– the data from the document is sent to a printer driver\n",
      "– once the printer buffer is empty, the printer sends an interrupt to the CPU\n",
      "in the computer; this is a request for more data to be sent to the printer; the\n",
      "whole process continues until the whole of the document has been printed\n",
      "E – the printer driver ensures that the data is in a format that the chosen\n",
      "printer can understand\n",
      "– at the end of each full pass of the print head, the paper is advanced very\n",
      "slightly to allow the next line to be printed; this continues until the whole page\n",
      "has been printed\n",
      "G – a check is made by the printer driver to ensure that the chosen printer is\n",
      "available to print (e.g. is it busy, is it off-line, is it out of ink, and so on)\n",
      "H – if there is more data in the printer buffer, then the whole process from\n",
      "stage 5 is repeated until the buffer is finally empty\n",
      "I – a sheet of paper is then fed into the main body of the printer; a sensor\n",
      "detects whether paper is available in the paper feed tray – if it is out of paper\n",
      "(or the paper is jammed) then an error message is sent back to the computer\n",
      "3 a Explain the difference between LED screens and LCD-LED backlit screens.\n",
      "b Modern LCD screens use blue-white LEDs as backlighting. Cold cathode ray\n",
      "(fluorescent) tubes were used. Give three advantages of using LEDs.\n",
      "4 Filipe has music stored on his computer’s backing store. He wishes to listen\n",
      "to his music through a pair of loudspeakers. Describe how the music, which is\n",
      "digitally stored can be played through his two analogue loudspeakers.\n",
      "\n",
      " 3 Hardware\n",
      "Inkjet and laser printers\n",
      "Inkjet printers\n",
      "Inkjet printers are essentially made up of:\n",
      "» a print head, which consists of nozzles that spray droplets of ink onto the\n",
      "paper to form characters\n",
      "» an ink cartridge or cartridges; either one cartridge for each colour (blue,\n",
      "yellow and magenta) and a black cartridge or one single cartridge containing\n",
      "all three colours + black (Note: some systems use six colours)\n",
      "» a stepper motor and belt, which moves the print head assembly across the\n",
      "page from side to side\n",
      "» a paper feed, which automatically feeds the printer with pages as they are\n",
      "required.\n",
      "The ink droplets are produced currently using two different technologies:\n",
      "Thermal bubble – tiny resistors create localised heat which makes the ink\n",
      "vaporise. This causes the ink to form a tiny bubble; as the bubble expands,\n",
      "some of the ink is ejected from the print head onto the paper. When the bubble\n",
      "collapses, a small vacuum is created which allows fresh ink to be drawn into the\n",
      "print head. This continues until the printing cycle is completed.\n",
      "Piezoelectric – a crystal is located at the back of the ink reservoir for each\n",
      "nozzle. The crystal is given a tiny electric charge which makes it vibrate. This\n",
      "vibration forces ink to be ejected onto the paper; at the same time more ink is\n",
      "drawn in for further printing.\n",
      "When a user wishes to print a document using an inkjet printer, the following\n",
      "sequence of events takes place. Whatever technology is used, the basic steps in\n",
      "the printing process are the same.\n",
      "\n",
      "\n",
      "\n",
      "<table border=\"1\" class=\"dataframe\">\n",
      "  <tbody>\n",
      "    <tr>\n",
      "      <td>Stage in\\nprocess</td>\n",
      "      <td>Description of what happens</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>1</td>\n",
      "      <td>the data from the document is sent to a printer driver</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>2</td>\n",
      "      <td>the printer driver ensures that the data is in a format that the chosen printer can understand</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>3</td>\n",
      "      <td>a check is made by the printer driver to ensure that the chosen printer is available to print (e.g. is it busy, is it off-\\nline, is it out of ink, and so on)</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>4</td>\n",
      "      <td>the data is then sent to the printer and it is stored in a temporary memory known as a printer buffer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>5</td>\n",
      "      <td>a sheet of paper is then fed into the main body of the printer; a sensor detects whether paper is available in the\\npaper feed tray – if it is out of paper (or the paper is jammed) then an error message is sent back to the computer</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>6</td>\n",
      "      <td>as the sheet of paper is fed through the printer, the print head moves from side to side across the paper printing\\nthe text or image; the four ink colours are sprayed in their exact amounts to produce the desired final colour</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>7</td>\n",
      "      <td>at the end of each full pass of the print head, the paper is advanced very slightly to allow the next line to be\\nprinted; this continues until the whole page has been printed</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>8</td>\n",
      "      <td>if there is more data in the printer buffer, then the whole process from stage 5 is repeated until the buffer is\\nfinally empty</td>\n",
      "    </tr>\n",
      "    <tr>\n",
      "      <td>9</td>\n",
      "      <td>once the printer buffer is empty, the printer sends an interrupt to the CPU in the computer; this is a request\\nfor more data to be sent to the printer; the whole process continues until the whole of the document has been\\nprinted</td>\n",
      "    </tr>\n",
      "  </tbody>\n",
      "</table>\n",
      "\n",
      "Question: \n",
      "How many steps are there in a laser printing process?\n",
      "\n",
      "Answer:\n",
      "\n",
      "\n",
      "\n",
      "Page Numbers:  105 110 104\n",
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    }
   ],
   "source": [
    "query = \"\"\"\n",
    "How many steps are there in a laser printing process?\n",
    "\"\"\"\n",
    "\n",
    "docs = retriever.get_relevant_documents(query)\n",
    "merged_context = ''\n",
    "reference = 'Page Numbers: '\n",
    "for doc in docs:\n",
    "    merged_context = merged_context + ' ' + doc.page_content\n",
    "    reference = reference + ' ' + str(doc.metadata['page'])\n",
    "    print('\\n\\nPage Number: ' + str(doc.metadata['page']))\n",
    "    print(doc.page_content)\n",
    "\n",
    "\n",
    "final_prompt = prompt_template.format(context=merged_context, question=query)\n",
    "print('\\n*\\n*\\n*\\n')\n",
    "print('FINAL PROMPT')\n",
    "print(final_prompt)\n",
    "print('\\n\\n')\n",
    "print(reference)\n",
    "print('\\n\\n')\n",
    "import pyperclip\n",
    "pyperclip.copy(final_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}