From c3db74b49ede052b1bacaf68dba23e0f41f71959 Mon Sep 17 00:00:00 2001 From: tryptofanik <49122854+tryptofanik@users.noreply.github.com> Date: Mon, 13 Jan 2025 13:38:56 +0100 Subject: [PATCH] Update docs (#7986) GitOrigin-RevId: 05d090e6e88eead576b75b28f6a09b82c106778a --- .../10.introduction/10.welcome.md | 2 + .../10.introduction/20.installation.md | 11 +++ .../10.introduction/30.pathway-overview.md | 39 ++++++++--- .../40.first_realtime_app_with_pathway.md | 58 ++++++++++----- .../10.introduction/50.concepts.md | 4 +- .../.iterate/article.py | 2 +- .../.vectorstore_pipeline/article.py | 4 +- .../4.user-guide/50.llm-xpack/10.overview.md | 70 ++++++++++++++----- .../6.ai-pipelines/20.quick-start.md | 3 +- 9 files changed, 144 insertions(+), 49 deletions(-) diff --git a/docs/2.developers/4.user-guide/10.introduction/10.welcome.md b/docs/2.developers/4.user-guide/10.introduction/10.welcome.md index 88153561..e0d2f2d2 100644 --- a/docs/2.developers/4.user-guide/10.introduction/10.welcome.md +++ b/docs/2.developers/4.user-guide/10.introduction/10.welcome.md @@ -59,6 +59,8 @@ width: '500' - [Examples](/developers/user-guide/introduction/first_realtime_app_with_pathway) - [Core concepts](/developers/user-guide/introduction/concepts) - [Why Pathway](/developers/user-guide/introduction/why-pathway) +- [Streaming and Static Modes](/developers/user-guide/introduction/streaming-and-static-modes) +- [Batch Processing](/developers/user-guide/introduction/batch-processing) - [Deployment](/developers/user-guide/deployment/cloud-deployment) - [LLM tooling](/developers/user-guide/llm-xpack/overview) diff --git a/docs/2.developers/4.user-guide/10.introduction/20.installation.md b/docs/2.developers/4.user-guide/10.introduction/20.installation.md index 6f6c86df..1992af0a 100644 --- a/docs/2.developers/4.user-guide/10.introduction/20.installation.md +++ b/docs/2.developers/4.user-guide/10.introduction/20.installation.md @@ -16,6 +16,8 @@ To quickly get started with Pathway, you can install it via pip with the followi pip install -U pathway ``` +It will install all basic dependencies to run your Pathway pipelines, including our powerful Rust engine. + ::article-img @@ -54,6 +56,15 @@ Pick one and start your hands-on experience with Pathway today! ## Optional packages +We separated dependencies into several groups to allow users to have better control over what is installed. + +- For a standard usage of the Live Data Framework, run `pip install pathway` . No external LLM-based libraries will be installed. + +- To run the AI pipelines or to build Live AI systems, consider running `pip install "pathway[xpack-llm]"`, which will install all common LLM libraries, such as OpenAI, Langchain, etc. + +For more information, please visit our [pyproject.toml](https://github.com/pathwaycom/pathway/blob/main/pyproject.toml) file, which describes the content of all groups. + + | **Package** | **Installation Command** | **Description** | **Notes** | |--------------|--------------------------|------------------|-----------| | **Basic LLM Tooling** | `pip install "pathway[xpack-llm]"` | Install common LLM libraries (OpenAI, Langchain, LlamaIndex) | [Learn more](/developers/user-guide/llm-xpack/overview) / [Examples](/developers/templates?category=llm#llm) | diff --git a/docs/2.developers/4.user-guide/10.introduction/30.pathway-overview.md b/docs/2.developers/4.user-guide/10.introduction/30.pathway-overview.md index 407480af..88bd21e7 100644 --- a/docs/2.developers/4.user-guide/10.introduction/30.pathway-overview.md +++ b/docs/2.developers/4.user-guide/10.introduction/30.pathway-overview.md @@ -16,23 +16,40 @@ Then, you simply have to import Pathway as any other Python library: import pathway as pw ``` -## Connect to your data sources -In Pathway, you need to use a **connector** to create a table from a data source. -Connectors rely on [schemas](/developers/user-guide/connect/schema) to structure the data: +## Define your Data Schema: + +[Schemas](/developers/user-guide/connect/schema) in Pathway define the structure of your data tables. They describe the data types and names of the columns, ensuring that your data is well-organized and consistent. + +For instance, when reading a data source, you specify a schema to map the incoming data: + ```python class InputSchema(pw.Schema): colA: int colB: float colC: str - - -input_table = pw.io.csv.read('./data/', schema=InputSchema) ``` +Here, the InputSchema specifies three columns: colA (an integer), colB (a float) and colC (a string). +Schemas define the structure of the data, ensuring type safety and optimizing runtime performance. + Pathway supports the following basic [data types](/developers/user-guide/connect/datatypes): `bool`, `str`, `bytes`, `int`, and `float`. Pathway also supports more complex data types, such as the `Optional` data type or temporal data types (`datetime.datetime`). +## Tables: + +[Tables](https://pathway.com/developers/api-docs/pathway-table) are Pathway objects that can actually store the data. These are composed of columns, each of which keeps data of the same type, just like in relational databases. + + +## Connectors: +In Pathway, you need to use a **connector** to create a table from a data source. Connectors read and ingest, in real-time, data from your chosen data sources. + +Here's an example of a connector that uses `InputSchema` to read **CSV** files from the `./data/` directory and outputs a table: + +```python +input_table = pw.io.csv.read('./data/', schema=InputSchema) +``` + Here is a small sample of Pathway input connectors: | Input Connectors | Example | @@ -46,11 +63,17 @@ Pathway comes with many more connectors, including an [Airbyte connector](/devel You can find the list of available connectors on our [connector page](/developers/user-guide/connect/pathway-connectors). ## Transformations -Once your input data is defined, you can define your data pipeline using [Pathway transformations](/developers/user-guide/introduction/concepts#processing-the-data-with-transformations): +Once your input data is specified, you can now define your data pipeline using Pathway [transformations](/developers/user-guide/introduction/concepts#processing-the-data-with-transformations). These, under the hood, are written in Rust meaning that they are very efficient. + +Here is an example of a simple transformation composed of filtering and summing by groups: ```python filtered_table = input_table.filter(input_table.colA > 0) -result_table = filtered_table.groupby(filtered_table.colB).reduce(sum_val=pw.Reducers.sum(pw.this.colC)) +result_table = ( + filtered_table + .groupby(filtered_table.colB) + .reduce(sum_val=pw.Reducers.sum(pw.this.colC)) +) ``` Here is a small sample of the operations you can do in Pathway: diff --git a/docs/2.developers/4.user-guide/10.introduction/40.first_realtime_app_with_pathway.md b/docs/2.developers/4.user-guide/10.introduction/40.first_realtime_app_with_pathway.md index 617279c8..87674dbb 100644 --- a/docs/2.developers/4.user-guide/10.introduction/40.first_realtime_app_with_pathway.md +++ b/docs/2.developers/4.user-guide/10.introduction/40.first_realtime_app_with_pathway.md @@ -15,7 +15,7 @@ on a Python 3.10+ installation, and you are ready to roll! ## A simple sum example -Let's start with a simple sum over positive values stored in CSV files, and written to a JSON Lines file: +Let's start with a simple sum over positive values stored in CSV files, and written to a [JSON Lines](https://jsonlines.org/) file: ::article-img --- @@ -38,10 +38,11 @@ The aim is to combine the data from those two data sources and find the live mea This is how you can do the whole pipeline in Pathway: ```python -import pathway as pw # import Pathway +import pathway as pw # Declare the Schema of your tables using pw.Schema. -# There are two input tables: measurements and threshold. +# There are two input tables: (1) measurements which is +# live stream and (2) threshold which is a CSV that might be modified over time. # Both have two columns: a name (str) and a float. class MeasurementSchema(pw.Schema): name: str @@ -80,21 +81,36 @@ thresholds_table = pw.io.csv( ) # Joining tables on the column name -joined_table = measurements_table.join( # The left table is measurements_table (pw.left) - thresholds_table, # The right table is thresholds_table (pw.right) - pw.left.name==pw.right.name, # The join is done on the column name of each table -).select( # The columns of the joined table are chosen using select - *pw.left, # All the columns of measurements are kept - pw.right.threshold # The threshold column of the threshold table is kept +joined_table = ( + # The left table is measurements_table (referred as pw.left) + measurements_table + .join( + # The right table is thresholds_table (referred as pw.right) + thresholds_table, + # The join is done on the column name of each table + pw.left.name==pw.right.name, + ) + # The columns of the joined table are chosen using select + .select( + # All the columns of measurements are kept + *pw.left, + # The threshold column of the threshold table is kept + pw.right.threshold + ) ) -# Filtering value strictly higher than the threshold. -alerts_table = joined_values.filter( - pw.this.value > pw.this.threshold -).select(pw.this.name, pw.this.value) # Only name and value fields are kept +alerts_table = ( + joined_values + # Filtering value strictly higher than the threshold. + .filter(pw.this.value > pw.this.threshold) + # Only name and value fields are kept + .select(pw.this.name, pw.this.value) +) # Sending the results to another Kafka topic, on the same Kafka instance -pw.io.kafka.write(alerts_table, rdkafka_settings, topic_name="alerts_topic", format="json") +pw.io.kafka.write( + alerts_table, rdkafka_settings, topic_name="alerts_topic", format="json" +) # Launching Pathway computation. pw.run() @@ -148,20 +164,22 @@ Then the output is: {"name": "B", "value":10, "time":1, "diff":1} ``` The output contains two more fields: `time` and `diff`: -* `time` represents the time at which the update has happened. In practice, the time is a timestamp. -* `diff` represents whether the row represents an addition or a deletion. An update is represented by two rows: one to remove the old value, one to add the new values. Those two rows have the same time to ensure the atomicity of the operation. +* `time` represents the time at which the update happened. In practice, the time is a regular timestamp. +* `diff` informs whether the row should be an insertion (`diff = 1`) or a deletion (`diff = -1`). An update is represented by two rows: one to remove the old value, one to add the new values. Those two rows have the same time to ensure the atomicity of the operation. In this case, we assume the first values were computed at time 1. The value `diff` is equal to `1` as it is an insertion. -Suppose now that the thresholds are updated: +Suppose now that the thresholds have changed, the file got an update and now looks like that: ``` name, threshold "A", 7 "B", 11 ``` -Then the output is: +Connector will automatically detect any new files or modifications within `./threshold-data/` and update the tables accordingly. + +That will trigger the reexecution of join and filter giving this output: ``` {"name": "B", "value":10, "time":1, "diff":1} {"name": "B", "value":10, "time":2, "diff":-1} @@ -169,7 +187,9 @@ Then the output is: ``` There are two more lines: the old alert is removed (`diff=-1`) and a new one is inserted for the other event (`diff=1`). -Old values are still kept as the output is a log of insertion and suppression. +Old values are still kept as the output is a log of insertion and suppression allowing to have exhaustive information about what happened to our data. + +Keep in mind that some output connectors to external data storage system might take these `-1` and `+1` rows, link them by time, and represent as update operation (like in case of SQL database). ## Other examples ::container{.flex .gap-8 .items-center .w-full .justify-center} diff --git a/docs/2.developers/4.user-guide/10.introduction/50.concepts.md b/docs/2.developers/4.user-guide/10.introduction/50.concepts.md index eedec509..83259e40 100644 --- a/docs/2.developers/4.user-guide/10.introduction/50.concepts.md +++ b/docs/2.developers/4.user-guide/10.introduction/50.concepts.md @@ -149,7 +149,9 @@ Here is a way to do it with Pathway, assuming a correct input table called `inpu ```python filtered_table = input_table.filter(input_table.age >= 0) -result_table = filtered_table.reduce(sum_age = pw.reducers.sum(filtered_table.age)) +result_table = filtered_table.reduce( + sum_age = pw.reducers.sum(filtered_table.age) +) ``` It's okay if you don't understand everything for now. diff --git a/docs/2.developers/4.user-guide/30.data-transformation/.iterate/article.py b/docs/2.developers/4.user-guide/30.data-transformation/.iterate/article.py index bd592de9..0ba889d7 100644 --- a/docs/2.developers/4.user-guide/30.data-transformation/.iterate/article.py +++ b/docs/2.developers/4.user-guide/30.data-transformation/.iterate/article.py @@ -175,7 +175,7 @@ def cc(vertices: pw.Table, edges: pw.Table) -> pw.Table: # %% [markdown] -# In an iteration step, the `edges` table is joined with the `vertices` table to get the representatives of neighbors in the graph. Then `groupby` is performed on `edges_with_repr` to get a minimal representative for each vertex. A new ID is assigned based on column `a` - vertex label. It is assigned in exactly the same way it is done above when creating a table. It allows you to have the same set of keys in the `vertices_updated` table as in the `vertices` table. However, Pathway is not that clever to deduce that the keys are exactly the same in these two tables. That's why it has to be additionally told they are the same, by using [`with_universe_of`](/developers/api-docs/pathway-table#pathway.Table.with_universe_of). +# In an iteration step, the `edges` table is joined with the `vertices` table to get the representatives of neighbors in the graph. Then `groupby` is performed on `edges_with_repr` to get a minimal representative for each vertex. A new ID is assigned based on column `a` - vertex label. It is assigned in exactly the same way it is done above when creating a table. It allows you to have the same set of keys in the `vertices_updated` table as in the `vertices` table. However, Pathway is not that clever to deduce that the keys are exactly the same in these two tables. That's why it has to be additionaly told they are the same, by using [`with_universe_of`](/developers/api-docs/pathway-table#pathway.Table.with_universe_of). # Preserving the set of keys is important in `iterate`. The iteration can stop only stop if there are no updates in any of the records. The records correspondence between iterations is determined using their IDs. If a record with one ID disappears and a record with a new ID appears, Pathway decides that something is still changing and the computation has to continue (even if the contents of the two rows are the same). It is possible to change the set of keys used in `iterate` but in the end the set of keys has to stop changing anyway. You can see that in the next example on computing shortest distances in a graph. diff --git a/docs/2.developers/4.user-guide/50.llm-xpack/.vectorstore_pipeline/article.py b/docs/2.developers/4.user-guide/50.llm-xpack/.vectorstore_pipeline/article.py index d1952e1a..559572d8 100644 --- a/docs/2.developers/4.user-guide/50.llm-xpack/.vectorstore_pipeline/article.py +++ b/docs/2.developers/4.user-guide/50.llm-xpack/.vectorstore_pipeline/article.py @@ -26,7 +26,7 @@ # %% [markdown] # # Always Up-to-date Data Indexing pipeline # -# This showcase shows how to use Pathway to deploy a live data indexing pipeline, which can be queried like a typical vector store. However, under the hood, Pathway updates the index on each data change, always giving up-to-date answers. +# This showcase demonstrates how to use Pathway to deploy a live data indexing pipeline that can be queried similarly to a typical vector store. Unlike traditional approaches, Pathway updates the index with every data change, ensuring consistently up-to-date answers. # # ::article-img # --- @@ -61,7 +61,7 @@ # Then download sample data. # %% -# _MD_SHOW_!pip install pathway litellm +# _MD_SHOW_!pip install "pathway[xpack-llm,xpack-llm-docs]" # _MD_SHOW_ !pip install unstructured[all-docs] # _MD_SHOW_!mkdir -p sample_documents # _MD_SHOW_![ -f sample_documents/repo_readme.md ] || wget 'https://gist.githubusercontent.com/janchorowski/dd22a293f3d99d1b726eedc7d46d2fc0/raw/pathway_readme.md' -O 'sample_documents/repo_readme.md' diff --git a/docs/2.developers/4.user-guide/50.llm-xpack/10.overview.md b/docs/2.developers/4.user-guide/50.llm-xpack/10.overview.md index 340ca3ef..fbc19adc 100644 --- a/docs/2.developers/4.user-guide/50.llm-xpack/10.overview.md +++ b/docs/2.developers/4.user-guide/50.llm-xpack/10.overview.md @@ -13,12 +13,49 @@ The LLM xpack provides you all the tools you need to use Large Language Models i You can find ready-to-run LLM and RAG examples on our [App Templates page](/developers/templates?category=llm#llm). +In order to install Pathway along with LLM xpack functionality simply run: +``` +`pip install "pathway[xpack-llm]"` +``` + +or + +``` +`pip install "pathway[all]"` +``` + + +## Queries table + +Let's start with preparing a Pathway table containing queries that we want to send to the LLM. + +```python +import pathway as pw +query = pw.debug.table_from_markdown(''' +message +How many 'r' there are in 'strawberry'? +''') +``` + ## Wrappers for LLMs -Out of the box, the LLM xpack provides wrappers for text generation and embedding LLMs. For text generation, you can use native wrappers for the OpenAI chat model and HuggingFace models running locally. Many other popular models, including Azure OpenAI, HuggingFace (when using their API) or Gemini can be used with the wrapper for LiteLLM. To check the full list of providers supported by LiteLLM check [LiteLLM documentation](https://docs.litellm.ai/docs/providers). +Out of the box, the LLM xpack provides wrappers for text generation and embedding LLMs. For text generation, you can use native wrappers for the OpenAI chat model and HuggingFace models running locally. Many other popular models, including Azure OpenAI, HuggingFace (when using their API) or Gemini can be used with the `LiteLLM` wrapper. To check the full list of providers supported by LiteLLM check [LiteLLM documentation](https://docs.litellm.ai/docs/providers). + +### UDFs + +Each wrapper is a [UDFClass](/developers/api-docs/pathway#pathway.UDF) (User Defined Function), which allows users to define their own functions to interact with Pathway objects. A UDF, in general, is any function that takes some input, processes it, and returns an output. In the context of the Pathway library, UDFs enable seamless integration of custom logic, such as invoking LLMs for specific tasks. + +In particular a UDF can serve as a wrapper for LLM calls, allowing users to pass prompts or other inputs to a model and retrieve the corresponding outputs. This design makes it easy to interact with Pathway tables and columns while incorporating the power of LLMs. + +### OpenAIChat + +Pathway comes with many predefined UDFs, including those that manage connections with OpenAI models. + +To use a wrapper, first create an instance of the wrapper, which you can then apply to a column containing prompts. For OpenAI, you create a wrapper using the [`OpenAIChat` class](/developers/api-docs/pathway-xpacks-llm/llms#pathway.xpacks.llm.llms.OpenAIChat). -Each wrapper is a [UDFClass](/developers/api-docs/pathway#pathway.UDF). To use it, first create an instance of the wrapper, which you can then apply to a column with prompts. For OpenAI, you create a wrapper with [`OpenAIChat` class](/developers/api-docs/pathway-xpacks-llm/llms#pathway.xpacks.llm.llms.OpenAIChat). ```python +from pathway.xpacks.llm.llms import OpenAIChat + model = OpenAIChat( model="gpt-3.5-turbo", api_key=os.environ["OPENAI_API_KEY"], # Read OpenAI API key from environmental variables @@ -28,16 +65,11 @@ responses = query.select(result=model(pw.this.messages)) ``` ### Preparing queries -`OpenAIChat` expects messages to be in the format required by [OpenAI API](https://platform.openai.com/docs/api-reference/chat/create) - that is a list of dictionaries, where each dictionary is one message in the conversation so far. If you want to ask single questions use [`pw.xpacks.llm.llm.prompt_chat_single_qa`](/developers/api-docs/pathway-xpacks-llm/llms#pathway.xpacks.llm.llms.prompt_chat_single_qa) to wrap them. +`OpenAIChat` expects messages to be in the format required by [OpenAI API](https://platform.openai.com/docs/api-reference/chat/create) - that is a list of dictionaries, where each dictionary is one message in the conversation so far. If you want to ask single questions use [`pw.xpacks.llm.llm.prompt_chat_single_qa`](/developers/api-docs/pathway-xpacks-llm/llms#pathway.xpacks.llm.llms.prompt_chat_single_qa) to wrap them so that they match the format expected by OpenAI API. ```python -from pathway.xpack.llm.llms import prompt_chat_single_qa +from pathway.xpacks.llm.llms import prompt_chat_single_qa - -model = OpenAIChat( - model="gpt-3.5-turbo", - api_key=os.environ["OPENAI_API_KEY"], # Read OpenAI API key from environmental variables -) # Column `prompt` holds strings with questions to be sent to OpenAI chat responses = query.select(result=model(prompt_chat_single_qa(pw.this.prompt))) ``` @@ -91,7 +123,7 @@ embedder = OpenAIEmbedder( responses = documents.select(result=embedder(pw.this.text)) ``` -### Asynchrony +## Asynchrony Wrapper for OpenAI and LiteLLM, both for chat and embedding, are asynchronous, and Pathway allows you to set three parameters to set their behavior. These are: - `capacity`, which sets the number of concurrent operations allowed, - `retry_strategy`, which sets the strategy for handling retries in case of failures, @@ -101,13 +133,16 @@ These three parameters need to be set during the initialization of the wrapper. ```python model = OpenAIChat( - capacity=5, # maximum concurrent operations is 5 + # maximum concurrent operations is 10 + capacity=10, # in case of failure, retry 5 times, each time waiting twice as long before retrying retry_strategy=pw.udfs.ExponentialBackoffRetryStrategy(max_retries=5, initial_delay=1000, backoff_factor=2), # if PATHWAY_PERSISTENT_STORAGE is set, then it is used to cache the calls cache_strategy=pw.udfs.DefaultCache(), + # select the model model="gpt-3.5-turbo", - api_key=os.environ["OPENAI_API_KEY"], # Read OpenAI API key from environmental variables + # read OpenAI API key from environmental variables + api_key=os.environ["OPENAI_API_KEY"], ) responses = query.select(result=model(prompt_chat_single_qa(pw.this.prompt))) ``` @@ -117,10 +152,10 @@ You can now combine these wrappers to create an LLM pipeline using Pathway. To l ## Preparing documents for LLMs -The Pathway xpack for LLMs provides tools for preparing your documents and texts in order to use them with LLMs. You can use [`ParseUnstructured`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.ParseUnstructured) for parsing your documents into texts and [`TokenCountSplitter`](/developers/api-docs/pathway-xpacks-llm/splitters#pathway.xpacks.llm.splitters.TokenCountSplitter) for dividing texts into smaller chunks. +The Pathway xpack for LLMs provides tools for preparing your documents and texts in order to use them with LLMs. You can use one of our UDFs like [`ParseUnstructured`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.ParseUnstructured) for parsing your documents into texts and [`TokenCountSplitter`](/developers/api-docs/pathway-xpacks-llm/splitters#pathway.xpacks.llm.splitters.TokenCountSplitter) for dividing texts into smaller chunks. ### Parsing documents -Use the [`ParseUnstructured` class](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.ParseUnstructured) to parse documents in Pathway. Underneath, it uses the [Unstructured](https://unstructured.io/) library to parse your documents. To use it, you need to read the contents of a file into a Pathway Table using any connector of your choice. Then, apply an instance of the `ParseUnstructured` class to get a Pathway Table with parsed content of documents. `ParseUnstructured` has an argument `mode` which takes one of three values: `single`, `paged` or `elements`. If set to `single`, the whole document is returned as one string, if set to `paged` then there is a string for each page in the document, and if set to `elements` then Unstructured's division into elements is preserved. The `mode` argument can be set either during initialization or application of `ParseUnstructured`. +Use the [`ParseUnstructured` class](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.ParseUnstructured) to parse documents in Pathway. Underneath, it utilizes the [Unstructured](https://unstructured.io/) library to parse your documents. To use it, you need to read the contents of a file into a Pathway Table using any connector of your choice. Then, apply an instance of the `ParseUnstructured` class to get a Pathway Table with parsed content of documents. `ParseUnstructured` has an argument `mode` which takes one of three values: `single`, `paged` or `elements`. If set to `single`, the whole document is returned as one string, if set to `paged` then there is a string for each page in the document, and if set to `elements` then Unstructured's division into elements is preserved. The `mode` argument can be set either during initialization or execution of `ParseUnstructured`. ```python import os @@ -176,10 +211,11 @@ Vector Store offer integrations with both LlamaIndex and LangChain. These allow ## Rerankers -Rerankers allow you to evaluate whether a document is relevant to the question asked. A typical application of rerankers is to implement a two-stage retrieval. First, some number of documents is retrieved from a vector store, purposely more than you wish to embed in the query as a context. Then all these documents are ranked with a reranker, and then only the best of them are left, while the rest are discarded. +Rerankers evaluate the relevance of documents to a given query, commonly used in a two-stage retrieval process. Initially, a vector store retrieves a broad set of documents, typically more than needed for query context, many of which may be irrelevant. This occurs because indexing flattens a document's entire meaning into a single vector, which can reduce accuracy. +Rerankers refine this by reassessing each document’s relevance. Running a reranker is more expensive than running an index-based retrieval, but it usually provides a significant improvement in accuracy. Pathway offers three rerankers: -- [`LLMReranker`](/developers/api-docs/pathway-xpacks-llm/rerankers#pathway.xpacks.llm.rerankers.LLMReranker) asks an LLN chat of your choice to rank the relevance of a document against a query on a scale from 1 to 5, +- [`LLMReranker`](/developers/api-docs/pathway-xpacks-llm/rerankers#pathway.xpacks.llm.rerankers.LLMReranker) asks an LLM chat of your choice to rank the relevance of a document against a query on a scale from 1 to 5, - [`CrossEncoderReranker`](/developers/api-docs/pathway-xpacks-llm/rerankers#pathway.xpacks.llm.rerankers.CrossEncoderReranker) is a wrapper on [CrossEncoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) from the `sentence_transformers`, - [`EncoderReranker`](/developers/api-docs/pathway-xpacks-llm/rerankers#pathway.xpacks.llm.rerankers.EncoderReranker) uses embeddings from the [sentence_transformers library](https://www.sbert.net/docs/sentence_transformer/usage/usage.html) to calculate the relevance of a document against a query. @@ -187,7 +223,7 @@ Additionally, once you rank the documents, you can use [`rerank_topk_filter`](/d ## Modular RAGs -To combine all the pieces into a RAG, you can use one of modular RAGs available in the LLM xpack. [`BaseRAGQuestionAnswerer`](/developers/api-docs/pathway-xpacks-llm/question_answering#pathway.xpacks.llm.question_answering.BaseRAGQuestionAnswerer) is a standard RAG, that given a query obtains `k` best documents from the vector store, and sends them along the question to the LLM chat. [`AdaptiveRAGQuestionAnswerer`](/developers/api-docs/pathway-xpacks-llm/question_answering#pathway.xpacks.llm.question_answering.AdaptiveRAGQuestionAnswerer) tries to limit the number of documents sent to the LLM chat to save tokens - to do that it initially sends only a small number of documents to the chat, which is increased until an answer is found. To read more, why that can save tokens without sacrificing accuracy, check our [showcase on Adaptive RAG](/developers/templates/adaptive-rag). +To combine all the pieces into a [RAG](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) (Retrieval Augmented Generation), you can use one of modular RAGs available in the LLM xpack. [`BaseRAGQuestionAnswerer`](/developers/api-docs/pathway-xpacks-llm/question_answering#pathway.xpacks.llm.question_answering.BaseRAGQuestionAnswerer) is a standard RAG, that given a query obtains `k` best documents from the vector store, and sends them along the question to the LLM chat. [`AdaptiveRAGQuestionAnswerer`](/developers/api-docs/pathway-xpacks-llm/question_answering#pathway.xpacks.llm.question_answering.AdaptiveRAGQuestionAnswerer) tries to limit the number of documents sent to the LLM chat to save tokens - to do that it initially sends only a small number of documents to the chat, which is increased until an answer is found. To read more, why that can save tokens without sacrificing accuracy, check our [showcase on Adaptive RAG](/developers/templates/adaptive-rag). Both these RAGs are customizable with an LLM model used to answer questions, a vector store for retrieving documents and templates for embedding context chunks in the question. diff --git a/docs/2.developers/6.ai-pipelines/20.quick-start.md b/docs/2.developers/6.ai-pipelines/20.quick-start.md index 20745eb8..56e78361 100644 --- a/docs/2.developers/6.ai-pipelines/20.quick-start.md +++ b/docs/2.developers/6.ai-pipelines/20.quick-start.md @@ -22,7 +22,8 @@ To get started, you'll need: 1. Docker (recommended) will install all dependencies automatically 2. Python 3.8+ with [Pathway](/developers/user-guide/introduction/installation) if you prefer a local setup. -**Note**: if you are using Pathway locally, you will need to install Pathway LLM xpack: +**Note**: if you are using Pathway locally, you will need to install Pathway LLM xpack with: + ``` pip install pathway[all] ```