+ +
+ +--- + +## 4. Prepare Data, Create Collection, and Insert Data + +### 4.1 Prepare the Data +We use the FAQ pages from the [Milvus Documentation 2.4.x](https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip) as the dataset for this example. + +#### Download and Extract Data: +```bash +wget https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip +unzip -q milvus_docs_2.4.x_en.zip -d milvus_docs +``` + +#### Process Markdown Files: +```python +from glob import glob + +text_lines = [] +for file_path in glob("milvus_docs/en/faq/*.md", recursive=True): + with open(file_path, "r") as file: + file_text = file.read() + text_lines += file_text.split("# ") +``` + +--- + +### 4.2 Generate Embeddings +Define a embedding model to generate text embeddings using the `milvus_model`. We use the `DefaultEmbeddingFunction` model as an example, which is a pre-trained and lightweight embedding model. +```python +from pymilvus import model as milvus_model + +embedding_model = milvus_model.DefaultEmbeddingFunction() + +# Generate test embedding +test_embedding = embedding_model.encode_queries(["This is a test"])[0] +embedding_dim = len(test_embedding) +print(embedding_dim) +print(test_embedding[:10]) +``` +#### Output: +``` +768 +[-0.04836066 0.07163023 -0.01130064 -0.03789345 -0.03320649 -0.01318448 + -0.03041712 -0.02269499 -0.02317863 -0.00426028] +``` +--- + +### 4.3 Create Collection +Connect to Milvus and create a collection: +```python +from pymilvus import MilvusClient + +# Connect to Milvus Standalone +client = MilvusClient(uri="http://localhost:19530") + +collection_name = "attu_tutorial" + +# Drop collection if it exists +if client.has_collection(collection_name): + client.drop_collection(collection_name) + +# Create a new collection +client.create_collection( + collection_name=collection_name, + dimension=embedding_dim, + metric_type="IP", # Inner product distance + consistency_level="Strong" +) +``` + +--- + +### 4.4 Insert Data +Iterate through the text lines, create embeddings, and insert the data into Milvus: +```python +from tqdm import tqdm + +data = [] +doc_embeddings = embedding_model.encode_documents(text_lines) + +for i, line in enumerate(tqdm(text_lines, desc="Creating embeddings")): + data.append({"id": i, "vector": doc_embeddings[i], "text": line}) + +client.insert(collection_name=collection_name, data=data) +``` +--- + +### 4.5 Visualize Data and Schema + +Now we can visualize the data schema and inserted entities using Attu's interface. The schema displays defined fields, including an `id` field of type `Int64` and a `vector` field of type `FloatVector(768)` with an `Inner Product (IP)` metric. The collection is loaded with **72 entities**. + +Additionally, we can view the inserted data, including ID, vector embeddings, and dynamic fields storing metadata such as text content. The interface supports filtering and querying based on specified conditions or dynamic fields. ++ + +
+ + +## 5. Visualizing Search Results and Relationships + +Attu provides a powerful interface for visualizing and exploring data relationships. To examine the inserted data points and their similarity relationships, follow these steps: + +### 5.1 **Perform a Search** +Navigate to the **Vector Search** tab in Attu. +1. Click on the **Generate Random Data** button to create test queries. +2. Click **Search** to retrieve results based on the generated data. + +The results are displayed in a table, showing IDs, similarity scores, and dynamic fields for each matching entity. + ++ +
+ +--- + +### 5.2 **Explore Data Relationships** +Click the **Explore** button in the results panel to visualize the relationships between the query vector and the search results in a **knowledge graph-like structure**. + +- The **central node** represents the search vector. +- The **connected nodes** represent the search results, clicking them will display the detailed information of the corresponding node. + ++ +
+ +--- + +### 5.3 **Expand the Graph** +Double-click on any result node to expand its connections. This action reveals additional relationships between the selected node and other data points in the collection, creating a **larger, interconnected knowledge graph**. + +This expanded view allows for deeper exploration of how data points are related, based on vector similarity. + ++ +
+ +--- + +## 6. Conclusion + +Attu simplifies the management and visualization of vector data stored in Milvus. From data insertion to query execution and interactive exploration, it provides an intuitive interface for handling complex vector search tasks. With features like dynamic schema support, graphical search visualizations, and flexible query filters, Attu empowers users to analyze large-scale datasets effectively. + +By leveraging Attu’s visual exploration tools, users can better understand their data, identify hidden relationships, and make data-driven decisions. Start exploring your own datasets today with Attu and Milvus! + + +--- + + diff --git a/v2.5.x/site/en/tutorials/tutorials-overview.md b/v2.5.x/site/en/tutorials/tutorials-overview.md index 4cd4828c0..99aed5fd7 100644 --- a/v2.5.x/site/en/tutorials/tutorials-overview.md +++ b/v2.5.x/site/en/tutorials/tutorials-overview.md @@ -32,4 +32,6 @@ This page provides a list of tutorials for you to interact with Milvus. | [Text Search Engine](text_search_engine.md) | Semantic Search | vector search | | [Search Image by Text](text_image_search.md) | Semantic Search | vector search | | [Image Deduplication](image_deduplication_system.md) | Deduplication | vector search | +| [Quickstart with Attu](quickstart_with_attu.md) | Quickstart | vector search | +| [Use AsyncMilvusClient with asyncio](use-async-milvus-client-with-asyncio.md) | AsyncIO | AsyncIO, vector search | diff --git a/v2.5.x/site/en/tutorials/use-async-milvus-client-with-asyncio.md b/v2.5.x/site/en/tutorials/use-async-milvus-client-with-asyncio.md new file mode 100644 index 000000000..91128df62 --- /dev/null +++ b/v2.5.x/site/en/tutorials/use-async-milvus-client-with-asyncio.md @@ -0,0 +1,325 @@ +--- +id: use-async-milvus-client-with-asyncio.md +summary: AsyncMilvusClient is an asynchronous MilvusClient that offers a coroutine-based API for non-blocking access to Milvus via asyncio. In this article, you will learn about the process for calling the APIs that AsyncMilvusClient provides and the aspects to which you need to pay attention to. +title: Question Answering System +--- + +# Tutorial: Use AsyncMilvusClient with asyncio + +**AsyncMilvusClient** is an asynchronous MilvusClient that offers a coroutine-based API for non-blocking access to Milvus via [asyncio](https://docs.python.org/3/library/asyncio.html). In this article, you will learn about the process for calling the APIs that AsyncMilvusClient provides and the aspects to which you need to pay attention to. + +## Overview + +Asyncio is a library for writing concurrent code using **async**/**await** syntax and serves as the foundation for Milvus' high-performance asynchronous client, which will fit into your code library running on top of asyncio. + +The methods that AsyncMilvusClient provides have identical parameter sets and behaviors as those of MilvusClient. The only difference lies in the way you call them. The following table lists the methods available in AsyncMilvusClient. + +**Client** + + | ||
---|---|---|
| + + | + + |
**Collection & Partition** + + | ||
|
|
|
| + + | + + |
**Index** + + | ||
|
|
|
|
|
|
**Vector** + + | ||
|
|
|
|
|
|
| + + | + + |