Inverted Index Web Server

A web server for building and serving an inverted index of text documents. The system provides fast text search capabilities with support for parallel processing and real-time index updates.

System Overview

This is a web server that:

• Builds an inverted index from text files
• Provides fast search capabilities
• Automatically updates the index when new files appear
• Efficiently utilizes available CPU resources through a thread pool

Technical Specifications

• Programming Language: Rust
• Concurrency: Multi-threaded processing with configurable thread count
• Storage: In-memory with thread-safe access
• API: REST HTTP

Dependencies

• Rust 1.70 or newer
• Operating System: Linux, macOS, or Windows
• Sufficient RAM for index storage (depends on document size)

API Endpoints

Search

GET /search?q={term}

Returns a list of documents containing the search term, along with relevance information and term positions.

Document Download

GET /document?docID={docID}

Returns the content of a document by its name.

Installation and Running

1. Clone the repository:

git clone https://github.com/twentyone212121/search-engine
cd search-engine

2. Build the project:

cargo build --release

3. Run the server with required parameters:

./target/release/inverted-index-server --corpus-dir /path/to/documents --port 8080 --ip 127.0.0.1 --thread-num 4

Command Line Parameters

The server accepts the following command line arguments:

• --ip: IP address to bind to (default: 127.0.0.1)
• --port: HTTP server port (default: 8080)
• --corpus-dir: Path to documents directory (required)
• --thread-num: Number of worker threads (default: 4)

Example Usage

# Run with all parameters specified
./target/release/inverted-index-server --corpus-dir ./documents --port 3000 --ip 0.0.0.0 --thread-num 8

# Run with minimal parameters (uses defaults)
./target/release/inverted-index-server --corpus-dir ./documents

License

MIT