This is a web scraper for a RAG AI implementation. If you are building a RAG AI, the data you get back is BY FAR the most important piece.

For more on this scraper and why it does what it does:

• Building a Speedy Resilient Web Scraper for RAG AI - Part 1: Preparing
• Building a Fast and Resilient Web Scraper for Your RAG AI - Part 2: Scaling Up

For more on the RAG AI it's built to support:

• Inside the Virtual College Advisor: A Deep Dive into RAG AI and Agent Technology

How it uses Agents:

• Agents in a RAG AI with Code Examples

Some of the evaluation steps done on the RAG AI:

• RAG AI Performance Evaluation: With and Without Agents and Reranking

Performance monitoring:

• Evaluating Vector Search Performance on a RAG AI: A Detailed Look