web-reader

A Cloudflare Workers service for converting web pages to clean, LLM-friendly markdown content. Inspired by Jina Reader, this service provides an easy way to convert any webpage into clean, readable markdown format.

Features

• Simple URL-based Conversion: Simply prepend your deployed worker URL to any webpage URL
• Caching with D1: Automatically caches converted content in Cloudflare D1 for faster responses
• Cache Control: Use X-No-Cache header to bypass cache when needed
• Multiple Input Methods: Support for both GET and POST requests

Deployment

Prerequisites

• TinyGo
• Wrangler

Steps

1. Clone the repository:

git clone https://github.com/AdjectiveAllison/web-reader.git
cd web-reader

2. Deploy the worker:

wrangler deploy

3. Initialize the D1 cache database:

wrangler d1 migrations apply web-reader-cache --remote

Usage

Simple GET Request

Simply prepend your worker URL to the target webpage URL:

curl "https://<your-worker>.workers.dev/https://example.com"

POST Request

You can also send a POST request with the URL in the request body:

curl -X POST "https://<your-worker>.workers.dev/convert" \
-H "Content-Type: application/json" \
-d '{
    "url": "https://example.com"
}'

Bypass Cache

To skip cached content and force a fresh fetch, use the X-No-Cache header:

curl -H "X-No-Cache: true" "https://<your-worker>.workers.dev/https://example.com"

Response Format

The service returns the content in a clean, standardized format:

Title: Example Domain

URL Source: https://example.com

Markdown Content:
[converted markdown content here]

License

MIT License - See LICENSE for details