Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

we need proxy on self-hosted version #1129

Open
mr-v-v-v opened this issue Feb 4, 2025 · 2 comments
Open

we need proxy on self-hosted version #1129

mr-v-v-v opened this issue Feb 4, 2025 · 2 comments

Comments

@mr-v-v-v
Copy link

mr-v-v-v commented Feb 4, 2025

We urgently need proxy support for the self-hosted version of the software. Without a proxy, it is impossible to scrape websites reliably since most sites will block servers after repeated requests from a single IP address.

Several related issues have already been created, including:

Issue #1035
Issue #925
Despite these reports, the issue has yet to be addressed. Proxy support is a critical feature for users relying on this tool for web scraping, as it significantly impacts the tool's usability and effectiveness. Could we please get an update on when this feature will be implemented?

Thank you for your attention to this matter.

@namhnz
Copy link

namhnz commented Feb 11, 2025

Any website using Cloudflare, after I successfully scrape it once, gets blocked on subsequent attempts.

@dcapeluto
Copy link

Not happening. They don't want you using it without needing them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants