Skip to content
This repository has been archived by the owner on Aug 9, 2018. It is now read-only.

Update links/reports #3

Open
kamikazechaser opened this issue Feb 20, 2018 · 1 comment
Open

Update links/reports #3

kamikazechaser opened this issue Feb 20, 2018 · 1 comment
Labels
enhancement New feature or request

Comments

@kamikazechaser
Copy link

It would be great to have a feature to update the stats of an OD. One approach is to queue a task every few days or to manually trigger to update the contents of the OD.

@kamikazechaser kamikazechaser changed the title [Faeture Suggestion] Update links/reports [Feature Suggestion] Update links/reports Feb 20, 2018
@simon987 simon987 changed the title [Feature Suggestion] Update links/reports Update links/reports Feb 20, 2018
@simon987 simon987 added the enhancement New feature or request label Feb 20, 2018
@simon987
Copy link
Owner

Not sure how useful that would be, given that the open directories posted usually don't last more than 2-3 months. Best way to do it IMO would be to schedule a re-scan every X days, as you said, and keep the older reports, however I'm very limited in space so that's not really an option.
Taking into consideration that I can only save the latest report, I think it's best to take a snapshot at the time of posting and leave it there, as OD gets reposted often (especially if their content change)

Another option would be to save the reports somewhere else (pastebin would be perfect, if it didn't have CAPTCHAs)

I also have to take into consideration that scanning a OD can take up to 8h, so rescanning every one of them at set intervals would inevitably mean that the script would be running 24/7, which is not ideal since I use the VPS it's hosted on for other projects.

If you need up-to-date indexing of ODs, you can take a look at filepursuit.com

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants