Skip to content

Aspasht/Divider

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
Aspasht
Oct 14, 2022
b7023f8 · Oct 14, 2022

History

24 Commits
Oct 10, 2022
Sep 16, 2022
Sep 26, 2022
Sep 17, 2022
Oct 14, 2022
Sep 26, 2022
Sep 26, 2022

Repository files navigation

Divider

Gather interesting urls with extensions from a given file.

What this will do

This will process files generated by tools like waybackurls,gau,etc containing urls and grab only urls ending with file extensions.

Installation

Clone the repository:
        git clone https://github.com/Aspasht/Divider.git

Install dependencies:
        pip install -r Requirements.txt

Required tools assigned to fetch flag:
        chmod +x install.sh
        ./install.sh
        Or you can install it manually by using following commands:
        go install github.com/lc/gau/v2/cmd/gau@latest
        go install github.com/tomnomnom/waybackurls@latest

Available Commands

  $ python divider.py -f myurlfile.txt
  $ python divider.py --fetch hosts.txt --req

Example

$ cat myUrls.txt
    https://example.com/test/test3
    https://example.com/test.css
    https://example.com/robots.txt
    https://example.com/sitemap.xml
    https://example.com/test?q=testvalue

$ python divider.py -f myUrls.txt
    https://example.com/robots.txt
    https://example.com/sitemap.xml

Usage

usage: divider.py [-h] -f FILE [-req REQUEST]
options:
  -h, --help                             show this help message and exit
  -f FILE, --file FILE                   Please add target file as argument!
  -o OUTPUT, --output OUTPU              Save output to a file!
  --req REQUEST, --request REQUEST       Send request for previously generated urls! (Default=False)
  --greq GREQUEST, --grequest GREQUEST   Send request for previously generated urls using grequest! (Default=False)
  --fetch FETCH, --fetch FETCH           Fetch urls using tools like waybackurls and gau {must be preinstalled}!

Commands

$ python divider.py -f myurls.txt # Grab urls with extensions without sending any requests.
$ python divider.py -f myurls.txt --req # Send normal requests using httpx module.
$ python divider.py -f myurls.txt --greq # Send requests using grequests module.
    Note: Use grequests flag if your resulting urls are less in number!
$ python divider.py --fetch hosts.txt --req -o out.txt # Save output to a file.

About

Gather interesting urls with extensions

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

Packages

No packages published