File tree Expand file tree Collapse file tree 1 file changed +9
-6
lines changed Expand file tree Collapse file tree 1 file changed +9
-6
lines changed Original file line number Diff line number Diff line change 11# Cyclone's URL Spider
2- Website spider to crawl website and create a wordlist that is sorted by frequency
2+ Wordlist & ngram creation tool to crawl a given url and create a wordlist / ngrams (depending on flags given).
33
44Usage Instructions:
5- - ./spider.bin -url https://github.com/cyclone-github
6- - to set crawl depth, use flag "-crawl" (defaults to -crawl 1)
7- - to set phrase depth, use flag "-phrase" (defaults to -phrase 1)
8- - ./spider -url https://github.com/cyclone-github -crawl 2 -phrase 2
9- - defaults output to url_wordlist.txt, but can also be specified by flag "-o"
5+ - To create a simple wordlist from a specified url (will save deduplicated wordlist to url_wordlist.txt):
6+ - - ./spider.bin -url https://github.com/cyclone-github
7+ - To create ngrams, use flag "-ngram" and set ngram level such as "-ngram 2" or a range "-ngram 1-3"
8+ - - ./spider.bin -url https://github.com/cyclone-github -ngram 1-3
9+ - To set url crawl url depth of 2, use flag "-crawl 2"
10+ - - ./spider.bin -url https://github.com/cyclone-github -crawl 2
11+ - To set a custom output file, use flag "-o filename"
12+ - - ./spider.bin -url https://github.com/cyclone-github -o wordlist.txt
1013
1114Compile from source code info:
1215- https://github.com/cyclone-github/scripts/blob/main/intro_to_go.txt
You can’t perform that action at this time.
0 commit comments