Skip to content

Commit 3eaa1d9

Browse files
Update README for v0.6.2
1 parent 0098289 commit 3eaa1d9

File tree

1 file changed

+9
-6
lines changed

1 file changed

+9
-6
lines changed

README.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,15 @@
11
# Cyclone's URL Spider
2-
Website spider to crawl website and create a wordlist that is sorted by frequency
2+
Wordlist & ngram creation tool to crawl a given url and create a wordlist / ngrams (depending on flags given).
33

44
Usage Instructions:
5-
- ./spider.bin -url https://github.com/cyclone-github
6-
- to set crawl depth, use flag "-crawl" (defaults to -crawl 1)
7-
- to set phrase depth, use flag "-phrase" (defaults to -phrase 1)
8-
- ./spider -url https://github.com/cyclone-github -crawl 2 -phrase 2
9-
- defaults output to url_wordlist.txt, but can also be specified by flag "-o"
5+
- To create a simple wordlist from a specified url (will save deduplicated wordlist to url_wordlist.txt):
6+
- - ./spider.bin -url https://github.com/cyclone-github
7+
- To create ngrams, use flag "-ngram" and set ngram level such as "-ngram 2" or a range "-ngram 1-3"
8+
- - ./spider.bin -url https://github.com/cyclone-github -ngram 1-3
9+
- To set url crawl url depth of 2, use flag "-crawl 2"
10+
- - ./spider.bin -url https://github.com/cyclone-github -crawl 2
11+
- To set a custom output file, use flag "-o filename"
12+
- - ./spider.bin -url https://github.com/cyclone-github -o wordlist.txt
1013

1114
Compile from source code info:
1215
- https://github.com/cyclone-github/scripts/blob/main/intro_to_go.txt

0 commit comments

Comments
 (0)