Pagodo - Automate Google Hacking Database Scraping And Searching


The goal of this project was to develop a passive Google dork script to collect potentially vulnerable web pages and applications on the Internet. There are 2 parts. The first is ghdb_scraper.py that retrieves Google Dorks and the second portion is pagodo.py that leverages the information gathered by ghdb_scraper.py.

What are Google Dorks?
The awesome folks at Offensive Security maintain the Google Hacking Database (GHDB) found here: https://www.exploit-db.com/google-hacking-database. It is a collection of Google searches, called dorks, that can be used to find potentially vulnerable boxes or other juicy info that is picked up by Google's search bots.

Installation
Scripts are written for Python 3.6+. Clone the git repository and install the requirements.
git clone https://github.com/opsdisk/pagodo.git
cd pagodo
virtualenv -p python3 .venv # If using a virtual environment.
source .venv/bin/activate # If using a virtual environment.
pip install -r requirements.txt

Google is blocking me!
If you start getting HTTP 503 errors, Google has rightfully detected you as a bot and will block your IP for a set period of time. The solution is to use proxychains and a bank of proxies to round robin the lookups.
Install proxychains4
apt install proxychains4 -y
Edit the /etc/proxychains4.conf configuration file to round robin the look ups through different proxy servers. In the example below, 2 different dynamic socks proxies have been set up with different local listening ports (9050 and 9051). Don't know how to utilize SSH and dynamic socks proxies? Do yourself a favor and pick up a copy of The Cyber Plumber's Handbook to learn all about Secure Shell (SSH) tunneling, port redirection, and bending traffic like a boss.
vim /etc/proxychains4.conf
round_robin
chain_len = 1
proxy_dns
remote_dns_subnet 224
tcp_read_time_out 15000
tcp_connect_time_out 8000
[ProxyList]
socks4 127.0.0.1 9050
socks4 127.0.0.1 9051
Throw proxychains4 in front of the Python script and each lookup will go through a different proxy (and thus source from a different IP). You could even tune down the -e delay time because you will be leveraging different proxy boxes.
proxychains4 python3 pagodo.py -g ALL_dorks.txt -s -e 17.0 -l 700 -j 1.1

ghdb_scraper.py
To start off, pagodo.py needs a list of all the current Google dorks. A datetimestamped file with the Google dorks and the indididual dork category dorks are also provided in the repo. Fortunately, the entire database can be pulled back with 1 GET request using ghdb_scraper.py. You can dump all dorks to a file, the individual dork categories to separate dork files, or the entire json blob if you want more contextual data about the dork.
To retrieve all dorks
python3 ghdb_scraper.py -j -s
To retrieve all dorks and write them to individual categories:
python3 ghdb_scraper.py -i
Dork categories:
categories = {      1: "Footholds",      2: "File Containing Usernames",      3: "Sensitives Directories",      4: "Web Server Detection",      5: "Vulnerable Files",      6: "Vulnerable Servers",      7: "Error Messages",      8: "File Containing Juicy Info",      9: "File Containing Passwords",      10: "Sensitive Online Shopping Info",      11: "Network or Vulnerability Data",      12: "Pages Containing Login Portals",      13: "Various Online devices",      14: "Advisories and Vulnerabilities",  }  

pagodo.py
Now that a file with the most recent Google dorks exists, it can be fed into pagodo.py using the -g switch to start collecting potentially vulnerable public applications. pagodo.py leverages the google python library to search Google for sites with the Google dork, such as:
intitle:"ListMail Login" admin -demo  
The -d switch can be used to specify a domain and functions as the Google search operator:
site:example.com  
Performing ~4600 search requests to Google as fast as possible will simply not work. Google will rightfully detect it as a bot and block your IP for a set period of time. In order to make the search queries appear more human, a couple of enhancements have been made. A pull request was made and accepted by the maintainer of the Python google module to allow for User-Agent randomization in the Google search queries. This feature is available in 1.9.3 and allows you to randomize the different user agents used for each search. This emulates the different browsers used in a large corporate environment.
The second enhancement focuses on randomizing the time between search queries. A minimum delay is specified using the -e option and a jitter factor is used to add time on to the minimum delay number. A list of 50 jitter times is created and one is randomly appended to the minimum delay time for each Google dork search.
categories = {
1: "Footholds",
2: "File Containing Usernames",
3: "Sensitives Directories",
4: "Web Server Detection",
5: "Vulnerable Files",
6: "Vulnerable Servers",
7: "Error Messages",
8: "File Containing Juicy Info",
9: "File Containing Passwords",
10: "Sensitive Online Shopping Info",
11: "Network or Vulnerability Data",
12: "Pages Containing Login Portals",
13: "Various Online devices",
14: "Advisories and Vulnerabilities",
}
Latter in the script, a random time is selected from the jitter array and added to the delay.
intitle:"ListMail Login" admin -demo
Experiment with the values, but the defaults successfully worked without Google blocking my IP. Note that it could take a few days (3 on average) to run so be sure you have the time.
To run it:
site:example.com

Conclusion
Comments, suggestions, and improvements are always welcome. Be sure to follow @opsdisk on Twitter for the latest updates.




via KitPloit

Related articles


  1. Hackrf Tools
  2. Hacking Tools For Games
  3. Hacking Tools For Pc
  4. Pentest Tools Subdomain
  5. Hack Tools
  6. Hackrf Tools
  7. Pentest Tools For Ubuntu
  8. Hack Tools For Games
  9. Pentest Tools Android
  10. Nsa Hack Tools Download
  11. Hacking Tools For Beginners
  12. Beginner Hacker Tools
  13. Github Hacking Tools
  14. Pentest Automation Tools
  15. Hacking Tools Windows
  16. How To Install Pentest Tools In Ubuntu
  17. Growth Hacker Tools
  18. Hacker Tools Linux
  19. Pentest Tools Tcp Port Scanner
  20. Pentest Automation Tools
  21. New Hack Tools
  22. Kik Hack Tools
  23. Hacking Tools Windows 10
  24. Pentest Tools Website Vulnerability
  25. Pentest Tools Bluekeep
  26. Hacker Tools Github
  27. Hacking Tools Hardware
  28. Nsa Hack Tools Download
  29. Underground Hacker Sites
  30. Blackhat Hacker Tools
  31. Pentest Recon Tools
  32. Pentest Tools Android
  33. Hack Tools
  34. How To Install Pentest Tools In Ubuntu
  35. Hacker Tools Software
  36. Free Pentest Tools For Windows
  37. Pentest Reporting Tools
  38. Hack Rom Tools
  39. Top Pentest Tools
  40. Hacker Tools For Mac
  41. Hacker Hardware Tools
  42. What Is Hacking Tools
  43. Computer Hacker
  44. Pentest Box Tools Download
  45. Hacking Tools Software
  46. Hack App
  47. Hacking Tools Free Download
  48. Hacking Tools Mac
  49. Pentest Tools For Android
  50. Hackers Toolbox
  51. Hack Website Online Tool
  52. Pentest Tools Alternative
  53. Nsa Hack Tools
  54. Hacking Tools For Beginners
  55. Underground Hacker Sites
  56. Pentest Tools Subdomain
  57. Blackhat Hacker Tools
  58. Hacking Tools For Games
  59. Hacking Tools Kit
  60. Best Hacking Tools 2020
  61. Hacking Tools For Mac
  62. Pentest Tools Find Subdomains
  63. Hacker Tools 2020
  64. Pentest Tools Apk
  65. Hacking Tools Online
  66. Hacks And Tools
  67. Hacking App
  68. Growth Hacker Tools
  69. How To Make Hacking Tools
  70. World No 1 Hacker Software
  71. Hacker Tools For Mac
  72. How To Install Pentest Tools In Ubuntu
  73. Wifi Hacker Tools For Windows
  74. Hacking Tools Kit
  75. Bluetooth Hacking Tools Kali
  76. Pentest Tools For Android
  77. Pentest Tools Kali Linux
  78. Hack Tools For Mac
  79. Hacker Tools Software
  80. Hack Tools Download
  81. Game Hacking
  82. Hacker Tools Free Download
  83. Hacker Tools For Pc
  84. Physical Pentest Tools
  85. Android Hack Tools Github
  86. Hacking Tools Download
  87. Install Pentest Tools Ubuntu
  88. Pentest Tools For Ubuntu
  89. Hacking Tools For Windows 7
  90. Hacker Tools Github
  91. Pentest Tools Online
  92. Hacking Tools For Games
  93. Physical Pentest Tools
  94. Pentest Tools Open Source
  95. Physical Pentest Tools
  96. Hacking Tools For Games
  97. Pentest Tools Kali Linux
  98. Hacking Tools Usb
  99. Hacker Tools Online
  100. Hacking Tools For Mac
  101. Hacks And Tools
  102. Pentest Tools Free
  103. Hak5 Tools
  104. Pentest Tools
  105. Github Hacking Tools
  106. Easy Hack Tools
  107. Hack Tools For Pc
  108. Hack Tool Apk No Root
  109. Hacker Search Tools
  110. Nsa Hack Tools
  111. Hacking Tools For Kali Linux
  112. Physical Pentest Tools
  113. Pentest Tools List
  114. Hacking Tools And Software
  115. Hacking Tools For Windows
  116. Computer Hacker
  117. Hack Tools For Pc
  118. Hacking Tools Kit
  119. Hacking Tools Windows
  120. Growth Hacker Tools
  121. Pentest Tools Find Subdomains
  122. Hack Tools Download
  123. Hacker Tools Software
  124. Hacker Tools For Pc
  125. Pentest Tools Framework
  126. Hack App
  127. Tools 4 Hack
  128. Hack Tools For Pc
  129. Hacking Tools Windows 10
  130. Pentest Tools Free
  131. Hacker Tools
  132. Hacking Tools Github
  133. Pentest Box Tools Download
  134. Pentest Tools For Windows
  135. Hacking Tools Windows
  136. Pentest Tools List
  137. Hacker Tools 2020
  138. Top Pentest Tools
  139. Pentest Tools Port Scanner
  140. Hacker Tools 2020
  141. World No 1 Hacker Software
  142. Best Hacking Tools 2019
  143. Pentest Tools For Mac
  144. Growth Hacker Tools
  145. What Is Hacking Tools
  146. Pentest Tools Linux
  147. Hacking Tools Download
  148. Pentest Tools For Ubuntu
  149. Hacking Tools Online
  150. Hacker Tools Apk Download
  151. Hack Tools
  152. Hacking Tools Usb
  153. Hacking Tools For Beginners
  154. Pentest Tools Tcp Port Scanner

沒有留言: