[Active – Information Gathering] Automated screenshot of websites with goWitness

gowitness is a website screenshot utility written in Golang, that uses Chrome Headless to generate screenshots of web interfaces using the command line, with a handy report viewer to process results. Both Linux and macOS is supported, with Windows support mostly working.

https://github.com/sensepost/gowitness.git

Inspiration for gowitness comes from Eyewitness. If you are looking for something with lots of extra features, be sure to check it out along with these other projects

https://github.com/RedSiege/EyeWitness

https://github.com/breenmachine/httpscreenshot

goWitness requires Chrome or Chromium installed

Installation

1. For installation information and other documentation, please refer to the wiki

https://github.com/sensepost/gowitness/wiki

https://github.com/sensepost/gowitness/wiki/Installation

2. To install run

  • go install gorm.io/gorm@latest
  • go install github.com/sensepost/gowitness@latest
  • sudo apt install chromium

3. Display the menu

  • gowitness -h

4. Take a sincreenshow to a single URL, the ourput is saved in ./screenshots/filename.png

  • gowitness single https://vk9-sec.com/

5. Run against a file containing a list

  • gowitness file -f URL-list.txt

[Information Gathering] Gathering old information from WayBackMachine using waybackurls

Waybackurls is a command-line tool used for scraping URLs from the Wayback Machine.

Waybackurls is important for cybersecurity professionals because it allows them to uncover historical data about a website, identify potential vulnerabilities, and assess the security posture of a target.

https://github.com/tomnomnom/waybackurls

Basic Usage

  • waybackurls <target>: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target.
  • waybackurls <target> -json: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target in JSON format.
  • waybackurls <target> | grep <keyword>: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target that contain the specified keyword.
  • waybackurls <target> | sort -u: This command retrieves all the unique URLs of the Wayback Machine archive for the specified domain or target.
  • waybackurls <target> | httprobe: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target and tests them for HTTP/HTTPS connectivity.

Advanced Usage

  • waybackurls <target> | grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*"|sort -u: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target, and uses regex to extract only the URLs that begin with "http" or "https".
  • waybackurls <target> -exclude <exclude-file>: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target, but excludes the URLs listed in the specified file.
  • waybackurls <target> -filter "status_code:200"|sort -u: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target that return a 200 status code.
  • waybackurls <target> | unfurl paths | sort | uniq -c | sort -rn: This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target, extracts only the paths, and sorts them by the number of occurrences to identify the most commonly accessed paths.
  • waybackurls <target> | xargs -I{} curl -s -L -I -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0" {} | grep -iE "x-frame-options|content-security-policy": This command retrieves all the URLs of the Wayback Machine archive for the specified domain or target, and tests them for X-Frame-Options and Content-Security-Policy headers.

Integrations

  • Using waybackurls with Nmap: By combining the results of Nmap with waybackurls, a cybersecurity professional can gather information about any web-based services that may be exposed on the target network.
  • Using waybackurls with Gobuster: Gobuster is a tool used for directory and file brute-forcing on web servers. By using the URLs gathered by waybackurls, a cybersecurity professional can perform more targeted directory and file brute-forcing.
  • Using waybackurls with Sublist3r: By combining the results of Sublist3r with waybackurls, a cybersecurity professional can gather information about the web-based services running on subdomains of the target domain.
  • Using waybackurls with Burp Suite: By feeding the URLs gathered by waybackurls into Burp Suite’s spidering feature, a cybersecurity professional can identify additional web application endpoints that may be vulnerable to attack.

Installation

1. Having GoLang already installed in your system run

  • go install github.com/tomnomnom/waybackurls@latest

2. Display the help menu

  • waybackurls -h

How to use

1. Query a domain

  • echo "https://vk9-sec.com" | waybackurls

2. Run it against a file that contains a list of URLs

  • cat file.txt | waybackurls

Domain info using Robtex

Robtex is a service which gathers public information about IP addresses, domain names, host names, Autonomous systems, and more.

How to use

1. Access https://www.robtex.com/dns-lookup and search for the domain

2. In the results we can find

Analysis

  • DNS servers
  • Mail servers
  • IP address

Quick Info

  • FQDN
  • DNS servers

Records

  • Analysis of DNS query record type (MX, A, NS, etc)

Shared

  • IP addresses
  • DNS servers
  • DNS servers IP address
  • Mail Servers
  • Mail Servers IP address

Graph

  • Displays a map of the results

 

Website infrastructure with Netcraft Site Report

Find out the infrastructure and technologies used by any site using results from Netcraft

The information that netcraft provides includes:

  • Background — This includes basic domain information.
  • Network — This includes information from IP Address to Domain names to nameservers.
  • SSL/TLS — This gives the ssl/tls status of the target
  • Hosting History -This gives the information on the hosting history of the target
  • Sender Policy Framework (SPF) — This describes who can send mail on the domains behalf
  • DMARC -This is a mechanism for domain owners to indicate how mail purporting to originate from their domain should be authenticated
  • Web Trackers — This trackers can be used to monitor individual user behavior across the web
  • Site Technology — This section includes details on :
  • Cloud & PaaS: Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). Platform as a service (PaaS) is a category of cloud computing services that provide a computing platform and a solution stack as a service.
  • Server-Side: Includes all the main technologies that Netcraft detects as running on the server such as PHP.
  • Client-Side Includes all the main technologies that run on the browser (such as JavaScript and Adobe Flash).
  • Content Delivery Network: A content delivery network or content distribution network (CDN) is a large distributed system of servers deployed in multiple data centers on the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance.
  • Content Management System: A content management system (CMS) is a computer program that allows publishing, editing and modifying content as well as maintenance from a central interface.
  • Mobile Technologies: Mobile technology is the technology used for hand held mobile devices.
  • Web Stats: Web analytics is the measurement, collection, analysis and reporting of internet data for purposes of understanding and optimizing web usage.
  • Character Encoding: A character encoding system consists of a code that pairs each character from a given repertoire with something else such as a bit pattern, sequence of natural numbers, octets, or electrical pulses in order to facilitate the transmission of data (generally numbers or text) through telecommunication networks or for data storage.
  • Web Browser Targeting: Web browser targeting enables software applications to make use of specific functions of the browser as well as optimizing the application for specific browser versions.

Using Netcraft

1. Access https://sitereport.netcraft.com/ and search for the domain you want

 

How to use whois

Information gathering is the first step of Ethical Hacking, where the penetration tester or even hackers gather information on their target victims. To increase your chances of a “successful” hacking, you will need to do a good job and spend time on this stage.

There is a couple of information that one can gather during the process. These include but are not limited to:

  • Victim IP addresses
  • Domain Name Information
  • Technologies used by the website/web applications
  • Other websites on the same server
  • DNS records
  • Unlisted files, subdomains and directories

Whois is an online tool that helps one to lookup information of the target website/web app such as Domain name, IP address block or an autonomous system but it is also used to query for a wider range of information. The information that is provided in the Whois lookup is publicly available unless the website is using domain privacy.

Whois

There are various webpages that offer whois services among those:

How to use

1. Access the website of your choice and search for the domain name of the site, in this case, I’d use vk9-sec.com, the result should look like this. (I’m using domain protection so my real information is hidden)

2. Information that can be found

  • Domain:This field will give you the domain name which we are querying the WHOIS details.
  • Registrar:This is the details of the registrar with whom the domain name is registered.
  • Registration Date: This is the date when the domain name was first registered.
  • Expiration Date:This is the date when the domain will expire.
  • Updated Date: This is the date when the WHOIS details last updated.
  • Status: This is the registrar status of the domain. This will be “OK” if there is no restriction and the domain is free to transfer from one registrar to another.
  • Name Servers: This field will provide the details of the nameservers used by the domain.
  • IP Address & IP Location
  • Contact

whois on CLI

1. Get help

  • whois -h

2. Run whois command and the domain you want to search

  • whois vk9-sec.com

 

Find internet accessible devices – Shodan

Shodan’s a search engine which helps find systems on the internet. It’s a great resource to provide passive reconnaissance.

Some have described Shodan as a search engine for hackers, and have even called it "the world's most dangerous search engine".

Devices that Shodan can find:

  • Servers
  • Routers
  • Switches
  • Printers on public ip
  • Webcams
  • Gas station pumps
  • Voip phones
  • Scada devices
  • Etc

How to

1. visit https://shodan.io/

2. Click on “Login or Registration”. Create an account and log in

3. Once logged in, use the search bar to look for devices

Also, under “My account” you can do and find the following

  • API information
  • Change account password

Basic search Filters

  • Port: Search by specific port
  • net: Search based on an IP/CIDR
  • hostname: Locate devices by hostname
  • os: Search by Operating System
  • city: Locate devices by city
  • country: Locate devices by country
  • geo: Locate devices by coordinates
  • org: Search by organization
  • before/after: Timeframe delimiter
  • hash: Search based on banner hash
  • has_screenshot:true Filter search based on a screenshot being present
  • title: Search based on text within the title

Searching

1. Find devices that run a specific port

  • port:6667

2. Look for devices that have a specific OS and a port

  • os:windows 2003 port:445

3. Look for device per country, and a keyword

  • cisco country:CR

4. We can also search by company, OS type and their location

  • org:apple os:linux country:IN

5. Search for title details and keywords

  • title:camera default

Analyzing the results

1. We are going to look for a Linux server with port 80 opened

  • os:linux port:80

2. Opening the first result takes us to the details of that

Device details displays basic information like the one below

  • IP
  • City
  • Country
  • Organization
  • ISP
  • Operating System
  • Last Update
  • ASN

The results may sometimes show vulnerability scanning results.

Opened port are also displayed

Each port discovered shows more info about the protocol. Sometimes useful configuration information can be displayed in there.

Reporting

1. Download Data

Clicking on “Download Results” data lets you download the results, this requires credits, use them wisely

2. Creating a Report

Clicking on “Create Report” lets you download a report of results

3. Going to “Reports” you can see generated file

CLI

Shodan also provides an easy to use command line (https://cli.shodan.io/), it comes already install in kali, but, you can visit the official site to learn hot to install it.

1. Display help

  • Shodan -h

Using -h over the command will display further help

  • shodan scan -h

2. Show product version

  • shodan version

3. Set up your API for starting the searches, API is found under account settings

  • shodan init <API>

4. First basic search

  • shodan search org:apple os:linux

5. Show count of the searches

  • shodan count org:apple os:linux

6. Show stats of the search

  • shodan stats org:apple os:linux

7. Check host info

  • shodan host 17.69.152.26

Shodan Maps

Shodan also provides a graphical map that helps you visualize the location of the result

(https://maps.shodan.io)

  • org:apple os:linux

Shodan images

It helps display screenshot or images captured (https://images.shodan.io/). Image data is gathered from 5 different sources: VNC, Remote Desktop (RDP), RTSP, Webcams and X Windows.

  • port:3389

Exploit search

Basic exploit search within the database (https://exploits.shodan.io/)

View API stats

This tool shows details about your API usage (https://developer.shodan.io/dashboard)

Network monitor

Keep track of the devices that you have exposed to the Internet. Setup notifications, launch scans and gain complete visibility into what you have connected. (https://monitor.shodan.io/dashboard)

ICS radar

ICS Radar is a 3D map of Industrial Control Systems (ICS) devices found by Shodan crawlers. (https://ics-radar.shodan.io/)

Honeypot score

The service called Honeypot or not? will attribute a Honeyscore to an IP address, a probability of being a honeypot. (https://honeyscore.shodan.io/)

Useful Shodan queries

https://github.com/jakejarvis/awesome-shodan-queries

https://beta.shodan.io/search/filters

Industrial Control Systems

Samsung Electronic Billboards

  • "Server: Prismview Player"

Gas Station Pump Controllers

  • "in-tank inventory" port:10001

Automatic License Plate Readers

  • P372 "ANPR enabled"

Traffic Light Controllers / Red Light Cameras

  • mikrotik streetlight

Voting Machines in the United States

  • "voter system serial"

Telcos Running Cisco Lawful Intercept Wiretaps

  • "Cisco IOS" "ADVIPSERVICESK9_LI-M"

Prison Pay Phones

  • "[2J[H Encartele Confidential"

Tesla PowerPack Charging Status

  • http.title:"Tesla PowerPack System" http.component:"d3" -ga3ca4f2

Electric Vehicle Chargers

  • "Server: gSOAP/2.8" "Content-Length: 583"

Maritime Satellites

  • "Cobham SATCOM" OR ("Sailor" "VSAT")

Submarine Mission Control Dashboards

  • title:"Slocum Fleet Mission Control"

CAREL PlantVisor Refrigeration Units

  • "Server: CarelDataServer" "200 Document follows"

Nordex Wind Turbine Farms

  • http.title:"Nordex Control"
  • "Jetty/3.1 (JSP 1.1; Servlet 2.2; java 1.6.0_14)"

C4 Max Commercial Vehicle GPS Trackers

  • "[1m[35mWelcome on console"

DICOM Medical X-Ray Machines

  • "DICOM Server Response" port:104

GaugeTech Electricity Meters

  • "Server: EIG Embedded Web Server" "200 Document follows"

Siemens Industrial Automation

  • "Siemens, SIMATIC" port:161

Siemens HVAC Controllers

  • "Server: Microsoft-WinCE" "Content-Length: 12581"

Door / Lock Access Controllers

  • "HID VertX" port:4070

Railroad Management

  • "log off" "select the appropriate"

Remote Desktop

Unprotected VNC

  • "authentication disabled" "RFB 003.008"

Windows RDP

"\x03\x00\x00\x0b\x06\xd0\x00\x00\x124\x00"

Network Infrastructure

MongoDB

  • "MongoDB Server Information" port:27017 -authentication
  • "Set-Cookie: mongo-express=" "200 OK"
  • "Home - Mongo Express"
  • all:”mongodb server information” all:”metrics”

Jenkins CI

  • "X-Jenkins" "Set-Cookie: JSESSIONID" http.title:"Dashboard"

Docker APIs

  • "Docker Containers:" port:2375

Pi-hole Open DNS Servers

  • "dnsmasq-pi-hole" "Recursion: enabled"

Already Logged-In as root via Telnet

  • "root@" port:23 -login -password -name -Session

Android Root Bridges

  • "Android Debug Bridge" "Device" port:5555

Lantronix Serial-to-Ethernet Adapter Leaking Telnet Passwords

  • Lantronix password port:30718 -secured

Citrix Virtual Apps

  • "Citrix Applications:" port:1604

Cisco Smart Install

  • "smart install client active"

PBX IP Phone Gateways

  • PBX "gateway console" -password port:23

Polycom Video Conferencing

  • http.title:"- Polycom" "Server: lighttpd"

Telnet Configuration:

  • "Polycom Command Shell" -failed port:23

Bomgar Help Desk Portal

  • "Server: Bomgar" "200 OK"

Intel Active Management CVE-2017-5689

  • "Intel(R) Active Management Technology" port:623,664,16992,16993,16994,16995

HP iLO 4 CVE-2017-12542

  • HP-ILO-4 !"HP-ILO-4/2.53" !"HP-ILO-4/2.54" !"HP-ILO-4/2.55" !"HP-ILO-4/2.60" !"HP-ILO-4/2.61" !"HP-ILO-4/2.62" !"HP-iLO-4/2.70" port:1900

Outlook Web Access

Exchange 2007

  • "x-owa-version" "IE=EmulateIE7"

Exchange 2010

  • "x-owa-version" "IE=EmulateIE7" http.favicon.hash:442749392

Exchange 2013 / 2016

  • "X-AspNet-Version" http.title:"Outlook" -"x-owa-version"

Lync / Skype for Business

  • "X-MS-Server-Fqdn"

Network Attached Storage (NAS)

SMB (Samba) File Shares

  • "Authentication: disabled" port:445

Specifically domain controllers:

  • "Authentication: disabled" NETLOGON SYSVOL -unix port:445

FTP Servers with Anonymous Login

  • "220" "230 Login successful." port:21

Iomega / LenovoEMC NAS Drives

  • "Set-Cookie: iomega=" -"manage/login.html" -http.title:"Log In"

Buffalo TeraStation NAS Drives

  • Redirecting sencha port:9000

Logitech Media Servers

  • "Server: Logitech Media Server" "200 OK"

Plex Media Servers

  • "X-Plex-Protocol" "200 OK" port:32400

Tautulli / PlexPy Dashboards

  • "CherryPy/5.1.0" "/home"

FTP

  • "230 login successful" port:"21"
  • “Anonymous+access+allowed” connected
  • Anonymous+access+allowed

Webcams

Yawcams

  • "Server: yawcam" "Mime-Type: text/html"

webcamXP/webcam7

  • ("webcam 7" OR "webcamXP") http.component:"mootools" -401

SQWebcam

  • “SQ-WEBCAM”

Android IP Webcam Server

  • "Server: IP Webcam Server" "200 OK"

Canon

  • title:"Network Camera VB-M600" 200 ok server: vb

Security DVRs

  • html:"DVR_H264 ActiveX"

d-link

  • d-Link Internet Camera, 200 OK
  • "Default Login" Authenticate

Printers & Copiers

HP Printers

  • "Serial Number:" "Built:" "Server: HP HTTP"
  • HP LaserJet 4250

Xerox Copiers/Printers

  • ssl:"Xerox Generic Root"

Epson Printers

  • "SERVER: EPSON_Linux UPnP" "200 OK"
  • "Server: EPSON-HTTP" "200 OK"

Canon Printers

  • "Server: KS_HTTP" "200 OK"
  • "Server: CANON HTTP Server"

Home Devices

Yamaha Stereos

  • "Server: AV_Receiver" "HTTP/1.1 406"

Apple AirPlay Receivers, Apple TVs, HomePods, etc.

  • "\x08_airplay" port:5353

Chromecasts / Smart TVs

  • "Chromecast:" port:8008

Crestron Smart Home Controllers

  • "Model: PYNG-HUB"

Random Stuff

OctoPrint 3D Printer Controllers

  • title:"OctoPrint" -title:"Login" http.favicon.hash:1307375944

Etherium Miners

  • "ETH - Total speed"

Apache Directory Listings

  • http.title:"Index of /" http.html:".pem"

Too Many Minecraft Servers

  • "Minecraft Server" "protocol 340" port:25565

NetBotz (net monitoring tool)

  • title:"netbotz appliance"

Scada

  • EIG Embedded Web Server
  • title:”RAKO Bridge Control Panel”

Vulnerabilities

  • CVE-2020-0796 os:windows

OOB

  • ”Active Management Technology”