Pegasy
Pegasy's Blog

Pegasy's Blog

Explanation, overview and usage of SpiderFoot

Explanation, overview and usage of SpiderFoot

Pegasy's photo
Pegasy
·Dec 17, 2021·

4 min read

Subscribe to my newsletter and never miss my upcoming articles

SpiderFoot is an OSINT tool, but first of all, what's OSINT?

OSINT stands for Open Source INTelligence. It refers to any information that can legally be obtained from free, open and/or public sources.

SpiderFoot is a tool that automates the OSINT scan of a given target for us. The data retrieved from the scan is processed by the built-in modules to gain even more informations from the results.

SpiderFoot has different pricing :

  1. Spider Foot - Free, self-hosted and open-source version - SpiderFoot's developers claim that the free plan only uses 25% of the ability of SpiderFoot HX. It is limited to only one user and can only scan 1 target per scan and it can only run for 3 hours.
  2. SpiderFoot HX - Paid, web-hosted version - It is not limited to only one user and has no limits regarding a scan's target amount.

If you choose to pay for SpiderFoot, let's have a look at the different offers:

Freelancer :

  • $79/month or $749/year
  • Limited to 15 scans per month
  • Limited to 1 user account
  • Limited to a 24 hours scan duration
  • Limited to 256 targets per scan
  • Limited to 6 months of data retention
  • Has official support
  • Has investigations, investigation is a type of scan that creates a dynamic graph and you can use Maltego-like transforms

Business :

  • $249/month or $2399/year
  • Limited to 50 scans per month
  • Limited to 3 user accounts
  • Limited to a 72 hours scan duration
  • Limited to 1024 targets per scan
  • Limited to 12 months of data retention
  • Has the same advantages of the Freelancer
  • Has Attack Surface Monitoring
  • Has Screenshotting
  • Has SpiderFoot HX API

Enterprise :

  • You will need to contact SpiderFoot's developers for your custom price
  • You will have custom amounts of scans per month, user accounts, scan duration limit, targets per scan and data retention time.
  • Has the same advantages of the Business
  • Has an EU option
  • Supports ElasticSearch, Splunk and REST feeds
  • Supports writing and running custom modules in Python

This guide assumes that you chose the free, open-source and self-hosted version.

Now that you have a background knowledge of what's OSINT and what's SpiderFoot let's dive into the tutorial.

If SpiderFoot is not installed on your machine, simply run "# apt-get install spiderfoot" if you are root else run "$ sudo apt-get install spiderfoot", assuming you are running a Kali Linux or any Debian-based Linux.

apt-install-spiderfoot.png

Once it is installed, let's determine our local IP address by typing "$ ip a"

ip-a-modified.png

As you can see, the IP address of this machine is "192.168.100.21", it will then be used during this tutorial. Don't forget to change it to your own IP address!

Let's now launch SpiderFoot by typing "spiderfoot -l 192.168.100.21:4444" the first part is the IP address the server has to listen on and after the ":" is the port it will need to listen on.

spiderfoot-l.png

Now, launch Firefox or any other browser that you like and go to http://192.168.100.21:4444

firefox.png

Congrats! You successfully started your SpiderFoot instance. Now let's scan something.

Click on "New Scan"

scan-empty.png

Fill in the name of the scan and the target, for this tutorial we will use "scanme.nmap.org" and determine the type of the scan, for this tutorial we will use "All".

scan.png

If you want to, you can customize your scan and activate or deactivate modules by clicking on "By Module", same for Required Data.

Now, click on Run Scan Now.

scan_run_now.png

You will be redirected to the scan, click on "Scans" and you can see our scan is starting. You can pause or stop it by clicking on the associated buttons under "Action".

scans.png

Let's have an overview of our scan, click on its name.

nmap_scan.png

As you can see, it automatically generated a dynamic vertical bar graph, you can click on any information that you want and it will redirect you to it. Or, you can click on "Browse".

browse.png

You can now click on any retrieved data and you will see its item, its source, the module that discovered it and the time it was identified.

retrieved_data.png

SpiderFoot also generated a dynamic graph accessible via the "Graph" tab.

graph.png

The layout can be changed by clicking on the R (Random) or F (Forced), it can then be exported as a PNG image by clicking on the image icon or as an Excel or CSV file by clicking on the Download icon.

graph-export.png

SpiderFoot keeps a track of everything that happened during the scan, it is located in the "Log" section, click on it. You'll see the logs.

log.png

That concludes this tutorial, if you wish to see more, please refer to the official documentation : https://www.spiderfoot.net/documentation/

 
Share this