When sizing up potential targets, attackers try to get as much information as possible without raising any alarms. The ability to passively research the details of online resources and their associated humans has never been easier. If you're wondering what kind of information about you and your network is available online right now, the best way to find out is to look for it yourself.

This article examines some online services that tabulate known information on users and websites. Some of these services use information that is freely available through online sources; others delve into the dark web to find data that has turned up in security breaches. For privacy, and in order to demonstrate richer examples, identifying information in the output of the tools described in this article will be redacted.

Certifiable

A few years ago, the mighty Google announced [1] that it was putting more weight on websites running HTTPS, as opposed to the unencrypted HTTP alternative, for its search engine indexing results.

Google stated authoritatively (as the main player in the search space): "Browsing the web should be a private experience between the user and the website and must not be subject to eavesdropping, man-in-the-middle attacks, or data modification."

And, while this announcement provided an excellent incentive for website owners to move to solely using HTTPS, it had an unwelcome side effect that made life a little easier for attackers. Attackers soon realized that, if each website uses HTTPS, the SSL certificates (now TLS certificates) for every website could be captured and scrutinized. Unlike a simple DNS entry, certificates hold much more information.

The first online tool that I will look at is called crt.sh. The crt.sh service [2], which is run by the certificate company Sectigo Limited [3], maintains a massive database of certificates that were discovered on websites (and potentially other services). Its splash page [2] tempts users with broad search criteria: "Enter an Identity (Domain Name, Organization Name, etc), a Certificate Fingerprint (SHA-1 or SHA-256), or a crt.sh ID."

In other words, it is possible to search for companies (not just domain names), as well as by certificate fingerprints and other criteria. There are also a number of advanced search options that I'd recommend testing. It is possible to automate usage of the tool by passing the search query directly to the main URL, such as:

https://crt.sh/?q=domain.tld

Figure 1 shows an example of a search. For a relatively quiet website, there's lots of information available for an attacker. It is immediately obvious that over the years, the site used a variety of certificate providers, including Let's Encrypt, DigiCert, and RapidSSL.

Figure 1: This website has used a number of different certificates.

The wealth of information available just from the abbreviated output in Figure 1 would surprise most crt.sh users. Click on a link on the right side of Figure 1, and you'll see a tiny sample of what is known, including information on which applications can make use of the certificate authority (Figure 2).

Figure 2: There's a lot of information stored at crt.sh.

Back to the results in Figure 1, the column entitled Common Name provides a plethora of information that just keeps on giving. The field reports hundreds, maybe thousands, of hostnames that certificates have supported over the years, along with timestamps to check for the likely status. These hostnames could include valuable information on the domain path, such as accounts.domain.com or mail3.domain.org.

Each of these fully qualified names present an attack surface that you can extract from crt.sh. I'd encourage you to try tools like this yourself to see if your website has publicly leaked any unwelcome information.

Digging into DNS

Another tool that can reveal a lot of information about resources related to a domain name is called DNSDumpster [4] by Hacker Target Pty Ltd. According to the website, this service offers "…a FREE domain research tool that can discover hosts related to a domain. Finding visible hosts from the attackers perspective is an important part of the security assessment process."

Figure 3 shows what happens when you add a domain name to the DNSDumpster search box.

Figure 3: A redacted example of DNSDumpster in action.

Even querying against a relatively quiet domain name, DNSDumpster can fill the screen with information that points the user in all sorts of directions. Figure 4 shows a visualization of the relationships among the discovered resources in the form of graph.

Figure 4: DNSDumpster offers clever visualizations.

If you are new to the term, OSINT (Open Source Intelligence) provides "legally gathered information about an individual or organization from free, public sources." If you discover intriguing information within DNS for a domain name, I recommend visiting an excellent OSINT resource called OSINT Framework [5]. OSINT Framework is an eye-watering resource that you could spend several days exploring. The site pulls together a vast array of free online tools and resources.

Consider PassiveDNS as an example. Figure 5 shows that, by expanding various menu options relating to DNS, you can see there are many pointers to free online tools to help you perform passive reconnaissance for DNS queries.

Figure 5: OSINT Framework delivering on suggested DNS tools.

Click the DNS History button on the right side of Figure 5, and you are pointed at a site called DNS History [6], which is run by 8086 Consultancy [7]. Figure 6 shows how simple it is to use the site if you need to query when changes took place for a DNS entry.

Figure 6: You should experiment with the clever DNS History service.

Scroll a little further down the search page to see some historical record references that might be useful to you or an attacker (Figure 7). In some cases, apparently obsolete and no longer public IP Addresses might currently have other live systems using them.

Figure 7: Useful DNS A records with IP Addresses and the dates they were in use.

The sophisticated DNS History service also displays easy-to-read representations of the number of registrations for domain names. As shown in Figure 8, a new feature, currently in Beta, shows a heat map of where on the planet registrations are taking place. DNS History is an impressive site that deserves much more time dedicated to exploring it.

Figure 8: A heat map lets you see where domain names are commonly registered. (Source http://dnshistory.org/p/heatmaps)

Going Dark

So far I have focused on certificates and DNS, along with the outstanding OSINT Framework, which is a topic all of its own. If you're willing to take a step down into the darkness, you can also find information by rummaging deeper on the dark web. Suppose I wanted to find information relating to a specific user via their email address. There are a number of services that collect information from the dark web for security professionals (and attackers obviously) to query. One service I've used professionally is Dehashed [8]. Figure 9 shows the mind-blowing number of compromised resources visible to Dehashed.

Figure 9: Dehashed: That's a lot of assets. (Source: https://www.dehashed.com)

You need to register for a free account to query the database. Signing up for a reasonably priced subscription will provide much more information in relation to the queries you perform.

To demonstrate what is available for free, I have used an email I know has been exposed a number of times, thanks to the inimitable Have I Been Pwned website [9] (for more on Have I Been Pwned, see the box entitled "Pwn Check")

Figure 10: Using Have I Been Pwned. (Source https://haveibeenpwned.com)

When I checked the email address in Dehashed, I got the results shows in Figure 11. Figure 12 shows some of the details from the 31 results Figure 11 mentions.

Figure 11: Dehashed has found some worrying data relating to the email address. (Source: https://www.dehashed.com)

Figure 12: The breaches where the findings were discovered. (Source: https://www.dehashed.com)

A careful look at Figure 12 shows some of the Sourced from data sources. These sources are well-known data breaches that contained the email address. (If you're interested, check out the article at the CSO site on the 15 biggest data breaches this century [11].)

Back to Dehashed, if you have an active subscription, you can click on any of the items relating to breaches on the left side of the screen, and the service will reveal what data was present in the data breach (relating to the email address).

Findings can include all sorts of data, including: usernames, email addresses, IP Addresses, postal addresses, telephone numbers, passwords (hashed and in plain text), and human names. The data is not just alluded to either – the findings are displayed for all to see.

Dehashed lets you request that an entry be removed from its database, but of course, the data could still be present in many other places online, including the dark web. Removing the visibility of the data in Dehashed only hides it from some security researchers and others who are using Dehashed.

Dehashed also provides a comprehensive (subscription-based) monitoring service, alongside a fully fledged API. The Dehashed Data Wells page [12] shows how much data was retrieved from specific data breaches, along with a narrative that provides some additional and useful context.

Figure 13: The breaches that Dehashed references (Source: https://dehashed.com/data): The "17 Database Breach," where apparently, in 2016, data related to a streaming app was exposed, including information about four million users.