CompTIA Pentest+ PT0-002 – Section 4: Passive Reconnaissance Part 3

28. OSINT Tools (OBJ 2.1)

Open-source intelligence tools are used during the reconnaissance phase to find actionable intelligence from various publicly available sources. This intelligence can help the penetration tester to be more precise and targeted in their attack development and future exploits during their engagement. because OSINT is publicly available. You’re not going to raise any alerts at your target organization, while you’re gathering information using only these OSINT tools. Now when working with OSINT, you can collect information from any public websites, including the target’s own website.

You can also check the WHOIS database to find out who owns or operates a given domain, as well as querying the public DNS servers to get copies of the target organization’s DNS records. This can include all of their subdomains, email servers and other software of service technologies that are being associated with their domain name. Now job postings, blogs, news articles and social media are also useful in gathering your intelligence. From a technical perspective, inspecting the target organization’s SSL and TLS certificates with the registry authority can also be a useful form of OSINT.

Now in this lesson, we’re going to take a quick look at some popular OSINT tools, that are covered by the exam. For the exam, you do not need to know how to actually use these tools and all of their command line options, but you should know why you would use each tool, and the basic case of OSINT, and then more specifically what type of OSINT information you could get from these tools. Now, the tools we’re going to cover include Metagoofil, FOCA, The Harvester, Shodan, Maltego, Recon-ng, and Censys. It’s important to remember that open-source intelligence is not trying to connect to our targeted servers.

Instead, we’re trying to find out information about them by connecting to other servers on the internet or by searching elsewhere on the internet. The first tool we have is known as Metagoofil. Metagoofil is a Linux-based tool that can search the metadata associated with public documents located on a target’s website. This tool relies on the Python scripting language to locate metadata with the different types of files,including Microsoft Word, Excel and PowerPoint as well as OpenOffice Writer, Calc and BASE files. Metadata is not the data itself that’s contained within the file though, such as the report you wrote for your college English class but instead it’s data about the data in that file. For example, this video consists of all the ones and zeros that make up the images on the screen that you’re viewing and the words you’re hearing.

But, the original video file also contains metadata about the video file itself. This metadata provides information such as the author, the company who created it, title and subject. Other kinds of metadata can also be included in the file, such as how much time is spent editing a Word document, the number of words in that document, or the number of changes or revisions made to that document. To use Metagoofil, you’re going to enter the command Metagoofil, and the appropriate options. For example, if you wanted to search the Apple website for any doc and PDF files limit that search to the first 200 documents found and only download up to 50 of those files each time and then create a working directory where you’re going to save those 50 files to and output the results to a file. You can do that by using the command metagoofil -d apple.com -t doc.pdf -l 200 -n 50 -o applefiles -f and then press the enter key. Metagoofil can run on any system because it is run by the Python interpreter. So you can install it on Linux, macOS or windows systems. The next tool we have is called FOCA, which stands for the Fingerprinting Organizations with Collected Archives.

FOCA’s used to find metadata and hidden information and documents that we collected from that organization. If our target organization is Udemy for example we might go onto Google and find out everything we can about Udemy by finding Word and Excel documents, pictures, and anything else that company may have put out onto the internet. We collect this information and then we run it through FOCA to find all the metadata things like GPS locations, authors names, email addresses internal codes, and things like that. We can then use this as part of our social engineering or to base our future exploits on. If we determine the software versions that were used to create those particular files. In recent versions of FOCA, they’ve also added the ability to use Google, Bing and DuckDuckGo to find downloadable files for you to analyze, just like you can with Metagoofil.

Now the nice thing about FOCA is that it uses a graphical user interface and this makes it easier to work with than the command line Metagoofil tool. Unfortunately though, FOCA does not work on Linux. So you have to be running Windows. If you want to use FOCA. The Harvester is our next tool and it’s a wonderful program for gathering emails, subdomains, hosts, employee names, email addresses, PGP key entries open ports and service banners off of different servers. As you can see, the Harvester is going to start with some of the basic open-source intelligence things like emails, subdomains, hosts, employees and things like that. But then we could take it further by scanning and enumerating things like PGP key entries open ports and banner grabbing. Conveniently the Harvester does come installed by default inside of Kali Linux. Now the Harvester is pretty easy to use. Once you learn at syntax to open up the Harvester go to your command line and then enter the harvester and hit enter. And the command line options will actually show up and tell you how to use it.

For example, if I wanted to search for all the email addresses for domain like Udemy.com and then limit my results to the first 250 that I find using Google I can do that by entering the command theharvester -d udemy.com -l 250 -b google and hitting enter. The Harvester has the ability to use multiple search engines, such as Google and Bing to find this open source intelligence but it can also search social media sites like Twitter and LinkedIn as well. If you want to do that, simply replace the word Google after dash B with a search engine or social media site. You want to search. If you want to check a digital certificate you can also obtain the certificate information from Kumamoto’s certificate search engine as well. You can even conduct banner grabbing in conjunction with using something like Shodan. Another tool that has a lot of great capabilities. Like the Harvester is known as Recon-ng. In fact, Recon-ng has much more capabilities than the Harvester because it uses the system of modules to have additional features and functions for your use. For example, there’s a module to conduct WHOIS queries to identify administrative and technical points of contacts.

There’s another one to conduct PGP key searches and another for social media profile searching yet another for file crawling of websites, and yet another one for DNS record searching and enumeration. If you have a list of emails for your reconnaissance efforts, you can also use Recon-ng to check. If those emails have been the victim of a data breach previously by checking, have I been pawned. And if so you might be able to find those passwords available as part of a password dump for sale over on the dark web. Now Recon-ng is a Python script and it makes it cross-platform and supported by Linux, macOS and windows. This web reconnaissance framework does come pre-installed in Kali Linux by default to start it up just type in recon-ng and press enter, then it’s going to pop up and you’ll be able to see all the commands on the screen.

If you want to try it out for yourself go ahead into your Kali virtual machine and enter recon-ng at the terminal prompt and you’ll be able to start playing with it. When you load up Recon-ng I recommend you set up a new workspace to contain everything that you’re going to be searching for and saving. For example, you can do this with the command line when launching it by typing reco-ng -w dion and that’s going to create a new workspace called Dion for me to use when I conduct my reconnaissance if you’re already in the Recon-ng prompt though, you can also just type in workspace, create and the name and it will actually create it inside of Recon-ng as well. Once you do this, you’re now going to be inside of this new text based environment and you’re going to be able to use all of the different features and functions of Recon-ng. Now similar to how Metasploit is used as an exploit framework and social engineering toolkit is used for social engineering. Recon-ng is used for reconnaissance and open-source intelligence gathering. It is a really complete all in one type of tool.

Next we have Shodan and Shodan is actually a website that contains a search engine that lets us find things like webcams routers servers and other devices that are considered part of the internet of things. Both researchers and attackers, both love Shodan because of the wealth of information it provides. For instance, today I found there are over 10,000 webcams that are actually open to the internet even though they shouldn’t be. Further we can see there are 1500 things using default passwords making them very easy targets. So if we’re targeting a company we can actually search for things owned by that company and see what they have opened to the internet while staying at arms length away because we’re connecting to Shodan and Shodan is the one doing the searching. Now Shodan is looking for everything on the internet.

And so it’s not targeting a particular company but it does add it to its database. Think of about Shodan like Google for different devices. In fact, Shodan is really good at finding things like IOT devices and many IOT devices are not very well secured. So as you come across things like webcams, refrigerators, thermostats or even alarm systems, that may be a great way for you to get into your targeted organization, because maybe they just the new Wi-Fi based security camera system. And you’re going to be able to use that as your gateway and pivot point into their network. The next tool we’re going to talk about is Censys. Now this is much like Shodan and it’s another search engine. The difference is it’s only used for finding hosts and networks across the internet with data about their configurations. It is a really nice search interface and it can build reports for us and even has an SQL engine that we can tie into.

If we pay for a subscription as a network defender Censys can also be configured to continually discover unknown assets on your network and help you mitigate the risk of virtual machine or cloud sprawl that can occur in those type of environments. The last tool we’re going to cover is called Maltego. Maltego is a piece of commercial software that’s used for conducting open-source intelligence and it helps us visually connect those relationships between pieces of information. For instance, if I gather a bunch of emails or a bunch of social media profiles of people who work for an organization, I can start mapping them out with this tool and figure out that if I want to get to the CEO, I first have to go through his receptionist. Then I might find out that to get through the receptionist. I need to go through her sister and if I figure out that her sister likes the movie Twilight, I might be able to start a conversation with her and start connecting all those dots backed up and into the CEO’s life.

Maltego is all about training from one person to the next, or one server to the next or one terminal to the next. Whether you’re doing email, social networking or even machines themselves. You can collect all this information and then map it out for exploitation using Maltego. Maltego also has the ability to automate the querying of public sources of data and then compare the data with other sets of information to provide these commonalities between different sources. This data could be names, physical addresses, network IP address ranges, phone numbers, email addresses, external URLs, DNS records, subdomains, social media profiles, or downloaded files. All of these results are graphically displayed and all of the links are visually created similar to a detective trying to solve a murder case whose strings tie together a bunch of photographs on a court board. So he can easily visualize the bigger picture. That’s what Maltego will do for you.

29. Using OSINT Tools (OBJ 2.1)

In this lesson, I’m going to show you how to use a couple of the open-source intelligence tools. Specifically, we’re going to take a quick look at Metagoofil, theHarvester, and Recon-NG. Now, first let’s look at Metagoofil. Let’s say for example, I want to search GitMe’s website for any Word, PowerPoint and PDF files. And I want to limit that search to the first 50 documents found, download up to 25 of those files, put them into a working directory, and then I can save those 25 files in there and output the results to a file that tells me everything we found. I can do that pretty easily by using Metagoofil. Now, first though, we have to install Metagoofil because it’s not installed by default in our version of Kali Linux, which at the time of this recording is 2021.4a. To do that, you can try to just run Metagoofil and it will ask if you want to install it, or you can use sudo apt-get install Metagoofil.

I’m going to do it the easy way and just type in Metagoofil first. Now, if it’s installed it would give me the usage and syntax of what I would have to do. If it’s not, it’s going to gimme an error message. And in this case it’s not installed, but Kali Linux is smart to say, would you like me to install it? In this case I’m going to say yes and let them do all the hard work of installing it for me. You do need to put in your password for the sudo user, which is the root user and the case of the default virtual machine that comes from kali.org, it’s just going to be K-A-L-I or Kali. It’s going to go through; it’s going to install. It’s going to say, would you like to install it? Say, yes. And there we go. It’s downloading it and off it goes, installing it, and in about another 10 seconds, boom, we’re done. Okay, we’re back to the command prompt. So at this point, we can now use this tool. Now, just to bring myself back up to the top of the screen I’m just going to type in the word clear and in a Linux terminal that just brings you right back to the top of your screen. Now, what I want to do is go ahead and do that Udemy search that I was talking about.

We’re going to go ahead and type in Metagoofil, then dash D for the domain, udemy.com is the domain we’re going to be searching dash T for the file type, and then the file types we want to search for. I’m going to do a doc, a docx, a PowerPoint or PPT, a PPTX, which is the newer version of PowerPoint, and let’s go ahead and throw PDF in there for good measure. Then we’re going to do dash L for limiting the number we want. And I’m just going to go ahead and make it up to 50 of each. And then I’m going to do dash N, which is the number I want to download, and I’m going to do up to 25. And then I want to put dash O for my output directory. And in this case, I’m just going to go ahead and call it Udemy files, and it’ll make a new directory in the directory I’m currently in called Udemy files. And then dash F and dash F is going to be to say, I want to create a file that gives me the results in one combined HTML file. Now, once I do this, I’m just going to go ahead and hit Enter, and it’s going to go off and start that search. You could see here that it is going to go ahead and download those files.

It’s going to save them into that new file folder that I just created, which is called Udemy files. It created the folder, and now we’re out there searching for 50 doc files and we’re going to wait 30 seconds and then it’s going to try the next search. And it’s going to continue to do this through each of those five file types. Now I’m going to go ahead and fast forward through this search because it does take a little bit of time to do all the searching, and we’re going to come back once the search is complete. Okay, our search is complete or in this case, our search actually failed. Now, why did it fail? Well, Google actually will block you if you’re making too many requests because it detects that you’re a bot. And in this case, it did just that. To be able to overcome this, you can actually install something like Tor into your Linux machine, and then you’ll route all your traffic through Tor so you’re coming from different IP addresses between each and every one of your searches. But for the purposes of this demonstration, this gives us the idea of what we were trying to do.

Now notice here at the top, we did search not just udemy.com, but all of its sub domains as well. So we found that they were actually trying to find some files and the about.udemy.com site the investors.udemy.com site and other things like that. All right, now that we did that, I’m going to go and clear my screen to bring me to the top again. Okay, once we clear the screen, I’m just going to do the LS command, which will list out the files. And you’ll see there is a text file with the HTML links based on what it found from all the metadata it was searching. And you also see there is one called Udemy files, which is a folder. And if I go into the Udemy files, you’re going to see that we do have a couple of files here that were downloaded as part of our reconnaissance using Metagoofil, and we can go through those files to look at what we found. The next tool we’re going to use is known as theHarvester.

And theHarvester is spelled as one word with a capital H. Now theHarvester is a wonderful tool and it’s used to gather emails, sub domains, hosts, employee names, email addresses, PGP key entries, open ports and service banners off of the servers. Now, again, theHarvester is another command line tool, and it’s pretty easy to use once you learn the syntax. It’s very similar to what we just used with Metagoofil. Now, when we want to use this command, we can just type in theHarvester and then hit Enter and it’s going to tell us how to use it. Here you could see the usage on the screen. So to use the tool, we’re just going to go ahead and type in theHarvester -d, the domain name that we want to search, in this case I’m going to use udemy.com, -l to limit the number of searches we want to return. I’m going to use five as the number of things I want to return. And then I want to go ahead and enter in -b and the search engine I want to use. For instance, I’m going to use Google. I could just as easily use something like LinkedIn or Bing or something else if I wanted to as well.

Once you’re done with that, go ahead and hit Enter and it’s going to run off and run that command. You’ll see here, we see the banner for theHarvester. We see the fact that it’s going in searching Google. We’re getting zero results, no IPs, no emails, and one host that was found with two IP addresses. Now, why is that? Well, again, I just did the scan from this computer when I was using Metagoofil and so Google at this point is already blocking me because they think that I am a bot or I’m doing something bad and so therefore, they’re blocking me. Again, if you go ahead and change your IP addresses, you go ahead and do something like Tor, it’s going to keep you from having this same problem. Let me go ahead and clear the screen. The next thing we’re going to go ahead and do is work with Recon-NG. Now, Recon-NG is a great tool but it is a little bit more complicated than the two I just showed you. So to startup Recon-NG, you just type in Recon-NG, and hit Enter. When you do that, it’s going to load up. And now you’re in a special command prompt within the Recon-NG environment.

Notice here, we’re in Recon-NG, and we are in the default workspace. Now Recon-NG is a wonderful web reconnaissance framework, and it works on lot like Metasploit does for exploits and the Social Engineer Toolkit works for social engineering. It brings a lot of tools and a lot of capabilities into one place. But because of that, it is a little bit more complicated. Now, also the other thing to note is a lot of the things you’re going to find online for Recon-NG are going to be wrong and they’re not going to work. The reason for that is there was a switch and a lot of the syntax going between version four and version five. You could see here, I’m operating with version 5.1.2, which is the latest at the time of this recording. So all the things I’m going to show you now are based on that version. These will not work in the older version four and the old version four commands will not always work in the newer version five.

So keep that in mind if you’re looking into any tutorials online or YouTube videos. If they’re older than about 2020, you may run into some issues there. Now to use this, we’re going to launch into Recon-NG just like we did here. And now, I like to personally set up my own workspace instead of using the default workspace. This gives me almost a place to store and save all my different commands and tools and information that I find in one area. And so to do this, we are going to use the workspaces command. Now, if you don’t know any of the commands in Recon-NG yet, which you probably don’t, you might want to first type in help. When you type in help it’ll list out all the commands you can use. And you’ll see there that we have one called workspaces at the bottom of the list. And this is used to manage workspaces. A workspace is just a defined area to keep your different information from different engagements into different pockets so they’re not mixing between clients. So what I’m going to do is I’m just going to type in the word workspaces, and then from workspaces if I hit Enter, it’s going to tell me I didn’t give enough syntax because we didn’t know what to do with it yet. Here we can do a create, list, load or remove. In our case, we can see if there’s any workspaces already. And there shouldn’t be because this is a brand new installation. But by doing that, we do workspaces space list and hit Enter, and you see we only have the default one.

I’m going to go ahead and create one called Dion. And to do that, we’re just going to type in workspaces create, and then I’m going to use the folder name of Dion. Now, if I go ahead and do workspaces, list, you’re going to see two default and Dion. All right, now that we’ve done that we can go ahead and select that workspace to work in it. And in this case, because I just created that workspace, it automatically put me into the Dion workspace. But let’s say I had another one because I was going to do workspaces create Udemy, now I’m going to have three different workspaces. Oh, I actually typed that wrong so make sure you type it correctly. Everything in Linux is case sensitive and obviously spelling sensitive. I forgot the S workspace and that’s why I got that error. So here we go, I listed it out and you see there are three workspaces now, Dion, Udemy and default. When you create a new workspace by default, it moves you into that workspace. So you’ll notice when I created Dion right next to Recon-NG, there was this thing that said bracket Dion bracket and that told me that I was in the Dion space.

And you can see that just below the first table where I entered, workspaces create Dion, it moved me into that Dion workspace. Then when I created the Udemy workspace, it then changed me from Dion into Udemy. But let’s say I wanted to go back into Dion. How would I do that? Well, to do that, we’re just going to type in workspaces load and then the name of the workspace, in my case, it’s Dion. Now you can see I am back into Recon-NG inside the workspace Dion. All right, the next thing we have to do is we have to have some modules installed to be able to do some functionality inside of Recon-NG. Now by default, there is no modules installed when you first get Recon-NG and this is a brand new install and if you just loaded up your virtual machine, you have a brand new install as well.

And so these are things we have to work through. So what we’re going to do is we’re actually going to go into the module section, which is another one of those key words. Again, if you get lost at any time, just type in help. And you’ll notice there we have a modules command and it says that it interfaces with installed modules. So to check if there’s any installed modules, I’m just going to type in modules, and then I would use the command search. Now, if you don’t know any of the commands for modules, again, just type in modules and hit Enter. There are your different options. You can load, reload or search. Search is essentially like listing but you can list all of them if you just type in search and hit Enter, or you can actually search for a keyword because there are many modules and maybe you just want a specific module for a specific use case. If I go ahead and hit Search and hit Enter, you’re going to see that I have no modules found and it has a red error there. That’s because I haven’t installed anything yet. And so now we need to go and find some modules. How do you do that? Well, looking back up at the Help area, you’ll see a command called Marketplace.

This interface is with the module marketplace. And this is where you can and search for and download a single module or all the modules. Now, some modules you’re going to find are going to require you to get an API key, to be able to associate with the service. For example, there’s a module to be able to search Twitter and Twitter requires you to register with them as a developer to get an API key, so you can then make that API connection from Recon-NG into Twitter and at searching their stuff. Now to make things easy for our demonstration, we’re not going to go into that. And I’m going to pick a module that does not require API keys. If you’re going to use Recon-NG for real, go online, look at the Recon-NG manual, you’ll be able to walk through how to do all of those things because there’s a lot of capability in this tool. For the exam, you do not need to know how to use Recon-NG. I just wanted to show you so you can get comfortable with it, so you can use it in the real world a little bit and then take it from there. Now, what we want to do first is we want to find a module that we can use.

The one I’m going to use is known as recon slash domain contacts slash whois POCs. I know this one doesn’t require an API. And the purpose of this is for us to be able to use this to look up the, whois data for different domain names. Now to do this we’re just going to use the command marketplace. And then we’re going to use the term install. Now, just like before, if you don’t know how to do something, you can simply type in marketplace and Enter and it will give you the syntax. And then you’re going to put the one that you’re looking for. In my case, I know exactly which one I’m looking for, it’s recon slash domains dash contacts slash whois underscore POCs, hit Enter, and it’s going to go ahead and install that module. You’ll see right here, it installed the module and then it reloaded it. If I want to validate that actually happened, I can do that by using the module search command like I did before to show there was no modules found. So let’s go ahead and do that.

And now, instead of no modules being found, I see that I have one module found. It’s under the recon category and it is the one I just installed. Now, if you want to install all of the modules and there are a lot of them out there, if you want to see them all, just type in marketplace and then hit Search and then hit Enter, and you’ll see there is a bunch. There are so many that is actually going off of my screen. And so if you wanted to actually scroll up here, you can go through and see that there are a ton of different modules, and all of these have different functionality. If you want to know what each one does, you can actually look them up inside of Recon-NG using the info command and you’ll be able to learn more about those particular tools. Or again, go online, go ahead and Google or Bing or DuckDuckGo or whatever your favorite search engine or choice is and look up those and you’ll figure out which ones they are, what they do, and which ones may be helpful in your reconnaissance. Now, if you want to install all of these, you certainly can. And to do that, you just type in marketplace, install all. And if I hit Enter right now it’s going to go and install every single one of those.

Now, the reason I’m not going to do that is because it’s going to clutter up our screen, it’s going to throw a bunch of errors because we haven’t set up all the API keys for all 40 or 50 different recon modules. And so I’m going to not do that right now. But if you want to do that and you want to set up your system fully, you can install all and then go find API keys for every single one of those modules. Now, the next thing we need to do is actually take that module and load it. Now again, if I type in modules search, you’re going to see that I have installed that module. But it doesn’t mean it’s loaded and ready for me to go, because you can see where I am inside of the recon structure. I’m at Recon-NG Dion. I’m not inside a module. I’m not inside any of the options. I’m just inside the workspace. So again, we’re just going to type in modules and hit Enter. You’re going to see the syntax.

We have load, reload, and search. I’m going to go ahead and type in modules, load, and then I want to use the, whois underscore POCs, which is the short name for the module I loaded. There we go. Notice how my prompt change. Now I’m inside this module. Now that we’re in the module, we need to specify the options that we want to use with the module, and we’re going to do that by entering information into our database. Now, inside of Recon-NG, there’s actually a database installed with it. And you’re able to put information into the database both things you want to search for and information that’s going to come back when you do those searches. Now to do this and work with the database, we need to use the DB command. And again, if you get lost at any time, just type in help, you’ll notice there the third line down is DB which interfaces with the workspaces database.

Each workspace has its own database and that’s another reason you want to have a workspace dedicated to whatever engagement you’re working on. So we’re going to go ahead and use DB. And then again, we can hit Enter and it’ll tell us what things we can do with the database. We can delete a database, we can insert a line in a database, we can add notes to a database, we can query it using SQL commands or we can look at the database schema. Now, if I wanted to see the schema of the database, I can just type in DB schema and hit Enter. And when I do that, again, it scrolls off the screen a little bit but as we scroll up you can see the different tables that we’re using. Let me get up here to the top. There we go. So the first one we have is domains, and this will hold domains, notes, and the module that found the information. Under companies it has company description, notes, and module. And you’re going to see each of these are going to interact with different parts of Recon-NG and different modules.

Domains is the one we’re actually going to be working with here because I’m using the, whois, point of contact module. So let me go ahead and get back here to the bottom. I’m just going to hit Enter and it’ll drop me right down. Okay, and now what we want to do is go back to our database command, and what we want to do is insert information into that database, specifically, the domain names that we want to look at. Now, what I’m going to do is I’m just going to type in DB insert and then domains, which is the field that I want to insert information in. And at that point I hit Enter and it’s going to say, what do you want to enter in here? Well, the thing I want to enter is the domain name. So if I wanted to search diontraining.com, I can enter that in. If there’s any notes I want to put in, I could put them here in too, I’m just going to hit Enter and make that blank and it’s going to insert that into one row.

Now, if I want to enter another domain to search, I can do that here as well. Let’s go ahead and do insert domains and in this case I’m going to use udemy.com. Another one I want to enter. Let’s go ahead and do DB insert and we’ll do domains and I’m going to use tesla.com. And again, no notes. All right, we have now inserted three different domain names, diontraining.com, udemy.com and tesla.com into our database. Now, if we want to see that and verify it took, we can use the command show and then the name of the table in this case domains. So show domains and here is our table. Boom! We have three rows, diontraining.com, udemy.com, tesla.com, no notes on any of them, and the module was user defined because I manually entered that information. All right, now that we have loaded our module, now that we have entered our information into the database, we are ready to start searching the whois database for these points of contact.

Now, what we’re going to do is we are going to use this whois POC module. If you don’t know what that module is, well, we’re inside of it now so we can just type in the command info. By typing in info, it’s going to tell me about the module I’m currently in. Notice, when I typed info for whois POCs it tells me this is the, whois POC harvester, who wrote it, the version, and a short description of it. It’s going to use the ARIN whois database to be able to harvest POC data from the whois queries for the given domain. This is also going to update our contact table with the results inside of our workspace. So that data we get back is going to fill part of our database in the workspace, so we can go back and look at that information later. Now, we have some options that we have to have. We have a source, we have the value of default, it’s required field, and the description is the source of input. Now, by default, that means it’s going to go and grab it from the table.

Those three domain names we just put in in rows one, two, and three, using the domains table. If I wanted to do this from a file, I could do that as well by changing this default value from default to the file name. In our case, we’re going to do it right from the database. It keeps it nice, it keeps it clean and that’s the way we’re going to do it. Now, if I wanted to change that, I could do that by changing that by saying options set because I’m changing the options in this case, I want to set the value of that from default to something else. Now, in our case I don’t want to do that so I’m going to go ahead and delete that but you could do that if you wanted to go ahead and read it from a file, read it from an SQL query or something else. As you can see here, the default is to select distinct domain from domains, where domain is not known. Which just basically means go into the database, look for the domains table and any domain inside of the domains table I want to grab each one that’s not blank and we’re going to test it. So this is going to allow me test all three of them with one command. Now that we know that our options are good, to run this command, we are just going to simply type run and hit Enter. When we do that it’s going to go off and it is pulling that information and it’s grabbing all that information and it dumps it to the screen.

Now that’s helpful, but the screen makes it pretty hard to read because I have to scroll up and look at that because we just found 16 new record and 12 of those were new contacts that were either not duplicated or were new things that we wanted to add. Those all got put into our database. Now, if you remember back I said, we can show things from the database using the show command. Before we used show with domains to show the three domains. Now I want to go ahead and use show with contacts to see the contacts table. Here it is. So we have those 12 entries going from row one all the way down to row 12. Now my screen is a little bit zoomed in to make it easier for you to read in the video. If I was zoomed out, it would all fit in one nice table. Here you could see that three columns, the phone, notes, and module row, went to the second line. Now let’s go ahead and read the first line. The first line we have is based on Udemy. You could see here that we found no first name, no middle name, the last name was operations architect, the email was netops+ARIN@udemy.com. The title was whois contact because that’s the type of information we got. The region, San Francisco, California, the country United States, and then phone blank, notes blank, module whois POCs, which is the module we use to find this information. Now, there is lots of modules in Recon-NG.

You might be finding contacts by doing Twitter searches, LinkedIn searches, Google searches, whatever it is, all that will go into this database to create our table of people. But this tells us which module found that information and which type of information it was in this case, a whois contact. Next, we have lines two through 12, and these are all based on Tesla. Now you’ll notice Dion training didn’t show up. The reason for that is we actually have our whois records set up with privacy. And so those are not going to be shared in the whois database. It just says this is a private record. So there was no way to grab that information and add it to the table. That’s why we have the one for Udemy, which is a very common way of doing it for a large organization. They actually have a group email, not a person’s email. But if we look at Tesla, they chose not to do that. Tesla actually has individual people’s names. Now, as I look at them, what is this information that’s going to be useful? Well, for one, I have names of people and emails I can use. That’s the obvious one. But in addition to that, I might be able to figure out what naming scheme that company uses. For example, maybe you can’t find everybody’s email when you’re doing your open-source research, but you found their first and last name on LinkedIn. Well, if you know that the company uses first name dot last name, like elon.musk@tesla.com, then you could put that in for everybody you find on LinkedIn and now you have their email. Conversely, though, we actually don’t see that here. With Tesla, I am seeing multiple different naming schemes. The first one, Anna, actually shows up as Ann, which is three letters from her first name and then her full last name martinez@tesla.com. The next one is just an abbreviation for the name Cameron, and they called it cam@tesla.com. The next one is Cheri, but it’s abbreviated down to C-H-E the first three letters again and then her last name lewis@tesla.com. We get down to Elon Musk, it’s first name dot last name.

We get down to Jian Gu, we’re seeing his full name slapped together with no period. So it’s another different convention. We go down to line nine and we see Mahdesai. and this is the first three letters and the last name@tesla.com. We get down to Paul Smicker and we just see paul@tesla.com. We get down to Terry Chi and we see tchi@tesla.com. So we’re seeing a little bit of differentiation here, but several of them, I saw at least three that had the first three letters and then the last name. So I’m thinking that might be the naming convention at Tesla for their employees. Now, if I wanted to validate that I could take some people’s names who I find on LinkedIn, and I know their first name and last name put them into that format of the first three letters and their last name@tesla.com and then see is it a valid email by checking something like email dossier at CentralOps or other things like that. So hopefully you could start seeing how we put all these different tools together and get information from different places, consolidate it, and then we can start doing things with it like spear phishing campaigns, or whaling campaign, or social engineering in general.

Lots of different ways to start using this information. Now, as I said at the beginning, there are a lot of different modules to Recon-NG. And I just wanted to show you the basic usage because all the modules work the same way. As you go into a workspace and then as you go into a module, you’re going to keep going through the directory structure like you saw here, Recon-NG, Dion, whois POCs. Now, if I wanted to go back, I could just type in the word back and it will bring me up a level as well. And then I can go ahead and load a different workspace, and then I can go ahead and do another assessment. Now, remember, you can always use the show command to show anything you want. Just like I did show domains or show contacts, you can also show company, you can show credentials, you can show hosts, leaks, locations, net blocks, ports, profiles, push pins, repositories, and vulnerabilities because all of those have tables in the database as you saw when we looked at the database schema. Now, all of those get things from various modules that you may or may not install.

So that’s the important thing is to have the right module for the right thing you’re trying to grab. In this demonstration, I only used one module and I only searched for contacts. So that’s why we saw things in this show contacts to display my findings as you see here on the screen. But the great thing, is if I ran other searches from other modules and they found contacts, they would also be in this table and I could easily see those too. Now, hopefully you’re starting to see how all of this starts coming together, and you’re going to play with it a little bit more on your own. As I said before, the thing that’s going to make you a great penetration tester is hands on the keyboard, practicing with this stuff, doing different reconnaissance evolutions, trying to find information that’s out there. Now the great thing about open-source intelligence and learning how to do this, is that all the information is out there. You don’t need permission from any of these companies to go look up this information because it’s all public information that’s sitting online. And this is a great way for you to start building up your skills early on in the reconnaissance process. Because as long as you’re doing passive reconnaissance, you’re not touching that company servers and you are not conducting any kind of hacking. You’re just in the preparation phases, you’re just learning information, and using these tools and getting better at them is going to make you a better penetration tester in the long run.

• Category: Uncategorized