· video · 14 min read
nMap Explorations - A Great Time - DEF CON 31 (Blue Team Village)
My Talk from DEF CON 31 in 2023
(n)Map Exploration: A Great Time in Remote Destinations
Let’s take a look at activity within a corporate environment. Can we find actions that stand out or might be suspicious?
Video Walkthrough
Overview
What will we learn?
There are a number of concepts we will go over and learn in this walkthrough:
- What is reconnaissance in a security/threat hunt context?
- What are some ways that adversaries can identify points of exploitation within an environment?
- What are some commonly abused services that adversaries use?
- How can we walk through the thought process associated with a Threat Hunt from hypothesis to tangible results?
- What tools do we have at our disposal and what can we do with them?
- Can we go deeper and find out more after validating our hypothesis?
Initial Required Concepts
In order to dive into the hunt, we need to have some baseline information to better understand what we are seeking to find. Let us take a look at some of these now.
What is reconnaissance?
In a security context, reconnaissance is the process of gathering information about a particular target. It’s important to recognize that reconnaissance is not always malicious in nature. It is not unexpected or unheard of to see some tools in use within the organization that perform this activity. Reconnaissance is often utilized prior to potential exploitation as an adversary would build out a “map” of items that exists in a particular environment.
What are some ways that adversaries can identify points of exploitation?
An adversary can begin with some basic web searches to find information that is freely available on the internet to begin building a model associated with a particular “entity” - often in this context the “entity” is an organization. We call this freely-available information on the internet Open Source Intelligence, or OSINT for short. There is often a limit to the usefulness of the information provided by OSINT resources as it pertains to actively using it against an entity.
What if the adversary is already connected to the network?
If the adversary already has a limited foothold into the network, there are more effective things that the adversary can do:
- Build a network map of the environment - what systems are there? what operating systems are there?
- Identify services that the adversary could then utilize to move deeper into systems/the network.
- Exfiltrate data
From a security point of view, we call this building a list of systems “Enumeration”
What tools could be used to do this enumeration within the network?
Some tools for this include:
- Network Scanning Tools: Used for building out systems associated with a particular network, including open ports, operating system type, and more. A few examples of applications in this group include the popular tool nmap and as well masscan.
- Vulnerability Scanners: OpenVAS/Nessus are two vulnerability scanners that could theoretically be used to identify hosts within a network.
- Built-in OS Tools: The ping tool could be used to identify available hosts within a network, though
Now that we understand more about reconnaissance, let’s put these concepts together to see if we can build a Threat Hunt with these ideas in mind!
Threat Hunting
What is Threat Hunting?
Threat Hunting (TH) is a process of being proactive of unveiling unknown-knowns and unknown-unknowns to better our security posture
Hypothesis
In order to begin the Threat Hunt, we need to have a reason to start the hunt. To start, we will need to come up with a hypothesis.
What is a hypothesis?
A hypothesis can be described as something we think might be occurring or something that we think might be taking place in an environment. A hypothesis focuses on the 6 W’s:
- Who: Who is doing the activity?
- What: What is happening?
- Where: Where (What systems/networks) is this happening?
- When: What time/time period did this happen?
- Why: What is the end goal for the activity being performed?
- How: How is the activity occurring in the system?
How do we create a hypothesis?
For HOW to create a hypothesis for Threat Hunting, you can read an in-depth guide here: (insert link).
For now we will go over the hypothesis I have created for this scenario.
Let’s create a hypothesis to hunt for reconnaissance!
Broad Hypothesis
The company has reconnaissance activity within the network.
Narrow hypothesis
Reconnaissance activity is occurring in the Magnum Tempus environment, and some of the activity is malicious.
Transitioning from a Hypothesis to a Query
How can we formulate a query based on our hypothesis?
Our initial hypothesis presumes that there is reconnaissance activity within the environment. While this is most certainly true, we need to dig deeper. There may indeed be this ocurring, but as we noted above, not all reconnaissance activity is outright malicious.
First we need to understand what tools we have available. For this Hunt, we are using Splunk, however many of the procedures/methods we use can and should be used with other query languages/log platforms.
Since we know our log data is in Splunk, we first need to determine what log sources we have available to us.
We could modify our index to include ALL log source data using a wildcard query like this:
index="*"
HOWEVER, this is considered bad practice as doing so is an expensive (resource-intensive) query and could cause undesirable effects on the Splunk (or other Log Aggregator) server and could hinder other searches being done on the system.
Thanks to our friend, Cereal Killer, we can utilize the following query to find the indexes (source) of data to query against:
| eventcount summarize=false index=* | dedup index | fields index
While you may not know this, when I think of nmap usage, I think of it being used on Linux distributions. As we can see above, there is an index for Linux, so let’s start there!
In Splunk queries, we start by calling the index (source) of the data we are querying against:
index="linux"
We can query with just this, however we will get ALL data in the associated log:
NOTE: We need to change the time filter to the time period we are checking against for this specific scenario because we don’t know exactly when the log data starts/stops. In your own Threat Hunting (outside of this scenario), you will want to identify a timeframe to conduct your searches. This could be 24 Hours, 1 Week, 1 Month.
In this scenario for this hunt, we are focusing on the whole day of 2023-04-29.
In Linux, an example of a basic host nmap scan command line entry would look like this:
nmap -sn 192.168.1.0/24
To make it easier, we can expand on our existing query and add just the text “nmap” to it, since the nmap application is just simply nmap.
index="linux" (nmap)
Okay, so for some reason this did not provide any results.
Did we do something wrong here? Not exactly. We’re conducting a threat hunt for something we THINK might be in the environment. A threat hunt will not always be successful, but sometimes it can be.
Let’s look at the index list again. So we see there are a few items that might be interesting to try. We see sysmonforlinux. Sysmon is a tool that is used to get telemetry data (information about what’s happening on the computer, even activities that the end user doesn’t ever see) and forwards this data to a log collector. This might also be a great source to query against.
We can also try adding some of the other tools we referenced above to the query as well.
How can we refine this query?
Let’s start with the previous query:
index="linux" (nmap)
So there was no nmap activity in the linux logs in Splunk. Let’s pivot to include another potential log source as well as another command line reconnaissance tool, masscan.
Masscan might not be as commonly used as nmap and if we think like a potential adversary, we might want to use a tool that is less commonly used and therefore potentially less likely to be recognized.
An example of a masscan command line would be:
masscan -p 80,443 192.168.1.0/24
This scan is specifically looking for open HTTP (80) and HTTPS (443 Secure!) ports between 192.168.1.0 and 192.168.1.254.
We can modify our search as seen here:
index IN (linux,sysmonforlinux) (masscan OR nmap)
This query did not net us any relevant hits, either.
Let’s pivot again - this is normal. Everything is fine.
How can we refine this query since the previous did not get us what we expected?
Let’s start with the initial query:
index="linux" (nmap)
Our initial hypothesis presumes that we have reconnaissance activity in the environment. We started looking at Linux systems to see if there was activity there, but perhaps we made a problematic first assumption that the reconnaissance would come from a Linux system. Let’s go with the “sysmon” and “windows” index choices. Much like Sysmon for Linux, Windows Sysmon can provide telemetry from Microsoft Windows systems and there actually is a nmap version that can be used in Windows. We will not include masscan at this point because it’s less common in Windows.
Let’s go!
Here’s what we should modify our search query to:
index IN (sysmon,windows) (nmap)
Let’s run this query!
We have hits!
Let’s look at the overall results:
182 events - digging through this, we could probably refine this list down. Let’s take a look at the hosts that show nmap activity. If we click on the host field on the left side, we see that 5 systems have been identified with something related to “nmap” on it:
What we also notice is that a LOT of activity related to nmap appears to be by one particular system:
wkst16.magnumtempus.financial
Let’s dig into this system a bit to see what’s happening.
We need to modify the query again
index IN (windows,sysmon) (nmap) AND host="wkst16.magnumtempus.financial"
Running this, we see 152 events that match this query.
Looking through these entries the only directly interesting entry is one associated with Powershell, but it appears to be activity associated with installing nmap, so I don’t see this as particularly nefarious.
We also see this system is associated with Seth Morgan, an IT Engineer. Without knowing much more about what an IT Engineer is within the orgainzation, let’s take a moment to presume this is expected activity by this user.
Let’s compare with Excluding the host from the results:
index IN (windows,sysmon) (nmap) AND NOT host="wkst16.magnumtempus.financial"
29 events!
Let’s take another look at the breakdown of the hosts associated with this query:
We see now that the top talker for this activity is:
iot-eng-wkst.magnumtempus.financial
Let’s refine our query and dig more into this system:
index IN (windows,sysmon) (nmap) AND NOT host="wkst16.magnumtempus.financial" AND host="iot-eng-wkst.magnumtempus.financial"
So now we have found some interesting activity that seems like it would be somewhat questionable. Many of the top few events we see are from the user we saw previously, Seth Morgan, connecting over the network from the source IP Address 172.16.50.20 to the destination IP Address 172.16.50.19 via port 3389.
What is port 3389?
Port 3389 is commonly associated with Microsoft Windows Remote Desktop Protocol (RDP). This is a service that allows users to connect to a computer remotely via another computer.
!!!IMPORTANT NOTE!!! Remote Desktop Protocol service should never be exposed to the public Internet. If this is required, please be sure to have this service hidden behind a secure VPN or some other access restriction.
Moving forward with this in mind, we need to determine what the system 172.16.50.19 is. For this, we’ll need to craft another query.
Don’t worry, this one will be simple and we’ll get back to our main query shortly.
index=* 172.16.50.19
When we run the query, we see can look at the hosts that are somewhat associated with this IP:
Over 40% of the traffic is associated with this asset:
iot-jumpbox.magnumtempus.financial
For those who may not be aware, a “jumpbox” is a system that is used as an intermediary between networks that may be highly restricted. This can be a normal practice and based on the hostname, we can presume this is an expected system on the network.
Let’s make a modification to the original query to only look for nmap activity from this system:
index IN (windows,sysmon) (nmap) AND host="iot-jumpbox.magnumtempus.financial"
Running this query, we see two entries this time:
The first entry appears to be a file creation in the C:\Windows\SysWOW64 folder for nmap. Interesting, we may need to understand this more.
The next entry is very intriguing. Looking at the metadata within this, we see command line activity from this system probing the overall network using nmap:
CommandLine: nmap 172.16.60.0/24 -p 22,5900,8080,80,443,502 -oA a -v
Also, of note, is that this command was being run by a local user:
IOT-JUMPBOX\iotadmin
We know this is a local user due to the fact that the username includes the system name “IOT-JUMPBOX”. If this were a domain account, we would theoretically expect to see “MAGNUM.TEMPUS\iotadmin”
Let’s look at the command and dissect this so we understand what the command is looking for.
As we discussed previously, the command goes across a subset of the network, looking at the entry
nmap 172.16.60.0/24
This will cover all systems between the IP 172.16.60.1 and 172.16.60.254
-p 22,5900,8080,80,443,502
This is an interesting part of the command because of the wide range of open services that this is looking for. Let’s break each one down:
- 22: This is for a remote service, Secure Shell, commonly called SSH. This is often used for remote access to a system, commonly over command line.
- 5900: This is another remote service, Virtual Network Computing, VNC. This is used for Graphical remote access to systems, similar to Windows Remote Desktop Protocol.
- 8080: This is looking for a web server. While this is not commonly used, 8080 is used in situations where the normally default web port 80 is unavailable.
- 80: See above. This is the more common port used.
- 443: This is for Secure HTTP, HTTPS. This adds a secure layer to regular HTTP.
- 502: This port is the most intriguing of all of the ports for this query. This is not a port we would expect to be queried for except in specific scenarios. It is specifically used for Modbus TCP/IP, allowing devices in industrial automation systems to communicate over TCP/IP networks. Modbus is widely used for exchanging data between devices such as PLCs, SCADA systems, and other industrial controllers.
-oA a -v
- -oA a: This flag is used to specify the output format and file naming convention for the scan results. In this case, the letter “a” is used as the argument after -oA. It instructs Nmap to save the results in three different formats: normal format (a.nmap), XML format (a.xml), and grepable format (a.gnmap). This makes sense as to why we saw the file being created in the Windows SysWOW64 folder, and based on the filename, matches.
- -v: The -v flag is useful for obtaining a more comprehensive view of the scanning process and gathering additional insights about the targets being scanned.
Where does this leave us?
What stands out for this is that a local admin account on the jumpbox is attempting to enumerate a network, specifically looking for exposed webhosts, remote services, and curiously enough a port associated with industrial systems.
While this could be legitimate activity, I would not expect to see this type of a probe with all these seemingly unrelated ports being queried in a normal business operation. An administrator would not be looking for remote access ports, web servers, and an industrial-specific port, but instead have separate queries to do this.
What’s important to note is that our Threat Hunt focused only on computer/asset logs and we did not investigate any network traffic logs that could have provided additional context and support for our Hunt.
This activity definitely raises some red flags, so we need to provide our findings to the next team for them to dig in deeper.
Let’s build our Threat Hunter Template now!
THREAT HUNTER TEMPLATE
Playbook Title: Detecting Reconnaissance in Company Environment
Mitre Tactic: T1046 - Network Service Discovery
Mitre Sub Technique: N/A
Hypothesis: Reconnaissance activity is occurring in the Magnum Tempus environment, and some of the activity is malicious.
Proposed Detection Query: index IN (windows,sysmon) (nmap) AND host="iot-jumpbox.magnumtempus.financial"
Simulation Details: NONE
Hunter Limitations/Observation Notes: Network traffic flow logs were not available during this hunt. Having access to these logs could add additional context for the search. Also of note, all of the fields were not parsed within the logging platform, making queries slightly less efficient and not as readable.
Hunt Findings: There may be some fishy activity occurring, since we detected probing on an uncommon port associated with PLC/SCADA