Firewalls are a critical part of any organization’s defense in depth strategy. They serve a key role in protecting your network against malicious actors, and they do this well. But they also have a dirty little secret.
Firewalls aren’t a set-it-and-forget-it device. They need tending and maintenance.
Many organizations don’t devote enough time to maintaining and tuning their security infrastructure. They turn on their firewalls and assume they’re protected. But the truth is, you need to monitor and analyze firewall logs to get the most out of these critical network devices.
In this post, we’re going to dive into five tips for improving your firewall log analysis skills. I’m going to talk about ways you can maximize the value from the time you spend analyzing your firewall logs and ways you can use those skills to automate a lot of this analysis. By the time we’re done, you should know what you need to add to your current process to improve your organization’s security.
1. Keep the Documentation Handy
I know this seems simple, but you’d be surprised how often people overlook documentation. Most firewall logs aren’t designed to be read by humans. They’re written in a foreign language with its own syntax and style.
What’s more, each vendor has its own style and syntax for firewall logs. This means devices from one vendor will have their own unique peculiarities. If you’ve worked with one vendor for a long time and you switch to another, you’re likely to experience some growing pains.
By keeping the documentation for your firewall vendor handy, you make it easy on yourself when you find a log entry you don’t immediately recognize. Most of the log messages you’ll see will be mundane. However, some messages might look unusual at first glance. These log entries may be signs of malicious behavior, or they may be nothing. The only way to tell the difference is to investigate, and the best place to start your investigation is with documentation.
2. Understand Your Network Topology
Most firewall logs don’t provide much information to the uninitiated. You’ll see a couple of IP addresses, ports, and information about whether the firewall blocked the request. If you’re scanning through firewall logs looking for indications of bad behavior, there’s not much to go on.
Unfortunately, bad behavior often hides in plain sight. A request on port 80 or 443 headed toward a web server is no big deal. But the same request headed toward an employee desktop is bad news. Your job while looking at firewall logs is to be able to tell the difference between the two at a glance. To do this, you need to understand your organization’s network topology inside and out.
One key mistake many experienced engineers make is not keeping up to speed on network changes. They memorize the network map at some point and rarely go back to it. Weeks or months later, they see a firewall entry that looks a little funny. They only then discover they’ve missed some critical server in their logs up to this point. If they weren’t a part of the provision process, they miss it completely, and then they need to go back and figure out if anything they missed exposed the organization to unacceptable risk.
If you end up in this situation, the extra work often takes you away from something important. And I can say from experience, it feels terrible. Instead, regularly check in on your network topology to understand new servers and servers your organization decommissions.
Frustration-free log management.
(It’s a thing.)
Aggregate, organize, and manage your logs with Papertrail
3. Look for Calls From Inside the House
Many engineers think of firewalls as tools only designed to keep malicious requests from getting into their organization. They’re also useful for keeping the malicious requests your organization is making from traveling out to the wider internet.
Obviously, the best situation is when you don’t have any malicious requests coming from inside your network. But the reality is, users make mistakes. They install things they shouldn’t. When they do this, one of the first warnings you might see about your infected system is an odd connection request. For instance, back in 2011, the Morto worm spread by targeting the port used for RDP and attempting to log on to any systems it found using common administrator passwords.
If you’re a security engineer, you know it’s your job to make sure your organization stays secure. It’s also your job to make sure you’re not spreading an infection if your security is breached. By watching for suspicious requests coming from inside the network, you’ll make sure that even if your security slips, the problem won’t affect another organization.
4. Watch for Spoofed Requests
Sometimes, you’ll find requests on your external firewall that look like they’re traveling between nodes on your internal network. Obviously, this isn’t the kind of connection your firewall should catch.
When I was a less experienced engineer, my first impulse when finding a request like this was to think I’d discovered evidence of malware. I only discovered IP spoofing was occurring after some investigation.
This practice isn’t necessarily malicious on its own. However, it often serves as a precursor to a more malicious action on the part of an attacker. If your firewall logs an attempt to connect between two internal services, it’s usually a good sign something else might be happening, and you should pay special attention to the other log entries around it.
5. Watch for Port Scans
Much like spoofed IP addresses, port scans can show an attacker is about to try something malicious. By scanning the ports on your network to see which ones are open, an attacker can craft a more tailored plan to target your network. In a well-configured firewall, a port scan will often look like a series of many requests to a single IP on a wide range of ports. Alternatively, other port scans will look at just a few ports, but they’ll target an entire IP range.
If you detect a port scan in your firewall logs, you should consider blocking the IP that originated the scan, at least for a little while.
Conclusion: Know Your Traffic, Then Automate
Once you’ve spent a while getting to know your network, you need to graduate to the next step. You can’t spend all your time looking at firewall logs; I’m sure you have other work to do. So once you understand your firewall logs and the traffic they represent, look into a log analysis tool like SolarWinds® Papertrail™.
By integrating a log analysis tool like Papertrail, you can make many of these steps automatic. For example, you can filter out much of the ordinary traffic in your firewall logs to quickly focus on unusual behavior, search by IPs or event types, and save your frequent searches. Using your searches, you can set up alerts and notifications and eliminate manually scrolling through your logs. If something looks suspicious in a notification, you can use a log visualization chart to visually explore the volume of different events or time periods,
But you need to understand your network traffic before you can teach Papertrail how to handle it. Without your knowledge and skill, no tool alone can effectively monitor and analyze firewall logs.
So get out there and read your firewall logs. When you’re ready to automate collection and alerting, give Papertrail a try.
This post was written by Eric Boersma. Eric is a software developer and development manager who’s done everything from IT security in pharmaceuticals and writing intelligence software for the US government to building international development teams for nonprofits. He loves to talk about the things he’s learned along the way, and he enjoys listening to and learning from others.