by Rik Farrow <rik@spirit.com>
Imagine network security the way it should be. You show up for work in the morning, and get a report via email from your IDS: everything is okay. You might feel a little dubious about this, but why should you? After all, isn't the goal of network security to prevent problems in the first place?
Today's IDS (Intrusion Detection Systems) don't come anywhere near reaching that goal. Instead, the main allure of IDS is that it will let you know about what might be wrong in your network, and most products focus on producing lots of alerts about incidents so that you get the feeling that you didn't waste your time and money deploying the system. And while knowing what is happening on your network is a worthwhile goal, it doesn't in itself do anything to solve the problems manifesting through the medium of the IDS console.
Perhaps I am being a bit unfair. But if you do nothing but listen to vendors rave about their products, you are bound to be mislead. As well as disappointed when the reality of what your shiny new IDS can't do sets in. IDS products are getting better, but still have a long ways to go.
This article is not about products. Producing a useful review of IDS products is an extreme undertaking, and I suggest that you visit Neohapsis' site (see Resources) for a link to just such a review. I talked to Greg Shipley of Neohapsis about the progress of the review, and discovered that it should be published about the same time this issue hits the streets.
First, some terminology. There are at least two general types of IDS: host-based and network-based. Host-based systems get installed on the server or desktop they are designed to protect, and generally watch logfiles and monitor files for changes. Some host-based systems are hybrids, in that they will also monitor the network traffic sent to the host where they are installed. Host-based IDS send alerts to a central console, as do network-based systems.
Network-based IDS, or NIDS, sniff network traffic using a system which is called a sensor. The sensor's main task is to collect all packets and evaluate both the network headers and the data looking for signs of misuse. Many sensors focus on attack signatures, that is, a pattern in the header or data that matches a known attack. Some sensors go beyond mere signature matching, attempting to match traffic with correct layer four and seven protocols.
What Shipley could disclose about the results of the review were striking. For example, many NIDS sensors crashed during the testing under real, not test, network loads. The fastest sensor caught almost every attack, but had a user interface that only a UNIX sysadmin would adore. A different product with an even faster sensor had a great user interface, but only a tiny signature database--so it missed many attacks.
Another issue for existing NIDS is that some of the best known vendors are only now catching up to some of the tricks used to evade IDS. In early 1998, Thomas Ptacek and Timothy Newsham published a paper about bypassing NIDS by using fragmentation and out-of-order packets. And what was true in 1998 remains true to some degree today. While many NIDS do attempt to defragment packets sniffed off a network, it is just not as simple as that. You can read my July 1999 Network Defense column to learn more about fragmentation issues and NIDS. And while vendors might claim it is not so easy to fragment attacks, Doug Song, while an employee of Anzen, an IDS company, wrote fragrouter, a tool that makes it easy to fragment any network traffic.
A related issue has to do with out-of-order packets. Another way of
playing hob with NIDS is by sending small packets containing
an attack split among them so that the attack signature does
not appear in any one of them, and sending the first packet last.
The task for the NIDS sensor is
to reassemble the data stream, then analyse it for attacks. IDS
vendors have taken to calling this maintaining state, and adding
this to a NIDS sensor already operating at its performance limits
often pushes it right over the edge.
A sensor that a vendor claims can monitor 100 megabits per second can often only do this part of the time. Shipley wrote, in another article, that part of the reason has to do with packet size. An 100 Mbs ethernet filled with maximum sized packets can carry, at most, 8,120 packets per second. But send the smallest possible packets, for example pure TCP ACK packets, and the packets per second goes up to 144,880. Now, to make things really interesting, let's make many of those packets acknowledgements to ongoing TCP connections that the sensor is supposed to be monitoring in its state table, and you can really begin to understand why many sensors experience meltdown.
It is one thing to sniff 144,880
packets per second (no mean task), another to check them for known
attack signatures, but matching them to existing data streams before
checking them is quite another. And an attacker can, through the
use of a port scanner, quickly set up thousands of connections in
the state table, leading to memory overload. According to Shipley,
many products had memory leaks, places in the code where memory
was used but never freed, that lead to crashes over time.
The failings of IDS' may be legion, but I am not claiming they are useless. In fact, a properly working IDS brings up the next big issue for proper deployment. Who is going to handle the potential security incident that the IDS is reporting?
I talked to a person who uses the free NIDS tool snort (http://www.snort.org) to monitor the internal backbone at a large colocation facility about practical uses of NIDS. At this center, snort collects between five and six thousand alerts each weekday (fewer over weekends), and this is with a trimmed list of attack signatures. With so many potential incidents, all they do at this site is filter through the alert logs on an hourly basis trying to determine exactly what is significant and what, if anything, can be done about it.
Most sites won't have to deal with thousands of alerts each day. But even dealing with several alerts each day is a fulltime job for someone skilled in network and operating system security. The person monitoring the IDS console must first perform triage--deciding which alert deserves immediate attention, and which can wait. Then this person must locate the possibly compromised system, determine if the attack has succeeded, and if so, decide what to do next. Should the system be further analysed? Saved for evidence? Or just reformatted and a new, patched, operating system and applications installed? And does this attack hint at more undiscovered vulnerabilities on other systems that should also be patched immediately?
Just installing a single sensor and NIDS console can easily consume weeks. Proper installation involves not just the physical process of installing the sensor and installing the console software, but also configuring the NIDS itself. Almost of IDS products support some level of customization. Sometimes, the IDS will do this 'automatically'--you launch a tool, and the tool scans the network, attempts to identify operating systems, then only watches for attacks specific to those IP addresses and the operating system associated with a particular address. Of course, you must remember to launch this configuration process anytime a system is added to your network.
More likely, you will need to manually select sets of attack signatures that you are most interested in. For example, if your organization has nothing but Microsoft systems, you don't want the IDS console annoying you with alerts about UNIX/Linux exploits directed at internal systems. You might want the IDS to alert you to attacks against external UNIX systems coming from your own network, though. And as you use a tool, you will begin to learn which alerts will (most likely) always be false positives--something the IDS picks up as an attack, but which represents normal network activity at your site.
And the console will be of critical importance to you. The console
must be capable of providing a high-level view, and still permit
you to drill down to collect more detailed information about a
particular alert. There must be a mechanism for clearing alerts
when you are done with them, or annotating the alerts, so you can
remember later what you have done about an alert. Other nice
features include the ability of the console software to logically
group related alerts, and allow you to associate a trouble ticket
to ones that require further action.
And hopefully, the console won't just present you
with a terse log-style message, but will provide you with information
about what the alert means, how serious it might be, and how
to go about actually checking to see if the alert represents
a real event, and finally, how to patch the system involved.
Okay, I am dreaming. While some IDS systems do give you a hint about what an alert means, and a few even make suggestions about how to check a system and see if the attack has been successful, that is far from the dream IDS system I alluded to in the beginning of this column. I'd like to enter a fantasy world for a minute, and imagine what a Manhattan Project approach ($2 billion 1945 dollars) could do to implement the dream IDS.
The dream IDS would be a hybrid, with both host-based and network -based sensors. The host-based module would also permit upgrading the operating system or software on monitored machines, include the ability to adjust file permissions/ownerships, registry keys/configuration, and OS tuning parameters, all without rebooting. And this host-based IDS would consume few CPU cycles, so installing it even on heavily loaded servers won't be an issue.
The NIDS sensors would be installed in switches (two vendors already can do this), so that all traffic can easily be monitored. The sensors are not just passive monitors of network traffic, but keep track of in use IP addresses, use passive fingerprinting to identify operating systems, and active fingerprinting if necessary. When a new system appears, the sensor goes into active mode, and performs a vulnerability scan of the new system, installing any OS or application updates that have been previously approved for that platform. The central console also gets informed of the presence of a new system on the network.
The NIDS sensor keeps a history of all network traffic. If a new service suddenly appears on a system, this could be an indication of a successful attack, or installation of a trojan. The NIDS not only alerts the console, it also may take countermeasures against the possible trojan--especially if the NIDS sensor detects other suspicious activity, like a sudden flood or port scan originating from the suspect system.
While this system is certainly a dream, it is not mine alone. DoD research projects, with names like Emerald and Sapphire, focus on assimilating reports from an array of sensors to provide the big picture, correlation between reports, and the ability to drill down to get details from individual sensors or hosts. Someday, the dream may come true.
Thomas H. Ptacek and Timothy N. Newsham ID paper: http://secinf.net/info/ids/idspaper/idspaper.html
Greg Shipley's article about ID systems in Network Computing: http://www.nwc.com/1122/1122f3.html
Grep Shipley's and Neohapsis' review of ID products will appear at http://www.neohapsis.com/articles/default.php.
Dug Song wrote fragrouter, a tool for testing network IDS while he worked for Anzen: http://www.anzen.com/research/nidsbench/
CERIAS page that defines terms and provides references: http://www.cerias.purdue.edu/coast/intrusion-detection/ids.html
Nice summary of different types of ID (including vulnerability scanners): www.icsa.net/html/communities/ids/buyers_guide/guide/Technology/technology.shtml
CSI ID system roundtable: http://www.gocsi.com/roundtable.htm