PATCHING SYSTEMS

by Rik Farrow <rik@spirit.com>

As I penned last month's column about distributed denial of service attacks, I realized that I had been ignoring something. Something that was not sexy, or exciting, but still vitally important to network security.

Patches. The automated attacks that installed the agents for the distributed denial of service attacks relied on old holes and the intrusions could have been prevented by installing patches.

An article that appeared in the November 11, 1999 Detroit News mentions that the Chinese launched an assault on US government Web sites after the accidental bombing of the Chinese embassy in Belgrade. The attacks revealed that there were three to four thousand backdoors already installed in US computer systems. One distributed denial of service attack, against the University of Minnesota, used two thousand different systems as agents. In both of these cases, we know how many systems were compromised because the backdoors or agents were uncovered as a result of the attacks. The question remains, how many other thousands of compromised systems are out there?

This is not just a problem for UNIX and Linux systems. A couple of weaknesses in sample IIS files led to the defacing of hundreds of NT web servers last year. And for every backdoor package written for UNIX systems, there are ten for NT and Windows systems.

There are two solutions to these problems. First, use antivirus software to scan existing Microsoft systems for previously installed Trojans, and look for signs of rootkits or backdoors on UNIX systems. Second, and equally important, install those security patches. You are much better off not being broken into because of some easy to fix problem than being the source of an attack on some third party because YOU were vulnerable.

Trouble

In 1998, an attack using an old vulnerability to become root on a target system made the international news. Two boys, using the login names MAK and STIMPY, broke into a Department of Defense site using the statd vulnerability in a Sun Solaris system. The boys were being guided by Analyzer, allegedly an Israeli. A patch for the security hole that the boys used had been on Sun's patch site for over two years before the attack.

Last Spring, someone running an automated tool used the Winnuke denial of service attack to crash over 6,000 unpatched Windows systems. There had been patches posted at Microsoft's Web sites for this problem for over a year.

Vendors put out dozens of security patches each year. Last year, Microsoft published 61 security bulletins, many focused on component applications, such as Internet Explorer and IIS (See Sidebar). Sun
Microsystems published eleven and Hewlett-Packard 16. RedHat, a distributer (and support organization) for Linux, published eight security fixes for its version 6.1, which was released in late summer.

The point behind looking at these numbers is not to judge a vendor or its product as good or bad based on the number of security advisories in a given period. It is much better that the vendors publish advisories instead of pretending not to have security vulnerabilities in their software. All software has bugs, and a more realistic correlation is between the lines of code a vendor supports and the number of bugs discovered.

Popularity does come into play. Earlier versions of Microsoft's Windows NT rarely had security problems reported. But that was back before NT was widely used, or well enough understood by the hacker community. That is no longer true today.

The real trouble is inertia. If you have a Web server that is working just fine, do you want to pause it to install a security patch? Or suppose you have 1200 workstations, all with slightly different hardware configurations, sitting on desktops throughout your organization. How willing will you be to install a security patch on those 1200 systems?

And inertia has a very good basis. Suppose you do install the patch, and then the Web server becomes unstable the next day? Or, a minority of your workstations blue screen, or an important application quits working? These are real fears, and can happen when patches are installed.

Problems of Scale

And then there is that other pesky problem--how to go about actually installing a patch on those 1200 workstations. In the world of UNIX, many research projects have focused on this very topic, and several excellent approaches have been developed. These are free, although will take intelligent and skilled individuals to set them up correctly.

One traditional way to distribute software across a network of UNIX systems was to use rdist. Rdist, short for remote distribution, follows instructions in a file called the distfile, and can detect if a file needs updating, upload the file, and adjust permissions and ownership on the target file.

The downside to rdist has been the use of the 'r' command, rsh (remote shell), to start rdist on the client system. The 'r' commands in general have very weak security, and this use of rsh requires establishing a trust relationship between the server and the clients--the type of trust relationship often abused by attackers.

There are several ways around this conundrum. One is to use Kerberized rsh on systems that support it. Kerberos is a single sign-on system that uses encrypted credentials for authentication. Another potential way around the 'r' commands problem is to replace the 'r' commands with SSH (Secure Shell). The Secure Shell uses RSA authentication, and comes in both commercial and open source versions.

There is also a replacement for rdist, an open source program named rsync. Rsync also uses the 'r' commands (or their replacements), but is somewhat simpler to use as it matches directories hierarchies instead of using a distfile. Rsync can also perform MD5 checksums to assure that the client's version of the file has not been tampered with in a subtle fashion, as done by many rootkits that install trojan commands on UNIX systems.

Besides these tools, there are more complete solutions, some that will pull instead of pushing updates . Rdist and rsync "push" new distributions to the clients, while some approaches allow clients to poll the servers, requesting or "pulling" in the needed patches. Published tools include lcfg, GeNUAdmin, OMNICONF, Config, and synctree, and papers about these tools can be found using the search engine at the USENIX Web site.

Microsoft has developed SMS, System Management Service, to perform a similar, or though not exactly the same, task as the UNIX toolkits. There are also commercial products designed to tackle this problem, that is, the secure remote installation of software patches and upgrades.

And there is also the simple approach, suitable for small to medium sites. Download the software patches yourself, post them to an internal Web server, and send email to notify your users that there is a new software patch that they need to install. You can provide instructions, as well as motivation, along with the patches, and check your Web server's logs to see which systems have downloaded the patches. But with this system, you cannot insure that the patches have been installed properly, or even at all.

One of the biggest potential pitfalls of using any of these techniques is that the server itself, if vulnerable, can be used to install backdoors or other exploits on all of its clients. If an attacker gains access to your central repository for software patches, the game is over. Not only can they install the software of their choice on all your systems, but the server itself will often be trusted by the client systems, and this trust by itself can be exploited simply to break in. In other words, any server you use for storing patches or upgrades must be carefully protected and monitored.

Don't let your systems be safe havens for attackers. Patch your systems. You will have less work to do in the long run.

Sidebar: The Danger of Active Scripting

Back in September, Network Magazine published an article that I had written in July 1999 about the wonders of MIME and Active Scripting. About the same time that column was published, Richard Smith, Owner of Phar Lap Software, made a presentation during the USENIX security symposium about how Active Scripting could be used to launch ActiveX components on some Windows installations, and do anything that the person sending you the email had in mind. Smith, in case you did not recognize the name, was one of the people who identified the author of the Melissa virus by reverse engineering the virus.

One of Smith's sources, Georgi Guninski of Bulgaria, began posting other flaws in Internet Explorer and Outlook in September, and has posted almost one a week since then. This has contributed to the large number of Microsoft security advisories, as Microsoft dutifully puts out patches.

CERT/CC raised the ante on February 2, 2000, by putting out an advisory about other problems with Active Scripting. The advisory has warnings for Web server script writers as well as users, but their main point was this: disable Active Scripting. In the advisory, CERT personnel claim that although these exploits are not in wide use, you should do this anyway. They provide examples of how a Web site could include scripts in links, so that when you click through this link the scripting would be included in the URL. This scripting could then be resent from a trusted site and executed on your system. In other words, simply by clicking on the wrong link, you could be giving an attacker access to your own system or internal network because you trust a particular Web site (which could even be internal).

I had disabled Java and Java Script in my versions of Netscape Navigator years ago, and suggest that you do likewise. Disabling Active Scripting will occasionally cause you problems, especially on Microsoft sites, as they use it liberally. What would truly be nice would be if the browser vendors would make it possible to enable/disable scripting with a button right on the menu bar, so you don't have to go digging through layers of menus. This button should always be visible, and show when Active Scripting is enabled by flashing red (or something equally unmissable). The browser should be configurable so that Active Scripting can automatically be disabled as soon as you leave the site where you had it enabled (so that you can forget, and the browser will turn off Active Scripting for you).

Active Scripting is a useful extension to the Web, but it is also a very dangerous one. You give control of your system to remote Web sites and to anyone who sends you email if you use Outlook, Outlook Express, or Netscape Navigator to read you email.

[end sidebar]

Resources:

Microsoft's Web page listing recent patches: http://www.microsoft.com/technet/security/current.asp

Sun support site with list of security bulletins and links to free patches: http://sunsolve.Sun.COM/pub-cgi/secBulletin.pl

The Hewlett-Packard sites that provide access to security patches require a login, but is free: US, Canada, Asia-Pacific, & Latin-America http://us-support.external.hp.com; Europe, http://europe-support.external.hp.com

URL for seraching USENIX site for configuration tools: http://www.usenix.org/Excite/AT-usenixquery.html

Fftp site for rsync is ftp://samba.anu.edu.au/pub/rsync, with a mirror in Europe: ftp://sunsite.auc.dk/pub/unix/rsync.

SSH commercial site: http://www.datafellows.com/, and the Open Source version at: http://www.openssh.com/

Information on turning off Active Scripting: http://www.cert.org/tech_tips/malicious_code_FAQ.html

CERT advisory about several dangers involving Active Scripting and Web FORMS: http://www.cert.org/advisories/CA-2000-02.html