Network Defense by Rik Farrow A little preparation goes a long way Although it has probably been years since you practiced a fire drill, there is another type of drill that you should consider. While even without practice, you and your coworkers could find the fire escapes, what would you do if you had an actual computer break-in? Panic? Pull the plug? Most people have not yet had to face the reality of computer crime, so they will not be prepared to handle an incident. The truth is that it makes sense to have a plan and to actually practice what you will do, so that when the time comes you won't have a chinese fire drill instead. The place to start is by modifying your organization's security policy to support the activities of a incident handling group, and then to dissemintate this information so that everyone knows what his or her responsibilities are, when to contact someone and who to contact. And, in the event that you suspect that a crime has occurred, how to go about contacting the appropriate agency that will investigate the incident. Plan Ahead Any changes in the way you handle your security should start with the support of your management. With your management behind you, you will be free to take whatever actions are necessary to handle an incident, whether it is from internal or external sources. This is where your policy comes into play, by having a written document that formalizes incident handling. Keep in mind that if you have never handled an incident before, then the policy you initially create will most likely need to be changed after you have the experience of handling a real incident. With a incident handling policy in hand, you then need to share this information with everyone in your organization. They need to know how to recognize a security incident, who they should call, and what they should do. Your own definition of an incident will vary from that of other organizations, as some organizations consider harassment by computer an incident. For the purposes of this article, we will focus on those incidents that involve breaking into or abusing computer or network resources. Next to recognising an incident, the most important information that everyone needs to know is what to do next. Unless the incident is currently in progress and causing significant damage, the person who discovers the incident should _leave things along and contact the Incident Handling Team__ (IHT). To make this possible, provide one or more telephone numbers that will surely reach a member of the team. For organizations with help desks, the help desk itself will be the point of contact. The help desk then contacts the IHT. Using the help desk has the side effect of generating an initial log of when the incident occurred as well generating an internal case number that can be assigned to the incident. What you don't want to have happen is for someone to try to clean up after the incident. The traces left behind after a breakin are often subtle, and doing anything might erase this evidence. Just imagine how much trouble someone would cause by cleaning up a crime scene before calling the police: carefully wiping all surfaces, replacing any broken glass or forced locks, cleaning the rugs, and then sending any witnesses on an ocean voyage. You don't want to do this, so let your people know that hand's off is the best advice. Sherlock Holmes The IHT itself should be composed of members with different areas of expertise. If your shop includes Solaris, Digital UNIX, MVS, as well as Windows systems, your team must also include people familiar with those operating systems. You won't necessarily need all of the team for every incident, but knowing where to find the expertise is critical. In handling a particular incident, one or more IHT members familiar with the system involved should do the initial investigation. They need to determine if this really is an incident at all. Many times, the "incident" will turn out to be a system malfunction or user error that mimics an incident. My personal "favorite" is a changed root password that occurred because one owner of root's password changed it without informing other owners. The investigating team must also be careful not to destroy evidence. If it is clear from the start that an incident has occurred, the first step is to create an image backup of the affected hard drives if possible. An image backup includes _everything_ on the hard drive, including deleted files, page files, file metadata, in other words, a sector-by- sector copy. UNIX system include the dd (dump device) command that can copy raw hard drives to tape or other hard drives. There are commercial products available for other operating systems, such as Windows NT, that can be used to make image backups. You want this because the image backup contains information that is not accessible from file systems, such as deleted files. Of course, if the affected system is a server with gigabytes of information, you might want to delay the backup until you have focused on areas that you will want to backup. If it is not clear, and you don't have the time to perform the backup first, carefully look for information supporting the assertion that a security incident has occurred. Look at logs, network traces, change dates on files and directories, changes to startup files or the registry, new users (and new group members on NT systems). Check for trojans (virus scanning software does a good job of this for Windows systems) as well as checking the Run keys in the registry. Whatever the team does, they must keep good records. The teams' actions could later be interpreted as actions of the intruder if not properly logged. With the initial analyses complete, the team may need to take further action to secure the system. This might be as simple as physically disconnecting the system from the network. In other cases, disabling the network service that provided the means for the attack might be all that is necessary. Keep in mind that it is common for attacks to leave trojan horses or rootkits behind that may provide for additional means of network access that do not show up as part of standard configuration. You might try portscanning the system looking for listening services that are not reported by programs like _netstat_. If you have discovered that the breakin or abuse occurred because of buggy service that has not yet been patched, apply the patches. Of course, this may imply that you have other systems without the patches, and that they may also have been compromised. This is where having a good intrusion detection system that monitors and logs network traffic becomes invaluable, as it can aid in discovering other potentially affected systems. The initial analyses may not have conclusively pinpointed the cause of the attack. Once you have completed the initial analysis and backed up the hard drive, you now have time to review what has been done and decide what other steps need to be taken. You have also reached the point where you need to decide whether or not to involve law enforcement. The Law I am not a lawyer, and (most likely) neither are you. Plus, laws involving computer crime vary from country to country. But there are two general principles involved in deciding when to contact law enforcement. The first is your own company policy about bringing in outsiders when an incident has occurred. Regardless of your internal company policy, consider that many countries have laws about not reporting crimes, especially when the crime endangered life or wellbeing or caused considerable damage. The second, but equally important determinant is the amount of damages caused by the incident. You must quantify the damage as this is used (at least in the US and Canada) as a means for determine whether resources should be spent investigating your incident. When tabulating your damages, you should include the time spent by the IHT members, time spent restoring systems to their previous state, the cost of downtime, loss of revenue, any lost time by users, as well as the potential damage to reputation if the incident became known. If intellectual property may have been stolen, the estimated value of that property. According to the FBI, the threshold before they will generally consider beginning an investigation is $5000. This amount does vary from district to district, but you must be able to provide an estimated cost for the incident. Remember that law enforcement agencies are already very busy, and you don't want to waste time with trivial incidents. You should also be prepared to provide a succinct summary of the incident written in terms that they can understand. You need to spell out in detail what was done, how you determined this was done, as well as how you came up with your financial damage assessment. After the summary, you can include a more detailed summary of the technical steps that you took to collect the evidence, along with a timeline for all activities. You cannot expect any outside party to be able to analyse and understand your systems as well as you can, so be prepared to provide your analyses as well as evidence to back this up (your hard drive images and printouts of appropriate log entries). Your participation with law enforcement will not stop there if they decide to investigate. They will likely have further questions or information gathering assignments for you, or aid in preparing a search warrant for collecting evidence. You can help with this process by designating a single point of contact, someone with authority, familiar with the incident, and usually available on-site. In the US, the first point of contact in most cases of computer crime is the FBI (see Resources for the Web site listing Field Offices). In Canada, it is the RCMP. In most countries, you will need to contact a national agency, as computer crime is rarely a local event. But not always. For example, an insider who has possibly committed fraud would be handled by local authorities. But any incident involving the Internet also will involve national authorities, such as the FBI. Keystone Cops I suggest that you prepare for the inevitable by creating policy and an IHT. You don't really want to "handle" an incident with panicky actions that destroy potential evidence, while amusing your attackers. You can also prepare by locating the local and national agencies that would handle criminal investigation of an incident. And you can try your own "incident drills" by staging an incident yourself. Hopefully, what you will see is a well ordered process, and not another late night run of the Keystone Cops. Sidebar: Preparing to Handle a Computer Security Incident Create policy that supports appropriate Incident Handling (IH) Notify all employees/users of policy for IH Create an IH team Practice handling incidents When an incident occurs: 1. Begin taking notes about everything that is done, who is talked to, when any event happens 2. Stabilize the incident (disconnect from network if necessary) 3. Backup the hard drive (preserve potential evidence) 4. Analyse the incident to more thoroughly understand what happened 5. Take additional steps if needed to prevent a reoccurance 6. Decide if contacting legal authorities is appropriate 7. If so, created a detailed summary of the incident, including estimated financial loss 8. Be prepared to provide your own analysis of the incident and to participate in any investigation 9. Review how the incident was handled, and adjust policy and/or team activities as necessary Resources: The FBI Field Office Locator: http://www.fbi.gov/contact/fo/fo.htm The National Infrastructure Protection Center, includes links to the US laws regarding computer crime, other useful links. http://www.nipc.gov/ Forum of Incident Response and Security Teams, a membership organization dedicated to incident handling and prevention. http://www.first.org/ Forming an Incident Response Team, Australian CERT, ftp://ftp.auscert.org.au/information/papers.html Recovering from incidents, ftp://info.cert.org/pub/tech_tips/, intruder_detection_checklist and root_compromise An excellent research paper on how to quantify the costs of incidents, including sample spreadsheets, available for a small fee ($23) by emailing a request to skwillia@cic.uiuc.edu.