Stranger in a Strange Land
by Richard Power and Rik Farrow

Grown-up enterprises take information security seriously. Of course,
few people-- whether IS professionals or board of directors
members--would admit to
 not taking it seriously. But there are certain rules of thumb,
there is a litmus test. For example, grown-up enterprises dedicate
from three to five percent of the total IS budget to information
security. They also have dedicated information
 security staffs with at the very least one full-time staff member
for every thousand users. In grown-up companies, information
security doesn't report within the IS department--typically, there
is a director of information security who
 reports directly to the CIO to ensure that the message gets
delivered.

Furthermore, recent CSI studies indicate that the numbers aren't
static. CSI's "Information Security Staffing Survey," conducted
with Charles Cresson Wood of Baseline Software (Sausalito, CA)
indicated that budgets for infosec st aff were going to rise by
17.8% the end of 1997. Fifty-eight percent of responde nts to the
"CSI Information Market Survey" reported that their infosec bud
gets were going to increase in 1998.

How would your enterprise fare if held to such a litmus test? Don't
feel too bad. Most other organizations wouldn't pass it either.
But if you were to ask the IS director or the CEO, you would be
assured that XYZ, Inc. takes s ecurity "very seriously." The sad
reality is that, in most organizations, security receives lip
service--at best.

So what happens when a security-savvy sys admin gets hired into a
corporate culture where security concerns are barely given lip-service?
We're going to tell you. But we can't reveal our source. You've
probably heard the old
 saw about intelligence work, "If I told you what I know, I'd have
to kill you." Well, in regard to network security, it's more like,
"If I told you what I know, I'd lose my job." So let's just call
him "Anonymous." The following vignettes from Anonymous' experiences
at a large, far-flung corporation provide some invaluable insights
into what's wrong with network security.

Maybe a blessing, maybe a curse

In many shops, you meet opposition from others who exercise
quasi-owner ship over certain systems. They don't want you to take
over. It's really tough to assess network vulnerability when people
on your own team overtly try to keep you away from "their" systems.
But in Anonymous' case nobody cared--at least in the beginning.
The indifference he encountered was both a blessing and a curse.
"The blessing was that everyone was saying, 'These are your systems,
find out how they're running and fix anything that's broken.' The
curse was that nobody really cared about the network's fundamental
security problems." Undaunted, he rolled up his sleeves and went
to work.

Searching for the source of the Nile

Anonymous knew that network security is predicated on asking tough
questions.  For example, where does your network end and the Internet
begin? Do you even know how large your network really is? Do you
know which versions of what operating systems are running on all
your systems? If you don't know, how can you tell if you've
implemented the various security patches that they probably need?
Do you know how many modems are on users desktops? Could you even
guess at how many digits the actual number would contain? If you've
purchased a firewall but haven't attempted to [answer] any these
questions, you have thrown your money away.  How can you secure a
perimeter if you don't know where it is? 

Anonymous spent his first few days crawling around on the floor, following
wires, drawing maps on his yellow pad, detailing the physical
location and contents of the systems on the network. He needed to
see how many physical disk arrays were attached to each unit.
After his map-making and inventory-taking was done, he logged on
each system and asked more questions. What do you think you have
attached to you? How much memory do you have? How many disks? How many
processors? He ran reports to see what the system thinks it has
logically and how it is using resources.  

"You can't possibly protect systems if you don't know what they
look like.  What's the system load average when they're operating
normally? If you don't know something as simple as that you have
no way of knowing if someone has started running some rogue process.
You need to be able to benchmark your systems. You need to know
the hardware, operating system version, and applications. What does
this system look like? What does it do for a living? What is it's
personality on a normal day? Are they running okay? Do they do
their jobs okay? That's the first thing to look at."

What did Anonymous find on his elbow-to-the-carpet expedition?
Fifteen large servers with many internal users as well as number
of Web sites with a fairly limited number of internal users but
thousands of users over the Internet.  Confidently, Anonymous
moved on to the next big question: "Is anyone breaking into these
systems? Are they safe? Who are the users with access to this
system and what level of access do they have?"

Falling through the cracks

Anonymous was amazed to find how many users he simply could not
identify. There were no password and user ID policies or procedures
in place. There were no naming conventions. You wouldn't uniformly
have first initial and last name.  Instead, you would have user
IDs like "jenny" or "jack." There was no notation that would say
for example, "On Jan. 1, 1997, I entered Bob Jones under user ID
'bjones,' he is a consulting working for the firm of XYZ LLP and
this account should expire six months from today." Thirty to forty
percent of the users in the domain that Anonymous inherited could
not be identified as legitimate users who worked for the company--
not through the company phone list, nor through the etc/password
file's notation field?

Anonymous' gut instinct as a security administrator told him to
delete any accounts that he could not verify as actual authorized
users. He couldn't tell who was logging on under these user names.
But there was tremendous internal pressure not to mess with them.
His colleagues were saying, in effect, "Yes, we have a house that
is in disarray, but it's working and if you start cleaning this
mess up by disabling accounts you may make something not work by
doing that and the price of making something not work--for example,
preventing a Web site from being updated--is more severe than the
security risk of not knowing who this person is poses to the
company." "That struck me as insane," A nonymous confides. "It took
me quite awhile to drum up the support to get rid of those we
couldn't verify."

As all worthy sys admins and security administrators should,
Anonymous had UNIX- based password crackers, NT-based password
crackers as well as a run-of the-mill DOS password crackers in his
tool kit. He had big dictionaries and word lists that included a
variety of items--for example, popular movie star name s, foul
language and common number sequences. When he ran one of them, he
was shocked at the sheer number of passwords it turned over--many
within one or two minutes. So, in addition to not knowing who the
users were, there was no password aging and no password checking.
Indeed, Anonymous' predecessor had accepted user passwords that
were clearly bad--for example, he had sent e-mail saying, "I'm
entering James smith, user ID 'jimmy,' password John' (for his
son). Anonymous was appalled. "You can't reasonably expect the
average user to be educated on the relative strengths and weaknesses
of passwords, but you certainly don't expect IS to endorse such
naivete."

Shell games

There was a user within the IS group who felt that he should have
root access to one of the primary servers--in his previous job he
had done some sys admin work.  He was told that it wasn't appropriate
for him to have root access, because it he didn't need it to do
his job.  It was an ongoing battle. Anonymous made the business
case that the user did not need root access, and that should have
been the end of it. But while performing routine checks, Anonymous
discovered that there were entries in root' s shell history file
that he had not made. They were alarming not only because had he
not entered them, but because the commands themselves were copies
of Korn shells or other shells to other places on the system under
other names.

Somebody had logged in as root and walked away from a terminal,
the user desirous of root swooped down and set the uid bit to run
[Richard:  The SUID or set-user-id bit. This is an obvious sign
of an attack or intrusion on UNIX systems, sometimes done by
legitimate sysadmins to provide themselves with a backdoor in 
case the root password is lost.  Never a good idea, though.  Also,
it is very difficult to discover who has done this (root is an
anonymous user).  RIK]
as root and tucked it away somewhere else under another users name
so that he could log into the system as just an average joe user,
then run the shell that effectively granted him root privileges.

Anonymous went to his boss and said, "We've had an internal security
breach." Of course, the IS Director, trying to look at both sides
of the issue aske d, "Was it possible that this was an accident?"
Anonymous explained that it had been done maliciously. And he had
the keystroke record to prove it. There was no reason for someone
to do this, Anonymous insisted, unless he wanted to create a backdoor
so that he could get root access any time he wanted it. How did
this sordid little affair get resolved?

"Because we didn't have any policy in place, and had never expressly
told this person or anyone else in IS that it was against the rules
to try to break into super user access on any of our systems, there
really wasn't much that could be done to him besides giving him a
stern warning. He was told that he had
 hacked the system, that if he did it again he would be fired and
that a formal letter was going into his personnel file."

To the best of my recollection, I don't recall

One bright sunny morning, there was a routine, external audit by
one of the big six accounting firms. Anonymous had been unofficially
told give curt "yes" or "no" answers and offer as little information
as possible. But Anonymous stood his ground.  "I was very forthcoming
with the auditor, and answered all the questions honestly. When
the report came back, it pointed to a lack of security."

Those who participated were asked how the audit turned out so badly?
Anonymous simply said he didn't feel comfortable lying, and that
it audit was worse than previous one, either the people conducting
the previous audits did not ask right questions or the people
answering the questions had lied Several people in the IS group
stood up and said "The results of this year's audit are correct,
we do have these problems," the audit was contested.

Between the Alpha and the Omega

If you are a diligent sys admin concerned about security but work
in a somewhat hostile environment, Anonymous recommends that you
make the Alpha pager your best friend. Rather than spending sleepless
nights and weekends lying in wait for accidental mishaps or
interlopers causing mischief, simply program your Alpha pager to
start squawking when something unseemly occurs.

Have there been a lot of failed attempts to su to the root account?
Is somebody trying to log in on the console when only you are
authorized to? Have any file systems changed dramatically in size?
For example, has one of your file servers just lost 300 megabytes
within the last hour? Are there suddenly new accounts in your
etc/password or etc/group files that you did not add? If you get
paged while your out at dinner on a Saturday night and the message
reads "Warning!  Your etc/password file has changed," you've probably
got a problem. Just set-up some shell scripts that test for certain
conditions and when those conditions are met, you'll get an e-mail,
saying "Warning! The /xxx file syste m on system x has just filled
or is at 95%" or "Warning! Web directories now have
permissions =917-7-7' for read/write/execute all the way across."

You can program your page to alert you if you there is suddenly a zero 
ID file
or more than one zero ID file (which makes it in effect the root user).
[Richard: These would be "set-user-id zero programs", and many already
exist on UNIX systems.  You could have a program which detected new
SUID zero programs and get paged.  On paging, this is not always
a good idea, as getting paged by false alarms at 4am, or during your
Saturday evening date, quickly teaches you to ignore your many
pages.  A carefully tuned paging system is essential; still, most
people disable this, as it requires better AI than currently exists.
Still, this person obviously is intensely motivated.  RIK]

You can also test whether or not there are suddenly new set uid
root files (files than run effectively with root's permission while
they're executing) that have appeared without your authorization.
[Richard:  Same thing as above.  As for the script, the standard method
is to use find, compare the output to a previous result, and if
any new files show up, report those files.  Dead give aways are
files with names like ksh, or files the same size as ksh or other shells.
Some new SUID files will appear during software installation. RIK]
These are all really easy things to test for. If through no other
way than simply using the find command, you say find these permissions
that would denote a file that is set uid root, you would know that
on a average day on your systems there should be about 70 of those,
and suddenly there are 72. You need to log on to your system and
find out what's going on.

Conclusion

Anonymous' Waterloo came within one year of his hiring. He wrote
a comprehensive policy on backup procedures and change control.
Hardly a paranoid or Draconian issue. If you can maintain orderly
backups and get control of operation systems version, etc., you
might has well hang it up. Well, the folks who didn't care suddenly
cared--not about security, but about who was responsible for what.
When his policy proposals got mired in inter-departmental debates,
Anonymous resigned.

Is there really any hope of pushing the security agenda in such an
immature environment? It won't be easy. If you find yourself in a
similar position, look for allies-- not within IS, but within other
departments, e.g., audit or corporate security or legal. If you
find anyone else who gives a damn, get together look for to someone
high up on the food chain and ask them to be your champion. Choose
someone with a golden parachute or a strong box full of stock shares
to lose, e.g. the CFO. Such a person will want you to not only give
them an assessment of the current security posture (or lack of
one), they will also demand that you quantify the risk. You'll also
want to have a modest list of proposals that would easily and
economically improve security immediately. It's your best shot.

Anonymous had good, initial proposals (i.e., change control and backup
procedures), but he lacked allies or a champion.
Good luck.