blank.gif (43 bytes)

Church Of The
Swimming Elephant


The Sky is Falling!
Cotse OpEd
By John Holstein



Due to this update to the security advisory, I would like to add a few notes to the previously written article you see below. Please read the modified advisory before proceeding.

The principle surrounding the way in which this vulnerability was presented is the same, should they have remained quiet? Yes. Should they have told us that this exploit was noted "internally" only and not in the wild? Yes. Did they? A hint was made, but they did not specifically say they had a full grasp on the situation.

While I don't agree 100% with the way they did things, I'll give them credit for handling it _almost_ appropriately by not releasing too much information..... Supplemental information should have came with it... A typical case of trying to be "too secure" and breaking the application, or in this case, raising the tempers of many users/admins for not giving a bit more information.

I think if they'd simply told us "...we found a bug. To the best of our knowledge we're the only ones that know anything about it. If we release more information, we'll get you into more trouble than if we don't. We'll have a fix in X number of days, until such time, we're gonna keep a lid on this, do "THIS" to fix an issue that will be patched in X days."

I believe this falls back to the same-ole-problem of finding a "happy medium". No matter if it's IT Security or Installing Widgets for ACME Ltd., finding the "happy medium" is always a problem. While we are looking toward the future for full disclosure, we should also note that in circumstances such as this, we may in fact be better off with "Almost-Full" Disclosure.

-- John Holstein, Cotse


--- Begin Original Article ---

Let me first point out that I am not trying to voice my opinion concerning "full disclosure" or the manner in which it is presented. I am merely stating what, in my honest opinion, is blatantly obvious concerning the recent OpenSSH exploit as disclosed by Theo de Raadt, *the* programmer for the OpenSSH project. To quote Mr. de Raadt, I would like to present to you the message I received:

There is an up coming OpenSSH vulnerability that we're working on with ISS. Details will be published early next week.

However, I can say that when OpenSSH's sshd(8) is running with priv seperation, the bug cannot be exploited.

OpenSSH 3.3p was released a few days ago, with various improvements but in particular, it significantly improves the Linux and Solaris support for priv sep. However, it is not yet perfect. Compression is disabled on some systems, and the many varieties of PAM are causing major headaches.

However, everyone should update to OpenSSH 3.3 immediately, and enable priv seperation in their ssh daemons, by setting this in your /etc/ssh/sshd_config file:

UsePrivilegeSeparation yes

Depending on what your system is, privsep may break some ssh functionality. However, with privsep turned on, you are immune from at least one remote hole. Understand?

3.3 does not contain a fix for this up coming bug.

If priv seperation does not work on your operating system, you need to work with your vendor so that we get patches to make it work on your system. Our developers are swamped enough without trying to support the myriad of PAM and other issues which exist in various systems. You must call on your vendors to help us.

Basically, OpenSSH sshd(8) is something like 27000 lines of code. A lot of that runs as root. But when UsePrivilegeSeparation is enabled, the daemon splits into two parts. A part containing about 2500 lines of code remains as root, and the rest of the code is shoved into a chroot-jail without any privs. This makes the daemon less vulnerable to attack.

We've been trying to warn vendors about 3.3 and the need for privsep, but they really have not heeded our call for assistance. They have basically ignored us. Some, like Alan Cox, even went further stating that privsep was not being worked on because "Nobody provided any info which proves the problem, and many people dont trust you theo" and suggested I "might be feeding everyone a trojan" (I think I'll publish that letter -- it is just so funny). HP's representative was downright rude, but that is OK because Compaq is retiring him. Except for Solar Designer, I think none of them has helped the OpenSSH portable developers make privsep work better on their systems. Apparently Solar Designer is the only person who understands the need for this stuff.

So, if vendors would JUMP and get it working better, and send us patches IMMEDIATELY, we can perhaps make a 3.3.1p release on Friday which supports these systems better. So send patches by Thursday night please. Then on Tuesday or Wednesday the complete bug report with patches (and exploits soon after I am sure) will hit BUGTRAQ.

Let me repeat: even if the bug exists in a privsep'd sshd, it is not exploitable. Clearly we cannot yet publish what the bug is, or provide anyone with the real patch, but we can try to get maximum deployement of privsep, and therefore make it hurt less when the problem is published.

So please push your vendor to get us maximally working privsep patches as soon as possible!

We've given most vendors since Friday last week until Thursday to get privsep working well for you so that when the announcement comes out next week their customers are immunized. That is nearly a full week (but they have already wasted a weekend and a Monday). Really I think this is the best we can hope to do (this thing will eventually leak, at which point the details will be published).

Customers can judge their vendors by how they respond to this issue.

OpenBSD and NetBSD users should also update to OpenSSH 3.3 right away. On OpenBSD privsep works flawlessly, and I have reports that is also true on NetBSD. All other systems appear to have minor or major weaknesses when this code is running.

(securityfocus postmaster; please post this through immediately, since i have bcc'd over 30 other places..)

Ok, what we know so far:

1) There's an exploit, not yet "in the wild", for various versions of OpenSSH.

2) We do not know what the _exact_ exploit is, although "hints" have been made.

3) We know there's no specific patch available, as of yet, however, a make-shift "fix" is possible.

4) We are currently expecting a new version of OpenSSH to be released within the next few days, yet this newest release will still be vulnerable, whereas (I have learned this since the initial message), the -stable version, not the -release version, will not be vulnerable.

5) When OpenSSH's sshd(8) is running with priv separation, the bug cannot be exploited.

6) OpenSSH developers are trying to get everyone else to patch their operating systems in such ways as to make "everything kosher" with OpenSSH: "...If priv seperation does not work on your operating system, you need to work with your vendor so that we get patches to make it work on your system. Our developers are swamped enough without trying to support the myriad of PAM and other issues which exist in various systems. You must call on your vendors to help us."

Folks, I don't know about you, but i have sampled a lot of the posts made to several newsgroups/mailing lists concerning this report. Although I believe that disclosing vulnerabilities in a "more timely manner" is needed to develop proper patches and to force vendors to modify code more quickly, I can't see that reporting vulnerabilities in this manner is the best route to take.

What we have here is a basic "cry wolf", "do as I say, not as I do". What I see is, a vulnerability in OpenSSH that is correctable, yet their disclosure comes before a fix is in -release mode. Therefore, to properly fix the problem, one must trust "un-trusted" software by installing the -stable branch OR applying a make-shift fix for the vulnerability.

I am all for trying to patch a problem BEFORE it becomes a problem. That's not the point. The point is releasing information without giving the end-user or administrators the opportunity to look at it logically and possibly come up with another work-around for the vulnerability.

By releasing the information in the manner above, the administrator only knows the very basic problem. By releasing the information after a fix is in place, a patch is available or more information is given, the administrator would then have the ability to come up with a fix themselves, rather than leaving them hanging with an "I said it would, just do it" type situation.

The basic problem boils down to this: Open source software allows coders around the world to see the code and possibly fix the problem, among other things. Most open-source-conscientious people install open source because they like to have more control over their software. When you release information that is restricted, constrained and non-specific, the open source user/administrator doesn't have the chance to look at the information and define a proper patch for the situation. Or at the very least, decide if completely shutting down a daemon is the correct path to take.

There are administrators with 60, 80, 100 machines in a server farm. What happens when you announce "The Sky is Falling" and these administrators do what "they think" is correct, based on improper information, only a week later, they must "do something else" because the previous information was incorrect and the person releasing the information was "crying wolf"?

In this type of situation, you have people thinking:

"...I'll be rethinking my use of OpenSSH for the very same reason. You're not my dad, my cop, my priest, my lawyer or firefighter. NOR are you the Unix version of 'install wizard'. I expect code from you. That's it. Write code."

-- Petr Swedock

I am not the only one that feels this way, to justify the writing of this article, let's take a look at some other "more educated" opinions, feel free to look up these articles at a usenet server nearest you.:

Date: 27 Jun 2002 06:27:15 +0000
Subject: Meta (was Re: Wow)

In response to the thread initiated by Theo:

From: (Theo de Raadt)
Subject: Wow
Date: Wed, 26 Jun 2002 17:39:09 +0000 (UTC)
Message-ID: afcu7t$2nrv$>

Feel free to dig around a bit and see the entire fiasco for yourself. This is, in my honest opinion, a case specifically reflecting the negative side of "full disclosure, in a timely manner." Folks, in order to get vulnerabilities disclosed quickly, this is EXACTLY what we shouldn't do.


Comments, Questions, Bugs?


Protect yourself from cyberstalkers, identity thieves, and those who would snoop on you.
Stop spam from invading your inbox without losing the mail you want. We give you more control over your e-mail than any other service.
Block popups, ads, and malicious scripts while you surf the net through our anonymous proxies.
Participate in Usenet, host your web files, easily send anonymous messages, and more, much more.
All private, all encrypted, all secure, all in an easy to use service, and all for only $5.95 a month!

Service Details

Have you gone to church today?
All pages ©1999, 2000, 2001, 2002, 2003 Church of the Swimming Elephant unless otherwise stated
Church of the Swimming Elephant©1999, 2000, 2001, 2002, 2003 is a wholly owned subsidiary of Packetderm, LLC.

Packetderm, LLC
210 Park Ave #308
Worcester, MA 01609