I and just about
every designer of Common Lisp and CLOS has had extreme exposure to the
MIT/Stanford style of design. The essence of this style can be captured by the
phrase ``the right thing.'' To such a designer it is important to get all of
the following characteristics right:
·
Simplicity-the
design must be simple, both in implementation and interface. It is more
important for the interface to be simple than the implementation.
·
Correctness-the
design must be correct in all observable aspects. Incorrectness is simply not
allowed.
·
Consistency-the
design must not be inconsistent. A design is allowed to be slightly less simple
and less complete to avoid inconsistency. Consistency is as important as
correctness.
·
Completeness-the
design must cover as many important situations as is practical. All reasonably
expected cases must be covered. Simplicity is not allowed to overly reduce
completeness.
I believe most
people would agree that these are good characteristics. I will call the use of
this philosophy of design the ``MIT approach.'' Common Lisp (with CLOS) and
Scheme represent the MIT approach to design and implementation.
The worse-is-better
philosophy is only slightly different:
·
Simplicity-the
design must be simple, both in implementation and interface. It is more
important for the implementation to be simple than the interface. Simplicity is
the most important consideration in a design.
·
Correctness-the
design must be correct in all observable aspects. It is slightly better to be
simple than correct.
·
Consistency-the
design must not be overly inconsistent. Consistency can be sacrificed for
simplicity in some cases, but it is better to drop those parts of the design
that deal with less common circumstances than to introduce either
implementational complexity or inconsistency.
·
Completeness-the
design must cover as many important situations as is practical. All reasonably
expected cases should be covered. Completeness can be sacrificed in favor of
any other quality. In fact, completeness must sacrificed whenever
implementation simplicity is jeopardized. Consistency can be sacrificed to
achieve completeness if simplicity is retained; especially worthless is
consistency of interface.
Early Unix and C are
examples of the use of this school of design, and I will call the use of this
design strategy the ``New Jersey approach.'' I have intentionally caricatured
the worse-is-better philosophy to convince you that it is obviously a bad
philosophy and that the New Jersey approach is a bad approach.
However, I believe
that worse-is-better, even in its strawman form, has better survival
characteristics than the-right-thing, and that the New Jersey approach when
used for software is a better approach than the MIT approach.
Let me start out by
retelling a story that shows that the MIT/New-Jersey distinction is valid and
that proponents of each philosophy actually believe their philosophy is better.
Two famous people,
one from MIT and another from Berkeley (but working on Unix) once met to
discuss operating system issues. The person from MIT was knowledgeable about
ITS (the MIT AI Lab operating system) and had been reading the Unix sources. He
was interested in how Unix solved the PC loser-ing problem. The PC loser-ing
problem occurs when a user program invokes a system routine to perform a
lengthy operation that might have significant state, such as IO buffers. If an
interrupt occurs during the operation, the state of the user program must be
saved. Because the invocation of the system routine is usually a single instruction,
the PC of the user program does not adequately capture the state of the
process. The system routine must either back out or press forward. The right
thing is to back out and restore the user program PC to the instruction that
invoked the system routine so that resumption of the user program after the
interrupt, for example, re-enters the system routine. It is called ``PC
loser-ing'' because the PC is being coerced into ``loser mode,'' where
``loser'' is the affectionate name for ``user'' at MIT.
The MIT guy did not
see any code that handled this case and asked the New Jersey guy how the
problem was handled. The New Jersey guy said that the Unix folks were aware of
the problem, but the solution was for the system routine to always finish, but
sometimes an error code would be returned that signaled that the system routine
had failed to complete its action. A correct user program, then, had to check
the error code to determine whether to simply try the system routine again. The
MIT guy did not like this solution because it was not the right thing.
The New Jersey guy
said that the Unix solution was right because the design philosophy of Unix was
simplicity and that the right thing was too complex. Besides, programmers could
easily insert this extra test and loop. The MIT guy pointed out that the
implementation was simple but the interface to the functionality was complex.
The New Jersey guy said that the right tradeoff has been selected in
Unix-namely, implementation simplicity was more important than interface
simplicity.
The MIT guy then
muttered that sometimes it takes a tough man to make a tender chicken, but the
New Jersey guy didn't understand (I'm not sure I do either).
Now I want to argue
that worse-is-better is better. C is a programming language designed for
writing Unix, and it was designed using the New Jersey approach. C is therefore
a language for which it is easy to write a decent compiler, and it requires the
programmer to write text that is easy for the compiler to interpret. Some have
called C a fancy assembly language. Both early Unix and C compilers had simple
structures, are easy to port, require few machine resources to run, and provide
about 50%--80% of what you want from an operating system and programming
language.
Half the computers that
exist at any point are worse than median (smaller or slower). Unix and C work
fine on them. The worse-is-better philosophy means that implementation
simplicity has highest priority, which means Unix and C are easy to port on
such machines. Therefore, one expects that if the 50% functionality Unix and C
support is satisfactory, they will start to appear everywhere. And they have,
haven't they?
Unix and C are the
ultimate computer viruses.
A further benefit of
the worse-is-better philosophy is that the programmer is conditioned to
sacrifice some safety, convenience, and hassle to get good performance and
modest resource use. Programs written using the New Jersey approach will work
well both in small machines and large ones, and the code will be portable
because it is written on top of a virus.
It is important to
remember that the initial virus has to be basically good. If so, the viral
spread is assured as long as it is portable. Once the virus has spread, there
will be pressure to improve it, possibly by increasing its functionality closer
to 90%, but users have already been conditioned to accept worse than the right
thing. Therefore, the worse-is-better software first will gain acceptance,
second will condition its users to expect less, and third will be improved to a
point that is almost the right thing. In concrete terms, even though Lisp
compilers in 1987 were about as good as C compilers, there are many more
compiler experts who want to make C compilers better than want to make Lisp
compilers better.
The good news is
that in 1995 we will have a good operating system and programming language; the
bad news is that they will be Unix and C++.
There is a final
benefit to worse-is-better. Because a New Jersey language and system are not
really powerful enough to build complex monolithic software, large systems must
be designed to reuse components. Therefore, a tradition of integration springs
up.
How does the right
thing stack up? There are two basic scenarios: the ``big complex system
scenario'' and the ``diamond-like jewel'' scenario.
The ``big complex
system'' scenario goes like this:
First, the right
thing needs to be designed. Then its implementation needs to be designed.
Finally it is implemented. Because it is the right thing, it has nearly 100% of
desired functionality, and implementation simplicity was never a concern so it
takes a long time to implement. It is large and complex. It requires complex
tools to use properly. The last 20% takes 80% of the effort, and so the right
thing takes a long time to get out, and it only runs satisfactorily on the most
sophisticated hardware.
The ``diamond-like
jewel'' scenario goes like this:
The right thing
takes forever to design, but it is quite small at every point along the way. To
implement it to run fast is either impossible or beyond the capabilities of
most implementors.
The two scenarios
correspond to Common Lisp and Scheme.
The first scenario
is also the scenario for classic artificial intelligence software.
The right thing is
frequently a monolithic piece of software, but for no reason other than that
the right thing is often designed monolithically. That is, this characteristic
is a happenstance.
The lesson to be
learned from this is that it is often undesirable to go for the right thing
first. It is better to get half of the right thing available so that it spreads
like a virus. Once people are hooked on it, take the time to improve it to 90%
of the right thing.
A wrong lesson is to
take the parable literally and to conclude that C is the right vehicle for AI
software. The 50% solution has to be basically right, and in this case it
isn't.
But, one can
conclude only that the Lisp community needs to seriously rethink its position
on Lisp design. I will say more about this later.