Re: Serious Performance Problem with Sun NIT/DLPI
Author:   Van Jacobson <van@ell.ee.lbl.gov>
Date:     1995/06/05
Forums:   comp.protocols.tcp-ip, comp.sys.sun.misc

In article <3qd3jr$2nv@guardian.EnGarde.com>, Mike Neuman wrote:

>In preparation for the commercial release of our IP-Watcher network
>monitoring and control program (see http://nad.infostructure.com/watcher.html),
>we've been doing a great deal of performance testing on a variety of network
>packet filters. The results of which are interesting, although they only
>seem to support my theory that STREAMS *REALLY* stinks for realtime (or
>at least time-critical) applications.

Mike,

You might be interested in the paper "The BSD Packet Filter: A New
Architecture for User-level Packet Capture" by Steve McCanne &
me in the 1993 Winter USENIX proceedings (also available as
ftp://ftp.ee.lbl.gov/papers/bpf-usenix93.ps.Z).  It includes
detailed NIT vs. in-kernel BPF performance measurements and
some explanation of why you got the results you did.  Using the
in-kernel BPF (so you totally avoid STREAMS) the differences
are much more dramatic:  BPF is always at least 10 times faster
than NIT and for the case where most packets are discarded by the
filter (which is common for a lot of monitoring applications) BPF
is roughly 100 times faster than NIT (5us/packet vs 450us/packet
on an SS-2).  Essentially all the difference is due to STREAMS.

I agree with your conclusion that STREAMS stink.  Anyone who's
measured the code reaches the same conclusion.  It's widely
believed that the `designers' of STREAMS were all former COBOL
programmers taken off writing long distance billing programs
for AT&T and ordered to learn C and turn v7 Unix into System-V.
Even Dennis Ritchie, after looking at what had been done to turn
his elegant v8 Stream I/O system into `STREAMS', remarked that
"the idea loses something when it's shouted".

However, as I think the BPF USENIX paper shows, the following isn't
right:

>Why not just implement the filtering stuff in libraries--it apparently is
>faster to copy it out of the kernel than filter it in-kernel? :-)

Just because AT&T forced some garbage code down Sun's throat
(literally -- Sun was `encouraged' to adopt STREAMS in the 80s
when AT&T owned a 25% equity interest in Sun and had a seat on
the board) that resulted in serious performance problems that Sun,
although trying valiantly, still hasn't recovered from, it doesn't
mean the poor performance is intrinsic.  With decent code in the
kernel (*not* STEAMS) an in-kernel filter is *much* faster.

 - Van