From van@helios.ee.lbl.gov  Mon Jul 18 15:03:25 1988
Posted-Date: Mon, 18 Jul 88 15:03:13 PDT
Received-Date: Mon, 18 Jul 88 15:03:25 PDT
Received: from vs.ee.lbl.gov by venera.isi.edu (5.54/5.51)
	id AA19686; Mon, 18 Jul 88 15:03:25 PDT
Received: by helios.ee.lbl.gov (5.59/s2.2)
	id AA00277; Mon, 18 Jul 88 15:03:14 PDT
Message-Id: <8807182203.AA00277@helios.ee.lbl.gov>
To: end2end-interest@venera.isi.edu
Cc: end2end-interest@venera.isi.edu
Subject: a new monitoring tool & an ethical question
Date: Mon, 18 Jul 88 15:03:13 PDT
From: Van Jacobson <van@helios.ee.lbl.gov>
Status: R

I've had a summer student working on a project to make a
"programming language" for network monitoring.  A first cut is
ready (we're running it & it's giving us useful information - an
example is attached) but the language is sufficiently powerful
that I wonder if distributing it might not be letting a monster
loose on an unsuspecting world.  In particular, it took me all
of 10 minutes last night to code up a "password grabber" that
watched the start of telnet sessions (conveniently signaled by a
SYN) and recorded the user name (triggered by the prompts
"login:"  or "username:") and password (prompt "password:").

I suspect that computer security isn't going to improve much in
the near future and the broadcast media used for virtually all
local networking make life awfully easy for the bad guys.  So,
before I announce this thing is available, should I announce
that this thing is available?

 - Van

 -------------------
A brief description:

We've grafted some network monitoring extensions onto the Icon
language (if you're not familiar with Icon, look up the book
"The Icon Programming Language" by Griswold & Griswold).  Icon
has some nice features for making monitoring programs --
associative arrays, a "set" data type, excellent string
handling and a goal-directed evaluation mechanism ("generators")
that makes it easy to code quite complex filters.  To get
reasonable performance, packet capture and "first level"
filtering is not done in Icon but in a heavily tuned C "filter
machine" (the code is three times faster than the Stanford
packet filter, which was no slouch to begin with.  The filter is
lifted from the new tcpdump).  A captured packet appears to the
programmer as an Icon record -- e.g., `p.tcp.ack' gets the ack
from a tcp packet.  Some serious magic involving lazy evaluation
dynamically fixes up offsets to account for variable length
headers & different types of encapsulation.  This makes the
world look simple to the user yet keeps the performance up.

As an example, here's a program that watches arp traffic and
maintains a database of IP - Ethernet address mappings.  Since
hosts may have to change their ethernet address (e.g., to run
Decnet) and may have to change their IP address (e.g., it's
dynamically assigned or a gateway's running proxy arp), both
mappings are one-to-many.  So, the basic data structures are two
tables, ether2ip and ip2ether, each of which contains a set of
addresses (Icon's "table" is an associative array, an array you
can index with anything).

The database is kept on a flat file with lines of the form
	ether	<ether addr> <ip addr1> <ip addr2> ...
and
	ip	<ip addr> <ether addr1> <ether addr2> ...
The program initializes its tables from this file when it starts
and appends a line to this file whenever a new ether-ip
combination appears.  (The code to read and write is boring so I
omitted it).  The program also logs a message whenever a new
ether-ip combination appears.

With that lengthy introduction, the program looks like: (BTW,
this is a program that we're actually running -- it has been a
great help finding people who mis-configure their workstations
and a help tracking down network problems)

# arpwatch

global e2i, i2e

procedure main()

	initialize_tables()

	# make a filter to grab only arp packets
	filter := make_filter ("arp")

	while (packet := read_packet(filter)) {
		# handle the ethernet-to-ip mapping
		if (ip_set := \e2i[packet.arp.xtha]) {
			# we've seen this ethernet address before;
			# check if the associated ip address is new.
			if (not member(ip_set, packet.arp.xtpa)) {
				insert(ip_set, packet.arp.xtpa)
				update_db ("ether", packet.arp.xtha, ip_set)
			}
		} else {
			# new ethernet address, make a new table entry
			# with a singleton set of the arp ip address.
			e2i[packet.arp.xtha] := set([packet.arp.xtpa])
			update_db ("ether", packet.arp.xtha, ip_set)
		}
		# same as above but in the ip-to-ethernet direction
		if (ether_set := \i2e[packet.arp.xtpa]) {
			if (not member(ether_set, packet.arp.xtha)) {
				insert(ether_set, packet.arp.xtha)
				update_db ("ip", packet.arp.xtpa, ether_set)
			}
		} else {
			i2e[packet.arp.xtpa] := set([packet.arp.xtha])
			update_db ("ip", packet.arp.xtpa, ether_set)
		}
	}
end