Frank da Cruz,
fdc@kermitproject.org
Most recent update:
17 July 2024
RECENT:
Effective
1 July 2011...
- The Kermit Project at Columbia University canceled
- A new Kermit Project website opened
at http://www.kermitproject.org.
- All Kermit software has an
Open Source license.
- The Kermit software archive and website at Columbia will remain in place.
Welcome to the new Open-Source Kermit Project.
WHAT IS KERMIT?
Kermit is the name of a file-transfer and -management protocol and a suite
of computer programs for many types of computers that implements that
protocol as well as other communication functions ranging from terminal
emulation to automation of communications tasks through a high-level
cross-platform scripting language. The software is transport-independent,
operating over TCP/IP connections in traditional clear-text mode or secured
by SSH, SSL/TLS, or Kerberos IV or V, as well as over serial-port
connections, modems, and other communication methods (X.25, DECnet, various
LAN protocols such as NETBIOS and LAT, parallel ports, etc, on particular
platforms).
The Kermit Project was founded at the Columbia University Computer Center
(now CUIT) in 1981 to meet a specific need, and until the mid- to late
1990s, Kermit was Columbia's standard desktop connectivity software, used
universally by students, faculty, and staff to connect from desktop
microcomputers, PCs, Macintoshes, and Unix workstations to the central
computing facilities: the IBM
mainframes (1963-2017), the DECSYSTEM-20s
(1977-1988), CLIO
(Columbia's first online library information system, 1984-2003), and Cunix
(our Unix-based servers, 1986-present), and to departmental VAXes, PDP-11s,
Suns, and other minicomputers. In the early days of microcomputers and PCs
but before widespread deployment of local area networks and desktop
workstations that connected to them, Kermit software linked the desktop to
e-mail, bulletin boards, file sharing, text processing, messaging, and other
aspects of the new on-line culture that is now taken for granted, long
before the experience was available at most other institutions. At
Columbia, the DEC-20s and the departmental minicomputers are long gone and
the IBM mainframes are now only for backoffice use, but Kermit software is
still used for SSH sessions from the desktop to CUNIX, and by the technical
staff for system and network administration tasks; for example, configuring racks
full of HP blade servers as they arrive, management of the
University's telephone system, CGI scripting,
alpha
paging of on-call staff, and so on. Plus, of course, by old-timers who
just plain prefer the safety and efficiency of text-mode
shell sessions for email and to get their work done; for example,
software development and website management.
Over the years, the Kermit Project grew into a worldwide cooperative
nonprofit software development and distribution effort, headquartered at and
coordinated from Columbia University,
as Kermit software was ported to or developed for more and more computers
and operating systems (see list). The Kermit
Project is dedicated to production of cross-platform,
long-lasting, stable, standards-conformant, interoperable
communications software, and has been actively engaged in
the standards process. Kermit software is used
all over the world in every sector of the economy: national government,
state and local government, academic, medicine and health care, engineering,
aerospace, nonprofit, and commercial.
Although terminal emulation has been largely supplanted by the Web for
online access, Kermit software continues to play a role in other
applications such as remote sensing and data collection, management and
troubleshooting of networking and telecommunications equipment, back office
work, cargo and inventory management, medical insurance claim submission,
electronic funds transfer, and online filing of income tax returns. Kermit
software is embedded in network routers and switches, in cell-phone towers,
in medical diagnostic and monitoring equipment, even in cardiac pacemakers, not
to mention the cash registers of quite a few big-name "big box"
retailers. In 2002 Kermit flew on the International
Space Station, and Kermit software is the communication method used by
EM APEX ocean floats (left) supplying realtime
data to hurricane researchers and trackers to this day (the hurricane
project entered a new expanded phase in 2010 based on a new version of Embedded Kermit).
Since the 1980s, Kermit protocol and software have been used on the
factory floor in programmable die-cutting, press brake, laminating,
flat roll, shearing, metal- and plastic-processing, woodworking, and other
machines. For example, in the manufacture of the Boeing 787, where
Kermit is used to control a Tape
Layer that forms certain body components. You can read more about how
Kermit is used on the factory floor here and here.
In the 1990s Kermit software was used in US Post Office automation, it played a
key role in the 1994 Brazilian national
election (the biggest in the history of the world up to that
time), and it was central to the UN relief
mission to Bosnia, “linking the entire spectrum of the project
operation, from mainframe, minicomputer, PCs, to handheld devices and
barcode readers.”
In the 1980s the robustness of the Kermit protocol suited it ideally
for service in the
Green Revolution
in Africa, the joint European-USSR Giotto space
mission, and perhaps most notably in reestablishing data communication
between US research stations in Antarctica and the mainland after they were
cut off in 1986 in a computer mishap during the 9-month Antarctic winter.
In 1989 an international conference on Kermit
was hosted in Moscow, USSR, and Kermit sessions were featured at other
conferences throughout the 1980s in Tokyo, Bern, Paris, Nashville, and
elsewhere.
The Kermit protocol and software are named after Kermit the Frog,
star of the television series, The Muppet Show; the name Kermit is used by
permission of Henson Associates, Inc. Why is it named after Kermit the
Frog? In May of 1981 we already had first implementations of the
protocol working, but we didn't have a name for the protocol or the software
yet. A group of us was discussing it (me, Bill Catchings, Bill Schilit,
Jeff Damens, I think that was the group), without actually caring too much
since we never expected the software to spread all over the world and last
for decades. I happened to be facing the wall that had a Muppets calendar
on it, and since my children were such big fans of the Muppet Show I
said, How about Kermit
? Thirty years later (May 2011) I found
the calendar page that I was looking at when I said that, you can see it on
the left and you can click on it to see a bigger image.
KERMIT SOFTWARE
Kermit software has been written for
hundreds
of different computers
and operating systems, some of it by volunteer programmers all over the world,
some of it by the Kermit Project professional staff.
The major features of the most popular Kermit programs are:
- Connection establishment and maintenance for a wide variety of
connection methods (TCP/IP, X.25, LAN, serial port, modem, etc).
- Terminal emulation.
- Error-free file transfer.
- Internet protocols including Telnet, Rlogin,
FTP, and HTTP.
- Internet security methods
including Kerberos, SSL/TLS, SSH, and SRP.
- Character-set conversion
during both terminal emulation and file
transfer – a unique feature of Kermit software.
- Numeric and alphanumeric paging.
- Script programming to automate complicated or repetitive tasks.
Kermit's user interface and script programming language are consistent across
platforms and communication methods, allowing the investment in learning to
pay off time and again as you move from one platform to another, one
communication method to another.
Our premiere Kermit software implementations are:
C-Kermit and
IBM Mainframe Kermit are
host-based packages with an unequaled range of versatility.
Kermit 95 and MS-DOS
Kermit are full-featured desktop communication software programs
rivaling the quality of anything else on (or off) the market, except perhaps
in flashiness of user interface: Kermit programs follow the text-mode
prompt-and-command style of yesteryear, which is baffling to some people
until they realize the advantages:
- The command set is reasonably consistent across all platforms, and
almost totally consistent across modern platforms such as Windows, Mac OS X,
Linux, and VMS. Learn it once, use it everywhere.
- Commands may be combined into "macros" or "programs" to automate any
task that can be done by hand, as described here.
In fact in C-Kermit and Kermit 95, the command language is a full-blown
programming language with variables, control structures, functions,
"subroutines", plus a few surprises.
- You don't have to know the commands in advance nor type them out in full.
The command style is called “context-sensitive menu on demand”
(you see the available choices when you type a question mark), and keywords
can be abbreviated. There is plenty of built-in help, and plenty more help to
be found on the Kermit website; for example the C-Kermit tutorial and the the Kermit 95 tutorial, just for starters.
- Touch typists can work faster when they don't have to move their hands
away from the home keys, and they suffer less repetitive strain injury.
- Certain things just can't be done efficiently or at all using a GUI
interface. Here's a completely random example, but it makes the point:
On a PC I have directory that contains thousands of images, together
with their thumbnails. For every image
xxx.jpg there is a thumbnail
xxx-t.jpg. I want to load all the thumbnails into Photoshop. Using
the mouse, this would take all day. With Kermit you can do it like this
(at the Kermit command prompt):
mkdir thumbnails
rename *-t.jpg thumbnails/
And then in the thumbnails subdirectory, Ctrl-A to "select all" and drag
to Photoshop (and then, if desired, drag the thumbnails back to the original
directory with one mouse motion, or rename them back with one Kermit command).
Kermit 95 was developed not only to meet Columbia's need for
connectivity from Windows 95 (and later) to the central text-based
services, but also to raise money to support the Kermit Project. Unlike
other Kermit programs, K95 was strictly commercial, available in both a
retail shrinkwrapped version (right) and in bulk
right-to-copy licenses. From its release in 1995 until mid-2011, over a
quarter million bulk license seats were purchased in over 1000 licenses
licenses ranging in size from 100 seats to 10,000. About 30,000
shrinkwrapped copies were sold, many thousands more purchased for download
from e-academy, and K95 was
site-licensed by over 100 universities as well as by entire statewide
university systems such as SUNY (64 campuses with about 400,000 students).
The Kermit Project was put on a self-funding basis in 1984, and from
then until its cancellation in 2011, it realized $8,894,912.00 in
revenue for the University, plus an equipment grant
(the Hermit
Project) valued at $3,000,000.00. Between 1984, when the Kermit
"business" began, until 1998, when the Internet took over the world, we made
31,591 shipments of Kermit software on magnetic media (mainly 10-inch reels
of 9-track magnetic tape); 4679 of them international to 107 different
countries including some that no longer exist such as the USSR and
Yugoslavia, and to others you might not expect such as New Caledonia.
KERMIT PROTOCOL
Since its inception in 1981, the
Kermit protocol has developed into a
sophisticated, powerful, and
extensible transport-independent tool
for file transfer and management, incorporating, among other things:
Kermit protocol uses well-defined, sequenced, error-checked packets in each
direction to effect a file-transfer session, following standard rules of
protocol layering. Packets are designed for maximum transparency, so they
can pass though any communication medium, no matter how restrictive.
Half-duplex (stop and wait), full-duplex (sliding windows with selective
retransmission), and continuous streaming transport can be used to adapt to
any connection.
The feature that distinguishes Kermit protocol from most others is its wide
range of settings to allow adaptation to any kind and quality of connection
between any two kinds of computer — packet length, packet encoding,
window size, character set, error-detection method, timeouts, pauses. Most
other protocols are designed to work only on certain kinds or qualities of
connections, and/or between certain kinds of computers or like file systems,
and therefore work poorly (or not at all) elsewhere and offer few if any
methods to adapt to unplanned-for situations. Kermit, on the other hand,
allows you to achieve successful file transfer and the highest possible
performance on any given connection.
Unlike FTP or X-, Y-, and ZMODEM (the other protocols with which Kermit is most
often compared) Kermit protocol does not assume or require:
- a full-duplex connection;
- a connection that is transparent to control characters;
- an 8-bit connection;
- a clean connection;
- big buffers all along the communication path;
- physical-link-layer flow control.
(although Kermit does not require any of these conditions, it can take
advantage of them when they are available).
A feature article on Kermit protocol by Tim Kientzle in the February 1996
issue of Dr. Dobb's Journal noted that
“Kermit's windowing approach is faster than protocols such as XModem
and YModem . . . What many people don't realize is that under
less-than-ideal conditions, Kermit's windowing approach is significantly
faster than ZModem, a protocol with a well-deserved reputation for fast
transfers over good-quality lines.” The efficiency of the Kermit
protocol is analyzed in depth here and here.
Thus Kermit transfers work "out of the box" almost every time. And at a
higher level, the Kermit command language allows all sorts of handy file
selection criteria to be used in any combination, for example:
- Wildcards and patterns to match filenames
- Selection by date ranges
- Selection by size ranges
- Only text files
- Only binary files
- Only files that don't exist on the other end or that are newer
- Exception lists and patterns
to accomplish almost any grouping you can imagine. In transit, a file can
have its character-set converted, it can be passed through a filter, etc, and
upon successful transfer, the source file can be deleted or renamed, the
destination file can be renamed or mailed, and so on.
The Kermit Protocol Specification
The Kermit file transfer protocol specification is given in the book,
Kermit, A File Transfer Protocol
by Frank da Cruz, with a foreword by
Donald Knuth.
As of 2016 the book is also available online in
PDF format).
The specification is also available online in the Sixth edition of the
online
Kermit Protocol Manual (1986). Both of
these lack the later refinements, but do include server mode, long packets,
sliding windows, etc.
Documentation for the later additions is
collected and publicly
available
HERE.
A
formal specification and verification of the Kermit protocol was
published by James Huggins of the University of Michigan in 1995; you can
download it
HERE.
KERMIT FILE TRANSFER EXAMPLE
Let's look the common case where you have a Windows desktop computer with a
connection — any kind of connection (modem, serial port, regular
Telnet, secure Telnet, rlogin, secure rlogin, SSH) — to a shell
session on a Unix server ("Unix" = Linux, Mac OS X, FreeBSD, Solaris, AIX,
HP-UX,
etc) and you want to transfer a file between
your PC and the Unix server. Your terminal emulator on Windows is
Kermit 95 and the Unix server has
C-Kermit or
G-Kermit
installed, which can be invoked simply by typing “kermit” at the
shell prompt (or perhaps “ckermit” or “gkermit”).
To download a file, say, message.txt,
you type the following command at the shell prompt:
kermit -s message.txt
The file is sent to Kermit 95's current directory on your PC (or to its
DOWNLOAD DIRECTORY if you have defined one). It doesn't
matter if the file is text or binary; Kermit figures it out and transfers it
automatically in the appropriate mode.
Similarly if you want to transfer a group of files, say, all the files
whose names start with “daily.”:
kermit -s daily.*
Kermit sends each file that matches, switching automatically
between text and binary mode as appropriate for each file (daily.jpg,
daily.xls, daily.txt, ...)
Uploading a file from your PC to Unix is just as easy. Suppose you
have a file called “budget.xls” in
Kermit 95's current directory on your PC. To upload it to UNIX, type
this at the Unix shell prompt:
kermit -g budget.xls
Those are the basics; there are many variations and refinements; for example:
- Only transfer files that are newer than the counterparts on the other end.
- Convert character sets of text files appropriately (e.g. between ISO
8859-1 and Unicode UTF-8).
- Recover a partial transfer from the point of failure (binary mode only).
To save yourself some typing, you can define aliases on Unix (in your
shell profile):
alias s="kermit -Ys"
alias g="kermit -Yg"
(
s for Send,
g for Get). And then:
s message.txt
g budget.xls
It's worth noting that you are transferring your files over the same
connection you already have; thus there is no need make a new connection,
re-authenticate yourself, or similar bureaucracy. If the connection is
secured by SSH, Kerberos, SSL, TLS, or SRP, then the file transfer is also
secure, automatically.
This marks an unparalleled degree of convenience. When you tell C-Kermit on
Unix to send or get a file, its first file-transfer packet is recognized
automatically by Kermit 95's terminal emulator and K95 pops into either
receive mode or server mode, depending on the direction, and when the
transfer is finished, K95 returns to its terminal emulation screen. If
there is an error (for example, if you do not have write permission in the
destination directory) K95 remains in its file-transfer screen so you can
see what the problem was.
The same procedures also work Unix-to-Unix, K95-to-VMS, Unix-to-VMS, VMS to
Unix, or OS/2 to VMS or Unix, as long as you are using K95 or C-Kermit as
your terminal program.
CONTROVERSIES
Also see: Popular
Misconceptions.
Over the years, the Kermit Project and software were the subject of various
controversies, notably:
- License
- From the very beginning we wanted Kermit software to be free to
everybody. But starting in 1984, Columbia University compelled us to find a
way to make it pay for itself; that is, to pay the salaries of the full- and
part-time staff, and for equipment, supplies, phone, etc. Otherwise we
would not be allowed to continue developing, maintaining, distributing, and
supporting the software, which by then had become popular all over the
world.
Our solution was to keep the software free for every individual and
organization for his/her/its own use, but to require companies to license it
if they were going to bundle it with a product or otherwise furnish it to
customers or clients; that is, if they were looking to make money from our
labor. This way they could make money but they would have to share
it with those who did the work.
As the Free Software movement took root, its proponents objected strenously
to this approach, but it allowed the Kermit project to continue another 10
years. Then in 1994, with the upcoming release of Microsoft Windows 95, we
decided to release the one and only Kermit program that was 100% commercial:
Kermit 95. This product allowed the
Kermit Project to flourish until about 2003, when the US and world economy
began to crash, and to continue to exist in increasingly diminished form
until 2011 when the Kermit Project at Columbia University was finally
canceled. At that point, since nobody's job depended on it any more, all
Kermit software that we had complete rights to was placed under an Open Source license, and now everybody is happy
except those who lost their jobs and those who called our free tech-support
number whenever they needed help. And those who wonder why there was never
another Kermit 95 release.
- Kermit vs X/Y/ZMODEM
- The XMODEM file
transfer protocol was developed elsewhere in 1977 for transferring files
over telephone connections from one microcomputer to another, and thus found
wide use among computer hobbyists, BYTE magazines fans, users and admins of
BBS
systems, and the like. Its successors, such as YMODEM and ZMODEM, grew
up in the same culture, serving approximately the same user base. In the BBS
world, communication links were always 100% transparent to all 256 byte
values, allowing these protocols to be relatively simple and still work well
in that environment; thus inhabitants of the BBS/hobbyist culture had
no reason to need or learn about Kermit.
The Kermit protocol, on the other hand, was designed for micro-mainframe
connections, which were much less tolerant and much more demanding because
the connections were rarely transparent, and the underlying computers were
radically different; for example, they might use different file formats
and character sets for file storage. Kermit, then, was aimed more towards
institutions — universities, hospitals, corporations, government
agencies — that had machine rooms with big central shared computers or
a diversity of departmental minicomputers plus individual users with PCs or
workstations on their desks, rather than hobbyists all with relatively
homogeneous personal microcomputers.
XMODEM was a painfully slow protocol, so the impetus was to evolve it into
faster and faster protocols; hence YMODEM and ZMODEM. But the newer MODEM
protocols still assumed a (more or less) 100% transparent connection between
two identical or very similar computers.
As YMODEM and ZMODEM appeared, people began to criticize Kermit protocol for
being slow, as indeed it was in its original form: short packets because
most mainframes could not withstand long bursts of incoming data from a
terminal; half-duplex stop-and-wait because IBM mainframes did not support
full-duplex communication; printable encodings for control characters and
8-bit characters because these could not pass through the mainframe's
terminal driver. Thus the original Kermit protocol was a “least
common denominator” among all the platforms where it needed to run
(and many more, besides, as it turned out). Its major strength was that it
was adaptable to any platform or communication method, including those where
XMODEM family did not fit at all; for example, in the IBM mainframe world.
Meanwhile, some BBS software packages offered Kermit protocol on their
Upload and Download menus, but those Kermit implementations were invariably
minimal (i.e. slow), often buggy, and occasionally totally
nonfunctional (see the Misconceptions page
about third-party Kermit protocol implementations). This tended to
reinforce the impression within the hobbyist culture that Kermit protocol
was slow.
To address the performance complaints, we took advantage of the intrinsic
extensibility of the Kermit protocol design (in which transfers begin with a
feature-negotiation phase) to add options for longer packets and for
full-duplex sliding windows with selective retransmission, as well as
options for compression and for taking advantage of transparent and/or
error-free connections (for example, network connections) when they were
available. These changes made Kermit protocol as fast or faster than ZMODEM
without sacrificing its universality, data conversion features,
robustness, and (most important) backwards compatibility (which is why you
don't see separate protocols: XKERMIT, YKERMIT, ZKERMIT).
The performance changes date back to about 1993;
see benchmarks.
Neverthess, each camp had its adherents based largely on its own culture and
each tended to dismiss the other, a trend that continues until today. Most
critics of Kermit base their observations on Kermit software from early
1980s, or upon 3rd-party Kermit protocol implementations, which tend to work
poorly. For a more detailed discussion, see the Misconceptions page.
In 2013 I noticed the Slashdot
discussion of the cancelation of the Kermit Project at Columbia
University. It illustrates the present topic quite well, as the
discussion is dominated by hobbyists and BBS users. But a few knowledgeable
Kermit users also contributed; here are some examples:
- Wow, in my college and post college days I used that protocol in so many
places and so many ways I can't even begin to count. That was a very
conservative protocol that was able to go through almost anything. One time
I had it go from a portable computer over a modem connection to an Equinox
data switch to an AT&T 3b5 Unix, to a cu back to the Equinox (to change the
speed from 300 baud to 9600 baud) to an IBM 7171 protocol converter to an
IBM 4361. And it could actually transfer files. Another time I had to stress
test a DECNET terminal simulator on a Sun (the old version would fail in the
middle of the day on the busiest of days) So I used kermit to connect to
host1, then to host 2, back to host 1, back to host 2, I think something
like 40 times. Then I did a file transfer through all the connections. It
worked.
- Wow. In the early 90s, I was responsible to connect the first Rumanian
universities (Bucharest, in particular) to the Internet. Since we couldn't
get IP going for various technical reasons, we decided to get them email in
the mean time, at least.
The first try was with uucp, but they couldn't handle its operations on the
Bucharest side. Phone lines weren't stable enough, then. So, for the 1st 6
months, email was sent to Bucharest by Kermit file transfer, triggered by a
hodge-podge of MDA scripts, invoked by sendmail. Kermit was way more robust
than any other file transfer protocol at this time, we believed eventually
it could handle bit transfers over wet clothes lines.
- Yes, it is used a lot in the embedded world. One of the few tools
available to recover a bricked RS232-only based device. Used on things like
the gumstix, beagleboard, and lots of other SBC like ARM based embedded
devices. If you make/order custom versions or your own shipping product does
not contain alternatives like MMC/SD card boot capabilities, c-kermit is one
of the few things out there to allow you to boot, load code, and then go to
console all from one tool on such devices. Saved my (and my employers) ass
many times on bricked or buggy embedded devices.
In the same discussion there is some complaining that no adequate
explanation was given for why some modules of Kermit 95 could not be
released in Open Source. The explanation was, and is, HERE.
Onsite Links
Offsite Links
- Kermit
Oral History Panel: Jeffrey Altman, Bill Catchings, and Frank
da Cruz, 6 April 2012, Watson Laboratory, Columbia University,
New York, Computer History Museum [Locally
archived copy].
- Oral
History of Joe Doupnik, 23 July 2012, Mountain View, California,
Computer History Museum [Locally archived
copy]. Note: On p.8, Joe refers to Daphne Tzoar as one of the
original MS-DOS programmers, which is correct, but then mistakenly credits
her with with the subsequent MS-DOS Kermit books, which were actually
written by Christine Gianone. Also on that page "Jeff Damons" should be
"Jeff Damens".
- Kermit
Project materials index at at the Computer History Museum.
-
Guide to the Frank da Cruz Kermit records,
Online Archive of California, January 2020.
- Kermit
(Protocol), Wikipedia, accessed 9 October 2019.
Translations of this page courtesy of...
[about translations]
The Kermit Project and Software 1981-present
| 23 September 2011
| Updated: Thu May 30 11:11:58 2024
|
|