Reading MIME Messages

[previous] [next] [table of contents] [index]

When you type show to display a non-MIME message, MH will run a showproc program) to display the message header and body. For MIME messages, show calls its mhnproc program to display the message -- by default, that's mhn. This section explains what happens overall and how you can control it. To learn about changing the way mhn works, read the Section Configuring mhn.

Getting Test MIME Messages

If you'd like to experiment as you read, you can get a sample MIME message from the sample MIME server. Also, as of this writing, there were lots of sample messages available for anonymous FTP on thumper.bellcore.com in the directory pub/nsb/samples. Most of those samples on thumper were written as MIME was being developed; some of them use old syntax that may make mhn complain, but they're still useful. You can copy the messages into an MH mail folder and rename the files (which have strange names) by following these steps:
% folder +mime-samples
Create folder "/yourMHdir/mime-samples"? y
% cd `mhpath`    ...cd to the MH folder
% ftp thumper.bellcore.com
Name (thumper.bellcore.com:ambar): anonymous
Password: xxx@yyy.com (use your user name and host here)
ftp> cd pub/nsb/samples
ftp> prompt      ...ftp won't ask whether to get each file
ftp> mget *
    ...lots of files transfer...
ftp> quit
% ls
+0cqBCNW0M2YtQhngB3   +0dYMHpa0M2YtABA8E7   +0eCBvVy0M2YtAoUABO
    ...lots of strange filenames appear...
Next, use a shell loop to rename the files to 1, 2, and so on, so that MH will recognize them as messages. Here are both C shell and Bourne/Korn shell syntax:

C shell:

% set n=1
% foreach f (*)
? mv "$f" $n
? @ n++
? end

Bourne shell:

$ n=1
$ for f in *
> do mv "$f" $n
> n=`expr $n + 1`
> done
Finally, type scan +mime-samples and you should see the sample messages.

The mhnproc

When show reads a MIME message, it runs the mhnproc program to analyze and display the message. The default mhnproc is mhn. (The mhn -show switch isn't needed because that's the default.) The following sections explain how mhn works.

Unfortunately, if you're showing more than one message (as in show 3-7), show will hand all the messages to its mhnproc, even if only one message needs the mhnproc. mhn will show the plain-text messages -- but their headers and bodies will be shown separately, without your normal showproc.

If you want to use a different mhnproc, or if you don't want show to call its mhnproc for MIME messages, ther are alternatives to mhn.

How mhn Shows a Message

mhn looks at the message header and body parts of each message to find out how to display them. Then, mhn starts at least two programs to show the message: one for the header, and one or more for the body. (If the message is a partial message, mhn won't show it at all. Another section explains,)

mhn uses the mhlproc and the format file mhl.headers to show the message header. The default mhlproc is mhl. The Section Using mhl introduces mhl and shows how you can change the header format for non-MIME messages. You can use the same techniques for changing the mhl.headers file. For instance, if you don't want the default empty line between the To: and From: fields, or if you want to see the Content-xxx: header fields, make your own mhl.headers file to do that. The Section mhl explains how to configure mhl. You can use a different form file than that used for mhl.headers; give the -formfile filename switch to mhn or add it to the mhn: entry in your MH profile.

If there's a moreproc: entry in your MH profile, mhn will use it to page through the message header. This usually isn't necessary because the header is probably shorter than a screenful after it's been filtered through mhl. In some cases, it's irritating: If you use a pager program that pauses and prints (END) at the end of a file, you have to exit the pager after the message header is shown. To keep mhn from using a pager program on the message header, add the undocumented -nomoreproc switch to the mhn: entry in your MH profile. Because -nomoreproc is undocumented, you shouldn't depend on it to work in all versions of mhn. Also, it may have effects that I haven't noticed. Still, I think it's really useful.

After showing the header, mhn shows the body parts one-by-one (or in parallel, if it can, for a multipart/parallel message -- the %e escape, listed in the Table Display String Escapes, controls that). In a MIME message, each body part has a content-type. The default content-type (for messages without any Content-type: field) is text/plain; charset=us-ascii.

mhn can adjust to different character sets. If you don't use us-ascii, you should set the MM_CHARSET environment variable -- usually in your login shell's startup file. MM_CHARSET tells mhn which character set your current UNIX session is using. If the message charset= parameter is different than your current MM_CHARSET, mhn can translate the characters; it uses a mhn-charset- profile entry to do that. Once you set up this stuff, you can forget about it; mhn will "do the right thing." The Section Displaying Other Character Sets: mhn-charset- has the details.

To display a content, mhn runs a UNIX command. It finds the commands in profile entries like these:

mhn-show-type/subtype: command
mhn-show-type: command
For example, to display a text/enriched content, mhn looks for an entry mhn-show-text/enriched:. If it finds that entry, mhn runs the command listed. Otherwise, mhn looks for an entry mhn-show-text:. There are defaults for a few cases. If mhn runs out of choices, it prints an error like this:
mhn: don't know how to display content
     (content application/x-foobar in message 6, part 2)
If mhn is showing a multipart message, it may list content information on the terminal (like mhn -list would), then prompt and wait:
part 3     image/x-pbm                40K Sample waveform
Press <return> to show content...
When you see that prompt, you can skip the part (not display it) by pressing your interrupt key -- often, that's CTRL-C. Or, you can skip the rest of the message, and get back to a shell prompt, by pressing your QUIT key -- typically CTRL-\ (control-backslash). You can prevent the pause by giving mhn its -nopause switch.

Section Showing Contents: mhn-show- has all the gory details of these content-displaying entries. But you might want to do some basic setup right now. Why?

Partial Messages

The MIME content-type message/partial is designed for messages that need to be split into pieces. That's because some mailers and transport systems can't handle messages that are "too long." The safe size depends on the system and gateways, but 50,000 ASCII characters is a good guess. If someone sends you a partial message and you try to show it, here's what will happen:
% show
(Message inbox:131)
%
When you don't see a header or a body, show (actually, mhn) is probably telling you that the message isn't complete. There are several ways to see what's happening. The command show -noshowproc | head -20 will spit the first 20 lines of the current message onto your screen, but that's probably more than you need to see. The undocumented switch mhn -debug is probably the most interesting. It's also useful for figuring out why mhn is doing something you don't understand. But, because the switch is undocumented, you must not rely on -debug staying in future MH versions -- or for it to work in the same way.
% mhn -debug 131
big endian architecture
MIME-Version: 1.0
Content-Type: message/partial; id="<6374.782021163@ora.com>"; number=1; total=5
The message "big endian architecture" shows mhn checking my computer and getting ready to decode message parts. If the MIME-Version: field has a MIME version that mhn doesn't understand, it would complain and try to read the message anyway... but mhn with MH 6.8.3 does understand MIME version 1.0. Finally, mhn shows the Content-Type: field... and quits because the message isn't complete. It needs all the parts with the id= parameter shown. The number=1; total=5 tells us that this message is part 1 of a total 5 parts.

How can you find all the parts -- especially if you don't want to rely on the undocumented -debug switch? The easiest way is to scan your folder and check for five messages with the same subject -- most mailers seem to use the same Subject: field for all parts of a message (and MH does). Or, if you have a lot of messages and you don't want to wade through a long scan listing, you can get the Content-Type: field of one part and use pick to search for the other parts.

Here's how to do that search. You can ask scan to show you just the Content-Type:. (If you're interested in the details, see the Section scan Format Strings. Otherwise, just follow along.):

% scan -format '%{content-type}' 131
message/partial; id="<6374.782021163@ora.com>"; number=1; total=5
Next, pick can search all Content-Type: fields in the folder for that id= string. Using the -sequence switch will "remember" the messages pick finds by using a sequence. Notice my quoting: single quotes (') around the whole string, which contains double quotes ("):
% pick --content-type 'id="<6374.782021163@ora.com>"' -sequence picked
5 hits
Good. pick found five parts. Let's list them:
% mhn -list picked
 msg part  type/subtype              size description
 131       message/partial            47K part 1 of 5
 132       message/partial            46K part 2 of 5
 133       message/partial            46K part 3 of 5
 134       message/partial            46K part 4 of 5
 135       message/partial            15K part 5 of 5
In this case, the parts arrived in order. That doesn't matter to mhn, though. If you give it all the message numbers with its -store option, it'll be happy:
% mhn -store picked
storing message 131 as file /u/jerry/Mail/inbox/143
storing message 132 as file /u/jerry/Mail/inbox/143
storing message 133 as file /u/jerry/Mail/inbox/143
storing message 134 as file /u/jerry/Mail/inbox/143
storing message 135 as file /u/jerry/Mail/inbox/143
mhn glued the parts together into a new message, number 143, in the current folder. If you don't give mhn all the parts, it will complain. For example, if I don't name message 133:
% mhn -store 131 132 134 135
mhn: missing part 3 of 5 part multipart message
You can find and store all the parts in one step by using the storeparts shell script in this book's online archive. This section explains how to install it. When you find a partial message, you can find and store the other parts by simply typing storeparts. The script will search all messages in the folder unless you give it a list or range of messages. For instance, to check all messages from the current to last (for the parts of the current message):
% storeparts cur-last
storeparts: searching for 'id="<6374.782021163@ora.com>"'.
5 hits
storing message 131 as file /u/jerry/Mail/inbox/144
storing message 132 as file /u/jerry/Mail/inbox/144
storing message 133 as file /u/jerry/Mail/inbox/144
storing message 134 as file /u/jerry/Mail/inbox/144
storing message 135 as file /u/jerry/Mail/inbox/144
Finally, you can remove the partial messages if you'd like to. If you've followed the steps above, their numbers are stored in the picked sequence. (The storeparts script uses the temp sequence.)
% rmm picked
It's a good idea to use the mhn -check switch when you compose a split message. Then, the person who combines the parts (with mhn -store) can check the combined message to be sure nothing was lost.

If you want to experiment with partial messages, send yourself some test messages. The Section on Partial Messages explains how.

External Parts

Your mailbox has probably been cluttered by some really long messages that you didn't want. Even worse, you might have to pay for your email or network access by the amount of data you receive. External body parts shorten MIME messages by not sending the data. Instead, an external part has instructions for getting the data by FTP, from a mail server, or from a local file. A MIME-capable mail reader can ask you whether you want to retrieve the data. If you do, your MUA will do "the right thing": start an FTP job, email a request to a mail server, or retrieve the local file. When it has the data, the MUA will display it. (Data from a mail-server takes another step. See below.)

In case you're curious, an external body part looks like this:

Content-type: message/external-body;
        access-type="anon-ftp";
        site="ftp.xyz.com";
        directory="pub/speeches";
        name="jones.941024.au"
Content-Description: Jones lecture on new polymers

Content-type: audio/basic
Content-ID: <AAE28372.941024213426@xyz.com>
The first section, before the blank line, gives information about the external part: how to get it (in this case, by anonymous FTP), the filename, and so on. The second section has information that's used to display the data once it has been received. (Of course, you can't "display" sound parts. "Render" is more correct.) The Content-ID: is a unique identifier that helps the MUA match the data with the original message; it's especially important for data stored in a cache or fetched from a mail server.

When mhn sees an external body part, it prompts you like this:

Retrieve jones.941024.au (content 1.4)
    using anonymous FTP from site ftp.xyz.com? y
If you answer y, mhn will run anonymous FTP and display the result. If you answer n, mhn will check to see whether there are alternative parts (for example, a plain-text version of the audio speech) and use them instead; otherwise, it will issue a complaint like "mhn: don't know how to display any of the contents."

There's an even more efficient way to handle external body parts: a cache. Only one person at a site has to retrieve the external body part. Then everyone can share a copy of it.

For messages with an access-type of mail-server, there's an extra step. mhn will ask:

% show 1
    ....
Retrieve content by asking mime-server@foo.bar.sf.ca.us

binary
send meeting-summary.93.ps.gz

? y
A y answer will send a mail message to the server; the message header will include the same Content-ID: field used in the external body part. When the server replies, it's supposed to include that same Content-ID: field in its reply. (Unfortunately, most mail servers don't. Surprisingly, the MIME mail server from Metamail version 2.7 and before doesn't return the correct Content-ID: field, either. The bug is supposed to be fixed when version 2.8 comes out. I guess that very few people use mail-server external body parts...)

When the reply comes, store it in the content cache:

% mhn -cache 2
Then, you can re-process the original message. It will find the cached content:
% show 1
    ...
Use cached copy of content (size 77742 octets)
    ...? y

Cached Contents

Sending the same long message to several people at the same host can be inefficient. The host's disks will have multiple copies of the same file. If you send the long message with external parts, each person who wants the long parts will have to fetch the same data; if the data comes over a network, those multiple copies waste network bandwidth and CPU cycles on the host that serves the data. If you get a message with external body parts, and you want to view the same message several times, mhn would be fetching the same data over the network again and again; that's also a waste.

MH can cache (store) body parts in a directory on your computer's filesystem. Before it tries to fetch a body part over the network, mhn will check the caches to see if the body part has already been cached. The body part is cached by its Content-ID: header field; that field's value is supposed to be unique for every external body part in the world.

mhn maintains two caches:

When you read an external body part, mhn will ask you whether to use the cached value:
% show
  ...
Use cached copy of content 1.4 (size 117722 octets)
    in file /usr/local/lib/mhn-cache/<AAE28372.941024213426@xyz.com>? y
You may not always want to use the cached value. For example, it might be old. If it's important for the external data to be current, you can list the cache file (with ls -l) to see when it was created. If the cache is old, answer n; mhn will ask whether you want to fetch a fresh copy of the data.

The -rcache and -wcache switches control mhn caching. They define the policy, respectively, reading from and writing to the cache. There are four policies:

When you read a message with external body parts, you can save them in the public or private cache. Use the public cache if you think that someone else will want the data too; use your private cache if the message was intended for you.

Here's an example. You want to write part 1.4 of a message into your system's public cache, so you use -wcache public:

% mhn -wcache public -part 1.4
Date:    Mon, 24 Oct 1994 21:34:07 EST
	...
Retrieve jones.941024.au (content 1.4)
    using anonymous FTP from site ftp.xyz.com? y
	...
part    audio/basic           114K    Jones lecture on new polymers
Press <return> to show content...
	...
% ls -l /usr/local/lib/mhn-cache
total 116
-r -- r--r--  1 jerry      117722 Oct 25 09:20 <AAE28372.941024213426@xyz.com>
That ls -l command shows the cached file.

CAUTION:

The cached filenames use angle-bracket characters < and >. These are legal in filenames, but the shell uses them as redirection characters. If you want to use a cached file directly, without mhn, remember to put a pair of single quotes around the filename:

% play < '/usr/local/lib/mhn-cache/<AAE28372.941024213426@xyz.com>'
Otherwise, you might destroy files accidentally.

The Section Caching External Body Parts: mhn-cache and mhn-private-cache has more information about caches.

Showing Part of a MIME Message

By default, show calls its mhnproc (typically, mhn) to show all the parts of a MIME message. To see only one part of a message, you can call mhn directly and use its -part switch. (You don't need to use the mhn -show switch because it's the default.) If you don't give a message number, mhn will show the current message. For example, to show part 1.2 of message 33, type:
% mhn -part 1.2
mhn will show the message header followed by part 1.2.

When you show parts of messages, you may not want to see the message header. You may not want mhn to pause before it shows the part. The MH command version named showpart does all that.

Decoding Messages with mimecat

MH makes it easy to get the content of a non-MIME message into a pipe or file: just redirect the output of show with the shell's > or | operators. MIME messages aren't as easy to handle because they may need to be decoded and because you may want only one part of a multipart message. You can play some tricks with mhn-store- profile entries, but there's no good general-purpose way to get message contents to the standard output.

A more detailed explanation of the mimecat script (in this book's online archive) can help. It decodes a message -- or part of a message, with the -part switch -- and writes the content to standard output. For example, to decode the current message and search for the word taco in part 3.1 (the part numbers are the same as mhn -list uses):

% mimecat -part 3.1 | grep taco
Emma, that was a <italic>huge</italic> taco!  How did you finish it?
From there, you can pipe the decoded text to printers, processors like richtext (from the Metamail package), and so on:
% mimecat -part 3.1 | grep taco | richtext -e -f
Emma, that was a huge taco!  How did you finish it?
mimecat makes a temporary mhn profile with a series of profile entries for different content subtypes. It's kind of an ugly hack, but what can you do? :-)

Checking a MIME Message

MIME has safety features to get through message-eating gateways and transport. But messages can still be corrupted. For instance, your computer might have a bad disk. When a MIME message is sent, the MUA can add a Content-MD5: header field with a message checksum. A checksum is a string of letters and numbers that's a function of the content of a message (before the message is sent, of course). If you have a message with an Content-MD5: header field, mhn can compare the checksum to the message body. If they don't match, the message body has probably been changed.

The fastest way to check a message is mhn -list -check. If the checksum matches, \fImhn\fP won't complain. Otherwise, here's what will happen:

% mhn -list -check
 msg part  type/subtype              size description
  49       application/octet-stream   44K
mhn: content integrity suspect (digest mismatch) -- continuing
     (content application/octet-stream in message 49)
Here's information about adding an integrity check when you send a MIME message.

Alternatives to mhn

If you don't want show to give MIME messages to mhn, the default mhnproc, you have three choices:

[Table of Contents] [Index] [Previous: Showing and Printing Messages] [Next: Listing MIME Message Parts]


Last change $Date: 1996/06/06 15:13:15 $

This file is from the third edition of the book MH & xmh: Email for Users & Programmers, ISBN 1-56592-093-7, by Jerry Peek. Copyright 1991, 1992, 1995 by O'Reilly & Associates, Inc. This file is freely-available; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation. For more information, see the file copying.htm.

Suggestions are welcome: Jerry Peek <jerry@ora.com>

Questions or problems regarding this web site should be directed to Steve Gielda.
Copyright 1999 www.cotse.com.  All rights reserved.
Last modified: Friday April 02, 1999.