Having become increasingly dissatisfied with existing Mail User Agents I decided to write my own. My first attempt is a single Tcl/Tk script, less than 4000 lines in length, which you can download from the attachments below. This script uses the following packages, all of which are included with ActiveTcl:
package require sqlite3 package require mime package require pop3 package require smtp package require md4
(Added 2006-01-29:) The use of the pop3 package has been discontinued. The TCL pop3 package is unreliable. I wrote my own substitute that seems to work much better. The replacement pop3 package is included in the code archive that is attached to this wiki page.
The Problem To Be Solved
I have several complaints with existing MUAs. They generally have too much eye-candy for my taste. And I find them difficult to configure and use. But my biggest complaint is that they do not do a good job of managing historical mail archives.
I work on both fixed desktop systems and on my laptop. I would like to be able to check email from any system I am working on and yet have all my historical emails archived in the same database. To be more specific, I want to use my desktop in my office and my laptop when I'm travelling - I want to be able to check email from either machine - and I want to have access to the (gigabytes) of historical email from either system, potentially while disconnected from the internet.
I have not found a good way to do that with previous MUAs.
My other big complaint is that it is difficult to find historical email messages. I want the equivalent of a google search for my email archives. (Note: gmail is not really an option for me because I need disconnected operation - I need to read historical emails on airplanes and in other places where internet is not available.)
Design Philosophy
This program is deliberately simple. It speaks POP3 and SMTP only. All email messages are stored in a single SQLite database.
The use of a single-file SQLite database allows me to easily move the data from laptop to desktop and back again as I move around. Furthermore, each email message is saved with a universally unique hash so that even if the two databases diverge, they can be easily resynchornized with each other.
The use of an SQL database makes searching much easier as well. As each message is inserted into the database, every word of text in the message (and in the subject and several other header lines) is indexed. This makes searching for old emails using keywords very fast, even with thousands and thousands of messages consuming gigabytes of space.
The current user interface will certainly not win any design awards. It is ugly. But it is functional. And functionality should come first. Beauty is nice to have too, and I would welcome any patches or suggestions to improve the appearence. But functionality needs to be the overriding principle.
Everything is shown in a single window. I find the usual 3-window layout with folders on the left, a list of message on top, and the current message on the bottom right to be needlessly complex. This MUA does not support folders because to my mind, folder management is way too much trouble than it is worth. Instead of folders, all email is stored in an SQLite database with full text searching built in. This allows me to find historical email much more quickly.
Emails are displayed in as text/plain only. I deliberately do not support text/html because I find that most HTML email is spam anyway. Anybody with something interesting to say usually sends plain text.
Implementation Notes
Tcl/Tk was used with Tcllib to simplify development.
The source code was original composed in multiple *.tcl files which are concatenated (in no particular order) to generate the finished program.
You can interactively change the running Tcl/Tk code using the "Tcl Proc Editor" available under the "Debug" menu option. You can also interactively type Tcl/Tk commands using the built-in Tcl console, or interact with the SQLite database using the built-in SQL console. Both consoles are also available under the Debug menu.
Comments Welcomed
As of this writing, the program works well enough that I am now using it for all of my personal and business email. But there is, of course, much room for improvement. Suggestions and/or patches are welcomed.
User comments
2005-Sep-30
If I understood you correctly I should be able to install ActiveTcl and run this app straight
away?
I installed ActiveTcl 8.4.11.1 on Windows2000 but sqlite3 package is missing. Is there anything
special that needs to be done to install sqlite3 from source so that ActiveTcl can find it?
(drh replies:) SQLite was added to ActiveTcl recently. Maybe
it didn't get into version 8.4.11. Try the new 8.5 beta.
I tried 8.5.0.0 beta3 and still no luck. There is no mention of sqlite in any documentation that came
with ActiveTcl and searching for "sqlite" in Tcl install dir returned no files. Can you confirm
that sqlite really is included with ActiveTcl for Linux? Or perhaps sqlite didn't make it into
beta3 but will in beta4 or whatever it will be called?
(bas) I have sqlite as part of AS Tcl on windows 8.4.11, and Linux 8.4.12. Are you sure you are using the AS tclsh, and not one that was installed from the Linux distro?
When all that was sorted out
2005-Sep-30
I like the idea of coming up with a standard mail database schema that any mail client could then
interact with. For example, I might write a mail client in PHP or REALbasic, but the database itself will be
the same for both. So, perhaps the really cool thing about this project is the database that gets created.
2006-Jan-27
Does/could this support SSL authentication for POP3 access? The only POP
account I have is a gmail one which requires SSL access. How hard would this
be to add -- just a question of package require tls and a couple of lines, or
a more substantial change? -- Neil Madden
drh replies: This is probably just a "package require tls" plus a few lines of
code at the point where it sets up the TCP/IP connection to the POP3
mail server, in the fetch.tcl source file. But I have never used TLS nor
POP3 over SSL so I really don't know.
2006-Jan-27
I actually started writing a outlook clone based on Sqlite, for some of the same reasons that you describe (although I wanted calendar too) (bas).
2006-Jan-28
I'm the original poster with windows problem from 2005-Sep-30 so here's the
update on getting this MUA to work on windows.
SQLite3 was included with ActiveTcl 8.4.11.2 and is included in newer
releases of 8.4. But 8.5 (beta4 ATM) still doesn't come with SQLite3, nor the
mime package (perhaps some other packages are missing too).
See ExmuaOnWindows for more windows related info.
Is there a way to have messages left on server after retrival rather then deleting them?
btw. Very usefull little MUA, thanks drh!
2006-Mar-28
Andreas Kupries - ActiveTcl 8.5 (a beta distribution) does not come with any packages at all, except some 8.5 specific stuff. It is meant to be used in conjunction with an ActiveTcl 8.4.x, whose packages it can use. Simply install into the same directory as the AT 8.4.
2006-Aug-01
Any chance of a screenshot?
2006-Oct-17
Any chance that we'll see the full-text-enabled version that you showed at the conference? Pretty please?
2007-Jan-16 (linebreak) Any chance of submitting specific bug reports on the tcllib pop3 problems, or at least a feature enhancment proposing to replace/augment tcllib with your code?
Attachments:
- ex-mua.zip 31441 bytes added by drh on 2005-Sep-30 13:48:08 UTC.
- ex-mua-2006-01-27.zip 43927 bytes added by drh on 2006-Jan-27 13:57:14 UTC.
The source code as of 2006-01-29