Web OS: POP3, IMAP, and SMTP

Abstract

The purpose of this project is to demonstrate the possibility of extending HTTP's functionality. Since HTTP is used as a means to indirectly communicate with individuals around the world, it seems logical to incorporate direct communication as well. Adding email handling capability to HTTP to existing protocols allows us to bypass the need for an external mailing interface and accomplish our goal. Rather than implementing a client or server based server, we have taken a "middle-man" approach by using servlets. The servlets will accept requests from the client WWW GUI and forward them to the desired server on the backend. The backend, in turn, attempts to perform the desired request and responds back to the GUI with the appropriate status.

Introduction

The proposed JSBMS (Java Servlet Based Mailing System) makes use of the POP3, IMAP, and SMTP protocols. POP3 and IMAP are protocols used to handle, view, and organize received email. SMTP is used to send out email to a desired recipient. Our JSBMS communicates with these services through HTTP extentions by means of servlets which are located on the webserver. These servlets
are invoked by the URLs which we designed. End users can access these URLs from any browser connected to the internet and perform various email functions. There are no custom configuration required on the users end for these services.

The following are links connected to the various aspect of this project:

Related Work
Background
Architecture
Program Documentation
Measurements
Task List
Advantages/Disadvantages
References

Related Work

There are currently many applications for viewing and sending Email messages (eg. Eudora, Netscape, Elm, etc..), and each of theses use one or more internet protocols (including POP3, IMAP, and SMTP). However, their means of communicating with the server using theses protocols will vary from application (although most will be direct). So in essence, any Email handling application outthere can be considered Related Work.

Background

SMTP

(Simple Mail Transfer Protocol) is used for transportation of email from one host to another. The protocol was designed with openess in mind such that there is no security in verifying the sender. It also avoids traffic by allowing multiple receipients with only one copy of the data being sent from one host to another.

POP3

(Post Office Protocol 3) is the protocol that allows email retrieval and deletion/undelete.

IMAP

(Interner Message Access Protocol, version 4rev1) is another protocol that handles email retrieval, but with increased manipulation capability. IMAP allows the manipulation of remote messages mailboxes, very much as if these mailboxes were accessed locally. IMAP's functionality includes creation, deletion, and the renaming of mailboxes, and the checking and setting of email messages statuses (FLAGS). The messages themselves can be partially retrieved and searched given various parameters.

Java Servlets

The use of Java servlets to produce dynamic output on the web is an alternative to: CGI scripts, Netscape's server push/client pull, and server-side includes (SSI). Its use is similar to SSI in that it is also server-side parsed. However, because it operates in the Java Runtime Environment's "sandbox", it offers greater security. Some of the downloaded servlet's restrictions include:

Its inability to access the local file system
It cannot make any other network connections other than the machine where it's connected from

It's similar to CGI scripts in the sense that it's full blown script that includes certain functionalities (in our case, the functionalities of the 3 protocols). It is different from CGI scripts in that

It does not need to occupy a port by itself and can only be invoked by accessing that port. Instead it'll be accessed a similar way as an HTML document, by using the port of the web server.
Added security as mentioned above.
A standard way of accessing the request from the HTML page and a standard way of sending responses to the HTML document.

Servlets are considered "server extentions", because once made a standard, they can be included in the distribution of the web server and made into known services such as POP3, IMAP, and/or SMTP.

Architecture

SMTP Servlets

This part of the project is comprised of 1 servlets and 1 HTML page.

SMTP.java: This servlet is called by the HTML page and accepts the following parameters:

- email address: This would be the input of email address(es) (this can be a series delimited by ,)
- cc: This would be input of email address(es) to also be receipients ( this can also be a series delimited by ,)
- subject: This would be the subject of the email.
- data: This would be the body of the email message.

The servlet establishes an SMTP connection to cs.columbia.edu and sends the message to all receipients from the email address and cc list.

POP3 Servlets

This part of the project is comprised of 3 servlets and 1 HTML page as a client to demonstrate the functionalities of the servlets. The 3 servlets are:

POP.java: This servlet is called by the HTML page and accepts 3 parameters from the page: username, password, and the host where the user wishes to check email on. POP.java will log onto the server and displays a list of emails that the user has, and some primary information that's associated with the email such as subject, date, from, and to fields. There will be a link to each of the email, and these links calls the RetrieveMail.java (to be explained next). There's also checkboxes where the user can choose to delete/undelete one of more of his emails, and the Submit botton associated with the input form calls the Update.java.
RetrieveMail.java: This servlet will take the username, password, and hostname parameters from POP.java via cookies and log on to the server. It takes an addition parameter from POP.java that tells it which email to display. It will then displays the appropriate email to the browser.
Update.java: This servlet will also takes the username, password, and hostname parameters from POP.java via cookies. Additional parameters include the emails that the user wishes to delete. Update.java then logs onto the server and deletes the appropriate emails, and displays the result back to the browser.

IMAP Servlets

This part of the project is comprised of 10 servlets and 1 html page (the logon screen). I've used Java, HTML and made use of cookies for these servlets. Each servlet incorporates the main action of the user, as well as the actions necessary to get back to the appropriate screen (see IMAP extension to HTTP in Program Documentation).

The 10 servlets are: ImapClean, ImapCopyBox, ImapCreateBox, ImapDel, ImapDelBox, ImapListBox, ImapLogon, ImapRead, ImapRenBox, and ImapView. Each of the modules will establish a new connection with server using the logon information in the cookies (which are set by the ImapLogon servlet). Additionally, Each of the modules will let the user know whether his/her action was done successfully or not. The following are brief descriptions of theses modules (the desired action of the user is the only one explained in the following modules).

ImapClean accepts 1 parameter: mailbox. It will logon, go to the chosen mailbox and send the expunge command to remove all email messages within containing the /deleted flag. It will then proceed to display the updated email list screen.

ImapCopyBox accepts 3 parameters: mailbox, tomailbox, and email. It will Logon, go to the chosen mailbox and email, and copy this email to the chosen destination tomailbox by using the copy command. It will then proceed to display the read email screen.

ImapCreateBox accepts 1 parameter: mailbox. It will logon, and send the create command with the mailbox parameter used and attempt to create this mailbox. It will then proceed to display the updated mailboxes view screen.

ImapDel accepts 2 parameters: mailbox and email. It will logon, go to the chosen mailbox and send the store w/ flag \deleted command on the email to be deleted and thus set this email for deletion. It will then proceed to display the email list screen.

ImapDelBox accepts 1 parameter: mailbox. It will logon, and send the delete command on the chosen mailbox in an attempt to delete that mailbox. It will then proceed to display the updated mailboxes view screen.

ImapListBox takes no parameters. It will logon, and send the list W/ \NoInferiors flag to display the mailboxes view screen.

ImapLogon accepts 3 parameters: username, password, and hostname. It will attempt to logon to the desired server using the login command and record the logon information on cookies to be used by the other modules. It will then go the inbox via the select inbox command and look for the amount of emails with the \recent flag and display that information on the intermediate logon screen.

ImapRead accepts 2 parameters: mailbox and email. It will logon, go to the chosen mailbox and send a fetch (body[header.fields (from date subject)] body[text]) command on the chosen email, thus getting the contents of the email and displaying that information on the read email screen.

ImapRenBox accepts 2 parameters: mailbox and tomailbox. It will logon, and attempt to rename the chosen mailbox to the tomailbox name using the rename command. It will then proceed to display the updated mailboxes view screen.

ImapView accepts 1 parameter: mailbox. It will logon, go to the chosen mailbox using the select command, do a fetch (body[header.fields (from date subject)]) command on all the messages found in this mailbox to get header information on all the emails found within this mailbox. It will then display this information to the email list screen.

Program Documentation

SMTP extension to HTTP

The webpage allows users to write and send an email message to anyone on the internet. A status page is displayed to show whether the email was sent via SMTP to cs.columbia.edu with or without any problems.

POP3 extension to HTTP

This part will allow the user to log onto his POP server and view his emails via an HTTP browser. The initial logon page contains 3 lines of input: username, password, and hostname. If the logon succeeds, the user will see a page that displays how many emails he has and some information associated with each of the email such as subject, date, from, and to fields. If the user misses any of the field or inputs an incorrect value, a servlet exception is caught and the error message is displayed to the user.

If the user has emails on the server, there will be a link associated with each email and when the user clicks, that email will be displayed.

There's also one checkbox associated with each email where the user can delete or undelete his email. When the user decides which emails to delete, he can click on the "update" botton to delete those emails. A confirmation page will then be displayed which tells the user how many emails are deleted and how many are kept on the server.

IMAP extension to HTTP

The main purpose of this project is to show how various protocols can be incorporated by HTTP and not to represent a full-size client. Due to the myriad possible commands, sub-commands, and mostly options, I've opted to only extend some of IMAP's functionality to HTTP in order to achieve our main objective.

In order to use this functionality one must have:

A web browser that understands java and cookies (such as Netscape)
An account on a server which uses the IMAP version 4rev1 on port 143

The functionality of IMAP included in this part of the project are: viewing, creation, deletion, and renaming of mailboxes, checking new mail, viewing, deleting, and moving mail. The general idea is to allow a user to connect to the desired server, and log on (you must have an email account on this server) to its IMAP port (the server must be running IMAP version 4rev1 on it) and proceed with mail and mailbox handling.

The servlets themselves are comprised of Java and HTML code, with some usage of cookies. There are basically 5 screen modes in this setup: A logon screen, An intermediate logon screen, a mailboxes view screen, an email list screen, and a read email screen.

The logon screen is a simple html form screen that requires the user to enter a username, a (hidden) password, and a destination server. The user then submits this information and proceeds to the intermediate logon screen.

The intermediate logon screen pops up if a connection was successfully established, and reports on the number of new messages since the user's last logon. You can only proceed to the mailboxes view screen from here.

The mailboxes view screen contains a list of all the mailboxes found in this user's account. From here you can delete, rename, or create mailboxes. You can also enter one of these mailboxes to bring you to the email list screen.

The email list screen contains a list of all emails (including some headers) stored in the chosen mailbox. From here you can set a message for deletion or permanently delete all messages set for deletion. You can choose to read a particular email which will bring you to the read email screen. Finally, you can choose to leave this mailbox and go back to the mailbox view screen.

The read email screen contains the contents of the chosen email message. You can make a copy of this email message to another mailbox from here. Or you can go back to the email list screen from the same mailbox.

The main problem with the current setup is the general slowness of every action. A new connection is required for every actions the user makes... The connection is broken upon completion of every action so as to free up the resources on the network. This is just one possible implementation, albeit not necessarely the best. I used this method because of running into difficulties in attempting to keep the connection open from servlet to servlet.

Measurements

As the purpose of our project is to demonstrate functionality, and because of the unknowns between the two sites' connections and conditions, measurements are meaningless here.

Task List

POP3	Joan Shen
IMAP	Pascal Guillot
SMTP	Siu Kwan Lo

Advantages/Disadvantages

Advantage:

Browser independent: can be accessed from any kind of browser, lynx or netscape, and not limited by browser specific features.
Extends server side functionality: the servlets can be distributed with the server and become a standarized service.
Standarization of accessing: access by connecting to Java Web Server port.
Added security over cgi scripts: utilizes Java virtual machine's build-in security features.
No effect of server or client: since this system acts as a "middle man", no applications (other than a browser) or special configurations are needed on the client side, nor are they needed on the server side (other than the protocols being active on the server).

Disadvantage:

Only available through Java Web Server at this point - Since our servlets api functions are specific to the JavaWebServer developed by Sun Microsystem, our servlets were not directly crossed platformed to Webservers developed by other vendors.
Congestion issue - If these servlets are widely used, congestion will occur around the "middle man" location, since all requests are passed through it.
Heavier workload on the JavaWebServer - Since each request will invoke a servlet, if there are many users accessing the email services at the same time, there would be many concurrent instances of the servlets running at the same time which may cause a heavy CPU workload on the JavaWebServer.
Unable to process same email operation with same destination simultaneously - If two users perform the same email service (SMTP, POP3, or IMAP) to the same destination at the same time, one user will fail. This is due to limitations of sockets. A socket connection must be unique by the (source address, port, dest address). In this case, both will have common source address (JavaWebServer), common port (ie. IMAP) and common destination address.

References

RFC 2060 - Internet Message Access Protocol (IMAP) - version 4rev1
By M. Crispin, Dec 96

1939 Post Office Protocol - Version 3. J. Myers & M. Rose. May 1996.
     (Format: TXT=47018 bytes) (Obsoletes RFC1725) (Updated by RFC1957)
     (Also STD0053) (Status: STANDARD)

0821 Simple Mail Transfer Protocol. J. Postel. Aug-01-1982. (Format:
     TXT=124482 bytes) (Obsoletes RFC0788) (Also STD0010) (Status:
     STANDARD)

JAVA in a nutshell - a Desktop Quick Reference
By David Flanagan, O'Reilly & Associates, Inc. May 97,  (2nd edition)

JAVA web server 1.1 HTML doc. (developer-servlets)
Instant Netscape Dynamic HTML (programmer's reference)
By Alex Homer and Chris Ullman, Wrox Press, Ltd. 97, (NC 4 edition)