APPENDIX C

Telnet lets you mimic the role of a browser. Telnet can make a connection to any port on a computer and, once a connection is made, any characters you type are sent to the remote port, and any text returned by the remote port are typed to the display. Thus, by using telnet to connect to a remote Web server, you can type HTTP request header fields (and data) into the telnet client, mimicking the behavior of a browser, and then see the data sent back by the remote server. This procedure is described in Section C.2.

Monitor lets you monitor the data sent between a browser and a remote Web server. It does this by acting as a simpleminded proxy server—using suitably written URLs, the Web browser connects (and sends data) to monitor, which forwards the data to a designated Web server, but also copies this information to a designated file. Subsequently, when the remote Web server returns data to monitor, monitor forwards this data to the browser and also copies the data to the designated file. As a result, the data file retains a record of all the data sent between the browser and the remote server.

The programs listen, backtalk, and monitor are available for downloading, as described in Section C.4, both as source code (in C) and as compiled executables for a number of different platforms. The telnet program is a standard tool on all Unix, Windows 95/98/2000/NT, and Macintosh systems.

C.1 Listen and Backtalk to Mimic a Server

Listen is a simple program that listens at a TCP/IP port and prints to the display any characters received at the port. Thus, by pointing a browser at the port being monitored by listen, this program can be used to observe the data that Web browsers send when making an HTTP request. Similar to listen, backtalk listens at a TCP/IP port and prints to the display any characters received at the port. However, backtalk also lets the user type characters (enter data) at the console, with these characters being sent back to the remote program that is talking to that port. Thus, backtalk can act like a simpleminded HTTP server—or rather, the user and backtalk can act like a simpleminded server, because the user has to type, by hand, the data to be returned to the browser.

To run either program, simply type the program name at the command prompt. Listen (or backtalk) then prints the port number it is listening at and goes silent. Here is a typical example (the typed command is shown in boldface):

At this point, any data sent to port 2055 on this computer will be printed on the screen just below the string listening on 2055. For example, if the preceding command were executed on the machine with domain name www.testing.org, then a browser could be pointed to the running listen program simply by accessing the URL

where test_data is added simply to place some data in the location portion of the URL. Note that this could be done in several ways:

• By typing the URL directly into a browser’s location or address window
• By creating an X/HTML document containing hypertext links that point to the correct URL
• By writing an X/HTML document containing a form that references the correct URL

When a browser accesses the indicated URL, it sends an HTTP message to the indicated port—this message (the HTTP header plus any subsequent data) is then displayed at the console, as illustrated in Figure C.1. Thus, you can use these programs to observe essentially all forms of interaction between a browser and a server.

With backtalk, any characters typed by the user are sent to the remote client attached to this port—the italicized boldface text in Figure C.1 is an example of the type of text a person might type. With backtalk, the character Ctrl-D (indicated in Figure C.1 by the symbol ^D) tells backtalk to close the connection to the browser. This informs the browser that all data has arrived and causes it to render the page. To stop backtalk (or listen), you simply type Ctrl-C.

Source code and executable versions of listen and backtalk can be downloaded from the book’s supporting Web site, as described in Section C.4.

Back to top

C.2 Using Telnet to Mimic a Browser

You can use a telnet client to play the role of a Web browser. For example, on a Unix system, typing the command:

makes a connection to port 80 on the machine www.testserver.org (this can be similarly done using Windows or Macintosh telnet clients, simply by entering the server’s host name and the desired port number in the appropriate boxes). If there is a Web server running on this port, any characters typed by the user are sent to the Web server, which interprets them as commands coming from a remote client (such as a Web browser). After a complete request is sent, the remote server will send data back, which the telnet client simply types to the screen. An example is shown in Figure C.2—here, the HTTP message information typed in by the user consists solely of the string GET /foo.html HTTP/1.0, followed by a single blank line (noted here by the symbol <CRLF> that marks the end of the header. The data that follows (in normal font) is that returned by the Web server.

The last line, Connection closed by foreign host, is produced by the telnet program itself and indicates that the connection between the telnet client and the server has been broken.

If you want to save the information into a file, as opposed to seeing it on the display, you can type the command:

telnet www.testserver.org 80 > filename

which saves everything the user types — and everything the server sends back — in the file named filename. However, note that this also means you won’t be able to see what you type, as this text is also stored in the file, as opposed to being displayed on the screen.

Back to top

C.3 Using Monitor to Monitor Client-Server Transactions

Monitor lets you monitor the data sent between a browser and a remote Web server. It does this by acting as a simpleminded proxy server. You access URLs that connect the Web browser (and sends data) to monitor, which, in turn, forwards the data to a designated remote server. At the same time, monitor copies this information to a designated file, or directly types it to the display. Then, when the remote Web server returns data to monitor, monitor forwards this data to the browser and copies the data to the designated file. As a result, the file retains a record of all the data sent between the browser and the remote server.

When you run monitor, you can specify a number of options to define the remote server (and port) to which it connects, the name of the file to which data should be stored, and the local port to which the browser should connect. The possible options (with parameters in italics) are

• -o filename specifies a file into which the monitored data should be saved. If filename exists, then the contents are erased when monitor starts. If this option is absent, then the monitored data is displayed on the terminal. Note that this latter option can cause problems with the terminal if binary data (such as image files) is being transferred.
• -s remote-server is a mandatory parameter specifying the domain name of the remote server to contact. At present, numeric IP addresses are not supported.
• -p remote-port is an optional parameter specifying the port number to contact on the remote server. If this is absent, then monitor assumes a default value of 80.
• -l loc-port specifies a local port (on the machine where monitor is running) by which monitor will communicate with the remote browser. Note that this number must be greater than 1024 if monitor is run by nonprivileged users. If not specified, then monitor chooses an available port number. In either case, monitor displays the selected port number to the user.
• -v specifies verbose output, useful for debugging purposes. Verbose output is sent to the display (stderr), and is not stored in filename.

An example is

monitor -o test.dat -s www.utoronto.ca

which means that monitor will connect to the HTTP server running on port 80 at the machine www.utoronto.ca, and that the session will be archived in the file test.dat.

When monitor starts up, it initially types data to the terminal giving the location at which monitor is running (so you know where to point the browser) and the name of the remote server to which monitor connects. Example output is

The first line tells where monitor is running, and the second says that monitor connects onto the Web server running on port 80 at www.utoronto.ca. Now, any URLs starting with the string

will connect a browser (or any other client) to monitor, and via monitor to the server running on www.utoronto.ca.

The data stored in the file test.dat will look something like that shown in Figure C.3. The lines --- BEGIN DATA FROM CLIENT --- and so on (shown in boldface) are not part of the actual HTTP transactions, but are added by monitor to show when the browser and remote server began (and ended) sending data to each other. The lines ... omitted data ... (in boldface italics) indicate portions of the data that were omitted to make this figure smaller. The sequence ^M at the end of each text line is an artifact of the editor used to display the file and denotes the carriage return character—each text line is actually ended by a carriage return line feed (CRLF) character pair.

Note, too, near the bottom of the file, that the second piece of data sent to the browser is a binary (GIF image) file. This produces odd sequences of characters on a text editor, and the file may look quite different, depending on the editor you use. Indeed, if your files contain binary data, you may need to configure your editor to acceptably display the data.

Back to top

C.4 Where to Obtain the Programs

Listen and backtalk were originally written by Norman Wilson, formerly of the Canadian Centre for Theoretical Astrophysics at the University of Toronto, and subsequently were slightly modified by the author. Monitor was written by the author, using the code of backtalk as a starting point. These programs were written for Unix systems, and they run under most variants of Unix. Unfortunately, they have not been ported to Windows 95/NT or MacOS. The programs are available, both as source code and as compiled executables, for SGI IRIX (6.x), Solaris (2.5), and Linux (2.2) at http://www.iangraham.org/books/xhtml2/appc/.