How modems work. (part 1) (Column)
by Mark Minasi

If you spend much time on computer data services such as CompuServe, GEnie, America Online, or, for that matter, any of the thousands of private bulletin board systems (BBSs), you've had to set up some kind of communications software. Are terms such as MNP, XMODEM, YMODEM, V.32, and the like confusing you? Then read on.

It may help to start the discussion with a statement of the overall purpose of communications. The point of communications is to get bytes of data from one point to another as quickly and reliably as possible, and that's where this whole mess started. So when you hear a new communications buzzword, ask, "How does this help me get data from point X to point Y?"

In the late seventies and the early eighties, the first BBSs appeared. Modems then typically transmitted at 300 bps, and the kind of data that was being transferred was usually text: either messages or text listings of BASIC programs.

In those early days, you'd transfer a file by setting your computer to capture mode, which meant that it would copy every incoming byte to a disk file. Then you'd tell the distant computer to list the file you wanted to receive, as if your only desire were to read the file as it scrolled by on the screen. The capture feature saved the file to disk as it appeared on your screen. When it was finished listing, you'd close the file, and you'd have your BASIC program transferred. There would be some extra trash above and below the file, but you'd just load the file into a text editor and trim that off.

Now and then, however, the transfer would become garbled because of phone line noise. In that case, you'd see the garbled characters, and you'd tell the remote system to resend the file.

Hopefully, it would come through OK the second time. If not, you might conclude that you'd just happened to get a noisy line today, and you'd just hang up and dial again, hoping for a better connection. This was called phone line roulette.

In this case, there were error-detection and error-correction mechanisms. The detection mechanism was you, the human operator, noticing that the incoming data looked wrong. The correction mechanism was also you, requesting a resend. This crude system worked for two reasons.

First, you were transmitting at 300 bps. (Never say baud; no matter what you've been told, it doesn't stand for bits per second.) The lower the data rate, the lower the error rate. Bad bits appear on a normal phone line once in a blue moon at 300 bps; they can appear every minute or so at 9600 bps. Error correction was less necessary because errors just plain didn't occur.

Second, the kind of data that you were transmitting could stand up to an error or two--it was mainly English text. If you saw an incoming line of BASIC that read PRONT 'Hello,' you knew that PRONT was really PRINT garbled. The built-in predictability of English helped the error-correction process.

As time went on, 1200-bps modems became more popular. BASIC listings were replaced on bulletin boards by COM and EXE files, files that don't mean anything when viewed by the human eye.

So a BBS operator named Ward Christensen in the Chicago area invented a simple method for tranferring data, a method that transfers data while simultaneously finding and correcting data garbled by transmission noise. He called the method XMODEM, and it looks something like this.

Say station S (the sender) is sending a 400-byte file to station R (the receiver). Both sides of the conversation must be running a program that manages a transfer according to the rules of the transfer. You do that whenever you tell your system to transfer with XMODEM. This insistence on both sides agreeing is the linchpin of communications protocols. Here's how the XMODEM technique works. (Because both sides are controlled by a computer, people don't have to worry about this stuff, except for issuing the commands to S and R to get going.)

First, the sender must ensure that the receiver is ready to receive the data. It does this by waiting for a particular one-byte signal from the receiver, an ASCII code 21, also known as NAK (negative acknowledgment).

There's no significance to this character--it was just picked at random. When R says NAK to S, R is saying to go ahead and start transferring. If you wanted to, you could generate a NAK from your keyboard by typing Ctrl-U.

S is going to send the data to R in 128-byte blocks. Before the 128 bytes of each block, however, it first sends an SOH (Start Of Header) signal, which is just an ASCII 1, something you could generate with a Ctrl-A. You've seen ASCII 1 before; it's the smiley face that shows up when you type a COM or EXE file to the screen.

After the SOH, S sends another byte, a block number byte. The value of the byte is 1 for the 1st block, 2 for the 2nd, and so on. As bytes can only hold values up to 255, the value wraps around after 255. The 256th block would be numbered 0, the 257th would be numbered 1, and so on.

For the sake of redundancy, the third byte repeats the block number -- but in another way. S subtracts the block number from 255! The second block number for our first block is, then, 255 minus 1, or 254. Then S sends the 128 bytes of data. The block's almost finished, but there's one more byte to go.

Recall that the whole purpose of this exercise is to be able to detect errors. S now adds something called a checksum that will allow R to detect errors. The sender S takes all 128 bytes of the data block and treats them as if they weren't ASCII characters or binary data, but just a string of 128 numbers. It then adds up the 128 numbers to get a sum. That sum is then divided by 256. The remainder, believe it or not, is the checksum. (This makes better sense when you do it in machine language; Ward knew what he was doing.)

Here's an example. Suppose we had a block of not 128 characters, but, for ease of use, 11 characters. Let's use my name in uppercase letters: MARK MINASI. Remember that it's 11 characters, not 10, because of the space between the first and last names. The ASCII code for M is 77, A is 65, R is 82, K is 75, a space is 32, M is 77, I is 73, N is 78, A is 65, S is 83, and I is 73. Add them up, and you get a total of 780. Divide 780 by 256, and you get 3 with a remainder of 12. So 12 is the checksum.

S has now sent the first block. It was 132 bytes long--3 bytes in the header, 128 bytes of data, and 1 byte of checksum. Now let's see what R does with the block.

First, it looks at the block number to make sure that it makes sense--if block 27 arrives right after block 10, something's wrong. Assuming there's no trouble, it next examines 128 bytes of data, ignoring the checksum for the moment. Using the 128 bytes of data, it computes its own checksum in the same way that the sender did. Now R examines the checksum that S sent to be sure it's identical to its own.

This is the error-detection part--the checksums must be the same. If they are, R signals S that all is well by sending a single byte, an ACK character. ACK (positive acknowledgment) is ASCII code 6, or Ctrl-F. If the checksums don't match, then the error-correction part comes in, and R sends a NAK code. When S receives the negative acknowledgment, it resends the data block. R checks the resent data block, and, hopefully, all is well. If not, S just keeps resending and R just keeps NAKing until they reach max NAKs, at which point the transfer is terminated. Assuming the first block has been received correctly, S then sends the next 128 bytes of our 400-byte file. Once that's been acknowledged, there's a third block, and finally there are only 16 bytes left. The 16 bytes are filled out with either end-of-file characters or null characters and then sent. Once that has been acknowledged by R, S says, "That's all, folks," by sending an EOT (End Of Transmission) character, which is ASCII 4, or Ctrl-D. A final ACK from R ends things.

A couple of things may be nagging at you about this protocol. First, why send the block number twice, and second, why subtract from 255? The first question is easier: The block number is sent twice simply for insurance. Remember that we don't trust the phone lines. This is a fairly common practice in communications and, for that matter, in real life. If you've ever heard an announcement like "Mr. Jones, Mr. Jones, please pick up the white courtesy phone" in an airport, you might ask yourself why the announcer said Mr. Jone's name twice--was he supposed to pick up the phone twice? No, of course not. The announcer just wanted to be sure that Mr. Jones heard his name.

But why subtract from 255? I thought I'd try asking the source himself, so I dropped a note on CompuServe to Ward Christensen. His answers were very helpful. "Why not?" he responded with a grin. "Actually, if a line glitch occurred that changed a particular block number to something else, there's a chance it might do two at a time." Thus, anything that affected, say, all the 1 bits would damage both copies of a block number, if both block numbers in XMODEM were equal. By subtracting from 255, there's a side effect of converting all the 1s to 0s and vice versa. Taking a binary number and converting its 1s to 0s and 0s to 1s is called taking its complement.

Ward observed that, in the case of the systematic error that always damages 0s or 1s, "the complement is not likely to get similarly zapped. Also, the block number isn't in the checksum, because I wanted it separately checkable." This provides an internal block check mechanism--but that's not the only reason for this method. Ward explained that one of the popular processors of the time was the 8080 (no, that's not the 8088; the 8080 predates the 8088 by about eight years). It had a built-in instruction that made using the complement for checking a simple matter.

That's XMODEM in some detail. We needed that detail to understand how you can speed up your file transfers without buying a new modem or new software. See you next month.