C/C++/VC++

Applied Cryptography, Protocols, Algorthms, Source Code in C
-Bruce Schneier-

C++ Neural Networks and Fuzzy Logic
-Valluru B. Rao-

Data Compression
-Mark Nelson, Jean-Loup Gailly-

Digital Imaging in C and the World Wide Web
-David Schwaderer-

Learn Encryption Techniques with BASIC and C++
-Gil Held-


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Preface

There are two kinds of cryptography in this world: cryptography that will stop your kid sister from reading your files, and cryptography that will stop major governments from reading your files. This book is about the latter.

If I take a letter, lock it in a safe, hide the safe somewhere in New York, then tell you to read the letter, that's not security. That's obscurity. On the other hand, if I take a letter and lock it in a safe, and then give you the safe along with the design specifications of the safe and a hundred identical safes with their combinations so that you and the world's best safecrackers can study the locking mechanism—and you still can't open the safe and read the letter—that's security.

For many years, this sort of cryptography was the exclusive domain of the military. The United States' National Security Agency (NSA), and its counterparts in the former Soviet Union, England, France, Israel, and elsewhere, have spent billions of dollars in the very serious game of securing their own communications while trying to break everyone else's. Private individuals, with far less expertise and budget, have been powerless to protect their own privacy against these governments.

During the last 20 years, public academic research in cryptography has exploded. While classical cryptography has been long used by ordinary citizens, computer cryptography was the exclusive domain of the world's militaries since World War II. Today, state–of–the–art computer cryptography is practiced outside the secured walls of the military agencies. The layperson can now employ security practices that can protect against the most powerful of adversaries—security that may protect against military agencies for years to come.

Do average people really need this kind of security? Yes. They may be planning a political campaign, discussing taxes, or having an illicit affair. They may be designing a new product, discussing a marketing strategy, or planning a hostile business takeover. Or they may be living in a country that does not respect the rights of privacy of its citizens. They may be doing something that they feel shouldn't be illegal, but is. For whatever reason, the data and communications are personal, private, and no one else's business.

This book is being published in a tumultuous time. In 1994, the Clinton administration approved the Escrowed Encryption Standard (including the Clipper chip and Fortezza card) and signed the Digital Telephony bill into law. Both of these initiatives try to ensure the government's ability to conduct electronic surveillance.

Some dangerously Orwellian assumptions are at work here: that the government has the right to listen to private communications, and that there is something wrong with a private citizen trying to keep a secret from the government. Law enforcement has always been able to conduct court–authorized surveillance if possible, but this is the first time that the people have been forced to take active measures to make themselves available for surveillance. These initiatives are not simply government proposals in some obscure area; they are preemptive and unilateral attempts to usurp powers that previously belonged to the people.

Clipper and Digital Telephony do not protect privacy; they force individuals to unconditionally trust that the government will respect their privacy. The same law enforcement authorities who illegally tapped Martin Luther King Jr.'s phones can easily tap a phone protected with Clipper. In the recent past, local police authorities have either been charged criminally or sued civilly in numerous jurisdictions—Maryland, Connecticut, Vermont, Georgia, Missouri, and Nevada—for conducting illegal wiretaps. It's a poor idea to deploy a technology that could some day facilitate a police state.

The lesson here is that it is insufficient to protect ourselves with laws; we need to protect ourselves with mathematics. Encryption is too important to be left solely to governments.

This book gives you the tools you need to protect your own privacy; cryptography products may be declared illegal, but the information will never be.

How to Read This Book

I wrote Applied Cryptography to be both a lively introduction to the field of cryptography and a comprehensive reference. I have tried to keep the text readable without sacrificing accuracy. This book is not intended to be a mathematical text. Although I have not deliberately given any false information, I do play fast and loose with theory. For those interested in formalism, there are copious references to the academic literature.

Chapter 1 introduces cryptography, defines many terms, and briefly discusses precomputer cryptography.

Chapters 2 through 6 (Part I) describe cryptographic protocols: what people can do with cryptography. The protocols range from the simple (sending encrypted messages from one person to another) to the complex (flipping a coin over the telephone) to the esoteric (secure and anonymous digital money exchange). Some of these protocols are obvious; others are almost amazing. Cryptography can solve a lot of problems that most people never realized it could.

Chapters 7 through 10 (Part II) discuss cryptographic techniques. All four chapters in this section are important for even the most basic uses of cryptography. Chapters 7 and 8 are about keys: how long a key should be in order to be secure, how to generate keys, how to store keys, how to dispose of keys, and so on. Key management is the hardest part of cryptography and often the Achilles' heel of an otherwise secure system. Chapter 9 discusses different ways of using cryptographic algorithms, and Chapter 10 gives the odds and ends of algorithms: how to choose, implement, and use algorithms.

Chapters 11 through 23 (Part III) list algorithms. Chapter 11 provides the mathematical background. This chapter is only required if you are interested in public–key algorithms. If you just want to implement DES (or something similar), you can skip ahead. Chapter 12 discusses DES: the algorithm, its history, its security, and some variants. Chapters 13, 14, and 15 discuss other block algorithms; if you want something more secure than DES, skip to the section on IDEA and triple–DES. If you want to read about a bunch of algorithms, some of which may be more secure than DES, read the whole chapter. Chapters 16 and 17 discuss stream algorithms. Chapter 18 focuses on one–way hash functions; MD5 and SHA are the most common, although I discuss many more. Chapter 19 discusses public–key encryption algorithms, Chapter 20 discusses public–key digital signature algorithms, Chapter 21 discusses public–key identification algorithms, and Chapter 22 discusses public–key key exchange algorithms. The important algorithms are RSA, DSA, Fiat–Shamir, and Diffie–Hellman, respectively. Chapter 23 has more esoteric public–key algorithms and protocols; the math in this chapter is quite complicated, so wear your seat belt.

Chapters 24 and 25 (Part IV) turn to the real world of cryptography. Chapter 24 discusses some of the current implementations of these algorithms and protocols, while Chapter 25 touches on some of the political issues surrounding cryptography. These chapters are by no means intended to be comprehensive.

Also included are source code listings for 10 algorithms discussed in Part III. I was unable to include all the code I wanted to due to space limitations, and cryptographic source code cannot otherwise be exported. (Amazingly enough, the State Department allowed export of the first edition of this book with source code, but denied export for a computer disk with the exact same source code on it. Go figure.) An associated source code disk set includes much more source code than I could fit in this book; it is probably the largest collection of cryptographic source code outside a military institution. I can only send source code disks to U.S. and Canadian citizens living in the U.S. and Canada, but hopefully that will change someday. If you are interested in implementing or playing with the cryptographic algorithms in this book, get the disk. See the last page of the book for details.

One criticism of this book is that its encyclopedic nature takes away from its readability. This is true, but I wanted to provide a single reference for those who might come across an algorithm in the academic literature or in a product. For those who are more interested in a tutorial, I apologize. A lot is being done in the field; this is the first time so much of it has been gathered between two covers. Even so, space considerations forced me to leave many things out. I covered topics that I felt were important, practical, or interesting. If I couldn't cover a topic in depth, I gave references to articles and papers that did.

I have done my best to hunt down and eradicate all errors in this book, but many have assured me that it is an impossible task. Certainly, the second edition has far fewer errors than the first. An errata listing is available from me and will be periodically posted to the Usenet newsgroup sci.crypt. If any reader finds an error, please let me know. I'll send the first person to find each error in the book a free copy of the source code disk.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

About the Author

BRUCE SCHNEIER is president of Counterpane Systems, an Oak Park, Illinois consulting firm specializing in cryptography and computer security. Bruce is also the author of E–Mail Security (John Wiley & Sons, 1995) and Protect Your Macintosh (Peachpit Press, 1994); and has written dozens of articles on cryptography for major magazines. He is a contributing editor to Dr. Dobb's Journal, where he edits the "Algorithms Alley" column, and a contributing editor to Computer and Communications Security Reviews. Bruce serves on the board of directors of the International Association for Cryptologic Research, is a member of the Advisory Board for the Electronic Privacy Information Center, and is on the program committee for the New Security Paradigms Workshop. In addition, he finds time to give frequent lectures on cryptography, computer security, and privacy.

Acknowledgments

The list of people who had a hand in this book may seem unending, but all are worthy of mention. I would like to thank Don Alvarez, Ross Anderson, Dave Balenson, Karl Barrus, Steve Bellovin, Dan Bernstein, Eli Biham, Joan Boyar, Karen Cooper, Whit Diffie, Joan Feigenbaum, Phil Karn, Neal Koblitz, Xuejia Lai, Tom Leranth, Mike Markowitz, Ralph Merkle, Bill Patton, Peter Pearson, Charles Pfleeger, Ken Pizzini, Bart Preneel, Mark Riordan, Joachim Schurman, and Marc Schwartz for reading and editing all or parts of the first edition; Marc Vauclair for translating the first edition into French; Abe Abraham, Ross Anderson, Dave Banisar, Steve Bellovin, Eli Biham, Matt Bishop, Matt Blaze, Gary Carter, Jan Camenisch, Claude CrŽpeau, Joan Daemen, Jorge Davila, Ed Dawson, Whit Diffie, Carl Ellison, Joan Feigenbaum, Niels Ferguson, Matt Franklin, Rosario Gennaro, Dieter Gollmann, Mark Goresky, Richard Graveman, Stuart Haber, Jingman He, Bob Hogue, Kenneth Iversen, Markus Jakobsson, Burt Kaliski, Phil Karn, John Kelsey, John Kennedy, Lars Knudsen, Paul Kocher, John Ladwig, Xuejia Lai, Arjen Lenstra, Paul Leyland, Mike Markowitz, Jim Massey, Bruce McNair, William Hugh Murray, Roger Needham, Clif Neuman, Kaisa Nyberg, Luke O'Connor, Peter Pearson, RenŽ Peralta, Bart Preneel, Yisrael Radai, Matt Robshaw, Michael Roe, Phil Rogaway, Avi Rubin, Paul Rubin, Selwyn Russell, Kazue Sako, Mahmoud Salmasizadeh, Markus Stadler, Dmitry Titov, Jimmy Upton, Marc Vauclair, Serge Vaudenay, Gideon Yuval, Glen Zorn, and several anonymous government employees for reading and editing all or parts of the second edition; Lawrie Brown, Leisa Condie, Joan Daemen, Peter Gutmann, Alan Insley, Chris Johnston, John Kelsey, Xuejia Lai, Bill Leininger, Mike Markowitz, Richard Outerbridge, Peter Pearson, Ken Pizzini, Colin Plumb, RSA Data Security, Inc., Michael Roe, Michael Wood, and Phil Zimmermann for providing source code; Paul MacNerland for creating the figures for the first edition; Karen Cooper for copyediting the second edition; Beth Friedman for proofreading the second edition; Carol Kennedy for indexing the second edition; the readers of sci.crypt and the Cypherpunks mailing list for commenting on ideas, answering questions, and finding errors in the first edition; Randy Seuss for providing Internet access; Jeff Duntemann and Jon Erickson for helping me get started; assorted random Insleys for the impetus, encouragement, support, conversations, friendship, and dinners; and AT&T Bell Labs for firing me and making this all possible. All these people helped to create a far better book than I could have created alone.

Bruce Schneier
Oak Park, Ill.
schneier@counterpane.com


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Afterword by Matt Blaze

One of the most dangerous aspects of cryptology (and, by extension, of this book), is that you can almost measure it. Knowledge of key lengths, factoring methods, and cryptanalytic techniques makes it possible to estimate (in the absence of a real theory of cipher design) the "work factor" required to break a particular cipher. It's all too tempting to misuse these estimates as if they were overall security metrics for the systems in which they are used. The real world offers the attacker a richer menu of options than mere cryptanalysis. Often more worrisome are protocol attacks, Trojan horses, viruses, electromagnetic monitoring, physical compromise, blackmail and intimidation of key holders, operating system bugs, application program bugs, hardware bugs, user errors, physical eavesdropping, social engineering, and dumpster diving, to name just a few.

High-quality ciphers and protocols are important tools, but by themselves make poor substitutes for realistic, critical thinking about what is actually being protected and how various defenses might fail (attackers, after all, rarely restrict themselves to the clean, well-defined threat models of the academic world). Ross Anderson gives examples of cryptographically strong systems (in the banking industry) that fail when exposed to the threats of the real world [43, 44]. Even when the attacker has access only to ciphertext, seemingly minor breaches in other parts of the system can leak enough information to render good cryptosystems useless. The Allies in World War II broke the German Enigma traffic largely by carefully exploiting operator errors [1587].

An NSA-employed acquaintance, when asked whether the government can crack DES traffic, quipped that real systems are so insecure that they never need to bother. Unfortunately, there are no easy recipes for making a system secure, no substitute for careful design and critical, ongoing scrutiny. Good cryptosystems have the nice property of making life much harder for the attacker than for the legitimate user; this is not the case for almost every other aspect of computer and communication security. Consider the following (quite incomplete) "Top Ten Threats to Security in Real Systems" list; all are easier to exploit than to prevent.

1. The sorry state of software. Everyone knows that nobody knows how to write software. Modern systems are complex, with hundreds of thousands of lines of code; any one of them has the chance to compromise security. Fatal bugs may even be far-removed from the security portion of the software.
2. Ineffective protection against denial-of-service attacks. Some cryptographic protocols allow anonymity. It may be especially dangerous to deploy anonymous protocols if they increase the opportunities for unidentified vandals to disrupt service; anonymous systems therefore need to be especially resistant to denial-of-service attacks. Robust networks can more easily support anonymity; consider that hardly anyone worries very much about the millions of anonymous entry points to more robust networks like the telephone system or the postal service, where it's relatively difficult (or expensive) for an individual to cause large-scale failures.
3. No place to store secrets. Cryptosystems protect large secrets with smaller ones (keys). Unfortunately, modern computers aren't especially good at protecting even the smallest secrets. Multi-user networked workstations can be broken into and their memories compromised. Standalone, single-user machines can be stolen or compromised through viruses that leak secrets asynchronously. Remote servers, where there may be no user available to enter a passphrase (but see threat #5), are an especially hard problem.
4. Poor random-number generation. Keys and session variables need good sources of unpredictable bits. A running computer has a lot of entropy in it but rarely provides applications with a convenient or reliable way to exploit it. A number of techniques have been proposed for getting true random numbers in software (taking advantage of unpredictability in things like I/O interarrival timing, clock and timer skew, and even air turbulence inside disk enclosures), but all these are very sensitive to slight changes in the environments in which they are used.
5. Weak passphrases. Most cryptographic software addresses the key storage and key generation problems by relying on user-generated passphrase strings, which are presumed to be unpredictable enough to produce good key material and are also easy enough to remember that they do not require secure storage. While dictionary attacks are a well-known problem with short passwords, much less is known about lines of attack against user-selected passphrase-based keys. Shannon tells us that English text has only just over 1 bit of entropy per character, which would seem to leave most passphrases well within reach of brute-force search. Less is known, however, about good techniques for enumerating passphrases in order to exploit this. Until we have a better understanding of how to attack passphrases, we really have no idea how weak or strong they are.
6. Mismatched trust. Almost all currently available cryptographic software assumes that the user is in direct control over the systems on which it runs and has a secure path to it. For example, the interfaces to programs like PGP assume that their passphrase input always comes from the user over a secure path like the local console. This is not always the case, of course; consider the problem of reading your encrypted mail when logged in over a network connection. What the system designer assumes is trusted may not match the needs or expectations of the real users, especially when software can be controlled remotely over insecure networks.
7. Poorly understood protocol and service interactions. As systems get bigger and more complex, benign features frequently come back to haunt us, and it's hard to know even where to look when things fail. The Internet worm was propagated via an obscure and innocent-looking feature in the sendmail program; how many more features in how many more programs have unexpected consequences just waiting to be discovered?
8. Unrealistic threat and risks assessment. Security experts tend to focus on the threats they know how to model and prevent. Unfortunately, attackers focus on what they know how to exploit, and the two are rarely exactly the same. Too many "secure" systems are designed without considering what the attacker is actually likely to do.
9. Interfaces that make security expensive and special. If security features are to be used, they must be convenient and transparent enough that people actually turn them on. It's easy to design encryption mechanisms that come only at the expense of performance or ease of use, and even easier to design mechanisms that invite mistakes. Security should be harder to turn off than on; unfortunately, few systems actually work this way.
10. Little broad-based demand for security. This is a well-known problem among almost everyone who has tied his or her fortune to selling security products and services. Until there is widespread demand for transparent security, the tools and infrastructure needed to support it will be expensive and inaccessible to many applications. This is partly a problem of understanding and exposing the threats and risks in real applications and partly a problem of not designing systems that include security as a basic feature rather than as a later add-on.

A more complete list and discussion of these kinds of threats could easily fill a book of this size and barely scratch the surface. What makes them especially difficult and dangerous is that there are no magic techniques, beyond good engineering and ongoing scrutiny, for avoiding them. The lesson for the aspiring cryptographer is to respect the limits of the art.

Matt Blaze
New York, NY


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part V
SOURCE CODE

1. DES
2. LOKI91
3. IDEA
4. GOST
5. Blowfish
6. 3–Way
7. RC5
8. A5
9. SEAL

DES

#define EN0   0      /* MODE == encrypt */
#define DE1   1      /* MODE == decrypt */

typedef struct {
        unsigned long ek[32];
        unsigned long dk[32];
} des_ctx;

extern void deskey(unsigned char *, short);
/*                  hexkey[8]     MODE
 * Sets the internal key register according to the hexadecimal
 * key contained in the 8 bytes of hexkey, according to the DES,
 * for encryption or decryption according to MODE.
 */

extern void usekey(unsigned long *);
/*                cookedkey[32]
 * Loads the internal key register with the data in cookedkey.
 */

extern void cpkey(unsigned long *);
/*               cookedkey[32]
 * Copies the contents of the internal key register into the storage
 * located at &cookedkey[0].
 */

extern void des(unsigned char *, unsigned char *);
/*                from[8]         to[8]
 * Encrypts/Decrypts (according to the key currently loaded in the
 * internal key register) one block of eight bytes at address 'from'
 * into the block at address 'to'.  They can be the same.
 */

static void scrunch(unsigned char *, unsigned long *);
static void unscrun(unsigned long *, unsigned char *);
static void desfunc(unsigned long *, unsigned long *);
static void cookey(unsigned long *);

static unsigned long KnL[32] = { 0L };
static unsigned long KnR[32] = { 0L };
static unsigned long Kn3[32] = { 0L };
static unsigned char Df_Key[24] = {
       0×01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef,
       0xfe,0xdc,0xba,0x98,0x76,0x54,0x32,0x10,
       0x89,0xab,0xcd,0xef,0×01,0x23,0x45,0x67 };

static unsigned short bytebit[8]    = {
       0200, 0100, 040, 020, 010, 04, 02, 01 };

static unsigned long bigbyte[24] = {
       0x800000L,    0x400000L,     0x200000L,    0x100000L,
       0x80000L,     0x40000L,      0x20000L,     0x10000L,
       0x8000L,      0x4000L,       0x2000L,      0x1000L,
       0x800L,              0x400L,               0x200L,              0x100L,
       0x80L,               0x40L,                0x20L,               0x10L,
       0x8L,         0x4L,          0x2L,         0x1L   };

/* Use the key schedule specified in the Standard (ANSI X3.92–1981). */

static unsigned char pc1[56] = {
       56, 48, 40, 32, 24, 16,  8,   0, 57, 49, 41, 33, 25, 17,
        9,  1, 58, 50, 42, 34, 26,  18, 10,  2, 59, 51, 43, 35,
       62, 54, 46, 38, 30, 22, 14,   6, 61, 53, 45, 37, 29, 21,
       13,  5, 60, 52, 44, 36, 28,  20, 12,  4, 27, 19, 11,  3 };

static unsigned char totrot[16] = {
       1,2,4,6,8,10,12,14,15,17,19,21,23,25,27,28 };

static unsigned char pc2[48] = {
       13, 16, 10, 23,  0,  4,       2, 27, 14,  5, 20,  9,
       22, 18, 11,  3, 25,  7,      15,  6, 26, 19, 12,  1,
       40, 51, 30, 36, 46, 54,      29, 39, 50, 44, 32, 47,
       43, 48, 38, 55, 33, 52,      45, 41, 49, 35, 28, 31 };
void deskey(key, edf)       /* Thanks to James Gillogly & Phil Karn! */
unsigned char *key;
short edf;
{
       register int i, j, l, m, n;
       unsigned char pc1m[56], pcr[56];
       unsigned long kn[32];

       for ( j = 0; j < 56; j++ ) {
              l = pc1[j];
              m = l & 07;
              pc1m[j] = (key[l >> 3] & bytebit[m]) ? 1 : 0;
              }
       for( i = 0; i < 16; i++ ) {
              if( edf == DE1 ) m = (15 – i) << 1;
              else m = i << 1;
              n = m + 1;
              kn[m] = kn[n] = 0L;
              for( j = 0; j < 28; j++ ) {
                     l = j + totrot[i];
                     if( l < 28 ) pcr[j] = pc1m[l];
                     else pcr[j] = pc1m[l – 28];
                     }
              for( j = 28; j < 56; j++ ) {
                  l = j + totrot[i];
                  if( l < 56 ) pcr[j] = pc1m[l];
                  else pcr[j] = pc1m[l – 28];
                  }
              for( j = 0; j < 24; j++ ) {
                     if( pcr[pc2[j]] ) kn[m] |= bigbyte[j];
                     if( pcr[pc2[j+24]] ) kn[n] |= bigbyte[j];
                     }
              }
       cookey(kn);
       return;
}

static void cookey(raw1)
register unsigned long *raw1;
{
       register unsigned long *cook, *raw0;
       unsigned long dough[32];
       register int i;

       cook = dough;
       for( i = 0; i < 16; i++, raw1++ ) {
              raw0 = raw1++;
              *cook   = (*raw0 & 0×00fc0000L) << 6;
              *cook  |= (*raw0 & 0×00000fc0L) << 10;
              *cook  |= (*raw1 & 0×00fc0000L) >> 10;
              *cook++       |= (*raw1 & 0×00000fc0L) >> 6;
              *cook   = (*raw0 & 0×0003f000L) << 12;
              *cook  |= (*raw0 & 0×0000003fL) << 16;
              *cook  |= (*raw1 & 0×0003f000L) >> 4;
              *cook++       |= (*raw1 & 0×0000003fL);
              }
       usekey(dough);
       return;
}

void cpkey(into)
register unsigned long *into;
{
       register unsigned long *from, *endp;
       from = KnL, endp = &KnL[32];
       while( from < endp ) *into++ = *from++;
       return;
}

void usekey(from)
register unsigned long *from;
{
       register unsigned long *to, *endp;
       to = KnL, endp = &KnL[32];
       while( to < endp ) *to++ = *from++;
       return;
}

void des(inblock, outblock)
unsigned char *inblock, *outblock;
{
       unsigned long work[2];

       scrunch(inblock, work);
       desfunc(work, KnL);
       unscrun(work, outblock);
       return;
}

static void scrunch(outof, into)
register unsigned char *outof;
register unsigned long *into;
{
       *into   = (*outof++ & 0xffL) << 24;
       *into  |= (*outof++ & 0xffL) << 16;
       *into  |= (*outof++ & 0xffL) << 8;
       *into++ |= (*outof++ & 0xffL);
       *into   = (*outof++ & 0xffL) << 24;
       *into  |= (*outof++ & 0xffL) << 16;
       *into  |= (*outof++ & 0xffL) << 8;
       *into  |= (*outof   & 0xffL);
       return;
}

static void unscrun(outof, into)
register unsigned long *outof;
register unsigned char *into;
{
       *into++ = (*outof >> 24) & 0xffL;
       *into++ = (*outof >> 16) & 0xffL;
       *into++ = (*outof >>  8) & 0xffL;
       *into++ =  *outof++           & 0xffL;
       *into++ = (*outof >> 24) & 0xffL;
       *into++ = (*outof >> 16) & 0xffL;
       *into++ = (*outof >>  8) & 0xffL;
       *into   =  *outof     & 0xffL;
       return;
}
static unsigned long SP1[64] = {
       0×01010400L, 0×00000000L, 0×00010000L, 0×01010404L,
       0×01010004L, 0×00010404L, 0×00000004L, 0×00010000L,
       0×00000400L, 0×01010400L, 0×01010404L, 0×00000400L,
       0×01000404L, 0×01010004L, 0×01000000L, 0×00000004L,
       0×00000404L, 0×01000400L, 0×01000400L, 0×00010400L,
       0×00010400L, 0×01010000L, 0×01010000L, 0×01000404L,
       0×00010004L, 0×01000004L, 0×01000004L, 0×00010004L,
       0×00000000L, 0×00000404L, 0×00010404L, 0×01000000L,
       0×00010000L, 0×01010404L, 0×00000004L, 0×01010000L,
       0×01010400L, 0×01000000L, 0×01000000L, 0×00000400L,
       0×01010004L, 0×00010000L, 0×00010400L, 0×01000004L,
       0×00000400L, 0×00000004L, 0×01000404L, 0×00010404L,
       0×01010404L, 0×00010004L, 0×01010000L, 0×01000404L,
       0×01000004L, 0×00000404L, 0×00010404L, 0×01010400L,
       0×00000404L, 0×01000400L, 0×01000400L, 0×00000000L,
       0×00010004L, 0×00010400L, 0×00000000L, 0×01010004L };

static unsigned long SP2[64] = {
       0x80108020L, 0x80008000L, 0×00008000L, 0×00108020L,
       0×00100000L, 0×00000020L, 0x80100020L, 0x80008020L,
       0x80000020L, 0x80108020L, 0x80108000L, 0x80000000L,
       0x80008000L, 0×00100000L, 0×00000020L, 0x80100020L,
       0×00108000L, 0×00100020L, 0x80008020L, 0×00000000L,
       0x80000000L, 0×00008000L, 0×00108020L, 0x80100000L,
       0×00100020L, 0x80000020L, 0×00000000L, 0×00108000L,
       0×00008020L, 0x80108000L, 0x80100000L, 0×00008020L,
       0×00000000L, 0×00108020L, 0x80100020L, 0×00100000L,
       0x80008020L, 0x80100000L, 0x80108000L, 0×00008000L,
       0x80100000L, 0x80008000L, 0×00000020L, 0x80108020L,
       0×00108020L, 0×00000020L, 0×00008000L, 0x80000000L,
       0×00008020L, 0x80108000L, 0×00100000L, 0x80000020L,
       0×00100020L, 0x80008020L, 0x80000020L, 0×00100020L,
       0×00108000L, 0×00000000L, 0x80008000L, 0×00008020L,
       0x80000000L, 0x80100020L, 0x80108020L, 0×00108000L };

static unsigned long SP3[64] = {
       0×00000208L, 0×08020200L, 0×00000000L, 0×08020008L,
       0×08000200L, 0×00000000L, 0×00020208L, 0×08000200L,
       0×00020008L, 0×08000008L, 0×08000008L, 0×00020000L,
       0×08020208L, 0×00020008L, 0×08020000L, 0×00000208L,
       0×08000000L, 0×00000008L, 0×08020200L, 0×00000200L,
       0×00020200L, 0×08020000L, 0×08020008L, 0×00020208L,
       0×08000208L, 0×00020200L, 0×00020000L, 0×08000208L,
       0×00000008L, 0×08020208L, 0×00000200L, 0×08000000L,
       0×08020200L, 0×08000000L, 0×00020008L, 0×00000208L,
       0×00020000L, 0×08020200L, 0×08000200L, 0×00000000L,
       0×00000200L, 0×00020008L, 0×08020208L, 0×08000200L,
       0×08000008L, 0×00000200L, 0×00000000L, 0×08020008L,
       0×08000208L, 0×00020000L, 0×08000000L, 0×08020208L,
       0×00000008L, 0×00020208L, 0×00020200L, 0×08000008L,
       0×08020000L, 0×08000208L, 0×00000208L, 0×08020000L,
       0×00020208L, 0×00000008L, 0×08020008L, 0×00020200L };

static unsigned long SP4[64] = {
       0×00802001L, 0×00002081L, 0×00002081L, 0×00000080L,
       0×00802080L, 0×00800081L, 0×00800001L, 0×00002001L,
       0×00000000L, 0×00802000L, 0×00802000L, 0×00802081L,
       0×00000081L, 0×00000000L, 0×00800080L, 0×00800001L,
       0×00000001L, 0×00002000L, 0×00800000L, 0×00802001L,
       0×00000080L, 0×00800000L, 0×00002001L, 0×00002080L,
       0×00800081L, 0×00000001L, 0×00002080L, 0×00800080L,
       0×00002000L, 0×00802080L, 0×00802081L, 0×00000081L,
       0×00800080L, 0×00800001L, 0×00802000L, 0×00802081L,
       0×00000081L, 0×00000000L, 0×00000000L, 0×00802000L,
       0×00002080L, 0×00800080L, 0×00800081L, 0×00000001L,
       0×00802001L, 0×00002081L, 0×00002081L, 0×00000080L,
       0×00802081L, 0×00000081L, 0×00000001L, 0×00002000L,
       0×00800001L, 0×00002001L, 0×00802080L, 0×00800081L,
       0×00002001L, 0×00002080L, 0×00800000L, 0×00802001L,
       0×00000080L, 0×00800000L, 0×00002000L, 0×00802080L };

static unsigned long SP5[64] = {
       0×00000100L, 0×02080100L, 0×02080000L, 0x42000100L,
       0×00080000L, 0×00000100L, 0x40000000L, 0×02080000L,
       0x40080100L, 0×00080000L, 0×02000100L, 0x40080100L,
       0x42000100L, 0x42080000L, 0×00080100L, 0x40000000L,
       0×02000000L, 0x40080000L, 0x40080000L, 0×00000000L,
       0x40000100L, 0x42080100L, 0x42080100L, 0×02000100L,
       0x42080000L, 0x40000100L, 0×00000000L, 0x42000000L,
       0×02080100L, 0×02000000L, 0x42000000L, 0×00080100L,
       0×00080000L, 0x42000100L, 0×00000100L, 0×02000000L,
       0x40000000L, 0×02080000L, 0x42000100L, 0x40080100L,
       0×02000100L, 0x40000000L, 0x42080000L, 0×02080100L,
       0x40080100L, 0×00000100L, 0×02000000L, 0x42080000L,
       0x42080100L, 0×00080100L, 0x42000000L, 0x42080100L,
       0×02080000L, 0×00000000L, 0x40080000L, 0x42000000L,
       0×00080100L, 0×02000100L, 0x40000100L, 0×00080000L,
       0×00000000L, 0x40080000L, 0×02080100L, 0x40000100L };

static unsigned long SP6[64] = {
       0x20000010L, 0x20400000L, 0×00004000L, 0x20404010L,
       0x20400000L, 0×00000010L, 0x20404010L, 0×00400000L,
       0x20004000L, 0×00404010L, 0×00400000L, 0x20000010L,
       0×00400010L, 0x20004000L, 0x20000000L, 0×00004010L,
       0×00000000L, 0×00400010L, 0x20004010L, 0×00004000L,
       0×00404000L, 0x20004010L, 0×00000010L, 0x20400010L,
       0x20400010L, 0×00000000L, 0×00404010L, 0x20404000L,
       0×00004010L, 0×00404000L, 0x20404000L, 0x20000000L,
       0x20004000L, 0×00000010L, 0x20400010L, 0×00404000L,
       0x20404010L, 0×00400000L, 0×00004010L, 0x20000010L,
       0×00400000L, 0x20004000L, 0x20000000L, 0×00004010L,
       0x20000010L, 0x20404010L, 0×00404000L, 0x20400000L,
       0×00404010L, 0x20404000L, 0×00000000L, 0x20400010L,
       0×00000010L, 0×00004000L, 0x20400000L, 0×00404010L,
       0×00004000L, 0×00400010L, 0x20004010L, 0×00000000L,
       0x20404000L, 0x20000000L, 0×00400010L, 0x20004010L };

static unsigned long SP7[64] = {
       0×00200000L, 0×04200002L, 0×04000802L, 0×00000000L,
       0×00000800L, 0×04000802L, 0×00200802L, 0×04200800L,
       0×04200802L, 0×00200000L, 0×00000000L, 0×04000002L,
       0×00000002L, 0×04000000L, 0×04200002L, 0×00000802L,
       0×04000800L, 0×00200802L, 0×00200002L, 0×04000800L,
       0×04000002L, 0×04200000L, 0×04200800L, 0×00200002L,
       0×04200000L, 0×00000800L, 0×00000802L, 0×04200802L,
       0×00200800L, 0×00000002L, 0×04000000L, 0×00200800L,
       0×04000000L, 0×00200800L, 0×00200000L, 0×04000802L,
       0×04000802L, 0×04200002L, 0×04200002L, 0×00000002L,
       0×00200002L, 0×04000000L, 0×04000800L, 0×00200000L,
       0×04200800L, 0×00000802L, 0×00200802L, 0×04200800L,
       0×00000802L, 0×04000002L, 0×04200802L, 0×04200000L,
       0×00200800L, 0×00000000L, 0×00000002L, 0×04200802L,
       0×00000000L, 0×00200802L, 0×04200000L, 0×00000800L,
       0×04000002L, 0×04000800L, 0×00000800L, 0×00200002L };

static unsigned long SP8[64] = {
       0x10001040L, 0×00001000L, 0×00040000L, 0x10041040L,
       0x10000000L, 0x10001040L, 0×00000040L, 0x10000000L,
       0×00040040L, 0x10040000L, 0x10041040L, 0×00041000L,
       0x10041000L, 0×00041040L, 0×00001000L, 0×00000040L,
       0x10040000L, 0x10000040L, 0x10001000L, 0×00001040L,
       0×00041000L, 0×00040040L, 0x10040040L, 0x10041000L,
       0×00001040L, 0×00000000L, 0×00000000L, 0x10040040L,
       0x10000040L, 0x10001000L, 0×00041040L, 0×00040000L,
       0×00041040L, 0×00040000L, 0x10041000L, 0×00001000L,
       0×00000040L, 0x10040040L, 0×00001000L, 0×00041040L,
       0x10001000L, 0×00000040L, 0x10000040L, 0x10040000L,
       0x10040040L, 0x10000000L, 0×00040000L, 0x10001040L,
       0×00000000L, 0x10041040L, 0×00040040L, 0x10000040L,
       0x10040000L, 0x10001000L, 0x10001040L, 0×00000000L,
       0x10041040L, 0×00041000L, 0×00041000L, 0×00001040L,
       0×00001040L, 0×00040040L, 0x10000000L, 0x10041000L };

static void desfunc(block, keys)
register unsigned long *block, *keys;
{
       register unsigned long fval, work, right, leftt;
       register int round;

       leftt = block[0];
       right = block[1];
       work = ((leftt >> 4) ^ right) & 0×0f0f0f0fL;
       right ^= work;
       leftt ^= (work << 4);
       work = ((leftt >> 16) ^ right) & 0×0000ffffL;
       right ^= work;
       leftt ^= (work << 16);
       work = ((right >> 2) ^ leftt) & 0x33333333L;
       leftt ^= work;
       right ^= (work << 2);
       work = ((right >> 8) ^ leftt) & 0×00ff00ffL;
       leftt ^= work;
       right ^= (work << 8);
       right = ((right << 1) | ((right >> 31) & 1L)) & 0xffffffffL;
       work = (leftt ^ right) & 0xaaaaaaaaL;
       leftt ^= work;
       right ^= work;
       leftt = ((leftt << 1) | ((leftt >> 31) & 1L)) & 0xffffffffL;

       for( round = 0; round < 8; round++ ) {
              work  = (right << 28) | (right >> 4);
              work ^= *keys++;
              fval  = SP7[ work             & 0x3fL];
              fval |= SP5[(work >>  8) & 0x3fL];
              fval |= SP3[(work >> 16) & 0x3fL];
              fval |= SP1[(work >> 24) & 0x3fL];
              work  = right ^ *keys++;
              fval |= SP8[ work             & 0x3fL];
              fval |= SP6[(work >>  8) & 0x3fL];
              fval |= SP4[(work >> 16) & 0x3fL];
              fval |= SP2[(work >> 24) & 0x3fL];
              leftt ^= fval;
              work  = (leftt << 28) | (leftt >> 4);
              work ^= *keys++;
              fval  = SP7[ work             & 0x3fL];
              fval |= SP5[(work >>  8) & 0x3fL];
              fval |= SP3[(work >> 16) & 0x3fL];
              fval |= SP1[(work >> 24) & 0x3fL];
              work  = leftt ^ *keys++;
              fval |= SP8[ work             & 0x3fL];
              fval |= SP6[(work >>  8) & 0x3fL];
              fval |= SP4[(work >> 16) & 0x3fL];
              fval |= SP2[(work >> 24) & 0x3fL];
              right ^= fval;
              }

       right = (right << 31) | (right >> 1);
       work = (leftt ^ right) & 0xaaaaaaaaL;
       leftt ^= work;
       right ^= work;
       leftt = (leftt << 31) | (leftt >> 1);
       work = ((leftt >> 8) ^ right) & 0×00ff00ffL;
       right ^= work;
       leftt ^= (work << 8);
       work = ((leftt >> 2) ^ right) & 0x33333333L;
       right ^= work;
       leftt ^= (work << 2);
       work = ((right >> 16) ^ leftt) & 0×0000ffffL;
       leftt ^= work;
       right ^= (work << 16);
       work = ((right >> 4) ^ leftt) & 0×0f0f0f0fL;
       leftt ^= work;
       right ^= (work << 4);
       *block++ = right;
       *block = leftt;
       return;
}
/* Validation sets:
 *
 * Single–length key, single–length plaintext –
 * Key    : 0123 4567 89ab cdef
 * Plain  : 0123 4567 89ab cde7
 * Cipher : c957 4425 6a5e d31d
 *
 **********************************************************************/

void des_key(des_ctx *dc, unsigned char *key){
        deskey(key,EN0);
        cpkey(dc–>ek);
        deskey(key,DE1);
        cpkey(dc–>dk);
}
/* Encrypt several blocks in ECB mode.  Caller is responsible for
   short blocks. */
void des_enc(des_ctx *dc, unsigned char *data, int blocks){
        unsigned long work[2];
        int i;
        unsigned char *cp;
        cp = data;
        for(i=0;i<blocks;i++){
                scrunch(cp,work);
                desfunc(work,dc–>ek);
                unscrun(work,cp);
                cp+=8;
        }
}

void des_dec(des_ctx *dc, unsigned char *data, int blocks){
        unsigned long work[2];
        int i;
        unsigned char *cp;

        cp = data;
        for(i=0;i<blocks;i++){
                scrunch(cp,work);
                desfunc(work,dc–>dk);
                unscrun(work,cp);
                cp+=8;
        }
}
void main(void){
        des_ctx dc;
        int i;
        unsigned long data[10];
        char *cp,key[8] = {0×01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
        char x[8] = {0×01,0x23,0x45,0x67,0x89,0xab,0xcd,0xe7};

        cp = x;

        des_key(&dc,key);
        des_enc(&dc,cp,1);
        printf("Enc(0..7,0..7) = ");
        for(i=0;i<8;i++) printf("%02x ", ((unsigned int) cp[i])&0×00ff);
        printf("\n");

        des_dec(&dc,cp,1);

        printf("Dec(above,0..7) = ");
        for(i=0;i<8;i++) printf("%02x ",((unsigned int)cp[i])&0×00ff);
        printf("\n");

        cp = (char *) data;
        for(i=0;i<10;i++)data[i]=i;

        des_enc(&dc,cp,5); /* Enc 5 blocks. */
        for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx.\n",
                                i/2,data[i],data[i+1]);

        des_dec(&dc,cp,1);
        des_dec(&dc,cp+8,4);
        for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx.\n",
                                i/2,data[i],data[i+1]);
}

LOKI91

#include <stdio.h>

#define LOKIBLK        8              /* No of bytes in a LOKI data–block          */
#define ROUNDS         16             /* No of LOKI rounds                        */

typedef unsigned long          Long;   /* type specification for aligned LOKI blocks */

extern Long    lokikey[2];     /* 64–bit key used by LOKI routines          */
extern char    *loki_lib_ver;         /* String with version no. & copyright       */

#ifdef __STDC__                       /* declare prototypes for library functions  */
extern void enloki(char *b);
extern void deloki(char *b);
extern void setlokikey(char key[LOKIBLK]);
#else                          /* else just declare library functions extern */
extern void enloki(), deloki(), setlokikey();
#endif __STDC__

char P[32] = {
        31, 23, 15, 7, 30, 22, 14, 6,
        29, 21, 13, 5, 28, 20, 12, 4,
        27, 19, 11, 3, 26, 18, 10, 2,
        25, 17, 9, 1, 24, 16, 8, 0
        };

typedef        struct {
        short  gen;            /* irreducible polynomial used in this field */
        short  exp;            /* exponent used to generate this s function */

        } sfn_desc;
sfn_desc sfn[] = {
        { /* 101110111 */ 375, 31}, { /* 101111011 */ 379, 31},
        { /* 110000111 */ 391, 31}, { /* 110001011 */ 395, 31},
        { /* 110001101 */ 397, 31}, { /* 110011111 */ 415, 31},
        { /* 110100011 */ 419, 31}, { /* 110101001 */ 425, 31},
        { /* 110110001 */ 433, 31}, { /* 110111101 */ 445, 31},
        { /* 111000011 */ 451, 31}, { /* 111001111 */ 463, 31},
        { /* 111010111 */ 471, 31}, { /* 111011101 */ 477, 31},
        { /* 111100111 */ 487, 31}, { /* 111110011 */ 499, 31},
        { 00, 00}      };

typedef struct {
        Long loki_subkeys[ROUNDS];
} loki_ctx;

static Long    f();                   /* declare LOKI function f */
static short   s();                   /* declare LOKI S–box fn s */

#define ROL12(b) b = ((b << 12) | (b >> 20));
#define ROL13(b) b = ((b << 13) | (b >> 19));

#ifdef  LITTLE_ENDIAN
#define bswap(cb) {                             \
        register char   c;                      \
        c = cb[0]; cb[0] = cb[3]; cb[3] = c;    \
        c = cb[1]; cb[1] = cb[2]; cb[2] = c;    \
        c = cb[4]; cb[4] = cb[7]; cb[7] = c;    \
        c = cb[5]; cb[5] = cb[6]; cb[6] = c;    \
}
#endif

void
setlokikey(loki_ctx *c, char *key)
{
        register       i;
        register Long  KL, KR;
#ifdef LITTLE_ENDIAN
        bswap(key);                   /* swap bytes round if little–endian */
#endif
        KL = ((Long *)key)[0];
        KR = ((Long *)key)[1];
        for (i=0; i<ROUNDS; i+=4) {           /* Generate the 16 subkeys */
            c–>loki_subkeys[i] = KL;
            ROL12 (KL);
            c–>loki_subkeys[i+1] = KL;
            ROL13 (KL);
            c–>loki_subkeys[i+2] = KR;
            ROL12 (KR);
            c–>loki_subkeys[i+3] = KR;
            ROL13 (KR);
        }

#ifdef LITTLE_ENDIAN
        bswap(key);                   /* swap bytes back if little–endian */
#endif
}
void
enloki (loki_ctx *c, char *b)
{
        register       i;
        register Long  L, R;          /* left & right data halves  */

#ifdef LITTLE_ENDIAN
        bswap(b);                     /* swap bytes round if little–endian */
#endif
        L = ((Long *)b)[0];
        R = ((Long *)b)[1];

        for (i=0; i<ROUNDS; i+=2) {           /* Encrypt with the 16 subkeys */
            L ^= f (R, c–>loki_subkeys[i]);
            R ^= f (L, c–>loki_subkeys[i+1]);
        }

        ((Long *)b)[0] = R;           /* Y = swap(LR) */
        ((Long *)b)[1] = L;

#ifdef LITTLE_ENDIAN
        bswap(b);                     /* swap bytes round if little–endian */
#endif
}

void
deloki(loki_ctx *c, char *b)
{
        register       i;
        register Long  L, R;                  /* left & right data halves  */

#ifdef LITTLE_ENDIAN
        bswap(b);                     /* swap bytes round if little–endian */
#endif

        L = ((Long *)b)[0];                   /* LR = X XOR K */
        R = ((Long *)b)[1];

        for (i=ROUNDS; i>0; i–=2) {                   /* subkeys in reverse order */
            L ^= f(R, c–>loki_subkeys[i–1]);
            R ^= f(L, c–>loki_subkeys[i–2]);
        }

        ((Long *)b)[0] = R;                   /* Y = LR XOR K */
        ((Long *)b)[1] = L;
}

#define MASK12         0×0fff                 /* 12 bit mask for expansion E */

static Long
f(r, k)
register Long  r;      /* Data value R(i–1) */
Long           k;      /* Key     K(i)   */
{
        Long   a, b, c;               /* 32 bit S–box output, & P output */
        a = r ^ k;                    /* A = R(i–1) XOR K(i) */

        /* want to use slow speed/small size version */
        b = ((Long)s((a         & MASK12))      ) | /* B = S(E(R(i–1))^K(i)) */
            ((Long)s(((a >>  8) & MASK12)) <<  8) |
            ((Long)s(((a >> 16) & MASK12)) << 16) |
            ((Long)s((((a >> 24) | (a << 8)) & MASK12)) << 24);

        perm32(&c, &b, P);            /* C = P(S( E(R(i–1)) XOR K(i))) */

        return(c);                    /* f returns the result C */
}

static short s(i)
register Long i;       /* return S–box value for input i */
{
        register short r, c, v, t;
        short  exp8();                /* exponentiation routine for GF(2^8) */

        r = ((i>>8) & 0xc) | (i & 0x3);               /* row value–top 2 & bottom 2 */
        c = (i>>2) & 0xff;                            /* column value–middle 8 bits */
        t = (c + ((r * 17) ^ 0xff)) & 0xff;           /* base value for Sfn */
        v = exp8(t, sfn[r].exp, sfn[r].gen);          /* Sfn[r] = t ^ exp mod gen */
        return(v);
}

#define        MSB     0x80000000L            /* MSB of 32–bit word */

perm32(out, in , perm)
Long    *out;          /* Output 32–bit block to be permuted                */
Long    *in;           /* Input  32–bit block after permutation             */
char    perm[32];      /* Permutation array                                 */
{
        Long   mask = MSB;                    /* mask used to set bit in output    */
        register int   i, o, b;       /* input bit no, output bit no, value */
        register char  *p = perm;     /* ptr to permutation array  */

        *out = 0;                     /* clear output block */
        for (o=0; o<32; o++) {                /* For each output bit position o */
               i =(int)*p++;                  /* get input bit permuted to output o */
               b = (*in >> i) & 01;           /* value of input bit i */
               if (b)                 /* If the input bit i is set */
                       *out |= mask;                  /*  OR in mask to output i */
               mask >>= 1;                            /* Shift mask to next bit    */
        }
}

#define SIZE 256               /* 256 elements in GF(2^8) */

short mult8(a, b, gen)
short   a, b;          /* operands for multiply */
short   gen;           /* irreducible polynomial generating Galois Field */
{
        short  product = 0;           /* result of multiplication */

        while(b != 0) {                       /* while multiplier is non–zero */
               if (b & 01)
                       product ^= a;          /*   add multiplicand if LSB of b set */
               a <<= 1;               /*   shift multiplicand one place */
               if (a >= SIZE)
                       a ^= gen;      /*   and modulo reduce if needed */
               b >>= 1;               /*   shift multiplier one place  */
        }
        return(product);
}

short exp8(base, exponent, gen)
short   base;          /* base of exponentiation      */
short   exponent;      /* exponent                    */
short   gen;           /* irreducible polynomial generating Galois Field */
{
        short  accum = base;          /* superincreasing sequence of base */
        short  result = 1;     /* result of exponentiation       */

        if (base == 0)                /* if zero base specified then      */
               return(0);      /* the result is "0" if base = 0    */

        while (exponent != 0) {       /* repeat while exponent non–zero */
               if (( exponent & 0×0001) == 0×0001)           /* multiply if exp 1 */
                       result = mult8(result, accum, gen);
               exponent >>= 1;                /* shift exponent to next digit */
               accum = mult8(accum, accum, gen);             /* & square  */
        }
        return(result);
}

void loki_key(loki_ctx *c, unsigned char *key){
        setlokikey(c,key);
}
void loki_enc(loki_ctx *c, unsigned char *data, int blocks){
        unsigned char *cp;
        int i;

        cp = data;
        for(i=0;i<blocks;i++){
                enloki(c,cp);
                cp+=8;
        }
}

void loki_dec(loki_ctx *c, unsigned char *data, int blocks){
        unsigned char *cp;
        int i;

        cp = data;
        for(i=0;i<blocks;i++){
                deloki(c,cp);
                cp+=8;
        }
}

void main(void){
        loki_ctx lc;
        unsigned long data[10];
        unsigned char *cp;
        unsigned char key[] = {0,1,2,3,4,5,6,7};
        int i;

        for(i=0;i<10;i++) data[i]=i;

        loki_key(&lc,key);
        cp = (char *)data;
        loki_enc(&lc,cp,5);
        for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx\n",
                        i/2,data[i],data[i+1]);
        loki_dec(&lc,cp,1);
        loki_dec(&lc,cp+8,4);
        for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx\n",
                        i/2,data[i],data[i+1]);
}

IDEA

typedef unsigned char boolean;      /* values are TRUE or FALSE */
typedef unsigned char byte; /* values are 0–255 */
typedef byte *byteptr;      /* pointer to byte */
typedef char *string;/* pointer to ASCII character string */
typedef unsigned short word16;      /* values are 0–65535 */
typedef unsigned long word32;       /* values are 0–4294967295 */

#ifndef TRUE
#define FALSE 0
#define TRUE (!FALSE)
#endif /* if TRUE not already defined */

#ifndef min   /* if min macro not already defined */
#define min(a,b) ( (a)<(b) ? (a) : (b) )
#define max(a,b) ( (a)>(b) ? (a) : (b) )
#endif /* if min macro not already defined */

#define IDEAKEYSIZE 16
#define IDEABLOCKSIZE 8

#define IDEAROUNDS 8
#define IDEAKEYLEN (6*IDEAROUNDS+4)

typedef struct{
       word16 ek[IDEAKEYLEN],dk[IDEAKEYLEN];
}idea_ctx;

/* End includes for IDEA.C */
#ifdef IDEA32        /* Use >16–bit temporaries */
#define low16(x) ((x) & 0xFFFF)
typedef unsigned int uint16;/* at LEAST 16 bits, maybe more */
#else
#define low16(x) (x) /* this is only ever applied to uint16's */
typedef word16 uint16;
#endif

#ifdef SMALL_CACHE
static uint16
mul(register uint16 a, register uint16 b)
{
       register word32 p;

       p = (word32)a * b;
       if (p) {
              b = low16(p);
              a = p>>16;
              return (b – a) + (b < a);
       } else if (a) {
              return 1–b;
       } else {
              return 1–a;
       }
} /* mul */
#endif /* SMALL_CACHE */

static uint16
mulInv(uint16 x)
{
       uint16 t0, t1;
       uint16 q, y;

       if (x <= 1)
              return x;     /* 0 and 1 are self–inverse */
       t1 = 0x10001L / x;   /* Since x >= 2, this fits into 16 bits */
       y = 0x10001L % x;
       if (y == 1)
              return low16(1–t1);
       t0 = 1;
       do {
              q = x / y;
              x = x % y;
              t0 += q * t1;
              if (x == 1)
                     return t0;
              q = y / x;
              y = y % x;
              t1 += q * t0;
       } while (y != 1);
       return low16(1–t1);
} /* mukInv */

static void
ideaExpandKey(byte const *userkey, word16 *EK)
{
       int i,j;
       for (j=0; j<8; j++) {
              EK[j] = (userkey[0]<<8) + userkey[1];
              userkey += 2;
       }
       for (i=0; j < IDEAKEYLEN; j++) {
              i++;
              EK[i+7] = EK[i & 7] << 9 | EK[i+1 & 7] >> 7;
              EK += i & 8;
              i &= 7;
       }
} /* ideaExpandKey */

static void
ideaInvertKey(word16 const *EK, word16 DK[IDEAKEYLEN])
{
       int i;
       uint16 t1, t2, t3;
       word16 temp[IDEAKEYLEN];
       word16 *p = temp + IDEAKEYLEN;

       t1 = mulInv(*EK++);
       t2 = –*EK++;
       t3 = –*EK++;
       *––p = mulInv(*EK++);
       *––p = t3;
       *––p = t2;
       *––p = t1;
       for (i = 0; i < IDEAROUNDS–1; i++) {
              t1 = *EK++;
              *––p = *EK++;
              *––p = t1;

              t1 = mulInv(*EK++);
              t2 = –*EK++;
              t3 = –*EK++;
              *––p = mulInv(*EK++);
              *––p = t2;
              *––p = t3;
              *––p = t1;
       }
       t1 = *EK++;
       *––p = *EK++;
       *––p = t1;

       t1 = mulInv(*EK++);
       t2 = –*EK++;
       t3 = –*EK++;
       *––p = mulInv(*EK++);
       *––p = t3;
       *––p = t2;
       *––p = t1;
/* Copy and destroy temp copy */
       memcpy(DK, temp, sizeof(temp));
       for(i=0;i<IDEAKEYLEN;i++)temp[i]=0;
} /* ideaInvertKey */

#ifdef SMALL_CACHE
#define MUL(x,y) (x = mul(low16(x),y))
#else /* !SMALL_CACHE */
#ifdef AVOID_JUMPS
#define MUL(x,y) (x = low16(x–1), t16 = low16((y)–1), \
              t32 = (word32)x*t16 + x + t16 + 1, x = low16(t32), \
              t16 = t32>>16, x = (x–t16) + (x<t16) )
#else /* !AVOID_JUMPS (default) */
#define MUL(x,y) \
       ((t16 = (y)) ? \
              (x=low16(x)) ? \
                     t32 = (word32)x*t16, \
                     x = low16(t32), \
                     t16 = t32>>16, \
                     x = (x–t16)+(x<t16) \
              : \
                     (x = 1–t16) \
       : \
              (x = 1–x))
#endif
#endif

static void
ideaCipher(byte *inbuf, byte *outbuf, word16 *key)
{
       register uint16 x1, x2, x3, x4, s2, s3;
       word16 *in, *out;
#ifndef SMALL_CACHE
       register uint16 t16; /* Temporaries needed by MUL macro */
       register word32 t32;
#endif
       int r = IDEAROUNDS;

       in = (word16 *)inbuf;
       x1 = *in++;  x2 = *in++;
       x3 = *in++;  x4 = *in;
#ifndef HIGHFIRST
       x1 = (x1 >>8) | (x1<<8);
       x2 = (x2 >>8) | (x2<<8);
       x3 = (x3 >>8) | (x3<<8);
       x4 = (x4 >>8) | (x4<<8);
#endif
       do {
              MUL(x1,*key++);
              x2 += *key++;
              x3 += *key++;
              MUL(x4, *key++);

              s3 = x3;
              x3 ^= x1;
              MUL(x3, *key++);
              s2 = x2;
              x2 ^= x4;
              x2 += x3;
              MUL(x2, *key++);
              x3 += x2;

              x1 ^= x2;  x4 ^= x3;

              x2 ^= s3;  x3 ^= s2;
       } while (––r);
       MUL(x1, *key++);
       x3 += *key++;
       x2 += *key++;
       MUL(x4, *key);

       out = (word16 *)outbuf;
#ifdef HIGHFIRST
       *out++ = x1;
       *out++ = x3;
       *out++ = x2;
       *out = x4;
#else /* !HIGHFIRST */
       *out++ = (x1 >>8) | (x1<<8);
       *out++ = (x3 >>8) | (x3<<8);
       *out++ = (x2 >>8) | (x2<<8);
       *out = (x4 >>8) | (x4<<8);
#endif
} /* ideaCipher */

void idea_key(idea_ctx *c, unsigned char *key){
       ideaExpandKey(key,c–>ek);
       ideaInvertKey(c–>ek,c–>dk);
}

void idea_enc(idea_ctx *c, unsigned char *data, int blocks){
       int i;
       unsigned char *d = data;
       for(i=0;i<blocks;i++){
              ideaCipher(d,d,c–>ek);
              d+=8;
       }
}

void idea_dec(idea_ctx *c, unsigned char *data, int blocks){
       int i;
       unsigned char *d = data;
       for(i=0;i<blocks;i++){
              ideaCipher(d,d,c–>dk);
              d+=8;
       }
}

#include <stdio.h>

#ifndef BLOCKS
#ifndef KBYTES
#define KBYTES 1024
#endif
#define BLOCKS (64*KBYTES)
#endif

int
main(void)
{      /* Test driver for IDEA cipher */
       int i, j, k;
       idea_ctx c;
       byte userkey[16];
       word16 EK[IDEAKEYLEN], DK[IDEAKEYLEN];
       byte XX[8], YY[8], ZZ[8];
       word32 long_block[10]; /* 5 blocks */
       long l;
       char *lbp;

       /* Make a sample user key for testing... */
       for(i=0; i<16; i++)
              userkey[i] = i+1;

       idea_key(&c,userkey);

       /* Make a sample plaintext pattern for testing... */
       for (k=0; k<8; k++)
              XX[k] = k;

       idea_enc(&c,XX,1); /* encrypt */

       lbp = (unsigned char *) long_block;
       for(i=0;i<10;i++) long_block[i] = i;
       idea_enc(&c,lbp,5);
       for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx.\n",
                               i/2,long_block[i],long_block[i+1]);

       idea_dec(&c,lbp,3);
       idea_dec(&c,lbp+24,2);

       for(i=0;i<10;i+=2) printf("Block %01d = %08lx %08lx.\n",
                               i/2,long_block[i],long_block[i+1]);

       return 0;       /* normal exit */
} /* main */

GOST

typedef unsigned long u4;
typedef unsigned char byte;

typedef struct {
        u4 k[8];
        /* Constant s–boxes –– set up in gost_init(). */
        char k87[256],k65[256],k43[256],k21[256];
} gost_ctx;

/* Note:  encrypt and decrypt expect full blocks––padding blocks is
          caller's responsibility.  All bulk encryption is done in
          ECB mode by these calls.  Other modes may be added easily
          enough.                                                    */
void gost_enc(gost_ctx *, u4 *, int);
void gost_dec(gost_ctx *, u4 *, int);
void gost_key(gost_ctx *, u4 *);
void gost_init(gost_ctx *);
void gost_destroy(gost_ctx *);

#ifdef __alpha  /* Any other 64–bit machines? */
typedef unsigned int word32;
#else
typedef unsigned long word32;
#endif

kboxinit(gost_ctx *c)
{
        int i;

        byte k8[16] = {14,  4, 13,  1,  2, 15, 11,  8,  3, 10,  6,
                       12,  5,  9,  0,  7 };

        byte k7[16] = {15,  1,  8, 14,  6, 11,  3,  4,  9,  7,  2,
                       13, 12,  0,  5, 10 };
        byte k6[16] = {10,  0,  9, 14,  6,  3, 15,  5,  1, 13, 12,
                        7, 11,  4,  2,  8 };
        byte k5[16] = { 7, 13, 14,  3,  0,  6,  9, 10,  1,  2,  8,
                        5, 11, 12,  4, 15 };
        byte k4[16] = { 2, 12,  4,  1,  7, 10, 11,  6,  8,  5,  3,
                       15, 13,  0, 14,  9 };
        byte k3[16] = {12,  1, 10, 15,  9,  2,  6,  8,  0, 13,  3,
                        4, 14,  7,  5, 11 };
        byte k2[16] = { 4, 11,  2, 14, 15,  0,  8, 13,  3, 12,  9,
                        7,  5, 10,  6,  1 };
        byte k1[16] = {13,  2,  8,  4,  6, 15, 11,  1, 10,  9,  3,
                       14,  5,  0, 12,  7 };

     for (i = 0; i < 256; i++) {
                c–>k87[i] = k8[i >> 4] << 4 | k7[i & 15];
                c–>k65[i] = k6[i >> 4] << 4 | k5[i & 15];
                c–>k43[i] = k4[i >> 4] << 4 | k3[i & 15];
                c–>k21[i] = k2[i >> 4] << 4 | k1[i & 15];
     }
}

static word32
f(gost_ctx *c,word32 x)
{
        x = c–>k87[x>>24 & 255] << 24 | c–>k65[x>>16 & 255] << 16 |
            c–>k43[x>> 8 & 255] <<  8 | c–>k21[x & 255];

     /* Rotate left 11 bits */
     return x<<11 | x>>(32–11);
}

void gostcrypt(gost_ctx *c, word32 *d){
     register word32 n1, n2; /* As named in the GOST */

        n1 = d[0];
        n2 = d[1];

     /* Instead of swapping halves, swap names each round */
        n2 ^= f(c,n1+c–>k[0]); n1 ^= f(c,n2+c–>k[1]);
        n2 ^= f(c,n1+c–>k[2]); n1 ^= f(c,n2+c–>k[3]);
        n2 ^= f(c,n1+c–>k[4]); n1 ^= f(c,n2+c–>k[5]);
        n2 ^= f(c,n1+c–>k[6]); n1 ^= f(c,n2+c–>k[7]);

        n2 ^= f(c,n1+c–>k[0]); n1 ^= f(c,n2+c–>k[1]);
        n2 ^= f(c,n1+c–>k[2]); n1 ^= f(c,n2+c–>k[3]);
        n2 ^= f(c,n1+c–>k[4]); n1 ^= f(c,n2+c–>k[5]);
        n2 ^= f(c,n1+c–>k[6]); n1 ^= f(c,n2+c–>k[7]);

        n2 ^= f(c,n1+c–>k[0]); n1 ^= f(c,n2+c–>k[1]);
        n2 ^= f(c,n1+c–>k[2]); n1 ^= f(c,n2+c–>k[3]);
        n2 ^= f(c,n1+c–>k[4]); n1 ^= f(c,n2+c–>k[5]);
        n2 ^= f(c,n1+c–>k[6]); n1 ^= f(c,n2+c–>k[7]);

        n2 ^= f(c,n1+c–>k[7]); n1 ^= f(c,n2+c–>k[6]);
        n2 ^= f(c,n1+c–>k[5]); n1 ^= f(c,n2+c–>k[4]);
        n2 ^= f(c,n1+c–>k[3]); n1 ^= f(c,n2+c–>k[2]);
        n2 ^= f(c,n1+c–>k[1]); n1 ^= f(c,n2+c–>k[0]);

        d[0] = n2; d[1] = n1;
}

void
gostdecrypt(gost_ctx *c, u4 *d){
     register word32 n1, n2; /* As named in the GOST */

        n1 = d[0]; n2 = d[1];

        n2 ^= f(c,n1+c–>k[0]); n1 ^= f(c,n2+c–>k[1]);
        n2 ^= f(c,n1+c–>k[2]); n1 ^= f(c,n2+c–>k[3]);
        n2 ^= f(c,n1+c–>k[4]); n1 ^= f(c,n2+c–>k[5]);
        n2 ^= f(c,n1+c–>k[6]); n1 ^= f(c,n2+c–>k[7]);

        n2 ^= f(c,n1+c–>k[7]); n1 ^= f(c,n2+c–>k[6]);
        n2 ^= f(c,n1+c–>k[5]); n1 ^= f(c,n2+c–>k[4]);
        n2 ^= f(c,n1+c–>k[3]); n1 ^= f(c,n2+c–>k[2]);
        n2 ^= f(c,n1+c–>k[1]); n1 ^= f(c,n2+c–>k[0]);

        n2 ^= f(c,n1+c–>k[7]); n1 ^= f(c,n2+c–>k[6]);
        n2 ^= f(c,n1+c–>k[5]); n1 ^= f(c,n2+c–>k[4]);
        n2 ^= f(c,n1+c–>k[3]); n1 ^= f(c,n2+c–>k[2]);
        n2 ^= f(c,n1+c–>k[1]); n1 ^= f(c,n2+c–>k[0]);

        n2 ^= f(c,n1+c–>k[7]); n1 ^= f(c,n2+c–>k[6]);
        n2 ^= f(c,n1+c–>k[5]); n1 ^= f(c,n2+c–>k[4]);
        n2 ^= f(c,n1+c–>k[3]); n1 ^= f(c,n2+c–>k[2]);
        n2 ^= f(c,n1+c–>k[1]); n1 ^= f(c,n2+c–>k[0]);

        d[0] = n2; d[1] = n1;
}
void gost_enc(gost_ctx *c, u4 *d, int blocks){
        int i;

        for(i=0;i<blocks;i++){
                gostcrypt(c,d);
                d+=2;
        }
}

void gost_dec(gost_ctx *c, u4 *d, int blocks){
        int i;

        for(i=0;i<blocks;i++){
                gostdecrypt(c,d);
                d+=2;
        }
}

void gost_key(gost_ctx *c, u4 *k){
        int i;
        for(i=0;i<8;i++) c–>k[i]=k[i];
}

void gost_init(gost_ctx *c){
        kboxinit(c);
}

void gost_destroy(gost_ctx *c){
        int i;
        for(i=0;i<8;i++) c–>k[i]=0;
}

void main(void){
        gost_ctx gc;
        u4 k[8],data[10];
        int i;

        /* Initialize GOST context. */
        gost_init(&gc);

        /* Prepare key––a simple key should be OK, with this many rounds! */
        for(i=0;i<8;i++) k[i] = i;
        gost_key(&gc,k);

        /* Try some test vectors. */
        data[0] = 0; data[1] = 0;
        gostcrypt(&gc,data);
        printf("Enc of zero vector:  %08lx %08lx\n",data[0],data[1]);
        gostcrypt(&gc,data);
        printf("Enc of above:        %08lx %08lx\n",data[0],data[1]);
        data[0] = 0xffffffff; data[1] = 0xffffffff;
        gostcrypt(&gc,data);
        printf("Enc of ones vector:  %08lx %08lx\n",data[0],data[1]);
        gostcrypt(&gc,data);
        printf("Enc of above:        %08lx %08lx\n",data[0],data[1]);

        /* Does gost_dec() properly reverse gost_enc()?  Do
           we deal OK with single–block lengths passed in gost_dec()?
           Do we deal OK with different lengths passed in? */

        /* Init data */
        for(i=0;i<10;i++) data[i]=i;
        /* Encrypt data as 5 blocks. */

        gost_enc(&gc,data,5);
        /* Display encrypted data. */

        for(i=0;i<10;i+=2) printf("Block %02d = %08lx %08lx\n",
                                 i/2,data[i],data[i+1]);

        /* Decrypt in different sized chunks. */
        gost_dec(&gc,data,1);
        gost_dec(&gc,data+2,4);
        printf("\n");

        /* Display decrypted data. */
        for(i=0;i<10;i+=2) printf("Block %02d = %08lx %08lx\n",
                                 i/2,data[i],data[i+1]);

        gost_destroy(&gc);
}

BLOWFISH

#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

#ifdef little_endian   /* Eg: Intel */
   #include <alloc.h>
#endif

#include <ctype.h>

#ifdef little_endian   /* Eg: Intel */
   #include <dir.h>
   #include <bios.h>
#endif

#ifdef big_endian
   #include <Types.h>
#endif

typedef struct {
        unsigned long S[4][256],P[18];
} blf_ctx;

#define MAXKEYBYTES 56          /* 448 bits */
// #define little_endian 1              /* Eg: Intel */
#define big_endian 1            /* Eg: Motorola */

void Blowfish_encipher(blf_ctx *,unsigned long *xl, unsigned long *xr);
void Blowfish_decipher(blf_ctx *,unsigned long *xl, unsigned long *xr);

#define N               16
#define noErr            0
#define DATAERROR         –1
#define KEYBYTES         8

FILE*         SubkeyFile;

unsigned long F(blf_ctx *bc, unsigned long x)
{
   unsigned short a;
   unsigned short b;
   unsigned short c;
   unsigned short d;
   unsigned long  y;
   d = x & 0×00FF;
   x >>= 8;
   c = x & 0×00FF;
   x >>= 8;
   b = x & 0×00FF;
   x >>= 8;
   a = x & 0×00FF;
   //y = ((S[0][a] + S[1][b]) ^ S[2][c]) + S[3][d];
   y = bc–>S[0][a] + bc–>S[1][b];
   y = y ^ bc–>S[2][c];
   y = y + bc–>S[3][d];

   return y;
}

void Blowfish_encipher(blf_ctx *c,unsigned long *xl, unsigned long *xr)
{
   unsigned long  Xl;
   unsigned long  Xr;
   unsigned long  temp;
   short          i;

   Xl = *xl;
   Xr = *xr;

   for (i = 0; i < N; ++i) {
      Xl = Xl ^ c–>P[i];
      Xr = F(c,Xl) ^ Xr;

      temp = Xl;
      Xl = Xr;
      Xr = temp;
   }

   temp = Xl;
   Xl = Xr;
   Xr = temp;

   Xr = Xr ^ c–>P[N];
   Xl = Xl ^ c–>P[N + 1];

   *xl = Xl;
   *xr = Xr;
}

void Blowfish_decipher(blf_ctx *c, unsigned long *xl, unsigned long *xr)
{
   unsigned long  Xl;
   unsigned long  Xr;
   unsigned long  temp;
   short          i;

   Xl = *xl;
   Xr = *xr;
   for (i = N + 1; i > 1; ––i) {
      Xl = Xl ^ c–>P[i];
      Xr = F(c,Xl) ^ Xr;

      /* Exchange Xl and Xr */
      temp = Xl;
      Xl = Xr;
      Xr = temp;
   }

   /* Exchange Xl and Xr */
   temp = Xl;
   Xl = Xr;
   Xr = temp;

   Xr = Xr ^ c–>P[1];
   Xl = Xl ^ c–>P[0];

   *xl = Xl;
   *xr = Xr;
}

short InitializeBlowfish(blf_ctx *c, char key[], short keybytes)
{
   short          i;
   short          j;
   short          k;
   short          error;
   short          numread;
   unsigned long  data;
   unsigned long  datal;
   unsigned long  datar;

unsigned long ks0[] = {
0xd1310ba6, 0x98dfb5ac, 0x2ffd72db, 0xd01adfb7, 0xb8e1afed, 0x6a267e96,
0xba7c9045, 0xf12c7f99, 0x24a19947, 0xb3916cf7, 0×0801f2e2, 0x858efc16,
0x636920d8, 0x71574e69, 0xa458fea3, 0xf4933d7e, 0×0d95748f, 0x728eb658,
0x718bcd58, 0x82154aee, 0x7b54a41d, 0xc25a59b5, 0x9c30d539, 0x2af26013,
0xc5d1b023, 0x286085f0, 0xca417918, 0xb8db38ef, 0x8e79dcb0, 0x603a180e,
0x6c9e0e8b, 0xb01e8a3e, 0xd71577c1, 0xbd314b27, 0x78af2fda, 0x55605c60,
0xe65525f3, 0xaa55ab94, 0x57489862, 0x63e81440, 0x55ca396a, 0x2aab10b6,
0xb4cc5c34, 0x1141e8ce, 0xa15486af, 0x7c72e993, 0xb3ee1411, 0x636fbc2a,
0x2ba9c55d, 0x741831f6, 0xce5c3e16, 0x9b87931e, 0xafd6ba33, 0x6c24cf5c,
0x7a325381, 0x28958677, 0x3b8f4898, 0x6b4bb9af, 0xc4bfe81b, 0x66282193,
0x61d809cc, 0xfb21a991, 0x487cac60, 0x5dec8032, 0xef845d5d, 0xe98575b1,
0xdc262302, 0xeb651b88, 0x23893e81, 0xd396acc5, 0×0f6d6ff3, 0x83f44239,
0x2e0b4482, 0xa4842004, 0x69c8f04a, 0x9e1f9b5e, 0x21c66842, 0xf6e96c9a,
0x670c9c61, 0xabd388f0, 0x6a51a0d2, 0xd8542f68, 0x960fa728, 0xab5133a3,
0x6eef0b6c, 0x137a3be4, 0xba3bf050, 0x7efb2a98, 0xa1f1651d, 0x39af0176,
0x66ca593e, 0x82430e88, 0x8cee8619, 0x456f9fb4, 0x7d84a5c3, 0x3b8b5ebe,
0xe06f75d8, 0x85c12073, 0x401a449f, 0x56c16aa6, 0x4ed3aa62, 0x363f7706,
0x1bfedf72, 0x429b023d, 0x37d0d724, 0xd00a1248, 0xdb0fead3, 0x49f1c09b,
0×075372c9, 0x80991b7b, 0x25d479d8, 0xf6e8def7, 0xe3fe501a, 0xb6794c3b,
0x976ce0bd, 0×04c006ba, 0xc1a94fb6, 0x409f60c4, 0x5e5c9ec2, 0x196a2463,
0x68fb6faf, 0x3e6c53b5, 0x1339b2eb, 0x3b52ec6f, 0x6dfc511f, 0x9b30952c,
0xcc814544, 0xaf5ebd09, 0xbee3d004, 0xde334afd, 0x660f2807, 0x192e4bb3,
0xc0cba857, 0x45c8740f, 0xd20b5f39, 0xb9d3fbdb, 0x5579c0bd, 0x1a60320a,
0xd6a100c6, 0x402c7279, 0x679f25fe, 0xfb1fa3cc, 0x8ea5e9f8, 0xdb3222f8,
0x3c7516df, 0xfd616b15, 0x2f501ec8, 0xad0552ab, 0x323db5fa, 0xfd238760,
0x53317b48, 0x3e00df82, 0x9e5c57bb, 0xca6f8ca0, 0x1a87562e, 0xdf1769db,
0xd542a8f6, 0x287effc3, 0xac6732c6, 0x8c4f5573, 0x695b27b0, 0xbbca58c8,
0xe1ffa35d, 0xb8f011a0, 0x10fa3d98, 0xfd2183b8, 0x4afcb56c, 0x2dd1d35b,
0x9a53e479, 0xb6f84565, 0xd28e49bc, 0x4bfb9790, 0xe1ddf2da, 0xa4cb7e33,
0x62fb1341, 0xcee4c6e8, 0xef20cada, 0x36774c01, 0xd07e9efe, 0x2bf11fb4,
0x95dbda4d, 0xae909198, 0xeaad8e71, 0x6b93d5a0, 0xd08ed1d0, 0xafc725e0,
0x8e3c5b2f, 0x8e7594b7, 0x8ff6e2fb, 0xf2122b64, 0x8888b812, 0x900df01c,
0x4fad5ea0, 0x688fc31c, 0xd1cff191, 0xb3a8c1ad, 0x2f2f2218, 0xbe0e1777,
0xea752dfe, 0x8b021fa1, 0xe5a0cc0f, 0xb56f74e8, 0x18acf3d6, 0xce89e299,
0xb4a84fe0, 0xfd13e0b7, 0x7cc43b81, 0xd2ada8d9, 0x165fa266, 0x80957705,
0x93cc7314, 0x211a1477, 0xe6ad2065, 0x77b5fa86, 0xc75442f5, 0xfb9d35cf,
0xebcdaf0c, 0x7b3e89a0, 0xd6411bd3, 0xae1e7e49, 0×00250e2d, 0x2071b35e,
0x226800bb, 0x57b8e0af, 0x2464369b, 0xf009b91e, 0x5563911d, 0x59dfa6aa,
0x78c14389, 0xd95a537f, 0x207d5ba2, 0×02e5b9c5, 0x83260376, 0x6295cfa9,
0x11c81968, 0x4e734a41, 0xb3472dca, 0x7b14a94a, 0x1b510052, 0x9a532915,
0xd60f573f, 0xbc9bc6e4, 0x2b60a476, 0x81e67400, 0×08ba6fb5, 0x571be91f,
0xf296ec6b, 0x2a0dd915, 0xb6636521, 0xe7b9f9b6, 0xff34052e, 0xc5855664,
0x53b02d5d, 0xa99f8fa1, 0×08ba4799, 0x6e85076a};
unsigned long ks1[] = {
0x4b7a70e9, 0xb5b32944, 0xdb75092e, 0xc4192623, 0xad6ea6b0, 0x49a7df7d,
0x9cee60b8, 0x8fedb266, 0xecaa8c71, 0x699a17ff, 0x5664526c, 0xc2b19ee1,
0x193602a5, 0x75094c29, 0xa0591340, 0xe4183a3e, 0x3f54989a, 0x5b429d65,
0x6b8fe4d6, 0x99f73fd6, 0xa1d29c07, 0xefe830f5, 0x4d2d38e6, 0xf0255dc1,
0x4cdd2086, 0x8470eb26, 0x6382e9c6, 0×021ecc5e, 0×09686b3f, 0x3ebaefc9,
0x3c971814, 0x6b6a70a1, 0x687f3584, 0x52a0e286, 0xb79c5305, 0xaa500737,
0x3e07841c, 0x7fdeae5c, 0x8e7d44ec, 0x5716f2b8, 0xb03ada37, 0xf0500c0d,
0xf01c1f04, 0×0200b3ff, 0xae0cf51a, 0x3cb574b2, 0x25837a58, 0xdc0921bd,
0xd19113f9, 0x7ca92ff6, 0x94324773, 0x22f54701, 0x3ae5e581, 0x37c2dadc,
0xc8b57634, 0x9af3dda7, 0xa9446146, 0×0fd0030e, 0xecc8c73e, 0xa4751e41,
0xe238cd99, 0x3bea0e2f, 0x3280bba1, 0x183eb331, 0x4e548b38, 0x4f6db908,
0x6f420d03, 0xf60a04bf, 0x2cb81290, 0x24977c79, 0x5679b072, 0xbcaf89af,
0xde9a771f, 0xd9930810, 0xb38bae12, 0xdccf3f2e, 0x5512721f, 0x2e6b7124,
0x501adde6, 0x9f84cd87, 0x7a584718, 0x7408da17, 0xbc9f9abc, 0xe94b7d8c,
0xec7aec3a, 0xdb851dfa, 0x63094366, 0xc464c3d2, 0xef1c1847, 0x3215d908,
0xdd433b37, 0x24c2ba16, 0x12a14d43, 0x2a65c451, 0x50940002, 0x133ae4dd,
0x71dff89e, 0x10314e55, 0x81ac77d6, 0x5f11199b, 0×043556f1, 0xd7a3c76b,
0x3c11183b, 0x5924a509, 0xf28fe6ed, 0x97f1fbfa, 0x9ebabf2c, 0x1e153c6e,
0x86e34570, 0xeae96fb1, 0x860e5e0a, 0x5a3e2ab3, 0x771fe71c, 0x4e3d06fa,
0x2965dcb9, 0x99e71d0f, 0x803e89d6, 0x5266c825, 0x2e4cc978, 0x9c10b36a,
0xc6150eba, 0x94e2ea78, 0xa5fc3c53, 0x1e0a2df4, 0xf2f74ea7, 0x361d2b3d,
0x1939260f, 0x19c27960, 0x5223a708, 0xf71312b6, 0xebadfe6e, 0xeac31f66,
0xe3bc4595, 0xa67bc883, 0xb17f37d1, 0×018cff28, 0xc332ddef, 0xbe6c5aa5,
0x65582185, 0x68ab9802, 0xeecea50f, 0xdb2f953b, 0x2aef7dad, 0x5b6e2f84,
0x1521b628, 0x29076170, 0xecdd4775, 0x619f1510, 0x13cca830, 0xeb61bd96,
0×0334fe1e, 0xaa0363cf, 0xb5735c90, 0x4c70a239, 0xd59e9e0b, 0xcbaade14,
0xeecc86bc, 0x60622ca7, 0x9cab5cab, 0xb2f3846e, 0x648b1eaf, 0x19bdf0ca,
0xa02369b9, 0x655abb50, 0x40685a32, 0x3c2ab4b3, 0x319ee9d5, 0xc021b8f7,
0x9b540b19, 0x875fa099, 0x95f7997e, 0x623d7da8, 0xf837889a, 0x97e32d77,
0x11ed935f, 0x16681281, 0×0e358829, 0xc7e61fd6, 0x96dedfa1, 0x7858ba99,
0x57f584a5, 0x1b227263, 0x9b83c3ff, 0x1ac24696, 0xcdb30aeb, 0x532e3054,
0x8fd948e4, 0x6dbc3128, 0x58ebf2ef, 0x34c6ffea, 0xfe28ed61, 0xee7c3c73,
0x5d4a14d9, 0xe864b7e3, 0x42105d14, 0x203e13e0, 0x45eee2b6, 0xa3aaabea,
0xdb6c4f15, 0xfacb4fd0, 0xc742f442, 0xef6abbb5, 0x654f3b1d, 0x41cd2105,
0xd81e799e, 0x86854dc7, 0xe44b476a, 0x3d816250, 0xcf62a1f2, 0x5b8d2646,
0xfc8883a0, 0xc1c7b6a3, 0x7f1524c3, 0x69cb7492, 0x47848a0b, 0x5692b285,
0×095bbf00, 0xad19489d, 0x1462b174, 0x23820e00, 0x58428d2a, 0×0c55f5ea,
0x1dadf43e, 0x233f7061, 0x3372f092, 0x8d937e41, 0xd65fecf1, 0x6c223bdb,
0x7cde3759, 0xcbee7460, 0x4085f2a7, 0xce77326e, 0xa6078084, 0x19f8509e,
0xe8efd855, 0x61d99735, 0xa969a7aa, 0xc50c06c2, 0x5a04abfc, 0x800bcadc,
0x9e447a2e, 0xc3453484, 0xfdd56705, 0×0e1e9ec9, 0xdb73dbd3, 0x105588cd,
0x675fda79, 0xe3674340, 0xc5c43465, 0x713e38d8, 0x3d28f89e, 0xf16dff20,
0x153e21e7, 0x8fb03d4a, 0xe6e39f2b, 0xdb83adf7};
unsigned long ks2[] = {
0xe93d5a68, 0x948140f7, 0xf64c261c, 0x94692934, 0x411520f7, 0x7602d4f7,
0xbcf46b2e, 0xd4a20068, 0xd4082471, 0x3320f46a, 0x43b7d4b7, 0x500061af,
0x1e39f62e, 0x97244546, 0x14214f74, 0xbf8b8840, 0x4d95fc1d, 0x96b591af,
0x70f4ddd3, 0x66a02f45, 0xbfbc09ec, 0×03bd9785, 0x7fac6dd0, 0x31cb8504,
0x96eb27b3, 0x55fd3941, 0xda2547e6, 0xabca0a9a, 0x28507825, 0x530429f4,
0×0a2c86da, 0xe9b66dfb, 0x68dc1462, 0xd7486900, 0x680ec0a4, 0x27a18dee,
0x4f3ffea2, 0xe887ad8c, 0xb58ce006, 0x7af4d6b6, 0xaace1e7c, 0xd3375fec,
0xce78a399, 0x406b2a42, 0x20fe9e35, 0xd9f385b9, 0xee39d7ab, 0x3b124e8b,
0x1dc9faf7, 0x4b6d1856, 0x26a36631, 0xeae397b2, 0x3a6efa74, 0xdd5b4332,
0x6841e7f7, 0xca7820fb, 0xfb0af54e, 0xd8feb397, 0x454056ac, 0xba489527,
0x55533a3a, 0x20838d87, 0xfe6ba9b7, 0xd096954b, 0x55a867bc, 0xa1159a58,
0xcca92963, 0x99e1db33, 0xa62a4a56, 0x3f3125f9, 0x5ef47e1c, 0x9029317c,
0xfdf8e802, 0×04272f70, 0x80bb155c, 0×05282ce3, 0x95c11548, 0xe4c66d22,
0x48c1133f, 0xc70f86dc, 0×07f9c9ee, 0x41041f0f, 0x404779a4, 0x5d886e17,
0x325f51eb, 0xd59bc0d1, 0xf2bcc18f, 0x41113564, 0x257b7834, 0x602a9c60,
0xdff8e8a3, 0x1f636c1b, 0×0e12b4c2, 0×02e1329e, 0xaf664fd1, 0xcad18115,
0x6b2395e0, 0x333e92e1, 0x3b240b62, 0xeebeb922, 0x85b2a20e, 0xe6ba0d99,
0xde720c8c, 0x2da2f728, 0xd0127845, 0x95b794fd, 0x647d0862, 0xe7ccf5f0,
0x5449a36f, 0x877d48fa, 0xc39dfd27, 0xf33e8d1e, 0×0a476341, 0x992eff74,
0x3a6f6eab, 0xf4f8fd37, 0xa812dc60, 0xa1ebddf8, 0x991be14c, 0xdb6e6b0d,
0xc67b5510, 0x6d672c37, 0x2765d43b, 0xdcd0e804, 0xf1290dc7, 0xcc00ffa3,
0xb5390f92, 0x690fed0b, 0x667b9ffb, 0xcedb7d9c, 0xa091cf0b, 0xd9155ea3,
0xbb132f88, 0x515bad24, 0x7b9479bf, 0x763bd6eb, 0x37392eb3, 0xcc115979,
0x8026e297, 0xf42e312d, 0x6842ada7, 0xc66a2b3b, 0x12754ccc, 0x782ef11c,
0x6a124237, 0xb79251e7, 0×06a1bbe6, 0x4bfb6350, 0x1a6b1018, 0x11caedfa,
0x3d25bdd8, 0xe2e1c3c9, 0x44421659, 0×0a121386, 0xd90cec6e, 0xd5abea2a,
0x64af674e, 0xda86a85f, 0xbebfe988, 0x64e4c3fe, 0x9dbc8057, 0xf0f7c086,
0x60787bf8, 0x6003604d, 0xd1fd8346, 0xf6381fb0, 0x7745ae04, 0xd736fccc,
0x83426b33, 0xf01eab71, 0xb0804187, 0x3c005e5f, 0x77a057be, 0xbde8ae24,
0x55464299, 0xbf582e61, 0x4e58f48f, 0xf2ddfda2, 0xf474ef38, 0x8789bdc2,
0x5366f9c3, 0xc8b38e74, 0xb475f255, 0x46fcd9b9, 0x7aeb2661, 0x8b1ddf84,
0x846a0e79, 0x915f95e2, 0x466e598e, 0x20b45770, 0x8cd55591, 0xc902de4c,
0xb90bace1, 0xbb8205d0, 0x11a86248, 0x7574a99e, 0xb77f19b6, 0xe0a9dc09,
0x662d09a1, 0xc4324633, 0xe85a1f02, 0×09f0be8c, 0x4a99a025, 0x1d6efe10,
0x1ab93d1d, 0×0ba5a4df, 0xa186f20f, 0x2868f169, 0xdcb7da83, 0x573906fe,
0xa1e2ce9b, 0x4fcd7f52, 0x50115e01, 0xa70683fa, 0xa002b5c4, 0×0de6d027,
0x9af88c27, 0x773f8641, 0xc3604c06, 0x61a806b5, 0xf0177a28, 0xc0f586e0,
0×006058aa, 0x30dc7d62, 0x11e69ed7, 0x2338ea63, 0x53c2dd94, 0xc2c21634,
0xbbcbee56, 0x90bcb6de, 0xebfc7da1, 0xce591d76, 0x6f05e409, 0x4b7c0188,
0x39720a3d, 0x7c927c24, 0x86e3725f, 0x724d9db9, 0x1ac15bb4, 0xd39eb8fc,
0xed545578, 0×08fca5b5, 0xd83d7cd3, 0x4dad0fc4, 0x1e50ef5e, 0xb161e6f8,
0xa28514d9, 0x6c51133c, 0x6fd5c7e7, 0x56e14ec4, 0x362abfce, 0xddc6c837,
0xd79a3234, 0x92638212, 0x670efa8e, 0x406000e0};
unsigned long ks3[] = {
0x3a39ce37, 0xd3faf5cf, 0xabc27737, 0x5ac52d1b, 0x5cb0679e, 0x4fa33742,
0xd3822740, 0x99bc9bbe, 0xd5118e9d, 0xbf0f7315, 0xd62d1c7e, 0xc700c47b,
0xb78c1b6b, 0x21a19045, 0xb26eb1be, 0x6a366eb4, 0x5748ab2f, 0xbc946e79,
0xc6a376d2, 0x6549c2c8, 0x530ff8ee, 0x468dde7d, 0xd5730a1d, 0x4cd04dc6,
0x2939bbdb, 0xa9ba4650, 0xac9526e8, 0xbe5ee304, 0xa1fad5f0, 0x6a2d519a,
0x63ef8ce2, 0x9a86ee22, 0xc089c2b8, 0x43242ef6, 0xa51e03aa, 0x9cf2d0a4,
0x83c061ba, 0x9be96a4d, 0x8fe51550, 0xba645bd6, 0x2826a2f9, 0xa73a3ae1,
0x4ba99586, 0xef5562e9, 0xc72fefd3, 0xf752f7da, 0x3f046f69, 0x77fa0a59,
0x80e4a915, 0x87b08601, 0x9b09e6ad, 0x3b3ee593, 0xe990fd5a, 0x9e34d797,
0x2cf0b7d9, 0×022b8b51, 0x96d5ac3a, 0×017da67d, 0xd1cf3ed6, 0x7c7d2d28,
0x1f9f25cf, 0xadf2b89b, 0x5ad6b472, 0x5a88f54c, 0xe029ac71, 0xe019a5e6,
0x47b0acfd, 0xed93fa9b, 0xe8d3c48d, 0x283b57cc, 0xf8d56629, 0x79132e28,
0x785f0191, 0xed756055, 0xf7960e44, 0xe3d35e8c, 0x15056dd4, 0x88f46dba,
0×03a16125, 0×0564f0bd, 0xc3eb9e15, 0x3c9057a2, 0x97271aec, 0xa93a072a,
0x1b3f6d9b, 0x1e6321f5, 0xf59c66fb, 0x26dcf319, 0x7533d928, 0xb155fdf5,
0×03563482, 0x8aba3cbb, 0x28517711, 0xc20ad9f8, 0xabcc5167, 0xccad925f,
0x4de81751, 0x3830dc8e, 0x379d5862, 0x9320f991, 0xea7a90c2, 0xfb3e7bce,
0x5121ce64, 0x774fbe32, 0xa8b6e37e, 0xc3293d46, 0x48de5369, 0x6413e680,
0xa2ae0810, 0xdd6db224, 0x69852dfd, 0×09072166, 0xb39a460a, 0x6445c0dd,
0x586cdecf, 0x1c20c8ae, 0x5bbef7dd, 0x1b588d40, 0xccd2017f, 0x6bb4e3bb,
0xdda26a7e, 0x3a59ff45, 0x3e350a44, 0xbcb4cdd5, 0x72eacea8, 0xfa6484bb,
0x8d6612ae, 0xbf3c6f47, 0xd29be463, 0x542f5d9e, 0xaec2771b, 0xf64e6370,
0x740e0d8d, 0xe75b1357, 0xf8721671, 0xaf537d5d, 0x4040cb08, 0x4eb4e2cc,
0x34d2466a, 0×0115af84, 0xe1b00428, 0x95983a1d, 0×06b89fb4, 0xce6ea048,
0x6f3f3b82, 0x3520ab82, 0×011a1d4b, 0x277227f8, 0x611560b1, 0xe7933fdc,
0xbb3a792b, 0x344525bd, 0xa08839e1, 0x51ce794b, 0x2f32c9b7, 0xa01fbac9,
0xe01cc87e, 0xbcc7d1f6, 0xcf0111c3, 0xa1e8aac7, 0x1a908749, 0xd44fbd9a,
0xd0dadecb, 0xd50ada38, 0×0339c32a, 0xc6913667, 0x8df9317c, 0xe0b12b4f,
0xf79e59b7, 0x43f5bb3a, 0xf2d519ff, 0x27d9459c, 0xbf97222c, 0x15e6fc2a,
0×0f91fc71, 0x9b941525, 0xfae59361, 0xceb69ceb, 0xc2a86459, 0x12baa8d1,
0xb6c1075e, 0xe3056a0c, 0x10d25065, 0xcb03a442, 0xe0ec6e0e, 0x1698db3b,
0x4c98a0be, 0x3278e964, 0x9f1f9532, 0xe0d392df, 0xd3a0342b, 0x8971f21e,
0x1b0a7441, 0x4ba3348c, 0xc5be7120, 0xc37632d8, 0xdf359f8d, 0x9b992f2e,
0xe60b6f47, 0×0fe3f11d, 0xe54cda54, 0x1edad891, 0xce6279cf, 0xcd3e7e6f,
0x1618b166, 0xfd2c1d05, 0x848fd2c5, 0xf6fb2299, 0xf523f357, 0xa6327623,
0x93a83531, 0x56cccd02, 0xacf08162, 0x5a75ebb5, 0x6e163697, 0x88d273cc,
0xde966292, 0x81b949d0, 0x4c50901b, 0x71c65614, 0xe6c6c7bd, 0x327a140a,
0x45e1d006, 0xc3f27b9a, 0xc9aa53fd, 0x62a80f00, 0xbb25bfe2, 0x35bdd2f6,
0x71126905, 0xb2040222, 0xb6cbcf7c, 0xcd769c2b, 0x53113ec0, 0x1640e3d3,
0x38abbd60, 0x2547adf0, 0xba38209c, 0xf746ce76, 0x77afa1c5, 0x20756060,
0x85cbfe4e, 0x8ae88dd8, 0x7aaaf9b0, 0x4cf9aa7e, 0x1948c25c, 0×02fb8a8c,
0×01c36ae4, 0xd6ebe1f9, 0x90d4f869, 0xa65cdea0, 0x3f09252d, 0xc208e69f,
0xb74e6132, 0xce77e25b, 0x578fdfe3, 0x3ac372e6};

/* Initialize s–boxes without file read. */
        for(i=0;i<256;i++){
                c–>S[0][i] = ks0[i];
                c–>S[1][i] = ks1[i];
                c–>S[2][i] = ks2[i];
                c–>S[3][i] = ks3[i];
        }

        j = 0;
        for (i = 0; i < N + 2; ++i) {
                data = 0×00000000;
                for (k = 0; k < 4; ++k) {
                        data = (data << 8) | key[j];
                        j = j + 1;
                        if (j >= keybytes) {
                                j = 0;
                        }
                }
        c–>P[i] = c–>P[i] ^ data;
        }

     datal = 0×00000000;
        datar = 0×00000000;

        for (i = 0; i < N + 2; i += 2) {
                Blowfish_encipher(c,&datal, &datar);

                c–>P[i] = datal;
                c–>P[i + 1] = datar;
        }

        for (i = 0; i < 4; ++i) {
                for (j = 0; j < 256; j += 2) {

                        Blowfish_encipher(c,&datal, &datar);

                        c–>S[i][j] = datal;
                        c–>S[i][j + 1] = datar;
                }
        }
}

void blf_key(blf_ctx *c, char *k, int len){
        InitializeBlowfish(c,k,len);
}

void blf_enc(blf_ctx *c, unsigned long *data, int blocks){
        unsigned long *d;
        int i;

        d = data;
        for(i=0;i<blocks;i++){
                Blowfish_encipher(c,d,d+1);
                d += 2;
        }
}

void blf_dec(blf_ctx *c, unsigned long *data, int blocks){
        unsigned long *d;
        int i;
        d = data;
        for(i=0;i<blocks;i++){
                Blowfish_decipher(c,d,d+1);
                d += 2;
        }
}

void main(void){
        blf_ctx c;
        char key[]="AAAAA";
        unsigned long data[10];
        int i;

        for(i=0;i<10;i++) data[i] = i;

        blf_key(&c,key,5);
        blf_enc(&c,data,5);
        blf_dec(&c,data,1);
        blf_dec(&c,data+2,4);
        for(i=0;i<10;i+=2) printf("Block %01d decrypts to: %08lx %08lx.\n",
                                  i/2,data[i],data[i+1]);
}

3–Way

#define   STRT_E   0×0b0b /* round constant of first encryption round */
#define   STRT_D   0xb1b1 /* round constant of first decryption round */
#define     NMBR       11 /* number of rounds is 11                   */

typedef   unsigned long int  word32 ;
                 /* the program only works correctly if long = 32bits */
typedef unsigned long u4;
typedef unsigned char u1;

typedef struct {
        u4 k[3],ki[3], ercon[NMBR+1],drcon[NMBR+1];
} twy_ctx;

/* Note:  encrypt and decrypt expect full blocks––padding blocks is
          caller's responsibility.  All bulk encryption is done in
          ECB mode by these calls.  Other modes may be added easily
          enough.                                                    */

/* destroy:  Context. */
/* Scrub context of all sensitive data. */
void twy_destroy(twy_ctx *);

/* encrypt:  Context, ptr to data block, # of blocks. */
void twy_enc(twy_ctx *, u4 *, int);

/* decrypt:  Context, ptr to data block, # of blocks. */
void twy_dec(twy_ctx *, u4 *, int);
/* key:  Context, ptr to key data. */
void twy_key(twy_ctx *, u4 *);
/* ACCODE––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– */
/* End of AC code prototypes and structures.                         */
/* ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– */
void mu(word32 *a)       /* inverts the order of the bits of a */
{
int i ;
word32 b[3] ;

b[0] = b[1] = b[2] = 0 ;
for( i=0 ; i<32 ; i++ )
   {
   b[0] <<= 1 ; b[1] <<= 1 ; b[2] <<= 1 ;
   if(a[0]&1) b[2] |= 1 ;
   if(a[1]&1) b[1] |= 1 ;
   if(a[2]&1) b[0] |= 1 ;

   a[0] >>= 1 ; a[1] >>= 1 ; a[2] >>= 1 ;
   }
a[0] = b[0] ;      a[1] = b[1] ;      a[2] = b[2] ;
}

void gamma(word32 *a)   /* the nonlinear step */
{
word32 b[3] ;

b[0] = a[0] ^ (a[1]|(~a[2])) ;
b[1] = a[1] ^ (a[2]|(~a[0])) ;
b[2] = a[2] ^ (a[0]|(~a[1])) ;

a[0] = b[0] ;      a[1] = b[1] ;      a[2] = b[2] ;
}

void theta(word32 *a)    /* the linear step */
{
word32 b[3];

b[0] = a[0] ^  (a[0]>>16) ^ (a[1]<<16) ^     (a[1]>>16) ^ (a[2]<<16) ^
               (a[1]>>24) ^ (a[2]<<8)  ^     (a[2]>>8)  ^ (a[0]<<24) ^
               (a[2]>>16) ^ (a[0]<<16) ^     (a[2]>>24) ^ (a[0]<<8)  ;
b[1] = a[1] ^  (a[1]>>16) ^ (a[2]<<16) ^     (a[2]>>16) ^ (a[0]<<16) ^
               (a[2]>>24) ^ (a[0]<<8)  ^     (a[0]>>8)  ^ (a[1]<<24) ^
               (a[0]>>16) ^ (a[1]<<16) ^     (a[0]>>24) ^ (a[1]<<8)  ;
b[2] = a[2] ^  (a[2]>>16) ^ (a[0]<<16) ^     (a[0]>>16) ^ (a[1]<<16) ^
               (a[0]>>24) ^ (a[1]<<8)  ^     (a[1]>>8)  ^ (a[2]<<24) ^
               (a[1]>>16) ^ (a[2]<<16) ^     (a[1]>>24) ^ (a[2]<<8)  ;
a[0] = b[0] ;      a[1] = b[1] ;      a[2] = b[2] ;
}

void pi_1(word32 *a)
{
a[0] = (a[0]>>10) ^ (a[0]<<22);
a[2] = (a[2]<<1)  ^ (a[2]>>31);
}

void pi_2(word32 *a)
{
a[0] = (a[0]<<1)  ^ (a[0]>>31);
a[2] = (a[2]>>10) ^ (a[2]<<22);
}

void rho(word32 *a)    /* the round function       */
{
theta(a) ;
pi_1(a) ;
gamma(a) ;
pi_2(a) ;
}

void rndcon_gen(word32 strt,word32 *rtab)
{                           /* generates the round constants */
int i ;

for(i=0 ; i<=NMBR ; i++ )
   {
   rtab[i] = strt ;
   strt <<= 1 ;
   if( strt&0x10000 ) strt ^= 0x11011 ;
   }
}

/* Modified slightly to fit the caller's needs. */
void encrypt(twy_ctx *c, word32 *a)
{
char i ;
for( i=0 ; i<NMBR ; i++ )
   {
   a[0] ^= c–>k[0] ^ (c–>ercon[i]<<16) ;
   a[1] ^= c–>k[1] ;
   a[2] ^= c–>k[2] ^ c–>ercon[i] ;
   rho(a) ;
   }
a[0] ^= c–>k[0] ^ (c–>ercon[NMBR]<<16) ;
a[1] ^= c–>k[1] ;
a[2] ^= c–>k[2] ^ c–>ercon[NMBR] ;
theta(a) ;
}

/* Modified slightly to meet caller's needs. */
void decrypt(twy_ctx *c, word32 *a)
{
char i ;

mu(a) ;
for( i=0 ; i<NMBR ; i++ )
   {
   a[0] ^= c–>ki[0] ^ (c–>drcon[i]<<16) ;
   a[1] ^= c–>ki[1] ;
   a[2] ^= c–>ki[2] ^ c–>drcon[i] ;
   rho(a) ;
   }
a[0] ^= c–>ki[0] ^ (c–>drcon[NMBR]<<16) ;
a[1] ^= c–>ki[1] ;
a[2] ^= c–>ki[2] ^ c–>drcon[NMBR] ;
theta(a) ;
mu(a) ;
}

void twy_key(twy_ctx *c, u4 *key){
        c–>ki[0] = c–>k[0] = key[0];
        c–>ki[1] = c–>k[1] = key[1];
        c–>ki[2] = c–>k[2] = key[2];
        theta(c–>ki);
        mu(c–>ki);
        rndcon_gen(STRT_E,c–>ercon);
        rndcon_gen(STRT_D,c–>drcon);

}

/* Encrypt in ECB mode. */
void twy_enc(twy_ctx *c, u4 *data, int blkcnt){
        u4 *d;
        int i;

        d = data;
        for(i=0;i<blkcnt;i++) {
                encrypt(c,d);
                d +=3;
        }
}

/* Decrypt in ECB mode. */
void twy_dec(twy_ctx *c, u4 *data, int blkcnt){
        u4 *d;
        int i;

        d = data;
        for(i=0;i<blkcnt;i++){
                decrypt(c,d);
                d+=3;
        }
}

/* Scrub sensitive values from memory before deallocating. */
void twy_destroy(twy_ctx *c){
        int i;

        for(i=0;i<3;i++) c–>k[i] = c–>ki[i] = 0;
}
void printvec(char *chrs, word32 *d){
        printf("%20s : %08lx %08lx %08lx \n",chrs,d[2],d[1],d[0]);
}

main()
{
twy_ctx gc;
word32 a[9],k[3];
int i;

/* Test vector 1. */

k[0]=k[1]=k[2]=0;
a[0]=a[1]=a[2]=1;
twy_key(&gc,k);

printf("**********\n");
printvec("KEY = ",k);
printvec("PLAIN = ",a);
encrypt(&gc,a);
printvec("CIPHER = ",a);

/* Test vector 2. */

k[0]=6;k[1]=5;k[2]=4;
a[0]=3;a[1]=2;a[2]=1;
twy_key(&gc,k);

printf("**********\n");
printvec("KEY = ",k);
printvec("PLAIN = ",a);
encrypt(&gc,a);
printvec("CIPHER = ",a);

/* Test vector 3. */

k[2]=0xbcdef012;k[1]=0x456789ab;k[0]=0xdef01234;
a[2]=0×01234567;a[1]=0x9abcdef0;a[0]=0x23456789;
twy_key(&gc,k);

printf("**********\n");
printvec("KEY = ",k);
printvec("PLAIN = ",a);
encrypt(&gc,a);
printvec("CIPHER = ",a);

/* Test vector 4. */

k[2]=0xcab920cd;k[1]=0xd6144138;k[0]=0xd2f05b5e;
a[2]=0xad21ecf7;a[1]=0x83ae9dc4;a[0]=0x4059c76e;
twy_key(&gc,k);

printf("**********\n");
printvec("KEY = ",k);
printvec("PLAIN = ",a);
encrypt(&gc,a);
printvec("CIPHER = ",a);

/*  TEST VALUES

key        : 00000000 00000000 00000000
plaintext  : 00000001 00000001 00000001
ciphertext : ad21ecf7 83ae9dc4 4059c76e

key        : 00000004 00000005 00000006
plaintext  : 00000001 00000002 00000003
ciphertext : cab920cd d6144138 d2f05b5e

key        : bcdef012 456789ab def01234
plaintext  : 01234567 9abcdef0 23456789
ciphertext : 7cdb76b2 9cdddb6d 0aa55dbb

key        : cab920cd d6144138 d2f05b5e
plaintext  : ad21ecf7 83ae9dc4 4059c76e
ciphertext : 15b155ed 6b13f17c 478ea871

*/

/* Enc/dec test: */
for(i=0;i<9;i++) a[i]=i;
twy_enc(&gc,a,3);
for(i=0;i<9;i+=3) printf("Block %01d encrypts to %08lx %08lx %08lx\n",
                         i/3,a[i],a[i+1],a[i+2]);

twy_dec(&gc,a,2);
twy_dec(&gc,a+6,1);

 for(i=0;i<9;i+=3) printf("Block %01d decrypts to %08lx %08lx %08lx\n",
                         i/3,a[i],a[i+1],a[i+2]);
}

RC5

#include <stdio.h>

/* An RC5 context needs to know how many rounds it has, and its subkeys. */
typedef struct {
        u4 *xk;
        int nr;
} rc5_ctx;

/* Where possible, these should be replaced with actual rotate instructions.
   For Turbo C++, this is done with _lrotl and _lrotr. */

#define ROTL32(X,C) (((X)<<(C))|((X)>>(32–(C))))
#define ROTR32(X,C) (((X)>>(C))|((X)<<(32–(C))))
/* Function prototypes for dealing with RC5 basic operations. */
void rc5_init(rc5_ctx *, int);
void rc5_destroy(rc5_ctx *);
void rc5_key(rc5_ctx *, u1 *, int);
void rc5_encrypt(rc5_ctx *, u4 *, int);
void rc5_decrypt(rc5_ctx *, u4 *, int);

/* Function implementations for RC5. */

/* Scrub out all sensitive values. */
void rc5_destroy(rc5_ctx *c){
        int i;
     for(i=0;i<(c–>nr)*2+2;i++) c–>xk[i]=0;
     free(c–>xk);
}

/* Allocate memory for rc5 context's xk and such. */
void rc5_init(rc5_ctx *c, int rounds){
     c–>nr = rounds;
     c–>xk = (u4 *) malloc(4*(rounds*2+2));
}

void rc5_encrypt(rc5_ctx *c, u4 *data, int blocks){
        u4 *d,*sk;
        int h,i,rc;

     d = data;
        sk = (c–>xk)+2;
        for(h=0;h<blocks;h++){
                d[0] += c–>xk[0];
                d[1] += c–>xk[1];
                for(i=0;i<c–>nr*2;i+=2){
                        d[0] ^= d[1];
                        rc = d[1] & 31;
                        d[0] = ROTL32(d[0],rc);
                        d[0] += sk[i];
               d[1] ^= d[0];
                        rc = d[0] & 31;
                        d[1] = ROTL32(d[1],rc);
                        d[1] += sk[i+1];
/*printf("Round %03d : %08lx %08lx  sk= %08lx %08lx\n",i/2,
                                d[0],d[1],sk[i],sk[i+1]);*/
                }
          d+=2;
        }
}

void rc5_decrypt(rc5_ctx *c, u4 *data, int blocks){
     u4 *d,*sk;
        int h,i,rc;

     d = data;
        sk = (c–>xk)+2;
     for(h=0;h<blocks;h++){
                for(i=c–>nr*2–2;i>=0;i–=2){
/*printf("Round %03d: %08lx %08lx  sk: %08lx %08lx\n",
        i/2,d[0],d[1],sk[i],sk[i+1]); */
                        d[1] –= sk[i+1];
                        rc = d[0] & 31;
                        d[1] = ROTR32(d[1],rc);
                        d[1] ^= d[0];
                        d[0] –= sk[i];
                        rc = d[1] & 31;
                        d[0] = ROTR32(d[0],rc);
               d[0] ^= d[1];
                }
                d[0] –= c–>xk[0];
                d[1] –= c–>xk[1];
        d+=2;
     }
}

void rc5_key(rc5_ctx *c, u1 *key, int keylen){
     u4 *pk,A,B; /* padded key */
     int xk_len, pk_len, i, num_steps,rc;
     u1 *cp;

     xk_len = c–>nr*2 + 2;
     pk_len = keylen/4;
     if((keylen%4)!=0) pk_len += 1;

     pk = (u4 *) malloc(pk_len * 4);
     if(pk==NULL) {
          printf("An error occurred!\n");
          exit(–1);
     }

     /* Initialize pk –– this should work on Intel machines, anyway.... */
     for(i=0;i<pk_len;i++) pk[i]=0;
     cp = (u1 *)pk;
     for(i=0;i<keylen;i++) cp[i]=key[i];

     /* Initialize xk. */
     c–>xk[0] = 0xb7e15163; /* P32 */
     for(i=1;i<xk_len;i++) c–>xk[i] = c–>xk[i–1] + 0x9e3779b9; /* Q32 */

     /* TESTING */
     A = B = 0;
     for(i=0;i<xk_len;i++) {
          A = A + c–>xk[i];
          B = B ^ c–>xk[i];
     }

     /* Expand key into xk. */
     if(pk_len>xk_len) num_steps = 3*pk_len;else num_steps = 3*xk_len;

     A = B = 0;
     for(i=0;i<num_steps;i++){
          A = c–>xk[i%xk_len] = ROTL32(c–>xk[i%xk_len] + A + B,3);
          rc = (A+B) & 31;
          B = pk[i%pk_len] = ROTL32(pk[i%pk_len] + A + B,rc);

     }

     /* Clobber sensitive data before deallocating memory. */
     for(i=0;i<pk_len;i++) pk[i] =0;

     free(pk);
}

void main(void){
     rc5_ctx c;
     u4 data[8];
     char key[] = "ABCDE";
     int i;

     printf("–––––––––––––––––––––––––––––––––––––––––––––––––\n");

        for(i=0;i<8;i++) data[i] = i;
     rc5_init(&c,10); /* 10 rounds */
     rc5_key(&c,key,5);

        rc5_encrypt(&c,data,4);
        printf("Encryptions:\n");
        for(i=0;i<8;i+=2) printf("Block %01d = %08lx %08lx\n",
                                 i/2,data[i],data[i+1]);

        rc5_decrypt(&c,data,2);
     rc5_decrypt(&c,data+4,2);
        printf("Decryptions:\n");
        for(i=0;i<8;i+=2) printf("Block %01d = %08lx %08lx\n",
                                 i/2,data[i],data[i+1]);
}

A5

typedef struct {
        unsigned long r1,r2,r3;
} a5_ctx;

static int threshold(r1, r2, r3)
unsigned int r1;
unsigned int r2;
unsigned int r3;
{
int total;

  total = (((r1 >>  9) & 0x1) == 1) +
          (((r2 >> 11) & 0x1) == 1) +
          (((r3 >> 11) & 0x1) == 1);

  if (total > 1)
    return (0);
  else
    return (1);
}

unsigned long clock_r1(ctl, r1)
int ctl;
unsigned long r1;
{
unsigned long feedback;

  ctl ^= ((r1 >> 9) & 0x1);
  if (ctl)
  {
    feedback = (r1 >> 18) ^ (r1 >> 17) ^ (r1 >> 16) ^ (r1 >> 13);
    r1 = (r1 << 1) & 0x7ffff;
    if (feedback & 0×01)
      r1 ^= 0×01;
  }
  return (r1);
}

unsigned long clock_r2(ctl, r2)
int ctl;
unsigned long r2;
{
unsigned long feedback;

  ctl ^= ((r2 >> 11) & 0x1);
  if (ctl)
  {
    feedback = (r2 >> 21) ^ (r2 >> 20) ^ (r2 >> 16) ^ (r2 >> 12);
    r2 = (r2 << 1) & 0x3fffff;
    if (feedback & 0×01)
      r2 ^= 0×01;
  }
  return (r2);
}

unsigned long clock_r3(ctl, r3)
int ctl;
unsigned long r3;
{
unsigned long feedback;

  ctl ^= ((r3 >> 11) & 0x1);
  if (ctl)
  {
    feedback = (r3 >> 22) ^ (r3 >> 21) ^ (r3 >> 18) ^ (r3 >> 17);
    r3 = (r3 << 1) & 0x7fffff;
    if (feedback & 0×01)
      r3 ^= 0×01;
  }
  return (r3);
}
int keystream(key, frame, alice, bob)
unsigned char *key;   /* 64 bit session key              */
unsigned long frame;  /* 22 bit frame sequence number    */
unsigned char *alice; /* 114 bit Alice to Bob key stream */
unsigned char *bob;   /* 114 bit Bob to Alice key stream */
{
unsigned long r1;   /* 19 bit shift register */
unsigned long r2;   /* 22 bit shift register */
unsigned long r3;   /* 23 bit shift register */
int i;              /* counter for loops     */
int clock_ctl;      /* xored with clock enable on each shift register */
unsigned char *ptr; /* current position in keystream */
unsigned char byte; /* byte of keystream being assembled */
unsigned int bits;  /* number of bits of keystream in byte */
unsigned int bit;   /* bit output from keystream generator */
  /* Initialise shift registers from session key */
  r1 = (key[0] | (key[1] << 8) | (key[2] << 16) ) & 0x7ffff;
  r2 = ((key[2] >> 3) | (key[3] << 5) | (key[4] << 13) | (key[5] << 21)) &
0x3fffff;
  r3 = ((key[5] >> 1) | (key[6] << 7) | (key[7] << 15) ) & 0x7fffff;

  /* Merge frame sequence number into shift register state, by xor'ing it
   * into the feedback path
   */

  for (i=0;i<22;i++)
  {
    clock_ctl = threshold(r1, r2, r2);
    r1 = clock_r1(clock_ctl, r1);
    r2 = clock_r2(clock_ctl, r2);
    r3 = clock_r3(clock_ctl, r3);
    if (frame & 1)
    {
      r1 ^= 1;
      r2 ^= 1;
      r3 ^= 1;
    }
    frame = frame >> 1;
  }

  /* Run shift registers for 100 clock ticks to allow frame number to
   * be diffused into all the bits of the shift registers
   */

  for (i=0;i<100;i++)
  {
    clock_ctl = threshold(r1, r2, r2);
    r1 = clock_r1(clock_ctl, r1);
    r2 = clock_r2(clock_ctl, r2);
    r3 = clock_r3(clock_ctl, r3);
  }

  /* Produce 114 bits of Alice–>Bob key stream */
  ptr = alice;
  bits = 0;
  byte = 0;
  for (i=0;i<114;i++)
  {
    clock_ctl = threshold(r1, r2, r2);
    r1 = clock_r1(clock_ctl, r1);
    r2 = clock_r2(clock_ctl, r2);
    r3 = clock_r3(clock_ctl, r3);

    bit = ((r1 >> 18) ^ (r2 >> 21) ^ (r3 >> 22)) & 0×01;
    byte = (byte << 1) | bit;
    bits++;
    if (bits == 8)
    {
      *ptr = byte;
      ptr++;
      bits = 0;
      byte = 0;
    }
  }
  if (bits)
    *ptr = byte;

  /* Run shift registers for another 100 bits to hide relationship between
   * Alice–>Bob key stream and Bob–>Alice key stream.
   */

  for (i=0;i<100;i++)
  {
    clock_ctl = threshold(r1, r2, r2);
    r1 = clock_r1(clock_ctl, r1);
    r2 = clock_r2(clock_ctl, r2);
    r3 = clock_r3(clock_ctl, r3);
  }

  /* Produce 114 bits of Bob–>Alice key stream */

  ptr = bob;
  bits = 0;
  byte = 0;
  for (i=0;i<114;i++)
  {
    clock_ctl = threshold(r1, r2, r2);
    r1 = clock_r1(clock_ctl, r1);
    r2 = clock_r2(clock_ctl, r2);
    r3 = clock_r3(clock_ctl, r3);

    bit = ((r1 >> 18) ^ (r2 >> 21) ^ (r3 >> 22)) & 0×01;
    byte = (byte << 1) | bit;
    bits++;
    if (bits == 8)
    {
      *ptr = byte;
      ptr++;
      bits = 0;
      byte = 0;
    }
  }
  if (bits)
    *ptr = byte;

  return (0);
}
void a5_key(a5_ctx *c, char *k){
        c–>r1 = k[0]<<11|k[1]<<3 | k[2]>>5          ; /* 19 */
        c–>r2 = k[2]<<17|k[3]<<9 | k[4]<<1 | k[5]>>7; /* 22 */
        c–>r3 = k[5]<<15|k[6]<<8 | k[7]             ; /* 23 */
}

/* Step one bit in A5, return 0 or 1 as output bit. */
int a5_step(a5_ctx *c){
        int control;
        control = threshold(c–>r1,c–>r2,c–>r3);
        c–>r1 = clock_r1(control,c–>r1);
        c–>r2 = clock_r2(control,c–>r2);
        c–>r3 = clock_r3(control,c–>r3);
        return( (c–>r1^c–>r2^c–>r3)&1);
}

/* Encrypts a buffer of len bytes. */
void a5_encrypt(a5_ctx *c, char *data, int len){
        int i,j;
        char t;

        for(i=0;i<len;i++){
                for(j=0;j<8;j++) t = t<<1 | a5_step(c);
                data[i]^=t;
        }
}

void a5_decrypt(a5_ctx *c, char *data, int len){
        a5_encrypt(c,data,len);
}

void main(void){
        a5_ctx c;
        char data[100];
        char key[] = {1,2,3,4,5,6,7,8};
        int i,flag;

        for(i=0;i<100;i++) data[i] = i;

        a5_key(&c,key);
        a5_encrypt(&c,data,100);

        a5_key(&c,key);
        a5_decrypt(&c,data,1);
        a5_decrypt(&c,data+1,99);

        flag = 0;
        for(i=0;i<100;i++) if(data[i]!=i)flag = 1;
        if(flag)printf("Decrypt failed\n"); else printf("Decrypt succeeded\n");
}

SEAL

#undef SEAL_DEBUG

#define ALG_OK 0
#define ALG_NOTOK 1
#define WORDS_PER_SEAL_CALL 1024

typedef struct {
     unsigned long t[520]; /* 512 rounded up to a multiple of 5 + 5*/
     unsigned long s[265]; /* 256 rounded up to a multiple of 5 + 5*/
     unsigned long r[20];  /* 16 rounded up to multiple of 5 */
        unsigned long counter; /* 32–bit synch value. */
        unsigned long ks_buf[WORDS_PER_SEAL_CALL];
        int ks_pos;
} seal_ctx;

#define ROT2(x) (((x) >> 2) | ((x) << 30))
#define ROT9(x) (((x) >> 9) | ((x) << 23))
#define ROT8(x) (((x) >> 8) | ((x) << 24))
#define ROT16(x) (((x) >> 16) | ((x) << 16))
#define ROT24(x) (((x) >> 24) | ((x) << 8))
#define ROT27(x) (((x) >> 27) | ((x) << 5))

#define WORD(cp)  ((cp[0] << 24)|(cp[1] << 16)|(cp[2] << 8)|(cp[3]))

#define F1(x, y, z) (((x) & (y)) | ((~(x)) & (z)))
#define F2(x, y, z) ((x)^(y)^(z))
#define F3(x, y, z) (((x) & (y)) | ((x) & (z)) | ((y) & (z)))
#define F4(x, y, z) ((x)^(y)^(z))

int g(in, i, h)
unsigned char *in;
int i;
unsigned long *h;
{
unsigned long h0;
unsigned long h1;
unsigned long h2;
unsigned long h3;
unsigned long h4;
unsigned long a;
unsigned long b;unsigned long c;
unsigned long d;
unsigned long e;
unsigned char *kp;
unsigned long w[80];
unsigned long temp;

     kp = in;
     h0 = WORD(kp); kp += 4;
     h1 = WORD(kp); kp += 4;
     h2 = WORD(kp); kp += 4;
     h3 = WORD(kp); kp += 4;
     h4 = WORD(kp); kp += 4;

     w[0] = i;
     for (i=1;i<16;i++)
          w[i] = 0;
     for (i=16;i<80;i++)
          w[i] = w[i–3]^w[i–8]^w[i–14]^w[i–16];

     a = h0;
     b = h1;
     c = h2;
     d = h3;
     e = h4;

     for (i=0;i<20;i++)
     {
          temp = ROT27(a) + F1(b, c, d) + e + w[i] + 0x5a827999;
          e = d;
          d = c;
          c = ROT2(b);
          b = a;
          a = temp;
     }
     for (i=20;i<40;i++)
     {
          temp = ROT27(a) + F2(b, c, d) + e + w[i] + 0x6ed9eba1;
          e = d;
          d = c;
          c = ROT2(b);
          b = a;
          a = temp;
     }
     for (i=40;i<60;i++)
     {
          temp = ROT27(a) + F3(b, c, d) + e + w[i] + 0x8f1bbcdc;
          e = d;
          d = c;
          c = ROT2(b);
          b = a;
          a = temp;
     }
     for (i=60;i<80;i++)
     {
          temp = ROT27(a) + F4(b, c, d) + e + w[i] + 0xca62c1d6;
          e = d;
          d = c;
          c = ROT2(b);
          b = a;
          a = temp;
     }
     h[0] = h0+a;
     h[1] = h1+b;
     h[2] = h2+c;
     h[3] = h3+d;
     h[4] = h4+e;

     return (ALG_OK);
}

unsigned long gamma(a, i)
unsigned char *a;
int i;
{
unsigned long h[5];
     (void) g(a, i/5, h);
     return h[i % 5];
}

int  seal_init(seal_ctx *result, unsigned char *key)
{
int i;
unsigned long h[5];

     for (i=0;i<510;i+=5)
          g(key, i/5, &(result–>t[i]));
     /* horrible special case for the end */
     g(key, 510/5, h);
     for (i=510;i<512;i++)
          result–>t[i] = h[i–510];
     /* 0x1000 mod 5 is +1, so have horrible special case for the start */
     g(key, (–1+0x1000)/5, h);
     for (i=0;i<4;i++)
          result–>s[i] = h[i+1];
     for (i=4;i<254;i+=5)
          g(key, (i+0x1000)/5, &(result–>s[i]));
     /* horrible special case for the end */
     g(key, (254+0x1000)/5, h);
     for (i=254;i<256;i++)
          result–>s[i] = h[i–254];
     /* 0x2000 mod 5 is +2, so have horrible special case at the start */
     g(key, (–2+0x2000)/5, h);
     for (i=0;i<3;i++)
          result–>r[i] = h[i+2];
     for (i=3;i<13;i+=5)
          g(key, (i+0x2000)/5, &(result–>r[i]));
     /* horrible special case for the end */
     g(key, (13+0x2000)/5, h);
     for (i=13;i<16;i++)
          result–>r[i] = h[i–13];
     return (ALG_OK);
}

int seal(seal_ctx *key, unsigned long in, unsigned long *out)
{
int i;
int j;
int l;
unsigned long a;
unsigned long b;
unsigned long c;
unsigned long d;
unsigned short p;
unsigned short q;
unsigned long n1;
unsigned long n2;
unsigned long n3;
unsigned long n4;
unsigned long *wp;

     wp = out;

     for (l=0;l<4;l++)
     {
          a = in ^ key–>r[4*l];
          b = ROT8(in) ^ key–>r[4*l+1];
          c = ROT16(in) ^ key–>r[4*l+2];
          d = ROT24(in) ^ key–>r[4*l+3];

          for (j=0;j<2;j++)
          {
               p = a & 0x7fc;
               b += key–>t[p/4];
               a = ROT9(a);

               p = b & 0x7fc;
               c += key–>t[p/4];
               b = ROT9(b);

               p = c & 0x7fc;
               d += key–>t[p/4];
               c = ROT9(c);

               p = d & 0x7fc;
               a += key–>t[p/4];
               d = ROT9(d);

          }
          n1 = d;
          n2 = b;
          n3 = a;
          n4 = c;

          p = a & 0x7fc;
          b += key–>t[p/4];
          a = ROT9(a);

          p = b & 0x7fc;
          c += key–>t[p/4];
          b = ROT9(b);

          p = c & 0x7fc;
          d += key–>t[p/4];
          c = ROT9(c);

          p = d & 0x7fc;
          a += key–>t[p/4];
          d = ROT9(d);

        /* This generates 64 32–bit words, or 256 bytes of keystream. */
          for (i=0;i<64;i++)
          {
               p = a & 0x7fc;
               b += key–>t[p/4];
               a = ROT9(a);
               b ^= a;

               q = b & 0x7fc;
               c ^= key–>t[q/4];
               b = ROT9(b);
               c += b;

               p = (p+c) & 0x7fc;
               d += key–>t[p/4];
               c = ROT9(c);
               d ^= c;

               q = (q+d) & 0x7fc;
               a ^= key–>t[q/4];
               d = ROT9(d);
               a += d;

               p = (p+a) & 0x7fc;
               b ^= key–>t[p/4];
               a = ROT9(a);

               q = (q+b) & 0x7fc;
               c += key–>t[q/4];
               b = ROT9(b);

               p = (p+c) & 0x7fc;
               d ^= key–>t[p/4];
               c = ROT9(c);

               q = (q+d) & 0x7fc;
               a += key–>t[q/4];
               d = ROT9(d);

               *wp = b + key–>s[4*i];
               wp++;
               *wp = c ^ key–>s[4*i+1];
               wp++;
               *wp = d + key–>s[4*i+2];
               wp++;
               *wp = a ^ key–>s[4*i+3];
               wp++;

               if (i & 1)
               {
                    a += n3;
                    c += n4;
               }
               else
               {
                    a += n1;
                    c += n2;
               }

          }
     }
     return (ALG_OK);
}

/* Added call to refill ks_buf and reset counter and ks_pos. */
void seal_refill_buffer(seal_ctx *c){
        seal(c,c–>counter,c–>ks_buf);
        c–>counter++;
        c–>ks_pos = 0;
}

void seal_key(seal_ctx *c, unsigned char *key){
        seal_init(c,key);
        c–>counter = 0;  /* By default, init to zero. */
        c–>ks_pos = WORDS_PER_SEAL_CALL;
                /* Refill keystream buffer on next call. */
}

/* This encrypts the next w words with SEAL. */
void seal_encrypt(seal_ctx *c, unsigned long *data_ptr, int w){
        int i;

        for(i=0;i<w;i++){
                if(c–>ks_pos>=WORDS_PER_SEAL_CALL) seal_refill_buffer(c);
                data_ptr[i]^=c–>ks_buf[c–>ks_pos];
                c–>ks_pos++;
        }
}

void seal_decrypt(seal_ctx *c, unsigned long *data_ptr, int w) {
        seal_encrypt(c,data_ptr,w);
}

void seal_resynch(seal_ctx *c, unsigned long synch_word){
        c–>counter = synch_word;
        c–>ks_pos = WORDS_PER_SEAL_CALL;
}

void main(void){
        seal_ctx sc;
        unsigned long buf[1000],t;
        int i,flag;
        unsigned char key[] =
                {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19};

        printf("1\n");
        seal_key(&sc,key);

        printf("2\n");
        for(i=0;i<1000;i++) buf[i]=0;
        printf("3\n");
        seal_encrypt(&sc,buf,1000);
        printf("4\n");
        t = 0;
        for(i=0;i<1000;i++) t = t ^ buf[i];
                printf("XOR of buf is %08lx.\n",t);

        seal_key(&sc,key);
        seal_decrypt(&sc,buf,1);
        seal_decrypt(&sc,buf+1,999);
        flag = 0;
        for(i=0;i<1000;i++) if(buf[i]!=0)flag=1;
        if(flag) printf("Decrypt failed.\n");
        else printf("Decrypt succeeded.\n");
}

References

1. ABA Bank Card Standard, "Management and Use of Personal Information Numbers," Aids from ABA, Catalog no. 207213, American Bankers Association, 1979.
2. ABA Document 4.3, "Key Management Standard," American Bankers Association, 1980.
3. M. Abadi, J. Feigenbaum, and J. Kilian, "On Hiding Information from an Oracle," Proceedings of the 19th ACM Symposium on the Theory of Computing, 1987, pp. 195–203.
4. M. Abadi, J. Feigenbaum, and J. Kilian, "On Hiding Information from an Oracle," Journal of Computer and System Sciences, v. 39, n. 1, Aug 1989, pp. 21–50.
5. M. Abadi and R. Needham, "Prudent Engineering Practice for Cryptographic Protocols," Research Report 125, Digital Equipment Corp Systems Research Center, Jun 1994.
6. C.M. Adams, "On Immunity Against Biham and Shamir's 'Differential Cryptanalysis,'" Information Processing Letters, v. 41, 14 Feb 1992, pp. 77–80.
7. C.M. Adams, "Simple and Effective Key Scheduling for Symmetric Ciphers," Workshop on Selected Areas in Cryptography—Workshop Record, Kingston, Ontario, 5–6 May 1994, pp. 129–133.
8. C.M. Adams and H. Meijer, "Security–Related Comments Regarding McEliece's Public–Key Cryptosystem," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 224–230.
9. C.M. Adams and S.E. Tavares, "The Structured Design of Cryptographically Good S–Boxes," Journal of Cryptology, v. 3, n. 1, 1990, pp. 27–41.
10. C.M. Adams and S.E. Tavares, "Designing S–Boxes for Ciphers Resistant to Differential Cryptanalysis," Proceedings of the 3rd Symposium on State and Progress of Research in Cryptography, Rome, Italy, 15–16 Feb 1993, pp. 181–190.
11. W. Adams and D. Shanks, "Strong Primality Tests That Are Not Sufficient," Mathematics of Computation, v. 39, 1982, pp. 255–300.
12. W.W. Adams and L.J. Goldstein, Introduction to Number Theory, Englewood Cliffs, N.J.: Prentice–Hall, 1976.
13. B.S. Adiga and P. Shankar, "Modified Lu–Lee Cryptosystem," Electronics Letters, v. 21, n. 18, 29 Aug 1985, pp. 794–795.
14. L.M. Adleman, "A Subexponential Algorithm for the Discrete Logarithm Problem with Applications to Cryptography," Proceedings of the IEEE 20th Annual Symposium of Foundations of Computer Science, 1979, pp. 55–60.
15. L.M. Adleman, "On Breaking Generalized Knapsack Public Key Cryptosystems," Proceedings of the 15th ACM Symposium on Theory of Computing, 1983, pp. 402– 412.
16. L.M. Adleman, "Factoring Numbers Using Singular Integers," Proceedings of the 23rd Annual ACM Symposium on the Theory of Computing, 1991, pp. 64–71.
17. L.M. Adleman, "Molecular Computation of Solutions to Combinatorial Problems," Science, v. 266, n. 11, Nov 1994, p. 1021.
18. L.M. Adleman, D. Estes, and K. McCurley, "Solving Bivariate Quadratic Congruences in Random Polynomial Time," Mathematics of Computation, v. 48, n. 177, Jan 1987, pp. 17–28.
19. L.M. Adleman, C. Pomerance, and R.S. Rumeley, "On Distinguishing Prime Numbers from Composite Numbers," Annals of Mathematics, v. 117, n. 1, 1983, pp. 173–206.
20. L.M. Adleman and R.L. Rivest, "How to Break the Lu–Lee (COMSAT) Public–Key Cryptosystem," MIT Laboratory for Computer Science, Jul 1979.
21. G.B. Agnew, "Random Sources for Cryptographic Systems," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 77–81.
22. G.B. Agnew, R.C. Mullin, I.M. Onyszchuk, and S.A. Vanstone, "An Implementation for a Fast Public–Key Cryptosystem," Journal of Cryptology, v. 3, n. 2, 1991, pp. 63–79.
23. G.B. Agnew, R.C. Mullin, and S.A. Vanstone, "A Fast Elliptic Curve Cryptosystem," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 706–708.
24. G.B. Agnew, R.C. Mullin, and S.A. Vanstone, "Improved Digital Signature Scheme Based on Discrete Exponentiation," Electronics Letters, v. 26, n. 14, 5 Jul 1990, pp. 1024–1025.
25. G.B. Agnew, R.C. Mullin, and S.A. Vanstone, "On the Development of a Fast Elliptic Curve Cryptosystem," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 482– 287.
26. G.B. Agnew, R.C. Mullin, and S.A. Vanstone, "An Implementation of Elliptic Curve Cryptosystems over F2155," IEEE Selected Areas of Communications, v. 11, n. 5, Jun 1993, pp. 804–813.
27. A. Aho, J. Hopcroft, and J. Ullman, The Design and Analysis of Computer Algorithms, Addison–Wesley, 1974.
28. S.G. Akl, "Digital Signatures: A Tutorial Survey," Computer, v. 16, n. 2, Feb 1983, pp. 15–24.
29. S.G. Akl, "On the Security of Compressed Encodings," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 209–230.
30. S.G. Akl and H. Meijer, "A Fast Pseudo–Random Permutation Generator with Applications to Cryptology," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 269–275.
31. M. Alabbadi and S.B. Wicker, "Security of Xinmei Digital Signature Scheme," Electronics Letters, v. 28, n. 9, 23 Apr 1992, pp. 890–891.
32. M. Alabbadi and S.B. Wicker, "Digital Signature Schemes Based on Error–Correcting Codes," Proceedings of the 1993 IEEE–ISIT, IEEE Press, 1993, p. 199.
33. M. Alabbadi and S.B. Wicker, "Cryptanalysis of the Harn and Wang Modification of the Xinmei Digital Signature Scheme," Electronics Letters, v. 28, n. 18, 27 Aug 1992, pp. 1756–1758.
34. K. Alagappan and J. Tardo, "SPX Guide: Prototype Public Key Authentication Service," Digital Equipment Corp., May 1991.
35. W. Alexi, B.–Z. Chor, O. Goldreich, and C.P. Schnorr, "RSA and Rabin Functions: Certain Parts Are as Hard as the Whole," Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science, 1984, pp. 449–457.
36. W. Alexi, B.–Z. Chor, O. Goldreich, and C.P. Schnorr, "RSA and Rabin Functions: Certain Parts are as Hard as the Whole," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 194–209.
37. Ameritech Mobile Communications et al., "Cellular Digital Packet Data System Specifications: Part 406: Airlink Security," CDPD Industry Input Coordinator, Costa Mesa, Calif., Jul 1993.
38. H.R. Amirazizi, E.D. Karnin, and J.M. Reyneri, "Compact Knapsacks are Polynomial Solvable," ACM SIGACT News, v. 15, 1983, pp. 20–22.
39. R.J. Anderson, "Solving a Class of Stream Ciphers," Cryptologia, v. 14, n. 3, Jul 1990, pp. 285–288.
40. R.J. Anderson, "A Second Generation Electronic Wallet," ESORICS 92, Proceedings of the Second European Symposium on Research in Computer Security, Springer–Verlag, 1992, pp. 411–418.
41. R.J. Anderson, "Faster Attack on Certain Stream Ciphers," Electronics Letters, v. 29, n. 15, 22 Jul 1993, pp. 1322–1323.
42. R.J. Anderson, "Derived Sequence Attacks on Stream Ciphers," presented at the rump session of CRYPTO '93, Aug 1993.
43. R.J. Anderson, "Why Cryptosystems Fail," 1st ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 215–227.
44. R.J. Anderson, "Why Cryptosystems Fail," Communications of the ACM, v. 37, n. 11, Nov 1994, pp. 32–40.
45. R.J. Anderson, "On Fibonacci Keystream Generators," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
46. R.J. Anderson, "Searching for the Optimum Correlation Attack," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
47. R.J. Anderson and T.M.A. Lomas, "Fortifying Key Negotiation Schemes with Poorly Chosen Passwords," Electronics Letters, v. 30, n. 13, 23 Jun 1994, pp. 1040–1041.
48. R.J. Anderson and R. Needham, "Robustness Principles for Public Key Protocols," Advances in Cryptology—CRYPTO '95 Proceedings, Springer–Verlag, 1995, to appear.
49. D. Andleman and J. Reeds, "On the Cryptanalysis of Rotor Machines and Substitution–Permutation Networks," IEEE Transactions on Information Theory, v. IT–28, n. 4, Jul 1982, pp. 578–584.
50. ANSI X3.92, "American National Standard for Data Encryption Algorithm (DEA)," American National Standards Institute, 1981.
51. ANSI X3.105, "American National Standard for Information Systems—Data Link Encryption," American National Standards Institute, 1983.
52. ANSI X3.106, "American National Standard for Information Systems—Data Encryption Algorithm—Modes of Operation," American National Standards Institute, 1983.
53. ANSI X9.8, "American National Standard for Personal Information Number (PIN) Management and Security," American Bankers Association, 1982.
54. ANSI X9.9 (Revised), "American National Standard for Financial Institution Message Authentication (Wholesale)," American Bankers Association, 1986.
55. ANSI X9.17 (Revised), "American National Standard for Financial Institution Key Management (Wholesale)," American Bankers Association, 1985.
56. ANSI X9.19, "American National Standard for Retail Message Authentication," American Bankers Association, 1985.
57. ANSI X9.23, "American National Standard for Financial Institution Message Encryption," American Bankers Association, 1988.
58. ANSI X9.24, "Draft Proposed American National Standard for Retail Key Management," American Bankers Association, 1988.
59. ANSI X9.26 (Revised), "American National Standard for Financial Institution Sign–On Authentication for Wholesale Financial Transaction," American Bankers Association, 1990.
60. ANSI X9.30, "Working Draft: Public Key Cryptography Using Irreversible Algorithms for the Financial Services Industry," American Bankers Association, Aug 1994.
61. ANSI X9.31, "Working Draft: Public Key Cryptography Using Reversible Algorithms for the Financial Services Industry," American Bankers Association, Mar 1993.
62. K. Aoki and K. Ohta, "Differential–Linear Cryptanalysis of FEAL–8," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. A3.4.1–11. (In Japanese.)
63. K. Araki and T. Sekine, "On the Conspiracy Problem of the Generalized Tanaka's Cryptosystem," IEICE Transactions, v. E74, n. 8, Aug 1991, pp. 2176–2178.
64. S. Araki, K. Aoki, and K. Ohta, "The Best Linear Expression Search for FEAL," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. A4.4.1–10.
65. C. Asmuth and J. Bloom, "A Modular Approach to Key Safeguarding," IEEE Transactions on Information Theory, v. IT–29, n. 2, Mar 1983, pp. 208–210.
66. D. Atkins, M. Graff, A.K. Lenstra, and P.C. Leyland, "The Magic Words are Squeamish Ossifrage," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 263–277.
67. AT&T, "T7001 Random Number Generator," Data Sheet, Aug 1986.
68. AT&T, "AT&T Readying New Spy–Proof Phone for Big Military and Civilian Markets," The Report on AT&T, 2 Jun 1986, pp. 6–7.
69. AT&T, "T7002/T7003 Bit Slice Multiplier," product announcement, 1987.
70. AT&T, "Telephone Security Device TSD 3600—User's Manual," AT&T, 20 Sep 1992.
71. Y. Aumann and U. Feige, "On Message Proof Systems with Known Space Verifiers," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 85–99.
72. R.G. Ayoub, An Introduction to the Theory of Numbers, Providence, RI: American Mathematical Society, 1963.
73. A. Aziz and W. Diffie, "Privacy and Authentication for Wireless Local Area Networks," IEEE Personal Communications, v. 1, n. 1, 1994, pp. 25–31.
74. A. Bahreman and J.D. Tygar, "Certified Electronic Mail," Proceedings of the Internet Society 1994 Workshop on Network and Distributed System Security, The Internet Society, 1994, pp. 3–19.
75. D. Balenson, "Automated Distribution of Cryptographic Keys Using the Financial Institution Key Management Standard," IEEE Communications Magazine, v. 23, n. 9, Sep 1985, pp. 41–46.
76. D. Balenson, "Privacy Enhancement for Internet Electronic Mail: Part III: Algorithms, Modes, and Identifiers," RFC 1423, Feb 1993.
77. D. Balenson, C.M. Ellison, S.B. Lipner, and S.T. Walker, "A New Approach to Software Key Escrow Encryption," TIS Report #520, Trusted Information Systems, Aug 94.
78. R. Ball, Mathematical Recreations and Essays, New York: MacMillan, 1960.
79. J. Bamford, The Puzzle Palace, Boston: Houghton Mifflin, 1982.
80. J. Bamford and W. Madsen, The Puzzle Palace, Second Edition, Penguin Books, 1995.
81. S.K. Banerjee, "High Speed Implementation of DES," Computers & Security, v. 1, 1982, pp. 261–267.
82. Z. Baodong, "MC–Veiled Linear Transform Public Key Cryptosystem," Acta Electronica Sinica, v. 20, n. 4, Apr 1992, pp. 21–24. (In Chinese.)
83. P.H. Bardell, "Analysis of Cellular Automata Used as Pseudorandom Pattern Generators," Proceedings of 1990 International Test Conference, pp. 762–768.
84. T. Baritaud, H. Gilbert, and M. Girault, "FFT Hashing is not Collision–Free," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 35–44.
85. C. Barker, "An Industry Perspective of the CCEP," 2nd Annual AIAA Computer Security Conference Proceedings, 1986.
86. W.G. Barker, Cryptanalysis of the Hagelin Cryptograph, Aegean Park Press, 1977.
87. P. Barrett, "Implementing the Rivest Shamir and Adleman Public Key Encryption Algorithm on a Standard Digital Signal Processor," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 311–323.
88. T.C. Bartee and D.I. Schneider, "Computation with Finite Fields," Information and Control, v. 6, n. 2, Jun 1963, pp. 79–98.
89. U. Baum and S. Blackburn, "Clock–Controlled Pseudorandom Generators on Finite Groups," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
90. K.R. Bauer, T.A. Bersen, and R.J. Feiertag, "A Key Distribution Protocol Using Event Markers," ACM Transactions on Computer Systems, v. 1, n. 3, 1983, pp. 249–255.
91. F. Bauspiess and F. Damm, "Requirements for Cryptographic Hash Functions," Computers & Security, v. 11, n. 5, Sep 1992, pp. 427–437.
92. D. Bayer, S. Haber, and W.S. Stornetta, "Improving the Efficiency and Reliability of Digital Time–Stamping," Sequences '91: Methods in Communication, Security, and Computer Science, Springer–Verlag, 1992, pp. 329–334.
93. R. Bayer and J.K. Metzger, "On the Encipherment of Search Trees and Random Access Files," ACM Transactions on Database Systems, v. 1, n. 1, Mar 1976, pp. 37–52.
94. M. Beale and M.F. Monaghan, "Encrytion Using Random Boolean Functions," Cryptography and Coding, H.J. Beker and F.C. Piper, eds., Oxford: Clarendon Press, 1989, pp. 219–230.
95. P. Beauchemin and G. Brassard, "A Generalization of Hellman's Extension to Shannon's Approach to Cryptography," Journal of Cryptology, v. 1, n. 2, 1988, pp. 129–132.
96. P. Beauchemin, G. Brassard, C. Crépeau, C. Goutier, and C. Pomerance, "The Generation of Random Numbers that are Probably Prime," Journal of Cryptology, v. 1, n. 1, 1988, pp. 53–64.
97. D. Beaver, J. Feigenbaum, and V. Shoup, "Hiding Instances in Zero–Knowledge Proofs," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 326–338.
98. H. Beker, J. Friend, and P. Halliden, "Simplifying Key Management in Electronic Funds Transfer Points of Sale Systems," Electronics Letters, v. 19, n. 12, Jun 1983, pp. 442–444.
99. H. Beker and F. Piper, Cipher Systems: The Protection of Communications, London: Northwood Books, 1982.
100. D.E. Bell and L.J. LaPadula, "Secure Computer Systems: Mathematical Foundations," Report ESD–TR–73–275, MITRE Corp., 1973.
101. D.E. Bell and L.J. LaPadula, "Secure Computer Systems: A Mathematical Model," Report MTR–2547, MITRE Corp., 1973.
102. D.E. Bell and L.J. LaPadula, "Secure Computer Systems: A Refinement of the Mathematical Model," Report ESD–TR–73–278, MITRE Corp., 1974.
103. D.E. Bell and L.J. LaPadula, "Secure Computer Systems: Unified Exposition and Multics Interpretation," Report ESD–TR–75–306, MITRE Corp., 1976.
104. M. Bellare and S. Goldwasser, "New Paradigms for Digital Signatures and Message Authentication Based on Non–Interactive Zero Knowledge Proofs," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 194–211.
105. M. Bellare and S. Micali, "Non–Interactive Oblivious Transfer and Applications," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 547–557.
106. M. Bellare, S. Micali, and R. Ostrovsky, "Perfect Zero–Knowledge in Constant Rounds," Proceedings of the 22nd ACM Symposium on the Theory of Computing, 1990, pp. 482–493.
107. S.M. Bellovin, "A Preliminary Technical Analysis of Clipper and Skipjack," unpublished manuscript, 20 Apr 1993.
108. S.M. Bellovin and M. Merritt, "Limitations of the Kerberos Protocol," Winter 1991 USENIX Conference Proceedings, USENIX Association, 1991, pp. 253–267.
109. S.M. Bellovin and M. Merritt, "Encrypted Key Exchange: Password–Based Protocols Secure Against Dictionary Attacks," Proceedings of the 1992 IEEE Computer Society Conference on Research in Security and Privacy, 1992, pp. 72–84.
110. S.M. Bellovin and M. Merritt, "An Attack on the Interlock Protocol When Used for Authentication," IEEE Transactions on Information Theory, v. 40, n. 1, Jan 1994, pp. 273–275.
111. S.M. Bellovin and M. Merritt, "Cryptographic Protocol for Secure Communications," U.S. Patent #5,241,599, 31 Aug 93.
112. I. Ben–Aroya and E. Biham, "Differential Cryptanalysis of Lucifer," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 187–199.
113. J.C. Benaloh, "Cryptographic Capsules: A Disjunctive Primitive for Interactive Protocols," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, 213–222.
114. J.C. Benaloh, "Secret Sharing Homorphisms: Keeping Shares of a Secret Secret," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 251–260.
115. J.C. Benaloh, "Verifiable Secret–Ballot Elections," Ph.D. dissertation, Yale University, YALEU/DCS/TR–561, Dec 1987.
116. J.C. Benaloh and M. de Mare, "One–Way Accumulators: A Decentralized Alternative to Digital Signatures," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 274–285.
117. J.C. Benaloh and D. Tuinstra, "Receipt–Free Secret Ballot Elections," Proceedings of the 26th ACM Symposium on the Theory of Computing, 1994, pp. 544–553.
118. J.C. Benaloh and M. Yung, "Distributing the Power of a Government to Enhance the Privacy of Voters," Proceedings of the 5th ACM Symposium on the Principles in Distributed Computing, 1986, pp. 52–62.
119. A. Bender and G. Castagnoli, "On the Implementation of Elliptic Curve Cryptosystems," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 186–192.
120. S. Bengio, G. Brassard, Y.G. Desmedt, C. Goutier, and J.–J. Quisquater, "Secure Implementation of Identification Systems," Journal of Cryptology, v. 4, n. 3, 1991, pp. 175–184.
121. C.H. Bennett, F. Bessette, G. Brassard, L. Salvail, and J. Smolin, "Experimental Quantum Cryptography," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 253–265.
122. C.H. Bennett, F. Bessette, G. Brassard, L. Salvail, and J. Smolin, "Experimental Quantum Cryptography," Journal of Cryptology, v. 5, n. 1, 1992, pp. 3–28.
123. C.H. Bennett and G. Brassard, "Quantum Cryptography: Public Key Distribution and Coin Tossing," Proceedings of the IEEE International Conference on Computers, Systems, and Signal Processing, Banjalore, India, Dec 1984, pp. 175–179.
124. C.H. Bennett and G. Brassard, "An Update on Quantum Cryptography," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 475–480.
125. C.H. Bennett and G. Brassard, "Quantum Public–Key Distribution System," IBM Technical Disclosure Bulletin, v. 28, 1985, pp. 3153–3163.
126. C.H. Bennett and G. Brassard, "Quantum Public Key Distribution Reinvented," SIGACT News, v. 18, n. 4, 1987, pp. 51–53.
127. C.H. Bennett and G. Brassard, "The Dawn of a New Era for Quantum Cryptography: The Experimental Prototype is Working!" SIGACT News, v. 20, n. 4, Fall 1989, pp. 78–82.
128. C.H. Bennett, G. Brassard, and S. Breidbart, Quantum Cryptography II: How to Re–Use a One–Time Pad Safely Even if P=NP, unpublished manuscript, Nov 1982.
129. C.H. Bennett, G. Brassard, S. Breidbart, and S. Weisner, "Quantum Cryptography, or Unforgeable Subway Tokens," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 267–275.
130. C.H. Bennett, G. Brassard, C. Crépeau, and M.–H. Skubiszewska, "Practical Quantum Oblivious Transfer," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 351–366.
131. C.H. Bennett, G. Brassard, and A.K. Ekert, "Quantum Cryptography," Scientific American, v. 267, n. 4, Oct 1992, pp. 50–57.
132. C.H. Bennett, G. Brassard, and N.D. Mermin, "Quantum Cryptography Without Bell's Theorem," Physical Review Letters, v. 68, n. 5, 3 Feb 1992, pp. 557–559.
133. C.H. Bennett, G. Brassard, and J.–M. Robert, "How to Reduce Your Enemy's Information," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 468–476.
134. C.H. Bennett, G. Brassard, and J.–M. Robert, "Privacy Amplification by Public Discussion," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 210–229.
135. J. Bennett, "Analysis of the Encryption Algorithm Used in WordPerfect Word Processing Program," Cryptologia, v. 11, n. 4, Oct 1987, pp. 206–210.
136. M. Ben–Or, S. Goldwasser, and A. Wigderson, "Completeness Theorems for Non–Cryptographic Fault–Tolerant Distributed Computation," Proceedings of the 20th ACM Symposium on the Theory of Computing, 1988, pp. 1–10.
137. M. Ben–Or, O. Goldreich, S. Goldwasser, J. Håstad, J. Kilian, S. Micali, and P. Rogaway, "Everything Provable is Provable in Zero–Knowledge," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 37–56.
138. M. Ben–Or, O. Goldreich, S. Micali, and R.L. Rivest, "A Fair Protocol for Signing Contracts," IEEE Transactions on Information Theory, v. 36, n. 1, Jan 1990, pp. 40–46.
139. H.A. Bergen and W.J. Caelli, "File Security in WordPerfect 5.0," Cryptologia, v. 15, n. 1, Jan 1991, pp. 57–66.
140. E.R. Berlekamp, Algebraic Coding Theory, Aegean Park Press, 1984.
141. S. Berkovits, "How to Broadcast a Secret," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 535–541.
142. S. Berkovits, J. Kowalchuk, and B. Schanning, "Implementing Public–Key Scheme," IEEE Communications Magazine, v. 17, n. 3, May 1979, pp. 2–3.
143. D.J. Bernstein, Bernstein vs. U.S. Department of State et al., Civil Action No. C95–0582–MHP, United States District Court for the Northern District of California, 21 Feb 1995.
144. T. Berson, "Differential Cryptanalysis Mod 232 with Applications to MD5," Advances in Cryptology—EUROCRYPT '92 Proceedings, 1992, pp. 71–80.
145. T. Beth, Verfahren der schnellen Fourier–Transformation, Teubner, Stuttgart, 1984. (In German.)
146. T. Beth, "Efficient Zero–Knowledge Identification Scheme for Smart Cards," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 77–84.
147. T. Beth, B.M. Cook, and D. Gollmann, "Architectures for Exponentiation in GF(2n)," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 302–310.
148. T. Beth and Y. Desmedt, "Identification Tokens—or: Solving the Chess Grandmaster Problem," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 169–176.
149. T. Beth and C. Ding, "On Almost Nonlinear Permutations," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 65–76.
150. T. Beth, M. Frisch, and G.J. Simmons, eds., Lecture Notes in Computer Science 578; Public Key Cryptography: State of the Art and Future Directions, Springer–Verlag, 1992.
151. T. Beth and F.C. Piper, "The Stop–and–Go Generator," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1984, pp. 88–92.
152. T. Beth and F. Schaefer, "Non Supersingular Elliptic Curves for Public Key Cryptosystems," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 316–327.
153. A. Beutelspacher, "How to Say 'No'," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 491–496.
154. J. Bidzos, letter to NIST regarding DSS, 20 Sep 1991.
155. J. Bidzos, personal communication, 1993.
156. P. Bieber, "A Logic of Communication in a Hostile Environment," Proceedings of the Computer Security Foundations Workshop III, IEEE Computer Society Press, 1990, pp. 14–22.
157. E. Biham, "Cryptanalysis of the Chaotic–Map Cryptosystem Suggested at EUROCRYPT '91," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer– Verlag, 1991, pp. 532–534.
158. E. Biham, "New Types of Cryptanalytic Attacks Using Related Keys," Technical Report #753, Computer Science Department, Technion—Israel Institute of Technology, Sep 1992.
159. E. Biham, "On the Applicability of Differential Cryptanalysis to Hash Functions," lecture at EIES Workshop on Cryptographic Hash Functions, Mar 1992.
160. E. Biham, personal communication, 1993.
161. E. Biham, "Higher Order Differential Cryptanalysis," unpublished manuscript, Jan 1994.
162. E. Biham, "On Modes of Operation," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 116–120.
163. E. Biham, "New Types of Cryptanalytic Attacks Using Related Keys," Journal of Cryptology, v. 7, n. 4, 1994, pp. 229–246.
164. E. Biham, "On Matsui's Linear Cryptanalysis," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 398–412.
165. E. Biham and A. Biryukov, "How to Strengthen DES Using Existing Hardware," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
166. E. Biham and P.C. Kocher, "A Known Plaintext Attack on the PKZIP Encryption," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
167. E. Biham and A. Shamir, "Differential Cryptanalysis of DES–like Cryptosystems," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 2–21.
168. E. Biham and A. Shamir, "Differential Cryptanalysis of DES–like Cryptosystems," Journal of Cryptology, v. 4, n. 1, 1991, pp 3–72.
169. E. Biham and A. Shamir, "Differential Cryptanalysis of Feal and N–Hash," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 1–16.
170. E. Biham and A. Shamir, "Differential Cryptanalysis of Snefru, Khafre, REDOC–II, LOKI, and Lucifer," Advances in Cryptology—CRYPTO '91 Proceedings, 1992, pp. 156–171.
171. E. Biham and A. Shamir, "Differential Cryptanalysis of the Full 16–Round DES," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, 487– 496.
172. E. Biham and A. Shamir, Differential Cryptanalysis of the Data Encryption Standard, Springer–Verlag, 1993.
173. R. Bird, I. Gopal, A. Herzberg, P. Janson, S. Kutten, R. Molva, and M. Yung, "Systematic Design of Two–Party Authentication Protocols," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 44–61.
174. R. Bird, I. Gopal, A. Herzberg, P. Janson, S. Kutten, R. Molva, and M. Yung, "Systematic Design of a Family of Attack–Resistant Authentication Protocols," IEEE Journal of Selected Areas in Communication, to appear.
175. R. Bird, I. Gopal, A. Herzberg, P. Janson, S. Kutten, R. Molva, and M. Yung, "A Modular Family of Secure Protocols for Authentication and Key Distribution," IEEE/ACM Transactions on Networking, to appear.
176. M. Bishop, "An Application for a Fast Data Encryption Standard Implementation," Computing Systems, v. 1, n. 3, 1988, pp. 221–254.
177. M. Bishop, "Privacy–Enhanced Electronic Mail," Distributed Computing and Cryptography, J. Feigenbaum and M. Merritt, eds., American Mathematical Society, 1991, pp. 93–106.
178. M. Bishop, "Privacy–Enhanced Electronic Mail," Internetworking: Research and Experience, v. 2, n. 4, Dec 1991, pp. 199–233.
179. M. Bishop, "Recent Changes to Privacy Enhanced Electronic Mail," Internetworking: Research and Experience, v. 4, n. 1, Mar 1993, pp. 47–59.
180. I.F. Blake, R. Fuji–Hara, R.C. Mullin, and S.A. Vanstone, "Computing Logarithms in Finite Fields of Characteristic Two," SIAM Journal on Algebraic Discrete Methods, v. 5, 1984, pp. 276–285.
181. I.F. Blake, R.C. Mullin, and S.A. Vanstone, "Computing Logarithms in GF (2n)," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 73–82.
182. G.R. Blakley, "Safeguarding Cryptographic Keys," Proceedings of the National Computer Conference, 1979, American Federation of Information Processing Societies, v. 48, 1979, pp. 313–317.
183. G.R. Blakley, "One–Time Pads are Key Safeguarding Schemes, Not Cryptosystems—Fast Key Safeguarding Schemes (Threshold Schemes) Exist," Proceedings of the 1980 Symposium on Security and Privacy, IEEE Computer Society, Apr 1980, pp. 108–113.
184. G.R. Blakley and I. Borosh, "Rivest–Shamir–Adleman Public Key Cryptosystems Do Not Always Conceal Messages," Computers and Mathematics with Applications, v. 5, n. 3, 1979, pp. 169–178.
185. G.R. Blakley and C. Meadows, "A Database Encryption Scheme which Allows the Computation of Statistics Using Encrypted Data," Proceedings of the 1985 Symposium on Security and Privacy, IEEE Computer Society, Apr 1985, pp. 116–122.
186. M. Blaze, "A Cryptographic File System for UNIX," 1st ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 9–16.
187. M. Blaze, "Protocol Failure in the Escrowed Encryption Standard," 2nd ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 59–67.
188. M. Blaze, "Key Management in an Encrypting File System," Proceedings of the Summer 94 USENIX Conference, USENIX Association, 1994, pp. 27–35.
189. M. Blaze and B. Schneier, "The MacGuffin Block Cipher Algorithm," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
190. U. Blöcher and M. Dichtl, "Fish: A Fast Software Stream Cipher," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 41–44.
191. R. Blom, "Non–Public Key Distribution," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 231–236.
192. K.J. Blow and S.J.D. Phoenix, "On a Fundamental Theorem of Quantum Cryptography," Journal of Modern Optics, v. 40, n. 1, Jan 1993, pp. 33–36.
193. L. Blum, M. Blum, and M. Shub, "A Simple Unpredictable Pseudo–Random Number Generator," SIAM Journal on Computing, v. 15, n. 2, 1986, pp. 364–383.
194. M. Blum, "Coin Flipping by Telephone: A Protocol for Solving Impossible Problems," Proceedings of the 24th IEEE Computer Conference (CompCon), 1982, pp. 133–137.
195. M. Blum, "How to Exchange (Secret) Keys," ACM Transactions on Computer Systems, v. 1, n. 2, May 1983, pp. 175–193.
196. M. Blum, "How to Prove a Theorem So No One Else Can Claim It," Proceedings of the International Congress of Mathematicians, Berkeley, CA, 1986, pp. 1444–1451.
197. M. Blum, A. De Santis, S. Micali, and G. Persiano, "Noninteractive Zero–Knowledge," SIAM Journal on Computing, v. 20, n. 6, Dec 1991, pp. 1084–1118.
198. M. Blum, P. Feldman, and S. Micali, "Non–Interactive Zero–Knowledge and Its Applications," Proceedings of the 20th ACM Symposium on Theory of Computing, 1988, pp. 103–112.
199. M. Blum and S. Goldwasser, "An Efficient Probabilistic Public–Key Encryption Scheme Which Hides All Partial Information," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 289–299.
200. M. Blum and S. Micali, "How to Generate Cryptographically–Strong Sequences of Pseudo–Random Bits," SIAM Journal on Computing, v. 13, n. 4, Nov 1984, pp. 850–864.
201. B. den Boer, "Cryptanalysis of F.E.A.L.," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 293–300.
202. B. den Boer and A. Bosselaers, "An Attack on the Last Two Rounds of MD4," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 194–203.
203. B. den Boer and A. Bosselaers, "Collisions for the Compression Function of MD5," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 293–304.
204. J.–P. Boly, A. Bosselaers, R. Cramer, R. Michelsen, S. Mjølsnes, F. Muller, T. Pedersen, B. Pfitzmann, P. de Rooij, B. Schoenmakers, M. Schunter, L. Vallée, and M. Waidner, "Digital Payment Systems in the ESPRIT Project CAFE," Securicom 94, Paris, France, 2–6 Jan 1994, pp. 35–45.
205. J.–P. Boly, A. Bosselaers, R. Cramer, R. Michelsen, S. Mjølsnes, F. Muller, T. Pedersen, B. Pfitzmann, P. de Rooij, B. Schoenmakers, M. Schunter, L. Vallée, and M. Waidner, "The ESPRIT Project CAFE—High Security Digital Payment System," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 217–230.
206. D.J. Bond, "Practical Primality Testing," Proceedings of IEE International Conference on Secure Communications Systems, 22–23 Feb 1984, pp. 50–53.
207. H. Bonnenberg, Secure Testing of VSLI Cryptographic Equipment, Series in Microelectronics, Vol. 25, Konstanz: Hartung Gorre Verlag, 1993.
208. H. Bonnenberg, A. Curiger, N. Felber, H. Kaeslin, and X. Lai, "VLSI Implementation of a New Block Cipher," Proceedings of the IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD 91), Oct 1991, pp. 510–513.
209. K.S. Booth, "Authentication of Signatures Using Public Key Encryption," Communications of the ACM, v. 24, n. 11, Nov 1981, pp. 772–774.
210. A. Bosselaers, R. Govaerts, and J. Vanderwalle, Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 175–186.
211. D.P. Bovet and P. Crescenzi, Introduction to the Theory of Complexity, Englewood Cliffs, N.J.: Prentice–Hall, 1994.
212. J. Boyar, "Inferring Sequences Produced by a Linear Congruential Generator Missing Low–Order Bits," Journal of Cryptology, v. 1, n. 3, 1989, pp. 177–184.
213. J. Boyar, D. Chaum, and I. Damgård, "Convertible Undeniable Signatures," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 189–205.
214. J. Boyar, K. Friedl, and C. Lund, "Practical Zero–Knowledge Proofs: Giving Hints and Using Deficiencies," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 155–172.
215. J. Boyar, C. Lund, and R. Peralta, "On the Communication Complexity of Zero–Knowledge Proofs," Journal of Cryptology, v. 6, n. 2, 1993, pp. 65–85.
216. J. Boyar and R. Peralta, "On the Concrete Complexity of Zero–Knowledge Proofs," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 507–525.
217. C. Boyd, "Some Applications of Multiple Key Ciphers," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 455–467.
218. C. Boyd, "Digital Multisignatures," Cryptography and Coding, H.J. Beker and F.C. Piper, eds., Oxford: Clarendon Press, 1989, pp. 241–246.
219. C. Boyd, "A New Multiple Key Cipher and an Improved Voting Scheme," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 617–625.
220. C. Boyd, "Multisignatures Revisited," Cryptography and Coding III, M.J. Ganley, ed., Oxford: Clarendon Press, 1993, pp. 21–30.
221. C. Boyd and W. Mao, "On the Limitation of BAN Logic," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 240–247.
222. C. Boyd and W. Mao, "Designing Secure Key Exchange Protocols," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 217–230.
223. B.O. Brachtl, D. Coppersmith, M.M. Hyden, S.M. Matyas, C.H. Meyer, J. Oseas, S. Pilpel, and M. Schilling, "Data Authentication Using Modification Detection Codes Based on a Public One Way Function," U.S. Patent #4,908,861, 13 Mar 1990.
224. J. Brandt, I.B. Damgård, P. Landrock, and T. Pederson, "Zero–Knowledge Authentication Scheme with Secret Key Exchange," Advances in Cryptology—CRYPTO '88, Springer–Verlag, 1990, pp. 583–588.
225. S.A. Brands, "An Efficient Off–Line Electronic Cash System Based on the Representation Problem," Report CS–R9323, Computer Science/Department of Algorithms and Architecture, CWI, Mar 1993.
226. S.A. Brands, "Untraceable Off–line Cash in Wallet with Observers," Advances in Cryptology—CRYPTO '93, Springer–Verlag, 1994, pp. 302–318.
227. S.A. Brands, "Electronic Cash on the Internet," Proceedings of the Internet Society 1995 Symposium on Network and Distributed Systems Security, IEEE Computer Society Press 1995, pp 64–84.
228. D.K. Branstad, "Hellman's Data Does Not Support His Conclusion," IEEE Spectrum, v. 16, n. 7, Jul 1979, p. 39.
229. D.K. Branstad, J. Gait, and S. Katzke, "Report on the Workshop on Cryptography in Support of Computer Security," NBSIR 77–1291, National Bureau of Standards, Sep 21–22, 1976, September 1977.
230. G. Brassard, "A Note on the Complexity of Cryptography," IEEE Transactions on Information Theory, v. IT–25, n. 2, Mar 1979, pp. 232–233.
231. G. Brassard, "Relativized Cryptography," Proceedings of the IEEE 20th Annual Symposium on the Foundations of Computer Science, 1979, pp. 383–391.
232. G. Brassard, "A Time–Luck Tradeoff in Relativized Cryptography," Proceedings of the IEEE 21st Annual Symposium on the Foundations of Computer Science, 1980, pp. 380–386.
233. G. Brassard, "A Time–Luck Tradeoff in Relativized Cryptography," Journal of Computer and System Sciences, v. 22, n. 3, Jun 1981, pp. 280–311.
234. G. Brassard, "An Optimally Secure Relativized Cryptosystem," SIGACT News, v. 15, n. 1, 1983, pp. 28–33.
235. G. Brassard, "Relativized Cryptography," IEEE Transactions on Information Theory, v. IT–29, n. 6, Nov 1983, pp. 877–894.
236. G. Brassard, Modern Cryptology: A Tutorial, Springer–Verlag, 1988.
237. G. Brassard, "Quantum Cryptography: A Bibliography," SIGACT News, v. 24, n. 3, Oct 1993, pp. 16–20.
238. G. Brassard, D. Chaum, and C. Crépeau, "An Introduction to Minimum Disclosure," CWI Quarterly, v. 1, 1988, pp. 3–17.
239. G. Brassard, D. Chaum, and C. Crépeau, "Minimum Disclosure Proofs of Knowledge," Journal of Computer and System Sciences, v. 37, n. 2, Oct 1988, pp. 156–189.
240. G. Brassard and C. Crépeau, "Non–Transitive Transfer of Confidence: A Perfect Zero–Knowledge Interactive Protocol for SAT and Beyond," Proceedings of the 27th IEEE Symposium on Foundations of Computer Science, 1986, pp. 188–195.
241. G. Brassard and C. Crépeau, "Zero–Knowledge Simulation of Boolean Circuits," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 223–233.
242. G. Brassard and C. Crépeau, "Sorting Out Zero–Knowledge," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 181–191.
243. G. Brassard and C. Crépeau, "Quantum Bit Commitment and Coin Tossing Protocols," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 49–61.
244. G. Brassard, C. Crépeau, R. Jozsa, and D. Langlois, "A Quantum Bit Commitment Scheme Provably Unbreakable by Both Parties," Proceedings of the 34th IEEE Symposium on Foundations of Computer Science, 1993, pp. 362–371.
245. G. Brassard, C. Crépeau, and J.–M. Robert, "Information Theoretic Reductions Among Disclosure Problems," Proceedings of the 27th IEEE Symposium on Foundations of Computer Science, 1986, pp. 168–173.
246. G. Brassard, C. Crépeau, and J.–M. Robert, "All–or–Nothing Disclosure of Secrets," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 234–238.
247. G. Brassard, C. Crépeau, and M. Yung, "Everything in NP Can Be Argued in Perfect Zero–Knowledge in a Bounded Number of Rounds," Proceedings on the 16th International Colloquium on Automata, Languages, and Programming, Springer–Verlag, 1989, pp. 123–136.
248. R.P. Brent, "An Improved Monte–Carlo Factorization Algorithm," BIT, v. 20, n. 2, 1980, pp. 176–184.
249. R.P. Brent, "On the Periods of Generalized Fibonacci Recurrences, Mathematics of Computation, v. 63, n. 207, Jul 1994, pp. 389–401.
250. R.P. Brent, "Parallel Algorithms for Integer Factorization," Research Report CMA–R49–89, Computer Science Laboratory, The Australian National University, Oct 1989.
251. D.M. Bressoud, Factorization and Primality Testing, Springer–Verlag, 1989.
252. E.F. Brickell, "A Fast Modular Multiplication Algorithm with Applications to Two Key Cryptography," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1982, pp. 51–60.
253. E.F. Brickell, "Are Most Low Density Polynomial Knapsacks Solvable in Polynomial Time?" Proceedings of the 14th Southeastern Conference on Combinatorics, Graph Theory, and Computing, 1983.
254. E.F. Brickell, "Solving Low Density Knapsacks," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 25–37.
255. E.F. Brickell, "Breaking Iterated Knapsacks," Advances in Cryptology: Proceedings of Crypto 84, Springer–Verlag, 1985, pp. 342–358.
256. E.F. Brickell, "Cryptanalysis of the Uagisawa Public Key Cryptosystem," Abstracts of Papers, EUROCRYPT '86, 20–22 May 1986.
257. E.F. Brickell, "The Cryptanalysis of Knapsack Cryptosystems," Applications of Discrete Mathematics, R.D. Ringeisen and F.S. Roberts, eds., Society for Industrial and Applied Mathematics, Philadelphia, 1988, pp. 3–23.
258. E.F. Brickell, "Survey of Hardware Implementations of RSA," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 368–370.
259. E.F. Brickell, D. Chaum, I.B. Damgård, and J. van de Graff, "Gradual and Verifiable Release of a Secret," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 156–166.
260. E.F. Brickell, J.A. Davis, and G.J. Simmons, "A Preliminary Report on the Cryptanalysis of Merkle–Hellman Knapsack," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 289–303.
261. E.F. Brickell and J. DeLaurentis, "An Attack on a Signature Scheme Proposed by Okamoto and Shiraishi," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 28–32.
262. E.F. Brickell, D.E. Denning, S.T. Kent, D.P. Maher, and W. Tuchman, "SKIPJACK Review—Interim Report," unpublished manuscript, 28 Jul 1993.
263. E.F. Brickell, J.C. Lagarias, and A.M. Odlyzko, "Evaluation of the Adleman Attack of Multiple Iterated Knapsack Cryptosystems," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 39–42.
264. E.F. Brickell, P.J. Lee, and Y. Yacobi, "Secure Audio Teleconference," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 418–426.
265. E.F. Brickell and K.S. McCurley, "An Interactive Identification Scheme Based on Discrete Logarithms and Factoring," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 63–71.
266. E.F. Brickell, J.H. Moore, and M.R. Purtill, "Structure in the S–Boxes of the DES," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 3–8.
267. E.F. Brickell and A.M. Odlyzko, "Cryptanalysis: A Survey of Recent Results," Proceedings of the IEEE, v. 76, n. 5, May 1988, pp. 578–593.
268. E.F. Brickell and A.M. Odlyzko, "Cryptanalysis: A Survey of Recent Results," Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1991, pp. 501–540.
269. E.F. Brickell and G.J. Simmons, "A Status Report on Knapsack Based Public Key Cryptosystems," Congressus Numerantium, v. 7, 1983, pp. 3–72.
270. E.F. Brickell and D.R. Stinson, "The Detection of Cheaters in Threshold Schemes," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 564–577.
271. A.G. Broscius and J.M. Smith, "Exploiting Parallelism in Hardware Implementation of the DES," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 367–376.
272. L. Brown, M. Kwan, J. Pieprzyk, and J. Seberry, "Improving Resistance to Differential Cryptanalysis and the Redesign of LOKI," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 36–50.
273. L. Brown, J. Pieprzyk, and J. Seberry, "LOKI: A Cryptographic Primitive for Authentication and Secrecy Applications," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 229–236.
274. L. Brown, J. Pieprzyk, and J. Seberry, "Key Scheduling in DES Type Cryptosystems," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 221–228.
275. L. Brown and J. Seberry, "On the Design of Permutation P in DES Type Cryptosystems," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 696–705. 276. W. Brown, "A Quantum Leap in Secret Communications," New Scientist, n. 1585, 30 Jan 1993, p. 21.
277. J.O. Brüer, "On Pseudo Random Sequences as Crypto Generators," Proceedings of the International Zurich Seminar on Digital Communication, Switzerland, 1984.
278. L. Brynielsson "On the Linear Complexity of Combined Shift Register Sequences," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 156–166.
279. J. Buchmann, J. Loho, and J. Zayer, "An Implementation of the General Number Field Sieve," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 159–165.
280. M. Burmester and Y. Desmedt, "Broadcast Interactive Proofs," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 81–95.
281. M. Burmester and Y. Desmedt, "A Secure and Efficient Conference Key Distribution System," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
282. D. Burnham, "NSA Seeking 500,000 'Secure' Telephones," The New York Times, 6 Oct 1994.
283. M. Burrows, M. Abadi, and R. Needham, "A Logic of Authentication," Research Report 39, Digital Equipment Corp. Systems Research Center, Feb 1989.
284. M. Burrows, M. Abadi, and R. Needham, "A Logic of Authentication," ACM Transactions on Computer Systems, v. 8, n. 1, Feb 1990, pp. 18–36.
285. M. Burrows, M. Abadi, and R. Needham, "Rejoinder to Nessett," Operating System Review, v. 20, n. 2, Apr 1990, pp. 39–40.
286. J.J. Cade, "A Modification of a Broken Public–Key Cipher," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 64–83.
287. T.R. Cain and A.T. Sherman, "How to Break Gifford's Cipher," Proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 198–209.
288. C. Calvelli and V. Varadharajan, "An Analysis of Some Delegation Protocols for Distributed Systems," Proceedings of the Computer Security Foundations Workshop V, IEEE Computer Society Press, 1992, pp. 92–110.
289. J.L. Camenisch, J.–M. Piveteau, and M.A. Stadler, "An Efficient Electronic Payment System Protecting Privacy," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 207–215.
290. P. Camion and J. Patarin, "The Knapsack Hash Function Proposed at Crypto '89 Can Be Broken," Advances in Cryptology—EUROCRYPT '91, Springer–Verlag, 1991, pp. 39–53.
291. C.M. Campbell, "Design and Specification of Cryptographic Capabilities," IEEE Computer Society Magazine, v. 16, n. 6, Nov 1978, pp. 15–19.
292. E.A. Campbell, R. Safavi–Naini, and P.A. Pleasants, "Partial Belief and Probabilistic Reasoning in the Analysis of Secure Protocols," Proceedings of the Computer Security Foundations Workshop V, IEEE Computer Society Press, 1992, pp. 92–110.
293. K.W. Campbell and M.J. Wiener, "DES Is Not a Group," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, pp. 512–520.
294. Z.F. Cao and G. Zhao, "Some New MC Knapsack Cryptosystems," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 70–75. (In Chinese).
295. C. Carlet, "Partially–Bent Functions," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 280–291.
296. C. Carlet, "Partially Bent Functions," Designs, Codes and Cryptography, v. 3, 1993, pp. 135–145.
297. C. Carlet, "Two New Classes of Bent Functions" Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 77–101.
298. C. Carlet, J. Seberry, and X.M. Zhang, "Comments on 'Generating and Counting Binary Bent Sequences,'" IEEE Transactions on Information Theory, v. IT–40, n. 2, Mar 1994, p. 600.
299. J.M. Carroll, Computer Security, 2nd edition, Butterworths, 1987.
300. J.M. Carroll, "The Three Faces of Information Security," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 433–450.
301. J.M. Carroll, "'Do–it–yourself' Cryptography," Computers & Security, v. 9, n. 7, Nov 1990, pp. 613–619.
302. T.R. Caron and R.D. Silverman, "Parallel Implementation of the Quadratic Scheme," Journal of Supercomputing, v. 1, n. 3, 1988, pp. 273–290.
303. CCITT, Draft Recommendation X.509, "The Directory—Authentication Framework," Consultation Committee, International Telephone and Telegraph, International Telecommunications Union, Geneva, 1987.
304. CCITT, Recommendation X.509, "The Directory—Authentication Framework," Consultation Committee, International Telephone and Telegraph, International Telecommunications Union, Geneva, 1989.
305. CCITT, Recommendation X.800, "Security Architecture for Open Systems Interconnection for CCITT Applications," International Telephone and Telegraph, International Telecommunications Union, Geneva, 1991.
306. F. Chabaud, "On the Security of Some Cryptosystems Based on Error–Correcting Codes," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
307. F. Chabaud and S. Vaudenay, "Links Between Differential and Linear Cryptanalysis," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
308. W.G. Chambers and D. Gollmann, "Generators for Sequences with Near–Maximal Linear Equivalence," IEE Proceedings, V. 135, Pt. E, n. 1, Jan 1988, pp. 67–69.
309. W.G. Chambers and D. Gollmann, "Lock–In Effect in Cascades of Clock–Controlled Shirt Registers," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 331–343.
310. A. Chan and R. Games, "On the Linear Span of Binary Sequences from Finite Geometries," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 405–417.
311. J.P. Chandler, D.C. Arrington, D.R. Berkelhammer, and W.L. Gill, "Identification and Analysis of Foreign Laws and Regulations Pertaining to the Use of Commercial Encryption Products for Voice and Data Communications," National Intellectual Property Law Institute, George Washington University, Washington, D.C., Jan 1994.
312. C.C. Chang and S.J. Hwang, "Cryptographic Authentication of Passwords," Proceedings of the 25th Annual 1991 IEEE International Carnahan Conference on Security Technology, Taipei, Taiwan, 1–3 Oct 1991, pp. 126–130.
313. C.C. Chang and S.J. Hwang, "A Strategy for Transforming Public–Key Cryptosystems into Identity–Based Cryptosystems," Proceedings of the 25th Annual 1991 IEEE International Carnahan Conference on Security Technology, Taipei, Taiwan, 1–3 Oct 1991, pp. 68–72.
314. C.C. Chang and C.H. Lin, "An ID–Based Signature Scheme Based upon Rabin's Public Key Cryptosystem," Proceedings of the 25th Annual 1991 IEEE International Carnahan Conference on Security Technology, Taipei, Taiwan, 1–3 Oct 1991, pp. 139–141.
315. C. Charnes and J. Pieprzyk, "Attacking the SL2 Hashing Scheme," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 322–330.
316. D. Chaum, "Untraceable Electronic Mail, Return Addresses, and Digital Pseudonyms," Communications of the ACM, v. 24, n. 2, Feb 1981, pp. 84–88.
317. D. Chaum, "Blind Signatures for Untraceable Payments," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 199–203.
318. D. Chaum, "Security Without Identification: Transaction Systems to Make Big Brother Obsolete," Communications of the ACM, v. 28, n. 10, Oct 1985, pp. 1030–1044. 319. D. Chaum, "Demonstrating that a Public Predicate Can Be Satisfied without Revealing Any Information about How," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 159–199.
320. D. Chaum, "Blinding for Unanticipated Signatures," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 227–233.
321. D. Chaum, "The Dining Cryptographers Problem: Unconditional Sender and Receiver Untraceability," Journal of Cryptology, v. 1, n. 1, 1988, pp. 65–75.
322. D. Chaum, "Elections with Unconditionally Secret Ballots and Disruptions Equivalent to Breaking RSA," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 177–181.
323. D. Chaum, "Blind Signature Systems," U.S. Patent #4,759,063, 19 Jul 1988.
324. D. Chaum, "Blind Unanticipated Signature Systems," U.S. Patent #4,759,064, 19 Jul 1988.
325. D. Chaum, "Online Cash Checks," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 288–293.
326. D. Chaum, "One–Show Blind Signature Systems," U.S. Patent #4,914,698, 3 Apr 1990.
327. D. Chaum, "Undeniable Signature Systems," U.S. Patent #4,947,430, 7 Aug 1990.
328. D. Chaum, "Returned–Value Blind Signature Systems," U.S. Patent #4,949,380, 14 Aug 1990.
329. D. Chaum, "Zero–Knowledge Undeniable Signatures," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 458–464.
330. D. Chaum, "Group Signatures," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 257–265. 331. D. Chaum, "Unpredictable Blind Signature Systems," U.S. Patent #4,991,210, 5 Feb 1991.
332. D. Chaum, "Achieving Electronic Privacy," Scientific American, v. 267, n. 2, Aug 1992, pp. 96–101.
333. D. Chaum, "Designated Confirmer Signatures," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
334. D. Chaum, C. Crépeau, and I.B. Damgård, "Multiparty Unconditionally Secure Protocols," Proceedings of the 20th ACM Symposium on the Theory of Computing, 1988, pp. 11–19.
335. D. Chaum, B. den Boer, E. van Heyst, S. Mjølsnes, and A. Steenbeek, "Efficient Offline Electronic Checks," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 294–301.
336. D. Chaum and J.–H. Evertse, "Cryptanalysis of DES with a Reduced Number of Rounds; Sequences of Linear Factors in Block Ciphers," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 192–211.
337. D. Chaum, J.–H. Evertse, and J. van de Graff, "An Improved Protocol for Demonstrating Possession of Discrete Logarithms and Some Generalizations," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 127–141.
338. D. Chaum, J.–H. Evertse, J. van de Graff, and R. Peralta, "Demonstrating Possession of a Discrete Logarithm without Revealing It," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 200–212.
339. D. Chaum, A. Fiat, and M. Naor, "Untraceable Electronic Cash," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 319–327.
340. D. Chaum and T. Pedersen, "Transferred Cash Grows in Size," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 391–407.
341. D. Chaum and T. Pedersen, "Wallet Databases with Observers," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 89–105.
342. D. Chaum and I. Schaumuller–Bichel, eds., Smart Card 2000, North Holland: Elsevier Science Publishers, 1989.
343. D. Chaum and H. van Antwerpen, "Undeniable Signatures," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 212–216.
344. D. Chaum, E. van Heijst, and B. Pfitzmann, "Cryptographically Strong Undeniable Signatures, Unconditionally Secure for the Signer," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 470–484.
345. T.M. Chee, "The Cryptanalysis of a New Public–Key Cryptosystem Based on Modular Knapsacks," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 204–212.
346. L. Chen, "Oblivious Signatures," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 161–172.
347. L. Chen and M. Burminster, "A Practical Secret Voting Scheme which Allows Voters to Abstain," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 100–107.
348. L. Chen and T.P. Pedersen "New Group Signature Schemes," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
349. J. Chenhui, "Spectral Characteristics of Partially–Bent Functions," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 48–51.
350. V. Chepyzhov and B. Smeets, "On a Fast Correlation Attack on Certain Stream Ciphers," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 176–185.
351. T.C. Cheung, "Management of PEM Public Key Certificates Using X.500 Directory Service: Some Problems and Solutions," Proceedings of the Internet Society 1994 Workshop on Network and Distributed System Security, The Internet Society, 1994, pp. 35–42.
352. G.C. Chiou and W.C. Chen, "Secure Broadcasting Using the Secure Lock," IEEE Transactions on Software Engineering, v. SE–15, n. 8, Aug 1989, pp. 929–934.
353. Y.J. Choie and H.S. Hwoang, "On the Cryptosystem Using Elliptic Curves," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 105–113.
354. B. Chor and O. Goldreich, "RSA/Rabin Least Significant Bits are 1/2+1/poly(log N) Secure," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 303–313.
355. B. Chor, S. Goldwasser, S. Micali, and B. Awerbuch, "Verifiable Secret Sharing and Achieving Simultaneity in the Presence of Faults," Proceedings of the 26th Annual IEEE Symposium on the Foundations of Computer Science, 1985, pp. 383–395.
356. B. Chor and R.L. Rivest, "A Knapsack Type Public Key Cryptosystem Based on Arithmetic in Finite Fields," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 54–65.
357. P. Christoffersson, S.–A. Ekahll, V. Fåk, S. Herda, P. Mattila, W. Price, and H.–O. Widman, Crypto Users' Handbook: A Guide for Implementors of Cryptographic Protection in Computer Systems, North Holland: Elsevier Science Publishers, 1988.
358. R. Cleve, "Controlled Gradual Disclosure Schemes for Random Bits and Their Applications," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 572–588.
359. J.D. Cohen, "Improving Privacy in Cryptographic Elections," Yale University Computer Science Department Technical Report YALEU/DCS/TR–454, Feb 1986.
360. J.D. Cohen and M.H. Fischer, "A Robust and Verifiable Cryptographically Secure Election Scheme," Proceedings of the 26th Annual IEEE Symposium on the Foundations of Computer Science, 1985, pp. 372–382.
361. R. Cole, "A Model for Security in Distributed Systems," Computers and Security, v. 9, n. 4, Apr 1990, pp. 319–330.
362. Comptroller General of the United States, "Matter of National Institute of Standards and Technology—Use of Electronic Data Interchange Technology to Create Valid Obligations," File B–245714, 13 Dec 1991.
363. M.S. Conn, letter to Joe Abernathy, National Security Agency, Ser: Q43–111–92, 10 Jun 1992.
364. C. Connell, "An Analysis of NewDES: A Modified Version of DES," Cryptologia, v. 14, n. 3, Jul 1990, pp. 217–223.
365. S.A. Cook, "The Complexity of Theorem–Proving Procedures," Proceedings of the 3rd Annual ACM Symposium on the Theory of Computing, 1971, pp. 151–158.
366. R.H. Cooper and W. Patterson, "A Generalization of the Knapsack Method Using Galois Fields," Cryptologia, v. 8, n. 4, Oct 1984, pp. 343–347.
367. R.H. Cooper and W. Patterson, "RSA as a Benchmark for Multiprocessor Machines," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 356–359.
368. D. Coppersmith, "Fast Evaluation of Logarithms in Fields of Characteristic Two," IEEE Transactions on Information Theory, v. 30, n. 4, Jul 1984, pp. 587–594.
369. D. Coppersmith, "Another Birthday Attack," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 14–17.
370. D. Coppersmith, "Cheating at Mental Poker," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 104–107.
371. D. Coppersmith, "The Real Reason for Rivest's Phenomenon," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 535–536.
372. D. Coppersmith, "Two Broken Hash Functions," Research Report RD 18397, IBM T.J. Watson Center, Oct 1992.
373. D. Coppersmith, "The Data Encryption Standard (DES) and Its Strength against Attacks," Technical Report RC 18613, IBM T.J. Watson Center, Dec 1992.
374. D. Coppersmith, "The Data Encryption Standard (DES) and its Strength against Attacks," IBM Journal of Research and Development, v. 38, n. 3, May 1994, pp. 243–250.
375. D. Coppersmith, "Attack on the Cryptographic Scheme NIKS–TAS," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 294–307.
376. D. Coppersmith, personal communication, 1994.
377. D. Coppersmith and E. Grossman, "Generators for Certain Alternating Groups with Applications to Cryptography," SIAM Journal on Applied Mathematics, v. 29, n. 4, Dec 1975, pp. 624–627.
378. D. Coppersmith, H. Krawczyk, and Y. Mansour, "The Shrinking Generator," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 22–39.
379. D. Coppersmith, A. Odlykzo, and R. Schroeppel, "Discrete Logarithms in GF(p)," Algorithmica, v. 1, n. 1, 1986, pp. 1–16.
380. D. Coppersmith and P. Rogaway, "Software Efficient Pseudo Random Function and the Use Thereof for Encryption," U.S. Patent pending, 1995.
381. D. Coppersmith, J. Stern, and S. Vaudenay, "Attacks on the Birational Signature Schemes," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 435–443.
382. V. Cordonnier and J.–J. Quisquater, eds., CARDIS '94—Proceedings of the First Smart Card Research and Advanced Application Conference, Lille, France, 24–26 Oct 1994.
383. C. Couvreur and J.–J. Quisquater, "An Introduction to Fast Generation of Large Prime Numbers," Philips Journal Research, v. 37, n. 5–6, 1982, pp. 231–264.
384. C. Couvreur and J.–J. Quisquater, "An Introduction to Fast Generation of Large Prime Numbers," Philips Journal Research, v. 38, 1983, p. 77.
385. C. Coveyou and R.D. MacPherson, "Fourier Analysis of Uniform Random Number Generators," Journal of the ACM, v. 14, n. 1, 1967, pp. 100–119.
386. T.M. Cover and R.C. King, "A Convergent Gambling Estimate of the Entropy of English," IEEE Transactions on Information Theory, v. IT–24, n. 4, Jul 1978, pp. 413–421.
387. R.J.F. Cramer and T.P. Pedersen, "Improved Privacy in Wallets with Observers," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 329–343.
388. R.E. Crandell, "Method and Apparatus for Public Key Exchange in a Cryptographic System," U.S. Patent #5,159,632, 27 Oct 1992.
389. C. Crépeau, "A Secure Poker Protocol That Minimizes the Effect of Player Coalitions," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 73–86.
390. C. Crépeau, "A Zero–Knowledge Poker Protocol that Achieves Confidentiality of the Players' Strategy, or How to Achieve an Electronic Poker Face," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 239–247.
391. C. Crépeau, "Equivalence Between Two Flavours of Oblivious Transfer," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 350–354.
392. C. Crépeau, "Correct and Private Reductions among Oblivious Transfers," Ph.D. dissertation, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, 1990.
393. C. Crépeau, "Quantum Oblivious Transfer," Journal of Modern Optics, v. 41, n. 12, Dec 1994, pp. 2445–2454.
394. C. Crépeau and J. Kilian, "Achieving Oblivious Transfer Using Weakened Security Assumptions," Proceedings of the 29th Annual Symposium on the Foundations of Computer Science, 1988, pp. 42–52.
395. C. Crépeau and J. Kilian, "Weakening Security Assumptions and Oblivious Transfer," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 2–7.
396. C. Crépeau and L. Salvail, "Quantum Oblivious Mutual Identification," Advances in Cryptology—EUROCRYPT '95 Proceedings, Springer–Verlag, 1995, pp. 133–146.
397. A. Curiger, H. Bonnenberg, R. Zimmermann, N. Felber, H. Kaeslin and W. Fichtner, "VINCI: VLSI Implementation of the New Block Cipher IDEA," Proceedings of IEEE CICC '93, San Diego, CA, May 1993, pp. 15.5.1–15.5.4.
398. A. Curiger and B. Stuber, "Specification for the IDEA Chip," Technical Report No. 92/03, Institut für Integrierte Systeme, ETH Zurich, Feb 1992.
399. T. Cusick, "Boolean Functions Satisfying a Higher Order Strict Avalanche Criterion," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 102–117.
400. T.W. Cusick and M.C. Wood, "The REDOC–II Cryptosystem," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 545–563.
401. Cylink Corporation, Cylink Corporation vs. RSA Data Security, Inc., Civil Action No. C94–02332–CW, United States District Court for the Northern District of California, 30 Jun 1994.
402. J. Daeman, "Cipher and Hash Function Design," Ph.D. Thesis, Katholieke Universiteit Leuven, Mar 95.
403. J. Daeman, A. Bosselaers, R. Govaerts, and J. Vandewalle, "Collisions for Schnorr's Hash Function FFT–Hash Presented at Crypto '91," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 477–480.
404. J. Daeman, R. Govaerts, and J. Vandewalle, "A Framework for the Design of One–Way Hash Functions Including Cryptanalysis of Damgård's One–Way Function Based on Cellular Automata," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 82–96.
405. J. Daeman, R. Govaerts, and J. Vandewalle, "A Hardware Design Model for Cryptographic Algorithms," ESORICS 92, Proceedings of the Second European Symposium on Research in Computer Security, Springer–Verlag, 1992, pp. 419–434.
406. J. Daemen, R. Govaerts, and J. Vandewalle, "Block Ciphers Based on Modular Arithmetic," Proceedings of the 3rd Symposium on State and Progress of Research in Cryptography, Rome, Italy, 15–16 Feb 1993, pp. 80–89.
407. J. Daemen, R. Govaerts, and J. Vandewalle, "Fast Hashing Both in Hardware and Software," presented at the rump session of CRYPTO '93, Aug 1993.
408. J. Daeman, R. Govaerts, and J. Vandewalle, "Resynchronization Weaknesses in Synchronous Stream Ciphers," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 159–167.
409. J. Daeman, R. Govaerts, and J. Vandewalle, "Weak Keys for IDEA," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 224–230.
410. J. Daemen, R. Govaerts, and J. Vandewalle, "A New Approach to Block Cipher Design," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 18–32.
411. Z.–D. Dai, "Proof of Rueppel's Linear Complexity Conjecture," IEEE Transactions on Information Theory, v. IT–32, n. 3, May 1986, pp. 440–443.
412. I.B. Damgård, "Collision Free Hash Functions and Public Key Signature Schemes," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 203–216.
413. I.B. Damgård, "Payment Systems and Credential Mechanisms with Provable Security Against Abuse by Individuals," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 328–335.
414. I.B. Damgård, "A Design Principle for Hash Functions," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 416–427.
415. I.B. Damgård, "Practical and Provably Secure Release of a Secret and Exchange of Signatures," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 200–217.
416. I.B. Damgård and L.R. Knudsen, "The Breaking of the AR Hash Function," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 286–292.
417. I.B. Damgård and P. Landrock, "Improved Bounds for the Rabin Primality Test," Cryptography and Coding III, M.J. Ganley, ed., Oxford: Clarendon Press, 1993, pp. 117–128.
418. I.B. Damgård, P. Landrock and C. Pomerance, "Average Case Error Estimates for the Strong Probable Prime Test," Mathematics of Computation, v. 61, n. 203, Jul 1993, pp. 177–194.
419. H.E. Daniels, Jr., letter to Datapro Research Corporation regarding CCEP, 23 Dec 1985.
420. H. Davenport, The Higher Arithmetic, Dover Books, 1983.
421. G.I. Davida, "Inverse of Elements of a Galois Field," Electronics Letters, v. 8, n. 21, 19 Oct 1972, pp. 518–520.
422. G.I. Davida, "Hellman's Scheme Breaks DES in Its Basic Form," IEEE Spectrum, v. 16, n. 7, Jul 1979, p. 39.
423. G.I. Davida, "Chosen Signature Cryptanalysis of the RSA (MIT) Public Key Cryptosystem," Technical Report TR–CS–82–2, Department of EECS, University of Wisconsin, 1982.
424. G.I. Davida and G.G. Walter, "A Public Key Analog Cryptosystem," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 143–147.
425. G.I. Davida, D. Wells, and J. Kam, "A Database Encryption System with Subkeys," ACM Transactions on Database Systems, v. 6, n. 2, Jun 1981, pp. 312–328.
426. D.W. Davies, "Applying the RSA Digital Signature to Electronic Mail," Computer, v. 16, n. 2, Feb 1983, pp. 55–62.
427. D.W. Davies, "Some Regular Properties of the DES," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 89–96.
428. D.W. Davies, "A Message Authentication Algorithm Suitable for a Mainframe Computer," Advances in Cryptology: Proceedings of Crypto 82, Springer–Verlag, 1985, pp. 393–400.
429. D.W. Davies and S. Murphy, "Pairs and Triplets of DES S–boxes," Cryptologia, v. 8, n. 1, 1995, pp. 1–25.
430. D.W. Davies and G.I.P. Parkin, "The Average Size of the Key Stream in Output Feedback Encipherment," Cryptography, Proceedings of the Workshop on Cryptography, Burg Feuerstein, Germany, March 29–April 2, 1982, Springer–Verlag, 1983, pp. 263–279.
431. D.W. Davies and G.I.P. Parkin, "The Average Size of the Key Stream in Output Feedback Mode," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 97–98.
432. D.W. Davies and W.L. Price, "The Application of Digital Signatures Based on Public–Key Cryptosystems," Proceedings of the Fifth International Computer Communications Conference, Oct 1980, pp. 525–530.
433. D.W. Davies and W.L. Price, "The Application of Digital Signatures Based on Public–Key Cryptosystems," National Physical Laboratory Report DNACS 39/80, Dec 1980.
434. D.W. Davies and W.L. Price, "Digital Signature—An Update," Proceedings of International Conference on Computer Communications, Sydney, Oct 1984, North Holland: Elsevier, 1985, pp. 843–847.
435. D.W. Davies and W.L. Price, Security for Computer Networks, second edition, John Wiley & Sons, 1989.
436. M. Davio, Y. Desmedt, M. Fosseprez, R. Govaerts, J. Hulsbrosch, P. Neutjens, P. Piret, J.–J. Quisquater, J. Vandewalle, and S. Wouters, "Analytical Characteristics of the Data Encryption Standard," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 171–202.
437. M. Davio, Y. Desmedt, J. Goubert, F. Hoornaert, and J.–J. Quisquater, "Efficient Hardware and Software Implementation of the DES," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 144–146.
438. M. Davio, Y. Desmedt, and J.–J. Quisquater, "Propagation Characteristics of the DES," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, 62–73.
439. D. Davis, R. Ihaka, and P. Fenstermacher, "Cryptographic Randomness from Air Turbulence in Disk Drives," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 114–120.
440. J.A. Davis, D.B. Holdbridge, and G.J. Simmons, "Status Report on Factoring (at the Sandia National Laboratories)," Advances in Cryptology: Proceedings of Crypto 84, Springer–Verlag, 1985, pp. 183–215.
441. R.M. Davis, "The Data Encryption Standard in Perspective," Computer Security and the Data Encryption Standard, National Bureau of Standards Special Publication 500–27, Feb 1978.
442. E. Dawson and A. Clark, "Cryptanalysis of Universal Logic Sequences," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, to appear.
443. M.H. Dawson and S.E. Tavares, "An Expanded Set of Design Criteria for Substitution Boxes and Their Use in Strengthening DES–Like Cryptosystems," IEEE Pacific Rim Conference on Communications, Computers, and Signal Processing, Victoria, BC, Canada, 9–10 May 1991, pp. 191–195.
444. M.H. Dawson and S.E. Tavares, "An Expanded Set of S–Box Design Criteria Based on Information Theory and Its Relation to Differential–like Attacks," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 352–367.
445. C.A. Deavours, "Unicity Points in Cryptanalysis," Cryptologia, v. 1, n. 1, 1977, pp. 46–68.
446. C.A. Deavours, "The Black Chamber: A Column; How the British Broke Enigma," Cryptologia, v. 4, n. 3, Jul 1980, pp. 129– 132.
447. C.A. Deavours, "The Black Chamber: A Column; La Méthode des Bâtons," Cryptologia, v. 4, n. 4, Oct 1980, pp. 240–247.
448. C.A. Deavours and L. Kruh, Machine Cryptography and Modern Cryptanalysis, Norwood MA: Artech House, 1985.
449. J.M. DeLaurentis, "A Further Weakness in the Common Modulus Protocol for the RSA Cryptosystem," Cryptologia, v. 8, n. 3, Jul 1984, pp. 253–259.
450. P. Delsarte, Y. Desmedt, A. Odlyzko, and P. Piret, "Fast Cryptanalysis of the Matsumoto–Imai Public–Key Scheme," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 142–149.
451. P. Delsarte and P. Piret, "Comment on 'Extension of RSA Cryptostructure: A Galois Approach'," Electronics Letters, v. 18, n. 13, 24 Jun 1982, pp. 582–583.
452. R. DeMillo, N. Lynch, and M. Merritt, "Cryptographic Protocols," Proceedings of the 14th Annual Symposium on the Theory of Computing, 1982, pp. 383–400.
453. R. DeMillo and M. Merritt, "Protocols for Data Security," Computer, v. 16, n. 2, Feb 1983, pp. 39–50.
454. N. Demytko, "A New Elliptic Curve Based Analogue of RSA," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 40–49.
455. D.E. Denning, "Secure Personal Computing in an Insecure Network," Communications of the ACM, v. 22, n. 8, Aug 1979, pp. 476–482.
456. D.E. Denning, Cryptography and Data Security, Addison–Wesley, 1982.
457. D.E. Denning, "Protecting Public Keys and Signature Keys," Computer, v. 16, n. 2, Feb 1983, pp. 27–35.
458. D.E. Denning, "Digital Signatures with RSA and Other Public–Key Cryptosystems," Communications of the ACM, v. 27, n. 4, Apr 1984, pp. 388–392.
459. D.E. Denning, "The Data Encryption Standard: Fifteen Years of Public Scrutiny," Proceedings of the Sixth Annual Computer Security Applications Conference, IEEE Computer Society Press, 1990.
460. D.E. Denning, "The Clipper Chip: A Technical Summary," unpublished manuscript, 21 Apr 1993.
461. D.E. Denning and G.M. Sacco, "Timestamps in Key Distribution Protocols," Communications of the ACM, v. 24, n. 8, Aug 1981, pp. 533–536.
462. D.E. Denning and M. Smid, "Key Escrowing Today," IEEE Communications Magazine, v. 32, n. 9, Sep 1994, pp. 58–68.
463. T. Denny, B. Dodson, A.K. Lenstra, and M.S. Manasse, "On the Factorization of RSA–120," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 166–174.
464. W.F. Denny, "Encryptions Using Linear and Non–Linear Codes: Implementations and Security Considerations," Ph.D. dissertation, The Center for Advanced Computer Studies, University of Southern Louisiana, Spring 1988.
465. Department of Defense, "Department of Defense Trusted Computer System Evaluation Criteria," DOD 5200.28–STD, Dec 1985.
466. Department of State, "International Traffic in Arms Regulations (ITAR)," 22 CFR 120–130, Office of Munitions Control, Nov 1989.
467. Department of State, "Defense Trade Regulations," 22 CFR 120–130, Office of Defense Trade Controls, May 1992.
468. Department of the Treasury, "Electronic Funds and Securities Transfer Policy," Department of the Treasury Directives Manual, Chapter TD 81, Section 80, Department of the Treasury, 16 Aug 1984.
469. Department of the Treasury, "Criteria and Procedures for Testing, Evaluating, and Certifying Message Authentication Decisions for Federal E.F.T. Use," Department of the Treasury, 1 May 1985.
470. Department of the Treasury, "Electronic Funds and Securities Transfer Policy—Message Authentication and Enhanced Security," Order No. 106–09, Department of the Treasury, 2 Oct 1986.
471. H. Dobbertin, "A Survey on the Construction of Bent Functions," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
472. B. Dodson and A.K. Lenstra, "NFS with Four Large Primes: An Explosive Experiment," draft manuscript.
473. D. Dolev and A. Yao, "On the Security of Public–Key Protocols," Communications of the ACM, v. 29, n. 8, Aug 1983, pp. 198–208.
474. J. Domingo–Ferrer, "Probabilistic Authentication Analysis," CARDIS 94—Proceedings of the First Smart Card Research and Applications Conference, Lille, France, 24–26 Oct 1994, pp. 49–60.
475. P. de Rooij, "On the Security of the Schnorr Scheme Using Preprocessing," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 71–80.
476. A. De Santis, G. Di Crescenzo, and G. Persiano, "Secret Sharing and Perfect Zero Knowledge," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 73–84.
477. A. De Santis, S. Micali, and G. Persiano, "Non–Interactive Zero–Knowledge Proof Systems," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 52–72.
478. A. De Santis, S. Micali, and G. Persiano, "Non–Interactive Zero–Knowledge with Preprocessing," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 269–282.
479. Y. Desmedt, "What Happened with Knapsack Cryptographic Schemes" Performance Limits in Communication, Theory and Practice, NATO ASI Series E: Applied Sciences, v. 142, Kluwer Academic Publishers, 1988, pp. 113–134.
480. Y. Desmedt, "Subliminal–Free Authentication and Signature," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 23–33.
481. Y. Desmedt, "Abuses in Cryptography and How to Fight Them," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 375–389.
482. Y. Desmedt and M. Burmester, "An Efficient Zero–Knowledge Scheme for the Discrete Logarithm Based on Smooth Numbers," Advances in Cryptology— ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 360–367.
483. Y. Desmedt and Y. Frankel, "Threshold Cryptosystems," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 307–315.
484. Y. Desmedt and Y. Frankel, "Shared Generation of Authentication and Signatures," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 457–469.
485. Y. Desmedt, C. Goutier, and S. Bengio, "Special Uses and Abuses of the Fiat–Shamir Passport Protocol," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 21–39.
486. Y. Desmedt and A.M. Odlykzo, "A Chosen Text Attack on the RSA Cryptosystem and Some Discrete Logarithm Problems," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 516–522.
487. Y. Desmedt, J.–J. Quisquater, and M. Davio, "Dependence of Output on Input in DES: Small Avalanche Characteristics," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 359–376.
488. Y. Desmedt, J. Vandewalle, and R. Govaerts, "Critical Analysis of the Security of Knapsack Public Key Algorithms," IEEE Transactions on Information Theory, v. IT–30, n. 4, Jul 1984, pp. 601–611.
489. Y. Desmedt and M. Yung, "Weaknesses of Undeniable Signature Schemes," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 205–220.
490. W. Diffie, lecture at IEEE Information Theory Workshop, Ithaca, N.Y., 1977.
491. W. Diffie, "Cryptographic Technology: Fifteen Year Forecast," BNR Inc., Jan 1981.
492. W. Diffie, "The First Ten Years of Public–Key Cryptography," Proceedings of the IEEE, v. 76, n. 5, May 1988, pp. 560–577.
493. W. Diffie, "Authenticated Key Exchange and Secure Interactive Communication," Proceedings of SECURICOM '90, 1990.
494. W. Diffie, "The First Ten Years of Public–Key Cryptography," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 135–175.
495. W. Diffie and M.E. Hellman, "Multiuser Cryptographic Techniques," Proceedings of AFIPS National Computer Conference, 1976, pp. 109–112.
496. W. Diffie and M.E. Hellman, "New Directions in Cryptography," IEEE Transactions on Information Theory, v. IT–22, n. 6, Nov 1976, pp. 644–654.
497. W. Diffie and M.E. Hellman, "Exhaustive Cryptanalysis of the NBS Data Encryption Standard," Computer, v. 10, n. 6, Jun 1977, pp. 74–84.
498. W. Diffie and M.E. Hellman, "Privacy and Authentication: An Introduction to Cryptography," Proceedings of the IEEE, v. 67, n. 3, Mar 1979, pp. 397–427.
499. W. Diffie, L. Strawczynski, B. O'Higgins, and D. Steer, "An ISDN Secure Telephone Unit," Proceedings of the National Telecommunications Forum, v. 41, n. 1, 1987, pp. 473–477.
500. W. Diffie, P.C. van Oorschot, and M.J. Wiener, "Authentication and Authenticated Key Exchanges," Designs, Codes and Cryptography, v. 2, 1992, 107–125.
501. C. Ding, "The Differential Cryptanalysis and Design of Natural Stream Ciphers," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 101–115.
502. C. Ding, G. Xiao, and W. Shan, The Stability Theory of Stream Ciphers, Springer–Verlag, 1991.
503. A. Di Porto and W. Wolfowicz, "VINO: A Block Cipher Including Variable Permutations," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 205–210.
504. B. Dixon and A.K. Lenstra, "Factoring Integers Using SIMD Sieves," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 28–39.
505. J.D. Dixon, "Factorization and Primality Tests," American Mathematical Monthly, v. 91, n. 6, 1984, pp. 333–352.
506. D. Dolev and A. Yao, "On the Security of Public Key Protocols," Proceedings of the 22nd Annual Symposium on the Foundations of Computer Science, 1981, pp. 350– 357.
507. L.X. Duan and C.C. Nian, "Modified Lu–Lee Cryptosystems," Electronics Letters, v. 25, n. 13, 22 Jun 1989, p. 826.
508. R. Durstenfeld, "Algorithm 235: Random Permutation," Communications of the ACM, v. 7, n. 7, Jul 1964, p. 420.
509. S. Dussé and B. Kaliski, Jr., "A Cryptographic Library for the Motorola DSP56000," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 230–244.
510. C. Dwork and L. Stockmeyer, "Zero–Knowledge with Finite State Verifiers," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 71–75.
511. D.E. Eastlake, S.D. Crocker, and J.I. Schiller, "Randomness Requirements for Security," RFC 1750, Dec 1994.
512. H. Eberle, "A High–Speed DES Implementation for Network Applications," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, pp. 521–539.
513. J. Edwards, "Implementing Electronic Poker: A Practical Exercise in Zero–Knowledge Interactive Proofs," Master's thesis, Department of Computer Science, University of Kentucky, May 1994.
514. W.F. Ehrsam, C.H.W. Meyer, R.L. Powers, J.L. Smith, and W.L. Tuchman, "Product Block Cipher for Data Security," U.S. Patent #3,962,539, 8 Jun 1976.
515. W.F. Ehrsam, C.H.W. Meyer, and W.L. Tuchman, "A Cryptographic Key Management Scheme for Implementing the Data Encryption Standard," IBM Systems Journal, v. 17, n. 2, 1978, pp. 106–125.
516. R. Eier and H. Lagger, "Trapdoors in Knapsack Cryptosystems," Lecture Notes in Computer Science 149; Cryptography—Proceedings, Burg Feuerstein 1982, Springer–Verlag, 1983, pp. 316–322.
517. A.K. Ekert, "Quantum Cryptography Based on Bell's Theorem," Physical Review Letters, v. 67, n. 6, Aug 1991, pp. 661–663.
518. T. ElGamal, "A Public–Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 10–18.
519. T. ElGamal, "A Public–Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms," IEEE Transactions on Information Theory, v. IT–31, n. 4, 1985, pp. 469–472.
520. T. ElGamal, "On Computing Logarithms Over Finite Fields," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 396–402.
521. T. ElGamal and B. Kaliski, letter to the editor regarding LUC, Dr. Dobb's Journal, v. 18, n. 5, May 1993, p. 10.
522. T. Eng and T. Okamoto, "Single–Term Divisible Electronic Coins," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
523. M.H. Er, D.J. Wong, A.A. Sethu, and K.S. Ngeow, "Design and Implementation of RSA Cryptosystem Using Multiple DSP Chips," 1991 IEEE International Symposium on Circuits and Systems, v. 1, Singapore, 11–14 Jun 1991, pp. 49–52.
524. D. Estes, L.M. Adleman, K. Konpella, K.S. McCurley, and G.L. Miller, "Breaking the Ong–Schnorr–Shamir Signature Schemes for Quadratic Number Fields," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 3–13.
525. ETEBAC, "Échanges Télématiques Entre Les Banques et Leurs Clients," Standard ETEBAC 5, Comité Français d'Organisation et de Normalisation Bancaires, Apr 1989. (In French.)
526. A. Evans, W. Kantrowitz, and E. Weiss, "A User Identification Scheme Not Requiring Secrecy in the Computer," Communications of the ACM, v. 17, n. 8, Aug 1974, pp. 437–472.
527. S. Even and O. Goldreich, "DES–Like Functions Can Generate the Alternating Group," IEEE Transactions on Information Theory, v. IT–29, n. 6, Nov 1983, pp. 863–865.
528. S. Even and O. Goldreich, "On the Power of Cascade Ciphers," ACM Transactions on Computer Systems, v. 3, n. 2, May 1985, pp. 108–116.
529. S. Even, O. Goldreich, and A. Lempel, "A Randomizing Protocol for Signing Contracts," Communications of the ACM, v. 28, n. 6, Jun 1985, pp. 637–647.
530. S. Even and Y. Yacobi, "Cryptography and NP–Completeness," Proceedings of the 7th International Colloquium on Automata, Languages, and Programming, Springer–Verlag, 1980, pp. 195–207.
531. H.–H. Evertse, "Linear Structures in Block Ciphers," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 249–266.
532. P. Fahn and M.J.B. Robshaw, "Results from the RSA Factoring Challenge," Technical Report TR–501, Version 1.3, RSA Laboratories, Jan 1995.
533. R.C. Fairfield, A. Matusevich, and J. Plany, "An LSI Digital Encryption Processor (DEP)," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 115–143.
534. R.C. Fairfield, A. Matusevich, and J. Plany, "An LSI Digital Encryption Processor (DEP)," IEEE Communications, v. 23, n. 7, Jul 1985, pp. 30–41.
535. R.C. Fairfield, R.L. Mortenson, and K.B. Koulthart, "An LSI Random Number Generator (RNG)," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 203–230.
536. "International Business Machines Corp. License Under Patents," Federal Register, v. 40, n. 52, 17 Mar 1975, p. 12067.
537. "Solicitation for Public Key Cryptographic Algorithms," Federal Register, v. 47, n. 126, 30 Jun 1982, p. 28445.
538. "Proposed Federal Information Processing Standard for Digital Signature Standard (DSS)," Federal Register, v. 56, n. 169, 30 Aug 1991, pp. 42980–42982.
539. "Proposed Federal Information Processing Standard for Secure Hash Standard," Federal Register, v. 57, n. 21, 31 Jan 1992, pp. 3747–3749.
540. "Proposed Reaffirmation of Federal Information Processing Standard (FIPS) 46–1, Data Encryption Standard (DES)," Federal Register, v. 57, n. 177, 11 Sep 1992, p. 41727.
541. "Notice of Proposal for Grant of Exclusive Patent License," Federal Register, v. 58, n. 108, 8 Jun 1993, pp. 23105–23106.
542. "Approval of Federal Information Processing Standards Publication 186, Digital Signature Standard (DSS)," Federal Register, v. 58, n. 96, 19 May 1994, pp. 26208–26211.
543. "Proposed Revision of Federal Information Processing Standard (FIPS) 180, Secure Hash Standard," Federal Register, v. 59, n. 131, 11 Jul 1994, pp. 35317–35318.
544. U. Feige, A. Fiat, and A. Shamir, "Zero Knowledge Proofs of Identity," Proceedings of the 19th Annual ACM Symposium on the Theory of Computing, 1987, pp. 210–217.
545. U. Feige, A. Fiat, and A. Shamir, "Zero Knowledge Proofs of Identity," Journal of Cryptology, v. 1, n. 2, 1988, pp. 77–94.
546. U. Feige and A. Shamir, "Zero Knowledge Proofs of Knowledge in Two Rounds," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 526–544.
547. J. Feigenbaum, "Encrypting Problem Instances, or,..., Can You Take Advantage of Someone Without Having to Trust Him," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 477–488.
548. J. Feigenbaum, "Overview of Interactive Proof Systems and Zero–Knowledge," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 423–439.
549. J. Feigenbaum, M.Y. Liberman, E. Grosse, and J.A. Reeds, "Cryptographic Protection of Membership Lists," Newsletter of the International Association of Cryptologic Research, v. 9, 1992, pp. 16–20.
550. J. Feigenbaum, M.Y. Liverman, and R.N. Wright, "Cryptographic Protection of Databases and Software," Distributed Computing and Cryptography, J. Feigenbaum and M. Merritt, eds., American Mathematical Society, 1991, pp. 161–172.
551. H. Feistel, "Cryptographic Coding for Data–Bank Privacy," RC 2827, Yorktown Heights, NY: IBM Research, Mar 1970.
552. H. Feistel, "Cryptography and Computer Privacy," Scientific American, v. 228, n. 5, May 1973, pp. 15–23.
553. H. Feistel, "Block Cipher Cryptographic System," U.S. Patent #3,798,359, 19 Mar 1974.
554. H. Feistel, "Step Code Ciphering System," U.S. Patent #3,798,360, 19 Mar 1974.
555. H. Feistel, "Centralized Verification System," U.S. Patent #3,798,605, 19 Mar 1974.
556. H. Feistel, W.A. Notz, and J.L. Smith, "Cryptographic Techniques for Machine to Machine Data Communications," RC 3663, Yorktown Heights, N.Y.: IBM Research, Dec 1971.
557. H. Feistel, W.A. Notz, and J.L. Smith, "Some Cryptographic Techniques for Machine to Machine Data Communications," Proceedings of the IEEE, v. 63, n. 11, Nov 1975, pp. 1545–1554.
558. P. Feldman, "A Practical Scheme for Non–interactive Verifiable Secret Sharing," Proceedings of the 28th Annual Symposium on the Foundations of Computer Science, 1987, pp. 427–437.
559. R.A. Feldman, "Fast Spectral Test for Measuring Nonrandomness and the DES," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 243–254.
560. R.A. Feldman, "A New Spectral Test for Nonrandomness and the DES," IEEE Transactions on Software Engineering, v. 16, n. 3, Mar 1990, pp. 261–267.
561. D.C. Feldmeier and P.R. Karn, "UNIX Password Security—Ten Years Later," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 44–63.
562. H. Fell and W. Diffie, "Analysis of a Public Key Approach Based on Polynomial Substitution," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 427–437.
563. N.T. Ferguson, "Single Term Off–Line Coins," Report CS–R9318, Computer Science/Department of Algorithms and Architecture, CWI, Mar 1993.
564. N.T. Ferguson, "Single Term Off–Line Coins," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 318–328.
565. N.T. Ferguson, "Extensions of Single–term Coins," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 292–301.
566. A. Fiat and A. Shamir, "How to Prove Yourself: Practical Solutions to Identification and Signature Problems," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 186–194.
567. A. Fiat and A. Shamir, "Unforgeable Proofs of Identity," Proceedings of Securicom 87, Paris, 1987, pp. 147–153.
568. P. Finch, "A Study of the Blowfish Encryption Algorithm," Ph.D. dissertation, Department of Computer Science, City University of New York Graduate School and University Center, Feb 1995.
569. R. Flynn and A.S. Campasano, "Data Dependent Keys for Selective Encryption Terminal," Proceedings of NCC, vol. 47, AFIPS Press, 1978, pp. 1127–1129.
570. R.H. Follett, letter to NIST regarding DSS, 25 Nov 1991.
571. R. Forré, "The Strict Avalanche Criterion: Spectral Properties and an Extended Definition," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 450–468.
572. R. Forré, "A Fast Correlation Attack on Nonlinearity Feedforward Filtered Shift Register Sequences," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 568–595.
573. S. Fortune and M. Merritt, "Poker Protocols," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 454–464.
574. R.B. Fougner, "Public Key Standards and Licenses," RFC 1170, Jan 1991.
575. Y. Frankel and M. Yung, "Escrowed Encryption Systems Visited: Threats, Attacks, Analysis and Designs," Advances in Cryptology—CRYPTO '95 Proceedings, Springer–Verlag, 1995, to appear.
576. W.F. Friedman, Methods for the Solution of Running–Key Ciphers, Riverbank Publication No. 16, Riverbank Labs, 1918.
577. W.F. Friedman, The Index of Coincidence and Its Applications in Cryptography, Riverbank Publication No. 22, Riverbank Labs, 1920. Reprinted by Aegean Park Press, 1987.
578. W.F. Friedman, Elements of Cryptanalysis, Laguna Hills, CA: Aegean Park Press, 1976.
579. W.F. Friedman, "Cryptology," Encyclopedia Britannica, v. 6, pp. 844–851, 1967.
580. A.M. Frieze, J. Hastad, R. Kannan, J.C. Lagarias, and A. Shamir, "Reconstructing Truncated Integer Variables Satisfying Linear Congruences," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 262–280.
581. A.M. Frieze, R. Kannan, and J.C. Lagarias, "Linear Congruential Generators Do not Produce Random Sequences," Proceedings of the 25th IEEE Symposium on Foundations of Computer Science, 1984, pp. 480–484.
582. E. Fujiaski and T. Okamoto, "On Comparison of Practical Digitial Signature Schemes," Proceedings of the 1992 Symposium on Cryptography and Information Security (SCIS 92), Tateshina, Japan, 2–4 Apr 1994, pp. 1A.1–12.
583. A. Fujioka, T. Okamoto, and S. Miyaguchi, "ESIGN: An Efficient Digital Signature Implementation for Smart Cards," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 446–457.
584. A. Fujioka, T. Okamoto, and K. Ohta, "Interactive Bi–Proof Systems and Undeniable Signature Schemes," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 243–256.
585. A. Fujioka, T. Okamoto, and K. Ohta, "A Practical Secret Voting Scheme for Large Scale Elections," Advances in Cryptology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 244–251.
586. K. Gaardner and E. Snekkenes, "Applying a Formal Analysis Technique to the CCITT X.509 Strong Two–Way Authentication Protocol," Journal of Cryptology, v. 3, n. 2, 1991, pp. 81–98.
587. H.F. Gaines, Cryptanalysis, American Photographic Press, 1937. (Reprinted by Dover Publications, 1956.)
588. J. Gait, "A New Nonlinear Pseudorandom Number Generator," IEEE Transactions on Software Engineering, v. SE–3, n. 5, Sep 1977, pp. 359–363.
589. J. Gait, "Short Cycling in the Kravitz–Reed Public Key Encryption System," Electronics Letters, v. 18, n. 16, 5 Aug 1982, pp. 706–707.
590. Z. Galil, S. Haber, and M. Yung, "A Private Interactive Test of a Boolean Predicate and Minimum–Knowledge Public–Key Cryptosystems," Proceedings of the 26th IEEE Symposium on Foundations of Computer Science, 1985, pp. 360–371.
591. Z. Galil, S. Haber, and M. Yung, "Cryptographic Computation: Secure Fault–Tolerant Protocols and the Public–Key Model," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 135–155.
592. Z. Galil, S. Haber, and M. Yung, "Minimum–Knowledge Interactive Proofs for Decision Problems," SIAM Journal on Computing, v. 18, n. 4, 1989, pp. 711–739.
593. R.G. Gallager, Information Theory and Reliable Communications, New York: John Wiley & Sons, 1968.
594. P. Gallay and E. Depret, "A Cryptography Microprocessor," 1988 IEEE International Solid–State Circuits Conference Digest of Technical Papers, 1988, pp. 148–149.
595. R.A. Games, "There are no de Bruijn Sequences of Span n with Complexity 2n–1 + n + 1," Journal of Combinatorical Theory, Series A, v. 34, n. 2, Mar 1983, pp. 248–251.
596. R.A. Games and A.H. Chan, "A Fast Algorithm for Determining the Complexity of a Binary Sequence with 2n," IEEE Transactions on Information Theory, v. IT–29, n. 1, Jan 1983, pp. 144–146.
597. R.A. Games, A.H. Chan, and E.L. Key, "On the Complexity of de Bruijn Sequences," Journal of Combinatorical Theory, Series A, v. 33, n. 1, Nov 1982, pp. 233–246.
598. S.H. Gao and G.L. Mullen, "Dickson Polynomials and Irreducible Polynomials over Finite Fields," Journal of Number Theory, v. 49, n. 1, Oct 1994, pp. 18–132.
599. M. Gardner, "A New Kind of Cipher That Would Take Millions of Years to Break," Scientific American, v. 237, n. 8, Aug 1977, pp. 120–124.
600. M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP–Completeness, W.H. Freeman and Co., 1979.
601. S.L. Garfinkel, PGP: Pretty Good Privacy, Sebastopol, CA: O'Reilly and Associates, 1995.
602. C.W. Gardiner, "Distributed Public Key Certificate Management," Proceedings of the Privacy and Security Research Group 1993 Workshop on Network and Distributed System Security, The Internet Society, 1993, pp. 69–73.
603. G. Garon and R. Outerbridge, "DES Watch: An Examination of the Sufficiency of the Data Encryption Standard for Financial Institution Information Security in the 1990's," Cryptologia, v. 15, n. 3, Jul 1991, pp. 177–193.
604. M. Gasser, A. Goldstein, C. Kaufman, and B. Lampson, "The Digital Distributed Systems Security Architecture," Proceedings of the 12th National Computer Security Conference, NIST, 1989, pp. 305–319.
605. J. von zur Gathen, D. Kozen, and S. Landau, "Functional Decomposition of Polynomials," Proceedings of the 28th IEEE Symposium on the Foundations of Computer Science, IEEE Press, 1987, pp. 127– 131.
606. P.R. Geffe, "How to Protect Data With Ciphers That are Really Hard to Break," Electronics, v. 46, n. 1, Jan 1973, pp. 99–101.
607. D.K. Gifford, D. Heitmann, D.A. Segal, R.G. Cote, K. Tanacea, and D.E. Burmaster, "Boston Community Information System 1986 Experimental Test Results," MIT/LCS/TR–397, MIT Laboratory for Computer Science, Aug 1987.
608. D.K. Gifford, J.M. Lucassen, and S.T. Berlin, "The Application of Digital Broadcast Communication to Large Scale Information Systems," IEEE Journal on Selected Areas in Communications, v. 3, n. 3, May 1985, pp. 457–467.
609. D.K. Gifford and D.A. Segal, "Boston Community Information System 1987–1988 Experimental Test Results," MIT/LCS/TR–422, MIT Laboratory for Computer Science, May 1989.
610. H. Gilbert and G. Chase, "A Statistical Attack on the Feal–8 Cryptosystem," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 22–33.
611. H. Gilbert and P. Chauvaud, "A Chosen Plaintext Attack of the 16–Round Khufu Cryptosystem," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 259–268.
612. M. Girault, "Hash–Functions Using Modulo–N Operations," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 217–226.
613. J. Gleick, "A New Approach to Protecting Secrets is Discovered," The New York Times, 18 Feb 1987, pp. C1 and C3.
614. J.–M. Goethals and C. Couvreur, "A Cryptanalytic Attack on the Lu–Lee Public–Key Cryptosystem," Philips Journal of Research, v. 35, 1980, pp. 301–306.
615. O. Goldreich, "A Uniform–Complexity Treatment of Encryption and Zero–Knowledge, Journal of Cryptology, v. 6, n. 1, 1993, pp. 21–53.
616. O. Goldreich and H. Krawczyk, "On the Composition of Zero Knowledge Proof Systems," Proceedings on the 17th International Colloquium on Automata, Languages, and Programming, Springer–Verlag, 1990, pp. 268–282.
617. O. Goldreich and E. Kushilevitz, "A Perfect Zero–Knowledge Proof for a Problem Equivalent to Discrete Logarithm," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 58–70.
618. O. Goldreich and E. Kushilevitz, "A Perfect Zero–Knowledge Proof for a Problem Equivalent to Discrete Logarithm," Journal of Cryptology, v. 6, n. 2, 1993, pp. 97–116.
619. O. Goldreich, S. Micali, and A. Wigderson, "Proofs That Yield Nothing but Their Validity and a Methodology of Cryptographic Protocol Design," Proceedings of the 27th IEEE Symposium on the Foundations of Computer Science, 1986, pp. 174–187.
620. O. Goldreich, S. Micali, and A. Wigderson, "How to Prove All NP Statements in Zero Knowledge and a Methodology of Cryptographic Protocol Design," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 171–185.
621. O. Goldreich, S. Micali, and A. Wigderson, "How to Play Any Mental Game," Proceedings of the 19th ACM Symposium on the Theory of Computing, 1987, pp. 218–229.
622. O. Goldreich, S. Micali, and A. Wigderson, "Proofs That Yield Nothing but Their Validity and a Methodology of Cryptographic Protocol Design," Journal of the ACM, v. 38, n. 1, Jul 1991, pp. 691–729.
623. S. Goldwasser and J. Kilian, "Almost All Primes Can Be Quickly Certified," Proceedings of the 18th ACM Symposium on the Theory of Computing, 1986, pp. 316– 329.
624. S. Goldwasser and S. Micali, "Probabilistic Encryption and How to Play Mental Poker Keeping Secret All Partial Information," Proceedings of the 14th ACM Symposium on the Theory of Computing, 1982, pp. 270–299.
625. S. Goldwasser and S. Micali, "Probabilistic Encryption," Journal of Computer and System Sciences, v. 28, n. 2, Apr 1984, pp. 270–299.
626. S. Goldwasser, S. Micali, and C. Rackoff, "The Knowledge Complexity of Interactive Proof Systems," Proceedings of the 17th ACM Symposium on Theory of Computing, 1985, pp. 291–304.
627. S. Goldwasser, S. Micali, and C. Rackoff, "The Knowledge Complexity of Interactive Proof Systems," SIAM Journal on Computing, v. 18, n. 1, Feb 1989, pp. 186– 208.
628. S. Goldwasser, S. Micali, and R.L. Rivest, "A Digital Signature Scheme Secure Against Adaptive Chosen–Message Attacks," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 281–308.
629. S. Goldwasser, S. Micali, and A.C. Yao, "On Signatures and Authentication," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 211–215.
630. J.D. Golic, "On the Linear Complexity of Functions of Periodic GF(q) Sequences," IEEE Transactions on Information Theory, v. IT–35, n. 1, Jan 1989, pp. 69–75.
631. J.D. Golic, "Linear Cryptanalysis of Stream Ciphers," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, pp. 262–282.
632. J.D. Golic, "Towards Fast Correlation Attacks on Irregularly Clocked Shift Registers," Advances in Cryptology—EUROCRYPT '95 Proceedings, Springer–Verlag, 1995, to appear.
633. J.D. Golic and M.J. Mihajlevic, "A Generalized Correlation Attack on a Class of Stream Ciphers Based on the Levenshtein Distance," Journal of Cryptology, v. 3, n. 3, 1991, pp. 201–212.
634. J.D. Golic and L. O'Connor, "Embedding and Probabilistic Correlation Attacks on Clock–Controlled Shift Registers," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
635. R. Golliver, A.K. Lenstra, K.S. McCurley, "Lattice Sieving and Trial Division," Proceedings of the Algorithmic Number Theory Symposium, Cornell, 1994, to appear.
636. D. Gollmann, "Kaskadenschaltungen taktgesteuerter Schieberegister als Pseudozufallszahlengeneratoren," Ph.D. dissertation, Universität Linz, 1983. (In German.)
637. D. Gollmann, "Pseudo Random Properties of Cascade Connections of Clock Controlled Shift Registers," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 93–98.
638. D. Gollmann, "Correlation Analysis of Cascaded Sequences," Cryptography and Coding, H.J. Beker and F.C. Piper, eds., Oxford: Clarendon Press, 1989, pp. 289–297.
639. D. Gollmann, "Transformation Matrices of Clock–Controlled Shift Registers," Cryptography and Coding III, M.J. Ganley, ed., Oxford: Clarendon Press, 1993, pp. 197–210.
640. D. Gollmann and W.G. Chambers, "Lock–In Effect in Cascades of Clock–Controlled Shift–Registers," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 331–343.
641. D. Gollmann and W.G. Chambers, "Clock–Controlled Shift Registers: A Review," IEEE Journal on Selected Areas in Communications, v. 7, n. 4, May 1989, pp. 525–533.
642. D. Gollmann and W.G. Chambers, "A Cryptanalysis of Stepk,m–cascades," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 680–687.
643. S.W. Golomb, Shift Register Sequences, San Francisco: Holden–Day, 1967. (Reprinted by Aegean Park Press, 1982.)
644. L. Gong, "A Security Risk of Depending on Synchronized Clocks," Operating Systems Review, v. 26, n. 1, Jan 1992, pp. 49–53.
645. L. Gong, R. Needham, and R. Yahalom, "Reasoning About Belief in Cryptographic Protocols," Proceedings of the 1991 IEEE Computer Society Symposium on Research in Security and Privacy, 1991, pp. 234–248.
646. R.M. Goodman and A.J. McAuley, "A New Trapdoor Knapsack Public Key Cryptosystem," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 150–158.
647. R.M. Goodman and A.J. McAuley, "A New Trapdoor Knapsack Public Key Cryptosystem," IEE Proceedings, v. 132, pt. E, n. 6, Nov 1985, pp. 289–292.
648. D.M. Gordon, "Discrete Logarithms Using the Number Field Sieve," Preprint, 28 Mar 1991.
649. D.M. Gordon and K.S. McCurley, "Computation of Discrete Logarithms in Fields of Characteristic Two," presented at the rump session of CRYPTO '91, Aug 1991.
650. D.M. Gordon and K.S. McCurley, "Massively Parallel Computation of Discrete Logarithms," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 312–323.
651. J.A. Gordon, "Strong Primes are Easy to Find," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 216–223.
652. J.A. Gordon, "Very Simple Method to Find the Minimal Polynomial of an Arbitrary Non–Zero Element of a Finite Field," Electronics Letters, v. 12, n. 25, 9 Dec 1976, pp. 663–664.
653. J.A. Gordon and R. Retkin, "Are Big S–Boxes Best?" Cryptography, Proceedings of the Workshop on Cryptography, Burg Feuerstein, Germany, March 29–April 2, 1982, Springer–Verlag, 1983, pp. 257–262.
654. M. Goresky and A. Klapper, "Feedback Registers Based on Ramified Extension of the 2–adic Numbers," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
655. GOST, Gosudarstvennyi Standard 28147–89, "Cryptographic Protection for Data Processing Systems," Government Committee of the USSR for Standards, 1989. (In Russian.)
656. GOST R 34.10–94, Gosudarstvennyi Standard of Russian Federation, "Information technology. Cryptographic Data Security. Produce and check procedures of Electronic Digital Signature based on Asymmetric Cryptographic Algorithm." Government Committee of the Russia for Standards, 1994. (In Russian.)
657. GOST R 34.11–94, Gosudarstvennyi Standard of Russian Federation, "Information technology. Cryptographic Data Security. Hashing function." Government Committee of the Russia for Standards, 1994. (In Russian.)
658. R. Göttfert and H. Niederreiter, "On the Linear Complexity of Products of Shift–Register Sequences," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 151–158.
659. R. Göttfert and H. Niederreiter, "A General Lower Bound for the Linear Complexity of the Product of Shift–Register Sequences," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
660. J. van de Graaf and R. Peralta, "A Simple and Secure Way to Show the Validity of Your Public Key," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 128–134.
661. J. Grollman and A.L. Selman, "Complexity Measures for Public–Key Cryptosystems," Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science, 1984, pp. 495–503.
662. GSA Federal Standard 1026, "Telecommunications: General Security Requirements for Equipment Using the Data Encryption Standard," General Services Administration, Apr 1982.
663. GSA Federal Standard 1027, "Telecommunications: Interoperability and Security Requirements for Use of the Data Encryption Standard in the Physical and Data Link Layers of Data Communications," General Services Administration, Jan 1983.
664. GSA Federal Standard 1028, "Interoperability and Security Requirements for Use of the Data Encryption Standard with CCITT Group 3 Facsimile Equipment," General Services Administration, Apr 1985.
665. P. Guam, "Cellular Automaton Public Key Cryptosystems," Complex Systems, v. 1, 1987, pp. 51–56.
666. H. Guan, "An Analysis of the Finite Automata Public Key Algorithm," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 120–126. (In Chinese.)
667. G. Guanella, "Means for and Method for Secret Signalling," U.S. Patent #2,405,500, 6 Aug 1946.
668. M. Gude, "Concept for a High–Performance Random Number Generator Based on Physical Random Phenomena," Frequenz, v. 39, 1985, pp. 187–190.
669. M. Gude, "Ein quasi–idealer Gleichverteilungsgenerator basierend auf physikalischen Zufallsphänomenen," Ph.D. dissertation, Aachen University of Technology, 1987. (In German.)
670. L.C. Guillou and J.–J. Quisquater, "A Practical Zero–Knowledge Protocol Fitted to Security Microprocessor Minimizing Both Transmission and Memory," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 123–128.
671. L.C. Guillou and J.–J. Quisquater, "A 'Paradoxical' Identity–Based Signature Scheme Resulting from Zero–Knowledge," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 216– 231.
672. L.C. Guillou, M. Ugon, and J.–J. Quisquater, "The Smart Card: A Standardized Security Device Dedicated to Public Cryptology," Contemporary Cryptology: The Science of Information Integrity, G. Simmons, ed., IEEE Press, 1992, pp. 561–613.
673. C.G. Günther, "Alternating Step Generators Controlled by de Bruijn Sequences," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 5–14.
674. C.G. Günther, "An Identity–based Key–exchange Protocol," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 29–37.
675. H. Gustafson, E. Dawson, and B. Caelli, "Comparison of Block Ciphers," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 208–220.
676. P. Gutmann, personal communication, 1993.
677. H. Gutowitz, "A Cellular Automaton Cryptosystem: Specification and Call for Attack," unpublished manuscript, Aug 1992.
678. H. Gutowitz, "Method and Apparatus for Encryption, Decryption, and Authentication Using Dynamical Systems," U.S. Patent #5,365,589, 15 Nov 1994.
679. H. Gutowitz, "Cryptography with Dynamical Systems," Cellular Automata and Cooperative Phenomenon, Kluwer Academic Press, 1993.
680. R.K. Guy, "How to Factor a Number," Fifth Manitoba Conference on Numeral Mathematics Congressus Numerantium, v. 16, 1976, pp. 49–89.
681. R.K. Guy, Unsolved Problems in Number Theory, Springer–Verlag, 1981.
682. S. Haber and W.S. Stornetta, "How to Time–Stamp a Digital Document," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 437–455.
683. S. Haber and W.S. Stornetta, "How to Time–Stamp a Digital Document," Journal of Cryptology, v. 3, n. 2, 1991, pp. 99–112.
684. S. Haber and W.S. Stornetta, "Digital Document Time–Stamping with Catenate Certificate," U.S. Patent #5,136,646, 4 Aug 1992.
685. S. Haber and W.S. Stornetta, "Method for Secure Time–Stamping of Digital Documents," U.S. Patent #5,136,647, 4 Aug 1992.
686. S. Haber and W.S. Stornetta, "Method of Extending the Validity of a Cryptographic Certificate," U.S. Patent #5,373,561, 13 Dec 1994.
687. T. Habutsu, Y. Nishio, I. Sasase, and S. Mori, "A Secret Key Cryptosystem by Iterating a Chaotic Map," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E73, n. 7, Jul 1990, pp. 1041–1044.
688. T. Habutsu, Y. Nishio, I. Sasase, and S. Mori, "A Secret Key Cryptosystem by Iterating a Chaotic Map," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 127–140.
689. S. Hada and H. Tanaka, "An Improvement Scheme of DES against Differential Cryptanalysis," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp 14A.1–11. (In Japanese.)
690. B.C.W. Hagelin, "The Story of the Hagelin Cryptos," Cryptologia, v. 18, n. 3, Jul 1994, pp. 204–242.
691. T. Hansen and G.L. Mullen, "Primitive Polynomials over Finite Fields," Mathematics of Computation, v. 59, n. 200, Oct 1992, pp. 639–643.
692. S. Harada and S. Kasahara, "An ID–Based Key Sharing Scheme Without Preliminary Communication," IEICE Japan, Technical Report, ISEC89–38, 1989. (In Japanese.)
693. S. Harari, "A Correlation Cryptographic Scheme," EUROCODE '90—International Symposium on Coding Theory, Springer–Verlag, 1991, pp. 180–192.
694. T. Hardjono and J. Seberry, "Authentication via Multi–Service Tickets in the Kuperee Server," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 144–160.
695. L. Harn and T. Kiesler, "New Scheme for Digital Multisignatures," Electronics Letters, v. 25, n. 15, 20 Jul 1989, pp. 1002– 1003.
696. L. Harn and T. Kiesler, "Improved Rabin's Scheme with High Efficiency," Electronics Letters, v. 25, n. 15, 20 Jul 1989, p. 1016.
697. L. Harn and T. Kiesler, "Two New Efficient Cryptosystems Based on Rabin's Scheme," Fifth Annual Computer Security Applications Conference, IEEE Computer Society Press, 1990, pp. 263–270.
698. L. Harn and D.–C. Wang, "Cryptanalysis and Modification of Digital Signature Scheme Based on Error–Correcting Codes," Electronics Letters, v. 28, n. 2, 10 Jan 1992, p. 157–159.
699. L. Harn and Y. Xu, "Design of Generalized ElGamal Type Digital Signature Schemes Based on Discrete Logarithm," Electronics Letters, v. 30, n. 24, 24 Nov 1994, p. 2025–2026.
700. L. Harn and S. Yang, "Group–Oriented Undeniable Signature Schemes without the Assistance of a Mutually Trusted Party," Advances in Cryptology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 133–142.
701. G. Harper, A. Menezes, and S. Vanstone, "Public–Key Cryptosystems with Very Small Key Lengths," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 163–173.
702. C. Harpes, "Notes on High Order Differential Cryptanalysis of DES," internal report, Signal and Information Processing Laboratory, Swiss Federal Institute of Technology, Aug 1993.
703. G.W. Hart, "To Decode Short Cryptograms," Communications of the ACM, v. 37, n. 9, Sep 1994, pp. 102–108.
704. J. Hastad, "On Using RSA with Low Exponent in a Public Key Network," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 403–408.
705. J. Hastad and A. Shamir, "The Cryptographic Security of Truncated Linearly Related Variables," Proceedings of the 17th Annual ACM Symposium on the Theory of Computing, 1985, pp. 356–362.
706. R.C. Hauser and E.S. Lee, "Verification and Modelling of Authentication Protocols," ESORICS 92, Proceedings of the Second European Symposium on Research in Computer Security, Springer–Verlag, 1992, pp. 131–154.
707. B. Hayes, "Anonymous One–Time Signatures and Flexible Untraceable Electronic Cash," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 294–305.
708. D.K. He, "LUC Public Key Cryptosystem and its Properties," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 60–69. (In Chinese.)
709. J. He and T. Kiesler, "Enhancing the Security of ElGamal's Signature Scheme," IEE Proceedings on Computers and Digital Techniques, v. 141, n. 3, 1994, pp. 193–195.
710. E.H. Hebern, "Electronic Coding Machine," U.S. Patent #1,510,441, 30 Sep 1924.
711. N. Heintze and J.D. Tygar, "A Model for Secure Protocols and their Compositions," Proceedings of the 1994 IEEE Computer Society Symposium on Research in Security and Privacy, 1994, pp. 2–13.
712. M.E. Hellman, "An Extension of the Shannon Theory Approach to Cryptography," IEEE Transactions on Information Theory, v. IT–23, n. 3, May 1977, pp. 289–294.
713. M.E. Hellman, "The Mathematics of Public–Key Cryptography," Scientific American, v. 241, n. 8, Aug 1979, pp. 146–157.
714. M.E. Hellman, "DES Will Be Totally Insecure within Ten Years," IEEE Spectrum, v. 16, n. 7, Jul 1979, pp. 32–39.
715. M.E. Hellman, "On DES–Based Synchronous Encryption," Dept. of Electrical Engineering, Stanford University, 1980.
716. M.E. Hellman, "A Cryptanalytic Time–Memory Trade Off," IEEE Transactions on Information Theory, v. 26, n. 4, Jul 1980, pp. 401–406.
717. M.E. Hellman, "Another Cryptanalytic Attack on 'A Cryptosystem for Multiple Communications'," Information Processing Letters, v. 12, 1981, pp. 182–183.
718. M.E. Hellman, W. Diffie, and R.C. Merkle, "Cryptographic Apparatus and Method," U.S. Patent #4,200,770, 29 Apr 1980.
719. M.E. Hellman, W. Diffie, and R.C. Merkle, "Cryptographic Apparatus and Method," Canada Patent #1,121,480, 6 Apr 1982.
720. M.E. Hellman and R.C. Merkle, "Public Key Cryptographic Apparatus and Method," U.S. Patent #4,218,582, 19 Aug 1980.
721. M.E. Hellman, R. Merkle, R. Schroeppel, L. Washington, W. Diffie, S. Pohlig, and P. Schweitzer, "Results of an Initial Attempt to Cryptanalyze the NBS Data Encryption Standard," Technical Report SEL 76–042, Information Systems Lab, Department of Electrical Engineering, Stanford University, 1976.
722. M.E. Hellman and S.C. Pohlig, "Exponentiation Cryptographic Apparatus and Method," U.S. Patent #4,424,414, 3 Jan 1984.
723. M.E. Hellman and J.M. Reyneri, "Distribution of Drainage in the DES," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 129–131.
724. F. Hendessi and M.R. Aref, "A Successful Attack Against the DES," Third Canadian Workshop on Information Theory and Applications, Springer–Verlag, 1994, pp. 78–90.
725. T. Herlestam, "Critical Remarks on Some Public–Key Cryptosystems," BIT, v. 18, 1978, pp. 493–496.
726. T. Herlestam, "On Functions of Linear Shift Register Sequences", Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 119–129.
727. T. Herlestam and R. Johannesson, "On Computing Logarithms over GF(2p)," BIT, v. 21, 1981, pp. 326–334.
728. H.M. Heys and S.E. Tavares, "On the Security of the CAST Encryption Algorithm," Proceedings of the Canadian Conference on Electrical and Computer Engineering, Halifax, Nova Scotia, Sep 1994, pp. 332–335.
729. H.M. Heys and S.E. Tavares, "The Design of Substitution–Permutation Networks Resistant to Differential and Linear Cryptanalysis," Proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 148–155.
730. E. Heyst and T.P. Pederson, "How to Make Fail–Stop Signatures," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 366–377.
731. E. Heyst, T.P. Pederson, and B. Pfitzmann, "New Construction of Fail–Stop Signatures and Lower Bounds," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 15–30.
732. L.S. Hill, "Cryptography in an Algebraic Alphabet," American Mathematical Monthly, v. 36, Jun–Jul 1929, pp. 306–312.
733. P.J.M. Hin, "Channel–Error–Correcting Privacy Cryptosystems," Ph.D. dissertation, Delft University of Technology, 1986. (In Dutch.)
734. R. Hirschfeld, "Making Electronic Refunds Safer," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 106–112.
735. A. Hodges, Alan Turing: The Enigma of Intelligence, Simon and Schuster, 1983.
736. W. Hohl, X. Lai, T. Meier, and C. Waldvogel, "Security of Iterated Hash Functions Based on Block Ciphers," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 379–390.
737. F. Hoornaert, M. Decroos, J. Vandewalle, and R. Govaerts, "Fast RSA–Hardware: Dream or Reality?" Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 257–264.
738. F. Hoornaert, J. Goubert, and Y. Desmedt, "Efficient Hardware Implementation of the DES," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 147–173.
739. E. Horowitz and S. Sahni, Fundamentals of Computer Algorithms, Rockville, MD: Computer Science Press, 1978.
740. P. Horster, H. Petersen, and M. Michels, "Meta–ElGamal Signature Schemes," Proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 96–107.
741. P. Horster, H. Petersen, and M. Michels, "Meta Message Recovery and Meta Blind Signature Schemes Based on the Discrete Logarithm Problem and their Applications," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 224–237.
742. L.K. Hua, Introduction to Number Theory, Springer–Verlag, 1982.
743. K. Huber, "Specialized Attack on Chor–Rivest Public Key Cryptosystem," Electronics Letters, v. 27, n. 23, 7 Nov 1991, pp. 2130–2131.
744. E. Hughes, "A Cypherpunk's Manifesto," 9 Mar 1993.
745. E. Hughes, "An Encrypted Key Transmission Protocol," presented at the rump session of CRYPTO '94, Aug 1994.
746. H. Hule and W.B. Müller, "On the RSA–Cryptosystem with Wrong Keys," Contributions to General Algebra 6, Vienna: Verlag Hölder–Pichler–Tempsky, 1988, pp. 103–109.
747. H.A. Hussain, J.W.A. Sada, and S.M. Kalipha, "New Multistage Knapsack Public–Key Cryptosystem," International Journal of Systems Science, v. 22, n. 11, Nov 1991, pp. 2313–2320.
748. T. Hwang, "Attacks on Okamoto and Tanaka's One–Way ID–Based Key Distribution System," Information Processing Letters, v. 43, n. 2, Aug 1992, pp. 83–86.
749. T. Hwang and T.R.N. Rao, "Secret Error–Correcting Codes (SECC)," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 540–563.
750. C. I'Anson and C. Mitchell, "Security Defects in CCITT Recommendation X.509—the Directory Authentication Framework," Computer Communications Review, v. 20, n. 2, Apr 1990, pp. 30–34.
751. IBM, "Common Cryptographic Architecture: Cryptographic Application Programming Interface Reference," SC40–1675–1, IBM Corp., Nov 1990.
752. IBM, "Common Cryptographic Architecture: Cryptographic Application Programming Interface Reference—Public Key Algorithm," IBM Corp., Mar 1993.
753. R. Impagliazzo and M. Yung, "Direct Minimum–Knowledge Computations," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 40–51.
754. I. Ingemarsson, "A New Algorithm for the Solution of the Knapsack Problem," Lecture Notes in Computer Science 149; Cryptography: Proceedings of the Workshop on Cryptography, Springer–Verlag, 1983, pp. 309–315.
755. I. Ingemarsson, "Delay Estimation for Truly Random Binary Sequences or How to Measure the Length of Rip van Winkle's Sleep," Communications and Cryptography: Two Sides of One Tapestry, R.E. Blahut et al., eds., Kluwer Adademic Publishers, 1994, pp. 179–186.
756. I. Ingemarsson and G.J. Simmons, "A Protocol to Set Up Shared Secret Schemes without the Assistance of a Mutually Trusted Party," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 266–282.
757. I. Ingemarsson, D.T. Tang, and C.K. Wong, "A Conference Key Distribution System," IEEE Transactions on Information Theory, v. IT–28, n. 5, Sep 1982, pp. 714–720.
758. ISO DIS 8730, "Banking—Requirements for Message Authentication (Wholesale)," Association for Payment Clearing Services, London, Jul 1987.
759. ISO DIS 8731–1, "Banking—Approved Algorithms for Message Authentication—Part 1: DEA," Association for Payment Clearing Services, London, 1987.
760. ISO DIS 8731–2, "Banking—Approved Algorithms for Message Authentication—Part 2: Message Authenticator Algorithm," Association for Payment Clearing Services, London, 1987.
761. ISO DIS 8732, "Banking—Key Management (Wholesale)," Association for Payment Clearing Services, London, Dec 1987.
762. ISO/IEC 9796, "Information Technology—Security Techniques—Digital Signature Scheme Giving Message Recovery," International Organization for Standardization, Jul 1991.
763. ISO/IEC 9797, "Data Cryptographic Techniques—Data Integrity Mechanism Using a Cryptographic Check Function Employing a Block Cipher Algorithm," International Organization for Standardization, 1989.
764. ISO DIS 10118 DRAFT, "Information Technology—Security Techniques—Hash Functions," International Organization for Standardization, 1989.
765. ISO DIS 10118 DRAFT, "Information Technology—Security Techniques—Hash Functions," International Organization for Standardization, April 1991.
766. ISO N98, "Hash Functions Using a Pseudo Random Algorithm," working document, ISO–IEC/JTC1/SC27/WG2, International Organization for Standardization, 1992.
767. ISO N179, "AR Fingerprint Function," working document, ISO–IEC/JTC1/SC27/ WG2, International Organization for Standardization, 1992.
768. ISO/IEC 10118, "Information Technology—Security Techniques—Hash Functions—Part 1: General and Part 2: Hash–Functions Using an n–Bit Block Cipher Algorithm," International Organization for Standardization, 1993.
769. K. Ito, S. Kondo, and Y. Mitsuoka, "SXAL8/MBAL Algorithm," Technical Report, ISEC93–68, IEICE Japan, 1993. (In Japanese.)
770. K.R. Iversen, "The Application of Cryptographic Zero–Knowledge Techniques in Computerized Secret Ballot Election Schemes," Ph.D. dissertation, IDT–report 1991:3, Norwegian Institute of Technology, Feb 1991.
771. K.R. Iversen, "A Cryptographic Scheme for Computerized General Elections," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 405–419.
772. K. Iwamura, T. Matsumoto, and H. Imai, "An Implementation Method for RSA Cryptosystem with Parallel Processing," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J75–A, n. 8, Aug 1992, pp. 1301–1311.
773. W.J. Jaburek, "A Generalization of ElGamal's Public Key Cryptosystem," Advances in Cryptology—EUROCRYPT '89 Proceedings, 1990, Springer–Verlag, pp. 23–28.
774. N.S. James, R. Lidi, and H. Niederreiter, "Breaking the Cade Cipher," Advances in Cryptology—CRYPTO '86 Proceedings, 1987, Springer–Verlag, pp. 60–63.
775. C.J.A. Jansen, "On the Key Storage Requirements for Secure Terminals," Computers and Security, v. 5, n. 2, Jun 1986, pp. 145–149.
776. C.J.A. Jansen, "Investigations on Nonlinear Streamcipher Systems: Construction and Evaluation Methods," Ph.D. dissertation, Technical University of Delft, 1989.
777. C.J.A. Jansen and D.E. Boekee, "Modes of Blockcipher Algorithms and their Protection against Active Eavesdropping," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 281–286.
778. S.M. Jennings, "A Special Class of Binary Sequences," Ph.D. dissertation, University of London, 1980.
779. S.M. Jennings, "Multiplexed Sequences: Some Properties of the Minimum Polynomial," Lecture Notes in Computer Science 149; Cryptography: Proceedings of the Workshop on Cryptography, Springer–Verlag, 1983, pp. 189–206.
780. S.M. Jennings, "Autocorrelation Function of the Multiplexed Sequence," IEE Proceedings, v. 131, n. 2, Apr 1984, pp. 169–172.
781. T. Jin, "Care and Feeding of Your Three–Headed Dog," Document Number IAG–90–011, Hewlett–Packard, May 1990.
782. T. Jin, "Living with Your Three–Headed Dog," Document Number IAG–90–012, Hewlett–Packard, May 1990.
783. A. Jiwa, J. Seberry, and Y. Zheng, "Beacon Based Authentication," Computer Security—ESORICS 94, Springer–Verlag, 1994, pp. 125–141.
784. D.B. Johnson, G.M. Dolan, M.J. Kelly, A.V. Le, and S.M. Matyas, "Common Cryptographic Architecture Cryptographic Application Programming Interface," IBM Systems Journal, v. 30, n. 2, 1991, pp. 130–150.
785. D.B. Johnson, S.M. Matyas, A.V. Le, and J.D. Wilkins, "Design of the Commercial Data Masking Facility Data Privacy Algorithm," 1st ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 93–96.
786. J.P. Jordan, "A Variant of a Public–Key Cryptosystem Based on Goppa Codes," Sigact News, v. 15, n. 1, 1983, pp. 61–66.
787. A. Joux and L. Granboulan, "A Practical Attack Against Knapsack Based Hash Functions," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
788. A. Joux and J. Stern, "Cryptanalysis of Another Knapsack Cryptosystem," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 470–476.
789. R.R. Jueneman, "Analysis of Certain Aspects of Output–Feedback Mode," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 99–127.
790. R.R. Jueneman, "Electronic Document Authentication," IEEE Network Magazine, v. 1, n. 2, Apr 1978, pp. 17–23.
791. R.R. Jueneman, "A High Speed Manipulation Detection Code," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 327–346.
792. R.R. Jueneman, S.M. Matyas, and C.H. Meyer, "Message Authentication with Manipulation Detection Codes," Proceedings of the 1983 IEEE Computer Society Symposium on Research in Security and Privacy, 1983, pp. 733–54.
793. R.R. Jueneman, S.M. Matyas, and C.H. Meyer, "Message Authentication," IEEE Communications Magazine, v. 23, n. 9, Sep 1985, pp. 29–40.
794. D. Kahn, The Codebreakers: The Story of Secret Writing, New York: Macmillan Publishing Co., 1967.
795. D. Kahn, Kahn on Codes, New York: Macmillan Publishing Co., 1983.
796. D. Kahn, Seizing the Enigma, Boston: Houghton Mifflin Co., 1991.
797. P. Kaijser, T. Parker, and D. Pinkas, "SESAME: The Solution to Security for Open Distributed Systems," Journal of Computer Communications, v. 17, n. 4, Jul 1994, pp. 501–518.
798. R. Kailar and V.D. Gilgor, "On Belief Evolution in Authentication Protocols," Proceedings of the Computer Security Foundations Workshop IV, IEEE Computer Society Press, 1991, pp. 102–116.
799. B.S. Kaliski, "A Pseudo Random Bit Generator Based on Elliptic Logarithms," Master's thesis, Massachusetts Institute of Technology, 1987.
800. B.S. Kaliski, letter to NIST regarding DSS, 4 Nov 1991.
801. B.S. Kaliski, "The MD2 Message Digest Algorithm," RFC 1319, Apr 1992.
802. B.S. Kaliski, "Privacy Enhancement for Internet Electronic Mail: Part IV: Key Certificates and Related Services," RFC 1424, Feb 1993.
803. B.S. Kaliski, "An Overview of the PKCS Standards," RSA Laboratories, Nov 1993.
804. B.S. Kaliski, "A Survey of Encryption Standards, IEEE Micro, v. 13, n. 6, Dec 1993, pp. 74–81.
805. B.S. Kaliski, personal communication, 1993.
806. B.S. Kaliski, "On the Security and Performance of Several Triple–DES Modes," RSA Laboratories, draft manuscript, Jan 1994.
807. B.S. Kaliski, R.L. Rivest, and A.T. Sherman, "Is the Data Encryption Standard a Group?", Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 81–95.
808. B.S. Kaliski, R.L. Rivest, and A.T. Sherman, "Is the Data Encryption Standard a Pure Cipher? (Results of More Cycling Experiments in DES)," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 212–226.
809. B.S. Kaliski, R.L. Rivest, and A.T. Sherman, "Is the Data Encryption Standard a Group? (Results of Cycling Experiments on DES)," Journal of Cryptology, v. 1, n. 1, 1988, pp. 3–36.
810. B.S. Kaliski and M.J.B. Robshaw, "Fast Block Cipher Proposal," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 33–40.
811. B.S. Kaliski and M.J.B. Robshaw, "Linear Cryptanalysis Using Multiple Approximations," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 26–39.
812. B.S. Kaliski and M.J.B. Robshaw, "Linear Cryptanalysis Using Multiple Approximations and FEAL," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
813. R.G. Kammer, statement before the U.S. government Subcommittee on Telecommunications and Finance, Committee on Energy and Commerce, 29 Apr 1993.
814. T. Kaneko, K. Koyama, and R. Terada, "Dynamic Swapping Schemes and Differential Cryptanalysis, Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 292–301.
815. T. Kaneko, K. Koyama, and R. Terada, "Dynamic Swapping Schemes and Differential Cryptanalysis," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E77–A, n. 8, Aug 1994, pp. 1328–1336.
816. T. Kaneko and H. Miyano, "A Study on the Strength Evaluation of Randomized DES–Like Cryptosystems against Chosen Plaintext Attacks," Proceedings of the 1993 Symposium on Cryptography and Information Security (SCIS 93), Shuzenji, Japan, 28–30 Jan 1993, pp. 15C.1–10.
817. J. Kari, "A Cryptosystem Based on Propositional Logic," Machines, Languages, and Complexity: 5th International Meeting of Young Computer Scientists, Selected Contributions, Springer–Verlag, 1989, pp. 210–219.
818. E.D. Karnin, J.W. Greene, and M.E. Hellman, "On Sharing Secret Systems," IEEE Transactions on Information Theory, v. IT–29, 1983, pp. 35–41.
819. F.W. Kasiski, Die Geheimschriften und die Dechiffrir–kunst, E.S. Miller und Sohn, 1863. (In German.)
820. A. Kehne, J. Schonwalder, and H. Langendorfer, "A Nonce–Based Protocol for Multiple Authentications," Operating Systems Review, v. 26, n. 4, Oct 1992, pp. 84–89.
821. J. Kelsey, personal communication, 1994.
822. R. Kemmerer, "Analyzing Encryption Protocols Using Formal Verification Techniques," IEEE Journal on Selected Areas in Communications, v. 7, n. 4, May 1989, pp. 448–457.
823. R. Kemmerer, C.A. Meadows, and J. Millen, "Three Systems for Cryptographic Protocol Analysis," Journal of Cryptology, v. 7, n. 2, 1994, pp. 79–130.
824. S.T. Kent, "Encryption–Based Protection Protocols for Interactive User–Computer Communications," MIT/LCS/TR–162, MIT Laboratory for Computer Science, May 1976.
825. S.T. Kent, "Privacy Enhancement for Internet Electronic Mail: Part II: Certificate–Based Key Management," RFC 1422, Feb 1993.
826. S.T. Kent, "Understanding the Internet Certification System," Proceedings of INET '93, The Internet Society, 1993, pp. BAB1–BAB10.
827. S.T. Kent and J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part II: Certificate–Based Key Management," RFC 1114, Aug 1989.
828. V. Kessler and G. Wedel, "AUTOLOG—An Advanced Logic of Authentication," Proceedings of the Computer Security Foundations Workshop VII, IEEE Computer Society Press, 1994, pp. 90–99.
829. E.L. Key, "An Analysis of the Structure and Complexity of Nonlinear Binary Sequence Generators," IEEE Transactions on Information Theory, v. IT–22, n. 6, Nov 1976, pp. 732–736.
830. T. Kiesler and L. Harn, "RSA Blocking and Multisignature Schemes with No Bit Expansion," Electronics Letters, v. 26, n. 18, 30 Aug 1990, pp. 1490–1491.
831. J. Kilian, Uses of Randomness in Algorithms and Protocols, MIT Press, 1990.
832. J. Kilian, "Achieving Zero–Knowledge Robustly," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 313–325.
833. J. Kilian and T. Leighton, "Failsafe Key Escrow," MIT/LCS/TR–636, MIT Laboratory for Computer Science, Aug 1994.
834. K. Kim, "Construction of DES–Like S–Boxes Based on Boolean Functions Satisfying the SAC," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 59–72.
835. K. Kim, S. Lee, and S. Park, "Necessary Conditions to Strengthen DES S–Boxes Against Linear Cryptanalysis," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp. 15D.1–9.
836. K. Kim, S. Lee, and S. Park, "How to Strengthen DES against Differential Attack," unpublished manuscript, 1994.
837. K. Kim, S. Lee, S. Park, and D. Lee, "DES Can Be Immune to Differential Cryptanalysis," Workshop on Selected Areas in Cryptography—Workshop Record, Kingston, Ontario, 5–6 May 1994, pp. 70–81.
838. K. Kim, S. Park, and S. Lee, "How to Strengthen DES against Two Robust Attacks," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, 173–182.
839. K. Kim, S. Park, and S. Lee, "Reconstruction of s2DES S–Boxes and their Immunity to Differential Cryptanalysis," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 282–291.
840. S. Kim and B.S. Um, "A Multipurpose Membership Proof System Based on Discrete Logarithm," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 177–183.
841. P. Kinnucan, "Data Encryption Gurus: Tuchman and Meyer," Cryptologia, v. 2, n. 4, Oct 1978.
842. A. Klapper, "The Vulnerability of Geometric Sequences Based on Fields of Odd Characteristic," Journal of Cryptology, v. 7, n. 1, 1994, pp. 33–52.
843. A. Klapper, "Feedback with Carry Shift Registers over Finite Fields," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
844. A. Klapper and M. Goresky, "2–adic Shift Registers," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 174–178.
845. A. Klapper and M. Goresky, "2–adic Shift Registers," Technical Report #239–93, Department of Computer Science, University of Kentucky, 19 Apr 1994.
846. A. Klapper and M. Goresky, "Large Period Nearly de Bruijn FCSR Sequences," Advances in Cryptology—EUROCRYPT '95 Proceedings, Springer–Verlag, 1995, pp. 263–273.
847. D.V. Klein, "'Foiling the Cracker': A Survey of, and Implications to, Password Security," Proceedings of the USENIX UNIX Security Workshop, Aug 1990, pp. 5–14.
848. D.V. Klein, personal communication, 1994.
849. C.S. Kline and G.J. Popek, "Public Key vs. Conventional Key Cryptosystems," Proceedings of AFIPS National Computer Conference, pp. 831–837.
850. H.–J. Knobloch, "A Smart Card Implementation of the Fiat–Shamir Identification Scheme," Advances in Cryptology—EUROCRPYT '88 Proceedings, Springer–Verlag, 1988, pp. 87–95.
851. T. Knoph, J. Fröbl, W. Beller, and T. Giesler, "A Hardware Implementation of a Modified DES Algorithm," Microprocessing and Microprogramming, v. 30, 1990, pp. 59–66.
852. L.R. Knudsen, "Cryptanalysis of LOKI," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 22–35.
853. L.R. Knudsen, "Cryptanalysis of LOKI," Cryptography and Coding III, M.J. Ganley, ed., Oxford: Clarendon Press, 1993, pp. 223–236.
854. L.R. Knudsen, "Cryptanalysis of LOKI91," Advances in Cryptology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 196–208.
855. L.R. Knudsen, "Iterative Characteristics of DES and s2DES," Advances in Cryptology—CRYPTO '92, Springer–Verlag, 1993, pp. 497–511.
856. L.R. Knudsen, "An Analysis of Kim, Park, and Lee's DES–Like S–Boxes," unpublished manuscript, 1993.
857. L.R. Knudsen, "Practically Secure Feistel Ciphers," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 211–221.
858. L.R. Knudsen, "Block Ciphers—Analysis, Design, Applications," Ph.D. dissertation, Aarhus University, Nov 1994.
859. L.R. Knudsen, personal communication, 1994.
860. L.R. Knudsen, "Applications of Higher Order Differentials and Partial Differentials," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
861. L.R. Knudsen and X. Lai, "New Attacks on All Double Block Length Hash Functions of Hash Rate 1, Including the Parallel–DM," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
862. L.R. Knudsen, "A Weakness in SAFER K–64," Advances in Cryptology–CRYPTO '95 Proceedings, Springer–Verlag, 1995, to appear.
863. D. Knuth, The Art of Computer Programming: Volume 2, Seminumerical Algorithms, 2nd edition, Addison–Wesley, 1981.
864. D. Knuth, "Deciphering a Linear Congruential Encryption," IEEE Transactions on Information Theory, v. IT–31, n. 1, Jan 1985, pp. 49–52.
865. K. Kobayashi and L. Aoki, "On Linear Cryptanalysis of MBAL," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. A4.2.1–9.
866. K. Kobayashi, K. Tamura, and Y. Nemoto, "Two–dimensional Modified Rabin Cryptosystem," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J72–D, n. 5, May 1989, pp. 850–851. (In Japanese.)
867. N. Koblitz, "Elliptic Curve Cryptosystems," Mathematics of Computation, v. 48, n. 177, 1987, pp. 203–209.
868. N. Koblitz, "A Family of Jacobians Suitable for Discrete Log Cryptosystems," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 94–99.
869. N. Koblitz, "Constructing Elliptic Curve Cryptosystems in Characteristic 2," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 156–167.
870. N. Koblitz, "Hyperelliptic Cryptosystems," Journal of Cryptology, v. 1, n. 3, 1989, pp. 129–150.
871. N. Koblitz, "CM–Curves with Good Cryptographic Properties," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 279–287.
872. Ç.K. Koç, "High–Speed RSA Implementation," Version 2.0, RSA Laboratories, Nov 1994.
873. M.J. Kochanski, "Remarks on Lu and Lee's Proposals," Cryptologia, v. 4, n. 4, 1980, pp. 204–207.
874. M.J. Kochanski, "Developing an RSA Chip," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 350–357.
875. J.T. Kohl, "The Use of Encryption in Kerberos for Network Authentication," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 35–43.
876. J.T. Kohl, "The Evolution of the Kerberos Authentication Service," EurOpen Conference Proceedings, May 1991, pp. 295–313.
877. J.T. Kohl and B.C. Neuman, "The Kerberos Network Authentication Service," RFC 1510, Sep 1993.
878. J.T. Kohl, B.C. Neuman, and T. Ts'o, "The Evolution of the Kerberos Authentication System," Distributed Open Systems, IEEE Computer Society Press, 1994, pp. 78–94.
879. Kohnfelder, "Toward a Practical Public Key Cryptosystem," Bachelor's thesis, MIT Department of Electrical Engineering, May 1978.
880. A.G. Konheim, Cryptography: A Primer, New York: John Wiley & Sons, 1981.
881. A.G. Konheim, M.H. Mack, R.K. McNeill, B. Tuckerman, and G. Waldbaum, "The IPS Cryptographic Programs," IBM Systems Journal, v. 19, n. 2, 1980, pp. 253–283.
882. V.I. Korzhik and A.I. Turkin, "Cryptanalysis of McEliece's Public–Key Cryptosystem," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 68–70.
883. S.C. Kothari, "Generalized Linear Threshold Scheme," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 231–241.
884. J. Kowalchuk, B.P. Schanning, and S. Powers, "Communication Privacy: Integration of Public and Secret Key Cryptography," Proceedings of the National Telecommunication Conference, IEEE Press, 1980, pp. 49.1.1–49.1.5.
885. K. Koyama, "A Master Key for the RSA Public–Key Cryptosystem," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J65–D, n. 2, Feb 1982, pp. 163–170.
886. K. Koyama, "A Cryptosystem Using the Master Key for Multi–Address Communications," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J65–D, n. 9, Sep 1982, pp. 1151–1158.
887. K. Koyama, "Demonstrating Membership of a Group Using the Shizuya–Koyama–Itoh (SKI) Protocol," Proceedings of the 1989 Symposium on Cryptography and Information Security (SCIS 89), Gotenba, Japan, 1989.
888. K. Koyama, "Direct Demonstration of the Power to Break Public–Key Cryptosystems," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 14–21.
889. K. Koyama, "Security and Unique Decipherability of Two–dimensional Public Key Cryptosystems," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E73, n. 7, Jul 1990, pp. 1057–1067.
890. K. Koyama, U.M. Maurer, T. Okamoto, and S.A. Vanstone, "New Public–Key Schemes Based on Elliptic Curves over the Ring Zn," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 252–266.
891. K. Koyama and K. Ohta, "Identity–based Conference Key Distribution System," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 175–184.
892. K. Koyama and T. Okamoto, "Elliptic Curve Cryptosystems and Their Applications," IEICE Transactions on Information and Systems, v. E75–D, n. 1, Jan 1992, pp. 50–57.
893. K. Koyama and R. Terada, "How to Strengthen DES–Like Cryptosystems against Differential Cryptanalysis," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E76–A, n. 1, Jan 1993, pp. 63–69.
894. K. Koyama and R. Terada, "Probabilistic Swapping Schemes to Strengthen DES against Differential Cryptanalysis," Proceedings of the 1993 Symposium on Cryptography and Information Security (SCIS 93), Shuzenji, Japan, 28–30 Jan 1993, pp. 15D.1–12.
895. K. Koyama and Y. Tsuruoka, "Speeding up Elliptic Cryptosystems Using a Singled Binary Window Method," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 345–357.
896. E. Kranakis, Primality and Cryptography, Wiler–Teubner Series in Computer Science, 1986.
897. D. Kravitz, "Digital Signature Algorithm," U.S. Patent #5,231,668, 27 Jul 1993.
898. D. Kravitz and I. Reed, "Extension of RSA Cryptostructure: A Galois Approach," Electronics Letters, v. 18, n. 6, 18 Mar 1982, pp. 255–256.
899. H. Krawczyk, "How to Predict Congruential Generators," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 138–153.
900. H. Krawczyk, "How to Predict Congruential Generators," Journal of Algorithms, v. 13, n. 4, Dec 1992, pp. 527–545.
901. H. Krawczyk, "The Shrinking Generator: Some Practical Considerations," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 45–46.
902. G.J. Kühn, "Algorithms for Self–Synchronizing Ciphers," Proceedings of COMSIG 88, 1988.
903. G.J. Kühn, F. Bruwer, and W. Smit, "'n Vinnige Veeldoelige Enkripsievlokkie," Proceedings of Infosec 90, 1990. (In Afrikaans.)
904. S. Kullback, Statistical Methods in Cryptanalysis, U.S. Government Printing Office, 1935. Reprinted by Aegean Park Press, 1976.
905. P.V. Kumar, R.A. Scholtz, and L.R. Welch, "Generalized Bent Functions and their Properties," Journal of Combinational Theory, Series A, v. 40, n. 1, Sep 1985, pp. 90–107.
906. M. Kurosaki, T. Matsumoto, and H. Imai, "Simple Methods for Multipurpose Certification," Proceedings of the 1989 Symposium on Cryptography and Information Security (SCIS 89), Gotenba, Japan, 1989.
907. M. Kurosaki, T. Matsumoto, and H. Imai, "Proving that You Belong to at Least One of the Specified Groups," Proceedings of the 1990 Symposium on Cryptography and Information Security (SCIS 90), Hihondaira, Japan, 1990.
908. K. Kurosawa, "Key Changeable ID–Based Cryptosystem," Electronics Letters, v. 25, n. 9, 27 Apr 1989, pp. 577–578.
909. K. Kurosawa, T. Ito, and M. Takeuchi, "Public Key Cryptosystem Using a Reciprocal Number with the Same Intractability as Factoring a Large Number," Cryptologia, v. 12, n. 4, Oct 1988, pp. 225–233.
910. K. Kurosawa, C. Park, and K. Sakano, "Group Signer/Verifier Separation Scheme," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, 134–143.
911. G.C. Kurtz, D. Shanks, and H.C. Williams, "Fast Primality Tests for Numbers Less than 50*109," Mathematics of Computation, v. 46, n. 174, Apr 1986, pp. 691–701.
912. K. Kusuda and T. Matsumoto, "Optimization of the Time–Memory Trade–Off Cryptanalysis and Its Application to Block Ciphers," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. A3.2.1–11. (In Japanese.)
913. H. Kuwakado and K. Koyama, "Security of RSA–Type Cryptosystems Over Elliptic Curves against Hastad Attack," Electronics Letters, v. 30, n. 22, 27 Oct 1994, pp. 1843–1844.
914. H. Kuwakado and K. Koyama, "A New RSA–Type Cryptosystem over Singular Elliptic Curves," IMA Conference on Applications of Finite Fields, Oxford University Press, to appear.
915. H. Kuwakado and K. Koyama, "A New RSA–Type Scheme Based on Singular Cubic Curves," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, pp. 144–151.
916. M. Kwan, "An Eight Bit Weakness in the LOKI Cryptosystem," technical report, Australian Defense Force Academy, Apr 1991.
917. M. Kwan and J. Pieprzyk, "A General Purpose Technique for Locating Key Scheduling Weakness in DES–Like Cryptosystems," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 237–246.
918. J.B. Lacy, D.P. Mitchell, and W.M. Schell, "CryptoLib: Cryptography in Software," UNIX Security Symposium IV Proceedings, USENIX Association, 1993, pp. 1–17.
919. J.C. Lagarias, "Knapsack Public Key Cryptosystems and Diophantine Approximations," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 3–23.
920. J.C. Lagarias, "Performance Analysis of Shamir's Attack on the Basic Merkle–Hellman Knapsack Cryptosystem," Lecture Notes in Computer Science 172; Proceedings of the 11th International Colloquium on Automata, Languages, and Programming (ICALP), Springer–Verlag, 1984, pp. 312–323.
921. J.C. Lagarias and A.M. Odlyzko, "Solving Low–Density Subset Sum Problems," Proceedings of the 24th IEEE Symposium on Foundations of Computer Science, 1983, pp. 1–10.
922. J.C. Lagarias and A.M. Odlyzko, "Solving Low–Density Subset Sum Problems," Journal of the ACM, v. 32, n. 1, Jan 1985, pp. 229–246.
923. J.C. Lagarias and J. Reeds, "Unique Extrapolation of Polynomial Recurrences," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 342–362.
924. X. Lai, Detailed Description and a Software Implementation of the IPES Cipher, unpublished manuscript, 8 Nov 1991.
925. X. Lai, On the Design and Security of Block Ciphers, ETH Series in Information Processing, v. 1, Konstanz: Hartung–Gorre Verlag, 1992.
926. X. Lai, personal communication, 1993.
927. X. Lai, "Higher Order Derivatives and Differential Cryptanalysis," Communications and Cryptography: Two Sides of One Tapestry, R.E. Blahut et al., eds., Kluwer Adademic Publishers, 1994, pp. 227–233.
928. X. Lai and L. Knudsen, "Attacks on Double Block Length Hash Functions," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 157–165.
929. X. Lai and J. Massey, "A Proposal for a New Block Encryption Standard," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 389–404.
930. X. Lai and J. Massey, "Hash Functions Based on Block Ciphers," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1992, pp. 55–70.
931. X. Lai, J. Massey, and S. Murphy, "Markov Ciphers and Differential Cryptanalysis," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 17–38.
932. X. Lai, R.A. Rueppel, and J. Woollven, "A Fast Cryptographic Checksum Algorithm Based on Stream Ciphers," Advances in Cryptology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 339–348.
933. C.S. Laih, J.Y. Lee, C.H. Chen, and L. Harn, "A New Scheme for ID–based Cryptosystems and Signatures," Journal of the Chinese Institute of Engineers, v. 15, n. 2, Sep 1992, pp. 605–610.
934. B.A. LaMacchia and A.M. Odlyzko, "Computation of Discrete Logarithms in Prime Fields," Designs, Codes, and Cryptography, v. 1, 1991, pp. 46–62.
935. L. Lamport, "Password Identification with Insecure Communications," Communications of the ACM, v. 24, n. 11, Nov 1981, pp. 770–772.
936. S. Landau, "Zero–Knowledge and the Department of Defense," Notices of the American Mathematical Society, v. 35, n. 1, Jan 1988, pp. 5–12.
937. S. Landau, S. Kent, C. Brooks, S. Charney, D. Denning, W. Diffie, A. Lauck, D. Mikker, P. Neumann, and D. Sobel, "Codes, Keys, and Conflicts: Issues in U.S. Crypto Policy," Report of a Special Panel of the ACM U.S. Public Policy Committee (USACM), Association for Computing Machinery, Jun 1994.
938. S.K. Langford and M.E. Hellman, "Cryptanalysis of DES," presented at 1994 RSA Data Security conference, Redwood Shores, CA, 12–14 Jan 1994.
939. D. Lapidot and A. Shamir, "Publicly Verifiable Non–Interactive Zero–Knowledge Proofs," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 353–365.
940. A.V. Le, S.M. Matyas, D.B. Johnson, and J.D. Wilkins, "A Public–Key Extension to the Common Cryptographic Architecture," IBM Systems Journal, v. 32, n. 3, 1993, pp. 461–485.
941. P. L'Ecuyer, "Efficient and Portable Combined Random Number Generators," Communications of the ACM, v. 31, n. 6, Jun 1988, pp. 742–749, 774.
942. P. L'Ecuyer, "Random Numbers for Simulation," Communications of the ACM, v. 33, n. 10, Oct 1990, pp. 85–97.
943. P.J. Lee and E.F. Brickell, "An Observation on the Security of McEliece's Public–Key Cryptosystem," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 275–280.
944. S. Lee, S. Sung, and K. Kim, "An Efficient Method to Find the Linear Expressions for Linear Cryptanalysis," Proceedings of the 1995 Korea–Japan Workshop on Information Security and Cryptography, Inuyama, Japan, 24–26 Jan 1995, pp. 183– 190.
945. D.J. Lehmann, "On Primality Tests," SIAM Journal on Computing, v. 11, n. 2, May 1982, pp. 374–375.
946. T. Leighton, "Failsafe Key Escrow Systems," Technical Memo 483, MIT Laboratory for Computer Science, Aug 1994.
947. A. Lempel and M. Cohn, "Maximal Families of Bent Sequences," IEEE Transactions on Information Theory, v. IT–28, n. 6, Nov 1982, pp. 865–868.
948. A.K. Lenstra, "Factoring Multivariate Polynomials Over Finite Fields," Journal of Computer System Science, v. 30, n. 2, Apr 1985, pp. 235–248.
949. A.K. Lenstra, personal communication, 1995.
950. A.K. Lenstra and S. Haber, letter to NIST Regarding DSS, 26 Nov 1991.
951. A.K. Lenstra, H.W. Lenstra Jr., and L. Lovácz, "Factoring Polynomials with Rational Coefficients," Mathematische Annalen, v. 261, n. 4, 1982, pp. 515–534.
952. A.K. Lenstra, H.W. Lenstra, Jr., M.S. Manasse, and J.M. Pollard, "The Number Field Sieve," Proceedings of the 22nd ACM Symposium on the Theory of Computing, 1990, pp. 574–572.
953. A.K. Lenstra and H.W. Lenstra, Jr., eds., Lecture Notes in Mathematics 1554: The Development of the Number Field Sieve, Springer–Verlag, 1993.
954. A.K. Lenstra, H.W. Lenstra, Jr., M.S. Manasse, and J.M. Pollard, "The Factorization of the Ninth Fermat Number," Mathematics of Computation, v. 61, n. 203, 1993, pp. 319–349.
955. A.K. Lenstra and M.S. Manasse, "Factoring by Electronic Mail," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 355–371.
956. A.K. Lenstra and M.S. Manasse, "Factoring with Two Large Primes," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 72–82.
957. H.W. Lenstra Jr. "Elliptic Curves and Number–Theoretic Algorithms," Report 86–19, Mathematisch Instituut, Universiteit van Amsterdam, 1986.
958. H.W. Lenstra Jr. "On the Chor–Rivest Knapsack Cryptosystem," Journal of Cryptology, v. 3, n. 3, 1991, pp. 149–155.
959. W.J. LeVeque, Fundamentals of Number Theory, Addison–Wesley, 1977.
960. L.A. Levin, "One–Way Functions and Pseudo–Random Generators," Proceedings of the 17th ACM Symposium on Theory of Computing, 1985, pp. 363–365.
961. Lexar Corporation, "An Evaluation of the DES," Sep 1976.
962. D.–X. Li, "Cryptanalysis of Public–Key Distribution Systems Based on Dickson Polynomials," Electronics Letters, v. 27, n. 3, 1991, pp. 228–229.
963. F.–X. Li, "How to Break Okamoto's Cryptosystems by Continued Fraction Algorithm," ASIACRYPT '91 Abstracts, 1991, pp. 285–289.
964. Y.X. Li and X.M. Wang, "A Joint Authentication and Encryption Scheme Based on Algebraic Coding Theory," Applied Algebra, Algebraic Algorithms and Error Correcting Codes 9, Springer–Verlag, 1991, pp. 241–245.
965. R. Lidl, G.L. Mullen, and G. Turwald, Pitman Monographs and Surveys in Pure and Applied Mathematics 65: Dickson Polynomials, London: Longman Scientific and Technical, 1993.
966. R. Lidl and W.B. Müller, "Permutation Polynomials in RSA–Cryptosystems," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 293–301.
967. R. Lidl and W.B. Müller, "Generalizations of the Fibonacci Pseudoprimes Test," Discrete Mathematics, v. 92, 1991, pp. 211–220.
968. R. Lidl and W.B. Müller, "Primality Testing with Lucas Functions," Advances in Cryptology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 539–542.
969. R. Lidl, W.B. Müller, and A. Oswald, "Some Remarks on Strong Fibonacci Pseudoprimes," Applicable Algebra in Engineering, Communication and Computing, v. 1, n. 1, 1990, pp. 59–65.
970. R. Lidl and H. Niederreiter, "Finite Fields," Encyclopedia of Mathematics and its Applications, v. 20, Addison–Wesley, 1983.
971. R. Lidl and H. Niederreiter, Introduction to Finite Fields and Their Applications, London: Cambridge University Press, 1986.
972. K. Lieberherr, "Uniform Complexity and Digital Signatures," Theoretical Computer Science, v. 16, n. 1, Oct 1981, pp. 99–110.
973. C.H. Lim and P.J. Lee, "A Practical Electronic Cash System for Smart Cards," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 34–47.
974. C.H. Lim and P.J. Lee, "Security of Interactive DSA Batch Verification," Electronics Letters, v. 30, n. 19, 15 Sep 1994, pp. 1592–1593.
975. H.–Y. Lin and L. Harn, "A Generalized Secret Sharing Scheme with Cheater Detection," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 149–158.
976. M.–C. Lin, T.–C. Chang, and H.–L. Fu, "Information Rate of McEliece's Public–key Cryptosystem," Electronics Letters, v. 26, n. 1, 4 Jan 1990, pp. 16–18.
977. J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part I—Message Encipherment and Authentication Procedures," RFC 989, Feb 1987.
978. J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part I—Message Encipherment and Authentication Procedures," RFC 1040, Jan 1988.
979. J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part I—Message Encipherment and Authentication Procedures," RFC 1113, Aug 1989.
980. J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part III—Algorithms, Modes, and Identifiers," RFC 1115, Aug 1989.
981. J. Linn, "Privacy Enhancement for Internet Electronic Mail: Part I—Message Encipherment and Authentication Procedures," RFC 1421, Feb 1993.
982. S. Lloyd, "Counting Binary Functions with Certain Cryptographic Properties," Journal of Cryptology, v. 5, n. 2, 1992, pp. 107–131.
983. T.M.A. Lomas, "Collision–Freedom, Considered Harmful, or How to Boot a Computer," Proceedings of the 1995 Korea–Japan Workshop on Information Security and Cryptography, Inuyama, Japan, 24–26 Jan 1995, pp. 35–42.
984. T.M.A. Lomas and M. Roe, "Forging a Clipper Message," Communications of the ACM, v. 37, n. 12, 1994, p. 12.
985. D.L. Long, "The Security of Bits in the Discrete Logarithm," Ph.D. dissertation, Princeton University, Jan 1984.
986. D.L. Long and A. Wigderson, "How Discrete Is the Discrete Log," Proceedings of the 15th Annual ACM Syposium on the Theory of Computing, Apr 1983.
987. D. Longley and S. Rigby, "An Automatic Search for Security Flaws in Key Management Schemes," Computers and Security, v. 11, n. 1, Jan 1992. pp. 75–89.
988. S.H. Low, N.F. Maxemchuk, and S. Paul, "Anonymous Credit Cards," Proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 108–117.
989. J.H. Loxton, D.S.P. Khoo, G.J. Bird, and J. Seberry, "A Cubic RSA Code Equivalent to Factorization," Journal of Cryptology, v. 5, n. 2, 1992, pp. 139–150.
990. S.C. Lu and L.N. Lee, "A Simple and Effective Public–Key Cryptosystem," COMSAT Technical Review, 1979, pp. 15–24.
991. M. Luby, S. Micali, and C. Rackoff, "How to Simultaneously Exchange a Secret Bit by Flipping a Symmetrically–Biased Coin," Proceedings of the 24nd Annual Symposium on the Foundations of Computer Science, 1983, pp. 11–22.
992. M. Luby and C. Rackoff, "How to Construct Pseudo–Random Permutations from Pseudorandom Functions," SIAM Journal on Computing, Apr 1988, pp. 373–386.
993. F. Luccio and S. Mazzone, "A Cryptosystem for Multiple Communications," Information Processing Letters, v. 10, 1980, pp. 180–183.
994. V. Luchangco and K. Koyama, "An Attack on an ID–Based Key Sharing System, Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 262–271.
995. D.J.C. MacKay, "A Free Energy Minimization Framework for Inferring the State of a Shift Register Given the Noisy Output Sequence," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
996. M.D. MacLaren and G. Marsaglia, "Uniform Random Number Generators," Journal of the ACM v. 12, n. 1, Jan 1965, pp. 83–89.
997. D. MacMillan, "Single Chip Encrypts Data at 14Mb/s," Electronics, v. 54, n. 12, 16 June 1981, pp. 161–165.
998. R. Madhavan and L.E. Peppard, "A Multiprocessor GaAs RSA Cryptosystem," Proceedings CCVLSI–89: Canadian Conference on Very Large Scale Integration, Vancouver, BC, Canada, 22–24 Oct 1989, pp. 115–122.
999. W.E. Madryga, "A High Performance Encryption Algorithm," Computer Security: A Global Challenge, Elsevier Science Publishers, 1984, pp. 557–570.
1000. M. Mambo, A. Nishikawa, S. Tsujii, and E. Okamoto, "Efficient Secure Broadcast Communication System," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 23–33.
1001. M. Mambo, K. Usuda, and E. Okamoto, "Proxy Signatures," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. B1.1.1–17.
1002. W. Mao and C. Boyd, "Towards Formal Analysis of Security Protocols," Proceedings of the Computer Security Foundations Workshop VI, IEEE Computer Society Press, 1993, pp. 147–158.
1003. G. Marsaglia and T.A. Bray, "On–Line Random Number Generators and their Use in Combinations," Communications of the ACM, v. 11, n. 11, Nov 1968, p. 757–759.
1004. K.M. Martin, "Untrustworthy Participants in Perfect Secret Sharing Schemes," Cryptography and Coding III, M.J. Ganley, ed., Oxford: Clarendon Press, 1993, pp. 255–264.
1005. J.L. Massey, "Shift–Register Synthesis and BCH Decoding," IEEE Transactions on Information Theory, v. IT–15, n. 1, Jan 1969, pp. 122–127.
1006. J.L. Massey, "Cryptography and System Theory," Proceedings of the 24th Allerton Conference on Communication, Control, and Computers, 1–3 Oct 1986, pp. 1–8.
1007. J.L. Massey, "An Introduction to Contemporary Cryptology," Proceedings of the IEEE, v. 76, n. 5., May 1988, pp. 533–549.
1008. J.L. Massey, "Contemporary Cryptology: An Introduction," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 1–39.
1009. J.L. Massey, "SAFER K–64: A Byte–Oriented Block–Ciphering Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 1–17.
1010. J.L. Massey, "SAFER K–64: One Year Later," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
1011. J.L. Massey and I. Ingemarsson, "The Rip Van Winkle Cipher—A Simple and Provably Computationally Secure Cipher with a Finite Key," IEEE International Symposium on Information Theory, Brighton, UK, May 1985.
1012. J.L. Massey and X. Lai, "Device for Converting a Digital Block and the Use Thereof," International Patent PCT/ CH91/00117, 28 Nov 1991.
1013. J.L. Massey and X. Lai, "Device for the Conversion of a Digital Block and Use of Same," U.S. Patent #5,214,703, 25 May 1993.
1014. J.L. Massey and R.A. Rueppel, "Linear Ciphers and Random Sequence Generators with Multiple Clocks," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 74–87.
1015. M. Matsui, "Linear Cryptanalysis Method for DES Cipher," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 386–397.
1016. M. Matsui, "Linear Cryptanalysis of DES Cipher (I)," Proceedings of the 1993 Symposium on Cryptography and Information Security (SCIS 93), Shuzenji, Japan, 28–30 Jan 1993, pp. 3C.1–14. (In Japanese.)
1017. M. Matsui, "Linear Cryptanalysis Method for DES Cipher (III)," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp. 4A.1–11. (In Japanese.)
1018. M. Matsui, "On Correlation Between the Order of the S–Boxes and the Strength of DES," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
1019. M. Matsui, "The First Experimental Cryptanalysis of the Data Encryption Standard," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 1–11.
1020. M. Matsui and A. Yamagishi, "A New Method for Known Plaintext Attack of FEAL Cipher," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 81–91.
1021. T. Matsumoto and H. Imai, "A Class of Asymmetric Crypto–Systems Based on Polynomials Over Finite Rings," IEEE International Symposium on Information Theory, 1983, pp. 131–132.
1022. T. Matsumoto and H. Imai, "On the Key Production System: A Practical Solution to the Key Distribution Problem," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 185–193.
1023. T. Matsumoto and H. Imai, "On the Security of Some Key Sharing Schemes (Part 2)," IEICE Japan, Technical Report, ISEC90–28, 1990.
1024. S.M. Matyas, "Digital Signatures—An Overview," Computer Networks, v. 3, n. 2, Apr 1979, pp. 87–94.
1025. S.M. Matyas, "Key Handling with Control Vectors," IBM Systems Journal, v. 30, n. 2, 1991, pp. 151–174.
1026. S.M. Matyas, A.V. Le, and D.G. Abraham, "A Key Management Scheme Based on Control Vectors," IBM Systems Journal, v. 30, n. 2, 1991, pp. 175–191.
1027. S.M. Matyas and C.H. Meyer, "Generation, Distribution, and Installation of Cryptographic Keys," IBM Systems Journal, v. 17, n. 2, 1978, pp. 126–137.
1028. S.M. Matyas, C.H. Meyer, and J. Oseas, "Generating Strong One–Way Functions with Cryptographic Algorithm," IBM Technical Disclosure Bulletin, v. 27, n. 10A, Mar 1985, pp. 5658–5659.
1029. U.M. Maurer, "Provable Security in Cryptography," Ph.D. dissertation, ETH No. 9260, Swiss Federal Institute of Technology, Zürich, 1990.
1030. U.M. Maurer, "A Provable–Secure Strongly–Randomized Cipher," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 361–373.
1031. U.M. Maurer, "A Universal Statistical Test for Random Bit Generators," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 409–420.
1032. U.M. Maurer, "A Universal Statistical Test for Random Bit Generators," Journal of Cryptology, v. 5, n. 2, 1992, pp. 89–106.
1033. U.M. Maurer and J.L. Massey, "Cascade Ciphers: The Importance of Being First," Journal of Cryptology, v. 6, n. 1, 1993, pp. 55–61.
1034. U.M. Maurer and J.L. Massey, "Perfect Local Randomness in Pseudo–Random Sequences," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 110–112.
1035. U.M. Maurer and Y. Yacobi, "Non–interactive Public Key Cryptography," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 498–507.
1036. G. Mayhew, "A Low Cost, High Speed Encryption System and Method," Proceedings of the 1994 IEEE Computer Society Symposium on Research in Security and Privacy, 1994, pp. 147–154.
1037. G. Mayhew, R. Frazee, and M. Bianco, "The Kinetic Protection Device," Proceedings of the 15th National Computer Security Conference, NIST, 1994, pp. 147–154.
1038. K.S. McCurley, "A Key Distribution System Equivalent to Factoring," Journal of Cryptology, v. 1, n. 2, 1988, pp. 95–106.
1039. K.S. McCurley, "The Discrete Logarithm Problem," Cryptography and Computational Number Theory (Proceedings of the Symposium on Applied Mathematics), American Mathematics Society, 1990, pp. 49–74.
1040. K.S. McCurley, open letter from the Sandia National Laboratories on the DSA of the NIST, 7 Nov 1991.
1041. R.J. McEliece, "A Public–Key Cryptosystem Based on Algebraic Coding Theory," Deep Space Network Progress Report 42–44, Jet Propulsion Laboratory, California Institute of Technology, 1978, pp. 114–116.
1042. R.J. McEliece, Finite Fields for Computer Scientists and Engineers, Boston: Kluwer Academic Publishers, 1987.
1043. P. McMahon, "SESAME V2 Public Key and Authorization Extensions to Kerberos," Proceedings of the Internet Society 1995 Symposium on Network and Distributed Systems Security, IEEE Computer Society Press, 1995, pp. 114–131.
1044. C.A. Meadows, "A System for the Specification and Analysis of Key Management Protocols," Proceedings of the 1991 IEEE Computer Society Symposium on Research in Security and Privacy, 1991, pp. 182–195.
1045. C.A. Meadows, "Applying Formal Methods to the Analysis of a Key Management Protocol," Journal of Computer Security, v. 1, n. 1, 1992, pp. 5–35.
1046. C.A. Meadows, "A Model of Computation for the NRL Protocol Analyzer," Proceedings of the Computer Security Foundations Workshop VII, IEEE Computer Society Press, 1994, pp. 84–89.
1047. C.A. Meadows, "Formal Verification of Cryptographic Protocols: A Survey," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 133–150.
1048. G. Medvinsky and B.C. Neuman, "NetCash: A Design for Practical Electronic Currency on the Internet," Proceedings of the 1st Annual ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 102–106.
1049. G. Medvinsky and B.C. Neuman, "Electronic Currency for the Internet," Electronic Markets, v. 3, n. 9/10, Oct 1993, pp. 23–24.
1050. W. Meier, "On the Security of the IDEA Block Cipher," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 371–385.
1051. W. Meier and O. Staffelbach, "Fast Correlation Attacks on Stream Ciphers," Journal of Cryptology, v. 1, n. 3, 1989, pp. 159–176.
1052. W. Meier and O. Staffelbach, "Analysis of Pseudo Random Sequences Generated by Cellular Automata," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 186–199.
1053. W. Meier and O. Staffelbach, "Correlation Properties of Combiners with Memory in Stream Ciphers," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 204–213.
1054. W. Meier and O. Staffelbach, "Correlation Properties of Combiners with Memory in Stream Ciphers," Journal of Cryptology, v. 5, n. 1, 1992, pp. 67–86.
1055. W. Meier and O. Staffelbach, "The Self–Shrinking Generator," Communications and Cryptography: Two Sides of One Tapestry, R.E. Blahut et al., eds., Kluwer Adademic Publishers, 1994, pp. 287–295.
1056. J. Meijers, "Algebraic–Coded Cryptosystems," Master's thesis, Technical University Eindhoven, 1990.
1057. J. Meijers and J. van Tilburg, "On the Rao–Nam Private–Key Cryptosystem Using Linear Codes," International Symposium on Information Theory, Budapest, Hungary, 1991.
1058. J. Meijers and J. van Tilburg, "An Improved ST–Attack on the Rao–Nam Private–Key Cryptosystem," International Conference on Finite Fields, Coding Theory, and Advances in Communications and Computing, Las Vegas, NV, 1991.
1059. A. Menezes, Elliptic Curve Public Key Cryptosystems, Kluwer Academic Publishers, 1993.
1060. A. Menezes, ed., Applications of Finite Fields, Kluwer Academic Publishers, 1993.
1061. A. Menezes and S.A. Vanstone, "Elliptic Curve Cryptosystems and Their Implementations," Journal of Cryptology, v. 6, n. 4, 1993, pp. 209–224.
1062. A. Menezes and S.A. Vanstone, "The Implementation of Elliptic Curve Cryptosystems," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 2–13.
1063. R. Menicocci, "Short Gollmann Cascade Generators May Be Insecure," Codes and Ciphers, Institute of Mathematics and its Applications, 1995, pp. 281–297.
1064. R.C. Merkle, "Secure Communication Over Insecure Channels," Communications of the ACM, v. 21, n. 4, 1978, pp. 294–299.
1065. R.C. Merkle, "Secrecy, Authentication, and Public Key Systems," Ph.D. dissertation, Stanford University, 1979.
1066. R.C. Merkle, "Method of Providing Digital Signatures," U.S. Patent #4,309,569, 5 Jan 1982.
1067. R.C. Merkle, "A Digital Signature Based on a Conventional Encryption Function," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 369–378.
1068. R.C. Merkle, "A Certified Digital Signature," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 218–238.
1069. R.C. Merkle, "One Way Hash Functions and DES," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 428–446.
1070. R.C. Merkle, "A Fast Software One–Way Hash Function," Journal of Cryptology, v. 3, n. 1, 1990, pp. 43–58.
1071. R.C. Merkle, "Fast Software Encryption Functions," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 476–501.
1072. R.C. Merkle, "Method and Apparatus for Data Encryption," U.S. Patent #5,003,597, 26 Mar 1991.
1073. R.C. Merkle, personal communication, 1993.
1074. R.C. Merkle and M. Hellman, "Hiding Information and Signatures in Trapdoor Knapsacks," IEEE Transactions on Information Theory, v. 24, n. 5, Sep 1978, pp. 525–530.
1075. R.C. Merkle and M. Hellman, "On the Security of Multiple Encryption," Communications of the ACM, v. 24, n. 7, 1981, pp. 465–467.
1076. M. Merritt, "Cryptographic Protocols," Ph.D. dissertation, Georgia Institute of Technology, GIT–ICS–83/6, Feb 1983.
1077. M. Merritt, "Towards a Theory of Cryptographic Systems: A Critique of Crypto–Complexity," Distributed Computing and Cryptography, J. Feigenbaum and M. Merritt, eds., American Mathematical Society, 1991, pp. 203–212.
1078. C.H. Meyer, "Ciphertext/Plaintext and Ciphertext/Key Dependencies vs. Number of Rounds for Data Encryption Standard," AFIPS Conference Proceedings, 47, 1978, pp. 1119–1126.
1079. C.H. Meyer, "Cryptography—A State of the Art Review," Proceedings of Compeuro '89, VLSI and Computer Peripherals, 3rd Annual European Computer Conference, IEEE Press, 1989, pp. 150–154.
1080. C.H. Meyer and S.M. Matyas, Cryptography: A New Dimension in Computer Data Security, New York: John Wiley & Sons, 1982.
1081. C.H. Meyer and M. Schilling, "Secure Program Load with Manipulation Detection Code," Proceedings of Securicom '88, 1988, pp. 111–130.
1082. C.H. Meyer and W.L. Tuchman, "Pseudo–Random Codes Can Be Cracked," Electronic Design, v. 23, Nov 1972.
1083. C.H. Meyer and W.L. Tuchman, "Design Considerations for Cryptography," Proceedings of the NCC, v. 42, Montvale, NJ: AFIPS Press, Nov 1979, pp. 594–597.
1084. S. Micali, "Fair Public–Key Cryptosystems," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 113–138.
1085. S. Micali, "Fair Cryptosystems," MIT/ LCS/TR–579.b, MIT Laboratory for Computer Science, Nov 1993.
1086. S. Micali, "Fair Cryptosystems and Methods for Use," U.S. Patent #5,276,737, 4 Jan 1994.
1087. S. Micali, "Fair Cryptosystems and Methods for Use," U.S. Patent #5,315,658, 24 May 1994.
1088. S. Micali and A. Shamir, "An Improvement on the Fiat–Shamir Identification and Signature Scheme," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 244–247.
1089. M.J. Mihajlevic, "A Correlation Attack on the Binary Sequence Generators with Time–Varying Output Function," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 67–79.
1090. M.J. Mihajlevic and J.D. Golic, "A Fast Iterative Algorithm for a Shift Register Internal State Reconstruction Given the Noisy Output Sequence," Advances in Cryptology—AUSCRYPT '90 Proceedings, Springer–Verlag, 1990, pp. 165–175.
1091. M.J. Mihajlevic and J.D. Golic, "Convergence of a Bayesian Iterative Error–Correction Procedure to a Noisy Shift Register Sequence," Advances in Cryptology —EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 124–137.
1092. J.K. Millen, S.C. Clark, and S.B. Freedman, "The Interrogator: Protocol Security Analysis," IEEE Transactions on Software Engineering, v. SE–13, n. 2, Feb 1987, pp. 274–288.
1093. G.L. Miller, "Riemann's Hypothesis and Tests for Primality," Journal of Computer Systems Science, v. 13, n. 3, Dec 1976, pp. 300–317.
1094. S.P. Miller, B.C. Neuman, J.I. Schiller, and J.H. Saltzer, "Section E.2.1: Kerberos Authentication and Authorization System," MIT Project Athena, Dec 1987.
1095. V.S. Miller, "Use of Elliptic Curves in Cryptography," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 417–426.
1096. M. Minsky, Computation: Finite and Infinite Machines, Englewood Cliffs, NJ: Prentice–Hall, 1967.
1097. C.J. Mitchell, "Authenticating Multi–Cast Internet Electronic Mail Messages Using a Bidirectional MAC Is Insecure," draft manuscript, 1990.
1098. C.J. Mitchell, "Enumerating Boolean Functions of Cryptographic Significance," Journal of Cryptology, v. 2, n. 3, 1990, pp. 155–170.
1099. C.J. Mitchell, F. Piper, and P. Wild, "Digital Signatures," Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1991, pp. 325–378.
1100. C.J. Mitchell, M. Walker, and D. Rush, "CCITT/ISO Standards for Secure Message Handling," IEEE Journal on Selected Areas in Communications, v. 7, n. 4, May 1989, pp. 517–524.
1101. S. Miyaguchi, "Fast Encryption Algorithm for the RSA Cryptographic System," Proceedings of Compcon 82, IEEE Press, pp. 672–678.
1102. S. Miyaguchi, "The FEAL–8 Cryptosystem and Call for Attack," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 624–627.
1103. S. Miyaguchi, "Expansion of the FEAL Cipher," NTT Review, v. 2, n. 6, Nov 1990.
1104. S. Miyaguchi, "The FEAL Cipher Family," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 627–638.
1105. S. Miyaguchi, K. Ohta, and M. Iwata, "128–bit Hash Function (N–Hash)," Proceedings of SECURICOM '90, 1990, pp. 127–137.
1106. S. Miyaguchi, K. Ohta, and M. Iwata, "128–bit Hash Function (N–Hash)," NTT Review, v. 2, n. 6, Nov 1990, pp. 128–132.
1107. S. Miyaguchi, K. Ohta, and M. Iwata, "Confirmation that Some Hash Functions Are Not Collision Free," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 326–343.
1108. S. Miyaguchi, A. Shiraishi, and A. Shimizu, "Fast Data Encipherment Algorithm FEAL–8," Review of the Electrical Communication Laboratories, v. 36, n. 4, 1988.
1109. H. Miyano, "Differential Cryptanalysis on CALC and Its Evaluation," Proceedings of the 1992 Symposium on Cryptography and Information Security (SCIS 92), Tateshina, Japan, 2–4 Apr 1992, pp. 7B.1–8.
1110. R. Molva, G. Tsudik, E. van Herreweghen, and S. Zatti, "KryptoKnight Authentication and Key Distribution System," Proceedings of European Symposium on Research in Computer Security, Toulouse, France, Nov 1992.
1111. P.L. Montgomery, "Modular Multiplication without Trial Division," Mathematics of Computation, v. 44, n. 170, 1985, pp. 519–521.
1112. P.L. Montgomery, "Speeding the Pollard and Elliptic Curve Methods of Factorization," Mathematics of Computation, v. 48, n. 177, Jan 1987, pp. 243–264.
1113. P.L. Montgomery and R. Silverman, "An FFT Extension to the p–1 Factoring Algorithm," Mathematics of Computation, v. 54, n. 190, 1990, pp. 839–854.
1114. J.H. Moore, "Protocol Failures in Cryptosystems," Proceedings of the IEEE, v. 76, n. 5, May 1988.
1115. J.H. Moore, "Protocol Failures in Cryptosystems," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 541–558.
1116. J.H. Moore and G.J. Simmons, "Cycle Structure of the DES with Weak and Semi–Weak Keys," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 3–32.
1117. T. Moriyasu, M. Morii, and M. Kasahara, "Nonlinear Pseudorandom Number Generator with Dynamic Structure and Its Properties," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Biwako, Japan, 27–29 Jan 1994, pp. 8A.1–11.
1118. R. Morris, "The Data Encryption Standard—Retrospective and Prospects," IEEE Communications Magazine, v. 16, n. 6, Nov 1978, pp. 11–14.
1119. R. Morris, remarks at the 1993 Cambridge Protocols Workshop, 1993.
1120. R. Morris, N.J.A. Sloane, and A.D. Wyner, "Assessment of the NBS Proposed Data Encryption Standard," Cryptologia, v. 1, n. 3, Jul 1977, pp. 281–291.
1121. R. Morris and K. Thompson, "Password Security: A Case History," Communications of the ACM, v. 22, n. 11, Nov 1979, pp. 594–597.
1122. S.B. Morris, "Escrow Encryption," lecture at MIT Laboratory for Computer Science, 2 Jun 1994.
1123. M.N. Morrison and J. Brillhart, "A Method of Factoring and the Factorization of F7," Mathematics of Computation, v. 29, n. 129, Jan 1975, pp. 183–205.
1124. L.E. Moser, "A Logic of Knowledge and Belief for Reasoning About Computer Security," Proceedings of the Computer Security Foundations Workshop II, IEEE Computer Society Press, 1989, pp. 57–63.
1125. Motorola Government Electronics Division, Advanced Techniques in Network Security, Scottsdale, AZ, 1977.
1126. W.B. Müller, "Polynomial Functions in Modern Cryptology," Contributions to General Algebra 3: Proceedings of the Vienna Conference, Vienna: Verlag Hölder–Pichler–Tempsky, 1985, pp. 7–32.
1127. W.B. Müller and W. Nöbauer, "Some Remarks on Public–Key Cryptography," Studia Scientiarum Mathematicarum Hungarica, v. 16, 1981, pp. 71–76.
1128. W.B. Müller and W. Nöbauer, "Cryptanalysis of the Dickson Scheme," Advances in Cryptology—EUROCRYPT '85 Proceedings, Springer–Verlag, 1986, pp. 50–61.
1129. C. Muller–Scholer, "A Microprocessor–Based Cryptoprocessor," IEEE Micro, Oct 1983, pp. 5–15.
1130. R.C. Mullin, E. Nemeth, and N. Weidenhofer, "Will Public Key Cryptosystems Live Up to Their Expectations?—HEP Implementation of the Discrete Log Codebreaker," ICPP 85, pp. 193–196.
1131. Y. Murakami and S. Kasahara, "An ID–Based Key Distribution Scheme," IEICE Japan, Technical Report, ISEC90–26, 1990.
1132. S. Murphy, "The Cryptanalysis of FEAL–4 with 20 Chosen Plaintexts," Journal of Cryptology, v. 2, n. 3, 1990, pp. 145–154.
1133. E.D. Myers, "STU–III—Multilevel Secure Computer Interface," Proceedings of the Tenth Annual Computer Security Applications Conference, IEEE Computer Society Press, 1994, pp. 170–179.
1134. D. Naccache, "Can O.S.S. be Repaired? Proposal for a New Practical Signature Scheme," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 233–239.
1135. D. Naccache, D. M'Ra•hi, D. Raphaeli, and S. Vaudenay, "Can D.S.A. be Improved? Complexity Trade–Offs with the Digital Signature Standard," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
1136. Y. Nakao, T. Kaneko, K. Koyama, and R. Terada, "A Study on the Security of RDES–1 Cryptosystem against Linear Cryptanalysis," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, pp. 163–172.
1137. M. Naor, "Bit Commitment Using Pseudo–Randomness," Advances in Cryptology—CRYPTO '89 Proceedings., Springer–Verlag, 1990, pp. 128–136.
1138. M. Naor and M. Yung, "Universal One–Way Hash Functions and Their Cryptographic Application," Proceedings of the 21st Annual ACM Symposium on the Theory of Computing, 1989, pp. 33–43.
1139. National Bureau of Standards, "Report of the Workshop on Estimation of Significant Advances in Computer Technology," NBSIR76–1189, National Bureau of Standards, U.S. Department of Commerce, 21–22 Sep 1976, Dec 1977.
1140. National Bureau of Standards, NBS FIPS PUB 46, "Data Encryption Standard," National Bureau of Standards, U.S. Department of Commerce, Jan 1977.
1141. National Bureau of Standards, NBS FIPS PUB 46–1, "Data Encryption Standard," U.S. Department of Commerce, Jan 1988.
1142. National Bureau of Standards, NBS FIPS PUB 74, "Guidelines for Implementing and Using the NBS Data Encryption Standard," U.S. Department of Commerce, Apr 1981.
1143. National Bureau of Standards, NBS FIPS PUB 81, "DES Modes of Operation," U.S. Department of Commerce, Dec 1980.
1144. National Bureau of Standards, NBS FIPS PUB 112, "Password Usage," U.S. Department of Commerce, May 1985.
1145. National Bureau of Standards, NBS FIPS PUB 113, "Computer Data Authentication," U.S. Department of Commerce, May 1985.
1146. National Computer Security Center, "Trusted Network Interpretation of the Trusted Computer System Evaluation Criteria," NCSC–TG–005 Version 1, Jul 1987.
1147. National Computer Security Center, "Trusted Database Management System Interpretation of the Trusted Computer System Evaluation Criteria," NCSC–TG–021 Version 1, Apr 1991.
1148. National Computer Security Center, "A Guide to Understanding Data Rememberance in Automated Information Systems," NCSC–TG–025 Version 2, Sep 1991.
1149. National Institute of Standards and Technology, NIST FIPS PUB XX, "Digital Signature Standard," U.S. Department of Commerce, DRAFT, 19 Aug 1991.
1150. National Institute of Standards and Technology, NIST FIPS PUB 46–2, "Data Encryption Standard," U.S. Department of Commerce, Dec 93.
1151. National Institute of Standards and Technology, NIST FIPS PUB 171, "Key Management Using X9.17," U.S. Department of Commerce, Apr 92.
1152. National Institute of Standards and Technology, NIST FIPS PUB 180, "Secure Hash Standard," U.S. Department of Commerce, May 93.
1153. National Institute of Standards and Technology, NIST FIPS PUB 185, "Escrowed Encryption Standard," U.S. Department of Commerce, Feb 94.
1154. National Institute of Standards and Technology, NIST FIPS PUB 186, "Digital Signature Standard," U.S. Department of Commerce, May 1994.
1155. National Institute of Standards and Technology, "Clipper Chip Technology," 30 Apr 1993.
1156. National Institute of Standards and Technology, "Capstone Chip Technology," 30 Apr 1993.
1157. J. Nechvatal, "Public Key Cryptography," NIST Special Publication 800–2, National Institute of Standards and Technology, U.S. Department of Commerce, Apr 1991.
1158. J. Nechvatal, "Public Key Cryptography," Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 177–288.
1159. R.M. Needham and M.D. Schroeder, "Using Encryption for Authentication in Large Networks of Computers," Communications of the ACM, v. 21, n. 12, Dec 1978, pp. 993–999.
1160. R.M. Needham and M.D. Schroeder, "Authentication Revisited," Operating Systems Review, v. 21, n. 1, 1987, p. 7.
1161. D.M. Nessett, "A Critique of the Burrows, Abadi, and Needham Logic," Operating System Review, v. 20, n. 2, Apr 1990, pp. 35–38.
1162. B.C. Neuman and S. Stubblebine, "A Note on the Use of Timestamps as Nonces," Operating Systems Review, v. 27, n. 2, Apr 1993, pp. 10–14.
1163. B.C. Neuman and T. Ts'o, "Kerberos: An Authentication Service for Computer Networks," IEEE Communications Magazine, v. 32, n. 9, Sep 1994, pp. 33–38.
1164. L. Neuwirth, "Statement of Lee Neuwirth of Cylink on HR145," submitted to congressional committees considering HR145, Feb 1987.
1165. D.B. Newman, Jr. and R.L. Pickholtz, "Cryptography in the Private Sector," IEEE Communications Magazine, v. 24, n. 8, Aug 1986, pp. 7–10.
1166. H. Niederreiter, "A Public–Key Cryptosystem Based on Shift Register Sequences," Advances in Cryptology—EUROCRYPT '85 Proceedings, Springer–Verlag, 1986, pp. 35–39.
1167. H. Niederreiter, "Knapsack–Type Cryptosystems and Algebraic Coding Theory," Problems of Control and Information Theory, v. 15, n. 2, 1986, pp. 159–166. 1168. H. Niederreiter, "The Linear Complexity Profile and the Jump Complexity of Keystream Sequences," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 174–188.
1169. V. Niemi, "A New Trapdoor in Knapsacks," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 405–411.
1170. V. Niemi and A. Renvall, "How to Prevent Buying of Voters in Computer Elections," Advances in Cryptology—ASIACRYPT '94 Proceedings, Springer–Verlag, 1995, pp. 164–170.
1171. I. Niven and H.A. Zuckerman, An Introduction to the Theory of Numbers, New York: John Wiley & Sons, 1972.
1172. R. Nöbauer, "Cryptanalysis of the Rédei Scheme," Contributions to General Algebra 3: Proceedings of the Vienna Conference, Verlag Hölder–Pichler–Tempsky, Vienna, 1985, pp. 255–264.
1173. R. Nöbauer, "Cryptanalysis of a Public–Key Cryptosystem Based on Dickson–Polynomials," Mathematica Slovaca, v. 38, n. 4, 1988, pp. 309–323.
1174. K. Noguchi, H. Ashiya, Y. Sano, and T. Kaneko, "A Study on Differential Attack of MBAL Cryptosystem," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp. 14B.1–7. (In Japanese.)
1175. H. Nurmi, A. Salomaa, and L. Santean, "Secret Ballot Elections in Computer Networks," Computers & Security, v. 10, 1991, pp. 553–560.
1176. K. Nyberg, "Construction of Bent Functions and Difference Sets," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 151–160.
1177. K. Nyberg, "Perfect Nonlinear S–Boxes," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 378–386.
1178. K. Nyberg, "On the Construction of Highly Nonlinear Permutations," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1991, pp. 92–98.
1179. K. Nyberg, "Differentially Uniform Mappings for Cryptography," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 55–64.
1180. K. Nyberg, "Provable Security against Differential Cryptanalysis," presented at the rump session of Eurocrypt '94, May 1994.
1181. K. Nyberg and L.R. Knudsen, "Provable Security against Differential Cryptanalysis," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 566–574.
1182. K. Nyberg and L.R. Knudsen, "Provable Security against Differential Cryptanalysis," Journal of Cryptology, v. 8, n. 1, 1995, pp. 27–37.
1183. K. Nyberg and R.A. Rueppel, "A New Signature Scheme Based on the DSA Giving Message Recovery," 1st ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 58–61.
1184. K. Nyberg and R.A. Rueppel, "Message Recovery for Signature Schemes Based on the Discrete Logarithm Problem," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
1185. L. O'Connor, "Enumerating Nondegenerate Permutations," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 368–377.
1186. L. O'Connor, "On the Distribution of Characteristics in Bijective Mappings," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 360–370.
1187. L. O'Connor, "On the Distribution of Characteristics in Composite Permutations," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 403–412.
1188. L. O'Connor and A. Klapper, "Algebraic Nonlinearity and Its Application to Cryptography," Journal of Cryptology, v. 7, n. 3, 1994, pp. 133–151.
1189. A. Odlyzko, "Discrete Logarithms in Finite Fields and Their Cryptographic Significance," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 224–314.
1190. A. Odlyzko, "Progress in Integer Factorization and Discrete Logarithms," unpublished manuscript, Feb 1995.
1191. Office of Technology Assessment, U.S. Congress, "Defending Secrets, Sharing Data: New Locks and Keys for Electronic Communication," OTA–CIT–310, Washington, D.C.: U.S. Government Printing Office, Oct 1987.
1192. B. O'Higgins, W. Diffie, L. Strawczynski, and R. de Hoog, "Encryption and ISDN—a Natural Fit," Proceedings of the 1987 International Switching Symposium, 1987, pp. 863–869.
1193. Y. Ohnishi, "A Study on Data Security," Master's thesis, Tohuku University, Japan, 1988. (In Japanese.)
1194. K. Ohta, "A Secure and Efficient Encrypted Broadcast Communication System Using a Public Master Key," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J70–D, n. 8, Aug 1987, pp. 1616–1624.
1195. K. Ohta, "An Electrical Voting Scheme Using a Single Administrator," IEICE Spring National Convention, A–294, 1988, v. 1, p. 296. (In Japanese.)
1196. K. Ohta, "Identity–based Authentication Schemes Using the RSA Cryptosystem," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J72D–II, n. 8, Aug 1989, pp. 612–620.
1197. K. Ohta and M. Matsui, "Differential Attack on Message Authentication Codes," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 200–223.
1198. K. Ohta and T. Okamoto, "Practical Extension of Fiat–Shamir Scheme," Electronics Letters, v. 24, n. 15, 1988, pp. 955–956.
1199. K. Ohta and T. Okamoto, "A Modification of the Fiat–Shamir Scheme," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 232–243.
1200. K. Ohta and T. Okamoto, "A Digital Multisignature Scheme Based on the Fiat–Shamir Scheme," Advances in Cryptology—ASIACRYPT '91 Proceedings, Springer–Verlag, 1993, pp. 139–148.
1201. K. Ohta, T. Okamoto and K. Koyama, "Membership Authentication for Hierarchy Multigroups Using the Extended Fiat–Shamir Scheme," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 446–457.
1202. E. Okamoto and K. Tanaka, "Key Distribution Based on Identification Information," IEEE Journal on Selected Areas in Communication, v. 7, n. 4, May 1989, pp. 481–485.
1203. T. Okamoto, "Fast Public–Key Cryptosystems Using Congruent Polynomial Equations," Electronics Letters, v. 22, n. 11, 1986, pp. 581–582.
1204. T. Okamoto, "Modification of a Public–Key Cryptosystem," Electronics Letters, v. 23, n. 16, 1987, pp. 814–815.
1205. T. Okamoto, "A Fast Signature Scheme Based on Congruential Polynomial Operations," IEEE Transactions on Information Theory, v. 36, n. 1, 1990, pp. 47–53.
1206. T. Okamoto, "Provably Secure and Practical Identification Schemes and Corresponding Signature Schemes," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 31–53.
1207. T. Okamoto, A. Fujioka, and E. Fujisaki, "An Efficient Digital Signature Scheme Based on Elliptic Curve over the Ring Zn," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, 1993, pp. 54–65.
1208. T. Okamoto, S. Miyaguchi, A. Shiraishi, and T. Kawoaka, "Signed Document Transmission System," U.S. Patent #4,625,076, 25 Nov 1986.
1209. T. Okamoto and K. Ohta, "Disposable Zero–Knowledge Authentication and Their Applications to Untraceable Electronic Cash," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 134–149.
1210. T. Okamoto and K. Ohta, "How to Utilize the Randomness of Zero–Knowledge Proofs," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 456–475.
1211. T. Okamoto and K. Ohta, "Universal Electronic Cash," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 324–337.
1212. T. Okamoto and K. Ohta, "Survey of Digital Signature Schemes," Proceedings of the Third Symposium on State and Progress of Research in Cryptography, Fondazone Ugo Bordoni, Rome, 1993, pp. 17–29.
1213. T. Okamoto and K. Ohta, "Designated Confirmer Signatures Using Trapdoor Functions," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp. 16B.1–11.
1214. T. Okamoto and K. Sakurai, "Efficient Algorithms for the Construction of Hyperelliptic Cryptosystems," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 267–278.
1215. T. Okamoto and A. Shiraishi, "A Fast Signature Scheme Based on Quadratic Inequalities," Proceedings of the 1985 Symposium on Security and Privacy, IEEE, Apr 1985, pp. 123–132.
1216. J.D. Olsen, R.A. Scholtz, and L.R. Welch, "Bent Function Sequences," IEEE Transactions on Information Theory, v. IT–28, n. 6, Nov 1982, pp. 858–864.
1217. H. Ong and C.P. Schnorr, "Signatures through Approximate Representations by Quadratic Forms," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984.
1218. H. Ong and C.P. Schnorr, "Fast Signature Generation with a Fiat Shamir–Like Scheme," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 432–440.
1219. H. Ong, C.P. Schnorr, and A. Shamir, "An Efficient Signature Scheme Based on Polynomial Equations," Proceedings of the 16th Annual Symposium on the Theory of Computing, 1984, pp. 208–216.
1220. H. Ong, C.P. Schnorr, and A. Shamir, "Efficient Signature Schemes Based on Polynomial Equations," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 37–46.
1221. Open Shop Information Services, OSIS Security Aspects, OSIS European Working Group, WG1, final report, Oct 1985.
1222. G.A. Orton, M.P. Roy, P.A. Scott, L.E. Peppard, and S.E. Tavares, "VLSI Implementation of Public–Key Encryption Algorithms," Advances in Cryptology— CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 277–301.
1223. H. Orup, E. Svendsen, and E. Andreasen, "VICTOR—An Efficient RSA Hardware Implementation," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 245–252.
1224. D. Otway and O. Rees, "Efficient and Timely Mutual Authentication," Operating Systems Review, v. 21, n. 1, 1987, pp. 8–10.
1225. G. Pagels–Fick, "Implementation Issues for Master Key Distribution and Protected Keyload Procedures," Computers and Security: A Global Challenge, Proceedings of IFIP/SEC '83, North Holland: Elsevier Science Publishers, 1984, pp. 381–390.
1226. C.M. Papadimitriou, Computational Complexity, Addison–Wesley, 1994.
1227. C.S. Park, "Improving Code Rate of McEliece's Public–key Cryptosystem," Electronics Letters, v. 25, n. 21, 12 Oct 1989, pp. 1466–1467.
1228. S. Park, Y. Kim, S. Lee, and K. Kim, "Attacks on Tanaka's Non–interactive Key Sharing Scheme," Proceedings of the 1995 Symposium on Cryptography and Information Security (SCIS 95), Inuyama, Japan, 24–27 Jan 1995, pp. B3.4.1–4.
1229. S.J. Park, K.H. Lee, and D.H. Won, "An Entrusted Undeniable Signature," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, pp. 120–126.
1230. S.J. Park, K.H. Lee, and D.H. Won, "A Practical Group Signature," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, pp. 127–133.
1231. S.K. Park and K.W. Miller, "Random Number Generators: Good Ones Are Hard to Find," Communications of the ACM, v. 31, n. 10, Oct 1988, pp. 1192–1201.
1232. J. Patarin, "How to Find and Avoid Collisions for the Knapsack Hash Function," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 305–317.
1233. W. Patterson, Mathematical Cryptology for Computer Scientists and Mathematicians, Totowa, N.J.: Rowman & Littlefield, 1987.
1234. W.H. Payne, "Public Key Cryptography Is Easy to Break," William H. Payne, unpublished manuscript, 16 Oct 90.
1235. T.P. Pederson, "Distributed Provers with Applications to Undeniable Signatures," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 221–242.
1236. S. Peleg and A. Rosenfield, "Breaking Substitution Ciphers Using a Relaxation Algorithm," Communications of the ACM, v. 22, n. 11, Nov 1979, pp. 598–605.
1237. R. Peralta, "Simultaneous Security of Bits in the Discrete Log," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 62–72.
1238. I. Peterson, "Monte Carlo Physics: A Cautionary Lesson," Science News, v. 142, n. 25, 19 Dec 1992, p. 422.
1239. B. Pfitzmann, "Fail–Stop Signatures: Principles and Applications," Proceedings of COMPUSEC '91, Eighth World Conference on Computer Security, Audit, and Control, Elsevier Science Publishers, 1991, pp. 125–134.
1240. B. Pfitzmann and M. Waidner, "Formal Aspects of Fail–Stop Signatures," Fakultät für Informatik, University Karlsruhe, Report 22/90, 1990.
1241. B. Pfitzmann and M. Waidner, "Fail–Stop Signatures and Their Application," Securicom '91, 1991, pp. 145–160.
1242. B. Pfitzmann and M. Waidner, "Unconditional Concealment with Cryptographic Ruggedness," VIS '91 Verlassliche Informationsysteme Proceedings, Darmstadt, Germany, 13–15 March 1991, pp. 3–2–320. (In German.)
1243. B. Pfitzmann and M. Waidner, "How to Break and Repair a 'Provably Secure' Untraceable Payment System," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 338–350.
1244. C.P. Pfleeger, Security in Computing, Englewood Cliffs, N.J.: Prentice–Hall, 1989.
1245. S.J.D. Phoenix and P.D. Townsend, "Quantum Cryptography and Secure Optical Communication," BT Technology Journal, v. 11, n. 2, Apr 1993, pp. 65–75.
1246. J. Pieprzyk, "On Public–Key Cryptosystems Built Using Polynomial Rings," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 73–80.
1247. J. Pieprzyk, "Error Propagation Property and Applications in Cryptography," IEE Proceedings–E, Computers and Digital Techniques, v. 136, n. 4, Jul 1989, pp. 262–270.
1248. D. Pinkas, T. Parker, and P. Kaijser, "SESAME: An Introduction," Issue 1.2, Bull, ICL, and SNI, Sep 1993.
1249. F. Piper, "Stream Ciphers," Elektrotechnic und Maschinenbau, v. 104, n. 12, 1987, pp. 564–568.
1250. V.S. Pless, "Encryption Schemes for Computer Confidentiality," IEEE Transactions on Computing, v. C–26, n. 11, Nov 1977, pp. 1133–1136.
1251. J.B. Plumstead, "Inferring a Sequence Generated by a Linear Congruence," Proceedings of the 23rd IEEE Symposium on the Foundations of Computer Science, 1982, pp. 153–159.
1252. R. Poet, "The Design of Special Purpose Hardware to Factor Large Integers," Computer Physics Communications, v. 37, 1985, pp. 337–341.
1253. S.C. Pohlig and M.E. Hellman, "An Improved Algorithm for Computing Logarithms in GF(p) and Its Cryptographic Significance," IEEE Transactions on Information Theory, v. 24, n. 1, Jan 1978, pp. 106–111.
1254. J.M. Pollard, "A Monte Carlo Method for Factorization," BIT, v. 15, 1975, pp. 331–334.
1255. J.M. Pollard and C.P. Schnorr, "An Efficient Solution of the Congruence x2 + ky2 = m (mod n)," IEEE Transactions on Information Theory, v. IT–33, n. 5, Sep 1987, pp. 702–709.
1256. C. Pomerance, "Recent Developments in Primality Testing," The Mathematical Intelligencer, v. 3, n. 3, 1981, pp. 97–105.
1257. C. Pomerance, "The Quadratic Sieve Factoring Algorithm," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, 169–182.
1258. C. Pomerance, "Fast, Rigorous Factorization and Discrete Logarithm Algorithms," Discrete Algorithms and Complexity, New York: Academic Press, 1987, pp. 119–143.
1259. C. Pomerance, J.W. Smith, and R. Tuler, "A Pipe–Line Architecture for Factoring Large Integers with the Quadratic Sieve Algorithm," SIAM Journal on Computing, v. 17, n. 2, Apr 1988, pp. 387–403.
1260. G.J. Popek and C.S. Kline, "Encryption and Secure Computer Networks," ACM Computing Surveys, v. 11, n. 4, Dec 1979, pp. 331–356.
1261. F. Pratt, Secret and Urgent, Blue Ribbon Books, 1942.
1262. B. Preneel, "Analysis and Design of Cryptographic Hash Functions," Ph.D. dissertation, Katholieke Universiteit Leuven, Jan 1993.
1263. B. Preneel, "Differential Cryptanalysis of Hash Functions Based on Block Ciphers," Proceedings of the 1st ACM Conference on Computer and Communications Security, 1993, pp. 183–188.
1264. B. Preneel, "Cryptographic Hash Functions," European Transactions on Telecommunications, v 5, n. 4, Jul/Aug 1994, pp. 431–448.
1265. B. Preneel, personal communication, 1995.
1266. B. Preneel, A. Bosselaers, R. Govaerts, and J. Vandewalle, "Collision–Free Hash Functions Based on Block Cipher Algorithms," Proceedings of the 1989 Carnahan Conference on Security Technology, 1989, pp. 203–210.
1267. B. Preneel, R. Govaerts, and J. Vandewalle, "An Attack on Two Hash Functions by Zheng–Matsumoto–Imai," Advances in Cryptology—ASIACRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 535–538.
1268. B. Preneel, R. Govaerts, and J. Vandewalle, "Hash Functions Based on Block Ciphers: A Synthetic Approach," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 368–378.
1269. B. Preneel, M. Nuttin, V. Rijmen, and J. Buelens, "Cryptanalysis of the CFB mode of the DES with a Reduced Number of Rounds," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 212–223.
1270. B. Preneel and V. Rijmen, "On Using Maximum Likelihood to Optimize Recent Cryptanalytic Techniques," presented at the rump session of EUROCRYPT '94, May 1994.
1271. B. Preneel, W. Van Leekwijck, L. Van Linden, R. Govaerts, and J. Vandewalle, "Propagation Characteristics of Boolean Functions," Advances in Cryptology— EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 161–173.
1272. W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, 1988.
1273. W. Price, "Key Management for Data Encipherment," Security: Proceedings of IFIP/SEC '83, North Holland: Elsevier Science Publishers, 1983.
1274. G.P. Purdy, "A High–Security Log–in Procedure," Communications of the ACM, v. 17, n. 8, Aug 1974, pp. 442–445.
1275. J.–J. Quisquater, "Announcing the Smart Card with RSA Capability," Proceedings of the Conference: IC Cards and Applications, Today and Tomorrow, Amsterdam, 1989.
1276. J.–J. Quisquater and C. Couvreur, "Fast Decipherment Algorithm for RSA Public–Key Cryptosystem," Electronic Letters, v. 18, 1982, pp. 155–168.
1277. J.–J. Quisquater and J.–P. Delescaille, "Other Cycling Tests for DES," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 255–256.
1278. J.–J. Quisquater and Y.G. Desmedt, "Chinese Lotto as an Exhaustive Code–Breaking Machine," Computer, v. 24, n. 11, Nov 1991, pp. 14–22.
1279. J.–J. Quisquater and M. Girault, "2n–bit Hash Functions Using n–bit Symmetric Block Cipher Algorithms, Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 102–109.
1280. J.–J. Quisquater and L.C. Guillou, "Des Procédés d'Authentification Basés sur une Publication de Problmes Complexes et Personnalisés dont les Solutions Maintenues Secrtes Constituent autant d'Accréditations," Proceedings of SECURICOM '89: 7th Worldwide Congress on Computer and Communications Security and Protection, Société d'Édition et d'Organisation d'Expositions Professionnelles, 1989, pp. 149–158. (In French.)
1281. J.–J., Myriam, Muriel, and Micha‘l Quisquater; L., Marie Annick, Ga•d, Anna, Gwenolé, and Soazig Guillou; and T. Berson, "How to Explain Zero–Knowledge Protocols to Your Children," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 628–631.
1282. M.O. Rabin, "Digital Signatures," Foundations of Secure Communication, New York: Academic Press, 1978, pp. 155–168.
1283. M.O. Rabin, "Digital Signatures and Public–Key Functions as Intractable as Factorization," MIT Laboratory for Computer Science, Technical Report, MIT/LCS/TR–212, Jan 1979.
1284. M.O. Rabin, "Probabilistic Algorithm for Testing Primality," Journal of Number Theory, v. 12, n. 1, Feb 1980, pp. 128– 138.
1285. M.O. Rabin, "Probabilistic Algorithms in Finite Fields," SIAM Journal on Computing, v. 9, n. 2, May 1980, pp. 273–280.
1286. M.O. Rabin, "How to Exchange Secrets by Oblivious Transfer," Technical Memo TR–81, Aiken Computer Laboratory, Harvard University, 1981.
1287. M.O. Rabin, "Fingerprinting by Random Polynomials," Technical Report TR–15–81, Center for Research in Computing Technology, Harvard University, 1981.
1288. T. Rabin and M. Ben–Or, "Verifiable Secret Sharing and Multiparty Protocols with Honest Majority," Proceedings of the 21st ACM Symposium on the Theory of Computing, 1989, pp. 73–85.
1289. RAND Corporation, A Million Random Digits with 100,000 Normal Deviates, Glencoe, IL: Free Press Publishers, 1955.
1290. T.R.N. Rao, "Cryposystems Using Algebraic Codes," International Conference on Computer Systems and Signal Processing, Bangalore, India, Dec 1984.
1291. T.R.N. Rao, "On Struit–Tilburg Cryptanalysis of Rao–Nam Scheme," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 458–460.
1292. T.R.N. Rao and K.H. Nam, "Private–Key Algebraic–Coded Cryptosystems," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 35–48.
1293. T.R.N. Rao and K.H. Nam, "Private–Key Algebraic–Code Encryptions," IEEE Transactions on Information Theory, v. 35, n. 4, Jul 1989, pp. 829–833.
1294. J.A. Reeds, "Cracking Random Number Generator," Cryptologia, v. 1, n. 1, Jan 1977, pp. 20–26.
1295. J.A. Reeds, "Cracking a Multiplicative Congruential Encryption Algorithm," in Information Linkage Between Applied Mathematics and Industry, P.C.C. Wang, ed., Academic Press, 1979, pp. 467–472.
1296. J.A. Reeds, "Solution of Challenge Cipher," Cryptologia, v. 3, n. 2, Apr 1979, pp. 83–95.
1297. J.A. Reeds and J.L. Manferdelli, "DES Has No Per Round Linear Factors," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 377–389.
1298. J.A. Reeds and N.J.A. Sloane, "Shift Register Synthesis (Modulo m)," SIAM Journal on Computing, v. 14, n. 3, Aug 1985, pp. 505–513.
1299. J.A. Reeds and P.J. Weinberger, "File Security and the UNIX Crypt Command," AT&T Technical Journal, v. 63, n. 8, Oct 1984, pp. 1673–1683.
1300. T. Renji, "On Finite Automaton One–Key Cryptosystems," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 135–148.
1301. T. Renji and C. Shihua, "A Finite Automaton Public Key Cryptosystems and Digital Signature," Chinese Journal of Computers, v. 8, 1985, pp. 401–409. (In Chinese.)
1302. T. Renji and C. Shihua, "Two Varieties of Finite Automaton Public Key Cryptosystems and Digital Signature," Journal of Computer Science and Tecnology, v. 1, 1986, pp. 9–18. (In Chinese.)
1303. T. Renji and C. Shihua, "An Implementation of Identity–based Cryptosystems and Signature Schemes by Finite Automaton Public Key Cryptosystems," Advances in Cryptology—CHINACRYPT '92, Bejing: Science Press, 1992, pp. 87–104. (In Chinese.)
1304. T. Renji and C. Shihua, "Note on Finite Automaton Public Key Cryptosystems," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 76–80.
1305. Research and Development in Advanced Communication Technologies in Europe, RIPE Integrity Primitives: Final Report of RACE Integrity Primitives Evaluation (R1040), RACE, June 1992.
1306. J.M. Reyneri and E.D. Karnin, "Coin Flipping by Telephone," IEEE Transactions on Information Theory, v. IT–30, n. 5, Sep 1984, pp. 775–776.
1307. P. Ribenboim, The Book of Prime Number Records, Springer–Verlag, 1988.
1308. P. Ribenboim, The Little Book of Big Primes, Springer–Verlag, 1991.
1309. M. Richter, "Ein Rauschgenerator zur Gewinnung won quasi–idealen Zufallszahlen für die stochastische Simulation," Ph.D. dissertation, Aachen University of Technology, 1992. (In German.)
1310. R.F. Rieden, J.B. Snyder, R.J. Widman, and W.J. Barnard, "A Two–Chip Implementation of the RSA Public Encryption Algorithm," Proceedings of GOMAC (Government Microcircuit Applications Conference), Nov 1982, pp. 24–27.
1311. H. Riesel, Prime Numbers and Computer Methods for Factorization, Boston: Birkhaüser, 1985.
1312. K. Rihaczek, "Data Interchange and Legal Security—Signature Surrogates," Computers & Security, v. 13, n. 4, Sep 1994, pp. 287–293.
1313. V. Rijmen and B. Preneel, "Improved Characteristics for Differential Cryptanalysis of Hash Functions Based on Block Ciphers," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
1314. R.L. Rivest, "A Description of a Single–Chip Implementation of the RSA Cipher," LAMBDA Magazine, v. 1, n. 3, Fall 1980, pp. 14–18.
1315. R.L. Rivest, "Statistical Analysis of the Hagelin Cryptograph," Cryptologia, v. 5, n. 1, Jan 1981, pp. 27–32.
1316. R.L. Rivest, "A Short Report on the RSA Chip," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, p. 327.
1317. R.L. Rivest, "RSA Chips (Past/Present/ Future)," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 159–168.
1318. R.L. Rivest, "The MD4 Message Digest Algorithm," RFC 1186, Oct 1990.
1319. R.L. Rivest, "The MD4 Message Digest Algorithm," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 303–311.
1320. R.L. Rivest, "The RC4 Encryption Algorithm," RSA Data Security, Inc., Mar 1992.
1321. R.L. Rivest, "The MD4 Message Digest Algorithm," RFC 1320, Apr 1992.
1322. R.L. Rivest, "The MD5 Message Digest Algorithm," RFC 1321, Apr 1992.
1323. R.L. Rivest, "Dr. Ron Rivest on the Difficulty of Factoring," Ciphertext: The RSA Newsletter, v. 1, n. 1, Fall 1993, pp. 6, 8.
1324. R.L. Rivest, "The RC5 Encryption Algorithm," Dr. Dobb's Journal, v. 20, n. 1, Jan 95, pp. 146–148.
1325. R.L. Rivest, "The RC5 Encryption Algorithm," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
1326. R.L. Rivest, M.E. Hellman, J.C. Anderson, and J.W. Lyons, "Responses to NIST's Proposal," Communications of the ACM, v. 35, n. 7, Jul 1992, pp. 41–54.
1327. R.L. Rivest and A. Shamir, "How to Expose an Eavesdropper," Communications of the ACM, v. 27, n. 4, Apr 1984, pp. 393–395.
1328. R.L. Rivest, A. Shamir, and L.M. Adleman, "A Method for Obtaining Digital Signatures and Public–Key Cryptosystems," Communications of the ACM, v. 21, n. 2, Feb 1978, pp. 120–126.
1329. R.L. Rivest, A. Shamir, and L.M. Adleman, "On Digital Signatures and Public Key Cryptosystems," MIT Laboratory for Computer Science, Technical Report, MIT/LCS/TR–212, Jan 1979.
1330. R.L. Rivest, A. Shamir, and L.M. Adleman, "Cryptographic Communications System and Method," U.S. Patent #4,405,829, 20 Sep 1983.
1331. M.J.B. Robshaw, "Implementations of the Search for Pseudo–Collisions in MD5," Technical Report TR–103, Version 2.0, RSA Laboratories, Nov 1993.
1332. M.J.B. Robshaw, "The Final Report of RACE 1040: A Technical Summary," Technical Report TR–9001, Version 1.0, RSA Laboratories, Jul 1993.
1333. M.J.B. Robshaw, "On Evaluating the Linear Complexity of a Sequence of Least Period 2n," Designs, Codes and Cryptography, v. 4, n. 3, 1994, pp. 263–269.
1334. M.J.B. Robshaw, "Block Ciphers," Technical Report TR–601, RSA Laboratories, Jul 1994.
1335. M.J.B. Robshaw, "MD2, MD4, MD5, SHA, and Other Hash Functions," Technical Report TR–101, Version 3.0, RSA Laboratories, Jul 1994.
1336. M.J.B. Robshaw, "On Pseudo–Collisions in MD5," Technical Report TR–102, Version 1.1, RSA Laboratories, Jul 1994.
1337. M.J.B. Robshaw, "Security of RC4," Technical Report TR–401, RSA Laboratories, Jul 1994.
1338. M.J.B. Robshaw, personal communication, 1995.
1339. M. Roe, "Reverse Engineering of an EES Device," K.U. Leuven Workshop on Cryptographic Algorithms, Springer–Verlag, 1995, to appear.
1340. P. Rogaway and D. Coppersmith, "A Software–Oriented Encryption Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 56–63.
1341. H.L. Rogers, "An Overview of the Candware Program," Proceedings of the 3rd Annual Symposium on Physical/Electronic Security, Armed Forces Communications and Electronics Association, paper 31, Aug 1987.
1342. J. Rompel, "One–Way Functions Are Necessary and Sufficient for Secure Signatures," Proceedings of the 22nd Annual ACM Symposium on the Theory of Computing, 1990, pp. 387–394.
1343. T. Rosati, "A High Speed Data Encryption Processor for Public Key Cryptography," Proceedings of the IEEE Custom Integrated Circuits Conference, 1989, pp. 12.3.1–12.3.5.
1344. O.S. Rothaus, "On 'Bent' Functions," Journal of Combinational Theory, Series A, v. 20, n. 3, 1976, pp. 300–305.
1345. RSA Laboratories, "PKCS #1: RSA Encryption Standard," version 1.5, Nov 1993.
1346. RSA Laboratories, "PKCS #3: Diffie–Hellman Key–Agreement Standard," version 1.4, Nov 1993.
1347. RSA Laboratories, "PKCS #5: Password–Based Encryption Standard," version 1.5, Nov 1993.
1348. RSA Laboratories, "PKCS #6: Extended–Certificate Syntax Standard," version 1.5, Nov 1993.
1349. RSA Laboratories, "PKCS #7: Cryptographic Message Syntax Standard," version 1.5, Nov 1993.
1350. RSA Laboratories, "PKCS #8: Private Key Information Syntax Standard," version 1.2, Nov 1993.
1351. RSA Laboratories, "PKCS #9: Selected Attribute Types," version 1.1, Nov 1993. 1352. RSA Laboratories, "PKCS #10: Certification Request Syntax Standard," version 1.0, Nov 1993.
1353. RSA Laboratories, "PKCS #11: Cryptographic Token Interface Standard," version 1.0, Apr 95.
1354. RSA Laboratories, "PKCS #12: Public Key User Information Syntax Standard," version 1.0, 1995.
1355. A.D. Rubin and P. Honeyman, "Formal Methods for the Analysis of Authentication Protocols," draft manuscript, 1994.
1356. F. Rubin, "Decrypting a Stream Cipher Based on J–K Flip–Flops," IEEE Transactions on Computing, v. C–28, n. 7, Jul 1979, pp. 483–487.
1357. R.A. Rueppel, Analysis and Design of Stream Ciphers, Springer–Verlag, 1986.
1358. R.A. Rueppel, "Correlation Immunity and the Summation Combiner," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 260–272.
1359. R.A. Rueppel, "When Shift Registers Clock Themselves," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1987, pp. 53–64.
1360. R.A. Rueppel, "Security Models and Notions for Stream Ciphers," Cryptography and Coding II, C. Mitchell, ed., Oxford: Clarendon Press, 1992, pp. 213–230.
1361. R.A. Rueppel, "On the Security of Schnorr's Pseudo–Random Sequence Generator," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 423–428.
1362. R.A. Rueppel, "Stream Ciphers," Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 65–134.
1363. R.A. Rueppel and J.L. Massey, "The Knapsack as a Nonlinear Function," IEEE International Symposium on Information Theory, Brighton, UK, May 1985.
1364. R.A. Rueppel and O.J. Staffelbach, "Products of Linear Recurring Sequences with Maximum Complexity," IEEE Transactions on Information Theory, v. IT–33, n. 1, Jan 1987, pp. 124–131.
1365. D. Russell and G.T. Gangemi, Computer Security Basics, O'Reilly and Associates, Inc., 1991.
1366. S. Russell and P. Craig, "Privacy Enhanced Mail Modules for ELM," Proceedings of the Internet Society 1994 Workshop on Network and Distributed System Security, The Internet Society, 1994, pp. 21–34.
1367. D.F.H. Sadok and J. Kelner, "Privacy Enhanced Mail Design and Implementation Perspectives," Computer Communications Review, v. 24, n. 3, Jul 1994, pp. 38–46.
1368. K. Sakano, "Digital Signatures with User–Flexible Reliability," Proceedings of the 1993 Symposium on Cryptography and Information Security (SCIS 93), Shuzenji, Japan, 28–30 Jan 1993, pp. 5C.1–8.
1369. K. Sakano, C. Park, and K. Kurosawa, "(k,n) Threshold Undeniable Signature Scheme," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 184–193.
1370. K. Sako, "Electronic Voting Schemes Allowing Open Objection to the Tally," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E77–A, n. 1, 1994, pp. 24–30.
1371. K. Sako and J. Kilian, "Secure Voting Using Partially Compatible Homomorphisms," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, p. 411–424.
1372. K. Sako and J. Kilian, "Receipt–Free Mix–Type Voting Scheme—A Practical Solution to the Implementation of a Voting Booth," Advances in Cryptology—EUROCRYPT '95 Proceedings, Springer–Verlag, 1995, pp. 393–403.
1373. A. Salomaa, Public–Key Cryptography, Springer–Verlag, 1990.
1374. A. Salomaa and L. Santean, "Secret Selling of Secrets with Many Buyers," ETACS Bulletin, v. 42, 1990, pp. 178–186.
1375. M. Sántha and U.V. Vazirani, "Generating Quasi–Random Sequences from Slightly Random Sources," Proceedings of the 25th Annual Symposium on the Foundations of Computer Science, 1984, pp. 434–440.
1376. M. Sántha and U.V. Vazirani, "Generating Quasi–Random Sequences from Slightly Random Sources," Journal of Computer and System Sciences, v. 33, 1986, pp. 75–87.
1377. S. Saryazdi, "An Extension to ElGamal Public Key Cryptosystem with a New Signature Scheme," Proceedings of the 1990 Bilkent International Conference on New Trends in Communication, Control, and Signal Processing, North Holland: Elsevier Science Publishers, 1990, pp. 195–198.
1378. J.E. Savage, "Some Simple Self–Synchronizing Digital Data Scramblers," Bell System Technical Journal, v. 46, n. 2, Feb 1967, pp. 448–487.
1379. B.P. Schanning, "Applying Public Key Distribution to Local Area Networks," Computers & Security, v. 1, n. 3, Nov 1982, pp. 268–274.
1380. B.P. Schanning, S.A. Powers, and J. Kowalchuk, "MEMO: Privacy and Authentication for the Automated Office," Proceedings of the 5th Conference on Local Computer Networks, IEEE Press, 1980, pp. 21–30.
1381. Schaumuller–Bichl, "Zur Analyse des Data Encryption Standard und Synthese Verwandter Chiffriersysteme," Ph.D. dissertation, Linz University, May 1981. (In German.)
1382. Schaumuller–Bichl, "On the Design and Analysis of New Cipher Systems Related to the DES," Technical Report, Linz University, 1983.
1383. A. Scherbius, "Ciphering Machine," U.S. Patent #1,657,411, 24 Jan 1928.
1384. J.I. Schiller, "Secure Distributed Computing," Scientific American, v. 271, n. 5, Nov 1994, pp. 72–76.
1385. R. Schlafly, "Complaint Against Exclusive Federal Patent License," Civil Action File No. C–93 20450, United States District Court for the Northern District of California.
1386. B. Schneier, "One–Way Hash Functions," Dr. Dobb's Journal, v. 16, n. 9, Sep 1991, pp. 148–151.
1387. B. Schneier, "Data Guardians," MacWorld, v. 10, n. 2, Feb 1993, pp. 145–151.
1388. B. Schneier, "Description of a New Variable–Length Key, 64–Bit Block Cipher (Blowfish)," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 191–204.
1389. B. Schneier, "The Blowfish Encryption Algorithm," Dr. Dobb's Journal, v. 19, n. 4, Apr 1994, pp. 38–40.
1390. B. Schneier, Protect Your Macintosh, Peachpit Press, 1994.
1391. B. Schneier, "Designing Encryption Algorithms for Real People," Proceedings of the 1994 ACM SIGSAC New Security Paradigms Workshop, IEEE Computer Society Press, 1994, pp. 63–71.
1392. B. Schneier, "A Primer on Authentication and Digital Signatures," Computer Security Journal, v. 10, n. 2, 1994, pp. 38–40.
1393. B. Schneier, "The GOST Encryption Algorithm," Dr. Dobb's Journal, v. 20, n. 1, Jan 95, pp. 123–124.
1394. B. Schneier, E–Mail Security (with PGP and PEM) New York: John Wiley & Sons, 1995.
1395. C.P. Schnorr, "On the Construction of Random Number Generators and Random Function Generators," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, pp. 225–232.
1396. C.P. Schnorr, "Efficient Signature Generation for Smart Cards," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 239–252.
1397. C.P. Schnorr, "Efficient Signature Generation for Smart Cards," Journal of Cryptology, v. 4, n. 3, 1991, pp. 161–174.
1398. C.P. Schnorr, "Method for Identifying Subscribers and for Generating and Verifying Electronic Signatures in a Data Exchange System," U.S. Patent #4,995,082, 19 Feb 1991.
1399. C.P. Schnorr, "An Efficient Cryptographic Hash Function," presented at the rump session of CRYPTO '91, Aug 1991.
1400. C.P. Schnorr, "FFT–Hash II, Efficient Cryptographic Hashing," Advances in Cryptology—EUROCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 45–54.
1401. C.P. Schnorr and W. Alexi, "RSA–bits are 0.5 + e Secure," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 113–126.
1402. C.P. Schnorr and S. Vaudenay, "Parallel FFT–Hashing," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 149–156.
1403. C.P. Schnorr and S. Vaudenay, "Black Box Cryptanalysis of Hash Networks Based on Multipermutations," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
1404. W. Schwartau, Information Warfare: Chaos on the Electronic Superhighway, New York: Thunders Mouth Press, 1994.
1405. R. Scott, "Wide Open Encryption Design Offers Flexible Implementations," Cryptologia, v. 9, n. 1, Jan 1985, pp. 75–90.
1406. J. Seberry, "A Subliminal Channel in Codes for Authentication without Secrecy," Ars Combinatorica, v. 19A, 1985, pp. 337–342.
1407. J. Seberry and J. Pieprzyk, Cryptography: An Introduction to Computer Security, Englewood Cliffs, N.J.: Prentice–Hall, 1989.
1408. J. Seberry, X.–M. Zhang, and Y. Zheng, "Nonlinearly Balanced Boolean Functions and Their Propagation Characteristics," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1994, pp. 49–60.
1409. H. Sedlack, "The RSA Cryptography Processor: The First High Speed One–Chip Solution," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 95–105.
1410. H. Sedlack and U. Golze, "An RSA Cryptography Processor," Microprocessing and Microprogramming, v. 18, 1986, pp. 583–590.
1411. E.S. Selmer, Linear Recurrence over Finite Field, University of Bergen, Norway, 1966.
1412. J.O. Shallit, "On the Worst Case of Three Algorithms for Computing the Jacobi Symbol," Journal of Symbolic Computation, v. 10, n. 6, Dec 1990, pp. 593–610.
1413. A. Shamir, "A Fast Signature Scheme," MIT Laboratory for Computer Science, Technical Memorandum, MIT/LCS/TM–107, Massachusetts Institute of Technology, Jul 1978.
1414. A. Shamir, "How to Share a Secret," Communications of the ACM, v. 24, n. 11, Nov 1979, pp. 612–613.
1415. A. Shamir, "On the Cryptocomplexity of Knapsack Systems," Proceedings of the 11th ACM Symposium on the Theory of Computing, 1979, pp. 118–129.
1416. A. Shamir, "The Cryptographic Security of Compact Knapsacks," MIT Library for Computer Science, Technical Memorandum, MIT/LCS/TM–164, Massachusetts Institute of Technology, 1980.
1417. A. Shamir, "On the Generation of Cryptographically Strong Pseudo–Random Sequences," Lecture Notes in Computer Science 62: 8th International Colloquium on Automata, Languages, and Programming, Springer–Verlag, 1981.
1418. A. Shamir, "A Polynomial Time Algorithm for Breaking the Basic Merkle–Hellman Cryptosystem," Advances in Cryptology: Proceedings of Crypto 82, Plenum Press, 1983, pp. 279–288.
1419. A. Shamir, "A Polynomial Time Algorithm for Breaking the Basic Merkle–Hellman Cryptosystem," Proceedings of the 23rd IEEE Symposium on the Foundations of Computer Science, 1982, pp. 145–152.
1420. A. Shamir, "On the Generation of Cryptographically Strong Pseudo–Random Sequences," ACM Transactions on Computer Systems, v. 1, n. 1, Feb 1983, pp. 38–44.
1421. A. Shamir, "A Polynomial Time Algorithm for Breaking the Basic Merkle–Hellman Cryptosystem," IEEE Transactions on Information Theory, v. IT–30, n. 5, Sep 1984, pp. 699–704.
1422. A. Shamir, "Identity–Based Cryptosystems and Signature Schemes," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 47–53.
1423. A. Shamir, "On the Security of DES," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 280–281.
1424. A. Shamir, lecture at SECURICOM '89.
1425. A. Shamir, "Efficient Signature Schemes Based on Birational Permutations," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 1–12.
1426. A. Shamir, personal communication, 1993.
1427. A. Shamir and A. Fiat, "Method, Apparatus and Article for Identification and Signature," U.S. Patent #4,748,668, 31 May 1988.
1428. A. Shamir and R. Zippel, "On the Security of the Merkle–Hellman Cryptographic Scheme," IEEE Transactions on Information Theory, v. 26, n. 3, May 1980, pp. 339–340.
1429. M. Shand, P. Bertin, and J. Vuillemin, "Hardware Speedups in Long Integer Multiplication," Proceedings of the 2nd Annual ACM Symposium on Parallel Algorithms and Architectures, 1990, pp. 138–145.
1430. D. Shanks, Solved and Unsolved Problems in Number Theory, Washington D.C.: Spartan, 1962.
1431. C.E. Shannon, "A Mathematical Theory of Communication," Bell System Technical Journal, v. 27, n. 4, 1948, pp. 379–423, 623–656.
1432. C.E. Shannon, "Communication Theory of Secrecy Systems," Bell System Technical Journal, v. 28, n. 4, 1949, pp. 656–715.
1433. C.E. Shannon, Collected Papers: Claude Elmwood Shannon, N.J.A. Sloane and A.D. Wyner, eds., New York: IEEE Press, 1993.
1434. C.E. Shannon, "Predication and Entropy in Printed English," Bell System Technical Journal, v. 30, n. 1, 1951, pp. 50–64.
1435. A. Shimizu and S. Miyaguchi, "Fast Data Encipherment Algorithm FEAL," Transactions of IEICE of Japan, v. J70–D, n. 7, Jul 87, pp. 1413–1423. (In Japanese.)
1436. A. Shimizu and S. Miyaguchi, "Fast Data Encipherment Algorithm FEAL," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 267–278.
1437. A. Shimizu and S. Miyaguchi, "FEAL—Fast Data Encipherment Algorithm," Systems and Computers in Japan, v. 19, n. 7, 1988, pp. 20–34, 104–106.
1438. A. Shimizu and S. Miyaguchi, "Data Randomization Equipment," U.S. Patent #4,850,019, 18 Jul 1989.
1439. M. Shimada, "Another Practical Public–key Cryptosystem," Electronics Letters, v. 28, n. 23, 5 Nov 1992, pp. 2146–2147.
1440. K. Shirriff, personal communication, 1993.
1441. H. Shizuya, T. Itoh, and K. Sakurai, "On the Complexity of Hyperelliptic Discrete Logarithm Problem," Advances in Cryptology—EUROCRYPT '91 Proceedings, Springer–Verlag, 1991, pp. 337–351.
1442. Z. Shmuley, "Composite Diffie–Hellman Public–Key Generating Systems Are Hard to Break," Computer Science Department, Technion, Haifa, Israel, Technical Report 356, Feb 1985.
1443. P.W. Shor, "Algorithms for Quantum Computation: Discrete Log and Factoring," Proceedings of the 35th Symposium on Foundations of Computer Science, 1994, pp. 124–134.
1444. L. Shroyer, letter to NIST regarding DSS, 17 Feb 1992.
1445. C. Shu, T. Matsumoto, and H. Imai, "A Multi–Purpose Proof System, Transactions of the Institute of Electronics, Information, and Communication Engineers, v. E75–A, n. 6, Jun 1992, pp. 735–743.
1446. E.H. Sibley, "Random Number Generators: Good Ones Are Hard to Find," Communications of the ACM, v. 31, n. 10, Oct 1988, pp. 1192–1201.
1447. V.M. Sidenikov and S.O. Shestakov, "On Encryption Based on Generalized Reed–Solomon Codes," Diskretnaya Math, v. 4, 1992, pp. 57–63. (In Russian.)
1448. V.M. Sidenikov and S.O. Shestakov, "On Insecurity of Cryptosystems Based on Generalized Reed–Solomon Codes," unpublished manuscript, 1992.
1449. D.P. Sidhu, "Authentication Protocols for Computer Networks," Computer Networks and ISDN Systems, v. 11, n. 4, Apr 1986, pp. 297–310.
1450. T. Siegenthaler, "Correlation–Immunity of Nonlinear Combining Functions for Cryptographic Applications," IEEE Transactions on Information Theory, v. IT–30, n. 5, Sep 1984, pp. 776–780.
1451. T. Siegenthaler, "Decrypting a Class of Stream Ciphers Using Ciphertext Only," IEEE Transactions on Computing, v. C–34, Jan 1985, pp. 81–85.
1452. T. Siegenthaler, "Cryptanalyst's Representation of Nonlinearity Filtered ml–sequences," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 103–110.
1453. R.D. Silverman, "The Multiple Polynomial Quadratic Sieve," Mathematics of Computation, v. 48, n. 177, Jan 1987, pp. 329–339.
1454. G.J. Simmons, "Authentication without Secrecy: A Secure Communication Problem Uniquely Solvable by Asymmetric Encryption Techniques," Proceedings of IEEE EASCON '79, 1979, pp. 661–662.
1455. G.J. Simmons, "Some Number Theoretic Questions Arising in Asymmetric Encryption Techniques," Annual Meeting of the American Mathematical Society, AMS Abstract 763.94.1, 1979, pp. 136–151.
1456. G.J. Simmons, "High Speed Arithmetic Using Redundant Number Systems," Proceedings of the National Telecommunications Conference, 1980, pp. 49.3.1–49.3.2.
1457. G.J. Simmons, "A 'Weak' Privacy Protocol Using the RSA Cryptosystem," Cryptologia, v. 7, n. 2, Apr 1983, pp. 180–182.
1458. G.J. Simmons, "The Prisoner's Problem and the Subliminal Channel," Advances in Cryptology: Proceedings of CRYPTO '83, Plenum Press, 1984, pp. 51–67.
1459. G.J. Simmons, "The Subliminal Channel and Digital Signatures," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 364–378.
1460. G.J. Simmons, "A Secure Subliminal Channel (?)," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 33–41.
1461. G.J. Simmons, "Cryptology," Encyclopedia Britannica, 16th edition, 1986, pp. 913–924B.
1462. G.J. Simmons, "How to (Really) Share a Secret," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 390–448.
1463. G.J. Simmons, "Prepositioned Secret Sharing Schemes and/or Shared Control Schemes," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 436–467.
1464. G.J. Simmons, "Geometric Shares Secret and/or Shared Control Schemes," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 216–241.
1465. G.J. Simmons, ed., Contemporary Cryptology: The Science of Information Integrity, IEEE Press, 1992.
1466. G.J. Simmons, "An Introduction to Shared Secret and/or Shared Control Schemes and Their Application," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 441–497.
1467. G.J. Simmons, "How to Insure that Data Acquired to Verify Treaty Compliance Are Trustworthy," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 615–630.
1468. G.J. Simmons, "The Subliminal Channels of the U.S. Digital Signature Algorithm (DSA)," Proceedings of the Third Symposium on: State and Progress of Research in Cryptography, Rome: Fondazone Ugo Bordoni, 1993, pp. 35–54.
1469. G.J. Simmons, "Subliminal Communication is Easy Using the DSA," Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 218–232.
1470. G.J. Simmons, "An Introduction to the Mathematics of Trust in Security Protocols," Proceedings: Computer Security Foundations Workshop VI, IEEE Computer Society Press, 1993, pp. 121–127.
1471. G.J. Simmons, "Protocols that Ensure Fairness," Codes and Ciphers, Institute of Mathematics and its Applications, 1995, pp. 383–394.
1472. G.J. Simmons, "Cryptanalysis and Protocol Failures," Communications of the ACM, v. 37, n. 11, Nov 1994, pp. 56–65.
1473. G.J. Simmons, "Subliminal Channels: Past and Present," European Transactions on Telecommuncations, v. 4, n. 4, Jul/Aug 1994, pp. 459–473.
1474. G.J. Simmons and M.J. Norris, How to Cipher Fast Using Redundant Number Systems, SAND–80–1886, Sandia National Laboratories, Aug 1980.
1475. A. Sinkov, Elementary Cryptanalysis, Mathematical Association of America, 1966.
1476. R. Siromoney and L. Matthew, "A Public Key Cryptosystem Based on Lyndon Words," Information Processing Letters, v. 35, n. 1, 15 Jun 1990, pp. 33–36.
1477. B. Smeets, "A Note on Sequences Generated by Clock–Controlled Shift Registers," Advances in Cryptology—EUROCRYPT '85, Springer–Verlag, 1986, pp. 40–42.
1478. M.E. Smid, "A Key Notarization System for Computer Networks," NBS Special Report 500–54, U.S. Department of Commerce, Oct 1979.
1479. M.E. Smid, "The DSS and the SHS," Federal Digital Signature Applications Symposium, Rockville, MD, 17–18 Feb 1993.
1480. M.E. Smid and D.K. Branstad, "The Data Encryption Standard: Past and Future," Proceedings of the IEEE, v. 76, n. 5., May 1988, pp. 550–559.
1481. M.E. Smid and D.K. Branstad, "The Data Encryption Standard: Past and Future," in Contemporary Cryptology: The Science of Information Integrity, G.J. Simmons, ed., IEEE Press, 1992, pp. 43–64.
1482. J.L. Smith, "The Design of Lucifer, A Cryptographic Device for Data Communications," IBM Research Report RC3326, 1971.
1483. J.L. Smith, "Recirculating Block Cipher Cryptographic System," U.S. Patent #3,796,830, 12 Mar 1974.
1484. J.L. Smith, W.A. Notz, and P.R. Osseck, "An Experimental Application of Cryptography to a Remotely Accessed Data System," Proceedings of the ACM Annual Conference, Aug 1972, pp. 282–290.
1485. K. Smith, "Watch Out Hackers, Public Encryption Chips Are Coming," Electronics Week, 20 May 1985, pp. 30–31.
1486. P. Smith, "LUC Public–Key Encryption," Dr. Dobb's Journal, v. 18, n. 1, Jan 1993, pp. 44–49.
1487. P. Smith and M. Lennon, "LUC: A New Public Key System," Proceedings of the Ninth International Conference on Information Security, IFIP/Sec 1993, North Holland: Elsevier Science Publishers, 1993, pp. 91–111.
1488. E. Snekkenes, "Exploring the BAN Approach to Protocol Analysis," Proceedings of the 1991 IEEE Computer Society Symposium on Research in Security and Privacy, 1991, pp. 171–181.
1489. B. Snow, "Multiple Independent Binary Bit Stream Generator," U.S. Patent #5,237,615, 17 Aug 1993.
1490. R. Solovay and V. Strassen, "A Fast Monte–Carlo Test for Primality," SIAM Journal on Computing, v. 6, Mar 1977, pp. 84–85; erratum in ibid, v. 7, 1978, p. 118.
1491. T. Sorimachi, T. Tokita, and M. Matsui, "On a Cipher Evaluation Method Based on Differential Cryptanalysis," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 94), Lake Biwa, Japan, 27–29 Jan 1994, pp. 4C.1–9. (In Japanese.)
1492. A. Sorkin, "Lucifer, a Cryptographic Algorithm," Cryptologia, v. 8, n. 1, Jan 1984, pp. 22–41.
1493. W. Stallings, "Kerberos Keeps the Ethernet Secure," Data Communications, Oct 1994, pp. 103–111.
1494. W. Stallings, Network and Internetwork Security, Englewood Cliffs, N.J.: Prentice–Hall, 1995.
1495. W. Stallings, Protect Your Privacy: A Guide for PGP Users, Englewood Cliffs, N.J.: Prentice–Hall, 1995.
1496. Standards Association of Australia, "Australian Standard 2805.4 1985: Electronic Funds Transfer—Requirements for Interfaces: Part 4—Message Authentication," SAA, North Sydney, NSW, 1985.
1497. Standards Association of Australia, "Australian Standard 2805.5 1985: Electronic Funds Transfer—Requirements for Interfaces: Part 5—Data Encipherment Algorithm," SAA, North Sydney, NSW, 1985.
1498. Standards Association of Australia, "Australian Standard 2805.5.3: Electronic Data Transfer—Requirements for Interfaces: Part 5.3—Data Encipherment Algorithm 2," SAA, North Sydney, NSW, 1992.
1499. J.G. Steiner, B.C. Neuman, and J.I. Schiller, "Kerberos: An Authentication Service for Open Network Systems," USENIX Conference Proceedings, Feb 1988, pp. 191–202.
1500. J. Stern, "Secret Linear Congruential Generators Are Not Cryptographically Secure," Proceedings of the 28th Symposium on Foundations of Computer Science, 1987, pp. 421–426.
1501. J. Stern, "A New Identification Scheme Based on Syndrome Decoding," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 13–21.
1502. A. Stevens, "Hacks, Spooks, and Data Encryption," Dr. Dobb's Journal, v. 15, n. 9, Sep 1990, pp. 127–134, 147–149.
1503. R. Struik, "On the Rao–Nam Private–Key Cryptosystem Using Non–Linear Codes," IEEE 1991 Symposium on Information Theory, Budapest, Hungary, 1991.
1504. R. Struik and J. van Tilburg, "The Rao–Nam Scheme Is Insecure against a Chosen–Plaintext Attack," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 445–457.
1505. S.G. Stubblebine and V.G. Gligor, "Protecting the Integrity of Privacy–Enhanced Mail with DES–Based Authentication Codes," Proceedings of the Privacy and Security Research Group 1993 Workshop on Network and Distributed System Security, The Internet Society, 1993, pp. 75–80.
1506. R. Sugarman, "On Foiling Computer Crime," IEEE Spectrum, v. 16, n. 7, Jul 79, pp. 31–32.
1507. H.N. Sun and T. Hwang, "Public–key ID–Based Cryptosystem," Proceedings of the 25th Annual 1991 IEEE International Carnahan Conference on Security Technology, Taipei, Taiwan, 1–3 Oct 1991, pp. 142–144.
1508. P.F. Syverson, "Formal Semantics for Logics of Computer Protocols," Proceedings of the Computer Security Foundations Workshop III, IEEE Computer Society Press, 1990, pp. 32–41.
1509. P.F. Syverson, "The Use of Logic in the Analysis of Cryptographic Protocols," Proceedings of the 1991 IEEE Computer Society Symposium on Research in Security and Privacy, 1991, pp. 156–170.
1510. P.F. Syverson, "Knowledge, Belief, and Semantics in the Analysis of Cryptographic Protocols," Journal of Computer Security, v. 1, n. 3, 1992, pp. 317–334.
1511. P.F. Syverson, "Adding Time to a Logic Authentication," 1st ACM Conference on Computer and Communications Security, ACM Press, 1993, pp. 97–106.
1512. P.F. Syverson and C.A. Meadows, "A Logical Language for Specifying Cryptographic Protocol Requirements," Proceedings of the 1993 IEEE Computer Society Symposium on Research in Security and Privacy, 1993, pp. 14–28.
1513. P.F. Syverson and C.A. Meadows, "Formal Requirements for Key Distribution Protocols," Advances in Cryptology—EUROCRYPT '94 Proceedings, Springer–Verlag, 1995, to appear.
1514. P.F. Syverson and P.C. van Oorschot, "On Unifying Some Cryptographic Protocol Logics," Proceedings of the 1994 IEEE Computer Society Symposium on Research in Security and Privacy, 1994, pp. 165–177.
1515. H. Tanaka, "A Realization Scheme for the Identity–Based Cryptosystem," Advances in Cryptology—CRYPTO '87 Proceedings, Springer–Verlag, 1988, pp. 340–349.
1516. H. Tanaka, "A Realization Scheme for the Identity–Based Cryptosystem," Electronics and Communications in Japan, Part 3 (Fundamental Electronic Science), v. 73, n. 5, May 1990, pp. 1–7.
1517. H. Tanaka, "Identity–Based Noninteractive Common–Key Generation and Its Application to Cryptosystems," Transactions of the Institute of Electronics, Information, and Communication Engineers, v. J75–A, n. 4, Apr 1992, pp. 796–800.
1518. J. Tardo and K. Alagappan, "SPX: Global Authentication Using Public Key Certificates," Proceedings of the 1991 IEEE Computer Society Symposium on Security and Privacy, 1991, pp. 232–244.
1519. J. Tardo, K. Alagappan, and R. Pitkin, "Public Key Based Authentication Using Internet Certificates," USENIX Security II Workshop Proceedings, 1990, pp. 121–123.
1520. A. Tardy–Corfdir and H. Gilbert, "A Known Plaintext Attack of FEAL–4 and FEAL–6," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 172–182.
1521. M. Tatebayashi, N. Matsuzaki, and D.B. Newman, "Key Distribution Protocol for Digital Mobile Communication System," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 324–333.
1522. M. Taylor, "Implementing Privacy Enhanced Mail on VMS," Proceedings of the Privacy and Security Research Group 1993 Workshop on Network and Distributed System Security, The Internet Society, 1993, pp. 63–68.
1523. R. Taylor, "An Integrity Check Value Algorithm for Stream Ciphers," Advances in Cryptology—CRYPTO '93 Proceedings, Springer–Verlag, 1994, pp. 40–48.
1524. T. Tedrick, "Fair Exchange of Secrets," Advances in Cryptology: Proceedings of CRYPTO '84, Springer–Verlag, 1985, pp. 434–438.
1525. R. Terada and P.G. Pinheiro, "How to Strengthen FEAL against Differential Cryptanalysis," Proceedings of the 1995 Japan–Korea Workshop on Information Security and Cryptography, Inuyama, Japan, 24–27 Jan 1995, pp. 153–162.
1526. J.–P. Tillich and G. Zémor, "Hashing with SI2," Advances in Cryptology—CRYPTO '94 Proceedings, Springer–Verlag, 1994, pp. 40–49.
1527. T. Tokita, T. Sorimachi, and M. Matsui, "An Efficient Search Algorithm for the Best Expression on Linear Cryptanalysis," IEICE Japan, Technical Report, ISEC93–97, 1994.
1528. M. Tompa and H. Woll, "Random Self–Reducibility and Zero–Knowledge Interactive Proofs of Possession of Information," Proceedings of the 28th IEEE Symposium on the Foundations of Computer Science, 1987, pp. 472–482.
1529. M. Tompa and H. Woll, "How to Share a Secret with Cheaters," Journal of Cryptology, v. 1, n. 2, 1988, pp. 133–138.
1530. M.–J. Toussaint, "Verification of Cryptographic Protocols," Ph.D. dissertation, Université de Lige, 1991.
1531. M.–J. Toussaint, "Deriving the Complete Knowledge of Participants in Cryptographic Protocols," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 24–43.
1532. M.–J. Toussaint, "Separating the Specification and Implementation Phases in Cryptology," ESORICS 92, Proceedings of the Second European Symposium on Research in Computer Security, Springer–Verlag, 1992, pp. 77–101.
1533. P.D. Townsend, J.G. Rarity, and P.R. Tapster, "Enhanced Single Photon Fringe Visibility in a 10 km–Long Prototype Quantum Cryptography Channel," Electronics Letters, v. 28, n. 14, 8 Jul 1993, pp. 1291–1293.
1534. S.A. Tretter, "Properties of PN2 Sequences," IEEE Transactions on Information Theory, v. IT–20, n. 2, Mar 1974, pp. 295–297.
1535. H. Truman, "Memorandum for: The Secretary of State, The Secretary of Defense," A 20707 5/4/54/OSO, NSA TS CONTL. NO 73–00405, 24 Oct 1952.
1536. Y.W. Tsai and T. Hwang, "ID Based Public Key Cryptosystem Based on Okamoto and Tanaka's ID Based One–Way Communications Scheme," Electronics Letters, v. 26, n. 10, 1 May 1990, pp. 666–668.
1537. G. Tsudik, "Message Authentication with One–Way Hash Functions," ACM Computer Communications Review, v. 22, n. 5, 1992, pp. 29–38.
1538. S. Tsujii and K. Araki, "A Rebuttal to Coppersmith's Attacking Method," memorandum presented at Crypto '94, Aug 1994. 1539. S. Tsujii, K. Araki, J. Chao, T. Sekine, and Y. Matsuzaki, "ID–Based Key Sharing Scheme—Cancellation of Random Numbers by Iterative Addition," IEICE Japan, Technical Report, ISEC 92–47, Oct 1992.
1540. S. Tsujii, K. Araki, and T. Sekine, "A New Scheme of Noninteractive ID–Based Key Sharing with Explosively High Degree of Separability," Technical Report, Department of Computer Science, Tokyo Institute of Technology, 93TR–0016, May 1993.
1541. S. Tsujii, K. Araki, and T. Sekine, "A New Scheme of Non Interactive ID–Based key Sharing with Explosively High Degree of Separability (Second Version)," Technical Report, Department of Computer Science, Tokyo Institute of Technology, 93TR–0020, Jul 1993.
1542. S. Tsujii, K. Araki, T. Sekine, and K. Tanada, "A New Scheme of Non Interactive ID–Based Key Sharing with Explosively High Degree of Separability," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 49–58.
1543. S. Tsujii, K. Araki, H. Tanaki, J. Chao, T. Sekine, and Y. Matsuzaki, "ID–Based Key Sharing Scheme—Reply to Tanaka's Comment," IEICE Japan, Technical Report, ISEC 92–60, Dec 1992.
1544. S. Tsujii and J. Chao, "A New ID–based Key Sharing System," Advances in Cryptology—CRYPTO '91 Proceedings, Springer–Verlag, 1992, pp. 288–299.
1545. S. Tsujii, J. Chao, and K. Araki, "A Simple ID–Based Scheme for Key Sharing," IEICE Japan, Technical Report, ISEC 92–25, Aug 1992.
1546. S. Tsujii and T. Itoh, "An ID–Based Cryptosystem Based on the Discrete Logarithm Problem," IEEE Journal on Selected Areas in Communication, v. 7, n. 4, May 1989, pp. 467–473.
1547. S. Tsujii and T. Itoh, "An ID–Based Cryptosystem Based on the Discrete Logarithm Problem," Electronics Letters, v. 23, n. 24, Nov 1989, pp. 1318–1320.
1548. S. Tsujii, K. Kurosawa, T. Itoh, A. Fujioka, and T. Matsumoto, "A Public–Key Cryptosystem Based on the Difficulty of Solving a System of Non–Linear Equations," TSUJII Laboratory Technical Memorandum, n. 1, 1986.
1549. Y. Tsunoo, E. Okamoto, and H. Doi, "Analytical Known Plain–Text Attack for FEAL–4 and Its Improvement," Proceedings of the 1994 Symposium on Cryptography and Information Security (SCIS 93), 1993.
1550. Y. Tsunoo, E. Okamoto, T. Uyematsu, and M. Mambo, "Analytical Known Plain–Text Attack for FEAL–6" Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 253–261.
1551. W. Tuchman, "Hellman Presents No Shortcut Solutions to DES," IEEE Spectrum, v. 16, n. 7, July 1979, pp. 40–41.
1552. U.S. Senate Select Committee on Intelligence, "Unclassified Summary: Involvement of NSA in the Development of the Data Encryption Standard," IEEE Communications Magazine, v. 16, n. 6, Nov 1978, pp. 53–55.
1553. B. Vallée, M. Girault, and P. Toffin, "How to Break Okamoto's Cryptosystem by Reducing Lattice Values," Advances in Cryptology—EUROCRYPT '88 Proceedings, Springer–Verlag, 1988, p. 281–291.
1554. H. Van Antwerpen, "Electronic Cash," Master's thesis, CWI, Netherlands, 1990.
1555. K. Van Espen and J. Van Mieghem, "Evaluatie en Implementatie van Authentiseringsalgoritmen," graduate thesis, ESAT Laboratorium, Katholieke Universiteit Leuven, 1989. (In Dutch.)
1556. P.C. van Oorschot, "Extending Cryptographic Logics of Belief to Key Agreement Protocols," Proceedings of the 1st Annual ACM Conference on Computer and Communications Security, 1993, pp. 232–243.
1557. P.C. van Oorschot, "An Alternate Explanation for Two BAN–logic 'Failures,'" Advances in Cryptology—EUROCRYPT '93 Proceedings, Springer–Verlag, 1994, pp. 443–447.
1558. P.C. van Oorschot and M.J. Wiener, "A Known–Plaintext Attack on Two–Key Triple Encryption," Advances in Cryptology—EUROCRYPT '90 Proceedings, Springer–Verlag, 1991, pp. 318–325.
1559. J. van Tilburg, "On the McEliece Cryptosystem," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 119–131.
1560. J. van Tilburg, "Cryptanalysis of the Xinmei Digital Signature Scheme," Electronics Letters, v. 28, n. 20, 24 Sep 1992, pp. 1935–1938.
1561. J. van Tilburg, "Two Chosen–Plaintext Attacks on the Li Wang Joing Authentication and Encryption Scheme," Applied Algebra, Algebraic Algorithms and Error Correcting Codes 10, Springer–Verlag, 1993, pp. 332–343.
1562. J. van Tilburg, "Security–Analysis of a Class of Cryptosystems Based on Linear Error–Correcting Codes," Ph.D. dissertation, Technical University Eindhoven, 1994.
1563. A. Vandemeulebroecke, E. Vanzieleghem, T. Denayer, and P.G. Jespers, "A Single Chip 1024 Bits RSA Processor," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 219–236.
1564. J. Vanderwalle, D. Chaum, W. Fumy, C. Jansen, P. Landrock, and G. Roelofsen, "A European Call for Cryptographic Algorithms: RIPE; RACE Integrity Primitives Evaluation," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 267–271.
1565. V. Varadharajan, "Verification of Network Security Protocols," Computers and Security., v. 8, n. 8, Aug 1989, pp. 693–708.
1566. V. Varadharajan, "Use of a Formal Description Technique in the Specification of Authentication Protocols," Computer Standards and Interfaces, v. 9, 1990, pp. 203–215.
1567. S. Vaudenay, "FFT–Hash–II Is not Yet Collision–Free," Advances in Cryptology—CRYPTO '92 Proceedings, Springer–Verlag, pp. 587–593.
1568. S. Vaudenay, "Differential Cryptanalysis of Blowfish," unpublished manuscript, 1995.
1569. U.V. Vazirani and V.V. Vazirani, "Trapdoor Pseudo–Random Number Generators with Applications to Protocol Design," Proceedings of the 24th IEEE Symposium on the Foundations of Computer Science, 1983, pp. 23–30.
1570. U.V. Vazirani and V.V. Vazirani, "Efficient and Secure Pseudo–Random Number Generation," Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science, 1984, pp. 458–463.
1571. U.V. Vazirani and V.V. Vazirani, "Efficient and Secure Pseudo–Random Number Generation," Advances in Cryptology: Proceedings of CRYPTO '84, Springer–Verlag, 1985, pp. 193–202.
1572. I. Verbauwhede, F. Hoornaert, J. Vanderwalle, and H. De Man, "ASIC Cryptographical Processor Based on DES," Euro ASIC '91 Proceedings, 1991, pp. 292–295.
1573. I. Verbauwhede, F. Hoornaert, J. Vanderwalle, H. De Man, and R. Govaerts, "Security Considerations in the Design and Implementation of a New DES Chip," Advances in Cryptology—EUROCRYPT '87 Proceedings, Springer–Verlag, 1988, pp. 287–300.
1574. R. Vogel, "On the Linear Complexity of Cascaded Sequences," Advances in Cryptology: Proceedings of EUROCRYPT 84, Springer–Verlag, 1985, pp. 99–109.
1575. S. von Solms and D. Naccache, "On Blind Signatures and Perfect Crimes," Computers & Security, v. 11, 1992, pp. 581–583.
1576. V.L. Voydock and S.T. Kent, "Security Mechanisms in High–Level Networks," ACM Computing Surveys, v. 15, n. 2, Jun 1983, pp. 135–171.
1577. N.R. Wagner, P.S. Putter, and M.R. Cain, "Large–Scale Randomization Techniques," Advances in Cryptology—CRYPTO '86 Proceedings, Springer–Verlag, 1987, pp. 393–404.
1578. M. Waidner and B. Pfitzmann, "The Dining Cryptographers in the Disco: Unconditional Sender and Recipient Untraceability with Computationally Secure Serviceability," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, p. 690.
1579. S.T. Walker, "Software Key Escrow—A Better Solution for Law Enforcement's Needs?" TIS Report #533, Trusted Information Systems, Aug 1994.
1580. S.T. Walker, "Thoughts on Key Escrow Acceptability," TIS Report #534D, Trusted Information Systems, Nov 1994.
1581. S.T. Walker, S.B. Lipner, C.M. Ellison, D.K. Branstad, and D.M. Balenson, "Commercial Key Escrow—Something for Everyone—Now and for the Future," TIS Report #541, Trusted Information Systems, Jan 1995.
1582. M.Z. Wang and J.L. Massey, "The Characteristics of All Binary Sequences with Perfect Linear Complexity Profiles," Abstracts of Papers, EUROCRYPT '86, 20–22 May 1986.
1583. E.J. Watson, "Primitive Polynomials (Mod 2)," Mathematics of Computation, v. 16, 1962, p. 368.
1584. P. Wayner, "Mimic Functions," Cryptologia, v. 16, n. 3, Jul 1992, pp. 193–214.
1585. P. Wayner, "Mimic Functions and Tractability," draft manuscript, 1993.
1586. A.F. Webster and S.E. Tavares, "On the Design of S–Boxes," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 523–534.
1587. G. Welchman, The Hut Six Story: Breaking the Enigma Codes, New York: McGraw–Hill, 1982.
1588. A.L. Wells Jr., "A Polynomial Form for Logarithms Modulo a Prime," IEEE Transactions on Information Theory, Nov 1984, pp. 845–846.
1589. D.J. Wheeler, "A Bulk Data Encryption Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer–Verlag, 1994, pp. 127–134.
1590. D.J. Wheeler, personal communication, 1994.
1591. D.J. Wheeler and R. Needham, "A Large Block DES–Like Algorithm," Technical Report 355, "Two Cryptographic Notes," Computer Laboratory, University of Cambridge, Dec 1994, pp. 1–3.
1592. D.J. Wheeler and R. Needham, "TEA, A Tiny Encryption Algorithm," Technical Report 355, "Two Cryptographic Notes," Computer Laboratory, University of Cambridge, Dec 1994, pp. 1–3.
1593. S.R. White, "Covert Distributed Processing with Computer Viruses," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 616–619.
1594. White House, Office of the Press Secretary, "Statement by the Press Secretary," 16 Apr 1993.
1595. B.A. Wichman and I.D. Hill, "An Efficient and Portable Pseudo–Random Number Generator," Applied Statistics, v. 31, 1982, pp. 188–190.
1596. M.J. Wiener, "Cryptanalysis of Short RSA Secret Exponents," IEEE Transactions on Information Theory, v. 36, n. 3, May 1990, pp. 553–558.
1597. M.J. Wiener, "Efficient DES Key Search," presented at the rump session of CRYPTO '93, Aug 1993.
1598. M.J. Wiener, "Efficient DES Key Search," TR–244, School of Computer Science, Carleton University, May 1994.
1599. M.V. Wilkes, Time–Sharing Computer Systems, New York: American Elsevier, 1968.
1600. E.A. Williams, An Invitation to Cryptograms, New York: Simon and Schuster, 1959.
1601. H.C. Williams, "A Modification of the RSA Public–Key Encryption Procedure," IEEE Transactions on Information Theory, v. IT–26, n. 6, Nov 1980, pp. 726–729.
1602. H.C. Williams, "An Overview of Factoring," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 71–80.
1603. H.C. Williams, "Some Public–Key Crypto–Functions as Intractable as Factorization," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, pp. 66–70.
1604. H.C. Williams, "Some Public–Key Crypto–Functions as Intractable as Factorization," Cryptologia, v. 9, n. 3, Jul 1985, pp. 223–237.
1605. H.C. Williams, "An M3 Public–Key Encryption Scheme," Advances in Cryptology—CRYPTO '85, Springer–Verlag, 1986, pp. 358–368.
1606. R.S. Winternitz, "Producing One–Way Hash Functions from DES," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 203–207.
1607. R.S. Winternitz, "A Secure One–Way Hash Function Built from DES," Proceedings of the 1984 Symposium on Security and Privacy, 1984, pp. 88–90.
1608. S. Wolfram, "Random Sequence Generation by Cellular Automata," Advances in Applied Mathematics, v. 7, 1986, pp. 123–169.
1609. S. Wolfram, "Cryptography with Cellular Automata," Advances in Cryptology—CRYPTO '85 Proceedings, Springer–Verlag, 1986, pp. 429–432.
1610. T.Y.C. Woo and S.S. Lam, "Authentication for Distributed Systems," Computer, v. 25, n. 1, Jan 1992, pp. 39–52.
1611. T.Y.C. Woo and S.S. Lam, "'Authentication' Revisited," Computer, v. 25, n. 3, Mar 1992, p. 10.
1612. T.Y.C. Woo and S.S. Lam, "A Semantic Model for Authentication Protocols," Proceedings of the 1993 IEEE Computer Society Symposium on Research in Security and Privacy, 1993, pp. 178–194.
1613. M.C. Wood, technical report, Cryptech, Inc., Jamestown, NY, Jul 1990.
1614. M.C. Wood, "Method of Cryptographically Transforming Electronic Digital Data from One Form to Another," U.S. Patent #5,003,596, 26 Mar 1991.
1615. M.C. Wood, personal communication, 1993.
1616. C.K. Wu and X.M. Wang, "Determination of the True Value of the Euler Totient Function in the RSA Cryptosystem from a Set of Possibilities," Electronics Letters, v. 29, n. 1, 7 Jan 1993, pp. 84–85.
1617. M.C. Wunderlich, "Recent Advances in the Design and Implementation of Large Integer Factorization Algorithms," Proceedings of 1983 Symposium on Security and Privacy, IEEE Computer Society Press, 1983, pp. 67–71.
1618. Xerox Network System (XNS) Authentication Protocol, XSIS 098404, Xerox Corporation, Apr 1984.
1619. Y.Y. Xian, "New Public Key Distribution System," Electronics Letters, v. 23, n. 11, 1987, pp. 560–561.
1620. L.D. Xing and L.G. Sheng, "Cryptanalysis of New Modified Lu–Lee Cryptosystems," Electronics Letters, v. 26, n. 19, 13 Sep 1990, p. 1601–1602.
1621. W. Xinmei, "Digital Signature Scheme Based on Error–Correcting Codes," Electronics Letters, v. 26, n. 13, 21 Jun 1990, p. 898–899.
1622. S.B. Xu, D.K. He, and X.M. Wang, "An Implementation of the GSM General Data Encryption Algorithm A5," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 287–291. (In Chinese.)
1623. M. Yagisawa, "A New Method for Realizing Public–Key Cryptosystem," Cryptologia, v. 9, n. 4, Oct 1985, pp. 360–380.
1624. C.H. Yang, "Modular Arithmetic Algorithms for Smart Cards," IEICE Japan, Technical Report, ISEC92–16, 1992.
1625. C.H. Yang and H. Morita, "An Efficient Modular–Multiplication Algorithm for Smart–Card Software Implementation," IEICE Japan, Technical Report, ISEC91–58, 1991.
1626. J.H. Yang, K.C. Zeng, and Q.B. Di, "On the Construction of Large S–Boxes," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 24–32. (In Chinese.)
1627. A.C.–C. Yao, "Protocols for Secure Computations," Proceedings of the 23rd IEEE Symposium on the Foundations of Computer Science, 1982, pp. 160–164.
1628. B. Yee, "Using Secure Coprocessors," Ph.D. dissertation, School of Computer Science, Carnegie Mellon University, May 1994.
1629. S.–M. Yen, "Design and Computation of Public Key Cryptosystems," Ph.D. dissertation, National Cheng Hung University, Apr 1994.
1630. S.–M. Yen and C.–S. Lai, "New Digital Signature Scheme Based on the Discrete Logarithm," Electronics Letters, v. 29, n. 12, 1993, pp. 1120–1121.
1631. K. Yiu and K. Peterson, "A Single–Chip VLSI Implementation of the Discrete Exponential Public–Key Distribution System," IBM Systems Journal, v. 15, n. 1, 1982, pp. 102–116.
1632. K. Yiu and K. Peterson, "A Single–Chip VLSI Implementation of the Discrete Exponential Public–Key Distribution System," Proceedings of Government Microcircuit Applications Conference, 1982, pp. 18–23.
1633. H.Y. Youm, S.L. Lee, and M.Y. Rhee, "Practical Protocols for Electronic Cash," Proceedings of the 1993 Korea–Japan Workshop on Information Security and Cryptography, Seoul, Korea, 24–26 Oct 1993, pp. 10–22.
1634. M. Yung, "Cryptoprotocols: Subscriptions to a Public Key, the Secret Blocking, and the Multi–Player Mental Poker Game," Advances in Cryptology: Proceedings of CRYPTO 84, Springer–Verlag, 1985, 439–453.
1635. G. Yuval, "How to Swindle Rabin," Cryptologia, v. 3, n. 3, Jul 1979, pp. 187–190.
1636. K.C. Zeng and M. Huang, "On the Linear Syndrome Method in Cryptanalysis," Advances in Cryptology—CRYPTO '88 Proceedings, Springer–Verlag, 1990, pp. 469–478.
1637. K.C. Zeng, M. Huang, and T.R.N. Rao, "An Improved Linear Algorithm in Cryptanalysis with Applications," Advances in Cryptology—CRYPTO '90 Proceedings, Springer–Verlag, 1991, pp. 34–47.
1638. K.C. Zeng, C.–H. Yang, and T.R.N. Rao, "On the Linear Consistency Test (LCT) in Cryptanalysis with Applications," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 164–174.
1639. K.C. Zeng, C.–H. Yang, D.–Y. Wei, and T.R.N. Rao, "Pseudorandom Bit Generators in Stream–Cipher Cryptography," IEEE Computer, v. 24, n. 2, Feb 1991, pp. 8–17.
1640. M. Zhang, S.E. Tavares, and L.L. Campbell, "Information Leakage of Boolean Functions and Its Relationship to Other Cryptographic Criteria," Proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 156–165.
1641. M. Zhang and G. Xiao, "A Modified Design Criterion for Stream Ciphers," CHINACRYPT '94, Xidian, China, 11–15 Nov 1994, pp. 201–209. (In Chinese.)
1642. Y. Zheng, T. Matsumoto, and H. Imai, "Duality between two Cryptographic Primitives," Papers of Technical Group for Information Security, IEICE of Japan, Mar 1989, pp. 47–57.
1643. Y. Zheng, T. Matsumoto, and H. Imai, "Impossibility and Optimality Results in Constructing Pseudorandom Permutations," Advances in Cryptology—EUROCRYPT '89 Proceedings, Springer–Verlag, 1990, pp. 412–422.
1644. Y. Zheng, T. Matsumoto, and H. Imai, "On the Construction of Block Ciphers Provably Secure and Not Relying on Any Unproved Hypotheses," Advances in Cryptology—CRYPTO '89 Proceedings, Springer–Verlag, 1990, pp. 461–480.
1645. Y. Zheng, T. Matsumoto, and H. Imai, "Duality between two Cryptographic Primitives," Proceedings of the 8th International Conference on Applied Algebra, Algebraic Algorithms and Error–Correcting Codes, Springer–Verlag, 1991, pp. 379–390.
1646. Y. Zheng, J. Pieprzyk, and J. Seberry, "HAVAL—A One–Way Hashing Algorithm with Variable Length of Output," Advances in Crytology—AUSCRYPT '92 Proceedings, Springer–Verlag, 1993, pp. 83–104.
1647. N. Zierler, "Linear Recurring Sequences," Journal Soc. Indust. Appl. Math., v. 7, n. 1, Mar 1959, pp. 31–48.
1648. N. Zierler, "Primitive Trinomials Whose Degree Is a Mersenne Exponent," Information and Control, v. 15, 1969, pp. 67–69.
1649. N. Zierler and J. Brillhart, "On Primitive Trinomials (mod 2)," Information and Control, v. 13, n. 6, Dec 1968, pp. 541–544.
1650. N. Zierler and W.H. Mills, "Products of Linear Recurring Sequences," Journal of Algebra, v. 27, n. 1, Oct 1973, pp. 147–157.
1651. C. Zimmer, "Perfect Gibberish," Discover, v. 13, n. 12, Dec 1992, pp. 92–99.
1652. P.R. Zimmermann, The Official PGP User's Guide, Boston: MIT Press, 1995.
1653. P.R. Zimmermann, PGP Source Code and Internals, Boston: MIT Press, 1995.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Index

A5, 389, 662–667
Abadi, Martin, 66
Absolute rate, of language, 234
Accreditation, 103
Active attacks, 27
Active cheaters, 27
Adams, Carlisle, 334
Adaptive-chosen-plaintext attack, 6
Addition chaining, 244
Additive generators, 390–392
Adjudicated protocol, 26, 71
Adjudicator, 26
Adleman, Leonard M., 163–164, 467
Adler, Roy, 266
Agnew, G. B., 423
Algebraic structure, DES, 282–283
Algorithm M, 393–394
Algorithms, 2–4, 17
all-or-nothing disclosure of secrets, 543–546
Asmuth-Bloom, 529–530
Barrett's, 244
Berlekamp-Massey algorithm, 380, 404
block
chain mode, 206–207
choosing, 354–355
replay, 191–193
breaking, 8
CAST, 334–335
choosing, 214–216
cipher block chaining mode, 193–197, 208–210
cipher block chaining of plaintext difference mode, 208
cipher block chaining with checksum, 207–208
cipher-feedback mode, 200–202, 208–210
cipher mode
choosing, 208–210
summary, 209
classes, 217
coin flipping
using Blum integers, 543
using exponentiation modulo p, 542–543
using square roots, 541–542
complexity, 237–239
constant, 238
convertible undeniable signatures, 538–539
counter mode, 205–206, 209
cubic, 238
data compression, 226
designated confirmer signatures, 539–540
Diffie-Hellman, fair, 546–547
digital signatures, 39
exponential, 238
for export, 215–216
extended Euclidean, 246–248
factoring, 256
ISO/IEC 9979 registered, 607
Karnin-Greene-Hellman, 530
Khafre, 317–318
Khufu, 317
linear, 238
linear syndrome, 381
modes, DES, 277–278
multiple block
cascading, 367–368
combining, 368
multiple-key public-key cryptography, 527–528
oblivious transfer, 550
one-way accumulators, 543
output-feedback mode, 203–205, 208–210
output feedback with a nonlinear function, 208
plaintext block chaining mode, 208
plaintext feedback mode, 208
polynomial, 238
polynomial-time, 238
probabilistic encryption, 552–554
propagating cipher block chaining mode, 207
public-key, 4–5, 33
quadratic, 238
quantum cryptography, 554–557
restricted, 3
running times, 238–239
secret-sharing algorithms, 528–531
secure multiparty computation, 551–552
Algorithms (Cont.)
security, 8–9
self-synchronizing stream cipher, 198–199
stream ciphers, 197–198
subliminal-channel signature, 79
superpolynomial, 238
symmetric, 4
synchronous stream cipher, 202–203
TEA, 346
types, 189
unconditionally secure, 8
undeniable digital signatures, 536–539
using, 213–229
vector scheme, 529
zero-knowledge proofs, 548–550
See also Block ciphers; Stream ciphers
All-or-nothing disclosure of secrets, 96, 543–546
voting with a single central facility, 128–130
Alternating stop-and-go generator, 383, 385, 410–411
American National Standards Institute, DES approval, 267–268
Anderson, Ross, 391
ANDOS, see All-or-nothing disclosure of secrets
Anonymous message broadcast, 137–139
ANSI X3.105, 267
ANSI X3.106, 267
ANSI X9.8, 267
ANSI X9.17, 268, 359
key generation, 175
ANSI X9.19, 267
ANSI X9.26, 268
Arbitrated protocol, 23–26
Arbitration, timestamping, 75–76
Arbitrator, 23
document signing with, 35–37
group signatures with, 84–85
AR hash function, 453
Arithmetic, modular, 242–245
Arms Export Control Act, 610
Asmuth-Bloom scheme, 529–530
Association for Computing Machinery, 608
Asymmetric algorithms, see Public-key algorithms
Atomic Energy Act, 610
Attack, 5
AT&T Model 3600 Telephone Security Device, 594–595
Authentication, 2, 52–56
DASS, 62
Denning-Sacco protocol, 63
dictionary attacks, 52
ISO framework, 574–577
Kerberos, 60
message, 56
Needham-Schroeder protocol, 58–59
Neuman-Stubblebine protocol, 60–62
Otway-Rees protocol, 59–60
protocols, formal analysis, 65–68
salt, 52–53
Schnorr, 511
SESAME, 572
SKEY, 53
SKID, 55–56
using interlock protocol, 54–55
using one-way functions, 52
using public-key cryptography, 53–54
Wide-Mouth Frog protocol, 56–57
Woo-Lam protocol, 63–64
Yahalom, 57–58
Authenticators, 568
Avalanche effect, 273
Backup keys, 181–182
BAN logic, 66–67
Barrett's algorithm, 244
BaseKing, 346
Basis, polarization measurement, 555
Battista, Leon, 11
BBS generator, 417
add to spelled out, 553–554
Beacons, 64
Bellovin, Steve, 518, 520–521, 571
Bennett, Charles, 555, 557
Berlekamp-Massey algorithm, 380, 404
Bernstein, Dan, 616
Berson, Tom, 441
Best affine approximation attack, 381
Beth-Piper stop-and-go generator, 383–384
Bias, 425
Bidirectional message authentication codes, 457
Biham, Eli, 284–285, 288, 296, 301, 303, 306, 308, 311–312, 314, 316, 319, 354, 361, 434
Bilateral stop-and-go generator, 384–385
Binary trees, 78
Biotechnology, as cryptanalysis tool, 156–157
Birthday attack, 165–166, 430
Bit commitment, 86–88
using one-way functions, 87–88
using pseudo-random-sequence generators, 88
using symmetric cryptography, 86–87
Blakley, George, 72, 529
Blaze, Matt, 346, 364
Blinding factor, 112
Blind signatures, 112–115, 549–550
patents, 115
voting with, 126–127
Blobs, 88
Block algorithms, 4
Block chain mode, 206–207
Block ciphers, 4, 189
Blowfish, 336–339
CA-1.1, 327–328
cascading algorithms, 367–368
CAST, 334–335
CDMF key shortening, 366
choosing algorithms, 354–355
combining algorithms, 368
counter mode, 205–206, 209
Crab, 342–344
CRYPTO-MECCANO, 346
designing, 351
design theory, 346–351
Feistel networks, 347
group structure, 348
S-box, 349–351
simple relations, 347–348
strength against differential and linear cryptanalysis, 348–349
weak keys, 348
double encryption, 357–358
double OFB/counter, 363–364
doubling length, 363
electronic codebook mode, 189–191, 208–210
encryption speeds, 355
FEAL, 308–312
feedback, 193
GOST, 331–334
IDEA, 319–325
iterated, 347
Li-Wang algorithm, 346
LOKI, 314–316
Lucifer, 303–304
Madryga, 304–306
McEliece algorithm, 346
MMB, 325–327
multiple encryption, 357
NewDES, 306–308
Rao-Nam algorithm, 346
RC2, 318–319
RC5, 344–346
REDOC II, 311–313
REDOC III, 313
SAFER K-64, 339–341
security, based on one-way hash functions, 353–354
Skipjack, 328–329
versus stream ciphers, 210–211
SXAL8/MBAL, 344
triple encryption, 358–363
3–Way, 341–342
using one-way hash functions, 351–354
whitening, 366–367
xDES1, 365–366
Block length, doubling, 363
Block replay, 191–193
Blocks, 4
Blowfish, 336–339, 354, 647–654
Blum, Manuel, 89, 105, 108
Blum, Blum, and Shub generator, 417–418
Blum integers, 253
coin flipping, 543
zero-knowledge proofs, 549
Blum-Micali generator, 416–417
Boolean functions, in S-boxes, 350
Bosselaers, Antoon, 436, 441
Boyar, Joan, 369
Brassard, Gilles, 555, 557
Broadcasting:
anonymous, 137–139
secret, 523–524
Brute-force attack, 8, 151–152
software-based, 154–155
time and cost estimates, 152–154
Bureau of Export Administration, 610–611
Burrows, Michael, 66
CA-1.1, 327–328
Cade algorithm, 500–501
Caesar Cipher, 11
CAFE, 606–607
CALC, 346
Cantwell Bill, 615–616
Capstone, 593–594
Cascade generators, 405
Cascades, Gollmann, 387–388
Cascading:
multiple block algorithms, 367–368
multiple stream ciphers, 419–420
Cash, digital, see Digital cash
Cassells, Ian, 381
CAST, 334–335
S-boxes, 349
CBC, see Cipher block chaining mode
CCEP, 269, 598–599
CDMF, 366, 574
Cellhash, 446
Cellular automata, 500
Cellular automaton generator, 414
Certificates:
Privacy-Enhanced Mail, 579
public-key, 185–187
X.509, 574–575
Certification authority, 186
Certification path, 576
Certified mail, digital, 122–123
Chaining variables, 436
Chambers, Bill, 385–386
Characteristics, 286–288
Chaum, David, 84, 115, 133, 137, 536, 549
Cheater, 27
sharing secrets with, 531
Chess Grandmaster Problem, 109
Chinese Lottery, 156–157
Chinese remainder theorem, 249–250, 470
Chor-Rivest knapsack, 466
Chosen-ciphertext attack, 6–7, 471–472
Chosen-key attack, 7
Chosen-plaintext attack, 6–7, 359
Chosen-text attack, 7
Cipher:
substitution, 10–12
transposition, 12
Cipher block chaining mode, 193–197, 208–210
DES, 277–278
error extension, 196
error propagation, 195–196
initialization vector, 194
message authentication codes, 456
padding, 195
security, 196–197
self-recovering, 196
triple encryption, 360–361
Cipher block chaining of plaintext difference mode, 208
Cipher block chaining with checksum, 207–208
Cipher-feedback mode, 200–202, 208–210
DES, 277
error propagation, 201–202
initialization vector, 201
Cipher mode:
choosing, 208–210
summary, 208–210
Ciphertext, 1–2
auto key, 198
hiding in ciphertext, 227–228
pairs, differential cryptanalysis, 285
stealing, 191
Ciphertext-only attack, 5–6
Cleartext, see Plaintext
Clipper chip, 591–593
Clipper key-escrow, 328
Clipper phone, 594
Clock-controlled generators, 381
Clocking, 381
CoCom, 610
Code, 9
Coefficients, solving for, 248
Coin flipping, 89–92
fair, 541–543
into a well, 92
key generation, 92
using Blum integers, 543
using one-way functions, 90
using public-key cryptography, 90–91
using square roots, 541–542
Collision, 166
Collision-free, 30
Collision-resistance, 429
Combination generator, 381
Combining function, 381
Commercial COMSEC Endorsement Program, 269, 598–599
Commercial Data Masking Facility, 366, 574
Common Cryptographic Architecture, 573–574
Common modulus, dangers of, 493
Common modulus attack, RSA, 472
Communications:
using public-key cryptography, 31–34
using symmetric cryptography, 28–29
Communications channels, encryption, 216–220
Communications Setup, 517–518
Complementation property, 281
Complement keys, DES, 281–282
Completely blind signatures, 112–113
Complete set of residues, 242
Complexity-theoretic approach, stream ciphers, 415–418
Complexity theory, 237–242
algorithms, 237–239
complexity of problems, 239–241
Compression, 226
Compression function, 431
Compression permutation, 273–274
Compromise, 5
Compromised keys, 182–183
Computational complexity, 237
Computationally secure, 8
Computer algorithms, 17
Computer clock, as random-sequence generator, 424
Computer Security Act of 1987, 600–601
Computing, with encrypted data, 85–86, 540–541
COMSET, 517–518
Conditional Access for Europe, 606–607
Conference key distribution, 524
Confusion, 237, 346–347
Congruent, 242
Connection integer, 403
feedback with carry shift registers, maximal-period, 406–407
Continued fraction algorithm, 256
Contract signing, simultaneous:
with an arbitrator, 118
without an arbitrator
face-to-face, 118–119
not face-to-face, 119–120
using cryptography, 120–122
Control Vector, 180
Convertible undeniable signatures, 538–539
Coppersmith, Don, 94, 266, 280, 283, 293, 398, 457
Coppersmith's algorithm, 263
Correlation attack, 380
Correlation immunity, stream ciphers, 380
Correlations, random-sequence generators, 425
Counter mode, 205–206, 209
Counting coincidences, 14
Crab, 342–344
Credit cards, anonymous, 147
Crepeau, Claude, 555
Crypt(1), 414
CRYPT(3), 296
Cryptanalysis, 1, 5–8
differential, see Differential cryptanalysis
FEAL, 311–312
GOST, 333–334
IDEA, 323
linear, 290–293
LOKI91, 316
Madryga, 306
N-Hash, 434–435
related-key, 290
Snefru, 432
types, 5–7
Cryptanalysts, 1
Crypt Breakers Workbench, 414
Cryptographers, 1
Cryptographic algorithm, see Cipher
Cryptographically secure pseudo-random, 45
Cryptographic facility, 562
Cryptographic mode, 189
Cryptographic protection, databases, 73–74
Cryptographic protocol, 22
Cryptography, 1
CRYPTO-LEGGO, 414
Cryptologists, 1
Cryptology, 1
CRYPTO-MECCANO, 346
Cryptosystems, 4
fair, 97
finite automaton public-key, 482
hybrid, 32–34
security, 234–235
weak, 97
Cusick, Thomas, 312
Cut and choose, 103
Cypherpunks, 609
Daemen, Joan, 325, 341, 349, 414
Damgard, Ivan, 446
Damm, Arvid Gerhard, 13
Data, encrypted:
computing with, 85–86, 540–541
discrete logarithm problem, 540–541
for storage, 220–222
Databases, cryptographic protection, 73–74
Data complexity, 9
Data Encryption Algorithm, see Data Encryption Standard
Data Encryption Standard, 17, 265–301
adoption, 267–268
algorithm, brute-force attack efficiency, 152–153
characteristics, 286–288
commercial chips, 279
compared to GOST, 333–334
compression permutation, 273–274
CRYPT(3), 296
decryption, 277
description, 270
DESX, 295
development, 265–267
differential cryptanalysis, 284–290
DES variants, 298
expansion permutation, 273–275
final permutation, 277
generalized, 296–297
hardware and software implementation, 278–279
with independent subkeys, 295
initial permutation, 271
iterated block cipher, 347
key transformation, 272–273
linear cryptanalysis, 290–293
modes, 277–278
multiple, 294–295
1987 review, 268–269
1993 review, 269–270
outline of algorithm, 270–272
P-boxes
design criteria, 294
permutation, 275, 277
RDES, 297–298
related-key cryptanalysis, 290
RIPE-MAC, 457–458
S-boxes, 349
alternate, 296–298
design criteria, 294
key-dependent, 298, 300, 354
substitution, 274–276
security, 278, 280–285
algebraic structure, 282–283
complement keys, 281–282
current, 300–301
key length, 283–284
number of rounds, 284
possibly weak keys, 281–282
S-box design, 284–285
semiweak keys, 280–281
weak keys, 280–281
snDES, 298–299
source code, 623–632
speeds on microprocessors and computers, 279
validation and certification of equipment, 268
Data Exchange Key, 581
Data Keys, 176
Davies, Donald, 562
Davies-Meyer, 448
abreast, 452
modified, 449–450
parallel, 451
tandem, 451–452
Davies-Price, 358
Decoherence, 165
Decryption, 1
DES, 277
key, 3
key-error detection, 179
knapsack algorithms, 465
with a public key, 39
with symmetric algorithm, 4
den Boer, Bert, 434, 436, 441
Denning-Sacco protocol, 63
Dense, 378
Dereferencing keys, 221–222
Derived sequence attack, 381
Designated confirmer signatures, 82–83, 539–540
Desmedt, Yvo, 81
DES, see Data Encryption Standard
Destruction:
information, 228–229
of keys, 184–185
DESX, 295
Dictionary attack, 52, 171–173
Differential cryptanalysis, 284–290
attacks against
DES, 288–290
DES variants, 298
Lucifer, 303
extending to higher-order differentials, 293
strength against, block cipher design theory, 348–349
Differential-linear cryptanalysis, 293
Diffie, Whitfield, 31, 37, 122, 216, 283, 419, 461, 501, 565
Diffie-Hellman:
EKE implementation, 519–520
extended, 515
failsafe, 547–548fair, 546–547
Hughes variant, 515
key exchange without exchanging keys, 515
patents, 516
with three or more parties, 514
Diffie's randomized stream cipher, 419
Diffusion, 237, 346–347
Digital card, properties, 146
Digital cash, 139–147
anonymous, 139
credit cards, 147
money orders, 140
double spending problem, 140–141
off-line systems, 146
on-line systems, 145–146
other protocols, 145–147
perfect crime, 145
practical, 145
secret splitting, 142–145
Digital certified mail, 122–123
Digital Notary System, 78
Digital Signature Algorithm, 17, 483–494
attacks against k, 492
computation time comparison with RSA, 489
criticisms, 484–486
dangers of common modulus, 493
description, 486–488
ElGamal encryption with, 490–491
patents, 493–494
prime generation, 488–490
proposal for NIST standard, 483–486
RSA encryption with, 491
security, 491–492
speed precomputations, 487–488
subliminal channel, 493, 534–536
foiling, 536
variants, 494–495
Digital signatures, 34–41
algorithms, 39
applications, 41
blind, 112–115, 549–550
convertible undeniable signatures, 538–539
converting identification schemes to, 512
definition, 39
designated confirmer signatures, 82–83, 539–540
ElGamal, 476–478
with encryption, 41–44
entrusted undeniable, 82
fail-stop, 85
Fiat-Shamir signature scheme, 507–508
group signatures, 84–85
Guillou-Quisquater signature scheme, 509–510
improved arbitrated solution, 76
key exchange with, 50
multiple, 39–40
Guillou-Quisquater, 510
nonrepudiation, 40
oblivious, 117
protocol, 40
proxy, 83
public-key algorithms, 483–502
Cade algorithm, 500–501
cellular automata, 500
Digital Signature Algorithm, see Digital Signature Algorithm
discrete logarithm signature schemes, 496–498
ESIGN, 499–500
GOST digital signature algorithm, 495–496
Digital signatures (Cont.)
public-key algorithms (Cont.)
Matsumoto-Imai algorithm, 500
Ong-Schnorr-Shamir, 498–499
public-key cryptography, 37–38
attacks against, 43–44
one-way hash functions and, 38–39
resend attack, foiling, 43
RSA, 473–474
Schnorr signature scheme, 511–512
subliminal-free, 80
with symmetric cryptosystems and arbitrator, 35–37
terminology, 39
timestamps, 38
trees, 37
undeniable, 81–82, 536–539
Dining Cryptographers Problem, 137
Discrete logarithm, 245
in finite field, 261–263
zero-knowledge proofs, 548
Discrete Logarithm Problem, 501, 540–541
Discrete logarithm signature schemes, 496–498
Distributed Authentication Security Service, 62
Distributed convertible undeniable signatures, 539
Distributed key management, 187
DNA computing, 163–164
DNRSG, 387
DoD key generation, 175
Double encryption, 357–358
Double OFB/counter, 363–364
Double spending problem, 140–141
Driver-level encryption, 222–223
DSA, see Digital Signature Algorithm
Dynamic random-sequence generator, 387
E-box, 273
ECB, see Electronic codebook mode
Electronic checks, 146
Electronic codebook mode, 189–191, 208–210
combined with OFB, 364
DES, 277–278padding, 190–191
triple encryption, 362–363
Electronic coins, 146
Electronic Frontier Foundation, 608
Electronic-funds transfer, DES adoption, 268
Electronic Privacy Information Center, 608
ElGamal, 532–533
EKE implementation, 519
encryption, 478
with DSA, 490–491
patents, 479
signatures, 476–478
speed, 478–479
ElGamal, Taher, 263
Elliptic curve cryptosystems, 480–481
Elliptic curve method, 256
Ellison, Carl, 362
Encoding, 226
Encrypt-decrypt-encrypt mode, 359
Encrypted Key Exchange:
applications, 521–522
augmented, 520–521
basic protocol, 518–519
implementation with
Diffie-Hellman, 519–520
ElGamal, 519
RSA, 519
strengthening, 520
Encryption, 1
communication channels, 216–220
combining link-by-link and end-to-end, 219–221
with compression and error control, 226
data, for storage, 220–222
detection, 226–227
digital signatures with, 41–44
driver-level versus file-level, 222–223
ElGamal, 478
with DSA, 490–491
end-to-end, 217–220
with interleaving, 210–211
key, 3
knapsack algorithms, 464
link-by-link, 216–218
multiple, 357
with a private key, 39
probabilistic, 552–554
RSA, 468
with DSA, 491
with symmetric algorithm, 4
using public key, 5
End-to-end encryption, 217–220
combined with link-by-link, 219–221
Enigma, 13, 414
Entropy, 233–234
Entrusted undeniable signature, 82
Error detection:
during decryption, 179
during transmission, 178
Error extension, cipher block chaining mode, 196
Error propagation:
cipher block chaining mode, 195–196
cipher-feedback mode, 201–202
output-feedback mode, 204
Escrow agencies, 592
Escrowed Encryption Standard, 97, 593
ESIGN, 499–500, 533–534
Euclid's algorithm, 245
Euler totient function, 248–249
Expansion permutation, 273–275, 315
Export:
of algorithms, 215–216, 610–616
foreign, 617
Exportable Protection Device, 389
Export Administration Act, 610
EXPTIME, 241
Extended Euclidean algorithm, 246–248
Factoring, 255–258
general number field sieve, 159–160
long-range predictions, 162
public-key encryption algorithms, 158–159
special number field sieve, 160–161
using quadratic sieve, 159
Factoring Problem, 501
Failsafe:
Diffie-Hellman, 547–548
key escrowing, 98
Fail-stop digital signatures, 85
Fair cryptosystems, 97
Fait-Shamir, 508
FAPKC0, 482
FAPKC1, 482
FAPKC2, 482
FEAL, 308–312
cryptanalysis, 311–312
description, 308–10
patents, 311
Feedback:
cipher block chaining mode, 193, 195
internal, output-feedback mode, 203
Feedback function, 373
Feedback shift register, 373
Feedback with carry shift registers, 402–404
combining generators, 405, 410
maximal-length, tap sequences, 408–409
maximal-period, connection integers, 406–407
Feedforward, cipher block chaining mode, 195
Feige, Uriel, 503–504
Feige-Fiat-Shamir, 503–508
enhancements, 506–507
identification scheme, 504–505
simplified, 503–504
Feistel, Horst, 266, 303
Feistel network, 347
Blowfish, 337
practically secure, 349
Fermat's little theorem, 248
Euler's generalization, 248
FFT-Hash, 446
Fiat, Amos, 503–504
Fiat-Shamir signature scheme, 507–508
Fibonacci configuration, 373, 379
Fibonacci shrinking generator, 391
File-level encryption, 222–223
Filter generator, 381
Finite field, 254
discrete logarithms, 261–263
FIPS PUB 46, 267
FIPS PUB 74, 267
FIPS PUB 81, 267
FIPS PUB 112, 267
Fish, 391
Fixed bit index, 543
Flat keyspace, 176
Flipping coins, see Coin flipping
Fortified key negotiation, 522
Galois configuration, linear feedback shift registers, 378–379
Galois field, computing in, 254–255
Garey, Michael, 241
Gatekeeper, 278
Geffe generator, 382–383
General number field sieve, 159–160, 256
General Services Administration, DES adoption, 268
Generators, 253–254
Gifford, 392–393
Gifford, David, 392
Gill, J., 501
Global deduction, 8
Goldwasser, Shafi, 94, 552
Gollmann, Dieter, 386
Gollmann cascade, 387–388
Goodman-McAuley cryptosystem, 466
Goresky, Mark, 404
GOST, 331–334, 354
source code, 643–647
GOST digital signature algorithm, 495–496
GOST hash function, 454
GOST R 34.10–94, 495
Gosudarstvennyi Standard Soyuza SSR, 331–334
Graham-Shamir knapsacks, 465
Graph isomorphism, 104–105
Greatest common divisor, 245–246
Grossman, Edna, 266
Group signatures, 84–85
Group Special Mobile, 389
Group structure, block ciphers design theory, 348
GSM, 389
Guillou, Louis, 102, 508
Guillou-Quisquater:
identification scheme, 508–510
signature scheme, 509–510
Gutmann, Peter, 353
Guy, Richard, 159
Haber, Stuart, 75, 485, 488
Hamiltonian cycles, 105–106
Hard drive, encrypted, providing random access to, 222
Hardware:
DES implementation, 278–279
encryption, 223–225
RSA, 469
Hash functions, see One-way hash functions
Hash value, 30
HAVAL, 445–446
Hellman, Martin, 31–32, 37, 262, 283, 293, 358–359, 461–462
Hiding information from an oracle, 86
Historical terms, 9
Homophonic substitution cipher, 10–11
Hughes, 515
Hughes, Eric, 609
Hughes XPD/KPD, 389–390
Hybrid cryptosystems, 32–34, 461
IBC-Hash, 458
IBM Common Cryptographic Architecture, 573–574
IBM secret-key management protocol, 561–562
IDEA, 319–325, 354
cryptanalysis, 323
description, 320–322
modes of operation, 323–325
overview, 320–321
patents, 325
S-boxes, 349
source code, 637–643
speed, 322–323
strength against differential cryptanalysis, 348
variants, 325
Ideal secrecy, 236
Identification schemes:
converting to signature schemes, 512
Feige-Fiat-Shamir, 503–508
Guillou-Quisquater, 508–510
Ohta-Okamoto, 508
Schnorr authentication and signature scheme, 510–512
Identity-based cryptosystems, 115
Ignition key, 564
Import, foreign, 617
Index of coincidence, 14
Information:
amount, information theory definition, 233
deduction, 8
destruction, 228–229
Information-theoretic approach, 418
stream ciphers, 415
Information theory, 233–237
cryptosystem security, 234–235
entropy and uncertainty, 233–234
in practice, 236–237
rate of the language, 234
unicity distance, 235–236
Ingemarsson, Ingemar, 418
Initialization vector:
cipher block chaining mode, 194
cipher-feedback mode, 201
output-feedback mode, 204
Inner-CBC, 360, 363
Insertion attack, synchronous stream ciphers, 203
Instance deduction, 8
Institute of Electrical and Electronics Engineers, 608
Integrated Services Digital Network, 563–565
Integrity, 2
Interactive protocol, 103
Interchange Key, 581
Interleave, 210–211
Interlock protocol, mutual authentication using, 54–55
Internal feedback, 203
International Association for Cryptologic Research, 605
International Standards Organization:
authentication framework, 574–577
DES adoption, 268
International Traffic in Arms Regulations, 610–614
Internet, Privacy-Enhanced Mail, 577–584
Introducers, 187
Inverses modulo a number, 246–248
IPES, 319
ISDN, 563–565
ISO 8732, 359
ISO 9796, 472, 474, 486
ISO/IEC 9979, 607
ISO X.509 protocols, 574–577
Iterated block cipher, 347
Jacobi symbol, 252–253
J-algebras, 501
Jam, 414
Jennings generator, 383–384
Johnson, David, 241
Jueneman's methods, 457
Kaliski, Burt, 342
Karn, 351–352
Karn, Phil, 351
Karnin-Greene-Hellman, 530
Kerberos, 60, 566–571
abbreviations, 567
authentication steps, 567
credentials, 568
getting initial ticket, 569
getting server tickets, 569–570
licenses, 571
model, 566
requesting services, 570
security, 571
Version 4, 570–571
Version 5 messages, 568
Kerckhoffs, A., 5
Kerckhoffs's assumption, 7
Key, 3
backup, 181–182
CDMF shortening, 366
complement, DES, 281–282
compromised, 182–183
controlling usage, 180
dereferencing, 221–222
destroying, 184–185
distribution in large networks, 177
generating, 170–175
ANSI X9.17 standard, 175
DoD, 175
pass phrases, 174–175
poor choices, 171–173
random keys, 173–174
reduced keyspaces, 170–171
ISDN, 563–564
lifetime, 183–184
possibly weak, DES, 281–282
semiweak, DES, 280–281
session, 33, 180
storing, 180–181
transferring, 176–177
transmission, error detection, 178
updating, 180
using, 179–180
verification, 178–179
weak
block ciphers design theory, 348DES, 280–281
Key and message broadcast, 51–52
Key and message transmission, 51
Key Auto-Key, 202
Keyboard latency, as random-sequence generator, 424–425
Key Certification Authority, 43
Key control vectors, 562
Key distribution:
anonymous, 94–95
conference, 524
Key Distribution Center, 43–44
Key-Encryption Keys, 176, 184
Key escrow, 97–100, 181–182, 591
politics, 98–100
Key exchange, 47–52
DASS, 62
Denning-Sacco protocol, 63
with digital signatures, 50
interlock protocol, 49–50
Kerberos, 60
key and message broadcast, 51–52
key and message transmission, 51
man-in-the-middle attack, 48–49
Needham-Schroeder protocol, 58–59
Neuman-Stubblebine protocol, 60–62
Otway-Rees protocol, 59–60
protocols, formal analysis, 65–68
with public-key cryptography, 48
with symmetric cryptography, 47–48
Wide-Mouth Frog protocol, 56–57
without exchanging keys, 515
Woo-Lam protocol, 63–64
Yahalom, 57–58
Key-exchange algorithms:
COMSET, 517–518
conference key distribution and secret broadcasting, 523–525
Diffie-Hellman, 513–516
Encrypted Key Exchange, 518–522
fortified key negotiation, 522
Shamir's three-pass protocol, 516–517
station-to-station protocol, 516
Tatebayashi-Matsuzaki-Newman, 524–525
Key generation, using coin flipping, 92
Key length:
comparing symmetric and public-key, 165–166
deciding on, 166–167
DES, 283–284
public-key, 158–165
DNA computing, 163–164
quantum computing, 164–165
recommended lengths, 161–163
symmetric, 151–158
biotechnology as cryptanalysis tool, 156–157
brute-force attack, 151–154
Chinese Lottery, 156–157
neural networks, 155
software-based brute-force attacks, 154–155
thermodynamic limitations on brute-force attacks, 157–158
using viruses to spread cracking program, 155–156
Key management, 169–187
distributed, 187
public-key, 185–187
Key negotiation, fortified, 522
Key notarization, 562
Key revocation certificate, 585
Keyspace, 3
flat, 176
nonlinear, 175–176
reduced, 170–171
Keystream generator, 197–198
counter mode, 206
periodic, 202
Khafre, 317–318, 349
Khufu, 317, 349
Kilian, Joe, 116
Kim, Kwangjo, 298, 350
Kinetic Protection Device, 389–390
Klapper, Andy, 404
Klein, Daniel, 53, 171
Knapsack algorithms, 462–466
decryption, 465
encryption, 464
implementations, 465
patents, 466
public key created from private key, 464
security, 465
superincreasing, 463–464
variants, 465–466
Knapsack problem, 501
Known-plaintext attack, 6–7, 151, 359
Knudsen, Lars, 8, 293, 314, 316, 348–349
Knuth, 393, 501
Koblitz, Neal, 480
Konheim, Alan, 266, 280
Kravitz, David, 493
Kravitz-Reed, 481
KryptoKnight, 571–572
Lagged Fibonacci generators, 390
LaGrange interpolating polynomial scheme, 528–529
Lai, Xuejia, 319, 449
Langford, Susan, 293
Law Enforcement Access Field, 591
Legal issues, 618
Legendre symbol, 251
Lehmann, 259
Lehmann algorithm, 259
Length, shift register, 373
Lenstra, Arjen, 159, 162, 257, 485, 488
LFSR/FCSR summation/parity cascade, 410–411
Lidl, Rudolph, 481
Linear complexity:
profile, 380
stream ciphers, 380
Linear congruential generators, 369–372
combining, 371–372
constants, 370
Linear consistency test, 381
Linear cryptanalysis:
DES, 290–293
strength against, block cipher design theory, 348–349
Linear error-correcting codes, algorithms based on, 480
Linear feedback shift registers, 372–379
Galois, 378–379
primitive polynomials mod 2, 376–377
software, 378–379
stream ciphers using, see Stream ciphers
Linear syndrome algorithm, 381
Link-by-link encryption, 216–218
combined with end-to-end, 219–221
Linking protocol, timestamping, 76–77
Li-Wang algorithm, 346
Local deduction, 8
Lock-in, 388
Logarithms, discrete, see Discrete logarithm
LOKI, 314–316
S-boxes, 349
source code, 632–637
LOKI Double-Block, 451
Low decryption exponent attack, RSA, 473
Low encryption exponent attack, RSA, 472–473
Luby, Michael, 352
Luby-Rackoff, 352–353
xDES1, 365
LUC, 481
Lucas number, 481
Luccio-Mazzone, 501
Lucifer, 266, 303–304
Lu-Lee cryptosystem, 466
Lyndon words, 501
MacGuffin, 346
Madryga, W. E., 304
Mafia Fraud, 110
Magic numbers, 423
Manasse, Mark, 159, 257
Man-in-the-middle attack, 48–49
Masks, REDOC II, 312
Massey, James, 319, 339, 386, 418, 449
Master Key, 561
Master Terminal Key, 561
Matsui, Mitsuru, 290–291
Matsumoto-Imai algorithm, 500
Mauborgne, Joseph, 15
Maurer, Ueli, 419
Maurer's randomized stream cipher, 419
Maximal period generator, 369
MBAL, 344
McEliece, Robert, 479
McEliece algorithm, 346, 479–480
MD2, 441
MD3, 446
MD4, 435–436
MD5, 436–441
MDC, 353–354
MDC-2, 452–453
MDC-4, 452–454
MD-strengthening, 431
Meet-in-the-middle attack, 358, 381
Mental poker, 92–95
Merkle, Ralph, 34, 316–318, 358–359, 432, 455, 461–462
Merkle's puzzles, 34
Merritt, Michael, 67, 518, 520–521, 571
Message:
authentication, 56
broadcasting, 69
Privacy-Enhanced Mail, 579–582
recovery, 497–498
resending as receipt, 42–43
Message authentication codes, 31, 455–459
bidirectional, 457
CBC-MAC, 456
IBC-Hash, 458
Jueneman's methods, 457
message authenticator algorithm, 456–457
one-way hash functions as, 458–459
RIPE-MAC, 457–458
stream ciphers, 459
Message authenticator algorithm, 456–457
Message broadcast, anonymous, 137–139
Message Digest, 435–436
Message Digest Cipher, 353
Message Integrity Check, 578
Message-meaning rule, 66
Message Security Protocol, 584
Meyer, Carl, 266, 278
Meyer, Joseph A., 614
Meyer-Schilling, 452
Micali, Silvio, 94, 508, 546–547, 552
Miller, Gary, 259
Miller, V. S., 480
Mimic functions, 10
Minimum-disclosure proofs, 108
MITRENET, 562–563
Miyaguchi, Shoji, 308
MMB, 325–327
m*n-bit S box, 349
Modular arithmetic, 242–245
Modular Multiplication-based Block cipher, 325–327
Modular reduction, 242
Modulo, inverses, 246–248
Monoalphabetic cipher, 10
Montgomery's method, 244
Moore's Law, 153
m-sequence, 374
MSP, 584
Muller, Winfried, 481
Multiparty unconditionally secure protocols, 137
Multiple-bit generator, 421
Multiple encryption, 357
quintuple, 366
Multiple Identity Fraud, 111
Multiple-key public-key cryptography, 527–528
Multiple signatures, 39–40
Multiplier, 369
Multispeed inner-product generator, 386–387
Mush, 392
Mutual shrinking generator, 392
MYK-80, 593–594
Mykotronx Clipper chip, 328
MYK-78T, 591–593
Nanoteq, 390
National Bureau of Standards, see National Institute of Standards and Technology
National Computer Security Center, 599–600
National Institute of Standards and Technology, 600–603
DES development, 265–267
Memorandum of Understanding, 601–603
National Security Agency, 597–599
DES development, 266–267
export of cryptography, 614–615
Memorandum of Understanding, 601–603
S-box development role, 278, 280
Navy Research Laboratory, protocol analyzer, 67–68
Needham, Roger, 58, 66, 216
Needham-Schroeder protocol, 58–59
Networks, large, key distribution, 177
Neuman-Stubblebine protocol, 60–62
Neural networks, breaking algorithms, 155
NewDES, 306–308
N-Hash, 433–435
Niederreiter, Harald, 501
Niederreiter algorithm, 480
Niemi cryptosystem, 466
Nobauer, Wilfried, 481
Noise, random, using as random-sequence generator, 423–424
Nonce-verification rule, 66
Non-Interactive Key Sharing systems, 115
Nonlinear-feedback shift registers, 412–413
Nonlinear keyspace, 175–176
Nonrepudiation, 2
Notz, Bill, 266
NP-complete problem, 240–242
graph isomorphism, 104
knapsack algorithms, 462
McEliece algorithm, 479
solving, 163–164
NRL Protocol Analyzer, 67–68
NSDD-145, 268
Nuclear Non-Proliferation Act, 610
Number field sieve, 256
Numbers:
2–adic, 404
large, 17–18
Number theory, 242–255
Barrett's algorithm, 244
Blum integers, 253
Chinese remainder theorem, 249–250
Euclid's algorithm, 245
Euler totient function, 248–249
extended Euclidean algorithm, 246–248
Fermat's little theorem, 248
Galois field, computing in, 254–255
generators, 253–254
greatest common divisor, 245–246
inverses modulo a number, 246–248
Jacobi symbol, 252–253
Legendre symbol, 251
modular arithmetic, 242–245
Montgomery's method, 244
prime numbers, 245
quadratic residues, 250–251
solving for coefficients, 248
Nyberg, Kaisa, 348
Oblivious transfer, 116–117, 550
Oblivous signatures, 117
OFB, see Output-feedback mode
Ohta, Kazuo, 146, 501
Ohta-Okamoto identification scheme, 508
Okamoto, Tatsuaki, 146, 501
1/p generator, 414
One-time pad, 15–17
hiding ciphertext in ciphertext, 227–228
One-time tape, 418
One-way accumulators, 95–96, 543
One-way function, 29–30
authentication using, 52
bit commitment using, 87–88
coin flipping using, 90
trap-door, 158
One-way hash functions, 30–31, 351–354
background, 429–431
birthday attacks, 165–166, 430
choosing, 455
cipher security, 353–354
compression function, 431
encryption speeds, 456
HAVAL, 445–446
improved arbitrated solution, 76
Karn, 351–352
length, 430–431
Luby-Rackoff, 352–353
MD2, 441
MD3, 446
MD4, 435–436
MD5, 436–441
MD-strengthening, 431
message authentication codes, 455–459
Message Digest Cipher, 353–354
multiple signatures, 40
N-Hash, 433–435
RIPE-MD, 445
Secure Hash Algorithm, 442–445signing documents with, 38–39
Snefru, 432
as unbiased random-bit generator, 107
using public-key algorithms, 455
using symmetric block algorithms, 446–455
AR hash function, 453
GOST hash function, 454
hash length equals block size, 447–449
LOKI Double-Block, 451
MDC-2 and MDC-4, 452–454
modified Davies-Meyer, 449–450
parallel Davies-Meyer, 451
Preneel-Bosselaers-Govaerts-Vandewalle, 450
Quisquater-Girault, 450
tandem and abreast Davies-Meyer, 451–452
Ong-Schnorr-Shamir, 498–499, 531–532
Orange Book, 599–600
Otway-Rees protocol, 59–60
Outerbridge, Richard, 363
Outer-CBC, 360
Output-feedback mode, 203–205, 208–210
combined with ECB, 364
DES, 277
with a nonlinear function, 208
Overtake, 598
Overwriting, 229
Padding:
cipher block chaining mode, 195
electronic codebook mode, 190–191
MD5, 436
Secure Hash Algorithm, 442
triple encryption with, 362
Painvin, Georges, 12
Pass phrases, 174–175
Passive attack, 27
Passive cheaters, 27
Patents, 609–610; See also specific algorithms
P-boxes:
design criteria, 294
permutation, 275, 277, 316
PEM, see Privacy-Enhanced Mail
Perfect secrecy, 235
Period, 11
shift register, 373
Permutation, 237
key, DES, 272–273
PES, 319, 324
Pike, 391–392
PKZIP, 394–395
Plaintext, 1–2
Plaintext block chaining mode, 208
Plaintext feedback mode, 208
Plaintext pair, right and wrong pairs, 287
Pless generator, 413–414
p-NEW scheme, 498
Pohlig, Stephen, 262
Pohlig-Hellman encryption scheme, 474
Polarized photons, 555
Pollard's Monte Carlo algorithm, 256
Polyalphabetic substitution cipher, 10–11
Polygram substitution cipher, 10–11
Polynomials:
degree, shift register length, 374
dense, 378
irreducible, 255, 481
sparse, 378
Pomerance, Carl, 257
Powerline System, 466
Pre-image, 30
Preneel, Bart, 457
Preneel-Bosselaers-Govaerts-Vandewalle, 450
Pretty Good Privacy, 584–587
Price, William, 562
Prime numbers, 245
generation, 258–261
DSA, 488–490
practical considerations, 260–260
relatively prime, 245
strong, 261
Primitive, 253
Principal square root, 251
Privacy-Enhanced Mail, 577–584
certificates, 579
documents, 578
messages, 579–582
RIPEM, 583–584
security, 582–583
TIS/PEM, 583
Private key, 5
creating public key from, 464
for public-key cryptography, lifetime, 184
Probabilistic encryption, 552–554
Problems:
complexity, 239–241
EXPTIME, 241
hard, 239
intractable, 239
PSPACE, 241
Problems (Cont.)
tractable, 239
undecidable, 240
See also NP-complete problem
Processing complexity, 9
Product cipher, 347
Proofs of Membership, 111
Propagating cipher block chaining mode, 207
Proposed Encryption Standard, 319
Protocols, 21, 47
adjudicated, 26, 70–71
all-or-nothing disclosure of secrets, 96
analysis, approaches, 65–66
anonymous message broadcast, 137–139
arbitrated, 23–26
attacks against, 27
authentication, 576–577
authentication and key-exchange, formal analysis, 65–68
BAN logic, 66–67
basic zero-knowledge, 102–104
bit commitment, 86–88
blind signatures, 112–115
characteristics, 21
cryptographic, 22
DASS, 62
definition, 21
Denning-Sacco, 63
digital cash, see Digital cash
digital certified mail, 122–123
digital signatures, 40
distributed, timestamping, 77–78
fair coin flips, 89–92
IBM Common Cryptographic Architecture, 573–574
IBM secret-key management, 561–562
identity-based public-key cryptography, 115
interactive, 103
interlock, 49–50, 54–55
Kerberos, 60, 566–571
key escrow, 97–100
key exchange, 47–52
KryptoKnight, 571–572
lessons, 64–65
mental poker, 92–95
multiparty unconditionally secure, 137
Needham-Schroeder, 58
Neuman-Stubblebine, 60–62
oblivious signatures, 117
oblivious transfer, 116–117
one-way accumulators, 95–96
Otway-Rees, 59–60
purpose, 22–23
secret splitting, 70–71
secure circuit evaluation, 137
secure elections, see Secure elections
secure multiparty computation, 134–137
self-enforcing, 26–27
SESAME, 572
simultaneous contract signing, 118–122
simultaneous exchange of secrets, 123–124
subliminal channel, 79–80
timestamping, 75–79
types, 24
Wide-Mouth Frog, 56–57
Woo-Lam, 63–64
Yahalom, 57–58
See also Authentication; Zero-knowledge proofs
Pseudo-Hadamard Transform, 340
Pseudo-random function family, SEAL, 398–399
Pseudo-random-number generator, 78, 416
Pseudo-random sequence, 44–45
Pseudo-random-sequence generator, 44
bit commitment using, 88
generating multiple streams, 420–421
linear congruential generators, 369–372
linear feedback shift registers, 372–379
PSPACE, 241
Public key, 5
certificates, 185–187
creating from private key, 464
key length, 158–165
recommended lengths, 161–163
key management, 185–187
Public-key algorithms, 4–5, 33, 500–502
background, 461–462
based on linear error-correcting codes, 480
Diffie-Hellman, 513
ElGamal, 476–479
elliptic curve cryptosystems, 480–481
finite automaton cryptosystems, 482
knapsack algorithms, 462–466
LUC, 481
McEliece, 479–480
one-way hash functions using, 455
Pohlig-Hellman, 474
Rabin, 475–476
RSA, see RSA
security, 461–462
strength, 502
Public-key cryptography:
attacks against, 43–44
authentication using, 53–54
coin flipping using, 90–91
communications using, 31–34
identity-based, 115
key exchange with, 48
multiple-key, 68–69
private keys, lifetime, 184
signing documents with, 37–38
one-way hash functions, 38–39
versus symmetric cryptography, 216–217
Public-Key Cryptography Standards, 588–589
Public Key Partners, 604–605
Public-key ring, 585
Purchase-key attack, 7
Quadratic nonresidues, 251
Quadratic residues, 250–251
generator, 417
Quadratic sieve, 256
factoring, 159
Quantum computing, 164–165
Quantum cryptography, 554–557
Quintuple encryption, 366
Quisquater, Jean-Jacques, 102, 508
Quisquater-Girault, 450
Rabin, 475–476
Rabin, Michael, 103, 259, 518, 550
Rabin-Miller algorithm, 259–260
RACE Integrity Primitives Evaluation, 605–606
Rackoff, Charles, 352
Rainbow Books, 600
Rambutan, 390
Random keys, 173–174
Random noise, as random-sequence generator, 423–424
Random-number generation, 44
Random-sequence generators, 421–428
biases and correlations, 425–426
computer clock, 424
distilling randomness, 426–428
keyboard latency measurement, 424–425
RAND tables, 422–423
using random noise, 423–424
Random sequences, real, 45–46
Randomized approach, stream ciphers, 415
Randomized stream cipher, 419
Randomness, distilling, 426–428
RAND tables, 422–423
Rao-Nam algorithm, 346
Rate of the language, 234
RC2, 318–319
RC4, 319, 397–398
RC5, 344–346
source code, 659–662
RDES, 297–298
Receipt, resending message as, 42–43
REDOC II, 311–313
REDOC III, 313
Redundancy, of language, 234
Reeds, Jim, 369
Related-key cryptanalysis, 290
Renji, Tao, 482
Renting Passports, 111
Replay attacks, 58–59
Research and Development in Advanced Communication Technologies, Integrity Primitives Evaluation, 605–606
Resend attack, foiling, 43
Residue, 242
quadratic, 250–251
reduced set, 248
Restricted algorithms, 3
RFC 1421, 578
RFC 1422, 578
RFC 1423, 578
RFC 1424, 578
Richter, Manfield, 423
Riordan, Mark, 583–584
RIPE, 605–606
RIPEM, 583–584
RIPE-MAC, 457–458
RIPE-MD, 445
Rip van Winkle cipher, 418–419
Rivest, Ron, 159, 163, 318–319, 344, 397, 435, 440–441, 444, 446, 467
Rivest Cipher, 318
Robshaw, Matt, 342
Rogaway, Phil, 398
ROM key, 181
ROT13, 11
Rotor machines, 12–13
RSA, 17, 466–474
ability to break, zero-knowledge proofs, 548–549
attack on encrypting and signing with, 473–474
blind signatures, 548
chosen ciphertext attack, 471–472
common modulus attack, 472
compared to DSA, 485
computation time comparison with DSA, 489
as de facto standard, 485–486
EKE implementation, 519
encryption, 468
with DSA, 491
in hardware, 469
low decryption exponent attack, 473
low encryption exponent attack, 472–473
patents, 474
restrictions on use, 473
security, 470–471
speed, 469
standards, 474
RSA Data Security, Inc., 295, 603–604
RSA Factoring Challenge, 257
RSA generator, 417
Rubber-hose cryptanalysis, 7
Rueppel, Ranier, 385–386
Running-key cipher, 12
SAFER K-64, 339–341
SAFER K-128, 341
Salt, 52–53
S-boxes:
alternate, DES, 296–298
Blowfish, 336
Boolean functions in, 350
DES, key-dependent, 298, 300
design
criteria, 294
security questions, 284
theory, 349–351
Lucifer, 303
NSA role, 278, 280
substitution, 274–276
Scherbius, Arthur, 13
Schlafly, Roger, 394
Schneier, Bruce, 336, 346
Schnorr, Claus, 418, 446, 510
Schnorr authentication and signature scheme, 510–512
Schroeder, Michael, 58, 216
Schwartau, Winn, 300
Sci.crypt, 608–609
Scott, Robert, 306
SEAL, 398–400
source code, 667–673
Secrecy:
ideal, 236
perfect, 235
Secrets, simultaneous exchange, 123–124
Secret sharing, 71–73
without adjudication, 72
with cheaters, 72
with disenrollment, 73
without revealing shares, 73
schemes with prevention, 73
verifiable, 73
Secret-sharing algorithms, 528–531
advanced threshold schemes, 530–531
Asmuth-Bloom, 529–530
cheater detection, 531
Karnin-Greene-Hellman, 530
LaGrange interpolating polynomial scheme, 528–529
vector scheme, 529
Secret splitting, 70–71
digital cash, 142–145
Secure and Fast Encryption Routine, 339
Secure circuit evaluation, 137
Secure elections, 125–134
divided protocols, 133
multiple-key ciphers, 133
simplistic voting protocols, 125–126
voting with
blind signatures, 126–127
single central facility, 128–130
two central facilities, 127–128
Secure elections (Cont.)
voting without central tabulating facility, 130–133
Secure European System for Applications in a Multivendor Environment, 572
Secure Hash Algorithm, 442–445
Secure multiparty computation, 134–137, 551–552
Secure Telephone Unit, 565
Security:
of algorithms, 8–9
Blowfish, 339
cipher block chaining mode, 196–197
ciphers based on one-way hash functions, 353–354
cryptosystem, 234–235
DES, 278, 280–285
algebraic structure, 282–283
current, 300–301
key length, 283–284
weak keys, 280–281
DSA, 491–492
ESIGN, 500
Kerberos, 571
knapsack algorithms, 465
MD5, 440–441
MMB, 326–327
output-feedback mode, 205
PKZIP, 395
Privacy-Enhanced Mail, 582–583
requirements for different information, 167
RSA, 470–471
SEAL, 400
Secure Hash Algorithm, 444–445
self-synchronizing stream cipher, 199
Selector string, 143
Self-decimated generator, 385–387
Self-enforcing protocols, 26–27
Self-recovering, cipher block chaining mode, 196
Self-shrinking generator, 388
Self-synchronizing stream cipher, 198–199
Selmer, E. S., 381
Semiweak keys, DES, 280–281
SESAME, 572
Session keys, 33, 180
SHA, 442–445
Shadows, 71–72
Shamir, Adi, 72, 284–285, 288, 291, 296, 303, 311–312, 314, 319, 416, 434, 462, 467, 502–504, 508, 516, 528
Shamir's pseudo-random-number generator, 416
Shamir's three-pass protocol, 516–517
Shimizu, Akihiro, 308
Shor, Peter, 164
Shrinking generator, 388, 411–412
Signature equation, 496
Signatures, see Digital signatures
Silverman, Bob, 159
Simmons, Gustavus, 72, 79, 493, 501, 531
Simple columnar transposition cipher, 12
Simple relations, 347–348
Simple substitution cipher, 10–11
Simultaneous exchange of secrets, 123–124
Skew, 425
SKEY, 53
SKID, 55–56
Skipjack, 267, 328–329
Smart cards, 587
observer, 146
Universal Electronic Payment System, 589–591
Smith, Lynn, 266
snDES, 298–299
Snefru, 432
Software:
DES implementation, 278–279
encryption, 225
linear feedback shift registers, 378–379
RSA speedups, 469–470
Software-based brute-force attack, 154–155
Software Publishers Association, 608
Solovay, Robert, 259
Solovay-Strassen algorithm, 259
Space complexity, 237
Sparse, 378
Special number field sieve, 160–161
SP network, 347
Square roots:
coin flipping using, 541–542
modulo n, 258
Standards:
public-key cryptography, 588–589
RSA, 474
Station-to-station protocol, 516
Steganography, 9–10
StepRightUp, 414
Stereotyped beginnings, 190
Stereotyped endings, 190
Storage:
data encryption for, 220–222
keys, 180–181
requirements, 9
Stornetta, W. Scott, 75
Straight permutation, 275
Strassen, Volker, 259
Stream algorithms, 4
Stream ciphers, 4, 189, 197–198
A5, 389
additive generators, 390–392
Algorithm M, 393–394
versus block ciphers, 210–211
Blum, Blum, and Shub generator, 417–418
Blum-Micali generator, 416–417
cascading multiple, 419–420
cellular automaton generator, 414
choosing, 420
complexity-theoretic approach, 415–418
correlation immunity, 380
counter mode, 206
crypt(1), 414
design and analysis, 379–381
Diffie's randomized stream cipher, 419
encryption speeds, 420
feedback with carry shift registers, 402–404
Fish, 391
Gifford, 392–393
Hughes XPD/KPD, 389–390
information-theoretic approach, 418
linear complexity, 380
Maurer's randomized stream cipher, 419
message authentication codes, 459
multiple, generating from single pseudo-random-sequence generator, 420–421
Mush, 392
Nanoteq, 390
nonlinear-feedback shift registers, 412–413
1/p generator, 414
output-feedback mode, 205
Pike, 391–392
PKZIP, 394–395
Pless generator, 413–414
Rambutan, 390
random-sequence generators, 421–428
RC4, 397–398
Rip van Winkle cipher, 418–419
RSA generator, 417
SEAL, 398–400
self-synchronizing, 198–199
synchronous, 202–203
system-theoretic approach, 415–416
using feedback with carry shift registers, 405–412
alternating stop-and-go generators, 410–411
cascade generators, 405
FCSR combining generators, 405, 410
LFSR/FCSR summation/parity cascade, 410–411
shrinking generators, 411–412
using linear feedback shift registers, 381–388
alternating stop-and-go generator, 383, 385
Beth-Piper stop-and-go generator, 383–384
bilateral stop-and-go generator, 384–385
DNRSG, 387
Geffe generator, 382
generalized Geffe generator, 382–383
Gollmann cascade, 387–388
Jennings generator, 383–384
multispeed inner-product generator, 386–387
self-decimated generator, 385–387
self-shrinking generator, 388
shrinking generator, 388
summation generator, 386–387
threshold generator, 384–386
WAKE, 400–402
Strict avalanche criteria, 350
Strong primes, 261
STU-III, 565–566
Subkey, 272
Blowfish, 338–339
Crab, 342–343
IDEA, 322
independent, DES, 295
Subliminal channel, 79–80
applications, 80
DSA, 493, 534–536
ElGamal, 532–533
ESIGN, 533–534
foiling, 536
Ong-Schnorr-Shamir, 531–532
signature algorithm, 79
Subliminal-free signature schemes, 80
Subprotocols, 26
Substitution boxes, 274–276
Substitution ciphers, 10–12
Substitution-permutation network, 347
SubStream, 414
Summation generator, 386–387
Superincreasing knapsack, 463–464
Superincreasing sequence, 463–464
Suppress-replay, 61
Surety Technologies, 79
SXAL8, 344
Symmetric algorithms, 4
Symmetric block algorithms, one-way hash functions using, 446–455
Symmetric cryptography:
bit commitment using, 86–87
communication using, 28–29
key exchange with, 47–48
versus public-key cryptography, 216–217
Symmetric cryptosystems, document signing, 35–37
Symmetric key length, 151–158
Synchronous stream cipher, 202–203
System-theoretic approach, stream ciphers, 415–416
Tap sequence, 373
feedback with carry shift registers, maximal-length, 408–409
Tatebayashi-Matsuzaki-Newman, 524–525
Tavares, Stafford, 334
TEA, 346
TEMPEST, 224
Terminology, 1–9, 39
Terrorist Fraud, 110
Thermodynamics, limitations on brute-force attacks, 157–158
Three-pass protocol, Shamir's, 516–517
Three-Satisfiability, 242
3–Way, 341–342, 354
source code, 654–659
Three-Way Marriage Problem, 242
Threshold generator, 384–386
Threshold schemes, 71–72, 530–531
Ticket-Granting Service, 567
Ticket Granting Ticket, 569
Tickets, 568
Time complexity, 237
Timestamping, 75
arbitrated solution, 75–76
digital signatures, 38
distributed protocol, 77–78
improved arbitrated solution, 76
improvements, 78–79
linking protocol, 76–77
patented protocols, 78–79
protocols, 75–79
TIS/PEM, 583
Total break, 8
Traffic analysis, 219
Traffic-flow security, 217
Transfer, oblivious, 116–117
Transposition, 237
ciphers, 12
Trapdoor one-way function, 30
Traveling Salesman Problem, 241–242
Trees, digital signatures, 37
Trial division, 256
Triple encryption, 358–363
encrypt-decrypt-encrypt mode, 359
with minimum key, 360
modes, 360–362
with three keys, 360
with two keys, 358–359
variants, 362–363
TSD, 594–595
Tsujii-Kurosawa-Itoh-Fujioka-Matsumoto, 501
Tuchman, Walt, 266, 278, 280, 294, 303, 358
Tuckerman, Bryant, 266
Turing, Alan, 240
Turing machine, 239, 241
2–adic numbers, 404
UEPS, 589–591
Uncertainty, 234
Unconditional sender and recipient untraceability, 138
Undeniable digital signatures, 81–82, 536–539
Unicity distance, 235–236
Unit key, 591
United States, export rules, 610–616
Universal Electronic Payment System, 589–591
Unpredictable, to left and to right, 417
Updating, keys, 180
Utah Digital Signature Act, 618
van Oorschot, Paul, 359
Vector scheme, 529
Verification, keys, 178–179
Verification block, 179
Verification equation, 496
Vernam, Gilbert, 15
Vigenere cipher, 10–11, 14
Vino, 346
Viruses, to spread cracking program, 155–156
VLSI 6868, 278
Voting, see Secure elections
WAKE, 400–402
Wayner, Peter, 10
Weak keys:
block ciphers design theory, 348
DES, 280–281
Wheeler, David, 400
Whitening, 363, 366–367
Wide-Mouth Frog protocol, 56–57
Wiener, Michael, 153, 284, 359
Williams, 475–476
Wolfram, Steve, 414, 446
Wood, Michael, 311, 313
Woo-Lam protocol, 63–64
Word Auto Key Encryption, 400
Work factor, 9
xDES1, 365–366
XOR, 13–15
XPD, 389–390
Yagisawa algorithm, 501
Yahalom, 57–58
Yao's millionaire problem, 551
Yung, Moti, 81
Yuval, Gideon, 430
Zero-knowledge proofs, 101–109, 548–549
ability to break RSA, 548–549
Chess Grandmaster Problem, 109
computational, 108
discrete logarithm, 548
generalities, 108–109
identity, 109–111
Mafia Fraud, 110
minimum-disclosure, 108
Multiple Identity Fraud, 111
n is Blum integer, 549
noninteractive, 106–107
no-use, 108
parallel, 106
perfect, 108
Proofs of Membership, 111
Renting Passports, 111
statistical, 108
Terrorist Fraud, 110
Zero-knowledge protocol:
basic, 102–104
graph isomorphism, 104–105
Hamiltonian cycles, 105–106
Zierler, Neal, 381
Zimmermann, Philip, 584


Applied Cryptography, Second Edition

 [Spisak knjiga]
Applied Cryptography, Second Edition
Bruce Schneier
Foreword by Whitfield Diffie
Preface
About the Author
Chapter 1—Foundations
1.1 Terminology
1.2 Steganography
1.3 Substitution Ciphers and Transposition Ciphers
1.4 Simple XOR
1.5 One-Time Pads
1.6 Computer Algorithms
1.7 Large Numbers
Part I—Cryptographic Protocols
Chapter 2—Protocol Building Blocks
2.1 Introduction to Protocols
2.2 Communications Using Symmetric Cryptography
2.3 One-Way Functions
2.4 One-Way Hash Functions
2.5 Communications Using Public-Key Cryptography
2.6 Digital Signatures
2.7 Digital Signatures with Encryption
2.8 Random and Pseudo-Random-Sequence Generation
Chapter 3—Basic Protocols
3.1 Key Exchange
3.2 Authentication
3.3 Authentication and Key Exchange
3.4 Formal Analysis of Authentication and Key-Exchange Protocols
3.5 Multiple-Key Public-Key Cryptography
3.6 Secret Splitting
3.7 Secret Sharing
3.8 Cryptographic Protection of Databases
Chapter 4—Intermediate Protocols
4.1 Timestamping Services
4.2 Subliminal Channel
4.3 Undeniable Digital Signatures
4.4 Designated Confirmer Signatures
4.5 Proxy Signatures
4.6 Group Signatures
4.7 Fail-Stop Digital Signatures
4.8 Computing with Encrypted Data
4.9 Bit Commitment
4.10 Fair Coin Flips
4.11 Mental Poker
4.12 One-Way Accumulators
4.13 All-or-Nothing Disclosure of Secrets
4.14 Key Escrow
Chapter 5—Advanced Protocols
5.1 Zero-Knowledge Proofs
5.2 Zero-Knowledge Proofs of Identity
5.3 Blind Signatures
5.4 Identity-Based Public-Key Cryptography
5.5 Oblivious Transfer
5.6 Oblivious Signatures
5.7 Simultaneous Contract Signing
5.8 Digital Certified Mail
5.9 Simultaneous Exchange of Secrets
Chapter 6—Esoteric Protocols
6.1 Secure Elections
6.2 Secure Multiparty Computation
6.3 Anonymous Message Broadcast
6.4 Digital Cash
Part II—Cryptographic Techniques
Chapter 7—Key Length
7.1 Symmetric Key Length
7.2 Public-Key Key Length
7.3 Comparing Symmetric and Public-Key Key Length
7.4 Birthday Attacks against One-Way Hash Functions
7.5 How Long Should a Key Be?
7.6 Caveat Emptor
Chapter 8—Key Management
8.1 Generating Keys
8.2 Nonlinear Keyspaces
8.3 Transferring Keys
8.4 Verifying Keys
8.5 Using Keys
8.6 Updating Keys
8.7 Storing Keys
8.8 Backup Keys
8.9 Compromised Keys
8.10 Lifetime of Keys
8.11 Destroying Keys
8.12 Public-Key Key Management
Chapter 9—Algorithm Types and Modes
9.1 Electronic Codebook Mode
9.2 Block Replay
9.3 Cipher Block Chaining Mode
9.4 Stream Ciphers
9.5 Self-Synchronizing Stream Ciphers
9.6 Cipher-Feedback Mode
9.7 Synchronous Stream Ciphers
9.8 Output-Feedback Mode
9.9 Counter Mode
9.10 Other Block-Cipher Modes
9.11 Choosing a Cipher Mode
9.12 Interleaving
9.13 Block Ciphers versus Stream Ciphers
Chapter 10—Using Algorithms
10.1 Choosing an Algorithm
10.2 Public-Key Cryptography versus Symmetric Cryptography
10.3 Encrypting Communications Channels
10.4 Encrypting Data for Storage
10.5 Hardware Encryption versus Software Encryption
10.6 Compression, Encoding, and Encryption
10.7 Detecting Encryption
10.8 Hiding Ciphertext in Ciphertext
10.9 Destroying Information
Part III—Cryptographic Algorithms
Chapter 11—Mathematical Background
11.1 Information Theory
11.2 Complexity Theory
11.3 Number Theory
11.4 Factoring
11.5 Prime Number Generation
11.6 Discrete Logarithms in a Finite Field
Chapter 12—Data Encryption Standard (DES)
12.1 Background
12.2 Description of DES
12.3 Security of DES
12.4 Differential and Linear Cryptanalysis
12.5 The Real Design Criteria
12.6 DES Variants
12.7 How Secure Is DES Today?
Chapter 13—Other Block Ciphers
13.1 Lucifer
13.2 Madryga
13.3 NewDES
13.4 FEAL
13.5 REDOC
13.6 LOKI
13.7 Khufu and Khafre
13.8 RC2
13.9 IDEA
13.10 MMB
13.11 CA-1.1
13.12 Skipjack
Chapter 14—Still Other Block Ciphers
14.1 GOST
14.2 CAST
14.3 Blowfish
14.4 SAFER
14.5 3-Way
14.6 Crab
14.7 SXAL8/MBAL
14.8 RC5
14.9 Other Block Algorithms
14.10 Theory of Block Cipher Design
14.11 Using one-Way Hash Functions
14.12 Choosing a Block Algorithm
Chapter 15—Combining Block Ciphers
15.1 Double Encryption
15.2 Triple Encryption
15.3 Doubling the Block Length
15.4 Other Multiple Encryption Schemes
15.5 CDMF Key Shortening
15.6 Whitening
15.7 Cascading Multiple Block Algorithms
15.8 Combining Multiple Block Algorithms
Chapter 16—Pseudo-Random-Sequence Generators and Stream Ciphers
16.1 Linear Congruential Generators
16.2 Linear Feedback Shift Registers
16.3 Design and Analysis of Stream Ciphers
16.4 Stream Ciphers Using LFSRs
16.5 A5
16.6 Hughes XPD/KPD
16.7 Nanoteq
16.8 Rambutan
16.9 Additive Generators
16.10 Gifford
16.11 Algorithm M
16.12 PKZIP
Chapter 17—Other Stream Ciphers and Real Random-Sequence Generators
17.1 RC4
17.2 SEAL
17.3 WAKE
17.4 Feedback with Carry Shift Registers
17.5 Stream Ciphers Using FCSRs
17.6 Nonlinear-Feedback Shift Registers
17.7 Other Stream Ciphers
17.8 System-Theoretic Approach to Stream-Cipher Design
17.9 Complexity-Theoretic Approach to Stream-Cipher Design
17.10 Other Approaches to Stream-Cipher Design
17.11 Cascading Multiple Stream Ciphers
17.12 Choosing a Stream Cipher
17.13 Generating Multiple Streams from a Single Pseudo-Random-Sequence Generator
17.14 Real Random-Sequence Generators
Chapter 18—One-Way Hash Functions
18.1 Background
18.2 Snefru
18.3 N- Hash
18.4 MD4
18.5 MD5
18.6 MD2
18.7 Secure Hash Algorithm (SHA)
18.8 RIPE-MD
18.9 HAVAL
18.10 Other One-Way Hash Functions
18.11 One-Way Hash Functions Using Symmetric Block Algorithms
18.12 Using Public-Key Algorithms
18.13 Choosing a One-Way Hash Function
18.14 Message Authentication Codes
Chapter 19—Public-Key Algorithms
19.1 Background
19.2 Knapsack Algorithms
19.3 RSA
19.4 Pohlig-Hellman
19.5 Rabin
19.6 ElGamal
19.7 McEliece
19.8 Elliptic Curve Cryptosystems
19.9 LUC
19.10 Finite Automaton Public-Key Cryptosystems
Chapter 20—Public-Key Digital Signature Algorithms
20.1 Digital Signature Algorithm (DSA)
20.2 DSA Variants
20.3 Gost Digital Signature Algorithm
20.4 Discrete Logarithm Signature Schemes
20.5 Ong-Schnorr-Shamir
20.6 ESIGN
20.7 Cellular Automata
20.8 Other Public-Key Algorithms
Chapter 21—Identification Schemes
21.1 Feige-Fiat-Shamir
21.2 Guillou-Quisquater
21.3 Schnorr
21.4 Converting Identification Schemes to Signature Schemes
Chapter 22—Key-Exchange Algorithms
22.1 Diffie-Hellman
22.2 Station-to-Station Protocol
22.3 Shamir’s Three-Pass Protocol
22.4 COMSET
22.5 Encrypted Key Exchange
22.6 Fortified Key Negotiation
22.7 Conference Key Distribution and Secret Broadcasting
Chapter 23—Special Algorithms for Protocols
23.1 Multiple-Key Public-Key Cryptography
23.2 Secret-Sharing Algorithms
23.3 Subliminal Channel
23.4 Undeniable Digital Signatures
23.5 Designated Confirmer Signatures
23.6 Computing with Encrypted Data
23.7 Fair Coin Flips
23.8 One-Way Accumulators
23.9 All-or-Nothing Disclosure of Secrets
23.10 Fair and Failsafe Cryptosystems
23.11 Zero-Knowledge Proofs of Knowledge
23.12 Blind Signatures
23.13 Oblivious Transfer
23.14 Secure Multiparty Computation
23.15 Probabilistic Encryption
23.16 Quantum Cryptography
Part IV—The Real World
Chapter 24—Example Implementations
24.1 IBM Secret-Key Management Protocol
24.2 MITRENET
24.3 ISDN
24.4 STU-III
24.5 Kerberos
24.6 KryptoKnight
24.7 SESAME
24.8 IBM Common Cryptographic Architecture
24.9 ISO Authentication Framework
24.10 Privacy-Enhanced Mail (PEM)
24.11 Message Security Protocol (MSP)
24.12 Pretty Good Privacy (PGP)
24.13 Smart Cards
24.14 Public-Key Cryptography Standards (PKCS)
24.15 Universal Electronic Payment System (UEPS)
24.16 Clipper
24.17 Capstone
24.18 AT&ampT Model 3600 Telephone Security Device (TSD)
Chapter 25—Politics
25.1 National Security Agency (NSA)
25.2 National Computer Security Center (NCSC)
25.3 National Institute of Standards and Technology (NIST)
25.4 RSA Data Security, Inc.
25.5 Public Key Partners
25.6 International Association for Cryptologic Research (IACR)
25.7 RACE Integrity Primitives Evaluation (RIPE)
25.8 Conditional Access for Europe (CAFE)
25.9 ISO/IEC 9979
25.10 Professional, Civil Liberties, and Industry Groups
25.11 Sci.crypt
25.12 Cypherpunks
25.13 Patents
25.14 U.S. Export Rules
25.15 Foreign Import and Export of Cryptography
25.16 Legal Issues
Afterword by Matt Blaze
Part V—Source Code
References
Index


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Foreword By Whitfield Diffie

The literature of cryptography has a curious history. Secrecy, of course, has always played a central role, but until the First World War, important developments appeared in print in a more or less timely fashion and the field moved forward in much the same way as other specialized disciplines. As late as 1918, one of the most influential cryptanalytic papers of the twentieth century, William F. Friedman's monograph The Index of Coincidence and Its Applications in Cryptography, appeared as a research report of the private Riverbank Laboratories [577]. And this, despite the fact that the work had been done as part of the war effort. In the same year Edward H. Hebern of Oakland, California filed the first patent for a rotor machine [710], the device destined to be a mainstay of military cryptography for nearly 50 years.

After the First World War, however, things began to change. U.S. Army and Navy organizations, working entirely in secret, began to make fundamental advances in cryptography. During the thirties and forties a few basic papers did appear in the open literature and several treatises on the subject were published, but the latter were farther and farther behind the state of the art. By the end of the war the transition was complete. With one notable exception, the public literature had died. That exception was Claude Shannon's paper "The Communication Theory of Secrecy Systems," which appeared in the Bell System Technical Journal in 1949 [1432]. It was similar to Friedman's 1918 paper, in that it grew out of wartime work of Shannon's. After the Second World War ended it was declassified, possibly by mistake.

From 1949 until 1967 the cryptographic literature was barren. In that year a different sort of contribution appeared: David Kahn's history, The Codebreakers [794]. It didn't contain any new technical ideas, but it did contain a remarkably complete history of what had gone before, including mention of some things that the government still considered secret. The significance of The Codebreakers lay not just in its remarkable scope, but also in the fact that it enjoyed good sales and made tens of thousands of people, who had never given the matter a moment's thought, aware of cryptography. A trickle of new cryptographic papers began to be written.

At about the same time, Horst Feistel, who had earlier worked on identification friend or foe devices for the Air Force, took his lifelong passion for cryptography to the IBM Watson Laboratory in Yorktown Heights, New York. There, he began development of what was to become the U.S. Data Encryption Standard; by the early 1970s several technical reports on this subject by Feistel and his colleagues had been made public by IBM [1482,1484,552].

This was the situation when I entered the field in late 1972. The cryptographic literature wasn't abundant, but what there was included some very shiny nuggets.

Cryptology presents a difficulty not found in normal academic disciplines: the need for the proper interaction of cryptography and cryptanalysis. This arises out of the fact that in the absence of real communications requirements, it is easy to propose a system that appears unbreakable. Many academic designs are so complex that the would–be cryptanalyst doesn't know where to start; exposing flaws in these designs is far harder than designing them in the first place. The result is that the competitive process, which is one strong motivation in academic research, cannot take hold.

When Martin Hellman and I proposed public–key cryptography in 1975 [496], one of the indirect aspects of our contribution was to introduce a problem that does not even appear easy to solve. Now an aspiring cryptosystem designer could produce something that would be recognized as clever—something that did more than just turn meaningful text into nonsense. The result has been a spectacular increase in the number of people working in cryptography, the number of meetings held, and the number of books and papers published.

In my acceptance speech for the Donald E. Fink award—given for the best expository paper to appear in an IEEE journal—which I received jointly with Hellman in 1980, I told the audience that in writing "Privacy and Authentication," I had an experience that I suspected was rare even among the prominent scholars who populate the IEEE awards ceremony: I had written the paper I had wanted to study, but could not find, when I first became seriously interested in cryptography. Had I been able to go to the Stanford bookstore and pick up a modern cryptography text, I would probably have learned about the field years earlier. But the only things available in the fall of 1972 were a few classic papers and some obscure technical reports.

The contemporary researcher has no such problem. The problem now is choosing where to start among the thousands of papers and dozens of books. The contemporary researcher, yes, but what about the contemporary programmer or engineer who merely wants to use cryptography? Where does that person turn? Until now, it has been necessary to spend long hours hunting out and then studying the research literature before being able to design the sort of cryptographic utilities glibly described in popular articles.

This is the gap that Bruce Schneier's Applied Cryptography has come to fill. Beginning with the objectives of communication security and elementary examples of programs used to achieve these objectives, Schneier gives us a panoramic view of the fruits of 20 years of public research. The title says it all; from the mundane objective of having a secure conversation the very first time you call someone to the possibilities of digital money and cryptographically secure elections, this is where you'll find it.

Not satisfied that the book was about the real world merely because it went all the way down to the code, Schneier has included an account of the world in which cryptography is developed and applied, and discusses entities ranging from the International Association for Cryptologic Research to the NSA.

When public interest in cryptography was just emerging in the late seventies and early eighties, the National Security Agency (NSA), America's official cryptographic organ, made several attempts to quash it. The first was a letter from a long–time NSA employee allegedly, avowedly, and apparently acting on his own. The letter was sent to the IEEE and warned that the publication of cryptographic material was a violation of the International Traffic in Arms Regulations (ITAR). This viewpoint turned out not even to be supported by the regulations themselves—which contained an explicit exemption for published material—but gave both the public practice of cryptography and the 1977 Information Theory Workshop lots of unexpected publicity.

A more serious attempt occurred in 1980, when the NSA funded the American Council on Education to examine the issue with a view to persuading Congress to give it legal control of publications in the field of cryptography. The results fell far short of NSA's ambitions and resulted in a program of voluntary review of cryptographic papers; researchers were requested to ask the NSA's opinion on whether disclosure of results would adversely affect the national interest before publication.

As the eighties progressed, pressure focused more on the practice than the study of cryptography. Existing laws gave the NSA the power, through the Department of State, to regulate the export of cryptographic equipment. As business became more and more international and the American fraction of the world market declined, the pressure to have a single product in both domestic and offshore markets increased. Such single products were subject to export control and thus the NSA acquired substantial influence not only over what was exported, but also over what was sold in the United States.

As this is written, a new challenge confronts the public practice of cryptography. The government has augmented the widely published and available Data Encryption Standard, with a secret algorithm implemented in tamper–resistant chips. These chips will incorporate a codified mechanism of government monitoring. The negative aspects of this "key–escrow" program range from a potentially disastrous impact on personal privacy to the high cost of having to add hardware to products that had previously encrypted in software. So far key escrow products are enjoying less than stellar sales and the scheme has attracted widespread negative comment, especially from the independent cryptographers. Some people, however, see more future in programming than politicking and have redoubled their efforts to provide the world with strong cryptography that is accessible to public scrutiny.

A sharp step back from the notion that export control law could supersede the First Amendment seemed to have been taken in 1980 when the Federal Register announcement of a revision to ITAR included the statement: "...provision has been added to make it clear that the regulation of the export of technical data does not purport to interfere with the First Amendment rights of individuals." But the fact that tension between the First Amendment and the export control laws has not gone away should be evident from statements at a conference held by RSA Data Security. NSA's representative from the export control office expressed the opinion that people who published cryptographic programs were "in a grey area" with respect to the law. If that is so, it is a grey area on which the first edition of this book has shed some light. Export applications for the book itself have been granted, with acknowledgement that published material lay beyond the authority of the Munitions Control Board. Applications to export the enclosed programs on disk, however, have been denied.

The shift in the NSA's strategy, from attempting to control cryptographic research to tightening its grip on the development and deployment of cryptographic products, is presumably due to its realization that all the great cryptographic papers in the world do not protect a single bit of traffic. Sitting on the shelf, this volume may be able to do no better than the books and papers that preceded it, but sitting next to a workstation, where a programmer is writing cryptographic code, it just may.

Whitfield Diffie
Mountain View, CA


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 1
Foundations

1.1 Terminology

Sender and Receiver

Suppose a sender wants to send a message to a receiver. Moreover, this sender wants to send the message securely: She wants to make sure an eavesdropper cannot read the message.

Messages and Encryption

A message is plaintext (sometimes called cleartext). The process of disguising a message in such a way as to hide its substance is encryption. An encrypted message is ciphertext. The process of turning ciphertext back into plaintext is decryption. This is all shown in Figure 1.1.

(If you want to follow the ISO 7498-2 standard, use the terms "encipher" and "decipher." It seems that some cultures find the terms "encrypt" and "decrypt" offensive, as they refer to dead bodies.)

The art and science of keeping messages secure is cryptography, and it is practiced by cryptographers. Cryptanalysts are practitioners of cryptanalysis, the art and science of breaking ciphertext; that is, seeing through the disguise. The branch of mathematics encompassing both cryptography and cryptanalysis is cryptology and its practitioners are cryptologists. Modern cryptologists are generally trained in theoretical mathematics—they have to be.

Figure 1.1 Encryption and Decryption.

Plaintext is denoted by M, for message, or P, for plaintext. It can be a stream of bits, a text file, a bitmap, a stream of digitized voice, a digital video image...whatever. As far as a computer is concerned, M is simply binary data. (After this chapter, this book concerns itself with binary data and computer cryptography.) The plaintext can be intended for either transmission or storage. In any case, M is the message to be encrypted.

Ciphertext is denoted by C. It is also binary data: sometimes the same size as M, sometimes larger. (By combining encryption with compression, C may be smaller than M. However, encryption does not accomplish this.) The encryption function E, operates on M to produce C. Or, in mathematical notation:

E(M) = C

In the reverse process, the decryption function D operates on C to produce M:

D(C) = M

Since the whole point of encrypting and then decrypting a message is to recover the original plaintext, the following identity must hold true:

D(E(M)) = M

Authentication, Integrity, and Nonrepudiation

In addition to providing confidentiality, cryptography is often asked to do other jobs:

— Authentication. It should be possible for the receiver of a message to ascertain its origin; an intruder should not be able to masquerade as someone else.
— Integrity. It should be possible for the receiver of a message to verify that it has not been modified in transit; an intruder should not be able to substitute a false message for a legitimate one.
— Nonrepudiation. A sender should not be able to falsely deny later that he sent a message.

These are vital requirements for social interaction on computers, and are analogous to face-to-face interactions. That someone is who he says he is...that someone's credentials—whether a driver's license, a medical degree, or a passport—are valid...that a document purporting to come from a person actually came from that person.... These are the things that authentication, integrity, and nonrepudiation provide.

Algorithms and Keys

A cryptographic algorithm, also called a cipher, is the mathematical function used for encryption and decryption. (Generally, there are two related functions: one for encryption and the other for decryption.)

If the security of an algorithm is based on keeping the way that algorithm works a secret, it is a restricted algorithm. Restricted algorithms have historical interest, but are woefully inadequate by today's standards. A large or changing group of users cannot use them, because every time a user leaves the group everyone else must switch to a different algorithm. If someone accidentally reveals the secret, everyone must change their algorithm.

Even more damning, restricted algorithms allow no quality control or standardization. Every group of users must have their own unique algorithm. Such a group can't use off-the-shelf hardware or software products; an eavesdropper can buy the same product and learn the algorithm. They have to write their own algorithms and implementations. If no one in the group is a good cryptographer, then they won't know if they have a secure algorithm.

Despite these major drawbacks, restricted algorithms are enormously popular for low-security applications. Users either don't realize or don't care about the security problems inherent in their system.

Modern cryptography solves this problem with a key, denoted by K. This key might be any one of a large number of values. The range of possible values of the key is called the keyspace. Both the encryption and decryption operations use this key (i.e., they are dependent on the key and this fact is denoted by the k subscript), so the functions now become:

EK(M) = C
DK(C) = M

Those functions have the property that (see Figure 1.2):

DK(EK(M)) = M

Some algorithms use a different encryption key and decryption key (see Figure 1.3). That is, the encryption key, K1, is different from the corresponding decryption key, K2. In this case:

EK1(M) = C
DK2(C) = M
DK2(EK1 (M)) = M

All of the security in these algorithms is based in the key (or keys); none is based in the details of the algorithm. This means that the algorithm can be published and analyzed. Products using the algorithm can be mass-produced. It doesn't matter if an eavesdropper knows your algorithm; if she doesn't know your particular key, she can't read your messages.

Figure 1.2 Encryption and decryption with a key.

Figure 1.3 Encryption and decryption with two different keys.

A cryptosystem is an algorithm, plus all possible plaintexts, ciphertexts, and keys.

Symmetric Algorithms

There are two general types of key-based algorithms: symmetric and public-key. Symmetric algorithms, sometimes called conventional algorithms, are algorithms where the encryption key can be calculated from the decryption key and vice versa. In most symmetric algorithms, the encryption key and the decryption key are the same. These algorithms, also called secret-key algorithms, single-key algorithms, or one-key algorithms, require that the sender and receiver agree on a key before they can communicate securely. The security of a symmetric algorithm rests in the key; divulging the key means that anyone could encrypt and decrypt messages. As long as the communication needs to remain secret, the key must remain secret.

Encryption and decryption with a symmetric algorithm are denoted by:

EK(M) = C
DK(C) = M


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Symmetric algorithms can be divided into two categories. Some operate on the plaintext a single bit (or sometimes byte) at a time; these are called stream algorithms or stream ciphers. Others operate on the plaintext in groups of bits. The groups of bits are called blocks, and the algorithms are called block algorithms or block ciphers. For modern computer algorithms, a typical block size is 64 bits—large enough to preclude analysis and small enough to be workable. (Before computers, algorithms generally operated on plaintext one character at a time. You can think of this as a stream algorithm operating on a stream of characters.)

Public-Key Algorithms

Public-key algorithms (also called asymmetric algorithms) are designed so that the key used for encryption is different from the key used for decryption. Furthermore, the decryption key cannot (at least in any reasonable amount of time) be calculated from the encryption key. The algorithms are called "public-key" because the encryption key can be made public: A complete stranger can use the encryption key to encrypt a message, but only a specific person with the corresponding decryption key can decrypt the message. In these systems, the encryption key is often called the public key, and the decryption key is often called the private key. The private key is sometimes also called the secret key, but to avoid confusion with symmetric algorithms, that tag won't be used here.

Encryption using public key K is denoted by:

EK(M) = C

Even though the public key and private key are different, decryption with the corresponding private key is denoted by:

DK(C) = M

Sometimes, messages will be encrypted with the private key and decrypted with the public key; this is used in digital signatures (see Section 2.6). Despite the possible confusion, these operations are denoted by, respectively:

EK(M) = C
DK(C) = M

Cryptanalysis

The whole point of cryptography is to keep the plaintext (or the key, or both) secret from eavesdroppers (also called adversaries, attackers, interceptors, interlopers, intruders, opponents, or simply the enemy). Eavesdroppers are assumed to have complete access to the communications between the sender and receiver.

Cryptanalysis is the science of recovering the plaintext of a message without access to the key. Successful cryptanalysis may recover the plaintext or the key. It also may find weaknesses in a cryptosystem that eventually lead to the previous results. (The loss of a key through noncryptanalytic means is called a compromise.)

An attempted cryptanalysis is called an attack. A fundamental assumption in cryptanalysis, first enunciated by the Dutchman A. Kerckhoffs in the nineteenth century, is that the secrecy must reside entirely in the key [794]. Kerckhoffs assumes that the cryptanalyst has complete details of the cryptographic algorithm and implementation. (Of course, one would assume that the CIA does not make a habit of telling Mossad about its cryptographic algorithms, but Mossad probably finds out anyway.) While real-world cryptanalysts don't always have such detailed information, it's a good assumption to make. If others can't break an algorithm, even with knowledge of how it works, then they certainly won't be able to break it without that knowledge.

There are four general types of cryptanalytic attacks. Of course, each of them assumes that the cryptanalyst has complete knowledge of the encryption algorithm used:

1. Ciphertext-only attack. The cryptanalyst has the ciphertext of several messages, all of which have been encrypted using the same encryption algorithm. The cryptanalyst's job is to recover the plaintext of as many messages as possible, or better yet to deduce the key (or keys) used to encrypt the messages, in order to decrypt other messages encrypted with the same keys.
Given: C1 = Ek(P1), C2 = Ek(P2),...Ci = Ek(Pi)
Deduce: Either P1, P2,...Pi; k; or an algorithm to infer Pi+1 from Ci+1 = Ek(Pi+1)
2. Known-plaintext attack. The cryptanalyst has access not only to the ciphertext of several messages, but also to the plaintext of those messages. His job is to deduce the key (or keys) used to encrypt the messages or an algorithm to decrypt any new messages encrypted with the same key (or keys).
Given: P1, C1 = Ek(P1), P2, C2 = Ek(P2),...Pi, Ci = Ek(Pi)
Deduce: Either k, or an algorithm to infer Pi+1 from Ci+1 = Ek(Pi+1)
3. Chosen-plaintext attack. The cryptanalyst not only has access to the ciphertext and associated plaintext for several messages, but he also chooses the plaintext that gets encrypted. This is more powerful than a known-plaintext attack, because the cryptanalyst can choose specific plaintext blocks to encrypt, ones that might yield more information about the key. His job is to deduce the key (or keys) used to encrypt the messages or an algorithm to decrypt any new messages encrypted with the same key (or keys).
Given: P1, C1 = Ek(P1), P2, C2 = Ek(P2),...Pi, Ci = Ek(Pi), where the cryptanalyst gets to choose P1, P2,...Pi
Deduce: Either k, or an algorithm to infer Pi+1 from Ci+1 = Ek(Pi+1)
4. Adaptive-chosen-plaintext attack. This is a special case of a chosen-plaintext attack. Not only can the cryptanalyst choose the plaintext that is encrypted, but he can also modify his choice based on the results of previous encryption. In a chosen-plaintext attack, a cryptanalyst might just be able to choose one large block of plaintext to be encrypted; in an adaptive-chosen-plaintext attack he can choose a smaller block of plaintext and then choose another based on the results of the first, and so forth.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

There are at least three other types of cryptanalytic attack.

5. Chosen-ciphertext attack. The cryptanalyst can choose different ciphertexts to be decrypted and has access to the decrypted plaintext. For example, the cryptanalyst has access to a tamperproof box that does automatic decryption. His job is to deduce the key.
Given: C1, P1 = Dk(C1), C2, P2 = Dk(C2),...Ci, Pi = Dk(Ci)
Deduce: k

This attack is primarily applicable to public-key algorithms and will be discussed in Section 19.3. A chosen-ciphertext attack is sometimes effective against a symmetric algorithm as well. (Sometimes a chosen-plaintext attack and a chosen-ciphertext attack are together known as a chosen-text attack.)
6. Chosen-key attack. This attack doesn't mean that the cryptanalyst can choose the key; it means that he has some knowledge about the relationship between different keys. It's strange and obscure, not very practical, and discussed in Section 12.4.
7. Rubber-hose cryptanalysis. The cryptanalyst threatens, blackmails, or tortures someone until they give him the key. Bribery is sometimes referred to as a purchase-key attack. These are all very powerful attacks and often the best way to break an algorithm.

Known-plaintext attacks and chosen-plaintext attacks are more common than you might think. It is not unheard-of for a cryptanalyst to get a plaintext message that has been encrypted or to bribe someone to encrypt a chosen message. You may not even have to bribe someone; if you give a message to an ambassador, you will probably find that it gets encrypted and sent back to his country for consideration. Many messages have standard beginnings and endings that might be known to the cryptanalyst. Encrypted source code is especially vulnerable because of the regular appearance of keywords: #define, struct, else, return. Encrypted executable code has the same kinds of problems: functions, loop structures, and so on. Known-plaintext attacks (and even chosen-plaintext attacks) were successfully used against both the Germans and the Japanese during World War II. David Kahn's books [794,795,796] have historical examples of these kinds of attacks.

And don't forget Kerckhoffs's assumption: If the strength of your new cryptosystem relies on the fact that the attacker does not know the algorithm's inner workings, you're sunk. If you believe that keeping the algorithm's insides secret improves the security of your cryptosystem more than letting the academic community analyze it, you're wrong. And if you think that someone won't disassemble your code and reverse-engineer your algorithm, you're naïve. (In 1994 this happened with the RC4 algorithm—see Section 17.1.) The best algorithms we have are the ones that have been made public, have been attacked by the world's best cryptographers for years, and are still unbreakable. (The National Security Agency keeps their algorithms secret from outsiders, but they have the best cryptographers in the world working within their walls—you don't. Additionally, they discuss their algorithms with one another, relying on peer review to uncover any weaknesses in their work.)

Cryptanalysts don't always have access to the algorithms, as when the United States broke the Japanese diplomatic code PURPLE during World War II [794]—but they often do. If the algorithm is being used in a commercial security program, it is simply a matter of time and money to disassemble the program and recover the algorithm. If the algorithm is being used in a military communications system, it is simply a matter of time and money to buy (or steal) the equipment and reverse-engineer the algorithm.

Those who claim to have an unbreakable cipher simply because they can't break it are either geniuses or fools. Unfortunately, there are more of the latter in the world. Beware of people who extol the virtues of their algorithms, but refuse to make them public; trusting their algorithms is like trusting snake oil.

Good cryptographers rely on peer review to separate the good algorithms from the bad.

Security of Algorithms

Different algorithms offer different degrees of security; it depends on how hard they are to break. If the cost required to break an algorithm is greater than the value of the encrypted data, then you're probably safe. If the time required to break an algorithm is longer than the time the encrypted data must remain secret, then you're probably safe. If the amount of data encrypted with a single key is less than the amount of data necessary to break the algorithm, then you're probably safe.

I say "probably" because there is always a chance of new breakthroughs in cryptanalysis. On the other hand, the value of most data decreases over time. It is important that the value of the data always remain less than the cost to break the security protecting it.

Lars Knudsen classified these different categories of breaking an algorithm. In decreasing order of severity [858]:

1. Total break. A cryptanalyst finds the key, K, such that DK(C) = P.
2. Global deduction. A cryptanalyst finds an alternate algorithm, A, equivalent to DK(C), without knowing K.
3. Instance (or local) deduction. A cryptanalyst finds the plaintext of an intercepted ciphertext.
4. Information deduction. A cryptanalyst gains some information about the key or plaintext. This information could be a few bits of the key, some information about the form of the plaintext, and so forth.

An algorithm is unconditionally secure if, no matter how much ciphertext a cryptanalyst has, there is not enough information to recover the plaintext. In point of fact, only a one-time pad (see Section 1.5) is unbreakable given infinite resources. All other cryptosystems are breakable in a ciphertext-only attack, simply by trying every possible key one by one and checking whether the resulting plaintext is meaningful. This is called a brute-force attack (see Section 7.1).

Cryptography is more concerned with cryptosystems that are computationally infeasible to break. An algorithm is considered computationally secure (sometimes called strong) if it cannot be broken with available resources, either current or future. Exactly what constitutes "available resources" is open to interpretation.

You can measure the complexity (see Section 11.1) of an attack in different ways:

1. Data complexity. The amount of data needed as input to the attack.
2. Processing complexity. The time needed to perform the attack. This is often called the work factor.
3. Storage requirements. The amount of memory needed to do the attack.

As a rule of thumb, the complexity of an attack is taken to be the minimum of these three factors. Some attacks involve trading off the three complexities: A faster attack might be possible at the expense of a greater storage requirement.

Complexities are expressed as orders of magnitude. If an algorithm has a processing complexity of 2128, then 2128 operations are required to break the algorithm. (These operations may be complex and time-consuming.) Still, if you assume that you have enough computing speed to perform a million operations every second and you set a million parallel processors against the task, it will still take over 1019 years to recover the key. That's a billion times the age of the universe.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

While the complexity of an attack is constant (until some cryptanalyst finds a better attack, of course), computing power is anything but. There have been phenomenal advances in computing power during the last half-century and there is no reason to think this trend won't continue. Many cryptanalytic attacks are perfect for parallel machines: The task can be broken down into billions of tiny pieces and none of the processors need to interact with each other. Pronouncing an algorithm secure simply because it is infeasible to break, given current technology, is dicey at best. Good cryptosystems are designed to be infeasible to break with the computing power that is expected to evolve many years in the future.

Historical Terms

Historically, a code refers to a cryptosystem that deals with linguistic units: words, phrases, sentences, and so forth. For example, the word "OCELOT" might be the ciphertext for the entire phrase "TURN LEFT 90 DEGREES," the word "LOLLIPOP" might be the ciphertext for "TURN RIGHT 90 DEGREES," and the words "BENT EAR" might be the ciphertext for "HOWITZER." Codes of this type are not discussed in this book; see [794,795]. Codes are only useful for specialized circumstances. Ciphers are useful for any circumstance. If your code has no entry for "ANTEATERS," then you can't say it. You can say anything with a cipher.

1.2 Steganography

Steganography serves to hide secret messages in other messages, such that the secret's very existence is concealed. Generally the sender writes an innocuous message and then conceals a secret message on the same piece of paper. Historical tricks include invisible inks, tiny pin punctures on selected characters, minute differences between handwritten characters, pencil marks on typewritten characters, grilles which cover most of the message except for a few characters, and so on.

More recently, people are hiding secret messages in graphic images. Replace the least significant bit of each byte of the image with the bits of the message. The graphical image won't change appreciably—most graphics standards specify more gradations of color than the human eye can notice—and the message can be stripped out at the receiving end. You can store a 64-kilobyte message in a 1024 × 1024 grey-scale picture this way. Several public-domain programs do this sort of thing.

Peter Wayner's mimic functions obfuscate messages. These functions modify a message so that its statistical profile resembles that of something else: the classifieds section of The New York Times, a play by Shakespeare, or a newsgroup on the Internet [1584,1585]. This type of steganography won't fool a person, but it might fool some big computers scanning the Internet for interesting messages.

1.3 Substitution Ciphers and Transposition Ciphers

Before computers, cryptography consisted of character-based algorithms. Different cryptographic algorithms either substituted characters for one another or transposed characters with one another. The better algorithms did both, many times each.

Things are more complex these days, but the philosophy remains the same. The primary change is that algorithms work on bits instead of characters. This is actually just a change in the alphabet size: from 26 elements to two elements. Most good cryptographic algorithms still combine elements of substitution and transposition.

Substitution Ciphers

A substitution cipher is one in which each character in the plaintext is substituted for another character in the ciphertext. The receiver inverts the substitution on the ciphertext to recover the plaintext.

In classical cryptography, there are four types of substitution ciphers:

— A simple substitution cipher, or monoalphabetic cipher, is one in which each character of the plaintext is replaced with a corresponding character of ciphertext. The cryptograms in newspapers are simple substitution ciphers.
— A homophonic substitution cipher is like a simple substitution cryptosystem, except a single character of plaintext can map to one of several characters of ciphertext. For example, "A" could correspond to either 5, 13, 25, or 56, "B" could correspond to either 7, 19, 31, or 42, and so on.
— A polygram substitution cipher is one in which blocks of characters are encrypted in groups. For example, "ABA" could correspond to "RTQ," "ABB" could correspond to "SLL," and so on.
— A polyalphabetic substitution cipher is made up of multiple simple substitution ciphers. For example, there might be five different simple substitution ciphers used; the particular one used changes with the position of each character of the plaintext.

The famous Caesar Cipher, in which each plaintext character is replaced by the character three to the right modulo 26 ("A" is replaced by "D," "B" is replaced by "E,"..., "W" is replaced by "Z," "X" is replaced by "A," "Y" is replaced by "B," and "Z" is replaced by "C") is a simple substitution cipher. It's actually even simpler, because the ciphertext alphabet is a rotation of the plaintext alphabet and not an arbitrary permutation.

ROT13 is a simple encryption program commonly found on UNIX systems; it is also a simple substitution cipher. In this cipher, "A" is replaced by "N," "B" is replaced by "O," and so on. Every letter is rotated 13 places.

Encrypting a file twice with ROT13 restores the original file.

P = ROT13 (ROT13 (P))

ROT13 is not intended for security; it is often used in Usenet posts to hide potentially offensive text, to avoid giving away the solution to a puzzle, and so forth.

Simple substitution ciphers can be easily broken because the cipher does not hide the underlying frequencies of the different letters of the plaintext. All it takes is about 25 English characters before a good cryptanalyst can reconstruct the plaintext [1434]. An algorithm for solving these sorts of ciphers can be found in [578,587,1600,78,1475,1236,880]. A good computer algorithm is [703].

Homophonic substitution ciphers were used as early as 1401 by the Duchy of Mantua [794]. They are much more complicated to break than simple substitution ciphers, but still do not obscure all of the statistical properties of the plaintext language. With a known-plaintext attack, the ciphers are trivial to break. A ciphertext-only attack is harder, but only takes a few seconds on a computer. Details are in [1261].

Polygram substitution ciphers are ciphers in which groups of letters are encrypted together. The Playfair cipher, invented in 1854, was used by the British during World War I [794]. It encrypts pairs of letters together. Its cryptanalysis is discussed in [587,1475,880]. The Hill cipher is another example of a polygram substitution cipher [732]. Sometimes you see Huffman coding used as a cipher; this is an insecure polygram substitution cipher.

Polyalphabetic substitution ciphers were invented by Leon Battista in 1568 [794]. They were used by the Union army during the American Civil War. Despite the fact that they can be broken easily [819,577,587,794] (especially with the help of computers), many commercial computer security products use ciphers of this form [1387,1390,1502]. (Details on how to break this encryption scheme, as used in WordPerfect, can be found in [135,139].) The Vigenère cipher, first published in 1586, and the Beaufort cipher are also examples of polyalphabetic substitution ciphers.

Polyalphabetic substitution ciphers have multiple one-letter keys, each of which is used to encrypt one letter of the plaintext. The first key encrypts the first letter of the plaintext, the second key encrypts the second letter of the plaintext, and so on. After all the keys are used, the keys are recycled. If there were 20 one-letter keys, then every twentieth letter would be encrypted with the same key. This is called the period of the cipher. In classical cryptography, ciphers with longer periods were significantly harder to break than ciphers with short periods. There are computer techniques that can easily break substitution ciphers with very long periods.

A running-key cipher—sometimes called a book cipher—in which one text is used to encrypt another text, is another example of this sort of cipher. Even though this cipher has a period the length of the text, it can also be broken easily [576,794].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Transposition Ciphers

In a transposition cipher the plaintext remains the same, but the order of characters is shuffled around. In a simple columnar transposition cipher, the plaintext is written horizontally onto a piece of graph paper of fixed width and the ciphertext is read off vertically (see Figure 1.4). Decryption is a matter of writing the ciphertext vertically onto a piece of graph paper of identical width and then reading the plaintext off horizontally.

Cryptanalysis of these ciphers is discussed in [587,1475]. Since the letters of the ciphertext are the same as those of the plaintext, a frequency analysis on the ciphertext would reveal that each letter has approximately the same likelihood as in English. This gives a very good clue to a cryptanalyst, who can then use a variety of techniques to determine the right ordering of the letters to obtain the plaintext. Putting the ciphertext through a second transposition cipher greatly enhances security. There are even more complicated transposition ciphers, but computers can break almost all of them.

The German ADFGVX cipher, used during World War I, is a transposition cipher combined with a simple substitution. It was a very complex algorithm for its day but was broken by Georges Painvin, a French cryptanalyst [794].

Although many modern algorithms use transposition, it is troublesome because it requires a lot of memory and sometimes requires messages to be only certain lengths. Substitution is far more common.

Rotor Machines

In the 1920s, various mechanical encryption devices were invented to automate the process of encryption. Most were based on the concept of a rotor, a mechanical wheel wired to perform a general substitution.

A rotor machine has a keyboard and a series of rotors, and implements a version of the Vigenère cipher. Each rotor is an arbitrary permutation of the alphabet, has 26 positions, and performs a simple substitution. For example, a rotor might be wired to substitute "F" for "A," "U" for "B," "L" for "C," and so on. And the output pins of one rotor are connected to the input pins of the next.

Figure 1.4 Columnar transposition cipher.

For example, in a 4-rotor machine the first rotor might substitute "F" for "A," the second might substitute "Y" for "F," the third might substitute "E" for "Y," and the fourth might substitute "C" for "E"; "C" would be the output ciphertext. Then some of the rotors shift, so next time the substitutions will be different.

It is the combination of several rotors and the gears moving them that makes the machine secure. Because the rotors all move at different rates, the period for an n-rotor machine is 26n. Some rotor machines can also have a different number of positions on each rotor, further frustrating cryptanalysis.

The best-known rotor device is the Enigma. The Enigma was used by the Germans during World War II. The idea was invented by Arthur Scherbius and Arvid Gerhard Damm in Europe. It was patented in the United States by Arthur Scherbius [1383]. The Germans beefed up the basic design considerably for wartime use.

The German Enigma had three rotors, chosen from a set of five, a plugboard that slightly permuted the plaintext, and a reflecting rotor that caused each rotor to operate on each plaintext letter twice. As complicated as the Enigma was, it was broken during World War II. First, a team of Polish cryptographers broke the German Enigma and explained their attack to the British. The Germans modified their Enigma as the war progressed, and the British continued to cryptanalyze the new versions. For explanations of how rotor ciphers work and how they were broken, see [794,86,448,498,446,880,1315,1587,690]. Two fascinating accounts of how the Enigma was broken are [735,796].

Further Reading

This is not a book about classical cryptography, so I will not dwell further on these subjects. Two excellent precomputer cryptology books are [587,1475]; [448] presents some modern cryptanalysis of cipher machines. Dorothy Denning discusses many of these ciphers in [456] and [880] has some fairly complex mathematical analysis of the same ciphers. Another older cryptography text, which discusses analog cryptography, is [99]. An article that presents a good overview of the subject is [579]. David Kahn's historical cryptography books are also excellent [794,795,796].

1.4 Simple XOR

XOR is exclusive-or operation: '^' in C or ⊕ in mathematical notation. It's a standard operation on bits:

0 ⊕ 0 = 0
0 ⊕ 1 = 1
1 ⊕ 0 = 1
1 ⊕ 1 = 0

Also note that:

a ⊕ a = 0
a ⊕ b ⊕ b = a

The simple-XOR algorithm is really an embarrassment; it's nothing more than a Vigenère polyalphabetic cipher. It's here only because of its prevalence in commercial software packages, at least those in the MS-DOS and Macintosh worlds [1502,1387]. Unfortunately, if a software security program proclaims that it has a "proprietary" encryption algorithm—significantly faster than DES—the odds are that it is some variant of this.

/* Usage:  crypto key input_file output_file */

void main (int argc, char *argv[])

{
     FILE *fi, *fo;
     char *cp;
     int c;

     if ((cp = argv[1]) && *cp!='\0')  {
         if ((fi = fopen(argv[2], "rb")) != NULL)  {
             if ((fo = fopen(argv[3], "wb")) != NULL)  {
                  while ((c = getc(fi)) != EOF)  {
                       if (!*cp) cp = argv[1];
                       c ^= *(cp++);
                       putc(c,fo);
                  }
                  fclose(fo);
             }
             fclose(fi);
         }
     }
}

This is a symmetric algorithm. The plaintext is being XORed with a keyword to generate the ciphertext. Since XORing the same value twice restores the original, encryption and decryption use exactly the same program:

P ⊕ K = C
C ⊕ K = P

There's no real security here. This kind of encryption is trivial to break, even without computers [587,1475]. It will only take a few seconds with a computer.

Assume the plaintext is English. Furthermore, assume the key length is any small number of bytes. Here's how to break it:

1. Discover the length of the key by a procedure known as counting coincidences [577]. XOR the ciphertext against itself shifted various numbers of bytes, and count those bytes that are equal. If the displacement is a multiple of the key length, then something over 6 percent of the bytes will be equal. If it is not, then less than 0.4 percent will be equal (assuming a random key encrypting normal ASCII text; other plaintext will have different numbers). This is called the index of coincidence. The smallest displacement that indicates a multiple of the key length is the length of the key.
2. Shift the ciphertext by that length and XOR it with itself. This removes the key and leaves you with plaintext XORed with the plaintext shifted the length of the key. Since English has 1.3 bits of real information per byte (see Section 11.1), there is plenty of redundancy for determining a unique decryption.

Despite this, the list of software vendors that tout this toy algorithm as being "almost as secure as DES" is staggering [1387]. It is the algorithm (with a 160-bit repeated "key") that the NSA finally allowed the U.S. digital cellular phone industry to use for voice privacy. An XOR might keep your kid sister from reading your files, but it won't stop a cryptanalyst for more than a few minutes.

1.5 One-Time Pads

Believe it or not, there is a perfect encryption scheme. It's called a one-time pad, and was invented in 1917 by Major Joseph Mauborgne and AT&T's Gilbert Vernam [794]. (Actually, a one-time pad is a special case of a threshold scheme; see Section 3.7.) Classically, a one-time pad is nothing more than a large nonrepeating set of truly random key letters, written on sheets of paper, and glued together in a pad. In its original form, it was a one-time tape for teletypewriters. The sender uses each key letter on the pad to encrypt exactly one plaintext character. Encryption is the addition modulo 26 of the plaintext character and the one-time pad key character.

Each key letter is used exactly once, for only one message. The sender encrypts the message and then destroys the used pages of the pad or used section of the tape. The receiver has an identical pad and uses each key on the pad, in turn, to decrypt each letter of the ciphertext. The receiver destroys the same pad pages or tape section after decrypting the message. New message—new key letters. For example, if the message is:

    ONETIMEPAD

and the key sequence from the pad is

    TBFRGFARFM

then the ciphertext is

    IPKLPSFHGQ

because

O + T mod 26 = I
N + B mod 26 = P
E + F mod 26 = K
etc.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Assuming an eavesdropper can't get access to the one-time pad used to encrypt the message, this scheme is perfectly secure. A given ciphertext message is equally likely to correspond to any possible plaintext message of equal size.

Since every key sequence is equally likely (remember, the key letters are generated randomly), an adversary has no information with which to cryptanalyze the ciphertext. The key sequence could just as likely be:

    POYYAEAAZX

which would decrypt to:

    SALMONEGGS

or

    BXFGBMTMXM

which would decrypt to:

    GREENFLUID

This point bears repeating: Since every plaintext message is equally possible, there is no way for the cryptanalyst to determine which plaintext message is the correct one. A random key sequence added to a nonrandom plaintext message produces a completely random ciphertext message and no amount of computing power can change that.

The caveat, and this is a big one, is that the key letters have to be generated randomly. Any attacks against this scheme will be against the method used to generate the key letters. Using a pseudo-random number generator doesn't count; they always have nonrandom properties. If you use a real random source—this is much harder than it might first appear, see Section 17.14—it's secure.

The other important point is that you can never use the key sequence again, ever. Even if you use a multiple-gigabyte pad, if a cryptanalyst has multiple ciphertexts whose keys overlap, he can reconstruct the plaintext. He slides each pair of ciphertexts against each other and counts the number of matches at each position. If they are aligned right, the proportion of matches jumps suddenly—the exact percentages depend on the plaintext language. From this point cryptanalysis is easy. It's like the index of coincidence, but with just two "periods" to compare [904]. Don't do it.

The idea of a one-time pad can be easily extended to binary data. Instead of a one-time pad consisting of letters, use a one-time pad of bits. Instead of adding the plaintext to the one-time pad, use an XOR. To decrypt, XOR the ciphertext with the same one-time pad. Everything else remains the same and the security is just as perfect.

This all sounds good, but there are a few problems. Since the key bits must be random and can never be used again, the length of the key sequence must be equal to the length of the message. A one-time pad might be suitable for a few short messages, but it will never work for a 1.544 Mbps communications channel. You can store 650 megabytes worth of random bits on a CD-ROM, but there are problems. First, you want exactly two copies of the random bits, but CD-ROMs are economical only for large quantities. And second, you want to be able to destroy the bits already used. CD-ROM has no erase facilities except for physically destroying the entire disk. Digital tape is a much better medium for this sort of thing.

Even if you solve the key distribution and storage problem, you have to make sure the sender and receiver are perfectly synchronized. If the receiver is off by a bit (or if some bits are dropped during the transmission), the message won't make any sense. On the other hand, if some bits are altered during transmission (without any bits being added or removed—something far more likely to happen due to random noise), only those bits will be decrypted incorrectly. But on the other hand, a one-time pad provides no authenticity.

One-time pads have applications in today's world, primarily for ultra-secure low-bandwidth channels. The hotline between the United States and the former Soviet Union was (is it still active?) rumored to be encrypted with a one-time pad. Many Soviet spy messages to agents were encrypted using one-time pads. These messages are still secure today and will remain that way forever. It doesn't matter how long the supercomputers work on the problem. Even after the aliens from Andromeda land with their massive spaceships and undreamed-of computing power, they will not be able to read the Soviet spy messages encrypted with one-time pads (unless they can also go back in time and get the one-time pads).

1.6 Computer Algorithms

There are many cryptographic algorithms. These are three of the most common:

— DES (Data Encryption Standard) is the most popular computer encryption algorithm. DES is a U.S. and international standard. It is a symmetric algorithm; the same key is used for encryption and decryption.
— RSA (named for its creators—Rivest, Shamir, and Adleman) is the most popular public-key algorithm. It can be used for both encryption and digital signatures.
— DSA (Digital Signature Algorithm, used as part of the Digital Signature Standard) is another public-key algorithm. It cannot be used for encryption, but only for digital signatures.

These are the kinds of stuff this book is about.

1.7 Large Numbers

Throughout this book, I use various large numbers to describe different things in cryptography. Because it is so easy to lose sight of these numbers and what they signify, Table 1.1 gives physical analogues for some of them.

These numbers are order-of-magnitude estimates, and have been culled from a variety of sources. Many of the astrophysics numbers are explained in Freeman

TABLE 1.1 Large Numbers
Physical Analogue | Number
Odds of being killed by lightning (per day) | 1 in 9 billion (233)
Odds of winning the top prize in a U.S. state lottery | 1 in 4,000,000 (222)
Odds of winning the top prize in a U.S. state lottery and being killed by lightning in the same day | 1 in 255
Odds of drowning (in the U.S. per year) | 1 in 59,000 (216)
Odds of being killed in an automobile accident(in the U.S. in 1993) | 1 in 6100 (213)
Odds of being killed in an automobile accident(in the U.S. per lifetime) | 1 in 88 (27)
Time until the next ice age | 14,000 (214) years
Time until the sun goes nova | 109 (230) years
Age of the planet | 109 (230) years
Age of the Universe | 1010 (234) years
Number of atoms in the planet | 1051 (2170)
Number of atoms in the sun | 1057 (2190)
Number of atoms in the galaxy | 1067 (2223)
Number of atoms in the Universe (dark matter excluded) | 1077 (2265)
Volume of the Universe | 1084 (2280) cm3
If the Universe is Closed:
Total lifetime of the Universe | 1011 (237) years 1018 (261) seconds
If the Universe is Open:
Time until low-mass stars cool off | 1014 (247) years
Time until planets detach from stars | 1015 (250) years
Time until stars detach from galaxies | 1019 (264) years
Time until orbits decay by gravitational radiation | 1020 (267) years
Time until black holes decay by the Hawking process | 1064 (2213) years
Time until all matter is liquid at zero temperature | 1065 (2216) years
Time until all matter decays to iron | 101026 years
Time until all matter collapses to black holes | 101076 years

Dyson's paper, "Time Without End: Physics and Biology in an Open Universe," in Reviews of Modern Physics, v. 52, n. 3, July 1979, pp. 447–460. Automobile accident deaths are calculated from the Department of Transportation's statistic of 163 deaths per million people in 1993 and an average lifespan of 69.7 years.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part I
Cryptographic protocols

Chapter 2
Protocol Building Blocks

2.1 Introduction to Protocols

The whole point of cryptography is to solve problems. (Actually, that's the whole point of computers—something many people tend to forget.) Cryptography solves problems that involve secrecy, authentication, integrity, and dishonest people. You can learn all about cryptographic algorithms and techniques, but these are academic unless they can solve a problem. This is why we are going to look at protocols first.

A protocol is a series of steps, involving two or more parties, designed to accomplish a task. This is an important definition. A "series of steps" means that the protocol has a sequence, from start to finish. Every step must be executed in turn, and no step can be taken before the previous step is finished. "Involving two or more parties" means that at least two people are required to complete the protocol; one person alone does not make a protocol. A person alone can perform a series of steps to accomplish a task (like baking a cake), but this is not a protocol. (Someone else must eat the cake to make it a protocol.) Finally, "designed to accomplish a task" means that the protocol must achieve something. Something that looks like a protocol but does not accomplish a task is not a protocol—it's a waste of time.

Protocols have other characteristics as well:

— Everyone involved in the protocol must know the protocol and all of the steps to follow in advance.
— Everyone involved in the protocol must agree to follow it.
— The protocol must be unambiguous; each step must be well defined and there must be no chance of a misunderstanding.
— The protocol must be complete; there must be a specified action for every possible situation.

The protocols in this book are organized as a series of steps. Execution of the protocol proceeds linearly through the steps, unless there are instructions to branch to another step. Each step involves at least one of two things: computations by one or more of the parties, or messages sent among the parties.

A cryptographic protocol is a protocol that uses cryptography. The parties can be friends and trust each other implicitly or they can be adversaries and not trust one another to give the correct time of day. A cryptographic protocol involves some cryptographic algorithm, but generally the goal of the protocol is something beyond simple secrecy. The parties participating in the protocol might want to share parts of their secrets to compute a value, jointly generate a random sequence, convince one another of their identity, or simultaneously sign a contract. The whole point of using cryptography in a protocol is to prevent or detect eavesdropping and cheating. If you have never seen these protocols before, they will radically change your ideas of what mutually distrustful parties can accomplish over a computer network. In general, this can be stated as:

— It should not be possible to do more or learn more than what is specified in the protocol.

This is a lot harder than it looks. In the next few chapters I discuss a lot of protocols. In some of them it is possible for one of the participants to cheat the other. In others, it is possible for an eavesdropper to subvert the protocol or learn secret information. Some protocols fail because the designers weren't thorough enough in their requirements definitions. Others fail because their designers weren't thorough enough in their analysis. Like algorithms, it is much easier to prove insecurity than it is to prove security.

The Purpose of Protocols

In daily life, there are informal protocols for almost everything: ordering goods over the telephone, playing poker, voting in an election. No one thinks much about these protocols; they have evolved over time, everyone knows how to use them, and they work reasonably well.

These days, more and more human interaction takes place over computer networks instead of face-to-face. Computers need formal protocols to do the same things that people do without thinking. If you moved from one state to another and found a voting booth that looked completely different from the ones you were used to, you could easily adapt. Computers are not nearly so flexible.

Many face-to-face protocols rely on people's presence to ensure fairness and security. Would you send a stranger a pile of cash to buy groceries for you? Would you play poker with someone if you couldn't see him shuffle and deal? Would you mail the government your secret ballot without some assurance of anonymity?

It is naïve to assume that people on computer networks are honest. It is naïve to assume that the managers of computer networks are honest. It is even naïve to assume that the designers of computer networks are honest. Most are, but the dishonest few can do a lot of damage. By formalizing protocols, we can examine ways in which dishonest parties can subvert them. Then we can develop protocols that are immune to that subversion.

In addition to formalizing behavior, protocols abstract the process of accomplishing a task from the mechanism by which the task is accomplished. A communications protocol is the same whether implemented on PCs or VAXs. We can examine the protocol without getting bogged down in the implementation details. When we are convinced we have a good protocol, we can implement it in everything from computers to telephones to intelligent muffin toasters.

The Players

To help demonstrate protocols, I have enlisted the aid of several people (see Table 2.1). Alice and Bob are the first two. They will perform all general two-person protocols. As a rule, Alice will initiate all protocols and Bob will respond. If the protocol requires a third or fourth person, Carol and Dave will perform those roles. Other actors will play specialized supporting roles; they will be introduced later.

Arbitrated Protocols

An arbitrator is a disinterested third party trusted to complete a protocol (see Figure 2.1a). Disinterested means that the arbitrator has no vested interest in the protocol and no particular allegiance to any of the parties involved. Trusted means that all people involved in the protocol accept what he says as true, what he does as correct, and that he will complete his part of the protocol. Arbitrators can help complete protocols between two mutually distrustful parties.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In the real world, lawyers are often used as arbitrators. For example, Alice is selling a car to Bob, a stranger. Bob wants to pay by check, but Alice has no way of knowing if the check is good. Alice wants the check to clear before she turns the title over to Bob. Bob, who doesn't trust Alice any more than she trusts him, doesn't want to hand over a check without receiving a title.

TABLE 2.1 Dramatis Personae
Alice | First participant in all the protocols
Bob | Second participant in all the protocols
Carol | Participant in the three- and four-party protocols
Dave | Participant in the four-party protocols
Eve | Eavesdropper
Mallory | Malicious active attacker
Trent | Trusted arbitrator
Walter | Warden; he'll be guarding Alice and Bob in some protocols
Peggy | Prover
Victor | Verifier

Figure 2.1 Types of protocols.

Enter a lawyer trusted by both. With his help, Alice and Bob can use the following protocol to ensure that neither cheats the other:

(1) Alice gives the title to the lawyer.
(2) Bob gives the check to Alice.
(3) Alice deposits the check.
(4) After waiting a specified time period for the check to clear, the lawyer gives the title to Bob. If the check does not clear within the specified time period, Alice shows proof of this to the lawyer and the lawyer returns the title to Alice.

In this protocol, Alice trusts the lawyer not to give Bob the title unless the check has cleared, and to give it back to her if the check does not clear. Bob trusts the lawyer to hold the title until the check clears, and to give it to him once it does. The lawyer doesn't care if the check clears. He will do his part of the protocol in either case, because he will be paid in either case.

In the example, the lawyer is playing the part of an escrow agent. Lawyers also act as arbitrators for wills and sometimes for contract negotiations. The various stock exchanges act as arbitrators between buyers and sellers.

Bankers also arbitrate protocols. Bob can use a certified check to buy a car from Alice:

(1) Bob writes a check and gives it to the bank.
(2) After putting enough of Bob's money on hold to cover the check, the bank certifies the check and gives it back to Bob.
(3) Alice gives the title to Bob and Bob gives the certified check to Alice.
(4) Alice deposits the check.

This protocol works because Alice trusts the banker's certification. Alice trusts the bank to hold Bob's money for her, and not to use it to finance shaky real estate operations in mosquito-infested countries.

A notary public is another arbitrator. When Bob receives a notarized document from Alice, he is convinced that Alice signed the document voluntarily and with her own hand. The notary can, if necessary, stand up in court and attest to that fact.

The concept of an arbitrator is as old as society. There have always been people—rulers, priests, and so on—who have the authority to act fairly. Arbitrators have a certain social role and position in our society; betraying the public trust would jeopardize that. Lawyers who play games with escrow accounts face almost-certain disbarment, for example. This picture of trust doesn't always exist in the real world, but it's the ideal.

This ideal can translate to the computer world, but there are several problems with computer arbitrators:

— It is easier to find and trust a neutral third party if you know who the party is and can see his face. Two parties suspicious of each other are also likely to be suspicious of a faceless arbitrator somewhere else on the network.
— The computer network must bear the cost of maintaining an arbitrator. We all know what lawyers charge; who wants to bear that kind of network overhead?
— There is a delay inherent in any arbitrated protocol.
— The arbitrator must deal with every transaction; he is a bottleneck in large-scale implementations of any protocol. Increasing the number of arbitrators in the implementation can mitigate this problem, but that increases the cost.
— Since everyone on the network must trust the arbitrator, he represents a vulnerable point for anyone trying to subvert the network.

Even so, arbitrators still have a role to play. In protocols using a trusted arbitrator, the part will be played by Trent.

Adjudicated Protocols

Because of the high cost of hiring arbitrators, arbitrated protocols can be subdivided into two lower-level subprotocols. One is a nonarbitrated subprotocol, executed every time parties want to complete the protocol. The other is an arbitrated subprotocol, executed only in exceptional circumstances—when there is a dispute. This special type of arbitrator is called an adjudicator (see Figure 2.1b).

An adjudicator is also a disinterested and trusted third party. Unlike an arbitrator, he is not directly involved in every protocol. The adjudicator is called in only to determine whether a protocol was performed fairly.

Judges are professional adjudicators. Unlike a notary public, a judge is brought in only if there is a dispute. Alice and Bob can enter into a contract without a judge. A judge never sees the contract until one of them hauls the other into court.

This contract-signing protocol can be formalized in this way:

Nonarbitrated subprotocol (executed every time):

(1) Alice and Bob negotiate the terms of the contract.
(2) Alice signs the contract.
(3) Bob signs the contract.

Adjudicated subprotocol (executed only in case of a dispute):

(4) Alice and Bob appear before a judge.
(5) Alice presents her evidence.
(6) Bob presents his evidence.
(7) The judge rules on the evidence.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The difference between an adjudicator and an arbitrator (as used in this book) is that the adjudicator is not always necessary. In a dispute, a judge is called in to adjudicate. If there is no dispute, using a judge is unnecessary.

There are adjudicated computer protocols. These protocols rely on the parties to be honest; but if someone suspects cheating, a body of data exists so that a trusted third party could determine if someone cheated. In a good adjudicated protocol, the adjudicator could also determine the cheater's identity. Instead of preventing cheating, adjudicated protocols detect cheating. The inevitability of detection acts as a preventive and discourages cheating.

Self-Enforcing Protocols

A self-enforcing protocol is the best type of protocol. The protocol itself guarantees fairness (see Figure 2.1c). No arbitrator is required to complete the protocol. No adjudicator is required to resolve disputes. The protocol is constructed so that there cannot be any disputes. If one of the parties tries to cheat, the other party immediately detects the cheating and the protocol stops. Whatever the cheating party hoped would happen by cheating, doesn't happen.

In the best of all possible worlds, every protocol would be self-enforcing. Unfortunately, there is not a self-enforcing protocol for every situation.

Attacks against Protocols

Cryptographic attacks can be directed against the cryptographic algorithms used in protocols, against the cryptographic techniques used to implement the algorithms and protocols, or against the protocols themselves. Since this section of the book discusses protocols, I will assume that the cryptographic algorithms and techniques are secure. I will only examine attacks against the protocols.

People can try various ways to attack a protocol. Someone not involved in the protocol can eavesdrop on some or all of the protocol. This is called a passive attack, because the attacker does not affect the protocol. All he can do is observe the protocol and attempt to gain information. This kind of attack corresponds to a ciphertext-only attack, as discussed in Section 1.1. Since passive attacks are difficult to detect, protocols try to prevent passive attacks rather than detect them. In these protocols, the part of the eavesdropper will be played by Eve.

Alternatively, an attacker could try to alter the protocol to his own advantage. He could pretend to be someone else, introduce new messages in the protocol, delete existing messages, substitute one message for another, replay old messages, interrupt a communications channel, or alter stored information in a computer. These are called active attacks, because they require active intervention. The form of these attacks depends on the network.

Passive attackers try to gain information about the parties involved in the protocol. They collect messages passing among various parties and attempt to cryptanalyze them. Active attacks, on the other hand, can have much more diverse objectives. The attacker could be interested in obtaining information, degrading system performance, corrupting existing information, or gaining unauthorized access to resources.

Active attacks are much more serious, especially in protocols in which the different parties don't necessarily trust one another. The attacker does not have to be a complete outsider. He could be a legitimate system user. He could be the system administrator. There could even be many active attackers working together. Here, the part of the malicious active attacker will be played by Mallory.

It is also possible that the attacker could be one of the parties involved in the protocol. He may lie during the protocol or not follow the protocol at all. This type of attacker is called a cheater. Passive cheaters follow the protocol, but try to obtain more information than the protocol intends them to. Active cheaters disrupt the protocol in progress in an attempt to cheat.

It is very difficult to maintain a protocol's security if most of the parties involved are active cheaters, but sometimes it is possible for legitimate parties to detect that active cheating is going on. Certainly, protocols should be secure against passive cheating.

2.2 Communications Using Symmetric Cryptography

How do two parties communicate securely? They encrypt their communications, of course. The complete protocol is more complicated than that. Let's look at what must happen for Alice to send an encrypted message to Bob.

(1) Alice and Bob agree on a cryptosystem.
(2) Alice and Bob agree on a key.
(3) Alice takes her plaintext message and encrypts it using the encryption algorithm and the key. This creates a ciphertext message.
(4) Alice sends the ciphertext message to Bob.
(5) Bob decrypts the ciphertext message with the same algorithm and key and reads it.

What can Eve, sitting between Alice and Bob, learn from listening in on this protocol? If all she hears is the transmission in step (4), she must try to cryptanalyze the ciphertext. This passive attack is a ciphertext-only attack; we have algorithms that are resistant (as far as we know) to whatever computing power Eve could realistically bring to bear on the problem.

Eve isn't stupid, though. She also wants to listen in on steps (1) and (2). Then, she would know the algorithm and the key—just as well as Bob. When the message comes across the communications channel in step (4), all she has to do is decrypt it herself.

A good cryptosystem is one in which all the security is inherent in knowledge of the key and none is inherent in knowledge of the algorithm. This is why key management is so important in cryptography. With a symmetric algorithm, Alice and Bob can perform step (1) in public, but they must perform step (2) in secret. The key must remain secret before, during, and after the protocol—as long as the message must remain secret—otherwise the message will no longer be secure. (Public-key cryptography solves this problem another way, and will be discussed in Section 2.5.)

Mallory, an active attacker, could do a few other things. He could attempt to break the communications path in step (4), ensuring that Alice could not talk to Bob at all. Mallory could also intercept Alice's messages and substitute his own. If he knew the key (by intercepting the communication in step (2), or by breaking the cryptosystem), he could encrypt his own message and send it to Bob in place of the intercepted message. Bob would have no way of knowing that the message had not come from Alice. If Mallory didn't know the key, he could only create a replacement message that would decrypt to gibberish. Bob, thinking the message came from Alice, might conclude that either the network or Alice had some serious problems.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

What about Alice? What can she do to disrupt the protocol? She can give a copy of the key to Eve. Now Eve can read whatever Bob says. She can reprint his words in The New York Times. Although serious, this is not a problem with the protocol. There is nothing to stop Alice from giving Eve a copy of the plaintext at any point during the protocol. Of course, Bob could also do anything that Alice could. This protocol assumes that Alice and Bob trust each other.

In summary, symmetric cryptosystems have the following problems:

— Keys must be distributed in secret. They are as valuable as all the messages they encrypt, since knowledge of the key gives knowledge of all the messages. For encryption systems that span the world, this can be a daunting task. Often couriers hand-carry keys to their destinations.
— If a key is compromised (stolen, guessed, extorted, bribed, etc.), then Eve can decrypt all message traffic encrypted with that key. She can also pretend to be one of the parties and produce false messages to fool the other party.
— Assuming a separate key is used for each pair of users in a network, the total number of keys increases rapidly as the number of users increases. A network of n users requires n(n - 1)/2 keys. For example, 10 users require 45 different keys to talk with one another and 100 users require 4950 keys. This problem can be minimized by keeping the number of users small, but that is not always possible.

2.3 One-Way Functions

The notion of a one-way function is central to public-key cryptography. While not protocols in themselves, one-way functions are a fundamental building block for most of the protocols discussed in this book.

One-way functions are relatively easy to compute, but significantly harder to reverse. That is, given x it is easy to compute f(x), but given f(x) it is hard to compute x. In this context, "hard" is defined as something like: It would take millions of years to compute x from f(x), even if all the computers in the world were assigned to the problem.

Breaking a plate is a good example of a one-way function. It is easy to smash a plate into a thousand tiny pieces. However, it's not easy to put all of those tiny pieces back together into a plate.

This sounds good, but it's a lot of smoke and mirrors. If we are being strictly mathematical, we have no proof that one-way functions exist, nor any real evidence that they can be constructed [230,530,600,661]. Even so, many functions look and smell one-way: We can compute them efficiently and, as of yet, know of no easy way to reverse them. For example, in a finite field x2 is easy to compute, but x1/2 is much harder. For the rest of this section, I'm going to pretend that there are one-way functions. I'll talk more about this in Section 11.2.

So, what good are one-way functions? We can't use them for encryption as is. A message encrypted with the one-way function isn't useful; no one could decrypt it. (Exercise: Write a message on a plate, smash the plate into tiny bits, and then give the bits to a friend. Ask your friend to read the message. Observe how impressed he is with the one-way function.) For public-key cryptography, we need something else (although there are cryptographic applications for one-way functions—see Section 3.2).

A trapdoor one-way function is a special type of one-way function, one with a secret trapdoor. It is easy to compute in one direction and hard to compute in the other direction. But, if you know the secret, you can easily compute the function in the other direction. That is, it is easy to compute f(x) given x, and hard to compute x given f(x). However, there is some secret information, y, such that given f(x) and y it is easy to compute x.

Taking a watch apart is a good example of a trap-door one-way function. It is easy to disassemble a watch into hundreds of minuscule pieces. It is very difficult to put those tiny pieces back together into a working watch. However, with the secret information—the assembly instructions of the watch—it is much easier to put the watch back together.

2.4 One-Way Hash Functions

A one-way hash function has many names: compression function, contraction function, message digest, fingerprint, cryptographic checksum, message integrity check (MIC), and manipulation detection code (MDC). Whatever you call it, it is central to modern cryptography. One-way hash functions are another building block for many protocols.

Hash functions have been used in computer science for a long time. A hash function is a function, mathematical or otherwise, that takes a variable-length input string (called a pre-image) and converts it to a fixed-length (generally smaller) output string (called a hash value). A simple hash function would be a function that takes pre-image and returns a byte consisting of the XOR of all the input bytes.

The point here is to fingerprint the pre-image: to produce a value that indicates whether a candidate pre-image is likely to be the same as the real pre-image. Because hash functions are typically many-to-one, we cannot use them to determine with certainty that the two strings are equal, but we can use them to get a reasonable assurance of accuracy.

A one-way hash function is a hash function that works in one direction: It is easy to compute a hash value from pre-image, but it is hard to generate a pre-image that hashes to a particular value. The hash function previously mentioned is not one-way: Given a particular byte value, it is trivial to generate a string of bytes whose XOR is that value. You can't do that with a one-way hash function. A good one-way hash function is also collision-free: It is hard to generate two pre-images with the same hash value.

The hash function is public; there's no secrecy to the process. The security of a one-way hash function is its one-wayness. The output is not dependent on the input in any discernible way. A single bit change in the pre-image changes, on the average, half of the bits in the hash value. Given a hash value, it is computationally unfeasible to find a pre-image that hashes to that value.

Think of it as a way of fingerprinting files. If you want to verify that someone has a particular file (that you also have), but you don't want him to send it to you, then ask him for the hash value. If he sends you the correct hash value, then it is almost certain that he has that file. This is particularly useful in financial transactions, where you don't want a withdrawal of $100 to turn into a withdrawal of $1000 somewhere in the network. Normally, you would use a one-way hash function without a key, so that anyone can verify the hash. If you want only the recipient to be able to verify the hash, then read the next section.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Message Authentication Codes

A message authentication code (MAC), also known as a data authentication code (DAC), is a one-way hash function with the addition of a secret key (see Section 18.14). The hash value is a function of both the pre-image and the key. The theory is exactly the same as hash functions, except only someone with the key can verify the hash value. You can create a MAC out of a hash function or a block encryption algorithm; there are also dedicated MACs.

2.5 Communications Using Public-Key Cryptography

Think of a symmetric algorithm as a safe. The key is the combination. Someone with the combination can open the safe, put a document inside, and close it again. Someone else with the combination can open the safe and take the document out. Anyone without the combination is forced to learn safecracking.

In 1976, Whitfield Diffie and Martin Hellman changed that paradigm of cryptography forever [496]. (The NSA has claimed knowledge of the concept as early as 1966, but has offered no proof.) They described public-key cryptography. They used two different keys—one public and the other private. It is computationally hard to deduce the private key from the public key. Anyone with the public key can encrypt a message but not decrypt it. Only the person with the private key can decrypt the message. It is as if someone turned the cryptographic safe into a mailbox. Putting mail in the mailbox is analogous to encrypting with the public key; anyone can do it. Just open the slot and drop it in. Getting mail out of a mailbox is analogous to decrypting with the private key. Generally it's hard; you need welding torches. However, if you have the secret (the physical key to the mailbox), it's easy to get mail out of a mailbox.

Mathematically, the process is based on the trap-door one-way functions previously discussed. Encryption is the easy direction. Instructions for encryption are the public key; anyone can encrypt a message. Decryption is the hard direction. It's made hard enough that people with Cray computers and thousands (even millions) of years couldn't decrypt the message without the secret. The secret, or trapdoor, is the private key. With that secret, decryption is as easy as encryption.

This is how Alice can send a message to Bob using public-key cryptography:

(1) Alice and Bob agree on a public-key cryptosystem.
(2) Bob sends Alice his public key.
(3) Alice encrypts her message using Bob's public key and sends it to Bob.
(4) Bob decrypts Alice's message using his private key.

Notice how public-key cryptography solves the key-management problem with symmetric cryptosystems. Before, Alice and Bob had to agree on a key in secret. Alice could choose one at random, but she still had to get it to Bob. She could hand it to him sometime beforehand, but that requires foresight. She could send it to him by secure courier, but that takes time. Public-key cryptography makes it easy. With no prior arrangements, Alice can send a secure message to Bob. Eve, listening in on the entire exchange, has Bob's public key and a message encrypted in that key, but cannot recover either Bob's private key or the message.

More commonly, a network of users agrees on a public-key cryptosystem. Every user has his or her own public key and private key, and the public keys are all published in a database somewhere. Now the protocol is even easier:

(1) Alice gets Bob's public key from the database.
(2) Alice encrypts her message using Bob's public key and sends it to Bob.
(3) Bob then decrypts Alice's message using his private key.

In the first protocol, Bob had to send Alice his public key before she could send him a message. The second protocol is more like traditional mail. Bob is not involved in the protocol until he wants to read his message.

Hybrid Cryptosystems

The first public-key algorithms became public at the same time that DES was being discussed as a proposed standard. This resulted in some partisan politics in the cryptographic community. As Diffie described it [494]:

The excitement public key cryptosystems provoked in the popular and scientific press was not matched by corresponding acceptance in the cryptographic establishment, however. In the same year that public key cryptography was discovered, the National Security Agency (NSA), proposed a conventional cryptographic system, designed by International Business Machines (IBM), as a federal Data Encryption Standard (DES). Marty Hellman and I criticized the proposal on the ground that its key was too small, but manufacturers were gearing up to support the proposed standard and our criticism was seen by many as an attempt to disrupt the standards-making process to the advantage of our own work. Public key cryptography in its turn was attacked, in sales literature [1125] and technical papers [849,1159] alike, more as though it were a competing product than a recent research discovery. This, however, did not deter the NSA from claiming its share of the credit. Its director, in the words of the Encyclopedia Britannica [1461], pointed out that "two-key cryptography had been discovered at the agency a decade earlier," although no evidence for this claim was ever offered publicly.

In the real world, public-key algorithms are not a substitute for symmetric algorithms. They are not used to encrypt messages; they are used to encrypt keys. There are two reasons for this:

1. Public-key algorithms are slow. Symmetric algorithms are generally at least 1000 times faster than public-key algorithms. Yes, computers are getting faster and faster, and in 15 years computers will be able to do public-key cryptography at speeds comparable to symmetric cryptography today. But bandwidth requirements are also increasing, and there will always be the need to encrypt data faster than public-key cryptography can manage.
2. Public-key cryptosystems are vulnerable to chosen-plaintext attacks. If C = E(P), when P is one plaintext out of a set of n possible plaintexts, then a cryptanalyst only has to encrypt all n possible plaintexts and compare the results with C (remember, the encryption key is public). He won't be able to recover the decryption key this way, but he will be able to determine P.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

A chosen-plaintext attack can be particularly effective if there are relatively few possible encrypted messages. For example, if P were a dollar amount less than $1,000,000, this attack would work; the cryptanalyst tries all million possible dollar amounts. (Probabilistic encryption solves the problem; see Section 23.15.) Even if P is not as well-defined, this attack can be very effective. Simply knowing that a ciphertext does not correspond to a particular plaintext can be useful information. Symmetric cryptosystems are not vulnerable to this attack because a cryptanalyst cannot perform trial encryptions with an unknown key.

In most practical implementations public-key cryptography is used to secure and distribute session keys; those session keys are used with symmetric algorithms to secure message traffic [879]. This is sometimes called a hybrid cryptosystem.

(1) Bob sends Alice his public key.
(2) Alice generates a random session key, K, encrypts it using Bob's public key, and sends it to Bob.
EB(K)
(3) Bob decrypts Alice's message using his private key to recover the session key.
DB(EB(K)) = K
(4) Both of them encrypt their communications using the same session key.

Using public-key cryptography for key distribution solves a very important key-management problem. With symmetric cryptography, the data encryption key sits around until it is used. If Eve ever gets her hands on it, she can decrypt messages encrypted with it. With the previous protocol, the session key is created when it is needed to encrypt communications and destroyed when it is no longer needed. This drastically reduces the risk of compromising the session key. Of course, the private key is vulnerable to compromise, but it is at less risk because it is only used once per communication to encrypt a session key. This is further discussed in Section 3.1.

Merkle's Puzzles

Ralph Merkle invented the first construction of public-key cryptography. In 1974 he registered for a course in computer security at the University of California, Berkeley, taught by Lance Hoffman. His term paper topic, submitted early in the term, addressed the problem of "Secure Communication over Insecure Channels" [1064]. Hoffman could not understand Merkle's proposal and eventually Merkle dropped the course. He continued to work on the problem, despite continuing failure to make his results understood.

Merkle's technique was based on "puzzles" that were easier to solve for the sender and receiver than for an eavesdropper. Here's how Alice sends an encrypted message to Bob without first having to exchange a key with him.

(1) Bob generates 220, or about a million, messages of the form: "This is puzzle number x. This is the secret key number y," where x is a random number and y is a random secret key. Both x and y are different for each message. Using a symmetric algorithm, he encrypts each message with a different 20-bit key and sends them all to Alice.
(2) Alice chooses one message at random and performs a brute-force attack to recover the plaintext. This is a large, but not impossible, amount of work.
(3) Alice encrypts her secret message with the key she recovered and some symmetric algorithm, and sends it to Bob along with x.
(4) Bob knows which secret key y he encrypts in message x, so he can decrypt the message.

Eve can break this system, but she has to do far more work than either Alice or Bob. To recover the message in step (3), she has to perform a brute-force attack against each of Bob's 220 messages in step (1); this attack has a complexity of 240. The x values won't help Eve either; they were assigned randomly in step (1). In general, Eve has to expend approximately the square of the effort that Alice expends.

This n to n2 advantage is small by cryptographic standards, but in some circumstances it may be enough. If Alice and Bob can try ten thousand keys per second, it will take them a minute each to perform their steps and another minute to communicate the puzzles from Bob to Alice on a 1.544 MB link. If Eve had comparable computing facilities, it would take her about a year to break the system. Other algorithms are even harder to break.

2.6 Digital Signatures

Handwritten signatures have long been used as proof of authorship of, or at least agreement with, the contents of a document. What is it about a signature that is so compelling [1392]?

1. The signature is authentic. The signature convinces the document's recipient that the signer deliberately signed the document.
2. The signature is unforgeable. The signature is proof that the signer, and no one else, deliberately signed the document.
3. The signature is not reusable. The signature is part of the document; an unscrupulous person cannot move the signature to a different document.
4. The signed document is unalterable. After the document is signed, it cannot be altered.
5. The signature cannot be repudiated. The signature and the document are physical things. The signer cannot later claim that he or she didn't sign it.

In reality, none of these statements about signatures is completely true. Signatures can be forged, signatures can be lifted from one piece of paper and moved to another, and documents can be altered after signing. However, we are willing to live with these problems because of the difficulty in cheating and the risk of detection.

We would like to do this sort of thing on computers, but there are problems. First, computer files are trivial to copy. Even if a person's signature were difficult to forge (a graphical image of a written signature, for example), it would be easy to cut and paste a valid signature from one document to another document. The mere presence of such a signature means nothing. Second, computer files are easy to modify after they are signed, without leaving any evidence of modification.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Signing Documents with Symmetric Cryptosystems and an Arbitrator

Alice wants to sign a digital message and send it to Bob. With the help of Trent and a symmetric cryptosystem, she can.

Trent is a powerful, trusted arbitrator. He can communicate with both Alice and Bob (and everyone else who may want to sign a digital document). He shares a secret key, KA, with Alice, and a different secret key, KB, with Bob. These keys have been established long before the protocol begins and can be reused multiple times for multiple signings.

(1) Alice encrypts her message to Bob with KA and sends it to Trent.
(2) Trent decrypts the message with KA.
(3) Trent takes the decrypted message and a statement that he has received this message from Alice, and encrypts the whole bundle with KB.
(4) Trent sends the encrypted bundle to Bob.
(5) Bob decrypts the bundle with KB. He can now read both the message and Trent's certification that Alice sent it.

How does Trent know that the message is from Alice and not from some imposter? He infers it from the message's encryption. Since only he and Alice share their secret key, only Alice could encrypt a message using it.

Is this as good as a paper signature? Let's look at the characteristics we want:

1. This signature is authentic. Trent is a trusted arbitrator and Trent knows that the message came from Alice. Trent's certification serves as proof to Bob.
2. This signature is unforgeable. Only Alice (and Trent, but everyone trusts him) knows KA, so only Alice could have sent Trent a message encrypted with KA. If someone tried to impersonate Alice, Trent would have immediately realized this in step (2) and would not certify its authenticity.
3. This signature is not reusable. If Bob tried to take Trent's certification and attach it to another message, Alice would cry foul. An arbitrator (it could be Trent or it could be a completely different arbitrator with access to the same information) would ask Bob to produce both the message and Alice's encrypted message. The arbitrator would then encrypt the message with KA and see that it did not match the encrypted message that Bob gave him. Bob, of course, could not produce an encrypted message that matches because he does not know KA.
4. The signed document is unalterable. Were Bob to try to alter the document after receipt, Trent could prove foul play in exactly the same manner just described.
5. The signature cannot be repudiated. Even if Alice later claims that she never sent the message, Trent's certification says otherwise. Remember, Trent is trusted by everyone; what he says is true.

If Bob wants to show Carol a document signed by Alice, he can't reveal his secret key to her. He has to go through Trent again:

(1) Bob takes the message and Trent's statement that the message came from Alice, encrypts them with KB, and sends them back to Trent.
(2) Trent decrypts the bundle with KB.
(3) Trent checks his database and confirms that the original message came from Alice.
(4) Trent re-encrypts the bundle with the secret key he shares with Carol, KC, and sends it to Carol.
(5) Carol decrypts the bundle with KC. She can now read both the message and Trent's certification that Alice sent it.

These protocols work, but they're time-consuming for Trent. He must spend his days decrypting and encrypting messages, acting as the intermediary between every pair of people who want to send signed documents to one another. He must keep a database of messages (although this can be avoided by sending the recipient a copy of the sender's encrypted message). He is a bottleneck in any communications system, even if he's a mindless software program.

Harder still is creating and maintaining someone like Trent, someone that everyone on the network trusts. Trent has to be infallible; if he makes even one mistake in a million signatures, no one is going to trust him. Trent has to be completely secure. If his database of secret keys ever got out or if someone managed to modify his programming, everyone's signatures would be completely useless. False documents purported to be signed years ago could appear. Chaos would result. Governments would collapse. Anarchy would reign. This might work in theory, but it doesn't work very well in practice.

Digital Signature Trees

Ralph Merkle proposed a digital signature scheme based on secret-key cryptography, producing an infinite number of one-time signatures using a tree structure [1067,1068]. The basic idea of this scheme is to place the root of the tree in some public file, thereby authenticating it. The root signs one message and authenticates its sub-nodes in the tree. Each of these nodes signs one message and authenticates its sub-nodes, and so on.

Signing Documents with Public-Key Cryptography

There are public-key algorithms that can be used for digital signatures. In some algorithms—RSA is an example (see Section 19.3)—either the public key or the private key can be used for encryption. Encrypt a document using your private key, and you have a secure digital signature. In other cases—DSA is an example (see Section 20.1)—there is a separate algorithm for digital signatures that cannot be used for encryption. This idea was first invented by Diffie and Hellman [496] and further expanded and elaborated on in other texts [1282,1328,1024,1283,426]. See [1099] for a good survey of the field.

The basic protocol is simple:

(1) Alice encrypts the document with her private key, thereby signing the document.
(2) Alice sends the signed document to Bob.
(3) Bob decrypts the document with Alice's public key, thereby verifying the signature.

This protocol is far better than the previous one. Trent is not needed to either sign or verify signatures. (He is needed to certify that Alice's public key is indeed her public key.) The parties do not even need Trent to resolve disputes: If Bob cannot perform step (3), then he knows the signature is not valid.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This protocol also satisfies the characteristics we're looking for:

1. The signature is authentic; when Bob verifies the message with Alice's public key, he knows that she signed it.
2. The signature is unforgeable; only Alice knows her private key.
3. The signature is not reusable; the signature is a function of the document and cannot be transferred to any other document.
4. The signed document is unalterable; if there is any alteration to the document, the signature can no longer be verified with Alice's public key.
5. The signature cannot be repudiated. Bob doesn't need Alice's help to verify her signature.

Signing Documents and Timestamps

Actually, Bob can cheat Alice in certain circumstances. He can reuse the document and signature together. This is no problem if Alice signed a contract (what's another copy of the same contract, more or less?), but it can be very exciting if Alice signed a digital check.

Let's say Alice sends Bob a signed digital check for $100. Bob takes the check to the bank, which verifies the signature and moves the money from one account to the other. Bob, who is an unscrupulous character, saves a copy of the digital check. The following week, he again takes it to the bank (or maybe to a different bank). The bank verifies the signature and moves the money from one account to the other. If Alice never balances her checkbook, Bob can keep this up for years.

Consequently, digital signatures often include timestamps. The date and time of the signature are attached to the message and signed along with the rest of the message. The bank stores this timestamp in a database. Now, when Bob tries to cash Alice's check a second time, the bank checks the timestamp against its database. Since the bank already cashed a check from Alice with the same timestamp, the bank calls the police. Bob then spends 15 years in Leavenworth prison reading up on cryptographic protocols.

Signing Documents with Public-Key Cryptography and One-Way Hash Functions

In practical implementations, public-key algorithms are often too inefficient to sign long documents. To save time, digital signature protocols are often implemented with one-way hash functions [432,433]. Instead of signing a document, Alice signs the hash of the document. In this protocol, both the one-way hash function and the digital signature algorithm are agreed upon beforehand.

(1) Alice produces a one-way hash of a document.
(2) Alice encrypts the hash with her private key, thereby signing the document.
(3) Alice sends the document and the signed hash to Bob.
(4) Bob produces a one-way hash of the document that Alice sent. He then, using the digital signature algorithm, decrypts the signed hash with Alice's public key. If the signed hash matches the hash he generated, the signature is valid.

Speed increases drastically and, since the chances of two different documents having the same 160-bit hash are only one in 2160, anyone can safely equate a signature of the hash with a signature of the document. If a non-one-way hash function were used, it would be an easy matter to create multiple documents that hashed to the same value, so that anyone signing a particular document would be duped into signing a multitude of documents.

This protocol has other benefits. First, the signature can be kept separate from the document. Second, the recipient's storage requirements for the document and signature are much smaller. An archival system can use this type of protocol to verify the existence of documents without storing their contents. The central database could just store the hashes of files. It doesn't have to see the files at all; users submit their hashes to the database, and the database timestamps the submissions and stores them. If there is any disagreement in the future about who created a document and when, the database could resolve it by finding the hash in its files. This system has vast implications concerning privacy: Alice could copyright a document but still keep the document secret. Only if she wished to prove her copyright would she have to make the document public. (See Section 4.1).

Algorithms and Terminology

There are many digital signature algorithms. All of them are public-key algorithms with secret information to sign documents and public information to verify signatures. Sometimes the signing process is called encrypting with a private key and the verification process is called decrypting with a public key. This is misleading and is only true for one algorithm, RSA. And different algorithms have different implementations. For example, one-way hash functions and timestamps sometimes add extra steps to the process of signing and verifying. Many algorithms can be used for digital signatures, but not for encryption.

In general, I will refer to the signing and verifying processes without any details of the algorithms involved. Signing a message with private key K is:

SK(M)

and verifying a signature with the corresponding public key is:

VK(M)

The bit string attached to the document when signed (in the previous example, the one-way hash of the document encrypted with the private key) will be called the digital signature, or just the signature. The entire protocol, by which the receiver of a message is convinced of the identity of the sender and the integrity of the message, is called authentication. Further details on these protocols are in Section 3.2.

Multiple Signatures

How could Alice and Bob sign the same digital document? Without one-way hash functions, there are two options. One is that Alice and Bob sign separate copies of the document itself. The resultant message would be over twice the size of the original document. The second is that Alice signs the document first and then Bob signs Alice's signature. This works, but it is impossible to verify Alice's signature without also verifying Bob's.

With one-way hash functions, multiple signatures are easy:

(1) Alice signs the hash of the document.
(2) Bob signs the hash of the document.
(3) Bob sends his signature to Alice.
(4) Alice sends the document, her signature, and Bob's signature to Carol.
(5) Carol verifies both Alice's signature and Bob's signature.

Alice and Bob can do steps (1) and (2) either in parallel or in series. In step (5), Carol can verify one signature without having to verify the other.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Nonrepudiation and Digital Signatures

Alice can cheat with digital signatures and there's nothing that can be done about it. She can sign a document and then later claim that she did not. First, she signs the document normally. Then, she anonymously publishes her private key, conveniently loses it in a public place, or just pretends to do either one. Alice then claims that her signature has been compromised and that others are using it, pretending to be her. She disavows signing the document and any others that she signed using that private key. This is called repudiation.

Timestamps can limit the effects of this kind of cheating, but Alice can always claim that her key was compromised earlier. If Alice times things well, she can sign a document and then successfully claim that she didn't. This is why there is so much talk about private keys buried in tamper-resistant modules—so that Alice can't get at hers and abuse it.

Although nothing can be done about this possible abuse, one can take steps to guarantee that old signatures are not invalidated by actions taken in disputing new ones. (For example, Alice could "lose" her key to keep from paying Bob for the junk car he sold her yesterday and, in the process, invalidate her bank account.) The solution is for the receiver of a signed document to have it timestamped [453].

The general protocol is given in [28]:

(1) Alice signs a message.
(2) Alice generates a header containing some identifying information. She concatenates the header with the signed message, signs that, and sends it to Trent.
(3) Trent verifies the outside signature and confirms the identifying information. He adds a timestamp to Alice's signed message and the identifying information. Then he signs it all and sends it to both Alice and Bob.
(4) Bob verifies Trent's signature, the identifying information, and Alice's signature.
(5) Alice verifies the message Trent sent to Bob. If she did not originate the message, she speaks up quickly.

Another scheme uses Trent after the fact [209]. After receiving a signed message, Bob can send a copy to Trent for verification. Trent can attest to the validity of Alice's signature.

Applications of Digital Signatures

One of the earliest proposed applications of digital signatures was to facilitate the verification of nuclear test ban treaties [1454,1467]. The United States and the Soviet Union (anyone remember the Soviet Union?) permitted each other to put seismometers on the other's soil to monitor nuclear tests. The problem was that each country needed to assure itself that the host nation was not tampering with the data from the monitoring nation's seismometers. Simultaneously, the host nation needed to assure itself that the monitor was sending only the specific information needed for monitoring.

Conventional authentication techniques can solve the first problem, but only digital signatures can solve both problems. The host nation can read, but not alter, data from the seismometer, and the monitoring nation knows that the data has not been tampered with.

2.7 Digital Signatures with Encryption

By combining digital signatures with public-key cryptography, we develop a protocol that combines the security of encryption with the authenticity of digital signatures. Think of a letter from your mother: The signature provides proof of authorship and the envelope provides privacy.

(1) Alice signs the message with her private key.
SA(M)
(2) Alice encrypts the signed message with Bob's public key and sends it to Bob.
EB(SA(M))
(3) Bob decrypts the message with his private key.
DB(EB(SA(M))) = SA(M)
(4) Bob verifies with Alice's public key and recovers the message.
VA(SA(M)) = M

Signing before encrypting seems natural. When Alice writes a letter, she signs it and then puts it in an envelope. If she put the letter in the envelope unsigned and then signed the envelope, then Bob might worry if the letter hadn't been covertly replaced. If Bob showed to Carol Alice's letter and envelope, Carol might accuse Bob of lying about which letter arrived in which envelope.

In electronic correspondence as well, signing before encrypting is a prudent practice [48]. Not only is it more secure—an adversary can't remove a signature from an encrypted message and add his own—but there are legal considerations: If the text to be signed is not visible to the signer when he affixes his signature, then the signature may have little legal force [1312]. And there are some cryptanalytic attacks against this technique with RSA signatures (see Section 19.3).

There's no reason Alice has to use the same public-key/private-key key pair for encrypting and signing. She can have two key pairs: one for encryption and the other for signatures. Separation has its advantages: she can surrender her encryption key to the police without compromising her signature, one key can be escrowed (see Section 4.13) without affecting the other, and the keys can have different sizes and can expire at different times.

Of course, timestamps should be used with this protocol to prevent reuse of messages. Timestamps can also protect against other potential pitfalls, such as the one described below.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Resending the Message as a Receipt

Consider an implementation of this protocol, with the additional feature of confirmation messages. Whenever Bob receives a message, he returns it as a confirmation of receipt.

(1) Alice signs a message with her private key, encrypts it with Bob's public key, and sends it to Bob.
EB(SA(M))
(2) Bob decrypts the message with his private key and verifies the signature with Alice's public key, thereby verifying that Alice signed the message and recovering the message.
VA(DB(EB(SA(M)))) = M
(3) Bob signs the message with his private key, encrypts it with Alice's public key, and sends it back to Alice.
EA(SB(M))
(4) Alice decrypts the message with her private key and verifies the signature with Bob's public key. If the resultant message is the same one she sent to Bob, she knows that Bob received the message accurately.

If the same algorithm is used for both encryption and digital-signature verification there is a possible attack [506]. In these cases, the digital signature operation is the inverse of the encryption operation: VX = EX and SX = DX.

Assume that Mallory is a legitimate system user with his own public and private key. Now, let's watch as he reads Bob's mail. First, he records Alice's message to Bob in step (1). Then, at some later time, he sends that message to Bob, claiming that it came from him (Mallory). Bob thinks that it is a legitimate message from Mallory, so he decrypts the message with his private key and then tries to verify Mallory's signature by decrypting it with Mallory's public key. The resultant message, which is pure gibberish, is:

EM(DB(EB(DA(M)))) = EM(DA(M))

Even so, Bob goes on with the protocol and sends Mallory a receipt:

EM(DB(EM(DA M))))

Now, all Mallory has to do is decrypt the message with his private key, encrypt it with Bob's public key, decrypt it again with his private key, and encrypt it with Alice's public key. Voilà! Mallory has M.

It is not unreasonable to imagine that Bob may automatically send Mallory a receipt. This protocol may be embedded in his communications software, for example, and send receipts automatically. It is this willingness to acknowledge the receipt of gibberish that creates the insecurity. If Bob checked the message for comprehensibility before sending a receipt, he could avoid this security problem.

There are enhancements to this attack that allow Mallory to send Bob a different message from the one he eavesdropped on. Never sign arbitrary messages from other people or decrypt arbitrary messages and give the results to other people.

Foiling the Resend Attack

The attack just described works because the encrypting operation is the same as the signature-verifying operation and the decryption operation is the same as the signature operation. A secure protocol would use even a slightly different operation for encryption and digital signatures. Using different keys for each operation solves the problem, as does using different algorithms for each operation; as do timestamps, which make the incoming message and the outgoing message different; as do digital signatures with one-way hash functions (see Section 2.6).

In general, then, the following protocol is secure as the public-key algorithm used:

(1) Alice signs a message.
(2) Alice encrypts the message and signature with Bob's public key (using a different encryption algorithm than for the signature) and sends it to Bob.
(3) Bob decrypts the message with his private key.
(4) Bob verifies Alice's signature.

Attacks against Public-Key Cryptography

In all these public-key cryptography protocols, I glossed over how Alice gets Bob's public key. Section 3.1 discusses this in detail, but it is worth mentioning here.

The easiest way to get someone's public key is from a secure database somewhere. The database has to be public, so that anyone can get anyone else's public key. The database also has to be protected from write-access by anyone except Trent; otherwise Mallory could substitute any public key for Bob's. After he did that, Bob couldn't read messages addressed to him, but Mallory could.

Even if the public keys are stored in a secure database, Mallory could still substitute one for another during transmission. To prevent this, Trent can sign each public key with his own private key. Trent, when used in this manner, is often known as a Key Certification Authority or Key Distribution Center (KDC). In practical implementations, the KDC signs a compound message consisting of the user's name, his public key, and any other important information about the user. This signed compound message is stored in the KDC's database. When Alice gets Bob's key, she verifies the KDC's signature to assure herself of the key's validity.

In the final analysis, this is not making things impossible for Mallory, only more difficult. Alice still has the KDC's public key stored somewhere. Mallory would have to substitute his own public key for that key, corrupt the database, and substitute his own keys for the valid keys (all signed with his private key as if he were the KDC), and then he's in business. But, even paper-based signatures can be forged if Mallory goes to enough trouble. Key exchange will be discussed in minute detail in Section 3.1.

2.8 Random and Pseudo-Random-Sequence Generation

Why even bother with random-number generation in a book on cryptography? There's already a random-number generator built into most every compiler, a mere function call away. Why not use that? Unfortunately, those random-number generators are almost definitely not secure enough for cryptography, and probably not even very random. Most of them are embarrassingly bad.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Random-number generators are not random because they don't have to be. Most simple applications, like computer games, need so few random numbers that they hardly notice. However, cryptography is extremely sensitive to the properties of random-number generators. Use a poor random-number generator and you start getting weird correlations and strange results [1231,1238]. If you are depending on your random-number generator for security, weird correlations and strange results are the last things you want.

The problem is that a random-number generator doesn't produce a random sequence. It probably doesn't produce anything that looks even remotely like a random sequence. Of course, it is impossible to produce something truly random on a computer. Donald Knuth quotes John von Neumann as saying: "Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin" [863]. Computers are deterministic beasts: Stuff goes in one end, completely predictable operations occur inside, and different stuff comes out the other end. Put the same stuff in on two separate occasions and the same stuff comes out both times. Put the same stuff into two identical computers, and the same stuff comes out of both of them. A computer can only be in a finite number of states (a large finite number, but a finite number nonetheless), and the stuff that comes out will always be a deterministic function of the stuff that went in and the computer's current state. That means that any random-number generator on a computer (at least, on a finite-state machine) is, by definition, periodic. Anything that is periodic is, by definition, predictable. And if something is predictable, it can't be random. A true random-number generator requires some random input; a computer can't provide that.

Pseudo-Random Sequences

The best a computer can produce is a pseudo-random-sequence generator. What's that? Many people have taken a stab at defining this formally, but I'll hand-wave here. A pseudo-random sequence is one that looks random. The sequence's period should be long enough so that a finite sequence of reasonable length—that is, one that is actually used—is not periodic. If you need a billion random bits, don't choose a sequence generator that repeats after only sixteen thousand bits. These relatively short nonperiodic subsequences should be as indistinguishable as possible from random sequences. For example, they should have about the same number of ones and zeros, about half the runs (sequences of the same bit) should be of length one, one quarter of length two, one eighth of length three, and so on. They should not be compressible. The distribution of run lengths for zeros and ones should be the same [643,863,99,1357]. These properties can be empirically measured and then compared to statistical expectations using a chi-square test.

For our purposes, a sequence generator is pseudo-random if it has this property:

1. It looks random. This means that it passes all the statistical tests of randomness that we can find. (Start with the ones in [863].)

A lot of effort has gone into producing good pseudo-random sequences on computer. Discussions of generators abound in the academic literature, along with various tests of randomness. All of these generators are periodic (there's no escaping that); but with potential periods of 2256 bits and higher, they can be used for the largest applications.

The problem is still those weird correlations and strange results. Every pseudo-random-sequence generator is going to produce them if you use them in a certain way. And that's what a cryptanalyst will use to attack the system.

Cryptographically Secure Pseudo-Random Sequences

Cryptographic applications demand much more of a pseudo-random-sequence generator than do most other applications. Cryptographic randomness doesn't mean just statistical randomness, although that's part of it. For a sequence to be cryptographically secure pseudo-random, it must also have this property:

2. It is unpredictable. It must be computationally infeasible to predict what the next random bit will be, given complete knowledge of the algorithm or hardware generating the sequence and all of the previous bits in the stream.

Cryptographically secure pseudo-random sequences should not be compressible...unless you know the key. The key is generally the seed used to set the initial state of the generator.

Like any cryptographic algorithm, cryptographically secure pseudo-random-sequence generators are subject to attack. Just as it is possible to break an encryption algorithm, it is possible to break a cryptographically secure pseudo-random-sequence generator. Making generators resistant to attack is what cryptography is all about.

Real Random Sequences

Now we're drifting into the domain of philosophers. Is there such a thing as randomness? What is a random sequence? How do you know if a sequence is random? Is "101110100" more random than "101010101"? Quantum mechanics tells us that there is honest-to-goodness randomness in the real world. But can we preserve that randomness in the deterministic world of computer chips and finite-state machines?

Philosophy aside, from our point of view a sequence generator is real random if it has this additional third property:

3. It cannot be reliably reproduced. If you run the sequence generator twice with the exact same input (at least as exact as humanly possible), you will get two completely unrelated random sequences.

The output of a generator satisfying these three properties will be good enough for a one-time pad, key generation, and any other cryptographic applications that require a truly random sequence generator. The difficulty is in determining whether a sequence is really random. If I repeatedly encrypt a string with DES and a given key, I will get a nice, random-looking output; you won't be able to tell that it's nonrandom unless you rent time on the NSA's DES cracker.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 3
Basic Protocols

3.1 Key Exchange

A common cryptographic technique is to encrypt each individual conversation with a separate key. This is called a session key, because it is used for only one particular communications session. As discussed in Section 8.5, session keys are useful because they only exist for the duration of the communication. How this common session key gets into the hands of the conversants can be a complicated matter.

Key Exchange with Symmetric Cryptography

This protocol assumes that Alice and Bob, users on a network, each share a secret key with the Key Distribution Center (KDC) [1260]—Trent in our protocols. These keys must be in place before the start of the protocol. (The protocol ignores the very real problem of how to distribute these secret keys; just assume they are in place and Mallory has no idea what they are.)

(1) Alice calls Trent and requests a session key to communicate with Bob.
(2) Trent generates a random session key. He encrypts two copies of it: one in Alice's key and the other in Bob's key. Trent sends both copies to Alice.
(3) Alice decrypts her copy of the session key.
(4) Alice sends Bob his copy of the session key.
(5) Bob decrypts his copy of the session key.
(6) Both Alice and Bob use this session key to communicate securely.

This protocol relies on the absolute security of Trent, who is more likely to be a trusted computer program than a trusted individual. If Mallory corrupts Trent, the whole network is compromised. He has all of the secret keys that Trent shares with each of the users; he can read all past communications traffic that he has saved, and all future communications traffic. All he has to do is to tap the communications lines and listen to the encrypted message traffic.

The other problem with this system is that Trent is a potential bottleneck. He has to be involved in every key exchange. If Trent fails, that disrupts the entire system.

Key Exchange with Public-Key Cryptography

The basic hybrid cryptosystem was discussed in Section 2.5. Alice and Bob use public-key cryptography to agree on a session key, and use that session key to encrypt data. In some practical implementations, both Alice's and Bob's signed public keys will be available on a database. This makes the key-exchange protocol even easier, and Alice can send a secure message to Bob even if he has never heard of her:

(1) Alice gets Bob's public key from the KDC.
(2) Alice generates a random session key, encrypts it using Bob's public key, and sends it to Bob.
(3) Bob then decrypts Alice's message using his private key.
(4) Both of them encrypt their communications using the same session key.

Man-in-the-Middle Attack

While Eve cannot do better than try to break the public-key algorithm or attempt a ciphertext-only attack on the ciphertext, Mallory is a lot more powerful than Eve. Not only can he listen to messages between Alice and Bob, he can also modify messages, delete messages, and generate totally new ones. Mallory can imitate Bob when talking to Alice and imitate Alice when talking to Bob. Here's how the attack works:

(1) Alice sends Bob her public key. Mallory intercepts this key and sends Bob his own public key.
(2) Bob sends Alice his public key. Mallory intercepts this key and sends Alice his own public key.
(3) When Alice sends a message to Bob, encrypted in "Bob's" public key, Mallory intercepts it. Since the message is really encrypted with his own public key, he decrypts it with his private key, re-encrypts it with Bob's public key, and sends it on to Bob.
(4) When Bob sends a message to Alice, encrypted in "Alice's" public key, Mallory intercepts it. Since the message is really encrypted with his own public key, he decrypts it with his private key, re-encrypts it with Alice's public key, and sends it on to Alice.

Even if Alice's and Bob's public keys are stored on a database, this attack will work. Mallory can intercept Alice's database inquiry and substitute his own public key for Bob's. He can do the same to Bob and substitute his own public key for Alice's. Or better yet, he can break into the database surreptitiously and substitute his key for both Alice's and Bob's. Then he simply waits for Alice and Bob to talk with each other, intercepts and modifies the messages, and he has succeeded.

This man-in-the-middle attack works because Alice and Bob have no way to verify that they are talking to each other. Assuming Mallory doesn't cause any noticeable network delays, the two of them have no idea that someone sitting between them is reading all of their supposedly secret communications.

Interlock Protocol

The interlock protocol, invented by Ron Rivest and Adi Shamir [1327], has a good chance of foiling the man-in-the-middle attack. Here's how it works:

(1) Alice sends Bob her public key.
(2) Bob sends Alice his public key.
(3) Alice encrypts her message using Bob's public key. She sends half of the encrypted message to Bob.
(4) Bob encrypts his message using Alice's public key. He sends half of the encrypted message to Alice.
(5) Alice sends the other half of her encrypted message to Bob.
(6) Bob puts the two halves of Alice's message together and decrypts it with his private key. Bob sends the other half of his encrypted message to Alice.
(7) Alice puts the two halves of Bob's message together and decrypts it with her private key.

The important point is that half of the message is useless without the other half; it can't be decrypted. Bob cannot read any part of Alice's message until step (6); Alice cannot read any part of Bob's message until step (7). There are a number of ways to do this:

— If the encryption algorithm is a block algorithm, half of each block (e.g., every other bit) could be sent in each half message.
— Decryption of the message could be dependent on an initialization vector (see Section 9.3), which could be sent with the second half of the message.
— The first half of the message could be a one-way hash function of the encrypted message (see Section 2.4) and the encrypted message itself could be the second half.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

To see how this causes a problem for Mallory, let's review his attempt to subvert the protocol. He can still substitute his own public keys for Alice's and Bob's in steps (1) and (2). But now, when he intercepts half of Alice's message in step (3), he cannot decrypt it with his private key and re-encrypt it with Bob's public key. He must invent a totally new message and send half of it to Bob. When he intercepts half of Bob's message to Alice in step (4), he has the same problem. He cannot decrypt it with his private key and re-encrypt it with Alice's public key. He has to invent a totally new message and send half of it to Alice. By the time he intercepts the second halves of the real messages in steps (5) and (6), it is too late for him to change the new messages he invented. The conversation between Alice and Bob will necessarily be completely different.

Mallory could possibly get away with this scheme. If he knows Alice and Bob well enough to mimic both sides of a conversation between them, they might never realize that they are being duped. But surely this is much harder than sitting between the two of them, intercepting and reading their messages.

Key Exchange with Digital Signatures

Implementing digital signatures during a session-key exchange protocol circumvents this man-in-the-middle attack as well. Trent signs both Alice's and Bob's public keys. The signed keys include a signed certification of ownership. When Alice and Bob receive the keys, they each verify Trent's signature. Now they know that the public key belongs to that other person. The key exchange protocol can then proceed.

Mallory has serious problems. He cannot impersonate either Alice or Bob because he doesn't know either of their private keys. He cannot substitute his public key for either of theirs because, while he has one signed by Trent, it is signed as being Mallory's. All he can do is listen to the encrypted traffic go back and forth or disrupt the lines of communication and prevent Alice and Bob from talking.

This protocol uses Trent, but the risk of compromising the KDC is less than the first protocol. If Mallory compromises Trent (breaks into the KDC), all he gets is Trent's private key. This key enables him only to sign new keys; it does not let him decrypt any session keys or read any message traffic. To read the traffic, Mallory has to impersonate a user on the network and trick legitimate users into encrypting messages with his phony public key.

Mallory can launch that kind of attack. With Trent's private key, he can create phony signed keys to fool both Alice and Bob. Then, he can either exchange them in the database for real signed keys, or he can intercept users' database requests and reply with his phony keys. This enables him to launch a man-in-the-middle attack and read people's communications.

This attack will work, but remember that Mallory has to be able to intercept and modify messages. In some networks this is a lot more difficult than passively sitting on a network reading messages as they go by. On a broadcast channel, such as a radio network, it is almost impossible to replace one message with another—although the entire network can be jammed. On computer networks this is easier and seems to be getting easier every day. Consider IP spoofing, router attacks, and so forth; active attacks don't necessarily mean someone down a manhole with a datascope, and they are not limited to three-letter agencies.

Key and Message Transmission

Alice and Bob need not complete the key-exchange protocol before exchanging messages. In this protocol, Alice sends Bob the message, M, without any previous key exchange protocol:

(1) Alice generates a random session key, K, and encrypts M using K.
EK(M)
(2) Alice gets Bob's public key from the database.
(3) Alice encrypts K with Bob's public key.
EB(K)
(4) Alice sends both the encrypted message and encrypted session key to Bob.
EK(M), EB(K)

For added security against man-in-the-middle attacks, Alice can sign the transmission.
(5) Bob decrypts Alice's session key, K, using his private key.
(6) Bob decrypts Alice's message using the session key.

This hybrid system is how public-key cryptography is most often used in a communications system. It can be combined with digital signatures, timestamps, and any other security protocols.

Key and Message Broadcast

There is no reason Alice can't send the encrypted message to several people. In this example, Alice will send the encrypted message to Bob, Carol, and Dave:

(1) Alice generates a random session key, K, and encrypts M using K.
EK(M)
(2) Alice gets Bob's, Carol's, and Dave's public keys from the database.
(3) Alice encrypts K with Bob's public key, encrypts K with Carol's public key, and then encrypts K with Dave's public key.
EB(K), EC(K), ED(K)
(4) Alice broadcasts the encrypted message and all the encrypted keys to anybody who cares to receive it.
EB(K), EC(K), ED(K), EK(M)
(5) Only Bob, Carol, and Dave can decrypt the key, K, each using his or her private key.
(6) Only Bob, Carol, and Dave can decrypt Alice's message using K.

This protocol can be implemented on a store-and-forward network. A central server can forward Alice's message to Bob, Carol, and Dave along with their particular encrypted key. The server doesn't have to be secure or trusted, since it will not be able to decrypt any of the messages.

3.2 Authentication

When Alice logs into a host computer (or an automatic teller, or a telephone banking system, or any other type of terminal), how does the host know who she is? How does the host know she is not Eve trying to falsify Alice's identity? Traditionally, passwords solve this problem. Alice enters her password, and the host confirms that it is correct. Both Alice and the host know this secret piece of knowledge and the host requests it from Alice every time she tries to log in.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Authentication Using One-Way Functions

What Roger Needham and Mike Guy realized is that the host does not need to know the passwords; the host just has to be able to differentiate valid passwords from invalid passwords. This is easy with one-way functions [1599,526,1274,1121]. Instead of storing passwords, the host stores one-way functions of the passwords.

(1) Alice sends the host her password.
(2) The host performs a one-way function on the password.
(3) The host compares the result of the one-way function to the value it previously stored.

Since the host no longer stores a table of everybody's valid password, the threat of someone breaking into the host and stealing the password list is mitigated. The list of passwords operated on by the one-way function is useless, because the one-way function cannot be reversed to recover the passwords.

Dictionary Attacks and Salt

A file of passwords encrypted with a one-way function is still vulnerable. In his spare time, Mallory compiles a list of the 1,000,000 most common passwords. He operates on all 1,000,000 of them with the one-way function and stores the results. If each password is about 8 bytes, the resulting file will be no more than 8 megabytes; it will fit on a few floppy disks. Now, Mallory steals an encrypted password file. He compares that file with his file of encrypted possible passwords and sees what matches.

This is a dictionary attack, and it's surprisingly successful (see Section 8.1). Salt is a way to make it more difficult. Salt is a random string that is concatenated with passwords before being operated on by the one-way function. Then, both the salt value and the result of the one-way function are stored in a database on the host. If the number of possible salt values is large enough, this practically eliminates a dictionary attack against commonly used passwords because Mallory has to generate the one-way hash for each possible salt value. This is a simple attempt at an initialization vector (see Section 9.3).

The point here is to make sure that Mallory has to do a trial encryption of each password in his dictionary every time he tries to break another person's password, rather than just doing one massive precomputation for all possible passwords.

A lot of salt is needed. Most UNIX systems use only 12 bits of salt. Even with that, Daniel Klein developed a password-guessing program that often cracks 40 percent of the passwords on a given host system within a week [847,848] (see Section 8.1). David Feldmeier and Philip Karn compiled a list of about 732,000 common passwords concatenated with each of 4096 possible salt values. They estimate that 30 percent of passwords on any given host can be broken with this list [561].

Salt isn't a panacea; increasing the number of salt bits won't solve everything. Salt only protects against general dictionary attacks on a password file, not against a concerted attack on a single password. It protects people who have the same password on multiple machines, but doesn't make poorly chosen passwords any better.

SKEY

SKEY is an authentication program that relies on a one-way function for its security. It's easy to explain.

To set up the system, Alice enters a random number, R. The computer computes f(R), f(f(R)), f(f(f(R))), and so on, about a hundred times. Call these numbers x1, x2, x3,..., x100. The computer prints out this list of numbers, and Alice puts it in her pocket for safekeeping. The computer also stores x101, in the clear, in a login database next to Alice's name.

The first time Alice wants to log in, she types her name and x100. The computer calculates f(x100) and compares it with x101; if they match, Alice is authenticated. Then, the computer replaces x101 with x100 in the database. Alice crosses x100 off her list.

Every time Alice logs in, she enters the last uncrossed number on her list: xi. The computer calculates f(xi) and compares it with xi+1 stored in its database. Eve can't get any useful information because each number is only used once, and the function is one-way. Similarly, the database is not useful to an attacker. Of course, when Alice runs out of numbers on her list, she has to reinitialize the system.

Authentication Using Public-Key Cryptography

Even with salt, the first protocol has serious security problems. When Alice sends her password to her host, anyone who has access to her data path can read it. She might be accessing her host through a convoluted transmission path that passes through four industrial competitors, three foreign countries, and two forward-thinking universities. Eve can be at any one of those points, listening to Alice's login sequence. If Eve has access to the processor memory of the host, she can see the password before the host hashes it.

Public-key cryptography can solve this problem. The host keeps a file of every user's public key; all users keep their own private keys. Here is a na•ve attempt at a protocol. When logging in, the protocol proceeds as follows:

(1) The host sends Alice a random string.
(2) Alice encrypts the string with her private key and sends it back to the host, along with her name.
(3) The host looks up Alice's public key in its database and decrypts the message using that public key.
(4) If the decrypted string matches what the host sent Alice in the first place, the host allows Alice access to the system.

No one else has access to Alice's private key, so no one else can impersonate Alice. More important, Alice never sends her private key over the transmission line to the host. Eve, listening in on the interaction, cannot get any information that would enable her to deduce the private key and impersonate Alice.

The private key is both long and non-mnemonic, and will probably be processed automatically by the user's hardware or communications software. This requires an intelligent terminal that Alice trusts, but neither the host nor the communications path needs to be secure.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

It is foolish to encrypt arbitrary strings—not only those sent by untrusted third parties, but under any circumstances at all. Attacks similar to the one discussed in Section 19.3 can be mounted. Secure proof-of-identity protocols take the following, more complicated, form:

(1) Alice performs a computation based on some random numbers and her private key and sends the result to the host.
(2) The host sends Alice a different random number.
(3) Alice makes some computation based on the random numbers (both the ones she generated and the one she received from the host) and her private key, and sends the result to the host.
(4) The host does some computation on the various numbers received from Alice and her public key to verify that she knows her private key.
(5) If she does, her identity is verified.

If Alice does not trust the host any more than the host trusts Alice, then Alice will require the host to prove its identity in the same manner.

Step (1) might seem unnecessary and confusing, but it is required to prevent attacks against the protocol. Sections 21.1 and 21.2 mathematically describe several algorithms and protocols for proving identity. See also [935].

Mutual Authentication Using the Interlock Protocol

Alice and Bob are two users who want to authenticate each other. Each of them has a password that the other knows: Alice has PA and Bob has PB. Here's a protocol that will not work:

(1) Alice and Bob trade public keys.
(2) Alice encrypts PA with Bob's public key and sends it to him.
(3) Bob encrypts PB with Alice's public key and sends it to her.
(4) Alice decrypts what she received in step (2) and verifies that it is correct.
(5) Bob decrypts what he received in step (3) and verifies that it is correct.

Mallory can launch a successful man-in-the-middle attack (see Section 3.1):

(1) Alice and Bob trade public keys. Mallory intercepts both messages. He substitutes his public key for Bob's and sends it to Alice. Then he substitutes his public key for Alice's and sends it to Bob.
(2) Alice encrypts PA with "Bob's" public key and sends it to him. Mallory intercepts the message, decrypts PA with his private key, re-encrypts it with Bob's public key and sends it on to him.
(3) Bob encrypts PB with "Alice's" public key and sends it to her. Mallory intercepts the message, decrypts PB with his private key, re-encrypts it with Alice's public key, and sends it on to her.
(4) Alice decrypts PB and verifies that it is correct.
(5) Bob decrypts PA and verifies that it is correct.

Alice and Bob see nothing different. However, Mallory knows both PA and PB.

Donald Davies and Wyn Price describe how the interlock protocol (described in Section 3.1) can defeat this attack [435]. Steve Bellovin and Michael Merritt discuss ways to attack this protocol [110]. If Alice is a user and Bob is a host, Mallory can pretend to be Bob, complete the beginning steps of the protocol with Alice, and then drop the connection. True artistry demands Mallory do this by simulating line noise or network failure, but the final result is that Mallory has Alice's password. He can then connect with Bob and complete the protocol, thus getting Bob's password, too.

The protocol can be modified so that Bob gives his password before Alice, under the assumption that the user's password is much more sensitive than the host's password. This falls to a more complicated attack, also described in [110].

SKID

SKID2 and SKID3 are symmetric cryptography identification protocols developed for RACE's RIPE project [1305] (See Section 25.7). They use a MAC (see Section 2.4) to provide security and both assume that both Alice and Bob share a secret key, K.

SKID2 allows Bob to prove his identity to Alice. Here's the protocol:

(1) Alice chooses a random number, RA. (The RIPE document specifies a 64-bit number). She sends it to Bob.
(2) Bob chooses a random number, RB. (The RIPE document specifies a 64-bit number). He sends Alice:
RB,HK(RA,RB,B)

HK is the MAC. (The RIPE document suggests the RIPE-MAC function—see Section 18.14.) B is Bob's name.
(3) Alice computes HK(RA,RB,B) and compares it with what she received from Bob. If the results are identical, then Alice knows that she is communicating with Bob.

SKID3 provides mutual authentication between Alice and Bob. Steps (1) through (3) are identical to SKID2, and then the protocol proceeds with:

(4) Alice sends Bob:
HK(RB,A)

A is Alice's name.
(5) Bob computes HK(RB,A), and compares it with what he received from Alice. If the results are identical, then Bob knows that he is communicating with Alice.

This protocol is not secure against a man-in-the-middle attack. In general, a man-in-the-middle attack can defeat any protocol that doesn't involve a secret of some kind.

Message Authentication

When Bob receives a message from Alice, how does he know it is authentic? If Alice signed her message, this is easy. Alice's digital signature is enough to convince anyone that the message is authentic.

Symmetric cryptography provides some authentication. When Bob receives a message from Alice encrypted in their shared key, he knows it is from Alice. No one else knows their key. However, Bob has no way of convincing a third party of this fact. Bob can't show the message to Trent and convince him that it came from Alice. Trent can be convinced that the message came from either Alice or Bob (since no one else shared their secret key), but he has no way of knowing which one.

If the message is unencrypted, Alice could also use a MAC. This also convinces Bob that the message is authentic, but has the same problems as symmetric cryptography solutions.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

3.3 Authentication and Key Exchange

These protocols combine authentication with key exchange to solve a general computer problem: Alice and Bob are on opposite ends of a network and want to talk securely. How can Alice and Bob exchange a secret key and at the same time each be sure that he or she is talking to the other and not to Mallory? Most of the protocols assume that Trent shares a different secret key with each participant, and that all of these keys are in place before the protocol begins.

The symbols used in these protocols are summarized in Table 3.1.

Wide-Mouth Frog

The Wide-Mouth Frog protocol [283,284] is probably the simplest symmetric key-management protocol that uses a trusted server. Both Alice and Bob share a secret key with Trent. The keys are just used for key distribution and not to encrypt any actual messages between users. Just by using two messages, Alice transfers a session key to Bob:

TABLE 3.1 Symbols used in authentication and key exchange protocols
A | Alice's name
B | Bob's name
EA | Encryption with a key Trent shares with Alice
EB | Encryption with a key Trent shares with Bob
I | Index number
K | A random session key
L | Lifetime
TA,TB | A timestamp
RA,RB | A random number, sometimes called a nonce, chosen by Alice and Bob respectively
(1) Alice concatenates a timestamp, Bob's name, and a random session key and encrypts the whole message with the key she shares with Trent. She sends this to Trent, along with her name.
A,EA(TA,B,K)
(2) Trent decrypts the message from Alice. Then he concatenates a new timestamp, Alice's name, and the random session key; he encrypts the whole message with the key he shares with Bob. Trent sends to Bob:
EB(TB,A,K)

The biggest assumption made in this protocol is that Alice is competent enough to generate good session keys. Remember that random numbers aren't easy to generate; it might be more than Alice can be trusted to do properly.

Yahalom

In this protocol, both Alice and Bob share a secret key with Trent [283,284].

(1) Alice concatenates her name and a random number, and sends it to Bob.
A,RA
(2) Bob concatenates Alice's name, Alice's random number, his own random number, and encrypts it with the key he shares with Trent. He sends this to Trent, along with his name.
B,EB(A,RA,RB)
(3) Trent generates two messages. The first consists of Bob's name, a random session key, Alice's random number, and Bob's random number, all encrypted with the key he shares with Alice. The second consists of Alice's Zname and the random session key, encrypted with the key he shares with Bob. He sends both messages to Alice.
EA(B,K,RA,RB),EB(A,K)
(4) Alice decrypts the first message, extracts K, and confirms that RA has the same value as it did in step (1). Alice sends Bob two messages. The first is the message received from Trent, encrypted with Bob's key. The second is RB, encrypted with the session key.
EB(A,K),EK(RB)
(5) Bob decrypts the message encrypted with his key, extracts K, and confirms that RB has the same value as it did in step (2).

At the end, Alice and Bob are each convinced that they are talking to the other and not to a third party. The novelty here is that Bob is the first one to contact Trent, who only sends one message to Alice.

Needham-Schroeder

This protocol, invented by Roger Needham and Michael Schroeder [1159], also uses symmetric cryptography and Trent.

(1) Alice sends a message to Trent consisting of her name, Bob's name, and a random number.
A,B,RA
(2) Trent generates a random session key. He encrypts a message consisting of a random session key and Alice's name with the secret key he shares with Bob. Then he encrypts Alice's random value, Bob's name, the key, and the encrypted message with the secret key he shares with Alice. Finally, he sends her the encrypted message:
EA(RA,B,K,EB(K,A))
(3) Alice decrypts the message and extracts K. She confirms that RA is the same value that she sent Trent in step (1). Then she sends Bob the message that Trent encrypted in his key.
EB(K,A)
(4) Bob decrypts the message and extracts K. He then generates another random value, RB. He encrypts the message with K and sends it to Alice.
EK(RB)
(5) Alice decrypts the message with K. She generates RB - 1 and encrypts it with K. Then she sends the message back to Bob.
EK(RB - 1)
(6) Bob decrypts the message with K and verifies that it is RB - 1.

All of this fussing around with RA and RB and RB - 1 is to prevent replay attacks. In this attack, Mallory can record old messages and then use them later in an attempt to subvert the protocol. The presence of RA in step (2) assures Alice that Trent's message is legitimate and not a replay of a response from a previous execution of the protocol. When Alice successfully decrypts RB and sends Bob RB - 1 in step (5), Bob is ensured that Alice's messages are not replays from an earlier execution of the protocol.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The major security hole in this protocol is that old session keys are valuable. If Mallory gets access to an old K, he can launch a successful attack [461]. All he has to do is record Alice's messages to Bob in step (3). Then, once he has K, he can pretend to be Alice:

(1) Mallory sends Bob the following message:
EB(K,A)
(2) Bob extracts K, generates RB, and sends Alice:
EK(RB)
(3) Mallory intercepts the message, decrypts it with K, and sends Bob:
EK(RB - 1)
(4) Bob verifies that "Alice's" message is RB - 1.

Now, Mallory has Bob convinced that he is Alice.

A stronger protocol, using timestamps, can defeat this attack [461,456]. A time-stamp is added to Trent's message in step (2) encrypted with Bob's key: EB(K,A,T). Timestamps require a secure and accurate system clock—not a trivial problem in itself.

If the key Trent shares with Alice is ever compromised, the consequences are drastic. Mallory can use it to obtain session keys to talk with Bob (or anyone else he wishes to talk to). Even worse, Mallory can continue to do this even after Alice changes her key [90].

Needham and Schroeder attempted to correct these problems in a modified version of their protocol [1160]. Their new protocol is essentially the same as the Otway-Rees protocol, published in the same issue of the same journal.

Otway-Rees

This protocol also uses symmetric cryptography [1224].

(1) Alice generates a message consisting of an index number, her name, Bob's name, and a random number, all encrypted in the key she shares with Trent. She sends this message to Bob along with the index number, her name, and his name:
I,A,B,EA(RA,I,A,B)
(2) Bob generates a message consisting of a new random number, the index number, Alice's name, and Bob's name, all encrypted in the key he shares with Trent. He sends it to Trent, along with Alice's encrypted message, the index number, her name, and his name:
I,A,B,EA(RA,I,A,B),EB(RB,I,A,B)
(3) Trent generates a random session key. Then he creates two messages. One is Alice's random number and the session key, encrypted in the key he shares with Alice. The other is Bob's random number and the session key, encrypted in the key he shares with Bob. He sends these two messages, along with the index number, to Bob:
I,EA(RA,K),EB(RB,K)
(4) Bob sends Alice the message encrypted in her key, along with the index number:
I,EA(RA,K)
(5) Alice decrypts the message to recover her key and random number. She then confirms that both have not changed in the protocol.

Assuming that all the random numbers match, and the index number hasn't changed along the way, Alice and Bob are now convinced of each other's identity, and they have a secret key with which to communicate.

Kerberos

Kerberos is a variant of Needham-Schroeder and is discussed in detail in Section 24.5. In the basic Kerberos Version 5 protocol, Alice and Bob each share keys with Trent. Alice wants to generate a session key for a conversation with Bob.

(1) Alice sends a message to Trent with her identity and Bob's identity.
A,B
(2) Trent generates a message with a timestamp, a lifetime, L, a random session key, and Alice's identity. He encrypts this in the key he shares with Bob. Then he takes the timestamp, the lifetime, the session key, and Bob's identity, and encrypts these in the key he shares with Alice. He sends both encrypted messages to Alice.
EA(T,L,K,B),EB(T,L,K,A)
(3) Alice generates a message with her identity and the timestamp, encrypts it in K, and sends it to Bob. Alice also sends Bob the message encrypted in Bob's key from Trent.
EK(A,T),EB(T,L,K,A)
(4) Bob creates a message consisting of the timestamp plus one, encrypts it in K, and sends it to Alice.
EK(T + 1)

This protocol works, but it assumes that everyone's clocks are synchronized with Trent's clock. In practice, the effect is obtained by synchronizing clocks to within a few minutes of a secure time server and detecting replays within the time interval.

Neuman-Stubblebine

Whether by system faults or by sabotage, clocks can become unsynchronized. If the clocks get out of sync, there is a possible attack against most of these protocols [644]. If the sender's clock is ahead of the receiver's clock, Mallory can intercept a message from the sender and replay it later when the timestamp becomes current at the receiver's site. This attack is called suppress-replay and can have irritating consequences.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This protocol, first presented in [820] and corrected in [1162] attempts to counter the suppress-replay attack. It is an enhancement to Yahalom and is an excellent protocol.

(1) Alice concatenates her name and a random number and sends it to Bob.
A,RA
(2) Bob concatenates Alice's name, her random number, and a timestamp, and encrypts with the key he shares with Trent. He sends it to Trent along with his name and a new random number.
B,RB,EB(A,RA,TB)
(3) Trent generates a random session key. Then he creates two messages. The first is Bob's name, Alice's random number, a random session key, and the timestamp, all encrypted with the key he shares with Alice. The second is Alice's name, the session key, and the timestamp, all encrypted with the key he shares with Bob. He sends these both to Alice, along with Bob's random number.
EA(B,RA,K,TB),EA(A,K,TB),RB
(4) Alice decrypts the message encrypted with her key, extracts K, and confirms that RA has the same value as it did in step (1). Alice sends Bob two messages. The first is the message received from Trent, encrypted with Bob's key. The second is RB, encrypted with the session key.
EB(A,K,TB),EK(RB)
(5) Bob decrypts the message encrypted with his key, extracts K, and confirms that TB and RB have the same value they did in step (2).

Assuming both random numbers and the timestamp match, Alice and Bob are convinced of one another's identity and share a secret key. Synchronized clocks are not required because the timestamp is only relative to Bob's clock; Bob only checks the timestamp he generated himself.

One nice thing about this protocol is that Alice can use the message she received from Trent for subsequent authentication with Bob, within some predetermined time limit. Assume that Alice and Bob completed the above protocol, communicated, and then terminated the connection. Alice and Bob can reauthenticate in three steps, without having to rely on Trent.

(1) Alice sends Bob the message Trent sent her in step (3) and a new random number.
EB(A,K,TB),R'A
(2) Bob sends Alice another new random number, and Alice's new random number encrypted in their session key.
R'B,EK(R'A)
(3) Alice sends Bob his new random number, encrypted in their session key.
EK(R'B)

The new random numbers prevent replay attacks.

DASS

The Distributed Authentication Security Service (DASS) protocols, developed at Digital Equipment Corporation, also provide for mutual authentication and key exchange [604,1519,1518]. Unlike the previous protocols, DASS uses both public-key and symmetric cryptography. Alice and Bob each have a private key. Trent has signed copies of their public keys.

(1) Alice sends a message to Trent, consisting of Bob's name.
B
(2) Trent sends Alice Bob's public key, KB, signed with Trent's private key, T. The signed message includes Bob's name.
ST(B,KB)
(3) Alice verifies Trent's signature to confirm that the key she received is actually Bob's public key. She generates a random session key, and a random public-key/private-key key pair: KP. She encrypts a timestamp with K. Then she signs a key lifetime, L, her name, and KP with her private key, KA. Finally, she encrypts K with Bob's public key, and signs it with KP. She sends all of this to Bob.
EK(TA),SKA(L,A,KP),SKP(EKB(K))
(4) Bob sends a message to Trent (this may be a different Trent), consisting of Alice's name.
A
(5) Trent sends Bob Alice's public key, signed in Trent's private key. The signed message includes Alice's name.
ST(A,KA)
(6) Bob verifies Trent's signature to confirm that the key he received is actually Alice's public key. He then verifies Alice's signature and recovers KP. He verifies the signature and uses his private key to recover K. Then he decrypts TA to make sure this is a current message.
(7) If mutual authentication is required, Bob encrypts a new timestamp with K, and sends it to Alice.
EK(TB)
(8) Alice decrypts TB with K to make sure that the message is current.

SPX, a product by DEC, is based on DASS. Additional information can be found in [34].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Denning-Sacco

This protocol also uses public-key cryptography [461]. Trent keeps a database of everyone's public keys.

(1) Alice sends a message to Trent with her identity and Bob's identity:
A,B
(2) Trent sends Alice Bob's public key, KB, signed with Trent's private key, T. Trent also sends Alice her own public key, KA, signed with his private key.
ST(B,KB),ST(A,KA)
(3) Alice sends Bob a random session key and a timestamp, signed in her private key and encrypted in Bob's public key, along with both signed public keys.
EB(SA(K,TA)),ST(B,KB),ST(A,KA)
(4) Bob decrypts Alice's message with his private key and then verifies Alice's signature with her public key. He checks to make sure that the timestamp is still valid.

At this point both Alice and Bob have K, and can communicate securely.

This looks good, but it isn't. After completing the protocol with Alice, Bob can then masquerade as Alice [5]. Watch:

(1) Bob sends his name and Carol's name to Trent
B,C
(2) Trent sends Bob both Bob's and Carol's signed public keys.
ST(B,KB),ST(C,KC)
(3) Bob sends Carol the signed session key and timestamp he previously received from Alice, encrypted with Carol's public key, along with Alice's certificate and Carol's certificate.
EC(SA(K,TA)),ST(A,KA),ST(C,KC)
(4) Carol decrypts Alice's message with her private key and then verifies Alice's signature with her public key. She checks to make sure that the timestamp is still valid.

Carol now thinks she is talking to Alice; Bob has successfully fooled her. In fact, Bob can fool everyone on the network until the timestamp expires.

This is easy to fix. Add the names inside the encrypted message in step (3):

EB(SA(A,B,K,TA)),ST(A,KA),ST(B,KB)

Now Bob can't replay the old message to Carol, because it is clearly meant for communication between Alice and Bob.

Woo-Lam

This protocol also uses public-key cryptography [1610,1611]:

(1) Alice sends a message to Trent with her identity and Bob's identity:
A,B
(2) Trent sends Alice Bob's public key, KB, signed with Trent's private key, T.
ST(KB)
(3) Alice verifies Trent's signature. Then she sends Bob her name and a random number, encrypted with Bob's public key.
EKB(A,RA)
(4) Bob sends Trent his name, Alice's name, and Alice's random number encrypted with Trent's public key, KT.
A,B,EKT(RA)
(5) Trent sends Bob Alice's public key, KA, signed with Trent's private key. He also sends him Alice's random number, a random session key, Alice's name, and Bob's name, all signed with Trent's private key and encrypted with Bob's public key.
ST(KA),EKB(ST(RA,K,A,B))
(6) Bob verifies Trent's signatures. Then he sends Alice the second part of Trent's message from step (5) and a new random number—all encrypted in Alice's public key.
EKA(ST(RA,K,A,B),RB)
(7) Alice verifies Trent's signature and her random number. Then she sends Bob the second random number, encrypted in the session key.
EK(RB)
(8) Bob decrypts his random number and verifies that it unchanged.

Other Protocols

There are many other protocols in the literature. The X.509 protocols are discussed in Section 24.9, KryptoKnight is discussed in Section 24.6, and Encrypted Key Exchange is discussed in Section 22.5.

Another new public-key protocol is Kuperee [694]. And work is being done on protocols that use beacons, a trusted node on a network that continuously broadcasts authenticated nonces [783].

Lessons Learned

There are some important lessons in the previous protocols, both those which have been broken and those which have not:

— Many protocols failed because the designers tried to be too clever. They optimized their protocols by leaving out important pieces: names, random numbers, and so on. The remedy is to make everything explicit [43,44].
— Trying to optimize is an absolute tar pit and depends a whole lot on the assumptions you make. For example: If you have authenticated time, you can do a whole lot of things you can't do if you don't.
— The protocol of choice depends on the underlying communications architecture. Do you want to minimize the size of messages or the number of messages? Can all parties talk with each other or can only a few of them?

It's questions like these that led to the development of formal methods for analyzing protocols.

3.4 Formal Analysis of Authentication and Key-Exchange Protocols

The problem of establishing secure session keys between pairs of computers (and people) on a network is so fundamental that it has led to a great deal of research. Some of the research focused on the development of protocols like the ones discussed in Sections 3.1, 3.2, and 3.3. This, in turn, has led to a greater and more interesting problem: the formal analysis of authentication and key-exchange protocols. People have found flaws in seemingly secure protocols years after they were proposed, and researchers wanted tools that could prove a protocol's security from the start. Although much of this work can apply to general cryptographic protocols, the emphasis in research is almost exclusively on authentication and key exchange.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

There are four basic approaches to the analysis of cryptographic protocols [1045]:

1. Model and verify the protocol using specification languages and verification tools not specifically designed for the analysis of cryptographic protocols.
2. Develop expert systems that a protocol designer can use to develop and investigate different scenarios.
3. Model the requirements of a protocol family using logics for the analysis of knowledge and belief.
4. Develop a formal method based on the algebraic term-rewriting properties of cryptographic systems.

A full discussion on these four approaches and the research surrounding them is well beyond the scope of this book. See [1047,1355] for a good introduction to the topic; I am only going to touch on the major contributions to the field.

The first approach treats a cryptographic protocol as any other computer program and attempts to prove correctness. Some researchers represent a protocol as a finite-state machine [1449,1565], others use extensions of first-order predicate calculus [822], and still others use specification languages to analyze protocols [1566]. However, proving correctness is not the same as proving security and this approach fails to detect many flawed protocols. Although it was widely studied at first, most of the work in this area has been redirected as the third approach gained popularity.

The second approach uses expert systems to determine if a protocol can reach an undesirable state (the leaking of a key, for example). While this approach better identifies flaws, it neither guarantees security nor provides techniques for developing attacks. It is good at determining whether a protocol contains a given flaw, but is unlikely to discover unknown flaws in a protocol. Examples of this approach can be found in [987,1521]; [1092] discusses a rule-based system developed by the U.S. military, called the Interrogator.

The third approach is by far the most popular, and was pioneered by Michael Burrows, Martin Abadi, and Roger Needham. They developed a formal logic model for the analysis of knowledge and belief, called BAN logic [283,284]. BAN logic is the most widely used logic for analyzing authentication protocols. It assumes that authentication is a function of integrity and freshness, and uses logical rules to trace both of those attributes through the protocol. Although many variants and extensions have been proposed, most protocol designers still refer back to the original work.

BAN logic doesn't provide a proof of security; it can only reason about authentication. It has a simple, straightforward logic that is easy to apply and still useful for detecting flaws. Some of the statements in BAN logic include:

Alice believes X. (Alice acts as though X is true.)

Alice sees X. (Someone has sent a message containing X to Alice, who can read and repeat X—possibly after decrypting it.)

Alice said X. (At some time, Alice sent a message that includes the statement X. It is not known how long ago the message was sent or even that it was sent during the current run of the protocol. It is known that Alice believed X when she said it.)

X is fresh. (X has not been sent in a message at any time before the current run of the protocol.)

And so on. BAN logic also provides rules for reasoning about belief in a protocol. These rules can then be applied to the logical statements about the protocol to prove things or answer questions about the protocol. For example, one rule is the message-meaning rule:

IF Alice believes that Alice and Bob share a secret key, K, and Alice sees X, encrypted under K, and Alice did not encrypt X under K, THEN Alice believes that Bob once said X.

Another rule is the nonce-verification rule:

IF Alice believes that X could have been uttered only recently and that Bob once said X, THEN Alice believes that Bob believes X.

There are four steps in BAN analysis:

(1) Convert the protocol into idealized form, using the statements previously described.
(2) Add all assumptions about the initial state of the protocol.
(3) Attach logical formulas to the statements: assertions about the state of the system after each statement.
(4) Apply the logical postulates to the assertions and assumptions to discover the beliefs held by the parties in the protocol.

The authors of BAN logic "view the idealized protocols as clearer and more complete specifications than traditional descriptions found in the literature...." [283,284]. Others are not so impressed and criticize this step because it may not accurately reflect the real protocol [1161,1612]. Further debate is in [221,1557]. Other critics try to show that BAN logic can deduce characteristics about protocols that are obviously false [1161]—see [285,1509] for a rebuttal—and that BAN logic deals only with trust and not security [1509]. More debate is in [1488,706,1002].

Despite these criticisms, BAN logic has been a success. It has found flaws in several protocols, including Needham-Schroeder and an early draft of a CCITT X.509 protocol [303]. It has uncovered redundancies in many protocols, including Yahalom, Needham-Schroeder, and Kerberos. Many published papers use BAN logic to make claims about their protocol's security [40,1162,73].

Other logic systems have been published, some designed as extensions to BAN logic [645,586,1556,828] and others based on BAN to correct perceived weaknesses [1488,1002]. The most successful of these is GNY [645], although it has some shortcomings [40]. Probabalistic beliefs were added to BAN logic, with mixed success, by [292,474]. Other formal logics are [156,798,288]; [1514] attempts to combine the features of several logics. And [1124,1511] present logics where beliefs can change over time.

The fourth approach to the analysis of cryptographic protocols models the protocol as an algebraic system, expresses the state of the participants' knowledge about the protocol, and then analyzes the attainability of certain states. This approach has not received as much attention as formal logics, but that is changing. It was first used by Michael Merritt [1076], who showed that an algebraic model can be used to analyze cryptographic protocols. Other approaches are in [473,1508,1530,1531,1532,1510,1612].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The Navy Research Laboratory's (NRL) Protocol Analyzer is probably the most successful application of these techniques [1512,823,1046,1513]; it has been used to discover both new and known flaws in a variety of protocols [1044,1045,1047]. The Protocol Analyzer defines the following actions:

— Accept (Bob, Alice, M, N). (Bob accepts the message M as from Alice during Bob's local round N.)
— Learn (Eve, M). (Eve learns M.)
— Send (Alice, Bob, Q, M). (Alice sends M to Bob in response toquery, Q.)
— Request (Bob, Alice, Q, N). (Bob sends Q to Alice during Bob's local round N.)

From these actions, requirements can be specified. For example:

— If Bob accepted message M from Alice at some point in the past, then Eve did not learn M at some point in the past.
— If Bob accepted message M from Alice in Bob's local round N, then Alice sent M to Bob as a response to a query in Bob's local round N.

To use the NRL Protocol Analyzer, a protocol must be specified using the previous constructs. Then, there are four phases of analysis: defining transition rules for honest participants, describing operations available to all—honest and dishonest—participants, describing the basic building blocks of the protocol, and describing the reduction rules. The point of all this is to show that a given protocol meets its requirements. Tools like the NRL Protocol Analyzer could eventually lead to a protocol that can be proven secure.

While much of the work in formal methods involves applying the methods to existing protocols, there is some push towards using formal methods to design the protocols in the first place. Some preliminary steps in this direction are [711]. The NRL Protocol Analyzer also attempts to do this [1512,222,1513].

The application of formal methods to cryptographic protocols is still a fairly new idea and it's really hard to figure out where it is headed. At this point, the weakest link seems to be the formalization process.

3.5 Multiple-Key Public-Key Cryptography

Public-key cryptography uses two keys. A message encrypted with one key can be decrypted with the other. Usually one key is private and the other is public. However, let's assume that Alice has one key and Bob has the other. Now Alice can encrypt a message so that only Bob can decrypt it, and Bob can encrypt a message so that only Alice can read it.

This concept was generalized by Colin Boyd [217]. Imagine a variant of public-key cryptography with three keys: KA, KB, and KC, distributed as shown in Table 3.2.

Alice can encrypt a message with KA so that Ellen, with KB and KC, can decrypt it. So can Bob and Carol in collusion. Bob can encrypt a message so that Frank can read it, and Carol can encrypt a message so that Dave can read it. Dave can encrypt a message with KA so that Ellen can read it, with KB so that Frank can read it, or with both KA and KB so that Carol can read it. Similarly, Ellen can encrypt a message so that either Alice, Dave, or Frank can read it. All the possible combinations are summarized in Table 3.3; there are no other ones.

TABLE 3.2 Three-Key Key Distribution
Alice | KA
Bob | KB
Carol | KC
Dave | KA and KB
Ellen | KB and KC
Frank | KC and KA

This can be extended to n keys. If a given subset of the keys is used to encrypt the message, then the other keys are required to decrypt the message.

Broadcasting a Message

Imagine that you have 100 operatives out in the field. You want to be able to send messages to subsets of them, but don't know which subsets in advance. You can either encrypt the message separately for each person or give out keys for every possible combination of people. The first option requires a lot of messages; the second requires a lot of keys.

Multiple-key cryptography is much easier. We'll use three operatives: Alice, Bob, and Carol. You give Alice KA and KB, Bob KB and KC, and Carol KC and KA. Now you can talk to any subset you want. If you want to send a message so that only Alice can read it, encrypt it with KC. When Alice receives the message, she decrypts it with KA and then KB. If you want to send a message so that only Bob can read it, encrypt it with KA; so that only Carol can read it, with KB. If you want to send a message so that both Alice and Bob can read it, encrypt it with KA and KC, and so on.

This might not seem exciting, but with 100 operatives it is quite efficient. Individual messages mean a shared key with each operative (100 keys total) and each message. Keys for every possible subset means 2100 - 2 different keys (messages to all operatives and messages to no operatives are excluded). This scheme needs only one encrypted message and 100 different keys. The drawback of this scheme is that you also have to broadcast which subset of operatives can read the message, otherwise each operative would have to try every combination of possible keys looking for the correct one. Even just the names of the intended recipients may be significant. At least for the straightforward implementation of this, everyone gets a really large amount of key data.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

There are other techniques for message broadcasting, some of which avoid the previous problem. These are discussed in Section 22.7.

TABLE 3.3 Three-Key Message Encryption
Encrypted with Keys: | Must be Decrypted with Keys:
KA | KB and KC
KB | KA and KC
KC | KA and KB
KA and KB | KC
KA and KC | KB
KB and KC | KA

3.6 Secret Splitting

Imagine that you've invented a new, extra gooey, extra sweet, cream filling or a burger sauce that is even more tasteless than your competitors'. This is important; you have to keep it secret. You could tell only your most trusted employees the exact mixture of ingredients, but what if one of them defects to the competition? There goes the secret, and before long every grease palace on the block will be making burgers with sauce as tasteless as yours.

This calls for secret splitting. There are ways to take a message and divide it up into pieces [551]. Each piece by itself means nothing, but put them together and the message appears. If the message is the recipe and each employee has a piece, then only together can they make the sauce. If any employee resigns with his single piece of the recipe, his information is useless by itself.

The simplest sharing scheme splits a message between two people. Here's a protocol in which Trent can split a message between Alice and Bob:

(1) Trent generates a random-bit string, R, the same length as the message, M.
(2) Trent XORs M with R to generate S.
M ⊕ R = S
(3) Trent gives R to Alice and S to Bob.

To reconstruct the message, Alice and Bob have only one step to do:

(4) Alice and Bob XOR their pieces together to reconstruct the message:
R ⊕ S = M

This technique, if done properly, is absolutely secure. Each piece, by itself, is absolutely worthless. Essentially, Trent is encrypting the message with a one-time pad and giving the ciphertext to one person and the pad to the other person. Section 1.5 discusses one-time pads; they have perfect security. No amount of computing power can determine the message from one of the pieces.

It is easy to extend this scheme to more people. To split a message among more than two people, XOR more random-bit strings into the mixture. In this example, Trent divides up a message into four pieces:

(1) Trent generates three random-bit strings, R, S, and T, the same length as the message, M.
(2) Trent XORs M with the three strings to generate U:
M ⊕ R ⊕ S ⊕ T = U
(3) Trent gives R to Alice, S to Bob, T to Carol, and U to Dave.

Alice, Bob, Carol, and Dave, working together, can reconstruct the message:

(4) Alice, Bob, Carol, and Dave get together and compute:
R ⊕ S ⊕ T ⊕ U = M

This is an adjudicated protocol. Trent has absolute power and can do whatever he wants. He can hand out gibberish and claim that it is a valid piece of the secret; no one will know it until they try to reconstruct the secret. He can hand out a piece to Alice, Bob, Carol, and Dave, and later tell everyone that only Alice, Carol, and Dave are needed to reconstruct the secret, and then fire Bob. But since this is Trent's secret to divide up, this isn't a problem.

However, this protocol has a problem: If any of the pieces gets lost and Trent isn't around, so does the message. If Carol, who has a piece of the sauce recipe, goes to work for the competition and takes her piece with her, the rest of them are out of luck. She can't reproduce the recipe, but neither can Alice, Bob, and Dave working together. Her piece is as critical to the message as every other piece combined. All Alice, Bob, or Dave know is the length of the message—nothing more. This is true because R, S, T, U, and M all have the same length; seeing anyone of them gives the length of M. Remember, M isn't being split in the normal sense of the word; it is being XORed with random values.

3.7 Secret Sharing

You're setting up a launch program for a nuclear missile. You want to make sure that no single raving lunatic can initiate a launch. You want to make sure that no two raving lunatics can initiate a launch. You want at least three out of five officers to be raving lunatics before you allow a launch.

This is easy to solve. Make a mechanical launch controller. Give each of the five officers a key and require that at least three officers stick their keys in the proper slots before you'll allow them to blow up whomever we're blowing up this week. (If you're really worried, make the slots far apart and require the officers to insert the keys simultaneously—you wouldn't want an officer who steals two keys to be able to vaporize Toledo.)

We can get even more complicated. Maybe the general and two colonels are authorized to launch the missile, but if the general is busy playing golf then five colonels are required to initiate a launch. Make the launch controller so that it requires five keys. Give the general three keys and the colonels one each. The general together with any two colonels can launch the missile; so can the five colonels. However, a general and one colonel cannot; neither can four colonels.

A more complicated sharing scheme, called a threshold scheme, can do all of this and more—mathematically. At its simplest level, you can take any message (a secret recipe, launch codes, your laundry list, etc.) and divide it into n pieces, called shadows or shares, such that any m of them can be used to reconstruct the message. More precisely, this is called an (m,n)-threshold scheme.

With a (3,4)-threshold scheme, Trent can divide his secret sauce recipe among Alice, Bob, Carol, and Dave, such that any three of them can put their shadows together and reconstruct the message. If Carol is on vacation, Alice, Bob, and Dave can do it. If Bob gets run over by a bus, Alice, Carol, and Dave can do it. However, if Bob gets run over by a bus while Carol is on vacation, Alice and Dave can't reconstruct the message by themselves.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

General threshold schemes are even more versatile. Any sharing scenario you can imagine can be modeled. You can divide a message among the people in your building so that to reconstruct it, you need seven people from the first floor and five people from the second floor, unless there is someone from the third floor involved, in which case you only need that person and three people from the first floor and two people from the second floor, unless there is someone from the fourth floor involved, in which case you need that person and one person from the third floor, or that person and two people from the first floor and one person from the second floor, unless there is...well, you get the idea.

This idea was invented independently by Adi Shamir [1414] and George Blakley [182] and studied extensively by Gus Simmons [1466]. Several different algorithms are discussed in Section 23.2.

Secret Sharing with Cheaters

There are many ways to cheat with a threshold scheme. Here are just a few of them.

Scenario 1: Colonels Alice, Bob, and Carol are in a bunker deep below some isolated field. One day, they get a coded message from the president: "Launch the missiles. We're going to eradicate the last vestiges of neural network research in the country." Alice, Bob, and Carol reveal their shadows, but Carol enters a random number. She's actually a pacifist and doesn't want the missiles launched. Since Carol doesn't enter the correct shadow, the secret they recover is the wrong secret. The missiles stay in their silos. Even worse, no one knows why. Alice and Bob, even if they work together, cannot prove that Carol's shadow is invalid.

Scenario 2: Colonels Alice and Bob are sitting in the bunker with Mallory. Mallory has disguised himself as a colonel and none of the others is the wiser. The same message comes in from the president, and everyone reveals their shadows. "Bwa-ha-ha!" shouts Mallory. "I faked that message from the president. Now I know both of your shadows." He races up the staircase and escapes before anyone can catch him.

Scenario 3: Colonels Alice, Bob, and Carol are sitting in the bunker with Mallory, who is again disguised. (Remember, Mallory doesn't have a valid shadow.) The same message comes in from the president and everyone reveals their shadows. Mallory reveals his shadow only after he has heard the other three. Since only three shadows are needed to reconstruct the secret, he can quickly create a valid shadow and reveals that. Now, not only does he know the secret, but no one realizes that he isn't part of the scheme.

Some protocols that handle these sorts of cheaters are discussed in Section 23.2.

Secret Sharing without Trent

A bank wants its vault to open only if three out of five officers enter their keys. This sounds like a basic (3,5)-threshold scheme, but there's a catch. No one is to know the entire secret. There is no Trent to divide the secret up into five pieces. There are protocols by which the five officers can create a secret and each get a piece, such that none of the officers knows the secret until they all reconstruct it. I'm not going to discuss these protocols in this book; see [756] for details.

Sharing a Secret without Revealing the Shares

These schemes have a problem. When everyone gets together to reconstruct their secret, they reveal their shares. This need not be the case. If the shared secret is a private key (to a digital signature, for example), then n shareholders can each complete a partial signature of the document. After the nth partial signature, the document has been signed with the shared private key and none of the shareholders learns any other shares. The point is that the secret can be reused, and you don't need a trusted processor to handle it. This concept is explored further by Yvo Desmedt and Yair Frankel [483,484].

Verifiable Secret Sharing

Trent gives Alice, Bob, Carol, and Dave each a share or at least he says he does. The only way any of them know if they have a valid share is to try to reconstruct the secret. Maybe Trent sent Bob a bogus share or Bob accidentally received a bad share through communications error. Verifiable secret sharing allows each of them to individually verify that they have a valid share, without having to reconstruct the secret [558,1235].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Secret-Sharing Schemes with Prevention

A secret is divided up among 50 people so that any 10 can get together and reconstruct the secret. That's easy. But, can we implement the same secret-sharing scheme with the added constraint that 20 people can get together and prevent the others from reconstructing the secret, no matter how many of them there are? As it turns out, we can [153].

The math is complicated, but the basic idea is that everyone gets two shares: a "yes" share and a "no" share. When it comes time to reconstruct the secret, people submit one of their shares. The actual share they submit depends on whether they wish the secret reconstructed. If there are m or more "yes" shares and fewer than n "no" shares, the secret can be reconstructed. Otherwise, it cannot.

Of course, nothing prevents a sufficient number of "yes" people from going off in a corner without the "no" people (assuming they know who they are) and reconstructing the secret. But in a situation where everyone submits their shares into a central computer, this scheme will work.

Secret Sharing with Disenrollment

You've set up your secret-sharing system and now you want to fire one of your shareholders. You could set up a new scheme without that person, but that's time-consuming. There are methods for coping with this system. They allow a new sharing scheme to be activated instantly once one of the participants becomes untrustworthy [1004].

3.8 Cryptographic Protection of Databases

The membership database of an organization is a valuable commodity. On the one hand, you want to distribute the database to all members. You want them to communicate with one another, exchange ideas, and invite each other over for cucumber sandwiches. On the other hand, if you distribute the membership database to everyone, copies are bound to fall into the hands of insurance salesmen and other annoying purveyors of junk mail.

Cryptography can ameliorate this problem. We can encrypt the database so that it is easy to extract the address of a single person but hard to extract a mailing list of all the members.

The scheme, from [550,549], is straightforward. Choose a one-way hash function and a symmetric encryption algorithm. Each record of the database has two fields. The index field is the last name of the member, operated on by the one-way hash function. The data field is the full name and address of the member, encrypted using the last name as the key. Unless you know the last name, you can't decrypt the data field.

Searching a specific last name is easy. First, hash the last name and look for the hashed value in the index field of the database. If there is a match, then that last name is in the database. If there are several matches, then there are several people in the database with the last name. Finally, for each matching entry, decrypt the full name and address using the last name as the key.

In [550] the authors use this system to protect a dictionary of 6000 Spanish verbs. They report minimal performance degradation due to the encryption. Additional complications in [549] handle searches on multiple indexes, but the idea is the same. The primary problem with this system is that it's impossible to search for people when you don't know how to spell their name. You can try variant spellings until you find the correct one, but it isn't practical to scan through everyone whose name begins with "Sch" when looking for "Schneier."

This protection isn't perfect. It is possible for a particularly persistent insurance salesperson to reconstruct the membership database through brute-force by trying every possible last name. If he has a telephone database, he can use it as a list of possible last names. This might take a few weeks of dedicated number crunching, but it can be done. It makes his job harder and, in the world of junk mail, "harder" quickly becomes "too expensive."

Another approach, in [185], allows statistics to be compiled on encrypted data.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 4
Intermediate Protocols

4.1 Timestamping Services

In many situations, people need to certify that a document existed on a certain date. Think about a copyright or patent dispute: The party that produces the earliest copy of the disputed work wins the case. With paper documents, notaries can sign and lawyers can safeguard copies. If a dispute arises, the notary or the lawyer testifies that the letter existed on a certain date.

In the digital world, it's far more complicated. There is no way to examine a digital document for signs of tampering. It can be copied and modified endlessly without anyone being the wiser. It's trivial to change the date stamp on a computer file. No one can look at a digital document and say: "Yes, this document was created before November 4, 1952."

Stuart Haber and W. Scott Stornetta at Bellcore thought about the problem [682, 683, 92]. They wanted a digital timestamping protocol with the following properties:

— The data itself must be timestamped, without any regard to the physical medium on which it resides.
— It must be impossible to change a single bit of the document without that change being apparent.
— It must be impossible to timestamp a document with a date and time different from the present one.

Arbitrated Solution

This protocol uses Trent, who has a trusted timestamping service, and Alice, who wishes to timestamp a document.

(1) Alice transmits a copy of the document to Trent.
(2) Trent records the date and time he received the document and retains a copy of the document for safekeeping.

Now, if anyone calls into question Alice's claim of when the document was created, she just has to call up Trent. He will produce his copy of the document and verify that he received the document on the date and time stamped.

This protocol works, but has some obvious problems. First, there is no privacy. Alice has to give a copy of the document to Trent. Anyone listening in on the communications channel could read it. She could encrypt it, but still the document has to sit in Trent's database. Who knows how secure that database is?

Second, the database itself would have to be huge. And the bandwidth requirements to send large documents to Trent would be unwieldy.

The third problem has to do with the potential errors. An error in transmission, or an electromagnetic bomb detonating somewhere in Trent's central computers, could completely invalidate Alice's claim of a timestamp.

And fourth, there might not be someone as honest as Trent to run the timestamping service. Maybe Alice is using Bob's Timestamp and Taco Stand. There is nothing to stop Alice and Bob from colluding and timestamping a document with any time that they want.

Improved Arbitrated Solution

One-way hash functions and digital signatures can clear up most of these problems easily:

(1) Alice produces a one-way hash of the document.
(2) Alice transmits the hash to Trent.
(3) Trent appends the date and time he received the hash onto the hash and then digitally signs the result.
(4) Trent sends the signed hash with timestamp back to Alice.

This solves every problem but the last. Alice no longer has to worry about revealing the contents of her document; the hash is sufficient. Trent no longer has to store copies of the document (or even of the hash), so the massive storage requirements and security problems are solved (remember, one-way hash functions don't have a key). Alice can immediately examine the signed timestamped hash she receives in step (4), so she will immediately catch any transmission errors. The only problem remaining is that Alice and Trent can still collude to produce any timestamp they want.

Linking Protocol

One way to solve this problem is to link Alice's timestamp with timestamps previously generated by Trent. These timestamps will most probably be generated for people other than Alice. Since the order that Trent receives the different timestamp requests can't be known in advance, Alice's timestamp must have occurred after the previous one. And since the request that came after is linked with Alice's timestamp, then hers must have occurred before. This sandwiches Alice's request in time.

If A is Alice's name, the hash value that Alice wants timestamped is Hn, and the previous time stamp is Tn - 1, then the protocol is:

(1) Alice sends Trent Hn and A.
(2) Trent sends back to Alice:
Tn = SK(n,A,Hn,tn,In - 1,Hn - 1,Tn - 1,Ln)

where Ln consists of the following hashed linking information:
Ln = H(In - 1,Hn - 1,Tn - 1,Ln - 1)

SK indicates that the message is signed with Trent's private key. Alice's name identifies her as the originator of the request. The parameter n indicates the sequence of the request: This is the nth timestamp Trent has issued. The parameter tn is the time. The additional information is the identification, original hash, time, and hashed timestamp of the previous document Trent stamped.
(3) After Trent stamps the next document, he sends Alice the identification of the originator of that document: In + 1.

If someone challenges Alice's timestamp, she just contacts the originators of the previous and following documents: In - 1 and In + 1. If their documents are called into question, they can get in touch with In - 2 and In + 2, and so on. Every person can show that their document was timestamped after the one that came before and before the one that came after.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This protocol makes it very difficult for Alice and Trent to collude and produce a document stamped with a different time than the actual one. Trent cannot forward-date a document for Alice, since that would require knowing in advance what document request came before it. Even if he could fake that, he would have to know what document request came before that, and so on. He cannot back-date a document, because the timestamp must be embedded in the timestamps of the document issued immediately after, and that document has already been issued. The only possible way to break this scheme is to invent a fictitious chain of documents both before and after Alice's document, long enough to exhaust the patience of anyone challenging the timestamp.

Distributed Protocol

People die; timestamps get lost. Many things could happen between the timestamping and the challenge to make it impossible for Alice to get a copy of In - 1 's timestamp. This problem could be alleviated by embedding the previous 10 people's timestamps into Alice's, and then sending Alice the identities of the next 10 people. Alice has a greater chance of finding people who still have their timestamps.

Along a similar line, the following protocol does away with Trent altogether.

(1) Using Hn as input, Alice generates a string of random values using a cryptographically secure pseudo-random-number generator:
V1, V2, V3, ...Vk
(2) Alice interprets each of these values as the identification, I, of another person. She sends Hn to each of these people.
(3) Each of these people attaches the date and time to the hash, signs the result, and sends it back to Alice.
(4) Alice collects and stores all the signatures as the timestamp.

The cryptographically secure pseudo-random-number generator in step (1) prevents Alice from deliberately choosing corrupt Is as verifiers. Even if she makes trivial changes in her document in an attempt to construct a set of corrupt Is, her chances of getting away with this are negligible. The hash function randomizes the Is; Alice cannot force them.

This protocol works because the only way for Alice to fake a timestamp would be to convince all of the k people to cooperate. Since she chose them at random in step (1), the odds against this are very high. The more corrupt society is, the higher a number k should be.

Additionally, there should be some mechanism for dealing with people who can't promptly return the timestamp. Some subset of k is all that would be required for a valid timestamp. The details depend on the implementation.

Further Work

Further improvements to timestamping protocols are presented in [92]. The authors use binary trees to increase the number of timestamps that depend on a given timestamp, reducing even further the possibility that someone could create a chain of fictitious timestamps. They also recommend publishing a hash of the day's timestamps in a public place, such as a newspaper. This serves a function similar to sending the hash to random people in the distributed protocol. In fact, a timestamp has appeared in every Sunday's New York Times since 1992.

These timestamping protocols are patented [684, 685, 686]. A Bellcore spin-off company called Surety Technologies owns the patents and markets a Digital Notary System to support these protocols. In their first version, clients send "certify" requests to a central coordinating server. Following Merkle's technique of using hash functions to build trees [1066], the server builds a tree of hash values whose leaves are all the requests received during a given second, and sends back to each requester the list of hash values hanging off the path from its leaf to the root of the tree. The client software stores this locally, and can issue a Digital Notary "certificate" for any file that has been certified. The sequence of roots of these trees comprises the "Universal Validation Record" that will be available electronically at multiple repository sites (and also published on CD-ROM). The client software also includes a "validate" function, allowing the user to test whether a file has been certified in exactly its current form (by querying a repository for the appropriate tree root and comparing it against a hash value appropriately recomputed from the file and its certificate). For information contact Surety Technologies, 1 Main St., Chatham, NJ, 07928; (201) 701-0600; Fax: (201) 701-0601.

4.2 Subliminal Channel

Alice and Bob have been arrested and are going to prison. He's going to the men's prison and she's going to the women's prison. Walter, the warden, is willing to let Alice and Bob exchange messages, but he won't allow them to be encrypted. Walter expects them to coordinate an escape plan, so he wants to be able to read everything they say.

Walter also hopes to deceive either Alice or Bob. He wants one of them to accept a fraudulent message as a genuine message from the other. Alice and Bob go along with this risk of deception, otherwise they cannot communicate at all, and they have to coordinate their plans. To do this they have to deceive the warden and find a way of communicating secretly. They have to set up a subliminal channel, a covert communications channel between them in full view of Walter, even though the messages themselves contain no secret information. Through the exchange of perfectly innocuous signed messages they will pass secret information back and forth and fool Walter, even though Walter is watching all the communications.

An easy subliminal channel might be the number of words in a sentence. An odd number of words in a sentence might correspond to "1, " while an even number of words might correspond to "0." So, while you read this seemingly innocent paragraph, I have sent my operatives in the field the message "101." The problem with this technique is that it is mere steganography (see Section 1.2); there is no key and security depends on the secrecy of the algorithm.

Gustavus Simmons invented the concept of a subliminal channel in a conventional digital signature algorithm [1458, 1473]. Since the subliminal messages are hidden in what looks like normal digital signatures, this is a form of obfuscation. Walter sees signed innocuous messages pass back and forth, but he completely misses the information being sent over the subliminal channel. In fact, the subliminal-channel signature algorithm is indistinguishable from a normal signature algorithm, at least to Walter. Walter not only cannot read the subliminal message, but he also has no idea that one is even present.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In general the protocol looks like this:

(1) Alice generates an innocuous message, pretty much at random.
(2) Using a secret key shared with Bob, Alice signs the innocuous message in such a way that she hides her subliminal message in the signature. (This is the meat of the subliminal channel protocol; see Section 23.3.)
(3) Alice sends this signed message to Bob via Walter.
(4) Walter reads the innocuous message and checks the signature. Finding nothing amiss, he passes the signed message to Bob.
(5) Bob checks the signature on the innocuous message, confirming that the message came from Alice.
(6) Bob ignores the innocuous message and, using the secret key he shares with Alice, extracts the subliminal message.

What about cheating? Walter doesn't trust anyone and no one trusts him. He can always prevent communication, but he has no way of introducing phony messages. Since he can't generate any valid signatures, Bob will detect his attempt in step (5). And since he does not know the shared key, he can't read the subliminal messages. Even more important, he has no idea that the subliminal messages are there. Signed messages using a digital signature algorithm look no different from signed messages with subliminal messages embedded in the signature.

Cheating between Alice and Bob is more problematic. In some implementations of a subliminal channel, the secret information Bob needs to read the subliminal message is the same information Alice needs to sign the innocuous message. If this is the case, Bob can impersonate Alice. He can sign messages purporting to come from her, and there is nothing Alice can do about it. If she is to send him subliminal messages, she has to trust him not to abuse her private key.

Other subliminal channel implementations don't have this problem. A secret key shared by Alice and Bob allows Alice to send Bob subliminal messages, but it is not the same as Alice's private key and does not allow Bob to sign messages. Alice need not trust Bob not to abuse her private key.

Applications of Subliminal Channel

The most obvious application of the subliminal channel is in a spy network. If everyone sends and receives signed messages, spies will not be noticed sending subliminal messages in signed documents. Of course, the enemy's spies can do the same thing.

Using a subliminal channel, Alice could safely sign a document under threat. She would, when signing the document, imbed the subliminal message, saying, "I am being coerced." Other applications are more subtle. A company can sign documents and embed subliminal messages, allowing them to be tracked throughout the documents' lifespans. The government can "mark" digital cash. A malicious signature program can leak secret information in its signatures. The possibilities are endless.

Subliminal-Free Signatures

Alice and Bob are sending signed messages to each other, negotiating the terms of a contract. They use a digital signature protocol. However, this contract negotiation has been set up as a cover for Alice's and Bob's spying activities. When they use the digital signature algorithm, they don't care about the messages they are signing. They are using a subliminal channel in the signatures to send secret information to each other. The counterespionage service, however, doesn't know that the contract negotiations and the use of signed messages are just cover-ups. This concern has led people to create subliminal-free signature schemes. These digital signature schemes cannot be modified to contain a subliminal channel. See [480, 481] for details.

4.3 Undeniable Digital Signatures

Normal digital signatures can be copied exactly. Sometimes this property is useful, as in the dissemination of public announcements. Other times it could be a problem. Imagine a digitally signed personal or business letter. If many copies of that document were floating around, each of which could be verified by anyone, this could lead to embarrassment or blackmail. The best solution is a digital signature that can be proven valid, but that the recipient cannot show to a third party without the signer's consent.

The Alice Software Company distributes DEW (Do-Everything-Word). To ensure that their software is virus-free, they include a digital signature with each copy. However, they want only legitimate buyers of the software, not software pirates, to be able to verify the signature. At the same time, if copies of DEW are found to contain a virus, the Alice Software Company should be unable to deny a valid signature.

Undeniable signatures [343, 327] are suited to these sorts of tasks. Like a normal digital signature, an undeniable signature depends on the signed document and the signer's private key. But unlike normal digital signatures, an undeniable signature cannot be verified without the signer's consent. Although a better name for these signatures might be something like "nontransferable signatures, " the name comes from the fact that if Alice is forced to either acknowledge or deny a signature—perhaps in court—she cannot falsely deny her real signature.

The mathematics are complicated, but the basic idea is simple:

(1) Alice presents Bob with a signature.
(2) Bob generates a random number and sends it to Alice.
(3) Alice does a calculation using the random number and her private key and sends Bob the result. Alice could only do this calculation if the signature is valid.
(4) Bob confirms this.

There is also an additional protocol so that Alice can prove that she did not sign a document, and cannot falsely deny a signature.

Bob can't turn around and convince Carol that Alice's signature is valid, because Carol doesn't know that Bob's numbers are random. He could have easily worked the protocol backwards on paper, without any help from Alice, and then shown Carol the result. Carol can be convinced that Alice's signature is valid only if she completes the protocol with Alice herself. This might not make much sense now, but it will once you see the mathematics in Section 23.4.

This solution isn't perfect. Yvo Desmedt and Moti Yung show that it is possible, in some applications, for Bob to convince Carol that Alice's signature is valid [489].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

For instance, Bob buys a legal copy of DEW. He can validate the signature on the software package whenever he wants. Then, Bob convinces Carol that he's a salesman from the Alice Software Company. He sells her a pirated copy of DEW. When Carol tries to validate the signature with Bob, he simultaneously validates the signature with Alice. When Carol sends him the random number, he then sends it on to Alice. When Alice replies, he then sends the reply on to Carol. Carol is convinced that she is a legitimate buyer of the software, even though she isn't. This attack is an instance of the chess grandmaster problem and is discussed in detail in Section 5.2.

Even so, undeniable signatures have a lot of applications; in many instances Alice doesn't want anyone to be able to verify her signature. She might not want personal correspondence to be verifiable by the press, be shown and verified out of context, or even to be verified after things have changed. If she signs a piece of information she sold, she won't want someone who hasn't paid for the information to be able to verify its authenticity. Controlling who verifies her signature is a way for Alice to protect her personal privacy.

A variant of undeniable signatures separates the relation between signer and message from the relation between signer and signature [910]. In one signature scheme, anyone can verify that the signer actually created the signature, but the cooperation of the signer is required to verify that the signature is valid for the message.

A related notion is an entrusted undeniable signature [1229]. Imagine that Alice works for Toxins, Inc., and sends incriminating documents to a newspaper using an undeniable signature protocol. Alice can verify her signature to the newspaper reporter, but not to anyone else. However, CEO Bob suspects that Alice is the source of the documents. He demands that Alice run the disavowal protocol to clear her name, and Alice refuses. Bob maintains that the only reason Alice has to refuse is that she is guilty, and fires her.

Entrusted undeniable signatures are like undeniable signatures, except that the disavowal protocol can only be run by Trent. Bob cannot demand that Alice run the disavowal protocol; only Trent can. And if Trent is the court system, then he will only run the protocol to resolve a formal dispute.

4.4 Designated Confirmer Signatures

The Alice Software Company is doing a booming business selling DEW—so good, in fact, that Alice is spending more time verifying undeniable signatures than writing new features.

Alice would like a way to designate one particular person in the company to be in charge of signature verification for the whole company. Alice, or any other programmer, would be able to sign documents with an undeniable protocol. But the verifications would all be handled by Carol.

As it turns out, this is possible with designated confirmer signatures [333, 1213]. Alice can sign a document such that Bob is convinced the signature is valid, but he cannot convince a third party; at the same time Alice can designate Carol as the future confirmer of her signature. Alice doesn't even need to ask Carol's permission beforehand; she just has to use Carol's public key. And Carol can still verify Alice's signature if Alice is out of town, has left the company, or just upped and died.

Designated confirmer signatures are kind of a compromise between normal digital signatures and undeniable signatures. There are certainly instances where Alice might want to limit who can verify her signature. On the other hand, giving Alice complete control undermines the enforceability of signatures: Alice might refuse to cooperate in either confirming or denying, she might claim the loss of keys for confirming or denying, or she might just be unavailable. Designated confirmer signatures can give Alice the protection of an undeniable signature while not letting her abuse that protection. Alice might even prefer it that way: Designated confirmer signatures can help prevent false applications, protect her if she actually does lose her key, and step in if she is on vacation, in the hospital, or even dead.

This idea has all sorts of possible applications. Carol can set herself up as a notary public. She can publish her public key in some directory somewhere, and people can designate her as a confirmer for their signatures. She can charge a small fee for confirming signatures for the masses and make a nice living.

Carol can be a copyright office, a government agency, or a host of other things. This protocol allows organizations to separate the people who sign documents from the people who help verify signatures.

4.5 Proxy Signatures

Designated confirmer signatures allows a signer to designate someone else to verify his signature. Alice, for instance, needs to go on a business trip to someplace which doesn't have very good computer network access—to the jungles of Africa, for example. Or maybe she is incapacitated after major surgery. She expects to receive some important e-mail, and has instructed her secretary Bob to respond accordingly. How can Alice give Bob the power to sign messages for her, without giving him her private key?

Proxy signatures is a solution [1001]. Alice can give Bob a proxy, such that the following properties hold:

— Distinguishability. Proxy signatures are distinguishable from normal signatures by anyone.
— Unforgeability. Only the original signer and the designated proxy signer can create a valid proxy signature.
— Proxy signer's deviation. A proxy signer cannot create a valid proxy signature not detected as a proxy signature.
— Verifiability. From a proxy signature, a verifier can be convinced of the original signer's agreement on the signed message.
— Identifiability. An original signer can determine the proxy signer's identity from a proxy signature.
— Undeniability. A proxy signer cannot disavow an accepted proxy signature he created.

In some cases, a stronger form of identifiability is required—that anyone can determine the proxy signer's identity from the proxy signature. Proxy signature schemes, based on different digital signature schemes, are in [1001].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

4.6 Group Signatures

David Chaum introduces this problem in [330]:

A company has several computers, each connected to the local network. Each department of that company has its own printer (also connected to the network) and only persons of that department are allowed to use their department's printer. Before printing, therefore, the printer must be convinced that the user is working in that department. At the same time, the company wants privacy; the user's name may not be revealed. If, however, someone discovers at the end of the day that a printer has been used too often, the director must be able to discover who misused that printer, and send him a bill.

The solution to this problem is called a group signature. Group signatures have the following properties:

— Only members of the group can sign messages.
— The receiver of the signature can verify that it is a valid signature from the group.
— The receiver of the signature cannot determine which member of the group is the signer.
— In the case of a dispute, the signature can be "opened" to reveal the identity of the signer.

Group Signatures with a Trusted Arbitrator

This protocol uses a trusted arbitrator:

(1) Trent generates a large pile of public-key/private-key key pairs and gives every member of the group a different list of unique private keys. No keys on any list are identical. (If there are n members of the group, and each member gets m key pairs, then there are n*m total key pairs.)
(2) Trent publishes the master list of all public keys for the group, in random order. Trent keeps a secret record of which keys belong to whom.
(3) When group members want to sign a document, he chooses a key at random from his personal list.
(4) When someone wants to verify that a signature belongs to the group, he looks on the master list for the corresponding public key and verifies the signature.
(5) In the event of a dispute, Trent knows which public key corresponds to which group member.

The problem with this protocol is that it requires a trusted party. Trent knows everyone's private keys and can forge signatures. Also, m must be long enough to preclude attempts to analyze which keys each member uses.

Chaum [330] lists a number of other protocols, some in which Trent is unable to fake signatures and others in which Trent is not even required. Another protocol [348] not only hides the identity of the signer, but also allows new members to join the group. Yet another protocol is [1230].

4.7 Fail-Stop Digital Signatures

Let's say Eve is a very powerful adversary. She has vast computer networks and rooms full of Cray computers—orders of magnitude more computing power than Alice. All of these computers chug away, day and night, trying to break Alice's private key. Finally—success. Eve can now impersonate Alice, forging her signature on documents at will.

Fail-stop digital signatures, introduced by Birgit Pfitzmann and Michael Waidner [1240], prevent this kind of cheating. If Eve forges Alice's signatures after a brute-force attack, then Alice can prove they are forgeries. If Alice signs a document and then disavows the signature, claiming forgery, a court can verify that it is not a forgery.

The basic idea behind fail-stop signatures is that for every possible public key, many possible private keys work with it. Each of these private keys yields many different possible signatures. However, Alice has only one private key and can compute just one signature. Alice doesn't know any of the other private keys.

Eve wants to break Alice's private key. (Eve could also be Alice, trying to compute a second private key for herself.) She collects signed messages and, using her array of Cray computers, tries to recover Alice's private key. Even if she manages to recover a valid private key, there are so many possible private keys that it is far more likely that she has a different one. The probability of Eve's recovering the proper private key can be made so small as to be negligible.

Now, when Eve forges a signed document using the private key she generated, it will have a different signature than if Alice signs the document herself. When Alice is hauled off to court, she can produce two different signatures for the same message and public key (corresponding to her private key and to the private key Eve created) to prove forgery. On the other hand, if Alice cannot produce the two different signatures, there is no forgery and Alice is still bound by her signature.

This signature scheme protects against Eve breaking Alice's signature scheme by sheer computational power. It does nothing against Mallory's much more likely attack of breaking into Alice's house and stealing her private key or Alice's attack of signing a document and then conveniently losing her private key. To protect against the former, Alice should buy herself a good guard dog; that kind of thing is beyond the scope of cryptography.

Additional theory and applications of fail-stop signatures can be found in [1239, 1241, 730, 731].

4.8 Computing with Encrypted Data

Alice wants to know the solution to some function f(x), for some particular value of x. Unfortunately, her computer is broken. Bob is willing to compute f(x) for her, but Alice isn't keen on letting Bob know her x. How can Alice let Bob compute f(x) for her without telling him x?

This is the general problem of computing with encrypted data, also called hiding information from an oracle. (Bob is the oracle; he answers questions.) There are ways to do this for certain functions; they are discussed in Section 23.6.

4.9 Bit Commitment

The Amazing Alice, magician extraordinaire, will now perform a mystifying feat of mental prowess. She will guess the card Bob will choose before he chooses it! Watch as Alice writes her prediction on a piece of paper. Marvel as Alice puts that piece of paper in an envelope and seals it shut. Thrill as Alice hands that sealed envelope to a random member of the audience. "Pick a card, Bob, any card." He looks at it and shows it to Alice and the audience. It's the seven of diamonds. Alice now takes the envelope back from the audience. She rips it open. The prediction, written before Bob chose his card, says "seven of diamonds"! Applause.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

To make this work, Alice had to switch envelopes at the end of the trick. However, cryptographic protocols can provide a method immune from any sleight of hand. Why is this useful? Here's a more mundane story:

Stockbroker Alice wants to convince investor Bob that her method of picking winning stocks is sound.

Bob: "Pick five stocks for me. If they are all winners, I'll give you my business."

Alice: "If I pick five stocks for you, you could invest in them without paying me. Why don't I show you the stocks I picked last month?"

Bob: "How do I know you didn't change last month's picks after you knew their outcome? If you tell me your picks now, I'll know that you can't change them. I won't invest in those stocks until after I've purchased your method. Trust me."

Alice: "I'd rather show you my picks from last month. I didn't change them. Trust me."

Alice wants to commit to a prediction (i.e., a bit or series of bits) but does not want to reveal her prediction until sometime later. Bob, on the other hand, wants to make sure that Alice cannot change her mind after she has committed to her prediction.

Bit Commitment Using Symmetric Cryptography

This bit-commitment protocol uses symmetric cryptography:

(1) Bob generates a random-bit string, R, and sends it to Alice.
R
(2) Alice creates a message consisting of the bit she wishes to commit to, b (it can actually be several bits), and Bob's random string. She encrypts it with some random key, K, and sends the result back to Bob.
EK(R,b)

That is the commitment portion of the protocol. Bob cannot decrypt the message, so he does not know what the bit is.

When it comes time for Alice to reveal her bit, the protocol continues:

(3) Alice sends Bob the key.
(4) Bob decrypts the message to reveal the bit. He checks his random string to verify the bit's validity.

If the message did not contain Bob's random string, Alice could secretly decrypt the message she handed Bob with a variety of keys until she found one that gave her a bit other than the one she committed to. Since the bit has only two possible values, she is certain to find one after only a few tries. Bob's random string prevents her from using this attack; she has to find a new message that not only has her bit inverted, but also has Bob's random string exactly reproduced. If the encryption algorithm is good, the chance of her finding this is minuscule. Alice cannot change her bit after she commits to it.

Bit Commitment Using One-Way Functions

This protocol uses one-way functions:

(1) Alice generates two random-bit strings, R1 and R2.
R1,R2
(2) Alice creates a message consisting of her random strings and the bit she wishes to commit to (it can actually be several bits).
(R1,R2,b)
(3) Alice computes the one-way function on the message and sends the result, as well as one of the random strings, to Bob.
H(R1,R2,b),R1

This transmission from Alice is evidence of commitment. Alice's one-way function in step (3) prevents Bob from inverting the function and determining the bit.

When it comes time for Alice to reveal her bit, the protocol continues:

(4) Alice sends Bob the original message.
(R1,R2,b)
(5) Bob computes the one-way function on the message and compares it and R1, with the value and random string he received in step (3). If they match, the bit is valid.

The benefit of this protocol over the previous one is that Bob does not have to send any messages. Alice sends Bob one message to commit to a bit and another message to reveal the bit.

Bob's random string isn't required because the result of Alice's commitment is a message operated on by a one-way function. Alice cannot cheat and find another message (R1,R2´,b´), such that H(R1,R2´,b´) = H(R1,R2,b). By sending Bob R1 she is committing to the value of b. If Alice didn't keep R2 secret, then Bob could compute both H(R1,R2,b) and H(R1,R2,b´) and see which was equal to what he received from Alice.

Bit Commitment Using Pseudo-Random-Sequence Generators

This protocol is even easier [1137]:

(1) Bob generates a random-bit string and sends it to Alice.
RB
(2) Alice generates a random seed for a pseudo-random-bit generator. Then, for every bit in Bob's random-bit string, she sends Bob either:
(a) the output of the generator if Bob's bit is 0, or
(b) the XOR of output of the generator and her bit, if Bob's bit is 1.

When it comes time for Alice to reveal her bit, the protocol continues:

(3) Alice sends Bob her random seed.
(4) Bob completes step (2) to confirm that Alice was acting fairly.

If Bob's random-bit string is long enough, and the pseudo-random-bit generator is unpredictable, then there is no practical way Alice can cheat.

Blobs

These strings that Alice sends to Bob to commit to a bit are sometimes called blobs. A blob is a sequence of bits, although there is no reason in the protocols why it has to be. As Gilles Brassard said, "They could be made out of fairy dust if this were useful" [236]. Blobs have these four properties:

1. Alice can commit to blobs. By committing to a blob, she is committing to a bit.
2. Alice can open any blob she has committed to. When she opens a blob, she can convince Bob of the value of the bit she committed to when she committed to the blob. Thus, she cannot choose to open any blob as either a zero or a one.
3. Bob cannot learn how Alice is able to open any unopened blob she has committed to. This is true even after Alice has opened other blobs.
4. Blobs do not carry any information other than the bit Alice committed to. The blobs themselves, as well as the process by which Alice commits to and opens them, are uncorrelated to anything else that Alice might wish to keep secret from Bob.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

4.10 Fair Coin Flips

It's story time with Joe Kilian [831]:

Alice and Bob wanted to flip a fair coin, but had no physical coin to flip. Alice offered a simple way of flipping a fair coin mentally.

"First, you think up a random bit, then I'll think up a random bit. We'll then exclusive-or the two bits together, " she suggested.

"But what if one of us doesn't flip a coin at random?" Bob asked.

"It doesn't matter. As long as one of the bits is truly random, the exclusive-or of the bits should be truly random, " Alice replied, and after a moment's reflection, Bob agreed.

A short while later, Alice and Bob happened upon a book on artificial intelligence, lying abandoned by the roadside. A good citizen, Alice said, "One of us must pick this book up and find a suitable waste receptacle." Bob agreed, and suggested they use their coin-flipping protocol to determine who would have to throw the book away.

"If the final bit is a 0, then you will pick the book up, and if it is a 1, then I will, " said Alice. "What is your bit?"

Bob replied, "1."

"Why, so is mine, " said Alice, slyly, "I guess this isn't your lucky day."

Needless to say, this coin-flipping protocol had a serious bug. While it is true that a truly random bit, x, exclusive-ORed with any independently distributed bit, y, will yield a truly random bit, Alice's protocol did not ensure that the two bits were distributed independently. In fact, it is not hard to verify that no mental protocol can allow two infinitely powerful parties to flip a fair coin. Alice and Bob were in trouble until they received a letter from an obscure graduate student in cryptography. The information in the letter was too theoretical to be of any earthly use to anyone, but the envelope the letter came in was extremely handy.

The next time Alice and Bob wished to flip a coin, they played a modified version of the original protocol. First, Bob decided on a bit, but instead of announcing it immediately, he wrote it down on a piece of paper and placed the paper in the envelope. Next, Alice announced her bit. Finally, Alice and Bob took Bob's bit out of the envelope and computed the random bit. This bit was indeed truly random whenever at least one of them played honestly. Alice and Bob had a working protocol, the cryptographer's dream of social relevance was fulfilled, and they all lived happily ever after.

Those envelopes sound a lot like bit-commitment blobs. When Manuel Blum introduced the problem of flipping a fair coin over a modem [194], he solved it using a bit-commitment protocol:

(1) Alice commits to a random bit, using any of the bit-commitment schemes listed in Section 4.9.
(2) Bob tries to guess the bit.
(3) Alice reveals the bit to Bob. Bob wins the flip if he correctly guessed the bit.

In general, we need a protocol with these properties:

— Alice must flip the coin before Bob guesses.
— Alice must not be able to re-flip the coin after hearing Bob's guess.
— Bob must not be able to know how the coin landed before making his guess.

There are several ways in which we can do this.

Coin Flipping Using One-Way Functions

If Alice and Bob can agree on a one-way function, this protocol is simple:

(1) Alice chooses a random number, x. She computes y = f(x), where f(x) is the one-way function.
(2) Alice sends y to Bob.
(3) Bob guesses whether x is even or odd and sends his guess to Alice.
(4) If Bob's guess is correct, the result of the coin flip is heads. If Bob's guess is incorrect, the result of the coin flip is tails. Alice announces the result of the coin flip and sends x to Bob.
(5) Bob confirms that y = f(x).

The security of this protocol rests in the one-way function. If Alice can find x and x´, such that x is even and x´ is odd, and y = f(x) = f(x´), then she can cheat Bob every time. The least significant bit of f(x) must also be uncorrelated with x. If not, Bob can cheat Alice at least some of the time. For example, if f(x) produces even numbers 75 percent of the time if x is even, Bob has an advantage. (Sometimes the least significant bit is not the best one to use in this application, because it can be easier to compute.)

Coin Flipping Using Public-Key Cryptography

This protocol works with either public-key cryptography or symmetric cryptography. The only requirement is that the algorithm commute. That is:

DK1(EK2(EK1(M))) = EK2(M)

In general, this property is not true for symmetric algorithms, but it is true for some public-key algorithms (RSA with identical moduli, for example). This is the protocol:

(1) Alice and Bob each generate a public-key/private-key key pair.
(2) Alice generates two messages, one indicating heads and the other indicating tails. These messages should contain some unique random string, so that she can verify their authenticity later in the protocol. Alice encrypts both messages with her public key and sends them to Bob in a random order.
EA(M1), EA(M2)
(3) Bob, who cannot read either message, chooses one at random. (He can sing "eeny meeny miney moe, " engage a malicious computer intent on subverting the protocol, or consult the I Ching—it doesn't matter.) He encrypts it with his public key and sends it back to Alice.
EB(EA(M))

M is either M1 or M2.
(4) Alice, who cannot read the message sent back to her, decrypts it with her private key and then sends it back to Bob.
DA(EB(EA(M))) = EB(M1) if M = M1, or
EB(M2) if M = M2
(5) Bob decrypts the message with his private key to reveal the result of the coin flip. He sends the decrypted message to Alice.
DB(EB(M1)) = M1 or DB(EB(M2)) = M2
(6) Alice reads the result of the coin flip and verifies that the random string is correct.
(7) Both Alice and Bob reveal their key pairs so that both can verify that the other did not cheat.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This protocol is self-enforcing. Either party can immediately detect cheating by the other, and no trusted third party is required to participate in either the actual protocol or any adjudication after the protocol has been completed. To see how this works, let's try to cheat.

If Alice wanted to cheat and force heads, she has three potential ways of affecting the outcome. First, she could encrypt two "heads" messages in step (2). Bob would discover this when Alice revealed her keys at step (7). Second, she could use some other key to decrypt the message in step (4). This would result in gibberish, which Bob would discover in step (5). Third, she could lie about the validity of the message in step (6). Bob would also discover this in step (7), when Alice could not prove that the message was not valid. Of course, Alice could refuse to participate in the protocol at any step, at which point Alice's attempted deception would be obvious to Bob.

If Bob wanted to cheat and force "tails, " his options are just as poor. He could incorrectly encrypt a message at step (3), but Alice would discover this when she looked at the final message at step (6). He could improperly perform step (5), but this would also result in gibberish, which Alice would discover at step (6). He could claim that he could not properly perform step (5) because of some cheating on the part of Alice, but this form of cheating would be discovered at step (7). Finally, he could send a "tails" message to Alice at step (5), regardless of the message he decrypted, but Alice would immediately be able to check the message for authenticity at step (6).

Flipping Coins into a Well

It is interesting to note that in all these protocols, Alice and Bob don't learn the result of the coin flip at the same time. Each protocol has a point where one of the parties (Alice in the first two protocols and Bob in the last one) knows the result of the coin flip but cannot change it. That party can, however, delay disclosing the result to the other party. This is known as flipping coins into a well. Imagine a well. Alice is next to the well and Bob is far away. Bob throws the coin and it lands in the well. Alice can now look into the well and see the result, but she cannot reach down to change it. Bob cannot see the result until Alice lets him come close enough to look.

Key Generation Using Coin Flipping

A real application for this protocol is session-key generation. Coin-flipping protocols allow Alice and Bob to generate a random session key such that neither can influence what the session key will be. And assuming that Alice and Bob encrypt their exchanges, this key generation is secure from eavesdropping as well.

4.11 Mental Poker

A protocol similar to the public-key fair coin flip protocol allows Alice and Bob to play poker with each other via electronic mail. Instead of Alice making and encrypting two messages, one for heads and one for tails, she makes 52 messages, M1, M2,..., M52, one for each card in the deck. Bob chooses five messages at random, encrypts them with his public key, and then sends them back to Alice. Alice decrypts the messages and sends them back to Bob, who decrypts them to determine his hand. He then chooses five more messages at random and sends them back to Alice as he received them; she decrypts these and they become her hand. During the game, additional cards can be dealt to either player by repeating the procedure. At the end of the game, Alice and Bob both reveal their cards and key pairs so that each can be assured that the other did not cheat.

Mental Poker with Three Players

Poker is more fun with more players. The basic mental poker protocol can easily be extended to three or more players. In this case, too, the cryptographic algorithm must be commutative.

(1) Alice, Bob, and Carol each generate a public-key/private-key key pair.
(2) Alice generates 52 messages, one for each card in the deck. These messages should contain some unique random string, so that she can verify their authenticity later in the protocol. Alice encrypts all the messages with her public key and sends them to Bob.
EA(Mn)
(3) Bob, who cannot read any of the messages, chooses five at random. He encrypts them with his public key and sends them back to Alice.
EB(EA(Mn))
(4) Bob sends the other 47 messages to Carol.
EA(Mn)
(5) Carol, who cannot read any of the messages, chooses five at random. She encrypts them with her public key and sends them to Alice.
EC(EA(Mn))
(6) Alice, who cannot read any of the messages sent back to her, decrypts them with her private key and then sends them back to Bob or Carol (depending on where they came from).
DA(EB(EA(Mn))) = EB(Mn)
DA(EC(EA(Mn))) = EC(Mn)
(7) Bob and Carol decrypt the messages with their keys to reveal their hands.
DB(EB(Mn)) = Mn
DC(EC(Mn)) = Mn
(8) Carol chooses five more messages at random from the remaining 42. She sends them to Alice.
EA(Mn)
(9) Alice decrypts the messages with her private key to reveal her hand.
DA(EA(Mn)) = Mn
(10) At the end of the game Alice, Bob, and Carol all reveal their hands and all of their keys so that everyone can make sure that no one has cheated.

Additional cards can be dealt in the same manner. If Bob or Carol wants a card, either one can take the encrypted deck and go through the protocol with Alice. If Alice wants a card, whoever currently has the deck sends her a random card.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Ideally, step (10) would not be necessary. All players shouldn't be required to reveal their hands at the end of the protocol; only those who haven't folded. Since step (10) is part of the protocol designed only to catch cheaters, perhaps there are improvements.

In poker, one is only interested in whether the winner cheated. Everyone else can cheat as much as they want, as long as they still lose. (Actually, this is not really true. Someone can, while losing, collect data on another player's poker style.) So, let's look at cases in which different players win.

If Alice wins, she reveals her hand and her keys. Bob can use Alice's private key to confirm that Alice performed step (2) correctly—that each of the 52 messages corresponded to a different card. Carol can confirm that Alice is not lying about her hand by encrypting the cards with Alice's public key and verifying that they are the same as the encrypted messages she sent to her in step (8).

If either Bob or Carol wins, the winner reveals his hand and keys. Alice can confirm that the cards are legitimate by checking her random strings. She can also confirm that the cards are the ones dealt by encrypting the cards with the winner's public key and verifying that they are the same as the encrypted messages she received in step (3) or (5).

This protocol isn't secure against collusion among malicious players. Alice and another player can effectively gang up on the third and together swindle that player out of everything without raising suspicion. Therefore, it is important to check all the keys and random strings every time the players reveal their hands. And if you're sitting around the virtual table with two people who never reveal their hands whenever one of them is the dealer (Alice, in the previous protocol), stop playing.

Understand that while this is all interesting theory, actually implementing it on a computer is an arduous task. A Sparc implementation with three players on separate workstations takes eight hours to shuffle a deck of cards, let alone play an actual game [513].

Attacks against Poker Protocols

Cryptographers have shown that a small amount of information is leaked by these poker protocols if the RSA public-key algorithm is used [453, 573]. Specifically, if the binary representation of the card is a quadratic residue (see Section 11.3), then the encryption of the card is also a quadratic residue. This property can be used to "mark" some cards—all the aces, for example. This does not reveal much about the hands, but in a game such as poker even a tiny bit of information can be an advantage in the long run.

Shafi Goldwasser and Silvio Micali [624] developed a two-player mental-poker protocol that fixes this problem, although its complexity makes it far more theoretical than practical. A general n- player poker protocol that eliminates the problem of information leakage was developed in [389].

Other research on poker protocols can be found in [573, 1634, 389]. A complicated protocol that allows players to not reveal their hands can be found in [390]. Don Coppersmith discusses two ways to cheat at mental poker using the RSA algorithm [370].

Anonymous Key Distribution

While it is unlikely that anyone is going to use this protocol to play poker via modem, Charles Pfleeger discusses a situation in which this type of protocol would come in handy [1244].

Consider the problem of key distribution. If we assume that people cannot generate their own keys (they might have to be of a certain form, or have to be signed by some organization, or something similar), we have to set up a Key Distribution Center to generate and distribute keys. The problem is that we have to figure out some way of distributing keys such that no one, including the server, can figure out who got which key.

This protocol solves the problem:

(1) Alice generates a public-key/private-key key pair. For this protocol, she keeps both keys secret.
(2) The KDC generates a continuous stream of keys.
(3) The KDC encrypts the keys, one by one, with its own public key.
(4) The KDC transmits the encrypted keys, one by one, onto the network.
(5) Alice chooses a key at random.
(6) Alice encrypts the chosen key with her public key.
(7) Alice waits a while (long enough so the server has no idea which key she has chosen) and sends the double-encrypted key back to the KDC.
(8) The KDC decrypts the double-encrypted key with its private key, leaving a key encrypted with Alice's public key.
(9) The server sends the encrypted key back to Alice.
(10) Alice decrypts the key with her private key.

Eve, sitting in the middle of this protocol, has no idea what key Alice chose. She sees a continuous stream of keys go by in step (4). When Alice sends the key back to the server in step (7), it is encrypted with her public key, which is also secret during this protocol. Eve has no way of correlating it with the stream of keys. When the server sends the key back to Alice in step (9), it is also encrypted with Alice's public key. Only when Alice decrypts the key in step (10) is the key revealed.

If you use RSA, this protocol leaks information at the rate of one bit per message. It's the quadratic residues again. If you're going to distribute keys in this manner, make sure this leakage isn't enough to matter. Also, the stream of keys from the KDC must be great enough to preclude a brute-force attack. Of course, if Alice can't trust the KDC, then she shouldn't be getting keys from it. A malicious KDC could presumably keep records of every key it generates. Then, it could search them all to determine which is Alice's.

This protocol also assumes that Alice is going to act fairly. There are things she can do, using RSA, to get more information than she might otherwise. This is not a problem in our scenario, but can be in other circumstances.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

4.12 One-Way Accumulators

Alice is a member of Cabal, Inc. Occasionally she has to meet with other members in dimly lit restaurants and whisper secrets back and forth. The problem is that the restaurants are so dimly lit that she has trouble knowing if the person across the table from her is also a member.

Cabal Inc. can choose from several solutions. Every member can carry a membership list. This has two problems. One, everyone now has to carry a large database, and two, they have to guard that membership list pretty carefully. Alternatively, a trusted secretary could issue digitally signed ID cards. This has the added advantage of allowing outsiders to verify members (for discounts at the local grocery store, for example), but it requires a trusted secretary. Nobody at Cabal, Inc. can be trusted to that degree.

A novel solution is to use something called a one-way accumulator [116]. This is sort of like a one-way hash function, except that it is commutative. That is, it is possible to hash the database of members in any order and get the same value. Moreover, it is possible to add members into the hash and get a new hash, again without regard to order.

So, here's what Alice does. She calculates the accumulation of every member's name other than herself. Then she saves that single value along with her own name. Bob, and every other member, does the same. Now, when Alice and Bob meet in the dimly lit restaurant, they simply trade accumulations and names with each other. Alice confirms that Bob's name added to his accumulation is equal to Alice's name added to her accumulation. Bob does the same. Now they both know that the other is a member. And at the same time, neither can figure out the identities of any other member.

Even better, nonmembers can be given the accumulation of everybody. Now Alice can verify her membership to a nonmember (for membership discounts at their local counterspy shop, perhaps) without the nonmember being able to figure out the entire membership list.

New members can be added just by sending around the new names. Unfortunately, the only way to delete a member is to send everyone a new list and have them recompute their accumulations. But Cabal, Inc. only has to do that if a member resigns; dead members can remain on the list. (Oddly enough, this has never been a problem.)

This is a clever idea, and has applications whenever you want the same effect as digital signatures without a centralized signer.

4.13 All-or-Nothing Disclosure of Secrets

Imagine that Alice is a former agent of the former Soviet Union, now unemployed. In order to make money, Alice sells secrets. Anyone who is willing to pay the price can buy a secret. She even has a catalog. All her secrets are listed by number, with tantalizing titles: "Where is Jimmy Hoffa?", "Who is secretly controlling the Trilateral Commission?", "Why does Boris Yeltsin always look like he swallowed a live frog?", and so on.

Alice won't give away two secrets for the price of one or even partial information about any of the secrets. Bob, a potential buyer, doesn't want to pay for random secrets. He also doesn't want to tell Alice which secrets he wants. It's none of Alice's business, and besides, Alice could then add "what secrets Bob is interested in" to her catalog.

A poker protocol won't work in this case, because at the end of the protocol Alice and Bob have to reveal their hands to each other. There are also tricks Bob can do to learn more than one secret.

The solution is called all-or-nothing disclosure of secrets (ANDOS) [246] because, as soon as Bob has gained any information whatsoever about one of Alice's secrets, he has wasted his chance to learn anything about any of the other secrets.

There are several ANDOS protocols in the cryptographic literature. Some of them are discussed in Section 23.9.

4.14 Key Escrow

This excerpt is from Silvio Micali's introduction to the topic [1084]:

Currently, court-authorized line tapping is an effective method for securing criminals to justice. More importantly, in our opinion, it also prevents the further spread of crime by deterring the use of ordinary communication networks for unlawful purposes. Thus, there is a legitimate concern that widespread use of public-key cryptography may be a big boost for criminal and terrorist organizations. Indeed, many bills propose that a proper governmental agency, under circumstances allowed by law, should be able to obtain the clear text of any communication over a public network. At the present time, this requirement would translate into coercing citizens to either (1) using weak cryptosystems—i.e., cryptosystems that the proper authorities (but also everybody else!) could crack with a moderate effort—or (2) surrendering, a priori, their secret key to the authority. It is not surprising that such alternatives have legitimately alarmed many concerned citizens, generating as reaction the feeling that privacy should come before national security and law enforcement.

Key escrow is the heart of the U.S. government's Clipper program and its Escrowed Encryption Standard. The challenge here is to develop a cryptosystem that both protects individual privacy but at the same time allows for court-authorized wiretaps.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The Escrowed Encryption Standard gets its security from tamperproof hardware. Each encryption chip has a unique ID number and secret key. This key is split into two pieces and stored, along with the ID number, by two different escrow agencies. Every time the chip encrypts a data file, it first encrypts the session key with this unique secret key. Then it transmits this encrypted session key and its ID number over the communications channel. When some law enforcement agency wants to decrypt traffic encrypted with one of these chips, it listens for the ID number, collects the appropriate keys from the escrow agencies, XORs them together, decrypts the session key, and then uses the session key to decrypt the message traffic. There are more complications to make this scheme work in the face of cheaters; see Section 24.16 for details. The same thing can be done in software, using public-key cryptography [77, 1579, 1580, 1581].

Micali calls his idea fair cryptosystems [1084, 1085]. (The U.S. government reportedly paid Micali $1, 000, 000 for the use of his patents [1086, 1087] in their Escrowed Encryption Standard; Banker's Trust then bought Micali's patent.) In these cryptosystems, the private key is broken up into pieces and distributed to different authorities. Like a secret sharing scheme, the authorities can get together and reconstruct the private key. However, the pieces have the additional property that they can be individually verified to be correct, without reconstructing the private key.

Alice can create her own private key and give a piece to each of n trustees. None of these trustees can recover Alice's private key. However, each trustee can verify that his piece is a valid piece of the private key; Alice cannot send one of the trustees a random-bit string and hope to get away with it. If the courts authorize a wiretap, the relevant law enforcement authorities can serve a court order on the n trustees to surrender their pieces. With all n pieces, the authorities reconstruct the private key and can wiretap Alice's communications lines. On the other hand, Mallory has to corrupt all n trustees in order to be able to reconstruct Alice's key and violate her privacy.

Here's how the protocol works:

(1) Alice creates her private-key/public-key key pair. She splits the private key into several public pieces and private pieces.
(2) Alice sends a public piece and corresponding private piece to each of the trustees. These messages must be encrypted. She also sends the public key to the KDC.
(3) Each trustee, independently, performs a calculation on its public piece and its private piece to confirm that they are correct. Each trustee stores the private piece somewhere secure and sends the public piece to the KDC.
(4) The KDC performs another calculation on the public pieces and the public key. Assuming that everything is correct, it signs the public key and either sends it back to Alice or posts it in a database somewhere.

If the courts order a wiretap, then each of the trustees surrenders his or her piece to the KDC, and the KDC can reconstruct the private key. Before this surrender, neither the KDC nor any individual trustee can reconstruct the private key; all the trustees are required to reconstruct the key.

Any public-key cryptography algorithm can be made fair in this manner. Some particular algorithms are discussed in Section 23.10. Micali's paper [1084, 1085] discusses ways to combine this with a threshold scheme, so that a subset of the trustees (e.g., three out of five) is required to reconstruct the private key. He also shows how to combine this with oblivious transfer (see Section 5.5) so that the trustees do not know whose private key is being reconstructed.

Fair cryptosystems aren't perfect. A criminal can exploit the system, using a subliminal channel (see Section 4.2) to embed another secret key into his piece. This way, he can communicate securely with someone else using this subliminal key without having to worry about court-authorized wiretapping. Another protocol, called failsafe key escrowing, solves this problem [946, 833]. Section 23.10 describes the algorithm and protocol.

The Politics of Key Escrow

Aside from the government's key-escrow plans, several commercial key-escrow proposals are floating around. This leads to the obvious question: What are the advantages of key-escrow for the user?

Well, there really aren't any. The user gains nothing from key escrow that he couldn't provide himself. He can already back up his keys if he wants (see Section 8.8). Key-escrow guarantees that the police can eavesdrop on his conversations or read his data files even though they are encrypted. It guarantees that the NSA can eavesdrop on his international phone calls—without a warrant—even though they are encrypted. Perhaps he will be allowed to use cryptography in countries that now ban it, but that seems to be the only advantage.

Key escrow has considerable disadvantages. The user has to trust the escrow agents' security procedures, as well as the integrity of the people involved. He has to trust the escrow agents not to change their policies, the government not to change its laws, and those with lawful authority to get his keys to do so lawfully and responsibly. Imagine a major terrorist attack in New York; what sorts of limits on the police would be thrown aside in the aftermath?


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

It is hard to imagine escrowed encryption schemes working as their advocates imagine without some kind of legal pressure. The obvious next step is a ban on the use of non-escrowed encryption. This is probably the only way to make a commercial system pay, and it's certainly the only way to get technologically sophisticated criminals and terrorists to use it. It's not clear how difficult outlawing non-escrowed cryptography will be, or how it will affect cryptography as an academic discipline. How can I research software-oriented cryptography algorithms without having software non-escrowed encryption devices in my possession; will I need a special license?

And there are legal questions. How do escrowed keys affect users' liability, should some encrypted data get out? If the U.S. government is trying to protect the escrow agencies, will there be the implicit assumption that if the secret was compromised by either the user or the escrow agency, then it must have been the user?

What if a major key-escrow service, either government or commercial, had its entire escrowed key database stolen? What if the U.S. government tried to keep this quiet for a while? Clearly, this would have an impact on users' willingness to use key escrow. If it's not voluntary, a couple of scandals like this would increase political pressure to either make it voluntary, or to add complex new regulations to the industry.

Even more dangerous is a scandal where it becomes public that political opponent of the current administration, or some outspoken critic of some intelligence or police agencies has been under surveillance for years. This could raise public sentiment strongly against escrowed encryption.

If signature keys are escrowed as well as encryption keys, there are additional issues. Is it acceptable for the authorities to use signature keys to run operations against suspected criminals? Will the authenticity of signatures based on escrowed keys be accepted in courts? What recourse do users have if the authorities actually do use their signature keys to sign some unfavorable contract, to help out a state-supported industry, or just to steal money?

The globalization of cryptography raises an additional set of questions. Will key-escrow policies be compatible across national borders? Will multi-national corporations have to keep separate escrowed keys in every country to stay in compliance with the various local laws? Without some kind of compatibility, one of the supposed advantages of key-escrow schemes (international use of strong encryption) falls apart.

What if some countries don't accept the security of escrow agencies on faith? How do users do business there? Are their digital contracts upheld by their courts, or is the fact that their signature key is held in escrow in the U.S. going to allow them to claim in Switzerland that someone else could have signed this electronic contract? Or will there be special waivers for people who do business in such countries?

And what about industrial espionage? There is no reason to believe that countries which currently conduct industrial espionage for their important or state-run companies will refrain from doing so on key-escrowed encryption systems. Indeed, since virtually no country is going to allow other countries to oversee its intelligence operations, widespread use of escrowed encryption will probably increase the use of wiretaps.

Even if countries with good civil rights records use key escrow only for the legitimate pursuit of criminals and terrorists, it's certain to be used elsewhere to keep track of dissidents, blackmail political opponents, and so on. Digital communications offer the opportunity to do a much more thorough job of monitoring citizens' actions, opinions, purchases, and associations than is possible in an analog world.

It's not clear how this will affect commercial key escrow, except that 20 years from now, selling Turkey or China a ready-made key-escrow system may look a lot like selling shock batons to South Africa in 1970, or building a chemical plant for Iraq in 1980. Even worse, effortless and untraceable tapping of communications may tempt a number of governments into tracking many of their citizens' communications, even those which haven't generally tried to do so before. And there's no guarantee that liberal democracies will be immune to this temptation.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 5
Advanced Protocols

5.1 Zero-Knowledge Proofs

Here's another story:

Alice: "I know the password to the Federal Reserve System computer, the ingredients in McDonald's secret sauce, and the contents of Volume 4 of Knuth."

Bob: "No, you don't."

Alice: "Yes, I do."

Bob: "Do not!"

Alice: "Do too!"

Bob: "Prove it!"

Alice: "All right. I'll tell you." She whispers in Bob's ear.

Bob: "That's interesting. Now I know it, too. I'm going to tell The Washington Post."

Alice: "Oops."

Unfortunately, the usual way for Alice to prove something to Bob is for Alice to tell him. But then he knows it, too. Bob can then tell anyone else he wants to and Alice can do nothing about it. (In the literature, different characters are often used in these protocols. Peggy is usually cast as the prover and Victor is the verifier. These names appear in the upcoming examples, instead of Alice and Bob.)

Using one-way functions, Peggy could perform a zero-knowledge proof [626]. This protocol proves to Victor that Peggy does have a piece of information, but it does not give Victor any way to know what the information is.

These proofs take the form of interactive protocols. Victor asks Peggy a series of questions. If Peggy knows the secret, she can answer all the questions correctly. If she does not, she has some chance—50 percent in the following examples—of answering correctly. After 10 or so questions, Victor will be convinced that Peggy knows the secret. Yet none of the questions or answers gives Victor any information about Peggy's information—only about her knowledge of it.

Basic Zero-Knowledge Protocol

Jean-Jacques Quisquater and Louis Guillou explain zero-knowledge with a story about a cave [1281]. The cave, illustrated in Figure 5.1, has a secret. Someone who knows the magic words can open the secret door between C and D. To everyone else, both passages lead to dead ends.

Peggy knows the secret of the cave. She wants to prove her knowledge to Victor, but she doesn't want to reveal the magic words. Here's how she convinces him:

(1) Victor stands at point A.
(2) Peggy walks all the way into the cave, either to point C or point D.
(3) After Peggy has disappeared into the cave, Victor walks to point B.
(4) Victor shouts to Peggy, asking her either to:
(a) come out of the left passage or
(b) come out of the right passage.
(5) Peggy complies, using the magic words to open the secret door if she has to.
(6) Peggy and Victor repeat steps (1) through (5) n times.

Assume that Victor has a camcorder and records everything he sees. He records Peggy disappearing into the cave, he records when he shouts out where he wants Peggy to come out from, and he records Peggy coming out. He records all n trials. If he showed this recording to Carol, would she believe that Peggy knew the magic words to open the door? No. What if Peggy and Victor had agreed beforehand what Victor would call out, and Peggy would make sure that she went into that path. Then she could come out where Victor asked her every time, without knowing the magic words. Or maybe they couldn't do that. Peggy would go into one of the passages and Victor would call out a random request. If Victor guessed right, great; if he didn't, they would edit that trial out of the camcorder recording. Either way, Victor can get a recording showing exactly the same sequence of events as in a real proof where Peggy knew the magic words.

Figure 5.1 The zero-knowledge cave.

This shows two things. One, it is impossible for Victor to convince a third party of the proof's validity. And two, it proves that the protocol is zero-knowledge. In the case where Peggy did not know the magic words, Victor will obviously not learn anything from watching the recording. But since there is no way to distinguish a real recording from a faked recording, Victor cannot learn anything from the real proof—it must be zero knowledge.

The technique used in this protocol is called cut and choose, because of its similarity to the classic protocol for dividing anything fairly:

(1) Alice cuts the thing in half.
(2) Bob chooses one of the halves for himself.
(3) Alice takes the remaining half.

It is in Alice's best interest to divide fairly in step (1), because Bob will choose whichever half he wants in step (2). Michael Rabin was the first person to use the cut-and-choose technique in cryptography [1282]. The concepts of interactive protocol and zero-knowledge were formalized later [626,627].

The cut-and-choose protocol works because there is no way Peggy can repeatedly guess which side Victor will ask her to come out of. If Peggy doesn't know the secret, she can only come out the way she came in. She has a 50 percent chance of guessing which side Victor will ask in each round (sometimes called an accreditation) of the protocol, so she has a 50 percent chance of fooling him. The chance of her fooling him in two rounds is 25 percent, and the chance of her fooling him all n times is 1 in 2n. After 16 rounds, Peggy has a 1 in 65,536 chance of fooling Victor. Victor can safely assume that if all 16 of Peggy's proofs are valid, then she must know the secret words to open the door between points C and D. (The cave analogy isn't perfect. Peggy can simply walk in one side and out the other; there's no need for any cut-and-choose protocol. However, mathematical zero knowledge requires it.)

Assume that Peggy knows some information, and furthermore that the information is the solution to a hard problem. The basic zero-knowledge protocol consists of several rounds.

(1) Peggy uses her information and a random number to transform the hard problem into another hard problem, one that is isomorphic to the original problem. She then uses her information and the random number to solve this new instance of the hard problem.
(2) Peggy commits to the solution of the new instance, using a bit-commitment scheme.
(3) Peggy reveals to Victor the new instance. Victor cannot use this new problem to get any information about the original instance or its solution.
(4) Victor asks Peggy either to:
(a) prove to him that the old and new instances are isomorphic (i.e., two different solutions to two related problems), or
(b) open the solution she committed to in step (2) and prove that it is a solution to the new instance.
(5) Peggy complies.
(6) Peggy and Victor repeat steps (1) through (5) n times.

Remember the camcorder in the cave protocol? You can do the same thing here. Victor can make a transcript of the exchange between him and Peggy. He cannot use this transcript to convince Carol, because he can always collude with Peggy to build a simulator that fakes Peggy's knowledge. This argument can be used to prove that the proof is zero-knowledge.

The mathematics behind this type of proof is complicated. The problems and the random transformation must be chosen carefully, so that Victor does not get any information about the solution to the original problem, even after many iterations of the protocol. Not all hard problems can be used for zero-knowledge proofs, but a lot of them can.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Graph Isomorphism

An example might go a long way to explain this concept; this one comes from graph theory [619,622]. A graph is a network of lines connecting different points. If two graphs are identical except for the names of the points, they are called isomorphic. For an extremely large graph, finding whether two graphs are isomorphic can take centuries of computer time; it's one of those NP-complete problems discussed in Section 11.1.

Assume that Peggy knows the isomorphism between the two graphs, G1 and G2. The following protocol will convince Victor of Peggy's knowledge:

(1) Peggy randomly permutes G1 to produce another graph, H, that is isomorphic to G1. Because Peggy knows the isomorphism between H and G1, she also knows the isomorphism between H and G2. For anyone else, finding an isomorphism between G1 and H or between G2 and H is just as hard as finding an isomorphism between G1 and G2.
(2) Peggy sends H to Victor.
(3) Victor asks Peggy either to:
(a) prove that H and G1 are isomorphic, or
(b) prove that H and G2 are isomorphic.
(4) Peggy complies. She either:
(a) proves that H and G1 are isomorphic, without proving that H and G2 are isomorphic, or
(b) proves that H and G2 are isomorphic, without proving that H and G1 are isomorphic.
(5) Peggy and Victor repeat steps (1) through (4) n times.

If Peggy does not know an isomorphism between G1 and G2, she cannot create graph H which is isomorphic to both. She can create a graph that is either isomorphic to G1 or one that is isomorphic to G2. Like the previous example, she has only a 50 percent chance of guessing which proof Victor will ask her to perform in step (3).

This protocol doesn't give Victor any useful information to aid him in figuring out an isomorphism between G1 and G2. Because Peggy generates a new graph H for each round of the protocol, he can get no information no matter how many rounds they go through the protocol. He won't be able to figure out an isomorphism between G1 and G2 from Peggy's answers.

In each round, Victor receives a new random permutation of H, along with an isomorphism between H and either G1 or G2. Victor could just as well have generated this by himself. Because Victor can create a simulation of the protocol, it can be proven to be zero-knowledge.

Hamiltonian Cycles

A variant of this example was first presented by Manuel Blum [196]. Peggy knows a circular, continuous path along the lines of a graph that passes through each point exactly once. This is called a Hamiltonian cycle. Finding a Hamiltonian cycle is another hard problem. Peggy has this piece of information—she probably got it by creating the graph with a certain Hamiltonian cycle—and this is what she wants to convince Victor that she knows.

Peggy knows the Hamiltonian cycle of a graph, G. Victor knows G, but not the Hamiltonian cycle. Peggy wants to prove to Victor that she knows this Hamiltonian cycle without revealing it. This is how she does it:

(1) Peggy randomly permutes G. She moves the points around and changes their labels to make a new graph, H. Since G and H are topologically isomorphic (i.e., the same graph), if she knows the Hamiltonian cycle of G then she can easily find the Hamiltonian cycle of H. If she didn't create H herself, determining the isomorphism between two graphs would be another hard problem; it could also take centuries of computer time. She then encrypts H to get H´. (This has to be a probabilistic encryption of each line in H, that is, an encrypted 0 or an encrypted 1 for each line in H.)
(2) Peggy gives Victor a copy of H´.
(3) Victor asks Peggy either to:
(a) prove to him that H´ is an encryption of an isomorphic copy of G, or
(b) show him a Hamiltonian cycle for H.
(4) Peggy complies. She either:
(a) proves that H´ is an encryption of an isomorphic copy of G by revealing the permutations and decrypting everything, without showing a Hamiltonian cycle for either G or H, or
(b) shows a Hamiltonian cycle for H by decrypting only those lines that constitute a Hamiltonian cycle, without proving that G and H are topologically isomorphic.
(5) Peggy and Victor repeat steps (1) through (4) n times.

If Peggy is honest, she can provide either proof in step (4) to Victor. However, if she does not know a Hamiltonian cycle for G, she cannot create an encrypted graph H´ which can meet both challenges. The best she can do is to create a graph that is either isomorphic to G or one that has the same number of points and lines and a valid Hamiltonian cycle. While she has a 50 percent chance of guessing which proof Victor will ask her to perform in step (3), Victor can repeat the protocol enough times to convince himself that Peggy knows a Hamiltonian cycle for G.

Parallel Zero-Knowledge Proofs

The basic zero-knowledge protocol involves n exchanges between Peggy and Victor. Why not do them all in parallel:

(1) Peggy uses her information and n random numbers to transform the hard problem into n different isomorphic problems. She then uses her information and the random numbers to solve the n new hard problems.
(2) Peggy commits to the solution of the n new hard problems.
(3) Peggy reveals to Victor the n new hard problems. Victor cannot use these new problems to get any information about the original problems or its solutions.
(4) For each of the n new hard problems, Victor asks Peggy either to:
(a) prove to him that the old and new problems are isomorphic, or
(b) open the solution she committed to in step (2) and prove that it is a solution to the new problem.
(5) Peggy complies for each of the n new hard problems.

Unfortunately, it's not that simple. This protocol does not have the same zero-knowledge properties as the previous protocol. In step (4), Victor can choose the challenges as a one-way hash of all the values committed to in the second step, thus making the transcript nonsimulatable. It is still zero-knowledge, but of a different sort. It seems to be secure in practice, but no one knows how to prove it. We do know that in certain circumstances, certain protocols for certain problems can be run in parallel while retaining their zero-knowledge property [247,106,546,616].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Noninteractive Zero-Knowledge Proofs

Carol can't be convinced because the protocol is interactive, and she is not involved in the interaction. To convince Carol, and anyone else who may be interested, we need a noninteractive protocol.

Protocols have been invented for noninteractive zero-knowledge proofs [477,198,478,197]. These protocols do not require any interaction; Peggy could publish them and thereby prove to anyone who takes the time to check that the proof is valid.

The basic protocol is similar to the parallel zero-knowledge proof, but a one-way hash function takes the place of Victor:

(1) Peggy uses her information and n random numbers to transform the hard problem into n different isomorphic problems. She then uses her information and the random numbers to solve the n new hard problems.
(2) Peggy commits to the solution of the n new hard problems.
(3) Peggy uses all of these commitments together as a single input to a one-way hash function. (After all, the commitments are nothing more than bit strings.) She then saves the first n bits of the output of this one-way hash function.
(4) Peggy takes the n bits generated in step (3). For each ith new hard problem in turn, she takes the ith bit of those n bits and:
(a) if it is a 0, she proves that the old and new problems are isomorphic, or
(b) if it is a 1, she opens the solution she committed to in step (2) and proves that it is a solution to the new problem.
(5) Peggy publishes all the commitments from step (2) as well as the solutions in step (4).
(6) Victor or Carol or whoever else is interested, verifies that steps (1) through (5) were executed properly.

This is amazing: Peggy can publish some data that contains no information about her secret, but can be used to convince anyone of the secret's existence. The protocol can also be used for digital signature schemes, if the challenge is set as a one-way hash of both the initial messages and the message to be signed.

This works because the one-way hash function acts as an unbiased random-bit generator. For Peggy to cheat, she has to be able to predict the output of the one-way hash function. (Remember, if she doesn't know the solution to the hard problem, she can do either (a) or (b) of step (4), but not both.) If she somehow knew what the one-way hash function would ask her to do, then she could cheat. However, there is no way for Peggy to force the one-way function to produce certain bits or to guess which bits it will produce. The one-way function is, in effect, Victor's surrogate in the protocol—randomly choosing one of two proofs in step (4).

In a noninteractive protocol, there must be many more iterations of the challenge/reply sequence. Peggy, not Victor, picks the hard problems using random numbers. She can pick different problems, hence different commitment vectors, till the hash function produces something she likes. In an interactive protocol, 10 iterations—a probability of 1 in 210 (1 in 1024) that Peggy can cheat—may be fine. However, that's not enough for noninteractive zero-knowledge proofs. Remember that Mallory can always do either (a) or (b) of step (4). He can try to guess which he will be asked to do, go through steps (1) through (3), and see if he guessed right. If he didn't, he can try again—repeatedly. Making 1024 guesses is easy on a computer. To prevent this brute-force attack, noninteractive protocols need 64 iterations, or even 128 iterations, to be valid.

This is the whole point of using a one-way hash function: Peggy cannot predict the output of the hash function because she cannot predict its input. The commitments which are used as the input are only known after she solves the new problems.

Generalities

Blum proved that any mathematical theorem can be converted into a graph such that the proof of that theorem is equivalent to proving a Hamiltonian cycle in the graph. The general case that any NP statement has a zero-knowledge proof, assuming one-way functions and therefore good encryption algorithms, was proved in [620]. Any mathematical proof can be converted into a zero-knowledge proof. Using this technique, a researcher can prove to the world that he knows the proof of a particular theorem without revealing what that solution is. Blum could have published these results without revealing them.

There are also minimum-disclosure proofs [590]. In a minimum-disclosure proof, the following properties hold:

1. Peggy cannot cheat Victor. If Peggy does not know the proof, her chances of convincing Victor that she knows the proof are negligible.
2. Victor cannot cheat Peggy. He doesn't get the slightest hint of the proof, apart from the fact that Peggy knows the proof. In particular, Victor cannot demonstrate the proof to anyone else without proving it himself from scratch.

Zero-knowledge proofs have an additional condition:

3. Victor learns nothing from Peggy that he could not learn by himself without Peggy, apart from the fact that Peggy knows the proof.

There is considerable mathematical difference between proofs that are only minimum-disclosure and those that are zero-knowledge. That distinction is beyond the scope of this book, but more sophisticated readers are welcome to peruse the references. The concepts were introduced in [626,619,622]. Further elaboration on their ideas, based on different mathematical assumptions, were developed in [240,319,239].

There are also different kinds of zero-knowledge proofs:

— Perfect. There is a simulator that gives transcripts identically distributed to real transcripts (the Hamiltonian cycle and graph isomorphism examples).
— Statistical. There is a simulator that gives transcripts identically distributed to real transcripts, except for some constant number of exceptions.
— Computational. There is a simulator that gives transcripts indistinguishable from real transcripts.
— No-use. A simulator may not exist, but we can prove that Victor will not learn any polynomial amount of information from the proof (the parallel example).

Over the years, extensive work, both theoretical and applied, has been done on minimum-disclosure and zero-knowledge proofs. Mike Burmester and Yvo Desmedt invented broadcast interactive proofs, where one prover can broadcast a zero-knowledge interactive proof to a large group of verifiers [280]. Cryptographers proved that everything that can be proven with an interactive proof can also be proven with a zero-knowledge interactive proof [753,137].

A good survey article on the topic is [548]. For additional mathematical details, variations, protocols, and applications, consult [590,619,240,319,620,113,241,1528,660,238,591,617,510,592,214,104,216,832,
97,939,622,482,615,618,215,476,71]. A lot has been written on this subject.

5.2 Zero-Knowledge Proofs of Identity

In the real world, we often use physical tokens as proofs of identity: passports, driver's licenses, credit cards, and so on. The token contains something that links it to a person: a picture, usually, or a signature, but it could almost as easily be a thumbprint, a retinal scan, or a dental x-ray. Wouldn't it be nice to do the same thing digitally?

Using zero-knowledge proofs as proofs of identity was first proposed by Uriel Feige, Amos Fiat, and Adi Shamir [566,567]. Alice's private key becomes a function of her "identity." Using a zero-knowledge proof, she proves that she knows her private key and therefore proves her identity. Algorithms for this can be found in Section 23.11.

This idea is quite powerful. It allows a person to prove his identity without any physical token. However, it's not perfect. Here are some abuses.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The Chess Grandmaster Problem

Here's how Alice, who doesn't even know the rules to chess, can defeat a grandmaster. (This is sometimes called the Chess Grandmaster Problem.) She challenges both Gary Kasparov and Anatoly Karpov to a game, at the same time and place, but in separate rooms. She plays white against Kasparov and black against Karpov. Neither grandmaster knows about the other.

Karpov, as white, makes his first move. Alice records the move and walks into the room with Kasparov. Playing white, she makes the same move against Kasparov. Kasparov makes his first move as black. Alice records the move, walks into the room with Karpov, and makes the same move. This continues, until she wins one game and loses the other, or both games end in a draw.

In reality, Kasparov is playing Karpov and Alice is simply acting as the middleman, mimicking the moves of each grandmaster on the other's board. However, if neither Karpov nor Kasparov knows about the other's presence, each will be impressed with Alice's play.

This kind of fraud can be used against zero-knowledge proofs of identity [485,120]. While Alice is proving her identity to Mallory, Mallory can simultaneously prove to Bob that he is Alice.

The Mafia Fraud

When discussing his zero-knowledge identification protocol, Adi Shamir [1424] said: "I could go to a Mafia-owned store a million successive times and they will still not be able to misrepresent themselves as me."

Here's how the Mafia can. Alice is eating at Bob's Diner, a Mafia-owned restaurant. Carol is shopping at Dave's Emporium, an expensive jewelry store. Bob and Carol are both members of the Mafia and are communicating by a secret radio link. Alice and Dave are unaware of the fraud.

At the end of Alice's meal, when she is ready to pay and prove her identity to Bob, Bob signals Carol that the fraud is ready to begin. Carol chooses some expensive diamonds and gets ready to prove her identity to Dave. Now, as Alice proves her identity to Bob, Bob radios Carol and Carol performs the same protocol with Dave. When Dave asks a question in the protocol, Carol radios the question back to Bob, and Bob asks it of Alice. When Alice answers, Bob radios the correct answer to Carol. Actually, Alice is just proving her identity to Dave, and Bob and Carol are simply sitting in the middle of the protocol passing messages back and forth. When the protocol finishes, Alice has proved herself to Dave and has purchased some expensive diamonds (which Carol disappears with).

The Terrorist Fraud

If Alice is willing to collaborate with Carol, they can also defraud Dave. In this protocol, Carol is a well-known terrorist. Alice is helping her enter the country. Dave is the immigration officer. Alice and Carol are connected by a secret radio link.

When Dave asks Carol questions as part of the zero-knowledge protocol, Carol radios them back to Alice, who answers them herself. Carol recites these answers to Dave. In reality, Alice is proving her identity to Dave, with Carol acting as a communications path. When the protocol finishes, Dave thinks that Carol is Alice and lets her into the country. Three days later, Carol shows up at some government building with a minivan full of explosives.

Suggested Solutions

Both the Mafia and Terrorist frauds are possible because the conspirators can communicate via a secret radio. One way to prevent this requires all identifications to take place inside Faraday cages, which block all electromagnetic radiation. In the terrorist example, this assures immigration officer Dave that Carol was not receiving her answers from Alice. In the Mafia example, Bob could simply build a faulty Faraday cage in his restaurant, but jeweler Dave would have a working one; Bob and Carol would not be able to communicate. To solve the Chess Grandmaster Problem, Alice should be forced to sit in her seat until the end of a game.

Thomas Beth and Yvo Desmedt proposed another solution, one using accurate clocks [148]. If each step in the protocol must take place at a given time, no time would be available for the conspirators to communicate. In the Chess Grandmaster Problem, if every move in each game must be made as a clock strikes one minute, then Alice will have no time to run from room to room. In the Mafia story, Bob and Carol will have no time to pass questions and answers to one another.

The Multiple Identity Fraud

There are other possible abuses to zero-knowledge proofs of identity, also discussed in [485,120]. In some implementations, there is no check when an individual registers a public key. Hence, Alice can have several private keys and, therefore, several identities. This can be a great help if she wants to commit tax fraud. Alice can also commit a crime and disappear. First, she creates and publishes several identities. One of them she doesn't use. Then, she uses that identity once and commits a crime so that the person who identifies her is the witness. Then, she immediately stops using that identity. The witness knows the identity of the person who committed the crime, but if Alice never uses that identity again—she's untraceable.

To prevent this, there has to be some mechanism by which each person has only one identity. In [120] the authors suggest the bizarre idea of tamperproof babies who are impossible to clone and contain a unique number as part of their genetic code. They also suggested having each baby apply for an identity at birth. (Actually, the parents would have to do this as the baby would be otherwise occupied.) This could easily be abused; parents could apply for multiple identities at the child's birth. In the end, the uniqueness of an individual is based on trust.

Renting Passports

Alice wants to travel to Zaire, but that government won't give her a visa. Carol offers to rent her identity to Alice. (Bob offered first, but there were some obvious problems.) Carol sells Alice her private key and Alice goes off to Zaire pretending to be Carol.

Carol has not only been paid for her identity, but now she has a perfect alibi. She commits a crime while Alice is in Zaire. "Carol" has proved her identity in Zaire; how could she commit a crime back home?

Of course, Alice is free to commit crimes as well. She does so either before she leaves or after she returns, near Carol's home. First she identifies herself as Carol (she has Carol's private key, so she can easily do that), then she commits a crime and runs away. The police will come looking for Carol. Carol will claim she rented her identity to Alice, but who would believe such a nonsensical story?

The problem is that Alice isn't really proving her identity; she is proving that she knows a piece of secret information. It is the link between that information and the person it belongs to that is being abused. The tamperproof baby solution would protect against this type of fraud, as would a police state where all citizens would have to prove their identity very frequently (at the end of each day, at each street corner, etc.). Biometric methods—fingerprints, retinal scanning, voiceprints, and so on—may help solve this problem.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Proofs of Membership

Alice wants to prove to Bob that she is a member of some super-secret organization, but she does not want to reveal her identity. This problem is similar but different to proving identity, and has also been studied [887,906,907,1201,1445]. Some solutions are related to the problem of group signatures (see Section 4.6).

5.3 Blind Signatures

An essential feature of digital signature protocols is that the signer knows what he is signing. This is a good idea, except when we want the reverse.

We might want people to sign documents without ever seeing their contents. There are ways that a signer can almost, but not exactly, know what he is signing. But first things first.

Completely Blind Signatures

Bob is a notary public. Alice wants him to sign a document, but does not want him to have any idea what he is signing. Bob doesn't care what the document says; he is just certifying that he notarized it at a certain time. He is willing to go along with this.

(1) Alice takes the document and multiplies it by a random value. This random value is called a blinding factor.
(2) Alice sends the blinded document to Bob.
(3) Bob signs the blinded document.
(4) Alice divides out the blinding factor, leaving the original document signed by Bob.

This protocol only works if the signature function and multiplication are commutative. If they are not, there are other ways to modify the document other than by multiplying. Some relevant algorithms appear in Section 23.12. For now, assume that the operation is multiplication and all the math works.

Can Bob cheat? Can he collect any information about the document that he is signing? If the blinding factor is truly random and makes the blinded document truly random, he cannot. The blinded document Bob signs in step (2) looks nothing like the document Alice began with. The blinded document with Bob's signature on it in step (3) looks nothing like the signed document at the end of step (4). Even if Bob got his hands on the document, with his signature, after completing the protocol, he cannot prove (to himself or to anyone else) that he signed it in that particular protocol. He knows that his signature is valid. He can, like anyone else, verify his signature. However, there is no way for him to correlate any information he received during the signing protocol with the signed document. If he signed a million documents using this protocol, he would have no way of knowing in which instance he signed which document.

The properties of completely blind signatures are:

1. Bob's signature on the document is valid. The signature is a proof that Bob signed the document. It will convince Bob that he signed the document if it is ever shown to him. It also has all of the other properties of digital signatures discussed in Section 2.6.
2. Bob cannot correlate the signed document with the act of signing the document. Even if he keeps records of every blind signature he makes, he cannot determine when he signed any given document.

Eve, who is in the middle, watching this protocol, has even less information than Bob.

Blind Signatures

With the completely blind signature protocol, Alice can have Bob sign anything: "Bob owes Alice a million dollars," "Bob owes Alice his first-born child," "Bob owes Alice a bag of chocolates." The possibilities are endless. This protocol isn't useful in many applications.

However, there is a way that Bob can know what he is signing, while still maintaining the useful properties of a blind signature. The heart of this protocol is the cut-and-choose technique. Consider this example. Many people enter this country every day, and the Department of Immigration wants to make sure they are not smuggling cocaine. The officials could search everyone, but instead they use a probabilistic solution. They will search one-tenth of the people coming in. One person in ten has his belongings inspected; the other nine get through untouched. Chronic smugglers will get away with their misdeeds most of the time, but they have a 10 percent chance of getting caught. And if the court system is effective, the penalty for getting caught once will more than wipe out the gains from the other nine times.

If the Department of Immigration wants to increase the odds of catching smugglers, they have to search more people. If they want to decrease the odds, they have to search fewer people. By manipulating the probabilities, they control how successful the protocol is in catching smugglers.

The blind signature protocol works in a similar manner. Bob will be given a large pile of different blinded documents. He will open, that is examine, all but one and then sign the last.

Think of the blinded document as being in an envelope. The process of blinding the document is putting the document in an envelope and the process of removing the blinding factor is opening the envelope. When the document is in an envelope, nobody can read it. The document is signed by having a piece of carbon paper in the envelope: When the signer signs the envelope, his signature goes through the carbon paper and signs the document as well.

This scenario involves a group of counterintelligence agents. Their identities are secret; not even the counterintelligence agency knows who they are. The agency's director wants to give each agent a signed document stating: "The bearer of this signed document, (insert agent's cover name here), has full diplomatic immunity." Each of the agents has his own list of cover names, so the agency can't just hand out signed documents. The agents do not want to send their cover names to the agency; the enemy might have corrupted the agency's computer. On the other hand, the agency doesn't want to blindly sign any document an agent gives it. A clever agent might substitute a message like: "Agent (name) has retired and collects a million-dollar-a-year pension. Signed, Mr. President." In this case, blind signatures could be useful.

Assume that all the agents have 10 possible cover names, which they have chosen themselves and which no one else knows. Also assume that the agents don't care under which cover name they are going to get diplomatic immunity. Also assume that the agency's computer is the Agency's Large Intelligent Computing Engine, or ALICE, and that our particular agent is the Bogota Operations Branch: BOB.

(1) BOB prepares n documents, each using a different cover name, giving himself diplomatic immunity.
(2) BOB blinds each of these documents with a different blinding factor.
(3) BOB sends the n blinded documents to ALICE.
(4) ALICE chooses n – 1 documents at random and asks BOB for the blinding factors for each of those documents.
(5) BOB sends ALICE the appropriate blinding factors.
(6) ALICE opens (i.e., she removes the blinding factor) n – 1 documents and makes sure they are correct—and not pension authorizations.
(7) ALICE signs the remaining document and sends it to BOB.
(8) Agent removes the blinding factor and reads his new cover name: "The Crimson Streak." The signed document gives him diplomatic immunity under that name.

This protocol is secure against BOB cheating. For him to cheat, he would have to predict accurately which document ALICE would not examine. The odds of him doing this are 1 in n—not very good. ALICE knows this and feels confident signing a document that she is not able to examine. With this one document, the protocol is the same as the previous completely blinded signature protocol and maintains all of its properties of anonymity.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

There is a trick that makes BOB's chance of cheating even smaller. In step (4), ALICE randomly chooses n/2 of the documents to challenge, and BOB sends her the appropriate blinding factors in step (5). In step (7), ALICE multiplies together all of the unchallenged documents and signs the mega-document. In step (8), BOB strips off all the blinding factors. ALICE's signature is acceptable only if it is a valid signature of the product of n/2 identical documents. To cheat BOB has to be able to guess exactly which subset ALICE will challenge; the odds are much smaller than the odds of guessing which one document ALICE won't challenge.

BOB has another way to cheat. He can generate two different documents, one that ALICE is willing to sign and one that ALICE is not. Then he can find two different blinding factors that transform each document into the same blinded document. That way, if ALICE asks to examine the document, BOB gives her the blinding factor that transforms it into the benign document. If ALICE doesn't ask to see the document and signs it, he uses the blinding factor that transforms it into the malevolent document. While this is theoretically possible, the mathematics of the particular algorithms involved make the odds of BOB's being able to find such a pair negligibly small. In fact, it can be made as small as the odds of Bob being able to produce the signature on an arbitrary message himself. This issue is discussed further in Section 23.12.

Patents

Chaum has patents for several flavors of blind signatures (see Table 5.1).

5.4 Identity-Based Public-Key Cryptography

Alice wants to send a secure message to Bob. She doesn't want to get his public key from a key server; she doesn't want to verify some trusted third party's signature on his public-key certificate; and she doesn't even want to store Bob's public key on her own computer. She just wants to send him a secure message.

Identity-based cryptosystems, sometimes called Non-Interactive Key Sharing (NIKS) systems, solve this problem [1422]. Bob's public key is based on his name and network address (or telephone number, or physical street address, or whatever). With normal public-key cryptography, Alice needs a signed certificate that associates Bob's public key with his identity. With identity-based cryptography, Bob's public key is his identity. This is a really cool idea, and about as ideal as you can get for a mail system: If Alice knows Bob's address, she can send him secure mail. It makes the cryptography about as transparent as possible.

The system is based on Trent issuing private keys to users based on their identity. If Alice's private key is compromised, she has to change some aspect of her identity to get another one. A serious problem is designing a system in such a way that a collusion of dishonest users cannot forge a key.

A lot of work has been done on the mathematics of these sorts of schemes—most of it in Japan—which turn out to be infuriatingly complicated to make secure. Many of the proposed solutions involve Trent choosing a random number for each user—in my opinion this defeats the real point of the system. Some of the algorithms discussed in Chapters 19 and 20 can be identity-based. For details, algorithms, and cryptanalysis, see [191,1422,891,1022,1515,1202,1196,908,692,674,1131,1023,1516,1536,1544,63,
1210,314,313,1545,1539,1543,933,1517,748,1228]. An algorithm that does not rely on any random numbers is [1035]. The system discussed in [1546,1547,1507] is insecure against a chosen-public-key attack; so is the system proposed as NIKS-TAS [1542,1540,1541,993,375,1538]. Honestly, nothing proposed so far is both practical and secure.

TABLE 5.1 Chaum's Blind Signature Patents
U.S. PATENT # | DATE | TITLE
4,759,063 | 7/19/88 | Blind Signature Systems [323]
4,759,064 | 7/19/88 | Blind Unanticipated Signature Systems [324]
4,914,698 | 3/3/90 | One-Show Blind Signature Systems [326]
4,949,380 | 8/14/90 | Returned-Value Blind Signature Systems [328]
4,991,210 | 2/5/91 | Unpredictable Blind Signature Systems [331]

5.5 Oblivious Transfer

Cryptographer Bob is desperately trying to factor a 500-bit number, n. He knows it's the product of five 100-bit numbers, but nothing more. (This is a problem. If he can't recover the key he'll have to work overtime and he'll miss his weekly mental poker game with Alice.)

What do you know? Here comes Alice now:

"I happen to know one factor of the number," she says, "and I'll sell it to you for $100. That's a dollar a bit." To show she's serious, she uses a bit-commitment scheme and commits to each bit individually.

Bob is interested, but has only $50. Alice is unwilling to lower her price and offers to sell Bob half the bits for half the price. "It'll save you a considerable amount of work," she says.

"But how do I know that your number is actually a factor of n? If you show me the number and let me verify that it is a factor, then I will agree to your terms," says Bob.

They are at an impasse. Alice cannot convince Bob that her number is a factor of n without revealing it, and Bob is unwilling to buy 50 bits of a number that could very well be worthless.

This story, stolen from Joe Kilian [831], introduces the concept of oblivious transfer. Alice transmits a group of messages to Bob. Bob receives some subset of those messages, but Alice has no idea which ones he receives. This doesn't completely solve the problem, however. After Bob has received a random half of the bits, Alice has to convince him that the bits she sent are part of a factor of n, using a zero-knowledge proof.

In the following protocol, Alice will send Bob one of two messages. Bob will receive one, and Alice will not know which.

(1) Alice generates two public-key/private-key key pairs, or four keys in all. She sends both public keys to Bob.
(2) Bob chooses a key in a symmetric algorithm (DES, for example). He chooses one of Alice's public keys and encrypts his DES key with it. He sends the encrypted key to Alice without telling her which of her public keys he used to encrypt it.
(3) Alice decrypts Bob's key twice, once with each of her private keys. In one of the cases, she uses the correct key and successfully decrypts Bob's DES key. In the other case, she uses the wrong key and only manages to generate a meaningless pile of bits that nonetheless look like a random DES key. Since she does not know the correct plaintext, she has no idea which is which.
(4) Alice encrypts both of her messages, each with a different one of the DES keys she generated in the previous step (one real and one meaningless) and sends both of them to Bob.
(5) Bob gets one of Alice's messages encrypted with the proper DES key and the other one encrypted with the gibberish DES key. When Bob decrypts each of them with his DES key, he can read one of them; the other just looks like gibberish to him.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Bob now has one of the two messages from Alice and Alice does not know which one he was able to read successfully. Unfortunately, if the protocol stopped here it would be possible for Alice to cheat. Another step is necessary.

(6) After the protocol is complete and both possible results of the transfer are known, Alice must give Bob her private keys so that he can verify that she did not cheat. After all, she could have encrypted the same message with both keys in step (4).

At this point, of course, Bob can figure out the second message.

The protocol is secure against an attack by Alice because she has no way of knowing which of the two DES keys is the real one. She encrypts them both, but Bob only successfully recovers one of them—until step (6). It is secure against an attack by Bob because, before step (6), he cannot get Alice's private keys to determine the DES key that the other message was encrypted in. This may still seem like nothing more than a more complicated way to flip coins over a modem, but it has extensive implications when used in more complicated protocols.

Of course, nothing stops Alice from sending Bob two completely useless messages: "Nyah Nyah" and "You sucker." This protocol guarantees that Alice sends Bob one of two messages; it does nothing to ensure that Bob wants to receive either of them.

Other oblivious transfer protocols are found in the literature. Some of them are noninteractive, meaning that Alice can publish her two messages and Bob can learn only one of them. He can do this on his own; he doesn't have to communicate with Alice [105].

No one really cares about being able to do oblivious transfer in practice, but the notion is an important building block for other protocols. Although there are many types of oblivious transfer—I have two secrets and you get one; I have n secrets and you get one; I have one secret which you get with probability 1/2; and so on—they are all equivalent [245,391,395].

5.6 Oblivious Signatures

Honestly, I can't think of a good use for these, but there are two kinds [346]:

1. Alice has n different messages. Bob can choose one of the n messages for Alice to sign, and Alice will have no way of knowing which one she signed.
2. Alice has one message. Bob can choose one of n keys for Alice to use in signing the message, and Alice will have no way of knowing which key she used.

It's a neat idea; I'm sure it has a use somewhere.

5.7 Simultaneous Contract Signing

Contract Signing with an Arbitrator

Alice and Bob want to enter into a contract. They've agreed on the wording, but neither wishes to sign unless the other signs as well. Face to face, this is easy: Both sign together. Over a distance, they could use an arbitrator.

(1) Alice signs a copy of the contract and sends it to Trent.
(2) Bob signs a copy of the contract and sends it to Trent.
(3) Trent sends a message to both Alice and Bob indicating that the other has signed the contract.
(4) Alice signs two copies of the contract and sends them to Bob.
(5) Bob signs both copies of the contract, keeps one for himself, and sends the other to Alice.
(6) Alice and Bob both inform Trent that they each have a copy of the contract signed by both of them.
(7) Trent tears up his two copies of the contract with only one signature each.

This protocol works because Trent prevents either of the parties from cheating. If Bob were to refuse to sign the contract in step (5), Alice could appeal to Trent for a copy of the contract already signed by Bob. If Alice were to refuse to sign in step (4), Bob could do the same. When Trent indicates that he received both contracts in step (3), both Alice and Bob know that the other is bound by the contract. If Trent does not receive both contracts in steps (1) and (2), he tears up the one he received and neither party is bound.

Simultaneous Contract Signing without an Arbitrator (Face-to-Face)

If Alice and Bob were sitting face-to-face, they could sign the contract this way [1244]:

(1) Alice signs the first letter of her name and passes the contract to Bob.
(2) Bob signs the first letter of his name and passes the contract to Alice.
(3) Alice signs the second letter of her name and passes the contract to Bob.
(4) Bob signs the second letter of his name and passes the contract to Alice.
(5) This continues until both Alice and Bob have signed their entire names.

If you ignore the obvious problem with this protocol (Alice has a longer name than Bob), it works just fine. After signing only one letter, Alice knows that no judge will bind her to the terms of the contract. But the letter is an act of good faith, and Bob responds with a similar act of good faith.

After each party has signed several letters, a judge could probably be convinced that both parties had signed the contract. The details are murky, though. Surely they are not bound after only the first letter; just as surely they are bound after they sign their entire names. At what point in the protocol do they become bound? After signing one-half of their names? Two-thirds of their names? Three-quarters?

Since neither Alice nor Bob is certain of the exact point at which she or he is bound, each has at least some fear that she or he is bound throughout the protocol. At no point can Bob say: "You signed four letters and I only signed three. You are bound but I am not." Bob has no reason not to continue with the protocol. Furthermore, the longer they continue, the greater the probability that a judge will rule that they are bound. Again, there is no reason not to continue with the protocol. After all, they both wanted to sign the contract; they just didn't want to sign before the other one.

Simultaneous Contract Signing without an Arbitrator (Not Face-to-Face)

This protocol uses the same sort of uncertainty [138]. Alice and Bob alternate taking baby steps toward signing until both have signed.

In the protocol, Alice and Bob exchange a series of signed messages of the form: "I agree that with probability p, I am bound by this contract."

The recipient of this message can take it to a judge and, with probability p, the judge will consider the contract to be signed.

(1) Alice and Bob agree on a date by which the signing protocol should be completed.
(2) Alice and Bob decide on a probability difference that they are willing to live with. For example, Alice might decide that she is not willing to be bound with a greater probability than 2 percent over Bob's probability. Call Alice's difference a; call Bob's difference b.
(3) Alice sends Bob a signed message with p = a.
(4) Bob sends Alice a signed message with p = a + b.
(5) Let p be the probability of the message Alice received in the previous step from Bob. Alice sends Bob a signed message with p´ = p + a or 1, whichever is smaller.
(6) Let p be the probability of the message Bob received in the previous step from Alice. Bob sends Alice a signed message with p´ = p + b or 1, whichever is smaller.
(7) Alice and Bob continue alternating steps (5) and (6) until both have received messages with p = 1 or until the date agreed to in step (1) has passed.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

As the protocol proceeds, both Alice and Bob agree to be bound to the contract with a greater and greater probability. For example, Alice might define her a as 2 percent and Bob might define his b as 1 percent. (It would be nice if they had chosen larger increments; we will be here for a while.) Alice's first message might state that she is bound with 2 percent probability. Bob might respond that he is bound with 3 percent probability. Alice's next message might state that she is bound with 5 percent probability and so on, until both are bound with 100 percent probability.

If both Alice and Bob complete the protocol by the completion date, all is well. Otherwise, either party can take the contract to the judge, along with the other party's last signed message. The judge then randomly chooses a value between 0 and 1 before seeing the contract. If the value is less than the probability the other party signed, then both parties are bound. If the value is greater than the probability, then both parties are not bound. (The judge then saves the value, in case he has to rule on another matter regarding the same contract.) This is what is meant by being bound to the contract with probability p.

That's the basic protocol, but it can have more complications. The judge can rule in the absence of one of the parties. The judge's ruling either binds both or neither party; in no situation is one party bound and the other one not. Furthermore, as long as one party is willing to have a slightly higher probability of being bound than the other (no matter how small), the protocol will terminate.

Simultaneous Contract Signing without an Arbitrator (Using Cryptography)

This cryptographic protocol uses the same baby-step approach [529]. DES is used in the description, although any symmetric algorithm will do.

(1) Both Alice and Bob randomly select 2n DES keys, grouped in pairs. The pairs are nothing special; they are just grouped that way for the protocol.
(2) Both Alice and Bob generate n pairs of messages, Li and Ri: "This is the left half of my ith signature" and "This is the right half of my ith signature," for example. The identifier, i, runs from 1 to n. Each message will probably also include a digital signature of the contract and a timestamp. The contract is considered signed if the other party can produce both halves, Li and Ri, of a single signature pair.
(3) Both Alice and Bob encrypt their message pairs in each of the DES key pairs, the left message with the left key in the pair and the right message with the right key in the pair.
(4) Alice and Bob send each other their pile of 2n encrypted messages, making clear which messages are which halves of which pairs.
(5) Alice and Bob send each other every key pair using the oblivious transfer protocol for each pair. That is, Alice sends Bob either the key used to encrypt the left message or the key used to encrypt the right message, independently, for each of the n pairs. Bob does the same. They can either alternate sending halves or one can send 100 and then the other—it doesn't matter. Now both Alice and Bob have one key in each key pair, but neither knows which halves the other one has.
(6) Both Alice and Bob decrypt the message halves that they can, using the keys they received. They make sure that the decrypted messages are valid.
(7) Alice and Bob send each other the first bits of all 2n DES keys.
(8) Alice and Bob repeat step (7) for the second bits of all 2n DES keys, the third bits, and so on, until all the bits of all the DES keys have been transferred.
(9) Alice and Bob decrypt the remaining halves of the message pairs and the contract is signed.
(10) Alice and Bob exchange the private keys used during the oblivious transfer protocol in step (5) and each verifies that the other did not cheat.

Why do Alice and Bob have to go through all this work? Let's assume Alice wants to cheat and see what happens. In steps (4) and (5), Alice could disrupt the protocol by sending Bob nonsense bit strings. Bob would catch this in step (6), when he tried to decrypt whatever half he received. Bob could then stop safely, before Alice could decrypt any of Bob's message pairs.

If Alice were very clever, she could only disrupt half the protocol. She could send one half of each pair correctly, but send a gibberish string for the other half. Bob has only a 50 percent chance of receiving the correct half, so half the time Alice could cheat. However, this only works if there is one key pair. If there were only two pairs, this sort of deception would succeed 25 percent of the time. That is why n should be large. Alice has to guess correctly the outcome of n oblivious transfer protocols; she has a 1 in 2n chance of doing this. If n = 10, Alice has a 1 in 1024 chance of deceiving Bob.

Alice could also send Bob random bits in step (8). Perhaps Bob won't know that she is sending him random bits until he receives the whole key and tries to decrypt the message halves. But again, Bob has probability on his side. He has already received half of the keys, and Alice does not know which half. If n is large enough, Alice is sure to send him a nonsense bit to a key he has already received and he will know immediately that she is trying to deceive him.

Maybe Alice will just go along with step (8) until she has enough bits of the keys to mount a brute-force attack and then stop transmitting bits. DES has a 56-bit-long key. If she receives 40 of the 56 bits, she only has to try 216, or 65,536, keys in order to read the message—a task certainly within the realm of a computer's capabilities. But Bob will have exactly the same number of bits of her keys (or, at worst, one bit less), so he can do the same thing. Alice has no real choice but to continue the protocol.

The basic point is that Alice has to play fairly, because the odds of fooling Bob are just too small. At the end of the protocol, both parties have n signed message pairs, any one of which is sufficient for a valid signature.

There is one way Alice can cheat; she can send Bob identical messages in Step (5). Bob can't detect this until after the protocol is finished, but he can use a transcript of the protocol to convince a judge of Alice's duplicity.

There are two weaknesses with protocols of this type [138]. First, it's a problem if one of the parties has significantly more computing power than the other. If, for example, Alice can mount a brute-force attack faster than Bob can, then she can stop sending bits early in step (8), and figure out Bob's keys herself. Bob, who cannot do the same in a reasonable amount of time, will not be happy.

Second, it's a problem if one of the parties stops the protocol early. If Alice abruptly stops the protocol, both face similar computational efforts, but Bob does not have any real legal recourse. If, for example, the contract specifies that she do something in a week, and Alice terminates the protocol at a point when Bob would have to spend a year's worth of computing power before she is really committed, that's a problem. The real difficulty here is the lack of a near-term deadline by which the process cleanly terminates with either both or neither party bound.

These problems also apply to the protocols in Sections 5.8 and 5.9.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

5.8 Digital Certified Mail

The same simultaneous oblivious transfer protocol used for contract signing works, with some modifications, for computer certified mail [529]. Suppose Alice wants to send a message to Bob, but she does not want him to read it without signing a receipt. Surly postal workers handle this process in real life, but the same thing can be done with cryptography. Whitfield Diffie first discussed this problem in [490].

At first glance, the simultaneous contract-signing protocol can do this. Alice simply encrypts her message with a DES key. Her half of the protocol can be something like: "This is the left half of the DES key: 32f5," and Bob's half can be something like: "This is the left half of my receipt." Everything else stays the same.

To see why this won't work, remember that the protocol hinges on the fact that the oblivious transfer in step (5) keeps both parties honest. Both of them know that they sent the other party a valid half, but neither knows which. They don't cheat in step (8) because the odds of getting away with it are miniscule. If Alice is sending Bob not a message but half of a DES key, Bob can't check the validity of the DES key in step (6). Alice can still check the validity of Bob's receipt, so Bob is still forced to be honest. Alice can freely send Bob some garbage DES key, and he won't know the difference until she has a valid receipt. Tough luck, Bob.

Getting around this problem requires some adjustment of the protocol:

(1) Alice encrypts her message using a random DES key, and sends the message to Bob.
(2) Alice generates n pairs of DES keys. The first key of each pair is generated at random; the second key of each pair is the XOR of the first key and the message encryption key.
(3) Alice encrypts a dummy message with each of her 2n keys.
(4) Alice sends the whole pile of encrypted messages to Bob, making sure he knows which messages are which halves of which pairs.
(5) Bob generates n pairs of random DES keys.
(6) Bob generates a pair of messages that indicates a valid receipt. "This is the left half of my receipt" and "this is the right half of my receipt" are good candidates, with the addition of some kind of random-bit string. He makes n receipt pairs, each numbered. As with the previous protocol, the receipt is considered valid if Alice can produce both halves of a receipt (with the same number) and all of her encryption keys.
(7) Bob encrypts each of his message pairs with DES key pairs, the ith message pair with the ith key pair, the left message with the left key in the pair, and the right message with the right key in the pair.
(8) Bob sends his pile of message pairs to Alice, making sure that Alice knows which messages are which halves of which pairs.
(9) Alice and Bob send each other every key pair using the oblivious transfer protocol. That is, Alice sends Bob either the key used to encrypt the left message or the key used to encrypt the right message, for each of the n pairs. Bob does the same. They can either alternate sending halves or one can send n and then the other—it doesn't matter. Now both Alice and Bob have one key in each key pair, but neither knows which halves the other has.
(10) Both Alice and Bob decrypt the halves they can and make sure that the decrypted messages are valid.
(11) Alice and Bob send each other the first bits of all 2n DES keys. (If they are worried about Eve being able to read these mail messages, then they should encrypt their transmissions to each other.)
(12) Alice and Bob repeat step (11) for the second bits of all 2n DES keys, the third bits, and so on, until all the bits of all the DES keys have been transferred.
(13) Alice and Bob decrypt the remaining halves of the message pairs. Alice has a valid receipt from Bob, and Bob can XOR any key pair to get the original message encryption key.
(14) Alice and Bob exchange the private keys used during the oblivious transfer protocol and each verifies that the other did not cheat.

Steps (5) through (8) for Bob, and steps (9) through (12) for both Alice and Bob, are the same as the contract-signing protocol. The twist is all of Alice's dummy messages. They give Bob some way of checking the validity of her oblivious transfer in step (10), which forces her to stay honest during steps (11) through (13). And, as with the simultaneous contract-signing protocol, both a left and a right half of one of Alice's message pairs are required to complete the protocol.

5.9 Simultaneous Exchange of Secrets

Alice knows secret A; Bob knows secret B. Alice is willing to tell Bob A, if Bob tells her B. Bob is willing to tell Alice B, if Alice tells him A. This protocol, observed in a schoolyard, does not work:

(1) Alice: "I'll tell if you tell me first."
(2) Bob: "I'll tell if you tell me first."
(3) Alice: "No, you first."
(4) Bob: "Oh, all right." Bob whispers.
(5) Alice: "Ha! I won't tell you."
(6) Bob: "That's not fair."

Cryptography can make it fair. The previous two protocols are implementations of this more general protocol, one that lets Alice and Bob exchange secrets simultaneously [529]. Rather than repeat the whole protocol, I'll sketch the modifications to the certified mail protocol.

Alice performs steps (1) through (4) using A as the message. Bob goes through similar steps using B as his message. Alice and Bob perform the oblivious transfer in step (9), decrypt the halves they can in step (10), and go through the iterations in steps (11) and (12). If they are concerned about Eve, they should encrypt their messages. Finally, both Alice and Bob decrypt the remaining halves of the message pairs and XOR any key pair to get the original message encryption key.

This protocol allows Alice and Bob to exchange secrets simultaneously, but says nothing about the quality of the secrets exchanged. Alice could promise Bob the solution to the Minotaur's labyrinth, but actually send him a map of Boston's subway system. Bob will get whatever secret Alice sends him. Other protocols are [1286,195,991,1524,705,753,259,358,415].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 6
Esoteric Protocols

6.1 Secure Elections

Computerized voting will never be used for general elections unless there is a protocol that both maintains individual privacy and prevents cheating. The ideal protocol has, at the very least, these six requirements:

1. Only authorized voters can vote.
2. No one can vote more than once.
3. No one can determine for whom anyone else voted.
4. No one can duplicate anyone else's vote. (This turns out to be the hardest requirement.)
5. No one can change anyone else's vote without being discovered.
6. Every voter can make sure that his vote has been taken into account in the final tabulation.

Additionally, some voting schemes may have the following requirement:

7. Everyone knows who voted and who didn't.

Before describing the complicated voting protocols with these characteristics, let's look at some simpler protocols.

Simplistic Voting Protocol #1

(1) Each voter encrypts his vote with the public key of a Central Tabulating Facility (CTF).
(2) Each voter sends his vote in to the CTF.
(3) The CTF decrypts the votes, tabulates them, and makes the results public.

This protocol is rife with problems. The CTF has no idea where the votes are from, so it doesn't even know if the votes are coming from eligible voters. It has no idea if eligible voters are voting more than once. On the plus side, no one can change anyone else's vote; but no one would bother trying to modify someone else's vote when it is far easier to vote repeatedly for the result of your choice.

Simplistic Voting Protocol #2

(1) Each voter signs his vote with his private key.
(2) Each voter encrypts his signed vote with the CTF's public key.
(3) Each voter sends his vote to a CTF.
(4) The CTF decrypts the votes, checks the signatures, tabulates the votes, and makes the results public.

This protocol satisfies properties one and two: Only authorized voters can vote and no one can vote more than once—the CTF would record votes received in step (3). Each vote is signed with the voter's private key, so the CTF knows who voted, who didn't, and how often each voter voted. If a vote comes in that isn't signed by an eligible voter, or if a second vote comes in signed by a voter who has already voted, the facility ignores it. No one can change anyone else's vote either, even if they intercept it in step (3), because of the digital signature.

The problem with this protocol is that the signature is attached to the vote; the CTF knows who voted for whom. Encrypting the votes with the CTF's public key prevents anyone from eavesdropping on the protocol and figuring out who voted for whom, but you have to trust the CTF completely. It's analogous to having an election judge staring over your shoulder in the voting booth.

These two examples show how difficult it is to achieve the first three requirements of a secure voting protocol, let alone the others.

Voting with Blind Signatures

We need to somehow dissociate the vote from the voter, while still maintaining authentication. The blind signature protocol does just that.

(1) Each voter generates 10 sets of messages, each set containing a valid vote for each possible outcome (e.g., if the vote is a yes or no question, each set contains two votes, one for "yes" and the other for "no"). Each message also contains a randomly generated identification number, large enough to avoid duplicates with other voters.
(2) Each voter individually blinds all of the messages (see Section 5.3) and sends them, with their blinding factors, to the CTF.
(3) The CTF checks its database to make sure the voter has not submitted his blinded votes for signature previously. It opens nine of the sets to check that they are properly formed. Then it individually signs each message in the set. It sends them back to the voter, storing the name of the voter in its database.
(4) The voter unblinds the messages and is left with a set of votes signed by the CTF. (These votes are signed but unencrypted, so the voter can easily see which vote is "yes" and which is "no.")
(5) The voter chooses one of the votes (ah, democracy) and encrypts it with the CTF's public key.
(6) The voter sends his vote in.
(7) The CTF decrypts the votes, checks the signatures, checks its database for a duplicate identification number, saves the serial number, and tabulates the votes. It publishes the results of the election, along with every serial number and its associated vote.

A malicious voter, call him Mallory, cannot cheat this system. The blind signature protocol ensures that his votes are unique. If he tries to send in the same vote twice, the CTF will notice the duplicate serial number in step (7) and throw out the second vote. If he tries to get multiple votes signed in step (2), the CTF will discover this in step (3). Mallory cannot generate his own votes because he doesn't know the facility's private key. He can't intercept and change other people's votes for the same reason.

The cut-and-choose protocol in step (3) is to ensure that the votes are unique. Without that step, Mallory could create a set of votes that are the same except for the identification number, and have them all validated.

A malicious CTF cannot figure out how individuals voted. Because the blind signature protocol prevents the facility from seeing the serial numbers on the votes before they are cast, the CTF cannot link the blinded vote it signed with the vote eventually cast. Publishing a list of serial numbers and their associated votes allows voters to confirm that their vote was tabulated correctly.

There are still problems. If step (6) is not anonymous and the CTF can record who sent in which vote, then it can figure out who voted for whom. However, if it receives votes in a locked ballot box and then tabulates them later, it cannot. Also, while the CTF may not be able to link votes to individuals, it can generate a large number of signed, valid votes and cheat by submitting those itself. And if Alice discovers that the CTF changed her vote, she has no way to prove it. A similar protocol, which tries to correct these problems, is [1195, 1370].

Voting with Two Central Facilities

One solution is to divide the CTF in two. Neither party would have the power to cheat on its own.

The following protocol uses a Central Legitimization Agency (CLA) to certify voters and a separate CTF to count votes [1373].

(1) Each voter sends a message to the CLA asking for a validation number.
(2) The CLA sends the voter back a random validation number. The CLA maintains a list of validation numbers. The CLA also keeps a list of the validation numbers' recipients, in case someone tries to vote twice.
(3) The CLA sends the list of validation numbers to the CTF.
(4) Each voter chooses a random identification number. He creates a message with that number, the validation number he received from the CLA, and his vote. He sends this message to the CTF.
(5) The CTF checks the validation number against the list it received from the CLA in step (3). If the validation number is there, the CTF crosses it off (to prevent someone from voting twice). The CTF adds the identification number to the list of people who voted for a particular candidate and adds one to the tally.
(6) After all votes have been received, the CTF publishes the outcome, as well as the lists of identification numbers and for whom their owners voted.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Like the previous protocol, each voter can look at the lists of identification numbers and find his own. This gives him proof that his vote was counted. Of course, all messages passing among the parties in the protocol should be encrypted and signed to prevent someone from impersonating someone else or intercepting transmissions.

The CTF cannot modify votes because each voter will look for his identification string. If a voter doesn't find his identification string, or finds his identification string in a tally other than the one he voted for, he will immediately know there was foul play. The CTF cannot stuff the ballot box because it is being watched by the CLA. The CLA knows how many voters have been certified and their validation numbers, and will detect any modifications.

Mallory, who is not an eligible voter, can try to cheat by guessing a valid validation number. This threat can be minimized by making the number of possible validation numbers much larger than the number of actual validation numbers: 100-digit numbers for a million voters, for example. Of course, the validation numbers must be generated randomly.

Despite this, the CLA is still a trusted authority in some respects. It can certify ineligible voters. It can certify eligible voters multiple times. This risk could be minimized by having the CLA publish a list of certified voters (but not their validation numbers). If the number of voters on this list is less than the number of votes tabulated, then something is awry. However, if more voters were certified than votes tabulated, it probably means that some certified people didn't bother voting. Many people who are registered to vote don't bother to cast ballots.

This protocol is vulnerable to collusion between the CLA and the CTF. If the two of them got together, they could correlate databases and figure out who voted for whom.

Voting with a Single Central Facility

A more complex protocol can be used to overcome the danger of collusion between the CLA and the CTF [1373]. This protocol is identical to the previous one, with two modifications:

— The CLA and the CTF are one organization, and
— ANDOS (see Section 4.13) is used to anonymously distribute validation numbers in step (2).

Since the anonymous key distribution protocol prevents the CTF from knowing which voter got which validation number, there is no way for the CTF to correlate validation numbers with votes received. The CTF still has to be trusted not to give validation numbers to ineligible voters, though. You can also solve this problem with blind signatures.

Improved Voting with a Single Central Facility

This protocol also uses ANDOS [1175]. It satisfies all six requirements of a good voting protocol. It doesn't satisfy the seventh requirement, but has two properties additional to the six listed at the beginning of the section:

7. A voter can change his mind (i.e., retract his vote and vote again) within a given period of time.
8. If a voter finds out that his vote is miscounted, he can identify and correct the problem without jeopardizing the secrecy of his ballot.

Here's the protocol:

(1) The CTF publishes a list of all legitimate voters.
(2) Within a specified deadline, each voter tells the CTF whether he intends to vote.
(3) The CTF publishes a list of voters participating in the election.
(4) Each voter receives an identification number, I, using an ANDOS protocol.
(5) Each voter generates a public-key/private-key key pair: k, d. If v is the vote, he generates the following message and sends it to the CTF:
I,Ek(I,v)

This message must be sent anonymously.
(6) The CTF acknowledges receipt of the vote by publishing:
Ek(I,v)
(7) Each voter sends the CTF:
I,d
(8) The CTF decrypts the votes. At the end of the election, it publishes the results of the election and, for each different vote, the list of all Ek(I,v) values that contained that vote.
(9) If a voter observes that his vote is not properly counted, he protests by sending the CTF:
I,Ek(I,v),d
(10) If a voter wants to change his vote (possible, in some elections) from v to v', he sends the CTF:
I,Ek(I,v'),d

A different voting protocol uses blind signatures instead of ANDOS, but is essentially the same [585]. Steps (1) through (3) are preliminary to the actual voting. Their purpose is to find out and publicize the total number of actual voters. Although some of them probably will not participate, it reduces the ability of the CTF to add fraudulent votes.

In step (4), it is possible for two voters to get the same identification number. This possibility can be minimized by having far more possible identification numbers than actual voters. If two voters submit votes with the same identification tag, the CTF generates a new identification number, I', chooses one of the two votes, and publishes:

I',Ek(I,v)

The owner of that vote recognizes it and sends in a second vote, by repeating step (5), with the new identification number.

Step (6) gives each voter the capability to check that the CTF received his vote accurately. If his vote is miscounted, he can prove his case in step (9). Assuming a voter's vote is correct in step (6), the message he sends in step (9) constitutes a proof that his vote is miscounted.

One problem with the protocol is that a corrupt CTF could allocate the votes of people who respond in step (2) but who do not actually vote. Another problem is the complexity of the ANDOS protocol. The authors recommend dividing a large population of voters into smaller populations, such as election districts.

Another, more serious problem is that the CTF can neglect to count a vote. This problem cannot be resolved: Alice claims that the CTF intentionally neglected to count her vote, but the CTF claims that the voter never voted.

Voting without a Central Tabulating Facility

The following protocol does away with the CTF entirely; the voters watch each other. Designed by Michael Merritt [452, 1076, 453], it is so unwieldy that it cannot be implemented practically for more than a handful of people, but it is useful to learn from nevertheless.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Alice, Bob, Carol, and Dave are voting yes or no (0 or 1) on a particular issue. Assume each voter has a public and private key. Also assume that everyone knows everyone else's public keys.

(1) Each voter chooses his vote and does the following:
(a) He attaches a random string to his vote.
(b) He encrypts the result of step (a) with Dave's public key.
(c) He encrypts the result of step (b) with Carol's public key.
(d) He encrypts the result of step (c) with Bob's public key.
(e) He encrypts the result of step (d) with Alice's public key.
(f) He attaches a new random string to the result of step (e) and encrypts it with Dave's public key. He records the value of the random string.
(g) He attaches a new random string to the result of step (f) and encrypts it with Carol's public key. He records the value of the random string.
(h) He attaches a new random string to the result of step (g) and encrypts it with Bob's public key. He records the value of the random string.
(i) He attaches a new random string to the result of step (h) and encrypts it with Alice's public key. He records the value of the random string.
If E is the encryption function, R i is a random string, and V is the vote, his message looks like:
EA(R5,EB(R4,EC(R3,ED(R2,EA(E B(EC(ED(V,R1))))))))

Each voter saves the intermediate results at each point in the calculation. These results will be used later in the protocol to confirm that his vote is among those being counted.
(2) Each voter sends his message to Alice.
(3) Alice decrypts all of the votes with her private key and then removes all of the random strings at that level.
(4) Alice scrambles the order of all the votes and sends the result to Bob.
Each vote now looks like this:
EB(R4,EC(R3,ED(R2,EA(EB(EC(ED(V,R1)))))))
(5) Bob decrypts all of the votes with his private key, checks to see that his vote is among the set of votes, removes all the random strings at that level, scrambles all the votes, and then sends the result to Carol.
Each vote now looks like this:
EC(R3,ED (R2,EA(EB(EC(ED(V,R1))))))
(6) Carol decrypts all of the votes with her private key, checks to see that her vote is among the set of votes, removes all the random strings at that level, scrambles all the votes, and then sends the result to Dave.
Each vote now looks like this:
ED(R2,EA(EB(EC(ED(V,R1)))))
(7) Dave decrypts all of the votes with his private key, checks to see that his vote is among the set of votes, removes all the random strings at that level, scrambles all the votes, and sends them to Alice.
Each vote now looks like this:
EA(EB(EC(ED(V,R1))))
(8) Alice decrypts all the votes with her private key, checks to see that her vote is among the set of votes, signs all the votes, and then sends the result to Bob, Carol, and Dave.
Each vote now looks like this:
SA(EB(EC(ED(V,R1))))
(9) Bob verifies and deletes Alice's signatures. He decrypts all the votes with his private key, checks to see that his vote is among the set of votes, signs all the votes, and then sends the result to Alice, Carol, and Dave.
Each vote now looks like this:
SB(EC(ED(V,R1)))
(10) Carol verifies and deletes Bob's signatures. She decrypts all the votes with her private key, checks to see that her vote is among the set of votes, signs all the votes, and then sends the result to Alice, Bob, and Dave.
Each vote now looks like this:
SC(ED(V,R1))
(11) Dave verifies and deletes Carol's signatures. He decrypts all the votes with his private key, checks to see that his vote is among the set of votes, signs all the votes, and then sends the result to Alice, Bob, and Carol.
Each vote now looks like this:
SD(V,R1)
(12) All verify and delete Dave's signature. They check to make sure that their vote is among the set of votes (by looking for their random string among the votes).
(13) Everyone removes the random strings from each vote and tallies the votes.

Not only does this protocol work, it is also self-adjudicating. Alice, Bob, Carol, and Dave will immediately know if someone tries to cheat. No CTF or CLA is required. To see how this works, let's try to cheat.

If someone tries to stuff the ballot, Alice will detect the attempt in step (3) when she receives more votes than people. If Alice tries to stuff the ballot, Bob will notice in step (4).

More devious is to substitute one vote for another. Since the votes are encrypted with various public keys, anyone can create as many valid votes as needed. The decryption protocol has two rounds: round one consists of steps (3) through (7), and round two consists of steps (8) through (11). Vote substitution is detected differently in the different rounds.

If someone substitutes one vote for another in round two, his actions are discovered immediately. At every step the votes are signed and sent to all the voters. If one (or more) of the voters noticed that his vote is no longer in the set of votes, he immediately stops the protocol. Because the votes are signed at every step, and because everyone can backtrack through the second round of the protocol, it is easy to detect who substituted the votes.

Substituting one vote for another during round one of the protocol is more subtle. Alice can't do it in step (3), because Bob, Carol, or Dave will detect it in step (5), (6), or (7). Bob could try in step (5). If he replaces Carol's or Dave's vote (remember, he doesn't know which vote corresponds to which voter), Carol or Dave will notice in step (6) or (7). They wouldn't know who tampered with their vote (although it would have had to be someone who had already handled the votes), but they would know that their vote was tampered with. If Bob is lucky and picks Alice's vote to replace, she won't notice until the second round. Then, she will notice her vote missing in step (8). Still, she would not know who tampered with her vote. In the first round, the votes are shuffled from one step to the other and unsigned; it is impossible for anyone to backtrack through the protocol to determine who tampered with the votes.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Another form of cheating is to try to figure out who voted for whom. Because of the scrambling in the first round, it is impossible for someone to backtrack through the protocol and link votes with voters. The removal of the random strings during the first round is also crucial to preserving anonymity. If they are not removed, the scrambling of the votes could be reversed by re-encrypting the emerging votes with the scrambler's public key. As the protocol stands, the confidentiality of the votes is secure.

Even more strongly, because of the initial random string, R1, even identical votes are encrypted differently at every step of the protocol. No one knows the outcome of the vote until step (11).

What are the problems with this protocol? First, the protocol has an enormous amount of computation. The example described had only four voters and it was complicated. This would never work in a real election, with tens of thousands of voters. Second, Dave learns the results of the election before anyone else does. While he still can't affect the outcome, this gives him some power that the others do not have. On the other hand, this is also true with centralized voting schemes.

The third problem is that Alice can copy anyone else's vote, even though she does not know what it is beforehand. To see why this could be a problem, consider a three-person election between Alice, Bob, and Eve. Eve doesn't care about the result of the election, but she wants to know how Alice voted. So she copies Alice's vote, and the result of the election is guaranteed to be equal to Alice's vote.

Other Voting Schemes

Many complex secure election protocols have been proposed. They come in two basic flavors. There are mixing protocols, like "Voting without a Central Tabulating Facility, " where everyone's vote gets mixed up so that no one can associate a vote with a voter.

There are also divided protocols, where individual votes are divided up among different tabulating facilities such that no single one of them can cheat the voters [360, 359, 118, 115]. These protocols only protect the privacy of voters to the extent that different "parts" of the government (or whoever is administering the voting) do not conspire against the voter. (This idea of breaking a central authority into different parts, who are only trusted when together, comes from [316].)

One divided protocol is [1371]. The basic idea is that each voter breaks his vote into several shares. For example, if the vote were "yes" or "no, " a 1 could indicate "yes" and a 0 could indicate "no"; the voter would then generate several numbers whose sum was either 0 or 1. These shares are sent to tabulating facilities, one to each, and are also encrypted and posted. Each center tallies the shares it receives (there are protocols to verify that the tally is correct) and the final vote is the sum of all the tallies. There are also protocols to ensure that each voter's shares add up to 0 or 1.

Another protocol, by David Chaum [322], ensures that voters who attempt to disrupt the election can be traced. However, the election must then be restarted without the interfering voter; this approach is not practical for large-scale elections.

Another, more complex, voting protocol that solves some of these problems can be found in [770, 771]. There is even a voting protocol that uses multiple-key ciphers [219]. Yet another voting protocol, which claims to be practical for large-scale elections, is in [585]. And [347] allows voters to abstain.

Voting protocols work, but they make it easier to buy and sell votes. The incentives become considerably stronger as the buyer can be sure that the seller votes as promised. Some protocols are designed to be receipt-free, so that it is impossible for a voter to prove to someone else that he voted in a certain way [117, 1170, 1372].

6.2 Secure Multiparty Computation

Secure multiparty computation is a protocol in which a group of people can get together and compute any function of many variables in a special way. Each participant in the group provides one or more variables. The result of the function is known to everyone in the group, but no one learns anything about the inputs of any other members other than what is obvious from the output of the function. Here are some examples:

Protocol #1

How can a group of people calculate their average salary without anyone learning the salary of anyone else?

(1) Alice adds a secret random number to her salary, encrypts the result with Bob's public key, and sends it to Bob.
(2) Bob decrypts Alice's result with his private key. He adds his salary to what he received from Alice, encrypts the result with Carol's public key, and sends it to Carol.
(3) Carol decrypts Bob's result with her private key. She adds her salary to what she received from Bob, encrypts the result with Dave's public key, and sends it to Dave.
(4) Dave decrypts Carol's result with his private key. He adds his salary to what he received from Carol, encrypts the result with Alice's public key, and sends it to Alice.
(5) Alice decrypts Dave's result with her private key. She subtracts the random number from step (1) to recover the sum of everyone's salaries.
(6) Alice divides the result by the number of people (four, in this case) and announces the result.

This protocol assumes that everyone is honest; they may be curious, but they follow the protocol. If any participant lies about his salary, the average will be wrong. A more serious problem is that Alice can misrepresent the result to everyone. She can subtract any number she likes in step (5), and no one would be the wiser. Alice could be prevented from doing this by requiring her to commit to her random number using any of the bit-commitment schemes from Section 4.9, but when she revealed her random number at the end of the protocol Bob could learn her salary.

Protocol #2

Alice and Bob are at a restaurant together, having an argument over who is older. They don't, however, want to tell the other their age. They could each whisper their age into the ear of a trusted neutral party (the waiter, for example), who could compare the numbers in his head and announce the result to both Alice and Bob.

The above protocol has two problems. One, your average waiter doesn't have the computational ability to handle situations more complex than determining which of two numbers is greater. And two, if Alice and Bob were really concerned about the secrecy of their information, they would be forced to drown the waiter in a bowl of vichyssoise, lest he tell the wine steward.

Public-key cryptography offers a far less violent solution. There is a protocol by which Alice, who knows a value a, and Bob, who knows a value b, can together determine if a < b, so that Alice gets no additional information about b and Bob gets no additional information about a. And, both Alice and Bob are convinced of the validity of the computation. Since the cryptographic algorithm used is an essential part of the protocol, details can be found in Section 23.14.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Of course, this protocol doesn't protect against active cheaters. There's nothing to stop Alice (or Bob, for that matter) from lying about her age. If Bob were a computer program that blindly executed the protocol, Alice could learn his age (is the age of a computer program the length of time since it was written or the length of time since it started running?) by repeatedly executing the protocol. Alice might give her age as 60. After learning that she is older, she could execute the protocol again with her age as 30. After learning that Bob is older, she could execute the protocol again with her age as 45, and so on, until Alice discovers Bob's age to any degree of accuracy she wishes.

Assuming that the participants don't actively cheat, it is easy to extend this protocol to multiple participants. Any number of people can find out the order of their ages by a sequence of honest applications of the protocol; and no participant can learn the age of another.

Protocol #3

Alice likes to do kinky things with teddy bears. Bob has erotic fantasies about marble tables. Both are pretty embarrassed by their particular fetish, but would love to find a mate who shared in their...um...lifestyle.

Here at the Secure Multiparty Computation Dating Service, we've designed a protocol for people like them. We've numbered an astonishing list of fetishes, from "aardvarks" to "zoot suits." Discreetly separated by a modem link, Alice and Bob can participate in a secure multiparty protocol. Together, they can determine whether they share the same fetish. If they do, they might look forward to a lifetime of bliss together. If they don't, they can part company secure in the knowledge that their particular fetish remains confidential. No one, not even the Secure Multiparty Computation Dating Service, will ever know.

Here's how it works:

(1) Using a one-way function, Alice hashes her fetish into a seven-digit string.
(2) Alice uses the seven-digit string as a telephone number, calls the number, and leaves a message for Bob. If no one answers or the number is not in service, Alice applies a one-way function to the telephone number until she finds someone who can play along with the protocol.
(3) Alice tells Bob how many times she had to apply the one-way hash function to her fetish.
(4) Bob hashes his fetish the same number of times that Alice did. He also uses the seven-digit string as a telephone number, and asks the person at the other end whether there were any messages for him.

Note that Bob has a chosen-plaintext attack. He can hash common fetishes and call the resulting telephone numbers, looking for messages for him. This protocol only really works if there are enough possible plaintext messages for this to be impractical.

There's also a mathematical protocol, one similar to Protocol #2. Alice knows a, Bob knows b, and together they will determine whether a = b, such that Bob does not learn anything additional about a and Alice does not learn anything additional about b. Details are in Section 23.14.

Protocol #4

This is another problem for secure multiparty computation [1373]: A council of seven meets regularly to cast secret ballots on certain issues. (All right, they rule the world—don't tell anyone I told you.) All council members can vote yes or no. In addition, two parties have the option of casting "super votes": S-yes and S-no. They do not have to cast super votes; they can cast regular votes if they prefer. If no one casts any super votes, then the majority of votes decides the issue. In the case of a single or two equivalent super votes, all regular votes are ignored. In the case of two contradicting super votes, the majority of regular votes decides. We want a protocol that securely performs this style of voting.

Two examples should illustrate the voting process. Assume there are five regular voters, N1 through N5, and two super voters: S1 and S2. Here's the vote on issue #1:

S1        S2      N1    N2    N3    N4    N5
S-yes     no     no    no    no    yes    yes

In this instance the only vote that matters is S1 's, and the result is "yes."

Here is the vote on issue #2:

S1       S2     N1   N2   N3   N4   N5
S-yes    S-no   no   no   no   yes  yes

Here the two super votes cancel and the majority of regular "no" votes decide the issue.

If it isn't important to hide the knowledge of whether the super vote or the regular vote was the deciding vote, this is an easy application of a secure voting protocol. Hiding that knowledge requires a more complicated secure multiparty computation protocol.

This kind of voting could occur in real life. It could be part of a corporation's organizational structure, where certain people have more power than others, or it could be part of the United Nations's procedures, where certain nations have more power than others.

Multiparty Unconditionally Secure Protocols

This is just a simple case of a general theorem: Any function of n inputs can be computed by a set of n players in a way that will let all learn the value of the function, but any set of less than n/2 players will not get any additional information that does not follow from their own inputs and the value of the output information. For details, see [136, 334, 1288, 621].

Secure Circuit Evaluation

Alice has her input, a. Bob has his input, b. Together they wish to compute some general function, f (a,b), such that Alice learns nothing about Bob's input and Bob learns nothing about Alice's input. The general problem of secure multiparty computation is also called secure circuit evaluation. Here, Alice and Bob can create an arbitrary Boolean circuit. This circuit accepts inputs from Alice and from Bob and produces an output. Secure circuit evaluation is a protocol that accomplishes three things:

1. Alice can enter her input without Bob's being able to learn it.
2. Bob can enter his input without Alice's being able to learn it.
3. Both Alice and Bob can calculate the output, with both parties being sure the output is correct and that neither party has tampered with it.

Details on secure circuit evaluation can be found in [831].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

6.3 Anonymous Message Broadcast

You can't go out to dinner with a bunch of cryptographers without raising a ruckus. In [321], David Chaum introduced the Dining Cryptographers Problem:

Three cryptographers are sitting down to dinner at their favorite three-star restaurant. Their waiter informs them that arrangements have been made with the maître d'hôtel for the bill to be paid anonymously. One of the cryptographers might be paying for the dinner, or it might have been the NSA. The three cryptographers respect each other's right to make an anonymous payment, but they wonder if the NSA is paying.

How do the cryptographers, named Alice, Bob, and Carol, determine if one of them is paying for dinner, while at the same time preserving the anonymity of the payer?

Chaum goes on to solve the problem:

Each cryptographer flips an unbiased coin behind his menu, between him and the cryptographer to his right, so that only the two of them can see the outcome. Each cryptographer then states aloud whether the two coins he can see—the one he flipped and the one his left-hand neighbor flipped—fell on the same side or on different sides. If one of the cryptographers is the payer, he states the opposite of what he sees. An odd number of differences uttered at the table indicates that a cryptographer is paying; an even number of differences indicates that NSA is paying (assuming that the dinner was paid for only once). Yet, if a cryptographer is paying, neither of the other two learns anything from the utterances about which cryptographer it is.

To see that this works, imagine Alice trying to figure out which other cryptographer paid for dinner (assuming that neither she nor the NSA paid). If she sees two different coins, then either both of the other cryptographers, Bob and Carol, said, "same" or both said, "different." (Remember, an odd number of cryptographers saying "different" indicates that one of them paid.) If both said, "different, " then the payer is the cryptographer closest to the coin that is the same as the hidden coin (the one that Bob and Carol flipped). If both said, "same, " then the payer is the cryptographer closest to the coin that is different from the hidden coin. However, if Alice sees two coins that are the same, then either Bob said, "same" and Carol said, "different, " or Bob said, "different" and Carol said, "same." If the hidden coin is the same as the two coins she sees, then the cryptographer who said, "different" is the payer. If the hidden coin is different from the two coins she sees, then the cryptographer who said, "same" is the payer. In all of these cases, Alice needs to know the result of the coin flipped between Bob and Carol to determine which of them paid.

This protocol can be generalized to any number of cryptographers; they all sit in a ring and flip coins among them. Even two cryptographers can perform the protocol. Of course, they know who paid, but someone watching the protocol could tell only if one of the two paid or if the NSA paid; they could not tell which cryptographer paid.

The applications of this protocol go far beyond sitting around the dinner table. This is an example of unconditional sender and recipient untraceability. A group of users on a network can use this protocol to send anonymous messages.

(1) The users arrange themselves into a circle.
(2) At regular intervals, adjacent pairs of users flip coins between them, using some fair coin flip protocol secure from eavesdroppers.
(3) After every flip, each user announces either "same" or "different."

If Alice wishes to broadcast a message, she simply starts inverting her statement in those rounds corresponding to a 1 in the binary representation of her message. For example, if her message were "1001, " she would invert her statement, tell the truth, tell the truth, and then invert her statement. Assuming the result of her flips were "different, " "same, " "same, " "same, " she would say "same, " "same, " "same, " "different."

If Alice notices that the overall outcome of the protocol doesn't match the message she is trying to send, she knows that someone else is trying to send a message at the same time. She then stops sending the message and waits some random number of rounds before trying again. The exact parameters have to be worked out based on the amount of message traffic on this network, but the idea should be clear.

To make things even more interesting, these messages can be encrypted in another user's public keys. Then, when everyone receives the message (a real implementation of this should add some kind of standard message-beginning and message-ending strings), only the intended recipient can decrypt and read it. No one else knows who sent it. No one else knows who could read it. Traffic analysis, which traces and compiles patterns of people's communications even though the messages themselves may be encrypted, is useless.

An alternative to flipping coins between adjacent parties would be for them to keep a common file of random bits. Maybe they could keep them on a CD-ROM, or one member of the pair could generate a pile of them and send them to the other party (encrypted, of course). Alternatively, they could agree on a cryptographically secure pseudo-random-number generator between them, and they could each generate the same string of pseudo-random bits for the protocol.

One problem with this protocol is that while a malicious participant cannot read any messages, he can disrupt the system unobserved by lying in step (3). There is a modification to the previous protocol that detects disruption [1578, 1242]; the problem is called "The Dining Cryptographers in the Disco."

6.4 Digital Cash

Cash is a problem. It's annoying to carry, it spreads germs, and people can steal it from you. Checks and credit cards have reduced the amount of physical cash flowing through society, but the complete elimination of cash is virtually impossible. It'll never happen; drug dealers and politicians would never stand for it. Checks and credit cards have an audit trail; you can't hide to whom you gave money.

On the other hand, checks and credit cards allow people to invade your privacy to a degree never before imagined. You might never stand for the police following you your entire life, but the police can watch your financial transactions. They can see where you buy your gas, where you buy your food, who you call on the telephone—all without leaving their computer terminals. People need a way to protect their anonymity in order to protect their privacy.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Happily, there is a complicated protocol that allows for authenticated but untraceable messages. Lobbyist Alice can transfer digital cash to Congresscritter Bob so that newspaper reporter Eve does not know Alice's identity. Bob can then deposit that electronic money into his bank account, even though the bank has no idea who Alice is. But if Alice tries to buy cocaine with the same piece of digital cash she used to bribe Bob, she will be detected by the bank. And if Bob tries to deposit the same piece of digital cash into two different accounts, he will be detected—but Alice will remain anonymous. Sometimes this is called anonymous digital cash to differentiate it from digital money with an audit trail, such as credit cards.

A great social need exists for this kind of thing. With the growing use of the Internet for commercial transactions, there is more call for network-based privacy and anonymity in business. (There are good reasons people are reluctant to send their credit card numbers over the Internet.) On the other hand, banks and governments seem unwilling to give up the control that the current banking system's audit trail provides. They'll have to, though. All it will take for digital cash to catch on is for some trustworthy institution to be willing to convert the digits to real money.

Digital cash protocols are very complex. We'll build up to one, a step at a time. For more formal details, read [318, 339, 325, 335, 340]. Realize that this is just one digital cash protocol; there are others.

Protocol #1

The first few protocols are physical analogies of cryptographic protocols. This first protocol is a simplified physical protocol for anonymous money orders:

(1) Alice prepares 100 anonymous money orders for $1000 each.
(2) Alice puts one each, and a piece of carbon paper, into 100 different envelopes. She gives them all to the bank.
(3) The bank opens 99 envelopes and confirms that each is a money order for $1000.
(4) The bank signs the one remaining unopened envelope. The signature goes through the carbon paper to the money order. The bank hands the unopened envelope back to Alice, and deducts $1000 from her account.
(5) Alice opens the envelope and spends the money order with a merchant.
(6) The merchant checks for the bank's signature to make sure the money order is legitimate.
(7) The merchant takes the money order to the bank.
(8) The bank verifies its signature and credits $1000 to the merchant's account.

This protocol works. The bank never sees the money order it signed, so when the merchant brings it to the bank, the bank has no idea that it was Alice's. The bank is convinced that it is valid, though, because of the signature. The bank is confident that the unopened money order is for $1000 (and not for $100, 000 or $100, 000, 000) because of the cut-and-choose protocol (see Section 5.1). It verifies the other 99 envelopes, so Alice has only a 1 percent chance of cheating the bank. Of course, the bank will make the penalty for cheating great enough so that it isn't worth that chance. If the bank refuses to sign the last check (if Alice is caught cheating) without penalizing Alice, she will continue to try until she gets lucky. Prison terms are a better deterrent.

Protocol #2

The previous protocol prevents Alice from writing a money order for more than she claims to, but it doesn't prevent Alice from photocopying the money order and spending it twice. This is called the double spending problem; to solve it, we need a complication:

(1) Alice prepares 100 anonymous money orders for $1000 each. On each money order she includes a different random uniqueness string, one long enough to make the chance of another person also using it negligible.
(2) Alice puts one each, and a piece of carbon paper, into 100 different envelopes. She gives them all to the bank.
(3) The bank opens 99 envelopes and confirms that each is a money order for $1000.
(4) The bank signs the one remaining unopened envelope. The signature goes through the carbon paper to the money order. The bank hands the unopened envelope back to Alice and deducts $1000 from her account.
(5) Alice opens the envelope and spends the money order with a merchant.
(6) The merchant checks for the bank's signature to make sure the money order is legitimate.
(7) The merchant takes the money order to the bank.
(8) The bank verifies its signature and checks its database to make sure a money order with the same uniqueness string has not been previously deposited. If it hasn't, the bank credits $1000 to the merchant's account. The bank records the uniqueness string in a database.
(9) If it has been previously deposited, the bank doesn't accept the money order.

Now, if Alice tries to spend a photocopy of the money order, or if the merchant tries to deposit a photocopy of the money order, the bank will know about it.

Protocol #3

The previous protocol protects the bank from cheaters, but it doesn't identify them. The bank doesn't know if the person who bought the money order (the bank has no idea it's Alice) tried to cheat the merchant or if the merchant tried to cheat the bank. This protocol corrects that:

(1) Alice prepares 100 anonymous money orders for $1000 each. On each of the money orders she includes a different random uniqueness string, one long enough to make the chance of another person also using it negligible.
(2) Alice puts one each, and a piece of carbon paper, into 100 different envelopes. She gives them all to the bank.
(3) The bank opens 99 envelopes and confirms that each is a money order for $1000 and that all the random strings are different.
(4) The bank signs the one remaining unopened envelope. The signature goes through the carbon paper to the money order. The bank hands the unopened envelope back to Alice and deducts $1000 from her account.
(5) Alice opens the envelope and spends the money order with a merchant.
(6) The merchant checks for the bank's signature to make sure the money order is legitimate.
(7) The merchant asks Alice to write a random identity string on the money order.
(8) Alice complies.
(9) The merchant takes the money order to the bank.
(10) The bank verifies the signature and checks its database to make sure a money order with the same uniqueness string has not been previously deposited. If it hasn't, the bank credits $1000 to the merchant's account. The bank records the uniqueness string and the identity string in a database.
(11) If the uniqueness string is in the database, the bank refuses to accept the money order. Then, it compares the identity string on the money order with the one stored in the database. If it is the same, the bank knows that the merchant photocopied the money order. If it is different, the bank knows that the person who bought the money order photocopied it.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This protocol assumes that the merchant cannot change the identity string once Alice writes it on the money order. The money order might have a series of little squares, which the merchant would require Alice to fill in with either Xs or Os. The money order might be made out of paper that tears if erased.

Since the interaction between the merchant and the bank takes place after Alice spends the money, the merchant could be stuck with a bad money order. Practical implementations of this protocol might require Alice to wait near the cash register during the merchant-bank interaction, much the same way as credit-card purchases are handled today.

Alice could also frame the merchant. She could spend a copy of the money order a second time, giving the same identity string in step (7). Unless the merchant keeps a database of money orders it already received, he would be fooled. The next protocol eliminates that problem.

Protocol #4

If it turns out that the person who bought the money order tried to cheat the merchant, the bank would want to know who that person was. To do that requires moving away from a physical analogy and into the world of cryptography.

The technique of secret splitting can be used to hide Alice's name in the digital money order.

(1) Alice prepares n anonymous money orders for a given amount.
Each of the money orders contains a different random uniqueness string, X, one long enough to make the chance of two being identical negligible.
On each money order, there are also n pairs of identity bit strings, I1, I2,..., In. (Yes, that's n different pairs on each check.) Each of these pairs is generated as follows: Alice creates a string that gives her name, address, and any other piece of identifying information that the bank wants to see. Then, she splits it into two pieces using the secret splitting protocol (see Section 3.6). Then, she commits to each piece using a bit-commitment protocol.
For example, I37 consists of two parts: I37L and I37R. Each part is a bit-committed packet that Alice can be asked to open and whose proper opening can be instantly verified. Any pair (e.g., I37L and I37R, but not I37L and I38R), reveals Alice's identity.
Each of the money orders looks like this:
    Amount
    Uniqueness String: X
     Identity Strings:  I1 = (I1L,I1R)
                        I2 =       (I2L,I2R)
                         ....
                        In = (InL, InR)
(2) Alice blinds all n money orders, using a blind signature protocol. She gives them all to the bank.
(3) The bank asks Alice to unblind n - 1 of the money orders at random and confirms that they are all well formed. The bank checks the amount, the uniqueness string, and asks Alice to reveal all of the identity strings.
(4) If the bank is satisfied that Alice did not make any attempts to cheat, it signs the one remaining blinded money order. The bank hands the blinded money order back to Alice and deducts the amount from her account.
(5) Alice unblinds the money order and spends it with a merchant.
(6) The merchant verifies the bank's signature to make sure the money order is legitimate.
(7) The merchant asks Alice to randomly reveal either the left half or the right half of each identity string on the money order. In effect, the merchant gives Alice a random n- bit selector string, b1, b2,..., bn. Alice opens either the left or right half of Ii, depending on whether bi is a 0 or a 1.
(8) Alice complies.
(9) The merchant takes the money order to the bank.
(10) The bank verifies the signature and checks its database to make sure a money order with the same uniqueness string has not been previously deposited. If it hasn't, the bank credits the amount to the merchant's account. The bank records the uniqueness string and all of the identity information in a database.
(11) If the uniqueness string is in the database, the bank refuses to accept the money order. Then, it compares the identity string on the money order with the one stored in the database. If it is the same, the bank knows that the merchant copied the money order. If it is different, the bank knows that the person who bought the money order photocopied it. Since the second merchant who accepted the money order handed Alice a different selector string than did the first merchant, the bank finds a bit position where one merchant had Alice open the left half and the other merchant had Alice open the right half. The bank XORs the two halves together to reveal Alice's identity.

This is quite an amazing protocol, so let's look at it from various angles.

Can Alice cheat? Her digital money order is nothing more than a string of bits, so she can copy it. Spending it the first time won't be a problem; she'll just complete the protocol and everything will go smoothly. The merchant will give her a random n-bit selector string in step (7) and Alice will open either the left half or right half of each Ii in step (8). In step (10), the bank will record all of this data, as well as the money order's uniqueness string.

When she tries to use the same digital money order a second time, the merchant (either the same merchant or a different merchant) will give her a different random selector string in step (7). Alice must comply in step (8); not doing so will immediately alert the merchant that something is suspicious. Now, when the merchant brings the money order to the bank in step (10), the bank would immediately notice that a money order with the same uniqueness string was already deposited. The bank then compares the opened halves of the identity strings. The odds that the two random selector strings are the same is 1 in 2n; it isn't likely to happen before the next ice age. Now, the bank finds a pair with one half opened the first time and the other half opened the second time. It XORs the two halves together, and out pops Alice's name. The bank knows who tried to spend the money order twice.

Note that this protocol doesn't keep Alice from trying to cheat; it detects her cheating with almost certainty. Alice can't prevent her identity from being revealed if she cheats. She can't change either the uniqueness string or any of the identity strings, because then the bank's signature will no longer be valid. The merchant will immediately notice that in step (6).

Alice could try to sneak a bad money order past the bank, one on which the identity strings don't reveal her name; or better yet, one whose identity strings reveal someone else's name. The odds of her getting this ruse past the bank in step (3) are 1 in n . These aren't impossible odds, but if you make the penalty severe enough, Alice won't try it. Or, you could increase the number of redundant money orders that Alice makes in step (1).

Can the merchant cheat? His chances are even worse. He can't deposit the money order twice; the bank will notice the repeated use of the selector string. He can't fake blaming Alice; only she can open any of the identity strings.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Even collusion between Alice and the merchant can't cheat the bank. As long as the bank signs the money order with the uniqueness string, the bank is assured of only having to make good on the money order once.

What about the bank? Can it figure out that the money order it accepted from the merchant was the one it signed for Alice? Alice is protected by the blind signature protocol in steps (2) through (5). The bank cannot make the connection, even if it keeps complete records of every transaction. Even more strongly, there is no way for the bank and the merchant to get together to figure out who Alice is. Alice can walk in the store and, completely anonymously, make her purchase.

Eve can cheat. If she can eavesdrop on the communication between Alice and the merchant, and if she can get to the bank before the merchant does, she can deposit the digital cash first. The bank will accept it and, even worse, when the merchant tries to deposit the cash he will be identified as a cheater. If Eve steals and spends Alice's cash before Alice can, then Alice will be identified as a cheater. There's no way to prevent this; it is a direct result of the anonynimity of the cash. Both Alice and the merchant have to protect their bits as they would paper money.

This protocol lies somewhere between an arbitrated protocol and a self-enforcing protocol. Both Alice and the merchant trust the bank to make good on the money orders, but Alice does not have to trust the bank with knowledge of her purchases.

Digital Cash and the Perfect Crime

Digital cash has its dark side, too. Sometimes people don't want so much privacy. Watch Alice commit the perfect crime [1575]:

(1) Alice kidnaps a baby.
(2) Alice prepares 10, 000 anonymous money orders for $1000 (or as many as she wants for whatever denomination she wants).
(3) Alice blinds all 10, 000 money orders, using a blind signature protocol. She sends them to the authorities with the threat to kill the baby unless the following instructions are met:
(a) Have a bank sign all 10, 000 money orders.
(b) Publish the results in a newspaper.
(4) The authorities comply.
(5) Alice buys a newspaper, unblinds the money orders, and starts spending them. There is no way for the authorities to trace the money orders to her.
(6) Alice frees the baby.

Note that this situation is much worse than any involving physical tokens—cash, for example. Without physical contact, the police have less opportunity to apprehend the kidnapper.

In general, though, digital cash isn't a good deal for criminals. The problem is that the anonymity only works one way: The spender is anonymous, but the merchant is not. Moreover, the merchant cannot hide the fact that he received money. Digital cash will make it easy for the government to determine how much money you made, but impossible to determine what you spent it on.

Practical Digital Cash

A Dutch company, DigiCash, owns most of the digital cash patents and has implemented digital cash protocols in working products. Anyone interested should contact DigiCash BV, Kruislaan 419, 1098 VA Amsterdam, Netherlands.

Other Digital Cash Protocols

There are other digital cash protocols; see [707, 1554, 734, 1633, 973]. Some of them involve some pretty complicated mathematics. Generally, the various digital cash protocols can be divided into various categories. On-line systems require the merchant to communicate with the bank at every sale, much like today's credit-card protocols. If there is a problem, the bank doesn't accept the cash and Alice cannot cheat.

Off-line systems, like Protocol #4, require no communication between the merchant and the bank until after the transaction between the merchant and the customer. These systems do not prevent Alice from cheating, but instead detect her cheating. Protocol #4 detected her cheating by making Alice's identity known if she tried to cheat. Alice knows that this will happen, so she doesn't cheat.

Another way is to create a special smart card (see Section 24.13) containing a tamperproof chip called an observer [332, 341, 387]. The observer chip keeps a mini database of all the pieces of digital cash spent by that smart card. If Alice attempts to copy some digital cash and spend it twice, the imbedded observer chip would detect the attempt and would not allow the transaction. Since the observer chip is tamperproof, Alice cannot erase the mini-database without permanently damaging the smart card. The cash can wend its way through the economy; when it is finally deposited, the bank can examine the cash and determine who, if anyone, cheated.

Digital cash protocols can also be divided along another line. Electronic coins have a fixed value; people using this system will need several coins in different denominations. Electronic checks can be used for any amount up to a maximum value and then returned for a refund of the unspent portion.

Two excellent and completely different off-line electronic coin protocols are [225, 226, 227] and [563, 564, 565]. A system called NetCash, with weaker anonymity properties, has also been proposed [1048, 1049]. Another new system is [289].

In [1211], Tatsuaki Okamoto and Kazuo Ohta list six properties of an ideal digital cash system:

1. Independence. The security of the digital cash is not dependent on any physical location. The cash can be transferred through computer networks.
2. Security. The digital cash cannot be copied and reused.
3. Privacy (Untraceability). The privacy of the user is protected; no one can trace the relationship between the user and his purchases.
4. Off-line Payment. When a user pays for a purchase with electronic cash, the protocol between the user and the merchant is executed off-line. That is, the shop does not need to be linked to a host to process the user's payment.
5. Transferability. The digital cash can be transferred to other users.
6. Divisibility. A piece of digital cash in a given amount can be subdivided into smaller pieces of cash in smaller amounts. (Of course, everything has to total up properly in the end.)

The protocols previously discussed satisfy properties 1, 2, 3, and 4, but not 5 and 6. Some on-line digital cash systems satisfy all properties except 4 [318, 413, 1243]. The first off-line digital cash system that satisfies properties 1, 2, 3, and 4, similar to the one just discussed, was proposed in [339]. Okamoto and Ohta proposed a system that satisfies properties 1 through 5 [1209]; they also proposed a system that satisfies properties 1 through 6 as well, but the data requirement for a single purchase is approximately 200 megabytes. Another off-line divisible coin system is described in [522].

The digital cash scheme proposed in [1211], by the same authors, satisfies properties 1 through 6, without the enormous data requirements. The total data transfer for a payment is about 20 kilobytes, and the protocol can be completed in several seconds. The authors consider this the first ideal untraceable electronic cash system.

Anonymous Credit Cards

This protocol [988] uses several different banks to protect the identity of the customer. Each customer has an account at two different banks. The first bank knows the person's identity and is willing to extend him credit. The second bank knows the customer only under a pseudonym (similar to a numbered Swiss bank account).

The customer can withdraw funds from the second bank by proving that the account is his. However, the bank does not know the person and is unwilling to extend him credit. The first bank knows the customer and transfers funds to the second bank—without knowing the pseudonym. The customer then spends these funds anonymously. At the end of the month, the second bank gives the first bank a bill, which it trusts the bank to pay. The first bank passes the bill on to the customer, which it trusts the customer to pay. When the customer pays, the first bank transfers additional funds to the second bank. All transactions are handled through an intermediary, which acts sort of like an electronic Federal Reserve: settling accounts among banks, logging messages, and creating an audit trail.

Exchanges between the customer, merchant, and various banks are outlined in [988]. Unless everyone colludes against the customer, his anonymity is assured. However, this is not digital cash; it is easy for the bank to cheat. The protocol allows customers to keep the advantages of credit cards without giving up their privacy.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part II
Cryptographic Techniques

Chapter 7
Key Length

7.1 Symmetric Key Length

The security of a symmetric cryptosystem is a function of two things: the strength of the algorithm and the length of the key. The former is more important, but the latter is easier to demonstrate.

Assume that the strength of the algorithm is perfect. This is extremely difficult to achieve in practice, but easy enough for this example. By perfect, I mean that there is no better way to break the cryptosystem other than trying every possible key in a brute-force attack.

To launch this attack, a cryptanalyst needs a small amount of ciphertext and the corresponding plaintext; a brute-force attack is a known-plaintext attack. For a block cipher, the cryptanalyst would need a block of ciphertext and corresponding plaintext: generally 64 bits. Getting this plaintext and ciphertext is easier than you might imagine. A cryptanalyst might get a copy of a plaintext message by some means and intercept the corresponding ciphertext. He may know something about the format of the ciphertext: For example, it is a WordPerfect file, it has a standard electronic-mail message header, it is a UNIX directory file, it is a TIFF image, or it is a standard record in a customer database. All of these formats have some predefined bytes. The cryptanalyst doesn't need much plaintext to launch this attack.

Calculating the complexity of a brute-force attack is easy. If the key is 8 bits long, there are 28, or 256, possible keys. Therefore, it will take 256 attempts to find the correct key, with a 50 percent chance of finding the key after half of the attempts. If the key is 56 bits long, then there are 256 possible keys. Assuming a supercomputer can try a million keys a second, it will take 2285 years to find the correct key. If the key is 64 bits long, then it will take the same supercomputer about 585,000 years to find the correct key among the 264 possible keys. If the key is 128 bits long, it will take 1025 years. The universe is only 1010 years old, so 1025 years is a long time. With a 2048-bit key, a million million-attempts-per-second computers working in parallel will spend 10597 years finding the key. By that time the universe will have long collapsed or expanded into nothingness.

Before you rush to invent a cryptosystem with an 8-kilobyte key, remember the other side to the strength question: The algorithm must be so secure that there is no better way to break it than with a brute-force attack. This is not as easy as it might seem. Cryptography is a subtle art. Cryptosystems that look perfect are often extremely weak. Strong cryptosystems, with a couple of minor changes, can become weak. The warning to the amateur cryptographer is to have a healthy, almost paranoid, suspicion of any new algorithm. It is best to trust algorithms that professional cryptographers have scrutinized for years without cracking them and to be suspicious of algorithm designers' grandiose claims of security.

Recall an important point from Section 1.1: The security of a cryptosystem should rest in the key, not in the details of the algorithm. Assume that any cryptanalyst has access to all the details of your algorithm. Assume he has access to as much ciphertext as he wants and can mount an intensive ciphertext-only attack. Assume that he can mount a plaintext attack with as much data as he needs. Even assume that he can mount a chosen-plaintext attack. If your cryptosystem can remain secure, even in the face of all that knowledge, then you've got something.

That warning aside, there is still plenty of room in cryptography to maneuver. In reality, this kind of security isn't really necessary in many situations. Most adversaries don't have the knowledge and computing resources of a major government, and even the ones who do probably aren't that interested in breaking your cryptosystem. If you're plotting to overthrow a major government, stick with the tried and true algorithms in the back of the book. The rest of you, have fun.

Time and Cost Estimates for Brute-Force Attack

Remember that a brute-force attack is typically a known-plaintext attack; it requires a small amount of ciphertext and corresponding plaintext. If you assume that a brute-force attack is the most efficient attack possible against an algorithm—a big assumption—then the key must be long enough to make the attack infeasible. How long is that?

Two parameters determine the speed of a brute-force attack: the number of keys to be tested and the speed of each test. Most symmetric algorithms accept any fixed-length bit pattern as the key. DES has a 56-bit key; it has 256 possible keys. Some algorithms discussed in this book have a 64-bit key; these have 264 possible keys. Others have a 128-bit key.

The speed at which each possible key can be tested is also a factor, but a less important one. For the purposes of this analysis, I will assume that each different algorithm can be tested in the same amount of time. The reality may be that one algorithm may be tested two, three, or even ten times faster than another. But since we are looking for key lengths that are millions of times more difficult to crack than would be feasible, small differences due to test speed are irrelevant.

Most of the debate in the cryptologic community about the efficiency of brute-force attacks has centered on the DES algorithm. In 1977, Whitfield Diffie and Martin Hellman [497] postulated the existence of a special-purpose DES-cracking machine. This machine consisted of a million chips, each capable of testing a million keys per second. Such a machine could test 256 keys in 20 hours. If built to attack an algorithm with a 64-bit key, it could test all 264 keys in 214 days.

A brute-force attack is tailor-made for parallel processors. Each processor can test a subset of the keyspace. The processors do not have to communicate among themselves; the only communication required at all is a single message signifying success. There are no shared memory requirements. It is easy to design a machine with a million parallel processors, each working independent of the others.

More recently, Michael Wiener decided to design a brute-force cracking machine [1597,1598]. (He designed the machine for DES, but the analysis holds for most any algorithm.) He designed specialized chips, boards, and racks. He estimated prices. And he discovered that for $1 million, someone could build a machine that could crack a 56-bit DES key in an average of 3.5 hours (results guaranteed in 7 hours). And that the price/speed ratio is linear. Table 7.1 generalizes these numbers to a variety of key lengths. Remember Moore's Law: Computing power doubles approximately every 18 months. This means costs go down a factor of 10 every five years; what cost $1 million to build in 1995 will cost a mere $100,000 in the year 2000. Pipelined computers might do even better [724].

For 56-bit keys, these numbers are within the budgets of most large companies and many criminal organizations. The military budgets of most industrialized nations can afford to break 64-bit keys. Breaking an 80-bit key is still beyond the realm of possibility, but if current trends continue that will change in only 30 years.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Of course, it is ludicrous to estimate computing power 35 years in the future. Breakthroughs in some science-fiction technology could make these numbers look like a joke. Conversely, physical limitations unknown at the present time could make them unrealistically optimistic. In cryptography it is wise to be pessimistic. Fielding an algorithm with an 80-bit key seems extremely short-sighted. Insist on at least 112-bit keys.

Table 7.1 Average Time Estimates for a Hardware Brute-Force Attack in 1995
Length of Key in Bits
Cost | 40 | 56 | 64 | 80 | 112 | 128
$100 K | 2 seconds | 35 hours | 1 year | 70,000 years | 1014 years | 1019 years
$1 M | .2 seconds | 3.5 hours | 37 days | 7000 years | 1013 years | 1018 years
$10 M | .02 seconds | 21 minutes | 4 days | 700 years | 1012 years | 1017 years
$100 M | 2 milliseconds | 2 minutes | 9 hours | 70 years | 1011 years | 1016 years
$1 G | .2 milliseconds | 13 seconds | 1 hour | 7 years | 1010 years | 1015 years
$10 G | .02 milliseconds | 1 second | 5.4 minutes | 245 days | 109 years | 1014 years
$100 G | 2 microseconds | .1 second | 32 seconds | 24 days | 108 years | 1013 years
$1 T | .2 microseconds | .01 second | 3 seconds | 2.4 days | 107 years | 1012 years
$10 T | .02 microseconds | 1 millisecond | .3 second | 6 hours | 106 years | 1011 years

If an attacker wants to break a key badly enough, all he has to do is spend money. Consequently, it seems prudent to try to estimate the minimum "value" of a key: How much value can be trusted to a single key before it makes economic sense to try to break? To give an extreme example, if an encrypted message is worth $1.39, then it wouldn't make much financial sense to set a $10-million cracker to the task of recovering the key. On the other hand, if the plaintext message is worth $100 million, then decrypting that single message would justify the cost of building the cracker. Also, the value of some messages decreases rapidly with time.

Software Crackers

Without special-purpose hardware and massively parallel machines, brute-force attacks are significantly harder. A software attack is about a thousand times slower than a hardware attack.

The real threat of a software-based brute-force attack is not that it is certain, but that it is "free." It costs nothing to set up a microcomputer to test possible keys whenever it is idle. If it finds the correct key—great. If it doesn't, then nothing is lost. It costs nothing to set up an entire microcomputer network to do that. A recent experiment with DES used the collective idle time of 40 workstations to test 234 keys in a single day [603]. At this speed, it will take four million days to test all keys, but if enough people try attacks like this, then someone somewhere will get lucky. As was said in [603]:

The crux of the software threat is sheer bad luck. Imagine a university computer network of 512 workstations, networked together. On some campuses this would be a medium-sized network. They could even be spread around the world, coordinating their activity through electronic mail. Assume each workstation is capable of running [the algorithm] at a rate of 15,000 encryptions per second.... Allowing for the overhead of testing and changing keys, this comes down to...8192 tests per second per machine. To exhaust [a 56-bit] keyspace with this setup would take 545 years (assuming the network was dedicated to the task twenty-four hours per day). Notice, however, that the same calculations give our hypothetical student hackers one chance in 200,000 of cracking a key in one day. Over a long weekend their odds increase to one chance in sixty-six thousand. The faster their hardware, or the more machines involved, the better their chance becomes. These are not good odds for earning a living from horse racing, but they're not the stuff of good press releases either. They are much better odds than the Government gives on its lotteries, for instance. "One-in-a-million"? "Couldn't happen again in a thousand years"? It is no longer possible to say such things honestly. Is this an acceptable ongoing risk?

Using an algorithm with a 64-bit key instead of a 56-bit key makes this attack 256 times more difficult. With a 40-bit key, the picture is far more bleak. A network of 400 computers, each capable of performing 32,000 encryptions per second, can complete a brute-force attack against a 40-bit key in a single day. (In 1992, the RC2 and RC4 algorithms were approved for export with a 40-bit key—see Section 13.8.)

A 128-bit key makes a brute-force attack ridiculous even to contemplate. Industry experts estimate that by 1996 there will be 200 million computers in use worldwide. This estimate includes everything from giant Cray mainframes to subnotebooks. If every one of those computers worked together on this brute-force attack, and each computer performed a million encryptions per second every second, it would still take a million times the age of the universe to recover the key.

Neural Networks

Neural nets aren't terribly useful for cryptanalysis, primarily because of the shape of the solution space. Neural nets work best with problems that have a continuity of solutions, some better than others. This allows a neural net to learn, proposing better and better solutions as it does. Breaking an algorithm provides for very little in the way of learning opportunities: You either recover the key or you don't. (At least this is true if the algorithm is any good.) Neural nets work well in structured environments where there is something to learn, but not in the high-entropy, seemingly random world of cryptography.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Viruses

The greatest difficulty in getting millions of computers to work on a brute-force attack is convincing millions of computer owners to participate. You could ask politely, but that's time-consuming and they might say no. You could try breaking into their machines, but that's even more time-consuming and you might get arrested. You could also use a computer virus to spread the cracking program more efficiently over as many computers as possible.

This is a particularly insidious idea, first presented in [1593]. The attacker writes and lets loose a computer virus. This virus doesn't reformat the hard drive or delete files; it works on a brute-force cryptanalysis problem whenever the computer is idle. Various studies have shown that microcomputers are idle between 70 percent and 90 percent of the time, so the virus shouldn't have any trouble finding time to work on its task. If it is otherwise benign, it might even escape notice while it does its work.

Eventually, one machine will stumble on the correct key. At this point there are two ways of proceeding. First, the virus could spawn a different virus. It wouldn't do anything but reproduce and delete any copies of the cracking virus it finds but would contain the information about the correct key. This new virus would simply propagate through the computer world until it lands on the computer of the person who wrote the original virus.

A second, sneakier approach would be for the virus to display this message on the screen:

    There is a serious bug in this computer.
    Please call 1-800-123-4567 and read the
    following 64-bit number to the operator:

    xxxx xxxx xxxx xxxx

    There is a $100 reward for the first
    person to report this bug.

How efficient is this attack? Assume the typical infected computer tries a thousand keys per second. This rate is far less than the computer's maximum potential, because we assume it will be doing other things occasionally. Also assume that the typical virus infects 10 million machines. This virus can break a 56-bit key in 83 days and a 64-bit key in 58 years. You might have to bribe the antiviral software makers, but that's your problem. Any increase in computer speeds or the virus infection rate would, of course, make this attack more efficient.

The Chinese Lottery

The Chinese Lottery is an eclectic, but possible, suggestion for a massively parallel cryptanalysis machine [1278]. Imagine that a brute-force, million-test-per-second cracking chip was built into every radio and television sold. Each chip is programmed to test a different set of keys automatically upon receiving a plaintext/ciphertext pair over the airwaves. Every time the Chinese government wants to break a key, it broadcasts the data. All the radios and televisions in the country start chugging away. Eventually, the correct key will appear on someone's display, somewhere in the country. The Chinese government pays a prize to that person; this makes sure that the result is reported promptly and properly, and also helps the sale of radios and televisions with the cracking chips.

If every man, woman, and child in China owns a radio or television, then the correct key to a 56-bit algorithm will appear in 61 seconds. If only 1 in 10 Chinese owns a radio or television—closer to reality—the correct key will appear in 10 minutes. The correct key for a 64-bit algorithm will appear in 4.3 hours—43 hours if only 1 in 10 owns a radio or television.

Some modifications are required to make this attack practical. First, it would be easier to have each chip try random keys instead of a unique set of keys. This would make the attack about 39 percent slower—not much in light of the numbers we're working with. Also, the Chinese Communist party would have to mandate that every person listen to or watch a certain show at a certain time, just to make sure that all of the radios and televisions are operating when the plaintext/ciphertext pair is broadcast. Finally, everyone would have to be instructed to call a Central-Party-Whatever-It's-Called if a key ever shows up on their screen, and then to read off the string of numbers appearing there.

Table 7.2 shows the effectiveness of the Chinese Lottery for different countries and different key lengths. China would clearly be in the best position to launch such an attack if they have to outfit every man, woman, and child with their own television or radio. The United States has fewer people but a lot more equipment per capita. The state of Wyoming could break a 56-bit key all by itself in less than a day.

Biotechnology

If biochips are possible, then it would be foolish not to use them as a distributed brute-force cryptanalysis tool. Consider a hypothetical animal, unfortunately called a "DESosaur" [1278]. It consists of biological cells capable of testing possible keys. The plaintext/ciphertext pair is broadcast to the cells via some optical channel (these cells are transparent, you see). Solutions are carried to the DESosaur's speech organ via special cells that travel through the animal's circulatory system.

The typical dinosaur had about 1014 cells (excluding bacteria). If each of them can perform a million encryptions per second (granted, this is a big if), breaking a 56-bit key would take seven ten-thousandths of a second. Breaking a 64-bit key would take less than two tenths of a second. Breaking a 128-bit key would still take 1011 years, though.

Table 7.2 Brute-Force Cracking Estimates for Chinese Lottery
 |  |  | Time to Break
Country | Population | # of Televisions/Radios | 56-bit | 64-bit
China | 1,190,431,000 | 257,000,000 | 280 seconds | 20 hours
U.S. | 260,714,000 | 739,000,000 | 97 seconds | 6.9 hours
Iraq | 19,890,000 | 4,730,000 | 4.2 hours | 44 days
Israel | 5,051,000 | 3,640,000 | 5.5 hours | 58 days
Wyoming | 470,000 | 1,330,000 | 15 hours | 160 days
Winnemucca, NV | 6,100 | 17,300 | 48 days | 34 years
(All data is from the 1995 World Almanac and Book of Facts.)

Another biological approach is to use genetically engineered cryptanalytic algae that are capable of performing brute-force attacks against cryptographic algorithms [1278]. These organisms would make it possible to construct a distributed machine with more processors because they could cover a larger area. The plaintext/ciphertext pair could be broadcast by satellite. If an organism found the result, it could induce the nearby cells to change color to communicate the solution back to the satellite.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Assume the typical algae cell is the size of a cube 10 microns on a side (this is probably a large estimate), then 1015 of them can fill a cubic meter. Pump them into the ocean and cover 200 square miles (518 square kilometers) of water to a meter deep (you figure out how to do it—I'm just the idea man), and you'd have 1023 (over a hundred billion gallons) of them floating in the ocean. (For comparison, the Exxon Valdez spilled 10 million gallons of oil.) If each of them can try a million keys per second, they will recover the key for a 128-bit algorithm in just over 100 years. (The resulting algae bloom is your problem.) Breakthroughs in algae processing speed, algae diameter, or even the size puddle one could spread across the ocean, would reduce these numbers significantly.

Don't even ask me about nanotechnology.

Thermodynamic Limitations

One of the consequences of the second law of thermodynamics is that a certain amount of energy is necessary to represent information. To record a single bit by changing the state of a system requires an amount of energy no less than kT, where T is the absolute temperature of the system and k is the Boltzman constant. (Stick with me; the physics lesson is almost over.)

Given that k =1.38*10-16 erg/°Kelvin, and that the ambient temperature of the universe is 3.2°K, an ideal computer running at 3.2°K would consume 4.4*10-16 ergs every time it set or cleared a bit. To run a computer any colder than the cosmic background radiation would require extra energy to run a heat pump.

Now, the annual energy output of our sun is about 1.21*1041 ergs. This is enough to power about 2.7*1056 single bit changes on our ideal computer; enough state changes to put a 187-bit counter through all its values. If we built a Dyson sphere around the sun and captured all of its energy for 32 years, without any loss, we could power a computer to count up to 2192. Of course, it wouldn't have the energy left over to perform any useful calculations with this counter.

But that's just one star, and a measly one at that. A typical supernova releases something like 1051 ergs. (About a hundred times as much energy would be released in the form of neutrinos, but let them go for now.) If all of this energy could be channeled into a single orgy of computation, a 219-bit counter could be cycled through all of its states.

These numbers have nothing to do with the technology of the devices; they are the maximums that thermodynamics will allow. And they strongly imply that brute-force attacks against 256-bit keys will be infeasible until computers are built from something other than matter and occupy something other than space.

7.2 Public-Key Key Length

One-way functions were discussed in Section 2.3. Multiplying two large primes is a one-way function; it's easy to multiply the numbers to get a product but hard to factor the product and recover the two large primes (see Section 11.3). Public-key cryptography uses this idea to make a trap-door one-way function. Actually, that's a lie; factoring is conjectured to be a hard problem (see Section 11.4). As far as anyone knows, it seems to be. Even if it is, no one can prove that hard problems are actually hard. Most everyone assumes that factoring is hard, but it has never been mathematically proven one way or the other.

This is worth dwelling on. It is easy to imagine that 50 years in the future we will all sit around, reminiscing about the good old days when people used to think factoring was hard, cryptography was based on factoring, and companies actually made money from this stuff. It is easy to imagine that future developments in number theory will make factoring easier or that developments in complexity theory will make factoring trivial. There's no reason to believe this will happen—and most people who know enough to have an opinion will tell you that it is unlikely—but there's also no reason to believe it won't.

In any case, today's dominant public-key encryption algorithms are based on the difficulty of factoring large numbers that are the product of two large primes. (Other algorithms are based on something called the Discrete Logarithm Problem, but for the moment assume the same discussion applies.) These algorithms are also susceptible to a brute-force attack, but of a different type. Breaking these algorithms does not involve trying every possible key; breaking these algorithms involves trying to factor the large number (or taking discrete logarithms in a very large finite field—a similar problem). If the number is too small, you have no security. If the number is large enough, you have security against all the computing power in the world working from now until the sun goes nova—given today's understanding of the mathematics. Section 11.3 discusses factoring in more mathematical detail; here I will limit the discussion to how long it takes to factor numbers of various lengths.

Factoring large numbers is hard. Unfortunately for algorithm designers, it is getting easier. Even worse, it is getting easier faster than mathematicians expected. In 1976 Richard Guy wrote: "I shall be surprised if anyone regularly factors numbers of size 1080 without special form during the present century" [680]. In 1977 Ron Rivest said that factoring a 125-digit number would take 40 quadrillion years [599]. In 1994 a 129-digit number was factored [66]. If there is any lesson in all this, it is that making predictions is foolish.

Table 7.3 shows factoring records over the past dozen years. The fastest factoring algorithm during the time was the quadratic sieve (see Section 11.3).

These numbers are pretty frightening. Today it is not uncommon to see 512-bit numbers used in operational systems. Factoring them, and thereby completely compromising their security, is well in the range of possibility: A weekend-long worm on the Internet could do it.

Computing power is generally measured in mips-years: a one-million-instruction-per-second (mips) computer running for one year, or about 3*1013 instructions. By convention, a 1-mips machine is equivalent to the DEC VAX 11/780. Hence, a mips-year is a VAX 11/780 running for a year, or the equivalent. (A 100 MHz Pentium is about a 50 mips machine; a 1800-node Intel Paragon is about 50,000.)

The 1983 factorization of a 71-digit number required 0.1 mips-years; the 1994 factorization of a 129-digit number required 5000. This dramatic increase in computing power resulted largely from the introduction of distributed computing, using the idle time on a network of workstations. This trend was started by Bob Silverman and fully developed by Arjen Lenstra and Mark Manasse. The 1983 factorization used 9.5 CPU hours on a single Cray X-MP; the 1994 factorization took 5000 mips-years and used the idle time on 1600 computers around the world for about eight months. Modern factoring methods lend themselves to this kind of distributed implementation.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The picture gets even worse. A new factoring algorithm has taken over from the quadratic sieve: the general number field sieve. In 1989 mathematicians would have told you that the general number field sieve would never be practical. In 1992 they would have told you that it was practical, but only faster than the quadratic sieve for numbers greater than 130 to 150 digits or so. Today it is known to be faster than the quadratic sieve for numbers well below 116 digits [472,635]. The general number field sieve can factor a 512-bit number over 10 times faster than the quadratic sieve. The algorithm would require less than a year to run on an 1800-node Intel Paragon. Table 7.4 gives the number of mips-years required to factor numbers of different sizes, given current implementations of the general number field sieve [1190].

Table 7.3 Factoring Using the Quadratic Sieve
Year | # of decimal digits factored | How many times harder to factor a 512-bit number
1983 | 71 | >20 million
1985 | 80 | >2 million
1988 | 90 | 250,000
1989 | 100 | 30,000
1993 | 120 | 500
1994 | 129 | 100

And the general number field sieve is still getting faster. Mathematicians keep coming up with new tricks, new optimizations, new techniques. There's no reason to think this trend won't continue. A related algorithm, the special number field sieve, can already factor numbers of a certain specialized form—numbers not generally used for cryptography—much faster than the general number field sieve can factor general numbers of the same size. It is not unreasonable to assume that the general number field sieve can be optimized to run this fast [1190]; it is possible that the NSA already knows how to do this. Table 7.5 gives the number of mips-years required for the special number field sieve to factor numbers of different lengths [1190].

At a European Institute for System Security workshop in 1991, the participants agreed that a 1024-bit modulus should be sufficient for long-term secrets through 2002 [150]. However, they warned: "Although the participants of this workshop feel best qualified in their respective areas, this statement [with respect to lasting security] should be taken with caution." This is good advice.

The wise cryptographer is ultra-conservative when choosing public-key key lengths. To determine how long a key you need requires you to look at both the intended security and lifetime of the key, and the current state-of-the-art of factoring. Today you need a 1024-bit number to get the level of security you got from a 512-bit number in the early 1980s. If you want your keys to remain secure for 20 years, 1024 bits is likely too short.

Even if your particular secrets aren't worth the effort required to factor your modulus, you may be at risk. Imagine an automatic banking system that uses RSA for security. Mallory can stand up in court and say: "Did you read in the newspaper in 1994 that RSA-129 was broken, and that 512-bit numbers can be factored by any organization willing to spend a few million dollars and wait a few months? My bank uses 512-bit numbers for security and, by the way, I didn't make these seven withdrawals." Even if Mallory is lying, the judge will probably put the onus on the bank to prove it.

Table 7.4 Factoring Using the General Number Field Sieve
# of bits | Mips-years required to factor
512 | 30,000
768 | 2*108
1024 | 3*1011
1280 | 1*1014
1536 | 3*1016
2048 | 3*1020

Table 7.5 Factoring Using the Special Number Field Sieve
# of bits | Mips-years required to factor
512 | <200
768 | 100,000
1024 | 3*107
1280 | 3*109
1536 | 2*1011
2048 | 4*1014

Why not use 10,000-bit keys? You can, but remember that you pay a price in computation time as your keys get longer. You want a key long enough to be secure, but short enough to be computationally usable.

Earlier in this section I called making predictions foolish. Now I am about to make some. Table 7.6 gives my recommendations for public-key lengths, depending on how long you require the key to be secure. There are three key lengths for each year, one secure against an individual, one secure against a major corporation, and the third secure against a major government.

Here are some assumptions from [66]:

We believe that we could acquire 100 thousand machines without superhuman or unethical efforts. That is, we would not set free an Internet worm or virus to find resources for us. Many organizations have several thousand machines each on the net. Making use of their facilities would require skillful diplomacy, but should not be impossible. Assuming the 5 mips average power, and one year elapsed time, it is not too unreasonable to embark on a project which would require half a million mips years.

The project to factor the 129-digit number harnessed an estimated 0.03 percent of the total computing power of the Internet [1190], and they didn't even try very hard. It isn't unreasonable to assume that a well-publicized project can harness 2 percent of the world's computing power for a year.

Assume a dedicated cryptanalyst can get his hands on 10,000 mips-years, a large corporation can get 107 mips-years, and that a large government can get 109 mips-years. Also assume that computing power will increase by a factor of 10 every five years. And finally, assume that advances in factoring mathematics allow us to factor general numbers at the speeds of the special number field sieve. (This isn't possible yet, but the breakthrough could occur at any time.) Table 7.6 recommends different key lengths for security during different years.

Table 7.6 Recommended Public-key Key Lengths (in bits)
Year | vs. Individual | vs. Corporation | vs. Government
1995 | 768 | 1280 | 1536
2000 | 1024 | 1280 | 1536
2005 | 1280 | 1536 | 2048
2010 | 1280 | 1536 | 2048
2015 | 1536 | 2048 | 2048


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Remember to take the value of the key into account. Public keys are often used to secure things of great value for a long time: the bank's master key for a digital cash system, the key the government uses to certify its passports, or a notary public's digital signature key. It probably isn't worth the effort to spend months of computing time to break an individual's private key, but if you can print your own money with a broken key the idea becomes more attractive. A 1024-bit key is long enough to sign something that will be verified within the week, or month, or even a few years. But you don't want to stand up in court 20 years from now with a digitally signed document and have the opposition demonstrate how to forge documents with the same signature.

Making predictions beyond the near future is even more foolish. Who knows what kind of advances in computing, networking, and mathematics are going to happen by 2020? However, if you look at the broad picture, in every decade we can factor numbers twice as long as in the previous decade. This leads to Table 7.7.

On the other hand, factoring technology may reach its Omega point long before 2045. Twenty years from now, we may be able to factor anything. I think that is unlikely, though.

Not everyone will agree with my recommendations. The NSA has mandated 512-bit to 1024-bit keys for their Digital Signature Standard (see Section 20.1)—far less than I recommend for long-term security. Pretty Good Privacy (see Section 24.12) has a maximum RSA key length of 2047 bits. Arjen Lenstra, the world's most successful factorer, refuses to make predictions past 10 years [949]. Table 7.8 gives Ron Rivest's key-length recommendations, originally made in 1990, which I consider much too optimistic [1323]. While his analysis looks fine on paper, recent history illustrates that surprises regularly happen. It makes sense to choose your keys to be resilient against future surprises.

Table 7.7 Long-range Factoring Predictions
Year | Key Length (in bits)
1995 | 1024
2005 | 2048
2015 | 4096
2025 | 8192
2035 | 16,384
2045 | 32,768

Low estimates assume a budget of $25,000, the quadratic sieve algorithm, and a technology advance of 20 percent per year. Average estimates assume a budget of $25 million, the general number field sieve algorithm, and a technology advance of 33 percent per year. High estimates assume a budget of $25 billion, a general quadratic sieve algorithm running at the speed of the special number field sieve, and a technology advance of 45 percent per year.

There is always the possibility that an advance in factoring will surprise me as well, but I factored that into my calculations. But why trust me? I just proved my own foolishness by making predictions.

DNA Computing

Now it gets weird. In 1994 Leonard M. Adleman actually demonstrated a method for solving an NP-complete problem (see Section 11.2) in a biochemistry laboratory, using DNA molecules to represent guesses at solutions to the problem [17]. (That's "solutions" meaning "answers," not meaning "liquids containing solutes." Terminology in this field is going to be awkward.) The problem that Adleman solved was an instance of the Directed Hamiltonian Path problem: Given a map of cities connected by one-way roads, find a path from City A to City Z that passes exactly once through all other cities on the map. Each city was represented by a different random 20-base string of DNA; with conventional molecular biology techniques, Adleman synthesized 50 picomols (30 million million molecules) of the DNA string representing each city. Each road was also represented by a 20-base DNA string, but these strings were not chosen randomly: They were cleverly chosen so that the "beginning" end of the DNA string representing the road from City P to City K ("Road PK") would tend to stick to the DNA string representing City P, and the end of Road PK would tend to stick to City K.

Table 7.8 Rivest's Optimistic Key-length Recommendations (in bits)
Year | Low | Average | High
1990 | 398 | 515 | 1289
1995 | 405 | 542 | 1399
2000 | 422 | 572 | 1512
2005 | 439 | 602 | 1628
2010 | 455 | 631 | 1754
2015 | 472 | 661 | 1884
2020 | 489 | 677 | 2017

Adleman synthesized 50 picomols of the DNA representing each road, mixed them all together with the DNA representing all the cities, and added a ligase enzyme, which links together the ends of DNA molecules. The clever relationship between the road DNA strings and the city DNA strings causes the ligase to link the road DNA strings together in a legal fashion. That is, the "exit" end of the road from P to K will always be linked to the "entrance" end of some road that originates at City K, never to the "exit" end of any road and never to the "entrance" end of a road that originates at some city other than K. After a carefully limited reaction time, the ligase has built a large number of DNA strings representing legal but otherwise random multiroad paths within the map.

From this soup of random paths, Adleman can find the tiniest trace—perhaps even a single molecule—of the DNA that represents the answer to the problem. Using common techniques of molecular biology, he discards all the DNA strings representing paths that are too long or too short. (The number of roads in the desired path must equal the number of cities minus one.) Next he discards all the DNA strings that do not pass through City A, then those that miss City B, and so forth. If any DNA survives this screening, it is examined to find the sequence of roads that it represents: This is the solution to the directed Hamiltonian path problem.

By definition, an instance of any NP-complete problem can be transformed, in polynomial time, into an instance of any other NP-complete problem, and therefore into an instance of the directed Hamiltonian path problem. Since the 1970s, cryptologists have been trying to use NP-complete problems for public-key cryptography.

While the instance that Adleman solved was very modest (seven cities on his map, a problem that can be solved by inspection in a few minutes), the technique is in its infancy and has no forbidding obstacles keeping it from being extended to larger problems. Thus, arguments about the security of cryptographic protocols based on NP-complete problems, arguments that heretofore have begun, "Suppose an adversary has a million processors, each of which can perform a million tests each second," may soon have to be replaced with, "Suppose an adversary has a thousand fermentation vats, each 20,000 liters in capacity."


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Quantum Computing

Now, it gets even weirder. The underlying principle behind quantum computing involves Einstein's wave-particle duality. A photon can simultaneously exist in a large number of states. A classic example is that a photon behaves like a wave when it encounters a partially silvered mirror; it is both reflected and transmitted, just as an ocean wave striking a seawall with a small opening in it will both reflect off the wall and pass through it. However, when a photon is measured, it behaves like a particle and only a single state can be detected.

In [1443], Peter Shor outlines a design for a factoring machine based on quantum mechanical principles. Unlike a classical computer, which can be thought of as having a single, fixed state at a given time, a quantum computer has an internal wave function, which is a superposition of a combination of the possible basis states. Computations transform the wave function, altering the entire set of states in a single operation. In this way, a quantum computer is an improvement over classical finite-state automata: It uses quantum properties to allow it to factor in polynomial time, theoretically allowing one to break cryptosystems based on factoring or the discrete logarithm problem.

The consensus is that quantum computers are compatible with the fundamental laws of quantum mechanics. However, it is unlikely that a quantum factoring machine will be built in the foreseeable future...if ever. One major obstacle is the problem of decoherence, which causes superimposed waveforms to lose their distinctness and makes the computer fail. Decoherence will make a quantum computer running at 1° Kelvin fail after just one nanosecond. Additionally, an enormous number of gates would be required to build a quantum factoring device; this may render the machine impossible to build. Shor's design requires a complete modular exponentiator. No internal clock can be used, so millions or possibly billions of individual gates would be required to factor cryptographically significant numbers. If n quantum gates have some minimum probability p of failure, the average number of trials required per successful run is (1/(1 – p))n. The number of gates required presumably grows polynomially with the length (in bits) of the number, so the number of trials required would be superexponential with the length of the numbers used—worse than factoring by trial division!

So, while quantum factorization is an area of great academic excitement, it is extremely unlikely that it will be practical in the foreseeable future. But don't say I didn't warn you.

7.3 Comparing Symmetric and Public-Key Key Length

A system is going to be attacked at its weakest point. If you are designing a system that uses both symmetric and public-key cryptography, the key lengths for each type of cryptography should be chosen so that it is equally difficult to attack the system via each mechanism. It makes no sense to use a symmetric algorithm with a 128-bit key together with a public-key algorithm with a 386-bit key, just as it makes no sense to use a symmetric algorithm with a 56-bit key together with a public-key algorithm with a 1024-bit key.

Table 7.9 lists public-key modulus lengths whose factoring difficulty roughly equals the difficulty of a brute-force attack for popular symmetric key lengths.

This table says that if you are concerned enough about security to choose a symmetric algorithm with a 112-bit key, you should choose a modulus length for your public-key algorithm of about 1792 bits. In general, though, you should choose a public-key length that is more secure than your symmetric-key length. Public keys generally stay around longer, and are used to protect more information.

7.4 Birthday Attacks against One-Way Hash Functions

There are two brute-force attacks against a one-way hash function. The first is the most obvious: Given the hash of message, H(M), an adversary would like to be able to create another document, M´, such that H(M) = H(M´). The second attack is more subtle: An adversary would like to find two random messages, M, and M´, such that H(M) = H(M´). This is called a collision, and it is a far easier attack than the first one.

Table 7.9 Symmetric and Public-key Key Lengths with Similar Resistances to Brute-Force Attacks
 | Symmetric Key Length | Public-key Key Length
 | 56 bits | 384 bits
 | 64 bits | 512 bits
 | 80 bits | 768 bits
 | 112 bits | 1792 bits
 | 128 bits | 2304 bits

The birthday paradox is a standard statistics problem. How many people must be in a room for the chance to be greater than even that one of them shares your birthday? The answer is 253. Now, how many people must there be for the chance to be greater than even that at least two of them will share the same birthday? The answer is surprisingly low: 23. With only 23 people in the room, there are still 253 different pairs of people in the room.

Finding someone with a specific birthday is analogous to the first attack; finding two people with the same random birthday is analogous to the second attack. The second attack is commonly known as a birthday attack.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Assume that a one-way hash function is secure and the best way to attack it is by using brute force. It produces an m-bit output. Finding a message that hashes to a given hash value would require hashing 2m random messages. Finding two messages that hash to the same value would only require hashing 2m/2 random messages. A machine that hashes a million messages per second would take 600,000 years to find a second message that matched a given 64-bit hash. The same machine could find a pair of messages that hashed to the same value in about an hour.

This means that if you are worried about a birthday attack, you should choose a hash-value twice as long as you otherwise might think you need. For example, if you want to drop the odds of someone breaking your system to less than 1 in 280, use a 160-bit one-way hash function.

7.5 How Long Should a Key Be?

There's no single answer to this question; it depends on the situation. To determine how much security you need, you must ask yourself some questions. How much is your data worth? How long does it need to be secure? What are your adversaries' resources?

A customer list might be worth $1000. Financial data for an acrimonious divorce case might be worth $10,000. Advertising and marketing data for a large corporation might be worth $1 million. The master keys for a digital cash system might be worth billions.

In the world of commodities trading, secrets only need to be kept for minutes. In the newspaper business, today's secrets are tomorrow's headlines. Product development information might need to remain secret for a year or two. U.S. Census data are required by law to remain secret for 100 years.

The guest list for your sister's surprise birthday party is only interesting to your nosy relatives. Corporate trade secrets are interesting to rival companies. Military secrets are interesting to rival militaries.

You can even specify security requirements in these terms. For example:

The key length must be such that there is a probability of no more than 1 in 232 that an attacker with $100 million to spend could break the system within one year, even assuming technology advances at a rate of 30 percent per annum over the period.

Table 7.10, taken partially from [150], estimates the secrecy requirements for several kinds of information:

Future computing power is harder to estimate, but here is a reasonable rule of thumb: The efficiency of computing equipment divided by price doubles every 18 months and increases by a factor of 10 every five years. Thus, in 50 years the fastest computers will be 10 billion times faster than today's! Remember, too, that these numbers only relate to general-purpose computers; who knows what kind of specialized cryptosystem-breaking equipment will be developed in the next 50 years?

Assuming that a cryptographic algorithm will be in use for 30 years, you can get some idea how secure it must be. An algorithm designed today probably will not see general use until 2000, and will still be used in 2025 to encrypt messages that must remain secret until 2075 or later.

Table 7.10 Security Requirements for Different Information
Type of Traffic | Lifetime | Minimum Key Length
Tactical military information | minutes/hours | 56–64 bits
Product announcements, mergers, interest rates | days/weeks | 64 bits
Long-term business plans | years | 64 bits
Trade secrets (e.g., recipe for Coca-Cola) | decades | 112 bits
H-bomb secrets | >40 years | 128 bits
Identities of spies | >50 years | 128 bits
Personal affairs | >50 years | 128 bits
Diplomatic embarrassments | >65 years | at least 128 bits
U.S. census data | 100 years | at least 128 bits

7.6 Caveat Emptor

This entire chapter is a whole lot of nonsense. The very notion of predicting computing power 10 years in the future, let alone 50 years is absolutely ridiculous. These calculations are meant to be a guide, nothing more. If the past is any guide, the future will be vastly different from anything we can predict.

Be conservative. If your keys are longer than you imagine necessary, then fewer technological surprises can harm you.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 8
Key Management

Alice and Bob have a secure communications system. They play mental poker, simultaneously sign contracts, even exchange digital cash. Their protocols are secure. Their algorithms are top-notch. Unfortunately, they buy their keys from Eve's "Keys-R-Us," whose slogan is "You can trust us: Security is the middle name of someone our ex-mother-in-law's travel agent met at the Kwik-E-Mart."

Eve doesn't have to break the algorithms. She doesn't have to rely on subtle flaws in the protocols. She can use their keys to read all of Alice's and Bob's message traffic without lifting a cryptanalytic finger.

In the real world, key management is the hardest part of cryptography. Designing secure cryptographic algorithms and protocols isn't easy, but you can rely on a large body of academic research. Keeping the keys secret is much harder.

Cryptanalysts often attack both symmetric and public-key cryptosystems through their key management. Why should Eve bother going through all the trouble of trying to break the cryptographic algorithm if she can recover the key because of sloppy key storage procedures? Why should she spend $10 million building a cryptanalysis machine if she can spend $1000 bribing a clerk? Spending a million dollars to buy a well-placed communications clerk in a diplomatic embassy can be a bargain. The Walkers sold U.S. Navy encryption keys to the Soviets for years. The CIA's director of counterintelligence went for less than $2 million, wife included. That's far cheaper than building massive cracking machines and hiring brilliant cryptanalysts. Eve can steal the keys. She can arrest or abduct someone who knows the keys. She can seduce someone and get the keys that way. (The Marines who guarded the U.S. Embassy in Moscow were not immune to that attack.) It's a whole lot easier to find flaws in people than it is to find them in cryptosystems.

Alice and Bob must protect their key to the same degree as all the data it encrypts. If a key isn't changed regularly, this can be an enormous amount of data. Unfortunately, many commercial products simply proclaim "We use DES" and forget about everything else. The results are not very impressive.

For example, the DiskLock program for Macintosh (version 2.1), sold at most software stores, claims the security of DES encryption. It encrypts files using DES. Its implementation of the DES algorithm is correct. However, DiskLock stores the DES key with the encrypted file. If you know where to look for the key, and want to read a file encrypted with DiskLock's DES, recover the key from the encrypted file and then decrypt the file. It doesn't matter that this program uses DES encryption—the implementation is completely insecure.

Further information on key management can be found in [457,98,1273,1225,775,357]. The following sections discuss some of the issues and solutions.

8.1 Generating Keys

The security of an algorithm rests in the key. If you're using a cryptographically weak process to generate keys, then your whole system is weak. Eve need not cryptanalyze your encryption algorithm; she can cryptanalyze your key generation algorithm.

Reduced Keyspaces

DES has a 56-bit key. Implemented properly, any 56-bit string can be the key; there are 256 (1016) possible keys. Norton Discreet for MS-DOS (versions 8.0 and earlier) only allows ASCII keys, forcing the high-order bit of each byte to be zero. The program also converts lowercase letters to uppercase (so the fifth bit of each byte is always the opposite of the sixth bit) and ignores the low-order bit of each byte, resulting in only 240 possible keys. These poor key generation procedures have made its DES ten thousand times easier to break than a proper implementation.

Table 8.1 gives the number of possible keys with various constraints on the input strings. Table 8.2 gives the time required for an exhaustive search through all of those keys, given a million attempts per second. Remember, there is very little time differential between an exhaustive search for 8-byte keys and an exhaustive search of 4-, 5-, 6-, 7-, and 8-byte keys.

All specialized brute-force hardware and parallel implementations will work here. Testing a million keys per second (either with one machine or with multiple machines in parallel), it is feasible to crack lowercase-letter and lowercase-letter-and-number keys up to 8 bytes long, alphanumeric-character keys up to 7 bytes long, printable character and ASCII-character keys up to 6 bytes long, and 8-bit-ASCII-character keys up to 5 bytes long.

Table 8.1 Number of Possible Keys of Various Keyspaces
 | 4-Byte | 5-Byte | 6-Byte | 7-Byte | 8-Byte
Lowercase letters (26): | 460,000 | 1.2*107 | 3.1*108 | 8.0*109 | 2.1*1011
Lowercase letters and digits (36): | 1,700,000 | 6.0*107 | 2.2*109 | 7.8*1010 | 2.8*1012
Alphanumeric characters (62): | 1.5*107 | 9.2*108 | 5.7*1010 | 3.5*1012 | 2.2*1014
Printable characters (95): | 8.1*107 | 7.7*109 | 7.4*1011 | 7.0*1013 | 6.6*1015
ASCII characters (128): | 2.7*108 | 3.4*1010 | 4.4*1012 | 5.6*1014 | 7.2*1016
8-bit ASCII characters (256): | 4.3*109 | 1.1*1012 | 2.8*1014 | 7.2*1016 | 1.8*1019
Table 8.2 Exhaustive Search of Various Keyspaces (assume one million attempts per second)
 | 4-Byte | 5-Byte | 6-Byte | 7-Byte | 8-Byte
Lowercase letters (26): | .5 seconds | 12 seconds | 5 minutes | 2.2 hours | 2.4 days
Lowercase letters and digits (36): | 1.7 seconds | 1 minute | 36 minutes | 22 hours | 33 days
Alphanumeric characters (62): | 15 seconds | 15 minutes | 16 hours | 41 days | 6.9 years
Printable characters (95): | 1.4 minutes | 2.1 hours | 8.5 days | 2.2 years | 210 years
ASCII characters (128): | 4.5 minutes | 9.5 hours | 51 days | 18 years | 2300 years
8-bit ASCII characters (256): | 1.2 hours | 13 days | 8.9 years | 2300 years | 580,000 years


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

And remember, computing power doubles every 18 months. If you expect your keys to stand up against brute-force attacks for 10 years, you'd better plan accordingly.

Poor Key Choices

When people choose their own keys, they generally choose poor ones. They're far more likely to choose "Barney" than "*9 (hH/A." This is not always due to poor security practices; "Barney" is easier to remember than "*9 (hH/A." The world's most secure algorithm won't help much if the users habitually choose their spouse's names for keys or write their keys on little pieces of paper in their wallets. A smart brute-force attack doesn't try all possible keys in numerical order; it tries the obvious keys first.

This is called a dictionary attack, because the attacker uses a dictionary of common keys. Daniel Klein was able to crack 40 percent of the passwords on the average computer using this system [847,848]. No, he didn't try one password after another, trying to login. He copied the encrypted password file and mounted the attack offline. Here's what he tried:

1. The user's name, initials, account name, and other relevant personal information as a possible password. All in all, up to 130 different passwords were tried based on this information. For an account name klone with a user named "Daniel V. Klein," some of the passwords that would be tried were: klone, klone0, klone1, klone123, dvk, dvkdvk, dklein, DKlein leinad, nielk, dvklein, danielk, DvkkvD, DANIEL-KLEIN, (klone), KleinD, and so on.
2. Words from various databases. These included lists of men's and women's names (some 16,000 in all); places (including variations so that "spain," "spanish," and "spaniard" would all be considered); names of famous people; cartoons and cartoon characters; titles, characters, and locations from films and science fiction stories; mythical creatures (garnered from Bullfinch's Mythology and dictionaries of mythical beasts); sports (including team names, nicknames, and specialized terms); numbers (both as numerals—"2001," and written out—"twelve"); strings of letters and numbers ("a," "aa," "aaa," "aaaa," etc.); Chinese syllables (from the Pinyin Romanization of Chinese, an international standard system of writing Chinese on an English keyboard); the King James Bible; biological terms; colloquial and vulgar phrases (such as "fuckyou," "ibmsux," and "deadhead"); keyboard patterns (such as "qwerty," "asdf," and "zxcvbn"); abbreviations (such as "roygbiv"—the colors in the rainbow, and "ooottafagvah"—a mnemonic for remembering the 12 cranial nerves); machine names (acquired from /etc/hosts); characters, plays, and locations from Shakespeare; common Yiddish words; the names of asteroids; and a collection of words from various technical papers Klein previously published. All told, more than 60,000 separate words were considered per user (with any inter- and intra-dictionary duplicates being discarded).
3. Variations on the words from step 2. This included making the first letter uppercase or a control character, making the entire word uppercase, reversing the word (with and without the aforementioned capitalization), changing the letter 'o' to the digit '0' (so that the word "scholar" would also be checked as "sch0lar"), changing the letter 'l' to the digit '1' (so that the word "scholar" would also be checked as "scho1ar"), and performing similar manipulation to change the letter 'z' into the digit '2', and the letter 's' into the digit '5'. Another test was to make the word into a plural (irrespective of whether the word was actually a noun), with enough intelligence built in so that "dress" became "dresses," "house" became "houses," and "daisy" became "daisies." Klein did not consider pluralization rules exclusively, though, so that "datum" forgivably became "datums" (not "data"), while "sphynx" became "sphynxs" (and not "sphynges"). Similarly, the suffixes "-ed," "-er," and "-ing" were added to transform words like "phase" into "phased," "phaser," and "phasing." These additional tests added another 1,000,000 words to the list of possible passwords that were tested for each user.
4. Various capitalization variations on the words from step 2 that were not considered in step 3. This included all single-letter capitalization variations (so that "michael" would also be checked as "mIchael," "miChael," "micHael," "michAel," etc.), double-letter capitalization variations ("MIchael," "MiChael," "MicHael,"..., "mIChael," "mIcHael," etc.), triple-letter variations, etc. The single-letter variations added roughly another 400,000 words to be checked per user, while the double-letter variations added another 1,500,000 words. Three-letter variations would have added at least another 3,000,000 words per user had there been enough time to complete the tests. Tests of four-, five-, and six-letter variations were deemed to be impracticable without much more computational horsepower to carry them out.
5. Foreign language words on foreign users. The specific test that was performed was to try Chinese language passwords on users with Chinese names. The Pinyin Romanization of Chinese syllables was used, combining syllables together into one-, two-, and three-syllable words. Because no tests were done to determine whether the words actually made sense, an exhaustive search was initiated. Since there are 298 Chinese syllables in the Pinyin system, there are 158,404 two-syllable words, and slightly more than 16,000,000 three-syllable words. A similar mode of attack could as easily be used with English, using rules for building pronounceable nonsense words.
6. Word pairs. The magnitude of an exhaustive test of this nature is staggering. To simplify the test, only words of three or four characters in length from /usr/dict/words were used. Even so, the number of word pairs is about ten million.

A dictionary attack is much more powerful when it is used against a file of keys and not a single key. A single user may be smart enough to choose good keys. If a thousand people each choose their own key as a password to a computer system, the odds are excellent that at least one person will choose a key in the attacker's dictionary.

Random Keys

Good keys are random-bit strings generated by some automatic process. If the key is 64 bits long, every possible 64-bit key must be equally likely. Generate the key bits from either a reliably random source (see Section 17.14) or a cryptographically secure pseudo-random-bit generator (see Chapters 16 and 17.) If these automatic processes are unavailable, flip a coin or roll a die.

This is important, but don't get too caught up in arguing about whether random noise from audio sources is more random than random noise from radioactive decay. None of these random-noise sources will be perfect, but they will probably be good enough. It is important to use a good random-number generator for key generation, but it is far more important to use good encryption algorithms and key management procedures. If you are worried about the randomness of your keys, use the key-crunching technique described below.

Some encryption algorithms have weak keys: specific keys that are less secure than the other keys. I advise testing for these weak keys and generating a new one if you discover one. DES has only 16 weak keys out of 256, so the odds of generating any of these keys are incredibly small. It has been argued that a cryptanalyst would have no idea that a weak key is being used and therefore gains no advantage from their accidental use. It has also been argued that not using weak keys gives a cryptanalyst information. However, testing for the few weak keys is so easy that it seems imprudent not to do so.

Generating keys for public-key cryptography systems is harder, because often the keys must have certain mathematical properties (they may have to be prime, be a quadratic residue, etc.). Techniques for generating large random prime numbers are discussed in Section 11.5. The important thing to remember from a key management point of view is that the random seeds for those generators must be just that: random.

Generating a random key isn't always possible. Sometimes you need to remember your key. (See how long it takes you to remember 25e8 56f2 e8ba c820). If you have to generate an easy-to-remember key, make it obscure. The ideal would be something easy to remember, but difficult to guess. Here are some suggestions:

— Word pairs separated by a punctuation character, for example "turtle*moose" or "zorch!splat"
— Strings of letters that are an acronym of a longer phrase; for example, "Mein Luftkissenfahrzeug ist voller Aale!" generates the key "MLivA!"


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Pass Phrases

A better solution is to use an entire phrase instead of a word, and to convert that phrase into a key. These phrases are called pass phrases. A technique called key crunching converts the easy-to-remember phrases into random keys. Use a one-way hash function to transform an arbitrary-length text string into a pseudo-random-bit string.

For example, the easy-to-remember text string:

My name is Ozymandias, king of kings. Look on my works, ye mighty, and despair.

might crunch into this 64-bit key:

e6c1 4398 5ae9 0a9b

Of course, it can be difficult to type an entire phrase into a computer with the echo turned off. Clever suggestions to solve this problem would be appreciated.

If the phrase is long enough, the resulting key will be random. Exactly what "long enough" means is open to interpretation. Information theory tells us that standard English has about 1.3 bits of information per character (see Section 11.1). For a 64-bit key, a pass phrase of about 49 characters, or 10 normal English words, should be sufficient. As a rule of thumb, figure that you need five words for each 4 bytes of key. That's a conservative assumption, since it doesn't take into account case, spacing, and punctuation.

This technique can even be used to generate private keys for public-key cryptography systems: The text string could be crunched into a random seed, and that seed could be fed into a deterministic system that generates public-key/private-key key pairs.

If you are choosing a pass phrase, choose something unique and easy-to-remember. Don't choose phrases from literature—the example from "Ozymandias" is a bad one. Both the complete works of Shakespeare and the dialogue from Star Wars are available on-line and can be used in a dictionary attack. Choose something obscure, but personal. Include punctuation and capitalization; if you can, include numbers and non-alphanumeric symbols. Poor or improper English, or even a foreign language, makes the pass phrase less susceptible to a dictionary attack. One suggestion is to use a phrase that is "shocking nonsense": something offensive enough that you are likely to remember and unlikely to write down.

Despite everything written here, obscurity is no substitute for true randomness. The best keys are random keys, difficult as they are to remember.

X9.17 Key Generation

The ANSI X9.17 standard specifies a method of key generation (see Figure 8.1) [55]. This does not generate easy-to-remember keys; it is more suitable for generating session keys or pseudo-random numbers within a system. The cryptographic algorithm used to generate keys is triple-DES, but it could just as easily be any algorithm.

Let EK(X) be triple-DES encryption of X with key K. This is a special key reserved for secret key generation. V0 is a secret 64-bit seed. T is a timestamp. To generate the random key Ri, calculate:

Ri = EK(EK(Ti) ⊕ Vi)

To generate Vi,+1, calculate:

Vi+1 = EK(EK(Ti) ⊕ Ri)

To turn Ri into a DES key, simply adjust every eighth bit for parity. If you need a 64-bit key, use it as is. If you need a 128-bit key, generate a pair of keys and concatenate them together.

DoD Key Generation

The U.S. Department of Defense recommends using DES in OFB mode (see Section 9.8) to generate random keys [1144]. Generate a DES key from system interrupt vectors, system status registers, and system counters. Generate an initialization vector from the system clock, system ID, and date and time. For the plaintext, use an externally generated 64-bit quantity: eight characters typed in by a system administrator, for example. Use the output as your key.

8.2 Nonlinear Keyspaces

Imagine that you are a military cryptography organization, building a piece of cryptography equipment for your troops. You want to use a secure algorithm, but you are worried about the equipment falling into enemy hands. The last thing you want is for your enemy to be able to use the equipment to protect their secrets.

Figure 8.1 ANSI X9.17 key generation.

If you can put your algorithm in a tamperproof module, here's what you can do. You can require keys of a special and secret form; all other keys will cause the module to encrypt and decrypt using a severely weakened algorithm. You can make it so that the odds of someone, not knowing this special form but accidentally stumbling on a correct key, are vanishingly small.

This is called a nonlinear keyspace, because all the keys are not equally strong. (The opposite is a linear, or flat, keyspace.) An easy way to do this is to create the key as two parts: the key itself and some fixed string encrypted with that key. The module decrypts the string with the key; if it gets the fixed string it uses the key normally, if not it uses a different, weak algorithm. If the algorithm has a 128-bit key and a 64-bit block size, the overall key is 192 bits; this gives the algorithm an effective key of 2128, but makes the odds of randomly choosing a good key one in 264.

You can be even subtler. You can design an algorithm such that certain keys are stronger than others. An algorithm can have no weak keys—keys that are obviously very poor—and can still have a nonlinear keyspace.

This only works if the algorithm is secret and the enemy can't reverse-engineer it, or if the difference in key strength is subtle enough that the enemy can't figure it out. The NSA did this with the secret algorithms in their Overtake modules (see Section 25.1). Did they do the same thing with Skipjack (see Section 13.12)? No one knows.

8.3 Transferring Keys

Alice and Bob are going to use a symmetric cryptographic algorithm to communicate securely; they need the same key. Alice generates a key using a random-key generator. Now she has to give it to Bob—securely. If Alice can meet Bob somewhere (a back alley, a windowless room, or one of Jupiter's moons), she can give him a copy of the key. Otherwise, they have a problem. Public-key cryptography solves the problem nicely and with a minimum of prearrangement, but these techniques are not always available (see Section 3.1). Some systems use alternate channels known to be secure. Alice could send Bob the key with a trusted messenger. She could send it by certified mail or via an overnight delivery service. She could set up another communications channel with Bob and hope no one is eavesdropping on that one.

Alice could send Bob the symmetric key over their communications channel—the one they are going to encrypt. This is foolish; if the channel warrants encryption, sending the encryption key in the clear over the same channel guarantees that anyone eavesdropping on the channel can decrypt all communications.

The X9.17 standard [55] specifies two types of keys: key-encryption keys and data keys. Key-Encryption Keys encrypt other keys for distribution. Data Keys encrypt message traffic. These key-encrypting keys have to be distributed manually (although they can be secured in a tamperproof device, like a smart card), but only seldomly. Data keys are distributed more often. More details are in [75]. This two-tiered key concept is used a lot in key distribution.

Another solution to the distribution problem splits the key into several different parts (see Section 3.6) and sends each of those parts over a different channel. One part could be sent over the telephone, one by mail, one by overnight delivery service, one by carrier pigeon, and so on. (see Figure 8.2). Since an adversary could collect all but one of the parts and still have no idea what the key is, this method will work in all but extreme cases. Section 3.6 discusses schemes for splitting a key into several parts. Alice could even use a secret sharing scheme (see Section 3.7), allowing Bob to reconstruct the key if some of the shares are lost in transmission.

Alice sends Bob the key-encryption key securely, either by a face-to-face meeting or the splitting technique just discussed. Once Alice and Bob both have the key-encryption key, Alice can send Bob daily data keys over the same communications channel. Alice encrypts each data key with the key-encryption key. Since the amount of traffic being encrypted with the key-encryption key is low, it does not have to be changed as often. However, since compromise of the key-encryption key could compromise every message encrypted with every key that was encrypted with the key-encryption key, it must be stored securely.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Key Distribution in Large Networks

Key-encryption keys shared by pairs of users work well in small networks, but can quickly get cumbersome if the networks become large. Since every pair of users must exchange keys, the total number of key exchanges required in an n-person network is n(n – 1)/2.

In a six-person network, 15 key exchanges are required. In a 1000-person network, nearly 500,000 key exchanges are required. In these cases, creating a central key server (or servers) makes the operation much more efficient.

Alternatively, any of the symmetric-cryptography or public-key-cryptography protocols in Section 3.1 provides for secure key distribution.

Figure 8.2 Key distribution via parallel channels.

8.4 Verifying Keys

When Bob receives a key, how does he know it came from Alice and not from someone pretending to be Alice? If Alice gives it to him when they are face-to-face, it's easy. If Alice sends her key via a trusted courier, then Bob has to trust the courier. If the key is encrypted with a key-encryption key, then Bob has to trust the fact that only Alice has that key. If Alice uses a digital signature protocol to sign the key, Bob has to trust the public-key database when he verifies that signature. (He also has to trust that Alice has kept her key secure.) If a Key Distribution Center (KDC) signs Alice's public key, Bob has to trust that his copy of the KDC's public key has not been tampered with.

In the end, someone who controls the entire network around Bob can make him think whatever he likes. Mallory could send an encrypted and signed message purporting to be from Alice. When Bob tried to access the public-key database to verify Alice's signature, Mallory could substitute his own public key. Mallory could invent his own false KDC and exchange the real KDC's public key for his own creation. Bob wouldn't be the wiser.

Some people have used this argument to claim that public-key cryptography is useless. Since the only way for Alice and Bob to ensure that their keys have not been tampered with is to meet face-to-face, public-key cryptography doesn't enhance security at all.

This view is naïve. It is theoretically true, but reality is far more complicated. Public-key cryptography, used with digital signatures and trusted KDCs, makes it much more difficult to substitute one key for another. Bob can never be absolutely certain that Mallory isn't controlling his entire reality, but Bob can be confident that doing so requires more resources than most real-world Mallorys have access to.

Bob could also verify Alice's key over the telephone, where he can hear her voice. Voice recognition is a really good authentication scheme. If it's a public key, he can safely recite it in public. If it's a secret key, he can use a one-way hash function to verify the key. Both PGP (see Section 24.12) and the AT&T TSD (see Section 24.18) use this kind of key verification.

Sometimes, it may not even be important to verify exactly whom a public key belongs to. It may be necessary to verify that it belongs to the same person to whom it belonged last year. If someone sends a signed withdrawal message to a bank, the bank does not have to be concerned with who withdraws the money, only whether it is the same person who deposited the money in the first place.

Error Detection during Key Transmission

Sometimes keys get garbled in transmission. Since a garbled key can mean megabytes of undecryptable ciphertext, this is a problem. All keys should be transmitted with some kind of error detection and correction bits. This way errors in transmission can be easily detected and, if required, the key can be resent.

One of the most widely used methods is to encrypt a constant value with the key, and to send the first 2 to 4 bytes of that ciphertext along with the key. At the receiving end, do the same thing. If the encrypted constants match, then the key has been transmitted without error. The chance of an undetected error ranges from one in 216 to one in 232.

Key-error Detection during Decryption

Sometimes the receiver wants to check if a particular key he has is the correct symmetric decryption key. If the plaintext message is something like ASCII, he can try to decrypt and read the message. If the plaintext is random, there are other tricks.

The naïve approach is to attach a verification block: a known header to the plaintext message before encryption. At the receiving end, Bob decrypts the header and verifies that it is correct. This works, but it gives Eve a known plaintext to help cryptanalyze the system. It also makes attacks against short-key ciphers like DES and all exportable ciphers easy. Precalculate the checksum once for each key, then use that checksum to determine the key in any message you intercept after that. This is a feature of any key checksum that doesn't include random or at least different data in each checksum. It's very similar in concept to using salt when generating keys from passphrases.

Here's a better way to do this [821]:

(1) Generate an IV (not the one used for the message).
(2) Use that IV to generate a large block of bits: say, 512.
(3) Hash the result.
(4) Use the same fixed bits of the hash, say 32, for the key checksum.

This gives Eve some information, but very little. If she tries to use the low 32 bits of the final hash value to mount a brute-force attack, she has to do multiple encryptions plus a hash per candidate key; brute-force on the key itself would be quicker.

She also gets no known-plaintext values to try out, and even if she manages to choose our random value for us, she never gets a chosen-plaintext out of us, since it goes through the hash function before she sees it.

8.5 Using Keys

Software encryption is scary. Gone are the days of simple microcomputers under the control of single programs. Now there's Macintosh System 7, Windows NT, and UNIX. You can't tell when the operating system will suspend the encryption application in progress, write everything to disk, and take care of some pressing task. When the operating system finally gets back to encrypting whatever is being encrypted, everything will look just fine. No one will ever realize that the operating system wrote the encryption application to disk, and that it wrote the key along with it. The key will sit on the disk, unencrypted, until the computer writes over that area of memory again. It could be minutes or it could be months. It could even be never; the key could still be sitting there when an adversary goes over the hard drive with a fine-tooth comb. In a preemptive, multitasking environment, you can set your encryption operation to a high enough priority so it will not be interrupted. This would mitigate the risk. Even so, the whole thing is dicey at best.

Hardware implementations are safer. Many encryption devices are designed to erase the key if tampered with. For example, the IBM PS/2 encryption card has an epoxy unit containing the DES chip, battery, and memory. Of course, you have to trust the hardware manufacturer to implement the feature properly.

Some communications applications, such as telephone encryptors, can use session keys. A session key is a key that is just used for one communications session—a single telephone conversation—and then discarded. There is no reason to store the key after it has been used. And if you use some key-exchange protocol to transfer the key from one conversant to the other, the key doesn't have to be stored before it is used either. This makes it far less likely that the key might be compromised.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Controlling Key Usage

In some applications it may be desirable to control how a session key is used. Some users may need session keys only for encryption or only for decryption. Session keys might only be authorized for use on a certain machine or at a certain time. One scheme to handle these sorts of restrictions attaches a Control Vector (CV) to the key; the control vector specifies the uses and restrictions for that key (see Section 24.1) [1025,1026]. This CV is hashed and XORed with a master key; the result is used as an encryption key to encrypt the session key. The resultant encrypted session key is then stored with the CV. To recover the session key, hash the CV and XOR it with the master key, and use the result to decrypt the encrypted session key.

The advantages of this scheme are that the CV can be of arbitrary length and that it is always stored in the clear with the encrypted key. This scheme assumes quite a bit about tamperproof hardware and the inability of users to get at the keys directly. This system is discussed further in Sections 24.1 and 24.8.

8.6 Updating Keys

Imagine an encrypted data link where you want to change keys daily. Sometimes it's a pain to distribute a new key every day. An easier solution is to generate a new key from the old key; this is sometimes called key updating.

All it takes is a one-way function. If Alice and Bob share the same key and they both operate on it using the same one-way function, they will get the same result. Then they can take the bits they need from the results to create the new key.

Key updating works, but remember that the new key is only as secure as the old key was. If Eve managed to get her hands on the old key, she can perform the key updating function herself. However, if Eve doesn't have the old key and is trying a ciphertext-only attack on the encrypted traffic, this is a good way for Alice and Bob to protect themselves.

8.7 Storing Keys

The least complex key storage problem is that of a single user, Alice, encrypting files for later use. Since she is the only person involved, she is the only person responsible for the key. Some systems take the easy approach: The key is stored in Alice's brain and never on the system. Alice is responsible for remembering the key and entering it every time she needs a file encrypted or decrypted.

An example of this system is IPS [881]. Users can either directly enter the 64-bit key or enter the key as a longer character string. The system then generates a 64-bit key from the character string using a key-crunching technique.

Another solution is to store the key in a magnetic stripe card, plastic key with an embedded ROM chip (called a ROM key), or smart card [556,557,455]. A user could then enter his key into the system by inserting the physical token into a special reader in his encryption box or attached to his computer terminal. While the user can use the key, he does not know it and cannot compromise it. He can use it only in the way and for the purposes indicated by the control vector.

A ROM key is a very clever idea. People understand physical keys, what they signify and how to protect them. Putting a cryptographic key in the same physical form makes storing and protecting that key more intuitive.

This technique is made more secure by splitting the key into two halves, storing one half in the terminal and the other half in the ROM key. The U.S. government's STU-III secure telephone works this way. Losing the ROM key does not compromise the cryptographic key—change that key and everything is back to normal. The same is true with the loss of the terminal. This way, compromising either the ROM key or the system does not compromise the cryptographic key—an adversary must have both parts.

Hard-to-remember keys can be stored in encrypted form, using something similar to a key-encryption key. For example, an RSA private key could be encrypted with a DES key and stored on disk. To recover the RSA key, the user has to type in the DES key to a decryption program.

If the keys are generated deterministically (with a cryptographically secure pseudo-random-sequence generator), it might be easier to regenerate the keys from an easy-to-remember password every time they are required.

Ideally, a key should never appear unencrypted outside the encryption device. This isn't always possible, but it is a worthy goal.

8.8 Backup Keys

Alice is the chief financial officer at Secrets, Ltd.—"We don't tell you our motto." Like any good corporate officer, she follows the company's security guidelines and encrypts all her data. Unfortunately, she ignores the company's street-crossing guidelines and gets hit by a truck. What does the company's president, Bob, do?

Unless Alice left a copy of her key, he's in deep trouble. The whole point of encryption is to make files unrecoverable without the key. Unless Alice was a moron and used lousy encryption software, her files are gone forever.

Bob can avoid this in several ways. The simplest is sometimes called key escrow (see Section 4.14): He requires all employees to write their keys on paper and give them to the company's security officer, who will lock them in a safe somewhere (or encrypt them all with a master key). Now, when Alice is bowled over on the Interstate, Bob can ask his security officer for her key. Bob should make sure to have the combination to the safe himself as well; otherwise, if the security officer is run over by another truck, Bob will be out of luck again.

The problem with this key management system is that Bob has to trust his security officer not to misuse everyone's keys. Even more significantly, all the employees have to trust the security officer not to misuse their keys. A far better solution is to use a secret-sharing protocol (see Section 3.7).

When Alice generates a key, she also divides up that key into some number of pieces. She then sends each piece—encrypted, of course—to a different company officer. None of those pieces alone is the key, but someone can gather all the pieces together and reconstruct the key. Now Alice is protected against any one malicious person, and Bob is protected against losing all of Alice's data after her run-in with the truck. Or, she could just store the different pieces, encrypted with each of the officer's different public keys, on her own hard disk. That way, no one gets involved with key management until it becomes necessary.

Another backup scheme [188] uses smart cards (see Section 24.13) for the temporary escrow of keys. Alice can put the key to secure her hard drive onto the smart card and give it to Bob while she is away. Bob can use the card to get into Alice's hard drive, but because the key is stored in the card Bob cannot learn it. And the system is bilaterally auditable: Bob can verify that the key will open Alice's drive, and when Alice returns she can verify if Bob has used the key and how many times.

Such a scheme makes no sense for data transmission. On a secure telephone, the key should exist for the length of the call and no longer. For data storage, as just described, key escrow can be a good idea. I've lost about one key every five years, and my memory is better than most. If 200 million people were using cryptography, that same rate would equal 40 million lost keys per year. I keep copies of my house keys with a neighbor because I may lose mine. If house keys were like cryptographic keys, and I lost them, I could never get inside and recover my possessions, ever again. Just as I keep off-site backups of my data, it makes sense to keep backups of my data-encryption keys.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

8.9 Compromised Keys

All of the protocols, techniques, and algorithms in this book are secure only if the key (the private key in a public-key system) remains secret. If Alice's key is lost, stolen, printed in the newspaper, or otherwise compromised, then all her security is gone.

If the compromised key was for a symmetric cryptosystem, Alice has to change her key and hope the actual damage was minimal. If it was a private key, she has bigger problems; her public key is probably on servers all over the network. And if Eve gets access to Alice's private key, she can impersonate her on the network: reading encrypted mail, signing correspondence, entering into contracts, and so forth. Eve can, effectively, become Alice.

It is vital that news of a private key's compromise propagate quickly throughout the network. Any databases of public keys must immediately be notified that a particular private key has been compromised, lest some unsuspecting person encrypt a message in that compromised key.

One hopes Alice knows when her key was compromised. If a KDC is managing the keys, Alice should notify it that her key has been compromised. If there is no KDC, then she should notify all correspondents who might receive messages from her. Someone should publicize the fact that any message received after her key was lost is suspect, and that no one should send messages to Alice with the associated public key. The application should be using some sort of timestamp, and then users can determine which messages are legitimate and which are suspect.

If Alice doesn't know exactly when her key was compromised, things are more difficult. Alice may want to back out of a contract because the person who stole the key signed it instead of her. If the system allows this, then anyone can back out of a contract by claiming that his key was compromised before it was signed. It has to be a matter for an adjudicator to decide.

This is a serious problem and brings to light the dangers of Alice tying all of her identity to a single key. It would be better for Alice to have different keys for different applications—just as she has different physical keys in her pocket for different locks. Other solutions to this problem involve biometrics, limits on what can be done with a key, time delays, and countersigning.

These procedures and tips are hardly optimal, but are the best we can do. The moral of the story is to protect keys, and protect private keys above all else.

8.10 Lifetime of Keys

No encryption key should be used for an indefinite period. It should expire automatically like passports and licenses. There are several reasons for this:

— The longer a key is used, the greater the chance that it will be compromised. People write keys down; people lose them. Accidents happen. If you use the same key for a year, there's a far greater chance of compromise than if you use it for a day.
— The longer a key is used, the greater the loss if the key is compromised. If a key is used only to encrypt a single budgetary document on a file server, then the loss of the key means only the compromise of that document. If the same key is used to encrypt all the budgetary information on the file server, then its loss is much more devastating.
— The longer a key is used, the greater the temptation for someone to spend the effort necessary to break it—even if that effort is a brute-force attack. Breaking a key shared between two military units for a day would enable someone to read and fabricate messages between those units for that day. Breaking a key shared by an entire military command structure for a year would enable that same person to read and fabricate messages throughout the world for a year. In our budget-conscious, post-Cold War world, which key would you choose to attack?
— It is generally easier to do cryptanalysis with more ciphertext encrypted with the same key.

For any cryptographic application, there must be a policy that determines the permitted lifetime of a key. Different keys may have different lifetimes. For a connection-based system, like a telephone, it makes sense to use a key for the length of the telephone call and to use a new one with each call.

Systems on dedicated communications channels are not as obvious. Keys should have relatively short lifetimes, depending on the value of the data and the amount of data encrypted during a given period. The key for a gigabit-per-second communications link might have to be changed more often than the key for a 9600-baud modem link. Assuming there is an efficient method of transferring new keys, session keys should be changed at least daily.

Key-encryption keys don't have to be replaced as frequently. They are used only occasionally (roughly once per day) for key exchange. This generates little ciphertext for a cryptanalyst to work with, and the corresponding plaintext has no particular form. However, if a key-encryption key is compromised, the potential loss is extreme: all communications encrypted with every key encrypted with the key-encryption key. In some applications, key-encryption keys are replaced only once a month or once a year. You have to balance the inherent danger in keeping a key around for a while with the inherent danger in distributing a new one.

Encryption keys used to encrypt data files for storage cannot be changed often. The files may sit encrypted on disk for months or years before someone needs them again. Decrypting them and re-encrypting them with a new key every day doesn't enhance security in any way; it just gives a cryptanalyst more to work with. One solution might be to encrypt each file with a unique file key, and then encrypt all the file keys with a key-encryption key. The key-encryption key should then be either memorized or stored in a secure location, perhaps in a safe somewhere. Of course, losing this key would mean losing all the individual file keys.

Private keys for public-key cryptography applications have varying lifetimes, depending on the application. Private keys used for digital signatures and proofs of identity may have to last years (even a lifetime). Private keys used for coin-flipping protocols can be discarded immediately after the protocol is completed. Even if a key's security is expected to last a lifetime, it may be prudent to change the key every couple of years. The private keys in many networks are good only for two years; after that the user must get a new private key. The old key would still have to remain secret, in case the user needed to verify a signature from that period. But the new key would be used to sign new documents, reducing the number of signed documents a cryptanalyst would have for an attack.

8.11 Destroying Keys

Given that keys must be replaced regularly, old keys must be destroyed. Old keys are valuable, even if they are never used again. With them, an adversary can read old messages encrypted with those keys [65].

Keys must be destroyed securely (see Section 10.9). If the key is written on paper, the paper should be shredded or burned. Be careful to use a high-quality shredder; many lousy shredders are on the market. Algorithms in this book are secure against brute-force attacks costing millions of dollars and taking millions of years. If an adversary can recover your key by taking a bag of shredded documents from your trash and paying 100 unemployed workers in some backwater country ten cents per hour for a year to piece the shredded pages together, that would be $26,000 well spent.

If the key is in a hardware EEPROM, the key should be overwritten multiple times. If the key is in a hardware EPROM or PROM, the chip should be smashed into tiny bits and scattered to the four winds. If the key is stored on a computer disk, the actual bits of the storage should be overwritten multiple times (see Section 10.9) or the disk should be shredded.

A potential problem is that, in a computer, keys can be easily copied and stored in multiple locations. Any computer that does its own memory management, constantly swapping programs in and out of memory, exacerbates the problem. There is no way to ensure that successful key erasure has taken place in the computer, especially if the computer's operating system controls the erasure process. The more paranoid among you should consider writing a special erasure program that scans all disks looking for copies of the key's bit pattern on unused blocks and then erases those blocks. Also remember to erase the contents of any temporary, or "swap," files.

8.12 Public-Key Key Management

Public-key cryptography makes key management easier, but it has its own unique problems. Each person has only one public key, regardless of the number of people on the network. If Alice wants to send a message to Bob, she has to get Bob's public key. She can go about this several ways:

— She can get it from Bob.
— She can get it from a centralized database.
— She can get it from her own private database.

Section 2.5 discussed a number of possible attacks against public-key cryptography, based on Mallory substituting his key for Bob's. The scenario is that Alice wants to send a message to Bob. She goes to the public-key database and gets Bob's public key. But Mallory, who is sneaky, has substituted his own key for Bob's. (If Alice asks Bob directly, Mallory has to intercept Bob's transmission and substitute his key for Bob's.) Alice encrypts her message in Mallory's key and sends it to Bob. Mallory intercepts the message, decrypts it, and reads it. He re-encrypts it with Bob's real key and sends it on to Bob. Neither Alice nor Bob is the wiser.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Public-key Certificates

A public-key certificate is someone's public key, signed by a trustworthy person. Certificates are used to thwart attempts to substitute one key for another [879]. Bob's certificate, in the public-key database, contains a lot more than his public key. It contains information about Bob—his name, address, and so on—and it is signed by someone Alice trusts: Trent (usually known as a certification authority, or CA). By signing both the key and the information about Bob, Trent certifies that the information about Bob is correct and that the public key belongs to Bob. Alice checks Trent's signature and then uses the public key, secure in the knowledge that it is Bob's and no one else's. Certificates play an important role in a number of public-key protocols such as PEM [825] (see Section 24.10) and X.509 [304] (see Section 24.9).

A complicated noncryptographic issue surrounds this type of system. What is the meaning of certification? Or, to put it another way, who is trusted to issue certificates to whom? Anyone may sign anyone else's certificate, but there needs to be some way to filter out questionable certificates: for example, certificates for employees of one company signed by the CA of another company. Normally, a certification chain transfers trust: A single trusted entity certifies trusted agents, trusted agents certify company CAs, and company CAs certify their employees.

Here are some more things to think about:

— What level of trust in someone's identity is implied by his certificate?
— What are the relationships between a person and the CA that certified his public key, and how can those relationships be implied by the certificate?
— Who can be trusted to be the "single trusted entity" at the top of the certification chain?
— How long can a certification chain be?

Ideally, Bob would follow some kind of authentication procedure before the CA signs his certificate. Additionally, some kind of timestamp or an indication of the certificate's validity period is important to guard against compromised keys [461].

Timestamping is not enough. Keys may be invalidated before they have expired, either through compromise or for administrative reasons. Hence, it is important the CA keep a list of invalid certificates, and for users to regularly check that list. This key revocation problem is still a difficult one to solve.

And one public-key/private-key pair is not enough. Certainly any good implementation of public-key cryptography needs separate keys for encryption and digital signatures. This separation allows for different security levels, expiration times, backup procedures, and so on. Someone might sign messages with a 2048-bit key stored on a smart card and good for twenty years, while they might use a 768-bit key stored in the computer and good for six months for encryption.

And a single pair of encryption and signature keys isn't enough, either. A private key authenticates a relationship as well as an identity, and people have more than one relationship. Alice might want to sign one document as Alice the individual, another as Alice, vice-president of Monolith, Inc., and a third as Alice, president of her community organization. Some of these keys are more valuable than others, so they can be better protected. Alice might have to store a backup of her work key with the company's security officer; she doesn't want the company to have a copy of the key she signed her mortgage with. Just as Alice has multiple physical keys in her pocket, she is going to have multiple cryptographic keys.

Distributed Key Management

In some situations, this sort of centralized key management will not work. Perhaps there is no CA whom Alice and Bob both trust. Perhaps Alice and Bob trust only their friends. Perhaps Alice and Bob trust no one.

Distributed key management, used in PGP (see Section 24.12), solves this problem with introducers. Introducers are other users of the system who sign their friends' public keys. For example, when Bob generates his public key, he gives copies to his friends: Carol and Dave. They know Bob, so they each sign Bob's key and give Bob a copy of the signature. Now, when Bob presents his key to a stranger, Alice, he presents it with the signatures of these two introducers. If Alice also knows and trusts Carol, she has reason to believe that Bob's key is valid. If she knows and trusts Carol and Dave a little, she has reason to believe that Bob's key is valid. If she doesn't know either Carol or Dave, she has no reason to trust Bob's key.

Over time, Bob will collect many more introducers. If Alice and Bob travel in similar circles, the odds are good that Alice will know one of Bob's introducers. To prevent against Mallory's substituting one key for another, an introducer must be sure that Bob's key belongs to Bob before he signs it. Perhaps the introducer should require the key be given face-to-face or verified over the telephone.

The benefit of this mechanism is that there is no CA that everyone has to trust. The down side is that when Alice receives Bob's public key, she has no guarantee that she will know any of the introducers and therefore no guarantee that she will trust the validity of the key.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 9
Algorithm Types and Modes

There are two basic types of symmetric algorithms: block ciphers and stream ciphers. Block ciphers operate on blocks of plaintext and ciphertext—usually of 64 bits but sometimes longer. Stream ciphers operate on streams of plaintext and ciphertext one bit or byte (sometimes even one 32-bit word) at a time. With a block cipher, the same plaintext block will always encrypt to the same ciphertext block, using the same key. With a stream cipher, the same plaintext bit or byte will encrypt to a different bit or byte every time it is encrypted.

A cryptographic mode usually combines the basic cipher, some sort of feedback, and some simple operations. The operations are simple because the security is a function of the underlying cipher and not the mode. Even more strongly, the cipher mode should not compromise the security of the underlying algorithm.

There are other security considerations: Patterns in the plaintext should be concealed, input to the cipher should be randomized, manipulation of the plaintext by introducing errors in the ciphertext should be difficult, and encryption of more than one message with the same key should be possible. These will be discussed in detail in the next sections.

Efficiency is another consideration. The mode should not be significantly less efficient than the underlying cipher. In some circumstances it is important that the ciphertext be the same size as the plaintext.

A third consideration is fault-tolerance. Some applications need to parallelize encryption or decryption, while others need to be able to preprocess as much as possible. In still others it is important that the decrypting process be able to recover from bit errors in the ciphertext stream, or dropped or added bits. As we will see, different modes have different subsets of these characteristics.

9.1 Electronic Codebook Mode

Electronic codebook (ECB) mode is the most obvious way to use a block cipher: A block of plaintext encrypts into a block of ciphertext. Since the same block of plaintext always encrypts to the same block of ciphertext, it is theoretically possible to create a code book of plaintexts and corresponding ciphertexts. However, if the block size is 64 bits, the code book will have 264 entries—much too large to precompute and store. And remember, every key has a different code book.

This is the easiest mode to work with. Each plaintext block is encrypted independently. You don't have to encrypt a file linearly; you can encrypt the 10 blocks in the middle first, then the blocks at the end, and finally the blocks in the beginning. This is important for encrypted files that are accessed randomly, like a database. If a database is encrypted with ECB mode, then any record can be added, deleted, encrypted, or decrypted independently of any other record—assuming that a record consists of a discrete number of encryption blocks. And processing is parallizeable; if you have multiple encryption processors, they can encrypt or decrypt different blocks without regard for each other.

The problem with ECB mode is that if a cryptanalyst has the plaintext and ciphertext for several messages, he can start to compile a code book without knowing the key. In most real-world situations, fragments of messages tend to repeat. Different messages may have bit sequences in common. Computer-generated messages, like electronic mail, may have regular structures. Messages may be highly redundant or have long strings of zeros or spaces.

If a cryptanalyst learns that the plaintext block "5e081bc5" encrypts to the ciphertext block "7ea593a4," he can immediately decrypt that ciphertext block whenever it appears in another message. If the encrypted messages have a lot of redundancies, and these tend to show up in the same places in different messages, a cryptanalyst can get a lot of information. He can mount statistical attacks on the underlying plaintext, irrespective of the strength of the block cipher.

This vulnerability is greatest at the beginning and end of messages, where well-defined headers and footers contain information about the sender, receiver, date, and so on. This problem is sometimes called stereotyped beginnings and stereotyped endings.

On the plus side, there is no security risk in encrypting multiple messages with the same key. In fact, each block can be looked at as a separate message encrypted with the same key. Bit errors in the ciphertext, when decrypted, will cause the entire plaintext block to decrypt incorrectly but will not affect the rest of the plaintext. However, if a ciphertext bit is accidentally lost or added, all subsequent ciphertext will decrypt incorrectly unless there is some kind of frame structure to realign the block boundaries.

Padding

Most messages don't divide neatly into 64-bit (or whatever size) encryption blocks; there is usually a short block at the end. ECB requires 64-bit blocks. Padding is the way to deal with this problem.

Pad the last block with some regular pattern—zeros, ones, alternating ones and zeros—to make it a complete block. If you need to delete the padding after decryption, add the number of padding bytes as the last byte of the last block. For example, assume the block size is 64 bits and the last block consists of 3 bytes (24 bits). Five bytes of padding are required to make the last block 64 bits; add 4 bytes of zeros and a final byte with the number 5. After decryption, delete the last 5 bytes of the last decryption block. For this method to work correctly, every message must be padded. Even if the plaintext ends on a block boundary, you have to pad one complete block. Otherwise, you can use an end-of-file character to denote the final plaintext byte, and then pad after that character.

Figure 9.1 is an alternative, called ciphertext stealing [402]. Pn-1 is the last full plaintext block and Pn is the final, short, plaintext block. Cn-1 is the last full ciphertext block and Cn is the final, short, ciphertext block. C' is just an intermediate result and is not part of the transmitted ciphertext.

9.2 Block Replay

A more serious problem with ECB mode is that an adversary could modify encrypted messages without knowing the key, or even the algorithm, in such a way as to fool the intended recipient. This problem was first discussed in [291].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

To illustrate the problem, consider a money transfer system that moves money between accounts in different banks. To make life easier for the bank's computer systems, banks agree on a standard message format for money transfer that looks like this:

     Bank One: Sending         1.5 blocks
     Bank Two: Receiving       1.5 blocks
     Depositor's Name          6 blocks
     Depositor's Account       2 blocks
     Amount of Deposit         1 block

A block corresponds to an 8-byte encryption block. The messages are encrypted using some block algorithm in ECB mode.

Figure 9.1 Ciphertext stealing in ECB mode.

Mallory, who is listening on the communications line between two banks, Bank of Alice and Bank of Bob, can use this information to get rich. First, he sets up his computer to record all of the encrypted messages from Bank of Alice to Bank of Bob. Then, he transfers $100 from Bank of Alice to his account in Bank of Bob. Later, he does it again. Using his computer, he examines the recorded messages looking for a pair of identical messages. These messages are the ones authorizing the $100 transfers to his account. If he finds more than one pair of identical messages (which is most likely in real life), he does another money transfer and records those results. Eventually he can isolate the message that authorized his money transaction.

Now he can insert that message into the communications link at will. Every time he sends the message to Bank of Bob, another $100 will be credited to his account. When the two banks reconcile their transfers (probably at the end of the day), they will notice the phantom transfer authorizations; but if Mallory is clever, he will have already withdrawn the money and headed for some banana republic without extradition laws. And he probably did his scam with dollar amounts far larger than $100, and with lots of different banks.

At first glance, the banks could easily prevent this by adding a timestamp to their messages.

    Date/Time Stamp:                1 block
    Bank One: Sending               1.5 blocks
    Bank Two: Receiving             1.5 blocks
    Depositor's Name                6 blocks
    Depositor's Account             2 blocks
    Amount of Deposit               1 block

Two identical messages would be easy to spot using this system. Still, using a technique called block replay, Mallory can still get rich. Figure 9.2 shows that Mallory can pick out the eight ciphertext blocks that correspond to his own name and account number: blocks 5 through 12. A diabolical laugh is appropriate at this point, because Mallory is now ready.

He intercepts random messages from Bank of Alice to Bank of Bob and replaces blocks 5 through 12 in the message with the bytes that correspond to his name and account number. Then he sends them on to Bank of Bob. He doesn't have to know who the original depositor was; he doesn't even have to know what the amount was (although, he could correlate the messages he doctored with the various deposits into his account and determine the encrypted blocks corresponding to some dollar amount). He simply changes the name and account numbers to his own and watches his account balance grow. (I suppose Mallory has to be careful not to modify a withdrawal message, but assume for the moment that each is a different length or something.)

Figure 9.2 Encryption blocks for an example record.

This will take longer than a day for the banks to catch. When they reconcile their transfers at the end of the day, everything will match. It probably won't be until one of the legitimate depositors notices that his deposits are not being credited, or when someone flags unusual activity in Mallory's account, that the banks will figure out the scam. Mallory isn't stupid, and by then he will have closed his account, changed his name, and bought a villa in Argentina.

Banks can minimize the problem by changing their keys frequently, but this only means that Mallory is going to have to work more quickly. Adding a MAC, however, will also solve the problem. Even so, this is a fundamental problem with ECB mode. Mallory can remove, repeat, or interchange blocks at will. The solution is a technique called chaining.

9.3 Cipher Block Chaining Mode

Chaining adds a feedback mechanism to a block cipher: The results of the encryption of previous blocks are fed back into the encryption of the current block. In other words, each block is used to modify the encryption of the next block. Each ciphertext block is dependent not just on the plaintext block that generated it but on all the previous plaintext blocks.

In cipher block chaining (CBC) mode, the plaintext is XORed with the previous ciphertext block before it is encrypted. Figure 9.3a shows CBC encryption in action. After a plaintext block is encrypted, the resulting ciphertext is also stored in a feedback register. Before the next plaintext block is encrypted, it is XORed with the feedback register to become the next input to the encrypting routine. The resulting ciphertext is again stored in the feedback register, to be XORed with the next plaintext block, and so on until the end of the message. The encryption of each block depends on all the previous blocks.

Decryption is just as straightforward (see Figure 9.3b). A ciphertext block is decrypted normally and also saved in a feedback register. After the next block is decrypted, it is XORed with the results of the feedback register. Then the next ciphertext block is stored in the feedback register, and so on, until the end of the message.

Mathematically, this looks like:

Ci = EK(Pi ⊕ Ci-1)
Pi = Ci-1 ⊕ DK(Ci)

Figure 9.3 Cipher block chaining mode.

Initialization Vector

CBC mode forces identical plaintext blocks to encrypt to different ciphertext blocks only when some previous plaintext block is different. Two identical messages will still encrypt to the same ciphertext. Even worse, two messages that begin the same will encrypt in the same way up to the first difference.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Some messages have a common header: a letterhead, or a "From" line, or whatever. While block replay would still be impossible, this identical beginning might give a cryptanalyst some useful information.

Prevent this by encrypting random data as the first block. This block of random data is called the initialization vector (IV), initializing variable, or initial chaining value. The IV has no meaning; it's just there to make each message unique. When the receiver decrypts this block, he just uses it to fill the feedback register and otherwise ignores it. A timestamp often makes a good IV. Otherwise, use some random bits from someplace.

With the addition of IVs, identical plaintext messages encrypt to different ciphertext messages. Thus, it is impossible for an eavesdropper to attempt block replay, and more difficult for him to build a code book. While the IV should be unique for each message encrypted with the same key, it is not an absolute requirement.

The IV need not be secret; it can be transmitted in the clear with the ciphertext. If this seems wrong, consider the following argument. Assume that we have a message of several blocks: B1, B2,..., Bi. B1 is encrypted with the IV. B2 is encrypted using the ciphertext of B1 as the IV. B3 is encrypted using the ciphertext of B2 as the IV, and so on. So, if there are n blocks, there are n-1 exposed "IVs," even if the original IV is kept secret. So there's no reason to keep the IV secret; the IV is just a dummy ciphertext block—you can think of it as B0 to start the chaining.

Padding

Padding works just like ECB mode, but in some applications the ciphertext has to be exactly the same size as the plaintext. Perhaps a plaintext file has to be encrypted and then replaced in the exact same memory location. In this case, you have to encrypt the last short block differently. Assume the last block has j bits. After encrypting the last full block, encrypt the ciphertext again, select the left-most j bits of the encrypted ciphertext, and XOR that with the short block to generate the ciphertext. Figure 9.4 illustrates this.

The weakness here is that while Mallory cannot recover the last plaintext block, he can change it systematically by changing individual bits in the ciphertext. If the last few bits of the ciphertext contain essential information, this is a weakness. If the last bits simply contain housekeeping information, it isn't a problem.

Ciphertext stealing is a better way (see Figure 9.5) [402]. Pn-1 is the last full plaintext block, and Pn is the final, short, plaintext block. Cn-1 is the last full ciphertext block, and Cn is the final, short, ciphertext block. C' is just an intermediate result and is not part of the transmitted ciphertext. The benefit of this method is that all the bits of the plaintext message go through the encryption algorithm.

Error Propagation

CBC mode can be characterized as feedback of the ciphertext at the encryption end and feedforward of the ciphertext at the decryption end. This has implications having to do with errors. A single bit error in a plaintext block will affect that ciphertext block and all subsequent ciphertext blocks. This isn't significant because decryption will reverse that effect, and the recovered plaintext will have the same single error.

Ciphertext errors are more common. They can easily result from a noisy communications path or a malfunction in the storage medium. In CBC mode, a single-bit error in the ciphertext affects one block and one bit of the recovered plaintext. The block containing the error is completely garbled. The subsequent block has a 1-bit error in the same bit position as the error.

Figure 9.4 Encrypting the last short block in CBC mode.

Figure 9.5 Ciphertext stealing in CBC mode.

This property of taking a small ciphertext error and converting it into a large plaintext error is called error extension. It is a major annoyance. Blocks after the second are not affected by the error, so CBC mode is self-recovering. Two blocks are affected by an error, but the system recovers and continues to work correctly for all subsequent blocks. CBC is an example of a block cipher being used in a self-synchronizing manner, but only at the block level.

While CBC mode recovers quickly from bit errors, it doesn't recover at all from synchronization errors. If a bit is added or lost from the ciphertext stream, then all subsequent blocks are shifted one bit out of position and decryption will generate garbage indefinitely. Any cryptosystem that uses CBC mode must ensure that the block structure remains intact, either by framing or by storing data in multiple-block-sized chunks.

Security Problems

Some potential problems are caused by the structure of CBC. First, because a ciphertext block affects the following block in a simple way, Mallory can add blocks to the end of an encrypted message without being detected. Sure, it will probably decrypt to gibberish, but in some situations this is undesirable.

If you are using CBC, you should structure your plaintext so that you know where the message ends and can detect the addition of extra blocks.

Second, Mallory can alter a ciphertext block to introduce controlled changes in the following decrypted plaintext block. For example, if Mallory toggles a single ciphertext bit, the entire block will decrypt incorrectly, but the following block will have a 1-bit error in the corresponding bit position. There are situations where this is desirable. The entire plaintext message should include some kind of controlled redundancy or authentication.

Finally, although plaintext patterns are concealed by chaining, very long messages will still have patterns. The birthday paradox predicts that there will be identical blocks after 2m/2 blocks, where m is the block size. For a 64-bit block size, that's about 34 gigabytes. A message has to be pretty long before this is a problem.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

9.4 Stream Ciphers

Stream ciphers convert plaintext to ciphertext 1 bit at a time. The simplest implementation of a stream cipher is shown in Figure 9.6. A keystream generator (sometimes called a running-key generator) outputs a stream of bits: k1, k2, k3,..., ki. This keystream (sometimes called a running key) is XORed with a stream of plaintext bits, p1, p2, p3,..., pi, to produce the stream of ciphertext bits.

ci = pi ⊕ ki

At the decryption end, the ciphertext bits are XORed with an identical keystream to recover the plaintext bits.

pi = ci ⊕ ki

Since

pi ⊕ ki ⊕ ki = pi

this works nicely.

The system's security depends entirely on the insides of the keystream generator. If the keystream generator outputs an endless stream of zeros, the ciphertext will equal the plaintext and the whole operation will be worthless. If the keystream generator spits out a repeating 16-bit pattern, the algorithm will be a simple XOR with negligible security (see Section 1.4). If the keystream generator spits out an endless stream of random (not pseudo-random, but real random—see Section 2.8) bits, you have a one-time pad and perfect security.

The reality of stream cipher security lies somewhere between the simple XOR and the one-time pad. The keystream generator generates a bit stream that looks random, but is actually a deterministic stream that can be flawlessly reproduced at decryption time. The closer the keystream generator's output is to random, the harder time a cryptanalyst will have breaking it.

Figure 9.6 Stream cipher.

If, however, the keystream generator produces the same bit stream every time it is turned on, the resulting cryptosystem will be trivial to break. An example will show why.

If Eve has a ciphertext and associated plaintext, she can XOR the plaintext and the ciphertext to recover the keystream. Or, if she has two different ciphertexts encrypted with the same keystream, she can XOR them together and get two plaintext messages XORed with each other. This is easy to break, and then she can XOR one of the plaintexts with the ciphertext to get the keystream.

Now, whenever she intercepts another ciphertext message, she has the keystream bits necessary to decrypt it. In addition, she can decrypt and read any old ciphertext messages she has previously intercepted. When Eve gets a single plaintext/ciphertext pair, she can read everything.

This is why all stream ciphers have keys. The output of the keystream generator is a function of the key. Now, if Eve gets a plaintext/ciphertext pair, she can only read messages encrypted with a single key. Change the key, and the adversary is back to square one. Stream ciphers are especially useful to encrypt never-ending streams of communications traffic: a T-1 link between two computers, for example.

A keystream generator has three basic parts (see Figure 9.7). The internal state describes the current state of the keystream generator. Two keystream generators, with the same key and the same internal state, will produce the same keystream. The output function takes the internal state and generates a keystream bit. The next-state function takes the internal state and generates a new internal state.

9.5 Self-Synchronizing Stream Ciphers

For a self-synchronizing stream cipher, each keystream bit is a function of a fixed number of previous ciphertext bits [1378]. The military calls this ciphertext auto key (CTAK). The basic idea was patented in 1946 [667].

Figure 9.7 Inside a keystream generator.

Figure 9.8 shows a self-synchronizing stream cipher. The internal state is a function of the previous n ciphertext bits. The cryptographic complexity is in the output function, which takes the internal state and generates a keystream bit.

Since the internal state depends wholly on the previous n ciphertext bits, the decryption keystream generator will automatically synchronize with the encryption keystream generator after receiving n ciphertext bits.

In smart implementations of this mode, each message begins with a random header n bits long. That header is encrypted, transmitted, and then decrypted. The decryption will be incorrect, but after those n bits both keystream generators will be synchronized.

The down side of a self-synchronizing stream cipher is error propagation. For each ciphertext bit garbled in transmission, the decryption keystream generator will incorrectly produce n keystream bits. Therefore, for each ciphertext error, there will be n corresponding plaintext errors, until the garbled bit works its way out of the internal state.

Security Problems

Self-synchronizing stream ciphers are also vulnerable to a playback attack. First Mallory records some ciphertext bits. Then, at a later time, he substitutes this recording into current traffic. After some initial garbage while the receiving end resynchronizes, the old ciphertext will decrypt as normal. The receiving end has no way of knowing that this is not current data, but old data being replayed. Unless timestamps are used, Mallory can convince a bank to credit his account again and again, by replaying the same message (assuming the key hasn't been changed, of course). Other weaknesses in this type of scheme could be exploited in the cases of very frequent resynchronization [408].

Figure 9.8 A self-synchronizing keystream generator.

9.6 Cipher-Feedback Mode

Block ciphers can also be implemented as a self-synchronizing stream cipher; this is called cipher-feedback (CFB) mode. With CBC mode, encryption cannot begin until a complete block of data is received. This is a problem in some network applications. In a secure network environment, for example, a terminal must be able to transmit each character to the host as it is entered. When data has to be processed in byte-sized chunks, CBC mode just won't do.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In CFB mode, data can be encrypted in units smaller than the block size. The following example will encrypt one ASCII character at a time (this is called 8-bit CFB), but nothing is sacred about the number eight. You can encrypt data one bit at a time using 1-bit CFB, although using one complete encryption of a block cipher for a single bit seems like a whole lot of work; a stream cipher might be a better idea. (Reducing the number of rounds of the block cipher to speed things up is not recommended [1269].) You can also use 64-bit CFB, or any n-bit CFB where n is less than or equal to the block size.

Figure 9.9 shows 8-bit CFB mode working with a 64-bit block algorithm. A block algorithm in CFB mode operates on a queue the size of the input block. Initially, the queue is filled with an IV, as in CBC mode. The queue is encrypted and the left-most eight bits of the result are XORed with the first 8-bit character of the plaintext to become the first 8-bit character of the ciphertext. This character can now be transmitted. The same eight bits are also moved to the right-most eight bit positions of the queue, and all the other bits move eight to the left. The eight left-most bits are discarded. Then the next plaintext character is encrypted in the same manner. Decryption is the reverse of this process. On both the encryption and the decryption side, the block algorithm is used in its encryption mode.

Figure 9.9 8-bit cipher-feedback mode.

Figure 9.10 n-bit CFB with an n-bit algorithm.

If the block size of the algorithm is n, then n-bit CFB looks like (see Figure 9.10):

Ci = Pi ⊕ EK(Ci-1)
Pi = Ci ⊕ EK(Ci-1)

Like CBC mode, CFB mode links the plaintext characters together so that the ciphertext depends on all the preceding plaintext.

Initialization Vector

To initialize the CFB process, the input to the block algorithm must be initialized with an IV. Like the IV used in CBC mode, it need not be secret.

The IV must be unique, though. (This is different from the IV in CBC mode, which should be unique but does not have to be.) If the IV in CFB is not unique, a cryptanalyst can recover the corresponding plaintext. The IV must be changed with every message. It can be a serial number, which increments after each message and does not repeat during the lifetime of the key. For data encrypted for storage, it can be a function of the index used to look up the data.

Error Propagation

With CFB mode, an error in the plaintext affects all subsequent ciphertext and reverses itself at decryption. An error in the ciphertext is more interesting. The first effect of a single-bit error in the ciphertext is to cause a single error in the plaintext. After that, the error enters the shift register, where it causes ciphertext to be garbled until it falls off the other end of the register. In 8-bit CFB mode, 9 bytes of decrypted plaintext are garbled by a single-bit error in the ciphertext. After that, the system recovers and all subsequent ciphertext is decrypted correctly. In general, in n-bit CFB a single ciphertext error will affect the decryption of the current and following m/n-1 blocks, where m is the block size.

One subtle problem with this kind of error propagation is that if Mallory knows the plaintext of a transmission, he can toggle bits in a given block and make it decrypt to whatever he wants. The next block will decrypt to garbage, but the damage may already be done. And he can change the final bits of a message without detection.

CFB is self-recovering with respect to synchronization errors as well. The error enters the shift register, where it garbles 8 bytes of data until it falls off the other end. CFB is an example of block cipher being used as a self-synchronizing stream cipher (at the block level).

9.7 Synchronous Stream Ciphers

In a synchronous stream cipher the keystream is generated independent of the message stream. The military calls this Key Auto-Key (KAK). On the encryption side, a keystream generator spits out keystream bits, one after the other. On the decryption side, another keystream generator spits out the identical keystream bits, one after the other. This works, as long as the two keystream generators are synchronized. If one of them skips a cycle or if a ciphertext bit gets lost during transmission, then every ciphertext character after the error will decrypt incorrectly.

If this happens, the sender and receiver must resynchronize their keystream generators before they can proceed. Frustrating matters even further, they must do this in such a way as to ensure that no part of the keystream is repeated, so the obvious solution of resetting the keystream generator to an earlier state won't work.

On the plus side, synchronous ciphers do not propagate transmission errors. If a bit is garbled during transmission, which is far more likely than a bit being lost altogether, then only the garbled bit will be decrypted incorrectly. All preceding and subsequent bits will be unaffected.

Since a keystream generator must generate the same output on both the encryption and decryption ends, it must be deterministic. Because it is implemented in a finite-state machine (i.e., a computer), the sequence will eventually repeat. These keystream generators are called periodic. Except for one-time pads, all keystream generators are periodic.

The keystream generator must have a long period, one far longer than the number of bits the generator will output between key changes. If the period is less than the plaintext, then different parts of the plaintext will be encrypted the same way—a severe weakness. If a cryptanalyst knows a piece of the plaintext, he can recover a piece of the keystream and use that to recover more of the plaintext. Even if the analyst only has the ciphertext, he can XOR the sections encrypted with the same keystream and get the XOR of plaintext with plaintext. This is just the simple XOR algorithm with a very long key.

How long a period is long enough depends on the application. A keystream generator encrypting a continuous T-1 link will encrypt 237 bits per day. The keystream generator's period must be orders of magnitude larger than that, even if the key is changed daily. If the period is long enough, you might only have to change the key weekly or even monthly.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Synchronous stream ciphers also protect against any insertions and deletions in the ciphertext, because these cause a loss of synchronization and will be immediately detected. They do not, however, fully protect against bit toggling. Like block ciphers in CFB mode, Mallory can toggle individual bits in the stream. If he knows the plaintext, he can make those bits decrypt to whatever he wants. Subsequent bits will decrypt correctly, so in certain applications Mallory can still do considerable damage.

Insertion Attack

Synchronous stream ciphers are vulnerable to an insertion attack [93]. Mallory has recorded a ciphertext stream, but does not know the plaintext or the keystream used to encrypt the plaintext.

     Original plaintext:    p1 p2 p3 p4 ...
     Original keystream:    k1 k2 k3 k4 ...
     Original ciphertext:   c1 c2 c3 c4 ...

Mallory inserts a single known bit, p', into the plaintext after p1 and then manages to get the modified plaintext encrypted with the same keystream. He records the resultant new ciphertext:

New plaintext:           p1    p'   p2    p3    p4    ...
Original keystream:      k1    k2    k3    k4    k5    ...
Updated ciphertext:      c1    c'2   c'3   c'4   c'5      ...

Assuming he knows the value of p', he can determine the entire plaintext after that bit from the original ciphertext and new ciphertext:

k2 = c'2 ⊕ p', and then p2 = c2 ⊕ k2
k3 = c'3 ⊕ p2, and then p3 = c3 ⊕ k3
k4 = c'4 ⊕ p3, and then p4 = c4 ⊕ k4

Mallory doesn't even have to know the exact position in which the bit was inserted; he can just compare the original and updated ciphertexts to see where they begin to differ. To protect against this attack, never use the same keystream to encrypt two different messages.

9.8 Output-Feedback Mode

Output-feedback (OFB) mode is a method of running a block cipher as a synchronous stream cipher. It is similar to CFB mode, except that n bits of the previous output block are moved into the right-most positions of the queue (see Figure 9.11). Decryption is the reverse of this process. This is called n-bit OFB. On both the encryption and the decryption sides, the block algorithm is used in its encryption mode. This is sometimes called internal feedback, because the feedback mechanism is independent of both the plaintext and the ciphertext streams [291].

If n is the block size of the algorithm, then n-bit OFB looks like (see Figure 9.12):

Ci = Pi ⊕ Si; Si = EK(Si-1)
Pi = Ci ⊕ S1; Si = EK(Si-1)

Figure 9.11 8-bit output-feedback mode.

Si is the state, which is independent of either the plaintext or the ciphertext.

One nice feature of OFB mode is that most of the work can occur offline, before the plaintext message even exists. When the message finally arrives, it can be XORed with the output of the algorithm to produce the ciphertext.

Initialization Vector

The OFB shift register must also be initially loaded with an IV. It should be unique but does not have to be secret.

Error Propagation

OFB mode has no error extension. A single-bit error in the ciphertext causes a single-bit error in the recovered plaintext. This can be useful in some digitized analog transmissions, like digitized voice or video, where the occasional single-bit error can be tolerated but error extension cannot.

On the other hand, a loss of synchronization is fatal. If the shift registers on the encryption end and the decryption end are not identical, then the recovered plaintext will be gibberish. Any system that uses OFB mode must have a mechanism for detecting a synchronization loss and a mechanism to fill both shift registers with a new (or the same) IV to regain synchronization.

Figure 9.12 n-bit OFB with an n-bit algorithm.

Security Problems with OFB

An analysis of OFB mode [588,430,431,789] demonstrates that OFB should be used only when the feedback size is the same as the block size. For example, you should only use a 64-bit algorithm in 64-bit OFB mode. Even though the U.S. government authorizes other feedback sizes for DES [1143], avoid them.

OFB mode XORs a keystream with the text. This keystream will eventually repeat. It is important that it does not repeat with the same key; otherwise, there is no security. When the feedback size equals the block size, the block cipher acts as a permutation of m-bit values (where m is the block length) and the average cycle length is 2m- 1. For a 64-bit block length, this is a very long number. When the feedback size n is less than the block length, the average cycle length drops to around 2m/2. For a 64-bit block cipher, this is only 232—not long enough.

Stream Ciphers in OFB

A stream cipher can also run in OFB mode. In this case, the key affects the next-state function (see Figure 9.13). The output function does not depend on the key; very often it is something simple like a single bit of the internal state or the XOR of multiple bits of the internal state. The cryptographic complexity is in the next-state function; this function is key-dependent. This method is also called internal feedback [291], because the feedback mechanism is internal to the key generation algorithm.

In a variant of this mode, the key determines just the initial state of the keystream generator. After the key sets the internal state of the generator, the generator runs undisturbed from then on.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

9.9 Counter Mode

Block ciphers in counter mode use sequence numbers as the input to the algorithm [824,498,715]. Instead of using the output of the encryption algorithm to fill the register, the input to the register is a counter. After each block encryption, the counter increments by some constant, typically one. The synchronization and error propagation characteristics of this mode are identical to those of OFB. Counter mode solves the OFB mode problem of n-bit output where n is less than the block length.

Figure 9.13 A keystream generator in output-feedback mode.

Nothing is sacred about the counter; it does not have to count through all the possible inputs in order. You can use any of the random-sequence generators in Chapters 16 and 17, whether cryptographically secure or not, as input to the block algorithm.

Stream Ciphers in Counter Mode

Stream ciphers in counter mode have simple next-state functions and complicated output functions dependent on the key. This technique, illustrated in Figure 9.14, was suggested in [498,715]. The next-state function can be something as simple as a counter, adding one to the previous state.

With a counter mode stream cipher, it is possible to generate the ith key bit, ki, without first generating all the previous key bits. Simply set the counter manually to the ith internal state and generate the bit. This is useful to secure random-access data files; you can decrypt a specific block of data without decrypting the entire file.

9.10 Other Block-Cipher Modes

Block Chaining Mode

To use a block algorithm in block chaining (BC) mode, simply XOR the input to the block cipher with the XOR of all the previous ciphertext blocks. As with CBC, an IV starts the process.

Mathematically, this looks like:

Ci = EK(Pi ⊕ Fi); Fi+1 = Fi ⊕ Ci
Pi = Fi ⊕ DK(Ci); Fi+1 = Fi ⊕ Ci

Like CBC, BC's feedback process extends errors in the plaintext. The primary problem with BC is that because the decryption of a ciphertext block depends on all the previous ciphertext blocks, a single error in the ciphertext will result in the incorrect decryption of all subsequent ciphertext blocks.

Figure 9.14 A keystream generator in counter mode.

Propagating Cipher Block Chaining Mode

Propagating cipher block chaining (PCBC) [1080] mode is similar to CBC mode, except that both the previous plaintext block and the previous ciphertext block are XORed with the current plaintext block before encryption (or after decryption) (see Figure 9.15).

Ci = EK(Pi ⊕ Ci-1 ⊕ Pi-1)
Pi = Ci-1 ⊕ Pi-1 ⊕ DK(Ci)

PCBC was used in Kerberos version 4 (see Section 24.5) to perform both encryption and integrity checking in one pass. In PCBC mode, an error in the ciphertext will result in incorrect decryption of all blocks that follow. This means that checking a standard block at the end of a message will ensure the integrity of the entire message.

Unfortunately, there is a problem with this mode [875]. Swapping two ciphertext blocks results in the incorrect decryption of the two corresponding plaintext blocks, but due to the nature of the XOR with the plaintext and the ciphertext, the errors cancel. So if the integrity checker looks only at the last few blocks of the decrypted plaintext, it could be fooled into accepting a partially garbled message. Although no one has figured out how to exploit this weakness, Kerberos version 5 switched to CBC mode after the flaw was discovered.

Cipher Block Chaining with Checksum

Cipher block chaining with checksum (CBCC) is a CBC variant [1618]. Keep a running XOR of all the plaintext blocks, and XOR that with the last plaintext block before encryption. CBCC ensures that any change made to any ciphertext block changes the decrypted output of the last block. If the last block contains any sort of integrity check or a constant, then the integrity of the decrypted plaintext can be checked with very little additional overhead.

Figure 9.15 Propagating cipher block chaining mode.

Output Feedback with a Nonlinear Function

Output feedback with a nonlinear function (OFBNLF) [777] is a variant of both OFB and ECB where the key changes with every block:

Ci = EKi(Pi); Ki = EK(Ki-1)
Pi = DKi(Ci); Ki = EK(Ki-1)

A single bit error in the ciphertext propagates to only one plaintext block. However, if a single bit is lost or added, then there is infinite error extension. With a block algorithm that has a complicated key scheduling algorithm, like DES, this mode is slow. I know of no cryptanalysis of this mode.

More Modes

Other modes are possible, although they are not extensively used. Plaintext block chaining (PBC) is like CBC except the previous plaintext block is XORed with the plaintext block instead of with the ciphertext block. Plaintext feedback (PFB) is like CFB, except the plaintext, not the ciphertext, is used for feedback. These two modes allow chosen-plaintext attacks in order to resist known-plaintext attacks. There is also cipher block chaining of plaintext difference (CBCPD). I'm sure it gets even weirder.

If a cryptanalyst has a brute-force keysearch machine, then he can recover the key if he can guess one of the plaintext blocks. Some of these stranger modes amount to light encryption before applying the encryption algorithm: for example, XORing the text with a fixed secret string or permuting the text. Almost anything nonstandard will frustrate this sort of cryptanalysis.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

9.11 Choosing a Cipher Mode

If simplicity and speed are your main concerns, ECB is the easiest and fastest mode to use a block cipher. It is also the weakest. Besides being vulnerable to replay attacks, an algorithm in ECB mode is the easiest to cryptanalyze. I don't recommend ECB for message encryption.

For encrypting random data, such as other keys, ECB is a good mode to use. Since the data is short and random, none of the shortcomings of ECB matter for this application.

For normal plaintext, use CBC, CFB, or OFB. Which mode you choose depends on your specific requirements. Table 9.1 gives a summary of the security and efficiency of the various modes.

CBC is generally best for encrypting files. The increase in security is significant; and while there are sometimes bit errors in stored data, there are almost never synchronization errors. If your application is software-based, CBC is almost always the best choice.

Table 9.1
Summary of Block Cipher Modes
ECB: |  | CBC:
 | Security: |  | Security:
- Plaintext patterns are not concealed. |  | + Plaintext patterns are concealed by XORing with previous ciphertext block.
- Input to the block cipher is not randomized; it is the same as the plaintext. |  | +Input to the block cipher is randomized by XORing with the previous ciphertext block.
+More than one message can be encrypted with the same key. |  | + More than one message can be encrypted with the same key.
- Plaintext is easy to manipulate, blocks can be removed, repeated, or interchanged. |  | +/- Plaintext is somewhat difficult to manipulate; blocks can be removed from the beginning and end of the message, bits of the first block can be changed, and repetition allows some controlled changes.
 | Efficiency: |  | Efficiency:
+ Speed is the same as the block cipher. |  | + Speed is the same as the block cipher.
Ciphertext is up to one block longer than the plaintext, due to padding. |  | - Ciphertext is up to one block longer than the plaintext, not counting the IV.
- No preprocessing is possible. |  | - No preprocessing is possible.
+ Processing is parallelizable. |  | +/- Encryptions not parallelizable; decryption is parallelizable and has a random-access property.
 | Fault-tolerance: |  | Fault-tolerance:
- A ciphertext error affects one full block of plaintext. |  | - A ciphertext error affects one full block of plaintext and the corresponding bit in the next block.
- Synchronization error is unrecoverable. |  | - Synchronization error is unrecoverable.
CFB: |  | OFB/Counter:
 | Security: |  | Security:
+ Plaintext patterns are concealed. |  | + Plaintext patterns are concealed.
+ Input to the block cipher is randomized. |  | + Input to the block cipher is randomized.
+ More than one message can be encrypted with the same key provided that a different IV is used. |  | + More than one message can be encrypted with the same key, provided that a different IV is used.
+/- Plaintext is somewhat difficult to manipulate;blocks can be removed from the beginning and end of the message, bits of the first block can be changed, and repetition allows some controlled changes. |  | - Plaintext is very easy to manipulate, any change in ciphertext directly affects the plaintext.
 | Efficiency: |  | Efficiency:
+ Speed is the same as the block cipher. |  | + Speed is the same as the block cipher.
- Ciphertext is the same size as the plaintext, not counting the IV. |  | - Ciphertext is the same size as the plaintext, not counting the IV.
+/- Encryption is not parallelizable; decryption is parallelizable and has a random-access property. |  | + Processing is possible before the message is seen.
- Some preprocessing is possible before a block is seen; the previous ciphertext block can be encrypted. |  | -/+ OFB processing is not parallelizable; counter processing is parallelizable.
+/- Encryption is not parallelizable; decryption is parallelizable and has a random-access property.
 | Fault-tolerance: |  | Fault-tolerance:
- A ciphertext error affects the corresponding bit of plaintext and the next full block. |  | + A ciphertext error affects only the corresponding bit of plaintext.
+Synchronization errors of full block sizes are recoverable. 1-bit CFB can recover from the addition or loss of single bits. |  | -Synchronization error is unrecoverable.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

CFB—specifically 8-bit CFB—is generally the mode of choice for encrypting streams of characters when each character has to be treated individually, as in a link between a terminal and a host. OFB is most often used in high-speed synchronous systems where error propagation is intolerable. OFB is also the mode of choice if preprocessing is required.

OFB is the mode of choice in a error-prone environment, because it has no error extension.

Stay away from the weird modes. One of the four basic modes—ECB, CBC, OFB, and CFB—is suitable for almost any application. These modes are not overly complex and probably do not reduce the security of the system. While it is possible that a complicated mode might increase the security of a system, most likely it just increases the complexity. None of the weird modes has any better error propagation or error recovery characteristics.

9.12 Interleaving

With most modes, encryption of a bit (or block) depends on the encryption of the previous bits (or blocks). This can often make it impossible to parallelize encryption. For example, consider a hardware box that does encryption in CBC mode. Even if the box contains four encryption chips, only one can work at any time. The next chip needs the results of the previous chip before it starts working.

The solution is to interleave multiple encryption streams. (This is not multiple encryption; that's covered in Sections 15.1 and 15.2). Instead of a single CBC chain, use four. The first, fifth, and every fourth block thereafter are encrypted in CBC mode with one IV. The second, sixth, and every fourth block thereafter are encrypted in CBC mode with another IV, and so on. The total IV is much longer than it would have been without interleaving.

Think of it as encrypting four different messages with the same key and four different IVs. These messages are all interleaved.

This trick can also be used to increase the overall speed of hardware encryption. If you have three encryption chips, each capable of encrypting data at 33 megabits/second, you can interleave them to encrypt a single 100 megabit/second data channel.

Figure 9.16 shows three parallel streams interleaved in CFB mode. The idea can also work in CBC and OFB modes, and with any number of parallel streams. Just remember that each stream needs its own IV. Don't share.

9.13 Block Ciphers versus Stream Ciphers

Although block and stream ciphers are very different, block ciphers can be implemented as stream ciphers and stream ciphers can be implemented as block ciphers. The best definition of the difference I've found is from Rainer Rueppel [1362]:

Block ciphers operate on data with a fixed transformation on large blocks of plaintext data; stream ciphers operate with a time-varying transformation on individual plaintext digits.

Figure 9.16 Interleaving three CFB encryptions.

In the real world, block ciphers seem to be more general (i.e., they can be used in any of the four modes) and stream ciphers seem to be easier to analyze mathematically. There is a large body of theoretical work on the analysis and design of stream ciphers—most of it done in Europe, for some reason. They have been used by the world's militaries since the invention of electronics. This seems to be changing; recently a whole slew of theoretical papers have been written on block cipher design. Maybe soon there will be a theory of block cipher design as rich as our current theory of stream cipher design.

Otherwise, the differences between stream ciphers and block ciphers are in the implementation. Stream ciphers that only encrypt and decrypt data one bit at a time are not really suitable for software implementation. Block ciphers can be easier to implement in software, because they often avoid time-consuming bit manipulations and they operate on data in computer-sized blocks. On the other hand, stream ciphers can be more suitable for hardware implementation because they can be implemented very efficiently in silicon.

These are important considerations. It makes sense for a hardware encryption device on a digital communications channel to encrypt the individual bits as they go by. This is what the device sees. On the other hand, it makes no sense for a software encryption device to encrypt each individual bit separately. There are some specific instances where bit- and byte-wise encryption might be necessary in a computer system—encrypting the link between the keyboard and the CPU, for example—but generally the encryption block should be at least the width of the data bus.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 10
Using Algorithms

Think of security—data security, communications security, information security, whatever—as a chain. The security of the entire system is only as strong as the weakest link. Everything has to be secure: cryptographic algorithms, protocols, key management, and more. If your algorithms are great but your random-number generator stinks, any smart cryptanalyst is going to attack your system through the random-number generation. If you patch that hole but forget to securely erase a memory location that contains the key, a cryptanalyst will break your system via that route. If you do everything right and accidentally e-mail a copy of your secure files to The Wall Street Journal, you might as well not have bothered.

It's not fair. As the designer of a secure system, you have to think of every possible means of attack and protect against them all, but a cryptanalyst only has to find one hole in your security and exploit it.

Cryptography is only a part of security, and often a very small part. It is the mathematics of making a system secure, which is different from actually making a system secure. Cryptography has its "size queens": people who spend so much time arguing about how long a key should be that they forget about everything else. If the secret police want to know what is on your computer, it is far easier for them to break into your house and install a camera that can record what is on your computer screen than it is for them to cryptanalze your hard drive.

Additionally, the traditional view of computer cryptography as "spy versus spy" technology is becoming increasingly inappropriate. Over 99 percent of the cryptography used in the world is not protecting military secrets; it's in applications such as bank cards, pay-TV, road tolls, office building and computer access tokens, lottery terminals, and prepayment electricity meters [43,44]. In these applications, the role of cryptography is to make petty crime slightly more difficult; the paradigm of the well-funded adversary with a rabbit warren of cryptanalysts and roomsful of computers just doesn't apply.

Most of those applications have used lousy cryptography, but successful attacks against them had nothing to do with cryptanalysis. They involved crooked employees, clever sting operations, stupid implementations, integration blunders, and random idiocies. (I strongly recommend Ross Anderson's paper, "Why Cryptosytems Fail" [44]; it should be required reading for anyone involved in this field.) Even the NSA has admitted that most security failures in its area of interest are due to failures in implementation, and not failures in algorithms or protocols [1119]. In these instances it didn't matter how good the cryptography was; the successful attacks bypassed it completely.

10.1 Choosing an Algorithm

When it comes to evaluating and choosing algorithms, people have several alternatives:

— They can choose a published algorithm, based on the belief that a published algorithm has been scrutinized by many cryptographers; if no one has broken the algorithm yet, then it must be pretty good.
— They can trust a manufacturer, based on the belief that a well-known manufacturer has a reputation to uphold and is unlikely to risk that reputation by selling equipment or programs with inferior algorithms.
— They can trust a private consultant, based on the belief that an impartial consultant is best equipped to make a reliable evaluation of different algorithms.
— They can trust the government, based on the belief that the government is trustworthy and wouldn't steer its citizens wrong.
— They can write their own algorithms, based on the belief that their cryptographic ability is second-to-none and that they should trust nobody but themselves.

Any of these alternatives is problematic, but the first seems to be the most sensible. Putting your trust in a single manufacturer, consultant, or government is asking for trouble. Most people who call themselves security consultants (even those from big-name firms) usually don't know anything about encryption. Most security product manufacturers are no better. The NSA has some of the world's best cryptographers working for it, but they're not telling all they know. They have their own interests to further which are not congruent with those of their citizens. And even if you're a genius, writing your own algorithm and then using it without any peer review is just plain foolish.

The algorithms in this book are public. Most have appeared in the open literature and many have been cryptanalyzed by experts in the field. I list all published results, both positive and negative. I don't have access to the cryptanalysis done by any of the myriad military security organizations in the world (which are probably better than the academic institutions—they've been doing it longer and are better funded), so it is possible that these algorithms are easier to break than it appears. Even so, it is far more likely that they are more secure than an algorithm designed and implemented in secret in some corporate basement.

The hole in all this reasoning is that we don't know the abilities of the various military cryptanalysis organizations.

What algorithms can the NSA break? For the majority of us, there's really no way of knowing. If you are arrested with a DES-encrypted computer hard drive, the FBI is unlikely to introduce the decrypted plaintext at your trial; the fact that they can break an algorithm is often a bigger secret than any information that is recovered. During WWII, the Allies were forbidden from using decrypted German Ultra traffic unless they could have plausibly gotten the information elsewhere. The only way to get the NSA to admit to the ability to break a given algorithm is to encrypt something so valuable that its public dissemination is worth the admission. Or, better yet, create a really funny joke and send it via encrypted e-mail to shady characters in shadowy countries. NSA employees are people, too; I doubt even they can keep a good joke secret.

A good working assumption is that the NSA can read any message that it chooses, but that it cannot read all messages that it chooses. The NSA is limited by resources, and has to pick and choose among its various targets. Another good assumption is that they prefer breaking knuckles to breaking codes; this preference is so strong that they will only resort to breaking codes when they wish to preserve the secret that they have read the message.

In any case, the best most of us can do is to choose among public algorithms that have withstood a reasonable amount of public scrutiny and cryptanalysis.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Algorithms for Export

Algorithms for export out of the United States must be approved by the U.S. government (actually, by the NSA—see Section 25.1). It is widely believed that these export-approved algorithms can be broken by the NSA. Although no one has admitted this on the record, these are some of the things the NSA is rumored to privately suggest to companies wishing to export their cryptographic products:

— Leak a key bit once in a while, embedded in the ciphertext.
— "Dumb down" the effective key to something in the 30-bit range. For example, while the algorithm might accept a 100-bit key, most of those keys might be equivalent.
— Use a fixed IV, or encrypt a fixed header at the beginning of each encrypted message. This facilitates a known-plaintext attack.
— Generate a few random bytes, encrypt them with the key, and then put both the plaintext and the ciphertext of those random bytes at the beginning of the encrypted message. This also facilitates a known-plaintext attack.

NSA gets a copy of the source code, but the algorithm's details remain secret from everyone else. Certainly no one advertises any of these deliberate weaknesses, but beware if you buy a U.S. encryption product that has been approved for export.

10.2 Public-Key Cryptography versus Symmetric Cryptography

Which is better, public-key cryptography or symmetric cryptography? This question doesn't make any sense, but has been debated since public-key cryptography was invented. The debate assumes that the two types of cryptography can be compared on an equal footing. They can't.

Needham and Schroeder [1159] pointed out that the number and length of messages are far greater with public-key algorithms than with symmetric algorithms. Their conclusion was that the symmetric algorithm was more efficient than the public-key algorithm. While true, this analysis overlooks the significant security benefits of public-key cryptography.

Whitfield Diffie writes [492,494]:

In viewing public-key cryptography as a new form of cryptosystem rather than a new form of key management, I set the stage for criticism on grounds of both security and performance. Opponents were quick to point out that the RSA system ran about one-thousandth as fast as DES and required keys about ten times as large. Although it had been obvious from the beginning that the use of public key systems could be limited to exchanging keys for conventional [symmetric] cryptography, it was not immediately clear that this was necessary. In this context, the proposal to build hybrid systems [879] was hailed as a discovery in its own right.

Public-key cryptography and symmetric cryptography are different sorts of animals; they solve different sorts of problems. Symmetric cryptography is best for encrypting data. It is orders of magnitude faster and is not susceptible to chosen-ciphertext attacks. Public-key cryptography can do things that symmetric cryptography can't; it is best for key management and a myriad of protocols discussed in Part I.

Other primitives were discussed in Part I: one-way hash functions, message authentication codes, and so on. Table 10.1 lists different types of algorithms and their properties [804].

10.3 Encrypting Communications Channels

This is the classic Alice and Bob problem: Alice wants to send Bob a secure message. What does she do? She encrypts the message.

In theory, this encryption can take place at any layer in the OSI (Open Systems Interconnect) communications model. (See the OSI security architecture standard for more information [305].) In practice, it takes place either at the lowest layers (one and two) or at higher layers. If it takes place at the lowest layers, it is called link-by-link encryption; everything going through a particular data link is encrypted. If it takes place at higher layers, it is called end-to-end encryption; the data are encrypted selectively and stay encrypted until they are decrypted by the intended final recipient. Each approach has its own benefits and drawbacks.

Table 10.1 Classes of Algorithms
Algorithm | Confidentiality | Authentication | Integrity | Key Management
Symmetric encryption algorithms | Yes | No | No | Yes
Public-key encryption algorithms | Yes | No | No | Yes
Digital signature algorithms | No | Yes | Yes | No
Key-agreement algorithms | Yes | Optional | No | Yes
One-way hash functions | No | No | Yes | No
Message authentication codes | No | Yes | Yes | No

Link-by-Link Encryption

The easiest place to add encryption is at the physical layer (see Figure 10.1). This is called link-by-link encryption. The interfaces to the physical layer are generally standardized and it is easy to connect hardware encryption devices at this point. These devices encrypt all data passing through them, including data, routing information, and protocol information. They can be used on any type of digital communication link. On the other hand, any intelligent switching or storing nodes between the sender and the receiver need to decrypt the data stream before processing it.

This type of encryption is very effective. Because everything is encrypted, a cryptanalyst can get no information about the structure of the information. He has no idea who is talking to whom, how long the messages they are sending are, what times of day they communicate, and so on. This is called traffic-flow security: the enemy is not only denied access to the information, but also access to the knowledge of where and how much information is flowing.

Security does not depend on any traffic management techniques. Key management is also simple; only the two endpoints of the line need a common key, and they can change their key independently from the rest of the network.

Figure 10.1 Link encryption.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Imagine a synchronous communications line, encrypted using 1-bit CFB. After initialization, the line can run indefinitely, recovering automatically from bit or synchronization errors. The line encrypts whenever messages are sent from one end to the other; otherwise it just encrypts and decrypts random data. Eve has no idea when messages are being sent and when they are not; she has no idea when messages begin and end. All she sees is an endless stream of random-looking bits.

If the communications line is asynchronous, the same 1-bit CFB mode can be used. The difference is that the adversary can get information about the rate of transmission. If this information must be concealed, make some provision for passing dummy messages during idle times.

The biggest problem with encryption at the physical layer is that each physical link in the network needs to be encrypted: Leaving any link unencrypted jeopardizes the security of the entire network. If the network is large, the cost may quickly become prohibitive for this kind of encryption.

Additionally, every node in the network must be protected, since it processes unencrypted data. If all the network's users trust one another, and all nodes are in secure locations, this may be tolerable. But this is unlikely. Even in a single corporation, information might have to be kept secret within a department. If the network accidentally misroutes information, anyone can read it. Table 10.2 summarizes the pros and cons of link-by-link encryption.

End-to-End Encryption

Another approach is to put encryption equipment between the network layer and the transport layer. The encryption device must understand the data according to the protocols up to layer three and encrypt only the transport data units, which are then recombined with the unencrypted routing information and sent to lower layers for transmission.

This approach avoids the encryption/decryption problem at the physical layer. By providing end-to-end encryption, the data remains encrypted until it reaches its final destination (see Figure 10.2). The primary problem with end-to-end encryption is that the routing information for the data is not encrypted; a good cryptanalyst can learn much from who is talking to whom, at what times and for how long, without ever knowing the contents of those conversations. Key management is also more difficult, since individual users must make sure they have common keys.

Table 10.2 Link-by-Link Encryption: Advantages and Disadvantages
 | Advantages:
Easier operation, since it can be made transparent to the user. That is, everything is encrypted before being sent over the link.
Only one set of keys per link is required.
Provides traffic-flow security, since any routing information is encrypted.
Encryption is online.
 | Disadvantages:
Data is exposed in the intermediate nodes.

Figure 10.2 End-to-end encryption.

Building end-to-end encryption equipment is difficult. Each particular communications system has its own protocols. Sometimes the interfaces between the levels are not well-defined, making the task even more difficult.

If encryption takes place at a high layer of the communications architecture, like the applications layer or the presentation layer, then it can be independent of the type of communication network used. It is still end-to-end encryption, but the encryption implementation does not have to bother about line codes, synchronization between modems, physical interfaces, and so forth. In the early days of electromechanical cryptography, encryption and decryption took place entirely offline; this is only one step removed from that.

Encryption at these high layers interacts with the user software. This software is different for different computer architectures, and so the encryption must be optimized for different computer systems. Encryption can occur in the software itself or in specialized hardware. In the latter case, the computer will send the data to the specialized hardware for encryption before sending it to lower layers of the communication architecture for transmission. This process requires some intelligence and is not suitable for dumb terminals. Additionally, there may be compatibility problems with different types of computers.

The major disadvantage of end-to-end encryption is that it allows traffic analysis. Traffic analysis is the analysis of encrypted messages: where they come from, where they go to, how long they are, when they are sent, how frequent or infrequent they are, whether they coincide with outside events like meetings, and more. A lot of good information is buried in that data, and a cryptanalyst will want to get his hands on it. Table 10.3 presents the positive and negative aspects of end-to-end encryption.

Combining the Two

Table 10.4, primarily from [1244], compares link-by-link and end-to-end encryption. Combining the two, while most expensive, is the most effective way of securing a network. Encryption of each physical link makes any analysis of the routing information impossible, while end-to-end encryption reduces the threat of unencrypted data at the various nodes in the network. Key management for the two schemes can be completely separate: The network managers can take care of encryption at the physical level, while the individual users have responsibility for end-to-end encryption.

Table 10.3 End-to-End Encryption: Advantages and Disadvantages
 | Advantages:
Higher secrecy level.
 | Disadvantages:
Requires a more complex key-management system.
Traffic analysis is possible, since routing information is not encrypted.
Encryption is offline.

10.4 Encrypting Data for Storage

Encrypting data for storage and later retrieval can also be thought of in the Alice and Bob model. Alice is still sending a message to Bob, but in this case "Bob" is Alice at some future time. However, the problem is fundamentally different.

In communications channels, messages in transit have no intrinsic value. If Bob doesn't receive a particular message, Alice can always resend it. This is not true for data encrypted for storage. If Alice can't decrypt her message, she can't go back in time and re-encrypt it. She has lost it forever. This means that encryption applications for data storage should have some mechanisms to prevent unrecoverable errors from creeping into the ciphertext.

The encryption key has the same value as the message, only it is smaller. In effect, cryptography converts large secrets into smaller ones. Being smaller, they can be easily lost. Key management procedures should assume that the same keys will be used again and again, and that data may sit on a disk for years before being decrypted.

Furthermore, the keys will be around for a long time. A key used on a communications link should, ideally, exist only for the length of the communication. A key used for data storage might be needed for years, and hence must be stored securely for years.

Other problems particular to encrypting computer data for storage were listed in [357]:

— The data may also exist in plaintext form, either on another disk, in another computer, or on paper. There is much more opportunity for a cryptanalyst to perform a known-plaintext attack.
— In database applications, pieces of data may be smaller than the block size of most algorithms. This will cause the ciphertext to be considerably larger than the plaintext.
— The speed of I/O devices demands fast encryption and decryption, and will probably require encryption hardware. In some applications, special high-speed algorithms may be required.
— Safe, long-term storage for keys is required.
— Key management is much more complicated, since different people need access to different files, different portions of the same file, and so forth.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 10.4 Comparing Link-by-Link and End-to-End Encryption
Link-by-link encryption |  | End-to-end encryption
 | Security within Hosts |
Message exposed in sending host |  | Message encrypted in sending host
Message exposed in intermediate nodes |  | Message encrypted inintermediate nodes
 | Role of User
Applied by sending host |  | Applied by sending process
Invisible to user |  | User applies encryption
Host maintains encryption |  | User must find algorithm
One facility for all users |  | User selects encryption
Can be done in hardware |  | More easily done in software
All or no messages encrypted |  | User chooses to encrypt or not, for each message
 | Implementation Concerns |
Requires one key per host pair |  | Requires one key per user pair
Requires encryption hardware or software at each host |  | Requires encryption hardware or software at each node
Provides node authentication |  | Provides user authentication

If the encrypted files are not structured as records and fields, such as text files, retrieval is easier: The entire file is decrypted before use. If the encrypted files are database files, this solution is problematic. Decrypting the entire database to access a single record is inefficient, but encrypting records independently might be susceptible to a block-replay kind of attack.

In addition, you must make sure the unencrypted file is erased after encryption (see Section 10.9). For further details and insights, consult [425,569].

Dereferencing Keys

When encrypting a large hard drive, you have two options. You can encrypt all the data using a single key. This gives a cryptanalyst a large amount of ciphertext to analyze and makes it impossible to allow multiple users to see only parts of the drive. Or, you can encrypt each file with a different key, forcing users to memorize a different key for each file.

The solution is to encrypt each file with a separate key, and to encrypt the keys with another key known by the users. Each user only has to remember that one key. Different users can have different subsets of the file-encryption keys encrypted with their key. And there can even be a master key under which every file-encryption key is encrypted. This is even more secure because the file-encryption keys are random and less susceptible to a dictionary attack.

Driver-Level vs. File-Level Encryption

There are two ways to encrypt a hard drive: at the file level and at the driver level. Encryption at the file level means that every file is encrypted separately. To use a file that's been encrypted, you must first decrypt the file, then use it, and then re-encrypt it.

Driver-level encryption maintains a logical drive on the user's machine that has all data on it encrypted. If done well, this can provide security that, beyond choosing good passwords, requires little worry on the part of the user. The driver must be considerably more complex than a simple file-encryption program, however, because it must deal with the issues of being an installed device driver, allocation of new sectors to files, recycling of old sectors from files, random-access read and update requests for any data on the logical disk, and so on.

Typically, the driver prompts the user for a password before starting up. This is used to generate the master decryption key, which may then be used to decrypt actual decryption keys used on different data.

Providing Random Access to an Encrypted Drive

Most systems expect to be able to access individual disk sectors randomly. This adds some complication for using many stream ciphers and block ciphers in any chaining mode. Several solutions are possible.

Use the sector address to generate a unique IV for each sector being encrypted or decrypted. The drawback is that each sector will always be encrypted with the same IV. Make sure this is not a security problem.

For the master key, generate a pseudo-random block as large as one sector. (You can do this by running an algorithm in OFB mode, for example.) To encrypt any sector, first XOR in this pseudo-random block, then encrypt normally with a block cipher in ECB mode. This is called ECB+OFB (see Section 15.4).

Since CBC and CFB are error-recovering modes, you can use all but the first block or two in the sector to generate the IV for that sector. For example, the IV for sector 3001 may be the hash of the all but the first 128 bits of the sector's data. After generating the IV, encrypt normally in CBC mode. To decrypt the sector, you use the second 64-bit block of the sector as an IV, and decrypt the remainder of the sector. Then, using the decrypted data, you regenerate the IV and decrypt the first 128 bits.

You can use a block cipher with a large enough block size that it can encrypt the whole sector at once. Crab (see Section 14.6) is an example.

10.5 Hardware Encryption versus Software Encryption

Hardware

Until very recently, all encryption products were in the form of specialized hardware. These encryption/decryption boxes plugged into a communications line and encrypted all the data going across that line. Although software encryption is becoming more prevalent today, hardware is still the embodiment of choice for military and serious commercial applications. The NSA, for example, only authorizes encryption in hardware. There are several reasons why this is so.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 10.5 Comparing File-Level and Driver-Level Encryption
File-Level Encryption |  | Driver-Level Encryption
 | Benefits: |
Ease of implementation and use. Flexible. Relatively small performance penalty. Users can move files between different machines without problems. Users can back files up without problems. |  | Temporary files, work files, and so forth can be kept on the secure drive. It's harder to forget to re-encrypt something on this kind of system.
 | Security Issues:
Potential leakage through security-unconscious programs. (Program may write file to disk for temporary storage, for example.) Bad implementations may always re-encrypt with same key for same password. |  | Lots of things can go wrong with a device-driver or memory-resident program. Bad implementations will allow chosen-plaintext, or even chosen-ciphertext attacks. If whole system is master-keyed under one password, loss of that password means that the attacker gets everything. A more limited set of ciphers can reasonably be used for this kind of application. For example, OFB stream ciphers would not work.
 | Usability Problems: |
User has to figure out what to do. There may be different passwords for different files. Manual encryption of selected files is the only access control. |  | There will be a performance penalty. The driver may interact in weird ways with Windows, OS/2 DOS emulation, device drivers, and so on.

The first is speed. As we will see in Part III, encryption algorithms consist of many complicated operations on plaintext bits. These are not the sorts of operations that are built into your run-of-the-mill computer. The two most common encryption algorithms, DES and RSA, run inefficiently on general-purpose processors. While some cryptographers have tried to make their algorithms more suitable for software implementation, specialized hardware will always win a speed race.

Additionally, encryption is often a computation-intensive task. Tying up the computer's primary processor for this is inefficient. Moving encryption to another chip, even if that chip is just another processor, makes the whole system faster.

The second reason is security. An encryption algorithm running on a generalized computer has no physical protection. Mallory can go in with various debugging tools and surreptitiously modify the algorithm without anyone ever realizing it. Hardware encryption devices can be securely encapsulated to prevent this. Tamperproof boxes can prevent someone from modifying a hardware encryption device. Special-purpose VLSI chips can be coated with a chemical such that any attempt to access their interior will result in the destruction of the chip's logic. The U.S. government's Clipper and Capstone chips (see Sections 24.16 and 24.17) are designed to be tamperproof. The chips can be designed so that it is impossible for Mallory to read the unencrypted key.

IBM developed a cryptographic system for encrypting data and communications on mainframe computers [515,1027]. It includes tamper-resistant modules to hold keys. This system is discussed in Section 24.1.

Electromagnetic radiation can sometimes reveal what is going on inside a piece of electronic equipment. Dedicated encryption boxes can be shielded, so that they leak no compromising information. General-purpose computers can be shielded as well, but it is a far more complex problem. The U.S. military calls this TEMPEST; it's a subject well beyond the scope of this book.

The final reason for the prevalence of hardware is the ease of installation. Most encryption applications don't involve general-purpose computers. People may wish to encrypt their telephone conversations, facsimile transmissions, or data links. It is cheaper to put special-purpose encryption hardware in the telephones, facsimile machines, and modems than it is to put in a microprocessor and software.

Even when the encrypted data comes from a computer, it is easier to install a dedicated hardware encryption device than it is to modify the computer's system software. Encryption should be invisible; it should not hamper the user. The only way to do this in software is to write encryption deep into the operating system. This isn't easy. On the other hand, even a computer neophyte can plug an encryption box between his computer and his external modem.

The three basic kinds of encryption hardware on the market today are: self-contained encryption modules (that perform functions such as password verification and key management for banks), dedicated encryption boxes for communications links, and boards that plug into personal computers.

Some encryption boxes are designed for certain types of communications links, such as T-1 encryption boxes that are designed not to encrypt synchronization bits. There are different boxes for synchronous and asynchronous communications lines. Newer boxes tend to accept higher bit rates and are more versatile.

Even so, many of these devices have some incompatibilities. Buyers should be aware of this and be well-versed in their particular needs, lest they find themselves the owners of encryption equipment unable to perform the task at hand. Pay attention to restrictions in hardware type, operating system, applications software, network, and so forth.

PC-board encryptors usually encrypt everything written to the hard disk and can be configured to encrypt everything sent to the floppy disk and serial port as well. These boards are not shielded against electromagnetic radiation or physical interference, since there would be no benefit in protecting the boards if the computer remained unaffected.

More companies are starting to put encryption hardware into their communications equipment. Secure telephones, facsimile machines, and modems are all available.

Internal key management for these devices is generally secure, although there are as many different schemes as there are equipment vendors. Some schemes are more suited for one situation than another, and buyers should know what kind of key management is incorporated into the encryption box and what they are expected to provide themselves.

Software

Any encryption algorithm can be implemented in software. The disadvantages are in speed, cost, and ease of modification (or manipulation). The advantages are in flexibility and portability, ease of use, and ease of upgrade. The algorithms written in C at the end of this book can be implemented, with little modification, on any computer. They can be inexpensively copied and installed on many machines. They can be incorporated into larger applications, such as communications programs or word processors.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Software encryption programs are popular and are available for all major operating systems. These are meant to protect individual files; the user generally has to manually encrypt and decrypt specific files. It is important that the key management scheme be secure: The keys should not be stored on disk anywhere (or even written to a place in memory from where the processor swaps out to disk). Keys and unencrypted files should be erased after encryption. Many programs are sloppy in this regard, and a user has to choose carefully.

Of course, Mallory can always replace the software encryption algorithm with something lousy. But for most users, that isn't a problem. If Mallory can break into our office and modify our encryption program, he can also put a hidden camera on the wall, a wiretap on the telephone, and a TEMPEST detector down the street. If Mallory is that much more powerful than the user, the user has lost the game before it starts.

10.6 Compression, Encoding, and Encryption

Using a data compression algorithm together with an encryption algorithm makes sense for two reasons:

Cryptanalysis relies on exploiting redundancies in the plaintext; compressing a file before encryption reduces these redundancies.
Encryption is time-consuming; compressing a file before encryption speeds up the entire process.

The important thing to remember is to compress before encryption. If the encryption algorithm is any good, the ciphertext will not be compressible; it will look like random data. (This makes a reasonable test of an encryption algorithm; if the ciphertext can be compressed, then the algorithm probably isn't very good.)

If you are going to add any type of transmission encoding or error detection and recovery, remember to add that after encryption. If there is noise in the communications path, decryption's error-extension properties will only make that noise worse. Figure 10.3 summarizes these steps.

10.7 Detecting Encryption

How does Eve detect an encrypted file? Eve is in the spy business, so this is an important question. Imagine that she's eavesdropping on a network where messages are flying in all directions at high speeds; she has to pick out the interesting ones. Encrypted files are certainly interesting, but how does she know they are encrypted?

Generally, she relies on the fact that most popular encryption programs have well-defined headers. Electronic-mail messages encrypted with either PEM or PGP (see Sections 24.10 and 24.12) are easy to identify for that reason.

Other file encryptors just produce a ciphertext file of seemingly random bits. How can she distinguish it from any other file of seemingly random bits? There is no sure way, but Eve can try a number of things:

— Examine the file. ASCII text is easy to spot. Other file formats, such as TIFF, TeX, C, Postscript, G3 facsimile, or Microsoft Excel, have standard identifying characteristics. Executable code is detectable, as well. UNIX files often have "magic numbers" that can be detected.

Figure 10.3 Encryption with compression and error control.

— Try to uncompress the file, using the major compression algorithms. If the file is compressed (and not encrypted), this should yield the original file.
— Try to compress the file. If the file is ciphertext (and the algorithm is good), then the probability that the file can be appreciably compressed by a general-purpose compression routine is small. (By appreciably, I mean more than 1 or 2 percent.) If it is something else (a binary image or a binary data file, for example) it probably can be compressed.

Any file that cannot be compressed and is not already compressed is probably ciphertext. (Of course, it is possible to specifically make ciphertext that is compressible.) Identifying the algorithm is a whole lot harder. If the algorithm is good, you can't. If the algorithm has some slight biases, it might be possible to recognize those biases in the file. However, the biases have to be pretty significant or the file has to be pretty big in order for this to work.

10.8 Hiding Ciphertext in Ciphertext

Alice and Bob have been sending encrypted messages to each other for the past year. Eve has been collecting them all, but she cannot decrypt any of them. Finally, the secret police tire of all this unreadable ciphertext and arrest the pair. "Give us your encryption keys," they demand. Alice and Bob refuse, but then they notice the thumbscrews. What can they do?

Wouldn't it be nice to be able to encrypt a file such that there are two possible decryptions, each with a different key. Alice could encrypt a real message to Bob in one of the keys and some innocuous message in the other key. If Alice were caught, she could surrender the key to the innocuous message and keep the real key secret.

The easiest way to do this is with one-time pads. Let P be the plaintext, D the dummy plaintext, C the ciphertext, K the real key, and K' the dummy key. Alice encrypts P:

P ⊕ K = C

Alice and Bob share K, so Bob can decrypt C:

C ⊕ K = P

If the secret police ever force them to surrender their key, they don't surrender K, but instead surrender:

K' = C ⊕ D

The police then recover the dummy plaintext:

C ⊕ K' = D


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Since these are one-time pads and K is completely random, there is no way to prove that K' was not the real key. To make matters more convincing, Alice and Bob should concoct some mildly incriminating dummy messages to take the place of the really incriminating real messages. A pair of Israeli spies once did this.

Alice could take P and encrypt it with her favorite algorithm and key K to get C. Then she takes C and XORs it with some piece of mundane plaintext—Pride and Prejudice for example, to get K'. She stores both C and the XOR on her hard disk. Now, when the secret police interrogate her, she can explain that she is an amateur cryptographer and that K' is a merely one-time pad for C. The secret police might suspect something, but unless they know K they cannot prove that Alice's explanation isn't valid.

Another method is to encrypt P with a symmetric algorithm and K, and D with K'. Intertwine bits (or bytes) of the ciphertext to make the final ciphertexts. If the secret police demand the key, Alice gives them K' and says that the alternating bits (or bytes) are random noise designed to frustrate cryptanalysis. The trouble is the explanation is so implausible that the secret police will probably not believe her (especially considering it is suggested in this book).

A better way is for Alice to create a dummy message, D, such that the concatenation of P and D, compressed, is about the same size as D. Call this concatenation P'. Alice then encrypts P' with whatever algorithm she and Bob share to get C. Then she sends C to Bob. Bob decrypts C to get P', and then P and D. Then they both compute C ⊕ D = K'. This K' becomes the dummy one-time pad they use in case the secret police break their doors down. Alice has to transmit D so that hers and Bob's alibis match.

Another method is for Alice to take an innocuous message and run it through some error-correcting code. Then she can introduce errors that correspond to the secret encrypted message. On the receiving end, Bob can extract the errors to reconstruct the secret message and decrypt it. He can also use the error-correcting code to recover the innocuous message. Alice and Bob might be hard pressed to explain to the secret police why they consistently get a 30 percent bit-error rate on an otherwise noise-free computer network, but in some circumstances this scheme can work.

Finally, Alice and Bob can use the subliminal channels in their digital signature algorithms (see Sections 4.2 and 23.3). This is undetectable, works great, but has the drawback of only allowing 20 or so characters of subliminal text to be sent per signed innocuous message. It really isn't good for much more than sending keys.

10.9 Destroying Information

When you delete a file on most computers, the file isn't really deleted. The only thing deleted is an entry in the disk's index file, telling the machine that the file is there. Many software vendors have made a fortune selling file-recovery software that recovers files after they have been deleted.

And there's yet another worry: Virtual memory means your computer can read and write memory to disk any time. Even if you don't save it, you never know when a sensitive document you are working on is shipped off to disk. This means that even if you never save your plaintext data, your computer might do it for you. And driver-level compression programs like Stacker and DoubleSpace can make it even harder to predict how and where information is stored on a disk.

To erase a file so that file-recovery software cannot read it, you have to physically write over all of the file's bits on the disk. According to the National Computer Security Center [1148]:

Overwriting is a process by which unclassified data are written to storage locations that previously held sensitive data.... To purge the...storage media, the DoD requires overwriting with a pattern, then its complement, and finally with another pattern; e.g., overwrite first with 0011 0101, followed by 1100 1010, then 1001 0111. The number of times an overwrite must be accomplished depends on the storage media, sometimes on its sensitivity, and sometimes on different DoD component requirements. In any case, a purge is not complete until a final overwrite is made using unclassified data.

You may have to erase files or you may have to erase entire drives. You should also erase all unused space on your hard disk.

Most commercial programs that claim to implement the DoD standard overwrite three times: first with all ones, then with all zeros, and finally with a repeating one-zero pattern. Given my general level of paranoia, I recommend overwriting a deleted file seven times: the first time with all ones, the second time with all zeros, and five times with a cryptographically secure pseudo-random sequence. Recent developments at the National Institute of Standards and Technology with electron-tunneling microscopes suggest even that might not be enough. Honestly, if your data is sufficiently valuable, assume that it is impossible to erase data completely off magnetic media. Burn or shred the media; it's cheaper to buy media new than to lose your secrets.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part III
Cryptographic Algorithms

Chapter 11
Mathematical Background

11.1 Information Theory

Modern information theory was first published in 1948 by Claude Elmwood Shannon [1431, 1432]. (His papers have been reprinted by the IEEE Press [1433].) For a good mathematical treatment of the topic, consult [593]. In this section, I will just sketch some important ideas.

Entropy and Uncertainty

Information theory defines the amount of information in a message as the minimum number of bits needed to encode all possible meanings of that message, assuming all messages are equally likely. For example, the day-of-the-week field in a database contains no more than 3 bits of information, because the information can be encoded with 3 bits:

    000 = Sunday
    001 = Monday
    010 = Tuesday
    011 = Wednesday
    100 = Thursday
    101 = Friday
    110 = Saturday
    111 is unused

If this information were represented by corresponding ASCII character strings, it would take up more memory space but would not contain any more information. Similarly, the "sex" field of a database contains only 1 bit of information, even though it might be stored as one of two 6-byte ASCII strings: "MALE" or "FEMALE."

Formally, the amount of information in a message M is measured by the entropy of a message, denoted by H(M). The entropy of a message indicating sex is 1 bit; the entropy of a message indicating the day of the week is slightly less than 3 bits. In general, the entropy of a message measured in bits is log2 n, in which n is the number of possible meanings. This assumes that each meaning is equally likely.

The entropy of a message also measures its uncertainty. This is the number of plaintext bits needed to be recovered when the message is scrambled in ciphertext in order to learn the plaintext. For example, if the ciphertext block "QHP*5M" is either "MALE" or "FEMALE, " then the uncertainty of the message is 1. A cryptanalyst has to learn only one well-chosen bit to recover the message.

Rate of a Language

For a given language, the rate of the language is

r = H(M)/N

in which N is the length of the message. The rate of normal English takes various values between 1.0 bits/letter and 1.5 bits/letter, for large values of N. Shannon, in [1434], said that the entropy depends on the length of the text. Specifically he indicated a rate of 2.3 bits/letter for 8-letter chunks, but the rate drops to between 1.3 and 1.5 for 16-letter chunks. Thomas Cover used a gambling estimating technique and found an entropy of 1.3 bits/character [386]. (I'll use 1.3 in this book.) The absolute rate of a language is the maximum number of bits that can be coded in each character, assuming each character sequence is equally likely. If there are L characters in a language, the absolute rate is:

R = log2 L

This is the maximum entropy of the individual characters.

For English, with 26 letters, the absolute rate is log2 26, or about 4.7 bits/letter. It should come as no surprise to anyone that the actual rate of English is much less than the absolute rate; natural language is highly redundant.

The redundancy of a language, denoted D, is defined by:

D = R - r

Given that the rate of English is 1.3, the redundancy is 3.4 bits/letter. This means that each English character carries 3.4 bits of redundant information.

An ASCII message that is nothing more than printed English has 1.3 bits of information per byte of message. This means it has 6.7 bits of redundant information, giving it an overall redundancy of 0.84 bits of information per bit of ASCII text, and an entropy of 0.16 bits of information per bit of ASCII text. The same message in BAUDOT, at 5 bits per character, has a redundancy of 0.74 bits per bit and an entropy of 0.26 bits per bit. Spacing, punctuation, numbers, and formatting modify these results.

Security of a Cryptosystem

Shannon defined a precise mathematical model of what it means for a cryptosystem to be secure. The goal of a cryptanalyst is to determine the key K, the plaintext P, or both. However, he may be satisfied with some probabilistic information about P: whether it is digitized audio, German text, spreadsheet data, or something else.

In most real-world cryptanalysis, the cryptanalyst has some probabilistic information about P before he even starts. He probably knows the language of the plaintext. This language has a certain redundancy associated with it. If it is a message to Bob, it probably begins with "Dear Bob." Certainly "Dear Bob" is more probable than "e8T&g [, m." The purpose of cryptanalysis is to modify the probabilities associated with each possible plaintext. Eventually one plaintext will emerge from the pile of possible plaintexts as certain (or at least, very probable).

There is such a thing as a cryptosystem that achieves perfect secrecy: a cryptosystem in which the ciphertext yields no possible information about the plaintext (except possibly its length). Shannon theorized that it is only possible if the number of possible keys is at least as large as the number of possible messages. In other words, the key must be at least as long as the message itself, and no key can be reused. In still other words, the one-time pad (see Section 1.5) is the only cryptosystem that achieves perfect secrecy.

Perfect secrecy aside, the ciphertext unavoidably yields some information about the corresponding plaintext. A good cryptographic algorithm keeps this information to a minimum; a good cryptanalyst exploits this information to determine the plaintext.

Cryptanalysts use the natural redundancy of language to reduce the number of possible plaintexts. The more redundant the language, the easier it is to cryptanalyze. This is the reason that many real-world cryptographic implementations use a compression program to reduce the size of the text before encrypting it. Compression reduces the redundancy of a message as well as the work required to encrypt and decrypt.

The entropy of a cryptosystem is a measure of the size of the keyspace, K. It is approximated by the base two logarithm of the number of keys:

H(K) = log2 K

A cryptosystem with a 64-bit key has an entropy of 64 bits; a cryptosystem with a 56-bit key has an entropy of 56 bits. In general, the greater the entropy, the harder it is to break a cryptosystem.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Unicity Distance

For a message of length n, the number of different keys that will decipher a ciphertext message to some intelligible plaintext in the same language as the original plaintext (such as an English text string) is given by the following formula [712,95]:

2H(K)- nD - 1

Shannon [1432] defined the unicity distance, U, also called the unicity point, as an approximation of the amount of ciphertext such that the sum of the real information (entropy) in the corresponding plaintext plus the entropy of the encryption key equals the number of ciphertext bits used. He then went on to show that ciphertexts longer than this distance are reasonably certain to have only one meaningful decryption. Ciphertexts significantly shorter than this are likely to have multiple, equally valid decryptions and therefore gain security from the opponent's difficulty in choosing the correct one.

For most symmetric cryptosystems, the unicity distance is defined as the entropy of the cryptosystem divided by the redundancy of the language.

U = H(K)/D

Unicity distance does not make deterministic predictions, but gives probabilistic results. Unicity distance estimates the minimum amount of ciphertext for which it is likely that there is only a single intelligible plaintext decryption when a brute-force attack is attempted. Generally, the longer the unicity distance, the better the cryptosystem. For DES, with a 56-bit key, and an ASCII English message, the unicity distance is about 8.2 ASCII characters or 66 bits. Table 11.1 gives the unicity distances for varying key lengths. The unicity distances for some classical cryptosystems are found in [445].

Unicity distance is not a measure of how much ciphertext is required for cryptanalysis, but how much ciphertext is required for there to be only one reasonable solution for cryptanalysis. A cryptosystem may be computationally infeasible to break even if it is theoretically possible to break it with a small amount of ciphertext. (The largely esoteric theory of relativized cryptography is relevant here [230, 231, 232, 233, 234, 235].) The unicity distance is inversely proportional to the redundancy. As redundancy approaches zero, even a trivial cipher can be unbreakable with a ciphertext-only attack.

Shannon defined a cryptosystem whose unicity distance is infinite as one that has ideal secrecy. Note that an ideal cryptosystem is not necessarily a perfect cryptosystem, although a perfect cryptosystem would necessarily be an ideal cryptosystem. If a cryptosystem has ideal secrecy, even successful cryptanalysis will leave some uncertainty about whether the recovered plaintext is the real plaintext.

Information Theory in Practice

While these concepts have great theoretical value, actual cryptanalysis seldom proceeds along these lines. Unicity distance guarantees insecurity if it's too small but does not guarantee security if it's high. Few practical algorithms are absolutely impervious to analysis; all manner of characteristics might serve as entering wedges to crack some encrypted messages. However, similar information theory considerations are occasionally useful, for example, to determine a recommended key change interval for a particular algorithm. Cryptanalysts also employ a variety of statistical and information theory tests to help guide the analysis in the most promising directions. Unfortunately, most literature on applying information theory to cryptanalysis remains classified, including the seminal 1940 work of Alan Turing.

Table 11.1 Unicity Distances of ASCII Text Encrypted with Algorithms with Varying Key Lengths
Key Length (in bits) | Unicity Distance (in characters)
40 | 5.9
56 | 8.2
64 | 9.4
80 | 11.8
128 | 18.8
256 | 37.6

Confusion and Diffusion

The two basic techniques for obscuring the redundancies in a plaintext message are, according to Shannon, confusion and diffusion [1432].

Confusion obscures the relationship between the plaintext and the ciphertext. This frustrates attempts to study the ciphertext looking for redundancies and statistical patterns. The easiest way to do this is through substitution. A simple substitution cipher, like the Caesar Cipher, is one in which every identical letter of plaintext is substituted for a single letter of ciphertext. Modern substitution ciphers are more complex: A long block of plaintext is substituted for a different block of ciphertext, and the mechanics of the substitution change with each bit in the plaintext or key. This type of substitution is not necessarily enough; the German Enigma is a complex substitution algorithm that was broken during World War II.

Diffusion dissipates the redundancy of the plaintext by spreading it out over the ciphertext. A cryptanalyst looking for those redundancies will have a harder time finding them. The simplest way to cause diffusion is through transposition (also called permutation). A simple transposition cipher, like columnar transposition, simply rearranges the letters of the plaintext. Modern ciphers do this type of permutation, but they also employ other forms of diffusion that can diffuse parts of the message throughout the entire message.

Stream ciphers rely on confusion alone, although some feedback schemes add diffusion. Block algorithms use both confusion and diffusion. As a general rule, diffusion alone is easily cracked (although double transposition ciphers hold up better than many other pencil-and-paper systems).

11.2 Complexity Theory

Complexity theory provides a methodology for analyzing the computational complexity of different cryptographic techniques and algorithms. It compares cryptographic algorithms and techniques and determines their security. Information theory tells us that all cryptographic algorithms (except one-time pads) can be broken. Complexity theory tells us whether they can be broken before the heat death of the universe.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Complexity of Algorithms

An algorithm's complexity is determined by the computational power needed to execute it. The computational complexity of an algorithm is often measured by two variables: T (for time complexity) and S (for space complexity, or memory requirement). Both T and S are commonly expressed as functions of n, where n is the size of the input. (There are other measures of complexity: the number of random bits, the communications bandwidth, the amount of data, and so on.)

Generally, the computational complexity of an algorithm is expressed in what is called "big O" notation: the order of magnitude of the computational complexity. It's just the term of the complexity function which grows the fastest as n gets larger; all lower-order terms are ignored. For example, if the time complexity of a given algorithm is 4n2 + 7n + 12, then the computational complexity is on the order of n2, expressed O(n2).

Measuring time complexity this way is system-independent. You don't have to know the exact timings of various instructions or the number of bits used to represent different variables or even the speed of the processor. One computer might be 50 percent faster than another and a third might have a data path twice as wide, but the order-of-magnitude complexity of an algorithm remains the same. This isn't cheating; when you're dealing with algorithms as complex as the ones presented here, the other stuff is negligible (is a constant factor) compared to the order-of-magnitude complexity.

This notation allows you to see how the input size affects the time and space requirements. For example, if T = O(n), then doubling the input size doubles the running time of the algorithm. If T = O(2n), then adding one bit to the input size doubles the running time of the algorithm (within a constant factor).

Generally, algorithms are classified according to their time or space complexities. An algorithm is constant if its complexity is independent of n: O(1). An algorithm is linear, if its time complexity is O(n). Algorithms can also be quadratic, cubic, and so on. All these algorithms are polynomial; their complexity is O(nm), when m is a constant. The class of algorithms that have a polynomial time complexity are called polynomial-time algorithms.

Algorithms whose complexities are O(t f(n)), where t is a constant greater than 1 and f (n) is some polynomial function of n, are called exponential. The subset of exponential algorithms whose complexities are O(c f(n)), where c is a constant and f (n) is more than constant but less than linear, is called superpolynomial.

Ideally, a cryptographer would like to be able to say that the best algorithm to break this encryption algorithm is of exponential-time complexity. In practice, the strongest statements that can be made, given the current state of the art of computational complexity theory, are of the form "all known cracking algorithms for this cryptosystem are of superpolynomial-time complexity." That is, the cracking algorithms that we know are of superpolynomial-time complexity, but it is not yet possible to prove that no polynomial-time cracking algorithm could ever be discovered. Advances in computational complexity may some day make it possible to design algorithms for which the existence of polynomial-time cracking algorithms can be ruled out with mathematical certainty.

As n grows, the time complexity of an algorithm can make an enormous difference in whether the algorithm is practical. Table 11.2 shows the running times for different algorithm classes in which n equals one million. The table ignores constants, but also shows why ignoring constants is reasonable.

Table 11.2 Running Times of Different Classes of Algorithms
Class | Complexity | # of Operations for n = 106 | Time at 106 O/S
Constant | O(1) | 1 | 1 µsec.
Linear | O(n) | 106 | 1 sec.
Quadratic | O(n2) | 1012 | 11.6 days
Cubic | O(n3) | 1018 | 32, 000 yrs.
Exponential | O(2n) | 10301,030 | 10301,006 times the age of the universe

Assuming that the unit of "time" for our computer is a microsecond, the computer can complete a constant algorithm in a microsecond, a linear algorithm in a second, and a quadratic algorithm in 11.6 days. It would take 32, 000 years to complete a cubic algorithm; not terribly practical, but a computer built to withstand the next ice age would deliver a solution eventually. Performing the exponential algorithm is futile, no matter how well you extrapolate computing power, parallel processing, or contact with superintelligent aliens.

Look at the problem of a brute-force attack against an encryption algorithm. The time complexity of this attack is proportional to the number of possible keys, which is an exponential function of the key length. If n is the length of the key, then the complexity of a brute-force attack is O(2n). Section 12.3 discusses the controversy surrounding a 56-bit key for DES instead of a 112-bit key. The complexity of a brute-force attack against a 56-bit key is 256; against a 112-bit key the complexity is 2112. The former is possible; the latter isn't.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Complexity of Problems

Complexity theory also classifies the inherent complexity of problems, not just the complexity of particular algorithms used to solve problems. (Excellent introductions to this topic are [600, 211, 1226]; see also [1096, 27, 739].) The theory looks at the minimum time and space required to solve the hardest instance of a problem on a theoretical computer known as a Turing machine . A Turing machine is a finite-state machine with an infinite read-write memory tape. It turns out that a Turing machine is a realistic model of computation.

Problems that can be solved with polynomial-time algorithms are called tractable, because they can usually be solved in a reasonable amount of time for reasonable-sized inputs. (The exact definition of "reasonable" depends on the circumstance.) Problems that cannot be solved in polynomial time are called intractable, because calculating their solution quickly becomes infeasible. Intractable problems are sometimes just called hard. Problems that can only be solved with algorithms that are superpolynomial are computationally intractable, even for relatively small values of n.

It gets worse. Alan Turing proved that some problems are undecidable . It is impossible to devise any algorithm to solve them, regardless of the algorithm's time complexity.

Problems can be divided into complexity classes, which depend on the complexity of their solutions. Figure 11.1 shows the more important complexity classes and their presumed relationships. (Unfortunately, not much about this material has been proved mathematically.)

On the bottom, the class P consists of all problems that can be solved in polynomial time. The class NP consists of all problems that can be solved in polynomial time only on a nondeterministic Turing machine: a variant of a normal Turing machine that can make guesses. The machine guesses the solution to the problem—either by making "lucky guesses" or by trying all guesses in parallel—and checks its guess in polynomial time.

NP 's relevance to cryptography is this: Many symmetric algorithms and all public-key algorithms can be cracked in nondeterministic polynomial time. Given a ciphertext C, the cryptanalyst simply guesses a plaintext, X, and a key, k, and in polynomial time runs the encryption algorithm on inputs X and k and checks whether the result is equal to C. This is important theoretically, because it puts an upper bound on the complexity of cryptanalysis for these algorithms. In practice, of course, it is a deterministic polynomial-time algorithm that the cryptanalyst seeks. Furthermore, this argument is not applicable to all classes of ciphers; in particular, it is not applicable to one-time pads—for any C, there are many X, k pairs that yield C when run through the encryption algorithm, but most of these X s are nonsense, not legitimate plaintexts.

Figure 11.1 Complexity classes.

The class NP includes the class P, because any problem solvable in polynomial time on a deterministic Turing machine is also solvable in polynomial time on a nondeterministic Turing machine; the guessing stage can simply be omitted.

If all NP problems are solvable in polynomial time on a deterministic machine, then P = NP. Although it seems obvious that some NP problems are much harder than others (a brute-force attack against an encryption algorithm versus encrypting a random block of plaintext), it has never been proven that P ≠ NP (or that P = NP). However, most people working in complexity theory believe that they are unequal.

Stranger still, specific problems in NP can be proven to be as difficult as any problem in the class. Steven Cook [365] proved that the Satisfiability problem (given a propositional Boolean formula, is there a way to assign truth values to the variables that makes the formula true?) is NP-complete . This means that, if Satisfiability is solvable in polynomial time, then P = NP. Conversely, if any problem in NP can be proven not to have a deterministic polynomial-time algorithm, the proof will show that Satisfiability does not have a deterministic polynomial-time algorithm either. No problem is harder than Satisfiability in NP.

Since Cook's seminal paper was published, a huge number of problems have been shown to be equivalent to Satisfiability; hundreds are listed in [600], and some examples follow. By equivalent, I mean that these problems are also NP-complete; they are in NP and also as hard as any problem in NP . If their solvability in deterministic polynomial time were resolved, the P versus NP question would be solved. The question of whether P = NP is the central unsolved question of computational complexity theory, and no one expects it to be solved anytime soon. If someone showed that P = NP, then most of this book would be irrelevant: As previously explained, many classes of ciphers are trivially breakable in nondeterministic polynomial time. If P = NP, they are breakable by feasible, deterministic algorithms.

Further out in the complexity hierarchy is PSPACE . Problems in PSPACE can be solved in polynomial space, but not necessarily polynomial time. PSPACE includes NP, but some problems in PSPACE are thought to be harder than NP. Of course, this isn't proven either. There is a class of problems, the so-called PSPACE-complete problems, with the property that, if any one of them is in NP then PSPACE = NP and if any one of them is in P then PSPACE = P .

And finally, there is the class of problems called EXPTIME . These problems are solvable in exponential time. The EXPTIME-complete problems can actually be proven not to be solvable in deterministic polynomial time. It has been shown that P does not equal EXPTIME .


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

NP-Complete Problems

Michael Garey and David Johnson compiled a list of over 300 NP-complete problems [600]. Here are just a few of them:

— Traveling Salesman Problem. A traveling salesman has to visit n different cities using only one tank of gas (there is a maximum distance he can travel). Is there a route that allows him to visit each city exactly once on that single tank of gas? (This is a generalization of the Hamiltonian Cycle problem—see Section 5.1.)
— Three-Way Marriage Problem. In a room are n men, n women, and n clergymen (priests, rabbis, whatever). There is also a list of acceptable marriages, which consists of one man, one woman, and one clergyman willing to officiate. Given this list of possible triples, is it possible to arrange n marriages such that everyone is either marrying one person or officiating at one marriage?
— Three-Satisfiability. There is a list of n logical statements, each with three variables. For example: if (x and y) then z, (x and w) or (not z), if ((not u and not x) or (z and (u or not x))) then (not z and u) or x), and so on. Is there a truth assignment for all the variables that satisfies all the statements? (This is a special case of the Satisfiability problem previously mentioned.)

11.3 Number Theory

This isn't a book on number theory, so I'm just going to sketch a few ideas that apply to cryptography. If you want a detailed mathematical text on number theory, consult one of these books: [1430, 72, 1171, 12, 959, 681, 742, 420]. My two favorite books on the mathematics of finite fields are [971, 1042]. See also [88, 1157, 1158, 1060].

Modular Arithmetic

You all learned modular arithmetic in school; it was called "clock arithmetic." Remember these word problems? If Mildred says she'll be home by 10:00, and she's 13 hours late, what time does she get home and for how many years does her father ground her? That's arithmetic modulo 12. Twenty-three modulo 12 equals 11.

(10 + 13) mod 12 = 23 mod 12 = 11 mod 12

Another way of writing this is to say that 23 and 11 are equivalent, modulo 12:

23 ≡ 11 (mod 12)

Basically, a ≡ b (mod n) if a = b + kn for some integer k. If a is non-negative and b is between 0 and n, you can think of b as the remainder of a when divided by n. Sometimes, b is called the residue of a, modulo n. Sometimes a is called congruent to b, modulo n (the triple equals sign, ≡, denotes congruence). These are just different ways of saying the same thing.

The set of integers from 0 to n - 1 form what is called a complete set of residues modulo n. This means that, for every integer a, its residue modulo n is some number from 0 to n - 1.

The operation a mod n denotes the residue of a, such that the residue is some integer from 0 to n - 1. This operation is modular reduction. For example, 5 mod 3 = 2.

This definition of mod may be different from the definition used in some programming languages. For example, PASCAL's modulo operator sometimes returns a negative number. It returns a number between -(n - 1) and n - 1. In C, the % operator returns the remainder from the division of the first expression by the second; this can be a negative number if either operand is negative. For all the algorithms in this book, make sure you add n to the result of the modulo operator if it returns a negative number.

Modular arithmetic is just like normal arithmetic: It's commutative, associative, and distributive. Also, reducing each intermediate result modulo n yields the same result as doing the whole calculation and then reducing the end result modulo n.

a + b) mod n = ((a mod n) + (b mod n)) mod n
(a - b) mod n = ((a mod n) - (b mod n)) mod n
(a*b) mod n = ((a mod n)*(b mod n)) mod n
(a*(b + c)) mod n = (((a*b) mod n) + ((a*c) mod n)) mod n

Cryptography uses computation mod n a lot, because calculating discrete logarithms and square roots mod n can be hard problems. Modular arithmetic is also easier to work with on computers, because it restricts the range of all intermediate values and the result. For a k- bit modulus, n, the intermediate results of any addition, subtraction, or multiplication will not be more than 2k- bits long. So we can perform exponentiation in modular arithmetic without generating huge intermediate results. Calculating the power of some number modulo some number,

ax mod n,

is just a series of multiplications and divisions, but there are speedups. One kind of speedup aims to minimize the number of modular multiplications; another kind aims to optimize the individual modular multiplications. Because the operations are distributive, it is faster to do the exponentiation as a stream of successive multiplications, taking the modulus every time. It doesn't make much difference now, but it will when you're working with 200-bit numbers.

For example, if you want to calculate a8 mod n, don't use the naïve approach and perform seven multiplications and one huge modular reduction:

(a*a*a*a*a*a*a*a) mod n

Instead, perform three smaller multiplications and three smaller modular reductions:

((a2 mod n)2 mod n)2 mod n

By the same token,

a16 mod n = (((a2 mod n)2 mod n)2 mod n)2 mod n


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Computing ax mod n, where x is not a power of 2, is only slightly harder. Binary notation expresses x as a sum of powers of 2: 25 is 11001 in binary, so 25 = 24 + 23 + 20. So

a25 mod n = (a*a24) mod n = (a*a8*a16) mod n
= (a*((a2)2)2*(((a2)2)2)2) mod n = ((((a2*a)2)2)2*a) mod n

With judicious storing of intermediate results, you only need six multiplications:

(((((((a2 mod n)*a) mod n)2 mod n)2 mod n)2 mod n)*a) mod n

This is called addition chaining [863], or the binary square and multiply method. It uses a simple and obvious addition chain based on the binary representation. In C, it looks like:

    unsigned long qe2(unsigned long x, unsigned long y, unsigned long n) {
        unsigned long s,t,u;
        int i;

        s = 1; t = x; u = y;
        while(u) {
            if(u&1) s = (s* t)%n;
            u>>=1;
            t = (t* t)%n;
        }
        return(s);
    }

Another, recursive, algorithm is:

    unsigned long fast_exp(unsigned long x, unsigned long y, unsigned long N) {
        unsigned long tmp;
         if(y==1) return(x % N);
         if ((y&1)==0) {
               tmp = fast_exp(x,y/2,N);
               return ((tmp* tmp)%N);
         }
         else {
               tmp = fast_exp(x,(y-1)/2,N);
               tmp = (tmp* tmp)%N;
               tmp = (tmp* x)%N;
               return (tmp);
         }
    }

This technique reduces the operation to, on the average, 1.5*k operations, if k is the length of the number x in bits. Finding the calculation with the fewest operations is a hard problem (it has been proven that the sequence must contain at least k- 1 operations), but it is not too hard to get the number of operations down to 1.1*k or better, as k grows.

An efficient way to do modular reductions many times using the same n is Montgomery's method [1111]. Another method is called Barrett's algorithm [87]. The software performance of these two algorithms and the algorithm previously discussed is in [210]: The algorithm I've discussed is the best choice for singular modular reductions; Barrett's algorithm is the best choice for small arguments; and Montgomery's method is the best choice for general modular exponentiations. (Montgomery's method can also take advantage of small exponents, using something called mixed arithmetic.)

The inverse of exponentiation modulo n is calculating a discrete logarithm . I'll discuss this shortly.

Prime Numbers

A prime number is an integer greater than 1 whose only factors are 1 and itself: No other number evenly divides it. Two is a prime number. So are 73, 2521, 2365347734339, and 2756839 - 1. There are an infinite number of primes. Cryptography, especially public-key cryptography, uses large primes (512 bits and even larger) often.

Evangelos Kranakis wrote an excellent book on number theory, prime numbers, and their applications to cryptography [896]. Paulo Ribenboim wrote two excellent references on prime numbers in general [1307, 1308].

Greatest Common Divisor

Two numbers are relatively prime when they share no factors in common other than 1. In other words, if the greatest common divisor of a and n is equal to 1. This is written:

gcd(a,n) = 1

The numbers 15 and 28 are relatively prime, 15 and 27 are not, and 13 and 500 are. A prime number is relatively prime to all other numbers except its multiples.

One way to compute the greatest common divisor of two numbers is with Euclid's algorithm . Euclid described the algorithm in his book, Elements, written around 300 B.C. He didn't invent it. Historians believe the algorithm could be 200 years older. It is the oldest nontrivial algorithm that has survived to the present day, and it is still a good one. Knuth describes the algorithm and some modern modifications [863].

In C:

    /* returns gcd of x and y */
    int gcd (int x, int y)
    {
         int g;
         if (x < 0)
          x = -x;
         if (y < 0)
          y = -y;
         if (x + y == 0)
          ERROR;
         g = y;
         while (x > 0) {
          g = x;
          x = y % x;
          y = g;
         }
         return g;
    }

This algorithm can be generalized to return the gcd of an array of m numbers:

    /* returns the gcd of x1, x2...xm */
    int multiple_gcd (int m, int *x)
    {
         size_t i;
         int g;

         if (m < 1)
               return 0;
         g = x[0];
         for (i=1; i<m; ++i) {
               g = gcd(g, x[i]);
    /* optimization, since for random x[i], g==1 60% of the time: */
                if (g == 1)
                     return 1;
         }
         return g;
    }

Inverses Modulo a Number

Remember inverses? The multiplicative inverse of 4 is 1/4, because 4*1/4 = 1. In the modulo world, the problem is more complicated:

4*x ≡ 1 (mod 7)

This equation is equivalent to finding an x and k such that

4x = 7k + 1

where both x and k are integers.

The general problem is finding an x such that

1 = (a*x) mod n

This is also written as

a -1 ≡ x (mod n)

The modular inverse problem is a lot more difficult to solve. Sometimes it has a solution, sometimes not. For example, the inverse of 5, modulo 14, is 3. On the other hand, 2 has no inverse modulo 14.

In general, a-1 ≡ x (mod n) has a unique solution if a and n are relatively prime. If a and n are not relatively prime, then a-1 ≡ x (mod n) has no solution. If n is a prime number, then every number from 1 to n- 1 is relatively prime to n and has exactly one inverse modulo n in that range.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

So far, so good. Now, how do you go about finding the inverse of a modulo n? There are a couple of ways. Euclid's algorithm can also compute the inverse of a number modulo n. Sometimes this is called the extended Euclidean algorithm.

Here's the algorithm in C++:

    #define isEven(x)   ((x & 0x01) == 0)
    #define isOdd(x)    (x & 0x01)
    #define swap(x,y)   (x ^= y, y ^= x, x ^= y)

    void ExtBinEuclid(int *u, int *v, int *u1, int *u2, int *u3)
    {
        // warning: u and v will be swapped if u < v
        int k, t1, t2, t3;

        if (*u < *v) swap(*u,*v);
        for (k = 0; isEven(*u) && isEven(*v); ++k) {
        *u >>= 1; *v >>= 1;
        }
        *u1 = 1; *u2 = 0; *u3 = *u; t1 = *v; t2 = *u-1; t3 = *v;
        do {
        do {
            if (isEven(*u3)) {
            if (isOdd(*u1) || isOdd(*u2)) {
                *u1 += *v; *u2 += *u;
            }
            *u1 >>= 1; *u2 >>= 1; *u3 >>= 1;
            }
            if (isEven(t3) || *u3 < t3) {
            swap(*u1,t1); swap(*u2,t2); swap(*u3,t3);
            }
        } while (isEven(*u3));
        while (*u1 < t1 || *u2 < t2) {
            *u1 += *v; *u2 += *u;
        }
        *u1 -= t1; *u2 -= t2; *u3 -= t3;
        } while (t3 > 0);
        while (*u1 >= *v && *u2 >= *u) {
        *u1 -= *v; *u2 -= *u;
        }
        *u1 <<= k; *u2 <<= k; *u3 <<= k;
    }

    main(int argc, char **argv) {
        int a, b, gcd;

        if (argc < 3) {
        cerr << "Usage: xeuclid u v" << endl;
        return -1;
        }
        int u = atoi(argv[1]);
        int v = atoi(argv[2]);
        if (u <= 0 || v <= 0) {
        cerr << "Arguments must be positive!" << endl;
        return -2;
        }
        // warning: u and v will be swapped if u < v
        ExtBinEuclid(&u, &v, &a, &b, &gcd);
        cout << a << " * " << u << " + (-"
        << b << ") * " << v << " = " << gcd << endl;
        if (gcd == 1)
        cout << "the inverse of " << v << " mod " << u << " is: "
            << u - b << endl;
        return 0;
    }

I'm not going to prove that it works or give the theory behind it. Details can be found in [863], or in any of the number theory texts previously listed.

The algorithm is iterative and can be slow for large numbers. Knuth showed that the average number of divisions performed by the algorithm is:

.843*log2(n) + 1.47

Solving for Coefficients

Euclid's algorithm can be used to solve this class of problems: Given an array of m variables x1, x2,...xm, find an array of m coefficients, u1, u2...um, such that

u1*x1 + ...+ um*xm = 1

Fermat's Little Theorem

If m is a prime, and a is not a multiple of m, then Fermat's little theorem says

am -1 ≡ 1 (mod m)

(Pierre de Fermat, pronounced "Fair-ma, " was a French mathematician who lived from 1601 to 1665. This theorem has nothing to do with his last theorem.)

The Euler Totient Function

There is another method for calculating the inverse modulo n, but it's not always possible to use it. The reduced set of residues mod n is the subset of the complete set of residues that is relatively prime to n. For example, the reduced set of residues mod 12 is {1, 5, 7, 11}. If n is prime, then the reduced set of residues mod n is the set of all numbers from 1 to n- 1. The number 0 is never part of the reduced set of residues for any n not equal to 1.

The Euler totient function, also called the Euler phi function and written as φ(n), is the number of elements in the reduced set of residues modulo n. In other words, φ(n) is the number of positive integers less than n that are relatively prime to n (for any n greater than 1). (Leonhard Euler, pronounced "Oiler, " was a Swiss mathematician who lived from 1707 to 1783.)

If n is prime, then φ(n) = n- 1. If n = pq, where p and q are prime, then φ(n) = (p- 1)(q- 1). These numbers appear in some public-key algorithms; this is why.

According to Euler's generalization of Fermat's little theorem, if gcd(a,n) = 1, then

aφ(n) mod n = 1

Now it is easy to compute a-1 mod n:

x = aφ(n)-1 mod n

For example, what is the inverse of 5, modulo 7? Since 7 is prime, φ(7) = 7- 1 = 6. So, the inverse of 5, modulo 7, is

56-1 mod 7 = 55 mod 7 = 3

Both methods for calculating inverses can be extended to solve for x in the general problem (if gcd(a,n) = 1):

(a*x) mod n = b

Using Euler's generalization, solve

x = (b*a φ(n)-1) mod n

Using Euclid's algorithm, solve

x = (b*(a-1 mod n)) mod n

In general, Euclid's algorithm is faster than Euler's generalization for calculating inverses, especially for numbers in the 500-bit range. If gcd(a,n) ≠ 1, all is not lost. In this general case, (a*x) mod n = b, can have multiple solutions or no solution.

Chinese Remainder Theorem

If you know the prime factorization of n, then you can use something called the Chinese remainder theorem to solve a whole system of equations. The basic version of this theorem was discovered by the first-century Chinese mathematician, Sun Tse.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In general, if the prime factorization of n is p1*p2*...*pt, then the system of equations

(x mod pi) = ai, where i = 1, 2,..., t

has a unique solution, x, where x is less than n. (Note that some primes can appear more than once. For example, p1 might be equal to p2 .) In other words, a number (less than the product of some primes) is uniquely identified by its residues mod those primes.

For example, use 3 and 5 as primes, and 14 as the number. 14 mod 3 = 2, and 14 mod 5 = 4. There is only one number less than 3*5 = 15 which has those residues: 14. The two residues uniquely determine the number.

So, for an arbitrary a < p and b < q (where p and q are prime), there exists a unique x, where x is less than pq, such that

x ≡ a (mod p), and x ≡ b (mod q)

To find this x, first use Euclid's algorithm to find u, such that

u*q ≡ 1 (mod p)

Then compute:

x = (((a - b)*u) mod p)*q + b

Here is the Chinese remainder theorem in C:

    /* r is the number of elements in arrays m and u;
    m is the array of (pairwise relatively prime) moduli
    u is the array of coefficients
    return value is n such than n == u[k]%m[k] (k=0..r-1) and
        n < m[0]*m[1]*...*m[r-1]
    */

    /* totient() is left as an exercise to the reader. */
    int chinese_remainder (size_t r, int *m, int *u)
    {
        size_t i;
        int modulus;
        int n;
        modulus = 1;
        for (i=0; i<r; ++i)
        modulus *= m[i];

        n = 0;
        for (i=0; i<r; ++i) {
        n += u[i] * modexp(modulus / m[i], totient(m[i]),
        m[i]);
        n %= modulus;
        }

        return n;
    }

The converse of the Chinese remainder theorem can also be used to find the solution to the problem: if p and q are primes, and p is less than q, then there exists a unique x less than pq, such that

a ≡ x (mod p), and b ≡ x (mod q)

If a ≥ b mod p, then

x = (((a - (b mod p))*u) mod p)*q + b

If a < b mod p, then

x = (((a + p - (b mod p))*u) mod p)*q + b

Quadratic Residues

If p is prime, and a is greater than 0 and less than p, then a is a quadratic residue mod p if

x2 ≡ a (mod p), for some x

Not all values of a satisfy this property. For a to be a quadratic residue modulo n, it must be a quadratic residue modulo all the prime factors of n. For example, if p = 7, the quadratic residues are 1, 2, and 4:

12 = 1 ≡ 1 (mod 7)
22 = 4 ≡ 4 (mod 7)
32 = 9 ≡ 2 (mod 7)
42 = 16 ≡ 2 (mod 7)
52 = 25 ≡ 4 (mod 7)
62 = 36 ≡ 1 (mod 7)

Note that each quadratic residue appears twice on this list.

There are no values of x which satisfy any of these equations:

x2 ≡ 3 (mod 7)
x2 ≡ 5 (mod 7)
x2 ≡ 6 (mod 7)

The quadratic nonresidues modulo 7, the numbers that are not quadratic residues, are 3, 5, and 6.

Although I will not do so here, it is easy to prove that, when p is odd, there are exactly (p- 1)/2 quadratic residues mod p and the same number of quadratic nonresidues mod p. Also, if a is a quadratic residue mod p, then a has exactly two square roots, one of them between 0 and (p- 1)/2, and the other between (p- 1)/2 and (p- 1). One of these square roots is also a quadratic residue mod p; this is called the principal square root.

If n is the product of two primes, p and q, there are exactly (p- 1)(q- 1)/4 quadratic residues mod n. A quadratic residue mod n is a perfect square modulo n. This is because to be a square mod n, the residue must be a square mod p and a square mod q. For example, there are 11 quadratic residues mod 35: 1, 4, 9, 11, 14, 15, 16, 21, 25, 29, and 30. Each quadratic residue has exactly four square roots.

Legendre Symbol

The Legendre symbol, written L(a,p), is defined when a is any integer and p is a prime greater than 2. It is equal to 0, 1, or -1.

L(a,p) = 0 if a is divisible by p.
L(a,p) = 1 if a is a quadratic residue mod p.
L(a,p) = - 1 is a is a quadratic nonresidue mod p.

One way to calculate L(a,p) is:

L(a,p) = a(p- 1)/2 mod p

Or you can use the following algorithm:

1. If a = 1, then L(a,p) = 1
2. If a is even, then L(a,p) = L(a /2,p)*(- 1)(p2- 1)/8
3. If a is odd (and ≠ 1), then L(a,p) = L(p mod a,a)*(- 1)(a- 1)*(p- 1)/4

Note that this is also an efficient way to determine whether a is a quadratic residue mod p (when p is prime).


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Jacobi Symbol

The Jacobi symbol, written J(a,n), is a generalization of the Legendre symbol to composite moduli; it is defined for any integer a and any odd integer n. The function shows up in primality testing. The Jacobi symbol is a function on the set of reduced residues of the divisors of n and can be calculated by several formulas [1412]. This is one method:

Definition 1: J(a,n) is only defined if n is odd.
Definition 2: J(0,n) = 0.
Definition 3: If n is prime, then the Jacobi symbol J(a,n) = 0 if n divides a.
Definition 4: If n is prime, then the Jacobi symbol J(a,n) = 1 if a is a quadratic residue modulo n.
Definition 5: If n is prime, then the Jacobi symbol J(a,n) = - 1 if a is a quadratic nonresidue modulo n.
Definition 6: If n is composite, then the Jacobi symbol J(a,n) = J(a,p1) *...* J(a,pm), where p1...pm is the prime factorization of n.

The following algorithm computes the Jacobi symbol recursively:

Rule 1: J(1,n) = 1
Rule 2: J(a*b,n) = J(a,n)*J(b,n)
Rule 3: J(2,n) = 1 if (n2 - 1)/8 is even, and - 1 otherwise
Rule 4: J(a,n) = J((a mod n),n)
Rule 5: J(a,b1*b2) = J(a,b1)*J(a,b2)
Rule 6: If the greatest common divisor of a and b = 1, and a and b are odd:
Rule 6a: J(a,b) = J(b,a) if (a - 1)(b - 1)/4 is even
Rule 6b: J(a,b) = - J(b,a) if (a-1)(b-1)/4 is odd

Here is the algorithm in C:

    /* This algorithm computes the Jacobi symbol recursively */
    int jacobi(int a, int b)
    {
    int g;
        assert(odd(b));
        if (a >= b) a %= b;       /* by Rule 4 */
        if (a == 0) return 0;       /* by Definition 2 */
        if (a == 1) return 1;       /* by Rule 1 */
        if (a < 0)
        if (((b-1)/2 % 2 == 0)
            return jacobi(-a,b);
        else
            return -jacobi(-a,b);
        if (a % 2 == 0) /* a is even */
        if (((b*b - 1)/8) % 2 == 0)
            return +jacobi(a/2, b)
        else
            return -jacobi(a/2, b) /* by Rule 3 and Rule 2 */
        g = gcd(a,b);
        assert(odd(a)); /* this is guaranteed by the (a % 2 == 0)
    test */
        if (g == a) /* a exactly divides b */
        return 0; /* by Rules 5 and 4, and Definition 2 */
        else if (g != 1)
        return jacobi(g,b) * jacobi(a/g, b); /* by Rule 2 */
        else if (((a-1)*(b-1)/4) % 2 == 0)
        return +jacobi(b,a);    /* by Rule 6a */
        else
        return -jacobi(b,a);    /* by Rule 6b */
    }

If n is known to be prime beforehand, simply compute a((n-1)/2) mod n instead of running the previous algorithm; in this case J(a,n) is equivalent to the Legendre symbol.

The Jacobi symbol cannot be used to determine whether a is a quadratic residue mod n (unless n is prime, of course). Note that, if J(a,n) = 1 and n is composite, it is not necessarily true that a is a quadratic residue modulo n. For example:

J(7, 143) = J(7, 11)*J(7, 13) = (- 1)(- 1) = 1

However, there is no integer x such that x2 ≡ 7 (mod 143).

Blum Integers

If p and q are two primes, and both are congruent to 3 modulo 4, then n = pq is sometimes called a Blum integer. If n is a Blum integer, each quadratic residue has exactly four square roots, one of which is also a square; this is the principal square root. For example, the principal square root of 139 mod 437 is 24. The other three square roots are 185, 252, and 413.

Generators

If p is a prime, and g is less than p, then g is a generator mod p if

for each b from 1 to p - 1, there exists some a where ga ≡ b (mod p).

Another way of saying this is that g is primitive with respect to p.

For example, if p = 11, 2 is a generator mod 11:

210 = 1024 ≡ 1 (mod 11)
21 = 2 ≡ 2 (mod 11)
28 = 256 ≡ 3 (mod 11)
22 = 4 ≡ 4 (mod 11)
24 = 16 ≡ 5 (mod 11)
29 = 512 ≡ 6 (mod 11)
27 = 128 ≡ 7 (mod 11)
23 = 8 ≡ 8 (mod 11)
26 = 64 ≡ 9 (mod 11)
25 = 32 ≡ 10 (mod 11)

Every number from 1 to 10 can be expressed as 2a (mod p).

For p = 11, the generators are 2, 6, 7, and 8. The other numbers are not generators. For example, 3 is not a generator because there is no solution to

3a = 2 (mod 11)

In general, testing whether a given number is a generator is not an easy problem. It is easy, however, if you know the factorization of p - 1. Let q1, q2,..., qn be the distinct prime factors of p - 1. To test whether a number g is a generator mod p, calculate

g(p- 1)/q mod p

for all values of q = q1, q2,..., qn.

If that number equals 1 for some value of q, then g is not a generator. If that value does not equal 1 for any values of q, then g is a generator.

For example, let p = 11. The prime factors of p - 1 = 10 are 2 and 5. To test whether 2 is a generator:

2(11- 1)/5 (mod 11) = 4
2(11- 1)/2 (mod 11) = 10


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Neither result is 1, so 2 is a generator.

To test whether 3 is a generator:

3(11- 1)/5 (mod 11) = 9
3(11- 1)/2 (mod 11) = 1

Therefore, 3 is not a generator.

If you need to find a generator mod p, simply choose a random number from 1 to p - 1 and test whether it is a generator. Enough of them will be, so you'll probably find one fast.

Computing in a Galois Field

Don't be alarmed; that's what we were just doing. If n is prime or the power of a large prime, then we have what mathematicians call a finite field . In honor of that fact, we use p instead of n. In fact, this type of finite field is so exciting that mathematicians gave it its own name: a Galois field, denoted as GF(p). (ƒvariste Galois was a French mathematician who lived in the early nineteenth century and did a lot of work in number theory before he was killed at age 20 in a duel.)

In a Galois field, addition, subtraction, multiplication, and division by nonzero elements are all well-defined. There is an additive identity, 0, and a multiplicative identity, 1. Every nonzero number has a unique inverse (this would not be true if p were not prime). The commutative, associative, and distributive laws are true.

Arithmetic in a Galois field is used a great deal in cryptography. All of the number theory works; it keeps numbers a finite size, and division doesn't have any rounding errors. Many cryptosystems are based on GF(p), where p is a large prime.

To make matters even more complicated, cryptographers also use arithmetic modulo irreducible polynomials of degree n whose coefficients are integers modulo q, where q is prime. These fields are called GF(qn). All arithmetic is done modulo p (x), where p (x) is an irreducible polynomial of degree n.

The mathematical theory behind this is far beyond the scope of the book, although I will describe some cryptosystems that use it. If you want to try to work more with this, GF(23) has the following elements: 0, 1, x, x + 1, x2, x2 + 1, x2 + x, x2 + x + 1. There is an algorithm for computing inverses in GF(2n) that is suitable for parallel implementation [421].

When talking about polynomials, the term "prime" is replaced by "irreducible." A polynomial is irreducible if it cannot be expressed as the product of two other polynomials (except for 1 and itself, of course). The polynomial x2 + 1 is irreducible over the integers. The polynomial x3 + 2x2 + x is not; it can be expressed as x (x + 1)(x + 1).

A polynomial that is a generator in a given field is called primitive; all its coefficients are relatively prime. We'll see primitive polynomials again when we talk about linear-feedback shift registers (see Section 16.2).

Computation in GF(2n) can be quickly implemented in hardware with linear-feedback shift registers. For that reason, computation over GF(2n) is often quicker than computation over GF(p). Just as exponentiation is much more efficient in GF(2n), so is calculating discrete logarithms [180, 181, 368, 379]. If you want to learn more about this, read [140].

For a Galois field GF(2n), cryptographers like to use the trinomial p (x) = xn + x + 1 as the modulus, because the long string of zeros between the xn and x coefficients makes it easy to implement a fast modular multiplication [183]. The trinomial must be primitive, otherwise the math does not work. Values of n less than 1000 [1649, 1648] for which xn + x + 1 is primitive are:

1, 3, 4, 6, 9, 15, 22, 28, 30, 46, 60, 63, 127, 153, 172, 303, 471, 532,
865, 900

There exists a hardware implementation of GF(2127) where p (x) = x127 + x + 1 [1631, 1632, 1129]. Efficient hardware architectures for implementing exponentiation in GF(2n) are discussed in [147].

11.4 Factoring

Factoring a number means finding its prime factors.

10 = 2*5
60 = 2*2*3*5
252601 = 41*61*101
2113 - 1 = 3391*23279*65993*1868569*1066818132868207

The factoring problem is one of the oldest in number theory. It's simple to factor a number, but it's time-consuming. This is still true, but there have been some major advances in the state of the art.

Currently, the best factoring algorithm is:

Number field sieve (NFS) [953] (see also [952,16,279]). The general number field sieve is the fastest-known factoring algorithm for numbers larger than 110 digits or so [472,635]. It was impractical when originally proposed, but that has changed due to a series of improvements over the last few years [953]. The NFS is still too new to have broken any factoring records, but this will change soon. An early version was used to factor the ninth Fermat number: 2512 + 1 [955,954].

Other factoring algorithms have been supplanted by the NFS:

Quadratic sieve (QS) [1257,1617,1259]. This is the fastest-known algorithm for numbers less than 110 decimal digits long and has been used extensively [440]. A faster version of this algorithm is called the multiple polynomial quadratic sieve [1453,302]. The fastest version of this algorithm is called the double large prime variation of the multiple polynomial quadratic sieve.
Elliptic curve method (ECM) [957,1112,1113]. This method has been used to find 43-digit factors, but nothing larger.
Pollard's Monte Carlo algorithm [1254,248]. (This algorithm also appears in volume 2, page 370 of Knuth [863].)
Continued fraction algorithm. See [1123,1252,863]. This algorithm isn't even in the running.
Trial division. This is the oldest factoring algorithm and consists of testing every prime number less than or equal to the square root of the candidate number.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

See [251] for a good introduction to these different factoring algorithms, except for the NFS. The best discussion of the NFS is [953]. Other, older references are [505, 1602, 1258]. Information on parallel factoring can be found in [250].

If n is the number being factored, the fastest QS variants have a heuristic asymptotic run time of:

e(1+ 0(1))(ln (n))(1/2)(ln (ln (n)))(1/2)

The NFS is much faster, with a heuristic asymptotic time estimate of:

e(1.923+ 0(1))(ln (n))(1/3)(ln (ln (n)))(2/3)

In 1970, the big news was the factoring of a 41-digit hard number [1123]. (A "hard" number is one that does not have any small factors and is not of a special form that allows it to be factored more easily.) Ten years later, factoring hard numbers twice that size took a Cray computer just a few hours [440].

In 1988, Carl Pomerance designed a modular factoring machine, using custom VLSI chips [1259]. The size of the number you would be able to factor depends on how large a machine you can afford to build. He never built it.

In 1993, a 120-digit hard number was factored using the quadratic sieve; the calculation took 825 mips-years and was completed in three months real time [463]. Other results are [504].

Today's factoring attempts use computer networks [302, 955]. In factoring a 116-digit number, Arjen Lenstra and Mark Manasse used 400 mips-years—the spare time on an array of computers around the world for a few months.

In March 1994, a 129-digit (428-bit) number was factored using the double large prime variation of the multiple polynomial QS [66] by a team of mathematicians led by Lenstra. Volunteers on the Internet carried out the computation: 600 people and 1600 machines over the course of eight months, probably the largest ad hoc multiprocessor ever assembled. The calculation was the equivalent of 4000 to 6000 mips-years. The machines communicated via electronic mail, sending their individual results to a central repository where the final steps of analysis took place. This computation used the QS and five-year-old theory; it would have taken one-tenth the time using the NFS [949]. According to [66]: "We conclude that commonly used 512-bit RSA moduli are vulnerable to any organization prepared to spend a few million dollars and to wait a few months." They estimate that factoring a 512-bit number would be 100 times harder using the same technology, and only 10 times harder using the NFS and current technology [949].

To keep up on the state of the art of factoring, RSA Data Security, Inc. set up the RSA Factoring Challenge in March 1991 [532]. The challenge consists of a list of hard numbers, each the product of two primes of roughly equal size. Each prime was chosen to be congruent to 2 modulo 3. There are 42 numbers in the challenge, one each of length 100 digits through 500 digits in steps of 10 digits (plus one additional number, 129 digits long). At the time of writing, RSA-100, RSA-110, RSA-120, and RSA-129 have been factored, all using the QS. RSA-130 might be next (using the NFS), or the factoring champions might skip directly to RSA-140.

This is a fast-moving field. It is difficult to extrapolate factoring technology because no one can predict advances in mathematical theory. Before the NFS was discovered, many people conjectured that the QS was asymptotically as fast as any factoring method could be. They were wrong.

Near-term advances in the NFS are likely to come in the form of bringing down the constant: 1.923. Some numbers of a special form, like Fermat numbers, have a constant more along the lines of 1.5 [955, 954]. If the hard numbers used in public-key cryptography had that kind of constant, 1024-bit numbers could be factored today. One way to lower the constant is to find better ways of representing numbers as polynomials with small coefficients. The problem hasn't been studied very extensively yet, but it is probable that advances are coming [949].

For the most current results from the RSA Factoring Challenge, send e-mail to challenge-info@rsa.com.

Square Roots Modulo n

If n is the product of two primes, then the ability to calculate square roots mod n is computationally equivalent to the ability to factor n [1283, 35, 36, 193]. In other words, someone who knows the prime factors of n can easily compute the square roots of a number mod n, but for everyone else the computation has been proven to be as hard as computing the prime factors of n.

11.5 Prime Number Generation

Public-key algorithms need prime numbers. Any reasonably sized network needs lots of them. Before discussing the mathematics of prime number generation, I will answer a few obvious questions.

1. If everyone needs a different prime number, won't we run out? No. In fact, there are approximately 10151 primes 512 bits in length or less. For numbers near n, the probability that a random number is prime is approximately one in ln n. So the total number of primes less than n is n /(ln n). There are only 1077 atoms in the universe. If every atom in the universe needed a billion new primes every microsecond from the beginning of time until now, you would only need 10109 primes; there would still be approximately 10151 512-bit primes left.
2. What if two people accidentally pick the same prime number? It won't happen. With over 10151 prime numbers to choose from, the odds of that happening are significantly less than the odds of your computer spontaneously combusting at the exact moment you win the lottery.
3. If someone creates a database of all primes, won't he be able to use that database to break public-key algorithms? Yes, but he can't do it. If you could store one gigabyte of information on a drive weighing one gram, then a list of just the 512-bit primes would weigh so much that it would exceed the Chandrasekhar limit and collapse into a black hole...so you couldn't retrieve the data anyway.

But if factoring numbers is so hard, how can generating prime numbers be easy? The trick is that the yes/no question, "Is n prime?" is a much easier question to answer than the more complicated question, "What are the factors of n? "


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The wrong way to find primes is to generate random numbers and then try to factor them. The right way is to generate random numbers and test if they are prime. There are several probabilistic primality tests; tests that determine whether a number is prime with a given degree of confidence. Assuming this "degree of confidence" is large enough, these sorts of tests are good enough. I've heard primes generated in this manner called "industrial-grade primes": These are numbers that are probably prime with a controllably small chance of error.

Assume a test is set to fail once in 250 tries. This means that there is a 1 in 1015 chance that the test falsely indicates that a composite number is prime. (The test will never falsely indicate that a prime number is composite.) If for some reason you need more confidence that the number is prime, you can set the failure level even lower. On the other hand, if you consider that the odds of the number being composite are 300 million times less than the odds of winning top prize in a state lottery, you might not worry about it so much.

Overviews of recent developments in the field can be found in [1256, 206]. Other important papers are [1490, 384, 11, 19, 626, 651, 911].

Solovay-Strassen

Robert Solovay and Volker Strassen developed a probabilistic primality testing algorithm [1490]. Their algorithm uses the Jacobi symbol to test if p is prime:

(1) Choose a random number, a, less than p.
(2) If the gcd(a,p) ≠ 1, then p fails the test and is composite.
(3) Calculate j = a(p- 1)/2 mod p.
(4) Calculate the Jacobi symbol J(a,p).
(5) If j ≠ J(a,p), then p is definitely not prime.
(6) If j = J(a,p), then the likelihood that p is not prime is no more than 50 percent.

A number a that does not indicate that p is definitely not prime is called a witness. If p is composite, the odds of a random a being a witness is no less than 50 percent. Repeat this test t times, with t different random values for a. The odds of a composite number passing all t tests is no more than one in 2t.

Lehmann

Another, simpler, test was developed independently by Lehmann [945]. Here it tests if p is prime:

(1) Choose a random number a less than p.
(2) Calculate a(p- 1)/2 mod p.
(3) If a(p- 1)/2 ≠ 1 or - 1 (mod p), then p is definitely not prime.
(4) If a(p- 1)/2 ≡ 1 or - 1 (mod p), then the likelihood that p is not prime is no more than 50 percent.

Again, the odds of a random a being a witness to p 's compositeness is no less than 50 percent. Repeat this test t times. If the calculation equals 1 or -1, but does not always equal 1, then p is probably prime with an error rate of 1 in 2t .

Rabin-Miller

The algorithm everyone uses—it's easy—was developed by Michael Rabin, based in part on Gary Miller's ideas [1093, 1284]. Actually, this is a simplified version of the algorithm recommended in the DSS proposal [1149, 1154].

Choose a random number, p, to test. Calculate b, where b is the number of times 2 divides p - 1 (i.e., 2b is the largest power of 2 that divides p - 1). Then calculate m, such that p = 1 + 2b *m.

(1) Choose a random number, a, such that a is less than p.
(2) Set j = 0 and set z = am mod p.
(3) If z = 1, or if z = p - 1, then p passes the test and may be prime.
(4) If j > 0 and z = 1, then p is not prime.
(5) Set j = j + 1. If j < b and z ≠ p - 1, set z = z2 mod p and go back to step (4). If z = p - 1, then p passes the test and may be prime.
(6) If j = b and z ≠ p - 1, then p is not prime.

The odds of a composite passing decreases faster with this test than with previous ones. Three-quarters of the possible values of a are guaranteed to be witnesses. This means that a composite number will slip through t tests no more than ¼t of the time, where t is the number of iterations. Actually, these numbers are very pessimistic. For most random numbers, something like 99.9 percent of the possible a values are witnesses [96].

There are even better estimations [417]. For n- bit candidate primes (where n is more than 100), the odds of error in one test are less than 1 in 4n 2(k/2)(1/2). And for a 256-bit n, the odds of error in six tests are less than 1 in 251 . More theory is in [418].

Practical Considerations

In real-world implementations, prime generation goes quickly.

(1) Generate a random n- bit number, p.
(2) Set the high-order and low-order bit to 1. (The high-order bit ensures that the prime is of the required length and the low-order bit ensures that it is odd.)
(3) Check to make sure p is not divisible by any small primes: 3, 5, 7, 11, and so on. Many implementations test p for divisibility by all primes less than 256. The most efficient is to test for divisibility by all primes less than 2000 [949]. You can do this efficiently using a wheel [863].
(4) Perform the Rabin-Miller test for some random a. If p passes, generate another random a and go through the test again. Choose a small value of a to make the calculations go quicker. Do five tests [651]. (One might seem like enough, but do five.) If p fails one of the tests, generate another p and try again.

Another option is not to generate a random p each time, but to incrementally search through numbers starting at a random point until you find a prime.

Step (3) is optional, but it is a good idea. Testing a random odd p to make sure it is not divisible by 3, 5, and 7 eliminates 54 percent of the odd numbers before you get to step (4). Testing against all primes less than 100 eliminates 76 percent of the odd numbers; testing against all primes less than 256 eliminates 80 percent. In general, the fraction of odd candidates that is not a multiple of any prime less than n is 1.12/ln n. The larger the n you test up to, the more precomputation is required before you get to the Rabin-Miller test.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

One implementation of this method on a Sparc II was able to find 256-bit primes in an average of 2.8 seconds, 512-bit primes in an average of 24.0 seconds, 768-bit primes in an average of 2.0 minutes, and 1024-bit primes in an average of 5.1 minutes [918].

Strong Primes

If n is the product of two primes, p and q, it may be desirable to use strong primes for p and q. These are prime numbers with certain properties that make the product n difficult to factor by specific factoring methods. Among the properties suggested have been [1328,651]:

The greatest common divisor of p - 1 and q - 1 should be small.
Both p - 1 and q - 1 should have large prime factors, respectively p' and q'.
Both p' - 1 and q' - 1 should have large prime factors.
Both p + 1 and q + 1 should have large prime factors.
Both (p - 1)/2 and (q - 1)/2 should be prime [182]. (Note that if this condition is true, then so are the first two.)

Whether strong primes are necessary is a subject of debate. These properties were designed to thwart some older factoring algorithms. However, the fastest factoring algorithms have as good a chance of factoring numbers that meet these criteria as they do of factoring numbers that do not [831].

I recommend against specifically generating strong primes. The length of the primes is much more important than the structure. Moreover, structure may be damaging because it is less random.

This may change. New factoring techniques may be developed that work better on numbers with certain properties than on numbers without them. If so, strong primes may be required once again. Check current theoretical mathematics journals for updates.

11.6 Discrete Logarithms in a Finite Field

Modular exponentiation is another one-way function used frequently in cryptography. Evaluating this expression is easy:

ax mod n

The inverse problem of modular exponentiation is that of finding the discrete logarithm of a number. This is a hard problem:

Find x where ax ≡ b (mod n).

For example:

If 3x ≡ 15 mod 17, then x = 6

Not all discrete logarithms have solutions (remember, the only valid solutions are integers). It's easy to see that there is no solution, x, to the equation

3x = 7 (mod 13)

It's far more difficult to solve these problems using 1024-bit numbers.

Calculating Discrete Logarithms in a Finite Group

There are three main groups whose discrete logarithms are of interest to cryptographers:

— The multiplicative group of prime fields: GF(p)
— The multiplicative group of finite fields of characteristic 2: GF(2n)
— Elliptic curve groups over finite fields F : EC(F)

The security of many public-key algorithms is based on the problem of finding discrete logarithms, so the problem has been extensively studied. A good comprehensive overview of the problem, and the best solutions at the time, can be found in [1189, 1039]. The best current article on the topic is [934].

If p is the modulus and is prime, then the complexity of finding discrete logarithms in GF(p) is essentially the same as factoring an integer n of about the same size, when n is the product of two approximately equal-length primes [1378, 934]. This is:

e(1+ 0(1))(ln (p))(1/2)(ln (ln (p)))(1/2)

The number field sieve is faster, with an heuristic asymptotic time estimate of

e(1.923+ 0(1))(ln (p))(1/3)(ln (ln (p)))(2/3)

Stephen Pohlig and Martin Hellman found a fast way of computing discrete logarithms in GF(p) if p - 1 has only small prime factors [1253]. For this reason, only fields where p - 1 has at least one large factor are used in cryptography. Another algorithm [14] computes discrete logarithms at a speed comparable to factoring; it has been expanded to fields of the form GF(pn) [716]. This algorithm was criticized [727] for having some theoretical problems. Other articles [1588] show how difficult the problem really is.

Computing discrete logarithms is closely related to factoring. If you can solve the discrete logarithm problem, then you can factor. (The converse has never been proven to be true.) Currently, there are three methods for calculating discrete logarithms in a prime field [370, 934, 648]: the linear sieve, the Gaussian integer scheme, and the number field sieve.

The preliminary, extensive computing has to be done only once per field. Afterward, individual logarithms can be quickly calculated. This can be a security disadvantage for systems based on these fields. It is important that different applications use different prime fields. Multiple users in the same application can use a common field, though.

In the world of extension fields, GF(2n) hasn't been ignored by researchers. An algorithm was proposed in [727]. Coppersmith's algorithm makes finding discrete logarithms in fields such as GF(2127) reasonable and finding them in fields around GF(2400) possible [368]. This was based on work in [180]. The precomputation stage of this algorithm is enormous, but otherwise it is nice and efficient. A practical implementation of a less efficient version of the same algorithm, after a seven-hour precomputation period, found discrete logs in GF(2127) in several seconds each [1130, 180]. (This particular field, once used in some cryptosystems [142, 1631, 1632], is insecure.) For surveys of some of these results, consult [1189, 1039].

More recently, the precomputations for GF(2227), GF(2313), and GF(2401) are done, and significant progress has been made towards GF(2503). These calculations are being executed on an nCube-2 massively parallel computer with 1024 processors [649, 650]. Computing discrete logarithms in GF(2593) is still barely out of reach.

Like discrete logarithms in a prime field, the precomputation required to calculate discrete logarithms in a polynomial field has to be done only once. Taher ElGamal [520] gives an algorithm for calculating discrete logs in the field GF(p2).


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 12
Data Encryption Standard (DES)

12.1 Background

The Data Encryption Standard (DES), known as the Data Encryption Algorithm (DEA) by ANSI and the DEA-1 by the ISO, has been a worldwide standard for 20 years. Although it is showing signs of old age, it has held up remarkably well against years of cryptanalysis and is still secure against all but possibly the most powerful of adversaries.

Development of the Standard

In the early 1970s, nonmilitary cryptographic research was haphazard. Almost no research papers were published in the field. Most people knew that the military used special coding equipment to communicate, but few understood the science of cryptography. The National Security Agency (NSA) had considerable knowledge, but they did not even publicly admit their own existence.

Buyers didn't know what they were buying. Several small companies made and sold cryptographic equipment, primarily to overseas governments. The equipment was all different and couldn't interoperate. No one really knew if any of it was secure; there was no independent body to certify the security. As one government report said [441]:

The intricacies of relating key variations and working principles to the real strength of the encryption/decryption equipment were, and are, virtually unknown to almost all buyers, and informed decisions as to the right type of on-line, off-line, key generation, etc., which will meet buyers' security needs, have been most difficult to make.

In 1972, the National Bureau of Standards (NBS), now the National Institute of Standards and Technology (NIST), initiated a program to protect computer and communications data. As part of that program, they wanted to develop a single, standard cryptographic algorithm. A single algorithm could be tested and certified, and different cryptographic equipment using it could interoperate. It would also be cheaper to implement and readily available.

In the May 15, 1973 Federal Register, the NBS issued a public request for proposals for a standard cryptographic algorithm. They specified a series of design criteria:

— The algorithm must provide a high level of security.
— The algorithm must be completely specified and easy to understand.
— The security of the algorithm must reside in the key; the security should not depend on the secrecy of the algorithm.
— The algorithm must be available to all users.
— The algorithm must be adaptable for use in diverse applications.
— The algorithm must be economically implementable in electronic devices.
— The algorithm must be efficient to use.
— The algorithm must be able to be validated.
— The algorithm must be exportable.

Public response indicated that there was considerable interest in a cryptographic standard, but little public expertise in the field. None of the submissions came close to meeting the requirements.

The NBS issued a second request in the August 27, 1974 Federal Register. Eventually they received a promising candidate: an algorithm based on one developed by IBM during the early 1970s, called Lucifer (see Section 13.1). IBM had a team working on cryptography at both Kingston and Yorktown Heights, including Roy Adler, Don Coppersmith, Horst Feistel, Edna Grossman, Alan Konheim, Carl Meyer, Bill Notz, Lynn Smith, Walt Tuchman, and Bryant Tuckerman.

The algorithm, although complicated, was straightforward. It used only simple logical operations on small groups of bits and could be implemented fairly efficiently in hardware.

The NBS requested the NSA's help in evaluating the algorithm's security and determining its suitability as a federal standard. IBM had already filed for a patent [514], but was willing to make its intellectual property available to others for manufacture, implementation, and use. Eventually, the NBS worked out the terms of agreement with IBM and received a nonexclusive, royalty-free license to make, use, and sell equipment that implemented the algorithm.

Finally, in the March 17, 1975 Federal Register, the NBS published both the details of the algorithm and IBM's statement granting a nonexclusive, royalty-free license for the algorithm, and requested comment [536]. Another notice, in the August 1, 1975 Federal Register, again requested comments from agencies and the general public.

And there were comments [721,497,1120]. Many were wary of the NSA's "invisible hand" in the development of the algorithm. They were afraid that the NSA had modified the algorithm to install a trapdoor. They complained that the NSA reduced the key size from the original 128-bits to 56-bits (see Section 13.1). They complained about the inner workings of the algorithm. Much of NSA's reasoning became clear in the early 1990s, but in the 1970s this seemed mysterious and worrisome.

In 1976, the NBS held two workshops to evaluate the proposed standard. The first workshop discussed the mathematics of the algorithm and the possibility of a trapdoor [1139]. The second workshop discussed the possibility of increasing the algorithm's key length [229]. The algorithm's designers, evaluators, implementors, vendors, users, and critics were invited. From all reports, the workshops were lively [1118].

Despite criticism, the Data Encryption Standard was adopted as a federal standard on November 23, 1976 [229] and authorized for use on all unclassified government communications. The official description of the standard, FIPS PUB 46, "Data Encryption Standard," was published on January 15, 1977 and became effective six months later [1140]. FIPS PUB 81, "DES Modes of Operation," was published in 1980 [1143]. FIPS PUB 74, "Guidelines for Implementing and Using the NBS Data Encryption Standard," was published in 1981 [1142]. NBS also published FIPS PUB 112, specifying DES for password encryption [1144], and FIPS PUB 113, specifying DES for computer data authentication [1145]. (FIPS stands for Federal Information Processing Standard.)

These standards were unprecedented. Never before had an NSA-evaluated algorithm been made public. This was probably the result of a misunderstanding between NSA and NBS. The NSA thought DES was hardware-only. The standard mandated a hardware implementation, but NBS published enough details so that people could write DES software. Off the record, NSA has characterized DES as one of their biggest mistakes. If they knew the details would be released so that people could write software, they would never have agreed to it. DES did more to galvanize the field of cryptanalysis than anything else. Now there was an algorithm to study: one that the NSA said was secure. It is no accident that the next government standard algorithm, Skipjack (see Section 13.12), was classified.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Adoption of the Standard

The American National Standards Institute (ANSI) approved DES as a private-sector standard in 1981 (ANSI X3.92) [50]. They called it the Data Encryption Algorithm (DEA). ANSI published a standard for DEA modes of operation (ANSI X3.106) [52], similar to the NBS document, and a standard for network encryption that uses DES (ANSI X3.105) [51].

Two other groups within ANSI, representing retail and wholesale banking, developed DES-based standards. Retail banking involves transactions between financial institutions and private individuals, and wholesale banking involves transactions between financial institutions.

ANSI's Financial Institution Retail Security Working Group developed a standard for the management and security of PINs (ANSI X9.8) [53] and another DES-based standard for the authentication of retail financial messages (ANSI X9.19) [56]. The group has a draft standard for secure key distribution (ANSI X9.24) [58].

ANSI's Financial Institution Wholesale Security Working Group developed its own set of standards for message authentication (ANSI X9.9) [54], key management (ANSI X9.17) [55,1151], encryption (ANSI X9.23) [57], and secure personal and node authentication (ANSI X9.26) [59].

The American Bankers Association develops voluntary standards for the financial industry. They published a standard recommending DES for encryption [1], and another standard for managing cryptographic keys [2].

Before the Computer Security Act of 1987, the General Services Administration (GSA) was responsible for developing federal telecommunications standards. Since then, that responsibility transferred to NIST. The GSA published three standards that used DES: two for general security and interoperability requirements (Federal Standard 1026 [662] and Federal Standard 1027 [663]), and one for Group 3 facsimile equipment (Federal Standard 1028) [664].

The Department of the Treasury wrote policy directives requiring that all electronic-funds transfer messages be authenticated with DES [468,470]. They also wrote DES-based criteria that all authentication devices must meet [469].

The ISO first voted to approve DES—they called it the DEA-1—as an international standard, then decided not to play a role in the standardization of cryptography. However, in 1987 the International Wholesale Financial Standards group of ISO used DES in an international authentication standard [758] and for key management [761]. DES is also specified in an Australian banking standard [1497].

Validation and Certification of DES Equipment

As part of the DES standard, NIST validates implementations of DES. This validation confirms that the implementation follows the standard. Until 1994, NIST only validated hardware and firmware implementations—until then the standard prohibited software implementations. As of March 1995, 73 different implementations had been validated.

NIST also developed a program to certify that authentication equipment conformed to ANSI X9.9 and FIPS 113. As of March, 1995, 33 products had been validated. The Department of the Treasury has an additional certification procedure. NIST also has a program to confirm that equipment conforms to ANSI X9.17 for wholesale key management [1151]; four products have been validated as of March, 1995.

1987

The terms of the DES standard stipulate that it be reviewed every five years. In 1983 DES was recertified without a hitch. In the March 6, 1987 Federal Register, NBS published a request for comments on the second five-year review. NBS offered three alternatives for consideration [1480,1481]: reaffirm the standard for another five years, withdraw the standard, or revise the applicability of the standard.

NBS and NSA reviewed the standard. NSA was more involved this time. Because of an executive directive called NSDD-145, signed by Reagan, NSA had veto power over the NBS in matters of cryptography. Initially, the NSA announced that it would not recertify the standard. The problem was not that DES had been broken, or even that it was suspected of having been broken. It was simply increasingly likely that it would soon be broken.

In its place, the NSA proposed the Commercial COMSEC Endorsement Program (CCEP), which would eventually provide a series of algorithms to replace DES [85]. These NSA-designed algorithms would not be made public, and would only be available in tamper-proof VLSI chips (see Section 25.1).

This announcement wasn't well received. People pointed out that business (especially the financial industry) uses DES extensively, and that no adequate alternative is available. Withdrawal of the standard would leave many organizations with no data protection. After much debate, DES was reaffirmed as a U.S. government standard until 1992 [1141]. According to the NBS, DES would not be certified again [1480].

1993

Never say "not." In 1992, there was still no alternative for DES. The NBS, now called NIST, again solicited comments on DES in the Federal Register [540]:

The purpose of this notice is to announce the review to assess the continued adequacy of the standard to protect computer data. Comments from industry and the public are invited on the following alternatives for FIPS 46-1. The costs (impacts) and benefits of these alternatives should be included in the comments:

— Reaffirm the standard for another five (5) years. The National Institute of Standards and Technology would continue to validate equipment that implements the standard. FIPS 46-1 would continue to be the only approved method for protecting unclassified computer data.
— Withdraw the standard. The National Institute of Standards and Technology would no longer continue to support the standard. Organizations could continue to utilize existing equipment that implements the standard. Other standards could be issued by NIST as a replacement for the DES.
— Revise the applicability and/or implementation statements for the standard. Such revisions could include changing the standard to allow the use of implementations of the DES in software as well as hardware; to allow the iterative use of the DES in specific applications; to allow the use of alternative algorithms that are approved and registered by NIST.

The comment period closed on December 10, 1992. According to Raymond Kammer, then the acting director of NIST [813]:

Last year, NIST formally solicited comments on the recertification of DES. After reviewing those comments, and the other technical inputs that I have received, I plan to recommend to the Secretary of Commerce that he recertify DES for another five years. I also plan to suggest to the Secretary that when we announce the recertification we state our intention to consider alternatives to it over the next five years. By putting that announcement on the table, we hope to give people an opportunity to comment on orderly technological transitions. In the meantime, we need to consider the large installed base of systems that rely upon this proven standard.

Even though the Office of Technology Assessment quoted NIST's Dennis Branstead as saying that the useful lifetime of DES would end in the late 1990s [1191], the algorithm was recertified for another five years [1150]. Software implementations of DES were finally allowed to be certified.

Anyone want to guess what will happen in 1998?


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

12.2 Description of DES

DES is a block cipher; it encrypts data in 64-bit blocks. A 64-bit block of plaintext goes in one end of the algorithm and a 64-bit block of ciphertext comes out the other end. DES is a symmetric algorithm: The same algorithm and key are used for both encryption and decryption (except for minor differences in the key schedule).

The key length is 56 bits. (The key is usually expressed as a 64-bit number, but every eighth bit is used for parity checking and is ignored. These parity bits are the least-significant bits of the key bytes.) The key can be any 56-bit number and can be changed at any time. A handful of numbers are considered weak keys, but they can easily be avoided. All security rests within the key.

At its simplest level, the algorithm is nothing more than a combination of the two basic techniques of encryption: confusion and diffusion. The fundamental building block of DES is a single combination of these techniques (a substitution followed by a permutation) on the text, based on the key. This is known as a round. DES has 16 rounds; it applies the same combination of techniques on the plaintext block 16 times (see Figure 12.1).

The algorithm uses only standard arithmetic and logical operations on numbers of 64 bits at most, so it was easily implemented in late 1970s hardware technology. The repetitive nature of the algorithm makes it ideal for use on a special-purpose chip. Initial software implementations were clumsy, but current implementations are better.

Outline of the Algorithm

DES operates on a 64-bit block of plaintext. After an initial permutation, the block is broken into a right half and a left half, each 32 bits long. Then there are 16 rounds of identical operations, called Function f, in which the data are combined with the key. After the sixteenth round, the right and left halves are joined, and a final permutation (the inverse of the initial permutation) finishes off the algorithm.

In each round (see Figure 12.2), the key bits are shifted, and then 48 bits are selected from the 56 bits of the key. The right half of the data is expanded to 48 bits via an expansion permutation, combined with 48 bits of a shifted and permuted key via an XOR, sent through 8 S-boxes producing 32 new bits, and permuted again. These four operations make up Function f. The output of Function f is then combined with the left half via another XOR. The result of these operations becomes the new right half; the old right half becomes the new left half. These operations are repeated 16 times, making 16 rounds of DES.

If Bi is the result of the ith iteration, Li and Ri are the left and right halves of Bi, Ki is the 48-bit key for round i, and f is the function that does all the substituting and permuting and XORing with the key, then a round looks like:

Figure 12.1 DES.

Li = Ri-1
Ri = Li-1 ⊕ f (Ri-1, Ki)

The Initial Permutation

The initial permutation occurs before round 1; it transposes the input block as described in Table 12.1. This table, like all the other tables in this chapter, should be read left to right, top to bottom. For example, the initial permutation moves bit 58 of the plaintext to bit position 1, bit 50 to bit position 2, bit 42 to bit position 3, and so forth.

The initial permutation and the corresponding final permutation do not affect DES's security. (As near as anyone can tell, its primary purpose is to make it easier to load plaintext and ciphertext data into a DES chip in byte-sized pieces. Remember that DES predates 16-bit or 32-bit microprocessor busses.) Since this bit-wise permutation is difficult in software (although it is trivial in hardware), many software implementations of DES leave out both the initial and final permutations. While this new algorithm is no less secure than DES, it does not follow the DES standard and should not be called DES.

Figure 12.2 One round of DES.

The Key Transformation

Initially, the 64-bit DES key is reduced to a 56-bit key by ignoring every eighth bit. This is described in Table 12.2. These bits can be used as parity check to ensure the key is error-free. After the 56-bit key is extracted, a different 48-bit subkey is generated for each of the 16 rounds of DES. These subkeys, Ki are determined in the following manner.

First, the 56-bit key is divided into two 28-bit halves. Then, the halves are circularly shifted left by either one or two bits, depending on the round. This shift is given in Table 12.3.

Table 12.1 Initial Permutation
58, | 50, | 42, | 34, | 26, | 18, | 10, | 2, | 60, | 52, | 44, | 36, | 28, | 20, | 12, | 4,
62, | 54, | 46, | 38, | 30, | 22, | 14, | 6, | 64, | 56, | 48, | 40, | 32, | 24, | 16, | 8,
57, | 49, | 41, | 33, | 25, | 17, | 9, | 1, | 59, | 51, | 43, | 35, | 27, | 19, | 11, | 3,
61, | 53, | 45, | 37, | 29, | 21, | 13, | 5, | 63, | 55, | 47, | 39, | 31, | 23, | 15, | 7


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 12.2 Key Permutation
57, | 49, | 41, | 33, | 25, | 17, | 9, | 1, | 58, | 50, | 42, | 34, | 26, | 18,
10, | 2, | 59, | 51, | 43, | 35, | 27, | 19, | 11, | 3, | 60, | 52, | 44, | 36,
63, | 55, | 47, | 39, | 31, | 23, | 15, | 7, | 62, | 54, | 46, | 38, | 30, | 22,
14, | 6, | 61, | 53, | 45, | 37, | 29, | 21, | 13, | 5, | 28, | 20, | 12, | 4

After being shifted, 48 out of the 56 bits are selected. Because this operation permutes the order of the bits as well as selects a subset of bits, it is called a compression permutation. This operation provides a subset of 48 bits. Table 12.4 defines the compression permutation (also called the permuted choice). For example, the bit in position 33 of the shifted key moves to position 35 of the output, and the bit in position 18 of the shifted key is ignored.

Because of the shifting, a different subset of key bits is used in each subkey. Each bit is used in approximately 14 of the 16 subkeys, although not all bits are used exactly the same number of times.

The Expansion Permutation

This operation expands the right half of the data, Ri, from 32 bits to 48 bits. Because this operation changes the order of the bits as well as repeating certain bits, it is known as an expansion permutation. This operation has two purposes: It makes the right half the same size as the key for the XOR operation and it provides a longer result that can be compressed during the substitution operation. However, neither of those is its main cryptographic purpose. By allowing one bit to affect two substitutions, the dependency of the output bits on the input bits spreads faster. This is called an avalanche effect. DES is designed to reach the condition of having every bit of the ciphertext depend on every bit of the plaintext and every bit of the key as quickly as possible.

Figure 12.3 defines the expansion permutation. This is sometimes called the E-box. For each 4-bit input block, the first and fourth bits each represent two bits of the output block, while the second and third bits each represent one bit of the output block. Table 12.5 shows which output positions correspond to which input positions. For example, the bit in position 3 of the input block moves to position 4 of the output block, and the bit in position 21 of the input block moves to positions 30 and 32 of the output block.

Although the output block is larger than the input block, each input block generates a unique output block.

Table 12.3 Number of Key Bits Shifted per Round
Round | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16
Number | 1 | 1 | 2 | 2 | 2 | 2 | 2 | 2 | 1 | 2 | 2 | 2 | 2 | 2 | 2 | 1
Table 12.4 Compression Permutation
14, | 17, | 11, | 24, | 1, | 5, | 3, | 28, | 15, | 6, | 21, | 10,
23, | 19, | 12, | 4, | 26, | 8, | 16, | 7, | 27, | 20, | 13, | 2,
41, | 52, | 31, | 37, | 47, | 55, | 30, | 40, | 51, | 45, | 33, | 48,
44, | 49, | 39, | 56, | 34, | 53, | 46, | 42, | 50, | 36, | 29, | 32


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The S-Box Substitution

After the compressed key is XORed with the expanded block, the 48-bit result moves to a substitution operation. The substitutions are performed by eight substitution boxes, or S-boxes. Each S-box has a 6-bit input and a 4-bit output, and there are eight different S-boxes. (The total memory requirement for the eight DES S-boxes is 256 bytes.) The 48 bits are divided into eight 6-bit sub-blocks. Each separate block is operated on by a separate S-box: The first block is operated on by S-box 1, the second block is operated on by S-box 2, and so on. See Figure 12.4.

Each S-box is a table of 4 rows and 16 columns. Each entry in the box is a 4-bit number. The 6 input bits of the S-box specify under which row and column number to look for the output. Table 12.6 shows all eight S-boxes.

The input bits specify an entry in the S-box in a very particular manner. Consider an S-box input of 6 bits, labeled b1 b2 b3 b4 b5 and b6. Bits b1 and b6 are combined to form a 2-bit number, from 0 to 3, which corresponds to a row in the table. The middle 4 bits, b2 through b5 are combined to form a 4-bit number, from 0 to 15, which corresponds to a column in the table.

For example, assume that the input to the sixth S-box (i.e., bits 31 through 36 of the XOR function) is 110011. The first and last bits combine to form 11, which corresponds to row 3 of the sixth S-box. The middle 4 bits combine to form 1001, which corresponds to the column 9 of the same S-box. The entry under row 3, column 9 of S-box 6 is 14. (Remember to count rows and columns from 0 and not from 1.) The value 1110 is substituted for 110011.

Figure 12.3 Expansion permutation.

Table 12.5 Expansion Permutation
32, | 1, | 2, | 3, | 4, | 5, | 4, | 5, | 6, | 7, | 8, | 9,
8, | 9, | 10, | 11, | 12, | 13, | 12, | 13, | 14, | 15, | 16, | 17,
16, | 17, | 18, | 19, | 20, | 21, | 20, | 21, | 22, | 23, | 24, | 25,
24, | 25, | 26, | 27, | 28, | 29, | 28, | 29, | 30, | 31, | 32, | 1

It is, of course, far easier to implement the S-boxes in software as 64-entry arrays. It takes some rearranging of the entries to do this, but that's not hard. (Don't just change the indexing without rearranging the entries. The S-boxes are designed very carefully.) However, this way of describing the S-boxes helps visualize how they work. Each S-box can be viewed as a substitution function on a 4-bit entry: b2 through b5 go in, and a 4-bit result comes out. Bits b1 and b6 come from neighboring blocks; they select one out of four substitution functions available in the particular S-box.

The S-box substitution is the critical step in DES. The algorithm's other operations are linear and easy to analyze. The S-boxes are nonlinear and, more than anything else, give DES its security.

The result of this substitution phase is eight 4-bit blocks which are recombined into a single 32-bit block. This block moves to the next step: the P-box permutation.

The P-Box Permutation

The 32-bit output of the S-box substitution is permuted according to a P-box. This permutation maps each input bit to an output position; no bits are used twice and no bits are ignored. This is called a straight permutation or just a permutation. Table 12.7 shows the position to which each bit moves. For example, bit 21 moves to bit 4, while bit 4 moves to bit 31.

Figure 12.4 S-box substitution.

Table 12.6 S-Boxes
S-box 1:
14, | 4, | 13, | 1, | 2, | 15, | 11, | 8, | 3, | 10, | 6, | 12, | 5, | 9, | 0, | 7,
0, | 15, | 7, | 4, | 14, | 2, | 13, | 1, | 10, | 6, | 12, | 11, | 9, | 5, | 3, | 8,
4, | 1, | 14, | 8, | 13, | 6, | 2, | 11, | 15, | 12, | 9, | 7, | 3, | 10, | 5, | 0,
15, | 12, | 8, | 2, | 4, | 9, | 1, | 7, | 5, | 11, | 3, | 14, | 10, | 0, | 6, | 13,
S-box 2:
15, | 1, | 8, | 14, | 6, | 11, | 3, | 4, | 9, | 7, | 2, | 13, | 12, | 0, | 5, | 10,
3, | 13, | 4, | 7, | 15, | 2, | 8, | 14, | 12, | 0, | 1, | 10, | 6, | 9, | 11, | 5,
0, | 14, | 7, | 11, | 10, | 4, | 13, | 1, | 5, | 8, | 12, | 6, | 9, | 3, | 2, | 15,
13, | 8, | 10, | 1, | 3, | 15, | 4, | 2, | 11, | 6, | 7, | 12, | 0, | 5, | 14, | 9,
S-box 3:
10, | 0, | 9, | 14, | 6, | 3, | 15, | 5, | 1, | 13, | 12, | 7, | 11, | 4, | 2, | 8,
13, | 7, | 0, | 9, | 3, | 4, | 6, | 10, | 2, | 8, | 5, | 14, | 12, | 11, | 15, | 1,
13, | 6, | 4, | 9, | 8, | 15, | 3, | 0, | 11, | 1, | 2, | 12, | 5, | 10, | 14, | 7,
1, | 10, | 13, | 0, | 6, | 9, | 8, | 7, | 4, | 15, | 14, | 3, | 11, | 5, | 2, | 12,
S-box 4:
7, | 13, | 14, | 3, | 0, | 6, | 9, | 10, | 1, | 2, | 8, | 5, | 11, | 12, | 4, | 15,
13, | 8, | 11, | 5, | 6, | 15, | 0, | 3, | 4, | 7, | 2, | 12, | 1, | 10, | 14, | 9,
10, | 6, | 9, | 0, | 12, | 11, | 7, | 13, | 15, | 1, | 3, | 14, | 5, | 2, | 8, | 4,
3, | 15, | 0, | 6, | 10, | 1, | 13, | 8, | 9, | 4, | 5, | 11, | 12, | 7, | 2, | 14,
S-box 5:
2, | 12, | 4, | 1, | 7, | 10, | 11, | 6, | 8, | 5, | 3, | 15, | 13, | 0, | 14, | 9,
14, | 11, | 2, | 12, | 4, | 7, | 13, | 1, | 5, | 0, | 15, | 10, | 3, | 9, | 8, | 6,
4, | 2, | 1, | 11, | 10, | 13, | 7, | 8, | 15, | 9, | 12, | 5, | 6, | 3, | 0, | 14,
11, | 8, | 12, | 7, | 1, | 14, | 2, | 13, | 6, | 15, | 0, | 9, | 10, | 4, | 5, | 3,
S-box 6:
12, | 1, | 10, | 15, | 9, | 2, | 6, | 8, | 0, | 13, | 3, | 4, | 14, | 7, | 5, | 11,
10, | 15, | 4, | 2, | 7, | 12, | 9, | 5, | 6, | 1, | 13, | 14, | 0, | 11, | 3, | 8,
9, | 14, | 15, | 5, | 2, | 8, | 12, | 3, | 7, | 0, | 4, | 10, | 1, | 13, | 11, | 6,
4, | 3, | 2, | 12, | 9, | 5, | 15, | 10, | 11, | 14, | 1, | 7, | 6, | 0, | 8, | 13,
S-box 7:
4, | 11, | 2, | 14, | 15, | 0, | 8, | 13, | 3, | 12, | 9, | 7, | 5, | 10, | 6, | 1,
13, | 0, | 11, | 7, | 4, | 9, | 1, | 10, | 14, | 3, | 5, | 12, | 2, | 15, | 8, | 6,
1, | 4, | 11, | 13, | 12, | 3, | 7, | 14, | 10, | 15, | 6, | 8, | 0, | 5, | 9, | 2,
6, | 11, | 13, | 8, | 1, | 4, | 10, | 7, | 9, | 5, | 0, | 15, | 14, | 2, | 3, | 12,
S-box 8:
13, | 2, | 8, | 4, | 6, | 15, | 11, | 1, | 10, | 9, | 3, | 14, | 5, | 0, | 12, | 7,
1, | 15, | 13, | 8, | 10, | 3, | 7, | 4, | 12, | 5, | 6, | 11, | 0, | 14, | 9, | 2,
7, | 11, | 4, | 1, | 9, | 12, | 14, | 2, | 0, | 6, | 10, | 13, | 15, | 3, | 5, | 8,
2, | 1, | 14, | 7, | 4, | 10, | 8, | 13, | 15, | 12, | 9, | 0, | 3, | 5, | 6, | 11


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 12.7 P-Box Permutation
16, | 7, | 20, | 21, | 29, | 12, | 28, | 17, | 1, | 15, | 23, | 26, | 5, | 18, | 31, | 10,
2, | 8, | 24, | 14, | 32, | 27, | 3, | 9, | 19, | 13, | 30, | 6, | 22, | 11, | 4, | 25

Finally, the result of the P-box permutation is XORed with the left half of the initial 64-bit block. Then the left and right halves are switched and another round begins.

The Final Permutation

The final permutation is the inverse of the initial permutation and is described in Table 12.8. Note that the left and right halves are not exchanged after the last round of DES; instead the concatenated block R16L16 is used as the input to the final permutation. There's nothing going on here; exchanging the halves and shifting around the permutation would yield exactly the same result. This is so that the algorithm can be used to both encrypt and decrypt.

Decrypting DES

After all the substitutions, permutations, XORs, and shifting around, you might think that the decryption algorithm is completely different and just as confusing as the encryption algorithm. On the contrary, the various operations were chosen to produce a very useful property: The same algorithm works for both encryption and decryption.

With DES it is possible to use the same function to encrypt or decrypt a block. The only difference is that the keys must be used in the reverse order. That is, if the encryption keys for each round are K1 K2 K3,..., K16 then the decryption keys are K16 K15 K14, ..., K1. The algorithm that generates the key used for each round is circular as well. The key shift is a right shift and the number of positions shifted is 0,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1.

Modes of DES

FIPS PUB 81 specifies four modes of operation: ECB, CBC, OFB, and CFB (see Chapter 9) [1143]. The ANSI banking standards specify ECB and CBC for encryption, and CBC and n-bit CFB for authentication [52].

In the software world, certification is usually not an issue. Because of its simplicity, ECB is most often used in off-the-shelf commercial software products, although it is the most vulnerable to attack. CBC is used occasionally, even though it is just slightly more complicated than ECB and provides much more security.

Table 12.8 Final Permutation
40, | 8, | 48, | 16, | 56, | 24, | 64, | 32, | 39, | 7, | 47, | 15, | 55, | 23, | 63, | 31,
38, | 6, | 46, | 14, | 54, | 22, | 62, | 30, | 37, | 5, | 45, | 13, | 53, | 21, | 61, | 29,
36, | 4, | 44, | 12, | 52, | 20, | 60, | 28, | 35, | 3, | 43, | 11, | 51, | 19, | 59, | 27,
34, | 2, | 42, | 10, | 50, | 18, | 58, | 26, | 33, | 1, | 41, | 9, | 49, | 17, | 57, | 25

Hardware and Software Implementations of DES

Much has been written on efficient hardware and software implementations of the algorithm [997,81,533,534,437,738,1573,176,271,1572]. At this writing, the recordholder for the fastest DES chip is a prototype developed at Digital Equipment Corporation [512]. It supports ECB and CBC modes and is based on a GaAs gate array of 50,000 transistors. Data can be encrypted and decrypted at a rate of 1 gigabit per second, which translates to 16.8 million blocks per second. This is impressive. Table 12.9 gives the specifications for some commercial DES chips. Seeming discrepancies between clock speed and data rate are due to pipelining within the chip; a chip might have multiple DES engines working in parallel.

The most impressive DES chip is VLSI's 6868 (formerly called "Gatekeeper"). Not only can it perform DES encryption in only 8 clock cycles (prototypes in the lab can do it in 4 clock cycles), but it can also do ECB triple-DES in 25 clock cycles, and OFB or CBC triple-DES in 35 clock cycles. This sounds impossible to me, too, but I assure you it works.

A software implementation of DES on an IBM 3090 mainframe can perform 32,000 DES encryptions per second. Most microcomputers are slower, but impressive nonetheless. Table 12.10 [603,793] gives actual results and estimates for various Intel and Motorola microprocessors.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

12.3 Security of DES

People have long questioned the security of DES [458]. There has been much speculation on the key length, number of iterations, and design of the S-boxes. The S-boxes were particularly mysterious—all those constants, without any apparent reason as to why or what they're for. Although IBM claimed that the inner workings were the result of 17 man-years of intensive cryptanalysis some people feared that the NSA embedded a trapdoor into the algorithm so they would have an easy means of decrypting messages.

The U.S. Senate Select Committee on Intelligence, with full top-secret clearances, investigated the matter in 1978. The findings of the committee are classified, but an unclassified summary of those findings exonerated the NSA from any improper involvement in the algorithm's design [1552]. "It was said to have convinced IBM that a shorter key was adequate, to have indirectly assisted in the development of the S-box structures and to have certified that the final DES algorithm was, to the best of their knowledge, free of any statistical or mathematical weaknesses" [435]. However, since the government never made the details of the investigation public, many people remained unconvinced.

Tuchman and Meyer, two of the IBM cryptographers who designed DES, said the NSA did not alter the design [841]:

Their basic approach was to look for strong substitution, permutation, and key scheduling functions.... IBM has classified the notes containing the selection criteria at the request of the NSA.... "The NSA told us we had inadvertently reinvented some of the deep secrets it uses to make its own algorithms," explains Tuchman.

Table 12.9 Commercial DES Chips
Manufacturer | Chip | Year | Clock | Data Rate | Availability
AMD | Am9518 | 1981 | 3 MHz | 1.3 MByte/s | N
AMD | Am9568 | ? | 4 MHz | 1.5 MByte/s | N
AMD | AmZ8068 | 1982 | 4 MHz | 1.7 MByte/s | N
AT&T | T7000A | 1985 | ? | 1.9 MByte/s | N
CE-Infosys | SuperCrypt | 1992 | 20 MHz | 12.5 MByte/s | Y
 | CE99C003 |  |  |  |
CE-Infosys | SuperCrypt | 1994 | 30 MHz | 20.0 MByte/s | Y
 | CE99C003A |  |  |  |
Cryptech | Cry12C102 | 1989 | 20 MHz | 2.8 MByte/s | Y
Newbridge | CA20C03A | 1991 | 25 MHz | 3.85 MByte/s | Y
Newbridge | CA20C03W | 1992 | 8 MHz | 0.64 MByte/s | Y
Newbridge | CA95C68/18/09 | 1993 | 33 MHz | 14.67 MByte/s | Y
Pijnenburg | PCC100 | ? | ? | 2.5 MByte/s | Y
Semaphore Communications | Roadrunner284 | ? | 40 MHz | 35.5 MByte/s | Y
VLSI Technology | VM007 | 1993 | 32 MHz | 200.0 MByte/s | Y
VLSI Technology | VM009 | 1993 | 33 MHz | 14.0 MByte/s | Y
VLSI Technology | 6868 | 1995 | 32 MHz | 64.0 MByte/s | Y
Western Digital | WD2001/2002 | 1984 | 3 MHz | 0.23 MByte/s | N
Table 12.10 DES Speeds on Different Microprocessors and Computers
Processor | Speed (in MHz) | DES Blocks (per second)
8088 | 4.7 | 370
68000 | 7.6 | 900
80286 | 6 | 1,100
68020 | 16 | 3,500
68030 | 16 | 3,900
80386 | 25 | 5,000
68030 | 50 | 10,000
68040 | 25 | 16,000
68040 | 40 | 23,000
80486 | 66 | 43,000
Sun ELC |  | 26,000
HyperSparc |  | 32,000
RS6000-350 |  | 53,000
Sparc 10/52 |  | 84,000
DEC Alpha 4000/610 |  | 154,000
HP 9000/887 | 125 | 196,000


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Later in the article, Tuchman is quoted: "We developed the DES algorithm entirely within IBM using IBMers. The NSA did not dictate a single wire!" Tuchman reaffirmed this when he spoke on the history of DES at the 1992 National Computer Security Conference.

On the other hand, Coppersmith wrote [373,374]: "The National Security Agency (NSA) also provided technical advice to IBM." And Konheim has been quoted as saying: "We sent the S-boxes off to Washington. They came back and were all different. We ran our tests and they passed." People have pointed to this as evidence that the NSA put a trapdoor in DES.

NSA, when questioned regarding any imposed weakness in DES, said [363]:

Regarding the Data Encryption Standard (DES), we believe that the public record from the Senate Committee for Intelligence's investigation in 1978 into NSA's role in the development of the DES is responsive to your question. That committee report indicated that NSA did not tamper with the design of the algorithm in any way and that the security afforded by the DES was more than adequate for at least a 5–10 year time span for the unclassified data for which it was intended. In short, NSA did not impose or attempt to impose any weakness on the DES.

Then why did they modify the S-boxes? Perhaps it was to ensure that IBM did not put a trapdoor in DES. The NSA had no reason to trust IBM's researchers, and would be lax in their duty if they did not make absolutely sure that DES was free of trapdoors. Dictating the S-boxes is one way they could make sure.

Very recently some new cryptanalysis results have shed some light on this issue, but for many years this has been the subject of much speculation.

Weak Keys

Because of the way the initial key is modified to get a subkey for each round of the algorithm, certain initial keys are weak keys [721,427]. Remember that the initial value is split into two halves, and each half is shifted independently. If all the bits in each half are either 0 or 1, then the key used for any cycle of the algorithm is the same for all the cycles of the algorithm. This can occur if the key is entirely 1s, entirely 0s, or if one half of the key is entirely 1s and the other half is entirely 0s. Also, two of the weak keys have other properties that make them less secure [427].

The four weak keys are shown in hexadecimal notation in Table 12.11. (Remember that every eighth bit is a parity bit.)

Additionally, some pairs of keys encrypt plaintext to the identical ciphertext. In other words, one key in the pair can decrypt messages encrypted with the other key in the pair. This is due to the way in which DES generates subkeys; instead of generating 16 different subkeys, these keys generate only two different subkeys. Each of these subkeys is used eight times in the algorithm. These keys are called semiweak keys, and are shown in hexadecimal notation in Table 12.12.

Table 12.11 DES Weak Keys
Weak | Key Value | (with parity bits) |  | Actual Key
0101 | 0101 | 0101 | 0101 | 0000000 0000000
1F1F | 1F1F | 0E0E | 0E0E | 0000000 FFFFFFF
E0E0 | E0E0 | F1F1 | F1F1 | FFFFFFF 0000000
FEFE | FEFE | FEFE | FEFE | FFFFFFF FFFFFFF

Some keys produce only four subkeys, each used four times in the algorithm. These possibly weak keys are listed in Table 12.13.

Before condemning DES for having weak keys, consider that this list of 64 keys is minuscule compared to the total set of 72,057,594,037,927,936 possible keys. If you select a random key, the odds of picking one of these keys is negligible. If you are truly paranoid, you could always check for weak keys during key generation. Some people don't think it's worth the bother. Others say that it's so easy to check, there's no reason not to.

There is further analysis on weak and semiweak keys in [1116], and additional key patterns have been investigated for weaknesses. None have been found.

Complement Keys

Take the bit-wise complement of a key; that is, replace all the 0s with 1s and the 1s with 0s. Now, if the original key encrypts a block of plaintext, then the complement of the key will encrypt the complement of the plaintext block into the complement of the ciphertext block.

If x' is the complement of x, then the identity is as follows:

EK(P) = C
EK'(P') = C'

This isn't anything mysterious. The subkeys are XORed with the right half after the expansion permutation in every round. This complementation property is a direct result of that fact.

Table 12.12 DES Semiweak Key Pairs
01FE | 01FE | 01FE | 01FE | and | FE01 | FE01 | FE01 | FE01
1FE0 | 1FE0 | 0EF1 | 0EF1 | and | E01F | E01F | F10E | F10E
01E0 | 01E0 | 01F1 | 01F1 | and | E001 | E001 | F101 | F101
1FFE | 1FFE | 0EFE | 0EFE | and | FE1F | FE1F | FE0E | FE0E
011F | 011F | 010E | 010E | and | 1F01 | 1F01 | 0E01 | 0E01
E0FE | E0FE | F1FE | F1FE | and | FEE0 | FEE0 | FEF1 | FEF1


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 12.13 DES Possibly Weak Keys
1F | 1F | 01 | 01 | 0E | 0E | 01 | 01 | E0 | 01 | 01 | E0 | F1 | 01 | 01 | F1
01 | 1F | 1F | 01 | 01 | 0E | 0E | 01 | FE | 1F | 01 | E0 | FE | 0E | 01 | F1
1F | 01 | 01 | 1F | 0E | 01 | 01 | 0E | FE | 01 | 1F | E0 | FE | 01 | 0E | F1
01 | 01 | 1F | 1F | 01 | 01 | 0E | 0E | E0 | 1F | 1F | E0 | F1 | 0E | 0E | F1
E0 | E0 | 01 | 01 | F1 | F1 | 01 | 01 | FE | 01 | 01 | FE | FE | 01 | 01 | FE
FE | FE | 01 | 01 | FE | FE | 01 | 01 | E0 | 1F | 01 | FE | F1 | 0E | 01 | FE
FE | E0 | 1F | 01 | FE | F1 | 0E | 01 | E0 | 01 | 1F | FE | F1 | 01 | 0E | FE
E0 | FE | 1F | 01 | F1 | FE | 0E | 01 | FE | 1F | 1F | FE | FE | 0E | 0E | FE
FE | E0 | 01 | 1F | FE | F1 | 01 | 0E | 1F | FE | 01 | E0 | 0E | FE | 01 | F1
E0 | FE | 01 | 1F | F1 | FE | 01 | 0E | 01 | FE | 1F | E0 | 01 | FE | 0E | F1
E0 | E0 | 1F | 1F | F1 | F1 | 0E | 0E | 1F | E0 | 01 | FE | 0E | F1 | 01 | FE
FE | FE | 1F | 1F | FE | FE | 0E | 0E | 01 | E0 | 1F | FE | 01 | F1 | 0E | FE
FE | 1F | E0 | 01 | FE | 0E | F1 | 01 | 01 | 01 | E0 | E0 | 01 | 01 | F1 | F1
E0 | 1F | FE | 01 | F1 | 0E | FE | 01 | 1F | 1F | E0 | E0 | 0E | 0E | F1 | F1
FE | 01 | E0 | 1F | FE | 01 | F1 | 0E | 1F | 01 | FE | E0 | 0E | 01 | FE | F1
E0 | 01 | FE | 1F | F1 | 01 | FE | 0E | 01 | 1F | FE | E0 | 01 | 0E | FE | F1
01 | E0 | E0 | 01 | 01 | F1 | F1 | 01 | 1F | 01 | E0 | FE | 0E | 01 | F1 | FE
1F | FE | E0 | 01 | 0E | FE | F0 | 01 | 01 | 1F | E0 | FE | 01 | 0E | F1 | FE
1F | E0 | FE | 01 | 0E | F1 | FE | 01 | 01 | 01 | FE | FE | 01 | 01 | FE | FE
01 | FE | FE | 01 | 01 | FE | FE | 01 | 1F | 1F | FE | FE | 0E | 0E | FE | FE
1F | E0 | E0 | 1F | 0E | F1 | F1 | 0E | FE | FE | E0 | E0 | FE | FE | F1 | F1
01 | FE | E0 | 1F | 01 | FE | F1 | 0E | E0 | FE | FE | E0 | F1 | FE | FE | F1
01 | E0 | FE | 1F | 01 | F1 | FE | 0E | FE | E0 | E0 | FE | FE | F1 | F1 | FE
1F | FE | FE | 1F | 0E | FE | FE | 0E | E0 | E0 | FE | FE | F1 | F1 | FE | FE


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

What this means is that a chosen-plaintext attack against DES only has to test half the possible keys: 255 keys instead of 256 [1080]. Eli Biham and Adi Shamir showed [172] that there is a known-plaintext attack of the same complexity, requiring at least 233 known plaintexts.

It is questionable whether this is a weakness, since most messages don't have complement blocks of plaintext (in random plaintext, the odds against it are extremely high) and users can be warned not to use complement keys.

Algebraic Structure

All possible 64-bit plaintext blocks can be mapped onto all possible 64-bit ciphertext blocks in 264! different ways. The DES algorithm, with its 56-bit key, gives us 256 (approximately 1017) of these mappings. Using multiple encryption, it seems possible to reach a larger portion of those possible mappings. However, this is only true if the DES operation does not have certain algebraic structures.

If DES were closed, then for any K1 and K2 there would always be a K3 such that

EK2(EK1(P)) = EK3(P)

In other words, the DES encryption operation would form a group, and encrypting a set of plaintext blocks with K1 followed by K2 would be identical to encrypting the blocks with K3. Even worse, DES would be vulnerable to a meet-in-the-middle known-plaintext attack that runs in only 228 steps [807].

If DES were pure, then for any K1 K2 and K3 there would always be a K4 such that

EK3(EK2(EK1(P))) = EK4(P)

Triple encryption would be useless. (Note that a closed cipher is necessarily pure but a pure cipher is not necessarily closed.)

An early theoretical paper by Don Coppersmith gave some hints, but it wasn't enough [377]. Various cryptographers wrestled with this question [588,427,431,527,723,789]. Cycling experiments gathered "overwhelming evidence" that DES is not a group [807,371,808,1116,809], but it wasn't until 1992 that cryptographers proved that DES is not a group [293]. Coppersmith said that the IBM team knew it all along.

Key Length

IBM's original submission to NBS had a 112-bit key. By the time the DES became a standard, that was reduced to a 56-bit key. Many cryptographers argued for the longer key. Their arguments centered on the possibility of a brute-force attack (see Section 7.1).

In 1976 and 1977, Diffie and Hellman argued that a special-purpose DES-cracking parallel computer could recover the key in a day and cost $20 million. In 1981, Diffie upped this to a two-day search time and a cost of $50 million [491]. Diffie and Hellman argued then that this was out of reach for everybody except organizations like the NSA, but that by 1990 DES would be totally insecure [714].

Hellman [716] presented another argument against the small key size: By trading memory space for time, it would be possible to speed up the searching process. He suggested the possibility of computing and storing 256 possible results of encrypting a single plaintext block under every possible key. Then, to break an unknown key, all that would be required would be for the cryptanalyst to insert the plaintext block into the encryption stream, recover the resulting ciphertext, and look the key up. Hellman pegged the cost of this cracking machine at $5 million.

Arguments for and against the existence of a DES-cracker lurking in some government basement somewhere have continued. Several people pointed out that the mean time between failures for the DES chips would never be high enough to ensure that the machine would work. This objection was shown to be superfluous in [1278]. Others suggested ways to speed the process even further and to reduce the effects of chip failures.

Meanwhile, hardware implementations of DES slowly approached the million-encryptions-per-second requirement of Diffie and Hellman's special-purpose machine. In 1984 DES chips capable of performing 256,000 encryptions per second had been produced [533,534]. By 1987 chips performing 512,000 encryptions per second were being developed, and a version capable of checking over a million keys per second was feasible [738,1573]. And in 1993 Michael Wiener designed a $1 million machine that could complete a brute-force attack against DES in an average of 3.5 hours (see Section 7.1).

No one has publicly admitted building this machine, although it is a reasonable assumption that someone has. A million dollars is not a lot of money to a large—or even a medium-sized—country.

It was not until 1990 that two Israeli mathematicians, Biham and Shamir, discovered differential cryptanalysis, a technique that put to rest the question of key length. Before we discuss that technique, let's turn to some other design criticisms of DES.

Number of Rounds

Why 16 rounds? Why not 32? After five rounds every ciphertext bit is a function of every plaintext bit and every key bit [1078,1080], and after eight rounds the ciphertext was essentially a random function of every plaintext bit and every key bit [880]. (This is called the avalanche effect.) So why not stop after eight rounds?

Over the years, variants of DES with a reduced number of rounds have been successfully attacked. DES with three or four rounds was easily broken in 1982 [49]. DES with six rounds fell some years later [336]. Biham and Shamir's differential cryptanalysis explained this as well: DES with any number of rounds fewer than 16 could be broken with a known-plaintext attack more efficiently than by a brute-force attack. Certainly brute-force is a much more likely attack, but it is interesting that the algorithm has exactly 16 rounds.

Design of the S-Boxes

In addition to being accused of reducing the key length, NSA was also accused of modifying the contents of the S-boxes. When pressed for design justification for the S-boxes, the NSA indicated that elements of the algorithm's design were "sensitive" and would not be made public. Many cryptographers were concerned that the NSA-designed S-boxes hid a trapdoor, making it possible for them to easily cryptanalyze the algorithm.

Since then, considerable effort has gone into analyzing the design and operation of the S-boxes. In the mid-1970s, Lexar Corporation [961,721] and Bell Laboratories [1120] examined the operation of the S-boxes. Neither analysis revealed any weaknesses, although both found inexplicable features. The S-boxes had more features in common with a linear transformation than one would expect if they were chosen at random. The Bell Laboratories team stated that the S-boxes may have hidden trapdoors, and the Lexar report concluded with:

Structures have been found in DES that were undoubtedly inserted to strengthen the system against certain types of attack. Structures have also been found that appear to weaken the system.

On the other hand, this report also warned:

...the problem [of the search for structure in the S-boxes] is complicated by the ability of the human mind to find apparent structure in random data, which is really not structure at all.

At the second workshop on DES, the National Security Agency revealed several design criteria behind the S-boxes [229]. This did nothing to quell people's suspicions, and the debate continued [228,422,714,1506,1551].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Various oddities about the S-boxes appeared in the literature. The last three output bits of the fourth S-box can be derived in the same way as the first by complementing some of the input bits [436,438]. Two different, but carefully chosen, inputs to S-boxes can produce the same output [436]. It is possible to obtain the same output of a single DES round by changing bits in only three neighboring S-boxes [487]. Shamir noticed that the S-boxes entries appeared to be somewhat imbalanced, but wasn't about to turn that imbalance into an attack [1423]. (He mentioned a feature of the fifth S-box, but it took another eight years before linear cryptanalysis exploited that feature.) Other researchers showed that publicly known design principles could be used to generate S-boxes with the observed characteristics [266].

Additional Results

There were other attempts to cryptanalyze DES. One cryptographer looked at nonrandomness based on spectral tests [559]. Others analyzed sequences of linear factors, but their attack failed after eight rounds [1297,336,531]. A 1987 unpublished attack by Donald Davies exploited the way the expansion permutation repeats bits into adjacent S-boxes; this attack is also impractical after eight rounds [172,429].

12.4 Differential and Linear Cryptanalysis

Differential Cryptanalysis

In 1990, Eli Biham and Adi Shamir introduced differential cryptanalysis [167,168,171,172]. This is a new method of cryptanalysis, heretofore unknown to the public. Using this method, Biham and Shamir found a chosen-plaintext attack against DES that was more efficient than brute force.

Differential cryptanalysis looks specifically at ciphertext pairs: pairs of ciphertexts whose plaintexts have particular differences. It analyzes the evolution of these differences as the plaintexts propagate through the rounds of DES when they are encrypted with the same key.

Simply, choose pairs of plaintexts with a fixed difference. The two plaintexts can be chosen at random, as long as they satisfy particular difference conditions; the cryptanalyst does not even have to know their values. (For DES, the term "difference" is defined using XOR. This can be different for different algorithms.) Then, using the differences in the resulting ciphertexts, assign different probabilities to different keys. As you analyze more and more ciphertext pairs, one key will emerge as the most probable. This is the correct key.

The details are more complicated. Figure 12.5 is the DES round function. Imagine a pair of inputs, X and X', that have the difference DX. The outputs, Y and Y' are known, and therefore so is the difference, Y. Both the expansion permutation and the P-box are known, so ΔA and ΔC are known. B and B' are not known, but their difference ΔB is known and equal to ΔA. (When looking at the difference, the XORing of Ki with A and A' cancels out.) So far, so good. Here's the trick: For any given ΔA, not all values of ΔC are equally likely. The combination of ΔA and ΔC suggests values for bits of A XOR Ki and A' XOR Ki. Since A and A' are known, this gives us information about Ki.

Look at the last round of DES. (Differential cryptanalysis ignores the initial and final permutation. They have no effect on the attack, except to make it harder to explain.) If we can identify K16 then we have 48 bits of the key. (Remember, the subkey in each round consists of 48 bits of the 56-bit key.) The other 8 bits we can get by brute force. Differential cryptanalysis will get us K16.

Certain differences in plaintext pairs have a high probability of causing certain differences in the resulting ciphertext pairs. These are called characteristics. Characteristics extend over a number of rounds and essentially define a path through these rounds. There is an input difference, a difference at each round, and an output difference—with a specific probability.

You can find these characteristics by generating a table where the rows represent the possible input XORs (the XOR of two different sets of input bits), the columns represent the possible output XORs, and the entries represent the number of times a particular output XOR occurs for a given input XOR. You can generate such a table for each of DES's eight S-boxes.

Figure 12.5 DES round function.

For example, Figure 12.6a is a one-round characteristic. They input difference of the left side is L; it could be anything. The input difference of the right side is 0. (The two inputs have the same right-hand side, so their difference is 0.) Since there is no difference going in to the round function, then there is no difference coming out of the round function. Therefore, the output difference of the left side is L ⊕ 0 = L, and the output difference of the right side is 0. This is a trivial characteristic, and is true with probability 1.

Figure 12.6b is a less obvious characteristic. Again, the input difference to the left side is arbitrary: L. The input difference to the right side is 0x60000000; the two inputs differ in only the second and third bits. With a probability of 14/64, the output difference of the round function is L ⊕ 0x00808200. This means that the output difference of the left side is L ⊕ 0x00808200 and the output difference of the right side is 0x60000000—with probability 14/64.

Different characteristics can be joined. And, assuming the rounds are independent, the probabilities can be multiplied. Figure 12.7 joins the two characteristics previously described. The input difference to the left side is 0x00808200 and the input difference to the right side is 0x60000000. At the end of the first round the input difference and the output of the round function cancel out, leaving an output difference of 0. This feeds into the second round; the final output difference of the left side is 0x60000000 and the final output difference of the right side is 0. This two-round characteristic has a probability of 14/64.

A plaintext pair that satisfies the characteristic is a right pair. A plaintext pair which does not is a wrong pair. A right pair will suggest the correct round key (for the last round of the characteristic); a wrong pair will suggest a random round key.

Figure 12.6 DES characteristics.

Figure 12.7 A two-round DES characteristic.

To find the correct round key, simply collect enough guesses so that one subkey is suggested more often than all the others. In effect, the correct subkey will rise out of all the random alternatives.

So, the basic differential attack on n-round DES will recover the 48-bit subkey used in round n, and the remaining 8 key bits are obtained by brute-force guessing.

There are still considerable problems. First, there is a negligible chance of success until you reach some threshold. That is, until you accumulate sufficient data you can't tell the correct subkey from all the noise. And the attack isn't practical: You have to use counters to assign different probabilities to 248 possible subkeys, and too much data is required to make this work.

At this point, Biham and Shamir tweaked their attack. Instead of a using a 15-round characteristic on 16-round DES, they used a 13-round characteristic and some tricks to get the last few rounds. A shorter characteristic with a higher probability worked better. And they used some clever mathematics to obtain 56-bit key candidates which could be tested immediately, eliminating the need for counters. This attack succeeds as soon as a right pair is found; this avoids the threshold and gives a linear success probability. If you have 1000 times fewer pairs, then you have 1000 times smaller chance of success. This sounds terrible, but it is a lot better than the threshold. There is always some chance of immediate success.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The results are most interesting. Table 12.14 is a summary of the best differential attack against DES with varying numbers of rounds [172]. The first column is the number of rounds. The next two columns are the numbers of chosen plaintexts or known plaintexts that must be examined for the attack, and the fourth column is the number of those plaintexts actually analyzed. The last column is the complexity of analysis, after the required plaintexts are found.

Table 12.14 Differential Cryptanalysis Attacks against DES
No. of Rounds | Chosen Plaintexts | Known Plaintexts | Analyzed Plaintexts | Complexity of Analysis
8 | 214 | 238 | 4 | 29
9 | 224 | 244 | 2 | 232†
10 | 224 | 243 | 214 | 215
11 | 231 | 247 | 2 | 232†
12 | 231 | 247 | 221 | 221
13 | 239 | 252 | 2 | 232†
14 | 239 | 251 | 229 | 229
15 | 247 | 256 | 27 | 237
16 | 247 | 255 | 236 | 237
†The complexity of the analysis can be greatly reduced for these variants by using about four times as many plaintexts with the clique method.

The best attack against full 16-round DES requires 247 chosen plaintexts. This can be converted to a known plaintext attack, but that requires 255 known plaintexts. And 237 DES operations are required during analysis.

Differential cryptanalysis works against DES and other similar algorithms with constant S-boxes. The attack is heavily dependent on the structure of the S-boxes; the ones in DES just happen to be optimized against differential cryptanalysis. And the attack works against DES in any of its operating modes—ECB, CBC, CFB, and OFB—with the same complexity [172].

DES's resistance can be improved by increasing the number of rounds. Chosen-plaintext differential cryptanalysis DES with 17 or 18 rounds takes about the same time as a brute-force search [160]. At 19 rounds or more, differential cryptanalysis becomes impossible because it requires more than 264 chosen plaintexts: Remember, DES has a 64-bit block size, so it only has 264 possible plaintext blocks. (In general, you can prove that an algorithm is resistant to differential cryptanalysis by showing that the amount of plaintext required to mount such an attack is greater than the amount of plaintext possible.)

Here are a few important points. First, this attack is largely theoretical. The enormous time and data requirements to mount a differential cryptanalytic attack put it beyond the reach of almost everyone. To get the requisite data for this attack against a full DES, you have to encrypt a 1.5 megabits-per-second data stream of chosen plaintext for almost three years. Second, this is primarily a chosen-plaintext attack. It can be converted to a known-plaintext attack, but you have to sift through all of the plaintext-ciphertext pairs looking for the useful ones. For full 16-round DES, this makes the attack slightly less efficient than brute force (the differential cryptanalytic attack requires 255.1 operations, and brute force requires 255). The consensus is that DES, when implemented properly, is still secure against differential cryptanalysis.

Why is DES so resistant to differential cryptanalysis? Why are the S-boxes optimized to make this attack as difficult as possible? Why are there as many rounds as required, but no more? Because the designers knew about it. IBM's Don Coppersmith recently wrote [373,374]:

The design took advantage of certain cryptanalytic techniques, most prominently the technique of "differential cryptanalysis," which were not known in the published literature. After discussions with NSA, it was decided that disclosure of the design consideration would reveal the technique of differential cryptanalysis, a powerful technique that can be used against many ciphers. This in turn would weaken the competitive advantage the United States enjoyed over other countries in the field of cryptography.

Adi Shamir responded to this, challenging Coppersmith to say that he hadn't found any stronger attacks against DES since then. Coppersmith has chosen to remain silent on that question [1426].

Related-Key Cryptanalysis

Table 12.3 showed the number of bits the DES key is rotated after each round: 2 bits after each round, except for 1 bit after rounds 1, 2, 9, and 16. Why?

Related-key cryptanalysis is similar to differential cryptanalysis, but it examines the difference between keys. The attack is different from any previously discussed: The cryptanalyst chooses a relationship between a pair of keys, but does not know the keys themselves. Data is encrypted with both keys. In the known-plaintext version, the cryptanalyst knows the plaintext and ciphertext of data encrypted with the two keys. In the chosen-plaintext version, the cryptanalyst gets to choose the plaintext encrypted with the two keys.

A modified DES, where the key is rotated two bits after every round, is less secure. Related-key cryptanalysis can break that variant using 217 chosen-key chosen plaintexts or 233 chosen-key known plaintexts [158,163].

This attack is not at all practical, but it is interesting for three reasons. One, it is the first cryptanalytic attack against DES's subkey-generation algorithm. Two, this attack is independent of the number of rounds of the cryptographic algorithm; it's just as effective against DES with 16 rounds, 32 rounds, or 1000 rounds. And three, DES is impervious to this attack. The variability in the rotation thwarts related-key cryptanalysis.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Linear Cryptanalysis

Linear cryptanalysis is another type of cryptanalytic attack, invented by Mitsuru Matsui [1016,1015,1017]. This attack uses linear approximations to describe the action of a block cipher (in this case, DES.)

This means that if you XOR some of the plaintext bits together, XOR some ciphertext bits together, and then XOR the result, you will get a single bit that is the XOR of some of the key bits. This is a linear approximation and will hold with some probability p. If p ≠ ½, then this bias can be exploited. Use collected plaintexts and associated ciphertexts to guess the values of the key bits. The more data you have, the more reliable the guess. The greater the bias, the greater the success rate with the same amount of data.

How do you identify good linear approximations for DES? Find good 1-round linear approximations and join them together. (Again, ignore the initial and final permutations; they don't affect the attack.) Look at the S-boxes. There are 6 input bits and 4 output bits. The input bits can be combined using XOR in 63 useful ways (26 - 1), and the output bits can be combined in 15 useful ways. Now, for each S-box you can evaluate the probability that for a randomly chosen input, an input XOR combination equals some output XOR combination. If there is a combination with a high enough bias, then linear cryptanalysis may work.

If the linear approximations are unbiased, then they would hold for 32 of the 64 possible inputs. I'll spare you the pages of tables, but the most biased S-box is S-box 5. In fact, the second input bit is equal to the XOR of all 4 output bits for only 12 inputs. This translates to a probability of 3/16, or a bias of 5/16, and is the most extreme bias in all the S-boxes. (Shamir noted this in [1423], but could not find a way to exploit it.)

Figure 12.8 shows how to turn this into an attack against the DES round function. The input bit into S-box 5 is b26. (I am numbering the bits from left to right and from 1 to 64. Matsui ignores this convention with DES and numbers his bits from right to left and from 0 to 63. It's enough to drive you mad.) The 4 output bits from S-box 5 are c17 c18 c19 and c20. We can trace b26 backwards from the input to the S-box. The bit a26 is XORed with a bit from the subkey Ki,26 to obtain b26. And bit X17 goes through the expansion permutation to become a26. After the S-box, the 4 output bits go through the P-box to become 4 output bits of the round function: Y3 Y8 Y14 and Y25. This means that with probability ½ - 5/16 :

Figure 12.8 A 1-round linear approximation for DES.

X17 ⊕ Y3 ⊕ Y8 ⊕ Y14 ⊕ Y25 = Ki,26

Linear approximations for different rounds can be joined in a manner similar to that discussed under differential cryptanalysis. Figure 12.9 is a 3-round approximation with a probability of ½ + .0061. The individual approximations are of varying quality: The last is very good, the first is pretty good, and the middle is bad. But together the three 1-round approximations give a very good three-round approximation.

The basic attack is to use the best linear approximation for 16-round DES. It requires 247 known plaintext blocks, and will result in 1 key bit. That's not very useful. If you interchange the role of plaintext and ciphertext and use decryption as well as encryption, you can get 2 key bits. That's still not very useful.

Figure 12.9 A 3-round linear approximation for DES.

There are refinements. Use a 14-round linear approximation for rounds 2 through 15. Guess the 6 subkey bits relevant to S-box 5 for the first and last rounds (12 key bits in all). Effectively you are doing 212 linear cryptanalyses in parallel and picking the correct one based on probabilities. This recovers the 12 bits plus the b26 and reversing plaintext and ciphertext recovers another 13 bits. To get the remaining 30 bits, use exhaustive search. There are other tricks, but that's basically it.

Against full 16-round DES, this attack can recover the key with an average of 243 known plaintexts. A software implementation of this attack recovered a DES key in 50 days using 12 HP9000/735 workstations [1019]. That is the most effective attack against DES at the time of this writing.

Linear cryptanalysis is heavily dependent on the structure of the S-boxes and the S-boxes in DES are not optimized against this attack. In fact, the ordering of the S-boxes chosen for DES lies among the 9 percent to 16 percent that offer the least protection against linear cryptanalysis [1018]. According to Don Coppersmith [373,374], resistance to linear cryptanalysis "was not part of the design criteria of DES." Either they didn't know about linear cryptanalysis or they knew about something else even more powerful whose resistance criteria took precedence.

Linear cryptanalysis is newer than differential cryptanalysis, and there may be more performance improvements in the coming years. Some ideas are proposed in [1270,811], but it is not clear that they can be used effectively against full DES. They work very well against reduced round variants, however.

Future Directions

Some work has been done to try to extend the concept of differential cryptanalysis to higher-order differentials [702,161,927,858,860]. Lars Knudsen uses something called partial differentials to attack 6-round DES; it requires 32 chosen plaintexts and 20,000 encryptions [860]. It is still too new to know if these extensions will make it easier to attack full 16-round DES.

Another avenue of attack is differential-linear cryptanalysis: combining differential and linear cryptanalysis. Susan Langford and Hellman have an attack on 8-round DES that recovers 10 key bits with an 80 percent probability of success with 512 chosen plaintexts and a 95 percent probability of success with 768 chosen plaintexts [938]. After the attack, a brute-force search of the remaining keyspace (246 possible keys) is required. While this attack is comparable in time to previous attacks, it requires far less plaintext. However, it doesn't seem to extend easily to more rounds.

But this attack is still new and work continues. It is possible that there may be a breakthrough some time during the next few years. Maybe there are benefits in combining this attack with higher-order differential cryptanalysis. Who knows?


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

12.5 The Real Design Criteria

After differential cryptanalysis became public, IBM published the design criteria for the S-boxes and the P-box [373,374]. The criteria for the S-boxes are:

— Each S-box has 6 input bits and 4 output bits. (This was the largest size that could be accommodated in a single chip with 1974 technology.)
— No output bit of an S-box should be too close to a linear function of the input bits.
— If you fix the left-most and right-most bits of an S-box and vary the 4 middle bits, each possible 4-bit output is attained exactly once.
— If two inputs to an S-box differ in exactly 1 bit, the outputs must differ in at least 2 bits.
— If two inputs to an S-box differ in the 2 middle bits exactly, the outputs must differ in at least 2 bits.
— If two inputs to an S-box differ in their first 2 bits and are identical in their last 2 bits, the two outputs must not be the same.
— For any nonzero 6-bit difference between inputs, no more than 8 of the 32 pairs of inputs exhibiting that difference may result in the same output difference.
— A criterion similar to the previous one, but for the case of three active S-boxes.

The criteria for the P-box are:

— The 4 output bits from each S-box in round i are distributed so that 2 of them affect the middle-bits of S-boxes at roundi + 1 and the other 2 affect end bits.
— The 4 output bits from each S-box affect six different S-boxes; no 2 affect the same S-box.
— If the output bit from one S-box affects a middle bit of another S-box, then an output bit from that other S-box cannot affect a middle bit of the first S-box.

The paper goes on to discuss the criteria. Generating S-boxes is pretty easy today, but was a complicated task in the early 1970s. Tuchman has been quoted as saying that they ran computer programs for months cooking up the S-boxes.

12.6 DES Variants

Multiple DES

Some DES implementations use triple-DES (see Figure 12.10) [55]. Since DES is not a group, then the resultant ciphertext is much harder to break using exhaustive search: 2112 attempts instead of 256 attempts. See Section 15.2 for more details.

Figure 12.10 Triple-DES.

DES with Independent Subkeys

Another variation is to use a different subkey for each round, instead of generating them from a single 56-bit key [851]. Since 48 key bits are used in each of 16 rounds, this means that the key length for this variant is 768 bits. This variant would drastically increase the difficulty of a brute-force attack against the algorithm; that attack would have a complexity of 2768.

However, a meet-in-the-middle attack (see Section 15.1) would be possible. This would reduce the complexity of attack to 2384; still long enough for any conceivable security needs.

Although independent subkeys foil linear cryptanalysis, this variant is susceptible to differential cryptanalysis and can be broken with 261 chosen plaintexts (see Table 12.15) [167,172]. It would seem that any modification of the key schedule cannot make DES much stronger.

DESX

DESX is a DES variant from RSA Data Security, Inc. that has been included in the MailSafe electronic mail security program since 1986 and the BSAFE toolkit since 1987. DESX uses a technique called whitening (see Section 15.6) to obscure the inputs and outputs to DES. In addition to a 56-bit DES key, DESX has an additional 64-bit whitening key. These 64 bits are XORed to the plaintext before the first round of DES. An additional 64 bits, computed as a one-way function of the entire 120-bit DES key, is XORed to the ciphertext after the last round [155]. Whitening makes DESX much stronger than DES against a brute-force attack; the attack requires (2120)/n operations with n known plaintexts. It also improves security against differential and linear cryptanalysis; the attacks require 261 chosen plaintexts and 260 known plaintexts, respectively [1338].

CRYPT(3)

CRYPT(3) is a DES variant found on UNIX systems. It is primarily used as a one-way function for passwords, but sometimes can also be used for encryption. The difference between CRYPT(3) and DES is that CRYPT(3) has a key-dependent expansion permutation with 212 possible permutations. This was done primarily so that off-the-shelf DES chips could not be used to construct a hardware password-cracker.

Generalized DES

Generalized DES (GDES) was designed both to speed up DES and to strengthen the algorithm [1381,1382]. The overall block size increases while the amount of computation remains constant.

Figure 12.11 is a block diagram of GDES. GDES operates on variable-sized blocks of plaintext. Encryption blocks are divided up into q 32-bit sub-blocks; the exact number depends on the total block size (this was variable in the design, but must be fixed for each implementation). In general, q equals the block size divided by 32.

Function f is calculated once per round on the right-most block. The result is XORed with all the other parts, which are then rotated to the right. GDES has a variable number of rounds, n. There is a slight modification to the last round, so that the encryption and decryption processes differ only in the order of the subkeys (just like DES). In fact, if q = 2 and n = 16, this is DES.

Biham and Shamir [167,168] showed that, using differential cryptanalysis, GDES with q = 8 and n = 16 is breakable with only six chosen plaintexts. If independent subkeys are also used, 16 chosen plaintexts are required. GDES with q = 8 and n = 22 is breakable with 48 chosen plaintexts, and GDES with q = 8 and n = 31 requires only 500,000 chosen plaintexts to break. Even GDES with q = 8 and n = 64 is weaker than DES; 249 chosen plaintexts are required to break it. In fact, any GDES scheme that is faster than DES is also less secure (see Table 12.15).

A variant of this scheme recently appeared [1591]. It is probably no more secure than the original GDES. In general, any large block DES variant that is faster than DES is probably also less secure than DES.

DES with Alternate S-Boxes

Other DES modifications centered around the S-boxes. Some designs made the order of the S-boxes variable. Other designers varied the contents of the S-boxes themselves. Biham and Shamir showed [170,172] that the design of the S-boxes, and even the order of the S-boxes themselves, were optimized against differential cryptanalysis:

The replacement of the order of the eight DES S-boxes (without changing their value) also makes DES much weaker: DES with 16 rounds of a particular replaced order is breakable in about 238 steps.... DES with random S-boxes is shown to be very easy to break. Even a minimal change of one entry of one of the DES S-boxes can make DES easier to break.

The DES S-boxes were not optimized against linear cryptanalysis. There are better S-boxes than the ones that come with DES, but blindly choosing new S-boxes isn't a good idea.

Figure 12.11 GDES.

Table 12.15 [167,169] lists some modifications to DES and the number of chosen plaintexts required for differential cryptanalysis. One change not listed, combining the left and right halves using addition mod 24 instead of XOR, is 217 times harder to break than DES [689].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

RDES

RDES is a variant that replaces swapping the left and right halves at the end of each round with a key-dependent swap [893]. The swappings are fixed, depending solely on the key. This means that the 15 key-dependent swaps occur with 215 possible instances, and that the variant is not resistant to differential cryptanalysis [816,894,112]. RDES has a large number of weak keys. In fact, almost every key is weaker than a typical DES key. This variant should not be used.

A better idea is to swap only within the right half, at the beginning of each round. Another better idea is to make the swapping dependent on the input data and not a static function of the key. There are a number of possible variants [813,815]. In RDES-1, there is a data-dependent swap of the 16-bit words at the beginning of each round. In RDES-2, there is a data-dependent swap of the bytes at the beginning of each round after the 16-bit swappings as in RDES-1. And so on through RDES-4. RDES-1 is secure against both differential cryptanalysis [815] and linear cryptanalysis [1136]. Presumably RDES-2 and greater are as well.

Table 12.15 Differential Cryptanalysis Attacks against DES Variants
Modified Operation | Chosen Plaintexts
Full DES (no modification) | 247
P permutation | Cannot strengthen
Identity permutation | 219
Order of S-boxes | 238
Replace XORs by additions | 239, 231
S-boxes:
Random | 218–220
Random permutations | 233–241
One entry | 233
Uniform tables | 226
Elimination of the E Expansion | 226
Order of E and subkey XOR | 244
GDES (width q = 8):
16 rounds | 6, 16
64 rounds | 249 (independent key)

sn DES

A group of Korean researchers, led by Kwangjo Kim, has attempted to find a set of S-boxes that are optimally secure against both linear and differential cryptanalysis. Their first attempt, known as s2DES, was presented in [834] and shown to be worse than DES against differential cryptanalysis in [855,858]. Their next attempt, s3DES, was presented in [839] and shown to be worse than DES against linear cryptanalysis [856,1491,1527,858,838]. Biham suggested a minor change to make s3DES secure against both linear and differential cryptanalysis [165]. The group went back to their computers and developed better techniques for S-box design [835,837]. They proposed s4DES [836] and then s5DES [838,944].

Table 12.16 gives the s3DES S-boxes with S-box 1 and S-box 2 reversed, which are secure against both differential and linear cryptanalysis. Sticking this variant in a triple-DES mix is sure to irritate cryptanalysts.

DES with Key-Dependent S-Boxes

Linear and differential cryptanalysis work only if the analyst knows the composition of the S-boxes. If the S-boxes are key-dependent and chosen by a cryptographically strong method, then linear and differential cryptanalysis are much more difficult. Remember though, that randomly generated S-boxes have very poor differential and linear characteristics; even if they are secret.

Table 12.16 s3DES S-Boxes (with S-box 1 and S-box 2 reversed)
S-box 1:
13 | 14 | 0 | 3 | 10 | 4 | 7 | 9 | 11 | 8 | 12 | 6 | 1 | 15 | 2 | 5
8 | 2 | 11 | 13 | 4 | 1 | 14 | 7 | 5 | 15 | 0 | 3 | 10 | 6 | 9 | 12
14 | 9 | 3 | 10 | 0 | 7 | 13 | 4 | 8 | 5 | 6 | 15 | 11 | 12 | 1 | 2
1 | 4 | 14 | 7 | 11 | 13 | 8 | 2 | 6 | 3 | 5 | 10 | 12 | 0 | 15 | 9
S-box 2:
15 | 8 | 3 | 14 | 4 | 2 | 9 | 5 | 0 | 11 | 10 | 1 | 13 | 7 | 6 | 12
6 | 15 | 9 | 5 | 3 | 12 | 10 | 0 | 13 | 8 | 4 | 11 | 14 | 2 | 1 | 7
9 | 14 | 5 | 8 | 2 | 4 | 15 | 3 | 10 | 7 | 6 | 13 | 1 | 11 | 12 | 0
10 | 5 | 3 | 15 | 12 | 9 | 0 | 6 | 1 | 2 | 8 | 4 | 11 | 14 | 7 | 13
S-box 3:
13 | 3 | 11 | 5 | 14 | 8 | 0 | 6 | 4 | 15 | 1 | 12 | 7 | 2 | 10 | 9
4 | 13 | 1 | 8 | 7 | 2 | 14 | 11 | 15 | 10 | 12 | 3 | 9 | 5 | 0 | 6
6 | 5 | 8 | 11 | 13 | 14 | 3 | 0 | 9 | 2 | 4 | 1 | 10 | 7 | 15 | 12
1 | 11 | 7 | 2 | 8 | 13 | 4 | 14 | 6 | 12 | 10 | 15 | 3 | 0 | 9 | 5
S-box 4:
9 | 0 | 7 | 11 | 12 | 5 | 10 | 6 | 15 | 3 | 1 | 14 | 2 | 8 | 4 | 13
5 | 10 | 12 | 6 | 0 | 15 | 3 | 9 | 8 | 13 | 11 | 1 | 7 | 2 | 14 | 4
10 | 7 | 9 | 12 | 5 | 0 | 6 | 11 | 3 | 14 | 4 | 2 | 8 | 13 | 15 | 1
3 | 9 | 15 | 0 | 6 | 10 | 5 | 12 | 14 | 2 | 1 | 7 | 13 | 4 | 8 | 11
S-box 5:
5 | 15 | 9 | 10 | 0 | 3 | 14 | 4 | 2 | 12 | 7 | 1 | 13 | 6 | 8 | 11
6 | 9 | 3 | 15 | 5 | 12 | 0 | 10 | 8 | 7 | 13 | 4 | 2 | 11 | 14 | 1
15 | 0 | 10 | 9 | 3 | 5 | 4 | 14 | 8 | 11 | 1 | 7 | 6 | 12 | 13 | 2
12 | 5 | 0 | 6 | 15 | 10 | 9 | 3 | 7 | 2 | 14 | 11 | 8 | 1 | 4 | 13
S-box 6:
4 | 3 | 7 | 10 | 9 | 0 | 14 | 13 | 15 | 5 | 12 | 6 | 2 | 11 | 1 | 8
14 | 13 | 11 | 4 | 2 | 7 | 1 | 8 | 9 | 10 | 5 | 3 | 15 | 0 | 12 | 6
13 | 0 | 10 | 9 | 4 | 3 | 7 | 14 | 1 | 15 | 6 | 12 | 8 | 5 | 11 | 2
1 | 7 | 4 | 14 | 11 | 8 | 13 | 2 | 10 | 12 | 3 | 5 | 6 | 15 | 0 | 9
S-box 7:
4 | 10 | 15 | 12 | 2 | 9 | 1 | 6 | 11 | 5 | 0 | 3 | 7 | 14 | 13 | 8
10 | 15 | 6 | 0 | 5 | 3 | 12 | 9 | 1 | 8 | 11 | 13 | 14 | 4 | 7 | 2
2 | 12 | 9 | 6 | 15 | 10 | 4 | 1 | 5 | 11 | 3 | 0 | 8 | 7 | 14 | 13
12 | 6 | 3 | 9 | 0 | 5 | 10 | 15 | 2 | 13 | 4 | 14 | 7 | 11 | 1 | 8
S-box 8:
13 | 10 | 0 | 7 | 3 | 9 | 14 | 4 | 2 | 15 | 12 | 1 | 5 | 6 | 11 | 8
2 | 7 | 13 | 1 | 4 | 14 | 11 | 8 | 15 | 12 | 6 | 10 | 9 | 5 | 0 | 3
4 | 13 | 14 | 0 | 9 | 3 | 7 | 10 | 1 | 8 | 2 | 11 | 15 | 5 | 12 | 6
8 | 11 | 7 | 14 | 2 | 4 | 13 | 1 | 6 | 5 | 9 | 0 | 12 | 15 | 3 | 10


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Here is a method to use 48 additional key bits to generate S-boxes that are resistant to both linear and differential cryptanalysis [165].

(1) Rearrange the DES S-boxes: 24673158.
(2) Select 16 of the remaining key bits. If the first bit is 1, swap the first two rows of S-box 1 with the last two rows of S-box 1. If the second bit is a 1, swap the first eight columns of S-box 1 with the second eight columns of S-box 1. Do the same to S-box 2 with the third and fourth key bits. Do the same with S-boxes 3 through 8.
(3) Take the remaining 32 key bits. XOR the first four with every entry of S-box 1, the second four with every entry of S-box 2 and so on.

The complexity of a differential cryptanalysis attack against this system is 251; the complexity of a linear cryptanalysis attack is 253. The complexity of exhaustive search is 2102.

What is neat about this DES variant is that it can be implemented in existing hardware. Several DES chip vendors sell DES chips with loadable S-boxes. This S-box generation method can be done outside the chip and then loaded in. Differential and linear cryptanalysis require so much known or chosen plaintext as to be unworkable, and a brute-force attack is inconceivable—with no speed penalties.

12.7 How Secure Is DES Today?

The answer is both easy and hard. The easy answer just looks at key length (see Section 7.1). A brute-force DES-cracking machine that can find a key in an average of 3.5 hours cost only $1 million in 1993 [1597,1598]. DES is so widespread that it is na•ve to pretend that the NSA and its counterparts haven't built such a machine. And remember that cost will drop by a factor of 5 every 10 years. DES will only become less secure as time goes on.

The hard answer tries to estimate cryptanalytic techniques. Differential cryptanalysis was known by the NSA long before the mid-1970s, when DES first became a standard. It is na•ve to pretend that the NSA theoreticians have been idle since then; almost certainly they have developed newer cryptanalytic techniques that can be applied against DES. But there are no facts, only rumors.

Winn Schwartau writes that the NSA had built a massively parallel DES-cracking machine as early as the mid-1980s [1404]. At least one such machine was built by Harris Corp. with a Cray Y-MP as a front end. Supposedly there are a series of algorithms that can reduce the complexity of a DES brute-force search by several orders of magnitude. Contextual algorithms, based on the inner workings of DES, can scrap sets of possible keys based on partial solutions. Statistical algorithms reduce the effective key size even further. And other algorithms choose likely keys—words, printable ASCII, and so on (see Section 8.1)—to test. The rumor is that the NSA can crack DES in 3 to 15 minutes, depending on how much preprocessing they can do. And these machines cost only $50,000 each, in quantity.

A different rumor is that if the NSA has a large amount of plaintext and ciphertext, its experts can perform some kind of statistical calculation and then go out to an array of optical disks and retrieve the key.

These are just rumors, but they don't give me a warm, fuzzy feeling about DES. It has just been too big a target for too long. Almost any change to DES will be more annoying; maybe the resultant cipher will be easier to break, but the NSA might not have the resources to devote to the problem.

My recommendation is to use Biham's construction for key-dependent S-boxes. It is easy to implement in software and in hardware chips that have loadable S-boxes, and has no performance penalty over DES. It increases the algorithm's resistance to a brute-force attack, makes differential and linear cryptanalysis harder, and gives the NSA something at least as strong as DES—but different—to worry about.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 13
Other Block Ciphers

13.1 Lucifer

In the late 1960s, led by Horst Feistel and later by Walt Tuchman, IBM initiated a research program in computer cryptography called Lucifer. Lucifer is also the name of a block algorithm that came out of that program in the early 1970s [1482, 1484]. In fact, there are at least two different algorithms with that name [552,1492]. And [552] leaves some gaps in the specification of the algorithm. All this has led to more than a little confusion.

Lucifer is a substitution-permutation network, with building blocks similar to DES. In DES, the output of the function f is XORed with the input of the previous round to form the input of the next round. Lucifer's S-boxes have 4-bit inputs and 4-bit outputs; the input of the S-boxes is the bit-permuted output of the S-boxes of the previous round; the input of the S-boxes of the first round is the plaintext. A key bit is used to choose the actual S-box from two possible S-boxes. (Lucifer represents this as a single T-box with 9 bits in and 8 bits out.) Unlike DES, there is no swapping between rounds and no block halves are used. Lucifer has 16 rounds, 128-bit blocks, and a key schedule simpler than DES.

Using differential cryptanalysis against the first incarnation of Lucifer, Biham and Shamir [170, 172] showed that Lucifer, with 32-bit blocks and 8 rounds, can be broken with 40 chosen plaintexts and 229 steps; the same attack can break Lucifer with 128-bit blocks and 8 rounds with 60 chosen plaintexts and 253 steps. Another differential cryptanalytic attack breaks 18-round, 128-bit Lucifer with 24 chosen plaintexts in 221 steps. All of these attacks used the strong DES S-boxes. Using differential cryptanalysis against the second incarnation, they found the S-boxes to be much weaker than DES. Further analysis showed that over half the possible keys are insecure [112]. Related-key cryptanalysis can break 128-bit Lucifer, with any number of rounds, with 233 chosen-key chosen plaintexts, or with 265 chosen-key known plaintexts [158]. The second incarnation of Lucifer is even weaker [170, 172, 112].

Some people feel that Lucifer is more secure than DES because of the longer key length and lack of published results. This is clearly not the case.

Lucifer is the subject of several U.S. patents: [553, 554, 555, 1483]. They have all expired.

13.2 Madryga

W. E. Madryga proposed this block algorithm in 1984 [999]. It is efficient for software: It has no irritating permutations and all its operations work on bytes.

His design objectives are worth repeating:

1. The plaintext cannot be derived from the ciphertext without using the key. (This just means that the algorithm is secure.)
2. The number of operations required to determine the key from a sample of plaintext and ciphertext should be statistically equal to the product of the operations in an encryption times the number of possible keys. (This means that no plaintext attack should be better than brute force.)
3. Knowledge of the algorithm should not defeat the strength of the cipher. (All the security should rest in the key.)
4. A one-bit change of the key should produce a radical change in the ciphertext using the same plaintext, and a 1-bit change of the plaintext should produce a radical change in the ciphertext using the same key. (This is the avalanche effect.)
5. The algorithm should contain a noncommutative combination of substitution and permutation.
6. The algorithm should include substitutions and permutations under the control of both the input data and the key.
7. Redundant bit groups in the plaintext should be totally obscured in the ciphertext.
8. The length of the ciphertext should be the same length as the plaintext.
9. There should be no simple relationships between any possible keys and ciphertext effects.
10. Any possible key should produce a strong cipher. (There should be no weak keys.)
11. The length of the key and the text should be adjustable to meet varying security requirements.
12. The algorithm should be efficiently implementable in software on large mainframes, minicomputers, and microcomputers, and in discrete logic. (In fact, the functions used in the algorithm are limited to XOR and bit-shifting.)

DES had already met objectives one through nine, but the next three were new. Assuming that the best way to break the algorithm was through brute force, a variable-length key would surely silence those who thought 56 bits was too low. They could implement this algorithm with any key length they desired. And, for anyone who has ever attempted to implement DES in software, an algorithm that took software implementations into account would be welcomed.

Description of Madryga

Madryga consists of two nested cycles. The outer cycle repeats eight times (although this could be increased if security warrants) and consists of an application of the inner cycle to the plaintext. The inner cycle transforms plaintext to ciphertext and repeats once for each 8-bit block (byte) of the plaintext. Thus, the algorithm passes through the entire plaintext eight successive times.

An iteration of the inner cycle operates on a 3-byte window of data, called the working frame (see Figure 13.1). This window advances 1 byte for each iteration. (The data are considered circular when dealing with the last 2 bytes.) The first 2 bytes of the working frame are together rotated a variable number of positions, while the last byte is XORed with some key bits. As the working frame advances, all bytes are successively rotated and XORed with key material. Successive rotations overlap the results of a previous XOR and rotation, and data from the XOR is used to influence the rotation. This makes the entire process reversible.

Because every byte of data influences the 2 bytes to its left and the 1 byte to its right, after eight passes every byte of the ciphertext is dependent on 16 bytes to the left and 8 bytes to the right.

When encrypting, each iteration of the inner cycle starts the working frame at the next-to-last byte of the plaintext and advances circularly through to the third-to-last byte of the plaintext. First, the entire key is XORed with a random constant and then rotated to the left 3 bits. The low-order 3 bits of the low-order byte of the working frame are saved; they will control the rotation of the other 2 bytes. Then, the low-order byte of the working frame is XORed with the low-order byte of the key. Next, the concatenation of the 2 high-order bytes are rotated to the left the variable number of bits (0 to 7). Finally, the working frame is shifted to the right 1 byte and the whole process repeats.

Figure 13.1 One iteration of Madryga.

The point of the random constant is to turn the key into a pseudo-random sequence. The length of this constant must be equal to the length of the key and must be the same for everyone who wishes to communicate with one another. For a 64-bit key, Madryga recommends the constant 0x0f1e2d3c4b5a6978.

Decryption reverses this process. Each iteration of the inner cycle starts the working frame at the third-to-last byte of the ciphertext and advances in the reverse direction circularly through to the second-to-last byte of the ciphertext. Both the key and the 2 ciphertext bytes are shifted to the right. And the XOR is done before the rotations.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Cryptanalysis of Madryga

Researchers at Queensland University of Technology [675] examined Madryga, along with several other block ciphers. They observed that the algorithm didn't exhibit the plaintext-ciphertext avalanche effect. Additionally, many ciphertexts had a higher percentage of ones than zeros.

Although I know of no formal analysis of the algorithm, it doesn't look terribly secure. A cursory review by Eli Biham led to the following observations [160]:

The algorithm consists only of linear operations (rotations and XOR), which are slightly modified depending on the data.

There is nothing like the strength of DES's S-boxes.

The parity of all the bits of the plaintext and the ciphertext is a constant, depending only on the key. So, if you have one plaintext and its corresponding ciphertext, you can predict the parity of the ciphertext for any plaintext.

None of this is damning in itself, but it doesn't leave me with a good feeling about the algorithm. I do not recommend Madryga.

13.3 NewDES

NewDES was designed in 1985 by Robert Scott as a possible DES replacement [1405, 364]. The algorithm is not a DES variant, as its name might imply. It operates on 64-bit blocks of plaintext, but it has a 120-bit key. NewDES is simpler than DES, with no initial or final permutations. All operations are on entire bytes. (Actually, NewDES isn't anything like a new version of DES; the name is unfortunate.)

The plaintext block is divided into eight 1-byte sub-blocks: B0, B1,..., B6, B7. Then the sub-blocks go through 17 rounds. Each round has eight steps. In each step, one of the sub-blocks is XORed with some key material (there is one exception), substituted with another byte via an f function, and then XORed with another sub-block to become that sub-block. The 120-bit key is divided into 15 key sub-blocks: K0, K1,..., K13, K14. The process is easier to understand visually than to describe. Figure 13.2 shows the NewDES encryption algorithm.

The f-function is derived from the Declaration of Independence. See [1405] for details.

Scott showed that every bit of the plaintext block affects every bit of the ciphertext block after only 7 rounds. He also analyzed the f function and found no obvious problems. NewDES has the same complementation property that DES has [364]: If EK(P) = C, then EK´(P´) = C´. This reduces the work required for a brute-force attack from 2120 steps to 2119 steps. Biham noticed that any change of a full byte, applied to all the key and data bytes, leads to another complementation property [160]. This reduces a brute-force attack further to 2112 steps.

Figure 13.2 NewDES.

This is not damning, but Biham's related-key cryptanalytic attack can break NewDES with 233 chosen-key chosen-plaintexts in 248 steps [160]. While this attack is time-consuming and largely theoretical, it shows that NewDES is weaker than DES.

13.4 FEAL

FEAL was designed by Akihiro Shimizu and Shoji Miyaguchi from NTT Japan [1435]. It uses a 64-bit block and a 64-bit key. The idea was to make a DES-like algorithm with a stronger round function. Needing fewer rounds, the algorithm would run faster. Unfortunately, reality fell far short of the design goals.

Description of FEAL

Figure 13.3 is a block diagram of one round of FEAL. The encryption process starts with a 64-bit block of plaintext. First, the data block is XORed with 64 key bits. The data block is then split into a left half and a right half. The left half is XORed with the right half to form a new right half. The left and new right halves go through n rounds (four, initially). In each round the right half is combined with 16 bits of key material (using function f) and XORed with the left half to form the new right half. The original right half (before the round) forms the new left half. After n rounds (remember not to switch the left and right halves after the nth round) the left half is again XORed with the right half to form a new right half, and then the left and right halves are concatenated together to form a 64-bit whole. The data block is XORed with another 64 bits of key material, and the algorithm terminates.

Figure 13.3 One round of FEAL.

Function f takes the 32 bits of data and 16 bits of key material and mixes them together. First the data block is broken up into 8-bit chunks, then the chunks are XORed and substituted with each other. Figure 13.4 is a block diagram of function f. The two functions S0 and S1, are defined as:

S0(a,b) = rotate left two bits ((a + b) mod 256)
S1(a,b) = rotate left two bits ((a + b + 1) mod 256)

The same algorithm can be used for decryption. The only difference is: When decrypting, the key material must be used in the reverse order.

Figure 13.5 is a block diagram of the key-generating function. First the 64-bit key is divided into two halves. The halves are XORed and operated on by function fk, as indicated in the diagram. Figure 13.6 is a block diagram of function fk. The two 32-bit inputs are broken up into 8-bit blocks and combined and substituted as shown. S0 and S1 are defined as just shown. The 16-bit key blocks are then used in the encryption/decryption algorithm.

On a 10 megahertz 80286 microprocessor, an assembly-language implementation of FEAL-32 can encrypt data at a speed of 220 kilobits per second. FEAL-64 can encrypt data at a speed of 120 kilobits per second [1104].

Figure 13.4 Function f.

Figure 13.5 Key processing part of FEAL.

Figure 13.6 Function fK.

Cryptanalysis of FEAL

FEAL-4, FEAL with four rounds, was successfully cryptanalyzed with a chosen-plaintext attack in [201] and later demolished [1132]. This later attack, by Sean Murphy, was the first published differential-cryptanalysis attack and required only 20 chosen plaintexts. The designers retaliated with 8-round FEAL [1436,1437,1108] which Biham and Shamir cryptanalyzed at the SECURICOM '89 conference[1427]. Another chosen-plaintext attack, using only 10,000 blocks, against FEAL-8 [610] forced the designers to throw up their hands and define FEAL-N [1102,1104], with a variable number of rounds (greater than 8, of course).

Biham and Shamir used differential cryptanalysis against FEAL-N; they could break it more quickly than by brute force (with fever than 264 chosen plaintext encryptions) for N less than 32 [169]. FEAL-16 required 228 chosen plaintexts or 246.5 known plaintexts to break. FEAL-8 required 2000 chosen plaintexts or 237.5 known plaintexts to break. FEAL-4 could be broken with just eight carefully selected chosen plaintexts.

The FEAL designers also defined FEAL-NX, a modification of FEAL, that accepts 128-bit keys (see Figure 13.7)[1103,1104]. Biham and Shamir showed that FEAL-NX with a 128-bit key is just as easy to break as FEAL-N with a 64-bit key, for any value of N [169]. Recently FEAL-N(X)S has been proposed, which strengthens FEAL with a dynamic swapping function [1525].

There's more. Another attack against FEAL-4, requiring only 1000 known plaintexts, and against FEAºL-8, requiring only 20,000 known plaintexts, was published in [1520]. Other attacks are in [1549,1550]. The best attack is by Mitsuru Matsui and Atshuiro Yamagishi [1020]. This is the first use of linear cryptanalysis, and can break FEAL-4 with 5 known plaintexts, FEAL-6 with 100 known plaintexts and FEAL-8 with 215 known plaintexts. Further refinements are in [64]. Differential-linear cryptanalysis can break FEAL-8 with only 12 chosen plaintexts [62]. Whenever someone discovers a new cryptanalytic attack, he always seems to try it out on FEAL first.

Patents

FEAL is patented in the United States [1438] and has patents pending in England, France, and Germany. Anyone wishing to license the algorithm should contact the Intellectual Property Department, NTT, 1-6 Uchisaiwai-cho, 1-chome, Chiyoda-ku, 100 Japan.

13.5 REDOC

REDOC II is another block algorithm, designed by Michael Wood for Cryptech, Inc. [1613,400]. It has a 20-byte (160-bit) key and an 80-bit block.

REDOC II performs all of its manipulations—permutations, substitutions, and key XORs—on bytes; the algorithm is efficient in software. REDOC II uses variable function tables. Unlike DES, which has a fixed (albeit optimized for security) set of permutation and substitution tables, REDOC II uses a key-dependent and plaintext-dependent set of tables (S-boxes, actually). REDOC II has 10 rounds; each round is a complicated series of manipulations on the block.

Figure 13.7 FEAL-NX key schedule.

Another unique feature in the design is the use of masks. These are numbers derived from the key table that are used to select the tables in a given function within a given round. Both the value of the data and the masks are used together to select the function tables.

Assuming that brute force is the most efficient means of attack, REDOC II is very secure: 2160 operations are required to recover the key. Thomas Cusick cryptanalyzed 1 round of REDOC II, but he was unable to extend the attack to multiple rounds [400]. Using differential cryptanalysis, Biham and Shamir were able to successfully cryptanalyze 1 round of REDOC II with 2300 chosen-plaintexts [170]. This attack cannot be extended to multiple rounds, but they were able to obtain three mask values after 4 rounds. I know of no other cryptanalysis.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

REDOC III

REDOC III is a streamlined version of REDOC II, also designed by Michael Wood [1615]. It operates on an 80-bit block. The key length is variable and can be as large as 2560 bytes (20, 480 bits). The algorithm consists solely of XORing key bytes with message bytes; there are no permutations or substitutions.

(1) Create a key table of 256 10-byte keys, using the secret key.
(2) Create two 10-byte mask blocks, M1 and M2. M1 is the XOR of the first 128 10-byte keys; M2 is the XOR of the second 128 10-byte keys.
(3) To encrypt a 10-byte block:
(a) XOR the first byte of the data block with the first byte of M1. Select a key from the key table computed in step (1). Use the computed XOR as the index into the table. XOR each byte in the data block with the corresponding byte in the chosen key, except for the first data byte.
(b) XOR the second byte of the data block with the second byte of M1. Select a key from the key table computed in step (1). Use the computed XOR as the index into the table. XOR each byte in the data block with the corresponding byte in the chosen key, except for the second data byte.
(c) Continue with the entire block (bytes 3 through 10), until each byte has been used to select a key from the key table after XORing it with the corresponding M1 value. Then XOR each byte with the key except for the byte used to select the key.
(d) Repeat steps (a) through (c) with M2.

The algorithm is easy and fast. On a 33 megahertz 80386, the algorithm encrypts data at 2.75 megabits per second. Wood estimates that a VLSI-pipelined design, with a 64-bit data path, woud encrypt data at over 1.28 gigabits per second with a 20 megahertz clock.

REDOC III is not secure [1440]. It is vulnerable to differential cryptanalysis. Only about 223 chosen plaintexts are required to reconstruct both masks.

Patents and Licenses

Both REDOC versions are patented in the United States [1614]. Foreign patents are pending. Anyone interested in licensing either REDOC II or REDOC III should contact Michael C. Wood, Delta Computec, Inc., 6647 Old Thompson Rd., Syracuse, NY 13211.

13.6 LOKI

LOKI is Australian and was first presented in 1990 as a potential alternative to DES [273]. It uses a 64-bit block and a 64-bit key. The general structure of the algorithm and key schedule were based on [274, 275], and the design of the S-boxes was based on [1247].

Using differential cryptanalysis, Biham and Shamir were able to break LOKI with 11 or fewer rounds faster than by brute force [170]. Furthermore, there is an 8-bit complementation property, which reduces the complexity of a brute-force attack by a factor of 256 [170, 916, 917].

Lars Knudsen showed that LOKI, with 14 rounds or fewer, is vulnerable to differential cryptanalysis [852, 853]. Additionally, if LOKI is implemented with alternate S-boxes, the resulting cipher will probably be vulnerable to differential cryptanalysis.

LOKI91

In response to these attacks, LOKI's designers went back to the drawing board and revised their algorithm. The result is LOKI91 [272]. (The previous version of LOKI was renamed LOKI89.)

To make the algorithm more resistant to differential cryptanalysis and to remove the complementation property, the following changes were made to the original design:

1. The subkey generation algorithm was changed so that the halves were swapped every second round, not every round.
2. The subkey generation algorithm was changed so that the rotation of the left subkey alternated between 12 and 13 bits to the left.
3. The initial and final XOR of the block with the key were eliminated.
4. The S-box function was altered to flatten out their XOR profile (to improve their resistance to differential cryptanalysis), and to eliminate any value of x such that f(x) = 0, where f is the combination of the E-, S-, and P-boxes.

Description of LOKI91

The mechanics of LOKI91 are similar to DES (see Figure 13.8). The data block is then divided into a left half and a right half and goes through 16 rounds, much like DES. In each round, the right half is first XORed with a piece of the key, then sent through an expansion permutation (see Table 13.1).

The 48-bit output is divided into four 12-bit blocks, and each block is sent through an S-box substitution. The S-box substitution is as follows: Take each 12-bit input; use the 2 left-most bits and the 2 right-most bits to form the number r, and the 8 innermost bits and form the number c. The output of the S-box, O, is as follows:

O(r,c) = (c + ((r*17) ⊕ 0xff) & 0xff)31 mod Pr

Figure 13.8 LOKI91.

Pr is given in Table 13.2.

Then, the four 8-bit outputs are recombined to form a single 32-bit number and sent through the permutation described in Table 13.3. Finally, the right half is XORed with the left half to become the new left half, and the left half becomes the new right half. After 16 rounds, the block is again XORed with the key to produce the ciphertext.

The subkeys are generated from the key in a straightforward manner. The 64-bit key is split into a left half and a right half. In each round, the subkey is the left half. This left half is then rotated 12 or 13 bits to the left, and then every two rounds the left and right halves are exchanged. As with DES, the same algorithm can be used for both encryption and decryption, with some modification in how the subkeys are used.

Table 13.1 Expansion Permutation
4, | 3, | 2, | 1, | 32, | 31, | 20, | 29, | 28, | 27, | 26, | 25,
28, | 27, | 26, | 25, | 24, | 23, | 22, | 21, | 20, | 19, | 18, | 17,
20, | 19, | 18, | 17, | 16, | 15, | 14, | 13, | 12, | 11, | 10, | 9,
12, | 11, | 10, | 9, | 8, | 7, | 6, | 5, | 4, | 3, | 2, | 1

Table 13.2 Pr
r: | 1, | 2, | 3, | 4, | 5, | 6, | 7, | 8, | 9, | 10, | 11, | 12, | 13, | 14, | 15, | 16
Pr: | 375, | 379, | 391, | 395, | 397, | 415, | 419, | 425, | 433, | 445, | 451, | 463, | 471, | 477, | 487, | 499

Cryptanalysis of LOKI91

Knudsen attempted to cryptanalyze LOKI91 [854, 858], but found it secure against differential cryptanalysis. However, he found a related-key chosen-plaintext attack that reduces the complexity of a brute-force search by almost a factor of four. This attack exploits a weakness in the key schedule and may also apply if the algorithm is used as a one-way hash function (see Section 18.11).

Another attack on related keys can break LOKI91 with 232 chosen-key chosen plaintexts, or 248 chosen-key known plaintexts [158]. The attack is independent of the number of rounds of the algorithm. (In the same paper, Biham breaks LOKI89 with 217 chosen-key chosen plaintexts or 233 known-key known plaintexts using related-key cryptanalysis.) It's easy to make LOKI91 resistant to this attack; avoid the simple key schedule.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Patents and Licenses

LOKI is not patented. Anyone can implement the algorithm and use it. The source code implementation in this book is copyrighted by the University of New South Wales. Anyone interested in using this implementation (or their other implementation, which is several orders of magnitude faster) in a commercial product should contact Director CITRAD, Department of Computer Science, University College, UNSW, Australian Defense Force Academy, Canberra ACT 2600, Australia; FAX: +61 6 268 8581.

13.7 Khufu and Khafre

In 1990 Ralph Merkle proposed two algorithms. The basic design principles behind them are [1071]:

1. DES's 56-bit key size is too small. Considering the negligible cost of increasing the key size (computer memory is cheap and plentiful), it should be increased.
2. DES's extensive use of permutations, while suitable for hardware implementations, is very difficult to implement in software. The faster software implementations of DES implement the permutations by table lookup. Table lookup can provide the same "diffusion" characteristics as permutation and can be much more flexible.
Table 13.3 P-Box Permutation
32, | 24, | 16, | 8, | 31, | 23, | 15, | 7, | 30, | 22, | 14, | 6, | 29, | 21, | 13, | 5,
28, | 20, | 12, | 4, | 27, | 19, | 11, | 3, | 26, | 18, | 10, | 2, | 25, | 17, | 9, | 1
3. The S-boxes in DES are small, with only 64 4-bit entries per box. Now that memory is larger, S-boxes should grow. Moreover, all eight S-boxes are used simultaneously. While this is suitable for hardware, it seems like an unreasonable restriction in software. A larger S-box size and sequential (rather than parallel) S-box usage should be employed.
4. The initial and final permutations in DES are widely viewed as cryptographically pointless and should be discarded.
5. All the faster implementations of DES precompute the keys for each round. Given this fact, there is no reason not to make this computation more complicated.
6. Unlike DES, the S-box design criteria should be public.

To this list, Merkle would probably now add "resistant to differential cryptanalysis and to linear attacks, " but those attacks were still unknown at the time.

Khufu

Khufu is a 64-bit block cipher. The 64-bit plaintext is first divided into two 32-bit halves, L and R. First, both halves are XORed with some key material. Then, they are subjected to a series of rounds similar to DES. In each round, the least significant byte of L is used as the input to an S-box. Each S-box has 8 input bits and 32 output bits. The selected 32-bit entry in the S-box is then XORed with R. L is then rotated some multiple of 8 bits, L and R are swapped, and the round ends. The S-box itself is not static, but changes every 8 rounds. Finally, after the last round, L and R are XORed with more key material, and then combined to form the ciphertext block.

Although parts of the key are XORed with the encryption block at the beginning and end of the algorithm, the primary purpose of the key is to generate the S-boxes. These S-boxes are secret and, in essence, part of the key. Khufu calls for a total key size of 512 bits (64 bytes) and gives an algorithm for generating S-boxes from the key. The number of rounds for the algorithm is left open. Merkle mentioned that 8-round Khufu is susceptible to a chosen-plaintext attack and recommended 16, 24, or 32 rounds [1071]. (He restricted the choice of rounds to a multiple of eight.)

Because Khufu has key-dependent and secret S-boxes, it is resistant to differential cryptanalysis. There is a differential attack against 16-round Khufu that recovers the key after 231 chosen plaintexts [611], but it cannot be extended to more rounds. If brute-force is the best way to attack Khufu, it is impressively secure. A 512-bit key gives a complexity of 2512—inconceivable under any circumstances.

Khafre

Khafre is the second of two cryptosystems proposed by Merkle [1071]. (Khufu and Khafre are names of Egyptian pharaohs.) It is similar in design to Khufu, except that it was designed for applications without precomputation time. The S-boxes are not key-dependent. Instead, Khafre uses fixed S-boxes. And the key is XORed with the encryption block not only before the first round and after the last round, but also after every 8 rounds of encryption.

Merkle speculated that key sizes of 64- or 128-bits would be used for Khafre and that more rounds of encryption would be required for Khafre than for Khufu. This, combined with the fact that each round of Khafre is more complex than for Khufu, makes Khafre slower. In compensation, Khafre does not require any precomputation and will encrypt small amounts of data more quickly.

In 1990 Biham and Shamir turned their differential cryptanalysis techniques against Khafre [170]. They were able to break 16-round Khafre with a chosen-plaintext attack using about 1500 different encryptions. It took about an hour, using their personal computer. Converting that to a known-plaintext attack would require about 238 encryptions. Khafre with 24 rounds can be broken by a chosen-plaintext attack using 253 encryptions, and a known-plaintext attack using 259 encryptions.

Patents

Both Khufu and Khafre are patented [1072]. Source code for the algorithms are in the patent. Anyone interested in licensing either or both algorithms should contact Director of Licensing, Xerox Corporation, P.O. Box 1600, Stamford, CT, 06904-1600.

13.8 RC2

RC2 is a variable-key-size encryption algorithm designed by Ron Rivest for RSA Data Security, Inc. (RSADSI). Apparently, "RC" stands for "Ron's Code, " although it officially stands for "Rivest Cipher." (RC3 was broken at RSADSI during development; RC1 never got further than Rivest's notebook.) It is proprietary, and its details have not been published. Don't think for a minute that this helps security. RC2 has already appeared in commercial products. As far as I know, RC2 has not been patented and is only protected as a trade secret.

RC2 is a variable-key-size 64-bit block cipher, designed to be a replacement for DES. According to the company, software implementations of RC2 are three times faster than DES. The algorithm accepts a variable-length key, from 0 bytes to the maximum string length the computer system supports; encryption speed is independent of key size. This key is preprocessed to yield a key-dependent table of 128 bytes. So the number of effectively different keys is 21024. RC2 has no S-boxes [805]; the two operations are "mix" and "mash, " and one is chosen in each round. According to their literature [1334]:

... RC2 is not an iterative block cipher. This suggests that RC2 offers more protection against differential and linear cryptanalysis than other block ciphers which have relied for their security on copying the design of DES.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

RSADSI's refusal to make RC2 public casts doubt on their claims. They are willing to provide details of the algorithm to most anyone willing to sign a nondisclosure agreement, and have claimed to allow cryptanalysts to publish any negative results they find. I don't know of any cryptanalyst outside the employ of the company who studied it, since it would amount to doing their analysis work for them.

Still, Ron Rivest is not the usual snake-oil peddler. He's a respected and competent cryptographer. I would put a fair degree of trust in the algorithm, even though I haven't personally inspected the code. RC4, once the proprietary intellectual property of RSADSI, was posted to the Internet (see Section 17.1), and it's probably just a matter of time before RC2 is posted as well.

An agreement between the Software Publishers Association (SPA) and the U.S. government gave RC2 and RC4 (see Section 17.1) special export status (see Section 25.14). Products that implement one of these two algorithms have a much simpler export approval process, provided that the keys are no more than 40 bits long.

Is a 40-bit key enough? There are a total of one trillion possible keys. Assuming that brute force is the most efficient method of cryptanalysis (a big assumption, considering that the algorithm has never been published), and assuming that a brute-force cryptanalysis chip can test one million keys per second, it will take him 12.7 days to find the correct key. One thousand machines working in parallel can produce the key in twenty minutes.

RSA Data Security, Inc., maintains that while encryption and decryption are quick, exhaustive key search is not. A significant amount of time is spent setting up the key schedule. While this time is negligible when encrypting and decrypting messages, it is not when trying every possible key.

The U.S. government would never allow export of any algorithm it couldn't, at least in theory, break. They could create a magnetic tape or CD of a specific plaintext block encrypted with every possible key. To break a given message, they could just run the tape and compare the ciphertext blocks in the message with the ciphertext blocks on the tape. If there is a match, they could try the candidate key and see if the message makes any sense. If they choose a common plaintext block (all zeros, the ASCII characters for a space, etc.), this method should work. The storage requirement for a 64-bit plaintext block encrypted with all 1012 possible keys is 8 terabytes—certainly possible.

For information on licensing RC2, contact RSADSI (see Section 25.4).

13.9 IDEA

The first incarnation of the IDEA cipher, by Xuejia Lai and James Massey, surfaced in 1990 [929]. It was called PES (Proposed Encryption Standard). The next year, after Biham and Shamir's demonstrated differential cryptanalysis, the authors strengthened their cipher against the attack and called the new algorithm IPES (Improved Proposed Encryption Standard) [931,924]. IPES changed its name to IDEA (International Data Encryption Algorithm) in 1992 [925].

IDEA is based on some impressive theoretical foundations and, although cryptanalysis has made some progress against reduced-round variants, the algorithm still seems strong. In my opinion, it is the best and most secure block algorithm available to the public at this time.

The future of IDEA is not yet clear. There has been no rush to adopt it as a replacement to DES, partly because it is patented and must be licensed for commercial applications, and partly because people are still waiting to see how well the algorithm fares during the coming years of cryptanalysis. Its current claim to fame is that it is part of PGP (see Section 24.12).

Overview of IDEA

IDEA is a block cipher; it operates on 64-bit plaintext blocks. The key is 128 bits long. The same algorithm is used for both encryption and decryption.

As with all the other block ciphers we've seen, IDEA uses both confusion and diffusion. The design philosophy behind the algorithm is one of "mixing operations from different algebraic groups." Three algebraic groups are being mixed, and they are all easily implemented in both hardware and software:

— XOR
— Addition modulo 216
— Multiplication modulo 216 + 1. (This operation can be viewed as IDEA's S-box.)

All these operations (and these are the only operations in the algorithm—there are no bit-level permutations) operate on 16-bit sub-blocks. This algorithm is even efficient on 16-bit processors.

Description of IDEA

Figure 13.9 is an overview of IDEA. The 64-bit data block is divided into four 16-bit sub-blocks: X1, X2, X3, and X4. These four sub-blocks become the input to the first round of the algorithm. There are eight rounds total. In each round the four sub-blocks are XORed, added, and multiplied with one another and with six 16-bit subkeys. Between rounds, the second and third sub-blocks are swapped. Finally, the four sub-blocks are combined with four subkeys in an output transformation.

In each round, the sequence of events is as follows:

(1) Multiply X1 and the first subkey.
(2) Add X2 and the second subkey.
(3) Add X3 and the third subkey.
(4) Multiply X4 and the fourth subkey.
(5) XOR the results of steps (1) and (3).
(6) XOR the results of steps (2) and (4).
(7) Multiply the results of step (5) with the fifth subkey.
(8) Add the results of steps (6) and (7).
(9) Multiply the results of step (8) with the sixth subkey.
(10) Add the results of steps (7) and (9).

Figure 13.9 IDEA.

(11) XOR the results of steps (1) and (9).
(12) XOR the results of steps (3) and (9).
(13) XOR the results of steps (2) and (10).
(14) XOR the results of steps (4) and (10).

The output of the round is the four sub-blocks that are the results of steps (11), (12), (13), and (14). Swap the two inner blocks (except for the last round) and that's the input to the next round.

After the eighth round, there is a final output transformation:

(1) Multiply X1 and the first subkey.
(2) Add X2 and the second subkey.
(3) Add X3 and the third subkey.
(4) Multiply X4 and the fourth subkey.

Finally, the four sub-blocks are reattached to produce the ciphertext.

Creating the subkeys is also easy. The algorithm uses 52 of them (six for each of the eight rounds and four more for the output transformation). First, the 128-bit key is divided into eight 16-bit subkeys. These are the first eight subkeys for the algorithm (the six for the first round, and the first two for the second round). Then, the key is rotated 25 bits to the left and again divided into eight subkeys. The first four are used in round 2; the last four are used in round 3. The key is rotated another 25 bits to the left for the next eight subkeys, and so on until the end of the algorithm.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Decryption is exactly the same, except that the subkeys are reversed and slightly different. The decryption subkeys are either the additive or multiplicative inverses of the encryption subkeys. (For the purposes of IDEA, the all-zero sub-block is considered to represent 216 = – 1 for multiplication modulo 216 + 1; thus the multiplicative inverse of 0 is 0.) Calculating these takes some doing, but you only have to do it once for each decryption key. Table 13.4 shows the encryption subkeys and the corresponding decryption subkeys.

Speed of IDEA

Current software implementations of IDEA are about twice as fast as DES. IDEA on a 33 megahertz 386 machine encrypts data at 880 kilobits per second, and 2400 kilobits per second on a 66 megahertz 486 machine. You might think IDEA should be faster, but multiplications aren't cheap. To multiply two 32-bit numbers on a 486 requires 40 clock cycles (10 on a Pentium).

A VLSI implementation of PES encrypts data at 55 megabits per second at 25 megahertz [208, 398]. Another VLSI chip developed at ETH Zurich, consisting of 251, 000 transistors on a chip 107.8 square millimeters, encrypts data using the IDEA algorithm at a 177 megabit-per-second data rate when clocked at 25 megahertz [926, 207, 397].

Table 13.4 IDEA Encryption and Decryption Subkeys
Round | Encryption Subkeys | Decryption Subkeys
1st | Z1(1) Z2(1) Z3(1) Z4(1) Z5(1) Z6(1) | Z1(9) - 1 –Z2(9) –Z3(9) Z4(9) - 1 Z5(8) Z6(8)
2nd | Z1(2) Z2(2) Z3(2) Z4(2) Z5(2) Z6(2) | Z1(8) - 1 –Z3(8) –Z2(8) Z4(8) - 1 Z5(7) Z6(7)
3rd | Z1(3) Z2(3) Z3(3) Z4(3) Z5(3) Z6(3) | Z1(7) - 1 –Z3(7) –Z2(7) Z4(7) - 1 Z5(6) Z6(6)
4th | Z1(4) Z2(4) Z3(4) Z4(4) Z5(4) Z6(4) | Z1(6) - 1 –Z3(6) –Z2(6) Z4(6) - 1 Z5(5) Z6(5)
5th | Z1(5) Z2(5) Z3(5) Z4(5) Z5(5) Z6(5) | Z1(5) - 1 –Z3(5) –Z2(5) Z4(5) - 1 Z55(4) Z6(4)
6th | Z1(6) Z2(6) Z3(6) Z4(6) Z5(6) Z6(6) | Z1(4) - 1 –Z3(4) –Z2(4) Z4(4) - 1 Z5(3) Z6(3)
7th | Z1(7) Z2(7) Z3(7) Z4(7) Z5(7) Z6(7) | Z1(3) - 1 –Z3(3) –Z2(3) Z4(3) - 1 Z5(2) Z6(2)
8th | Z1(8) Z2(8) Z3(8) Z4(8) Z5(8) Z6(8) | Z1(2) - 1 –Z3(2) –Z2(2) Z4(2) - 1 Z5(1) Z6(1)
output transformation | Z1(9) Z2(9) Z3(9) Z4(9) | Z1(1) - 1 –Z2(1) –Z3(1) Z4(1) - 1

Cryptanalysis of IDEA

IDEA's key length is 128 bits—over twice as long as DES. Assuming that a brute-force attack is the most efficient, it would require 2128(1038) encryptions to recover the key. Design a chip that can test a billion keys per second and throw a billion of them at the problem, and it will still take 1013 years—that's longer than the age of the universe. An array of 1024 such chips can find the key in a day, but there aren't enough silicon atoms in the universe to build such a machine. Now we're getting somewhere—although I'd keep my eye on the dark matter debate.

Perhaps brute force isn't the best way to attack IDEA. The algorithm is still too new for any definitive cryptanalytic results. The designers have done their best to make the algorithm immune to differential cryptanalysis; they defined the concept of a Markov cipher and showed that resistance to differential cryptanalysis can be modeled and quantified [931, 925]. (Figure 13.10 shows the original PES algorithm to be contrasted with the IDEA algorithm of Figure 13.9 which was strengthened against differential cryptanalysis. It's amazing how a few subtle changes can make such a big difference.) In [925], Lai argued (he gave evidence, not a proof) that IDEA is immune to differential cryptanalysis after only 4 of its 8 rounds. According to Biham, his related-key cryptanalytic attack doesn't work against IDEA, either [160].

Willi Meier examined the three algebraic operations of IDEA, and pointed out that while they are incompatible, there are instances where they can be simplified in such a way as to facilitate cryptanalysis some percentage of the time [1050]. His attack is more efficient than brute-force for 2-round IDEA (242 operations), but less efficient for 3-round IDEA or higher. Normal IDEA, with 8 rounds, is safe.

Joan Daemen discovered a class of weak keys for IDEA [406, 409]. These are not weak keys in the sense of the DES weak keys; that is, the encryption function is self-inverse. They are weak in the sense that if they are used, an attacker can easily identify them in a chosen-plaintext attack. For example, a weak key is (in hex):

0000, 0000, 0x 00, 0000, 0000, 000x,xxxx,x000


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The number at the positions of "x" can be any number. If this key is used, the bit-wise XOR of certain plaintext pairs guarantees the bit-wise XOR of the resultant ciphertext pairs.

In any case, the chance of accidentally generating one of these weak keys is very small: one in 296. There is no danger if you choose keys at random. And it is easy to modify IDEA so that it doesn't have any weak keys: XOR every subkey with the value 0x0dae [409].

I know of no other cryptanalytic results against IDEA, although many people have tried.

IDEA Modes of Operation and Variants

IDEA can work within any block cipher mode discussed in Chapter 9. Any double-IDEA implementation would be susceptible to the same meet-in-the-middle attack as DES (see Section 15.1). However, because IDEA's key length is more than double DES's, the attack is impractical. It would require a storage space of 64*2128 bits, or 1039 bytes. Maybe there's enough matter in the universe to create a memory device that large, but I doubt it.

Figure 13.10 PES.

If you're worried about parallel universes as well, use a triple-IDEA implementation (see Section 15.2):

C = EK3(DK2(EK1(P)))

It is immune to the meet-in-the-middle attack.

There's also no reason why you can't implement IDEA with independent subkeys, especially if you have key-management tools to handle the longer key. IDEA needs a total of 52 16-bit keys, for a total key length of 832 bits. This variant is definitely more secure, but no one knows by how much.

A naïve variation might double the block size. The algorithm would work just as well with 32-bit sub-blocks instead of 16-bit sub-blocks, and a 256-bit key. Encryption would be quicker and security would increase 232 times. Or would it? The theory behind the algorithm hinges on the fact that 216 + 1 is prime; 232 + 1 is not. Perhaps the algorithm could be modified to work, but it would have very different security properties. Lai says it would be difficult to make it work [926].

While IDEA appears to be significantly more secure than DES, it isn't always easy to substitute one for the other in an existing application. If your database and message templates are hardwired to accept a 64-bit key, it may be impossible to implement IDEA's 128-bit key.

For those applications, generate a 128-bit key by concatenating the 64-bit key with itself. Remember that IDEA is weakened considerably by this modification.

If you are more concerned with speed than security, you might consider a variant of IDEA with fewer rounds. Currently the best attack against IDEA is faster than brute force only for 2.5 rounds or less [1050]; 4 round IDEA would be twice as fast and, as far as I know, just as secure.

Caveat Emptor

IDEA is a relatively new algorithm, and many questions remain. Is IDEA a group? (Lai thinks not [926].) Are there any still-undiscovered ways of breaking this cipher? IDEA has a firm theoretical basis, but time and time again secure-looking algorithms have fallen to new forms of cryptanalysis. Several academic and military groups have cryptanalyzed IDEA. None of them has gone public about any successes they might have had. One might—someday.

Patents and Licenses

IDEA is patented in Europe and the United States [1012, 1013]. The patent is held by Ascom-Tech AG. No license fee is required for non-commercial use. Commercial users interested in licensing the algorithm should contact Ascom Systec AG, Dept CMVV, Gewerbepark, CH-5506, Mägenwil, Switzerland; +41 64 56 59 83; Fax:+41 64 56 59 90; idea@ascom.ch.

13.10 MMB

A complaint against IDEA, that it uses a 64-bit encryption block, was addressed by Joan Daemen in an algorithm called MMB (Modular Multiplication-based Block cipher) [385, 405, 406]. MMB is based on the same basic theory as IDEA: mixing operations of different algebraic groups. MMB is an iterative algorithm that mainly consists of linear steps (XOR and key applications) and the parallel applications of four large nonlinear invertible substitutions. These substitutions are determined by a multiplication modulo 232 – 1 with constant factors. The result is an algorithm that has both a 128-bit key and a 128-bit block size.

MMB operates on 32-bit sub-blocks of text (x0, x1, x2, x3) and 32-bit sub-blocks of key (k0, k1, k2, k3). This makes the algorithm well suited for implementation on modern, 32-bit processors. A nonlinear function, f, is applied six times alternating with XORing. Here it is (all index operations are mod 4):

xi = xi ⊕ ki, for i = 0 to 3
f(x0,x1,x2,x3)
xi = xi ⊕ ki + 1, for i = 0 to 3
f(x0,x1,x2,x3)
xi = xi ⊕ ki + 2, for i = 0 to 3
f(x0,x1,x2,x3)
xi = xi ⊕ ki, for i = 0 to 3
f(x0,x1,x2,x3)
xi = xi ⊕ ki + 1, for i = 0 to 3
f(x0,x1,x2,x3)
xi = xi ⊕ ki + 2, for i = 0 to 3
f(x0,x1,x2,x3)

The function f has three steps:

(1) xi = ci * xi, for i = 0 to 3 (If the input to the multiplication is all 1s, the output is also all 1s.)
(2) If the least significant bit of x0 = 1, then x0 = x0 ⊕ C. If the least significant byte of x3 = 0, then x3 = x3 ⊕ C.
(3) xi = xi – 1 ⊕ xi ⊕ xi + 1, for i = 0 to 3

All index operations are mod 4. The multiplication operation in step (1) is modulo 232 – 1. For the purposes of the algorithm, if the second operand is 232 – 1, then the result is 232 – 1. The various constants are:

C = 2aaaaaaa
c0 = 025f1cdb
c1 = 2 * c0
c2 = 23 * c0
c3 = 27 * c0

The constant C is the "simplest" constant with a high ternary weight, a least-significant bit of zero, and no circular symmetry. The constant c0 has certain other characteristics. The constants c1, c2, and c3 are shifted versions of c0, preventing attacks based on symmetry. See [405] for more details.

Decryption is the reverse process. Steps (2) and (3) are their own inverse. Step (1) uses ci-1 instead of ci. The value of c0-1 is 0dad4694.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Security of MMB

The design of MMB ensures that each round has considerable diffusion independent of the key. In IDEA, the amount of diffusion is to some extent dependent on the particular subkeys. MMB was also designed not to have any weak keys as IDEA has.

MMB is dead [402]. Although no cryptanalysis has been published, this is true for several reasons. First, it was not designed to be resistant to linear cryptanalysis. The multiplication factors were chosen to be resistant to differential cryptanalysis, but the algorithm's authors were unaware of linear cryptanalysis.

Second, Eli Biham has an effective chosen-key attack [160], which exploits the fact that all rounds are identical and that the key schedule is just a cyclic shift by 32 bits. Third, even though MMB would be very efficient in software, the algorithm would be less efficient than DES in hardware.

Daemen suggests that anyone interested in improving MMB should first do an analysis of modular multiplication with respect to linear cryptanalysis and choose a new multiplication factor, and then make the constant C different for each round [402]. Then, improve the key scheduling by adding constants to the round keys to remove the bias. He's not going to do it; he designed 3-Way instead (see Section 14.5).

13.11 CA-1.1

CA is a block cipher built on cellular automata, designed by Howard Gutowitz [677, 678, 679]. It encrypts plaintext in 384-bit blocks and has a 1088-bit key (it's really two keys, a 1024-bit key and a 64-bit key). Because of the nature of cellular automata, the algorithm is most efficient when implemented in massively parallel integrated circuits.

CA-1.1 uses both reversible and irreversible cellular automaton rules. Under a reversible rule, each state of the lattice comes from a unique predecessor state, while under an irreversible rule, each state can have many predecessor states. During encryption, irreversible rules are iterated backward in time. To go backward from a given state, one of the possible predecessor states is selected at random. This process can be repeated many times. Backward iteration thus serves to mix random information with the message information. CA-1.1 uses a particular kind of partially linear irreversible rule, which is such that a random predecessor state for any given state can be rapidly built. Reversible rules are also used for some stages of encryption.

The reversible rules (simple parallel permutations on sub-blocks of the state) are nonlinear. The irreversible rules are derived entirely from information in the key, while the reversible rules depend both on key information and on the random information inserted during the stages of encryption with irreversible rules.

CA-1.1 is built around a block-link structure. That is, the processing of the message block is partially segregated from the processing of the stream of random information inserted during encryption. This random information serves to link stages of encryption together. It can also be used to chain together a ciphertext stream. The information in the link is generated as part of encryption.

Because CA-1.1 is a new algorithm, it is too early to make any pronouncements on its security. Gutowitz discusses some possible attacks, including differential cryptanalysis, but is unable to break the algorithm. As an incentive, Gutowitz has offered a $1000 prize to "the first person who develops a tractable procedure to break CA-1.1."

CA-1.1 is patented [678], but is available free for non-commercial use. Anyone interested in either licensing the algorithm or in the cryptanalysis prize should contact Howard Gutowitz, ESPCI, Laboratoire d'ƒlectronique, 10 rue Vauquelin, 75005 Paris, France.

13.12 Skipjack

Skipjack is the NSA-developed encryption algorithm for the Clipper and Capstone chips (see Sections 24.16 and 24.17). Since the algorithm is classified Secret, its details have never been published. It will only be implemented in tamperproof hardware.

The algorithm is classified Secret, not because that enhances its security, but because the NSA doesn't want Skipjack being used without the Clipper key-escrow mechanism. They don't want the algorithm implemented in software and spread around the world.

Is Skipjack secure? If the NSA wants to produce a secure algorithm, they presumably can. On the other hand, if the NSA wants to design an algorithm with a trapdoor, they can do that as well.

Here's what has been published [1154, 462].

— It's an iterative block cipher.
— The block size is 64 bits.
— It has an 80-bit key.
— It can be used in ECB, CBC, 64-bit OFB, or 1-, 8-, 16-, 32- or 64-bit CFB modes.
— There are 32 rounds of processing per single encrypt or decrypt operation.
— NSA started the design in 1985 and completed the evaluation in 1990.

The documentation for the Mykotronx Clipper chip says that the latency for the Skipjack algorithm is 64 clock cycles. This means that each round consists of two clock cycles: presumably one for the S-box substitution and another for the final XOR at the end of the round. (Remember: permutations take no time in hardware.) The Mykotronx documentation calls this two-clock-cycle operation a "G-box, " and the whole thing a "shift." (Some part of the G-box is called an "F-table, " probably a table of constants but maybe a table of functions.)

I heard a rumor that Skipjack uses 16 S-boxes, and another that the total memory requirement for storing the S-boxes is 128 bytes. It is unlikely that both of these rumors are true.

Another rumor implies that Skipjack's rounds, unlike DES's, do not operate on half of the block size. This, combined with the notion of "shifts, " an inadvertent statement made at Crypto '94 that Skipjack has "a 48-bit internal structure, " implies that it is similar in design to SHA (see Section 18.7) but with four 16-bit sub-blocks: three sub-blocks go through a key-dependent one-way function to produce 16 bits, which are XORed with the remaining sub-block; then the whole block is circularly shifted 16 bits to become the input to the next round, or shift. This also implies 128 bytes of S-box data. I suspect that the S-boxes are key-dependent.

The structure of Skipjack is probably similar to DES. The NSA realizes that their tamperproof hardware will be reverse-engineered eventually; they won't risk any advanced cryptographic techniques.

The fact that the NSA is planning to use the Skipjack algorithm to encrypt their Defense Messaging System (DMS) implies that the algorithm is secure. To convince the skeptics, NIST allowed a panel of "respected experts from outside the government...access to the confidential details of the algorithm to assess its capabilities and publicly report its findings" [812].

The preliminary report of these experts [262] (there never was a final report, and probably never will be) concluded that:

Under an assumption that the cost of processing power is halved every 18 months, it will be 36 years before the difficulty of breaking Skipjack by exhaustive search will be equal to the difficulty of breaking DES today. Thus, there is no significant risk that Skipjack will be broken by exhaustive search in the next 30–40 years.

There is no significant risk that Skipjack can be broken through a shortcut method of attack, including differential cryptanalysis. There are no weak keys; there is no complementation property. The experts, not having time to evaluate the algorithm to any great extent, instead evaluated NSA's own design and evaluation process.

The strength of Skipjack against a cryptanalytic attack does not depend on the secrecy of the algorithm.

Of course, the panelists did not look at the algorithm long enough to come to any conclusions themselves. All they could do was to look at the results that the NSA showed to them.

One unanswered question is whether the Skipjack keyspace is flat (see Section 8.2). Even if Skipjack has no weak keys in the DES sense, some artifact of the key-scheduling process could make some keys stronger than others. Skipjack could have 270 strong keys, far more than DES; the odds of choosing one of those strong keys at random would still be about 1 in 1000. Personally, I think the Skipjack keyspace is flat, but the fact that no one has ever said this publicly is worrisome.

Skipjack is patented, but the patent is being withheld from distribution by a patent secrecy agreement [1122]. The patent will only be issued when and if the Skipjack algorithm is successfully reverse-engineered. This gives the government the best of both worlds: the protection of a patent and the confidentiality of a trade secret.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 14
Still Other Block Ciphers

14.1 GOST

GOST is a block algorithm from the former Soviet Union [655,1393]. "GOST" is an acronym for "Gosudarstvennyi Standard," or Government Standard, sort of similar to a FIPS, except that it can (and does) refer to just about any kind of standard. (Actually, the full name is Gosudarstvennyi Standard Soyuza SSR, or Government Standard of the Union of Soviet Socialist Republics.) This standard is number 28147-89. The Government Committee for Standards of the USSR authorized the standard, whoever they were.

I don't know whether GOST 28147-89 was used for classified traffic or just for civilian encryption. A remark at its beginning states that the algorithm "satisfies all cryptographic requirements and not limits the grade of information to be protected." I have heard claims that it was initially used for very high-grade communications, including classified military communications, but I have no confirmation.

Description of GOST

GOST is a 64-bit block algorithm with a 256-bit key. GOST also has some additional key material that will be discussed later. The algorithm iterates a simple encryption algorithm for 32 rounds.

To encrypt, first break the text up into a left half, L. and a right half, R. The subkey for round i is Ki. A round, i, of GOST is:

Li = Ri - 1
Ri = Li -1 ⊕ f(Ri - 1, Ki)

Figure 14.1 is a single round of GOST. Function f is straightforward. First, the right half and the ith subkey are added modulo 232. The result is broken into eight 4-bit chunks, and each chunk becomes the input to a different S-box. There are eight different S-boxes in GOST; the first 4 bits go into the first S-box, the second 4 bits go into the second S-box, and so on. Each S-box is a permutation of the numbers 0 through 15. For example, an S-box might be:

Figure 14.1 One round of GOST.

7, 10, 2, 4, 15, 9, 0, 3, 6, 12, 5, 13, 1, 8, 11

In this case, if the input to the S-box is 0, the output is 7. If the input is 1, the output is 10, and so on. All eight S-boxes are different; these are considered additional key material. The S-boxes are to be kept secret.

The outputs of the eight S-boxes are recombined into a 32-bit word, then the entire word undergoes an 11-bit left circular shift. Finally, the result XORed to the left half to become the new right half, and the right half becomes the new left half. Do this 32 times and you're done.

The subkeys are generated simply. The 256-bit key is divided into eight 32-bit blocks: k1, k2,..., k8. Each round uses a different subkey, as shown in Table 14.1. Decryption is the same as encryption with the order of the kis reversed.

The GOST standard does not discuss how to generate the S-boxes, only that they are somehow supplied [655]. This has led to speculation that some Soviet organization would supply good S-boxes to those organizations it liked and bad S-boxes to those organizations it wished to eavesdrop on. This may very well be true, but further conversations with a GOST chip manufacturer within Russia offered another alternative. He generated the S-box permutations himself, using a random-number generator.

Table 14.1 Use of GOST Subkeys in Different Rounds
Round: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16
Subkey: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8
Round: | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32
Subkey: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1

More recently, a set of S-boxes used in an application for the Central Bank of the Russian Federation surfaced. These S-boxes are also used in the GOSTone-way hash function (see section 18.11) [657].They are listed in Table 14.2.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Cryptanalysis of GOST

These are the major differences between DES and GOST.

— DES has a complicated procedure for generating the subkeys from the keys. GOST has a very simple procedure.
— DES has a 56-bit key; GOST has a 256-bit key. If you add in the secret S-box permutations, GOST has a total of about 610 bits of secret information.
Table 14.2 GOST S-Boxes
 | S-box 1: |  |  |  |  |  |  |  |  |  |  |  |  |  |
4 | 10 | 9 | 2 | 13 | 8 | 0 | 14 | 6 | 11 | 1 | 12 | 7 | 15 | 5 | 3
 | S-box 2: |  |  |  |  |  |  |  |  |  |  |  |  |  |
14 | 11 | 4 | 12 | 6 | 13 | 15 | 10 | 2 | 3 | 8 | 1 | 0 | 7 | 5 | 9
 | S-box 3: |  |  |  |  |  |  |  |  |  |  |  |  |  |
5 | 8 | 1 | 13 | 10 | 3 | 4 | 2 | 14 | 15 | 12 | 7 | 6 | 0 | 9 | 11
 | S-box 4: |  |  |  |  |  |  |  |  |  |  |  |  |  |
7 | 13 | 10 | 1 | 0 | 8 | 9 | 15 | 14 | 4 | 6 | 12 | 11 | 2 | 5 | 3
 | S-box 5: |  |  |  |  |  |  |  |  |  |  |  |  |  |
6 | 12 | 7 | 1 | 5 | 15 | 13 | 8 | 4 | 10 | 9 | 14 | 0 | 3 | 11 | 2
 | S-box 6: |  |  |  |  |  |  |  |  |  |  |  |  |  |
4 | 11 | 10 | 0 | 7 | 2 | 1 | 13 | 3 | 6 | 8 | 5 | 9 | 12 | 15 | 14
 | S-box 7: |  |  |  |  |  |  |  |  |  |  |  |  |  |
13 | 11 | 4 | 1 | 3 | 15 | 5 | 9 | 0 | 10 | 14 | 7 | 6 | 8 | 2 | 12
 | S-box 8: |  |  |  |  |  |  |  |  |  |  |  |  |  |
1 | 15 | 13 | 0 | 5 | 7 | 10 | 4 | 9 | 2 | 3 | 14 | 6 | 11 | 8 | 12


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

— The S-boxes in DES have 6-bit inputs and 4-bit outputs; the S-boxes in GOST have 4-bit inputs and outputs. Both algorithms have eight S-boxes, but an S-box in GOST is one-fourth the size of an S-box in DES.
— DES has an irregular permutation, called a P-box; GOST uses an 11-bit left circular shift.
— DES has 16 rounds; GOST has 32 rounds.

If there is no better way to break GOST other than brute force, it is a very secure algorithm. GOST has a 256-bit key—longer if you count the secret S-boxes. Against differential and linear cryptanalysis, GOST is probably stronger than DES. Although the random S-boxes in GOST are probably weaker than the fixed S-boxes in DES, their secrecy adds to GOST's resistance against differential and linear attacks. Also, both of these attacks depend on the number of rounds: the more rounds, the more difficult the attack. GOST has twice as many rounds as DES; this alone probably makes both differential and linear cryptanalysis infeasible.

The other parts of GOST are either on par or worse than DES. GOST doesn't have the same expansion permutation that DES has. Deleting this permutation from DES weakens it by reducing the avalanche effect; it is reasonable to believe that GOST is weaker for not having it. GOST's use of addition instead is no less secure than DES's XOR.

The greatest difference between them seems to be GOST's cyclic shift instead of a permutation. The DES permutation increases the avalanche effect. In GOST a change in one input bit affects one S-box in one round, which then affects two S-boxes in the next round, three the round after that, and so on. GOST requires 8 rounds before a single change in an input affects every output bit; DES only requires 5 rounds. This is certainly a weakness. But remember: GOST has 32 rounds to DES's 16.

GOST's designers tried to achieve a balance between efficiency and security. They modified DES's basic design to create an algorithm that is better suited for software implementation. They seem to have been less sure of their algorithm's security, and have tried to compensate by making the key length very large, keeping the S-boxes secret, and doubling the number of iterations. Whether their efforts have resulted in an algorithm more secure than DES remains to be seen.

14.2 CAST

CAST was designed in Canada by Carlisle Adams and Stafford Tavares [10,7]. They claim that the name refers to their design procedure and should conjure up images of randomness, but note the authors' initials. The example CAST algorithm uses a 64-bit block size and a 64-bit key.

The structure of CAST should be familiar. The algorithm uses six S-boxes with an 8-bit input and a 32-bit output. Construction of these S-boxes is implementation-dependent and complicated; see the references for details.

To encrypt, first divide the plaintext block into a left half and a right half. The algorithm has 8 rounds. In each round the right half is combined with some key material using function f and then XORed with the left half to form the new right half. The original right half (before the round) becomes the new left half. After 8 rounds (don't switch the left and right halves after the eighth round), the two halves are concatenated to form the ciphertext.

Function f is simple:

(1) Divide the 32-bit input into four 8-bit quarters: a, b, c, d.
(2) Divide the 16-bit subkey into two 8-bit halves: e, f.
(3) Process a through S-box 1, b through S-box 2, c through S-box 3, d through S-box 4, e through S-box 5, and f through S-box 6.
(4) XOR the six S-box outputs together to get the final 32-bit output.

Alternatively, the 32-bit input can be XORed with 32 bits of key, divided into four 8-bit quarters, processed through the S-boxes, and then XORed together [7]. N rounds of this appears to be as secure as N + 2 rounds of the other option.

The 16-bit subkey for each round is easily calculated from the 64-bit key. If k1, k2,..., k8 are the 8 bytes of the key, then the subkeys for each round are:

Round 1: | k1, k2
Round 2: | k3, k4
Round 3: | k5, k6
Round 4: | k7, k8
Round 5: | k4, k3
Round 6: | k2, k1
Round 7: | k8, k7
Round 8: | k6, k5

The strength of this algorithm lies in its S-boxes. CAST does not have fixed S-boxes; new ones are constructed for each application. Design criteria are in [10]; bent functions are the S-box columns, selected for a number of desirable S-box properties (see Section 14.10). Once a set of S-boxes has been constructed for a given implementation of CAST, they are fixed for all time. The S-boxes are implementation-dependent, but not key-dependent.

It was shown in [10] that CAST is resistant to differential cryptanalysis and in [728] that CAST is resistant to linear cryptanalysis. There is no known way to break CAST other than brute force.

Northern Telecom is using CAST in their Entrust security software package for Macintoshes, PCs, and UNIX workstations. The particular S-boxes they chose are not public. The Canadian government is evaluating CAST as a new encryption standard. CAST is patent-pending.

14.3 Blowfish

Blowfish is an algorithm of my own design, intended for implementation on large microprocessors [1388,1389]. The algorithm is unpatented, and the C code in the back of this book is in the public domain. I designed Blowfish to meet the following design criteria.

1. Fast. Blowfish encrypts data on 32-bit microprocessors at a rate of 26 clock cycles per byte.
2. Compact. Blowfish can run in less than 5K of memory.
3. Simple. Blowfish uses only simple operations: addition, XORs, and table lookups on 32-bit operands. Its design is easy to analyze which makes it resistant to implementation errors [1391].
4. Variably Secure. Blowfish's key length is variable and can be as long as 448 bits.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Blowfish is optimized for applications where the key does not change often, like a communications link or an automatic file encryptor. It is significantly faster than DES when implemented on 32-bit microprocessors with large data caches, such as the Pentium and the PowerPC. Blowfish is not suitable for applications, such as packet switching, with frequent key changes, or as a one-way hash function. Its large memory requirement makes it infeasible for smart card applications.

Description of Blowfish

Blowfish is a 64-bit block cipher with a variable-length key. The algorithm consists of two parts: key expansion and data encryption. Key expansion converts a key of up to 448 bits into several subkey arrays totaling 4168 bytes.

Data encryption consists of a simple function iterated 16 times. Each round consists of a key-dependent permutation, and a key- and data-dependent substitution. All operations are additions and XORs on 32-bit words. The only additional operations are four indexed array data lookups per round.

Blowfish uses a large number of subkeys. These keys must be precomputed before any data encryption or decryption.

The P-array consists of 18 32-bit subkeys:

P1, P2,..., P18

Four 32-bit S-boxes have 256 entries each:

S1,0, S1,1,..., S1,255
S2,0, S2,1,..., S2,255
S3,0, S3,1,..., S3,255
S4,0, S4,1,..., S4,255

The exact method used to calculate these subkeys will be described later in this section.

Figure 14.2 Blowfish.

Blowfish is a Feistel network (see Section 14.10) consisting of 16 rounds. The input is a 64-bit data element, x. To encrypt:

Divide x into two 32-bit halves: xL, xR
For i = 1 to 16:
xL = xL ⊕ Pi
xR = F(xL) ⊕ xR
Swap xL and xR
Swap xL and xR (Undo the last swap.)
xR = xR ⊕ P17
xL = xL ⊕ P18
Recombine xL and xR

Figure 14.3 Function F.

Function F is as follows (see Figure 14.3):

Divide xL into four eight-bit quarters:
a, b, c, and d F(xL) = ((S1,a + S2,b mod 232) ⊕ S3,c) + S4,d mod 232

Decryption is exactly the same as encryption, except that P1, P2,..., P18 are used in the reverse order.

Implementations of Blowfish that require the fastest speeds should unroll the loop and ensure that all subkeys are stored in cache. See [568] for details.

The subkeys are calculated using the Blowfish algorithm. The exact method follows.

(1) Initialize first the P-array and then the four S-boxes, in order, with a fixed string. This string consists of the hexadecimal digits of p.
(2) XOR P1 with the first 32 bits of the key, XOR P2 with the second 32-bits of the key, and so on for all bits of the key (up to P18). Repeatedly cycle through the key bits until the entire P-array has been XORed with key bits.
(3) Encrypt the all-zero string with the Blowfish algorithm, using the subkeys described in steps (1) and (2).
(4) Replace P1 and P2 with the output of step (3).
(5) Encrypt the output of step (3) using the Blowfish algorithm with the modified subkeys.
(6) Replace P3 and P4 with the output of step (5).
(7) Continue the process, replacing all elements of the P-array, and then all four S-boxes in order, with the output of the continuously changing Blowfish algorithm.

In total, 521 iterations are required to generate all required subkeys. Applications can store the subkeys—there's no need to execute this derivation process multiple times.

Security of Blowfish

Serge Vaudenay examined Blowfish with known S-boxes and r rounds; a differential attack can recover the P-array with 28r + 1 chosen plaintexts [1568]. For certain weak keys that generate bad S-boxes (the odds of getting them randomly are 1 in 214), the same attack requires only 24r + 1 chosen plaintexts to recover the P-array. With unknown S-boxes this attack can detect whether a weak key is being used, but cannot determine what it is (neither the S-boxes nor the P-array). This attack only works against reduced-round variants; it is completely ineffective against 16-round Blowfish.

Of course, the discovery of weak keys is significant, even though they seem impossible to exploit. A weak key is one in which two entries for a given S-box are identical. There is no way to check for weak keys before doing the key expansion. If you are worried, you have to do the key expansion and check for identical S-box entries. I don't think this is necessary, though.

I know of no successful cryptanalysis against Blowfish. To be safe, do not implement Blowfish with a reduced number of rounds.

Kent Marsh Ltd. has incorporated Blowfish in their FolderBolt security product for Microsoft Windows and Macintosh. It is also part of Nautilus and PGPfone.

14.4 SAFER

SAFER K-64 stands for Secure And Fast Encryption Routine with a Key of 64 bits [1009]. James Massey produced this nonproprietary algorithm for Cylink Corp. and it is incorporated into some of their products. The government of Singapore is planning to use this algorithm—with a 128-bit key [1010]—for a wide variety of applications. There are no patent, copyright, or other restrictions on its use.

The algorithm has a block and key size of 64 bits. It is not a Feistel network like DES (see Section 14.10), but an iterated block cipher: The same function is applied for some number of rounds. Each round uses two 64-bit subkeys, and the algorithm only uses operations on bytes.

Description of SAFER K-64

The plaintext block is divided into eight byte-length sub-blocks: B1, B2,..., B7, B8. Then the sub-blocks go through r rounds. Finally, an output transformation is applied to the sub-blocks. Each round uses two subkeys: K2i - 1 and K2i.

Figure 14.4 shows one round of SAFER K-64. First, sub-blocks are either XORed or added with bytes of subkey K2i - 1. Then, the eight sub-blocks are subjected to one of two nonlinear transformations:

y = 45x mod 257. (If x = 128, then y = 0.)
y = log45 x. (If x = 0, then y = 128.)


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

These are operations in the finite field GF(257), and 45 is a primitive element in that field. In practical implementations of SAFER K-64, it is quicker to implement this in a lookup table than to calculate new results all the time.

Then, sub-blocks are either XORed or added with bytes of subkey K2r. The results of this operation are fed through three layers of linear operations designed to increase the avalanche effect. Each operation is called a Pseudo-Hadamard Transform (PHT). If the inputs to a PHT are a1 and a2, then the outputs are:

b1 = (2a1 + a2) mod 256
b2 = (a1 + a2) mod 256

After r rounds, there is a final output transformation. This is the same as the first step of each round. B1, B4, B5, and B8 are XORed with the corresponding bytes of the last subkey, and B2, B3, B6, and B7 are added to the corresponding bytes of the last subkey. The result is the ciphertext.

Figure 14.4 One round of SAFER.

Decryption is the reverse process: the output transformation (with subtraction instead of addition), then r reverse rounds. The Inverse PHT (IPHT) is:

a1 = (b1 – b2) mod 256
a2 = (–b1 + 2b2) mod 256

Massey recommends 6 rounds, but you can increase that if you want greater security.

Generating subkeys is easy. The first subkey, K1, is simply the user key. Subsequent subkeys are generated by the following procedure:

Ki+1 = (K1 <<< 3i) + ci

The symbol "<<<" is a left circular shift or a left rotation. The rotation is byte by byte, and ci is a round constant. If cij is the jth byte of the ith round constant, then you can calculate all of the round constants by the formula

cij = 4545^((9i + j) mod 256) mod 257 mod 257

Generally, these values are stored in a table.

SAFER K-128

This alternate key schedule was developed by the Ministry of Home Affairs in Singapore, and then incorporated into SAFER by Massey [1010]. It uses two keys, Ka and Kb, each 64-bits long. The trick is to generate two subkey sequences in parallel, and then alternate subkeys from each sequence. This means that if you choose Ka = Kb, then the 128-bit key is compatible with the 64-bit key Ka.

Security of SAFER K-64

Massey showed that SAFER K-64 is immune to differential cryptanalysis after 8 rounds and is adequately secure against the attack after 6 rounds. After only 3 rounds linear cryptanalysis is ineffective against this algorithm [1010].

Knudsen found a weakness in the key schedule: For virtually every key, there exists at least one (and sometimes as many as nine) other key that encrypts some different plaintext to identical ciphertexts [862]. The number of different plaintexts that encrypt to identical ciphertexts after 6 rounds is anywhere from 222 to 228. While this attack may not impact SAFER's security when used as an encryption algorithm, it greatly reduces its security when used as a one-way hash function. In any case, Knudsen recommends at least 8 rounds.

SAFER was designed for Cylink, and Cylink is tainted by the NSA [80]. I recommend years of intense cryptanalysis before using SAFER in any form.

14.5 3-Way

3-Way is a block cipher designed by Joan Daemen [402,410]. It has a 96-bit block length and key length, and is designed to be very efficient in hardware.

3-Way is not a Feistel network, but it is an iterated block cipher. 3-Way can have n rounds; Daemen recommends 11.

Description of 3-Way

The algorithm is simple to describe. To encrypt a plaintext block, x:

For i = 0 to n – 1
x = x XOR Ki
x = theta (x)
x = pi – 1 (x)
x = gamma (x)
x = pi – 2 (x)
x = x ⊕ Kn
x = theta (x)

The functions are:

— theta(x) is a linear substitution function—basically a bunch of circular shifts and XORs.
— pi–1(x) and pi–2(x) are simple permutations.
— gamma(x) is a nonlinear substitution function. This is the step that gives 3-Way its name; it is the parallel execution of the substitution step on 3-bit blocks of the input.

Decryption is similar to encryption, except that the bits of the input have to be reversed and the bits of the output have to be reversed. Code to implement 3-Way can be found in the back of this book.

So far, there has been no successful cryptanalysis of 3-Way. The algorithm is unpatented.

14.6 Crab

This algorithm was developed by Burt Kaliski and Matt Robshaw of RSA Laboratories [810]. The idea behind Crab is to use techniques from one-way hash functions to make a fast encryption algorithm. Hence, Crab is very similar to MD5, and this section assumes you are familiar with Section 18.5.

Crab has a very large block: 1024 bytes. Since Crab is presented more as a research contribution than a real algorithm, no definitive key-generation routines are presented. The authors suggest a method that could turn an 80-bit key into three requisite subkeys, although the algorithm could easily accept variable-length keys.

Crab uses two sets of large subkeys:

A permutation of the numbers 0 through 255: P0, P1, P2,..., P255.
A 2048-entry array of 32-bit numbers: S0, S1, S2,..., S2047.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

These subkeys must all be calculated before encryption or decryption.

To encrypt a 1024-byte block X:

(1) Divide X into 256 32-bit sub-blocks: X0, X1, X2,..., X255.
(2) Permute the sub-blocks of X according to P.
(3) For r = 0 to 3
For g = 0 to 63
A = X(4g)<<<2r
B = X(4g+1)<<<2r
C = X(4g+2)<<<2r
D = X(4g+3)<<<2r
For step s = 0 to 7
A = A ⊕ (B + fr(B,C,D) + S512r+8 g+s)
TEMP = D
D = C
C = B
B = A <<< 5
A = TEMP
X(4g)<<<2r = A
X(4g+1)<<<2r = B
X(4g+2)<<<2r = C
X(4g+3)<<<2r = D
(4) Recombine X0, X1, X2,..., X255 to form the ciphertext.

The functions fr(B,C,D) are similar to those used in MD5:

f0(B,C,D) = (B Λ C) ν ((¬ B) Λ D)
f1(B,C,D) = (B Λ D) ν (C Λ (¬ D))
f2(B,C,D) = B ⊕ C ⊕ D
f3(B,C,D) = C ⊕ (B ν (¬ D))

Decryption is the reverse process.

Generating the subkeys is a large task. Here is how the permutation array, P, could be generated from an 80-bit key, K.

(1) Initialize K0, K1, K2,..., K9 with the 10 bytes of K.
(2) For i = 10 to 255
Ki = Ki - 2 ⊕ Ki - 6 ⊕ Ki - 7 ⊕ Ki - 10
(3) For i = 0 to 255, Pi = i
(4) m = 0
(5) For j = 0 to 1
For i = 256 to 1 step -1
m = (K256 - i + K257 - i) mod i
K257 - i = K257 - i <<< 3
Swap Pi and Pi - 1

The S-array of 2048 32-bit words could be generated in a similar manner, either from the same 80-bit key or from another key. The authors caution that these details should "be viewed as motivational; there may very well be alternative schemes which are both more efficient and offer improved security" [810].

Crab was proposed as a testbed of new ideas and not as a working algorithm. It uses many of the same techniques as MD5. Biham has argued that a very large block size makes an algorithm easier to cryptanalyze [160]. On the other hand, Crab may make efficient use of a very large key. In such a case, "easier to cryptanalyze" might not mean much.

14.7 SXAL8/MBAL

This is a 64-bit block algorithm from Japan [769]. SXAL8 is the basic algorithm; MBAL is an expanded version with a variable block length. Since MBAL does some clever things internally, the authors claim that they can get adequate security with only a few rounds. With a block length of 1024 bytes, MBAL is about 70 times faster than DES. Unfortunately, [1174] shows that MBAL is susceptible to differential cryptanalysis, and [865] shows that it is susceptible to linear cryptanalysis.

14.8 RC5

RC5 is a block cipher with a variety of parameters: block size, key size, and number of rounds. It was invented by Ron Rivest and analyzed by RSA Laboratories [1324,1325].

There are three operations: XOR, addition, and rotations. Rotations are constant-time operations on most processors and variable rotations are a nonlinear function. These rotations, which depend on both the key and the data, are the interesting operation.

RC5 has a variable-length block, but this example will focus on a 64-bit data block. Encryption uses 2r + 2 key-dependent 32-bit words—S0, S1, S2,..., S2r + 1—where r is the number of rounds. We'll generate those words later. To encrypt, first divide the plaintext block into two 32-bit words: A and B. (RC5 assumes a little-endian convention for packing bytes into words: The first byte goes into the low-order bit positions of register A, etc.) Then:

A = A + S0
B = B + S1
For i = 1 to r:
A = ((A ⊕ B) <<< B) + S2i
B = ((B ⊕ A) <<< A) + S2i + 1

The output is in the registers A and B.

Decryption is just as easy. Divide the plaintext block into two words, A and B, and then:

For i = r down to 1:
B = ((B – S2i + 1) >>> A) ⊕ A
A = ((A – S2i) >>> B) ⊕ B
B = B – S1
A = A – S0

The symbol ">>>" is a right circular shift. Of course, all addition and subtraction are mod 232.

Creating the array of keys is more complicated, but also straightforward. First, copy the bytes of the key into an array, L, of c 32-bit words, padding the final word with zeros if necessary. Then, initialize an array, S, using a linear congruential generator mod 232:

S0 = P
for i = 1 to 2(r + 1) – 1:
Si = (Si - 1 + Q) mod 232

P = 0xb7e15163 and Q = 0x9e3779b9; these constants are based on the binary representation of e and phi.

Finally, mix L into S:

i = j = 0
A = B = 0
do 3n times (where n is the maximum of 2(r + 1) and c):
A = Si = (Si + A + B) <<< 3
B = Lj = (Lj + A + B) <<< (A + B)
i = (i + 1) mod 2(r + 1)
j = (j + 1) mod c


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

RC5 is actually a family of algorithms. We just defined RC5 with a 32-bit word size and 64-bit block; there's no reason why the same algorithm can't have a 64-bit word size and 128-bit block size. For w = 64, P and Q are 0xb7e151628aed2a6b and 0x9e3779b97f4a7c15, respectively. Rivest designates particular implementations of RC5 as RC5-w/r/b, where w is the word size, r is the number of rounds, and b is the length of the key in bytes.

RC5 is new, but RSA Laboratories has spent considerable time analyzing it with a 64-bit block. After 5 rounds, the statistics look very good. After 8 rounds, every plaintext bit affects at least one rotation. There is a differential attack that requires 224 chosen plaintexts for 5 rounds, 245 for 10 rounds, 253 for 12 rounds, and 268 for 15 rounds. Of course, there are only 264 possible chosen plaintexts, so this attack won't work for 15 or more rounds. Linear cryptanalysis estimates indicate that it is secure after 6 rounds. Rivest recommends at least 12 rounds, and possibly 16 [1325]. This number may change.

RSADSI is in the process of patenting RC5, and the name is trademarked. The company claims that license fees will be very small, but you'd better check with them.

14.9 Other Block Algorithms

There is an algorithm called CRYPTO-MECCANO in the literature [301]; it is insecure. Four Japanese cryptographers presented an algorithm based on chaotic maps at Eurocrypt '91 [687, 688]; Biham cryptanalyzed the algorithm at the same conference [157]. Another algorithm relies on subsets of a particular set of random codes [693]. There are several algorithms based on the theory of error-correcting codes: a variant of the McEliece algorithm (see Section 19.7) [786,1290], the Rao-Nam algorithm [1292,733,1504,1291,1056,1057,1058,1293], variants of the Rao-Nam algorithm [464,749,1503], and the Li-Wang algorithm [964,1561]—they are all insecure. CALC is insecure [1109]. An algorithm called TEA, for Tiny Encryption Algorithm, is too new to comment on [1592]. Vino is another algorithm [503]. MacGuffin, a block algorithm by Matt Blaze and me, is also insecure [189]; it was broken at the same conference it was proposed. BaseKing, similar in design philosophy as 3-way but with a 192-bit block [402], is too new to comment on.

There are many more block algorithms outside the cryptology community. Some are used by various government and military organizations. I have no information about any of those. There are also dozens of proprietary commercial algorithms. Some might be good; most are probably not. If companies do not feel that their interests are served by making their algorithms public, it is best to assume they're right and avoid the algorithm.

14.10 Theory of Block Cipher Design

In Section 11.1, I described Shannon's principles of confusion and diffusion. Fifty years after these principles were first written, they remain the cornerstone of good block cipher design.

Confusion serves to hide any relationship between the plaintext, the ciphertext, and the key. Remember how linear and differential cryptanalysis can exploit even a slight relationship between these three things? Good confusion makes the relationship statistics so complicated that even these powerful cryptanalytic tools won't work.

Diffusion spreads the influence of individual plaintext or key bits over as much of the ciphertext as possible. This also hides statistical relationships and makes cryptanalysis more difficult.

Confusion alone is enough for security. An algorithm consisting of a single key-dependent lookup table of 64 bits of plaintext to 64 bits of ciphertext would be plenty strong. The problem is that large lookup tables require lots of memory to implement: 1020 bytes of memory for the table just mentioned. The whole point of block cipher design is to create something that looks like a large lookup table, but with much smaller memory requirements.

The trick is to repeatedly mix confusion (with much smaller tables) and diffusion in a single cipher in different combinations. This is called a product cipher. Sometimes a block cipher that incorporates layers of substitution and permutation is called a substitution-permutation network, or even an SP network.

Look back at function f of DES. The expansion permutation and P-box perform diffusion; the S-boxes perform confusion. The expansion permutation and P-box are linear; the S-boxes are nonlinear. Each operation is pretty simple on its own; together they work pretty well.

DES also illustrates a few more principles of block cipher design. The first is the idea of an iterated block cipher. This simply means taking a simple round function and iterating it multiple times. Two-round DES isn't very strong; it takes 5 rounds before all of the output bits are dependent on all of the input bits and all of the key bits [1078,1080]. Sixteen-round DES is strong; 32-round DES is even stronger.

Feistel Networks

Most block algorithms are Feistel networks. This idea dates from the early 1970s [552,553]. Take a block of length n and divide it into two halves of length n/2: L and R. Of course, n must be even. You can define an iterated block cipher where the output of the ith round is determined from the output of the previous round:

Li = Ri - 1
Ri = Li - 1 ⊕ f(Ri - 1,Ki)

Ki is the subkey used in the ith round and f is an arbitrary round function.

You've seen this concept in DES, Lucifer, FEAL, Khufu, Khafre, LOKI, GOST, CAST, Blowfish, and others. Why is it such a big deal? The function is guaranteed to be reversible. Because XOR is used to combine the left half with the output of the round function, it is necessarily true that

Li - 1 ⊕ f(Ri - 1,Ki) ⊕ f(Ri - 1,Ki) = Li - 1

A cipher that uses this construction is guaranteed to be invertible as long as the inputs to f in each round can be reconstructed. It doesn't matter what f is; f need not be invertible. We can design f to be as complicated as we please, and we don't have to implement two different algorithms—one for encryption and another for decryption. The structure of a Feistel network takes care of all this automatically.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Simple Relations

DES has the property that if EK(P) = C, then EK'(P') = C', where P', C', and K' are the bit-wise complements of P, C, and K. This property reduces the complexity of a brute-force attack by a factor of two. LOKI has complementation properties that reduce the complexity of a brute-force attack by a factor of 256.

A simple relation can be defined as [857]:

If EK(P) = C, then Ef(K) (g(P,K)) = h(C,K)

where f, g, and h are simple functions. By simple I mean that they are easy to compute, much easier than an iteration of the block cipher. In DES, f is the bit-wise complement of K, g is the bit-wise complement of P, and h is the bit-wise complement of C. This is a result of XoRing the key into part of the text.

In a good block cipher, there are no simple relations. Methods for finding some of these weaknesses are in [917].

Group Structure

When discussing an algorithm, the question of whether it is a group arises. The elements of the group are the ciphertext blocks with each possible key, and the group operation is composition. Looking at an algorithmos group structure is an attempt to get a handle on just how much extra scrambling happens under multiple encryption.

The useful question is, however, not whether an algorithm is actually a group, but just how close to a group it is. If it were only lacking one element, it wouldnot be a group; but double encryption would be—statistically speaking—a waste of time. The work on DES showed that DES is very far away from being a group. There are still some interesting questions about the semigroup that DES encryption generates. Does it contain the identity: That is, does it even generate a group? To put it another way, does some combination of encryption (not decryption) operations eventually generate the identity function? If so, how long is the shortest such combination?

The goal is to estimate the size of the keyspace for a theoretical brute-force attack, and the result is a greatest lower bound on the keyspace entropy.

Weak Keys

In a good block cipher, all keys are equally strong. Algorithms with a small number of weak keys, like DES, are generally no problem. The odds of picking one at random are very small, and it's easy to test for and discard them. However, these weak keys can sometimes be exploited if the block cipher is used as a one-way hash function (see Section 18.11).

Strength against Differential and Linear Cryptanalysis

The study of differential and linear cryptanalysis has shed significant light on the theory of good block cipher design. The inventors of IDEA introduced the concept of differentials, a generalization of the basic idea of characteristics [931]. They argued that block ciphers can be designed to resist this attack; IDEA is the result of that work [931]. This concept was further formalized in [1181,1182], when Kaisa Nyberg and Lars Knudsen showed how to make block ciphers provably secure against differential cryptanalysis. This theory has extensions to higher-order differentials [702,161,927,858,860] and partial differentials [860]. Higher-order differentials seem to apply only to ciphers with a small number of rounds, but partial differentials combine nicely with differentials.

Linear cryptanalysis is newer, and is still being improved. Notions of key ranking [1019] and multiple approximations [811,812] have been defined. other work that extends the idea of linear cryptanalysis can be found in [1270]; [938] tries to combine linear and differential cryptanalysis into one attack. It is unclear what design techniques will protect against these sorts of attacks.

Knudsen has made some progress, considering some necessary (but not perhaps sufficient) criteria for what he calls practically secure Feistel networks: ciphers that resist both linear and differential cryptanalysis [857]. Nyberg introduced in linear cryptanalysis an analogy to the concept of differentials from differential cryptanalysis [1180].

Interestingly enough, there seems to be a duality between differential and linear cryptanalysis. This duality becomes apparent both in the design of techniques to construct good differential characteristics and linear approximations [164,1018], and also in the design criteria for making algorithms that are secure against both attacks [307]. Exactly where this line of research will lead is still unknown. As a start, Daemen has developed an algorithm-design strategy based on linear and differential cryptanalysis [402].

S-Box Design

The strength of various Feistel networks—and specifically their resistance to differential and linear cryptanalysis—is tied directly to their S-boxes. This has prompted a spate of research on what constitutes a good S-box.

An S-box is simply a substitution: a mapping of m-bit inputs to n-bit outputs. Previously I talked about one large lookup table of 64-bit inputs to 64-bit outputs; that would be a 64*64-bit S-box. An S-box with an m-bit input and an n-bit output is called a m*n-bit S-box. S-boxes are generally the only nonlinear step in an algorithm; they are what give a block cipher its security. In general, the bigger they are, the better.

DES has eight different 6*4-bit S-boxes. Khufu and Khafre have a single 8*32-bit S-box, LoKI has a 12*8-bit S-box, and both Blowfish and CAST have 8*32-bit S-boxes. In IDEA the modular multiplication step is effectively the S-box; it is a 16*16-bit S-box. The larger this S-box, the harder it is to find useful statistics to attack using either differential or linear cryptanalysis [653,729,1626]. Also, while random S-boxes are usually not optimal to protect against differential and linear attacks, it is easier to find strong S-boxes if the S-boxes are larger. Most random S-boxes are nonlinear, nondegenerate, and have strong resistance to linear cryptanalysis—and the fraction that does not goes down rapidly as the number of input bits decreases [1185,1186,1187].

The size of m is more important than the size of n. Increasing the size of n reduces the effectiveness of differential cryptanalysis, but greatly increases the effectiveness of linear cryptanalysis. In fact, if n ≤ 2m – m, then there is definitely a linear relation of the input and output bits of the S-box. And if n ≤ 2m, then there is a linear relation of only the output bits [164].

Much of this work involves the study of Boolean functions [94,1098,1262,1408]. In order to be secure, the Boolean functions used in S-boxes must satisfy specific conditions. They should not be linear or affine, nor even close to linear or affine [9,1177,1178,1188]. There should be a balance of zeros and ones, and no correlations between different combinations of bits. The output bits should behave independently when any single input bit is complemented. These design criteria are also related to the study of bent functions: functions which can be shown to be optimally nonlinear. Although their definition is simple and natural, their study is very complicated [1344,1216,947,905,1176,1271,295,296,297,149,349,471,298].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

One property that seems very important is the avalanche effect: how many output bits of an S-box change when some subset of the input bits are changed. IT'S easy to impose conditions on Boolean functions so that they satisfy certain avalanche criteria, but constructing them is a harder task. The strict avalanche criteria (SAC) guarantees that exactly half of the output bits change when one input bit changes [1586]. See also [982,571,1262,399]. one paper attempts to look at all these criteria in terms of information leakage [1640].

A few years ago cryptographers proposed choosing S-boxes so that the difference distribution table for each S-box is uniform. This would provide immunity against differential cryptanalysis by smoothing out the differentials in any particular round [6,443,444,1177]. LoKI is an example of this design. However, this approach can sometimes aid in differential cryptanalysis [172]. Actually, a better approach is making sure that the maximum differential is as small as possible. Kwangjo Kim proposed five criteria for the construction of S-boxes [834], similar to the design criteria for the DES S-boxes.

Choosing good S-boxes is not an easy task; there are many competing ideas on how to do it. Four general approaches can be identified.

1. Choose randomly. It is clear that small random S-boxes are insecure, but large random S-boxes may be good enough. Random S-boxes with eight or more inputs are quite strong [1186,1187]. Twelve-bit S-boxes are better. Even more strength is added if the S-boxes are both random and key-dependent. IDEA uses both large and key-dependent S-boxes.
2. Choose and test. Some ciphers generate random S-boxes and then test them for the requisite properties. See [9,729] for examples of this approach.
3. Man-made. This technique uses little mathematics: S-boxes are generated using more intuitive techniques. Bart Preneel stated that "...theoretically interesting criteria are not sufficient [for choosing Boolean functions for S-boxes]..." and that "...ad hoc design criteria are required" [1262].
4. Math-made. Generate S-boxes according to mathematical principles so that they have proven security against differential and linear cryptanalysis, and good diffusive properties. See [1179] for an excellent example of this approach.

There has been some call for a combination of the "math-made" and "man-made" approaches [1334], but the real debate seems to be between randomly chosen S-boxes and S-boxes with certain properties. Certainly the latter approach has the advantage of being optimal against known attacks—linear and differential cryptanalysis—but it offers unknown protection against unknown attacks. The designers of DES knew about differential cryptanalysis, and its S-boxes were optimized against it. They did not seem to know about linear cryptanalysis, and the DES S-boxes are very weak against it [1018]. Randomly selected S-boxes in DES would be weaker against differential cryptanalysis and stronger against linear cryptanalysis.

On the other hand, random S-boxes may not be optimal against these attacks, but they can be made sufficiently large and therefore sufficiently resistant. Also, they are more likely to be sufficiently resistant against unknown attacks. The debate is still raging, but my personal feeling is that S-boxes should be as large as possible, random, and key-dependent.

Designing a Block Cipher

It is easy to design a block cipher. If you think of a 64-bit block cipher as a permutation of the 64-bit numbers, it is clear that almost all of those permutations are secure. What is difficult is to design a block cipher that is not only secure, but can also be easily described and simply implemented.

It's is easy to design a block cipher if you have sufficient memory for 48*32 S-boxes. It'S hard to design an insecure DES variant if you iterate it for 128 rounds. If the length of your key is 512 bits, you really donot care if there are key-complementation properties.

The real trick—and the reason that real-world block cipher design is very difficult—is to design a block cipher with the smallest possible key, the smallest possible memory requirement, and the fastest possible running time.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

14.11 Using one-Way Hash Functions

The simplest way to encrypt with a one-way function is to hash the previous ciphertext block concatenated with the key, then XoR the result with the current plaintext block:

Ci = Pi ⊕ H(K,Ci - 1)
Pi = Ci ⊕ H(K,Ci - 1)

Set the block length equal to the output of the one-way hash function. This, in effect uses the one-way function as a block cipher in CFB mode. A similar construction can use the one-way function in OFB mode:

Ci = Pi ⊕ Si; Si = H(K,Ci - 1)
Pi = Ci ⊕ Si; Si = H(K,Ci - 1)

The security of this scheme depends on the security of the one-way function.

Karn

This method, invented by Phil Karn and placed in the public domain, makes an invertible encryption algorithm out of certain one-way hash functions.

The algorithm operates on plaintext and ciphertext in 32-byte blocks. The key can be any length, although certain key lengths will be more efficient for certain one-way hash functions. For the one-way hash functions MD4 and MD5, 96-byte keys work best.

To encrypt, first split the plaintext into two 16-byte halves: P1 and Pr. Then, split the key into two 48-byte halves: K1 and Kr.

P = P1,Pr
K = K1,Kr

Append K1 to P1 and hash it with a one-way hash function, then XoR the result of the hash with Pr to produce Cr, the right half of the ciphertext. Then, append Kr to Cr and hash it with the one-way hash function. XoR the result with P1 to produce C1. Finally, append Cr to C1 to produce the ciphertext.

Cr = Pr ⊕ H(P1,K 1)
C1 = P1 ⊕ H(Cr,Kr)
C = C1,Cr

To decrypt, simply reverse the process. Append Kr to Cr, hash and XoR with C1 to produce P1. Append K1 to P1, hash and XoR with Cr to produce Pr.

P1 = C1 ⊕ H(Cr,Kr)
Pr = Cr ⊕ H(P1,K1)
P = P1,Pr

The overall structure of Karn is the same as many of the other block algorithms discussed in this section. It has only two rounds, because the complexity of the algorithm is embedded in the one-way hash function. And since the key is used only as the input to the hash function, it cannot be recovered even using a chosen-plaintext attack—assuming, of course, that the one-way hash function is secure.

Luby-Rackoff

Michael Luby and Charles Rackoff showed that Karn is not secure [992]. Consider two single-block messages: AB and AC. If a cryptanalyst knows both the plaintext and the ciphertext of the first message, and knows the first half of the plaintext of the second message, then he can easily compute the entire second message. This known-plaintext attack is useful only in certain circumstances, but it is a major security problem.

A three-round encryption algorithm avoids this problem [992,1643,1644]. It uses three different hash functions: H1, H2, and H3. Further work shows that H1 can equal H2, or that H2 can equal H3, but not both [1193]. Also, H1, H2, and H3 cannot be based on iterating the same basic function [1643]. Anyway, assuming that H(k,x) behaves like a pseudo-random function, here is a three-round version:

(1) Divide the key into two halves: K1 and Kr.
(2) Divide the plaintext block into two halves: L0 and R0.
(3) Append K1 to L0 and hash it. XoR the result of the hash with R0 to produce R1:
R1 = R0 ⊕ H(K1,L0)
(4) Append Kr to R1 and hash it. XOR the result of the hash with L0 to produce L1:
L1 = L0 ⊕ H(Kr,R1)
(5) Append K1 to L1 and hash it. XOR the result of the hash with R1 to produce R2:
R2 = R1 ⊕ H(K1,L1)
(6) Append L1 to R1 to generate the message.

Message Digest Cipher (MDC)

MDC, invented by Peter Gutmann [676], is a means of turning one-way hash functions into a block cipher that runs in CFB mode. The cipher runs almost as fast as the hash function and is at least as secure as the hash function. The rest of this section assumes you are familiar with Chapter 18.

Hash functions such as MD5 and SHA use a 512-bit text block to transform an input value (128 bits with MD5, and 160 bits with SHA) into an output value of equal size. This transformation is not reversible, but it is perfect for CFB mode: The same operation is used for both encryption and decryption.

Let's look at MDC with SHA. MDC has a 160-bit block size and a 512-bit key. The hash function is run "sideways," with the old hash state as the input plaintext block (160 bits) and the 512-bit hash input as a key (see Figure 14.5). Normally, when using the hash to simply hash some input, the 512-bit input to the hash is varied as each new 512-bit block is hashed. But in this case the 512-bit input becomes an unchanging key.

MDC can be used with any one-way hash function: MD4, MD5, Snefru, and others. It is unpatented. Anyone can use it at any time, in any way, royalty-free [676].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

However, I donot trust this construction. It is possible to attack the hash function in a way that hash functions are not designed to withstand. It is not important for hash functions to be able to resist a chosen-plaintext attack, where a cryptanalyst chooses several of those starting 160-bit values, has them "encrypted" by the same 512-bit "key," and uses this to learn some information about the 512-bit key used. Since the designers didnot have to worry about this, it seems like a bad idea to count on your cipher being able to resist this attack.

Security of Ciphers Based on one-Way Hash Functions

While these constructions can be secure, they depend on the choice of the underlying one-way hash function. A good one-way hash function doesnot necessarily make a secure encryption algorithm. Cryptographic requirements are different. For example, linear cryptanalysis is not a viable attack against one-way hash functions, but works against encryption algorithms. A one-way hash function such as SHA could have linear characteristics which, while not affecting its security as a one-way hash function, could make it insecure in an encryption algorithm such as MDC. I know of no cryptanalytic analysis of particular one-way hash functions as block ciphers; wait for such analysis before you trust any of them.

Figure 14.5 Message Digest Cipher (MDC).

14.12 Choosing a Block Algorithm

It's a tough decision. DES is almost certainly insecure against the major governments of the world unless you only encrypt very small chunks of data for a single key. IT'S probably all right against anyone else, but that is changing soon. Brute-force DES key search machines will quickly become economical for all sorts of organizations.

Biham's key-dependent S-boxes for DES should be secure for at least a few years against all but the most well-funded adversaries, and possibly even from them. If you need security that lasts decades or fear the cryptanalytic efforts of major governments, use triple-DES with three independent keys.

The other algorithms arenot worthless. I like Blowfish because it is fast and I wrote it. 3-WAY looks good, and GoST is probably okay. The problem with any recommendation is that the NSA almost certainly has an array of impressive cryptanalytic techniques that are still classified, and I donot know which algorithms they can break with them. Table 14.3 gives timing measurements for some algorithms. These are meant for comparison purposes only.

My favorite algorithm is IDEA. Its 128-bit key, combined with its resistance to any public means of cryptanalysis, gives me a warm, fuzzy feeling about the algorithm. The algorithm has been analyzed by a lot of different groups, and no serious results have been announced yet. Barring extraordinary cryptanalytic news tomorrow, I am betting on IDEA today.

Table 14.3 Encryption Speeds of Some Block Ciphers on a 33 MHz 486SX
Algorithm | Encryption Speed (Kilobytes/second) | Algorithm | Encryption Speed (Kilobytes/second)
Blowfish (12 rounds) | 182 | MDC (using MD4) | 186
Blowfish (16 rounds) | 135 | MDC (using MD5) | 135
Blowfish (20 rounds) | 110 | MDC (using SHA) | 23
DES | 35 | NewDES | 233
FEAL-8 | 300 | REDOC II | 1
FEAL-16 | 161 | REDOC III | 78
FEAL-32 | 91 | RC5-32/8 | 127
GOST | 53 | RC5-32/12 | 86
IDEA | 70 | RC5-32/16 | 65
Khufu (16 rounds) | 221 | RC5-32/20 | 52
Khufu (24 rounds) | 153 | SAFER (6 rounds) | 81
Khufu (32 rounds) | 115 | SAFER (8 rounds) | 61
Luby-Rackoff (using MD4) | 47 | SAFER (10 rounds) | 49
Luby-Rackoff (using MD5) | 34 | SAFER (12 rounds) | 41
Luby-Rackoff (using SHA) | 11 | 3-Way | 25
Lucifer | 52 | Triple-DES | 12


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 15
Combining Block Ciphers

There are many ways to combine block algorithms to get new algorithms. The impetus behind these schemes is to try to increase security without going through the trouble of designing a new algorithm. DES is a secure algorithm; it has been cryptanalyzed for a good 20 years and the most practical way to break it is still brute force. However, the key is too short. Wouldn't it be nice to use DES as a building block for another algorithm with a longer key? We'd have the best of both worlds: the assurance of two decades of cryptanalysis plus a long key.

Multiple encryption is one combination technique: using an algorithm to encrypt the same plaintext block multiple times with multiple keys. Cascading is like multiple encryption, but uses different algorithms. There are other techniques.

Encrypting a plaintext block twice with the same key, whether with the same algorithm or a different one, is not smart. For the same algorithm, it does not affect the complexity of a brute-force search. (Remember, you assume a cryptanalyst knows the algorithm including the number of encryptions used.) For different algorithms, it may or may not. If you are going to use any of the techniques in this chapter, make sure the multiple keys are different and independent.

15.1 Double Encryption

A naìve way of improving the security of a block algorithm is to encrypt a block twice with two different keys. First encrypt a block with the first key, then encrypt the resulting ciphertext with the second key. Decryption is the reverse process.

C = EK2(EK1(P))
P = DK1(DK2(C))

If the block algorithm is a group (see Section 11.3), then there is always a K3 such that

C = EK2(EK1(P)) = EK3(P)

If this is not the case, the resultant doubly-encrypted ciphertext block should be much harder to break using an exhaustive search. Instead of 2n attempts (where n is the bit length of the key), it would require 22n attempts. If the algorithm is a 64-bit algorithm, the doubly-encrypted ciphertext would require 2128 attempts to find the key.

This turns out not to be true for a known-plaintext attack. Merkle and Hellman [1075] developed a time-memory trade-off that could break this double-encryption scheme in 2n + 1 encryptions, not in 22n encryptions. (They showed this for DES, but the result can be generalized to any block algorithm.) The attack is called a meet-in-the-middle attack; it works by encrypting from one end, decrypting from the other, and matching the results in the middle.

In this attack, the cryptanalyst knows P1, C1, P2, and C2, such that

C1 = EK2(EK1(P1))
C2 = EK2(EK1(P2))

For each possible K, he computes EK(P1) and stores the result in memory. After collecting them all, he computes DK(C1) for each K and looks for the same result in memory. If he finds it, it is possible that the current key is K2 and the key in memory is K1. He tries encrypting P2 with K1 and K2; if he gets C2 he can be pretty sure (with a probability of success of 1 in 22m – 2n, where m is the block size) that he has both K1 and K2. If not, he keeps looking. The maximum number of encryption trials he will probably have to run is 2*2n, or 2n + 1. If the probability of error is too large, he can use a third ciphertext block to get a probability of success of 1 in 23m – 2n. There are still other optimizations [912].

This attack requires a lot of memory: 2n blocks. For a 56-bit algorithm, this translates to 256 64-bit blocks, or 1017 bytes. This is still considerably more memory storage than one could comfortably comprehend, but it's enough to convince the most paranoid of cryptographers that double encryption is not worth anything.

For a 128-bit key, the amount of memory required is an enormous 1039 bytes. If we assume that a way exists to store a bit of information on a single atom of aluminum, the memory device required to launch this attack would be a cube of solid aluminum over a kilometer on a side. And then you need some place to put it! The meet-in-the middle attack seems infeasible for keys this size.

Another double-encryption method, sometimes called Davies-Price, is a variant of CBC [435].

Ci = EK1(Pi ⊕ EK2(Ci - 1))
Pi = DK1(Ci) ⊕ EK2(Ci - 1)

They claim "no special virtue of this mode," but it seems to be vulnerable to the same meet-in-the-middle attacks as other double-encryption modes.

15.2 Triple Encryption

Triple Encryption with Two Keys

A better idea, proposed by Tuchman in [1551], operates on a block three times with two keys: with the first key, then with the second key, and finally with the first key again. He suggested that the sender first encrypt with the first key, then decrypt with the second key, and finally encrypt with the first key. The receiver decrypts with the first key, then encrypts with the second key, and finally decrypts with the first key.

C = EK1(DK2(EK1(P)))
P = DK1(EK2(DK1(C)))


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This is sometimes called encrypt-decrypt-encrypt (EDE) mode [55]. If the block algorithm has an n-bit key, then this scheme has a 2n-bit key. The curious encrypt-decrypt-encrypt pattern was designed by IBM to preserve compatibility with conventional implementations of the algorithm: Setting the two keys equal to each other is identical to encrypting once with the key. There is no security inherent in the encrypt-decrypt-encrypt pattern, but this mode has been adopted to improve the DES algorithm in the X9.17 and ISO 8732 standards [55,761].

K1 and K2 alternate to prevent the meet-in-the-middle attack previously described. If C = EK2(EK1(EK1(P))), then a cryptanalyst could precompute EK1(EK1(P))) for every possible K1 and then proceed with the attack. It only requires 2n + 2 encryptions.

Triple encryption with two keys is not susceptible to the same meet-in-the-middle attack described earlier. But Merkle and Hellman developed another time-memory trade-off that could break this technique in 2n - 1 steps using 2n blocks of memory [1075].

For each possible K2, decrypt 0 and store the result in memory. Then, decrypt 0 with each possible K1 to get P. Triple-encrypt P to get C, and then decrypt C with K1. If that decryption is a decryption of 0 with a K2 (stored in memory), the K1 K2 pair is a possible candidate. Check if it is right. If it's not, keep looking.

This is a chosen-plaintext attack, requiring an enormous amount of chosen plaintext to mount. It requires 2n time and memory, and 2m chosen plaintexts. It is not very practical, but it is a weakness.

Paul van Oorschot and Michael Wiener converted this to a known-plaintext attack, requiring p known plaintexts. This example assumes EDE mode.

(1) Guess the first intermediate value, a.
(2) Tabulate, for each possible K1, the second intermediate value, b, when the first intermediate value is a, using known plaintext:
b = DK1(C)

where C is the resulting ciphertext from a known plaintext.
(3) Look up in the table, for each possible K2, elements with a matching second intermediate value, b:
b = EK2(a)
(4) The probability of success is p/m, where p is the number of known plaintexts and m is the block size. If there is no match, try another a and start again.

The attack requires 2n + m/p time and p memory. For DES, this is 2120/p [1558]. For p greater than 256, this attack is faster than exhaustive search.

Triple Encryption with Three Keys

If you are going to use triple encryption, I recommend three different keys. The key length is longer, but key storage is usually not a problem. Bits are cheap.

C = EK3(DK2(EK1(P)))
P = DK1(EK2(DK3(C)))

The best time-memory trade-off attack takes 22n steps and requires 2n blocks of memory; it's a meet-in-the-middle attack [1075]. Triple encryption, with three independent keys, is as secure as one might naïvely expect double encryption to be.

Triple Encryption with Minimum Key (TEMK)

There is a secure way of using triple encryption with two keys that prevents the previous attack, called Triple Encryption with Minimum Key (TEMK) [858]. The trick is to derive three keys from two: X1 and X2:

K1 = EX1(DX2(EX1(T1)))
K2 = EX1(DX2(EX1(T2)))
K3 = EX1(DX2(EX1(T3)))

T1, T2, and T3 are constants, which do not have to be secret. This is a special construction that guarantees that for any particular pair of keys, the best attack is a known-plaintext attack.

Triple-Encryption Modes

It's not enough to just specify triple encryption; there are several ways to do it. The decision of which to use affects both security and efficiency.

Here are two possible triple-encryption modes:

Inner-CBC: Encrypt the entire file in CBC mode three different times (see Figure 15.1a). This requires three different IVs.
Ci = EK3(Si ⊕ Ci - 1); Si = DK2(Ti ⊕ Si - 1); Ti = EK1(Pi ⊕ Ti - 1)
Pi = Ti - 1 ⊕ DK1(Ti); Ti = Si - 1 ⊕ EK2(Si); Si = Ci - 1 ⊕ DK3(Ci)

C0, S0, and T0 are IVs.
Outer-CBC: Triple-encrypt the entire file in CBC mode (see Figure 15.1b). This requires one IV.
Ci = EK3(DK2(EK1(Pi ⊕ Ci - 1)))
Pi = Ci - 1 ⊕ DK1(EK2(DK3(Ci)))


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Both modes require more resources than single encryption: more hardware or more time. However, given three encryption chips, the throughput of inner-CBC is no slower than single encryption. Since the three CBC encryptions are independent, three chips can be kept busy all the time, each feeding back into itself.

On the other hand, outer-CBC feedback is outside the three encryptions. This means that even with three chips, the throughput is only one-third that of single encryption. To get the same throughput with outer-CBC, you need to interleave IVs (see Section 9.12):

Figure 15.1 Triple encryption in CBC mode.

Ci = EK3(DK2(EK1(Pi ⊕ Ci-3)))

In this case C0, C-1, and C-2 are IVs. This doesn't help software implementations any, unless you have a parallel machine.

Unfortunately, the simpler mode is also the least secure. Biham analyzed various modes with respect to chosen-ciphertext differential cryptanalysis and found that inner-CBC is only slightly more secure than single encryption against a differential attack. If you think of triple encryption as a single larger algorithm, then inner feedbacks allow the introduction of external and known information into the inner workings of the algorithm; this facilitates cryptanalysis. The differential attacks require enormous amounts of chosen ciphertext to mount and are not very practical, but the results should be enough to give the most paranoid pause. Another analysis against meet-in-the-middle and brute-force attacks concludes that they are all equally secure [806].

There are other modes, as well. You can encrypt the entire file once in ECB, then twice in CBC; or once in CBC, once in ECB, and once in CBC; or twice in CBC and once in ECB. Biham showed that these variants are no more secure than single DES against a chosen-plaintext differential cryptanalysis attack [162]. And he doesn't have high hopes for any other variants. If you are going to use triple encryption, use modes with outer feedback.

Variants on Triple Encryption

Before there were proofs that DES does not form a group, several schemes were proposed for multiple encryption. One way to guarantee that triple encryption doesn't reduce to single encryption is to change the effective block size. One simple method is to add a bit of padding. Pad the text with a string of random bits, half a block in length, between the first and second and between the second and third encryptions (see Figure 15.2). If p is the padding function, then:

C = EK3(p(EK2(p(EK1(P)))))

This padding not only disrupts patterns, but also overlaps encrypted blocks like bricks. It only adds one block to the length of the message.

Another technique, proposed by Carl Ellison, is to use some kind of keyless permutation function between the three encryptions. The permutation could work on large blocks—8 kilobytes or so—and would effectively give this variant a block size of 8 kilobytes. Assuming that the permutation is fast, this variant is not much slower than basic triple encryption.

C = EK3(T(EK2(T(EK1(P)))))

T collects a block of input (up to 8 kilobytes in length) and uses a pseudo-random-number generator to transpose it. A 1-bit change in the input causes 8 changed output bytes after the first encryption, up to 64 changed output bytes after the second encryption, and up to 512 changed output bytes after the third encryption. If each block algorithm is in CBC mode, as originally proposed, then the effect of a single changed input bit is likely to be the entire 8 kilobyte block, even in blocks other than the first.

Figure 15.2 Triple encryption with padding.

The most recent variant of this scheme responded to Biham's attack on inner-CBC by including a whitening pass to hide plaintext patterns. That pass is a stream XOR with a cryptographically secure random-number generator called R below. The T on either side of it prevents the cryptanalyst from knowing a priori which key was used to encrypt any given byte on input to the last encryption. The second encryption is labelled nE (encryption with one of n different keys, used cyclically):

C = EK3(R(T(nEK2(T(EK1(R))))))

All encryptions are in ECB mode and keys are provided at least for the n + 2 encryption keys and the cryptographically secure random-number generator.

This scheme was proposed with DES, but works with any block algorithm. I know of no analysis of the security of this scheme.

15.3 Doubling the Block Length

There is some argument in the academic community whether a 64-bit block is long enough. On the one hand, a 64-bit block length only diffuses plaintext over 8 bytes of ciphertext. On the other hand, a longer block length makes it harder to hide patterns securely; there is more room to make mistakes.

Some propose doubling the block length of an algorithm using multiple encryptions [299]. Before implementing any of these, look for the possibility of meet-in-the-middle attacks. Richard Outerbridge's scheme [300], illustrated in Figure 15.3, is no more secure than single-block, two-key triple encryption [859].

However, I advise against this sort of thing. It isn't faster than conventional triple encryption: six encryptions are still required to encrypt two blocks of data. We know the characteristics of triple encryption; constructions like this often have hidden problems.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

15.4 Other Multiple Encryption Schemes

The problem with two-key triple encryption is that it only doubles the size of the keyspace, but it requires three encryptions per block of plaintext. Wouldn't it be nice to find some clever way of combining two encryptions that would double the size of the keyspace?

Double OFB/Counter

This method uses a block algorithm to generate two keystreams, which are then used to encrypt the plaintext.

Si = EK1(Si - 1 ⊕ I1); I1 = I1 + 1
Ti = EK2(Ti - 1 ⊕ I2); I2 = I2 + 1
Ci = Pi ⊕ Si ⊕ Ti

Figure 15.3 Doubling the block length.

Si and Ti are internal variables, and I1 and I2 are counters. Two copies of the block algorithm run in a kind of hybrid OFB/counter mode, and the plaintext, Si, and Ti are XORed together. The two keys, K1 and K2, are independent. I know of no cryptanalysis of this variant.

ECB + OFB

This method was designed for encrypting multiple messages of a fixed length, for example, disk blocks [186,188]. Use two keys: K1 and K2. First, use the algorithm and K1 to generate a mask of the required block length. This mask will be used repeatedly to encrypt messages with the same keys. Then, XOR the plaintext message with the mask. Finally, encrypt the XORed plaintext with the algorithm and K2 in ECB mode.

This mode has not been analyzed outside the paper in which it was proposed. Clearly it is at least as strong as a single ECB encryption and may be as strong as two passes with the algorithm. Possibly, a cryptanalyst could search for the two keys independently, if several known plaintext files are encrypted with the same key.

To thwart analysis of identical blocks in the same positions of different messages, you can add an IV. Unlike an IV in any other mode, here the IV is XORed with every block of the message before ECB encryption.

Matt Blaze designed this mode for his UNIX Cryptographic File System (CFS). It is a nice mode because the latency is only one encryption in ECB mode; the mask can be generated once and stored. In CFS, DES is the block algorithm.

xDESi

In [1644,1645], DES is used as a building block for a series of block algorithms with both larger key sizes and larger block sizes. These constructions do not depend on DES in any way and can be used with any block algorithm.

The first, xDES1, is simply a Luby-Rackoff construction with the block cipher as the underlying function (see Section 14.11). The block size is twice the size of the underlying block cipher and the key size is three times the size of the underlying block cipher. In each of 3 rounds, encrypt the right half with the block algorithm and one of the keys, XOR the result with the left half, and swap the two halves.

This is faster than conventional triple encryption, since three encryptions encrypt a block twice as large as the underlying algorithm. But there is also a simple meet-in-the-middle attack that finds the key with a table the size of 2k, where k is the key size of the underlying algorithm. Encrypt the right half of a plaintext block with all possible values of K1, XOR the left half of the plaintext, and store these values in a table. Then, encrypt the right half of the ciphertext with all possible values of K3 and look for a match in the table. If you find one, the key pair K1 and K3 are possible candidates for the right key. Repeat the attack a few times, and only one candidate will remain. This shows that xDES1 is not an ideal solution. Even worse, there is a chosen plaintext attack that proves xDES1 is not much stronger than the underlying block cipher [858].

xDES2 extends this idea to a 5-round algorithm with a block size 4 times that of the underlying block cipher and a key size 10 times that of the underlying block cipher. Figure 15.4 is one round of xDES2; each of the four sub-blocks are the size of the underlying block ciphers and all 10 keys are independent.

This scheme is also faster than triple encryption: Ten encryptions are used to encrypt a block four times the size of the underlying block cipher. However, it is vulnerable to differential cryptanalysis [858] and should not be used. The scheme is even vulnerable if DES with independent round keys is used.

Figure 15.4 One round of xDES2.

For i ≥ 3, xDESi is probably too big to be useful as a block algorithm. For example, the block size for xDES3 is 6 times that of the underlying cipher, the key size is 21 times, and 21 encryptions are required to encrypt a block 6 times that of the underlying block cipher. Triple encryption is faster.

Quintuple Encryption

If triple encryption isn't secure enough—perhaps you need to encrypt triple-encryption keys using an even stronger algorithm—then higher multiples might be in order. Quintuple encryption is very strong against meet-in-the-middle attacks. (Similar arguments to the ones used with double encryption can show that quadruple encryption provides minimal security improvements over triple encryption.)

C = EK1(DK2(EK3(DK2(EK1(P)))))
P = DK1(EK2(DK3(EK2(DK1(C)))))

This construction is backwards compatible with triple encryption if K2 = K3, and is backwards compatible with single encryption if K1 = K2 = K3. Of course, it would be even stronger if all five keys were independent.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

15.5 CDMF Key Shortening

This method was designed by IBM for their Commercial Data Masking Facility or CDMF (see Section 24.8) to shrink a 56-bit DES key to a 40-bit key suitable for export [785]. It assumes that the original DES key includes the parity bits.

(1) Zero the parity bits: bits 8, 16, 24, 32, 40, 48, 56, 64.
(2) Encrypt the output of step (1) with DES and the key 0xc408b0540ba1e0ae, and XOR the result with the output of step (1).
(3) Take the output of step (2) and zero the following bits: 1, 2, 3, 4, 8, 16, 17, 18, 19, 20, 24, 32, 33, 34, 35, 36, 40, 48, 49, 50, 51, 52, 56, 64.
(4) Encrypt the output of step (3) with DES and the following key: 0xef2c041ce6382fe6. This key is then used for message encryption.

Remember that this method shortens the key length, and thereby weakens the algorithm.

15.6 Whitening

Whitening is the name given to the technique of XORing some key material with the input to a block algorithm, and XORing some other key material with the output. This was first done in the DESX variant developed by RSA Data Security, Inc., and then (presumably independently) in Khufu and Khafre. (Rivest named this technique; it's a nonstandard usage of the word.)

The idea is to prevent a cryptanalyst from obtaining a plaintext/ciphertext pair for the underlying algorithm. The technique forces a cryptanalyst to guess not only the algorithm key, but also one of the whitening values. Since there is an XOR both before and after the block algorithm, this technique is not susceptible to a meet-in-the-middle attack.

C = K3 ⊕ EK2(P ⊕ K1)
P = K1 ⊕ DK2(C ⊕ K3)

If K1 = K3, then a brute-force attack requires 2n + m/p operations, where n is the key size, m is the block size, and p is the number of known plaintexts. If K1 and K3 are different, then a brute-force attack requires 2n + m + 1 operations with three known plaintexts. Against differential and linear cryptanalysis, these measures only provide a few key bits of protection. But computationally this is a very cheap way to increase the security of a block algorithm.

15.7 Cascading Multiple Block Algorithms

What about encrypting a message once with Algorithm A and key KA, then again with Algorithm B and key KB? Maybe Alice and Bob have different ideas about which algorithms are secure: Alice wants to use Algorithm A and Bob wants to use Algorithm B. This technique is sometimes called cascading, and can be extended far beyond only two algorithms and keys.

Pessimists have said that there is no guarantee that the two algorithms will work together to increase security. There may be subtle interactions between the two algorithms that actually decrease security. Even triple encryption with three different algorithms may not be as secure as you think. Cryptography is a black art; if you don't know what you are doing, you can easily get into trouble.

Reality is much rosier. The previous warnings are true only if the different keys are related to each other. If all of the multiple keys are independent, then the resultant cascade is at least as difficult to break as the first algorithm in the cascade [1033]. If the second algorithm is vulnerable to a chosen-plaintext attack, then the first algorithm might facilitate that attack and make the second algorithm vulnerable to a known-plaintext attack when used in a cascade. This potential attack is not limited to encryption algorithms: If you let someone else specify any algorithm which is used on your message before encryption, then you had better be sure that your encryption will withstand a chosen-plaintext attack. (Note that the most common algorithm used for compressing and digitizing speech to modem speeds, used before any encryption, is CELP—designed by the NSA.)

This can be better phrased: Using a chosen-plaintext attack, a cascade of ciphers is at least as hard to break as any of its component ciphers [858]. A previous result showed that the cascade is at least as difficult to break as the strongest algorithm, but that result is based on some unstated assumptions [528]. Only if the algorithms commute, as they do in the case of cascaded stream ciphers (or block ciphers in OFB mode), is the cascade at least as strong as the strongest algorithm.

If Alice and Bob do not trust each other's algorithms, they can use a cascade. If these are stream algorithms, the order doesn't matter. If they are block algorithms, Alice can first use Algorithm A and then use Algorithm B. Bob, who trusts Algorithm B more, can use Algorithm B followed by Algorithm A. They might even add a good stream cipher between the two algorithms; it can't hurt and could very well increase security.

Remember that the keys for each algorithm in the cascade must be independent. If Algorithm A has a 64-bit key and Algorithm B has a 128-bit key, then the resultant cascade must have a 192-bit key. If you don't use independent keys, then the pessimists are much more likely to be right.

15.8 Combining Multiple Block Algorithms

Here's another way to combine multiple block algorithms, one that is guaranteed to be at least as secure as both algorithms. With two algorithms (and two independent keys):

(1) Generate a random-bit string, R, the same size as the message M.
(2) Encrypt R with the first algorithm.
(3) Encrypt M ⊕ R with the second algorithm.
(4) The ciphertext message is the results of steps (2) and (3).

Assuming the random-bit string is indeed random, this method encrypts M with a one-time pad and then encrypts both the pad and the encrypted message with each of the two algorithms. Since both are required to reconstruct M, a cryptanalyst must break both algorithms. The drawback is that the ciphertext is twice the size of the plaintext.

This method can be extended to multiple algorithms, but the ciphertext expands with each additional algorithm. It's a good idea, but I don't think it's very practical.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 16
Pseudo-Random-Sequence Generators and Stream Ciphers

16.1 Linear Congruential Generators

Linear congruential generators are pseudo-random-sequence generators of the form

Xn = (aXn-1 + b) mod m

in which Xn is the nth number of the sequence, and Xn-1 is the previous number of the sequence. The variables a, b, and m are constants: a is the multiplier, b is the increment, and m is the modulus. The key, or seed, is the value of X0.

This generator has a period no greater than m. If a, b, and m are properly chosen, then the generator will be a maximal period generator (sometimes called maximal length) and have period of m. (For example, b should be relatively prime to m.) Details on choosing constants to ensure maximal period can be found in [863,942]. Another good article on linear congruential generators and their theory is [1446].

Table 16.1, taken from [1272], gives a list of good constants for linear congruential generators. They all produce maximal period generators and even more important, pass the spectral test for randomness for dimensions 2, 3, 4, 5, and 6 [385,863]. They are organized by the largest product that does not overflow a specific word length.

The advantage of linear congruential generators is that they are fast, requiring few operations per bit.

Unfortunately, linear congruential generators cannot be used for cryptography; they are predictable. Linear congruential generators were first broken by Jim Reeds [1294,1295,1296] and then by Joan Boyar [1251]. She also broke quadratic generators:

Xn = (aXn-12 + bXn-1 + c) mod m

and cubic generators:

Xn = (aXn-13 + bXn-12 + cXn-1 + d) mod m

Other researchers extended Boyar's work to break any polynomial congruential generator [923,899,900]. Truncated linear congruential generators were also broken [581,705,580], as were truncated linear congruential generators with unknown parameters [1500,212]. The preponderance of evidence is that congruential generators aren't useful for cryptography.

Table 16.1 Constants for Linear Congruential Generators
Overflow At: | a | b | m
220 | 106 | 1283 | 6075
221 | 211 | 1663 | 7875
222 | 421 | 1663 | 7875
223 | 430 | 2531 | 11979
 | 936 | 1399 | 6655
 | 1366 | 1283 | 6075
224 | 171 | 11213 | 53125
 | 859 | 2531 | 11979
 | 419 | 6173 | 29282
 | 967 | 3041 | 14406
225 | 141 | 28411 | 134456
 | 625 | 6571 | 31104
 | 1541 | 2957 | 14000
 | 1741 | 2731 | 12960
 | 1291 | 4621 | 21870
 | 205 | 29573 | 139968
226 | 421 | 17117 | 81000
 | 1255 | 6173 | 29282
 | 281 | 28411 | 134456
227 | 1093 | 18257 | 86436
 | 421 | 54773 | 259200
 | 1021 | 24631 | 116640
 | 1021 | 25673 | 121500
228 | 1277 | 24749 | 117128
 | 741 | 66037 | 312500
 | 2041 | 25673 | 121500
229 | 2311 | 25367 | 120050
 | 1807 | 45289 | 214326
 | 1597 | 51749 | 244944
 | 1861 | 49297 | 233280
 | 2661 | 36979 | 175000
 | 4081 | 25673 | 121500
 | 3661 | 30809 | 145800
230 | 3877 | 29573 | 139968
 | 3613 | 45289 | 214326
 | 1366 | 150889 | 714025
231 | 8121 | 28411 | 134456
 | 4561 | 51349 | 243000
 | 7141 | 54773 | 259200
232 | 9301 | 49297 | 233280
 | 4096 | 150889 | 714025
233 | 2416 | 374441 | 1771875
234 | 17221 | 107839 | 510300
 | 36261 | 66037 | 312500
235 | 84589 | 45989 | 217728


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Linear congruential generators remain useful for noncryptographic applications, however, such as simulations. They are efficient and show good statistical behavior with respect to most reasonable empirical tests. Considerable information on linear congruential generators and their implementations can be found in [942].

Combining Linear Congruential Generators

Various people examined the combination of linear congruential generators [1595,941]. The results are no more cryptographically secure, but the combinations have longer periods and perform better in some randomness tests.

Use this generator for 32-bit computers [941]:

static long s1 = 1 ; /* A "long" must be 32 bits long. */ static long s2 = 1 ;

#define MODMULT(a,b,c,m,s) q = s/a; s = b*(s-a*q) - c*q; if (s<0) s+=m  ;
/* MODMULT(a,b,c,m,s) computes s*b mod m, provided that m=a*b+c and 0 <= c < m. */

/* combinedLCG returns a pseudorandom real value in the range
* (0,1). It combines linear congruential generators with
* periods of 231-85 and 231-249, and has a period that is the
* product of these two prime numbers. */

double combinedLCG ( void )
{
  long q ;
  long z ;

  MODMULT ( 53668, 40014, 12211, 2147483563L, s1 )
  MODMULT ( 52774, 40692, 3791, 2147483399L, s2 )
  z = s1 - s2 ;
  if ( z < 1 )
    z += 2147483562 ;
  return z * 4.656613e-10 ;
}

/* In general, call initLCG before using combinedLCG. */
void initLCG ( long InitS1, long InitS2 )
{
  s1 = InitS1 ;
  s2 = InitS2 ;
}

This generator works as long as the machine can represent all integers between-231 + 85 and 231 - 85. The variables, s1 and s2, are global; they hold the current state of the generator. Before the first call, they must be initialized. The variable s1 needs an initial value between 1 and 2147483562; the variable s2 needs an initial value between 1 and 2147483398. The generator has a period somewhere in the neighborhood of 1018.

If you only have a 16-bit computer, use this generator instead:

static int s1 = 1 ; /* An "int" must be 16 bits long. */
static int s2 = 1 ;
static int s3 = 1 ;

#define MODMULT(a,b,c,m,s) q = s/a; s = b*(s-a*q) - c*q; if
(s<0) s+=m  ;

/* combined LCG returns a pseudorandom real value in the
range
* (0,1). It combines linear congruential generators with
* periods of 215-405, 215-1041, and 215-1111, and has a period
* that is the product of these three prime numbers. */

double combinedLCG ( void )
{
  int q ;
  int z ;

  MODMULT ( 206, 157, 21, 32363, s1 )
  MODMULT ( 217, 146, 45, 31727, s2 )
  MODMULT ( 222, 142, 133, 31657, s3 )
  z = s1 - s2 ;
  if ( z > 706 )
    z -= 32362 ;
  z += s3 ;
  if ( z < 1 )
    z += 32362 ;
  return z * 3.0899e-5 ;
}

/* In general, call initLCG before using combinedLCG. */
void initLCG ( int InitS1, int InitS2, InitS3 )
{
  s1 = InitS1 ;
  s2 = InitS2 ;
  s3 = InitS3 ;
}

This generator works as long as the machine can represent all integers between-32363 and 32363. The variables, s1, s2, and s3, are global; they hold the current state of the generator. Before the first call, they must be initialized. The variable s1 needs an initial value between 1 and 32362. The variable s2 needs an initial value between 1 and 31726. The variable s3 needs an initial value between 1 and 31656. This generator has a period of 1.6*1013.

For both of these generators, the constant term b in the linear congruence is 0.

16.2 Linear Feedback Shift Registers

Shift register sequences are used in both cryptography and coding theory. There is a wealth of theory about them; stream ciphers based on shift registers have been the workhorse of military cryptography since the beginnings of electronics.

A feedback shift register is made up of two parts: a shift register and a feedback function (see Figure 16.1). The shift register is a sequence of bits. (The length of a shift register is figured in bits; if it is n bits long, it is called an n-bit shift register.) Each time a bit is needed, all of the bits in the shift register are shifted 1 bit to the right. The new left-most bit is computed as a function of the other bits in the register. The output of the shift register is 1 bit, often the least significant bit. The period of a shift register is the length of the output sequence before it starts repeating.

Cryptographers have liked stream ciphers made up of shift registers: They are easily implemented in digital hardware. I will only touch on the mathematical theory. Ernst Selmer, the Norwegian government's chief cryptographer, worked out the theory of shift register sequences in 1965 [1411]. Solomon Golomb, an NSA mathematician, wrote a book with Selmer's results and some of his own [643]. See also [970,971,1647].

The simplest kind of feedback shift register is a linear feedback shift register, or LFSR (see Figure 16.2). The feedback function is simply the XOR of certain bits in the register; the list of these bits is called a tap sequence. Sometimes this is called a Fibonacci configuration. Because of the simple feedback sequence, a large body of mathematical theory can be applied to analyzing LFSRs. Cryptographers like to analyze sequences to convince themselves that they are random enough to be secure. LFSRs are the most common type of shift registers used in cryptography.

Figure 16.3 is a 4-bit LFSR tapped at the first and fourth bit. If it is initialized with the value 1111, it produces the following sequence of internal states before repeating:

1 1 1 1
0 1 1 1
1 0 1 1
0 1 0 1
1 0 1 0
1 1 0 1
0 1 1 0
0 0 1 1
1 0 0 1
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
1 1 0 0
1 1 1 0

Figure 16.1 Feedback shift register.

Figure 16.2 Linear feedback shift register.

The output sequence is the string of least significant bits:

1 1 1 1 0 1 0 1 1 0 0 1 0 0 0....

An n-bit LFSR can be in one of 2n - 1 internal states. This means that it can, in theory, generate a 2n - 1-bit-long pseudo-random sequence before repeating. (It's 2n - 1 and not 2n because a shift register filled with zeros will cause the LFSR to output a neverending stream of zeros—this is not particularly useful.) Only LFSRs with certain tap sequences will cycle through all 2n - 1 internal states; these are the maximal-period LFSRs. The resulting output sequence is called an m-sequence.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In order for a particular LFSR to be a maximal-period LFSR, the polynomial formed from a tap sequence plus the constant 1 must be a primitive polynomial mod 2. The degree of the polynomial is the length of the shift register. A primitive polynomial of degree n is an irreducible polynomial that divides x2n-1 + 1, but not xd + 1 for any d that divides 2n - 1 (see Section 11.3). For the mathematical theory behind all this, consult [643,1649,1648].

Figure 16.3 4-bit LFSR.

In general, there is no easy way to generate primitive polynomials mod 2 for a given degree. The easiest way is to choose a random polynomial and test whether it is primitive. This is complicated—something like testing random numbers for primality—but many mathematical software packages do this. See [970,971] for some methods.

Table 16.2 lists some, but by no means all, primitive polynomials mod 2 of varying degrees [1583,643,1649,1648,1272,691]. For example, the listing (32, 7, 5, 3, 2, 1, 0) means that the following polynomial is primitive modulo 2:

x32 + x7 + x5 + x3 + x2 + x + 1

It's easy to turn this into a maximal-period LFSR. The first number is the length of the LFSR. The last number is always 0 and can be ignored. All the numbers, except the 0, specify the tap sequence, counting from the left of the shift register. That is, low degree terms in the polynomial correspond to taps near the left-hand side of the register.

To continue the example, the listing (32, 7, 5, 3, 2, 1, 0) means that if you take a 32-bit shift register and generate the new bit by XORing the thirty-second, seventh, fifth, third, second, and first bits together (see Figure 16.4), the resultant LFSR will be maximal length; it will cycle through 232 - 1 values before repeating.

The C code for this LFSR looks like:

int LFSR () {
     static unsigned long ShiftRegister = 1;
     /* Anything but 0. */
     ShiftRegister = ((((ShiftRegister >> 31)
                ^ (ShiftRegister >> 6)
                ^ (ShiftRegister >> 4)
                ^ (ShiftRegister >> 2)
                ^ (ShiftRegister >> 1)
                ^ ShiftRegister))
                & 0×00000001)
                << 31)
                | (ShiftRegister >> 1) ;
     return ShiftRegister & 0×00000001;
}

Figure 16.4 32-bit long maximal-length LFSR.

Table 16.2 Some Primitive Plynomials Mod 2
(1, 0) | (36, 11, 0) | (68, 9, 0) | (97, 6, 0)
(2, 1, 0) | (36, 6, 5, 4, 2, 1, 0) | (68, 7, 5, 1, 0) | (98, 11, 0)
(3, 1, 0) | (37, 6, 4, 1, 0) | (69, 6, 5, 2, 0) | (98, 7, 4, 3, 1, 0)
(4, 1, 0) | (37, 5, 4, 3, 2, 1, 0) | (70, 5, 3, 1, 0) | (99, 7, 5, 4, 0)
(5, 2, 0) | (38, 6, 5, 1, 0) | (71, 6, 0) | (100, 37, 0)
(6, 1, 0) | (39, 4, 0) | (71, 5, 3, 1, 0) | (100, 8, 7, 2, 0)
(7, 1, 0) | (40, 5, 4, 3, 0) | (72, 10, 9, 3, 0) | (101, 7, 6, 1, 0)
(7, 3, 0) | (41, 3, 0) | (72, 6, 4, 3, 2, 1, 0) | (102, 6 5 3 0)
(8, 4, 3, 2, 0) | (42, 7, 4, 3, 0) | (73, 25, 0) | (103, 9, 9)
(9, 4, 0) | (42, 5, 4, 3, 2, 1, 0) | (73, 4, 3, 2, 0) | (104, 11, 10, 1, 0)
(10, 3, 0) | (43, 6, 4, 3, 0) | (74, 7, 4, 3, 0) | (105, 16, 0)
(11, 2, 0) | (44, 6, 5, 2, 0) | (75, 6, 3, 1, 0) | (106, 15, 0)
(12, 6, 4, 1, 0) | (45, 4, 3, 1, 0) | (76, 5, 4, 2, 0) | (107, 9, 7, 4, 0)
(13, 4, 3, 1, 0) | (46, 8, 7, 6, 0) | (77, 6, 5, 2, 0) | (108, 31, 0)
(14, 5, 3, 1, 0) | (46, 8, 5, 3, 2, 1, 0) | (78, 7, 2, 1, 0) | (109, 5, 4, 2, 0)
(15, 1, 0) | (47, 5, 0) | (79, 9, 0) | (110, 6, 4, 1, 0)
(16, 5, 3, 2, 0) | (48, 9, 7, 4, 0) | (79, 4, 3, 2, 0) | (111, 10, 0)
(17, 3, 0) | (48, 7, 5, 4, 2, 1, 0) | (80, 9, 4, 2, 0) | (111, 49, 0)
(17, 5, 0) | (49, 9, 0) | (80, 7, 5, 3, 2, 1, 0) | (113, 9, 0)
(17, 6, 0) | (49, 6, 5, 4, 0) | (81, 4, 0) | (113, 15, 0)
(18, 7, 0) | (50, 4, 3, 2, 0) | (82, 9, 6, 4, 0) | (113, 30, 0)
(18, 5, 2, 1, 0) | (51, 6, 3, 1, 0) | (82, 8, 7, 6, 1, 0) | (114, 11, 2, 1, 0)
(19, 5, 2, 1, 0) | (52, 3, 0) | (83, 7, 4, 2, 0) | (115, 8, 7, 5, 0)
(20, 3, 0) | (53, 6, 2, 1, 0) | (84, 13, 0) | (116, 6, 5, 2, 0)
(21, 2, 0) | (54, 8, 6, 3, 0) | (84, 8, 7, 5, 3, 1, 0) | (117, 5, 2, 1, 0)
(22, 1, 0) | (54, 6, 5, 4, 3, 2, 0) | (85, 8, 2, 1, 0) | (118, 33, 0)
(23, 5, 0) | (55, 24, 0) | (86, 6, 5, 2, 0) | (119, 8, 0)
(24, 4, 3, 1, 0) | (55, 6, 2, 1, 0) | (87, 13, 0) | (119, 45, 0)
(25, 3, 0) | (56, 7, 4, 2, 0) | (87, 7, 5, 1, 0) | (120, 9, 6, 2, 0)
(26, 6, 2, 1, 0) | (57, 7, 0) | (88, 11, 9, 8, 0) | (121, 18, 0)
(27, 5, 2, 1, 0) | (57, 5, 3, 2, 0) | (88, 8, 5, 4, 3, 1, 0) | (122, 6, 2, 1, 0)
(28, 3, 0) | (58, 19, 0) | (89, 38, 0) | (123, 2, 0)
(29, 2, 0) | (58, 6, 5, 1, 0) | (89, 51, 0) | (124, 37, 0)
(30, 6, 4, 1, 0) | (59, 7, 4, 2, 0) | (89, 6, 5, 3, 0) | (125, 7, 6, 5, 0)
(31, 3, 0) | (59, 6, 5, 4, 3, 1, 0) | (90, 5, 3, 2, 0) | (126, 7, 4, 2, 0)
(31, 6, 0) | (60, 1, 0) | (91, 8, 5, 1, 0) | (127, 1, 0)
(31, 7, 0) | (61, 5, 2, 1, 0) | (91, 7, 6, 5, 3, 2, 0) | (127, 7, 0)
(31, 13, 0) | (62, 6, 5, 3, 0) | (92, 6, 5, 2, 0) | (127, 63, 0)
(32, 7, 6, 2, 0) | (63, 1, 0) | (93, 2, 0) | (128, 7, 2, 1, 0)
(32, 7, 5, 3, 2, 1, 0) | (64, 4, 3, 1, 0) | (94, 21, 0) | (129, 5, 0)
(33, 13, 0) | (65, 18, 0) | (94, 6, 5, 1, 0) | (130, 3, 0)
(33, 16, 4, 1, 0) | (65, 4, 3, 1, 0) | (95, 11, 0) | (131, 8, 3, 2, 0)
(34, 8, 4, 3, 0) | (66, 9, 8, 6, 0) | (95, 6, 5, 4, 2, 1, 0) | (132, 29, 0)
(34, 7, 6, 5, 2, 1, 0) | (66, 8, 6, 5, 3, 2, 0) | (96, 10, 9, 6, 0) | (133, 9, 8, 2, 0)
(35, 2, 0) | (67, 5, 2, 1, 0) | (96, 7, 6, 4, 3, 2, 0) | (134, 57, 0)
(135, 11, 0) | (152, 6, 3, 2, 0) | (178, 87, 0) | (270, 133, 0)
(135, 16, 0) | (153, 1, 0) | (183, 56, 0) | (282, 35, 0)
(135, 22, 0) | (153, 8, 0) | (194, 87, 0) | (282, 43, 0)
(136, 8, 3, 2, 0) | (154, 9, 5, 1, 0) | (198, 65, 0) | (286, 69, 0)
(137, 21, 0) | (155, 7, 5, 4, 0) | (201, 14, 0) | (286, 73, 0)
(138, 8, 7, 1, 0) | (156, 9, 5, 3, 0) | (201, 17, 0) | (294, 61, 0)
(139, 8, 5, 3, 0) | (157, 6, 5, 2, 0) | (201, 59, 0) | (322, 67, 0)
(140, 29, 0) | (158, 8, 6, 5, 0) | (201, 79, 0) | (333, 2, 0)
(141, 13, 6, 1, 0) | (159, 31, 0) | (202, 55, 0) | (350, 53, 0)
(142, 21, 0) | (159, 34, 0) | (207, 43, 0) | (366, 29, 0)
(143, 5, 3, 2, 0) | (159, 40, 0) | (212, 105, 0) | (378, 43, 0)
(144, 7, 4, 2, 0) | (160, 5, 3, 2, 0) | (218, 11, 0) | (378, 107, 0)
(145, 52, 0) | (161, 18, 0) | (218, 15, 0) | (390, 89, 0)
(145, 69, 0) | (161, 39, 0) | (218, 71, 0) | (462, 73, 0)
(146, 5, 3, 2, 0) | (161, 60, 0) | (218, 83, 0) | (521, 32, 0)
(147, 11, 4, 2, 0) | (162, 8, 7, 4, 0) | (225, 32, 0) | (521, 48, 0)
(148, 27, 0) | (163, 7, 6, 3, 0) | (225, 74, 0) | (521, 158, 0)
(149, 10, 9, 7, 0) | (164, 12, 6, 5, 0) | (225, 88, 0) | (521, 168, 0)
(150, 53, 0) | (165, 9, 8, 3, 0) | (225, 97, 0) | (607, 105, 0)
(151, 3, 0) | (166, 10, 3, 2, 0) | (225, 109, 0) | (607, 147, 0)
(151, 9, 0) | (167, 6, 0) | (231, 26, 0) | (607, 273, 0)
(151, 15, 0) | (170, 23, 0) | (231, 34, 0) | (1279, 216, 0)
(151, 31, 0) | (172, 2, 0) | (234, 31, 0) | (1279, 418, 0)
(151, 39, 0) | (174, 13, 0) | (234, 103, 0) | (2281, 715, 0)
(151, 43, 0) | (175, 6, 0) | (236, 5, 0) | (2281, 915, 0)
(151, 46, 0) | (175, 16, 0) | (250, 103, 0) | (2281, 1029, 0)
(151, 51, 0) | (175, 18, 0) | (255, 52, 0) | (3217, 67, 0)
(151, 63, 0) | (175, 57, 0) | (255, 56, 0) | (3217, 576, 0)
(151, 66, 0) | (177, 8, 0) | (255, 82, 0) | (4423, 271, 0)
(151, 67, 0) | (177, 22, 0) | (258, 83, 0) | (9689, 84, 0)
(151, 70, 0) | (177, 88, 0) | (266, 47, 0) |


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The code is a little more complicated when the shift register is longer than the computer's word size, but not significantly so.

Note that all of these listings have an odd number of coefficients. I have provided such a large table because LFSRs are often used for stream-cipher cryptography and I wanted many examples so that different people would pick different primitive polynomials. Since, if p(x) is primitive, then so is xnp(1/x); each entry on the table is actually two primitive polynomials.

For example, if (a, b, 0) is primitive, then (a, a - b, 0) is also primitive. If (a, b, c, d, 0) is primitive, then (a, a - d, a - c, a - b, 0) is also primitive. Mathematically:

if xa + xb + 1 is primitive, so is xa + xa-b + 1
if xa + xb + xc + xd + 1 is primitive, so is xa + xa-d + xa-c + xa-b + 1

Primitive trinomials are fastest in software, because only two bits of the shift register have to be XORed to generate each new bit. Actually, all the feedback polynomials listed in Table 16.2 are sparse, meaning that they only have a few coefficients. Sparseness is always a source of weakness, sometimes enough to break the algorithm. It is far better to use dense primitive polynomials, those with a lot of coefficients, for cryptographic applications. If you use dense polynomials, and especially if you make them part of the key, you can live with much shorter LFSRs.

Generating dense primitive polynomials modulo 2 is not easy. In general, to generate primitive polynomials of degree k you need to know the factorization of 2k - 1. Three good references for finding primitive polynomials are [652,1285,1287].

LFSRs are competent pseudo-random-sequence generators all by themselves, but they have some annoying nonrandom properties. Sequential bits are linear, which makes them useless for encryption. For an LFSR of length n, the internal state is the next n output bits of the generator. Even if the feedback scheme is unknown, it can be determined from only 2n output bits of the generator, by using the highly efficient Berlekamp-Massey algorithm [1082,1083]: see Section 16.3.

Also, large random numbers generated from sequential bits of this sequence are highly correlated and, for certain types of applications, not very random at all. Even so, LFSRs are often used as building blocks in encryption algorithms.

LFSRs in Software

LFSRs are slow in software, but they're faster in assembly language than in C. One solution is to run 16 LFSRs (or 32, depending on your computer's word size) in parallel. This scheme uses an array of words that is the length of the LFSR, with each bit position in the words representing a different LFSR. Assuming all the feedback polynomials are the same, this can run pretty quickly. In general, the best way to update shift registers is to multiply the current state by suitable binary matrices [901].

It is also possible to modify the LFSR's feedback scheme. The resultant generator is no better cryptographically, but it still has a maximal period and is easy to implement in software [1272]. Instead of using the bits in the tap sequence to generate the new left-most bit, each bit in the tap sequence is XORed with the output of the generator and replaced; then the output of the generator becomes the new left-most bit (see Figure 16.5). This is sometimes called a Galois configuration.

In C, this looks like:

#define mask 0×80000057

static unsigned long ShiftRegister=1;
void seed_LFSR (unsigned long seed)
{
     if (seed == 0) /* avoid calamity */
          seed = 1;
     ShiftRegister = seed;
}

int modified_LFSR (void)
{
     if (ShiftRegister & 0×00000001) {
          ShiftRegister = ((ShiftRegister ^ mask >> 1) |
     0×8000000;
          return 1;
     } else {
          ShiftRegister >>= 1;
          return 0;
     }
}

Figure 16.5 Galois LFSR.

The savings here is that all the XORs can be done as a single operation. This can also be parallelized, and the different feedback polynomials can be different. The Galois configuration can also be faster in hardware, especially in custom VLSI implementations. In general, if you are using hardware that is good at shifts, use a Fibonacci configuration; if you can exploit parallelism, use a Galois configuration.

16.3 Design and Analysis of Stream Ciphers

Most practical stream-cipher designs center around LFSRs. In the early days of electronics, they were very easy to build. A shift register is nothing more than an array of bit memories and the feedback sequence is just a series of XOR gates. Even in VLSI circuitry, a LFSR-based stream cipher can give you a lot of security with only a few logic gates.

The problem with LFSRs is that they are very inefficient in software. You want to avoid sparse feedback polynomials—they facilitate correlation attacks [1051,1090,350]—and dense feedback polynomials are inefficient. Any stream cipher outputs a bit at a time; you have to iterate the algorithm 64 times to encrypt what a single iteration of DES can encrypt. In fact, a simple LFSR algorithm like the shrinking generator described later is no faster in software than DES.

This branch of cryptography is fast-paced and very politically charged. Most designs are secret; a majority of military encryptions systems in use today are based on LFSRs. In fact, most Cray computers (Cray 1, Cray X-MP, Cray Y-MP) have a rather curious instruction generally known as "population count." It counts the 1 bits in a register and can be used both to efficiently calculate the Hamming distance between two binary words and to implement a vectorized version of a LFSR. I've heard this called the canonical NSA instruction, demanded by almost all computer contracts.

On the other hand, an astonishingly large number of seemingly complex shift-register-based generators have been cracked. And certainly military cryptanalysis institutions such as the NSA have cracked a lot more. Sometimes it's amazing to see the simple ones proposed again and again.

Linear Complexity

Analyzing stream ciphers is often easier than analyzing block ciphers. For example, one important metric used to analyze LFSR-based generators is linear complexity, or linear span. This is defined as the length, n, of the shortest LFSR that can mimic the generator output. Any sequence generated by a finite-state machine over a finite field has a finite linear complexity [1006]. Linear complexity is important because a simple algorithm, called the Berlekamp-Massey algorithm, can generate this LFSR after examining only 2n bits of the keystream [1005]. Once you've generated this LFSR, you've broken the stream cipher.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This idea has extensions from fields to rings [1298], and when the output sequence is viewed as numbers over fields of odd characteristic [842]. A further enhancement is the notion of a linear complexity profile, which measures the linear complexity of the sequence as it gets longer and longer [1357,1168,411,1582]. Another algorithm for computing linear complexity is useful only in very specialized circumstances [597,595,596,1333]. A generalization of linear complexity is in [776]. There is also the notion of sphere complexity [502] and 2-adic complexity [844].

In any case, remember that a high linear complexity does not necessarily indicate a secure generator, but a low linear complexity indicates an insecure one [1357,1249].

Correlation Immunity

Cryptographers try to get a high linear complexity by combining the output of several output sequences in some nonlinear manner. The danger here is that one or more of the internal output sequences—often just outputs of individual LFSRs—can be correlated with the combined keystream and attacked using linear algebra. Often this is called a correlation attack or a divide-and-conquer attack. Thomas Siegenthaler has shown that correlation immunity can be precisely defined, and that there is a trade-off between correlation immunity and linear complexity [1450].

The basic idea behind a correlation attack is to identify some correlation between the output of the generator and the output of one of its internal pieces. Then, by observing the output sequence, you can obtain information about that internal output. Using that information and other correlations, collect information about the other internal outputs until the entire generator is broken.

Correlation attacks and variations such as fast correlation attacks—these offer a trade-off between computational complexity and effectiveness—have been successfully applied to a number of LFSR-based keystream generators [1451,278,1452,572,1636,1051,1090,350,633,1054,1089,995]. Some interesting new ideas along these lines are in [46,1641].

Other Attacks

There are other general attacks against keystream generators. The linear consistency test attempts to identify some subset of the encryption key using matrix techniques [1638]. There is also the meet-in-the-middle consistency attack [39,41]. The linear syndrome algorithm relies on being able to write a fragment of the output sequence as a linear equation [1636,1637]. There is the best affine approximation attack [502] and the derived sequence attack [42]. The techniques of differential cryptanalysis have even been applied to stream ciphers [501], as has linear cryptanalysis [631].

16.4 Stream Ciphers Using LFSRs

The basic approach to designing a keystream generator using LFSRs is simple. First you take one or more LFSRs, generally of different lengths and with different feedback polynomials. (If the lengths are all relatively prime and the feedback polynomials are all primitive, the whole generator is maximal length.) The key is the initial state of the LFSRs. Every time you want a bit, shift the LFSRs once (this is sometimes called clocking). The output bit is a function, preferably a nonlinear function, of some of the bits of the LFSRs. This function is called the combining function, and the whole generator is called a combination generator. (If the output bit is a function of a single LFSR, the generator is called a filter generator.) Much of the theoretical background for this kind of thing was laid down by Selmer and Neal Zierler [1647].

Complications have been added. Some generators have LFSRs clocked at different rates; sometimes the clocking of one generator depends on the output of another. These are all electronic versions of pre-WWII cipher machine ideas, and are called clock-controlled generators [641]. Clock control can be feedforward, where the output of one LFSR controls the clocking of another, or feedback, where the output of one LFSR controls its own clocking.

Although these generators are, at least in theory, susceptible to embedding and probabilistic correlation attacks [634,632], many are secure for now. Additional theory on clock-controlled shift registers is in [89].

Ian Cassells, once the head of pure mathematics at Cambridge and a former Bletchly Park cryptanalyst, said that "cryptography is a mixture of mathematics and muddle, and without the muddle the mathematics can be used against you." What he meant was that in stream ciphers, you need some kind of mathematical structure—such as a LFSR—to guarantee maximal-length and other properties, and then some complicated nonlinear muddle to stop someone from getting at the register and solving it. This advice also holds true for block algorithms.

What follows is a smattering of LFSR-based keystream generators that have appeared in the literature. I don't know if any of them have been used in actual cryptographic products. Most of them are of theoretical interest only. Some have been broken; some may still be secure.

Since LFSR-based ciphers are generally implemented in hardware, electronics logic symbols will be used in the figures. In the text, ⊕ is XOR, ^ is AND, ⊦ is OR, and ¬ is NOT.

Geffe Generator

This keystream generator uses three LFSRs, combined in a nonlinear manner (see Figure 16.6) [606]. Two of the LFSRs are inputs into a multiplexer, and the third LFSR controls the output of the multiplexer. If a1, a2, and a3 are the outputs of the three LFSRs, the output of the Geffe generator can be described by:

b = (a1 ^ a2) ⊕ ((¬ a1) ^ a3)

If the LFSRs have lengths n1, n2, and n3, respectively, then the linear complexity of the generator is

(n1 + 1)n2 + n1n3

The period of the generator is the least common multiple of the periods of the three generators. Assuming the degrees of the three primitive feedback polynomials are relatively prime, the period of this generator is the product of the periods of the three LFSRs.

Although this generator looks good on paper, it is cryptographically weak and falls to a correlation attack [829,1638]. The output of the generator equals the output of LFSR-2 75 percent of the time. So, if the feedback taps are known, you can guess the initial value for LFSR-2 and generate the output sequence of that register. Then you can count the number of times the output of the LFSR-2 agrees with the output of the generator. If you guessed wrong, the two sequences will agree about 50 percent of the time; if you guessed right, the two sequences will agree about 75 percent of the time.

Similarly, the output of the generator equals the output of LFSR-3 about 75 percent of the time. With those correlations, the keystream generator can be easily cracked. For example, if the primitive polynomials only have three terms each, and the largest LFSR is of length n, it only takes a segment of the output sequence 37n-bits long to reconstruct the internal states of all three LFSRs [1639].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Generalized Geffe Generator

Instead of choosing between two LFSRs, this scheme chooses between k LFSRs, as long as k is a power of 2. There are k + 1 LFSRs total (see Figure 16.7). LFSR-1 must be clocked log2k times faster than the other k LFSRs.

Figure 16.6 Geffe generator.

Even though this scheme is more complex than the Geffe generator, the same kind of correlation attack is possible. I don't recommend this generator.

Jennings Generator

This scheme uses a multiplexer to combine two LFSRs [778,779,780]. The multiplexer, controlled by LFSR-1, selects 1 bit of LFSR-2 for each output bit. There is also a function that maps the output of LFSR-2 to the input of the multiplexer (see Figure 16.8).

The key is the initial state of the two LFSRs and the mapping function. Although this generator has great statistical properties, it fell to Ross Anderson's meet-in-the-middle consistency attack [39] and the linear consistency attack [1638,442]. Don't use this generator.

Beth-Piper Stop-and-Go Generator

This generator, shown in Figure 16.9, uses the output of one LFSR to control the clock of another LFSR [151]. The clock input of LFSR-2 is controlled by the output of LFSR-1, so that LFSR-2 can change its state at time t only if the output of LFSR-1 was 1 at time t - 1.

No one has been able to prove results about this generator's linear complexity in the general case. However, it falls to a correlation attack [1639].

Alternating Stop-and-Go Generator

This generator uses three LFSRs of different length. LFSR-2 is clocked when the output of LFSR-1 is 1; LFSR-3 is clocked when the output of LFSR-1 is 0. The output of the generator is the XOR of LFSR-2 and LFSR-3 (see Figure 16.10) [673].

This generator has a long period and large linear complexity. The authors found a correlation attack against LFSR-1, but it does not substantially weaken the generator. There have been other attempts at keystream generators along these lines [1534,1574,1477].

Figure 16.7 Generalized Geffe generator.

Figure 16.8 Jennings generator.

Bilateral Stop-and-Go Generator

This generator uses two LFSRs, both of length n (see Figure 16.11) [1638]. The output of the generator is the XOR of the outputs of each LFSR. If the output of LFSR-2 at time t – 1 is 0 and the output at time t – 2 is 1, then LFSR-2 does not clock at time t. Conversely, if the output of LFSR-1 at time t – 1 is 0 and the output at t – 2 is 1, and if LFSR-1 clocked at time t, then LFSR-2 does not clock at time t.

The linear complexity of this system is roughly equal to the period. According to [1638], "no evident key redundancy has been observed in this system."

Threshold Generator

This generator tries to get around the security problems of the previous generators by using a variable number of LFSRs [277]. The theory is that if you use a lot of LFSRs, it's harder to break the cipher.

This generator is illustrated in Figure 16.12. Take the output of a large number of LFSRs (use an odd number of them). Make sure the lengths of all the LFSRs are relatively prime and all the feedback polynomials are primitive: maximize the period. If more than half the output bits are 1, then the output of the generator is 1. If more than half the output bits are 0, then the output of the generator is 0.

Figure 16.9 Beth-Piper stop-and-go generator.

Figure 16.10 Alternating stop-and-go generator.

With three LFSRs, the output generator can be written as:

b = (a1 ^ a2) ⊕ (a1 ^ a3) ⊕ (a2 ^ a3)

This is very similar to the Geffe generator, except that it has a larger linear complexity of

n1n2 + n1n3 + n2n3

where n1, n2, and n3 are the lengths of the first, second, and third LFSRs.

This generator isn't great. Each output bit of the generator yields some information about the state of the LFSRs—0.189 bit to be exact—and the whole thing falls to a correlation attack. I don't recommend using it.

Self-Decimated Generators

Self-decimated generators are generators that control their own clock. Two have been proposed, one by Rainer Rueppel (see Figure 16.13) [1359] and another by Bill Chambers and Dieter Gollmann [308] (see Figure 16.14). In Rueppel's generator, when the output of the LFSR is 0, the LFSR is clocked d times. When the output of the LFSR is 1, the LFSR is clocked k times. Chambers's and Gollmann's generator is more complicated, but the idea is the same. Unfortunately, both generators are insecure [1639], although some modifications have been proposed that may correct the problems [1362].

Figure 16.11 Bilateral stop-and-go generator.

Figure 16.12 Threshold generator.

Multispeed Inner-Product Generator

This generator, by Massey and Rueppel [1014], uses two LFSRs clocked at two different speeds (see Figure 16.15). LFSR-2 is clocked d times as fast as LFSR-1. The individual bits of the two LFSRs are ANDed together and then XORed with each other to produce the final output bit of the generator.

Although this generator has high linear complexity and it possesses excellent statistical properties, it still falls to a linear consistency attack [1639]. If n1 is the length of LFSR-1, n2 is the length of the LFSR-2, and d is the speed multiple between the two, then the internal state of the generator can be recovered from an output sequence of length

n1 + n2 + log2d

Summation Generator

More work by Rainer Rueppel, this generator adds the output of two LFSRs (with carry) [1358,1357]. This operation is highly nonlinear. Through the late 1980s, this generator was the security front-runner, but it fell to a correlation attack [1053,1054,1091]. And it has been shown that this is an example of a feedback with carry shift register (see Section 17.4), and can be broken [844].

Figure 16.13 Rueppel's self-decimated generator.

Figure 16.14 Chambers's and Gollmann's self-decimated generator.

DNRSG

That stands for "dynamic random-sequence generator" [1117]. The idea is to have two different filter generators—threshold, summation, or whatever—fed by a single set of LFSRs and controlled by another LFSR.

First clock all the LFSRs. If the output of LFSR-0 is 1, then compute the output of the first filter generator. If the output of LFSR-0 is 0, then compute the output of the second filter generator. The final output is the first output XOR the second.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Gollmann Cascade

The Gollmann cascade (see Figure 16.16), described in [636,309], is a strengthened version of a stop-and-go generator. It consists of a series of LFSRs, with the clock of each controlled by the previous LFSR. If the output of LFSR-1 is 1 at time t - 1, then LFSR-2 clocks. If the output of LFSR-2 is 1 at time t - 1, then LFSR-3 clocks, and so on. The output of the final LFSR is the output of the generator. If all the LFSRs have the same length, n, the linear complexity of a system with k LFSRs is

n(2n - 1)k-1

Figure 16.15 Multispeed inner-product generator.

Cascades are a cool idea: They are conceptually very simple and they can be used to generate sequences with huge periods, huge linear complexities, and good statistical properties. They are vulnerable to an attack called lock-in [640]. This is a technique by which a cryptanalyst reconstructs the input to the last shift register in the cascade, then proceeds to break the cascade register by register. This is a serious problem in some situations and weakens the effective key length of the algorithm, but precautions can be taken to minimize the attack.

Further analysis has indicated that the sequence approaches random as k gets larger [637,638,642,639]. Based on recent attacks on short Gollmann cascades [1063], I recommend using a k of at least 15. You're better off using more LFSRs of shorter length than fewer LFSRs of longer length.

Shrinking Generator

The shrinking generator [378] uses a different form of clock control than the previous generators. Take two LFSRs: LFSR-1 and LFSR-2. Clock both of them. If the output of LFSR-1 is 1, then the output of the generator is LFSR-2. If the output of LFSR-1 is 0, discard the two bits, clock both LFSRs, and try again.

This idea is simple, reasonably efficient, and looks secure. If the feedback polynomials are sparse, the generator is vulnerable, but no other problems have been found. Even so, it's new. One implementation problem is that the output rate is not regular; if LFSR-1 has a long string of zeros then the generator outputs nothing. The authors suggest buffering to solve this problem [378]. Practical implementation of the shrinking generator is discussed in [901].

Self-Shrinking Generator

The self-shrinking generator [1050] is a variant of the shrinking generator. Instead of using two LFSRs, use pairs of bits from a single LFSR. Clock a LFSR twice. If the first bit in the pair is 1, the output of the generator is the second bit. If the first bit is 0, discard both bits and try again. While the self-shrinking generator requires about half the memory space as the shrinking generator, it is also half the speed.

While the self-shrinking generator also seems secure, it still has some unexplained behavior and unknown properties. This is a very new generator; give it some time.

Figure 16.16 Gollmann cascade.

16.5 A5

A5 is the stream cipher used to encrypt GSM (Group Special Mobile). That's the non-American standard for digital cellular mobile telephones. It is used to encrypt the link from the telephone to the base station. The rest of the link is unencrypted; the telephone company can easily eavesdrop on your conversations.

A lot of strange politics surrounds this one. Originally it was thought that GSM's cryptography would prohibit export of the phones to some countries. Now some officials are discussing whether A5 might harm export sales, implying that it is so weak as to be an embarrassment. Rumor has it that the various NATO intelligence agencies had a catfight in the mid-1980s over whether GSM encryption should be strong or weak. The Germans wanted strong cryptography, as they were sitting near the Soviet Union. The other countries overruled them, and A5 is a French design.

We know most of the details. A British telephone company gave all the documentation to Bradford University without remembering to get them to sign a nondisclosure agreement. It leaked here and there, and was eventually posted to the Internet. A paper describing A5 is [1622]; there is also code at the back of this book.

A5 consists of three LFSRs; the register lengths are 19, 22, and 23; all the feedback polynomials are sparse. The output is the XOR of the three LFSRs. A5 uses variable clock control. Each register is clocked based on its own middle bit, XORed with the inverse threshold function of the middle bits of all three registers. Usually, two of the LFSRs clock in each round.

There is a trivial attack requiring 240 encryptions: Guess the contents of the first two LFSRs, then try to determine the third LFSR from the keystream. (Whether this attack is actually feasible is under debate, but a hardware key search machine currently under design should resolve the matter soon [45].)

Nonetheless, it is becoming clear that the basic ideas behind A5 are good. It is very efficient. It passes all known statistical tests; its only known weakness is that its registers are short enough to make exhaustive search feasible. Variants of A5 with longer shift registers and denser feedback polynomials should be secure.

16.6 Hughes XPD/KPD

This algorithm is brought to you by Hughes Aircraft Corp. They put it in army tactical radios and direction-finding equipment for sale to foreign militaries. It was designed in 1986 and called XPD, for Exportable Protection Device. Later it was renamed KPD—Kinetic Protection Device—and declassified [1037,1036].

The algorithm uses a 61-bit LFSR. There are 210 different primitive feedback polynomials, which were approved by the NSA. The key selects one of these polynomials (they are all stored in ROM somewhere), as well as the initial state of the LFSR.

It has eight different nonlinear filters, each of which has six taps from the LFSR and which produces 1 bit. The bits combine to generate a byte, which is used to encrypt or decrypt the datastream.

This algorithm looks pretty impressive, but I doubt it is. The NSA allows export, so there must be some attack on the order of 240 or less. What is it?

16.7 Nanoteq

Nanoteq is a South African electronics company. This is their algorithm that has been fielded by the South African police to encrypt their fax transmissions, and presumably for other uses as well.

The algorithm is described, more or less, in [902,903]. It uses a 127-bit LFSR with a fixed feedback polynomial; the key is the initial state of the feedback register. The 127 bits of the register are reduced to a single keystream bit using 25 primitive cells. Each cell has five inputs and one output:

f(x1,x2,x3,x4,x5) = x1 + x2 + (x1 + x3) (x2 + x4 + x5) + (x1 + x4) (x2 + x3) + x5

Each input of the function is XORed with some bit of the key. There is also a secret permutation that depends on the particular implementation, and is not detailed in the papers. This algorithm is only available in hardware.

Is this algorithm secure? I doubt it. During the transition to majority rule, embarrassing faxes from one police station to another would sometimes turn up in the liberal newspapers. These could easily have been the results of U.S., U.K., or Soviet intelligence efforts. Ross Anderson took some initial steps towards cryptanalyzing this algorithm in [46]; I expect more results to come soon.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

16.8 Rambutan

Rambutan is a British algorithm, designed by the Communications Electronics Security Group (one of the aliases used by GCHQ). It is only sold as a hardware module and is approved for the protection of classified material up to "Confidential." The algorithm itself is secret, and the chip is not generally commercially available.

Rambutan has a 112-bit key (plus parity bits) and can operate in three modes: ECB, CBC, and 8-bit CFB. This strongly indicates that it is a block algorithm, but rumors point elsewhere. Supposedly, it is a LFSR stream cipher. It has five shift registers, each one of a different length around 80 bits. The feedback polynomials are fairly sparse, with only about 10 taps each. Each shift register provides four inputs to a very large and complex nonlinear function which eventually spits out a single bit.

Why call it Rambutan? Perhaps, like the fruit, it's spiny and forbidding on the outside but soft and yielding inside. On the other hand, maybe that's not the reason.

16.9 Additive Generators

Additive generators (sometimes called lagged Fibonacci generators) are extremely efficient because they produce random words instead of random bits [863]. They are not secure on their own, but can be used as building blocks for secure generators.

The initial state of the generator is an array of n-bit words: 8-bit words, 16-bit words, 32-bit words, whatever: X1, X2, X3,..., Xm. This initial state is the key. The ith word of the generator is

Xi = (Xi-a + Xi-b + Xi-c +...+ Xi-m) mod 2n

If the coefficients a, b, c,..., m are chosen right, the period of this generator is at least 2n - 1. One of the requirements on the coefficients is that the least significant bit forms a maximal-length LFSR.

For example, (55,24,0) is a primitive polynomial mod 2 from Table 16.2. This means that the following additive generator is maximal length.

Xi = (Xi-55 + Xi-24) mod 2n

This works because the primitive polynomial has three coefficients. If it has more than three, you need some additional requirements to make it maximal length. See [249] for details.

Fish

Fish is an additive generator based on techniques used in the shrinking generator [190]. It produces a stream of 32-bit words which can be XORed with a plaintext stream to produce ciphertext, or XORed with a ciphertext stream to produce plaintext. The algorithm is named as it is because it is a Fibonacci shrinking generator.

First, use these two additive generators. The key is the initial values of these generators.

Ai = (Ai-55 + Ai-24) mod 232
Bi = (Bi-52 + Bi-19) mod 232

These sequences are shrunk, as a pair, depending on the least significant bit of Bi: if it is 1, use the pair; if it is 0, ignore the pair. Cj is the sequence of used words from Ai, and Dj is the sequence of used words from Bi. These words are used in pairs—C2j, C2j+1, D2j, and D2j+1—to generate two 32-bit output words: K2j and K2j+1.

E2j = C2j ⊕ (D2j ^ D2j+1)
F2j = D2j+1 ^ (E2j ^ C2j+1)
K2j = E2j ⊕ F2j
K2i+1 = C2i+1 ⊕ F2j

This algorithm is fast. On a 33 megahertz 486, a C implementation of Fish encrypts data at 15 megabits per second. Unfortunately, it is also insecure; an attack has a work factor of about 240 [45].

Pike

Pike is a leaner, meaner version of Fish, brought to you by Ross Anderson, the man who broke Fish [45]. It uses three additive generators. For example:

Ai = (Ai-55 + Ai-24) mod 232
Bi = (Bi-57 + Bi-7) mod 232
Ci = (Ci-58 + Ci-19) mod 232

To generate the keystream word, look at the addition carry bits. If all three agree (all are 0 or all are 1), then clock all three generators. If they do not, just clock the two generators that agree. Save the carry bits for next time. The final output is the XOR of the three generators.

Pike is faster than Fish, since on the average 2.75 steps will be required per output rather than 3. It is far too new to trust, but looks good so far.

Mush

Mush is a mutual shrinking generator. It's easy to explain [1590]. Take two additive generators: A and B. If the carry bit of A is set, clock B. If the carry bit of B is set, clock A. Clock A, and set the carry bit if there is a carry. Clock B, and set the carry bit if there is a carry. The final output is the XOR of the output of A and B.

The easiest generators to use are the ones from Fish:

Ai = (Ai-55 + Ai-24) mod 232
Bi = (Bi-52 + Bi-19) mod 232

On the average, three generator iterations are required to produce one output word. And if the coefficients of the additive generators are chosen correctly and are relatively prime, the output sequence will be maximal length. I know of no successful attacks, but remember that this algorithm is very new.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

16.10 Gifford

David Gifford invented a stream cipher and used it to encrypt news wire reports in the Boston area from 1984 until 1988 [608,607,609]. The algorithm has a single 8-byte register: b0, b1,..., b7. The key is the initial state of the register. The algorithm works in OFB; the plaintext does not affect the algorithm at all. (See Figure 16.17).

To generate a key byte ki, concatenate b0 and b2 and concatenate b4 and b7. Multiply the two together to get a 32-bit number. The third byte from the left is ki.

To update the register, take b1 and sticky right shift it 1 bit. This means the left-most bit is both shifted and also remains in place. Take b7 and shift it 1 bit to the left; there should be a 0 in the right-most bit position. Take the XOR of the modified b1, the modified b7, and b0. Shift the original register 1 byte to the right and put this byte in the left-most position.

This algorithm remained secure throughout its life, but was broken in 1994 [287]. It turns out that the feedback polynomial isn't primitive and can be attacked that way—oops.

Figure 16.17 Gifford.

16.11 Algorithm M

The name is from Knuth [863]. It's a method for combining multiple pseudo-random streams that increases their security. One generator's output is used to select a delayed output from the other generator [996,1003]. In C:

#define ARR_SIZE (8192) /* for example — the larger the better
*/

static unsigned char delay[ ARR_SIZE ] ;

unsigned char prngA( void ) ;
long prngB( void ) ;

void init_algM( void )
{
  long i ;

  for ( i = 0 ; i < ARR_SIZE ; i++ )
    delay = prngA() ;

} /* init_algM */

unsigned char algM( void )
{
  long j,v ;
  j = prngB() % ARR_SIZE ;    /* get the delay[] index */
  v = delay[j] ;         /* get the value to return */
  delay[j] = prngA() ;        /* replace it */

  return ( v ) ;
} /* algM */

This has strength in that if prngA were truly random, one could not learn anything about prngB (and could therefore not cryptanalyze it). If prngA were of the form that it could be cryptanalyzed only if its output were available in order (i.e., only if prngB were cryptanalyzed first) and otherwise it was effectively truly random, then the combination would be secure.

16.12 PKZIP

Roger Schlafly designed the encryption algorithm built into the PKZIP data compression program. It's a stream cipher that encrypts data one byte at a time. At least, this is the algorithm in version 2.04g. I can't speak for later versions, but unless there is some announcement you can probably assume that they are identical.

The algorithm uses three 32-bit variables, initialized as follows:

K0 = 305419896
K1 = 591751049
K2 = 878082192

It has an 8-bit key, K3, derived from K2. Here is the algorithm (all symbols are standard C notation):

Ci = Pi ^ K3
K0 = crc32 (K0, Pi)
K1 = K1 + (K0 & 0×000000ff)
K1 = K1 * 134775813 + 1
K2 = crc32 (K2, K1 >> 24)
K3 = ((K2 | 2) * ((K2 | 2) ^ 1)) >> 8

The function crc32 takes the previous value and a byte, XORs them, and calculates the next value by the CRC polynomial denoted by 0×edb88320. In practice, a 256-entry table can be precomputed and the crc32 calculation becomes:

crc32 (a, b) = (a >> 8) ^ table [(a & 0×ff) ⊕ b]

The table is precomputed by the original definition of crc32:

table [i] = crc32 (i, 0)

To encrypt a plaintext stream, first loop the key bytes through the encryption algorithm to update the keys. Ignore the ciphertext output in this step. Then encrypt the plaintext, one byte at a time. Twelve random bytes are prepended to the plaintext, but that's not really important. Decryption is similar to encryption, except that Ci is used in the second step of the algorithm instead of Pi.

Security of PKZIP

Unfortunately, it's not that great. An attack requires 40 to 200 bytes of known plaintext and has a time complexity of about 227 [166]. You can do it in a few hours on your personal computer. If the compressed file has any standard headers, getting the known plaintext is no problem. Don't use the built-in encryption in PKZIP.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 17
Other Stream Ciphers and Real Random-Sequence Generators

17.1 RC4

RC4 is a variable-key-size stream cipher developed in 1987 by Ron Rivest for RSA Data Security, Inc. For seven years it was proprietary, and details of the algorithm were only available after signing a nondisclosure agreement.

In September, 1994 someone posted source code to the Cypherpunks mailing list—anonymously. It quickly spread to the Usenet newsgroup sci.crypt, and via the Internet to ftp sites around the world. Readers with legal copies of RC4 confirmed compatibility. RSA Data Security, Inc. tried to put the genie back into the bottle, claiming that it was still a trade secret even though it was public; it was too late. It has since been discussed and dissected on Usenet, distributed at conferences, and taught in cryptography courses.

RC4 is simple to describe. The algorithm works in OFB: The keystream is independent of the plaintext. It has a 8 * 8 S-box: S0, S1,..., S255. The entries are a permutation of the numbers 0 through 255, and the permutation is a function of the variable-length key. It has two counters, i and j, initialized to zero.

To generate a random byte, do the following:

i = (i + 1) mod 256
j = (j + Si) mod 256
swap Si and Sj
t = (Si + Sj) mod 256
K = St

The byte K is XORed with the plaintext to produce ciphertext or XORed with the ciphertext to produce plaintext. Encryption is fast—about 10 times faster than DES.

Initializing the S-box is also easy. First, fill it linearly: S0 = 0, S1 = 1,..., S255 = 255. Then fill another 256-byte array with the key, repeating the key as necessary to fill the entire array: K0, K1,..., K255. Set the index j to zero. Then:

for i = 0 to 255:
j = (j + Si + Ki) mod 256
swap Si and Sj

And that's it. RSADSI claims that the algorithm is immune to differential and linear cryptanalysis, doesn't seem to have any small cycles, and is highly nonlinear. (There are no public cryptanalytic results. RC4 can be in about 21700 (256! × 2562) possible states: an enormous number.) The S-box slowly evolves with use: i ensures that every element changes and j ensures that the elements change randomly. The algorithm is simple enough that most programmers can quickly code it from memory.

It should be possible to generalize this idea to larger S-boxes and word sizes. The previous version is 8-bit RC4. There's no reason why you can't define 16-bit RC4 with a 16 * 16 S-box (100K of memory) and a 16-bit word. You'd have to iterate the initial setup a lot more times—65,536 to keep with the stated design—but the resulting algorithm should be faster.

RC4 has special export status if its key length is 40 bits or under (see Section 13.8). This special export status has nothing to do with the secrecy of the algorithm, although RSA Data Security, Inc. has hinted for years that it does. The name is trademarked, so anyone who writes his own code has to call it something else. Various internal documents by RSA Data Security, Inc. have not yet been made public [1320,1337].

So, what's the deal with RC4? It's no longer a trade secret, so presumably anyone can use it. However, RSA Data Security, Inc. will almost certainly sue anyone who uses unlicensed RC4 in a commercial product. They probably won't win, but they will certainly make it cheaper for a company to license than fight.

RC4 is in dozens of commercial cryptography products, including Lotus Notes, Apple Computer's AOCE, and Oracle Secure SQL. It is part of the Cellular Digital Packet Data specification [37].

17.2 SEAL

SEAL is a software-efficient stream cipher designed at IBM by Phil Rogaway and Don Coppersmith [1340]. The algorithm was optimized for 32-bit processors: To run well it needs eight 32-bit registers and a cache of a few kilobytes. Using a relatively slow operation, SEAL preprocesses the key operation into a set of tables. These tables are then used to speed up encryption and decryption.

Pseudo-random Function Family

One novel feature of SEAL is that is isn't really a traditional stream cipher: it is a pseudo-random function family. Given a 160-bit key k, and a 32-bit n, SEAL stretches n into an L-bit string k(n). L can take any value less than 64 kilobytes. SEAL is supposed to enjoy the property that if k is selected at random, then k(n) should be computationally indistinguishable from a random L-bit function of n.

The practical effect of SEAL being a pseudo-random function family is that it is useful in applications where traditional stream ciphers are not. With most stream ciphers you generate a sequence of bits in one direction: Knowing the key and a position i, the only way to determine the ith bit generated is to generate all the bits up until the ith one. But a pseudo-random function family is different: You get easy access at any desired position in the key stream. This is very useful.

Imagine you need to secure a hard drive. You want to encrypt each and every 512-byte sector. With a pseudo-random function family like SEAL, you can encrypt the contents of sector n by XORing it with k(n). It is as though the entire disk is XORed with a long pseudo-random string, where any piece of that long string can be computed without any trouble.

A pseudo-random function family also simplifies the synchronization problem encountered with standard stream ciphers. Suppose you send encrypted messages over a channel that sometimes drops messages. With a pseudo-random function family, you can encrypt under k the nth message you transmit, xn, as n together with the XOR of xn and k(n). The receiver doesn't need to store any state to recover xn, nor does he need to worry about lost messages affecting the message decryption process.

Description of SEAL

The inner loop of SEAL is shown by Figure 17.1. Three key-derived tables, called R, S, and T, drive the algorithm. The preprocessing step maps the key k, to these tables using a procedure based on SHA (see Section 18.7). The 2-kilobyte table, T, is a 9 * 32 bit S-box.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

SEAL also uses four 32-bit registers, A, B, C, and D, whose initial values are determined by n and the k-derived tables R and T. These registers get modified over several iterations, each one involving 8 rounds. In each round 9 bits of a first register (either A, B, C, or D) are used to index into table T. The value retrieved from T is then added to or XORed with the contents of a second register: again one of A, B, C, or D. The first register is then circularly shifted by nine positions. In some rounds the second register is further modified by adding or XORing it with the (now shifted) first register. After 8 rounds of this, A, B, C, and D are added to the keystream, each masked first by adding or XORing it with a certain word from S. The iteration is completed by adding to A and C additional values dependent on n, n1, n2, n3, n4; exactly which one depends on the parity of the iteration number.

The important ideas in this design seem to be:

1. Use a large, secret, key-derived S-box (T).
2. Alternate arithmetic operations which don't commute (addition and XOR).
3. Use an internal state maintained by the cipher which is not directly manifest in the data stream (the ni values which modify A and C at the end of each iteration).
4. Vary the round function according to the round number, and vary the iteration function according to the iteration number.

Figure 17.1 The inner loop of SEAL.

SEAL requires about five elementary machine operations to encrypt each byte of text. It runs at 58 megabits per second on a 50 megahertz 486 machine. This is probably the fastest software algorithm in the book.

On the other hand, SEAL must preprocess its key into internal tables. These tables total roughly 3 kilobytes in size, and their calculation takes about 200 SHA computations. Thus, SEAL is not appropriate to use in situations where you don't have the time to perform the key setup or you don't have the memory to store the tables.

Security of SEAL

SEAL is a new algorithm and has yet to be subjected to any published cryptanalysis. This suggests caution. However, SEAL seems to be well thought through. Its peculiarities do, in the end, make a good deal of sense. And Don Coppersmith is generally regarded as the world's cleverest cryptanalyst.

Patents and Licenses

SEAL is being patented [380]. Anyone wishing to license SEAL should contact the Director of Licenses, IBM Corporation, 500 Columbus Ave., Thurnwood, NY, 10594.

17.3 WAKE

WAKE is the Word Auto Key Encryption algorithm, invented by David Wheeler [1589]. It produces a stream of 32-bit words which can be XORed with a plaintext stream to produce ciphertext, or XORed with a ciphertext stream to produce plaintext. And it's fast.

WAKE works in CFB; the previous ciphertext word is used to generate the next key word. It also uses an S-box of 256 32-bit values. This S-box has a special property: The high-order byte of all the entries is a permutation of all possible bytes, and the low-order 3 bytes are random.

First, generate the S-box entries, Si, from the key. Then initialize four registers with the key (or with another key): a0, b0, c0, and d0. To generate a 32-bit keystream word, Ki:

Ki = di

The ciphertext word Ci, is the plaintext word, Pi XORed with Ki.

Then, update the four registers:

ai+1 = M(ai,di)
bi+1 = M(bi,ai+1)
ci+1 = M(ci,bi+1)
di+1 = M(di,ci+1)

Function M is

M(x,y) = (x + y) >> 8 ⊕ S(x+y) ^ 255

This is shown in Figure 17.2. The operation >> is a right shift, not a rotation. The low-order 8 bits of x + y are the input into the S-box. Wheeler gives a procedure for generating the S-box, but it isn't really complete. Any algorithm to generate random bytes and a random permutation will work.

Figure 17.2 Wake.

WAKE's biggest asset is that it is fast. However, it's insecure against a chosen-plaintext or chosen-ciphertext attack. It is being used in the current version of Dr. Solomon's Anti-Virus program.

17.4 Feedback with Carry Shift Registers

A feedback with carry shift register, or FCSR, is similar to a LFSR. Both have a shift register and a feedback function; the difference is that a FCSR also has a carry register (see Figure 17.3). Instead of XORing all the bits in the tap sequence, add the bits together and add in the contents of the carry register. The result mod 2 becomes the new bit. The result divided by 2 becomes the new content of the carry register.

Figure 17.4 is an example of a 3-bit FCSR tapped at the first and second bit. Its initial value is 001, and the initial contents of the carry register is 0. The output bit is the right-most bit of the shift register.

Shift Register | Carry Register
0 0 1 | 0
1 0 0 | 0
0 1 0 | 0
1 0 1 | 0
1 1 0 | 0
1 1 1 | 0
0 1 1 | 1
1 0 1 | 1
0 1 0 | 1
0 0 1 | 1
0 0 0 | 1
1 0 0 | 0

Figure 17.3 Feedback with carry shift register.

Note that the final internal state (including the contents of the carry register) is the same as the second internal state. The sequence cycles at this point, and has a period of 10.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

There are a few things to note here. First, the carry register is not a single bit; it is a number. The size of the carry register must be at least log2t, where t is the number of taps. There are only two taps in the previous example, so the carry register only has to be 1 bit wide. If there were four taps, the carry register would have to be 2 bits wide, and could be either 0, 1, 2, or 3.

Second, there is an initial transient before the FCSR settles down into its repeating period. In the previous example, only one state never repeated. For larger and more complicated FCSRs, there may be more.

Third, the maximum period of a FCSR is not 2n - 1, where n is the length of the shift register. The maximum period is q - 1, where q is the connection integer. This number gives the taps and is defined by:

q = 2q1 + 22q2 + 24q4 +...+ 2nqn - 1

(Yes, the qis are numbered from left to right.) And even worse, q has to be a prime for which 2 is a primitive root. The rest of this discussion assumes q is of this form.

In this example, q = 2*0 + 4*1 + 8*1 - 1 = 11. And 11 is a prime with 2 as a primitive root. So the maximum period is 10.

Not all initial states give you the maximum period. For example, look at the FCSR when the initial value is 101 and the carry register is set to 4.

Figure 17.4 3-bit FCSR.

Shift Register | Carry Register
1 0 1 | 4
1 1 0 | 2
1 1 1 | 1
1 1 1 | 1

At this point the register spits out a neverending stream of 1s.

Any initial state will result in one of four things. First, it is part of the maximum period. Second, it will fall into the maximum period after an initial transient. Third, it will fall into a sequence of all zeros after an initial transient. Fourth, it will fall into a sequence of all ones after an initial transient.

There is a mathematical formula for determining what will happen to a given initial state, but it's much easier to just test it. Run the FCSR for a while. (If m is the initial memory, and t is the number of taps, then log2(t) + log2(m) + 1 steps are enough.) If it degenerates into a neverending stream of 0s or 1s within n bits, where n is the length of the FCSR, don't use it. If it doesn't, then use it. Since the initial state of a FCSR corresponds to the key of the stream cipher, this means that a FCSR-based generator will have a set of weak keys.

Table 17.1 lists all connection integers less than 10,000 for which 2 is a primitive root. These all have maximum period q - 1. To turn one of these numbers into a tap sequence, calculate the binary expansion of q + 1. For example, 9949 would translate to taps on bits 1, 2, 3, 4, 6, 7, 9, 10, and 13, because

9950 = 213 + 210 + 29 + 27 + 26 + 24 + 23 + 22 + 21

Table 17.2 lists all the 4-tap tap sequences that result in a maximal-length FCSR for shift register lengths of 32 bits, 64 bits, and 128 bits. Each of the four values, a, b, c, and d, combine to generate q, a prime for which 2 is primitive.

q = 2a + 2b + 2c + 2d - 1

Any of these tap sequences can be used to create a FCSR with period q - 1.

The idea of using FCSRs for cryptography is still very new; it is being pioneered by Andy Klapper and Mark Goresky [844,845,654,843,846]. Just as the analysis of LFSRs is based on the addition of primitive polynomials mod 2, analysis of FCSRs is based on addition over something called the 2-adic numbers. The theory is well beyond the scope of this book, but there seems to be a 2-adic analog for everything. Just as you can define linear complexity, you can define 2-adic complexity. There is even a 2-adic analog to the Berlekamp-Massey algorithm. What this means is that the list of potential stream ciphers has just doubled—at least. Anything you can do with a LFSR you can do with a FCSR.

There are further enhancements to this sort of idea, ones that involve multiple carry registers. The analysis of these sequence generators is based on addition over the ramified extensions of the 2-adic numbers [845,846].

17.5 Stream Ciphers Using FCSRs

There aren't any FCSR stream ciphers in the literature; the theory is still too new. In the interests of getting the ball rolling, I propose some here. I am taking two different tacks: I am proposing FCSR stream ciphers that are identical to previously proposed LFSR generators, and I am proposing stream ciphers that use both FCSRs and LFSRs. The security of the former can probably be analyzed using 2-adic numbers; the latter cannot be analyzed using algebraic techniques—they can probably only be analyzed indirectly. In any case, it is important to choose LFSRs and FCSRs whose periods are relatively prime.

All this will come later. Right now I know of no implementation or analysis of any of these ideas. Wait some years and scan the literature before you trust any of them.

Cascade Generators

There are two ways to use FCSRs in a cascade generator:

— FCSR Cascade. The Gollmann cascade with FCSRs instead of LFSRs.
— LFSR/FCSR Cascade. The Gollmann cascade with the generators alternating between LFSRs and FCSRs.

FCSR Combining Generators

These generators use a variable number of LFSRs and/or FCSRs, and a variety of functions to combine them. The XOR operation destroys the algebraic properties of FCSRs, so it makes sense to use it to combine them. The generator, shown in Figure 17.5, uses a variable number of FCSRs. Its output is the XOR of the outputs of the individual FCSRs.

Other generators along similar lines are:

— FCSR Parity Generator. All registers are FCSRs and the combining function is XOR.
— LFSR/FCSR Parity Generator. Registers are a mix of LFSRs and FCSRs and the combining function is XOR.
— FCSR Threshold Generator. All registers are FCSRs and the combining function is the majority function.
— LFSR/FCSR Threshold Generator. Registers are a mix of LFSRs and FCSRs and the combining function is the majority function.
— FCSR Summation Generator. All registers are FCSRs and the combining function is addition with carry.
— LFSR/FCSR Summation Generator. Registers are a mix of LFSRs and FCSRs and the combining function is addition with carry.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 17.1 Connection Integers for Maximal-period FCSRs
2 | 653 | 1549 | 2477 | 3539
5 | 659 | 1571 | 2531 | 3547
11 | 661 | 1619 | 2539 | 3557
13 | 677 | 1621 | 2549 | 3571
19 | 701 | 1637 | 2557 | 3581
29 | 709 | 1667 | 2579 | 3613
37 | 757 | 1669 | 2621 | 3637
53 | 773 | 1693 | 2659 | 3643
59 | 787 | 1733 | 2677 | 3659
61 | 797 | 1741 | 2683 | 3677
67 | 821 | 1747 | 2693 | 3691
83 | 827 | 1787 | 2699 | 3701
101 | 829 | 1861 | 2707 | 3709
107 | 853 | 1867 | 2741 | 3733
131 | 859 | 1877 | 2789 | 3779
139 | 877 | 1901 | 2797 | 3797
149 | 883 | 1907 | 2803 | 3803
163 | 907 | 1931 | 2819 | 3851
173 | 941 | 1949 | 2837 | 3853
179 | 947 | 1973 | 2843 | 3877
181 | 1019 | 1979 | 2851 | 3907
197 | 1061 | 1987 | 2861 | 3917
211 | 1091 | 1997 | 2909 | 3923
227 | 1109 | 2027 | 2939 | 3931
269 | 1117 | 2029 | 2957 | 3947
293 | 1123 | 2053 | 2963 | 3989
317 | 1171 | 2069 | 3011 | 4003
347 | 1187 | 2083 | 3019 | 4013
349 | 1213 | 2099 | 3037 | 4019
373 | 1229 | 2131 | 3067 | 4021
379 | 1237 | 2141 | 3083 | 4091
389 | 1259 | 2213 | 3187 | 4093
419 | 1277 | 2221 | 3203 | 4099
421 | 1283 | 2237 | 3253 | 4133
443 | 1291 | 2243 | 3299 | 4139
461 | 1301 | 2267 | 3307 | 4157
467 | 1307 | 2269 | 3323 | 4219
491 | 1373 | 2293 | 3347 | 4229
509 | 1381 | 2309 | 3371 | 4243
523 | 1427 | 2333 | 3413 | 4253
541 | 1451 | 2339 | 3461 | 4259
547 | 1453 | 2357 | 3467 | 4261
557 | 1483 | 2371 | 3469 | 4283
563 | 1493 | 2389 | 3491 | 4349
587 | 1499 | 2437 | 3499 | 4357
613 | 1523 | 2459 | 3517 | 4363
619 | 1531 | 2467 | 3533 | 4373
4397 | 5693 | 6781 | 7717 | 8861
4451 | 5701 | 6803 | 7757 | 8867
4483 | 5717 | 6827 | 7789 | 8923
4493 | 5741 | 6829 | 7829 | 8933
4507 | 5749 | 6869 | 7853 | 8963
4517 | 5779 | 6883 | 7877 | 8971
4547 | 5813 | 6899 | 7883 | 9011
4603 | 5827 | 6907 | 7901 | 9029
4621 | 5843 | 6917 | 7907 | 9059
4637 | 5851 | 6947 | 7933 | 9173
4691 | 5869 | 6949 | 7949 | 9181
4723 | 5923 | 6971 | 8053 | 9203
4787 | 5939 | 7013 | 8069 | 9221
4789 | 5987 | 7019 | 8093 | 9227
4813 | 6011 | 7027 | 8117 | 9283
4877 | 6029 | 7043 | 8123 | 9293
4933 | 6053 | 7069 | 8147 | 9323
4957 | 6067 | 7109 | 8171 | 9341
4973 | 6101 | 7187 | 8179 | 9349
4987 | 6131 | 7211 | 8219 | 9371
5003 | 6173 | 7219 | 8221 | 9397
5011 | 6197 | 7229 | 8237 | 9419
5051 | 6203 | 7237 | 8243 | 9421
5059 | 6211 | 7243 | 8269 | 9437
5077 | 6229 | 7253 | 8291 | 9467
5099 | 6269 | 7283 | 8293 | 9491
5107 | 6277 | 7307 | 8363 | 9533
5147 | 6299 | 7331 | 8387 | 9539
5171 | 6317 | 7349 | 8429 | 9547
5179 | 6323 | 7411 | 8443 | 9587
5189 | 6373 | 7451 | 8467 | 9613
5227 | 6379 | 7459 | 8539 | 9619
5261 | 6389 | 7477 | 8563 | 9629
5309 | 6397 | 7499 | 8573 | 9643
5333 | 6469 | 7507 | 8597 | 9661
5387 | 6491 | 7517 | 8627 | 9677
5443 | 6547 | 7523 | 8669 | 9733
5477 | 6619 | 7541 | 8677 | 9749
5483 | 6637 | 7547 | 8693 | 9803
5501 | 6653 | 7549 | 8699 | 9851
5507 | 6659 | 7573 | 8731 | 9859
5557 | 6691 | 7589 | 8741 | 9883
5563 | 6701 | 7603 | 8747 | 9901
5573 | 6709 | 7621 | 8803 | 9907
5651 | 6733 | 7643 | 8819 | 9923
5659 | 6763 | 7669 | 8821 | 9941
5683 | 6779 | 7691 | 8837 | 9949


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 17.2 Tap Sequences for Maximal-length FCSRs
(32, 6, 3, 2) | (64, 24, 19, 2) | (64, 59, 28, 2) | (96, 55, 53, 2)
(32, 7, 5, 2) | (64, 25, 3, 2) | (64, 59, 38, 2) | (96, 56, 9, 2)
(32, 8, 3, 2) | (64, 25, 4, 2) | (64, 59, 44, 2) | (96, 56, 51, 2)
(32, 13, 8, 2) | (64, 25, 11, 2) | (64, 60, 49, 2) | (96, 57, 3, 2)
(32, 13, 12, 2) | (64, 25, 19, 2) | (64, 61, 51, 2) | (96, 57, 17, 2)
(32, 15, 6, 2) | (64, 27, 5, 2) | (64, 63, 8, 2) | (96, 57, 47, 2)
(32, 16, 2, 1) | (64, 27, 16, 2) | (64, 63, 13, 2) | (96, 58, 35, 2)
(32, 16, 3, 2) | (64, 27, 22, 2) | (64, 63, 61, 2) | (96, 59, 46, 2)
(32, 16, 5, 2) | (64, 28, 19, 2) |  | (96, 60, 29, 2)
(32, 17, 5, 2) | (64, 28, 25, 2) | (96, 15, 5, 2) | (96, 60, 41, 2)
(32, 19, 2, 1) | (64, 29, 16, 2) | (96, 21, 17, 2) | (96, 60, 45, 2)
(32, 19, 5, 2) | (64, 29, 28, 2) | (96, 25, 19, 2) | (96, 61, 17, 2)
(32, 19, 9, 2) | (64, 31, 12, 2) | (96, 25, 20, 2) | (96, 63, 20, 2)
(32, 19, 12, 2) | (64, 32, 21, 2) | (96, 29, 15, 2) | (96, 65, 12, 2)
(32, 19, 17, 2) | (64, 35, 29, 2) | (96, 29, 17, 2) | (96, 65, 39, 2)
(32, 20, 17, 2) | (64, 36, 7, 2) | (96, 30, 3, 2) | (96, 65, 51, 2)
(32, 21, 9, 2) | (64, 37, 2, 1) | (96, 32, 21, 2) | (96, 67, 5, 2)
(32, 21, 15, 2) | (64, 37, 11, 2) | (96, 32, 27, 2) | (96, 67, 25, 2)
(32, 23, 8, 2) | (64, 39, 4, 2) | (96, 33, 5, 2) | (96, 67, 34, 2)
(32, 23, 21, 2) | (64, 39, 25, 2) | (96, 35, 17, 2) | (96, 68, 5, 2)
(32, 25, 5, 2) | (64, 41, 5, 2) | (96, 35, 33, 2) | (96, 68, 19, 2)
(32, 25, 12, 2) | (64, 41, 11, 2) | (96, 39, 21, 2) | (96, 69, 17, 2)
(32, 27, 25, 2) | (64, 41, 27, 2) | (96, 40, 25, 2) | (96, 69, 36, 2)
(32, 29, 19, 2) | (64, 43, 21, 2) | (96, 41, 12, 2) | (96, 70, 23, 2)
(32, 29, 20, 2) | (64, 43, 28, 2) | (96, 41, 27, 2) | (96, 71, 6, 2)
(32, 30, 3, 2) | (64, 45, 28, 2) | (96, 41, 35, 2) | (96, 71, 40, 2)
(32, 30, 7, 2) | (64, 45, 41, 2) | (96, 42, 35, 2) | (96, 72, 53, 2)
(32, 31, 5, 2) | (64, 47, 5, 2) | (96, 43, 14, 2) | (96, 73, 32, 2)
(32, 31, 9, 2) | (64, 47, 21, 2) | (96, 44, 23, 2) | (96, 77, 27, 2)
(32, 31, 30, 2) | (64, 47, 30, 2) | (96, 45, 41, 2) | (96, 77, 31, 2)
 | (64, 49, 19, 2) | (96, 47, 36, 2) | (96, 77, 32, 2)
(64, 3, 2, 1) | (64, 49, 20, 2) | (96, 49, 31, 2) | (96, 77, 33, 2)
(64, 14, 3, 2) | (64, 52, 29, 2) | (96, 51, 30, 2) | (96, 77, 71, 2)
(64, 15, 8, 2) | (64, 53, 8, 2) | (96, 53, 17, 2) | (96, 78, 39, 2)
(64, 17, 2, 1) | (64, 53, 43, 2) | (96, 53, 19, 2) | (96, 79, 4, 2)
(64, 17, 9, 2) | (64, 56, 39, 2) | (96, 53, 32, 2) | (96, 81, 80, 2)
(64, 17, 16, 2) | (64, 56, 45, 2) | (96, 53, 48, 2) | (96, 83, 14, 2)
(64, 19, 2, 1) | (64, 59, 5, 2) | (96, 54, 15, 2) | (96, 83, 26, 2)
(64, 19, 18, 2) | (64, 59, 8, 2) | (96, 55, 44, 2) | (96, 83, 54, 2)
(96, 83, 60, 2) | (128, 31, 25, 2) | (128, 81, 55, 2) | (128, 105, 11, 2)
(96, 83, 65, 2) | (128, 33, 21, 2) | (128, 82, 67, 2) | (128, 105, 31, 2)
(96, 83, 78, 2) | (128, 35, 22, 2) | (128, 83, 60, 2) | (128, 105, 48, 2)
(96, 84, 65, 2) | (128, 37, 8, 2) | (128, 83, 61, 2) | (128, 107, 40, 2)
(96, 85, 17, 2) | (128, 41, 12, 2) | (128, 83, 77, 2) | (128, 107, 62, 2)
(96, 85, 31, 2) | (128, 42, 35, 2) | (128, 84, 15, 2) | (128,, 107102, 2)
(96, 85, 76, 2) | (128, 43, 25, 2) | (128, 84, 43, 2) | (128, 108, 35, 2)
(96, 85, 79, 2) | (128, 43, 42, 2) | (128, 85, 63, 2) | (128, 108, 73, 2)
(96, 86, 39, 2) | (128, 45, 17, 2) | (128, 87, 57, 2) | (128, 108, 75, 2)
(96, 86, 71, 2) | (128, 45, 27, 2) | (128, 87, 81, 2) | (128, 108, 89, 2)
(96, 87, 9, 2) | (128, 49, 9, 2) | (128, 89, 81, 2) | (128, 109, 11, 2)
(96, 87, 44, 2) | (128, 51, 9, 2) | (128, 90, 43, 2) | (128,, 109108, 2)
(96, 87, 45, 2) | (128, 54, 51, 2) | (128, 91, 9, 2) | (128, 110, 23, 2)
(96, 88, 19, 2) | (128, 55, 45, 2) | (128, 91, 13, 2) | (128, 111, 61, 2)
(96, 88, 35, 2) | (128, 56, 15, 2) | (128, 91, 44, 2) | (128, 113, 59, 2)
(96, 88, 43, 2) | (128, 56, 19, 2) | (128, 92, 35, 2) | (128, 114, 83, 2)
(96, 88, 79, 2) | (128, 56, 55, 2) | (128, 95, 94, 2) | (128, 115, 73, 2)
(96, 89, 35, 2) | (128, 57, 21, 2) | (128, 96, 23, 2) | (128,, 117105, 2)
(96, 89, 51, 2) | (128, 57, 37, 2) | (128, 96, 61, 2) | (128, 119, 30, 2)
(96, 89, 69, 2) | (128, 59, 29, 2) | (128, 97, 25, 2) | (128,, 119101, 2)
(96, 89, 87, 2) | (128, 59, 49, 2) | (128, 97, 68, 2) | (128, 120, 9, 2)
(96, 92, 51, 2) | (128, 60, 57, 2) | (128, 97, 72, 2) | (128, 120, 27, 2)
(96, 92, 71, 2) | (128, 61, 9, 2) | (128, 97, 75, 2) | (128, 120, 37, 2)
(96, 93, 32, 2) | (128, 61, 23, 2) | (128, 99, 13, 2) | (128, 120, 41, 2)
(96, 93, 39, 2) | (128, 61, 52, 2) | (128, 99, 14, 2) | (128, 120, 79, 2)
(96, 94, 35, 2) | (128, 63, 40, 2) | (128, 99, 26, 2) | (128, 120, 81, 2)
(96, 95, 4, 2) | (128, 63, 62, 2) | (128, 99, 54, 2) | (128, 121, 5, 2)
(96, 95, 16, 2) | (128, 67, 41, 2) | (128, 99, 56, 2) | (128, 121, 67, 2)
(96, 95, 32, 2) | (128, 69, 33, 2) | (128, 99, 78, 2) | (128, 121, 95, 2)
(96, 95, 44, 2) | (128, 71, 53, 2) | (128, 100, 13, 2) | (128, 121, 96, 2)
(96, 95, 45, 2) | (128, 72, 15, 2) | (128, 100, 39, 2) | (128, 123, 40, 2)
 | (128, 72, 41, 2) | (128, 101, 44, 2) | (128, 123, 78, 2)
(128, 5, 4, 2) | (128, 73, 5, 2) | (128, 101, 97, 2) | (128, 124, 41, 2)
(128, 15, 4, 2) | (128, 73, 65, 2) | (128, 103, 46, 2) | (128, 124, 69, 2)
(128, 21, 19, 2) | (128, 73, 67, 2) | (128, 104, 13, 2) | (128, 124, 81, 2)
(128, 25, 5, 2) | (128, 75, 13, 2) | (128, 104, 19, 2) | (128, 125, 33, 2)
(128, 26, 11, 2) | (128, 80, 39, 2) | (128, 104, 35, 2) | (128, 125, 43, 2)
(128, 27, 25, 2) | (128, 80, 53, 2) | (128, 105, 7, 2) | (128, 127, 121, 2)

Figure 17.5 Combining Generators.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

LFSR/FCSR Summation/Parity Cascade

The theory is that addition with carry destroys the algebraic properties of LFSRs, and that XOR destroys the algebraic properties of FCSRs. This generator combines those ideas, as used in the LFSR/FCSR Summation Generator and the LFSR/FCSR Parity Generator just listed, with the Gollmann cascade.

The generator is a series of arrays of registers, with the clock of each array controlled by the output of the previous array. Figure 17.6 is one stage of this generator. The first array of LFSRs is clocked and the results are combined using addition with carry. If the output of this combining function is 1, then the next array (of FCSRs) is clocked and the output of those FCSRs is combined with the output of the previous combining function using XOR. If the output of the first combining function is 0, then the array of FCSRs is not clocked and the output is simply added to the carry from the previous round. If the output of this second combining function is 1, then the third array of LFSRs is clocked, and so on.

This generator uses a lot of registers: n*m, where n is the number of stages and m is the number of registers per stage. I recommend n = 10 and m = 5.

Alternating Stop-and-Go Generators

These generators are stop-and-go generators with FCSRs instead of some LFSRs. Additionally, the XOR operation can be replaced with an addition with carry (see Figure 17.7).

— FCSR Stop-and-Go Generator. Register-1, Register-2, and Register-3 are FCSRs. The combining operation is XOR.
— FCSR/LFSR Stop-and-Go Generator. Register-1 is a FCSR, and Registers-2 and -3 are LFSRs. The combining operation is addition with carry.

Figure 17.6 Concoction Generator.

— LFSR/FCSR Stop-and-Go Generator. Register-1 is a LFSR, and Registers-2 and -3 are FCSRs. The combining operation is XOR.

Shrinking Generators

There are four basic generator types using FCSRs:

— FCSR Shrinking Generator. A shrinking generator with FCSRs instead of LFSRs.
— FCSR/LFSR Shrinking Generator. A shrinking generator with a LFSR shrinking a FCSR.
— LFSR/FCSR Shrinking Generator: A shrinking generator with a FCSR shrinking a LFSR.

Figure 17.7 Alternating stop-and-go generators.

— FCSR Self-Shrinking Generator. A self-shrinking generator with a FCSR instead of a LFSR.

17.6 Nonlinear-Feedback Shift Registers

It is easy to imagine a more complicated feedback sequence than the ones used in LFSRs or FCSRs. The problem is that there isn't any mathematical theory that can analyze them. You'll get something, but who knows what it is? In particular, here are some problems with nonlinear-feedback shift register sequences.

— There may be biases, such as more ones than zeros or fewer runs than expected, in the output sequence.
— The maximum period of the sequence may be much lower than expected.
— The period of the sequence might be different for different starting values.
— The sequence may appear random for a while, but then "dead end" into a single value. (This can easily be solved by XORing the nonlinear function with the rightmost bit.)

On the plus side, if there is no theory to analyze nonlinear-feedback shift registers for security, there are few tools to cryptanalyze stream ciphers based on them. We can use nonlinear-feedback shift registers in stream-cipher design, but we have to be careful.

In a nonlinear-feedback shift register, the feedback function can be anything you want (see Figure 17.8).

Figure 17.8 A nonlinear-feedback shift register (probably insecure).

Figure 17.9 3-bit nonlinear feedback shift register.

Figure 17.9 is a 3-bit shift register with the following feedback function: The new bit is the first bit times the second bit. If it is initialized with the value 110, it produces the following sequence of internal states:

1 1 0
0 1 1
1 0 1
0 1 0
0 0 1
0 0 0
0 0 0

And so on forever.

The output sequence is the string of least significant bits:

0 1 1 0 1 0 0 0 0 0 0 0....

This isn't terribly useful.

It gets even worse. If the initial value is 100, it produces 010, 001, then repeats forever at 000. If the initial value is 111, it repeats itself forever right from the start.

Some work has been done on computing the linear complexity of the product of two LFSRs [1650,726,1364,630,658,659]. A construction that involved computing LFSRs over a field of odd characteristic [310] is insecure [842].

17.7 Other Stream Ciphers

Many other stream ciphers have appeared in the literature here and there. Here are some of them.

Pless Generator

This generator is designed around the capabilities of the J-K flip-flop [1250]. Eight LFSRs drive four J-K flip-flops; each flip-flop acts as a nonlinear combiner for two of the LFSRs. To avoid the problem that knowledge of an output of the flip-flop identifies both the source and value of the next output bit, clock the four flip-flops and then interleave the outputs to yield the final keystream.

This algorithm has been cryptanalyzed by attacking each of the four flip-flops independently [1356]. Additionally, combining J-K flip-flops is cryptographically weak; generators of this type succumb to correlation attacks [1451].

Cellular Automaton Generator

In [1608,1609], Steve Wolfram proposed using a one-dimensional cellular automaton as a pseudo-random-number generator. Cellular automata is not the subject of this book, but Wolfram's generator consisted of a one-dimensional array of bits, a1, a2, a3,..., ak,..., an, and an update function:

a'k = ak-1 ⊕ (ak ⊦ ak+1)

The bit is extracted from one of the ak values; which one really doesn't matter.

The generator's behavior appears to be quite random. However, there is a known-plaintext attack against these generators [1052]. This attack works on a PC with values of n up to 500 bits. Additionally, Paul Bardell proved that the output of a cellular automaton can also be generated by a linear-feedback shift register of equal length and is therefore no more secure [83].

1/p Generator

This generator was proposed, and then cryptanalyzed, in [193]. If the internal state of the generator at time t is xt, then

xt+1 = bxt mod p

The output of the generator is the least significant bit of xt div p, where div is the truncated integer division. For maximum period, the constants b and p should be chosen so that p is prime and b is a primitive root mod p. Unfortunately, this generator isn't secure. (Note that for b = 2, an FCSR with a connection integer p outputs the reverse of this sequence.)


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

crypt(1)

The original UNIX encryption algorithm, crypt(1), is a stream cipher based on the same ideas as the Enigma. This is a 256-element, single-rotor substitution cipher with a reflector. Both the rotor and the reflector are generated from the key. This algorithm is far simpler than the World War II German Enigma and, for a skilled cryptanalyst, very easy to break [1576,1299]. A public-domain UNIX program, called Crypt Breakers Workbench (CBW), can be used to break files encrypted with crypt(1).

Other Schemes

Another generator is based on the knapsack problem (see Section 19.2) [1363]. CRYPTO-LEGGO is insecure [301]. Joan Daemen has developed SubStream, Jam, and StepRightUp [402]; they are all too new to comment on. Many other algorithms are described in the literature, and even more are kept secret and incorporated into equipment.

17.8 System-Theoretic Approach to Stream-Cipher Design

In practice, stream-cipher design is a lot like block-cipher design. It involves more mathematical theory, but in the end a cryptographer proposes a design and then tries to analyze it.

According to Rainer Rueppel, there are four different approaches to the construction of stream ciphers [1360,1362]:

— System-theoretic approach. Try to make sure that each design creates a difficult and unknown problem for the cryptanalyst, using a set of fundamental design principles and criteria.
— Information-theoretic approach. Try to keep the cryptanalyst in the dark about the plaintext. No matter how much work the cryptanalyst invests, he will never get a unique solution.
— Complexity-theoretic approach. Try to base the cryptosystem on, or make it equivalent to, some known and difficult problem such as factoring or taking discrete logarithms.
— Randomized approach. Try to generate an unmanageably large problem by forcing the cryptanalyst to examine lots of useless data in his attempts at cryptanalysis.

The approaches differ in their assumptions about the capabilities and opportunities of the cryptanalyst, the definition of cryptographic success, and the notion of security. Most of the research in this field is theoretical, but there are some good stream ciphers among the impractical ones.

The system-theoretic approach was used in all the stream ciphers previously listed; it produces most of the stream ciphers that are practical enough to be used in the real world. A cryptographer designs keystream generators that have testable security properties—period, distribution of bit patterns, linear complexity, and so on—and not ciphers based on mathematical theory. The cryptographer also studies various cryptanalytic techniques against these generators and makes sure the generators are immune to these attacks.

Over the years, the approach has resulted in a set of design criteria for stream ciphers [1432,99,1357,1249]. These were discussed by Rueppel in [1362], in which he details the theory behind them.

— Long period, no repetitions.
— Linear complexity criteria—large linear complexity, linear complexity profile, local linear complexity, and so forth.
— Statistical criteria such as ideal k-tuple distributions.
— Confusion—every keystream bit must be a complex transformation of all or most of the key bits.
— Diffusion—redundancies in substructures must be dissipated into long-range statistics.
— Nonlinearity criteria for Boolean functions like mth-order correlation immunity, distance to linear functions, avalanche criterion, and so on.

This list of design criteria is not unique for stream ciphers designed by the system-theoretic approach; it is true for all stream ciphers. It is even true for all block ciphers. The unique point about the system-theoretic approach is that stream ciphers are designed to satisfy these goals directly.

The major problem with these cryptosystems is that nothing can be proven about their security; the design criteria have never been proved to be either necessary or sufficient for security. A keystream generator may satisfy all the design principles, but could still turn out to be insecure. Another could turn out to be secure. There is still some magic to the process.

On the other hand, breaking each of these keystream generators is a different problem for a cryptanalyst. If enough different generators are out there, it may not be worth the cryptanalyst's time to try to break each one. He may better achieve fame and glory by figuring out better ways to factor large numbers or calculating discrete logarithms.

17.9 Complexity-Theoretic Approach to Stream-Cipher Design

Rueppel also delineated a complexity-theoretic approach to stream-cipher design. Here, a cryptographer attempts to use complexity theory to prove that his generators are secure. Consequently, the generators tend to be more complicated, based on the same sorts of hard problems as public-key cryptography. And like public-key algorithms, they tend to be slow and cumbersome.

Shamir's Pseudo-Random-Number Generator

Adi Shamir used the RSA algorithm as a pseudo-random-number generator [1417]. While Shamir showed that predicting the output of the pseudo-random-number generator is equivalent to breaking RSA, potential biases in the output were demonstrated in [1401,200].

Blum-Micali Generator

This generator gets its security from the difficulty of computing discrete logarithms [200]. Let g be a prime and p be an odd prime. A key x0, starts off the process:

xi+1 = gxi mod p

The output of the generator is 1 if xi < (p - 1)/2, and 0 otherwise.

If p is large enough so that computing discrete logarithms mod p is infeasible, then this generator is secure. Additional theoretical results can be found in [1627,986,985,1237,896,799].

RSA

This RSA generator [35,36] is a modification of [200]. The initial parameters are a modulus N which is the product of two large primes p and q, an integer e which is relatively prime to (p - 1) (q - 1), and a random seed x0, where x0 is less than N.

xi+1 = xei mod N

The output of the generator is the least significant bit of xi. The security of this generator is based on the difficulty of breaking RSA. If N is large enough, then the generator is secure. Additional theory can be found in [1569,1570,1571,30,354].

Blum, Blum, and Shub

The simplest and most efficient complexity-theoretic generator is called the Blum, Blum, and Shub generator, after its inventors. Mercifully, we shall abbreviate it to BBS, although it is sometimes called the quadratic residue generator [193].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The theory behind the BBS generator has to do with quadratic residues modulo n (see Section 11.3). Here's how it works.

First find two large prime numbers, p and q, which are congruent to 3 modulo 4. The product of those numbers, n, is a Blum integer. Choose another random integer, x, which is relatively prime to n. Compute

x0 = x2 mod n

That's the seed for the generator.

Now you can start computing bits. The ith pseudo-random bit is the least significant bit of xi, where

xi = xi-12 mod n

The most intriguing property of this generator is that you don't have to iterate through all i - 1 bits to get the ith bit. If you know p and q, you can compute the ith bit directly.

bi is the least significant bit of xi, where xi = x0(2i) mod ((p-1)(q-1))

This property means you can use this cryptographically strong pseudo-random-bit generator as a stream cryptosystem for a random-access file.

The security of this scheme rests on the difficulty of factoring n. You can make n public, so anyone can generate bits using the generator. However, unless a cryptanalyst can factor n, he can never predict the output of the generator—not even with a statement like: "The next bit has a 51 percent chance of being a 1."

More strongly, the BBS generator is unpredictable to the left and unpredictable to the right. This means that given a sequence generated by the generator, a cryptanalyst cannot predict the next bit in the sequence nor the previous bit in the sequence. This is not security based on some complicated bit generator that no one understands, but the mathematics behind factoring n.

This algorithm is slow, but there are speedups. As it turns out, you can use more than the least significant bit of each xi as a pseudo-random bit. According to [1569,1570,1571,35,36], if n is the length of xi, the least significant log2n bits of xi can be used. The BBS generator is comparatively slow and isn't useful for stream ciphers. However, for high-security applications, such as key generation, this generator is the best of the lot.

17.10 Other Approaches to Stream-Cipher Design

In an information-theoretic approach to stream ciphers, the cryptanalyst is assumed to have unlimited time and computing power. The only practical stream cipher that is secure against an adversary like this is a one-time pad (see Section 1.5). Since bits would be impractical on a pad, this is sometimes called a one-time tape. Two magnetic tapes, one at the encryption end and the other at the decryption end, would have the same random keystream on them. To encrypt, simply XOR the plaintext with the bits on the tape. To decrypt, XOR the ciphertext with the bits on the other, identical, tape. You never use the same keystream bits twice. Since the keystream bits are truly random, no one can predict the keystream. If you burn the tapes when you are through with them, you've got perfect secrecy (assuming no one else has copies of the tape).

Another information-theoretic stream cipher, developed by Claus Schnorr, assumes that the cryptanalyst only has access to a limited number of ciphertext bits [1395]. The results are highly theoretical and have no practical value, at least not yet. For more details, consult [1361,1643,1193].

In a randomized stream cipher, the cryptographer tries to ensure that the cryptanalyst has an infeasibly large problem to solve. The objective is to increase the number of bits the cryptanalyst has to work with, while keeping the secret key small. This can be done by making use of a large public random string for encryption and decryption. The key would specify which parts of the large random string are to be used for encryption and decryption. The cryptanalyst, not knowing the key, is forced to pursue a brute-force search through the random string. The security of this sort of cipher can be expressed by the average number of bits a cryptanalyst must examine before the chances of determining the key improve over pure guessing.

Rip van Winkle Cipher

James Massey and Ingemar Ingemarsson proposed the Rip van Winkle cipher [1011], so named because the receiver has to receive 2n bits of ciphertext before attempting decryption. The algorithm, illustrated in Figure 17.10, is simple to implement, provably secure, and completely impractical. Simply XOR the plaintext with the keystream, and delay the keystream by 0 to 20 years—the exact delay is part of the key. In Massey's words: "One can easily guarantee that the enemy cryptanalyst will need thousands of years to break the cipher, if one is willing to wait millions of years to read the plaintext." Further work on this idea can be found in [1577,755].

Figure 17.10 Rip van Winkle cipher.

Diffie's Randomized Stream Cipher

This scheme was first proposed by Whitfield Diffie [1362]. The data are 2n random sequences. The key is k, a random n-bit string. To encrypt a message, Alice uses the kth random string as a one-time pad. She then sends the ciphertext plus the 2n random strings over 2n + 1 different communications channels.

Bob knows k, so he can easily choose which one-time pad to decrypt the message with. Eve has no choice but to examine the random sequences one at a time until she finds the correct one-time pad. Any attack must examine an expected number of bits which is in O(2n). Rueppel points out that if you send n random strings instead of 2n, and if the key is used to specify a linear combination of those random strings, the security is the same.

Maurer's Randomized Stream Cipher

Ueli Maurer described a scheme based on XORing the plaintext with several large public random-bit sequences [1034,1029,1030]. The key is the set of starting positions within each sequence. This turns out to be provably almost secure, with a calculable probability of being broken based on how much memory the attacker has at his disposal, without regard to the amount of computing power he has. Maurer claims that this scheme would be practical with about 100 different sequences of 1020 random bits each. Digitizing the face of the moon might be one way to get this many bits.

17.11 Cascading Multiple Stream Ciphers

If performance is no issue, there's no reason not to choose multiple stream ciphers and cascade them. Simply XOR the output of each generator with the plaintext to get the ciphertext. Ueli Maurer's result (see Section 15.7) says that if the generators have independent keys, then the security of the cascade is at least as secure as the strongest algorithm in the cascade. It is probably much more secure than that.

Stream ciphers can be combined in all the same ways as block ciphers (see Chapter 15). Stream ciphers can be cascaded (see Section 15.7) with other stream ciphers, or together with block ciphers.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

A clever trick is to use one algorithm, either a block or stream algorithm, to frequently rekey a fast stream algorithm (which could even be a block algorithm in OFB mode). The fast algorithm could be weak, since a cryptanalyst would never see very much plaintext encrypted with any one key.

There's a trade-off between the size of the fast algorithm's internal state (which may impact security) and how often you can afford to rekey. The rekey needs to be relatively fast; algorithms that have a long key setup routine aren't suitable for this kind of application. And the rekeying should be independent of the internal state of the fast algorithm.

17.12 Choosing a Stream Cipher

If the study of stream ciphers offers any lessons, it's that new types of attacks are invented with alarming regularity. Classically, stream ciphers have been based on considerable mathematical theory. This theory can be used to prove good properties about the cipher, but can also be used to find new attacks against the cipher. I worry about any stream cipher based solely on LFSRs for this reason.

I prefer stream ciphers that are designed more along the lines of block ciphers: nonlinear transformations, large S-boxes, and so on. RC4 is my favorite, and SEAL is a close second. I would be very interested in seeing cryptanalytic results against my generators that combine LFSRs and FCSRs; this seems to be a very fruitful area of stream-cipher research to mine for actual designs. Or, you can use a block cipher in OFB or CFB to get a stream cipher.

Table 17.3 gives some timing measurements for some algorithms. These are meant for comparison purposes only.

17.13 Generating Multiple Streams from a Single Pseudo-Random-Sequence Generator

If you need to encrypt multiple channels of communications in a single box—a multiplexer, for example—the easy solution is to use a different pseudo-random-sequence generator for each stream. This has two problems: It requires more hardware, and all the different generators have to be synchronized. It would be simpler to use a single generator.

Table 17.3 Encryption Speeds of Some Stream Ciphers on a 33MHz 486SX
Algorithm | Encryption Speed (Kilobytes/Second)
A5 | 5
PIKE | 62
RC4 | 164
SEAL | 381

One solution is to clock the generator multiple times. If you want three independent streams, clock the generator three times and send 1 bit into each stream. This technique works, but you may have trouble clocking the generator as fast as you would like. For example, if you can only clock the generator three times as fast as the data stream, you can only create three streams. Another way is to use the same sequence for each channel—perhaps with a variable time delay. This is insecure.

A really clever idea [1489], patented by the NSA, is shown in Figure 17.11. Dump the output of your favorite generator into an m-bit simple shift register. At each clock pulse, shift the register one to the right. Then, for each output stream, AND the register with a different m-bit control vector viewed as a unique identifier for the desired output stream, then XOR all the bits together to get the output bit for that stream. If you want several output streams in parallel, you need a separate control vector and an XOR/AND logic array for each output stream.

There are some things to watch out for. If any of the streams are linear combinations of other streams, then the system can be broken. But if you are clever, this is an easy and secure way to solve the problem.

17.14 Real Random-Sequence Generators

Sometimes cryptographically secure pseudo-random numbers are not good enough. Many times in cryptography, you want real random numbers. Key generation is a prime example. It's fine to generate random cryptographic keys based on a pseudo-random sequence generator, but if an adversary gets a copy of that generator and the master key, the adversary can create the same keys and break your cryptosystem, no matter how secure your algorithms are. A random-sequence generator's sequences cannot be reproduced. No one, not even you, can reproduce the bit sequence out of those generators.

Figure 17.11 Multiple-bit generator.

There is a large philosophical debate over whether any of these techniques actually produces real random bits. I am not going to address that debate. The point here is to produce bits that have the same statistical properties as random bits and are not reproducible.

The important thing about any real random-sequence generator is that it be tested. There is a wealth of literature on this topic. Tests of randomness can be found in [863,99]. Maurer showed that all these tests can be derived from trying to compress the sequence [1031,1032]. If you can compress a random sequence, then it is not truly random.

Anyhow, what we have here is a whole lot of black magic. The primary point is to generate a sequence of bits that your adversary is unlikely to guess. It doesn't sound like much, but it's harder than you think. I can't prove that any of these techniques generates random bits. These techniques produce a sequence of bits that cannot be easily reproduced. For some details, see [1375,1376,511].

RAND Tables

Back in 1955, when computers were still new, the Rand Corporation published a book that contained a million random digits [1289]. Their method is described in the book:

The random digits in the book were produced by rerandomization of a basic table generated by an electronic roulette wheel. Briefly, a random frequency pulse source, providing on the average about 100,000 pulses per second, was gated about once per second by a constant frequency pulse. Pulse standardization circuits passed the pulses through a 5-place binary counter. In principle the machine was a 32-place roulette wheel which made, on the average, about 3000 revolutions per trial and produced one number per second. A binary-to-decimal converter was used which converted 20 of the 32 numbers (the other twelve were discarded) and retained only the final digit of two-digit numbers; this final digit was fed into an IBM punch to produce finally a punched card table of random digits.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The book goes on to discuss the results of various randomness tests on the data. It also suggests how to use the book to find a random number:

The lines of the digit table are numbered from 00000 to 19999. In any use of the table, one should first find a random starting position. A common procedure for doing this is to open the book to an unselected page of the digit table and blindly choose a five-digit number; this number with the first digit reduced modulo 2 determines the starting line; the two digits to the right of the initially selected five-digit number are reduced modulo 50 to determine the starting column in the starting line. To guard against the tendency of books to open repeatedly at the same page and the natural tendency of a person to choose a number toward the center of the page: every five-digit number used to determine a starting position should be marked and not used a second time for this purpose.

The meat of the book is the "Table of Random Digits." It lists them in 5-digit groups—"10097 32533 76520 13586..."—50 on a line and 50 lines on a page. The table goes on for 400 pages and, except for a particularly racy section on page 283 which reads "69696," makes for a boring read. The book also includes a table of 100,000 normal deviates.

The interesting thing about the RAND book is not its million random digits, but that they were created before the computer revolution. Many cryptographic algorithms use arbitrary constants—so-called "magic numbers." Choosing magic numbers from the RAND tables ensures that they haven't been specially chosen for some nefarious reason. Khafre does this, for example.

Using Random Noise

The best way to collect a large number of random bits is to tap the natural randomness of the real world. Often this method requires specialized hardware, but you can play tricks with computers.

Find an event that happens regularly but randomly: atmospheric noise peaking at a certain threshold, a toddler falling while learning to walk, or some such. Measure the time interval between one event and the next event. Record it. Measure the time interval between the second event and the third event. Record it as well. If the first time interval is greater than the second, output 1 as the bit. If the second time interval is greater than the first, output 0 as the event. Do it again for the next event.

Throw a dart at the New York Stock Exchange closing prices in your local newspaper. Compare the closing price of the stock you hit with the closing price of the stock directly above it. If the one you hit is more, output 0; if it less, output 1.

Hook a Geiger counter up to your computer, count emissions over a fixed time interval, and keep the least significant bit. Or measure the time between successive ticks. (Since the radioactive source is decaying, the average time between successive ticks is continuously getting longer. You want to choose a source with the half life long enough to make this negligible—like plutonium. Or, if you're worried about your health, you can apply appropriate statistical corrections.)

G. B. Agnew proposed a real random-bit generator, suitable for integration into a VLSI device [21]. It is a metal insulator semiconduction capacitor (MISC). Two of them are placed in close proximity, and the random bit is a function of the difference in charge between the two. Another random-number generator generates a random-bit stream based on the frequency instability in a free-running oscillator [535]. A commercial chip from AT&T generates random numbers from the same phenomenon [67]. M. Gude built a random-number generator that collected random bits from physical phenomena, such as radioactive decay [668,669]. Manfield Richter developed a random-number generator based on thermal noise from a semiconductor diode [1309].

Supposedly the time intervals between successive 2e4 light emissions from a trapped mercury atom are random. Use that. Better yet, find a semiconductor company that makes random-number-generation chips; they are out there.

There is also a random-number generator that uses the computer's disk drive [439]. It measures the time required to read a disk block and uses the variation in that time as a random number source. It filters the timing data to remove structure that comes from quantization, then applies a fast Fourier transform to vectors of the numbers. This removes bias and correlation. Finally, it uses the spectral angles for frequencies in (0, π), normalized to the unit interval, as the random bits. A large part of the variation in disk rotation speed is caused by air turbulence, so there is randomness in the system. There are caveats, though. If you keep too many bits of the output, you are using the fast Fourier transform as a random-number generator and risk predictability. And it's best to read the same disk block over and over, so that your filtering doesn't have to remove structure that comes from the disk-scheduler. An implementation of this system was able to collect about 100 bits per minute [439].

Using the Computer's Clock

If you want a single random bit (or even a few), take the least significant bit from any clock register. This might not be terribly random in a UNIX system because of various potential synchronizations, but it works on some personal computers.

Beware of getting too many bits this way. Executing the same subroutine several times in succession could easily skew bits generated in this manner. For example, if each bit generation subroutine takes an even number of clock ticks to execute, you will get an endless stream of the same bit out of the generator. If each subroutine takes an odd number of clock ticks to execute, you will get an endless stream of alternating bits out of the generator. Even if the resonance isn't this obvious, the resultant bit stream will be far from random.

One random-number generator works this way [918]:

Our truly random number generator...works by setting an alarm and then incrementing a counter register rapidly in the CPU until an interrupt occurs. The contents of the register are then XORed with the contents of an output buffer byte (truncating the register's data to 8 bits). After each byte of the output buffer is filled, the buffer is further processed by doing a right, circular shift of each character by 2 bits. This has the effect of moving the most active (and random) least significant bits into the most significant positions. The entire process is then repeated 3 times. Finally each character of the buffer has been touched by the two most random bits of the counter register after interrupts. That is 4n interrupts have occurred where n is the number of desired random bytes.

This method is very sensitive to the randomness of system interrupts and the granularity of the clock. The output looked pretty good when tested on real UNIX machines.

Measuring Keyboard Latency

People's typing patterns are both random and nonrandom. They are nonrandom enough that they can be used as a means of identification, but they are random enough that they can be used to generate random bits. Measure the time between successive keystrokes, then take the least significant bits of those measurements. These bits are going to be pretty random. This technique may not work on a UNIX terminal, since the keystrokes pass through filters and other mechanisms before they get to your program, but it will work on most personal computers.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Ideally, you only want to collect one random bit per keystroke. Collecting more may skew the results, depending on how good a typist is sitting at the keyboard. This technique is limited, though. While it's easy to have someone type 100 words or so when it is time to generate a key, it isn't reasonable to ask the typist to type a 100,000-word essay to generate a keystream for a one-time pad.

Biases and Correlations

A major problem with all these systems is that there could be nonrandomness in the generated sequence. The underlying physical processes might be random, but many kinds of measuring instruments are between the digital part of the computer and the physical process. Those instruments could easily introduce problems.

A way to eliminate bias, or skew, is to XOR several bits together. If a random bit is biased toward 0 by a factor e, then the probability of 0 can be written as:

P(0) = .5 + e

XORing two of these bits together yields:

P(0) = (.5 + e)2 + (.5 - e)2 = .5 + 2e2

By the same calculation, XORing 4 bits together yields:

P(0) = .5 + 8e4

XORing m bits will exponentially converge to an equal probability of 0 and 1. If you know the maximum bias you are willing to accept for your application, you can calculate how many bits you need to XOR together to get random bits below that bias.

An even better method is to look at the bits in pairs. If the 2 bits are the same, discard them and look at the next pair. If the 2 bits are different, take the first bit as the output of the generator. This eliminates bias completely. Other techniques for reducing bias use transition mappings, compression, and fast Fourier transforms [511].

The potential problem with both methods is that if there is a correlation between adjacent bits, then these methods will increase the bias. One way to correct this is to use multiple random sources. Take four different random sources and XOR the bits together; or take two random sources, and look at those bits in pairs.

For example, take a radioactive source and hook a Geiger counter to your computer. Take a pair of noisy diodes and record as an event every time the noise exceeds a certain peak. Measure atmospheric noise. Get a random bit from each and XOR them together to produce the random bit. The possibilities are endless.

The mere fact that a random-number generator has a bias does not necessarily mean that it is unusable. It just means that it is less secure. For example, consider the problem of Alice generating a triple-DES 168-bit key. All she has is a random-bit generator with a bias toward 0: It produces 55 percent 0s and 45 percent 1s. This means that there are only 0.99277 bits of entropy per key bit, as opposed to 1 bit of entropy if the generator were perfect. Mallory, trying to break the key, can optimize his brute-force search to try the most probable key first (000...0), and work toward the least probable key (111...1). Because of the bias, Mallory can expect to find the key in 2109 attempts. If there were no bias, Mallory would expect to make 2111 attempts. The resultant key is less secure, but not appreciably so.

Distilling Randomness

In general, the best way to generate random numbers is to find a whole lot of seemingly random events and distill randomness from them. This randomness can then be stored in a pool or reservoir that applications can draw on as needed. One-way hash functions are ready-made for the job; they're fast, so you can shovel quite a bit through them without worrying too much about performance or the actual randomness of each observation. Hash almost anything you can find that has at least some randomness. Try:

— A copy of every keystroke
— Mouse commands
— The sector number, time of day, and seek latency for every disk operation
— Actual mouse position
— Number of current scanline of monitor
— Contents of the actually displayed image
— Contents of FATs, kernel tables, and so on
— Access/modify times of /dev/tty
— CPU load
— Arrival times of network packets
— Input from a microphone
— /dev/audio without a microphone attached

If your system uses separate crystal oscillators for its CPU and time-of-day clocks, try reading the time of day in a tight loop. On some (but not all) systems this will reflect the random phase jitter between the two oscillators.

Since much of the randomness in these events is in their timing, use the most finely grained time-of-day clock you can find. A standard PC uses an Intel 8254 clock chip (or equivalent) driven at 1.1931818 megahertz, so reading the counter register directly gives you 838-nanosecond resolution. To avoid skewing the results, avoid taking your event samples on a timer interrupt.

Here is the process in C with MD5 (see Section 18.5) as the hash function:

char Randpool[16];

/* Call early and call often on a wide variety of random or semi-
 * random system events to churn the randomness pool.
 * The exact format and length of randevent doesn't matter as long as
 * its contents are at least somewhat unpredictable.
 */
void churnrand(char *randevent,unsigned int randlen)
{
     MD5_CTX md5;
     MD5Init(&md5);
     MD5Update(&md5,Randpool,sizeof(Randpool));
     MD5Update(&md5,randevent,randlen);
     MD5Final(Randpool,&md5);
}

After calling churnrand() enough to build up sufficient randomness in Randpool, you can now generate random bits from it. MD5 again comes in handy, this time as a counter-mode pseudo-random byte-stream generator.

long Randcnt;
void genrand(char *buf,unsigned int buflen)
{
     MD5_CTX md5;
     char tmp[16];
     unsigned int n;

     while(buflen != 0) {
          /* Hash the pool with a counter */
          MD5Init(&md5);
          MD5Update(&md5,Randpool,sizeof(Randpool));
          MD5Update(&md5,(unsigned char *)&Randcnt,sizeof(Randcnt));
          MD5Final(tmp,&md5);
          Randcnt++; /* Increment counter */

          /* Copy 16 bytes or requested amount, whichever is less,
          * to the user's buffer */
          n = (buflen < 16) ? buflen : 16;
          memcpy(buf,tmp,n);
          buf += n;
          buflen -= n;
     }
}

The hash function is crucial here for several reasons. First, it provides an easy way to generate an arbitrary amount of pseudo-random data without having to call churnrand() each time. In effect, the system degrades gracefully from perfect to practical randomness when the demand exceeds the supply. In this case it becomes theoretically possible to use the result from one genrand() call to determine a previous or subsequent result. But this requires inverting MD5, which is computationally infeasible.

This is important since the routine doesn't know what each caller will do with the random data it returns. One call might generate a random number for a protocol that is sent in the clear, perhaps in response to a direct request by an attacker. The very next call might generate a secret key for an unrelated session that the attacker wishes to penetrate. Obviously, it is very important that an attacker not be able to deduce the secret key from the nonce.

One problem remains. There must be sufficient randomness in the Randpool[] array before the first call to genrand(). If the system has been running for a while with a local user typing on the keyboard, no problem. But what about a standalone system that reboots automatically without seeing any keyboard or mouse input?

This is a tough one. A partial solution would require the operator to type for a while after the very first reboot, and to create a seed file on disk before shutting down to carry the randomness in Randseed[] across reboots. But do not save the Randseed[] array directly. An attacker who steals this file could determine all of the results from genrand() after the last call to churnrand() prior to the file being created.

The fix to this problem is to hash the Randseed[] array before storing it, perhaps by just calling genrand(). When the system reboots, you read in the seed file, pass it to churnrand(), then promptly destroy it. Unfortunately, this does not deal with the threat of someone stealing the seed file between reboots and using it to guess future values of the genrand() function. I see no solution to this problem other than to wait until enough external random events have taken place after a reboot before allowing genrand() to produce results.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 18
One-Way Hash Functions

18.1 Background

A one-way hash function, H(M), operates on an arbitrary-length pre-image message, M. It returns a fixed-length hash value, h.

h = H(M), where h is of length m

Many functions can take an arbitrary-length input and return an output of fixed length, but one-way hash functions have additional characteristics that make them one-way [1065]:

Given M, it is easy to compute h.
Given h, it is hard to compute M such that H(M)= h.
Given M, it is hard to find another message, M', such that H(M) = H(M').

If Mallory could do the hard things, he would undermine the security of every protocol that uses the one-way hash function. The whole point of the one-way hash function is to provide a "fingerprint" of M that is unique. If Alice signed M by using a digital signature algorithm on H(M), and Bob could produce M', another message different from M where H(M) = H(M'), then Bob could claim that Alice signed M'.

In some applications, one-wayness is insufficient; we need an additional requirement called collision-resistance.

It is hard to find two random messages, M and M', such that H(M) = H(M').

Remember the birthday attack from Section 7.4? It is not based on finding another message M', such that H(M) = H(M'), but based on finding two random messages, M and M', such that H(M) = H(M').

The following protocol, first described by Gideon Yuval [1635], shows how—if the previous requirement were not true—Alice could use the birthday attack to swindle Bob.

(1) Alice prepares two versions of a contract: one is favorable to Bob; the other bankrupts him.
(2) Alice makes several subtle changes to each document and calculates the hash value for each. (These changes could be things like: replacing SPACE with SPACE-BACKSPACE-SPACE, putting a space or two before a carriage return, and so on. By either making or not making a single change on each of 32 lines, Alice can easily generate 232 different documents.)
(3) Alice compares the hash values for each change in each of the two documents, looking for a pair that matches. (If the hash function only outputs a 64-bit value, she would usually find a matching pair with 232 versions of each.) She reconstructs the two documents that hash to the same value.
(4) Alice has Bob sign the version of the contract that is favorable to him, using a protocol in which he only signs the hash value.
(5) At some time in the future, Alice substitutes the contract Bob signed with the one that he didn't. Now she can convince an adjudicator that Bob signed the other contract.

This is a big problem. (One moral is to always make a cosmetic change to any document you sign.)

Other similar attacks could be mounted assuming a successful birthday attack. For example, an adversary could send an automated control system (on a satellite, perhaps) random message strings with random signature strings. Eventually, one of those random messages will have a valid signature. The adversary would have no idea what the command would do, but if his only objective was to tamper with the satellite, this would do it.

Length of One-Way Hash Functions

Hash functions of 64 bits are just too small to survive a birthday attack. Most practical one-way hash functions produce 128-bit hashes. This forces anyone attempting the birthday attack to hash 264 random documents to find two that hash to the same value, not enough for lasting security. NIST, in its Secure Hash Standard (SHS), uses a 160-bit hash value. This makes the birthday attack even harder, requiring 280 random hashes.

The following method has been proposed to generate a longer hash value than a given hash function produces.

(1) Generate the hash value of a message, using a one-way hash function listed in this book.
(2) Prepend the hash value to the message.
(3) Generate the hash value of the concatenation of the message and the hash value.
(4) Create a larger hash value consisting of the hash value generated in step (1) concatenated with the hash value generated in step (3).
(5) Repeat steps (1) through (3) as many times as you wish, concatenating as you go.

Although this method has never been proved to be either secure or insecure, various people have some serious reservations about it [1262, 859].

Overview of One-Way Hash Functions

It's not easy to design a function that accepts an arbitrary-length input, let alone make it one-way. In the real world, one-way hash functions are built on the idea of a compression function . This one-way function outputs a hash value of length n given an input of some larger length m [1069, 414]. The inputs to the compression function are a message block and the output of the previous blocks of text (see Figure 18.1). The output is the hash of all blocks up to that point. That is, the hash of block Mi is

hi = f(Mi,hi- 1)

This hash value, along with the next message block, becomes the next input to the compression function. The hash of the entire message is the hash of the last block.

The pre-image should contain some kind of binary representation of the length of the entire message. This technique overcomes a potential security problem resulting from messages with different lengths possibly hashing to the same value [1069, 414]. This technique is sometimes called MD-strengthening [930].

Various researchers have theorized that if the compression function is secure, then this method of hashing an arbitrary-length pre-image is also secure—but nothing has been proved [1138, 1070, 414].

A lot has been written on the design of one-way hash functions. For more mathematical information, consult [1028, 793, 791, 1138, 1069, 414, 91, 858, 1264]. Bart Preneel's thesis [1262] is probably the most comprehensive treatment of one-way hash functions.

Figure 18.1 One-way function.

18.2 Snefru

Snefru is a one-way hash function designed by Ralph Merkle [1070]. (Snefru, like Khufu and Khafre, was an Egyptian pharaoh.) Snefru hashes arbitrary-length messages into either 128-bit or 256-bit values.

First the message is broken into chunks, each 512-m in length. (The variable m is the length of the hash value.) If the output is a 128-bit hash value, then the chunks are each 384 bits long; if the output is a 256-bit hash value, then the chunks are each 256 bits long.

The heart of the algorithm is function H, which hashes a 512-bit value into an m-bit value. The first m bits of H's output are the hash of the block; the rest are discarded. The next block is appended to the hash of the previous block and hashed again. (The initial block is appended to a string of zeros.) After the last block (if the message isn't an integer number of blocks long, zeros are used to pad the last block), the first m bits are appended to a binary representation of the length of the message and hashed one final time.

Function H is based on E, which is a reversible block-cipher function that operates on 512-bit blocks. H is the last m bits of the output of E XORed with the first m bits of the input of E.

The security of Snefru resides in function E, which randomizes data in several passes. Each pass is composed of 64 randomizing rounds. In each round a different byte of the data is used as an input to an S-box; the output word of the S-box is XORed with two neighboring words of the message. The S-boxes are constructed in a manner similar to those in Khafre (see Section 13.7). Some rotations are thrown in, too. Originally Snefru was designed with two passes.

Cryptanalysis of Snefru

Using differential cryptanalysis, Biham and Shamir demonstrated the insecurity of two-pass Snefru (128-bit hash value) [172]. Their attack finds pairs of messages that hash to the same value within minutes.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

On 128-bit Snefru, their attacks work better than brute force for four passes or less. A birthday attack against Snefru takes 264 operations; differential cryptanalysis can find a pair of messages that hash to the same value in 228.5 operations for three-pass Snefru and 244.5 operations for four-pass Snefru. Finding a message that hashes to a given value by brute force requires 2128 operations; differential cryptanalysis takes 256 operations for three-pass Snefru and 288 operations for four-pass Snefru.

Although Biham and Shamir didn't analyze 256-bit hash values, they extended their analysis to 224-bit hash values. Compared to a birthday attack that requires 2112 operations, they can find messages that hash to the same value in 212.5 operations for two-pass Snefru, 233 operations for three-pass Snefru, and 281 operations for four-pass Snefru.

Currently, Merkle recommends using Snefru with at least eight passes [1073]. However, with this many passes the algorithm is significantly slower than either MD5 or SHA.

18.3 N- Hash

N-Hash is an algorithm invented by researchers at Nippon Telephone and Telegraph, the same people who invented FEAL, in 1990 [1105, 1106]. N-Hash uses 128-bit message blocks, a complicated randomizing function similar to FEAL's, and produces a 128-bit hash value.

The hash of each 128-bit block is a function of the block and the hash of the previous block.

H0 = I, where I is a random initial value
Hi = g(Mi,Hi- 1) ⊕ Mi ⊕ Hi- 1

The hash of the entire message is the hash of the last message block. The random initial value, I, can be any value determined by the user (even all zeros).

The function g is a complicated one. Figure 18.2 is an overview of the algorithm. Initially, the 128-bit hash of the previous message block, Hi-1, has its 64-bit left half and 64-bit right half swapped; it is then XORed with a repeating one/zero pattern (128 bits worth), and then XORed with the current message block, Mi. This value then cascades into N(N = 8 in the figures) processing stages. The other input to the processing stage is the previous hash value XORed with one of eight binary constant values.

Figure 18.2 Outline of N-Hash.

One processing stage is given in Figure 18.3. The message block is broken into four 32-bit values. The previous hash value is also broken into four 32-bit values. The function f is given in Figure 18.4. Functions S0 and S1 are the same as they were in FEAL.

S0(a,b) = rotate left two bits ((a + b) mod 256)
S1(a,b) = rotate left two bits ((a + b + 1) mod 256)

The output of one processing stage becomes the input to the next processing stage. After the last processing stage, the output is XORed with the Mi and Hi-1, and then the next block is ready to be hashed.

Cryptanalysis of N- Hash

Bert den Boer discovered a way to produce collisions in the round function of N-Hash [1262]. Biham and Shamir used differential cryptanalysis to break 6-round N-Hash [169, 172]. Their particular attack (there certainly could be others) works for any N that is divisible by 3, and is more efficient than the birthday attack for any N less than 15.

Figure 18.3 One processing stage of N-Hash.

Figure 18.4 Function f.

The same attack can find pairs of messages that hash to the same value for 12-round N-Hash in 256 operations, compared to 264 operations for a brute-force attack. N-hash with 15 rounds is safe from differential cryptanalysis: The attack requires 272 operations.

The algorithm's designers recommend using N-Hash with at least 8 rounds [1106]. Given the proven insecurity of N-Hash and FEAL (and its speed with 8 rounds), I recommend using another algorithm entirely.

18.4 MD4

MD4 is a one-way hash function designed by Ron Rivest [1318, 1319, 1321]. MD stands for Message Digest; the algorithm produces a 128-bit hash, or message digest, of the input message.

In [1319], Rivest outlined his design goals for the algorithm:

Security. It is computationally infeasible to find two messages that hashed to the same value. No attack is more efficient than brute force.
Direct Security. MD4's security is not based on any assumption, like the difficulty of factoring.
Speed. MD4 is suitable for high-speed software implementations. It is based on a simple set of bit manipulations on 32-bit operands.
Simplicity and Compactness. MD4 is as simple as possible, without large data structures or a complicated program.
Favor Little-Endian Architectures. MD4 is optimized for microprocessor architectures (specifically Intel microprocessors); larger and faster computers make any necessary translations.

After the algorithm was first introduced, Bert den Boer and Antoon Bosselaers successfully cryptanalyzed the last two of the algorithm's three rounds [202]. In an unrelated cryptanalytic result, Ralph Merkle successfully attacked the first two rounds [202]. Eli Biham discussed a differential cryptanalysis attack against the first two rounds of MD4 [159]. Even though these attacks could not be extended to the full algorithm, Rivest strengthened the algorithm. The result is MD5.

18.5 MD5

MD5 is an improved version of MD4 [1386, 1322]. Although more complex than MD4, it is similar in design and also produces a 128-bit hash.

Description of MD5

After some initial processing, MD5 processes the input text in 512-bit blocks, divided into 16 32-bit sub-blocks. The output of the algorithm is a set of four 32-bit blocks, which concatenate to form a single 128-bit hash value.

First, the message is padded so that its length is just 64 bits short of being a multiple of 512. This padding is a single 1-bit added to the end of the message, followed by as many zeros as are required. Then, a 64-bit representation of the message's length (before padding bits were added) is appended to the result. These two steps serve to make the message length an exact multiple of 512 bits in length (required for the rest of the algorithm), while ensuring that different messages will not look the same after padding.

Four 32-bit variables are initialized:

A = 0x01234567
B = 0x89abcdef
C = 0xfedcba98
D = 0x76543210

These are called chaining variables.

Now, the main loop of the algorithm begins. This loop continues for as many 512-bit blocks as are in the message.

The four variables are copied into different variables: a gets A, b gets B, c gets C, and d gets D.

The main loop has four rounds (MD4 had only three rounds), all very similar. Each round uses a different operation 16 times. Each operation performs a nonlinear function on three of a, b, c, and d. Then it adds that result to the fourth variable, a sub-block of the text and a constant. Then it rotates that result to the right a variable number of bits and adds the result to one of a, b, c, or d. Finally the result replaces one of a, b, c, or d. See Figures 18.5 and 18.6.

Figure 18.5 MD5 main loop.

There are four nonlinear functions, one used in each operation (a different one for each round).

F(X,Y,Z) = (X ⊥ Y) ⊦ ((¬ X) ⊥ Z)
G(X,Y,Z) = (X ⊥ Z) ¬ (Y (¬ Z))
H(X,Y,Z) = X ⊕ Y ⊕ Z
I(X,Y,Z) = Y ⊕ (X ⊦ (¬ Z))

(⊕ is XOR,⊥ is AND, ⊦ is OR, and ¬ is NOT.)

These functions are designed so that if the corresponding bits of X, Y, and Z are independent and unbiased, then each bit of the result will also be independent and unbiased. The function F is the bit-wise conditional: If X then Y else Z. The function H is the bit-wise parity operator.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

If Mj represents the j th sub-block of the message (from 0 to 15), and <<<s represents a left circular shift of s bits, the four operations are:

FF(a,b,c,d,Mj,s,ti) denotes a = b + ((a + F(b,c,d ) + Mj + ti) <<< s)
GG(a,b,c,d,Mj,s,ti) denotes a = b + ((a + G(b,c,d ) + Mj + ti) <<< s)
HH(a,b,c,d,Mj,s,ti) denotes a = b + ((a + H(b,c,d) + Mj + ti) <<< s)
II(a,b,c,d,Mj,s,ti) denotes a = b + ((a + I(b,c,d ) + Mj + ti) <<< s)

Figure 18.6 One MD5 operation.

The four rounds (64 steps) look like:

Round 1:
FF (a, b, c, d, M0, 7, 0xd76aa478)
FF (d, a, b, c, M1, 12, 0xe8c7b756)
FF (c, d, a, b, M2, 17, 0x242070db)
FF (b, c, d, a, M3, 22, 0xc1bdceee)
FF (a, b, c, d, M4, 7, 0xf57c0faf)
FF (d, a, b, c, M5, 12, 0x4787c62a)
FF (c, d, a, b, M6, 17, 0xa8304613)
FF (b, c, d, a, M7, 22, 0xfd469501)
FF (a, b, c, d, M8, 7, 0x698098d8)
FF (d, a, b, c, M9, 12, 0x8b44f7af)
FF (c, d, a, b, M10, 17, 0xffff5bb1)
FF (b, c, d, a, M11, 22, 0x895cd7be)
FF (a, b, c, d, M12, 7, 0x6b901122)
FF (d, a, b, c, M13, 12, 0xfd987193)
FF (c, d, a, b, M14, 17, 0xa679438e)
FF (b, c, d, a, M15, 22, 0x49b40821)
Round 2:
GG (a, b, c, d, M1, 5, 0xf61e2562)
GG (d, a, b, c, M6, 9, 0xc040b340)
GG (c, d, a, b, M11, 14, 0x265e5a51)
GG (b, c, d, a, M0, 20, 0xe9b6c7aa)
GG (a, b, c, d, M5, 5, 0xd62f105d)
GG (d, a, b, c, M10, 9, 0x02441453)
GG (c, d, a, b, M15, 14, 0xd8a1e681)
GG (b, c, d, a, M4, 20, 0xe7d3fbc8)
GG (a, b, c, d, M9, 5, 0x21e1cde6)
GG (d, a, b, c, M14, 9, 0xc33707d6)
GG (c, d, a, b, M3, 14, 0xf4d50d87)
GG (b, c, d, a, M8, 20, 0x455a14ed)
GG (a, b, c, d, M13, 5, 0xa9e3e905)
GG (d, a, b, c, M2, 9, 0xfcefa3f8)
GG (c, d, a, b, M7, 14, 0x676f02d9)
GG (b, c, d, a, M12, 20, 0x8d2a4c8a)
Round 3:
HH (a, b, c, d, M5, 4, 0xfffa3942)
HH (d, a, b, c, M8, 11, 0x8771f681)
HH (c, d, a, b, M11, 16, 0x6d9d6122)
HH (b, c, d, a, M14, 23, 0xfde5380c)
HH (a, b, c, d, M1, 4, 0xa4beea44)
HH (d, a, b, c, M4, 11, 0x4bdecfa9)
HH (c, d, a, b, M7, 16, 0xf6bb4b60)
HH (b, c, d, a, M10, 23, 0xbebfbc70)
HH (a, b, c, d, M13, 4, 0x289b7ec6)
HH (d, a, b, c, M0, 11, 0xeaa127fa)
HH (c, d, a, b, M3, 16, 0xd4ef3085)
HH (b, c, d, a, M6, 23, 0x04881d05)
HH (a, b, c, d, M9, 4, 0xd9d4d039)
HH (d, a, b, c, M12, 11, 0xe6db99e5)
HH (c, d, a, b, M15, 16, 0x1fa27cf8)
HH (b, c, d, a, M2, 23, 0xc4ac5665)
Round 4:
II (a, b, c, d, M0, 6, 0xf4292244)
II (d, a, b, c, M7, 10, 0x432aff97)
II (c, d, a, b, M14, 15, 0xab9423a7)
II (b, c, d, a, M5, 21, 0xfc93a039)
II (a, b, c, d, M12, 6, 0x655b59c3)
II (d, a, b, c, M3, 10, 0x8f0ccc92)
II (c, d, a, b, M10, 15, 0xffeff47d)
II (b, c, d, a, M1, 21, 0x85845dd1)
II (a, b, c, d, M8, 6, 0x6fa87e4f)
II (d, a, b, c, M15, 10, 0xfe2ce6e0)
II (c, d, a, b, M6, 15, 0xa3014314)
II (b, c, d, a, M13, 21, 0x4e0811a1)
II (a, b, c, d, M4, 6, 0xf7537e82)
II (d, a, b, c, M11, 10, 0xbd3af235)
II (c, d, a, b, M2, 15, 0x2ad7d2bb)
II (b, c, d, a, M9, 21, 0xeb86d391)

Those constants, ti, were chosen as follows:

In step i, ti is the integer part of 232*abs(sin(i)), where i is in radians.

After all of this, a, b, c, and d are added to A, B, C, D, respectively, and the algorithm continues with the next block of data. The final output is the concatenation of A, B, C, and D.

Security of MD5

Ron Rivest outlined the improvements of MD5 over MD4 [1322]:

1. A fourth round has been added.
2. Each step now has a unique additive constant.
3. The function G in round 2 was changed from ((X⊥ Y ) ⊦ (X⊥ Z ) ⊦ (Y⊥ Z )) to ((X⊥ Z ) ⊦ (Y⊥ ¬ Z )) to make G less symmetric.
4. Each step now adds in the result of the previous step. This promotes a faster avalanche effect.
5. The order in which message sub-blocks are accessed in rounds 2 and 3 is changed, to make these patterns less alike.
6. The left circular shift amounts in each round have been approximately optimized, to yield a faster avalanche effect. The four shifts used in each round are different from the ones used in other rounds.

Tom Berson attempted to use differential cryptanalysis against a single round of MD5 [144], but his attack is ineffective against all four rounds. A more successful attack by den Boer and Bosselaers produces collisions using the compression function in MD5 [203, 1331, 1336]. This does not lend itself to attacks against MD5 in practical applications, and it does not affect the use of MD5 in Luby-Rackoff-like encryption algorithms (see Section 14.11). It does mean that one of the basic design principles of MD5—to design a collision-resistant compression function—has been violated. Although it is true that "there seems to be a weakness in the compression function, but it has no practical impact on the security of the hash function" [1336], I am wary of using MD5.

18.6 MD2

MD2 is another 128-bit one-way hash function designed by Ron Rivest [801, 1335]. It, along with MD5, is used in the PEM protocols (see Section 24.10). The security of MD2 is dependent on a random permutation of bytes. This permutation is fixed, and depends on the digits of π. S0, S1, S2,..., S255 is the permutation. To hash a message M:

(1) Pad the message with i bytes of value i so that the resulting message is a multiple of 16 bytes long.
(2) Append a 16-byte checksum to the message.
(3) Initialize a 48-byte block: X0, X1, X2,..., X47. Set the first 16 bytes of X to be 0, the second 16 bytes of X to be the first 16 bytes of the message, and the third 16 bytes of X to be the XOR of the first 16 bytes of X and the second 16 bytes of X.
(4) This is the compression function:
t = 0
For j = 0 to 17
For k = 0 to 47
t = Xk XOR St
Xk = t
t = (t + j ) mod 256
(5) Set the second 16 bytes of X to be the second 16 bytes of the message, and the third 16 bytes of X to be the XOR of the first 16 bytes of X and the second 16 bytes of X. Do step (4). Repeat steps (5) and (4) with every 16 bytes of the message, in turn.
(6) The output is the first 16 bytes of X.

Although no weaknesses in MD2 have been found (see [1262]), it is slower than most other suggested hash functions.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

18.7 Secure Hash Algorithm (SHA)

NIST, along with the NSA, designed the Secure Hash Algorithm (SHA) for use with the Digital Signature Standard (see Section 20.2) [1154]. (The standard is the Secure Hash Standard (SHS); SHA is the algorithm used in the standard.) According to the Federal Register [539]:

A Federal Information Processing Standard (FIPS) for Secure Hash Standard (SHS) is being proposed. This proposed standard specified a Secure Hash Algorithm (SHA) for use with the proposed Digital Signature Standard .... Additionally, for applications not requiring a digital signature, the SHA is to be used whenever a secure hash algorithm is required for Federal applications.

And

This Standard specifies a Secure Hash Algorithm (SHA), which is necessary to ensure the security of the Digital Signature Algorithm (DSA). When a message of any length < 264 bits is input, the SHA produces a 160-bit output called a message digest. The message digest is then input to the DSA, which computes the signature for the message. Signing the message digest rather than the message often improves the efficiency of the process, because the message digest is usually much smaller than the message. The same message digest should be obtained by the verifier of the signature when the received version of the message is used as input to SHA. The SHA is called secure because it is designed to be computationally infeasible to recover a message corresponding to a given message digest, or to find two different messages which produce the same message digest. Any change to a message in transit will, with a very high probability, result in a different message digest, and the signature will fail to verify. The SHA is based on principles similar to those used by Professor Ronald L. Rivest of MIT when designing the MD4 message digest algorithm [1319], and is closely modelled after that algorithm.

SHA produces a 160-bit hash, longer than MD5.

Description of SHA

First, the message is padded to make it a multiple of 512 bits long. Padding is exactly the same as in MD5: First append a one, then as many zeros as necessary to make it 64 bits short of a multiple of 512, and finally a 64-bit representation of the length of the message before padding.

Five 32-bit variables (MD5 has four variables, but this algorithm needs to produce a 160-bit hash) are initialized as follows:

A = 0x67452301
B = 0xefcdab89
C = 0x98badcfe
D = 0x10325476
E = 0xc3d2e1f0

The main loop of the algorithm then begins. It processes the message 512 bits at a time and continues for as many 512-bit blocks as are in the message.

First the five variables are copied into different variables: a gets A, b gets B, c gets C, d gets D, and e gets E.

The main loop has four rounds of 20 operations each (MD5 has four rounds of 16 operations each). Each operation performs a nonlinear function on three of a, b, c, d, and e, and then does shifting and adding similar to MD5.

SHA's set of nonlinear functions is:

ft(X,Y,Z ) = (X⊥ Y) ⊦ ((¬ X )⊥ Z), for t = 0 to 19.
ft (X,Y,Z ) = X ⊕ Y ⊕ Z, for t = 20 to 39.
ft (X,Y,Z ) = (X⊥ Y ) ⊦ (X⊥ Z) ⊦ (Y⊥ Z), for t = 40 to 59.
ft (X,Y,Z ) = X ⊕ Y ⊕ Z, for t = 60 to 79.

Four constants are used in the algorithm:

Kt = 0x5a827999, for t = 0 to 19.
Kt = 0x6ed9eba1, for t = 20 to 39.
Kt = 0x8f1bbcdc, for t = 40 to 59.
Kt = 0xca62c1d6, for t = 60 to 79.

(If you wonder where those numbers came from: 0x5a827999 = 21/2 /4, 0x6ed9eba1 = 31/2 /4, 0x8f1bbcdc = 51/2 /4, and 0xca62c1d6 = 101/2 /4; all times 232.)

The message block is transformed from 16 32-bit words (M0 to M15 ) to 80 32-bit words (W0 to W79) using the following algorithm:

Wt = Mt, for t = 0 to 15
Wt = (Wt- 3 ⊕ Wt - 8 ⊕ Wt - 14 ⊕ Wt - 16 ) <<< 1, for t = 16 to 79.

(As an interesting aside, the original SHA specification did not have the left circular shift. The change "corrects a technical flaw that made the standard less secure than had been thought" [543]. The NSA has refused to elaborate on the exact nature of the flaw.)

If t is the operation number (from 0 to 79), Wt represents the t th sub-block of the expanded message, and <<< s represents a left circular shift of s bits, then the main loop looks like:

FOR t = 0 to 79
TEMP = (a <<< 5) + ft (b,c,d) + e + Wt + Kt
e = d
d = c
c = b <<< 30
b = a
a = TEMP

Figure 18.7 One SHA operation.

Figure 18.7 shows one operation. Shifting the variables accomplishes the same thing as MD5 does by using different variables in different locations.

After all of this, a, b, c, d, and e are added to A, B, C, D, and E respectively, and the algorithm continues with the next block of data. The final output is the concatenation of A, B, C, D, and E.

Security of SHA

SHA is very similar to MD4, but has a 160-bit hash value. The main changes are the addition of an expand transformation and the addition of the previous step's output into the next step for a faster avalanche effect. Ron Rivest made public the design decisions behind MD5, but SHA's designers did not. Here are Rivest's MD5 improvements to MD4 and how they compare with SHA's:

1. "A fourth round has been added." SHA does this, too. However, in SHA the fourth round uses the same f function as the second round.
2. "Each step now has a unique additive constant." SHA keeps the MD4 scheme where it reuses the constants for each group of 20 rounds.
3. "The function G in round 2 was changed from ((X⊥ Y) ⊦ (X⊥ Z) ⊦ (Y⊥ Z)) to ((X⊥ Z) ⊦ (Y⊥ ¬ (Z))) to make G less symmetric." SHA uses the MD4 version: ((X⊥ Y) ⊦ (X⊥ Z) ⊦ (Y⊥ Z)).
4. "Each step now adds in the result of the previous step. This promotes a faster avalanche effect." This change has been made in SHA as well. The difference in SHA is that a fifth variable is added, and not b, c, or d, which is already used in ft. This subtle change makes the den Boer-Bosselaers attack against MD5 impossible against SHA.
5. "The order in which message sub-blocks are accessed in rounds 2 and 3 is changed, to make these patterns less alike." SHA is completely different, since it uses a cyclic error-correcting code.
6. "The left circular shift amounts in each round have been approximately optimized, to yield a faster avalanche effect. The four shifts used in each round are different from the ones used in other rounds." SHA uses a constant shift amount in each round. This shift amount is relatively prime to the word size, as in MD4.

This leads to the following comparison: SHA is MD4 with the addition of an expand transformation, an extra round, and better avalanche effect; MD5 is MD4 with improved bit hashing, an extra round, and better avalanche effect.

There are no known cryptographic attacks against SHA. Because it produces a 160-bit hash, it is more resistant to brute-force attacks (including birthday attacks) than 128-bit hash functions covered in this chapter.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

18.8 RIPE-MD

RIPE-MD was developed for the European Community's RIPE project [1305] (see Section 25.7). The algorithm is a variation of MD4, designed to resist known cryptanalytic attacks, and produce a 128-bit hash value. The rotations and the order of the message words are modified. Additionally, two instances of the algorithm, differing only in the constants, run in parallel. After each block, the output of both instances are added to the chaining variables. This seems to make the algorithm highly resistant to cryptanalysis.

18.9 HAVAL

HAVAL is a variable-length one-way hash function [1646]. It is a modification of MD5. HAVAL processes messages in blocks of 1024 bits, twice those of MD5. It has eight 32-bit chaining variables, twice those of MD5. It has a variable number of rounds, from three to five (each of which has 16 steps), and it can produce a hash length of 128, 160, 192, 224, or 256 bits.

HAVAL replaces MD5's simple nonlinear functions with highly nonlinear 7-variable functions, each of which satisfies the strict avalanche criterion. Each round uses a single function, but in every step a different permutation is applied to the inputs. It has a new message order and every step (except those in the first round) uses a different additive constant. The algorithm also has two rotations.

The core of the algorithm is

TEMP = (f(j,A,B,C,D,E,F,G) <<< 7) + (H <<< 11) + M[i][r (j)] + K(j)
H = G; G = F; F = E; E = D; D = C; C = B; B = A; A = TEMP

The variable number of rounds and variable-length output mean there are 15 versions of this algorithm. Den Boer's and Bosselaers's attack against MD5 [203] does not apply to HAVAL because of the rotation of H.

18.10 Other One-Way Hash Functions

MD3 is yet another hash function designed by Ron Rivest. It had several flaws and never really made it out of the laboratory, although a description was recently published in [1335].

A group of researchers at the University of Waterloo have proposed a one-way hash function based on iterated exponentiation in GF(2593) [22]. In this scheme, a message is divided into 593-bit blocks; beginning with the first block, the blocks are successively exponentiated. Each exponent is the result of the computation with the previous block; the first exponent is given by an IV.

Ivan Damgård designed a one-way hash function based on the knapsack problem (see Section 19.2) [414]; it can be broken in about 232 operations [290, 1232, 787].

Steve Wolfram's cellular automata [1608] have been proposed as a basis for one-way hash functions. An early implementation [414] is insecure [1052, 404]. Another one-way hash function, Cellhash [384,404], and an improved version, Subhash [384,402, 405], are based on cellular automata; both are designed for hardware. Boognish mixes the design principles of Cellhash with those of MD4 [402, 407]. StepRightUp can be implemented as a hash function as well [402].

Claus Schnorr proposed a one-way hash function based on the discrete Fourier transform, called FFT-Hash, in the summer of 1991 [1399]; it was broken a few months later by two independent groups [403, 84]. Schnorr proposed a revised version, called FFT-Hash II (the previous version was renamed FFT-Hash I) [1400], which was broken a few weeks later [1567]. Schnorr has proposed further modifications [1402, 1403] but, as it stands, the algorithm is much slower than the others in this chapter. Another hash function, called SL2 [1526], is insecure [315].

Additional theoretical work on constructing one-way hash functions from one-way functions and one-way permutations can be found in [412, 1138, 1342].

18.11 One-Way Hash Functions Using Symmetric Block Algorithms

It is possible to use a symmetric block cipher algorithm as a one-way hash function. The idea is that if the block algorithm is secure, then the one-way hash function will also be secure.

The most obvious method is to encrypt the message with the algorithm in CBC or CFB mode, a fixed key, and IV; the last ciphertext block is the hash value. These methods are described in various standards using DES: both modes in [1143], CFB in [1145], CBC in [55, 56, 54]. This just isn't good enough for one-way hash functions, although it will work for a MAC (see Section 18.14) [29].

A cleverer approach uses the message block as the key, the previous hash value as the input, and the current hash value as the output.

The actual hash functions proposed are even more complex. The block size is usually the key length, and the size of the hash value is the block size. Since most block algorithms are 64 bits, several schemes are designed around a hash that is twice the block size.

Assuming the hash function is correct, the security of the scheme is based on the security of the underlying block function. There are exceptions, though. Differential cryptanalysis is easier against block functions in hash functions than against block functions used for encryption: The key is known, so several tricks can be applied; only one right pair is needed for success; and you can generate as much chosen plaintext as you want. Some work on these lines is [1263, 858, 1313].

What follows is a summary of the various hash functions that have appeared in the literature [925, 1465, 1262]. Statements about attacks against these schemes assume that the underlying block cipher is secure; that is, the best attack against them is brute force.

One useful measure for hash functions based on block ciphers is the hash rate, or the number of n-bit messages blocks, where n is the block size of the algorithm, processed per encryption. The higher the hash rate, the faster the algorithm. (This measure was given the opposite definition in [1262], but the definition given here is more intuitive and is more widely used. This can be confusing.)

Schemes Where the Hash Length Equals the Block Size

The general scheme is as follows (see Figure 18.8):

H0 = IH, where IH is a random initial value
Hi = EA(B) ⊕ C

where A, B, and C can be either Mi, H- 1, (Mi ⊕ Hi - 1), or a constant (assumed to be 0). H0 is some random initial value: IH. The message is divided up into block-size chunks, Mi, and processed individually. And there is some kind of MD-strengthening, perhaps the same padding procedure used in MD5 and SHA.

Figure 18.8 General hash function where the hash length equals the block size.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 18.1
Secure Hash Functions Where the Block Length Equals the Hash Size
Hi = EHi- 1 (Mi) ⊕ Mi
Hi = EHi - 1(Mi ⊕ Hi- 1 ) ⊕ Mi ⊕ Hi- 1
Hi = EHi- 1(Mi) ⊕ Hi- 1 ⊕ Mi
Hi = EHi- 1(Mi ⊕ Hi- 1) ⊕ Mi
Hi = EMi(Hi - 1) ⊕ Hi- 1
Hi = EMi(Mi ⊕ Hi - 1 ) ⊕ Mi ⊕ Hi - 1
Hi = EMi(Hi - 1 ) ⊕ Mi ⊕ Hi - 1
Hi = EMi(Mi ⊕ Hi- 1) ⊕ Hi- 1
Hi = EMi⊕ Hi- 1(Mi) ⊕ Mi
Hi = EMi⊕ Hi- 1(Hi- 1) ⊕ Hi- 1
Hi = EMi⊕ Hi- 1(Mi ) ⊕ Hi- 1
Hi = EMi⊕ Hi- 1(Hi- 1 ) ⊕ Mi

The three different variables can take on one of four possible values, so there are 64 total schemes of this type. Bart Preneel studied them all [1262].

Fifteen are trivially weak because the result does not depend on one of the inputs. Thirty-seven are insecure for more subtle reasons. Table 18.1 lists the 12 secure schemes remaining: The first 4 are secure against all attacks (see Figure 18.9) and the last 8 are secure against all but a fixed-point attack, which is not really worth worrying about.

The first scheme was described in [1028]. The third scheme was described in [1555, 1105, 1106] and was proposed as an ISO standard [766]. The fifth scheme was proposed by Carl Meyer, but is commonly called Davies-Meyer in the literature [1606, 1607, 434, 1028]. The tenth scheme was proposed as a hash-function mode for LOKI [273].

The first, second, third, fourth, ninth, and eleventh schemes have a hash rate of 1; the key length equals the block length. The others have a rate of k/n, where k is the key length. This means that if the key length is shorter than the block length, then the message block can only be the length of the key. It is not recommended that the message block be longer than the key length, even if the encryption algorithm's key length is longer than the block length.

If the block algorithm has a DES-like complementation property and DES-like weak keys, there is an additional attack that is possible against all 12 schemes. The attack isn't very dangerous and not really worth worrying about. However, you can solve it by fixing bits 2 and 3 of the key to "01" or "10" [1081, 1107]. Of course, this reduces the length of k from 56 bits to 54 bits (in DES, for example) and decreases the hash rate.

The following schemes, proposed in the literature, have been shown to be insecure.

Figure 18.9 The four secure hash functions where the block length equals the hash size.

This scheme [1282] was broken in [369]:

Hi = EMi(Hi - 1)

Davies and Price proposed a variant which cycles the entire message through the algorithm twice [432, 433]. Coppersmith's attack works on this variant with not much larger computational requirements [369].

Another scheme [432, 458] was shown insecure in [1606]:

Hi = EMi⊕ Hi- 1 (Hi- 1)

This scheme was shown insecure in [1028] (c is a constant):

Hi = Ec (Mi ⊕ Hi - 1) ⊕ Mi ⊕ Hi- 1

Modified Davies-Meyer

Lai and Massey modified the Davies-Meyer technique to work with the IDEA cipher [930, 925]. IDEA has a 64-bit block size and 128-bit key size. Their scheme is

Ho = IH, where IH is a random initial value
Hi = EHi- 1,Mi(Hi- 1)

This function hashes the message in blocks of 64 bits and produces a 64-bit hash value (See Figure 18.10).

No known attack on this scheme is easier than brute force.

Figure 18.10 Modified Davies-Meyer.

Preneel-Bosselaers-Govaerts-Vandewalle

This hash function, first proposed in [1266], produces a hash value twice the block length of the encryption algorithm: A 64-bit algorithm produces a 128-bit hash.

With a 64-bit block algorithm, the scheme produces two 64-bit hash values, Gi and Hi, which are concatenated to produce the 128-bit hash. With most block algorithms, the block size is 64 bits. Two adjacent message blocks, Li and Ri, each the size of the block length, are hashed together.

G0 = IG, where IG is a random initial value
H0 = IH, where IH is another random initial value
Gi = ELi⊕ Hi- 1(Ri ⊕ Gi- 1 ) ⊕ Ri ⊕ Gi- 1 ⊕ Hi- 1
Hi = ELi⊕ Ri(Hi- 1 ⊕ Gi- 1 ) ⊕ Li ⊕ Gi- 1 ⊕ Hi- 1

Lai demonstrates attacks against this scheme that, in some instances, make the birthday attack trivially solvable [925, 926]. Preneel [1262] and Coppersmith [372] also have successful attacks against this scheme. Do not use it.

Quisquater-Girault

This scheme, first proposed in [1279], generates a hash that is twice the block length and has a hash rate of 1. It has two hash values, Gi and Hi, and two blocks, Li and Ri, are hashed together.

G0 = IG, where IG is a random initial value
H0 = IH, where IH is another random initial value
Wi = ELi(Gi - 1 ⊕ Ri) ⊕ Ri ⊕ Hi- 1
Gi = ERi(Wi ⊕ Li) ⊕ Gi- 1 ⊕ Hi- 1 ⊕ Li
Hi = Wi ⊕ Gi- 1


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

This scheme appeared in a 1989 draft ISO standard [764], but was dropped in a later version [765]. Security problems with this scheme were identified in [1107, 925, 1262, 372]. (Actually, the version in the proceedings was strengthened after the version presented at the conference was attacked.) In some instances the birthday attack is solvable with a complexity of 239, not 264, through brute force. Do not use this scheme.

LOKI Double-Block

This algorithm is a modification of Quisquater-Girault, specifically designed to work with LOKI [273]. All parameters are as in Quisquater-Girault.

G0 = IG, where IG is a random initial value
H0 = IH, where IH is another random initial value
Wi = ELi ⊕ Gi- 1 (Gi- 1 ⊕ Ri) ⊕ Ri ⊕ Hi- 1
Gi = ERi⊕ Hi- 1(Wi ⊕ Li) ⊕ Gi- 1 ⊕ Hi- 1 ⊕ Li
Hi = Wi ⊕ Gi- 1

Again, in some instances the birthday attack is trivially solvable [925, 926, 1262, 372, 736]. Do not use this scheme.

Parallel Davies-Meyer

This is yet another attempt at an algorithm with a hash rate of 1 that produces a hash twice the block length [736].

G0 = IG, where IG is a random initial value
H0 = IH, where IH is another random initial value
Gi = ELi⊕ Ri(Gi- 1 ⊕ Li) ⊕ Li ⊕ Hi- 1
Hi = ELi(Hi - 1 ⊕ Ri) ⊕ Ri ⊕ Hi- 1

Unfortunately, this scheme isn't secure either [928, 861]. As it turns out, a double-length hash function with a hash rate of 1 cannot be more secure than Davies-Meyer [861].

Tandem and Abreast Davies-Meyer

Another way around the inherent limitations of a block cipher with a 64-bit key uses an algorithm, like IDEA (see Section 13.9), with a 64-bit block and a 128-bit key. These two schemes produce a 128-bit hash value and have a hash rate of ½ [930, 925].

Figure 18.11 Tandem Davies-Meyer.

In this first scheme, two modified Davies-Meyer functions work in tandem (see Figure 18.11).

G0 = IG, where IG is some random initial value
H0 = IH, where IH is some other random initial value
Wi = EGi- 1, Mi(Hi- 1)
Gi = Gi- 1 ⊕ EMi,Wi(Gi- 1)
Hi = Wi ⊕ Hi- 1

The following scheme uses two modified Davies-Meyer functions side-by-side (see Figure 18.12).

G0 = IG, where IG is some random initial value
H0 = IH, where IH is some other random initial value
Gi = Gi- 1 ⊕ EMi,Hi- 1(¬Gi- 1)
Hi = Hi- 1 ⊕ EGi- 1,Mi(Hi- 1)

In both schemes, the two 64-bit hash values Gi and Hi are concatenated to produce a single 128-bit hash.

As far as anyone knows, these algorithms have ideal security for a 128-bit hash function: Finding a message that hashes to a given hash value requires 2128 attempts, and finding two random messages that hash to the same value requires 264 attempts—assuming that there is no better way to attack the block algorithm than by using brute force.

MDC-2 and MDC-4

MDC-2 and MDC-4 were first developed at IBM [1081, 1079]. MDC-2, sometimes called Meyer-Schilling, is under consideration as an ANSI and ISO standard [61, 765]; a variant was proposed in [762]. MDC-4 is specified for the RIPE project [1305] (see Section 25.7). The specifications use DES as the block function, although in theory any encryption algorithm could be used.

Figure 18.12 Abreast Davies-Meyer.

Figure 18.13 MDC-2.

MDC-2 has a hash rate of ½, and produces a hash value twice the length of the block size. It is shown in Figure 18.13. MDC-4 also produces a hash value twice the length of the block size, and has a hash rate of ¼ (see Figure 18.14).

These schemes have been analyzed in [925, 1262]. They are secure against current computing power, but they are not nearly as secure as the designers have estimated. If the block algorithm is DES, they have been looked at with respect to differential cryptanalysis [1262].

Both MDC-2 and MDC-4 are patented [223].

AR Hash Function

The AR hash function was developed by Algorithmic Research, Ltd. and has been distributed by the ISO for information purposes only [767]. Its basic structure is a variant of the underlying block cipher (DES in the reference) in CBC mode. The last two ciphertext blocks and a constant are XORed to the current message block and encrypted by the algorithm. The hash is the last two ciphertext blocks computed. The message is processed twice, with two different keys, so the hash function has a hash rate of ½. The first key is 0x0000000000000000, the second key is 0x2a41522f4446502a, and c is 0x0123456789abcdef. The result is compressed to a single 128-bit hash value. See [750] for the details.

Hi = EK (Mi ⊕ Hi- 1 ⊕ Hi- 2 ⊕ c ) ⊕ Mi

This sounds interesting, but it is insecure. After considerable preprocessing, it is possible to find collisions for this hash function easily [416].

Figure 18.14 MDC-4.

GOST Hash Function

This hash function comes from Russia, and is specified in the standard GOST R 34.11-94 [657]. It uses the GOST block algorithm (see Section 14.1), although in theory it could use any block algorithm with a 64-bit block size and a 256-bit key. The function produces a 256-bit hash value.

The compression function, Hi = f(Mi,Hi-1) (both operands are 256-bit quantities) is defined as follows:

(1) Generate four GOST encryption keys by some linear mixing of Mi, Hi - 1, and some constants.
(2) Use each key to encrypt a different 64 bits of Hi- 1 in ECB mode. Store the resulting 256 bits into a temporary variable, S.
(3) Hi is a complex, although linear, function of S, Mi, and Hi- 1.

The final hash of M is not the hash of the last block. There are actually three chaining variables: Hn is the hash of the last message block, Z is the sum mod 2256 of all the message blocks, and L is the length of the message. Given those variables and the padded last block, M', the final hash value is:

H = f(Z ⊕ M', f(L, f(M',Hn)))

The documentation is a bit confusing (and in Russian), but I think all that is correct. In any case, this hash function is specified for use with the Russian Digital Signature Standard (see Section 20.3).


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Other Schemes

Ralph Merkle proposed a scheme using DES, but it's slow; it only processes seven message bits per iteration and each iteration involves two DES encryptions [1065, 1069]. Another scheme [1642, 1645] is insecure [1267]; it was once proposed as an ISO standard.

18.12 Using Public-Key Algorithms

It is possible to use a public-key encryption algorithm in a block chaining mode as a one-way hash function. If you then throw away the private key, breaking the hash would be as difficult as reading the message without the private key.

Here's an example using RSA. If M is the message to be hashed, n is the product of two primes p and q, and e is another large number relatively prime to (p - 1)(q - 1), then the hash function, H(M ), would be

H(M ) = Me mod n

An even easier solution would be to use a single strong prime as the modulus p. Then:

H(M ) = Me mod p

Breaking this problem is probably as difficult as finding the discrete logarithm of e. The problem with this algorithm is that it's far slower than any others discussed here. I don't recommend it for that reason.

18.13 Choosing a One-Way Hash Function

The contenders seem to be SHA, MD5, and constructions based on block ciphers; the others really haven't been studied enough to be in the running. I vote for SHA. It has a longer hash value than MD5, is faster than the various block-cipher constructions, and was developed by the NSA. I trust the NSA's abilities at cryptanalysis, even if they don't make their results public.

Table 18.2 gives timing measurements for some hash functions.y They are meant for comparison purposes only.

18.14 Message Authentication Codes

A message authentication code, or MAC, is a key-dependent one-way hash function. MACs have the same properties as the one-way hash functions discussed previously, but they also include a key. Only someone with the identical key can verify the hash. They are very useful to provide authenticity without secrecy.

MACs can be used to authenticate files between users. They can also be used by a single user to determine if his files have been altered, perhaps by a virus. A user could compute the MAC of his files and store that value in a table. If the user used instead a one-way hash function, then the virus could compute the new hash value after infection and replace the table entry. A virus could not do that with a MAC, because the virus does not know the key.

Table 18.2 Speeds of Some Hash Functions on a 33 MHz 486SX
Algorithm | Hash Length | Encryption Speed (kilobytes/second)
Abreast Davies-Meyer (with IDEA) | 128 | 22
Davies-Meyer (with DES) | 64 | 9
GOST Hash | 256 | 11
HAVAL (3 passes) | variable | 168
HAVAL (4 passes) | variable | 118
HAVAL (5 passes) | variable | 95
MD2 | 128 | 23
MD4 | 128 | 236
MD5 | 128 | 174
N-HASH (12 rounds) | 128 | 29
N-HASH (15 rounds) | 128 | 24
RIPE-MD | 128 | 182
SHA | 160 | 75
SNEFRU (4 passes) | 128 | 48
SNEFRU (8 passes) | 128 | 23

An easy way to turn a one-way hash function into a MAC is to encrypt the hash value with a symmetric algorithm. Any MAC can be turned into a one-way hash function by making the key public.

CBC-MAC

The simplest way to make a key-dependent one-way hash function is to encrypt a message with a block algorithm in CBC or CFB modes. The hash is the last encrypted block, encrypted once more in CBC or CFB modes. The CBC method is specified in ANSI X9.9 [54], ANSI X9.19 [56], ISO 8731-1 [759], ISO 9797 [763], and an Australian standard [1496]. Differential cryptanalysis can break this scheme with reduced-round DES or FEAL as the underlying block algorithms [1197].

The potential security problem with this method is that the receiver must have the key, and that key allows him to generate messages with the same hash value as a given message by decrypting in the reverse direction.

Message Authenticator Algorithm (MAA)

This algorithm is an ISO standard [760]. It produces a 32-bit hash, and was designed for mainframe computers with a fast multiply instruction [428].

v = v <<< 1
e = v ⊕ w
x = ((((e + y ) mod 232) ⊦ A⊥ C) * (x ⊕ Mi)) mod 232 - 1
y = ((((e + x) mod 232) ⊦ B⊥ D) * (y ⊕ Mi)) mod 232 - 2

Iterate these for each message block, Mi, and the resultant hash is the XOR of x and y. The variables v and e are determined from the key. A, B, C, and D are constants.

This algorithm is probably in wide use, but I can't believe it is all that secure. It was designed a long time ago, and isn't very complicated.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Bidirectional MAC

This MAC produces a hash value twice the length of the block algorithm [978]. First, compute the CBC-MAC of the message. Then, compute the CBC-MAC of the message with the blocks in reverse order. The bidirectional MAC value is simply the concatenation of the two. Unfortunately, this construction is insecure [1097].

Jueneman's Methods

This MAC is also called a quadratic congruential manipulation detection code (QCMDC) [792, 789]. First, divide the message into m- bit blocks. Then:

H0 = IH, where IH is the secret key
Hi = (Hi- 1 + Mi)2 mod p, where p is a prime less than 2m - 1 and + denotes integer addition

Jueneman suggests n = 16 and p = 231 - 1. In [792] he also suggests that an additional key be used as H1, with the actual message starting at H2.

Because of a variety of birthday-type attacks discovered in conjunction with Don Coppersmith, Jueneman suggested computing the QCMDC four times, using the result of one iteration as the IV for the next iteration, and then concatenating the results to obtain a 128-bit hash value [793]. This was further strengthened by doing the four iterations in parallel and cross-linking them [790, 791]. This scheme was broken by Coppersmith [376].

Another variant [432, 434] replaced the addition operation with an XOR and used message blocks significantly smaller than p. H0 was also set, making it a keyless one-way hash function. After this scheme was attacked [612], it was strengthened as part of the European Open Shop Information-TeleTrust project [1221], quoted in CCITT X.509 [304], and adopted in ISO 10118 [764, 765]. Unfortunately, Coppersmith has broken this scheme as well [376]. There has been some research using exponents other than 2 [603], but none of it has been promising.

RIPE-MAC

RIPE-MAC was invented by Bart Preneel [1262] and adopted by the RIPE project [1305] (see Section 18.8). It is based on ISO 9797 [763], and uses DES as a block encryption function. RIPE-MAC has two flavors: one using normal DES, called RIPE-MAC1, and another using triple-DES for even greater security, called RIPE-MAC3. RIPE-MAC1 uses one DES encryption per 64-bit message block; RIPE-MAC3 uses three.

The algorithm consists of three parts. First, the message is expanded to a length that is a multiple of 64 bits. Next, the expanded message is divided up into 64-bit blocks. A keyed compression function is used to hash these blocks, under the control of a secret key, into a single block of 64 bits. This is the step that uses either DES or triple-DES. Finally, the output of this compression is subjected to another DES-based encryption with a different key, derived from the key used in the compression. See [1305] for details.

IBC-Hash

IBC-Hash is another MAC adopted by the RIPE project [1305] (see Section 18.8). It is interesting because it is provably secure; the chance of successful attack can be quantified. Unfortunately, every message must be hashed with a different key. The chosen level of security puts constraints on the maximum message size that can be hashed—something no other function in this chapter does. Given these considerations, the RIPE report recommends that IBC-Hash be used only for long, infrequently sent messages.

The heart of the function is

hi = ((Mi mod p) + v ) mod 2n

The secret key is the pair p and v, where p is an N- bit prime and v is a random number less than 2n. The Mi values are derived by a carefully specified padding procedure. The probabilities of breaking both the one-wayness and the collision-resistance can be quantified, and users can choose their security level by changing the parameters.

One-Way Hash Function MAC

A one-way hash function can also be used as a MAC [1537]. Assume Alice and Bob share a key K, and Alice wants to send Bob a MAC for message M. Alice concatenates K and M, and computes the one-way hash of the concatenation: H (K,M ). This hash is the MAC. Since Bob knows K, he can reproduce Alice's result. Mallory, who does not know K, can't.

This method works with MD-strengthening techniques, but has serious problems. Mallory can always add new blocks to the end of the message and compute a valid MAC. This attack can be thwarted if you put the message length at the beginning, but Preneel is suspicious of this scheme [1265]. It is better to put the key at the end of the message, H (M,K ), but this has some problems as well [1265]. If H is one-way but not collision-free, Mallory can forge messages. Still better is H (K,M,K ), or H (K1,M,K2 ), where K1 and K2 are different [1537]. Preneel is still suspicious [1265].

The following constructions seem secure:

H (K1,H(K2, M))
H (K, H (K,M))
H (K,p,M,K ), where p pads K to a full message block.

Figure 18.15 Stream cipher MAC.

The best approach is to concatenate at least 64 bits of the key with each message block. This makes the one-way hash function less efficient, because the message blocks are smaller, but it is much more secure [1265].

Alternatively, use a one-way hash function and a symmetric algorithm. Hash the file, then encrypt the hash. This is more secure than first encrypting the file and then hashing the encrypted file, but it is vulnerable to the same attack as the H (M,K ) approach [1265].

Stream Cipher MAC

This MAC scheme uses stream ciphers (see Figure 18.15) [932]. A cryptographically secure pseudo-random-bit generator demultiplexes the message stream into two substreams. If the output bit of the bit generator ki, is 1, then the current message bit mi, is routed to the first substream; if the ki is 0, the mi is routed to the second substream. The substreams are each fed into a different LFSR (see Section 16.2). The output of the MAC is simply the final states of the shift registers.

Unfortunately, this method is not secure against small changes in the message [1523]. For example, if you alter the last bit of the message, then only 2 bits in the corresponding MAC value need to be altered to create a fake MAC; this can be done with reasonable probability. The author presents a more secure, and more complicated, alternative.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 19
Public-Key Algorithms

19.1 Background

The concept of public-key cryptography was invented by Whitfield Diffie and Martin Hellman, and independently by Ralph Merkle. Their contribution to cryptography was the notion that keys could come in pairs—an encryption key and a decryption key—and that it could be infeasible to generate one key from the other (see Section 2.5). Diffie and Hellman first presented this concept at the 1976 National Computer Conference [495]; a few months later, their seminal paper "New Directions in Cryptography" was published [496]. (Due to a glacial publishing process, Merkle's first contribution to the field didn't appear until 1978 [1064].)

Since 1976, numerous public-key cryptography algorithms have been proposed. Many of these are insecure. Of those still considered secure, many are impractical. Either they have too large a key or the ciphertext is much larger than the plaintext.

Only a few algorithms are both secure and practical. These algorithms are generally based on one of the hard problems discussed in Section 11.2. Of these secure and practical public-key algorithms, some are only suitable for key distribution. Others are suitable for encryption (and by extension for key distribution). Still others are only useful for digital signatures. Only three algorithms work well for both encryption and digital signatures: RSA, ElGamal, and Rabin. All of these algorithms are slow. They encrypt and decrypt data much more slowly than symmetric algorithms; usually that's too slow to support bulk data encryption.

Hybrid cryptosystems (see Section 2.5) speed things up: A symmetric algorithm with a random session key is used to encrypt the message, and a public-key algorithm is used to encrypt the random session key.

Security of Public-Key Algorithms

Since a cryptanalyst has access to the public key, he can always choose any message to encrypt. This means that a cryptanalyst, given C = EK(P), can guess the value of P and easily check his guess. This is a serious problem if the number of possible plaintext messages is small enough to allow exhaustive search, but can be solved by padding messages with a string of random bits. This makes identical plaintext messages encrypt to different ciphertext messages. (For more about this concept, see Section 23.15.)

This is especially important if a public-key algorithm is used to encrypt a session key. Eve can generate a database of all possible session keys encrypted with Bob's public key. Sure, this requires a large amount of time and memory, but for a 40-bit exportable key or a 56-bit DES key, it's a whole lot less time and memory than breaking Bob's public key. Once Eve has generated the database, she will have his key and can read his mail at will.

Public-key algorithms are designed to resist chosen-plaintext attacks; their security is based both on the difficulty of deducing the secret key from the public key and the difficulty of deducing the plaintext from the ciphertext. However, most public-key algorithms are particularly susceptible to a chosen-ciphertext attack (see Section 1.1).

In systems where the digital signature operation is the inverse of the encryption operation, this attack is impossible to prevent unless different keys are used for encryption and signatures.

Consequently, it is important to look at the whole system and not just at the individual parts. Good public-key protocols are designed so that the various parties can't decrypt arbitrary messages generated by other parties—the proof-of-identity protocols are a good example (see Section 5.2).

19.2 Knapsack Algorithms

The first algorithm for generalized public-key encryption was the knapsack algorithm developed by Ralph Merkle and Martin Hellman [713, 1074]. It could only be used for encryption, although Adi Shamir later adapted the system for digital signatures [1413]. Knapsack algorithms get their security from the knapsack problem, an NP-complete problem. Although this algorithm was later found to be insecure, it is worth examining because it demonstrates how an NP-complete problem can be used for public-key cryptography.

The knapsack problem is a simple one. Given a pile of items, each with different weights, is it possible to put some of those items into a knapsack so that the knapsack weighs a given amount? More formally: Given a set of values M1, M2,..., Mn , and a sum S, compute the values of bi such that

S = b1M1 + b2M2 + ...+ bnMn

The values of bi can be either zero or one. A one indicates that the item is in the knapsack; a zero indicates that it isn't.

For example, the items might have weights of 1, 5, 6, 11, 14, and 20. You could pack a knapsack that weighs 22; use weights 5, 6, and 11. You could not pack a knapsack that weighs 24. In general, the time required to solve this problem seems to grow exponentially with the number of items in the pile.

The idea behind the Merkle-Hellman knapsack algorithm is to encode a message as a solution to a series of knapsack problems. A block of plaintext equal in length to the number of items in the pile would select the items in the knapsack (plaintext bits corresponding to the b values), and the ciphertext would be the resulting sum. Figure 19.1 shows a plaintext encrypted with a sample knapsack problem.

The trick is that there are actually two different knapsack problems, one solvable in linear time and the other believed not to be. The easy knapsack can be modified to create the hard knapsack. The public key is the hard knapsack, which can easily be used to encrypt but cannot be used to decrypt messages. The private key is the easy knapsack, which gives an easy way to decrypt messages. People who don't know the private key are forced to try to solve the hard knapsack problem.

Superincreasing Knapsacks

What is the easy knapsack problem? If the list of weights is a superincreasing sequence, then the resulting knapsack problem is easy to solve. A superincreasing sequence is a sequence in which every term is greater than the sum of all the previous terms. For example, {1, 3, 6, 13, 27, 52} is a superincreasing sequence, but {1, 3, 4, 9, 15, 25} is not.

The solution to a superincreasing knapsack is easy to find. Take the total weight and compare it with the largest number in the sequence. If the total weight is less than the number, then it is not in the knapsack. If the total weight is greater than or equal to the number, then it is in the knapsack. Reduce the weight of the knapsack by the value and move to the next largest number in the sequence. Repeat until finished. If the total weight has been brought to zero, then there is a solution. If the total weight has not, there isn't.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

For example, consider a total knapsack weight of 70 and a sequence of weights of {2, 3, 6, 13, 27, 52}. The largest weight, 52, is less than 70, so 52 is in the knapsack. Subtracting 52 from 70 leaves 18. The next weight, 27, is greater than 18, so 27 is not in the knapsack. The next weight, 13, is less than 18, so 13 is in the knapsack. Subtracting 13 from 18 leaves 5. The next weight, 6, is greater than 5, so 6 is not in the knapsack. Continuing this process will show that both 2 and 3 are in the knapsack and the total weight is brought to 0, which indicates that a solution has been found. Were this a Merkle-Hellman knapsack encryption block, the plaintext that resulted from a ciphertext value of 70 would be 110101.

Non-superincreasing, or normal, knapsacks are hard problems; they have no known quick algorithm. The only known way to determine which items are in the knapsack is to methodically test possible solutions until you stumble on the correct one. The fastest algorithms, taking into account the various heuristics, grow exponentially with the number of possible weights in the knapsack. Add one item to the sequence of weights, and it takes twice as long to find the solution. This is much more difficult than a superincreasing knapsack where, if you add one more weight to the sequence, it simply takes another operation to find the solution.

Figure 19.1 Encryption with knapsacks.

The Merkle-Hellman algorithm is based on this property. The private key is a sequence of weights for a superincreasing knapsack problem. The public key is a sequence of weights for a normal knapsack problem with the same solution. Merkle and Hellman developed a technique for converting a superincreasing knapsack problem into a normal knapsack problem. They did this using modular arithmetic.

Creating the Public Key from the Private Key

Without going into the number theory, this is how the algorithm works: To get a normal knapsack sequence, take a superincreasing knapsack sequence, for example {2, 3, 6, 13, 27, 52}, and multiply all of the values by a number n, mod m. The modulus should be a number greater than the sum of all the numbers in the sequence: for example, 105. The multiplier should have no factors in common with the modulus: for example, 31. The normal knapsack sequence would then be

2 * 31 mod 105 = 62
3 * 31 mod 105 = 93
6 * 31 mod 105 = 81
13 * 31 mod 105 = 88
27 * 31 mod 105 = 102
52 * 31 mod 105 = 37

The knapsack would then be {62, 93, 81, 88, 102, 37}.

The superincreasing knapsack sequence is the private key. The normal knapsack sequence is the public key.

Encryption

To encrypt a binary message, first break it up into blocks equal to the number of items in the knapsack sequence. Then, allowing a one to indicate the item is present and a zero to indicate that the item is absent, compute the total weights of the knapsacks—one for every message block.

For example, if the message were 011000110101101110 in binary, encryption using the previous knapsack would proceed like this:

message = 011000 110101 101110
011000 corresponds to 93 + 81 = 174
110101 corresponds to 62 + 93 + 88 + 37 = 280
101110 corresponds to 62 + 81 + 88 + 102 = 333

The ciphertext would be

174,280,333

Decryption

A legitimate recipient of this message knows the private key: the original superincreasing knapsack, as well as the values of n and m used to transform it into a normal knapsack. To decrypt the message, the recipient must first determine n-1 such that n(n-1 ) ≡ 1 (mod m). Multiply each of the ciphertext values by n-1 mod m, and then partition with the private knapsack to get the plaintext values.

In our example, the superincreasing knapsack is {2, 3, 6, 13, 27, 52}, m is equal to 105, and n is equal to 31. The ciphertext message is 174, 280, 333. In this case n-1 is equal to 61, so the ciphertext values must be multiplied by 61 mod 105.

174 * 61 mod 105 = 9 = 3 + 6, which corresponds to 011000
280 * 61 mod 105 = 70 = 2 + 3 + 13 + 52, which corresponds to 110101
333 * 61 mod 105 = 48 = 2 + 6 + 13 + 27, which corresponds to 101110

The recovered plaintext is 011000 110101 101110.

Practical Implementations

With a knapsack sequence of only six items, it's not hard to solve the problem even if it isn't superincreasing. Real knapsacks should contain at least 250 items. The value for each term in the superincreasing knapsack should be somewhere between 200 and 400 bits long, and the modulus should be somewhere between 100 to 200 bits long. Real implementations of the algorithm use random-sequence generators to produce these values.

With knapsacks like that, it's futile to try to solve them by brute force. If a computer could try a million possibilities per second, trying all possible knapsack values would take over 1046 years. Even a million machines working in parallel wouldn't solve this problem before the sun went nova.

Security of Knapsacks

It wasn't a million machines that broke the knapsack cryptosystem, but a pair of cryptographers. First a single bit of plaintext was recovered [725]. Then, Shamir showed that knapsacks can be broken in certain circumstances [1415, 1416]. There were other results—[1428, 38, 754, 516, 488]—but no one could break the general Merkle-Hellman system. Finally, Shamir and Zippel [1418, 1419, 1421] found flaws in the transformation that allowed them to reconstruct the superincreasing knapsack from the normal knapsack. The exact arguments are beyond the scope of this book, but a nice summary of them can be found in [1233, 1244]. At the conference where the results were presented, the attack was demonstrated on stage using an Apple II computer [492, 494].

Knapsack Variants

Since the original Merkle-Hellman scheme was broken, many other knapsack systems have been proposed: multiple iterated knapsacks, Graham-Shamir knapsacks, and others. These have all been analyzed and broken, generally using the same cryptographic techniques, and litter the cryptographic highway [260, 253, 269, 921, 15, 919, 920, 922, 366, 254, 263, 255]. Good overviews of these systems and their cryptanalyses can be found in [267, 479, 257, 268].

Other algorithms have been proposed that use ideas similar to those used in knapsack cryptosystems, but these too have been broken. The Lu-Lee cryptosystem [990, 13] was broken in [20, 614, 873]; a modification [507] is also insecure [1620]. Attacks on the Goodman-McAuley cryptosystem are in [646, 647, 267, 268]. The Pieprzyk cryptosystem [1246] can be broken by similar attacks. The Niemi cryptosystem [1169], based on modular knapsacks, was broken in [345, 788]. A newer multistage knapsack [747] has not yet been broken, but I am not optimistic. Another variant is [294].

While a variation of the knapsack algorithm is currently secure—the Chor-Rivest knapsack [356], despite a "specialized attack" [743]—the amount of computation required makes it far less useful than the other algorithms discussed here. A variant, called the Powerline System, is not secure [958]. Most important, considering the ease with which all the other variations fell, it doesn't seem prudent to trust them.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Patents

The original Merkle-Hellman algorithm is patented in the United States [720] and worldwide (see Table 19.1). Public Key Partners (PKP) licenses the patent, along with other public-key cryptography patents (see Section 25.5). The U.S. patent will expire on August 19, 1997.

19.3 RSA

Soon after Merkle's knapsack algorithm came the first full-fledged public-key algorithm, one that works for encryption and digital signatures: RSA [1328, 1329]. Of all the public-key algorithms proposed over the years, RSA is by far the easiest to understand and implement. (Martin Gardner published an early description of the algorithm in his "Mathematical Games" column in Scientific American [599].) It is also the most popular. Named after the three inventors—Ron Rivest, Adi Shamir, and Leonard Adleman—it has since withstood years of extensive cryptanalysis. Although the cryptanalysis neither proved nor disproved RSA's security, it does suggest a confidence level in the algorithm.

Table 19.1 Foreign Merkle-Hellman Knapsack Patents
Country | Number | Date of Issue
Belgium | 871039 | 5 Apr 1979
Netherlands | 7810063 | 10 Apr 1979
Great Britain | 2006580 | 2 May 1979
Germany | 2843583 | 10 May 1979
Sweden | 7810478 | 14 May 1979
France | 2405532 | 8 Jun 1979
Germany | 2843583 | 3 Jun 1982
Germany | 2857905 | 15 Jul 1982
Canada | 1128159 | 20 Jul 1982
Great Britain | 2006580 | 18 Aug 1982
Switzerland | 63416114 | 14 Jan 1983
Italy | 1099780 | 28 Sep 1985

RSA gets its security from the difficulty of factoring large numbers. The public and private keys are functions of a pair of large (100 to 200 digits or even larger) prime numbers. Recovering the plaintext from the public key and the ciphertext is conjectured to be equivalent to factoring the product of the two primes.

To generate the two keys, choose two random large prime numbers, p and q. For maximum security, choose p and q of equal length. Compute the product:

n = pq

Then randomly choose the encryption key, e, such that e and (p - 1)(q - 1) are relatively prime. Finally, use the extended Euclidean algorithm to compute the decryption key, d, such that

ed ≡ 1 mod (p - 1)(q - 1)

In other words,

d = e-1 mod ((p - 1)(q - 1))

Note that d and n are also relatively prime. The numbers e and n are the public key; the number d is the private key. The two primes, p and q, are no longer needed. They should be discarded, but never revealed.

To encrypt a message m, first divide it into numerical blocks smaller than n (with binary data, choose the largest power of 2 less than n). That is, if both p and q are 100-digit primes, then n will have just under 200 digits and each message block, mi , should be just under 200 digits long. (If you need to encrypt a fixed number of blocks, you can pad them with a few zeros on the left to ensure that they will always be less than n.) The encrypted message, c, will be made up of similarly sized message blocks, ci, of about the same length. The encryption formula is simply

ci = mie mod n

To decrypt a message, take each encrypted block ci and compute

mi = cid mod n

Since

cid = (mie)d = mied = mik(p - 1)(q- 1)+ 1 = mi mik(p- 1)(q- 1) = mi*1 = mi; all (mod n)

the formula recovers the message. This is summarized in Table 19.2.

The message could just as easily have been encrypted with d and decrypted with e; the choice is arbitrary. I will spare you the number theory that proves why this works; most current texts on cryptography cover it in detail.

A short example will probably go a long way to making this clearer. If p = 47 and q = 71, then

Table 19.2 RSA Encryption
 | Public Key:
n | product of two primes, p and q (p and q must remain secret)
e | relatively prime to (p - 1)(q - 1)
 | Private Key:
d | e-1 mod ((p - 1)(q - 1))
 | Encrypting:
c = me mod n
 | Decrypting:
m = cd mod n
n = pq = 3337

The encryption key, e, must have no factors in common with

(p - 1)(q - 1) = 46 * 70 = 3220

Choose e (at random) to be 79. In that case

d = 79-1 mod 3220 = 1019

This number was calculated using the extended Euclidean algorithm (see Section 11.3). Publish e and n, and keep d secret. Discard p and q.

To encrypt the message

m = 6882326879666683


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

first break it into small blocks. Three-digit blocks work nicely in this case. The message is split into six blocks, mi, in which

m1 = 688
m2 = 232
m3 = 687
m4 = 966
m5 = 668
m6 = 003

The first block is encrypted as

68879 mod 3337 = 1570 = c1

Performing the same operation on the subsequent blocks generates an encrypted message:

c = 1570 2756 2091 2276 2423 158

Decrypting the message requires performing the same exponentiation using the decryption key of 1019, so

15701019 mod 3337 = 688 = m1

The rest of the message can be recovered in this manner.

RSA in Hardware

Much has been written on the subject of hardware implementations of RSA [1314, 1474, 1456, 1316, 1485, 874, 1222, 87, 1410, 1409, 1343, 998, 367, 1429, 523, 772]. Good survey articles are [258, 872]. Many different chips perform RSA encryption [1310, 252, 1101, 1317, 874, 69, 737, 594, 1275, 1563, 509, 1223]. A partial list of currently available RSA chips, from [150, 258], is listed in Table 19.3. Not all are available on the open market.

Speed of RSA

In hardware, RSA is about 1000 times slower than DES. The fastest VLSI hardware implementation for RSA with a 512-bit modulus has a throughput of 64 kilobits per second [258]. There are also chips that perform 1024-bit RSA encryption. Currently chips are being planned that will approach 1 megabit per second using a 512-bit modulus; they will probably be available in 1995. Manufacturers have also implemented RSA in smart cards; these implementations are slower.

In software, DES is about 100 times faster than RSA. These numbers may change slightly as technology changes, but RSA will never approach the speed of symmetric algorithms. Table 19.4 gives sample software speeds of RSA [918].

Software Speedups

RSA encryption goes much faster if you're smart about choosing a value of e. The three most common choices are 3, 17, and 65537 (216 + 1). (The binary representation of 65537 has only two ones, so it takes only 17 multiplications to exponentiate.) X.509 recommends 65537 [304], PEM recommends 3 [76], and PKCS #1 (see Section 24.14) recommends 3 or 65537 [1345]. There are no security problems with using any of these three values for e (assuming you pad messages with random values—see later section), even if a whole group of users uses the same value for e.

Table 19.3 Existing RSA Chips
Company | Clock Speed | Baud Rate Per 512 Bits | Clock Cycles Per 512 Bit Encryption | Technology | Bits per Chip | Number of Transistors
Alpha Techn. | 25 MHz | 13 K | .98 M | 2 micron | 1024 | 180,000
AT&T | 15 MHz | 19 K | .4 M | 1.5 micron | 298 | 100,000
British Telecom | 10 MHz | 5.1 K | 1 M | 2.5 micron | 256 | ——
Business Sim. Ltd. | 5 MHz | 3.8 K | .67 M | Gate Array | 32 | ——
Calmos Syst. Inc. | 20 MHz | 28 K | .36 M | 2 micron | 593 | 95,000
CNET | 25 MHz | 5.3 K | 2.3 M | 1 micron | 1024 | 100,000
Cryptech | 14 MHz | 17 K | .4 M | Gate Array | 120 | 33,000
Cylink | 30 MHz | 6.8 K | 1.2 M | 1.5 micron | 1024 | 150,000
GEC Marconi | 25 MHz | 10.2 K | .67 M | 1.4 micron | 512 | 160,000
Pijnenburg | 25 MHz | 50 K | .256 M | 1 micron | 1024 | 400,000
Sandia | 8 MHz | 10 K | .4 M | 2 micron | 272 | 86,000
Siemens | 5 MHz | 8.5 K | .3 M | 1 micron | 512 | 60,000


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Table 19.4 RSA Speeds for Different Modulus Lengths with an 8-bit Public Key (on a SPARC II)
 | 512 bits | 768 bits | 1, 024 bits
Encrypt | 0.03 sec | 0.05 sec | 0.08 sec
Decrypt | 0.16 sec | 0.48 sec | 0.93 sec
Sign | 0.16 sec | 0.52 sec | 0.97 sec
Verify | 0.02 sec | 0.07 sec | 0.08 sec

Private key operations can be speeded up with the Chinese remainder theorem if you save the values of p and q, and additional values such as d mod (p - 1), d mod (q - 1), and q-1 mod p [1283, 1276]. These additional numbers can easily be calculated from the private and public keys.

Security of RSA

The security of RSA depends wholly on the problem of factoring large numbers. Technically, that's a lie. It is conjectured that the security of RSA depends on the problem of factoring large numbers. It has never been mathematically proven that you need to factor n to calculate m from c and e. It is conceivable that an entirely different way to cryptanalyze RSA might be discovered. However, if this new way allows the cryptanalyst to deduce d, it could also be used as a new way to factor large numbers. I wouldn't worry about it too much.

It is also possible to attack RSA by guessing the value of (p - 1)(q - 1). This attack is no easier than factoring n [1616].

For the ultraskeptical, some RSA variants have been proved to be as difficult as factoring (see Section 19.5). Also look at [36], which shows that recovering even certain bits of information from an RSA-encrypted ciphertext is as hard as decrypting the entire message.

Factoring n is the most obvious means of attack. Any adversary will have the public key, e, and the modulus, n. To find the decryption key, d, he has to factor n. Section 11.4 discusses the current state of factoring technology. Currently, a 129-decimal-digit modulus is at the edge of factoring technology. So, n must be larger than that. Read Section 7.2 on public key length.

It is certainly possible for a cryptanalyst to try every possible d until he stumbles on the correct one. This brute-force attack is even less efficient than trying to factor n.

From time to time, people claim to have found easy ways to break RSA, but to date no such claim has held up. For example, in 1993 a draft paper by William Payne proposed a method based on Fermat's little theorem [1234]. Unfortunately, this method is also slower than factoring the modulus.

There's another worry. Most common algorithms for computing primes p and q are probabilistic; what happens if p or q is composite? Well, first you can make the odds of that happening as small as you want. And if it does happen, the odds are that encryption and decryption won't work properly—you'll notice right away. There are a few numbers, called Carmichael numbers, which certain probabilistic primality algorithms will fail to detect. These are exceedingly rare, but they are insecure [746]. Honestly, I wouldn't worry about it.

Chosen Ciphertext Attack against RSA

Some attacks work against the implementation of RSA. These are not attacks against the basic algorithm, but against the protocol. It's important to realize that it's not enough to use RSA. Details matter.

Scenario 1: Eve, listening in on Alice's communications, manages to collect a ciphertext message, c, encrypted with RSA in her public key. Eve wants to be able to read the message. Mathematically, she wants m, in which

m = cd

To recover m, she first chooses a random number, r, such that r is less than n. She gets Alice's public key, e. Then she computes

x = re mod n
y = xc mod n
t = r-1 mod n

If x = re mod n, then r = xd mod n.

Now, Eve gets Alice to sign y with her private key, thereby decrypting y. (Alice has to sign the message, not the hash of the message.) Remember, Alice has never seen y before. Alice sends Eve

u = yd mod n

Now, Eve computes

tu mod n = r-1yd mod n = r-1xdcd mod n = cd mod n = m

Eve now has m.

Scenario 2: Trent is a computer notary public. If Alice wants a document notarized, she sends it to Trent. Trent signs it with an RSA digital signature and sends it back. (No one-way hash functions are used here; Trent encrypts the entire message with his private key.)

Mallory wants Trent to sign a message he otherwise wouldn't. Maybe it has a phony timestamp; maybe it purports to be from another person. Whatever the reason, Trent would never sign it if he had a choice. Let's call this message m'.

First, Mallory chooses an arbitrary value x and computes y = xe mod n. He can easily get e; it's Trent's public key and must be public to verify his signatures. Then he computes m = ym' mod n, and sends m to Trent to sign. Trent returns m'd mod n. Now Mallory calculates (md mod n)x-1 mod n, which equals n'd mod n and is the signature of m'.

Actually, Mallory can use several methods to accomplish these same things [423, 458, 486]. The weakness they all exploit is that exponentiation preserves the multiplicative structure of the input. That is:

(xm)d mod n = xdmd mod n

Scenario 3: Eve wants Alice to sign m3 . She generates two messages, m1 and m2, such that

m3 ≡ m1m2 (mod n)

If Eve can get Alice to sign m1 and m2, she can calculate m3:

m3d = (m1d mod n)(m2d mod n )


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Moral: Never use RSA to sign a random document presented to you by a stranger. Always use a one-way hash function first. The ISO 9796 block format prevents this attack.

Common Modulus Attack on RSA

A possible RSA implementation gives everyone the same n, but different values for the exponents e and d. Unfortunately, this doesn't work. The most obvious problem is that if the same message is ever encrypted with two different exponents (both having the same modulus), and those two exponents are relatively prime (which they generally would be), then the plaintext can be recovered without either of the decryption exponents [1457].

Let m be the plaintext message. The two encryption keys are e1 and e2. The common modulus is n. The two ciphertext messages are:

c1 = me1 mod n
c2 = me2 mod n

The cryptanalyst knows n, e1, e2, c1, and c2. Here's how he recovers m.

Since e1 and e2 are relatively prime, the extended Euclidean algorithm can find r and s, such that

re1 + se2 = 1

Assuming r is negative (either r or s has to be, so just call the negative one r), then the extended Euclidean algorithm can be used again to calculate c1-1. Then

(c1-1)-r * C2s = m mod n

There are two other, more subtle, attacks against this type of system. One attack uses a probabilistic method for factoring n. The other uses a deterministic algorithm for calculating someone's secret key without factoring the modulus. Both attacks are described in detail in [449].

Moral: Don't share a common n among a group of users.

Low Encryption Exponent Attack against RSA

RSA encryption and signature verification are faster if you use a low value for e, but that can also be insecure [704]. If you encrypt e(e + 1)/2 linearly dependent messages with different public keys having the same value of e, there is an attack against the system. If there are fewer than that many messages, or if the messages are unrelated, there is no problem. If the messages are identical, then e messages are enough. The easiest solution is to pad messages with independent random values. This also ensures that me mod n ≠ me. Most real-world RSA implementations—PEM and PGP (see Sections 24.10 and 24.12), for example—do this.

Moral: Pad messages with random values before encrypting them; make sure m is about the same size as n.

Low Decryption Exponent Attack against RSA

Another attack, this one by Michael Wiener, will recover d, when d is up to one quarter the size of n and e is less than n [1596]. This rarely occurs if e and d are chosen at random, and cannot occur if e has a small value.

Moral: Choose a large value for d.

Lessons Learned

Judith Moore lists several restrictions on the use of RSA, based on the success of these attacks [1114, 1115]:

— Knowledge of one encryption/decryption pair of exponents for a given modulus enables an attacker to factor the modulus.
— Knowledge of one encryption/decryption pair of exponents for a given modulus enables an attacker to calculate other encryption/decryption pairs without having to factor n.
— A common modulus should not be used in a protocol using RSA in a communications network. (This should be obvious from the previous two points.)
— Messages should be padded with random values to prevent attacks on low encryption exponents.
— The decryption exponent should be large.

Remember, it is not enough to have a secure cryptographic algorithm. The entire cryptosystem must be secure, and the cryptographic protocol must be secure. A failure in any of those three areas makes the overall system insecure.

Attack on Encrypting and Signing with RSA

It makes sense to sign a message before encrypting it (see Section 2.7), but not everyone follows this practice. With RSA, there is an attack against protocols that encrypt before signing [48].

Alice wants to send a message to Bob. First she encrypts it with Bob's public key; then she signs it with her private key. Her encrypted and signed message looks like:

(meB mod nB)dA mod nA

Here's how Bob can claim that Alice sent him m' and not m. Realize that since Bob knows the factorization of nB (it's his modulus), he can calculate discrete logarithms with respect to nB. Therefore, all he has to do is to find an x such that

m'x = m mod nB

Then, if he can publish xeB as his new public exponent and keep nB as his modulus, he can claim that Alice sent him message m' encrypted in this new exponent.

This is a particularly nasty attack in some circumstances. Note that hash functions don't solve the problem. However, forcing a fixed encryption exponent for every user does.

Standards

RSA is a de facto standard in much of the world. The ISO almost, but not quite, created an RSA digital-signature standard; RSA is in an information annex to ISO 9796 [762]. The French banking community standardized on RSA [525], as have the Australians [1498]. The United States currently has no standard for public-key encryption, because of pressure from the NSA and patent issues. Many U.S. companies use PKCS (see Section 24.14), written by RSA Data Security, Inc. A draft ANSI banking standard specifies RSA [61].

Patents

The RSA algorithm is patented in the United States [1330], but not in any other country. PKP licenses the patent, along with other public-key cryptography patents (see Section 25.5). The U.S. patent will expire on September 20, 2000.

19.4 Pohlig-Hellman

The Pohlig-Hellman encryption scheme [1253] is similar to RSA. It is not a symmetric algorithm, because different keys are used for encryption and decryption. It is not a public-key scheme, because the keys are easily derivable from each other; both the encryption and decryption keys must be kept secret.

Like RSA,

C = Pe mod n
P = Cd mod n

where

ed ≡ 1 (mod some complicated number)

Unlike RSA, n is not defined in terms of two large primes, it must remain part of the secret key. If someone had e and n, they could calculate d. Without knowledge of e or d, an adversary would be forced to calculate

e = logPC mod n

We have already seen that this is a hard problem.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Patents

The Pohlig-Hellman algorithm is patented in the United States [722] and also in Canada. PKP licenses the patent, along with other public-key cryptography patents (see Section 25.5).

19.5 Rabin

Rabin's scheme [1283,1601] gets its security from the difficulty of finding square roots modulo a composite number. This problem is equivalent to factoring. Here is one implementation of this scheme.

First choose two primes, p and q, both congruent to 3 mod 4. These primes are the private key; the product n = pq is the public key.

To encrypt a message, M (M must be less than n), simply compute

C = M2 mod n

Decrypting the message is just as easy, but slightly more annoying. Since the receiver knows p and q, he can solve the two congruences using the Chinese remainder theorem. Compute

m1 = C(p + 1)/4 mod p
m2 = (p - C(p+ 1)/4) mod p
m3 = C(q + 1)/4 mod q
m4 = (q - C(q + 1)/4) mod q

Then choose an integer a = q(q-1 mod p) and a integer b = p(p-1 mod q). The four possible solutions are:

M1 = (am1 + bm3) mod n
M2 = (am1 + bm4) mod n
M3 = (am2 + bm3) mod n
M4 = (am2 + bm4) mod n

One of those four results, M1, M2, M3, or M4, equals M. If the message is English text, it should be easy to choose the correct Mi. On the other hand, if the message is a random-bit stream (say, for key generation or a digital signature), there is no way to determine which Mi is correct. One way to solve this problem is to add a known header to the message before encrypting.

Williams

Hugh Williams redefined Rabin's schemes to eliminate these shortcomings [1601]. In his scheme, p and q are selected such that

p ≡ 3 mod 8
q ≡ 7 mod 8

and

N = pq

Also, there is a small integer, S, such that J(S,N) = -1. (J is the Jacobi symbol—see Section 11.3). N and S are public. The secret key is k, such that

k = 1/2 * (1/4 * (p - 1) * (q - 1) + 1)

To encrypt a message M, compute c1 such that J(M,N) = (-1)c 1. Then, compute M' = (Sc 1 * M) mod N. Like Rabin's scheme, C = M'2 mod N. And c2 = M' mod 2. The final ciphertext message is the triple:

(C, c1, c2)

To decrypt C, the receiver computes M" using

Ck ≡ ±M" (mod N)

The proper sign of M" is given by c2. Finally,

M = (Sc 1 * (-1)c 1 * M") mod N

Williams refined this scheme further in [1603, 1604, 1605]. Instead of squaring the plaintext message, cube it. The large primes must be congruent to 1 mod 3; otherwise the public and private keys are the same. Even better, there is only one unique decryption for each encryption.

Both Rabin and Williams have an advantage over RSA in that they are provably as secure as factoring. However, they are completely insecure against a chosen-ciphertext attack. If you are going to use these schemes in instances where an attacker can mount this attack (for example, as a digital signature algorithm where an attacker can choose messages to be signed), be sure to use a one-way hash function before signing. Rabin suggested another way of defeating this attack: Append a different random string to each message before hashing and signing. Unfortunately, once you add a one-way hash function to the system it is no longer provably as secure as factoring [628], although adding hashing cannot weaken the system in any practical sense.

Other Rabin variants are [972, 909, 696, 697, 1439, 989]. A two-dimensional variant is in [866, 889].

19.6 ElGamal

The ElGamal scheme [518,519] can be used for both digital signatures and encryption; it gets its security from the difficulty of calculating discrete logarithms in a finite field.

To generate a key pair, first choose a prime, p, and two random numbers, g and x, such that both g and x are less than p. Then calculate

y = gx mod p

The public key is y, g, and p. Both g and p can be shared among a group of users. The private key is x.

ElGamal Signatures

To sign a message, M, first choose a random number, k, such that k is relatively prime to p - 1. Then compute

a = gk mod p

and use the extended Euclidean algorithm to solve for b in the following equation:

M = (xa + kb) mod (p - 1)

The signature is the pair: a and b. The random value, k, must be kept secret.

To verify a signature, confirm that

yaab mod p = gM mod p

Each ElGamal signature or encryption requires a new value of k, and that value must be chosen randomly. If Eve ever recovers a k that Alice used, she can recover Alice's private key, x. If Eve ever gets two messages signed or encrypted using the same k, even if she doesn't know what it is, she can recover x.

This is summarized in Table 19.5.

For example, choose p = 11 and g = 2. Choose private key x = 8. Calculate

y = gx mod p = 28 mod 11 = 3

The public key is y = 3, g = 2, and p = 11.

To authenticate M = 5, first choose a random number k = 9. Confirm that gcd(9, 10) = 1. Compute

a = gk mod p = 29 mod 11 = 6

and use the extended Euclidean algorithm to solve for b:

M = (ax + kb) mod (p - 1)
5 = (8 * 6 + 9 * b) mod 10


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The solution is b = 3, and the signature is the pair: a = 6 and b = 3.

Table 19.5 ElGamal Signatures
 | Public Key:
p | prime (can be shared among a group of users)
g | <p (can be shared among a group of users)
y | = gx mod p
 | Private Key:
x | <p
 | Signing:
k | choose at random, relatively prime to p - 1
a | (signature) = gk mod p
b | (signature) such that M = (xa + kb) mod (p - 1)
 | Verifying:
Accept as valid if yaab mod p = gM mod p

To verify a signature, confirm that

yaab mod p = gM mod p
36 63 mod 11 = 25 mod 11

A variant of ElGamal for signatures is in [1377]. Thomas Beth invented a variant of the ElGamal scheme suitable for proofs of identity [146]. There are variants for password authentication [312], and for key exchange [773]. And there are thousands more (see Section 20.4).

ElGamal Encryption

A modification of ElGamal can encrypt messages. To encrypt message M, first choose a random k, such that k is relatively prime to p - 1. Then compute

a = gk mod p
b = ykM mod p

The pair, a and b, is the ciphertext. Note that the ciphertext is twice the size of the plaintext.

To decrypt a and b, compute

M = b/ax mod p

Since ax ≡ gkx (mod p), and b/ax ≡ ykM/ax ≡ gxkM/gxk ≡ M (mod p), this all works (see Table 19.6). This is really the same as Diffie-Hellman key exchange (see Section 22.1), except that y is part of the key, and the encryption is multiplied by yk.

Speed

Table 19.7 gives sample software speeds of ElGamal [918].

Table 19.6 ElGamal Encryption
 | Public Key:
p | prime (can be shared among a group of users)
g | < p (can be shared among a group of users)
y | = gx mod p
 | Private Key:
x | < p
 | Encrypting:
k | choose at random, relatively prime to p - 1.
a | (ciphertext) = gk mod p
b | (ciphertext) = ykM mod p
 | Decrypting:
M (plaintext) = b/ax mod p

Patents

ElGamal is unpatented. But, before you go ahead and implement the algorithm, realize that PKP feels that this algorithm is covered under the Diffie-Hellman patent [718]. However, the Diffie-Hellman patent will expire on April 29, 1997, making ElGamal the first public-key cryptography algorithm suitable for encryption and digital signatures unencumbered by patents in the United States. I can hardly wait.

19.7 McEliece

In 1978 Robert McEliece developed a public-key cryptosystem based on algebraic coding theory [1041]. The algorithm makes use of the existence of a class of error-correcting codes, known as Goppa codes. His idea was to construct a Goppa code and disguise it as a general linear code. There is a fast algorithm for decoding Goppa codes, but the general problem of finding a code word of a given weight in a linear binary code is NP-complete. A good description of this algorithm can be found in [1233]; see also [1562]. Following is just a quick summary.

Let dH(x,y) denote the Hamming distance between x and y. The numbers n, k, and t are system parameters.

The private key has three parts: G' is a k * n generator matrix for a Goppa code that can correct t errors. P is an n * n permutation matrix. S is a k * k nonsingular matrix.

The public key is a k * n matrix G: G = SG'P.

Plaintext messages are strings of k bits, in the form of k-element vectors over GF(2).

To encrypt a message, choose a random n-element vector over GF(2), z, with Hamming distance less than or equal to t.

c = mG + z

To decrypt the ciphertext, first compute c' = cP-1. Then, using the decoding algorithm for the Goppa code, find m' such that dH(m' G, c') is less than or equal to t. Finally, compute m = m'S-1.

In his original paper, McEliece suggested that n = 1024, t = 50, and k = 524. These are the minimum values required for security.

Table 19.7 ElGamal Speeds for Different Modulus Lengths with a 160-bit Exponent (on a SPARC II)
 | 512 bits | 768 bits | 1024 bits
Encrypt | 0.33 sec | 0.80 sec | 1.09 sec
Decrypt | 0.24 sec | 0.58 sec | 0.77 sec
Sign | 0.25 sec | 0.47 sec | 0.63 sec
Verify | 1.37 sec | 5.12 sec | 9.30 sec


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Although the algorithm was one of the first public-key algorithms, and there were no successful cryptanalytic results against the algorithm, it has never gained wide acceptance in the cryptographic community. The scheme is two to three orders of magnitude faster than RSA, but has some problems. The public key is enormous: 219 bits long. The data expansion is large: The ciphertext is twice as long as the plaintext.

Some attempts at cryptanalysis of this system can be found in [8, 943, 1559, 306]. None of these were successful in the general case, although the similarity between the McEliece algorithm and knapsacks worried some.

In 1991, two Russian cryptographers claimed to have broken the McEliece system with some parameters [882]. Their paper contained no evidence to substantiate their claim, and most cryptographers discount the result. Another Russian attack, one that cannot be used directly against the McEliece system, is in [1447, 1448]. Extensions to McEliece can be found in [424, 1227, 976].

Other Algorithms Based on Linear Error-Correcting Codes

The Niederreiter algorithm [1167] is closely related to the McEliece algorithm, and assumes that the public key is a random parity-check matrix of an error-correcting code. The private key is an efficient decoding algorithm for this matrix.

Another algorithm, used for identification and digital signatures, is based on syndrome decoding [1501]; see [306] for comments. An algorithm based on error-correcting codes [1621] is insecure [698, 33, 31, 1560, 32].

19.8 Elliptic Curve Cryptosystems

Elliptic curves have been studied for many years and there is an enormous amount of literature on the subject. In 1985, Neal Koblitz and V. S. Miller independently proposed using them for public-key cryptosystems [867, 1095]. They did not invent a new cryptographic algorithm with elliptic curves over finite fields, but they implemented existing public-key algorithms, like Diffie-Hellman, using elliptic curves.

Elliptic curves are interesting because they provide a way of constructing "elements" and "rules of combining" that produce groups. These groups have enough familiar properties to build cryptographic algorithms, but they don't have certain properties that may facilitate cryptanalysis. For example, there is no good notion of "smooth" with elliptic curves. That is, there is no set of small elements in terms of which a random element has a good chance of being expressed by a simple algorithm. Hence, index calculus discrete logarithm algorithms do not work. See [1095] for more details.

Elliptic curves over the finite field GF(2n ) are particularly interesting. The arithmetic processors for the underlying field are easy to construct and are relatively simple to implement for n in the range of 130 to 200. They have the potential to provide faster public-key cryptosystems with smaller key sizes. Many public-key algorithms, like Diffie-Hellman, ElGamal, and Schnorr, can be implemented in elliptic curves over finite fields.

The mathematics here are complex and beyond the scope of this book. Those interested in this topic are invited to read the two references previously mentioned, and the excellent book by Alfred Menezes [1059]. Two analogues of RSA work in elliptic curves [890, 454]. Other papers are [23, 119, 1062, 869, 152, 871, 892, 25, 895, 353, 1061, 26, 913, 914, 915]. Elliptic curve cryptosystems with small key lengths are discussed in [701]. Next Computer Inc.'s Fast Elliptic Encryption (FEE) algorithm also uses elliptic curves [388]. FEE has the nice feature that the private key can be any easy-to-remember string. There are proposed public-key cryptosystems using hyperelliptic curves [868, 870, 1441, 1214].

19.9 LUC

Some cryptographers have developed generalizations of RSA that use various permutation polynomials instead of exponentiation. A variation called Kravitz-Reed, using irreducible binary polynomials [898], is insecure [451, 589]. Winfried MŸller and Wilfried Nöbauer use Dickson polynomials [1127, 1128, 965]. Rudolph Lidl and MŸller generalized this approach in [966, 1126] (a variant is called the Réidi scheme), and Nöbauer looked at its security in [1172, 1173]. (Comments on prime generation with Lucas functions are in [969, 967, 968, 598].) Despite all of this prior art, a group of researchers from New Zealand managed to patent this scheme in 1993, calling it LUC [1486, 521,1487].

The nth Lucas number, Vn(P,1), is defined as

Vn(P, 1) = PVn-1 (P, 1) - Vn- 2 (P,1)

There's a lot more theory to Lucas numbers; I'm ignoring all of it. A good theoretical treatment of Lucas sequences is in [1307, 1308]. A particularly nice description of the mathematics of LUC is in [1494, 708].

In any case, to generate a public-key/private-key key pair, first choose two large primes, p and q. Calculate n, the product of p and q. The encryption key, e, is a random number that is relatively prime to p - 1, q - 1, p + 1, and q + 1.

There are four possible decryption keys,

d = e-1 mod (lcm((p + 1), (q + 1)))
d = e-1 mod (lcm((p + 1), (q - 1)))
d = e-1 mod (lcm((p - 1), (q + 1)))
d = e-1 mod (lcm((p - 1), (q - 1)))

where lcm is the least common multiple.

The public key is d and n; the private key is e and n. Discard p and q.

To encrypt a message, P (P must be less than n), calculate

C = Ve(P, 1) (mod n )

And to decrypt:

P = Vd P, 1) (mod n), with the proper d

At best, LUC is no more secure than RSA. And recent, still-unpublished results show how to break LUC in at least some implementations. I just don't trust it.

19.10 Finite Automaton Public-Key Cryptosystems

Chinese cryptographer Tao Renji has developed a public-key algorithm based on finite automata [1301, 1302, 1303, 1300, 1304, 666]. Just as it is hard to factor the product of two large primes, it is also hard to factor the composition of two finite automata. This is especially so if one or both of them is nonlinear.

Much of this research took place in China in the 1980s and was published in Chinese. Renji is starting to write in English. His main result was that certain nonlinear automata (the quasilinear automata) possess weak inverses if, and only if, they have a certain echelon matrix structure. This property disappears if they are composed with another automaton (even a linear one). In the public-key algorithm, the secret key is an invertible quasilinear automaton and a linear automaton, and the corresponding public key can be derived by multiplying them out term by term. Data is encrypted by passing it through the public automaton, and decrypted by passing it through the inverses of its components (in some cases provided they have been set to a suitable initial state). This scheme works for both encryption and digital signatures.

The performance of such systems can be summed up by saying that like McEliece's system, they run much faster than RSA, but require longer keys. The keylength thought to give similar security to 512-bit RSA is 2792 bits, and to 1024-bit RSA is 4152 bits. For the former case, the system encrypts data at 20, 869 bytes/sec and decrypts data at 17, 117 bytes/sec, running on a 33 Mhz 80486.

Renji has published three algorithms. The first is FAPKC0. This is a weak system which uses linear components, and is primarily illustrative. Two serious systems, FAPKC1 and FAPKC2, use one linear and one nonlinear component each. The latter is more complex, and was developed in order to support identity-based operation.

As for their strength, quite a lot of work has been done on them in China (where there are now over 30 institutes publishing cryptography and security papers). One can see from the considerable Chinese language literature that the problem has been studied.

One possible attraction of FAPKC1 and FAPKC2 is that they are not encumbered by any U.S. patents. Thus, once the Diffie-Hellman patent expires in 1997, they will unquestionably be in the public domain.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 20
Public-Key Digital Signature Algorithms

20.1 Digital Signature Algorithm (DSA)

In August 1991, The National Institute of Standards and Technology (NIST) proposed the Digital Signature Algorithm (DSA) for use in their Digital Signature Standard (DSS). According to the Federal Register [538]:

A Federal Information Processing Standard (FIPS) for Digital Signature Standard (DSS) is being proposed. This proposed standard specifies a public-key digital signature algorithm (DSA) appropriate for Federal digital signature applications. The proposed DSS uses a public key to verify to a recipient the integrity of data and identity of the sender of the data. The DSS can also be used by a third party to ascertain the authenticity of a signature and the data associated with it.

This proposed standard adopts a public-key signature scheme that uses a pair of transformations to generate and verify a digital value called a signature.

And:

This proposed FIPS is the result of evaluating a number of alternative digital signature techniques. In making the selection NIST has followed the mandate contained in section 2 of the Computer Security Act of 1987 that NIST develop standards to "...assure the cost-effective security and privacy of Federal information and, among technologies offering comparable protection, on selecting the option with the most desirable operating and use characteristics."

Among the factors that were considered during this process were the level of security provided, the ease of implementation in both hardware and software, the ease of export from the U.S., the applicability of patents, impact on national security and law enforcement and the level of efficiency in both the signing and verification functions. A number of techniques were deemed to provide appropriate protection for Federal systems. The technique selected has the following desirable characteristics:

NIST expects it to be available on a royalty-free basis. Broader use of this technique resulting from public availability should be an economic benefit to the government and the public.

The technique selected provides for efficient implementation of the signature operations in smart card applications. In these applications the signing operations are performed in the computationally modest environment of the smart card while the verification process is implemented in a more computationally rich environment such as a personal computer, a hardware cryptographic module, or a mainframe computer.

Before it gets too confusing, let me review the nomenclature: DSA is the algorithm; the DSS is the standard. The standard employs the algorithm. The algorithm is part of the standard.

Reaction to the Announcement

NIST's announcement created a maelstrom of criticisms and accusations. Unfortunately, it was more political than academic. RSA Data Security, Inc., purveyors of the RSA algorithm, led the criticism against DSS. They wanted RSA, and not another algorithm, used as the standard. RSADSI makes a lot of money licensing the RSA algorithm, and a royalty-free digital signature standard would directly affect their bottom line. (Note: DSA is not necessarily free of patent infringements; I'll discuss that later.)

Before the algorithm was announced, RSADSI campaigned against a "common modulus," which might have given the government the ability to forge signatures. When the algorithm was announced without this common modulus, they attacked it on other grounds [154], both in letters to NIST and statements to the press. (Four letters to NIST appeared in [1326]. When reading them, keep in mind that at least two of the authors, Rivest and Hellman, had a financial interest in DSS's not being approved.)

Many large software companies that already licensed the RSA algorithm came out against the DSS. In 1982, the government had solicited public-key algorithms for a standard [537]. After that, there wasn't a peep out of NIST for nine years. Companies such as IBM, Apple, Novell, Lotus, Northern Telecom, Microsoft, DEC, and Sun had already spent large amounts of money implementing the RSA algorithm. They were not interested in losing their investment.

In all, NIST received 109 comments by the end of the first comment period on February 28, 1992.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Let's look at the criticisms against DSA, one by one.

1. DSA cannot be used for encryption or key distribution.
True, but not the point of the standard. This is a signature standard. NIST should have a standard for public-key encryption. NIST is committing a grave injustice to the American people by not implementing a public-key encryption standard. It is suspicious that this proposed digital signature standard cannot be used for encryption. (As it turns out, though, it can—see Section 23.3.) That does not mean that a signature standard is useless.
2. DSA was developed by the NSA, and there may be a trapdoor in the algorithm.
Much of the initial comments were just paranoia: "NIST's denial of information with no apparent justification does not inspire confidence in DSS, but intensifies concern that there is a hidden agenda, such as laying the groundwork for a national public-key cryptosystem that is in fact vulnerable to being broken by NIST and/or NSA" [154]. One serious question about the security of DSA was raised by Arjen Lenstra and Stuart Haber at Bellcore. This will be discussed later.
3. DSA is slower than RSA [800].
True, more or less. Signature generation speeds are the same, but signature verification can be 10 to 40 times slower with DSA. Key generation, however, is faster. But key generation is irrelevant; a user rarely does it. On the other hand, signature verification is the most common operation.
The problem with this criticism is that there are many ways to play with the test parameters, depending on the results you want. Precomputations can speed up DSA signature generation, but don't always apply. Proponents of RSA use numbers optimized to make their calculations easier; proponents of DSA use their own optimizations. In any case, computers are getting faster all the time. While there is a speed difference, it will not be noticeable in most applications.
4. RSA is a de facto standard.
Here are two examples of this complaint. From Robert Follett, the program director of standards at IBM [570]:

IBM is concerned that NIST has proposed a standard with a different digital signature scheme rather than adopting the international standard. We have been convinced by users and user organizations that the international standards using RSA will be a prerequisite to the sales of security products in the very near future.

From Les Shroyer, vice president and director, corporate MIS and telecommunications, at Motorola [1444]:

We must have a single, robust, politically-accepted digital signature standard that is usable throughout the world, between both U.S. and non-U.S., and Motorola and non-Motorola entities. The lack of other viable digital signature technology for the last eight years has made RSA a de facto standard.... Motorola and many other companies... have committed millions of dollars to RSA. We have concern over the interoperability and support of two different standards, as that situation will lead to added costs, delays in deployment, and complication....

Many companies wanted NIST to adopt the ISO 9796, the international digital signature standard that uses RSA [762]. While this is a valid complaint, it is not a sufficient justification to make it a standard. A royalty-free standard would better serve the U.S. public interest.
5. The DSA selection process was not public; sufficient time for analysis has not been provided.
First NIST claimed that they designed the DSA; then they admitted that NSA helped them. Finally, they confirmed that NSA designed the algorithm. This worries many people; the NSA doesn't inspire trust. Even so, the algorithm is public and available for analysis; and NIST extended the time for analysis and comment.
6. DSA may infringe on other patents.
It may. This will be discussed in the section on patent issues.
7. The key size is too small.
This was the only valid criticism of DSS. The original implementation set the modulus at 512 bits [1149]. Since the algorithm gets its security from the difficulty of computing discrete logs in that modulus, this worried most cryptographers. There have since been advances in the problem of calculating discrete logarithms in a finite field, and 512 bits is too short for long-term security (see Section 7.2). According to Brian LaMacchia and Andrew Odlyzko, "...even 512-bit primes appear to offer only marginal security..." [934]. In response to this criticism, NIST made the key size variable, from 512 bits to 1024 bits. Not great, but better.

On May 19, 1994, the standard was finally issued [1154]. The issuing statement said [542]:

This standard is applicable to all Federal departments and agencies for the protection of unclassified information.... This standard shall be used in designing and implementing public-key based signature schemes which Federal departments and agencies operate or which are operated for them under contract. Adoption and use of this standard is available to private and commercial organizations.

Before you run out and implement this standard in your next product, read the section on patent issues below.

Description of DSA

DSA is a variant of the Schnorr and ElGamal signature algorithms, and is fully described in [1154]. The algorithm uses the following parameters:

p = a prime number L bits long, when L ranges from 512 to 1024 and is a multiple of 64. (In the original standard, the size of p was fixed at 512 bits [1149]. This was the source of much criticism and was changed by NIST [1154].)

q = a 160-bit prime factor of p – 1.

g = h(p-1)/q mod p, where h is any number less than p – 1 such that h(p - 1)/q mod p is greater than 1.

x = a number less than q.

y = gx mod p.

The algorithm also makes use of a one-way hash function: H(m). The standard specifies the Secure Hash Algorithm, discussed in Section 18.7.

The first three parameters, p, q, and g, are public and can be common across a network of users. The private key is x; the public key is y.

To sign a message, m:

(1) Alice generates a random number, k, less than q.
(2) Alice generates
r = (gk mod p) mod q
s = (k-1 (H(m) + xr)) mod q

The parameters r and s are her signature; she sends these to Bob.
(3) Bob verifies the signature by computing
w = s-1 mod q
u1 = (H(m) * w) mod q
u2 = (rw) mod q
v = ((gu 1 * yu 2) mod p) mod q

If v = r, then the signature is verified.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Proofs for the mathematical relationships are found in [1154]. Table 20.1 provides a summary.

Speed Precomputations

Table 20.2 gives sample software speeds of DSA [918].

Real-world implementations of DSA can often be speeded up through precomputations. Notice that the value r does not depend on the message. You can create a string of random k values, and then precompute r values for each of them. You can also precompute k-1 for each of those k values. Then, when a message comes along, you can compute s for a given r and k-1.

This precomputation speeds up DSA considerably. Table 20.3 is a comparison of DSA and RSA computation times for a particular smart card implementation [1479].

Table 20.1 DSA Signatures
 | Public Key:
p 512-bit to 1024-bit prime (can be shared among a group of users)
q 160-bit prime factor of p – 1 (can be shared among a group of users)
g = h(p - 1)/q mod p, where h is less than p – 1 and h(p - 1)/q mod p > 1 (can be shared among a group of users)
y = gx mod p (a p-bit number)
 | Private Key:
x < q (a 160-bit number)
 | Signing:
k choose at random, less than q
r (signature) = (gk mod p) mod q
s (signature) = (k-1 (H(m) + xr)) mod q
 | Verifying:
w = s-1 mod q
u1 = (H(m) * w) mod q
u2 = (rw) mod q
v = ((gu 1 * yu 2) mod p) mod q
If v = r, then the signature is verified.

DSA Prime Generation

Lenstra and Haber pointed out that certain moduli are much easier to crack than others [950]. If someone forced a network to use one of these "cooked" moduli, then their signatures would be easier to forge. This isn't a problem for two reasons: These moduli are easy to detect and they are so rare that the chances of using one when choosing a modulus randomly are almost negligible—smaller, in fact, than the chances of accidentally generating a composite number using a probabilistic prime generation routine.

In [1154] NIST recommended a specific method for generating the two primes, p and q, where q divides p – 1. The prime p is L bits long, between 512 and 1024 bits long, in some multiple of 64 bits. The prime q is 160 bits long. Let L – 1 = 160n + b, where L is the length of p, and n and b are two numbers and b is less than 160.

Table 20.2 DSA Speeds for Different Modulus Lengths with a 160-bit Exponent (on a SPARC II)
 | 512 bits | 768 bits | 1024 bits
Sign | 0.20 sec | 0.43 sec | 0.57 sec
Verify | 0.35 sec | 0.80 sec | 1.27 sec

Table 20.3 Comparison of RSA and DSA Computation Times
 | DSA | RSA | DSA with Common p, q, g
Global Computations | Off-card (P) | N/A | Off-card (P)
Key Generation | 14 sec | Off-card (S) | 4 sec
Precomputation | 14 sec | N/A | 4 sec
Signature | .03 sec | 15 sec | .03 sec
Verification | 16 sec | 1.5 sec | 10 sec
 | 1–5 sec off-card (P) | 1–3 sec off-card (P)
Off-card computations were performed on an 80386 33 mHz, personal computer. (P) indicates public parameters off-card and (S) indicates secret parameters off-card. Both algorithms use a 512-bit modulus.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

(1) Choose an arbitrary sequence of at least 160 bits and call it S. Let g be the length of S in bits.
(2) Compute U = SHA(S) ⊕ SHA ((S + 1) mod 2g), where SHA is the Secure Hash Algorithm (see Section 18.7).
(3) Form q by setting the most significant bit and the least significant bit of U to 1.
(4) Check whether q is prime.
(5) If q is not prime, go back to step (1).
(6) Let C = 0 and N = 2.
(7) For k = 0, 1,..., n, let Vk = SHA ((S + N + k) mod 2g)
(8) Let W be the integer
W = V0 + 2160V1 +...+ 2160(n - 1)Vn - 1 + 2160n(Vn mod 2b)

and let
X = W + 2L - 1

Note that X is an L-bit number.
(9) Let p = X – ((X mod 2q) – 1). Note that p is congruent to 1 mod 2q.
(10) If p < 2L - 1, then go to step (13).
(11) Check whether p is prime.
(12) If p is prime, go to step (15).
(13) Let C = C + 1 and N = N + n + 1.
(14) If C = 4096, then go to step (1). Otherwise, go to step (7).
(15) Save the value of S and the value of C used to generate p and q.

In [1154], the variable S is called the "seed," C is called the "counter," and N the "offset."

The point of this exercise is that there is a public means of generating p and q. For all practical purposes, this method prevents cooked values of p and q. If someone hands you a p and a q, you might wonder where that person got them. However, if someone hands you a value for S and C that generated the random p and q, you can go through this routine yourself. Using a one-way hash function, SHA in the standard, prevents someone from working backwards from a p and q to generate an S and C.

This security is better than what you get with RSA. In RSA, the prime numbers are kept secret. Someone could generate a fake prime or one of a special form that makes factoring easier. Unless you know the private key, you won't know that. Here, even if you don't know a person's private key, you can confirm that p and q have been generated randomly.

ElGamal Encryption with DSA

There have been allegations that the government likes the DSA because it is only a digital signature algorithm and can't be used for encryption. It is, however, possible to use the DSA function call to do ElGamal encryption.

Assume that the DSA algorithm is implemented with a single function call:

    DSAsign (p,q,g,k,x,h,r,s)

You supply the numbers p, q, g, k, x, and h, and the function returns the signature parameters: r and s.

To do ElGamal encryption of message m with public key y, choose a random number, k, and call

    DSAsign (p,p,g,k,0,0,r,s)

The value of r returned is a in the ElGamal scheme. Throw s away. Then, call

    DSAsign (p,p,y,k,0,0,r,s)

Rename the value of r to be u; throw s away. Call

    DSAsign (p,p,m,1,u,0,r,s)

Throw r away. The value of s returned is b in the ElGamal scheme. You now have the ciphertext, a and b.

Decryption is just as easy. Using secret key x, and ciphertext messages a and b, call

    DSAsign (p,p,a,x,0,0,r,s)

The value r is ax mod p. Call that e. Then call

    DSAsign (p,p,1,e,b,0,r,s)

The value s is the plaintext message, m.

This method will not work with all implementations of DSA. Some may fix the values of p and q, or the lengths of some of the other parameters. Still, if the implementation is general enough, this is a way to encrypt using nothing more than digital signature function.

RSA Encryption with DSA

RSA encryption is even easier. With a modulus n, message m, and public key e, call

    DSAsign (n,n,m,e,0,0,r,s)

The value of r returned is the ciphertext.

RSA decryption is the same thing. If d is the private key, then

    DSAsign (n,n,m,d,0,0,r,s)

returns the plaintext as the value of r.

Security of DSA

At 512-bits, DSA wasn't strong enough for long-term security. At 1024 bits, it is.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The NSA, in its first public interview on the subject, commented to Joe Abernathy of The Houston Chronicle on allegations about a trapdoor in DSS [363]:

Regarding the alleged trapdoor in the DSS. We find the term trapdoor somewhat misleading since it implies that the messages sent by the DSS are encrypted and with access via a trapdoor one could somehow decrypt (read) the message without the sender's knowledge.

The DSS does not encrypt any data. The real issue is whether the DSS is susceptible to someone forging a signature and therefore discrediting the entire system. We state categorically that the chances of anyone—including NSA—forging a signature with the DSS when it is properly used and implemented is infinitesimally small.

Furthermore, the alleged trapdoor vulnerability is true for any public key-based authentication system, including RSA. To imply somehow that this only affects the DSS (a popular argument in the press) is totally misleading. The issue is one of implementation and how one goes about selecting prime numbers. We call your attention to a recent EUROCRYPT conference which had a panel discussion on the issue of trapdoors in the DSS. Included on the panel was one of the Bellcore researchers who initially raised the trapdoor allegation, and our understanding is that the panel—including the person from Bellcore—concluded that the alleged trapdoor was not an issue for the DSS. Furthermore, the general consensus appeared to be that the trapdoor issue was trivial and had been overblown in the press. However, to try to respond to the trapdoor allegation, at NIST's request, we have designed a prime generation process which will ensure that one can avoid selection of the relatively few weak primes which could lead to weakness in using the DSS. Additionally, NIST intends to allow for larger modulus sizes up to 1024 which effectively negates the need to even use the prime generation process to avoid weak primes. An additional very important point that is often overlooked is that with the DSS the primes are public and therefore can be subject to public examination. Not all public key systems provide for this same type of examination.

The integrity of any information security system requires attention to proper implementation. With the myriad of vulnerabilities possible given the differences among users, NSA has traditionally insisted on centralized trusted centers as a way to minimize risk to the system. While we have designed technical modifications to the DSS to meet NIST's requests for a more decentralized approach, we still would emphasize that portion of the Federal Register notice for the DSS which states:

"While it is the intent of this standard to specify general security requirements for generating digital signatures, conformance to this standard does not assure that a particular implementation is secure. The responsible authority in each agency or department shall assure that an overall implementation provides an acceptable level of security. NIST will be working with government users to ensure appropriate implementations."

Finally, we have read all the arguments purporting insecurities with the DSS, and we remain unconvinced of their validity. The DSS has been subjected to intense evaluation within NSA which led to its being endorsed by our Director of Information Systems Security for use in signing unclassified data processed in certain intelligence systems and even for signing classified data in selected systems. We believe that this approval speaks to the lack of any credible attack on the integrity provided by the DSS given proper use and implementation. Based on the technical and security requirements of the U.S. government for digital signatures, we believe the DSS is the best choice. In fact, the DSS is being used in a pilot project for the Defense Message System to assure the authenticity of electronic messages of vital command and control information. This initial demonstration includes participation from the Joint Chiefs of Staff, the military services, and Defense Agencies and is being done in cooperation with NIST.

I'm not going to comment on the trustworthiness of the NSA. Take their comments for what you think they're worth.

Attacks against k

Each signature requires a new value of k, and that value must be chosen randomly. If Eve ever recovers a k that Alice used to sign a message, perhaps by exploiting some properties of the random-number generator that generated k, she can recover Alice's private key, x. If Eve ever gets two messages signed using the same k, even if she doesn't know what it is, she can recover x. And with x, Eve can generate undetectable forgeries of Alice's signature. In any implementation of the DSA, a good random-number generator is essential to the system's security [1468].

Dangers of a Common Modulus

Even though the DSS does not specify a common modulus to be shared by everyone, different implementations may. For example, the Internal Revenue Service is considering using the DSS for the electronic submission of tax returns. What if they require every taxpayer in the country to use a common p and q? Even though the standard doesn't require a common modulus, such an implementation accomplishes the same thing. A common modulus too easily becomes a tempting target for cryptanalysis. It is still too early to tell much about different DSS implementations, but there is some cause for concern.

Subliminal Channel in DSA

Gus Simmons discovered a subliminal channel in DSA [1468,1469] (see Section 23.3). This subliminal channel allows someone to embed a secret message in his signature that can only be read by another person who knows the key. According to Simmons, it is a "remarkable coincidence" that the "apparently inherent shortcomings of subliminal channels using the ElGamal scheme can all be overcome" in the DSS, and that the DSS "provides the most hospitable setting for subliminal communications discovered to date." NIST and NSA have not commented on this subliminal channel; no one knows if they even knew about it. Since this subliminal channel allows an unscrupulous implementer of DSS to leak a piece of the private key with each signature, it is important to never use an implementation of DSS if you don't trust the implementer.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Patents

David Kravitz, formerly of the NSA, holds a patent on DSA [897]. According to NIST [538]:

NIST intends to make this DSS technique available world-wide on a royalty-free basis to the public interest. We believe this technique is patentable and that no other patents would apply to the DSS, but we cannot give firm assurances to such effect in advance of issuance of the patent.

Even so, three patent holders claim that the DSA infringes on their patents: Diffie-Hellman (see Section 22.1) [718], Merkle-Hellman (see Section 19.2) [720], and Schnorr (see Section 21.3) [1398]. The Schnorr patent is the most troublesome. The other two patents expire in 1997; the Schnorr patent is valid until 2008. The Schnorr algorithm was not developed with government money; unlike the PKP patents, the U.S. government has no rights to the Schnorr patent; and Schnorr patented his algorithm worldwide. Even if the U.S. courts rule in favor of DSA, it is unclear what other courts around the world would do. Is an international company going to adopt a standard that may be legal in some countries but infringes on a patent in others? This issue will take time to resolve; at the time of this writing it isn't even resolved in the United States.

In June 1993 NIST proposed to give PKP an exclusive patent license to DSA [541]. The agreement fell through after public outcry and the standard was issued without any deal. NIST said [542]:

...NIST has addressed the possible patent infringement claims, and has concluded that there are no valid claims.

So the standard is official, lawsuits are threatened, and no one knows what to do. NIST has said that it would help defend people sued for patent infringement, if they were using DSA to satisfy a government contract. Everyone else, it seems, is on their own. ANSI has a draft banking standard that uses DSA [60]. NIST is working to standardize DSA within the government. Shell Oil has made DSA their international standard. I know of no other proposed DSA standards.

20.2 DSA Variants

This variant makes computation easier on the signer by not forcing him to compute k-1 [1135]. All the parameters are as in DSA. To sign a message, m, Alice generates two random numbers, k and d, both less than q. The signature is

r = (gk mod p) mod q
s = (H(m) + xr) * d mod q
t = kd mod q

Bob verifies the signature by computing

w = t/s mod q
u1 = (H(m) * w) mod q
u2 = (rw) mod q

If r = ((gu 1 * yu 2) mod p) mod q, then the signature is verified.

This next variant makes computation easier on the verifier [1040,1629]. All the parameters are as in DSA. To sign a message, m, Alice generates a random number, k, less than q. The signature is

r = (gk mod p) mod q
s = k * (H(m) + xr)-1 mod q

Bob verifies the signature by computing

u1 = (H(m) * s) mod q
u2 = (sr) mod q

If r = ((gu 1 * yu 2) mod p) mod q, then the signature is verified.

Another DSA variant allows for batch verification; Bob can verify signatures in batches [1135]. If they are all valid, he is done. If one isn't valid, then he still has to find it. Unfortunately, it is not secure; either the signer or the verifier can easily create a set of bogus signatures that satisfy the batch criteria [974].

There is also a variant for DSA prime generation, one that embeds q and the parameters used to generate the primes within p. Whether this scheme reduces the security of DSA is still unknown.

(1) Choose an arbitrary sequence of at least 160 bits and call it S. Let g be the length of S in bits.
(2) Compute U = SHA(S) ⊕ SHA ((S + 1) mod 2g), where SHA is the Secure Hash Algorithm (see Section 18.7).
(3) Form q by setting the most significant bit and the least significant bit of U to 1.
(4) Check whether q is prime.
(5) Let p be the concatenation of q, S, C, and SHA(S). C is set to 32 zero bits.
(6) p = p – (p mod q) + 1.
(7) p = p + q.
(8) If the C in p is 0x7fffffff, go to step (1).
(9) Check whether p is prime.
(10) If p is composite, go to step (7).

The neat thing about this variant is that you don't have to store the values of C and S used to generate p and q; they are embedded within p. For applications without a whole lot of memory, like smart cards, this can be a big deal.

20.3 GOST Digital Signature Algorithm

This is a Russian digital signature standard, officially called GOST R 34.10-94 [656]. The algorithm is very similar to DSA, and uses the following parameters

p = a prime number, either between 509 and 512 bits long, or between 1020 and 1024 bits long.
q = a 254- to 256-bit prime factor of p – 1.
a = any number less than p – 1 such that aq mod p = 1.
x = a number less than q.
y = ax mod p.

The algorithm also makes use of a one-way hash function: H(x). The standard specifies GOST R 34.11-94 (see Section 18.11), a function based on the GOST symmetric algorithm (see Section 14.1) [657].

The first three parameters, p, q, and a, are public and can be common across a network of users. The private key is x; the public key is y.

To sign a message, m

(1) Alice generates a random number, k, less than q
(2) Alice generates
r = (ak mod p) mod q
s = (xr + k(H(m))) mod q

If H(m) mod q = 0, then set it equal to 1. If r = 0, then choose another k and start again. The signature is two numbers: r mod 2256 and s mod 2256. She sends these to Bob.
(3) Bob verifies the signature by computing
v = H(m)q-2 mod q
z1 = (sv) mod q
z2 = ((q – r) * v) mod q
u = ((az 1 * yz 2) mod p) mod q

If u = r, then the signature is verified.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The difference between this scheme and DSA is that with DSA s = (xr + k-1(H(m))) mod q, which leads to a different verification equation. Curious, though, is that q is 256 bits. Most Western cryptographers seem satisfied with a q of around 160 bits. Perhaps this is just a reflection of the Russian tendency to play it ultrasafe.

The standard has been in use since the beginning of 1995, and is not classified "for special use"—whatever that means.

20.4 Discrete Logarithm Signature Schemes

ElGamal, Schnorr (see Section 21.3), and DSA signature schemes are very similar. In fact, they are just three examples of a general digital signature scheme based on the Discrete Logarithm Problem. Along with thousands of other signature schemes, they are part of the same family [740,741,699,1184].

Choose p, a large prime number, and q, either p – 1 or a large prime factor of p – 1. Then choose g, a number between 1 and p such that gq ≡ 1 (mod p). All these numbers are public, and can be common to a group of users. The private key is x, less than q. The public key is y = gx mod p.

To sign a message, m, first choose a random k less than and relatively prime to q. If q is also prime, any k less than q works. First compute

r = gk mod p

The generalized signature equation now becomes

ak = b + cx mod q

The coefficients a, b, and c can be any of a variety of things. Each line in Table 20.4 gives six possibilities.

To verify the signature, the receiver must confirm that

ra = gbyc mod p

This is called the verification equation.

Table 20.5 lists the signature and verifications possible from just the first line of potential values for a, b, and c, ignoring the effects of the ±

Table 20.4 Possible Permutations of a, b, and c (r' = r mod q)
±r' | ±s | m
±r'm | ±s | 1
±r'm | ±ms | 1
±mr' | ±r's | 1
±ms | ±r's | 1

That's six different signature schemes. Adding the negative signs brings the total to 24. Using the other possible values listed for a, b, and c brings the total to 120.

ElGamal [518,519] and DSA [1154] are essentially based on equation (4). Other schemes are based on equation (2) [24,1629]. Schnorr [1396,1397] is closely related to equation (5), as is another scheme [1183]. And equation (1) can be modified to yield the scheme proposed in [1630]. The rest of the equations are new.

There's more. You can make any of these schemes more DSA-like by defining r as

r = (gk mod p) mod q

Keep the same signature equation and make the verification equation

u1 = a-1b mod q
u2 = a-1c mod q
r = (gu 1 yu 2 mod p) mod q

There are two other possibilities along these lines [740,741]; you can do this with each of the 120 schemes, bringing the total to 480 discrete-logarithm-based digital signature schemes.

But wait—there's more. Additional generalizations and variations can generate more than 13,000 variants (not all of them terribly efficient) [740,741].

One of the nice things about using RSA for digital signatures is a feature called message recovery. When you verify an RSA signature you compute m. Then you compare the computed m with the message and see if the signature is valid for that message. With the previous schemes, you can't recover m when you compute the signature; you need a candidate m that you use in a verification equation. Well, as it turns out it is possible to construct a message recovery variant for all the above signature schemes.

Table 20.5 Discrete Logarithm Signature Schemes
Signature Equation | Verification Equation
(1) r'k = s + mx mod q | rr' = gsym mod p
(2) r'k = m + sx mod q | rr' = gmys mod p
(3) sk = r' + mx mod q | rs = gr'ym mod p
(4) sk = m + r'x mod q | rs = gmyr' mod p
(5) mk = s + r'x mod q | rm = gsyr' mod p
(6) mk = r' + sx mod q | rm = gr'ys mod p

To sign, first compute

r = mgk mod p

and replace m by 1 in the signature equation. Then you can reconstruct the verification equation such that m can be computed directly.

You can do the same with the DSA-like schemes:

r = (mgk mod p) mod q


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

All the variants are equally secure, so it makes sense to choose a scheme that is easy to compute with. The requirement to compute inverses slows most of these schemes. As it turns out, a scheme in this pile allows computing both the signature equation and the verification equation without inverses and also gives message recovery. It is called the p-NEW scheme [1184].

r = mg-k mod p
s = k – r'x mod q

And m is recovered (and the signature verified) by

m = gsyr'r mod p

Some variants sign two and three message blocks at the same time [740]; other variants can be used for blind signatures [741].

This is a remarkable piece of research. All of the various discrete-logarithm-based digital signature schemes have been put in one coherent framework. In my opinion this finally puts to rest any patent dispute between Schnorr [1398] and DSA [897]: DSA is not a derivative of Schnorr, nor even of ElGamal. All three are examples of this general construction, and this general construction is unpatented.

20.5 Ong-Schnorr-Shamir

This signature scheme uses polynomials modulo n [1219,1220]. Choose a large integer n (you need not know the factorization of n). Then choose a random integer, k, such that k and n are relatively prime. Calculate h such that

h = –k-2 mod n = -(k-1)2 mod n

The public key is h and n; k is the private key.

To sign a message, M, first generate a random number, r, such that r and n are relatively prime. Then calculate:

S1 = 1/2 * (M/r + r) mod n
S2 = k/2 * (M/r – r) mod n

The pair, S1 and S2, is the signature.

To verify a signature, confirm that

S12 + h * S22 ≡ M (mod n)

The version of the scheme described here is based on quadratic polynomials. When it was first proposed in [1217], a $100 reward was offered for successful cryptanalysis. It was proved insecure [1255,18], but its authors were not deterred. They proposed a modification of the algorithm based on cubic polynomials, which is also insecure [1255]. The authors then proposed a quartic version, which was also broken [524,1255]. A variant which fixes these problems is in [1134].

20.6 ESIGN

ESIGN is a digital signature scheme from NTT Japan [1205,583]. It is touted as being at least as secure and considerably faster than either RSA or DSA, with similar key and signature lengths.

The private key is a pair of large prime numbers, p and q. The public key is n, when

n = p2q

H is a hash function that operates on a message, m, such that H(m) is between 0 and n – 1. There is also a security parameter, k, which will be discussed shortly.

(1) Alice picks a random number x, where x is less than pq.
(2) Alice computes:
w, the least integer that is larger than or equal to
(H(m) – xk mod n)/pq
s = x + ((w/kxk - 1) mod p)pq
(3) Alice sends s to Bob.
(4) To verify the signature, Bob computes sk mod n. He also computes a, which is the least integer larger than or equal to two times the number of bits of n divided by 3. If H(m) is less than or equal to sk mod n, and if sk mod n is less than H(m) + 2a, then the signature is considered valid.

This algorithm works faster with precomputation. This precomputation can be done at any time and has nothing to do with the message being signed. After picking x, Alice could break step (2) into two partial steps. The first can be precomputed.

(2a) Alice computes:
u = xk mod n
v = 1/(kxk - 1) mod p
(2b) Alice computes:
w = the least integer that is larger than or equal to
(H(m) – u)/pq)
s = x + (wv mod p)pq

For the size of numbers generally used, this precomputation speeds up the signature process by a factor of 10. Almost all the hard work is done in the precomputation stage. A discussion of modular arithmetic operations to speed ESIGN can be found in [1625,1624]. This algorithm can also be extended to work with elliptic curves [1206].

Security of ESIGN

When this algorithm was originally proposed, k was set to 2 [1215]. This was quickly broken by Ernie Brickell and John DeLaurentis [261], who then extended their attack to k = 3. A modified version of this algorithm [1203] was broken by Shamir [1204]. The variant proposed in [1204] was broken in [1553]. ESIGN is the current incarnation of this family of algorithms. Another new attack [963] does not work against ESIGN.

The authors currently recommend these values for k: 8, 16, 32, 64, 128, 256, 512, and 1024. They also recommend that p and q each be of at least 192 bits, making n at least 576 bits long. (I think n should be twice that length.) With these parameters, the authors conjecture that ESIGN is as secure as RSA or Rabin. And their analysis shows favorable speed comparison to RSA, ElGamal, and DSA [582].

Patents

ESIGN is patented in the United States [1208], Canada, England, France, Germany, and Italy. Anyone who wishes to license the algorithm should contact Intellectual Property Department, NTT, 1–6 Uchisaiwai-cho, 1-chome, Chiyada-ku, 100 Japan.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

20.7 Cellular Automata

A new and novel idea, studied by Papua Guam [665], is the use of cellular automata in public-key cryptosystems. This system is still far too new and has not been studied extensively, but a preliminary examination suggests that it may have a cryptographic weakness similar to one seen in other cases [562]. Still, this is a promising area of research. Cellular automata have the property that, even if they are invertible, it is impossible to calculate the predecessor of an arbitrary state by reversing the rule for finding the successor. This sounds a whole lot like a trapdoor one-way function.

20.8 Other Public-Key Algorithms

Many other public-key algorithms have been proposed and broken over the years. The Matsumoto-Imai algorithm [1021] was broken in [450]. The Cade algorithm was first proposed in 1985, broken in 1986 [774], and then strengthened in the same year [286]. In addition to these attacks, there are general attacks for decomposing polynomials over finite fields [605]. Any algorithm that gets its security from the composition of polynomials over a finite field should be looked upon with skepticism, if not outright suspicion.

The Yagisawa algorithm combines exponentiation mod p with arithmetic mod p – 1 [1623]; it was broken in [256]. Another public-key algorithm, Tsujii-Kurosawa-Itoh-Fujioka-Matsumoto [1548] is insecure [948]. A third system, Luccio-Mazzone [993], is insecure [717]. A signature scheme based on birational permutations [1425] was broken the day after it was presented [381]. Tatsuaki Okamoto has several signature schemes: one is provably as secure as the Discrete Logarithm Problem, and another is provably as secure as the Discrete Logarithm Problem and the Factoring Problem [1206]. Similar schemes are in [709].

Gustavus Simmons suggested J-algebras as a basis for public-key algorithms [1455,145]. This idea was abandoned after efficient methods for factoring polynomials were invented [951]. Special polynomial semigroups have also been studied [1619,962], but so far nothing has come of it. Harald Niederreiter proposed a public-key algorithm based on shift-register sequences [1166]. Another is based on Lyndon words [1476] and another on propositional calculus [817]. And a recent public-key algorithm gets its security from the matrix cover problem [82]. Tatsuaki Okamoto and Kazuo Ohta compare a number of digital signature schemes in [1212].

Prospects for creating radically new and different public-key cryptography algorithms seem dim. In 1988 Whitfield Diffie noted that most public-key algorithms are based on one of three hard problems [492, 494]:

1. Knapsack: Given a set of unique numbers, find a subset whose sum is N.
2. Discrete logarithm: If p is a prime and g and m are integers, find x such that gx ≡ M (mod p).
3. Factoring: If N is the product of two primes, either
a) factor N,
b) given integers M and C, find d such that Md ≡ C (mod N),
c) given integers e and C, find M such that Me ≡ C (mod N), or
d) given an integer x, decide whether there exists an integer y such that x ≡ y2 (mod N).

According to Diffie [492,494], the Discrete Logarithm Problem was suggested by J. Gill, the Factoring Problem by Knuth, and the knapsack problem by Diffie himself.

This narrowness in the mathematical foundations of public-key cryptography is worrisome. A breakthrough in either the problem of factoring or of calculating discrete logarithms could render whole classes of public-key algorithms insecure. Diffie points out [492,494] that this risk is mitigated by two factors:

1. The operations on which public key cryptography currently depends—multiplying, exponentiating, and factoring—are all fundamental arithmetic phenomena. They have been the subject of intense mathematical scrutiny for centuries and the increased attention that has resulted from their use in public key cryptosystems has on balance enhanced rather than diminished our confidence.
2. Our ability to carry out large arithmetic computations has grown steadily and now permits us to implement our systems with numbers sufficient in size to be vulnerable only to a dramatic breakthrough in factoring, logarithms, or root extraction.

As we have seen, not all public-key algorithms based on these problems are secure. The strength of any public-key algorithm depends on more than the computational complexity of the problem upon which it is based; a hard problem does not necessarily imply a strong algorithm. Adi Shamir listed three reasons why this is so [1415]:

1. Complexity theory usually deals with single isolated instances of a problem. A cryptanalyst often has a large collection of statistically related problems to solve—several ciphertexts encrypted with the same key.
2. The computational complexity of a problem is typically measured by its worst-case or average-case behavior. To be useful as a cipher, the problem must be hard to solve in almost all cases.
3. An arbitrarily difficult problem cannot necessarily be transformed into a cryptosystem, and it must be possible to insert trapdoor information into the problem so that a shortcut solution is possible with this information and only with this information.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 21
Identification Schemes

21.1 Feige-Fiat-Shamir

Amos Fiat's and Adi Shamir's authentication and digital signature scheme is discussed in [566,567]. Uriel Feige, Fiat, and Shamir modified the algorithm to a zero-knowledge proof of identity [544,545]. This is the best-known zero-knowledge proof of identity.

On July 9, 1986 the three authors submitted a U.S. patent application [1427]. Because of its potential military applications, the application was reviewed by the military. Occasionally the Patent Office responds not with a patent, but with something called a secrecy order. On January 6, 1987, three days before the end of their six-month period, the Patent Office imposed that order at the request of the Army. They stated that "...the disclosure or publication of the subject matter...would be detrimental to the national security...." The authors were ordered to notify all Americans to whom the research had been disclosed that unauthorized disclosure could lead to two years' imprisonment, a $10,000 fine, or both. Furthermore, the authors had to inform the Commissioner of Patents and Trademarks of all foreign citizens to whom the information had been disclosed.

This was ludicrous. All through the second half of 1986, the authors had presented the work at conferences throughout Israel, Europe, and the United States. The authors weren't even American citizens, and all the work had been done at the Weizmann Institute in Israel.

Word spread through the academic community and the press. Within two days the secrecy order was rescinded; Shamir and others believe that the NSA pulled strings to rescind the order, although they officially had no comment. Further details of this bizarre story are in [936].

Simplified Feige-Fiat-Shamir Identification Scheme

Before issuing any private keys, the arbitrator chooses a random modulus, n, which is the product of two large primes. In real life, n should be at least 512 bits long and probably closer to 1024 bits. This n can be shared among a group of provers. (Choosing a Blum integer makes computation easier, but it is not required for security.)

To generate Peggy's public and private keys, a trusted arbitrator chooses a number, v, where v is a quadratic residue mod n. In other words, choose v such that x2 ≡ v (mod n) has a solution and v-1 mod n exists. This v is Peggy's public key. Then calculate the smallest s for which s ≡ sqrt (v-1) (mod n). This is Peggy's private key.

The identification protocol can now proceed.

(1) Peggy picks a random r, where r is less then n. She then computes x = r2 mod n, and sends x to Victor.
(2) Victor sends Peggy a random bit, b.
(3) If b = 0, then Peggy sends Victor r. If b = 1, then Peggy sends Victor y = r * s mod n.
(4) If b = 0, Victor verifies that x = r2 mod n, proving that Peggy knows sqrt (x). If b = 1, Victor verifies that x = y2 * v mod n, proving that Peggy knows sqrt (v-1).

This is a single round—called an accreditation—of the protocol. Peggy and Victor repeat this protocol t times, until Victor is convinced that Peggy knows s. It's a cut-and-choose protocol. If Peggy doesn't know s, she can pick r such that she can fool Victor if he sends her a 0, or she can pick r such that she can fool Victor if he sends her a 1. She can't do both. The odds of her fooling Victor once are 50 percent. The odds of her fooling him t times are 1 in 2t.

Another way for Victor to attack the protocol would be trying to impersonate Peggy. He could initiate the protocol with another verifier, Valerie. In step (1), instead of choosing a random r, he would just reuse an old r that he saw Peggy use. However, the odds of Valerie choosing the same value for b in step (2) that Victor did in the protocol with Peggy are 1 in 2. So, the odds of his fooling Valerie are 50 percent. The odds of his fooling her t times are 1 in 2t.

For this to work, Peggy must not reuse an r, ever. If she did, and Victor sent Peggy the other random bit in step (2), then he would have both of Peggy's responses. Then, from even one of these, he can calculate s and it's all over for Peggy.

Feige-Fiat-Shamir Identification Scheme

In their papers [544,545], Feige, Fiat and Shamir show how parallel construction can increase the number of accreditations per round and reduce Peggy and Victor's interactions.

First generate n as in the previous example, the product of two large primes. To generate Peggy's public and private keys, first choose k different numbers: v1, v2,..., vk, where each vi is a quadratic residue mod n. In other words, choose vi such that x2 = vi mod n has a solution and vi-1 mod n exists. This string, v1, v2,..., vk, is the public key. Then calculate the smallest si such that si = sqrt (vi-1) mod n. This string, s1, s2,..., sk, is the private key.

And the protocol is:

(1) Peggy picks a random r, when r is less than n. She then computes x = r2 mod n, and sends x to Victor.
(2) Victor sends Peggy a random binary string k-bits long: b1, b2,..., bk.
(3) Peggy computes y = r * (s1b1 * s2b2 *...* skbk) mod n. (She multiplies together whichever values of si that correspond to bi = 1. If Victor's first bit is a 1, then s1 is part of the product; if Victor's first bit is a 0, then s1 is not part of the product, and so on.) She sends y to Victor.
(4) Victor verifies that x = y2 * (v1b1 * v2b2 *...* vkbk) mod n. (He multiplies together the values of vi based on the random binary string. If his first bit is a 1, then v1 is part of the product; if his first bit is a 0, then v1 is not part of the product, and so on.)

Peggy and Victor repeat this protocol t times, until Victor is convinced that Peggy knows s1, s2,..., sk.

The chance that Peggy can fool Victor is 1 in 2kt. The authors recommend a 1 in 220 chance of a cheater fooling Victor and suggest that k = 5 and t = 4. If you are more paranoid, increase these numbers.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

An Example

Let's look at this protocol in action with small numbers.

If n = 35 (the two primes are 5 and 7), then the possible quadratic residues are:

1: x2 ≡ 1 (mod 35) has the solutions: x = 1, 6, 29, or 34.
4: x2 ≡ 4 (mod 35) has the solutions: x = 2, 12, 23, or 33.
9: x2 ≡ 9 (mod 35) has the solutions: x = 3, 17, 18, or 32.
11: x2 ≡ 11 (mod 35) has the solutions: x = 9, 16, 19, or 26.
14: x2 ≡ 14 (mod 35) has the solutions: x = 7 or 28.
15: x2 ≡ 15 (mod 35) has the solutions: x = 15 or 20.
16: x2 ≡ 16 (mod 35) has the solutions: x = 4, 11, 24, or 31.
21: x2 ≡ 21 (mod 35) has the solutions: x = 14 or 21.
25: x2 ≡ 25 (mod 35) has the solutions: x = 5 or 30.
29: x2 ≡ 29 (mod 35) has the solutions: x = 8, 13, 22 or 27.
30: x2 ≡ 30 (mod 35) has the solutions: x = 10 or 25.

The inverses (mod 35) and their square roots are:

v | v-1 | s = sqrt (v-1)
1 | 1 | 1
4 | 9 | 3
9 | 4 | 2
11 | 16 | 4
16 | 11 | 9
29 | 29 | 8

Note that 14, 15, 21, 25, and 30 do not have inverses mod 35, because they are not relatively prime to 35. This makes sense, because there should be (5 - 1) * (7 - 1)/4 quadratic residues mod 35 relatively prime to 35: That is gcd(x,35) = 1 (see Section 11.3).

So, Peggy gets the public key consisting of k = 4 values: {4,11,16,29}. The corresponding private key is {3,4,9,8}. Here's one round of the protocol.

(1) Peggy chooses a random r = 16, computes 162 mod 35 = 11, and sends it to Victor.
(2) Victor sends Peggy a random binary string {1,1,0,1}.
(3) Peggy computes 16 * ((31) * (41) * (90) * (81)) mod 35 = 31 and sends it to Victor.
(4) Victor verifies that 312 * ((41) * (111) * (160) * (291)) mod 35 = 11.

Peggy and Victor repeat the protocol t times, each time with a different random r, until Victor is satisfied.

With small values like these, there's no real security. But when n is 512 bits long or more, Victor cannot learn anything about Peggy's secret key except the fact that she knows it.

Enhancements

It is possible to embed identification information into the protocol. Assume that I is a binary string representing Peggy's identification: her name, address, social security number, hat size, preferred brand of soft drink, and other personal information. Use a one-way hash function H(x) to compute H(I,j), where j is a small number concatenated onto I. Find a set of js where H(I,j) is a quadratic residue mod n. These H(I,j)s become v1, v2,..., vk (the js need not be quadratic residues). Peggy's public key is now I and the list of js. She sends I and the list of js to Victor before step (1) of the protocol (or perhaps Victor downloads them from a public bulletin board someplace), and Victor generates v1, v2,..., vk from H(I,j).

Now, after Victor successfully completes the protocol with Peggy, he is assured that Trent, who knows the factorization of the modulus, has certified the association between I and Peggy by giving her the square roots of the vi derived from I. (See Section 5.2 for background information.)

Feige, Fiat, and Shamir include the following implementation remarks [544,545]:

For nonperfect hash functions, it may be advisable to randomize I by concatenating it with a long random string, R. This string is chosen by the arbitrator and is revealed to Victor along with I.

In typical implementations, k should be between 1 and 18. Larger values of k can reduce the time and communication complexity by reducing the number of rounds.

The value n should be at least 512 bits long. (Of course, there has been considerable progress in factoring since then.)

If each user chooses his own n and publishes it in a public key file, they can dispense with the arbitrator. However, this RSA-like variant makes the scheme considerably less convenient.

Fiat-Shamir Signature Scheme

Turning this identification scheme into a signature scheme is basically a matter of turning Victor into a hash function. The primary benefit of the Fiat-Shamir digital signature scheme over RSA is speed: Fiat-Shamir requires only 1 percent to 4 percent of the modular multiplications of RSA. For this protocol, we'll bring back Alice and Bob.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The setup is the same as the identification scheme. Choose n to be the product of two large primes. Generate the public key, v1, v2,..., vk, and the private key, s1, s2,..., sk, such that si = sqrt (vi-1) mod n.

(1) Alice picks t random integers between 1 and n: r1, r2,..., rt, and computes x1, x2,..., xt such that xi = ri2 mod n.
(2) Alice hashes the concatenation of the message and the string of xis to generate a bit stream: H(m, x1, x2,..., xt). She uses the first k * t bits of this string as values of bij, where i goes from 1 to t, and j goes from 1 to k.
(3) Alice computes y1, y2,..., yt, where
yi = ri * (s1bi1 * s2bi2 *...* skbik) mod n
(For each i, she multiplies together the values of the sj based on the random bi,j values. If bi,1 is a 1, then s1 is multiplied; if bi,1 is a 0, then s1 is not multiplied.)
(4) Alice sends Bob m, all the bit values of bi,j, and all the values of yi. He already has Alice's public key: v1, v2,..., vk.
(5) Bob computes z1, z2,..., zt, where
zi = yi2 * (v1bi1 * v2bi2 *...* vkbik) mod n
(Again, Bob multiplies based on the bi, j values.) Also note that zi should be equal to xi.
(6) Bob verifies that the first k * t bits of H(m, z1, z2,..., zt) are the bi, j values that Alice sent him.

As with the identification scheme, the security of this signature scheme is proportional to 1/2kt. It also depends on the difficulty of factoring n. Fiat and Shamir pointed out that forging a signature is easier when the complexity of factoring n is considerably lower than 2kt. And, because of birthday-type attacks (see Section 18.1), they recommend that k * t be increased from 20 to at least 72. They suggest k = 9 and t = 8.

Improved Fiat-Shamir Signature Scheme

Silvio Micali and Adi Shamir improved the Fiat-Shamir protocol in [1088]. They chose v1, v2,..., vk to be the first k prime numbers. So

v1 = 2, v2 = 3, v3 = 5, and so on.

This is the public key.

The private key, s1, s2,..., sk is a random square root, determined by

si = sqrt (vi-1) mod n

In this version, every person must have a different n. The modification makes it easier to verify signatures. The time required to generate signatures, and the security of those signatures, is unaffected.

Other Enhancements

There is also an N-party identification scheme, based on the Fiat-Shamir algorithm [264]. Two other improvements to the Fiat-Shamir scheme are proposed in [1218]. Another variant is [1368].

Ohta-Okamoto Identification Scheme

This protocol is a modification of the Feige-Fiat-Shamir identification scheme and gets its security from the difficulty of factoring [1198,1199]. The same authors also wrote a multisignature scheme (see Section 23.1), by which a number of different people can sequentially sign a message [1200]. This scheme has been proposed for smart-card implementation [850].

Patents

Fiat-Shamir is patented [1427]. Anyone interested in licensing the algorithm should contact Yeda Research and Development, The Weizmann Institute of Science, Rehovot 76100, Israel.

21.2 Guillou-Quisquater

Feige-Fiat-Shamir was the first practical identity-based protocol. It minimized computation by increasing the number of iterations and accreditations per iteration. For some implementations, like smart cards, this is less than ideal. Exchanges with the outside world are time-consuming, and the storage required for each accreditation can strain the limited resources of the card.

Louis Guillou and Jean-Jacques Quisquater developed a zero-knowledge identification algorithm more suited to applications like these [670,1280]. The exchanges between Peggy and Victor and the parallel accreditations in each exchange are both kept to an absolute minimum: There is only one exchange of one accreditation for each proof. For the same level of security, the computation required by Guillou-Quisquater is greater than by Feige-Fiat-Shamir by a factor of three. And like Feige-Fiat-Shamir, this identification algorithm can be converted to a digital signature algorithm.

Guillou-Quisquater Identification Scheme

Peggy is a smart card who wants to prove her identity to Victor. Peggy's identity consists of a set of credentials: a data string consisting of the card's name, validity period, a bank account number, and whatever else the application warrants. This bit string is called J. (Actually, the credentials can be a longer string and hashed to a J value. This complexity does not modify the protocol in any way.) This is analogous to the public key. Other public information, shared by all "Peggys" who could use this application, is an exponent v and a modulus n, where n is the product of two secret primes. The private key is B, calculated such that JBv ≡ 1 (mod n).

Peggy sends Victor her credentials, J. Now, she wants to prove to Victor that those credentials are hers. To do this, she has to convince Victor that she knows B. Here's the protocol:

(1) Peggy picks a random integer r, such that r is between 1 and n - 1. She computes T = rv mod n and sends it to Victor.
(2) Victor picks a random integer, d, such that d is between zero and v - 1. He sends d to Peggy.
(3) Peggy computes D = rBd mod n, and sends it to Victor.
(4) Victor computes T´ = DvJd mod n. If T ≡ T´ (mod n), then the authentication succeeds.

The math isn't that complex:

T´ = DvJd = (rBd)vJd = rvBdvJd = rv(JBv)d = rv ≡ T (mod n)

since B was constructed to satisfy

JBv ≡ 1 (mod n)


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Guillou-Quisquater Signature Scheme

This identification can be converted to a signature scheme, also suited for smart-card implementation [671,672].

The public and private key setup is the same as before. Here's the protocol:

(1) Alice picks a random integer r, such that r is between 1 and n - 1. She computes T = rv mod n.
(2) Alice computes d = H(M,T), where M is the message being signed and H(x) is a one-way hash function. The d produced by the hash function must be between 0 and v - 1 [1280]. If the output of the hash function is not within this range, it must be reduced modulo v.
(3) Alice computes D = rBd mod n. The signature consists of the message, M, the two calculated values, d and D, and her credentials, J. She sends this signature to Bob.
(4) Bob computes T´ = DvJd mod n. He then computes d´ = H(M,T´). If d = d´, then Alice must know B and the signature is valid.

Multiple Signatures

What if many people want to sign the same document? The easy solution has each of them signing separately, but this signature scheme can do better than that. Here Alice and Bob sign the same document and Carol verifies the signatures, but any number of people can be involved in the signature process. As before, Alice and Bob have their own unique J and B values: (JA, BA) and (JB, BB). The values n and v are common to the system.

(1) Alice picks a random integer, rA, such that rA is between 1 and n - 1. She computes TA = rAv mod n and sends TA to Bob.
(2) Bob picks a random integer, rB, such that rB is between 1 and n - 1. He computes TB = rBv mod n and sends TB to Alice.
(3) Alice and Bob each compute T = (TATB) mod n.
(4) Alice and Bob each compute d = H(M,T), where M is the message being signed and H(x) is a one-way hash function. The d produced by the hash function must be between 0 and v - 1 [1280]. If the output of the hash function is not within this range, it must be reduced modulo v.
(5) Alice computes DA = rABAd mod n and sends DA to Bob.
(6) Bob computes DB = rBBBd mod n and sends DB to Alice.
(7) Alice and Bob each compute D = DADB mod n. The signature consists of the message, M, the two calculated values, d and D, and both of their credentials: JA and JB.
(8) Carol computes J = JAJB mod n.
(9) Carol computes T´ = DvJd mod n. She then computes d´ = H(M,T´). If d ≡ d´, then the multiple signature is valid.

This protocol can be extended to any number of people. For multiple people to sign, they all multiply their individual Ti values together in step (3), and their individual Di values together in step (7). To verify a multiple signature, multiply all the signers Ji values together in step (8). Either all the signatures are valid or there is at least one invalid signature.

21.3 Schnorr

Claus Schnorr's authentication and signature scheme [1396,1397] gets its security from the difficulty of calculating discrete logarithms. To generate a key pair, first choose two primes, p and q, such that q is a prime factor of p - 1. Then, choose an a not equal to 1, such that aq ≡ 1 (mod p). All these numbers can be common to a group of users and can be freely published.

To generate a particular public-key/private-key key pair, choose a random number less than q. This is the private key, s. Then calculate v = a-s mod p. This is the public key.

Authentication Protocol

(1) Peggy picks a random number, r, less than q, and computes x = ar mod p. This is the preprocessing stage and can be done long before Victor is present.
(2) Peggy sends x to Victor.
(3) Victor sends Peggy a random number, e, between 0 and 2t - 1. (I'll discuss t in a moment.)
(4) Peggy computes y = (r + se) mod q and sends y to Victor.
(5) Victor verifies that x = ayve mod p.

The security is based on the parameter t. The difficulty of breaking the algorithm is about 2t. Schnorr recommended that p be about 512 bits, q be about 140 bits, and t be 72.

Digital Signature Protocol

Schnorr can also be used as a digital signature protocol on a message, M. The public-key/private-key key pair is the same, but we're now adding a one-way hash function, H(M).

(1) Alice picks a random number, r, less than q, and computes x = ar mod p. This computation is the preprocessing stage.
(2) Alice concatenates M and x, and hashes the result:
e = H(M,x)
(3) Alice computes y = (r + se) mod q. The signature is e and y; she sends these to Bob.
(4) Bob computes x´ = ayve mod p. He then confirms that the concatenation of M and x´ hashes to e.
e = H(M,x´)
If it does, he accepts the signature as valid.

In his paper, Schnorr cites these novel features of his algorithm:

Most of the computation for signature generation can be completed in a preprocessing stage, independent of the message being signed. Hence, it can be done during idle time and not affect the signature speed. An attack against this preprocessing stage is discussed in [475], but I don't think it's practical.

For the same level of security, the length of signatures is less for Schnorr than for RSA. For example, with a 140-bit q, signatures are only 212-bits long, less than half the length of RSA signatures. Schnorr's signatures are also much shorter than ElGamal signatures.

Of course, practical considerations may make even fewer bits suitable for a given scheme: For example, an identification scheme where the cheater must perform an on-line attack in only a few seconds, versus a signature scheme where the cheater can calculate for years off-line to come up with a forgery.

A modification of this algorithm, by Ernie Brickell and Kevin McCurley, enhances its security [265].

Patents

Schnorr is patented in the United States [1398] and in many other countries. In 1993, PKP acquired the worldwide rights to the patent (see Section 25.5). The U.S. patent expires on February 19, 2008.

21.4 Converting Identification Schemes to Signature Schemes

There is a standard method of converting an identification scheme into a signature scheme: Replace Victor with a one-way hash function. The message is not hashed before it is signed; instead the hashing is incorporated into the signing algorithm. In principle, this can be done with any identification scheme.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 22
Key-Exchange Algorithms

22.1 Diffie-Hellman

Diffie-Hellman was the first public-key algorithm ever invented, way back in 1976 [496]. It gets its security from the difficulty of calculating discrete logarithms in a finite field, as compared with the ease of calculating exponentiation in the same field. Diffie-Hellman can be used for key distribution—Alice and Bob can use this algorithm to generate a secret key—but it cannot be used to encrypt and decrypt messages.

The math is simple. First, Alice and Bob agree on a large prime, n and g, such that g is primitive mod n. These two integers don't have to be secret; Alice and Bob can agree to them over some insecure channel. They can even be common among a group of users. It doesn't matter.

Then, the protocol goes as follows:

(1) Alice chooses a random large integer x and sends Bob
X = gx mod n
(2) Bob chooses a random large integer y and sends Alice
Y = gy mod n
(3) Alice computes
k = Yx mod n
(4) Bob computes
k´ = Xy mod n

Both k and k´ are equal to gxy mod n. No one listening on the channel can compute that value; they only know n, g, X, and Y. Unless they can compute the discrete logarithm and recover x or y, they do not solve the problem. So, k is the secret key that both Alice and Bob computed independently.

The choice of g and n can have a substantial impact on the security of this system. The number (n - 1)/2 should also be a prime [1253]. And most important, n should be large: The security of the system is based on the difficulty of factoring numbers the same size as n. You can choose any g, such that g is primitive mod n; there's no reason not to choose the smallest g you can—generally a one-digit number. (And actually, g does not have to be primitive; it just has to generate a large subgroup of the multiplicitive group mod n.)

Diffie-Hellman with Three or More Parties

The Diffie-Hellman key-exchange protocol can easily be extended to work with three or more people. In this example, Alice, Bob, and Carol together generate a secret key.

(1) Alice chooses a random large integer x and sends Bob
X = gx mod n
(2) Bob chooses a random large integer y and sends Carol
Y = gy mod n
(3) Carol chooses a random large integer z and sends Alice
Z = gz mod n
(4) Alice sends Bob
Z´ = Zx mod n
(5) Bob sends Carol
X´ = Xy mod n
(6) Carol sends Alice
Y´ = Yz mod n
(7) Alice computes
k = Y´x mod n
(8) Bob computes
k = Z´y mod n
(9) Carol computes
k = X´z mod n

The secret key, k, is equal to gxyz mod n, and no one else listening in on the communications can compute that value. The protocol can be easily extended to four or more people; just add more people and more rounds of computation.

Extended Diffie-Hellman

Diffie-Hellman also works in commutitive rings [1253]. Z. Shmuley and Kevin McCurley studied a variant of the algorithm where the modulus is a composite number [1442,1038]. V. S. Miller and Neal Koblitz extended this algorithm to elliptic curves [1095,867]. Taher ElGamal used the basic idea to develop an encryption and digital signature algorithm (see Section 19.6).

This algorithm also works in the Galois field GF(2k) [1442,1038]. Some implementations take this approach [884,1631,1632], because the computation is much quicker. Similarly, cryptanalytic computation is equally fast, so it is important to carefully choose a field large enough to ensure security.

Hughes

This variant of Diffie-Hellman allows Alice to generate a key and send it to Bob [745].

(1) Alice chooses a random large integer x and generates
k = gx mod n
(2) Bob chooses a random large integer y and sends Alice
Y = gy mod n
(3) Alice sends Bob
X = Yx mod n
(4) Bob computes
z = y-1
k´ = Xz mod n

If everything goes correctly, k = k´.

The advantage of this protocol over Diffie-Hellman is that k can be computed before any interaction, and Alice can encrypt a message using k prior to contacting Bob. She can send it to a variety of people and interact with them to exchange the key individually later.

Key Exchange without Exchanging Keys

If you have a community of users, each could publish a public key, X = gx mod n, in a common database. If Alice wants to communicate with Bob, she just has to retrieve Bob's public key and generate their shared secret key. She could then encrypt a message with that key and send it to Bob. Bob would retrieve Alice's public key to generate the shared secret key.

Each pair of users would have a unique secret key, and no prior communication between users is required. The public keys have to be certified to prevent spoofing attacks and should be changed regularly, but otherwise this is a pretty clever idea.

Patents

The Diffie-Hellman key-exchange algorithm is patented in the United States [718] and Canada [719]. A group called Public Key Partners (PKP) licenses the patent, along with other public-key cryptography patents (see Section 25.5). The U.S. patent will expire on April 29, 1997.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

22.2 Station-to-Station Protocol

Diffie-Hellman key exchange is vulnerable to a man-in-the-middle attack. One way to prevent this problem is to have Alice and Bob sign their messages to each other [500].

This protocol assumes that Alice has a certificate with Bob's public key and that Bob has a certificate with Alice's public key. These certificates have been signed by some trusted authority outside this protocol. Here's how Alice and Bob generate a secret key, k.

(1) Alice generates a random number, x, and sends it to Bob.
(2) Bob generates a random number, y. Using the Diffie-Hellman protocol he computes their shared key based on x and y: k. He signs x and y, and encrypts the signature using k. He then sends that, along with y, to Alice.
y,Ek(SB(x,y))
(3) Alice also computes k. She decrypts the rest of Bob's message and verifies his signature. Then she sends Bob a signed message consisting of x and y, encrypted in their shared key.
Ek(SA(x,y))
(4) Bob decrypts the message and verifies Alice's signature.

22.3 Shamir's Three-Pass Protocol

This protocol, invented by Adi Shamir but never published, enables Alice and Bob to communicate securely without any advance exchange of either secret keys or public keys [1008].

This assumes the existence of a symmetric cipher that is commutative, that is:

EA(EB(P)) = EB(EA(P))

Alice's secret key is A; Bob's secret key is B. Alice wants to send a message, M, to Bob. Here's the protocol.

(1) Alice encrypts M with her key and sends Bob
C1 = EA(M)
(2) Bob encrypts C1 with his key and sends Alice
C2 = EB(EA(M))
(3) Alice decrypts C2 with her key and sends Bob
C3 = DA(EB(EA(M))) =DA(EA(EB(M))) = EB(M)
(4) Bob decrypts C3 with his key to recover M.

One-time pads are commutative and have perfect secrecy, but they will not work with this protocol. With a one-time pad, the three ciphertext messages would be:

C1 = P⊕ A
C2 = P⊕ A⊕ B
C3 = P⊕ B

Eve, who can record the three messages as they pass between Alice and Bob, simply XORs them together to retrieve the message:

C1 ⊕ C2 ⊕ C3 = (P ⊕ A) ⊕ (P ⊕ A ⊕ B) ⊕ (P ⊕ B) = P

This clearly won't work.

Shamir (and independently, Jim Omura) described an encryption algorithm that will work with this protocol, one similar to RSA. Let p be a large prime for which p - 1 has a large prime factor. Choose an encryption key, e, such that e is relatively prime to p - 1. Calculate d such that de ≡ 1 (mod p - 1).

To encrypt a message, calculate

C = Me mod p

To decrypt a message, calculate

M = Cd mod p

There seems to be no way for Eve to recover M without solving the discrete logarithm problem, but this has never been proved.

Like Diffie-Hellman, this protocol allows Alice to initiate secure communication with Bob without knowing any of his keys. For Alice to use a public-key algorithm, she has to know his public key. With Shamir's three-pass protocol, she just sends him a ciphertext message. The same thing with a public-key algorithm looks like:

(1) Alice asks Bob (or a KDC) for his public key.
(2) Bob (or the KDC) sends Alice his public key.
(3) Alice encrypts M with Bob's public key and sends it to Bob.

Shamir's three-pass protocol will fall to a man-in-the-middle attack.

22.4 COMSET

COMSET (COMmunications SETup) is a mutual identification and key exchange protocol developed for the RIPE project [1305] (see Section 25.7). Using public-key cryptography, it allows Alice and Bob to identify themselves to each other and also to exchange a secret key.

The mathematical principle behind COMSET is Rabin's scheme [1283] (see Section 19.5). The scheme itself was originally proposed in [224]. See [1305] for details.

22.5 Encrypted Key Exchange

The Encrypted Key Exchange (EKE) protocol was designed by Steve Bellovin and Michael Merritt [109]. It provides security and authentication on computer networks, using both symmetric and public-key cryptography in a novel way: A shared secret key is used to encrypt a randomly generated public key.

The Basic EKE Protocol

Alice and Bob (two users, a user and the host, or whoever) share a common password, P. Using this protocol, they can authenticate each other and generate a common session key, K.

(1) Alice generates a random public-key/private-key key pair. She encrypts the public key, K´, using a symmetric algorithm and P as the key: Ep(K´). She sends Bob
A, EP(K´)
(2) Bob knows P. He decrypts the message to obtain K´. Then, he generates a random session key, K, and encrypts it with the public key he received from Alice and P as the key. He sends Alice
EP(EK´(K))
(3) Alice decrypts the message to obtain K. She generates a random string, RA, encrypts it with K, and sends Bob
EK(RA)
(4) Bob decrypts the message to obtain RA. He generates another random string, RB, encrypts both strings with K, and sends Alice the result.
EK(RA, RB)
(5) Alice decrypts the message to obtain RA and RB. Assuming the RA she received from Bob is the same as the one she sent to Bob in step (3), she encrypts RB with K and sends it to Bob.
EK(RB)
(6) Bob decrypts the message to obtain RB. Assuming the RB he received from Alice is the same one he sent to Alice in step (4), the protocol is complete. Both parties now communicate using K as the session key.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

At step (3), both Alice and Bob know K´ and K. K is the session key and can be used to encrypt all other messages between Alice and Bob. Eve, sitting between Alice and Bob, only knows EP(K´), EP(EK´(K)), and some messages encrypted with K. In other protocols, Eve could make guesses at P (people choose bad passwords all the time, and if Eve is clever she can make some good guesses) and then test her guesses. In this protocol, Eve cannot test her guess without cracking the public-key algorithm as well. And if both K´ and K are chosen randomly, this can be an insurmountable problem.

The challenge-response portion of the protocol, steps (3) through (6), provides validation. Steps (3) through (5) prove to Alice that Bob knows K; steps (4) through (6) prove to Bob that Alice knows K. The Kerberos protocol timestamp exchange accomplishes the same thing.

EKE can be implemented with a variety of public-key algorithms: RSA, ElGamal, Diffie-Hellman. There are security problems with implementing EKE with a knapsack algorithm (aside from the inherent insecurity of knapsack algorithms): The normal distribution of the ciphertext messages negates the benefits of EKE.

Implementing EKE with RSA

The RSA algorithm seems perfect for this application, but there are some subtle problems. The authors recommend encrypting only the encryption exponent in step (1) and sending the modulus in the clear. An explanation of the reasoning behind this recommendation, as well as other subtleties involved in using RSA, is in [109].

Implementing EKE with ElGamal

Implementing EKE with the ElGamal algorithm is straightforward, and there is even a simplification of the basic protocol. Using the notation from Section 19.6, g and p are parts of the public key and are common to all users. The private key is a random number r. The public key is gr mod p. The message Alice sends to Bob in step (1) becomes

Alice, gr mod p

Note that this public key does not have to be encrypted with P. This is not true in general, but it is true for the ElGamal algorithm. Details are in [109].

Bob chooses a random number, R (for the ElGamal algorithm and independent of any random numbers chosen for EKE), and the message he sends to Alice in step (2) becomes

EP(gR mod p, KgRr mod p)

Refer back to Section 19.6 for restrictions on choosing the variables for ElGamal.

Implementing EKE with Diffie-Hellman

With the Diffie-Hellman protocol, K is generated automatically. The final protocol is even simpler. A value for g and n is set for all users on the network.

(1) Alice picks a random number, rA, and sends Bob
A, grA mod n

With Diffie-Hellman, Alice does not have to encrypt her first message with P.
(2) Bob picks a random number, rB, and calculates
K = grA*rB mod n

He generates a random string RB, then calculates and sends Alice:
EP(grB mod n), EK(RB)
(3) Alice decrypts the first half of Bob's message to obtain grB mod n. Then she calculates K and uses K to decrypt RB. She generates another random string, RA, encrypts both strings with K, and sends Bob the result.
EK(RA, RB)
(4) Bob decrypts the message to obtain RA and RB. Assuming the RB he received from Alice is the same as the one he sent to Alice in step (2), he encrypts RA with K and sends it to Alice.
EK(RA)
(5) Alice decrypts the message to maintain RA. Assuming the RA she received from Bob is the same as the one she sent to Bob in step (3), the protocol is complete. Both parties now communicate using K as the session key.

Strengthening EKE

Bellovin and Merritt suggest an enhancement of the challenge-and-response portion of the protocol—to prevent a possible attack if a cryptanalyst recovers an old K value.

Look at the basic EKE protocol. In step (3), Alice generates another random number, SA, and sends Bob

EK(RA, SA)

In step (4), Bob generates another random number, SB, and sends Alice

EK(RA, RB, SB)

Alice and Bob now can both calculate the true session key, SA ⊕ SB. This key is used for all future messages between Alice and Bob; K is just used as a key-exchange key.

Look at the levels of protection EKE provides. A recovered value of S gives Eve no information about P, because P is never used to encrypt anything that leads directly to S. A cryptanalytic attack on K is also not feasible; K is used only to encrypt random data, and S is never encrypted alone.

Augmented EKE

The EKE protocol suffers from one serious disadvantage: It requires that both parties possess the P. Most password-based authentication systems store a one-way hash of the user's password, not the password itself (see Section 3.2). The Augmented EKE (A-EKE) protocol uses a one-way hash of the user's password as the superencryption key in the Diffie-Hellman variant of EKE. The user then sends an extra message based on the original password; this message authenticates the newly chosen session key.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Here's how it works. As usual, Alice and Bob want to authenticate each other and generate a common key. They agree on some digital signature scheme where any number can serve as the private key, and where the public key is derived from the private key, rather than being generated along with it. The ElGamal and DSA algorithms work well for this. Alice's password P (or perhaps some simple hash of it) will serve as the private key and as P´.

(1) Alice picks her random exponent Ra and transmits
EP´(gRA mod n)
(2) Bob, who knows only P´ and cannot derive P from it, chooses Rb and sends
EP´(gRA mod n)
(3) Both Alice and Bob calculate the shared session key K = grA*rB mod n. Finally, Alice proves that she knows P itself, and not just P´, by sending
EK(SP(K))

Bob, who knows both K and P´, can decrypt and validate the signature. Only Alice could have sent this message, since only she knows P; an intruder who obtains a copy of Bob's password file can try guessing at P, but cannot otherwise sign the session key.

The A-EKE scheme does not work with the public-key variant of EKE, since in it one party chooses the session key and imposes it on the other. This permits a man-in-the-middle attack by an attacker who has captured P´.

Applications of EKE

Bellovin and Merritt suggest using this protocol for secure public telephones [109]:

Let us assume that encrypting public telephones are deployed. If someone wishes to use one of these phones, some sort of keying information must be provided. Conventional solutions...require that the caller possess a physical key. This is undesirable in many situations. EKE permits use of a short, keypad-entered password, but uses a much longer session key for the call.

EKE would also be useful with cellular phones. Fraud has been a problem in the cellular industry; EKE can defend against it (and ensure the privacy of the call) by rendering a phone useless if a PIN has not been entered. Since the PIN is not stored within the phone, it is not possible to retrieve one from a stolen unit.

EKE's primary strength is that both symmetric and public-key cryptography work together in a manner that strengthens them both:

From a general perspective, EKE functions as a privacy amplifier. That is, it can be used to strengthen comparatively weak symmetric and asymmetric systems when used together. Consider, for example, the key size needed to maintain security when using exponential key exchange. As LaMacchia and Odlyzko have shown [934], even modulus sizes once believed to be safe (to wit, 192 bits) are vulnerable to an attack requiring only a few minutes of computer time. But their attack is not feasible if one must first guess a password before applying it.

Conversely, the difficulty of cracking exponential key exchange can be used to frustrate attempts at password-guessing. Password-guessing attacks are feasible because of how rapidly each guess may be verified. If performing such verification requires solving an exponential key exchange, the total time, if not the conceptual difficulty, increases dramatically.

EKE is patented [111].

22.6 Fortified Key Negotiation

This scheme also protects key-negotiation schemes from poorly chosen passwords and man-in-the-middle attacks [47,983]. It uses a hash function of two variables that has a very special property: It has many collisions on the first variable while having effectively no collisions on the second variable.

H´(x, y) = H(H(k, x) mod 2m, x),
where H(k, x) is an ordinary hash function on k and x

Here's the protocol. Alice and Bob share a secret password, P, and have just exchanged a secret key, K, using Diffie-Hellman key exchange. They use P to check that their two session keys are the same (and that Eve is not attempting a man-in-the-middle attack), without giving P away to Eve.

(1) Alice sends Bob
H´ (P, K)
(2) Bob computes H´ (P, K) and compares his result with what he received from Alice. If they match he sends Alice
H´(H(P, K))
(3) Alice computes H´ (H(P, K)) and compares her result with what she received from Bob.

If Eve is trying a man-in-the-middle attack, she shares one key, K1, with Alice, and another key, K2, with Bob. To fool Bob in step (2), she has to figure out the shared password and then send Bob H´ * (P, K2). With a normal hash function she can try common passwords until she guesses the correct one, and then successfully infiltrate the protocol. But with this hash function, many passwords are likely to produce the same value when hashed with K1. So when she finds a match, she will probably have the wrong password, and hence Bob will not be fooled.

22.7 Conference Key Distribution and Secret Broadcasting

Alice wants to broadcast a message, M, from a single transmitter. However, she doesn't want it to be intelligible by every listener. In fact, she only wants a select subset of listeners to be able to recover M. Everyone else should get nonsense.

Alice can share a different key (secret or public) with each listener. She encrypts the message in some random key, K. Then she encrypts a copy of K with each of the keys of her intended recipients. Finally, she broadcasts the encrypted message and then all of the encrypted Ks. Bob, who is listening, either tries to decrypt all the Ks with his secret key, looking for one that is correct, or, if Alice doesn't mind everyone knowing who her message is for, he looks for his name followed by an encrypted key. Multiple-key cryptography, previously discussed, also works.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Another method is suggested in [352]. First, each listener shares a secret key with Alice, one that is larger than any possible encrypted message. All of those keys should be pairwise prime. She encrypts the message in a random key, K. Then, she computes a single integer, R, such that R modulo a secret key is congruent to K when that secret key is supposed to decrypt the message, and R modulo a secret key is otherwise congruent to zero.

For example, if Alice wants the secret to be received by Bob, Carol, and Ellen, but not by Dave and Frank, she encrypts the message with K and then computes R such that

R≡ K(mod KB)
R≡ K(mod KC)
R≡ 0 (mod KD)
R≡ K(mod KE)
R≡ 0 (mod KF)

This is a straightforward algebra problem, one that Alice can solve easily. When listeners receive the broadcast, they compute the received key modulo their secret key. If they were intended to receive the message, they recover the key. Otherwise, they recover nothing.

Yet a third way, using a threshold scheme (see Section 3.7), is suggested in [141]. Like the others, every potential receiver gets a secret key. This key is a shadow in a yet-uncreated threshold scheme. Alice saves some secret keys for herself, adding some randomness to the system. Let's say there are k people out there.

Then, to broadcast M, Alice encrypts M with key K and does the following.

(1) Alice chooses a random number, j. This number serves to hide the number of recipients of the message. It doesn't have to be very large; it can be as small as 0.
(2) Alice creates a (k + j + 1, 2k + j + 1) threshold scheme, with:
K as the secret.
The secret keys of the intended recipients as shadows.
The secret keys of nonrecipients not as shadows.
j randomly chosen shadows, not the same as any of the secret keys.
(3) Alice broadcasts k + j randomly chosen shadows, none of which is any of the shadows listed in step (2).
(4) All listeners who receive the broadcast add their shadow to the k + j shadows they received. If adding their shadow allows them to calculate the secret, then they have recovered the key. If it does not, then they haven't.

Another approach can be found in [885,886,1194]. For yet another approach, see [1000].

Conference Key Distribution

This protocol allows a group of n users to agree on a secret key using only insecure channels. The group shares two large primes, p and q, and a generator g the same size as q.

(1) User i, where i goes from 1 to n, chooses a random ri less than q, and broadcasts
zi = gri mod p
(2) Every user verifies that zjq≡ 1 (mod p), for all i from 1 to n.
(3) User i broadcasts
xi = (zi+1/zi-1)ri mod p
(4) User i computes
K = (zi-1)nri * xin-1 * xi+1n-2 *...* xi-2 mod p

All index computations in the above protocol—i - 1, i - 2, and i + 1—should be computed mod n. At the end of the protocol, all honest users have the same K. No one else gets anything. However, this protocol falls to a man-in-the-middle attack. Another protocol, not quite as pretty, is in [757].

Tatebayashi-Matsuzaki-Newman

This key distribution protocol is suitable for networks [1521]. Alice wants to generate a session key with Bob using Trent, the KDC. All parties know Trent's public key, n. Trent knows the two large primes that n factors to, and hence can easily take cube roots modulo n. A lot of the details are left out of the following protocol, but you get the idea.

(1) Alice chooses a random number, rA, and sends Trent
rA3 mod n
(2) Trent tells Bob that someone wants to exchange a key with him.
(3) Bob chooses a random number, rB, and sends Trent
rB3 mod n
(4) Trent uses his private key to recover rA and rB. He sends Alice
rA⊕ rB
(5) Alice calculates
(rA⊕ rB) ⊕ rA = rB

She uses this rB to communicate securely with Bob.

This protocol looks good, but it has a horrible flaw. Carol can listen in on step (3) and use that information, with the help of an unsuspecting Trent and another malicious user (Dave), to recover rB [1472].

(1) Carol chooses a random number, rC, and sends Trent
rB3rC3 mod n
(2) Trent tells Dave that someone wants to exchange a key with him.
(3) Dave chooses a random number, rD, and sends Trent
rD3 mod n
(4) Trent uses his private key to recover rC and rD. He sends Carol
(rBrC) mod n⊕ rD
(5) Dave sends rD to Carol.
(6) Carol uses rC and rD to recover rB. She uses rB to eavesdrop on Alice and Bob.

This is not good.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 23
Special Algorithms for Protocols

23.1 Multiple-Key Public-Key Cryptography

This is a generalization of RSA (see Section 19.3) [217,212]. The modulus, n, is the product of two primes, p and q. However, instead of choosing e and d such that ed ≡ 1 mod ((p - 1)(q - 1)), choose t keys, Ki, such that

K1 * K2 *...* Kt ≡ 1 mod ((p - 1)(q - 1))

Since

MK1*K2*...*Kt = M

this is a multiple-key scheme as described in Section 3.5.

If, for example, there are five keys, a message encrypted with K3 and K5 can be decrypted with K1 , K2 , and K4:

C = MK3*K5 mod n
M = CK1*K2*K4 mod n

One use for this is multisignatures. Imagine a situation where both Alice and Bob have to sign a document for it to be valid. Use three keys: K1, K2 , and K3. The first two are issued one each to Alice and Bob,and the third is made public.

(1) First Alice signs M and sends it to Bob.
M' = MK1 mod n
(2) Bob can recover M from M'.
M = M'K2*K3 mod n
(3) He can also add his signature.
M" = M'K2 mod n
(4) Anyone can verify the signature with K3 , the public key.
M = M"K3 mod n

Note that a trusted party is needed to set this system up and distribute the keys to Alice and Bob. Another scheme with the same problem is [484]. Yet a third scheme is [695,830,700], but the effort in verification is proportional to the number of signers. Newer schemes [220,1200] based on zero-knowledge identification schemes solve both shortcomings of the previous systems.

23.2 Secret-Sharing Algorithms

Back in Section 3.7 I discussed the idea behind secret-sharing schemes. The four different algorithms that follow are all particular cases of a general theoretical framework [883].

LaGrange Interpolating Polynomial Scheme

Adi Shamir uses polynomial equations in a finite field to construct a threshold scheme [1414]. Choose a prime, p, which is both larger than the number of possible shadows and larger than the largest possible secret. To share a secret, generate an arbitrary polynomial of degree m -1. For example, if you want to create a (3,n)-threshold scheme (three shadows are necessary to reconstruct M),generate a quadratic polynomial

(ax2 + bx + M) mod p

where p is a random prime larger than any of the coefficients. The coefficients a and b are chosen randomly; they are kept secret and discarded after the shadows are handed out. M is the message. The prime must be made public.

The shadows are obtained by evaluating the polynomial at n different points:

ki = F(xi)

In other words, the first shadow could be the polynomial evaluated at x = 1, the second shadow could be the polynomial evaluated at x = 2, and so forth.

Since the quadratic polynomial has three unknown coefficients, a, b, and M, any three shadows can be used to create three equations. Two shadows cannot. One shadow cannot. Four or five shadows are redundant.

For example, let M be 11. To construct a (3, 5)-threshold scheme, where any three of five people can reconstruct M, first generate a quadratic equation (7 and 8 were chosen randomly):

F(x) = (7x2 + 8x + 11) mod 13

The five shadows are:

k1 = F(1) = 7 + 8 + 11 ≡ 0 (mod 13)
k2 = F(2) = 28 + 16 + 11 ≡ 3 (mod 13)
k3 = F(3) = 63 + 24 + 11 ≡ 7 (mod 13)
k4 = F(4) = 112 + 32 + 11 ≡ 12 (mod 13)
k5 = F(5) = 175 + 40 + 11 ≡ 5 (mod 13)

To reconstruct M from three of the shadows, for example k2 , k3 , and k5 , solve the set of linear equations:

a * 22 + b * 2 + M ≡ 3 (mod 13)
a * 32 + b * 3 + M ≡ 7 (mod 13)
a * 52 + b * 5 + M ≡ 5 (mod 13)

The solution will be a =7, b =8, and M =11. So M is recovered.

This sharing scheme can be easily implemented for larger numbers. If you want to divide the message into 30 equal parts such that any six can get together and reproduce the message, give each of the 30 people the evaluation of a polynomial of degree 6.

F(x) = (ax6 + bx5 + cx4 + dx3 + ex2 + fx + M) mod p

Six people can solve for the six unknowns (including M); five people cannot learn anything about M.

The most mind-boggling aspect of secret sharing is that if the coefficients are picked randomly, five people with infinite computing power can't learn anything more than the length of the message (which each of them knows anyway). This is as secure as a one-time pad; an attempt at exhaustive search (that is, trying all possible sixth shadows) will reveal that any conceivable message could be the secret. This is true for all the secret-sharing schemes presented here.

Vector Scheme

George Blakley invented a scheme using points in space [182]. The message is defined as a point in m-dimensional space. Each shadow is the equation of an (m -1)-dimensional hyperplane that includes the point. The intersection of any m of the hyperplanes exactly determines the point.

For example, if three shadows are required to reconstruct the message, then it is a point in three-dimensional space. Each shadow is a different plane. With one shadow, you know the point is somewhere on the plane. With two shadows, you know the point is somewhere on the line formed where the two planes intersect. With three shadows, you can determine the point exactly: the intersection of the three planes.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Asmuth-Bloom

This scheme uses prime numbers [65]. For an (m, n)-threshold scheme, choose a large prime, p, greater than M. Then choose n numbers less than p, d1 , d2 , ..., dn, such that:

1. The d values are in increasing order; di < di+1
2. Each di is relatively prime to every other di
3. d1 * d2 * ... * dm > p * dn-m+2 * dn-m+3 * ... * dn

To distribute the shadows, first choose a random value r and compute

M' = M + rp

The shadows, ki, are

ki = M' mod di

Any m shadows can get together and reconstruct M using the Chinese remainder theorem, but any m -1 cannot. See [65] for details.

Karnin-Greene-Hellman

This scheme uses matrix multiplication [818]. Choose n +1 m-dimensional vectors, V0 , V1 , ..., Vn, such that any possible m * m matrix formed out of those vectors has rank m. The vector U is a row vector of dimension m +1.

M is the matrix product U·V0. The shadows are the products U·Vi, where i is a number from 1 to n.

Any m shadows can be used to solve the m * m system of linear equations, where the unknowns are the coefficients of U. UV0 can be computed from U. Any m -1 shadows cannot solve the system of linear equations and therefore cannot recover the secret.

Advanced Threshold Schemes

The previous examples illustrate only the simplest threshold schemes: Divide a secret into n shadows such that any m can be used to recover the secret. These algorithms can be used to create far more complicated schemes. The following examples will use Shamir's algorithm, although any of the others will work.

To create a scheme in which one person is more important than another, give that person more shadows. If it takes five shadows to recreate a secret and one person has three shadows while everyone else has only one, then that person and two other people can recreate the secret. Without that person, it takes five to recreate the secret.

Two or more people could get multiple shadows. Each person could have a different number of shadows. No matter how the shadows are distributed, any m of them can be used to reconstruct the secret. Someone with m -1 shadows, be it one person or a roomful of people, cannot do it.

In other types of schemes, imagine a scenario with two hostile delegations. You can share the secret so that two people from the 7 in Delegation A and 3 people from the 12 in Delegation B are required to reconstruct the secret. Make a polynomial of degree 3 that is the product of a linear expression and a quadratic expression. Give everyone from Delegation A a shadow that is the result of an evaluation of the linear equation; give everyone from Delegation B a shadow that is the evaluation of the quadratic equation.

Any two shadows from Delegation A can be used to reconstruct the linear equation, but no matter how many other shadows the group has, they cannot get any information about the secret. The same is true for Delegation B: They can get three shadows together to reconstruct the quadratic equation, but they cannot get any more information necessary to reconstruct the secret. Only when the two delegations share their equations can they be multiplied to reconstruct the secret.

In general, any type of sharing scheme that can be imagined can be implemented. All you have to do is to envision a system of equations that corresponds to the particular scheme. Some excellent papers on generalized secret-sharing schemes are [1462,1463,1464].

Sharing a Secret with Cheaters

This algorithm modifies the standard (m, n)-threshold scheme to detect cheaters [1529]. I demonstrate this using the Lagrange scheme, although it works with the others as well.

Choose a prime, p, that is both larger than n and larger than

(s - 1) (m - 1)/e + m

where s is the largest possible secret and e is the probability of successful cheating. You can make e as small as you want; it just makes the computation more complex. Construct your shadows as before, except instead of using 1, 2, 3,..., n for xi, choose random numbers between 1 and p - 1 for xi.

Now, when Mallory sneaks into the secret reconstruction meeting with his false share, his share has a high probability of not being possible. An impossible secret is, of course, a fake secret. See [1529] for the math.

Unfortunately, while Mallory is exposed as a cheater, he still learns the secret (assuming that there are m other valid shares). Another protocol, from [1529,975], prevents that. The basic idea is to have a series of k secrets, such that none of the participants knows beforehand which is correct. Each secret is larger than the one before, except for the real secret. The participants combine their shadows to generate one secret after the other, until they create a secret that is less than the previous secret. That's the correct one.

This scheme will expose cheaters early, before the secret is generated. There are complications when the participants deliver their shadows one at a time; refer to the papers for details. Other papers on the detection and prevention of cheaters in threshold schemes are [355,114,270].

23.3 Subliminal Channel

Ong-Schnorr-Shamir

This subliminal channel (see Section 4.2), designed by Gustavus Simmons [1458,1459,1460], uses the Ong-Schnorr-Shamir identification scheme (see Section 20.5). As in the original scheme, the sender (Alice) chooses a public modulus, n, and a private key, k, such that n and k are relatively prime. Unlike the original scheme, k is shared between Alice and Bob, the recipient of the subliminal message.

The public key is calculated:

h = -k2 mod n

If Alice wants to send the subliminal message M by means of the innocuous message M', she first confirms that M' and n are relatively prime, and that M and n are relatively prime.

Alice calculates

S1 = 1/2 * ((M' /M + M)) mod n
S2 = k/2 * ((M' /M - M)) mod n

Together, the pair, S1 and S2, is the signature under the traditional Ong-Schnorr-Shamir scheme and the carrier of the subliminal message.

Walter the warden (remember him?) can authenticate the message as described by the Ong-Schnorr-Shamir signature scheme, but Bob can do better. He can authenticate the message (it is always possible that Walter can make his own messages). He confirms that

S12 - S2 2/k2 ≡ M' (mod n)

If the message is authentic, the receiver can recover the subliminal message using this formula:

M = M'/(S1 + S2 k-1) mod n

This works, but remember that the basic Ong-Schnorr-Shamir has been broken.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

ElGamal

Simmons's second subliminal channel [1459], described in [1407,1473], is based on the ElGamal signature scheme (see Section 19.6).

Key generation is the same as the basic ElGamal signature scheme. First choose a prime, p, and two random numbers, g and r, such that both g and r are less than p. Then calculate

K = gr mod p

The public key is K, g, and p. The private key is r. Besides Alice, Bob also knows r; it is the key that is used to send and read the subliminal message in addition to being the key used to sign the innocuous message.

To send a subliminal message M using the innocuous message M', M and p must be all relatively prime to each other, and M and p -1 must be relatively prime. Alice calculates

X = gM mod p

and solves the following equation for Y (using the extended Euclidean algorithm):

M' = rX + MY mod (p - 1)

As in the basic ElGamal scheme, the signature is the pair: X and Y.

Walter can verify the ElGamal signature. He confirms that

KXXY ≡ gM' (mod p)

Bob can recover the subliminal message. First he confirms that

(gr)X XY ≡ gM' (mod p)

If it does, he accepts the message as genuine (not from Walter).

Then, to recover M, he computes

M = (Y–1 (M' - rX)) mod (p - 1)

For example, let p =11 and g =2. The private key, r, is chosen to be 8. This means the public key, which Walter can use to verify the signature, is gr mod p =28 mod 11 =3.

To send the subliminal message M =9, using innocuous message M'= 5, Alice confirms that 9 and 11 are relatively prime and that 5 and 11 are relatively prime. She also confirms that 9 and 11 -1 =10 are relatively prime. They are, so she calculates

X = gM mod p = 29 mod 11 = 6

Then, she solves the following equation for Y:

5 = 8 * 6 + 9 * Y mod 10

Y = 3, so the signature is the pair, X and Y: 6 and 3.

Bob confirms that

(gr)X XY ≡ gM' (mod p)
(28)663 ≡ 25 (mod 11)

It does (do the math yourself if you don't trust me), so he then recovers the subliminal message by calculating

M = (Y–1 (M' - rX)) mod (p - 1) = 3-1(5 - 8 * 6) mod 10 = 7(7) mod 10 = 49 mod 10 = 9

ESIGN

A subliminal channel can be added to ESIGN [1460] (see Section 20.6).

In ESIGN, the secret key is a pair of large prime numbers, p and q, and the public key is n =p2q . With a subliminal channel, the private key is three primes, p, q, and r, and the public key is n, such that

n = p2qr

The variable, r, is the extra piece of information that Bob needs to read the subliminal message.

To sign a normal message, Alice first picks a random number, x, such that x is less than pqr and computes:

w, the least integer that is larger than (H(m) - xk mod n)/pqr)
s = x + ((w/kxk-1) mod p)pqr

H(m) is the hash of the message; k is a security parameter. The value s is the signature.

To verify the signature, Bob computes sk mod n. He also computes a, which is the least integer larger than the number of bits of n divided by 3. If H(m) is less than or equal to sk mod n, and if sk mod n is less than H(m) +2a , then the signature is considered valid.

To send a subliminal message, M, using the innocuous message, M', Alice calculates s using M in place of H(m). This means that the message must be smaller than p2qr. She then chooses a random value, u, and calculates

x' = M' + ur

Then, use this x' value as the "random number" x to sign M'. This second s value is sent as a signature.

Walter can verify that s (the second s) is a valid signature of M'.

Bob can also authenticate the message in the same way. But, since he also knows r, he can calculate

s = x' + ypqr = M + ur + ypqr ≡ M (mod r)

This implementation of a subliminal channel is far better than the previous two. In the Ong-Schnorr-Shamir and ElGamal implementations, Bob has Alice's private key. Besides being able to read subliminal messages from Alice, Bob can impersonate Alice and sign normal documents. Alice can do nothing about this; she must trust Bob to set up this subliminal channel.

The ESIGN scheme doesn't suffer from this problem. Alice's private key is the set of three primes: p, q, and r. Bob's secret key is just r. He knows n =p2qr, but to recover p and q he has to factor that number. If the primes are large enough, Bob has just as much trouble impersonating Alice as would Walter or anyone else.

DSA

There is also a subliminal channel in DSA (see Section 20.1) [1468,1469,1473]. In fact, there are several. The simplest subliminal channel involves the choice of k. It is supposed to be a 160-bit random number. However, if Alice chooses a particular k, then Bob, who also knows Alice's private key, can recover it. Alice can send Bob a 160-bit subliminal message in each DSA signature; everyone else simply verifies Alice's signature. Another complication: Since k should be random, Alice and Bob need to share a one-time pad and encrypt the subliminal message with the one-time pad to generate a k.

DSA has subliminal channels that do not require Bob to know Alice's private key. These also involve choosing particular values of k, but cannot be used to send 160 bits of information. This scheme, presented in [1468,1469], allows Alice and Bob to exchange one bit of subliminal information per signed message.

(1) Alice and Bob agree on a random prime, P (different from the parameter p in the signature scheme). This is their secret key for the subliminal channel.
(2) Alice signs an innocuous message, M. If she wants to send Bob the subliminal bit, 1, she makes sure the r parameter of the signature is a quadratic residue modulo P. If she wants to send him a 0, she makes sure the r parameter is a quadratic nonresidue modulo P. She does this by signing the message with random k values until she gets a signature with an r with the requisite property. Since quadratic residues and quadratic nonresidues are equally likely, this shouldn't be too difficult.
(3) Alice sends the signed message to Bob.
(4) Bob verifies the signature to make sure the message is authentic. Then he checks whether r is a quadratic residue or a quadratic nonresidue modulo P and recovers the subliminal bit.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Sending multiple bits via this method involves making r either a quadratic residue or a quadratic nonresidue modulo a variety of parameters. See [1468,1469] for details.

This scheme can be easily extended to send multiple subliminal bits per signature. If Alice and Bob agree on two random primes, P and Q, Alice can send two bits by choosing a random k such that r is either a quadratic residue mod P or a quadratic nonresidue mod P, and either a quadratic residue mod Q or a quadratic nonresidue mod Q. A random value of k has a 25 percent chance of producing an r of the correct form.

Here's how Mallory, an unscrupulous implementer of DSA,can have the algorithm leak 10 bits of Alice's private key every time she signs a document.

(1) Mallory puts his implementation of DSA in a tamperproof VLSI chip, so that no one can examine its inner workings. He creates a 14-bit subliminal channel in his implementation of DSA. That is, he chooses 14 random primes, and has the chip choose a value of k such that r is either a quadratic residue or a quadratic nonresidue modulo each of those 14 primes, depending on the subliminal message.
(2) Mallory distributes the chips to Alice, Bob, and everyone else who wants them.
(3) Alice signs a message normally, using her 160-bit private key, x.
(4) The chip randomly chooses a 10-bit block of x: the first 10 bits, the second 10 bits, and so on. Since there are 16 possible 10-bit blocks, a 4-bit number can identify which block it is. This 4-bit identifier, plus the 10 bits of the key, is the 14-bit subliminal message.
(5) The chip tries random values of k until it finds one that has the correct quadratic residue properties to send the subliminal message. The odds of a random k being of the correct form are 1 in 16,384. Assuming the chip can test 10,000 values of k per second, it will find one in less than two seconds. This computation does not involve the message and can be performed off-line, before Alice wants to sign a message.
(6) The chip signs the message normally, using the value of k chosen in step (5).
(7) Alice sends the digital signature to Bob, or publishes it on the network, or whatever.
(8) Mallory recovers r and, because he knows the 14 primes, decrypts the subliminal message.

It's scary that even if Alice knows what is happening, she cannot prove it. As long as those 14 secret primes stay secret, Mallory is safe.

Foiling the DSA Subliminal Channel

The subliminal channel relies on the fact that Alice can choose k to transmit subliminal information. To foil the subliminal channel, Alice cannot be allowed to choose k. However, neither can anyone else; if someone else were allowed to choose k, it would allow that person to forge Alice's signature. The only solution is for Alice to jointly generate k with another party, call him Bob, in such a way that Alice cannot control a single bit of k and Bob cannot know a single bit of k. At the end of the protocol, Bob should be able to verify that Alice used the k that they jointly generated.

Here's the protocol [1470,1472,1473]:

(1) Alice chooses k' and sends Bob
u = gk' mod p
(2) Bob chooses k" and sends it to Alice.
(3) Alice calculates k = k'k" mod (p - 1). She uses k to sign her message, M, with the DSA and sends Bob the signature: r and s.
(4) Bob verifies that
((uk" mod p) mod q) = r

If it does, he knows that k was used to sign M.

After step (4), Bob knows that no subliminal information can be embedded in r. If he is a trusted party, he can certify that Alice's signature is subliminal-free. Others will have to trust his certification; Bob cannot prove this fact to a third party with a transcript of the protocol.

A surprising result is that if Bob wants to, he can use this protocol to create his own subliminal channel. Bob can embed a subliminal message in one of Alice's signatures by choosing k" with certain characteristics. When Simmons discovered this, he dubbed it the "Cuckoo's Channel." Details on how the Cuckoo's Channel works, and a three-pass protocol for generating k that prevents it, are discussed in [1471,1473].

Other Schemes

Any signature scheme can be converted into a subliminal channel [1458,1460,1406]. A protocol for embedding a subliminal channel in the Fiat-Shamir and Feige-Fiat-Shamir protocols, as well as possible abuses of the subliminal channel, can be found in [485].

23.4 Undeniable Digital Signatures

This undeniable signature algorithm (see Section 4.3) is by David Chaum [343,327]. First, a large prime, p, and a primitive element, g, are made public, and used by a group of signers. Alice has a private key, x, and a public key, gx mod p.

To sign a message, Alice computes z =mx mod p.That's all she has to do.

Verification is a little more complicated.

(1) Bob chooses two random numbers, a and b, both less than p, and sends Alice:
c = za(gx)b mod p
(2) Alice computes t=x–1 mod (p - 1), and sends Bob:
d = ct mod p
(3) Bob confirms that
d ≡ magb (mod p)

If it is, he accepts the signature as genuine.

Imagine that Alice and Bob went through this protocol, and Bob is now convinced that Alice signed the message. Bob wants to convince Carol, so he shows her a transcript of the protocol. Dave, however, wants to convince Carol that some other person signed the document. He creates a fake transcript of the protocol. First he generates the message in step (1). Then in step (3) he generates d and the fake transmission from this other person in step (2). Finally, he creates the message in step (2). To Carol, both Bob's and Dave's transcripts are identical. She cannot be convinced of the signature's validity unless she goes through the protocol herself.

Of course, if she were watching over Bob's shoulder as he completed the protocol, she would be convinced. Carol has to see the steps done in order, just as Bob does.

There may be a problem with this signature scheme, but I know of no details. Please pay attention to the literature before you use it.

Another protocol not only has a confirmation protocol—Alice can convince Bob that her signature is valid—but it also has a disavowal protocol; Alice can use a zero-knowledge interactive protocol to convince him that the signature is not valid, if it is not [329].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Like the previous protocol, a group of signers use a shared public large prime, p, and a primitive element, g. Alice has a unique private key, x, and a public key, gx mod p. To sign a message, Alice computes z =mx mod p.

To verify a signature:

(1) Bob chooses two random numbers, a and b, both less than p, and sends Alice:
c = magb mod p
(2) Alice chooses a random number, q, less than p, and computes and sends to Bob:
s1 = cgq mod p, s2 = (cgq)x mod p
(3) Bob sends Alice a and b, so that Alice can confirm that Bob did not cheat in step (1).
(4) Alice sends Bob q, so that Bob can use mx and reconstruct s1 and s2. If then the signature is valid.
s1 ≡ cgq (mod p)
s2 ≡ (gx)b +qza (mod p)

Alice can also disavow a signature, z, for a message, m. See [329] for details.

Additional protocols for undeniable signatures can be found in [584,344]. Lein Harn and Shoubao Yang proposed a group undeniable signature scheme [700].

Convertible Undeniable Signatures

An algorithm for a convertible undeniable signature, which can be verified, disavowed, and also converted to a conventional digital signature is given in [213]. It's based on the ElGamal digital signature algorithm.

Like ElGamal, first choose two primes, p and q, such that q divides p -1. Now you have to create a number, g, less than q. First choose a random number, h, between 2 and p -1. Calculate

g = h( p-1)/q mod p

If g equals the 1, choose another random h. If it doesn't, stick with the g you have.

The private keys are two different random numbers, x and z, both less than q. The public keys are p, q, g, y, and u, where

y = gx mod p
u = gz mod p

To compute the convertible undeniable signature of message m (which is actually the hash of a message), first choose a random number, t, between 1 and q -1. Then compute

T = gt mod p

and

m' = Ttzm mod q.

Now, compute the standard ElGamal signature on m'. Choose a random number, R, such that R is less than and relatively prime to p -1. Then compute r =gR mod p, and use the extended Euclidean algorithm to compute s, such that

m' ≡ rx + Rs (mod q)

The signature is the ElGamal signature (r, s), and T.

Here's how Alice verifies her signature to Bob:

(1) Bob generates two random numbers, a and b. He computes c = TTmagb mod p and sends that to Alice.
(2) Alice generates a random number, k, and computes h1 = cgk mod p, and h2 = h1 z mod p, and sends both of those numbers to Bob.
(3) Bob sends Alice a and b.
(4) Alice verifies that c = TTmagb mod p. She sends k to Bob.
(5) Bob verifies that h1 = TTmagb+k mod p, and that h2 = yrarsaub+k mod p.

Alice can convert all of her undeniable signatures to normal signatures by publishing z. Now, anyone can verify her signature without her help.

Undeniable signature schemes can be combined with secret-sharing schemes to create distributed convertible undeniable signatures [1235]. Someone can sign a message, then distribute the ability to confirm that the signature is valid. He might, for example, require three out of five people to participate in the protocol in order to convince Bob that the signature is valid. Improvements on this notion deleted the requirement for a trusted dealer [700,1369].

23.5 Designated Confirmer Signatures

Here's how Alice can sign a message and Bob can verify it, such that Carol can verify Alice's signature at some later time to Dave (see Section 4.4) [333].

First, a large prime, p, and a primitive element, g, are made public and used by a group of users. The product of two primes, n, is also public. Carol has a private key, z, and a public key is h =gx mod p.

In this protocol Alice can sign m such that Bob is convinced that the signature is valid,but cannot convince a third party.

(1) Alice chooses a random x and computes
a = gx mod p
b = hx mod p

She computes the hash of m, H(m), and the hash of a and b concatenated, H(a,b). She then computes
j = (H(m) ⊕ H(a, b))1/3 mod n

and sends a, b, and j to Bob.
(2) Bob chooses two random numbers, s and t, both less than p, and sends Alice
c = gsht mod p
(3) Alice chooses a random q less than p, and sends Bob
d = gq mod p
e = (cd)x mod p
(4) Bob sends Alice s and t.
(5) Alice confirms that
gsht ≡ c (mod p)

Then she sends Bob q.
(6) Bob confirms
d ≡ gq (mod p)
e/aq ≡ asbt (mod p)
H(m) ⊕ H(a, b) ≡ j1/3 mod n

If they all check out, he accepts the signature as genuine.

Bob cannot use a transcript of this proof to convince Dave that the signature is genuine, but Dave can conduct a protocol with Alice's designated confirmer, Carol. Here's how Carol convinces Dave that a and bconstitute a valid signature.

(1) Dave chooses a random u and v, both less than p, and sends Carol
k = guav mod p
(2) Carol chooses a random w, less than p, and sends Dave
l = gw mod p
y = (kl)z mod p
(3) Dave sends Carol u and v.
(4) Carol confirms that
guav ≡ k (mod p)

Then she sends Dave w.
(5) Dave confirms that
gw ≡ l (mod p)
y/hw ≡ hubv (mod p)

If they both check out, he accepts the signature as genuine.

In another protocol Carol can convert the designated-confirmer protocol into a conventional digital signature. See [333] for details.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

23.6 Computing with Encrypted Data

The Discrete Logarithm Problem

There is a large prime, p, and a generator, g. Alice has a particular value for x, and wants to know e, such that

ge ≡ x (mod p)

This is a hard problem, and Alice lacks the computational power to compute the result. Bob has the power to solve the problem—he represents the government, or a large computing organization, or whatever. Here's how Bob can do it without Alice revealing x [547,4]:

(1) Alice chooses a random number, r, less than p.
(2) Alice computes
x' = xgr mod p
(3) Alice asks Bob to solve
ge' ≡ x' (mod p)
(5) Bob computes e' and sends it to Alice.
(6) Alice recovers e by computing
e = (e' - r) mod (p - 1)

Similar protocols for the quadratic residuosity problem and for the primitive root problem are in [3,4]. (See also Section 4.8.)

23.7 Fair Coin Flips

The following protocols allow Alice and Bob to flip a fair coin over a data network (see Section 4.9) [194]. This is an example of flipping a coin into a well (see Section 4.10). At first, only Bob knows the result of the coin toss and tells it to Alice. Later, Alice may check to make sure that Bob told her the correct outcome of the toss.

Coin Flipping Using Square Roots

Coin-flip subprotocol:

(1) Alice chooses two large primes, p and q, and sends their product, n to Bob.
(2) Bob chooses a random positive integer, r, such that r is less than n/2. Bob computes
z = r2 mod n

and sends z to Alice.
(3) Alice computes the four square roots of z (mod n). She can do this because she knows the factorization of n. Let's call them +x, -x, +y, and -y. Call x' the smaller of these two numbers:
x mod n
-x mod n

Similarly, call y' the smaller of these two numbers:
y mod n
-y mod n

Note that r is equal either to x' or y'.
(4) Alice guesses whether r = x' or r = y', and sends her guess to Bob.
(5) If Alice's guess is correct, the result of the coin flip is heads. If Alice's guess is incorrect, the result of the coin flip is tails. Bob announces the result of the coin flip.

Verification subprotocol:

(6) Alice sends p and q to Bob.
(7) Bob computes x' and y' and sends them to Alice.
(8) Alice calculates r.

Alice has no way of knowing r, so her guess is real. She only tells Bob one bit of her guess in step (4) to prevent Bob from getting both x' and y'. If Bob has both of those numbers, he can change r after step (4).

Coin Flipping Using Exponentiation Modulo p

Exponentiation modulo a prime number, p, is used as a one-way function in this protocol [1306]:

Coin-flip subprotocol:

(1) Alice chooses a prime number, p, in such a way that the factorization of p - 1 is known and contains at least one large prime.
(2) Bob selects two primitive elements, h and t, in GF(p). He sends them to Alice.
(3) Alice checks that h and t are primitive and then chooses a random integer x, relatively prime to p - 1. She then computes one of the two values:
y = hx mod p, or y = tx mod p

She sends y to Bob.
(4) Bob guesses whether Alice calculated y as afunction of h or t, and sends his guess to Alice.
(5) If Bob's guess is correct, the result of the coin flip is heads. If Bob's guess is incorrect, the result of the coin flip is tails. Alice announces the result of the coin flip.

Verification subprotocol:

(6) Alice reveals x to Bob. Bob computes hx mod p and tx mod p, to confirm that Alice has played fairly and to verify the result of the toss. He also checks that x and p - 1 are relatively prime.

For Alice to cheat, she has to know two integers, x and x', such that hx ≡tx' (mod p). If she knew those values,she would be able to calculate:

logt h = x'x-1 mod p - 1 and logth = x-1x' mod p - 1

These are hard problems.

Alice would be able to do this if she knew logt h, but Bob chooses h and t in step (2). Alice has no other recourse except to try to compute the discrete logarithm. Alice could also attempt to cheat by choosing an x that is not relatively prime to p -1, but Bob will detect that in step (6).

Bob can cheat if h and t are not primitive in GF(p), but Alice can easily check that after step (2) because she knows the prime factorization of p -1.

One nice thing about this protocol is that if Alice and Bob want to flip multiple coins, they can use the same values for p, h, and t. Alice just generates a new x, and the protocol continues from step (3).

Coin Flipping Using Blum Integers

Blum integers can be used in a coin-flipping protocol.

(1) Alice generates a Blum integer, n, a random x relatively prime to n, x0 = x2 mod n, and x1 = x02 mod n. She sends n and x1 to Bob.
(2) Bob guesses whether x0 is even or odd.
(3) Alice sends x to Bob.
(4) Bob checks that n is a Blum integer (Alice would have to give Bob the factors of n and proofs of their primality, or execute some zero-knowledge protocol to convince him that n is a Blum integer), and he verifies that x0 = x2 mod n and x1 = x02 mod n. If all this checks out, Bob wins the flip if he guessed correctly.

It is crucial that n be a Blum integer. Otherwise, Alice may be able to find an x'0 such that x'02 mod n =x0 2 mod n =x1 , where x'0 is also a quadratic residue. If x0 were even and x'0 were odd (or vice versa), Alice could freely cheat.

23.8 One-Way Accumulators

There is a simple one-way accumulator function [116] (see Section 4.12):

A(xi, y) = xi-1y mod n


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The numbers n (n is the product of two primes) and x0 must be agreed upon in advance. Then, the accumulation of y1 , y2 , and y3 would be

((x0 y1 mod n)y2 mod n)y3 mod n

This computation is independent of the order of y1, y2 , and y3.

23.9 All-or-Nothing Disclosure of Secrets

This protocol allows multiple parties (at least two are required for the protocol to work) to buy individual secrets from a single seller (see Section 4.13) [1374,1175]. First, here's a definition. Take two bit strings, x and y. The fixed bit index (FBI) of x and y are the bits where the ith bit of x equals the ith bit of y.

For example:

x = 110101001011
y = 101010000110
FBI(x, y) = {1, 4, 5, 11}
(We're reading the bits from right to left, with the right-most bit as zero.)

Now, here's the protocol. Alice is the seller. Bob and Carol are buyers. Alice has k n-bit secrets: S1, S2 , ..., Sk. Bob wants to buy secret Sb; Carol wants to buy secret Sc.

(1) Alice generates a public-key/private-key key pair and tells Bob (but not Carol) the public key. She generates another public-key/private-key key pair and tells Carol (but not Bob) the public key.
(2) Bob generates k n-bit random numbers, B1 , B2 , ..., Bk, and tells them to Carol. Carol generates k n-bit random numbers, C1 , C2 , ..., Ck, and tells them to Bob.
(3) Bob encrypts Cb (remember, Sb is the secret he wants to buy) with the public key from Alice. He computes the FBI of Cb and the result he just encrypted. He sends this FBI to Carol.
Carol encrypts Bc (remember, Sc is the secret she wants to buy) with the public key from Alice. She computes the FBI of Bc and the result she just encrypted. She sends this FBI to Bob.
(4) Bob takes each of the n-bit numbers B1, B2 , ..., Bk, and replaces every bit whose index is not in the FBI he received from Carol with its complement. He sends this new list of n-bit numbers, B'1, B'2, ..., B'k, to Alice.
Carol takes each of the n-bit numbers C1 , C2 , ..., Ck, and replaces every bit whose index is not in the FBI she received from Bob with its complement. She sends this new list of n-bit numbers, C'1, C'2, ..., C'k, to Alice.
(5) Alice decrypts all C'i with Bob's private key, giving her k n-bit numbers: C"1, C"2, ..., C"k. She computes Si ⊕ C"i, for i = 1 to k, and sends the results to Bob.
Alice decrypts all B'i with Carol's private key, giving her k n-bit numbers: B"1, B"2, ..., B"k. She computes Si ⊕ B"i, for i = 1 to k, and sends the results to Carol.
(6) Bob computes Sb by XORing Cb and the bth number he received from Alice.
Carol computes Sc by XORing Bc and the cth number she received from Alice.

This is complicated. An example will go a long way to help.

Alice has the following eight 12-bit secrets for sale: S1 =1990, S2 =471, S3 =3860, S4 =1487, S5 =2235, S6 =3751, S7 =2546, and S8 =4043. Bob wants to buy S7. Carol wants to buy S2.

(1) Alice uses the RSA algorithm. The key pair she will use with Bob is: n = 7387, e = 5145, and d = 777. The key pair she will use with Carol is: n = 2747, e = 1421, and d = 2261. She tells Bob and Carol each their public key.
(2) Bob generates eight 12-bit random numbers, B1 = 743, B2 = 1988, B3 = 4001, B4 = 2942, B5 = 3421, B6 = 2210, B7 = 2306, and B8 = 222, and tells them to Carol. Carol generates eight 12-bit random numbers, C1 = 1708, C2 = 711, C3 = 1969, C4 = 3112, C5 = 4014, C6 = 2308, C7 = 2212, and C8 = 222, and tells them to Bob.
(3) Bob wants to buy S7 , so he encrypts C7 with the public key that Alice gave him.
22125145 mod 7387 = 5928

Now:
2212 = 0100010100100
5928 = 1011100101000

So, the FBI of those two numbers is {0, 1, 4, 5, 6}. He sends this to Carol.

Carol wants to buy S2 , so she encrypts B2 with the public key that Alice gave her and computes the FBI of B2 with the result of her encryption. She sends {0, 1, 2, 6, 9, 10} to Bob.

(4) Bob takes B1 , B2 , ..., B8 , and replaces every bit whose index is not in the set {0, 1, 2, 6, 9, 10} with its complement. For example:
B2 = 111111000100 = 1988
B'2 = 011001111100 = 1660

He sends B'1, B'2, ..., B'8, to Alice.
Carol takes C1 , C2 , ..., C8 , and replaces every bit whose index is not in the set {0, 1, 4, 5, 6} with its complement. For example:
C7 = 0100010100100 = 2212
C'7 = 1011100101000 = 5928

She sends C'1, C'2, ..., C'8, to Alice.
(5) Alice decrypts all C'i with Bob's private key and XORs the results with Si. For example, for i = 7:
5928777 mod 7387 = 2212; 2546 ⊕ 2212 = 342

She sends the results to Bob.
Alice decrypts all B'i with Carol's private key and XORs the results with Si. For example, for i = 2:
16602261 (mod 2747) = 1988; 471 ⊕ 1988 = 1555

She sends the results to Carol.
(6) Bob computes S7 by XORing C7 and the seventh number he received from Alice:
2212 ⊕ 342 = 2546

Carol computes S2 by XORing B2 and the second number she received from Alice.
1988 ⊕ 1555 = 471

The protocol works for any number of buyers. If Bob, Carol, and Dave want to buy secrets, Alice gives each buyer two public keys, one for each of the others. Each buyer gets a set of numbers from each other buyer. Then, they complete the protocol with Alice for each of their sets of numbers and XOR all of their final results from Alice to get their secret. More details are in [1374,1175].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Unfortunately, a pair of dishonest parties can cheat. Alice and Carol, working together, can easily find out what secret Bob is getting: If they know the FBI of Cb and Bob's encryption algorithm, they can find b such that Cb has the right FBI. And Bob and Carol, working together, can easily get all the secrets from Alice.

If you assume honest parties,there's an easier protocol [389].

(1) Alice encrypts all of the secrets with RSA and sends them to Bob: Ci = Sie mod n
(2) Bob chooses his secret Cb, picks a random r, and sends C', to Alice.
C' = Cbre mod n
(3) Alice sends Bob P'.
P' = C'd mod n
(4) Bob calculates Sb.
Sb = P'r-1 mod n

If the parties may be dishonest, Bob can prove in zero-knowledge that he knows some r such that C'=Cbre mod n and keep b secret until Alice gives him P' in step (3) [246].

23.10 Fair and Failsafe Cryptosystems

Fair Diffie-Hellman

Fair cryptosystems are a way to do key escrowing in software (see Section 4.14). This example is from Silvio Micali [1084,1085]. It is patented [1086,1087].

In the basic Diffie-Hellman scheme, a group of users share a prime, p, and a generator, g. Alice's private key is s, and her public key is t =gs mod p.

Here's how to make Diffie-Hellman fair (this example uses five trustees).

(1) Alice chooses five integers, s1 , s2, s3, s4, and s5, each less than p - 1. Alice's private key is
s = (s1 + s2 + s3 + s4 + s5) mod p - 1

and her public key is
t = gs mod p

Alice also computes
ti = gsi mod p, for i = 1 to 5

Alice's public shares are ti, and her private shares are si.
(2) Alice sends a private piece and corresponding public piece to each trustee. For example, she sends s1 and t1 to trustee 1. She sends t to the KDC.
(3) Each trustee verifies that
ti = gsi mod p

If it does, the trustee signs ti and sends it to the KDC. The trustee stores si in a secure place.
(4) After receiving all five public pieces, the KDC verifies that
t = (t1 * t2 * t3 * t4 * t5) mod p

If it does, the KDC approves the public key.

At this point, the KDC knows that the trustees each have a valid piece, and that they can reconstruct the private key if required. However, neither the KDC nor any four of the trustees working together can reconstruct Alice's private key.

Micali's papers [1084,1085] also contain a procedure for making RSA fair and for combining a threshold scheme with the fair cryptosystem, so that m out of n trustees can reconstruct the private key.

Failsafe Diffie-Hellman

Like the previous protocol, a group of users share a prime, p, and a generator, g. Alice's private key is s, and her public key is t =gs mod p.

(1) The KDC chooses a random number, B, between 0 and p - 2, and commits to B using a bit-commitment protocol (see Section 4.9).
(2) Alice chooses a random number, A, between 0 and p - 2. She sends gA mod p to the KDC.
(3) The user "shares" A with each trustee using a verifiable secret-sharing scheme (see Section 3.7).
(4) The KDC reveals B to Alice.
(5) Alice verifies the commitment from step (1). Then she sets her public key as
t = (gA)gB mod p

She sets her private key as
s = (A + B) mod (p - 1)

The trustees can reconstruct A. Since the KDC knows B, this is enough to reconstruct s. And Alice cannot make use of any subliminal channels to send unauthorized information. This protocol, discussed in [946,833] is being patented.

23.11 Zero-Knowledge Proofs of Knowledge

Zero-Knowledge Proof of a Discrete Logarithm

Peggy wants to prove to Victor that she knows an x that satisfies

Ax ≡ B (mod p)

where p is a prime, and x is a random number relatively prime to p - 1. The numbers A, B, and p are public, and x is secret. Here's how Peggy can prove she knows x without revealing it (see Section 5.1) [338,337].

(1) Peggy generates t random numbers, r1 , r2 , ..., rt, where all ri are less than p - 1.1
(2) Peggy computes hi = Ari mod p, for all values of i, and sends them to Victor.
(3) Peggy and Victor engage in a coin-flipping protocol to generate t bits: b1, b2 , ..., bt.
(4) For all t bits, Peggy does one of the following:
a) If bi = 0, she sends Victor ri
b) If bi = 1, she sends Victor si = (ri - rj) mod (p - 1), where j is the lowest value for which bj = 1
(5) For all t bits, Victor confirms one of the following:
a) If bi = 0, that Ari ≡ hi (mod p)
b) If bi = 1, that Asi ≡ hihj-1 (mod p)
(6) Peggy sends Victor Z, where
Z = (x - rj) mod (p - 1)
(7) Victor confirms that AZ ≡ Bhj-1 (mod p)

Peggy's probability of successfully cheating is 2-t.

Zero-Knowledge Proof of the Ability to Break RSA

Alice knows Carol's private key. Maybe she has broken RSA; maybe she has broken into Carol's house and stolen the key. Alice wants to convince Bob that she knows Carol's key. However, she doesn't want to tell Bob the key or even decrypt one of Carol's messages for Bob. Here's a zero-knowledge protocol by which Alice convinces Bob that she knows Carol's private key [888].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Carol's public key is e, her private key is d, and the RSA modulus is n.

(1) Alice and Bob agree on a random k and an m such that
km ≡ e (mod n)

They should choose the numbers randomly, using a coin-flip protocol to generate k and then computing m. If both k and m are greater than 3, the protocol continues. Otherwise, they choose again.
(2) Alice and Bob generate a random ciphertext, C. Again, they should use a coin-flip protocol.
(3) Alice, using Carol's private key, computes
M = Cd mod n

She then computes

X = Mk mod n

and sends X to Bob.
(4) Bob confirms that Xm mod n = C. If it does, he believes Alice.

A similar protocol can be used to demonstrate the ability to break a discrete logarithm problem [888].

Zero-Knowledge Proof that n Is a Blum Integer

There are no known truly practical zero-knowledge proofs that n =pq, where p and q are primes congruent to 3 modulo 4. However, if you allow n to be of the form prqs, where r and s are odd, then the properties which make Blum integers useful in cryptography still hold. And there exists a zero-knowledge proof that n is of that form.

Assume Alice knows the factorization of the Blum integer n, where n is of the form previously discussed. Here's how she can prove to Bob that n is of that form [660].

(1) Alice sends Bob a number u which has a Jacobi symbol -1 modulo n.
(2) Alice and Bob jointly agree on random bits: b1 , b2 , ..., bk.
(3) Alice and Bob jointly agree on random numbers: x1 , x2 , ..., xk.
(4) For each i = 1, 2,..., k, Alice sends Bob a square root modulo n, of one of the four numbers: xi, -xi, uxi, -uxi. The square root must have the Jacobi symbol bi.

The odds of Alice successfully cheating are one in 2k.

23.12 Blind Signatures

The notion of blind signatures (see Section 5.3) was invented by David Chaum [317,323], who also invented their first implementation [318]. It uses the RSA algorithm.

Bob has a public key, e, a private key, d, and a public modulus, n. Alice wants Bob to sign message m blindly.

(1) Alice chooses a random value, k, between 1 and n. Then she blinds m by computing
t = mke mod n
(2) Bob signs t
td = (mke)d mod n
(3) Alice unblinds td by computing
s = td/k mod n
(4) And the result is
s = md mod n

This can easily be shown
td ≡ (mke)d ≡ mdk (mod n), so td/k = mdk/k ≡ md (mod n).

Chaum invented a family of more complicated blind signature algorithms in [320,324], called blind unanticipated signatures. These signatures are more complex in construction, but more flexible.

23.13 Oblivious Transfer

In this protocol by Michael Rabin [1286], Alice has a 50 percent chance of sending Bob two primes, p, and q. Alice will not know whether the transfer is successful. (See Section 5.5.) (This protocol can be used to send Bob any message with a 50 percent success rate if p and q reveal an RSA private key.)

(1) Alice sends Bob the product of the two primes: n = pq.
(2) Bob chooses a random x less than n, such that x is relatively prime to n. He sends Alice:
a = x2 mod n
(3) Alice, knowing p and q, computes the four roots of a: x, n - x, y, and n - y. She chooses one of these roots at random and sends it to Bob.
(4) If Bob receives y or n - y, he can compute the greatest common divisor of x + y and n, which is either p or q. Then, of course, n/p = q.
If Bob receives x or n - x, he can't compute anything.

This protocol may have a weakness: It might be the case that Bob can compute a number a such that given the square root of a you can calculate a factor of n all the time.

23.14 Secure Multiparty Computation

This protocol is from [1373]. Alice knows the integer i; Bob knows the integer j. Alice and Bob together wish to know whether i ≤ j or if i > j, but neither Alice nor Bob wish to reveal the integer each knows. This special case of secure multiparty computation (see Section 6.2) is sometimes known as Yao's millionaire problem [1627].

For this example, assume that i and j range from 1 to 100. Bob has a public key and a private key.

(1) Alice chooses a large random number, x, and encrypts it in Bob's public key.
c = EB(x)
(2) Alice computes c - i and sends the result to Bob.
(3) Bob computes the following 100 numbers:
yu = DB(c - i + u), for 1 ≤ u ≤ 100

DB is the decryption algorithm with Bob's private key.
He chooses a large random prime, p. (The size of p should be somewhat smaller than x. Bob doesn't know x, but Alice could easily tell him the size of x.) He then computes the following 100 numbers:
zu = (yu mod p), for 1 ≤ u ≤ 100

He then verifies that, for all u ≠ v
|zu - zv| ≥ 2

and that for all u
0 < zu < p - 1

If this is not true, Bob chooses another prime and tries again.
(4) Bob sends Alice this sequence of numbers in this exact order:
z1 , z2 , ..., zj, zj+1 + 1, zj+2 + 1,..., z100 + 1, p
(5) Alice checks whether the ith number in the sequence is congruent to x mod p. If it is, she concludes that i ≤ j. If it is not, she concludes that i > j.
(6) Alice tells Bob the conclusion.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

All the verification that Bob goes through in step (3) is to guarantee that no number appears twice in the sequence generated in step (4). Otherwise, if za =zb, Alice knows that a ≤j <b.

The one drawback to this protocol is that Alice learns the result of the computation before Bob does. Nothing stops her from completing the protocol up to step (5) and then refusing to tell Bob the results in step (6). She could even lie to Bob in step (6).

Example of the Protocol

Assume they're using RSA. Bob's public key is 7 and his private key is 23; n =55. Alice's secret value, i, is 4; Bob's secret value, j, is 2. (Assume that only the values 1, 2, 3, and 4 are possible for i and j.)

(1) Alice chooses x = 39, and c = EB(39) = 19.
(2) Alice computes c - i = 19 - 4 = 15. She sends 15 to Bob.
(3) Bob computes the following 4 numbers:
y1 = DB(15 + 1) = 26
y2 = DB(15 + 2) = 18
y3 = DB(15 + 3) = 2
y4 = DB(15 + 4) = 39

He chooses p =31 and calculates:

z1 = (26 mod 31) = 26
z2 = (18 mod 31) = 18
z3 = (2 mod 31) = 2
z4 = (39 mod 31) = 8

He does all the verifications and confirms that the sequence is fine.
(4) Bob sends Alice this sequence of numbers in this exact order:
26, 18, 2 + 1, 8 + 1, 31 = 26, 18, 3, 9, 31
(5) Alice checks whether the 4th number in the sequence is congruent to x mod p. Since 9 ≠ 39 (mod 31), then i > j.
(6) Alice tells Bob.

This protocol can be used to create far more complicated protocols. A group of people can conduct a secret auction over a computer network. They arrange themselves in a logical circle, and through individual pairwise comparisons, determine which is offering the highest price. In order to prevent people from changing their bids in mid-auction, some sort of bit-commitment protocol could also be used. If the auction is a Dutch auction, then the highest bidder gets the item for his highest price. If it is an English auction, then he gets the item for the second-highest price. (This can be determined by a second round of pairwise comparisons.) Similar ideas have applications in bargaining, negotiations, and arbitration.

23.15 Probabilistic Encryption

The notion of probabilistic encryption was invented by Shafi Goldwasser and Silvio Micali [624]. Although its theory makes it the most secure cryptosystem invented, its early implementation was impractical [625]. More recent implementations have changed that.

The point behind probabilistic encryption is to eliminate any information leaked with public-key cryptography. Because a cryptanalyst can always encrypt random messages with a public key, he can get some information. Assuming he has ciphertext C =EK(M) and is trying to recover plaintext message M, he can pick a random message M' and encrypt it: C' =EK(M'). If C'=C, then he guessed the correct plaintext. If it's wrong, he just guesses again.

Also, no partial information is leaked about the original message. With public-key cryptography, sometimes a cryptanalyst can learn things about the bits: The XOR of bits 5, 17, and 39 is 1, and so on. With probabilistic encryption, even this type of information remains hidden.

Not a whole lot of information is to be gained here, but there are potential problems with allowing a cryptanalyst to encrypt random messages with your public key. Some information is being leaked to the cryptanalyst every time he encrypts a message. No one really knows how much.

Probabilistic encryption tries to eliminate that leakage. The goal is that no computation on the ciphertext, or on any other trial plaintexts, can give the cryptanalyst any information about the corresponding plaintext.

In probabilistic encryption, the encrypting algorithm is probabilistic rather than deterministic. In other words, a large number of ciphertexts will decrypt to a given plaintext, and the particular ciphertext used in any given encryption is randomly chosen.

C1 = EK(M), C2 = EK(M), C3 = EK(M),..., Ci = EK(M)
M = DK(C1) = DK(C2) = DK(C3) =...= DK(Ci)

With probabilistic encryption, a cryptanalyst can no longer encrypt random plaintexts looking for the correct ciphertext. To illustrate, assume the cryptanalyst has ciphertext Ci =EK(M). Even if he guesses M correctly, when he encrypts EK(M), the result will be a completely different C: Cj. He cannot compare Ci and Cj, and so cannot know that he has guessed the message correctly.

This is amazingly cool stuff. Even if a cryptanalyst has the public encryption key, the plaintext, and the ciphertext, he cannot prove that the ciphertext is the encryption of the plaintext without the private decryption key. Even if he tries exhaustive search, he can only prove that every conceivable plaintext is a possible plaintext.

Under this scheme, the ciphertext will always be larger than the plaintext. You can't get around this; it's a result of the fact that many ciphertexts decrypt to the same plaintexts. The first probabilistic encryption scheme [625] resulted in a ciphertext so much larger than the plaintext that it was unusable.

However, Manual Blum and Goldwasser have an efficient implementation of probabilistic encryption using the Blum Blum Shub (BBS) random-bit generator described in Section 17.9 [199].

The BBS generator is based on the theory of quadratic residues. In English, there are two primes, p and q, that are congruent to 3 modulo 4. That's the private key. Their product, pq =n, is the public key. (Mind your ps and qs; the security of this scheme rests in the difficulty of factoring n.)

To encrypt a message, M, first choose some random x, relatively prime to n. Then compute

x0 = x2 mod n

Use x0 as the seed of the BBS pseudo-random-bit generator and use the output of the generator as a stream cipher. XOR M, one bit at a time, with the output of the generator. The generator spits out bits bi (the least-significant bit of xi, where xi =xi-12 mod n), so

M = M1 , M2 , M3 , ..., Mt
C = M1 ⊕ b1 , M2 ⊕ b2 , M3 ⊕ b3 , ..., Mt ⊕ bt
where t is the length of the plaintext

Append the last computed value, xt, to the end of the message and you're done.

The only way to decrypt this message is to recover x0 and then set up the same BBS generator to XOR with the ciphertext. Because the BBS generator is secure to the left, the value xt is of no use to the cryptanalyst. Only someone who knows p and q can decrypt the message.

In C, the algorithm to recover x0 from xt is:

int x0 (int p, int q, int n, int t, int xt)
{
     int a, b, u, v, w, z;
     /* we already know that gcd(p, q) == 1 */
     (void)extended_euclidian(p, q, &a, &b);
     u = modexp ((p+1)/4, t, p-1);
     v = modexp ((q+1)/4, t, q-1);
     w = modexp (xt%p, u, p);
     z = modexp (xt%q, v, q);
     return (b*q*w + a*p*z) % n;


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Once you have x0, decryption is easy. Just set up the BBS generator and XOR the output with the ciphertext.

You can make this scheme go even faster by using all the known secure bits of xi, not just the least significant bit. With this improvement, Blum-Goldwasser probabilistic encryption is faster than RSA while leaking no partial information about the plaintext. You can also prove that the difficulty of breaking this scheme is the same as the difficulty of factoring n.

On the other hand, this scheme is totally insecure against a chosen-ciphertext attack. From the least significant bits of the right quadratic residues, it is possible to calculate the square root of any quadratic residue. If you can do this, then you can factor. For details, consult [1570,1571,35,36].

23.16 Quantum Cryptography

Quantum cryptography taps the natural uncertainty of the quantum world. With it, you can create a communications channel where it is impossible to eavesdrop without disturbing the transmission. The laws of physics secure this quantum channel: even if the eavesdropper can do whatever he wants, even if the eavesdropper has unlimited computing power, even if P = NP. Charles Bennett, Gilles Brassard, Claude Crépeau, and others have expanded on this idea, describing quantum key distribution, quantum coin flipping, quantum bit commitment, quantum oblivious transfer, and quantum secure multiparty computation. Their work is described in [128,129,123,124,125,133,126,394,134,392,396]. The best overview of quantum cryptography can be found in [131]; [1651] is another good nontechnical overview. A complete bibliography of quantum cryptography is [237].

This would still be on the lunatic fringe of cryptography, but Bennett and Brassard actually went and built a working model of the thing [127,121,122]. Now we have experimental quantum cryptography.

So sit back, get yourself something to drink, and relax. I'm going to explain what this is all about.

According to quantum mechanics, particles don't actually exist in any single place. They exist in several places at once, with probabilities of being in different places if someone looks. However, it isn't until a scientist comes along and measures the particle that it "collapses" into a single location. But you can't measure every aspect (for example, position and velocity) of a particle at the same time. If you measure one of those two quantities, the very act of measuring it destroys any possibility of measuring the other quantity. The quantum world has a fundamental uncertainty and there's no way to avoid it.

That uncertainty can be used to generate a secret key. As they travel, photons vibrate in some direction; up and down, left to right, or more likely at some angle. Normal sunlight is unpolarized; the photons vibrate every which way. When a large group of photons vibrate in the same direction they are polarized. Polarization filters allow only photons that are polarized in a certain direction through; the rest are blocked. For example, a horizontal polarization filter only allows horizontally polarized photons through. Turn that filter 90 degrees, and only vertically polarized photons can come through.

Let's say you have a pulse of horizontally polarized photons. If they try to pass through a horizontally polarized filter, they all get through. Slowly turn that filter 90 degrees; the number of photons getting through gets smaller and smaller, until none get through. This is counterintuitive. You'd think that turning the filter just a little will block all the photons, since the photons are horizontally polarized. But in quantum mechanics, each particle has a probability of suddenly switching its polarization to match the filter. If the angle is a little bit off, it has a high probability. If the angle is 90 degrees off, it has zero probability. And if the angle is 45 degrees off, it has a 50 percent probability of passing through the filter.

Polarization can be measured in any basis: two directions at right angles. An example basis is rectilinear: horizontal and vertical. Another is diagonal: left-diagonal and right-diagonal. If a photon pulse is polarized in a given basis and you measure it in the same basis, you learn the polarization. If you measure it in the wrong basis, you get a random result. We're going to use this property to generate a secret key:

(1) Alice sends Bob a string of photon pulses. Each of the pulses is randomly polarized in one of four directions: horizontal, vertical, left-diagonal, and right-diagonal.
For example, Alice sends Bob:
| | / — — \ — | — /
(2) Bob has a polarization detector. He can set his detector to measure rectilinear polarization or he can set his detector to measure diagonal polarization. He can't do both; quantum mechanics won't let him. Measuring one destroys any possibility of measuring the other. So, he sets his detectors at random, for example:
× + + × × × + × + +

Now, when Bob sets his detector correctly, he will record the correct polarization. If he sets his detector to measure rectilinear polarization and the pulse is polarized rectilinearly, he will learn which way Alice polarized the photon. If he sets his detector to measure diagonal polarization and the pulse is polarized rectilinearly, he will get a random measurement. He won't know the difference. In this example, he might get the result:
/ | — \ / \ — / — |
(3) Bob tells Alice, over an insecure channel, what settings he used.
(4) Alice tells Bob which settings were correct. In our example, the detector was correctly set for pulses 2, 6, 7, and 9.
(5) Alice and Bob keep only those polarizations that were correctly measured. In our example, they keep:
* | * * * \ — * — *

Using a prearranged code, Alice and Bob each translate those polarization measurements into bits. For example, horizontal and left-diagonal might equal one, and vertical and right-diagonal might equal zero. In our example, they both have:
0 0 1 1

So, Alice and Bob have generated four bits. They can generate as many as they like using this system. On the average, Bob will guess the correct setting 50 percent of the time, so Alice has to send 2n photon pulses to generate n bits. They can use these bits as a secret key for a symmetric algorithm or they can guarantee perfect secrecy and generate enough bits for a one-time pad.

The really cool thing is that Eve cannot eavesdrop. Just like Bob, she has to guess which type of polarization to measure; and like Bob, half of her guesses will be wrong. Since wrong guesses change the polarization of the photons, she can't help introducing errors in the pulses as she eavesdrops. If she does, Alice and Bob will end up with different bit strings. So,Alice and Bob finish the protocol like this:

(6) Alice and Bob compare a few bits in their strings. If there are discrepancies, they know they are being bugged. If there are none, they discard the bits they used for comparison and use the rest.

Enhancements to this protocol allow Alice and Bob to use their bits even in the presence of Eve [133,134,192]. They could compare only the parity of subsets of the bits. Then, if no discrepancies are found, they only have to discard one bit of the subset. This detects eavesdropping with only a 50 percent probability, but if they do this with n different subsets Eve's probability of eavesdropping without detection is only 1 in 2n .

There's no such thing as passive eavesdropping in the quantum world. If Eve tries to recover all the bits, she will necessarily disrupt the communications.

Bennett and Brassard built a working model of quantum key distribution and have exchanged secure bits on a laser table. The latest I heard, some folks at British Telecom were sending bits over a 10-kilometer fiber-optic link [276,1245,1533]. They figure ilometers is feasible. The mind boggles.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part IV
The Real World

Chapter 24
Example Implementations

It's one thing to design protocols and algorithms, but another thing to field them in operational systems. In theory, theory and practice are the same; in practice they are different. Often ideas that look good on paper don't work in real life. Maybe the bandwidth requirements are too large; maybe the protocol is too slow. Chapter 10 discusses some of the issues related to using cryptography; this chapter gives examples of how it has been done in practice.

24.1 IBM Secret-Key Management Protocol

In the late 1970s IBM developed a complete key management system for communications and file security on a computer network, using only symmetric cryptography [515, 1027]. This protocol is less important in the actual mechanisms and more in its overall philosophy: By automating the generation, distribution, installation, storage, changing, and destruction of keys, the protocol went a long way to ensure the security of the underlying cryptographic algorithms.

This protocol provides three things: secure communications between a server and several terminals, secure file storage at the server, and secure communication among servers. The protocol doesn't really provide for direct terminal-to-terminal communication, although it can be modified to do that.

Each server on the network is attached to a cryptographic facility, which does all of the encrypting and decrypting. Each server has a Master Key, KM0, and two variants, KM1 and KM2, both of which are simple variants of KM0. These keys are used to encrypt other keys and to generate new keys. Each terminal has a Master Terminal Key, KMT, which is used to exchange keys with other terminals.

The servers store KMT, encrypted with KM1. All other keys, such as those used to encrypt files of keys (called KNF), are stored in encrypted form under KM2. The master key, KM0, is stored in some nonvolatile security module. Today that could be either a ROM key or a magnetic card, or it could be typed in by the user (probably as a text string and then key crunched). KM1 and KM2 are not stored anywhere in the system, but are computed from KM0 whenever they are needed. Session keys, for communication among servers, are generated with a pseudo-random process in the server. Keys to encrypt files for storage (KNF) are generated in the same manner.

The heart of the protocol is a tamper-resistant module, called a cryptographic facility. At both the server and the terminal, all encryption and decryption takes place within this facility. The most important keys, those used to generate the actual encryption keys, are stored in this module. These keys can never be read once they are stored. And they are tagged by use: A key dedicated for one purpose cannot accidentally be used for another. This concept of key control vectors is probably the most significant contribution of this system. Donald Davies and William Price discuss this key management protocol in detail [435].

A Variation

A variation on this scheme of master and session keys can be found in [1478]. It's built around network nodes with key notarization facilities that serve local terminals. It is designed to:

— Secure two-way communication between any two terminal users.
— Secure communications using encrypted mail.
— Provide personal file protection.
— Provide a digital signature capability.

For communication and file transfer among users, the scheme uses keys generated in the key notarization facility and sent to the users encrypted under a master key. The identities of the users are incorporated with the key, to provide evidence that the session key has been used between a particular pair of users. This key notarization feature is central to the system. Although the system does not use public-key cryptography, it has a digital-signature-like capability: A key could have only come from a particular source and could only be read at a particular destination.

24.2 MITRENET

One of the earliest implementations of public-key cryptography was the experimental system MEMO (MITRE Encrypted Mail Office). MITRE is a DoD contractor, a government think tank, and an all-around bunch of smart guys. MEMO was a secure electronic mail system for users in the MITRENET network, using public-key cryptography for key exchange and DES for file encryption.

In the MEMO system, all public keys are stored in a Public Key Distribution Center, which is a separate node on the network. They are stored in an EPROM to prevent anyone from changing them. Private keys are generated by users or by the system.

For a user to send secure messages, the system first establishes a secure communications path with the Public Key Distribution Center. The user requests a file of all public keys from the Center. If the user passes an identification test using his private key, the Center sends this list to the user's workstation. The list is encrypted using DES to ensure file integrity.

The implementation uses DES to encrypt messages. The system generates a random DES key for file encryption; the user encrypts the file with the DES key and encrypts the DES key with the recipient's public key. Both the DES-encrypted file and the public-key-encrypted key are sent to the recipient.

MEMO makes no provision for lost keys. There is some provision for integrity checking of the messages, using checksums. No authentication is built into the system.

The particular public-key implementation used for this system—Diffie-Hellman key exchange over GF(2127)—was proven insecure before the system was implemented (see Section 11.6), although it is easy to modify the system to use larger numbers. MEMO was intended mainly for experimental purposes and was never made operational on the real MITRENET system.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

24.3 ISDN

Bell-Northern Research developed a prototype secure Integrated Services Digital Network (ISDN) telephone terminal [499, 1192, 493, 500]. As a telephone, it was never developed beyond prototype. The resulting product was the Packet Data Security Overlay. The terminal uses Diffie-Hellman key exchange, RSA digital signatures, and DES data encryption; it can transmit and receive voice and data at 64 kilobits per second.

Keys

A long-term public-key/private-key key pair is embedded in the phone. The private key is stored in a tamper-resistant area of the phone. The public key serves as the identification of the phone. These keys are part of the phone itself and cannot be altered in any way.

Additionally, two other public keys are stored in the phone. One of these keys is the owner's public key. This key is used to authenticate commands from the owner and can be changed via a command signed by the owner. In this way an owner can transfer ownership of the phone to someone else.

The public key of the network is also stored in the phone. This key is used to authenticate commands from the network's key management facility and to authenticate calls from other users on the network. This key can also be changed via a signed command from the owner. This permits the owner to move his phone from one network to another.

These keys are considered long-term keys: rarely, if ever, changed. A short-term public-key/private-key key pair is also stored on the phone. These are encapsulated in a certificate signed by the key management facility. When two phones set up a call, they exchange certificates. The public key of the network authenticates these certificates.

This exchange and verification of certificates only sets up a secure call from phone to phone. To set up a secure call from person to person, the protocol has an additional piece. The owner's private key is stored on a hardware ignition key, which is inserted into the telephone by the owner. This ignition key contains the owner's private key, encrypted under a secret password known only by the owner (not by the phone, not by the network's key management facility, not by anybody). It also contains a certificate signed by the network's key management facility that contains the owner's public key and some identifying information (name, company, job title, security clearance, favorite pizza toppings, sexual preference, or whatever). This is also encrypted. To decrypt this information and enter it into the phone, the owner types his secret password on the phone's keypad. After the phone uses this information to set up calls, it is erased after the owner removes his ignition key.

The phone also stores a set of certificates from the network's key management facility. These certificates authorize particular users to use particular phones.

Calling

A call from Alice to Bob works as follows.

(1) Alice inserts her ignition key into the phone and enters her password.
(2) The phone interrogates the ignition key to determine Alice's identity and gives Alice a dial tone.
(3) The phone checks its set of certificates to ensure that Alice is authorized to use the particular phone.
(4) Alice dials the number; the phone places the call.
(5) The two telephones use a public-key cryptography key-exchange protocol to generate a unique and random session key. All subsequent protocol steps are encrypted using this key.
(6) Alice's phone transmits its certificate and user authentication.
(7) Bob's phone authenticates the signatures on both the certificate and the user authentication using the network's public key.
(8) Bob's phone initiates a challenge-and-reply sequence. It demands real-time signed responses to time-dependent challenges. (This prevents an adversary from using certificates copied from a previous exchange.) One response must be signed by Alice's phone's private key; another must be signed by Alice's private key.
(9) Bob's phone rings, unless he is already on the phone.
(10) If Bob is home, he inserts his ignition key into the phone. His phone interrogates the ignition key and checks Bob's certificate as in steps (2) and (3).
(11) Bob transmits his certificate and user authentication.
(12) Alice's phone authenticates Bob's signatures as in step (7), and initiates a challenge-and-reply sequence as in step (8).
(13) Both phones display the identity of the other user and phone on their displays.
(14) The secure conversation begins.
(15) When one party hangs up, the session key is deleted, as are the certificates Bob's phone received from Alice's phone and the certificates Alice's phone received from Bob's phone.

Each DES key is unique to each call. It exists only inside the two phones for the duration of the call and is destroyed immediately afterward. If an adversary captures one or both of the phones involved in the call, he will not be able to decrypt any previous call between the two phones.

24.4 STU-III

STU stands for "Secure Telephone Unit, " an NSA-designed secure phone. The unit is about the size and shape of a conventional telephone, and can be used as such. The phones are also tamper-resistant, enough so that they are unclassified if unkeyed. They also have a data port and can be used to secure modem traffic as well as voice [1133].

Whitfield Diffie described the STU-III in [494]:

To make a call with a STU-III, the caller first places an ordinary call to another STU-III, then inserts a key-shaped device containing a cryptographic variable and pushes a "go secure" button. After an approximately 15-second wait for cryptographic setup, each phone shows information about the identity and clearance of the other party on its display and the call can proceed.

In an unprecedented move, Walter Deeley, NSA's deputy director for communications security, announced the STU-III or Future Secure Voice System in an exclusive interview given to The New York Times [282]. The objective of the new system was primarily to provide secure voice and low-speed data communications for the U.S. Defense Department and its contractors. The interview didn't say much about how it was going to work, but gradually the word began to leak out. The new system was using public key.

The new approach to key management was reported early on [68] and one article spoke of phones being "reprogrammed once a year by secure telephone link, " a turn of phrase strongly suggestive of a certificate passing protocol, similar to that described [in Section 24.3], that minimizes the need for phones to talk to the key management center. Recent reports have been more forthcoming, speaking of a key management system called FIREFLY that [1341] "evolved from public key technology and is used to establish pair-wise traffic encryption keys." Both this description and testimony submitted to the U.S. Congress by Lee Neuwirth of Cylink [1164] suggest a combination of key exchange and certificates similar to that used in the ISDN secure phone and it is plausible that FIREFLY too is based on exponentiation.

STU-IIIs are manufactured by AT&T and GE. Somewhere between 300, 000 and 400, 000 have been fielded through 1994. A new version, the Secure Terminal Equipment (STE), will work on ISDN lines.

24.5 Kerberos

Kerberos is a trusted third-party authentication protocol designed for TCP/IP networks. A Kerberos service, sitting on the network, acts as a trusted arbitrator. Kerberos provides secure network authentication, allowing a person to access different machines on the network. Kerberos is based on symmetric cryptography (DES as implemented, but other algorithms could be used instead). Kerberos shares a different secret key with every entity on the network and knowledge of that secret key equals proof of identity.

Kerberos was originally developed at MIT for Project Athena. The Kerberos model is based on Needham-Schroeder's trusted third-party protocol (see Section 3.3) [1159]. The original version of Kerberos, Version 4, is specified in [1094, 1499]. (Versions 1 through 3 were internal development versions.) Version 5, modified from Version 4, is specified in [876, 877, 878]. The best overview of Kerberos is [1163]. Other survey articles are [1384, 1493], and two good articles on using Kerberos in the real world are [781, 782].


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The Kerberos Model

The basic Kerberos protocol was outlined in Section 3.3. In the Kerberos model, there are entities—clients and servers—sitting on the network. Clients can be users, but can also be independent software programs that need to do things: download files, send messages, access databases, access printers, obtain administrative privileges, whatever.

Kerberos keeps a database of clients and their secret keys. For a human user, the secret key is an encrypted password. Network services requiring authentication, as well as clients who wish to use these services, register their secret key with Kerberos.

Because Kerberos knows everyone's secret key, it can create messages that convince one entity of another entity's identity. Kerberos also creates session keys which are given to a client and a server (or to two clients) and no one else. A session key is used to encrypt messages between the two parties, after which it is destroyed.

Kerberos uses DES for encryption. Kerberos Version 4 provided a nonstandard mode for authentication. This mode is weak: It fails to detect certain changes to the ciphertext (see Section 9.10). Kerberos Version 5 uses CBC mode.

Figure 24.1 Kerberos authentication steps.

How Kerberos Works

This section discusses Kerberos Version 5. I will outline the differences between Version 4 and Version 5 further on. The Kerberos protocol is straightforward (see Figure 24.1). A client requests a ticket for a Ticket-Granting Service (TGS) from Kerberos. This ticket is sent to the client, encrypted in the client's secret key. To use a particular server, the client requests a ticket for that server from the TGS. Assuming everything is in order, the TGS sends the ticket back to the client. The client then presents this ticket to the server along with an authenticator. Again, if there's nothing wrong with the client's credentials, the server lets the client have access to the service.

Table 24.1 Kerberos Table of Abbreviations
c | = client
s | = server
a | = client's network address
v | = beginning and ending validity time for a ticket
t | = timestamp
Kx | = x's secret key
Kx, y | = session key for x and y
{m}Kx | = m encrypted in x's secret key
Tx, y | = x's ticket to use y
Ax, y | = authenticator from x to y

Credentials

Kerberos uses two types of credentials: tickets and authenticators. (The rest of this section uses the notation used in Kerberos documents—see Table 24.1.) A ticket is used to pass securely to the server the identity of the client for whom the ticket was issued. It also contains information that the server can use to ensure that the client using the ticket is the same client to whom the ticket was issued. An authenticator is an additional credential, presented with the ticket.

A Kerberos ticket takes this form:

Tc, s = s, {c, a, v, Kc, s}Ks

A ticket is good for a single server and a single client. It contains the client's name and network address, the server's name, a timestamp, and a session key. This information is encrypted with the server's secret key. Once the client gets this ticket, she can use it multiple times to access the server—until the ticket expires. The client cannot decrypt the ticket (she does not know the server's secret key), but she can present it to the server in its encrypted form. No one listening on the network can read or modify the ticket as it passes through the network.

A Kerberos authenticator takes this form:

Ac, s = {c, t, key}Kc, s

The client generates it every time she wishes to use a service on the server. The authenticator contains the client's name, a timestamp, and an optional additional session key, all encrypted with the session key shared between the client and the server. Unlike a ticket, it can only be used once. However, since the client can generate authenticators as needed (it knows the shared secret key), this is not a problem.

The authenticator serves two purposes. First, it contains some plaintext encrypted in the session key. This proves that it also knows the key. Just as important, the sealed plaintext includes the timestamp. An eavesdropper who records both the ticket and the authenticator can't replay them two days later.

Kerberos Version 5 Messages

Kerberos Version 5 has five messages (see Figure 24.1):

1. Client to Kerberos: | c, tgs
2. Kerberos to client: | {Kc, tgs}Kc, {Tc, tgs}Ktgs
3. Client to TGS: | {Ac, s}Kc, tgs, {Tc, tgs}Ktgs
4. TGS to client: | {Kc, s}Kc, tgs, {Tc, s}Ks
5. Client to server: | {Ac, s}Kc, s, {Tc, s}Ks

These will now be discussed in detail.

Getting an Initial Ticket

The client has one piece of information that proves her identity: her password. Obviously we don't want her to send this password over the network. The Kerberos protocol minimizes the chance that this password will be compromised, while at the same time not allowing a user to properly authenticate herself unless she knows the password.

The client sends a message containing her name and the name of her TGS server to the Kerberos authentication server. (There can be many TGS servers.) In reality, the user probably just enters her name into the system and the login program sends the request.

The Kerberos authentication server looks up the client in his database. If the client is in the database, Kerberos generates a session key to be used between her and the TGS. This is called a Ticket Granting Ticket (TGT). Kerberos encrypts that session key with the client's secret key. Then it creates a TGT for the client to authenticate herself to the TGS, and encrypts that in the TGS's secret key. The authentication server sends both of these encrypted messages back to the client.

The client now decrypts the first message and retrieves the session key. The secret key is a one-way hash of her password, so a legitimate user will have no trouble doing this. If the user were an imposter, he would not know the correct password and therefore could not decrypt the response from the Kerberos authentication server. Access would be denied and he wouldn't be able to get the ticket or the session key.

The client saves the TGT and session key and erases the password and the one-way hash. This information is erased to reduce the chance of compromise. If an adversary manages to copy the client's memory, he will only get the TGT and the session key. These are valuable pieces of information, but only during the lifetime of the TGT. After the TGT expires, they will be worthless.

The client can now prove her identity to the TGS for the lifetime of the TGT.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Getting Server Tickets

A client has to obtain a separate ticket for each service she wants to use. The TGS grants tickets for individual servers.

When a client needs a ticket that she does not already have, she sends a request to the TGS. (In reality, the program would do this automatically, and it would be invisible to the user.)

The TGS, upon receiving the request, decrypts the TGT with his secret key. Then he uses the session key included in the TGT to decrypt the authenticator. Finally, he compares the information in the authenticator with the information in the ticket, the client's network address with the address the request was sent from, and the timestamp with the current time. If everything matches, he allows the request to proceed.

Checking timestamps assumes that all machines have synchronized clocks, at least to within several minutes. If the time in the request is too far in the future or the past, the TGS treats the request as an attempt to replay a previous request. The TGS should also keep track of all live authenticators, because past requests can have timestamps that are still valid. Another request with the same ticket and timestamp as one already received can be ignored.

The TGS responds to a valid request by returning a valid ticket for the client to present to the server. The TGS also creates a new session key for the client and the server, encrypted with the session key shared by the client and the TGS. Both of these messages are then sent back to the client. The client decrypts the message and extracts the session key.

Requesting a Service

Now the client is ready to authenticate herself to the server. She creates a message very similar to the one sent to the TGS (which makes sense, since the TGS is a service).

The client creates an authenticator, consisting of her name and network address, and a timestamp, encrypted with the session key for her and the server that the TGS generated. The request consists of the ticket received from Kerberos (already encrypted with the server's secret key) and the encrypted authenticator.

The server decrypts and checks the ticket and the authenticator, as discussed previously, and also checks the client's address and the timestamp. If everything checks out, the server knows that, according to Kerberos, the client is who she says she is.

For applications that require mutual authentication, the server sends the client back a message consisting of the timestamp, encrypted with the session key. This proves that the server knew his secret key and could decrypt the ticket and therefore the authenticator.

The client and the server can encrypt future messages with the shared key, if desired. Since only they share this key, they both can assume that a recent message encrypted in that key originated with the other party.

Kerberos Version 4

The previous sections discussed Kerberos Version 5. In the messages and the construction of the tickets and authenticators, Version 4 is slightly different.

In Kerberos Version 4, the five messages looked like:

1. Client to Kerberos: | c, tgs
2. Kerberos to client: | {Kc, tgs, {Tc, tgs}Ktgs}Kc
3. Client to TGS: | {Ac, s}Kc, tgs, {Tc, tgs}Ktgs, s
4. TGS to client: | {Kc, s, {Tc, s}Ks}Kc, tgs
5. Client to server: | {Ac, s}Kc, s, {Tc, s}Ks
 | Tc, s = {s, c, a, v, l, Kc, s}Ks
 | Ac, s = {c, a, t}Kc, s

Messages 1, 3, and 5 are identical. The double encryption of the ticket in steps 2 and 4 has been removed in Version 5. The Version 5 ticket adds the possibility of multiple addresses, and it replaces a "lifetime" field, l, with a beginning and ending time. The Version 5 authenticator adds the option of including an additional key.

Security of Kerberos

Steve Bellovin and Michael Merritt discussed several potential security vulnerabilities of Kerberos [108]. Although this paper was written about the Version 4 protocols, many of their comments also apply to Version 5.

It may be possible to cache and replay old authenticators. Although timestamps are supposed to prevent this, replays can be done during the lifetime of the ticket. Servers are supposed to store all valid tickets to prevent replays, but this is not always possible. And ticket lifetimes can be long; eight hours is typical.

Authenticators rely on the fact that all the clocks in the network are more or less synchronized. If a host can be fooled about the correct time, then an old authenticator can be replayed without any problem. Most network time protocols are insecure, so this can be a serious problem.

Kerberos is also vulnerable to password-guessing attacks. An intruder can collect tickets and then try to decrypt them. Remember that the average user doesn't usually choose good passwords. If Mallory collects enough tickets, his chances of recovering a password are good.

Perhaps the most serious attack involves malicious software. The Kerberos protocols rely on the fact that the Kerberos software is trustworthy. There's nothing to stop Mallory from surreptitiously replacing all client Kerberos software with a version that, in addition to completing the Kerberos protocols, records passwords. This is a problem with any cryptographic software package on an insecure computer, but the widespread use of Kerberos in these environments makes it a particularly tempting target.

Enhancements to Kerberos are in the works, including an implementation of public-key cryptography and a smart-card interface for key management.

Licenses

Kerberos is not in the public domain, but MIT's code is freely available. Actually implementing it into a working UNIX environment is another story. Several companies sell versions of Kerberos, but you can get a good version free from Cygnus Support, 814 University Ave., Palo Alto, CA, 94301; (415) 322-3811; fax: (415) 322-3270.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

24.6 KryptoKnight

KryptoKnight (Kryptonite—get it?) is an authentication and key distribution system designed by IBM. It is a secret-key protocol and uses either DES in CBC mode (see Section 9.3) or a modified version of MD5 (see Section 18.5).

KryptoKnight supports four security services:

— User authentication (called single sign-on)
— Two-party authentication
— Key distribution
— Authentication of data origin and content

From a user's perspective, KryptoKnight is similar to Kerberos. Some differences are:

— KryptoKnight uses a hash function for authentication and encrypting tickets.
— KryptoKnight does not rely on synchronized clocks; it uses nonces for challenges (see Section 3.3).
— If Alice wishes to communicate with Bob, KryptoKnight has the option of allowing Alice to send a message to Bob and then for Bob to initiate the key exchange protocol.

KryptoKnight has tickets and authenticators, just like Kerberos. It has TGSs, but KryptoKnight calls them authentication servers. KryptoKnight's designers spent considerable effort minimizing the number of messages, lengths of messages, and amount of encryption. For further information on KryptoKnight, read [1110, 173, 174, 175].

24.7 SESAME

SESAME stands for Secure European System for Applications in a Multivendor Environment. It's a European Community security project, 50 percent funded by RACE (see Section 25.7), whose primary objective is producing technology for user authentication with distributed access control. Think of it as kind of a European version of Kerberos. It's a two-part project: Stage one is a basic prototype of the architecture, and stage two is a set of commercial projects. The three companies with the greatest hand in development are ICL in the United Kingdom, Siemens in Germany, and Bull in France.

SESAME is an authentication and key-exchange system [361, 1248, 797, 1043]. It uses the Needham-Schroeder protocol, with public-key cryptography to communicate between different security domains. The system is seriously flawed in several respects. Instead of using a real encryption algorithm, they use XOR with a 64-bit key size. Even worse, they use XOR in CBC mode, which leaves half the plaintext unencrypted. In their defense, they planned on using DES until the French government complained; they validated the code with DES but then removed it, and expect people to add it back. I am unimpressed nonetheless.

Authentication in SESAME is a function on the first block of a message, not on the entire message. This has the effect of authenticating "Dear Sir" and not the body of a letter. Key generation consists of two calls to the UNIX rand function, which isn't very random. SESAME uses crc32 and MD5 as one-way hash functions. And of course, SESAME is vulnerable to Kerberos-like password-guessing.

24.8 IBM Common Cryptographic Architecture

The Common Cryptographic Architecture (CCA) was designed and developed by IBM to provide cryptographic primitives for confidentiality, integrity, key management, and personal identification number (PIN) processing [751, 784, 1025, 1026, 940,752]. Keys are managed by control vectors (CVs) (see Section 8.5). Every key has a CV XORed with it and is never separated from the vector unless inside secure hardware. The CV is a data structure providing an intuitive understanding of the privileges associated with a particular key.

The individual bits of the CV are defined to have specific meanings for using and handling each key managed by CCA. The CV is carried with the encrypted key in data structures called key tokens. Internal key tokens are used locally and contain keys encrypted under the local master key (MK). External key tokens are used to export and import encrypted keys between systems. Keys in external key tokens are encrypted under key-encrypting keys (KEK). The KEKs are managed in internal key tokens. Keys are separated according to their permitted uses.

Key length is also specified and enforced using bits in the CV. Single length keys are 56 bits and are used for such functions as privacy and message authentication. Double length keys are 112 bits and are used for key management, PIN functions, and other special uses. Keys can be required to be DOUBLE-ONLY in which both the left and right halves of the key must be different, DOUBLE in which the halves are permitted to be equal by chance, SINGLE-REPLICATED in which the left and right halves are equal, or SINGLE which contains only 56 bits. The CCA functions specify hardware enforcement of certain key types to be used for some operations.

The CV is checked in a secure hardware processor: It must conform to the permitted CCA rules for each CCA function. If the CV successfully passes the test requirements, a variant of the KEK or MK is obtained by the XOR of the KEK or MK with the CV, and the plaintext target key is recovered for use internally with the CCA function. When new keys are generated, the CV specifies the uses of the generated key. Those combinations of key types that could be used in attacking the system are not generated or imported into a CCA-compliant system.

CCA uses a combination of public-key cryptography and secret-key cryptography for key distribution. The KDC shares a secret master key with each user and encrypts session keys using that master key. Master keys are distributed using public-key cryptography.

The system's designers chose this hybrid approach for two reasons. The first is performance. Public-key cryptography is computationally intensive; if session keys are distributed using public-key cryptography, the system might bog down. The second is backwards compatibility; this system can be overlaid on existing secret-key schemes with minimal disruption.

CCA systems are designed to be interoperable. For systems that are non-CCA compliant, a Control Vector Translate (CVXLT) function permits keys to be passed between the two implementations. Initialization of the CVXLT function requires dual control. Two individuals must set up the required translation tables independently. Such dual control provides a high degree of assurance concerning the integrity and pedigree of any keys introduced into the system.

A key of type DATA is provided for compatibility with other systems. A DATA key is stored with a CV that identifies the key as a DATA key. DATA keys can have broad uses and as such must be regarded with suspicion and used with care. DATA keys may not be used for any key management functions.

The Commercial Data Masking Facility (CDMF) provides an exportable version of CCA. It has a special feature that reduces DES keys to an effective 40 bits for export (see Section 15.5) [785].

24.9 ISO Authentication Framework

Public-key cryptography has been recommended for use with the ISO authentication framework, also known as the X.509 protocols [304]. This framework provides for authentication across networks. Although no particular algorithms are specified for either security or authentication, the specification recommends RSA. There are provisions, however, for multiple algorithms and hash functions. X.509 was initially issued in 1988. After public review and comment, it was revised in 1993 to correct some security problems [1100, 750].

Figure 24.2 An X.509 certificate.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Certificates

The most important part of X.509 is its structure for public-key certificates. Each user has a distinct name. A trusted Certification Authority (CA) assigns a unique name to each user and issues a signed certificate containing the name and the user's public key. Figure 24.2 shows an X.509 certificate [304].

The version field identifies the certificate format. The serial number is unique within the CA. The next field identifies the algorithm used to sign the certificate, together with any necessary parameters. Issuer is the name of the CA. The period of validity is a pair of dates; the certificate is valid during the time period between the two. Subject is the name of the user. The subject's public key information includes the algorithm name, any necessary parameters, and the public key. The last field is the CA's signature.

If Alice wants to communicate with Bob, she first gets his certificate from a database. Then she verifies its authenticity. If both share the same CA, this is easy. Alice simply verifies the CA's signature on Bob's certificate.

If they use different CAs, it's more complicated. Think of a tree structure, with different CAs certifying other CAs and users. On the top is one master CA. Each CA has a certificate signed by the CA above it, and by the CAs below it. Alice uses these certificates to verify Bob's certificate.

Figure 24.3 illustrates this. Alice's certificate is certified by CAA ; Bob's is certified by CAB. Alice knows CAA's public key. CAC has a certificate signed by CAA, so Alice can verify that. CAD has a certificate signed by CAC. CAB has a certificate signed by CAD. And Bob's certificate is signed by CAB. By moving up the certification tree to a common point, in this case CAD, and then down to Bob, Alice can verify Bob's certificate.

Figure 24.3 Sample certification hierarchy.

Certificates can be stored on databases around the network. Users can send them to each other. When a certificate expires, it should be removed from any public directories. The issuing CA, however, should maintain a copy of the certificate. Should a dispute arise later, it will be required.

Certificates can also be revoked, either because the user's key has been compromised, the CA's key has been compromised, or because the CA no longer wants to certify the user. Each CA must maintain a list of all revoked but not expired certificates. When Alice receives a new certificate, she should check to see if it has been revoked. She can check a database of revoked keys on the network, but more likely she will check a locally cached list of revoked certificates. There are certainly possible abuses to this system; key revocation is probably its weakest part.

Authentication Protocols

Alice wants to communicate with Bob. First she goes to a database and obtains what is called a certification path from Alice to Bob, and Bob's public key. At this point Alice can initiate either a one-way, two-way, or three-way authentication protocol.

The one-way protocol is a single communication from Alice to Bob. It establishes the identities of both Alice and Bob and the integrity of any information communicated by Alice to Bob. It also prevents any replay attacks on the communication.

The two-way protocol adds a reply from Bob. It establishes that Bob, and not an imposter, sent the reply. It also establishes the secrecy of both communications and prevents replay attacks.

Both the one-way and two-way protocols use timestamps. A three-way protocol adds another message from Alice to Bob and obviates the need for timestamps (and therefore authenticated time).

The one-way protocol is:

(1) Alice generates a random number, RA.
(2) Alice constructs a message, M = (TA, RA, IB, d), where TA is Alice's timestamp, IB is Bob's identity, and d is an arbitrary piece of data. The data may be encrypted with Bob's public key, EB, for security.
(3) Alice sends (CA, DA (M)) to Bob. (CA is Alice's certificate; DA is Alice's private key.)
(4) Bob verifies CA and obtains EA. He makes sure these keys have not expired. (EA is Alice's public key.)
(5) Bob uses EA to decrypt DA(M). This verifies both Alice's signature and the integrity of the signed information.
(6) Bob checks the IB in M for accuracy.
(7) Bob checks the TA in M and confirms that the message is current.
(8) As an option, Bob can check RA in M against a database of old random numbers to ensure the message is not an old one being replayed.

The two-way protocol consists of the one-way protocol and then a similar one-way protocol from Bob to Alice. After executing steps (1) through (8) of the one-way protocol, the two-way protocol continues with:

(9) Bob generates another random number, RB.
(10) Bob constructs a message M' = (TB, RB, IA, RA, d), where TB is Bob's timestamp, IA is the identity of Alice and d is arbitrary data. The data may be encrypted with Alice's public key, EA, for security. RA is the random number Alice generated in step (1).
(11) Bob sends DB(M') to Alice.
(12) Alice uses EB to decrypt DB(M'). This verifies both Bob's signature and the integrity of the signed information.
(13) Alice checks the IA in M' for accuracy.
(14) Alice checks the TB in M' and confirms that the message is current.
(15) As an option, Alice can check the RB in M' to ensure the message is not an old one being replayed.

The three-way protocol accomplishes the same thing as the two-way protocol, but without timestamps. Steps (1) through (15) are identical to the two-way protocol, with TA = TB = 0.

(16) Alice checks the received version of RA against the RA she sent to Bob in step (3).
(17) Alice sends DA(RB) to Bob.
(18) Bob uses EA to decrypt DA(RB). This verifies both Alice's signature and the integrity of the signed information.
(19) Bob checks the received version of RB against the RB he sent to Alice in step (10).

24.10 Privacy-Enhanced Mail (PEM)

PEM is the Internet Privacy-Enhanced Mail standard, adopted by the Internet Architecture Board (IAB) to provide secure electronic mail over the Internet. It was initially designed by the Internet Research Task Force (IRTF) Privacy and Security Research Group (PSRG), and then handed over to the Internet Engineering Task Force (IETF) PEM Working Group. The PEM protocols provide for encryption, authentication, message integrity, and key management.

The complete PEM protocols were initially detailed in a series of RFCs (Requests for Comment) in [977] and then revised in [978]. The third iteration of the protocols [979, 827, 980] is summarized in [177, 178]. The protocols were modified and improved, and the final protocols are detailed in another series of RFCs [981, 825, 76, 802]. Another paper by Matthew Bishop [179] details the changes. Reports of attempts to implement PEM include [602, 1505, 1522, 74, 351, 1366, 1367]. See also [1394].

PEM is an inclusive standard. The PEM procedures and protocols are intended to be compatible with a wide range of key-management approaches, including both symmetric and public-key schemes to encrypt data-encrypting keys. Symmetric cryptography is used for message-text encryption. Cryptographic hash algorithms are used for message integrity. Other documents support key-management mechanisms using public-key certificates; algorithms, modes, and associated identifiers; and paper and electronic format details and procedures for the key-management infrastructure to support these services.

PEM supports only certain algorithms, but allows for different suites of algorithms to be specified later. Messages are encrypted with DES in CBC mode. Authentication, provided by something called a Message Integrity Check (MIC), uses either MD2 or MD5. Symmetric key management can use either DES in ECB mode or triple-DES using two keys (called EDE mode). PEM also supports public-key certificates for key management, using the RSA algorithm (key length up to 1024 bits) and the X.509 standard for certificate structure.

PEM provides three privacy-enhancement services: confidentiality, authentication, and message integrity. No special processing requirements are imposed on the electronic mail system. PEM can be incorporated selectively, by site or by user, without affecting the rest of the network.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

PEM Documents

The specifications for PEM come from four documents:

— RFC 1421: Part I, Message Encryption and Authentication Procedures. This document defines message encryption and authentication procedures in order to provide privacy-enhanced mail services for electronic mail transfer on the Internet.
— RFC 1422: Part II, Certificate-Based Key Management. This document defines a supporting key management architecture and infrastructure, based on public-key certificate techniques to provide keying information to message originators and recipients.
— RFC 1423: Part III, Algorithms, Modes, and Identifiers. This document provides definitions, formats, references, and citations for cryptographic algorithms, usage modes, and associated identifiers and parameters.
— RFC 1424: Part IV, Key Certification and Related Services. This document describes three types of service in support of PEM: key certification, certificate revocation list (CRL) storage, and CRL retrieval.

Certificates

PEM is compatible with the authentication framework described in [304]; see also [826]. PEM is a superset of X.509; it establishes procedures and conventions for a key-management infrastructure for use with PEM and with other protocols (from both the TCP/IP and OSI suites) in the future.

The key-management infrastructure establishes a single root for all Internet certification. The Internet Policy Registration Authority (IPRA) establishes global policies that apply to all certification under this hierarchy. Beneath the IPRA root are Policy Certification Authorities (PCAs), each of which establishes and publishes its policies for registering users or organizations. Each PCA is certified by the IPRA. Below PCAs, CAs certify users and subordinate organizational entities (such as departments, offices, subsidiaries). Initially, the majority of users are expected to be registered with some organization.

Some PCAs are expected to provide certification for users who wish to register independent of any organization. For users who wish anonymity while taking advantage of PEM privacy facilities, one or more PCAs are expected to be established with policies that allow for registration of users who do not wish to disclose their identities.

PEM Messages

PEM's heart is its message format. Figure 24.4 shows an encrypted message using symmetric key management, Figure 24.5 shows an authenticated and encrypted message using public-key key management, and Figure 24.6 shows an authenticated (but unencrypted) message using public-key key management.

The first field is "Proc-Type, " and identifies the type of processing performed on the message. There are three possible types of messages. The "ENCRYPTED" specifier says that the message is encrypted and signed. The "MIC-ONLY" and "MIC-CLEAR" specifiers would indicate that the message is signed, but not encrypted. MIC-CLEAR messages are not encoded and can be read using non-PEM software. MIC-ONLY messages need PEM software to transform them to a human-readable form. A PEM message is always signed; it is optionally encrypted.

   -----BEGIN PRIVACY-ENHANCED MESSAGE-----
   Proc-Type: 4,ENCRYPTED
   Content-Domain: RFC822
   DEK-Info: DES-CBC,F8143EDE5960C597
   Originator-ID-Symmetric: schneier@counterpane.com,,
   Recipient-ID-Symmetric: schneier@chinet.com,ptf-kmc,3
   Key-Info:
DES-ECB,RSA-MD2,9FD3AAD2F2691B9A,B70665BB9BF7CBCDA60195DB94F727D3
   Recipient-ID-Symmetric: pem-dev@tis.com,ptf-kmc,4
   Key-Info:
DES-ECB,RSA-MD2,161A3F75DC82EF26,E2EF532C65CBCFF79F83A2658132DB47
LLrHB0eJzyhP+/fSStdW8okeEnv47jxe7SJ/iN72ohNcUk2jHEUSoH1nvNSIWL9M
8tEjmF/zxB+bATMtPjCUWbz8Lr9wloXIkjHUlBLpvXR0UrUzYbkNpk0agV2IzUpk
J6UiRRGcDSvzrsoK+oNvqu6z7Xs5Xfz5rDqUcMlK1Z6720dcBWGGsDLpTpSCnpot
   dXd/H5LMDWnonNvPCwQUHt==
   -----END PRIVACY-ENHANCED MESSAGE-----

Figure 24.4 Example of an encapsulated message (symmetric case).

   -----BEGIN PRIVACY-ENHANCED MESSAGE-----
   Proc-Type: 4,ENCRYPTED
   Content-Domain: RFC822
   DEK-Info: DES-CBC,BFF968AA74691AC1
   Originator-Certificate:
MIIBlTCCAScCAWUwDQYJKoZIhvcNAQECBQAwUTELMAkGA1UEBhMCVVMxIDAeBgNV
BAoTF1JTQSBEYXRhIFNlY3VyaXR5LCBJbmMuMQ8wDQYDVQQLEwZCZXRhIDExDzAN
BgNVBAsTBk5PVEFSWTAeFw05MTA5MDQxODM4MTdaFw05MzA5MDMxODM4MTZaMEUx
CzAJBgNVBAYTAlVTMSAwHgYDVQQKExdSU0EgRGF0YSBTZWN1cml0eSwgSW5jLjEU
MBIGA1UEAxMLVGVzdCBVc2VyIDEwWTAKBgRVCAEBAgICAANLADBIAkEAwHZHl7i+
yJcqDtjJCowzTdBJrdAiLAnSC+CnnjOJELyuQiBgkGrgIh3j8/x0fM+YrsyF1u3F
LZPVtzlndhYFJQIDAQABMA0GCSqGSIb3DQEBAgUAA1kACKr0PqphJYw1j+YPtcIq
iWlFPuN5jJ79Khfg7ASFxskYkEMjRNZV/HZDZQEhtVaU7Jxfzs2wfX5byMp2X3U/
    5XUXGx7qusDgHQGs7Jk9W8CW1fuSWUgN4w==
   Key-Info: RSA,
I3rRIGXUGWAF8js5wCzRTkdhO34PTHdRZY9Tuvm03M+NM7fx6qc5udixps2Lng0+
    wGrtiUm/ovtKdinz6ZQ/aQ==
   Issuer-Certificate:
MIIB3DCCAUgCAQowDQYJKoZIhvcNAQECBQAwTzELMAkGA1UEBhMCVVMxIDAeBgNV
BAoTF1JTQSBEYXRhIFNlY3VyaXR5LCBJbmMuMQ8wDQYDVQQLEwZCZXRhIDExDTAL
BgNVBAsTBFRMQ0EwHhcNOTEwOTAxMDgwMDAwWhcNOTIwOTAxMDc1OTU5WjBRMQsw
CQYDVQQGEwJVUzEgMB4GA1UEChMXUlNBIERhdGEgU2VjdXJpdHksIEluYy4xDzAN
BgNVBAsTBkJldGEgMTEPMA0GA1UECxMGTk9UQVJZMHAwCgYEVQgBAQICArwDYgAw
XwJYCsnp6lQCxYykNlODwutF/jMJ3kL+3PjYyHOwk+/9rLg6X65B/LD4bJHtO5XW
cqAz/7R7XhjYCm0PcqbdzoACZtIlETrKrcJiDYoP+DkZ8k1gCk7hQHpbIwIDAQAB
MA0GCSqGSIb3DQEBAgUAA38AAICPv4f9Gx/tY4+p+4DB7MV+tKZnvBoy8zgoMGOx
dD2jMZ/3HsyWKWgSF0eH/AJB3qr9zosG47pyMnTf3aSy2nBO7CMxpUWRBcXUpE+x
EREZd9++32ofGBIXaialnOgVUn0OzSYgugiQ077nJLDUj0hQehCizEs5wUJ35a5h
   MIC-Info: RSA-MD5,RSA,
UdFJR8u/TIGhfH65ieewe21OW4tooa3vZCvVNGBZirf/7nrgzWDABz8w9NsXSexv
    AjRFbHoNPzBuxwmOAFeA0HJszL4yBvhG
   Recipient-ID-Asymmetric:
MFExCzAJBgNVBAYTAlVTMSAwHgYDVQQKExdSU0EgRGF0YSBTZWN1cml0eSwgSW5j
    LjEPMA0GA1UECxMGQmV0YSAxMQ8wDQYDVQQLEwZOT1RBUlk=,
    66
   Key-Info: RSA,
O6BS1ww9CTyHPtS3bMLD+L0hejdvX6Qv1HK2ds2sQPEaXhX8EhvVphHYTjwekdWv
    7x0Z3Jx2vTAhOYHMcqqCjA==
qeWlj/YJ2Uf5ng9yznPbtD0mYloSwIuV9FRYx+gzY+8iXd/NQrXHfi6/MhPfPF3d
   jIqCJAxvld2xgqQimUzoS1a4r7kQQ5c/Iua4LqKeq3ciFzEv/MbZhA==
   -----END PRIVACY-ENHANCED MESSAGE-----

Figure 24.5 Example of an encapsulated encrypted message (asymmetric case).

The next field, "Content-Domain, " specifies the type of mail message. It has nothing to do with security. The "DEK-Info" field gives information on the Data Exchange Key (DEK), the encryption algorithm used to encrypt the text, and any parameters associated with the encryption algorithm. Only DES in CBC mode is currently specified, or "DES-CBC." The second subfield specifies the IV. Other algorithms may be specified by PEM in the future; their use will be noted in DEK-Info and in other fields that identify algorithms.

For messages with symmetric key management (see Figure 24.4), the next field is "Originator-ID-Symmetric" with three subfields. The first subfield identifies the sender by a unique electronic mail address. The second subfield is optional and identifies the authority that issued the interchange key. The third is an optional Version/Expiration subfield.

Continuing with the symmetric key-management case, each recipient has two fields: "Recipient-ID-Symmetric" and "Key-Info." The "Recipient-ID-Symmetric" field has three subfields; these identify the receiver in the same way that "Originator-ID-Symmetric" identified the sender.

The "Key-Info" field specifies the key-management parameters. This field has four subfields. The first subfield gives the algorithm used to encrypt the DEK. Since the key management in this message is symmetric, the sender and receiver have to share a common key. This is called the Interchange Key (IK), which is used to encrypt the DEK. The DEK can be either encrypted using DES in ECB (denoted by "DES-ECB") or triple-DES (which would be denoted "DES-EDE"). The second subfield specifies the MIC algorithm. It can be either MD2 (denoted by "RSA-MD2") or MD5 (which would be denoted "RSA-MD5"). The third subfield, the DEK, and the fourth field, the MIC, are both encrypted with the IK.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Figures 24.5 and 24.6 show messages with public-key key management (called "asymmetric" in PEM nomenclature). The headers are different. In ENCRYPTED messages, after the "DEK-Info" field comes the "Originator-Certificate" field. The certificate follows the X.509 standard (see Section 24.9). The next field is "Key-Info" with two subfields. The first subfield specifies the public-key algorithm used to encrypt the DEK; currently only RSA is supported. The next subfield is the DEK, encrypted in the originator's public key. This is an optional field, intended to permit the originator to decrypt his own message in the event that it is returned by the mail system. The next field "Issuer-Certificate, " is the certificate of whomever signed the Originator-Certificate.

Continuing with the asymmetric key-management case, the next field is "MIC-Info." The first subfield gives the algorithm under which the MIC was computed. The second subfield shows the algorithm under which the MIC was signed. The third subfield consists of the MIC, signed by the sender's private key.

   -----BEGIN PRIVACY-ENHANCED MESSAGE-----
   Proc-Type: 4,MIC-ONLY
   Content-Domain: RFC822
   Originator-Certificate:
MIIBlTCCAScCAWUwDQYJKoZIhvcNAQECBQAwUTELMAkGA1UEBhMCVVMxIDAeBgNV
BAoTF1JTQSBEYXRhIFNlY3VyaXR5LCBJbmMuMQ8wDQYDVQQLEwZCZXRhIDExDzAN
BgNVBAsTBk5PVEFSWTAeFw05MTA5MDQxODM4MTdaFw05MzA5MDMxODM4MTZaMEUx
CzAJBgNVBAYTAlVTMSAwHgYDVQQKExdSU0EgRGF0YSBTZWN1cml0eSwgSW5jLjEU
MBIGA1UEAxMLVGVzdCBVc2VyIDEwWTAKBgRVCAEBAgICAANLADBIAkEAwHZHl7i+
yJcqDtjJCowzTdBJrdAiLAnSC+CnnjOJELyuQiBgkGrgIh3j8/x0fM+YrsyF1u3F
LZPVtzlndhYFJQIDAQABMA0GCSqGSIb3DQEBAgUAA1kACKr0PqphJYw1j+YPtcIq
iWlFPuN5jJ79Khfg7ASFxskYkEMjRNZV/HZDZQEhtVaU7Jxfzs2wfX5byMp2X3U/
    5XUXGx7qusDgHQGs7Jk9W8CW1fuSWUgN4w==
   Issuer-Certificate:
MIIB3DCCAUgCAQowDQYJKoZIhvcNAQECBQAwTzELMAkGA1UEBhMCVVMxIDAeBgNV
BAoTF1JTQSBEYXRhIFNlY3VyaXR5LCBJbmMuMQ8wDQYDVQQLEwZCZXRhIDExDTAL
BgNVBAsTBFRMQ0EwHhcNOTEwOTAxMDgwMDAwWhcNOTIwOTAxMDc1OTU5WjBRMQsw
CQYDVQQGEwJVUzEgMB4GA1UEChMXUlNBIERhdGEgU2VjdXJpdHksIEluYy4xDzAN
BgNVBAsTBkJldGEgMTEPMA0GA1UECxMGTk9UQVJZMHAwCgYEVQgBAQICArwDYgAw
XwJYCsnp6lQCxYykNlODwutF/jMJ3kL+3PjYyHOwk+/9rLg6X65B/LD4bJHtO5XW
cqAz/7R7XhjYCm0PcqbdzoACZtIlETrKrcJiDYoP+DkZ8k1gCk7hQHpbIwIDAQAB
MA0GCSqGSIb3DQEBAgUAA38AAICPv4f9Gx/tY4+p+4DB7MV+tKZnvBoy8zgoMGOx
dD2jMZ/3HsyWKWgSF0eH/AJB3qr9zosG47pyMnTf3aSy2nBO7CMxpUWRBcXUpE+x
EREZd9++32ofGBIXaialnOgVUn0OzSYgugiQ077nJLDUj0hQehCizEs5wUJ35a5h
   MIC-Info: RSA-MD5,RSA,
jV2OfH+nnXHU8bnL8kPAad/mSQlTDZlbVuxvZAOVRZ5q5+Ejl5bQvqNeqOUNQjr6
    EtE7K2QDeVMCyXsdJlA8fA==
LSBBIG1lc3NhZ2UgZm9yIHVzZSBpbiB0ZXN0aW5nLg0KLSBGb2xsb3dpbmcgaXMg
   YSBibGFuayBsaW5lOg0KDQpUaGlzIGlzIHRoZSBlbmQuDQo=
   -----END PRIVACY-ENHANCED MESSAGE-----

Figure 24.6 Example of an encapsulated MIC-ONLY message (asymmetric case).

Still continuing with asymmetric key management, the next fields deal with the recipients. There are two fields for each recipient: "Recipient-ID-Asymmetric" and "Key-Info." The "Recipient-ID-Asymmetric" field has two subfields. The first identifies the authority that issued the receiver's public key; the second is an optional Version/Expiration subfield. The "Key-Info" field specifies the key management parameters: The first subfield identifies the algorithm used to encrypt the message and the second subfield is the DEK encrypted with the receiver's public key.

Security of PEM

RSA keys in PEM can range from 508 bits to 1024 bits. This should be long enough for anyone's security needs. A more likely attack would be against the key-management protocols. Mallory could steal your private key—don't write it down anywhere—or attempt to fool you into accepting a bogus public key. The key certification provisions of PEM make this unlikely if everyone follows proper procedures, but people have been known to be sloppy.

A more insidious attack would be for Mallory to modify the PEM implementation running on your system. This modified implementation could surreptitiously send Mallory all of your mail, encrypted with his public key. It could even send him a copy of your private key. If the modified implementation works well, you will never know what is happening.

There's no real way to prevent this kind of attack. You could use a one-way hash function and fingerprint the PEM code. Then, each time you run it, you could check the fingerprint for modification. But Mallory could modify the fingerprint code at the same time he modifies the PEM code. You could fingerprint the fingerprint code, but Mallory could modify that as well. If Mallory can get access to your machine, he can subvert the security of PEM.

The moral is that you can never really trust a piece of software if you cannot trust the hardware it is running on. For most people, this kind of paranoia is unwarranted. For some, it is very real.

TIS/PEM

Trusted Information Systems, partially supported by the U.S. government Advanced Research Projects Agency, has designed and implemented a reference implementation of PEM (TIS/PEM). Developed for UNIX-based platforms, it has also been ported to VMS, DOS, and Windows.

Although the PEM specifications indicate a single certification hierarchy for use by the Internet, TIS/PEM supports the existence of multiple certification hierarchies. Sites may specify a set of certificates that are to be considered valid, including all certificates issued by them. A site need not join the Internet hierarchy in order to use TIS/PEM.

TIS/PEM is currently available to all U.S. and Canadian organizations and citizens upon request. It will be distributed in source code form. Interested parties should contact: Privacy-Enhanced Mail, Trusted Information Systems, Inc., 3060 Washington Road (Rte. 97), Glenwood, MD 21738; (301) 854-6889; fax: (301) 854-5363; Internet: pem-info@tis.com.

RIPEM

RIPEM is a program, written by Mark Riordan, that implements the PEM protocols. Although technically not public domain, the program is publicly available and can be used royalty-free for personal, noncommercial applications. A license for its use is included with the documentation.

The code cannot be exported. Of course, U.S. government laws don't apply outside the United States, and some people have ignored the export rules. RIPEM code is available on bulletin boards worldwide. Something called RIPEM/SIG, which only does digital signatures, is exportable.

At this writing, RIPEM is not a complete implementation of the PEM protocols; it does not implement certificates for authenticating keys.

Before writing RIPEM, Riordan wrote a similar program called RPEM. This was intended to be a public-domain electronic-mail encryption program. To try to avoid patent issues, Riordan used Rabin's algorithm (see Section 19.5). Public Key Partners claimed that their patents were broad enough to cover all of public-key cryptography and threatened to sue; Riordan stopped distributing the program.

RPEM isn't really used anymore. It is not compatible with RIPEM. Since RIPEM can be used with the full blessing of Public Key Partners, there is no reason to use RPEM instead.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

24.11 Message Security Protocol (MSP)

The Message Security Protocol (MSP) is the military equivalent of PEM. It was developed by the NSA in the late 1980s under the Secure Data Network System (SDNS) program. It is an X.400-compatible application-level protocol for securing electronic mail. MSP will be used for signing and encrypting messages in the Department of Defense's planned Defense Message System (DMS) network.

The Preliminary Message Security Protocol (PMSP), to be used for "unclassified but sensitive" messages, is a version of MSP adapted for use with both X.400 and TCP/IP. This protocol is also called Mosaic.

Like PEM, MSP and PMSP software applications are flexible and designed to accommodate a variety of algorithms for security functions including signing, hashing, and encryption. PSMP will work with the Capstone chip (see Section 24.17).

24.12 Pretty Good Privacy (PGP)

Pretty Good Privacy (PGP) is a freeware electronic-mail security program, originally designed by Philip Zimmermann [1652]. It uses IDEA for data encryption, RSA (with keys up to 2047 bits) for key management and digital signatures, and MD5 as a one-way hash function.

PGP's random public keys use a probabilistic primality tester, and get their initial seeds from measuring the user's keyboard latency while typing. PGP generates random IDEA keys using the method delineated in ANSI X9.17, Appendix C (see Section 8.1) [55], with IDEA as the symmetric algorithm instead of DES. PGP also encrypts the user's private key using a hashed pass phrase instead of a password.

PGP-encrypted messages have layered security. The only thing a cryptanalyst can learn about an encrypted message is who the recipient is, assuming he knows the recipient's key ID. Only after the recipient decrypts the message does he learn who signed the message, if it is signed. Contrast this approach with PEM, which leaves quite a bit of information about the sender, recipient, and message in the unencrypted header.

The most interesting aspect of PGP is its distributed approach to key management (see Section 8.12). There are no key certification authorities; PGP instead supports a "web of trust." Every user generates and distributes his own public key. Users sign each other's public keys, creating an interconnected community of PGP users.

For example, Alice might physically give her public key to Bob. Bob knows Alice, so he signs her public key. He then gives the signed key back to her and keeps a copy for himself. When Alice wants to communicate with Carol, Alice sends Carol a copy of the key Bob signed. Carol, who already has Bob's public key (she got it at some other time) and trusts Bob to certify other people's keys, verifies his signature on Alice's key and accepts it as valid. Bob has introduced Alice to Carol.

PGP does not specify a policy for establishing trust; users are free to decide who they trust and who they do not. PGP provides mechanisms for associating trust with public keys and for using trust. Each user keeps a collection of signed public keys in a file called a public-key ring. Each key in the ring has a key legitimacy field that indicates the degree to which the particular user trusts the validity of the key. The higher the trust level, the more the user believes the key is legitimate. A signature trust field measures how far the user trusts the signer to certify the public keys of other users. And finally, an owner trust field indicates the degree to which the particular user trusts the key's owner to sign other public keys; this field is set manually by the user. PGP continuously updates these fields as users supply new information.

Figure 24.7 shows how this model might look for a particular user, Alice. Alice's key is at the top, and the owner trust value is ultimate trust. Alice has signed Bob's, Carol's, Dave's, Ellen's, and Frank's keys. She trusts Bob and Carol to sign other people's public keys, and she partially trusts Dave and Ellen to sign other people's public keys. And she trusts Gail to sign other people's public keys, even though she has not signed Gail's key herself.

Two partially trusted signatures may be sufficient to certify a key. Alice believes that Kurt's key is legitimate because both Dave and Ellen have signed it. This is not automatic in PGP; Alice can set her own paranoia level.

Just because Alice believes a key to be valid, she does not have to trust it to sign other people's keys. She does not trust Frank to sign other people's public keys, even though she signed his key herself. And she does not trust Ivan's signature on Martin's key, or Kurt's signature on Nancy's key.

Owen's key doesn't fit into the web anywhere; perhaps Alice got it from a key server. PGP does not assume that the key is valid; Alice must either declare the key valid or decide to trust one of the key's signers.

Of course, nothing prevents Alice from using keys she does not trust. PGP's job is to alert Alice that the key is not trusted, not to prevent communications.

The weakest link of this whole system is key revocation: It is impossible to guarantee that no one will use a compromised key. If Alice's private key is stolen she can send out something called a key revocation certificate, but since key distribution is ad hoc and largely word of mouth there is no guarantee that it will reach everyone who has her public key on his key ring. And as Alice has to sign the key revocation certificate with her private key; if she loses the key altogether she cannot revoke it.

Figure 24.7 PGP trust model.

The current version of PGP is 2.6.2. A new version of PGP, PGP 3.0, is scheduled for release by the end of 1995. Changes in 3.0 include options for triple-DES, SHA, and other public-key algorithms, a split of the encryption and signature public-key/private-key key pairs, enhanced procedures for key revocation, improved key-ring management functions, an API for integrating PGP in other programs, and a completely rewritten code base.

PGP is available for MS-DOS, UNIX, Macintosh, Amiga, and Atari. It is free for personal, noncommercial use, and is available from many ftp sites on the Internet. To ftp PGP from MIT, telnet to net-dist.mit.edu, log in as getpgp, answer the questions, then ftp to net-dist.mit.edu and change to the directory named in the telnet session. It is also available from ftp.ox.ac.uk, ftp.dsi.unimi.it, ftp.funet.fi, ftp.demon.co.uk, Compuserve, AOL, and elsewhere. For U.S. commercial users, PGP can be bought—complete with licenses—for about $100 from a company called ViaCrypt, 9033 N 24th Ave., Phoenix, AZ, 85021; (602) 944-0773; viacrypt@acm.org. Several shareware front-ends are available to help integrate PGP into MS-DOS, Microsoft Windows, Macintosh, and UNIX.

There are several books about PGP [601, 1394, 1495]. The source code has even been published in book form [1653] in an attempt to frustrate the U.S. Department of State, which continues to maintain that source code is exportable on paper but not electronically. Assuming you trust IDEA, PGP is the closest you're likely to get to military-grade encryption.

24.13 Smart Cards

A smart card is a plastic card, the size and shape of a credit card, with an embedded computer chip. It's an old idea—the first patents were filed 20 years ago—but practical limitations made them feasible only five or so years ago. Since then they have taken off, mostly in Europe. Many countries use smart cards for pay telephones. There are also smart credit cards, smart cash cards, smart everything cards. The U.S. credit-card companies are looking at the technology, and within a few years even backwards Americans will have smart cards in their wallets.

A smart card contains a small computer (usually an 8-bit microprocessor), RAM (about a quarter kilobyte), ROM (about 6 or 8 kilobytes), and either EPROM or EEPROM (a few kilobytes). Future-generation smart cards will undoubtedly have more capacity, but some physical limitations on smart cards make expansion difficult. The card has its own operating system, programs, and data. (What it doesn't have is power; that comes when the card is plugged in to a reader.) And it is secure. In a world where you might not trust someone else's computer or telephone or whatever, you can still trust a card that you keep with you in your wallet.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Smart cards can have different cryptographic protocols and algorithms programmed into them. They might be configured as an electronic purse, and be able to spend and receive digital cash. They may be able to perform zero-knowledge authentication protocols; they may have their own encryption keys. They might be able to sign documents, or unlock applications on a computer.

Some smart cards are assumed to be tamperproof; this often protects the institution that issues the cards. A bank wouldn't want you to be able to hack their smart card to give yourself more money.

There is a lot of interest in smart cards, and a lot of information about them is available. A good survey article on the cryptography in smart cards is [672]. CARTES is a conference held in Paris every October; and CardTech is held in Washington, D.C. every April. The proceedings of two other smart-card conferences are [342, 382]. There are hundreds of smart-card patents, mostly owned by European companies. An interesting paper on possible future applications—integrity checking, audit trails, copy protection, digital cash, secure postage meters—is [1628].

24.14 Public-Key Cryptography Standards (PKCS)

The Public-Key Cryptography Standards (PKCS) are RSA Data Security, Inc.'s attempt to provide an industry standard interface for public-key cryptography. Traditionally, this sort of thing would be handled by ANSI, but, considering the current situation in cryptography politics, RSADSI figured that they had better do it on their own. Working with a variety of companies, they developed a series of standards. Some are compatible with other standards and some are not.

These are not standards in the traditional sense of the word; no standards body convened and voted on PKCS. According to its own materials, RSADSI will "retain sole decision-making authority on what each standard is" and will "publish revised standards when appropriate" [803].

Even so, there is a lot of good stuff here. If you're not sure what kind of syntax and data structures to use when programming public-key cryptography, these standards are probably as good as anything else you can come up with. And, since they're not really standards, you can tailor them to suit your needs.

Following is a short description of each PKCS (PKCS #2 and PKCS #4 have been incorporated into PKCS #1).

PKCS #1 [1345] describes a method for RSA encryption and decryption, primarily for constructing the digital signatures and digital envelopes described in PKCS #7. For digital signatures, the message is hashed and then the hash is encrypted with the private key of the signer. Both message and hash are represented together as detailed in PKCS #7. For digital envelopes (encrypted messages), the message is first encrypted with a symmetric algorithm, and then the message key is encrypted with the public key of the recipient. The encrypted message and encrypted key are represented together according to the syntax of PKCS #7. Both of these methods are compatible with PEM standards. PKCS #1 also describes a syntax, identical to the syntax in X.509 and PEM, for RSA public and private keys and three signature algorithms—MD2 and RSA, MD4 and RSA, and MD5 and RSA—for signing certificates and the like.

PKCS #3 [1346] describes a method for implementing Diffie-Hellman key exchange.

PKCS #5 [1347] describes a method for encrypting messages with a secret key derived from a password. It uses either MD2 or MD5 to derive the key from the password, and encrypts with DES in CBC mode. The method is intended primarily to encrypt private keys when transferring them from one computer system to another, but can be used to encrypt messages.

PKCS #6 [1348] describes a standard syntax for public key certificates. The syntax is a superset of an X.509 certificate, so that X.509 certificates can be extracted if necessary. Over and above the X.509 set, additional attributes extend the certification process beyond just the public key. These include other information, such as electronic mail address.

PKCS # 7 [1349] is a general syntax for data that may be encrypted or signed, such as digital envelopes or digital signatures. The syntax is recursive, so that envelopes can be nested, or someone can sign some previously encrypted data. The syntax also allows other attributes, such as timestamps, to be authenticated along with the message content. PKCS #7 is compatible with PEM so that signed and encrypted messages can be converted to PEM messages without any cryptographic operations, and vice versa. PKCS #7 can support a variety of architectures—PEM is one—for certificate-based key management.

PKCS #8 [1350] describes a syntax for private key information—including a private key and a set of attributes—and a syntax for encrypted private keys. PKCS #5 can be used to encrypt the private key information.

PKCS #9 [1351] defines selected attribute types for PKCS #6 extended certificates, PKCS #7 digitally signed messages, and PKCS #8 private-key information.

PKCS #10 [1352] describes a standard syntax for certification requests. A certification comprises a distinguished name, a public key, and (optionally) a set of attributes, collectively signed by the person requesting certification. Certification requests are sent to a certification authority, who either transforms the request into an X.509 public-key certificate or a PKCS #6 certificate.

PKCS #11 [1353], the Cryptographic Token API Standard, specifies a programming interface called "Cryptoki" for portable cryptographic devices of all kinds. Cryptoki presents a common logical model, enabling applications to perform cryptographic operations on portable devices without knowing details of the underlying technology. The standard also defines application profiles: sets of algorithms that a device may support.

PKCS #12 [1354] describes syntax for storing in software a user's public keys, protected private keys, certificates, and other related cryptographic information. The goal is to standardize on a single key file for use among a variety of applications.

These standards are comprehensive, but not exhaustive. Many things are outside their scope: the problem of naming, noncryptographic issues surrounding certification, key lengths, and conditions on various parameters. What the PKCS provide are a format for transferring data based on public-key cryptography and an infrastructure to support that transfer.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

24.15 Universal Electronic Payment System (UEPS)

The UEPS is a smart-card banking application initially developed for rural South Africa, but later adopted by all of that country's major banking groups. About 2 million cards were issued in that country by early 1995. It has also been adopted in Namibia, and is also being deployed by at least one bank in Russia.

The system provides a secure debit card suitable for regions where poor telephone service make on-line verification impossible. Both customers and merchants have cards; customers can use their cards to transfer money to merchants. Merchants can then take their cards to a telephone and deposit the money in their bank account; customers can take their cards to a telephone and have money moved onto their card. There is no intention to provide anonymity, only to prevent fraud.

Here is the communications protocol between customer Alice and merchant Bob. (Actually, Alice and Bob just plug their cards into a machine and wait for it to complete the transaction.) When Alice first gets her card, she is given a key pair, K1 and K2; the bank calculates them from her name and some secret function. Only the merchant cards have the secrets necessary to work out these customer keys.

(1) Alice sends Bob her name, A, his name, B, and a random number, RA, encrypted using DES: first with K2 and then with K1. She also sends her name in the clear.
A, EK1(EK2(A, B, RA))
(2) Bob calculates K1 and K2 from Alice's name. He decrypts the message, confirms that A and B are correct, then encrypts Alice's unencrypted second message with K2.
EK2(A, B, RA)

Bob does not send this message to Alice; 56 bits of the ciphertext become K3. Bob then sends Alice his name, her name, and another random number, RB, encrypted using DES: first with K3 and then with K1.
EK1(EK3(B, A, RB))
(3) Alice computes K3 in the same manner Bob did. She decrypts Bob's message, confirms that B and A are correct, then encrypts Bob's unencrypted message with K3.
EK3(B, A, RB)

Alice does not send this message to Bob; 56 bits of the ciphertext become K4. Alice then sends Bob her name, his name, and the digital check, C. This check contains the names of the sender and recipient, a date, a check number, an amount, and two MACs, all encrypted using DES: first with K4 and then with K1. One of the MACs can be verified by Alice's bank, and the other can only be verified by the clearing center. Alice debits her account by the correct amount.
EK1(EK4(A, B, C))
(4) Bob computes K4 in the same manner Alice did. Assuming all the names match and the check is correctly formed, he accepts it for payment.

A really clever thing about this protocol is that the encryption key for each message depends on the previous message. Each message doubles as an authenticator for all previous messages. This means that someone can't replay an old message; the receiver could never decrypt it. I am impressed with this idea and expect that it will see wider use once it becomes widely known.

Another clever thing about this protocol is that it enforces correct implementation. If the application developer doesn't implement this protocol correctly, it just won't work.

Both cards store records of every transaction. When the cards eventually go online to communicate with the bank—the merchant to deposit his money and the customer to get more money—the bank uploads these records for auditing purposes.

Tamperproof hardware prevents either participant from messing with the data; Alice cannot change the value of her card. Extensive audit trails provide data to identify and prosecute fraudulent transactions. There are universal secrets in the cards—MAC keys in the customer cards, functions to convert customer names to K1 and K2 in the merchant cards—but these are assumed to be difficult to reverse-engineer.

This scheme is not meant to be perfect, only more secure than either paper checks or traditional debit cards. The threat of fraud is not from rival militaries, but from opportunistic customers and merchants. UEPS protects against that kind of abuse.

The message exchange is an excellent example of a robust protocol: Every message names both parties, includes unique information to ensure freshness, and depends explicitly on all the messages that came before it.

24.16 Clipper

The Clipper chip (also known as the MYK-78T) is an NSA-designed, tamper-resistant VLSI chip designed for encrypting voice conversations; it is one of the two chips that implements the U.S. government's Escrowed Encryption Standard (EES) [1153]. VLSI Technologies, Inc. manufactures the chip, and Mykotronx, Inc. programs it. Initially, the Clipper chip will be available in the AT&T Model 3600 Telephone Security Device (see Section 24.18). The chip implements the Skipjack encryption algorithm (see Section 13.12), an NSA-designed classified secret-key encryption algorithm, in OFB only.

The most controversial aspect of the Clipper chip, and the entire EES, is the key-escrow protocol (see Section 4.14). Each chip has a special key, not needed for messages. This key is used to encrypt a copy of each user's message key. As part of the synchronization process, the sending Clipper chip generates and sends a Law Enforcement Access Field (LEAF) to the receiving Clipper chip. The LEAF contains a copy of the current session key, encrypted with a special key (called the unit key). This allows a government eavesdropper to recover the session key, and then recover the plaintext of the conversation.

According to the director of NIST [812]:

A "key-escrow" system is envisioned that would ensure that the "Clipper Chip" is used to protect the privacy of law-abiding Americans. Each device containing the chip will have two unique "keys, " numbers that will be needed by authorized government agencies to decode messages encoded by the device. When the device is manufactured, the two keys would be deposited separately in two "key-escrow" databases established by the attorney general. Access to these keys would be limited to government officials with legal authorization to conduct a wiretap.

The government also wants to encourage the sale of telephones with these devices abroad; no one knows what might happen to those key-escrow databases.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Politics aside, the internal structure of the LEAF is worth discussing [812, 1154, 1594, 459, 107, 462]. The LEAF is a 128-bit string containing enough information to allow law enforcement to recover the session key, KS, assuming the two escrow agencies in charge of those key-escrow databases cooperate. The LEAF contains a 32-bit unit identifier, U, unique to the Clipper chip. It also contains the current 80-bit session key encrypted with the chip's unique unit key, KU, and a 16-bit checksum, C, called an escrow identifier. This checksum is a function of the session key, the IV, and possibly other information. These three fields are encrypted with a fixed family key, KF, shared by all interoperable Clipper chips. The family key, the encryption modes used, the details of the checksum, and the exact structure of the LEAF are all secret. It probably looks something like this:

EKF(U,KU(KS, C))

KU is programmed into Clipper chips at the factory. This key is then split (see Section 3.6) and stored in two different key-escrow databases, guarded by two different escrow agencies.

For Eve to recover KS from the LEAF, she first has to decrypt the LEAF with KF and recover U. Then she has to take a court order to each escrow agency, who each return half of KU for the given U. Eve XORs the two halves together to recover KU, then she uses KU to recover KS, and KS to eavesdrop on the conversation.

The checksum is designed to prevent someone from circumventing this scheme; the receiving Clipper chip won't decrypt if the checksum doesn't check. However, there are only 216 possible checksum values, and a bogus LEAF with the right checksum but the wrong key can be found in about 42 minutes [187]. This isn't much help for Clipper voice conversations. Because the key exchange protocol is not part of the Clipper chip, the 42-minute brute-force attack must occur after key exchange; it cannot be done before making the telephone call. This attack may work for facsimile transmission or with the Fortezza card (see Section 24.17).

Supposedly, the Clipper chip will resist reverse-engineering by "a very sophisticated, well-funded adversary" [1154], but rumors are that Sandia National Laboratories successfully reverse-engineered one. Even if those rumors aren't true, I suspect that the largest chip manufacturers in the world can reverse-engineer Clipper; it's just a matter of time before someone with the right combination of resources and ethics comes along.

Enormous privacy issues are associated with this scheme. Numerous civil liberty advocacy groups are actively campaigning against any key-escrow mechanism that gives the government the right to eavesdrop on citizens. But the sneaky thing is that this idea never went through Congress; NIST published the Escrowed Encryption Standard as a FIPS [1153], bypassing that irritating legislative process. Right now it looks like the EES is dying a slow and quiet death, but standards have a way of creeping up on you.

Anyway, Table 24.2 lists the different agencies participating in this program. Anyone want to do a threat analysis on having both escrow agents in the executive branch? Or on having escrow agents who really don't know anything about the wiretap requests, and can do no more than blindly approve them? Or on having the government impose a secret algorithm as a commercial standard?

In any case, implementing Clipper raises enough problems to question its value in court. Remember, Clipper only works in OFB mode. Despite what you may have been told to the contrary, this does not provide integrity or authentication. Imagine that Alice is on trial, and a Clipper-encrypted telephone call is part of the evidence. Alice claims that she never made the call; the voice is not hers. The phone's compression algorithm is so bad that it is hard to recognize Alice's voice, but the prosecution argues that since only Alice's escrowed key will decipher the call it must have been made from her telephone.

Alice argues that the call was forged like so [984, 1339]: Given the ciphertext and the plaintext, it is possible to XOR them to get the keystream. This keystream can then be XORed with an entirely different plaintext to form a forged ciphertext, which can then be converted to forged plaintext when fed into the Clipper decryptor. True or not, this argument could easily put enough doubt in a jury's mind to disregard the telephone call as evidence.

Another attack, called the Squeeze attack, allows Alice to frame Bob. Here's how [575]: Alice calls Bob using Clipper. She saves a copy of his LEAF as well as the session key. Then, she calls Carol (who she knows is being wiretapped). During the key setup, Alice forces the session key to be identical to the one she used with Bob; this requires hacking the phone, but it is not hard. Then, instead of sending her LEAF she sends Bob's. It's a valid LEAF, so Carol's phone will not notice. Now she can say whatever she wants to Carol; when the police decrypt the LEAF, they will find that it is Bob's. Even if Bob wasn't framed by Alice, the mere fact that he can claim this in court undermines the purpose of the scheme.

The law enforcement authorities of the United States should not be in the business of collecting information in criminal investigations that is useless in court. Even if key escrow were a good idea, Clipper is a bad way of implementing it.

24.17 Capstone

Capstone (also known as the MYK-80) is the other NSA-developed VLSI cryptographic chip that implements the U.S. government's Escrowed Encryption Standard [1153]. Capstone includes the following functions [1155, 462]:

Table 24.2
EES Participating Agencies
Justice—System Sponsor and Family Key Agent
NIST—Program Manager and Escrow Agent
FBI—Decrypt User and Family Key Agent
Treasury—Escrow Agent
NSA—Program Developer
— The Skipjack algorithm in any of the four basic modes: ECB, CBC, CFB, and OFB.
— A public-key Key Exchange Algorithm (KEA), probably Diffie-Hellman.
— The Digital Signature Algorithm (DSA).
— The Secure Hash Algorithm (SHA).
— A general purpose exponentiation algorithm.
— A general purpose, random-number generator that uses a pure noise source.

Capstone provides the cryptographic functionality needed for secure electronic commerce and other computer-based applications. The first application of Capstone is in a PCMCIA card called Fortezza. (It was originally called Tessera until a company called Tessera, Inc. complained.)

NSA had considered lengthening Capstone's LEAF checksum in production versions for use in Fortezza cards, in order to foil the brute-force attack against the LEAF previously discussed. Instead, they added a feature that reset the card after 10 incorrect LEAFs. This only increases the time required to find a fake but valid LEAF by 10 percent, to 46 minutes. I am not impressed.

24.18 AT&T Model 3600 Telephone Security Device (TSD)

The AT&T Telephone Security Device (TSD) is the Clipper phone. Actually, there are four models of the TSD. One contains the Clipper chip, another contains an exportable proprietary AT&T encryption algorithm, the third contains a proprietary algorithm for domestic use plus the exportable algorithm, and the fourth contains the Clipper, domestic, and exportable algorithms.

TSDs use a different session key for each telephone call. A pair of TSDs generate a session key using Diffie-Hellman key exchange, independent of the Clipper chip. Since Diffie-Hellman incorporates no authentication, the TSD has two methods to thwart a man-in-the-middle attack.

The first is a screen. The TSD hashes the session key and displays that hash on a small screen as four Hex digits. The conversants should confirm that their screens show the same digits. The voice quality is good enough that they can recognize each other's voice.

Eve still has a possible attack. Imagine her in the middle of Alice and Bob's conversation. She uses one TSD on the line with Alice and a modified TSD on the line with Bob; in the middle she bridges the two phone calls. Alice tries to go secure. She generates a key as normal, except that Eve is acting as Bob. Eve recovers the key, and using the modified TSD, forces the key she generates with Bob to have the same hash value. This attack may not sound very likely, but the TSD uses a variant of the interlock protocol to prevent it.

The TSD generates random numbers using a noise source and a chaotic amplifier with digital feedback. This generates a bit stream, which is fed through a post-whitening filter using the digital signal processor.

Despite all of this, the TSD manual does not mention security at all. In fact, it says [70]:

AT&T makes no warranty that the TSD will prevent cryptanalytic attack on any encrypted transmission by any government agency, its agents, or any third party. Furthermore, AT&T makes no warranty that the TSD will prevent any attack on any communication by methods which bypass encryption.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Chapter 25
Politics

25.1 National Security Agency (NSA)

The NSA is the National Security Agency (once called "No Such Agency" or "Never Say Anything," but they've been more open recently), the official security body of the U.S. government. President Harry Truman created the agency in 1952 under the Department of Defense, and for many years its very existence was kept secret. The NSA is concerned with signals intelligence; its mandate is to listen in on and decode all foreign communications of interest to the security of the United States.

The following paragraphs are excerpted from NSA's original charter, signed by President Truman in 1952, and classified for many years thereafter [1535]:

The COMINT mission of the National Security Agency (NSA) shall be to provide an effective, unified organization and control of the communications intelligence activities of the United States conducted against foreign governments, to provide for integrated operational policies and procedures pertaining thereto. As used in this directive, the terms "communications intelligence" or "COMINT" shall be construed to mean all procedures and methods used in the interception of communications other than foreign press and propaganda broadcasts and the obtaining of information from such communications by other than intended recipients, but shall exclude censorship and the production and dissemination of finished intelligence.

The special nature of COMINT actives requires that they be treated in all respects as being outside the framework of other or general intelligence activities. Orders, directives, policies, or recommendations of any authority of the Executive Branch relating to the collection, production, security, handling, dissemination, or utilization of intelligence, and/or classified material, shall not be applicable to COMINT actives, unless specifically so stated and issued by competent department or agency authority represented on the Board. Other National Security Council Intelligence Directives to the Director of Central Intelligence and related implementing directives issued by the Director of Central Intelligence shall be construed as non-applicable to COMINT activities, unless the National Security Council has made its directive specifically applicable to COMINT.

NSA conducts research in cryptology, both in designing secure algorithms to protect U.S. communications and in designing cryptanalytic techniques to listen in on non-U.S. communications. The NSA is known to be the largest employer of mathematicians in the world; it is also the largest purchaser of computer hardware in the world. The NSA probably possesses cryptographic expertise many years ahead of the public state of the art (in algorithms, but probably not in protocols) and can undoubtedly break many of the systems used in practice. But, for reasons of national security, almost all information about the NSA—even its budget—is classified. (Its budget is rumored to be $13 billion per year—including military funding of NSA projects and personnel—and it is rumored to employ 16,000 people.)

The NSA uses its power to restrict the public availability of cryptography, so as to prevent national enemies from employing encryption methods too strong for the NSA to break. James Massey discusses this struggle between academic and military research in cryptography [1007]:

If one regards cryptology as the prerogative of government, one accepts that most cryptologic research will be conducted behind closed doors. Without doubt, the number of workers engaged today in such secret research in cryptology far exceeds that of those engaged in open research in cryptology. For only about 10 years has there in fact been widespread open research in cryptology. There have been, and will continue to be, conflicts between these two research communities. Open research is a common quest for knowledge that depends for its vitality on the open exchange of ideas via conference presentations and publications in scholarly journals. But can a government agency, charged with responsibilities of breaking the ciphers of other nations, countenance the publication of a cipher that it cannot break? Can a researcher in good conscience publish such a cipher that might undermine the effectiveness of his own government's code-breakers? One might argue that publication of a provably secure cipher would force all governments to behave like Stimson's "gentlemen," but one must be aware that open research in cryptography is fraught with political and ethical considerations of a severity more than in most scientific fields. The wonder is not that some conflicts have occurred between government agencies and open researchers in cryptology, but rather that these conflicts (at least those of which we are aware) have been so few and so mild.

James Bamford wrote a fascinating book about the NSA: The Puzzle Palace [79], recently updated by Bamford and Wayne Madsen [80].

The Commercial COMSEC Endorsement Program (CCEP)

The Commercial COMSEC Endorsement Program (CCEP), codenamed Overtake, is a 1984 NSA initiative to facilitate the development of computer and communications products with embedded cryptography [85,1165]. The military had always paid for this kind of thing for themselves, and it was very expensive. The NSA figured that if companies could sell equipment to both the military and to corporate users, even overseas, costs would go down and everyone would benefit. They would no longer endorse equipment as complying with Federal Standard 1027, and then CCEP would provide government-endorsed cryptographic equipment [419].

NSA developed a series of cryptographic modules for different purposes. Different algorithms would be used in the modules for different applications, and manufacturers would be able to pull one module out and plug in another depending on the customer. There were modules for military use (Type I), modules for "unclassified but sensitive" government use (Type II), modules for corporate use (Type III), and modules for export (Type IV). Table 25.1 summarizes the different modules, applications, and names.

This program is still around, but never became popular outside the government. All the modules were tamperproof, all the algorithms were classified, and you had to get your keys from NSA. Corporations never really bought into the idea of using classified algorithms dictated by the government. You'd think the NSA would have learned from this lesson and not even bothered with Clipper, Skipjack, and escrowed encryption chips.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

25.2 National Computer Security Center (NCSC)

The National Computer Security Center, a branch of the NSA, is responsible for the government's trusted computer program. Currently, the center evaluates commercial security products (both hardware and software), sponsors and publishes research, develops technical guidelines, and generally provides advice, support, and training.

The NCSC publishes the infamous "Orange Book" [465]. Its actual title is the Department of Defense Trusted Computer System Evaluation Criteria, but that's a mouthful to say and the book has an orange cover. The Orange Book attempts to define security requirements, gives computer manufacturers an objective way to measure the security of their systems, and guides them as to what to build into their secure products. It focuses on computer security and doesn't really say a lot about cryptography.

The Orange Book defines four broad divisions of security protection. It also defines classes of protection within some of those divisions. They are summarized in Table 25.2.

Table 25.1 CCEP Modules
Application | Type I | Type II
Voice/low-speed data | Winster | Edgeshot
Computer | Tepache | Bulletproof
High-speed data | Foresee | Brushstroke
Next Generation | Countersign I | Countersign II

Sometimes manufacturers say things like "we have C2 security." This is what they're talking about. For more information on this, read [1365]. The computer security model used in these criteria is called the Bell-LaPadula model [100,101,102,103].

The NCSC has published a whole series of books on computer security, sometimes called the Rainbow Books (all the covers have different colors). For example, Trusted Network Interpretation of the Trusted Computer System Evaluation Criteria [1146], sometimes called the "Red Book," interprets the Orange Book for networks and network equipment. The Trusted Database Management System Interpretation of the Trusted Computer System Evaluation Criteria [1147]—I can't even begin to describe the color of that cover—does the same for databases. There are now over 30 of these books, some with hideously colored covers.

For a complete set of the Rainbow Books, write Director, National Security Agency, INFOSEC Awareness, Attention: C81, 9800 Savage Road, Fort George G. Meade, MD 20755-6000; (410) 766-8729. Don't tell them I sent you.

25.3 National Institute of Standards and Technology (NIST)

The NIST is the National Institute of Standards and Technology, a division of the U.S. Department of Commerce. Formerly the NBS (National Bureau of Standards), it changed its name in 1988. Through its Computer Systems Laboratory (CSL), NIST promotes open standards and interoperability that it hopes will spur the economic development of computer-based industries. To this end, NIST issues standards and guidelines that it hopes will be adopted by all computer systems in the United States. Official standards are published as FIPS (Federal Information Processing Standards) publications.

If you want copies of any FIPS (or any other NIST publication), contact National Technical Information Service (NTIS), U.S. Department of Commerce, 5285 Port Royal Road, Springfield, VA 22161; (703) 487-4650; or visit gopher://csrc.ncsl.nist.gov.

When Congress passed the Computer Security Act of 1987, NIST was mandated to define standards for ensuring the security of sensitive but unclassified information in government computer systems. (Classified information and Warner Amendment data are under the jurisdiction of the NSA.) The Act authorizes NIST to work with other government agencies and private industry in evaluating proposed technology standards.

Table 25.2 Orange Book Classifications
D: Minimal Security C: Discretionary Protection
 | C1: Discretionary Security Protection C2: Controlled Access Protection
B: Mandatory Protection
 | B1: Labeled Security Protection B2: Structured Protection B3: Security Domains
A: Verified Protection
 | A1: Verified Design

NIST issues standards for cryptographic functions. U.S. government agencies are required to use them for sensitive but unclassified information. Often the private sector adopts these standards as well. NIST issued DES, DSS, SHS, and EES.

All these algorithms were developed with some help from the NSA, ranging from analyzing DES to designing DSS, SHS, and the Skipjack algorithm in EES. Some people have criticized NIST for allowing the NSA to have too much control over these standards, since the NSA's interests may not coincide with those of NIST. It is unclear how much actual influence NSA has on the design and development of the algorithms. Given NIST's limited staff, budget, and resources, NSA's involvement is probably considerable. NSA has significant resources to contribute, including a computer facility second-to-none.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

The official "Memorandum of Understanding" (MOU) between the two agencies reads:

MEMORANDUM OF UNDERSTANDING BETWEEN THE DIRECTOR OF THE NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY AND THE DIRECTOR OF THE NATIONAL SECURITY AGENCY CONCERNING THE IMPLEMENTATION OF PUBLIC LAW 100-235

Recognizing that:

A. Under Section 2 of the Computer Security Act of 1987 (Public Law 100-235), (the Act), the National Institute of Standards and Technology (NIST) has the responsibility within the Federal Government for:
1. Developing technical, management, physical, and administrative standards and guidelines for the cost-effective security and privacy of sensitive information in Federal computer systems as defined in the Act; and,
2. Drawing on the computer system technical security guidelines of the National Security Agency (NSA) in this regard where appropriate.
B. Under Section 3 of the Act, the NIST is to coordinate closely with other agencies and offices, including the NSA, to assure:
1. Maximum use of all existing and planned programs, materials, studies, and reports relating to computer systems security and privacy, in order to avoid unnecessary and costly duplication of effort; and,
2. To the maximum extent feasible, that standards developed by the NIST under the Act are consistent and compatible with standards and procedures developed for the protection of classified information in Federal computer systems.
C. Under the Act, the Secretary of Commerce has the responsibility, which he has delegated to the Director of NIST, for appointing the members of the Computer System Security and Privacy Advisory Board, at least one of whom shall be from the NSA.

Therefore, in furtherance of the purposes of this MOU, the Director of the NIST and the Director of the NSA hereby agree as follows:

I. The NIST will:
1. Appoint to the Computer Security and Privacy Advisory Board at least one representative nominated by the Director of the NSA.
2. Draw upon computer system technical security guidelines developed by the NSA to the extent that the NIST determines that such guidelines are consistent with the requirements for protecting sensitive information in Federal computer systems.
3. Recognize the NSA-certified rating of evaluated trusted systems under the Trusted Computer Security Evaluation Criteria Program without requiring additional evaluation.
4. Develop telecommunications security standards for protecting sensitive unclassified computer data, drawing upon the expertise and products of the National Security Agency, to the greatest extent possible, in meeting these responsibilities in a timely and cost-effective manner.
5. Avoid duplication where possible in entering into mutually agreeable arrangements with the NSA for the NSA support.
6. Request the NSA's assistance on all matters related to cryptographic algorithms and cryptographic techniques including but not limited to research, development evaluation, or endorsement.
II. The NSA will:
1. Provide the NIST with technical guidelines in trusted technology, telecommunications security, and personal identification that may be used in cost-effective systems for protecting sensitive computer data.
2. Conduct or initiate research and development programs in trusted technology, telecommunications security, cryptographic techniques and personal identification methods.
3. Be responsive to the NIST's requests for assistance in respect to all matters related to cryptographic algorithms and cryptographic techniques including but not limited to research, development, evaluation, or endorsement.
4. Establish the standards and endorse products for application to secure systems covered in 10 USC Section 2315 (the Warner Amendment).
5. Upon request by Federal agencies, their contractors and other government-sponsored entities, conduct assessments of the hostile intelligence threat to federal information systems, and provide technical assistance and recommend endorsed products for application to secure systems against that threat.
III. The NIST and the NSA shall:
1. Jointly review agency plans for the security and privacy of computer systems submitted to NIST and NSA pursuant to section 6(b) of the Act.
2. Exchange technical standards and guidelines as necessary to achieve the purposes of the Act.
3. Work together to achieve the purposes of this memorandum with the greatest efficiency possible, avoiding unnecessary duplication of effort.
4. Maintain an on-going open dialogue to ensure that each organization remains abreast of emerging technologies and issues affecting automated information system security in computer-based systems.
5. Establish a Technical Working Group to review and analyze issues of mutual interest pertinent to protection of systems that process sensitive or other unclassified information. The Group shall be composed of six federal employees, three each selected by NIST and NSA and to be augmented as necessary by representatives of other agencies. Issues may be referred to the group by either the NSA Deputy Director for Information Security or the NIST Deputy Director or may be generated and addressed by the group upon approval by the NSA DDI or NIST Deputy Director. Within days of the referral of an issue to the Group by either the NSA Deputy Director for Information Security or the NIST Deputy Director, the Group will respond with a progress report and plan for further analysis, if any.
6. Exchange work plans on an annual basis on all research and development projects pertinent to protection of systems that process sensitive or other unclassified information, including trusted technology, for protecting the integrity and availability of data, telecommunications security and personal identification methods. Project updates will be exchanged quarterly, and project reviews will be provided by either party upon request of the other party.
7. Ensure the Technical Working Group reviews prior to public disclosure all matters regarding technical systems security techniques to be developed for use in protecting sensitive information in federal computer systems to ensure they are consistent with the national security of the United States. If NIST and NSA are unable to resolve such an issue within 60 days, either agency may elect to raise the issue to the Secretary of Defense and the Secretary of Commerce. It is recognized that such an issue may be referred to the President through the NSC for resolution. No action shall be taken on such an issue until it is resolved.
8. Specify additional operational agreements in annexes to this MOU as they are agreed to by NSA and NIST.
IV. Either party may elect to terminate this MOU upon six months' written notice. This MOU is effective upon approval of both signatories.
/signed/
RAYMOND G. KAMMER
Acting Director, National Institute of Standards and Technology, 24 March 1989
W. O. STUDEMAN
Vice Admiral, U.S. Navy; Director, National Security Agency, 23 March 1989


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

25.4 RSA Data Security, Inc.

RSA Data Security, Inc. (RSADSI) was founded in 1982 to develop, license, and market the RSA patent. It has some commercial products, including a standalone e-mail security package, and various cryptographic libraries (available in either source or object form). RSADSI also markets the RC2 and RC4 symmetric algorithms (see Section 11.8). RSA Laboratories, a research lab associated with RSADSI, performs basic cryptographic research and provides consulting services.

Anyone interested in either their patents or products should contact Director of Sales, RSA Data Security, Inc., 100 Marine Parkway, Redwood City, CA 94065; (415) 595-8782; fax: (415) 595-1873.

25.5 Public Key Partners

The five patents in Table 25.3 are held by Public Key Partners (PKP) of Sunnyvale, California, a partnership between RSADSI and Caro-Kahn, Inc.—the parent company of Cylink. (RSADSI gets 65 percent of the profits and Caro-Kahn gets 35 percent.) PKP claims that these patents, and 4,218,582 in particular, apply to all uses of public-key cryptography.

In [574], PKP wrote:

These patents [4,200,770, 4,218,582, 4,405,829, and 4,424,414] cover all known methods of practicing the art of Public Key, including the variations collectively known as ElGamal.

Due to the broad acceptance of RSA digital signatures throughout the international community, Public Key Partners strongly endorses its incorporation in a digital signature standard. We assure all interested parties that Public Key Partners will comply with all of the policies of ANSI and the IEEE concerning the availability of licenses to practice this art. Specifically, in support of any RSA signature standard which may be adopted, Public Key Partners hereby gives its assurance that licenses to practice RSA signatures will be available under reasonable terms and conditions on a nondiscriminatory basis.

Whether this is true depends on who you talk to. PKP's licenses have mostly been secret, so there is no way to check if the licenses are standard. Although they claim to have never denied a license to anyone, at least two companies claim to have been denied a license. PKP guards its patents closely, threatening anyone who tries to use public-key cryptography without a license. In part, this is a reaction to U.S. patent law. If you hold a patent and fail to prosecute an infringement, you can lose your patent. There has been much talk about whether the patents are legal, but so far it has all been talk. All legal challenges to PKP's patents have been settled before judgment.

Table 25.3 Public Key Partners' Patents
Patent # | Date | Inventors | Patent Covers
4,200,770 | 4/29/80 | Hellman, Diffie, Merkle | Diffie-Hellman Key Exchange
4,218,582 | 8/19/80 | Hellman, Merkle | Merkle-Hellman Knapsacks
4,405,829 | 9/20/83 | Rivest, Shamir, Adleman | RSA
4,424,414 | 3/3/84 | Hellman, Pohlig | Pohlig-Hellman
4,995,082 | 2/19/91 | Schnorr | Schnorr Signatures

I am not going to dispense legal advice in this book. Maybe the RSA patent will not hold up in court. Maybe the patents do not apply to the entirety of public-key cryptography. (Honestly, I can't see how they cover ElGamal or elliptic curve cryptosystems.) Perhaps someone will eventually win a suit against PKP or RSADSI. But keep in mind that corporations with large legal departments like IBM, Microsoft, Lotus, Apple, Novell, Digital, National Semiconductor, AT&T, and Sun have all licensed RSA for use in their products rather than fight them in court. And Boeing, Shell Oil, DuPont, Raytheon, and Citicorp have all licensed RSA for their own internal use.

In one case, PKP brought suit against TRW Corporation for using the ElGamal algorithm without a license. TRW claimed they did not need a license. PKP and TRW reached a settlement in June 1992. The details of the settlement are unknown, but they included an agreement by TRW to license the patents. This does not bode well. TRW can afford good lawyers; I can only assume that if they thought they could win the suit without spending an unreasonable amount of money, they would have fought.

Meanwhile, PKP is having its own internal problems. In June 1994 Caro-Kahn sued RSADSI alleging, among other things, that the RSA patent is invalid and unenforceable [401]. Both partners are trying to have the partnership dissolved. Are the patents valid or not? Will users have to get a license from Caro-Kahn to use the RSA algorithm? Who will own the Schnorr patent? The matter will probably be sorted out by the time this book sees publication.

Patents are good for only 17 years, and cannot be renewed. On April 29, 1997, Diffie-Hellman key exchange (and the ElGamal algorithm) will enter the public domain. On September 20, 2000, RSA will enter the public domain. Mark your calendars.

25.6 International Association for Cryptologic Research (IACR)

The International Association for Cryptologic Research is the worldwide cryptographic research organization. Its stated purpose is to advance the theory and practice of cryptology and related fields. Membership is open to any person. The association sponsors two annual conferences, Crypto (held in Santa Barbara in August) and Eurocrypt (held in Europe in May), and publishes quarterly The Journal of Cryptology and the IACR Newsletter.

The address of the IACR Business Office changes whenever the president does. The current address is: IACR Business Office, Aarhus Science Park, Gustav Wieds Vej 10, DK-8000 Aarhus C, Denmark.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

25.7 RACE Integrity Primitives Evaluation (RIPE)

The Research and Development in Advanced Communication Technologies in Europe (RACE) program was launched by the European Community to support pre-competitive and pre-normative work in communications standards and technologies to support Integrated Broadband Communication (IBC). As part of that effort, RACE established the RACE Integrity Primitives Evaluation (RIPE) to put together a portfolio of techniques to meet the anticipated security requirements of IBC.

Six leading European cryptography research groups made up the RIPE consortium: Center for Mathematics and Computer Science, Amsterdam; Siemens AG; Philips Crypto BV; Royal PTT Nederland NV, PTT Research; Katholieke Universiteit Leuven; and Aarhus Universitet. After calls for algorithms in 1989 and 1991 [1564], 32 submissions from around the world, and a 350 man-month evaluation project, the consortium published RIPE Integrity Primitives [1305,1332]. The report included an introduction and some basic integrity concepts, and these primitives: MDC-4 (see Section 18.11), RIPE-MD (see Section 18.8), RIPE-MAC (see Section 18.14), IBC-HASH, SKID (see Section 3.2), RSA, COMSET (see Section 16.1), and RSA key generation.

25.8 Conditional Access for Europe (CAFE)

Conditional Access for Europe (CAFE) is a project in the European Community's ESPRIT program [204,205]. Work began in December 1992 and is scheduled to be finished by the end of 1995. The consortium involved consists of groups for social and market studies (Cardware, Institut für Sozialforschung), software and hardware manufacturers (DigiCash, Gemplus, Ingenico, Siemens), and cryptographers (CWI Amsterdam, PTT Research Netherlands, SPET, Sintef Delab Trondheim, Universities of Århus, Hildesheim and Leuven).

The goal is to develop systems for conditional access, particularly digital payment systems. Payment systems must give legal certainty to everybody at all times and require as little trust as possible—this certainty should not depend on the tamper-resistance of any devices.

The basic device for CAFE is an electronic wallet: a small computer that looks something like a pocket calculator. It has a battery, keyboard, screen, and an infrared channel for communicating with other wallets. Every user owns and uses his own wallet, which administers his rights and guarantees his security.

A device with a keyboard and screen has an advantage over a smart card; it can operate independent of a terminal. A user can directly enter his password and the amount of the payment. The user does not have to give his wallet up to complete a transaction, unlike the current situation with credit cards.

Additional features are:

— Offline transactions. The purpose of the system is to replace small cash transactions; an online system would be too cumbersome.
— Loss tolerance. If a user loses his wallet, or if it breaks or is stolen, he can recover his money.
— Support for different currencies.
— An open architecture and open system. A user should be able to pay for arbitrary services, such as shopping, telephone, and public transport, by a range of service providers. The system should be interoperable between any number of electronic money issuers, and between different wallet types and manufacturers.
— Low cost.

At this writing there is a software version of the system, and the consortium is hard at work on a hardware prototype.

25.9 ISO/IEC 9979

In the mid-80s, the ISO tried to standardize DES, which by then was already a FIPS and an ANSI standard. After some political wrangling, the ISO decided not to standardize cryptographic algorithms, but instead to register algorithms. Only encryption algorithms can be registered; hash functions and signature schemes cannot. Any national body can submit an algorithm for registration.

Currently only three algorithms have been submitted (see Table 25.4). A submission includes information about applications, parameters, implementations, modes, and test vectors. A detailed description is optional; it is possible to submit secret algorithms for registration.

The fact that an algorithm is registered does not imply anything about its quality, nor is registration an approval of the algorithm by the ISO/IEC. Registration merely indicates that a single national body wants to register the algorithm, based on whatever criteria that body uses.

I am not impressed with this idea. Registration obstructs the standardization process. Rather than agreeing on a few algorithms, the ISO is allowing any algorithm to be registered. With so little control over what is registered, stating that an algorithm is "ISO/IEC 9979 Registered" sounds a whole lot better than it is. In any case, the registry is maintained by the National Computer Centre Ltd., Oxford Road, Manchester, M1 7ED, United Kingdom.

Table 25.4 ISO/IEC 9979 Registered Algorithms
Name | Registration Number
B-CRYPT | 0001
IDEA | 0002
LUC | 0003

25.10 Professional, Civil Liberties, and Industry Groups

Electronic Privacy Information Center (EPIC)

EPIC was established in 1994 to focus public attention on emerging privacy issues relating to the National Information Infrastructure, such as the Clipper chip, the Digital Telephony proposal, national identity numbers and systems, medical records privacy, and the sale of consumer data. EPIC conducts litigation, sponsors conferences, produces reports, publishes the EPIC Alert, and leads campaigns on privacy issues. Anyone interested in joining should contact Electronic Privacy Information Center, 666 Pennsylvania Avenue SE, Suite 301, Washington, D.C. 20003; (202) 544-9240; fax: (202) 547-5482; Internet: info@epic.org.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Electronic Frontier Foundation (EFF)

The EFF is dedicated to protecting civil rights in cyberspace. With respect to cryptographic policy in the United States, they believe that information and access to cryptography are fundamental rights, and therefore should be free of government restriction. They organized the Digital Privacy and Security Working Group, a coalition of 50 organizations. The group opposed the Digital Telephony bill and the Clipper initiative. The EFF is also helping in a lawsuit against cryptography export controls [143]. Anyone interested in joining the EFF should contact Electronic Frontier Foundation, 1001 G Street NW, Suite 950E, Washington, D.C. 20001; (202) 347-5400; fax: (202) 393-5509; Internet: eff@eff.org.

Association for Computing Machinery (ACM)

The ACM is an international computer industry organization. In 1994 the U.S. ACM Public Policy Committee produced an excellent report on U.S. cryptography policy [935]. This should be required reading for anyone interested in the politics of cryptography. It is available via anonymous ftp from info.acm.org in /reports/acm_crypto/acm_crypto_study.ps.

Institute of Electrical and Electronics Engineers (IEEE)

The IEEE is another professional organization. The U.S. office investigates and makes recommendations on privacy-related issues including encryption policy, identity numbers, and privacy protections on the Internet.

Software Publishers Association (SPA)

The SPA is a trade association of over 1000 personal computer software companies. They have lobbied for relaxation of export controls on cryptography, and maintain a list of commercially available foreign cryptography products.

25.11 Sci.crypt

Sci.crypt is the Usenet newsgroup for cryptology. It is read by an estimated 100,000 people worldwide. Most of the posts are nonsense, bickering, or both; some are political, and most of the rest are requests for information or basic questions. Occasionally nuggets of new and useful information are posted to this newsgroup. If you follow sci.crypt regularly, you will learn how to use something called a kill file.

Another Usenet newsgroup is sci.crypt.research, a moderated newsgroup devoted to discussions about cryptology research. There are fewer posts and they are more interesting.

25.12 Cypherpunks

The Cypherpunks are an informal group of people interested in teaching and learning about cryptography. They also experiment with cryptography and try to put it into use. In their opinion, all the cryptographic research in the world doesn't do society any good unless it gets used.

In "A Cypherpunk's Manifesto," Eric Hughes writes [744]:

We the Cypherpunks are dedicated to building anonymous systems. We are defending our privacy with cryptography, with anonymous mail forwarding systems, with digital signatures, and with electronic money.

Cypherpunks write code. We know that someone has to write software to defend privacy, and since we can't get privacy unless we all do, we're going to write it. We publish our code so that our fellow Cypherpunks may practice and play with it. Our code is free for all to use, worldwide. We don't care much if you don't approve of the software we write. We know that software can't be destroyed and that widely dispersed systems can't be shut down.

People interested in joining the cypherpunks mailing list on the Internet should send mail to majordomo@toad.com. The mailing list is archived at ftp.csua.berkeley.edu in /pub/cypherpunks.

25.13 Patents

Software patents are an issue much larger than the scope of this book. Whether they're good or bad, they exist. Algorithms, cryptographic algorithms included, can be patented in the United States. IBM owned the DES patents [514]. IDEA is patented. Almost every public-key algorithm is patented. NIST even has a patent for the DSA. Some cryptography patents have been blocked by intervention from the NSA, under the authority of the Invention Secrecy Act of 1940 and the National Security Act of 1947. This means that instead of a patent, the inventor gets a secrecy order and is prohibited from discussing his invention with anybody.

The NSA has special dispensation when it comes to patents. They can apply for a patent and then block its issuance. It's a secrecy order again, but here the NSA is both the inventor and the issuer of the order. When, at some later date, the secrecy order is removed, the Patent Office issues the patent good for the standard 17 years. This rather clearly protects the invention while keeping it secret. If someone else invents the same thing, the NSA has already filed for the patent. If no one else invents it, then it remains secret.

Not only does this fly directly in the face of the patent process, which is supposed to disclose as well as protect inventions, it allows the NSA to keep a patent for more than 17 years. The 17-year clock starts ticking after the patent is issued, not when it is filed. How this will change, now that the United States has ratified the GATT treaty, is unclear.

25.14 U.S. Export Rules

According to the U.S. government, cryptography can be a munition. This means it is covered under the same rules as a TOW missile or an M1 Abrams tank. If you sell cryptography overseas without the proper export license, then you are an international arms smuggler. Unless you think time in a federal penitentiary would look good on your résumé, pay attention to the rules.

With the advent of the Cold War in 1949, all of the NATO countries (except Iceland), and later Australia, Japan, and Spain, formed CoCom, the Coordinating Committee for Multilateral Export Controls. This is an unofficial nontreaty organization, chartered to coordinate national restrictions on the export of sensitive military technologies to the Soviet Union, other Warsaw Pact countries, and the People's Republic of China. Examples of controlled technologies are computers, milling machinery, and cryptography. The goal here was to slow technology transfer into those countries, and thereby keep their militaries inferior.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Since the end of the Cold War, the CoCom countries realized that many of their controls were obsolete. They are supposedly in the process of defining something called the "New Forum," another multinational organization designed to stop the flow of military technologies to countries the members don't particularly like.

In any case, U.S. export policy on strategic goods is defined by the Export Administration Act, the Arms Export Control Act, the Atomic Energy Act, and the Nuclear Non-Proliferation Act. The controls established by all this legislation are implemented through a number of statutes, none of them coordinated with each other. Over a dozen agencies including the military services administer controls; often their regulatory programs overlap and contradict.

Controlled technologies appear on several lists. Cryptography has traditionally been classified as a munition and appears on the U.S. Munitions List (USML), the International Munitions List (IML), the Commerce Control List (CCL), and the International Industrial List (IIL). The Department of State is responsible for the USML; it is published as part of the International Traffic in Arms Regulations (ITAR) [466,467].

Two U.S. government agencies control export of cryptography. One is the Bureau of Export Administration (BXA) in the Department of Commerce, authorized by the Export Administration Regulations (EAR). The other is the Office of Defense Trade Controls (DTC) in the State Department, authorized by the ITAR. As a rule of thumb, the Commerce Department's BXA has far less stringent requirements, but State Department's DTC (which takes technical and national security advice from the NSA, and always seems to follow that advice) sees all cryptography exports first and can refuse to transfer jurisdiction to BXA.

The ITAR regulates this stuff. (Before 1990 the Office of Defense Trade Controls was called the Office of Munitions Controls; presumably this public relations effort is designed to help us forget that we're dealing with guns and bombs.) Historically, the DTC has been reluctant to grant export licenses for encryption products stronger than a certain level—not that they have ever been public about exactly what that level is.

The following sections are excerpted from the ITAR [466,467]:

§ 120.10 Technical data.

Technical data means, for purposes of this subchapter:
(1) Information, other than software as defined in 120.10(d), which is required for the design, development, production, processing, manufacture, assembly, operation, repair, maintenance or modification of defense articles. This includes, for example, information in the form of blueprints, drawings, photographs, plans, instructions and documentation;
(2) Classified information relating to defense articles and defense services;
(3) Information covered by an invention secrecy order;
(4) Software as defined in Sec. 121.8(f) directly related to defense articles;
(5) This definition does not include information concerning general scientific, mathematical or engineering principles commonly taught in schools, colleges and universities in the public domain as defined in § 120.11. It also does not include basic marketing information on function or purpose or general system descriptions of defense articles.

§ 120.11 Public domain.

Public domain means information which is published and which is generally accessible or available to the public:
(1) Through sales at newsstands and bookstores;
(2) Through subscriptions which are available without restriction to any individual who desires to obtain or purchase the published information;
(3) Through second class mailing privileges granted by the U.S. Government;
(4) At libraries open to the public or from which the public can obtain documents;
(5) Through patents available at any patent office;
(6) Through unlimited distribution at a conference, meeting, seminar, trade show or exhibition, generally accessible to the public, in the United States;
(7) Through public release (i.e., unlimited distribution) in any form (e.g., not necessarily in published form) after approval by the cognizant U.S. government department or agency (see also § 125.4(b)(13)).
(8) Through fundamental research in science and engineering at accredited institutions of higher learning in the U.S., where the resulting information is ordinarily published and shared broadly in the scientific community. Fundamental research is defined to mean basic and applied research in science and engineering where the resulting information is ordinarily published and shared broadly within the scientific community, as distinguished from research the results of which are restricted for proprietary reasons or specific U.S. Government access and dissemination controls. University research will not be considered fundamental research if:
(i) The University or its researchers accept other restrictions on publication of scientific and technical information resulting from the project or activity, or
(ii) The research is funded by the U.S. Government and specific access and dissemination controls protecting information resulting from the research are applicable.

§ 120.17 Export.

Export means:
(1) Sending or taking defense articles out of the United States in any manner, except by mere travel outside of the United States by a person whose personal knowledge includes technical data; or
(2) Transferring registration, control or ownership to a foreign person of any aircraft, vessel, or satellite covered by the U.S. Munitions List, whether in the United States or abroad; or
(3) Disclosing (including oral or visual disclosure) or transferring in the United States any defense articles to an embassy, any agency or subdivision of a foreign government (e.g., diplomatic missions); or
(4) Disclosing (including oral or visual disclosure) or transferring technical data to a foreign person, whether in the United States or abroad; or
(5) Performing a defense service on behalf of, or for the benefit of, a foreign person, whether in the United States or abroad.
(6) A launch vehicle or payload shall not, by the launching of such vehicle, be considered export for the purposes of this subchapter. However, for certain limited purposes (see § 126.1 of this subchapter), the controls of this subchapter apply to sales and other transfers of defense articles or defense services.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Part 121—The United States Munitions List

§ 121.1 General. The United States Munitions List

Category XIII—Auxiliary Military Equipment
(1) Cryptographic (including key management) systems, equipment, assemblies, modules, integrated circuits, components or software with the capability of maintaining secrecy or confidentiality of information or information systems, except cryptographic equipment and software as follows:
(i) Restricted to decryption functions specifically designed to allow the execution of copy protected software, provided the decryption functions are not user-accessible.
(ii) Specifically designed, developed or modified for use in machines for banking or money transactions, and restricted to use only in such transactions. Machines for banking or money transactions include automatic teller machines, self-service statement printers, point of sale terminals or equipment for the encryption of interbanking transactions.
(iii) Employing only analog techniques to provide the cryptographic processing that ensures information security in the following applications....
(iv) Personalized smart cards using cryptography restricted for use only in equipment or systems exempted from the controls of the USML.
(v) Limited to access control, such as automatic teller machines, self-service statement printers or point of sale terminals, which protects passwords or personal identification numbers (PIN) or similar data to prevent unauthorized access to facilities but does not allow for encryption or files or text, except as directly related to the password of PIN protection.
(vi) Limited to data authentication which calculates a Message Authentication Code (MAC) or similar result to ensure no alteration of text has taken place, or authenticate users, but does not allow for encryption of data, text or other media other than that needed for the authentication.
(vii) Restricted for fixed data compression or coding techniques.
(viii) Limited to receiving for radio broadcast, pay television or similar restricted audience television of the consumer type, without digital encryption and where digital decryption is limited to video, audio or management functions.
(ix) Software designed or modified to protect against malicious computer damage, (e.g., viruses).
(2) Cryptographic (including key management) systems, equipment, assemblies, modules, integrated circuits, components or software which have the capability of generating spreading or hopping codes for spread spectrum systems or equipment.
(3) Cryptographic systems, equipment, assemblies, modules, integrated circuits, components or software.

§ 125.2 Exports of unclassified technical data.

(a) General. A license (DSP-5) is required for the export of unclassified technical data unless the export is exempt from the licensing requirements of this subchapter. In the case of a plant visit, details of the proposed discussions must be transmitted to the Office of Defense Trade Controls for an appraisal of the technical data. Seven copies of the technical data or the details of the discussions must be provided.
(b) Patents. A license issued by the Office of Defense Trade Controls is required for the export of technical data whenever the data exceeds that which is used to support a domestic filing of a patent application or to support a foreign filing of a patent application whenever no domestic application has been filed. Requests for the filing of patent applications in a foreign country, and requests for the filing of amendments, modifications or supplements to such patents, should follow the regulations of the U.S. Patent and Trademark Office in accordance with 37 CFR part 5. The export of technical data to support the filing and processing of patent applications in foreign countries is subject to regulations issued by the U.S. Patent and Trademark Office pursuant to 35 U.S.C. 184.
(c) Disclosures. Unless otherwise expressly exempted in this subchapter, a license is required for the oral, visual or documentary disclosure of technical data by U.S. persons to foreign persons. A license is required regardless of the manner in which the technical data is transmitted (e.g., in person, by telephone, correspondence, electronic means, etc.). A license is required for such disclosures by U.S. persons in connection with visits to foreign diplomatic missions and consular offices.

And so on. There's a lot more information in this document. If you're going to try to export cryptography, I suggest you get a copy of the entire thing and a lawyer who speaks the language.

In reality, the NSA has control over the export of cryptographic products. If you want a Commodity Jurisdiction (CJ), you must submit your product to the NSA for approval and submit the CJ application to the State Department. After State Department approval, the matter moves under the jurisdiction of the Commerce Department, which has never cared much about the export of cryptography. However, the State Department will never grant a CJ without NSA approval.

In 1977 an NSA employee named Joseph A. Meyer wrote a letter—unauthorized, according to the official story of the incident—to the IEEE, warning them that the scheduled presentation of the original RSA paper would violate the ITAR. From The Puzzle Palace:

He had a point. The ITAR did cover any "unclassified information that can be used, or adapted for use, in the design, production, manufacture, repair, overhaul, processing, engineering, development, operation, maintenance, or reconstruction" of the listed materials, as well as "any technology which advances the state-of-the-art or establishes a new art in an area of significant military applicability in the United States." And export did include transferring the information both by writing and by either oral or visual means, including briefings and symposia in which foreign nationals are present.

But followed literally, the vague, overly broad regulations would seem to require that anyone planning to write or speak out publicly on a topic touching the Munitions List must first get approval from the State Department—a chilling prospect clearly at odds with the First Amendment and one as yet untested by the Supreme Court.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

In the end NSA disavowed Meyer's actions and the RSA paper was presented as planned. No actions were taken against any of the inventors, although their work arguably enhanced foreign cryptography capabilities more than anything released since.

The following statement by NSA discusses the export of cryptography [363]:

Cryptographic technology is deemed vital to national security interests. This includes economic, military, and foreign policy interests.

We do not agree with the implications from the House Judiciary Committee hearing of 7 May 1992 and recent news articles that allege that U.S. export laws prevent U.S. firms' manufacture and use of top encryption equipment. We are unaware of any case where a U.S. firm has been prevented from manufacturing and using encryption equipment within this country or for use by the U.S. firm or its subsidiaries in locations outside the U.S. because of U.S. export restrictions. In fact, NSA has always supported the use of encryption by U.S. businesses operating domestically and overseas to protect sensitive information.

For export to foreign countries, NSA as a component of the Department of Defense (along with the Department of State and the Department of Commerce) reviews export licenses for information security technologies controlled by the Export Administration Regulations or the International Traffic in Arms Regulations. Similar export control systems are in effect in all the Coordinating Committee for Multilateral Export Controls (CoCom) countries as well as many non-CoCom countries as these technologies are universally considered as sensitive. Such technologies are not banned from export and are reviewed on a case-by-case basis. As part of the export review process, licenses may be required for these systems and are reviewed to determine the effect such export could have on national security interests—including economic, military, and political security interests. Export licenses are approved or denied based upon the type of equipment involved, the proposed end use and the end user.

Our analysis indicates that the U.S. leads the world in the manufacture and export of information security technologies. Of those cryptologic products referred to NSA by the Department of State for export licenses, we consistently approve over 90%. Export licenses for information security products under the jurisdiction of the Department of Commerce are processed and approved without referral to NSA or DoD. This includes products using such techniques as the DSS and RSA which provide authentication and access control to computers or networks. In fact, in the past NSA has played a major role in successfully advocating the relaxation of export controls on RSA and related technologies for authentication purposes. Such techniques are extremely valuable against the hacker problem and unauthorized use of resources.

It is the stated policy of the NSA not to restrict the export of authentication products, only encryption products. If you want to export an authentication-only product, approval may merely be a matter of showing that your product cannot easily be used for encryption. Furthermore, the bureaucratic procedures are much simpler for authentication products than for encryption products. An authentication product needs State Department approval only once for a CJ; an encryption product may require approval for every product revision or even every sale.

Without a CJ, you must request export approval every time you wish to export the product. The State Department does not approve the export of products with strong encryption, even those using DES. Isolated exceptions include export to U.S. subsidiaries for the purposes of communicating to the U.S., exports for some banking applications, and export to appropriate U.S. military users. The Software Publishers Association (SPA) has been negotiating with the government to ease export license restrictions. A 1992 agreement between them and the State Department eased the export license rules for two algorithms, RC2 and RC4, as long as the key size is 40 bits or less. Refer to Section 7.1 for more information.

In 1993, Rep. Maria Cantwell (D-WA) introduced a bill at the behest of the software industry to relax export controls on encryption software. Sen. Patty Murray (D-WA) introduced a companion bill in the Senate. The Cantwell Bill was appended to the general export control legislation going through Congress, but was deleted by the House Intelligence Committee after a massive lobbying effort by the NSA. Whatever the NSA did, it was impressive; the committee voted unanimously to delete the wording. I can't remember the last time a bunch of legislators voted unanimously to do anything.

In 1995 Dan Bernstein, with the help of the EFF, sued the U.S. government, seeking to bar the government from restricting publication of cryptographic documents and software [143]. The suit claimed that the export control laws are unconstitutional, an "impermissible prior restraint on speech, in violation of the First Amendment." Specifically, the lawsuit charges that the current export control process:

— Allows bureaucrats to restrict publication without ever going to court.
— Provides too few procedural safeguards for First Amendment rights.
— Requires publishers to register with the government, creating in effect a "licensed press."
— Disallows general publication by requiring recipients to be individually identified.
— Is sufficiently vague that ordinary people cannot know what conduct is allowed and what conduct is prohibited.
— Is overbroad because it prohibits conduct that is clearly protected (such as speaking to foreigners within the United States).
— Is applied too broadly, by prohibiting export of software that contains no cryptography, on the theory that cryptography could be added to it later.
— Egregiously violates the First Amendment by prohibiting private speech on cryptography because the government wishes its own opinions on cryptography to guide the public instead.
— Exceeds the authority granted by Congress in the export control laws in many ways, as well as exceeding the authority granted by the Constitution.


Applied Cryptography, Second Edition

 Applied Cryptography, Second Edition - Bruce Schneier

Everyone anticipates that the case will take several years to settle, and no one has any idea how it will come out.

Meanwhile, the Computer Security and Privacy Advisory Board, an official advisory board to NIST, voted in March 1992 to recommend a national policy review of cryptographic issues, including export policy. They said that export policy is decided solely by agencies concerned with national security, without input from agencies concerned with encouraging commerce. Those agencies concerned with national security are doing everything possible to make sure this doesn't change, but eventually it has to.

25.15 Foreign Import and Export of Cryptography

Other countries have their own import and export rules [311]. This summary is incomplete and probably out of date. Countries could have rules and ignore them, or could have no rules but restrict import, export, and use anyway.

— Australia requires an import certificate for cryptography only upon request from the exporting country.
— Canada has no import controls, and export controls are similar to those of the United States. The exportation of items from Canada may be subject to restriction if they are included on the Export Control List pursuant to the Export and Import Permits Act. Canada follows the CoCom regulations in the regulation of cryptographic technology. Encryption devices are outlined in category five, part two of Canada's export regulations. These provisions are similar to U.S. category five in the Export Administration Regulations.
— China has a licensing scheme for importing commodities; exporters must file an application with the Ministry of Foreign Trade. Based on China's List of Prohibited and Restricted Imports and Exports enacted in 1987, China restricts the import and export of voice-encoding devices.
— France has no special rules for the import of cryptography, but they have rules regarding the sale and use of cryptography in their country. All products must be certified: Either they must meet a published specification, or the company proprietary specification must be provided to the government. The government may also ask for two units for their own use. Companies must have a license to sell cryptography within France; the license specifies the target market. Users must have a license to buy and use cryptography; the license includes a statement to the effect that users must be prepared to give up their keys to the government up to four months after use. This restriction may be waived in some cases: for banks, large companies, and so on. And there is no use license requirement for cryptography exportable from the U.S.
— Germany follows the CoCom guidelines, requiring a license to export cryptography. They specifically maintain control of public-domain and mass-market cryptography software.
— Israel has import restrictions, but no one seems to know what they are.
— Belgium, Italy, Japan, Netherlands, and the United Kingdom follow the CoCom guidelines on cryptography, requiring a license for export.
— Brazil, India, Mexico, Russia, Saudi Arabia, Spain, South Africa, Sweden, and Switzerland have no import or export controls on cryptography.

25.16 Legal Issues

Are digital signatures real signatures? Will they stand up in court? Some preliminary legal research has resulted in the opinion that digital signatures would meet the requirements of legally binding signatures for most purposes, including commercial use as defined in the Uniform Commercial Code (UCC). A GAO (General Accounting Office) decision, made at the request of NIST, opines that digital signatures will meet the legal standards of handwritten signatures [362].

The Utah Digital Signature Act went into effect on May 1, 1995, providing a legal framework for the use of digital signatures in the judicial system. California has a bill pending, while Oregon and Washington are still writing theirs. Texas and Florida are right behind. By this book's publication, more states will have followed suit.

The American Bar Association (EDI and Information Technology Division of the Science and Technology Section) produced a model act for states to use for their own legislation. The act attempts to incorporate digital signatures into the existing legal infrastructure for signatures: the Uniform Commercial Code, the United States Federal Reserve regulations, common law of contracts and signatures, the United Nations Convention on Contracts for the International Sale of Goods, and the United Nations Convention on International Bills of Exchange and International Promissory Committees. Included in the act are responsibilities and obligations of certification authorities, issues of liability, and limits and policies.

In the United States, laws about signatures, contracts, and commercial transactions are state laws, so this model act is designed for states. The eventual goal is a federal act, but if this all begins at the state level there is less chance of the NSA mucking up the works.

Even so, the validity of digital signatures has not been challenged in court; their legal status is still undefined. In order for digital signatures to carry the same authority as handwritten signatures, they must first be used to sign a legally binding document, and then be challenged in court by one party. The court would then consider the security of the signature scheme and issue a ruling. Over time, as this happened repeatedly, a body of precedent rulings would emerge regarding which digital signature methods and what key sizes are required for a digital signature to be legally binding. This is likely to take years.

Until then, if two people wish to use digital signatures for contracts (or purchase requests, or work orders, or whatever), it is recommended that they sign a paper contract in which they agree in the future to be bound by any documents digitally signed by them [1099]. This document would specify algorithm, key size, and any other parameters; it should also delineate how disputes would be resolved.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Preface

The number of models available in neural network literature is quite large. Very often the treatment is mathematical and complex. This book provides illustrative examples in C++ that the reader can use as a basis for further experimentation. A key to learning about neural networks to appreciate their inner workings is to experiment. Neural networks, in the end, are fun to learn about and discover. Although the language for description used is C++, you will not find extensive class libraries in this book. With the exception of the backpropagation simulator, you will find fairly simple example programs for many different neural network architectures and paradigms. Since backpropagation is widely used and also easy to tame, a simulator is provided with the capacity to handle large input data sets. You use the simulator in one of the chapters in this book to solve a financial forecasting problem. You will find ample room to expand and experiment with the code presented in this book.

There are many different angles to neural networks and fuzzy logic. The fields are expanding rapidly with ever-new results and applications. This book presents many of the different neural network topologies, including the BAM, the Perceptron, Hopfield memory, ART1, Kohonen's Self-Organizing map, Kosko's Fuzzy Associative memory, and, of course, the Feedforward Backpropagation network (aka Multilayer Perceptron). You should get a fairly broad picture of neural networks and fuzzy logic with this book. At the same time, you will have real code that shows you example usage of the models, to solidify your understanding. This is especially useful for the more complicated neural network architectures like the Adaptive Resonance Theory of Stephen Grossberg (ART).

The subjects are covered as follows:

• Chapter 1 gives you an overview of neural network terminology and nomenclature. You discover that neural nets are capable of solving complex problems with parallel computational architectures. The Hopfield network and feedforward network are introduced in this chapter.
• Chapter 2 introduces C++ and object orientation. You learn the benefits of object-oriented programming and its basic concepts.
• Chapter 3 introduces fuzzy logic, a technology that is fairly synergistic with neural network problem solving. You learn about math with fuzzy sets as well as how you can build a simple fuzzifier in C++.
• Chapter 4 introduces you to two of the simplest, yet very representative, models of: the Hopfield network, the Perceptron network, and their C++ implementations.
• Chapter 5 is a survey of neural network models. This chapter describes the features of several models, describes threshold functions, and develops concepts in neural networks.
• Chapter 6 focuses on learning and training paradigms. It introduces the concepts of supervised and unsupervised learning, self-organization and topics including backpropagation of errors, radial basis function networks, and conjugate gradient methods.
• Chapter 7 goes through the construction of a backpropagation simulator. You will find this simulator useful in later chapters also. C++ classes and features are detailed in this chapter.
• Chapter 8 covers the Bidirectional Associative memories for associating pairs of patterns.
• Chapter 9 introduces Fuzzy Associative memories for associating pairs of fuzzy sets.
• Chapter 10 covers the Adaptive Resonance Theory of Grossberg. You will have a chance to experiment with a program that illustrates the working of this theory.
• Chapters 11 and 12 discuss the Self-Organizing map of Teuvo Kohonen and its application to pattern recognition.
• Chapter 13 continues the discussion of the backpropagation simulator, with enhancements made to the simulator to include momentum and noise during training.
• Chapter 14 applies backpropagation to the problem of financial forecasting, discusses setting up a backpropagation network with 15 input variables and 200 test cases to run a simulation. The problem is approached via a systematic 12-step approach for preprocessing data and setting up the problem. You will find a number of examples of financial forecasting highlighted from the literature. A resource guide for neural networks in finance is included for people who would like more information about this area.
• Chapter 15 deals with nonlinear optimization with a thorough discussion of the Traveling Salesperson problem. You learn the formulation by Hopfield and the approach of Kohonen.
• Chapter 16 treats two application areas of fuzzy logic: fuzzy control systems and fuzzy databases. This chapter also expands on fuzzy relations and fuzzy set theory with several examples.
• Chapter 17 discusses some of the latest applications using neural networks and fuzzy logic.

In this second edition, we have followed readers' suggestions and included more explanations and material, as well as updated the material with the latest information and research. We have also corrected errors and omissions from the first edition.

Neural networks are now a subject of interest to professionals in many fields, and also a tool for many areas of problem solving. The applications are widespread in recent years, and the fruits of these applications are being reaped by many from diverse fields. This methodology has become an alternative to modeling of some physical and nonphysical systems with scientific or mathematical basis, and also to expert systems methodology. One of the reasons for it is that absence of full information is not as big a problem in neural networks as it is in the other methodologies mentioned earlier. The results are sometimes astounding, even phenomenal, with neural networks, and the effort is at times relatively modest to achieve such results. Image processing, vision, financial market analysis, and optimization are among the many areas of application of neural networks. To think that the modeling of neural networks is one of modeling a system that attempts to mimic human learning is somewhat exciting. Neural networks can learn in an unsupervised learning mode. Just as human brains can be trained to master some situations, neural networks can be trained to recognize patterns and to do optimization and other tasks.

In the early days of interest in neural networks, the researchers were mainly biologists and psychologists. Serious research now is done by not only biologists and psychologists, but by professionals from computer science, electrical engineering, computer engineering, mathematics, and physics as well. The latter have either joined forces, or are doing independent research parallel with the former, who opened up a new and promising field for everyone.

In this book, we aim to introduce the subject of neural networks as directly and simply as possible for an easy understanding of the methodology. Most of the important neural network architectures are covered, and we earnestly hope that our efforts have succeeded in presenting this subject matter in a clear and useful fashion.

We welcome your comments and suggestions for this book, from errors and oversights, to suggestions for improvements to future printings at the following E-mail addresses:

V. Rao rao@cse.bridgeport.edu

H. Rao ViaSW@aol.com

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Dedication

To the memory of

Vydehamma, Annapurnamma, Anandarao, Madanagopalarao, Govindarao, and Rajyalakshamma.

Acknowledgments

We thank everyone at MIS:Press/Henry Holt and Co. who has been associated with this project for their diligence and support, namely, the Technical Editors of this edition and the first edition for their suggestions and feedback; Laura Lewin, the Editor, and all of the other people at MIS:Press for making the book a reality.

We would also like to thank Dr. Tarek Kaylani for his helpful suggestions, Professor R. Haskell, and our other readers who wrote to us, and Dr. V. Rao's students whose suggestions were helpful. Please E-mail us more feedback!

Finally, thanks to Sarada and Rekha for encouragement and support. Most of all, thanks to Rohini and Pranav for their patience and understanding through many lost evenings and weekends.

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Appendix A
Compiling Your Programs

All of the programs included in the book have been compiled and tested on Turbo C++, Borland C++, and Microsoft C++/Visual C++ with either the small or medium memory model. You should not have any problems in using other compilers, since standard I/O routines are used. Your target should be a DOS executable. With the backpropagation simulator of Chapters 7, 13, and 14 you may run into a memory shortage situation. You should unload any TSR (Terminate and Stay Resident) programs and/or choose smaller architectures for your networks. By going to more hidden layers with fewer neurons per layer, you may be able to reduce the overall memory requirements.

The programs in this book make heavy use of floating point calculations, and you should compile your programs to take advantage of a math coprocessor, if you have one installed in your computer.

The organization of files on the accompanying diskette are according to chapter number. You will find relevant versions of files in the corresponding chapter directory.

Most of the files are self-contained, or include other needed files in them. You will not require a makefile to build the programs. Load the main file for example for backpropagation, the backprop.cpp file, into the development environment editor for your compiler and build a .exe file. That's it!

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Appendix B
Mathematical Background

Dot Product or Scalar Product of Two Vectors

Given vectors U and V, where U = (u1, …, un) and V = (v1, …, vn), their dot product or scalar product is U • V = u1v1 +… + unvn = Σ ui vi.

Matrices and Some Arithmetic Operations on Matrices

A real matrix is a rectangular array of real numbers. A matrix with m rows and n columns is referred to as an mxn matrix. The element in the ith row and jth column of the matrix is referred to as the ij element of the matrix and is denoted by aij.

The transpose of a matrix M is denoted by MT. The element in the ith row and jth column of MT is the same as the element of M in its jth row and ith column. MT is obtained from M by interchanging the rows and columns of M. For example, if

            2  7 -3              2  4

       M =          , then MT =  7  0

            4  0  9             -3  9

If X is a vector with m components, x1, …, xm, then it can be written as a column vector with components listed one below another. It can be written as a row vector, X = (x1, …, xm). The transpose of a row vector is the column vector with the same components, and the transpose of a column vector is the corresponding row vector.

The addition of matrices is possible if they have the same size, that is, the same number of rows and same number of columns. Then you just add the ij elements of the two matrices to get the ij elements of the sum matrix. For example,

       3 -4 5    5 2 -3      8 -2  2

               +         =

       2  3 7    6 0  4      8  3 11

Multiplication is defined for a given pair of matrices, only if a condition on their respective sizes is satisfied. Then too, it is not a commutative operation. This means that if you exchange the matrix on the left with the matrix on the right, the multiplication operation may not be defined, and even if it is, the result may not be the same as before such an exchange.

The condition to be satisfied for multiplying the matrices A, B as AB is, that the number of columns in A is equal to the number of rows in B. Then to get the ij element of the product matrix AB, you take the ith row of A as one vector and the jth column of B as a second vector and do a dot product of the two. For example, the two matrices given previously to illustrate the addition of two matrices are not compatible for multiplication in whichever order you take them. It is because there are three columns in each, which is different from the number of rows, which is 2 in each. Another example is given as follows.

                 3 -4 5              5  6

       Let A =            and   B =  2  0

                 2  3 7             -3  4

Then AB and BA are both defined, AB is a 2x2 matrix, whereas BA is 3x3.

                 -8  38              27 -2 67

       Also AB =         and BA  =    6 -8 10

                 -5  40              -1 24 13

Lyapunov Function

A Lyapunov function is a function that decreases with time, taking on non-negative values. It is used to correspond between the state variables of a system and real numbers. The state of the system changes as time changes, and the function decreases. Thus, the Lyapunov function decreases with each change of state of the system.

We can construct a simple example of a function with the property of decreasing with each change of state as follows. Suppose a real number, x, represents the state of a dynamic system at time t. Also suppose that x is bounded for any t by a positive real number M. That means x is less than M for every value of t.

Then the function,

       f(x,t) = exp(-|x|/(M+|x|+t))

is non-negative and decreases with increasing t.

Local Minimum

A function f(x) is defined to have a local minimum at y, with a value z, if

       f(y) = z, and f(x) ≥ z, for each x, such that there exists

       a positive real number h such that y – h ≤ x ≤ y + h.

In other words, there is no other value of x in a neighborhood of y, where the value of the function is smaller than z.

There can be more than one local minimum for a function in its domain. A Step function (with a graph resembling a staircase) is a simple example of a function with an infinite number of points in its domain with local minima.

Global Minimum

A function f(x) is defined to have a global minimum at y, with a value z, if

       f(y) = z, and f(x) ≥ z, for each x in the domain of

       the function f.

In other words, there is no other value of x in the domain of the function f, where the value of the function is smaller than z. Clearly, a global minimum is also a local minimum, but a local minimum may not be a global minimum.

There can be more than one global minimum for a function in its domain. The trigonometric function f(x) = sinx is a simple example of a function with an infinite number of points with global minima. You may recall that sin(3π/ 2), sin (7π/ 2), and so on are all –1, the smallest value for the sine function.

Kronecker Delta Function

The Kronecker delta function is a function of two variables. It has a value of 1 if the two arguments are equal, and 0 if they are not. Formally,

                       1 if x = y

       δ(x,y)=

                       0 if x ≠ y

Gaussian Density Distribution

The Gaussian Density distribution, also called the Normal distribution, has a density function of the following form. There is a constant parameter c, which can have any positive value.

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

References

Ahmadian, Mohamad, and Pimmel, Russell, "Recognizing Geometrical Features of Simulated Targets with Neural Networks," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 409–411.
Aleksander, Igor, and Morton, Helen, An Introduction to Neural Computing, Chapman and Hall, London, 1990.
Aiken, Milam, "Forecasting T-Bill Rates with a Neural Net," Technical Analysis of Stocks and Commodities, May 1995, Technical Analysis Inc., Seattle.
Anderson, James, and Rosenfeld, Edward, eds., Neurocomputing: Foundations of Research, MIT Press, Cambridge, MA, 1988.
Anzai, Yuichiro, Pattern Recognition and Machine Learning, Academic Press, Englewood Cliffs, NJ, 1992.
Azoff, E. Michael, Neural Network Time Series Forecasting of Financial Markets, John Wiley & Sons, New York, 1994.
Bauer, Richard J., Genetic Algorithms and Investment Strategies, John Wiley & Sons, New York, 1994.
Booch, Grady, Object Oriented Design with Applications, Benjamin-Cummings, Redwood City, CA, 1991.
Carpenter, Gail A., and Ross, William D., "ART-EMAP: A Neural Network Architecture for Object Recognition by Evidence Accumulation," IEEE Transactions on Neural Networks, Vol. 6., No. 4, July 1995, pp. 805–818.
Colby, Robert W., and Meyers, Thomas A., The Encyclopedia of Technical Market Indicators, Business One Irwin, Homewood, IL, 1988.
Collard, J. E., "Commodity Trading with a Three Year Old," Neural Networks in Finance and Investing, pp. 411–420, Probus Publishing, Chicago, 1993.
Cox, Earl, The Fuzzy Systems Handbook, Academic Press, Boston, 1994.
Dagli, Cihan, et al., eds., Intelligent Engineering Systems through Artificial Neural Networks, Volume 2, ASME Press, New York, 1992.
Davalo, Eric, and Naim, Patrick, Neural Networks, MacMillan, New York, 1991.
Ercal, F., et al., "Diagnosing Malignant Melanoma Using a Neural Network," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 553–555.
Frank, Deborah, and Pletta, J. Bryan, "Neural Network Sensor Fusion for Security Applications," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 745–748.
Freeman, James A., and Skapura, David M., Neural Networks Algorithms, Applications, and Programming Techniques, Addison-Wesley, Reading, MA, 1991.
Gader, Paul, et al., "Fuzzy and Crisp Handwritten Character Recognition Using Neural Networks," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 421–424.
Ganesh, C., et al., "A Neural Network-Based Object Identification System," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 471–474.
Glover, Fred, ORSA CSTS Newsletter Vol 15, No 2, Fall 1994.
Goldberg, David E., Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, MA, 1989.
Grossberg, Stephen, et al., Introduction and Foundations, Lecture Notes, Neural Network Courses and Conference, Boston University, May 1992.
Hammerstrom, Dan, "Neural Networks at Work," IEEE Spectrum, New York, June 1993.
Hertz, John, Krogh, Anders, and Palmer, Richard, Introduction to the Theory of Neural Computation, Addison-Wesley, Reading, MA, 1991.
Jagota, Arun, "Word Recognition with a Hopfield-Style Net," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 445–448.
Johnson, R. Colin, "Accuracy Moves OCR into the Mainstream," Electronic Engineering Times, CMP Publications, Manhasset, NY, January 16, 1995.
Johnson, R. Colin, "Making the Neural-Fuzzy Connection," Electronic Engineering Times, CMP Publications, Manhasset, NY, September 27, 1993.
Johnson, R. Colin, "Neural Immune System Nabs Viruses," Electronic Engineering Times, CMP Publications, Manhasset, NY, May 8, 1995.
Jurik, Mark, "The Care and Feeding of a Neural Network," Futures Magazine, Oster Communications, Cedar Falls, IA, October 1992.
Kimoto, Takashi, et al., "Stock Market Prediction System with Modular Neural Networks," Neural Networks in Finance and Investing, pp. 343–356, Probus Publishing, Chicago, 1993.
Kline, J., and Folger, T.A., Fuzzy Sets, Uncertainty and Information, Prentice Hall, New York, 1988.
Konstenius, Jeremy G., "Trading the S&P with a Neural Network," Technical Analysis of Stocks and Commodities, October 1994, Technical Analysis Inc., Seattle.
Kosaka, M., et al., "Applications of Fuzzy Logic/Neural Network to Securities Trading Decision Support System," Conference Proceedings of the 1991 IEEE International Conference on Systems, Man and Cybernetics, V.3, pp. 1913–1918.
Kosko, Bart, and Isaka, Satoru, Fuzzy Logic, Scientific American, New York, July 1993.
Kosko, Bart, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence, Prentice-Hall, New York, 1992.
Laing, Jonathan, "New Brains: How Smart Computers are Beating the Stock Market," Barron's, February 27, 1995.
Lederman, Jess, and Klein, Robert, eds., Virtual Trading, Probus Publishing, Chicago, 1995.
Lin, C.T. and Lee, C.S.G, "A Multi-Valued Boltzman Machine", IEEE Transactions on Systems, Man, and Cybernetics, Vol. 25, No. 4, April 1995 pp. 660-668.
MacGregor, Ronald J., Neural and Brain Modeling, Academic Press, Englewood Cliffs, NJ, 1987.
Mandelman, Avner, "The Computer's Bullish! A Money Manager's Love Affair with Neural Network Programs," Barron's, December 14, 1992.
Maren, Alianna, Harston, Craig, and Pap, Robert, Handbook of Neural Computing Applications, Academic Press, Englewood Cliffs, NJ, 1990.
Marquez, Leorey, et al., "Neural Network Models as an Alternative to Regression," Neural Networks in Finance and Investing, pp. 435–449, Probus Publishing, Chicago, 1993.
Mason, Anthony et al., "Diagnosing Faults in Circuit Boards—A Neural Net Approach," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 839–843.
McNeill, Daniel, and Freiberger, Paul, Fuzzy Logic, Simon & Schuster, New York, 1993.
McNeill, F. Martin, and Thro, Ellen, Fuzzy Logic: A Practical Approach, Academic Press, Boston, 1994.
Murphy, John J., Intermarket Technical Analysis, John Wiley & Sons, New York, 1991.
Murphy, John J., Technical Analysis of the Futures Markets, NYIF Corp., New York, 1986.
Nellis, J., and Stonham, T.J., "A Neural Network Character Recognition System that Dynamically Adapts to an Author's Writing Style," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 975–979.
Peters, Edgar E., Fractal Market Analysis: Applying Chaos Theory to Investment and Economics, John Wiley & Sons, New York, 1994.
Ressler, James L., and Augusteijn, Marijke F., "Weapon Target Assignment Accessibility Using Neural Networks," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 397–399.
Rao, Satyanaryana S,, and Sethuraman, Sriram, "A Modified Recurrent Learning Algorithm for Nonlinear Time Series Prediction," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 725–730.
Rao, Valluru, and Rao, Hayagriva, Power Programming Turbo C++, MIS:Press, New York, 1992.
Ruggiero, Murray, "How to Build an Artificial Trader," Futures Magazine, Oster Communications, Cedar Falls, IA, September 1994.
Ruggiero, Murray, "How to Build a System Framework," Futures Magazine, Oster Communications, Cedar Falls, IA, November 1994.
Ruggiero, Murray, "Nothing Like Net for Intermarket Analysis," Futures Magazine, Oster Communications, Cedar Falls, IA, May 1995.
Ruggiero, Murray, "Putting the Components before the System," Futures Magazine, Oster Communications, Cedar Falls, IA, October 1994.
Ruggiero, Murray, "Training Neural Networks for Intermarket Analysis," Futures Magazine, Oster Communications, Cedar Falls, IA, August 1994.
Rumbaugh, James, et al, Object-Oriented Modeling and Design, Prentice-Hall Inc, Englewood Cliffs, New Jersey, 1991.
Sharda, Ramesh, and Patil, Rajendra, "A Connectionist Approach to Time Series Prediction: An Empirical Test," Neural Networks in Finance and Investing, pp. 451–463, Probus Publishing, Chicago, 1993.
Sherry, Clifford, "Are Your Inputs Correlated?" Technical Analysis of Stocks and Commodities, February 1995, Technical Analysis Inc., Seattle.
Simpson, Patrick K., Artificial Neural Systems: Foundations, Paradigms, Applications and Implementations, Pergamon Press, London, 1990.
Soucek, Branko, and Soucek, Marina, Neural and Massively Parallel Computers: The Sixth Generation, John Wiley & Sons, New York, 1988.
Stein, Jon, "The Trader's Guide to Technical Indicators," Futures Magazine, Oster Communications, Cedar Falls, IA, August 1990.
Terano, Toshiro, et al., Fuzzy Systems Theory and Its Applications, Academic Press, Boston, 1993.
Trippi, Robert, and Turban, Efraim, eds., Neural Networks in Finance and Investing, Probus Publishing, Chicago, 1993.
Wasserman, Gary S., and Sudjianto, Agus, Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 901–904.
Wasserman, Philip D., Advanced Methods in Neural Computing, Van Nostrand Reinhold, New York, 1993.
Wasserman, Philip D., Neural Computing, Van Nostrand Reinhold, New York, 1989.
Wu, Fred, et al., "Neural Networks for EEG Diagnosis," Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, V.3, pp. 559–565.
Yager, R., ed., Fuzzy Sets and Applications: Selected Papers by L.Z. Zadeh, Wiley-Interscience, New York, 1987.
Yan, Jun, Ryan, Michael, and Power, James, Using Fuzzy Logic, Prentice-Hall, New York, 1994.
Yoon, Y., and Swales, G., "Predicting Stock Price Performance: A Neural Network Approach," Neural Networks in Finance and Investing, pp. 329–339, Probus Publishing, Chicago, 1993.

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Index

A

ABAM see Adaptive Bi-directional Associative Memory
abstract class, 138
abstract data type, 22
accumulation-distribution, 402, 404,
activation, 3, 18, 89
zero, 182
Adaline, 81, 82, 102, 103, 112
adaptation, 77, 120
Adaptive Bi-directional Associative Memory, 181, 212
Competitive, 212
Differential
Competitive, 212
Hebbian, 212
adaptive filters, 120
adaptive linear element, 102
adaptive models, 120
adaptive parameters, 373
adaptive resonance, 118
Adaptive Resonance Theory I, 104, 107, 108, 115, 117, 243
algorithm for calculations, 246
equations for, 246
F1 layer calculations, 247
F2 layer calculations, 247
modifying connection weights, 248
Top-down inputs, 247
two-thirds rule, 244, 245
Adaptive Resonance Theory II, 248
Adaptive Resonance Theory III, 248
additive composition method, 506
advancing issues, 387
aggregation, 82, 87, 98
Ahmadian, 513
Aiken, 405
algorithm, 1
backpropagation, 7, 103, 271, 373, 375
constructive, 121
data clustering, 245
encoding, 94, 96
learning algorithm , 61, 79, 102, 118
adaptive steepest descent, 410
alpha, 273, 274, 330, 372, 373, 384
alphabet classifier system, 320
Amari, 104
analog, 73, 74, 322
signal, 8
AND operation , 64
Anderson, 105
annealing
process, 430
schedule 113
simulated annealing, 113, 114
Anzai, 456, 472
application, 102, 374, 511
nature of , 74
approximation, 109
architecture, 7, 81, 120, 373
Artificial Intelligence , 6, 34
artificial neural network , 1
ART1 see Adaptive Resonance Theory I
ART2 see Adaptive Resonance Theory II
ART3 see Adaptive Resonance Theory III
artneuron class, 249
ASCII, 305, 306, 307, 329
graphic characters, 306
assocpair class
in BAM network, 186
association, 218
asynchronously , 1, 14
asynchronous update, 13, 62
attentional subsystem, 107, 243
Augusteijn, 512
autoassociation , 7, 8, 82,, 92, 102, 180
autoassociative network, 7, 64, 97, 375
average winner distance, 296
Azoff, 410

B

backpropagated errors, 144
Backpropagation, 10, 103, 104, 120, 123, 302, 325, 329, 374
algorithm, 7, 103, 271, 373, 375
beta, 330
calculating error, 396
calculations, 128, 130
changing beta while training, 337
choosing middle layer size, 372
convergence, 372
momentum term, 330
noise factor, 336
self-supervised, 375
simulator, 134, 173, 337, 375, 377, 396
training and testing, 396
training law, 333
variations of , 373
Baer, 516
BAM see Bi-directional Associative Memory
bar
chart, 403, 404
features, 513
Barron's, 377, 378, 388
base class , 25, 28, 138
beta, 136, 337, 372, 373, 384
bias, 16, 77, 125, 128, 378
Bi-directional Associative Memory, 81, 92, 104, 115, 117, 179, 185, 215
connection weight matrix, 212
continuous, 211
inputs, 180
network, 104
outputs, 180
training, 181
Unipolar Binary, 212
bin, 325
binary , 8, 15, 51, 65, 73, 74, 104
input pattern, 51, 98
patterns 11, 97
string, 16, 62
binary to bipolar mapping, 62, 63
binding
dynamic binding , 24, 139
late binding , 24
static binding, 139
bipolar, 15, 17, 104
mapping, 97
string, 16, 62, 180
bit pattern, 13
Blending problem, 418
block averages, 393
bmneuron class, 186
Boltzmann distribution, 113
Boltzmann machine, 92, 112, 113, 118, 419, 512
Booch, 21
boolean logic, 50
bottom-up
connection weight, 248
connections, 107, 244
Box-Jenkins methodology, 406
Brain-State-in-a-Box, 81, 82, 105
breadth, 387, 389
Buckles, 484
buy/sell signals, 409, 410

C

cache, 137
Cader, 406
CAM see Content-Addressable-Memory
Carpenter, 92, 107, 117, 243, 269, 517
car navigation, 374
cartesian product, 479
cascade-correlation, 512
Cash Register game, 3, 65
categorization of inputs, 261
category, 37
Cauchy distribution, 113
Cauchy machine, 112, 113, 419
cells
complex cells, 106
simple cells, 106
center of area method, 504
centroid, 507, 508
character recognition, 305
characters
alphabetic, 305
ASCII, 306, 307
garbled, 322
graphic, 306, 307
handwritten, 320
Chawla, 514
Chiang, 513
cin, 25, 58, 71
clamping probabilities, 114
Clinton, Hillary, 405
C language, 21
class, 22
abstract class, 138
base class, 25, 28, 138, 139
derived class, 23, 25, 26, 144
friend class, 23
hierarchy, 27, 138, 139
input_layer class, 138
iostream class, 71
network class, 53, 66
output_layer class, 138
parent class, 26
classification, 322
C layer, 106
codebook vectors, 116
Cohen, 212
Collard, 405
column vector, 97
combinatorial problem, 422
comparison layer, 244
competition, 9, 94, 97
competitive learning, 243
compilers
C++ compilers, 27
compilation error messages, 27
complement, 33, 185, 201, 202
complex cells, 106
composition
max-min, 220
compressor, 375
Computer Science, 34
conditional fuzzy
mean, 491
variance, 491
conjugate gradient methods, 373
conjunction operator, 505
connections, 2, 93, 102
bottom-up, 107
excitatory, 272
feedback , 82
inhibitory , 272
lateral, 93, 97, 107, 272, 276
recurrent, 82, 107, 179
top-down, 107
connection weight matrix, 220
connection weights, 89, 98
conscience factor, 302
constraints, 417
constructor, 23, 28, 55, 66
default constructor, 23
Consumer Price Index, 387
Content-Addressable-Memory, 5
continuous
Bi-directional Associative Memory, 211
models, 98
convergence, 78, 79, 96, 118, 119, 132, 323, 372, 373, 375
cooperation, 9, 94
correlation matrix, 9, 63
correlation-product encoding, 220
cost function, 124, 373
Cottrell, 374
counterpropagation, 106
network, 92, 93, 302
cout, 25, 58
C++, 21
classes, 138
code, 36
comments, 58
compilers, 27
implementation, 185
crisp, 31, 73, 74
data sets, 475
rules, 48, 217
values, 50
cube, 84, 87, 89, 90
cum_deltas, 331
cycle, 78, 125
learning cycle, 103
cyclic information, 380

D

data biasing, 378
data hiding, 21, 22
data clustering, 109, 245
data completion, 102
data compression, 102, 302
Deboeck, 406
Davalo, 457
de la Maza, 410
Decision support systems, 75
declining issues, 387
decompressor, 375
decorrelation, 384
default
constructor, 23, 66
destructor, 23
defuzzification, 504, 506
degenerate tour, 424
degree of
certainty, 31
membership, 32, 477
degrees of freedom, 383
delete, 24, 144
delta rule, 110-113
derived class, 23, 25, 26, 144
descendant, 139, 143
DeSieno, 302
destructor, 23, 24
digital signal processing boards, 325
dimensionality, 381, 382, 384
directed graph, 65
discount rate, 35
discretization of a character, 98
discrete models, 98
discriminator, 517
disjunction operator, 506
display_input_char function, 308
display_winner_weights function, 308
distance
Euclidean, 13
Hamming, 13
DJIA see Dow Jones Industrial Average
DOF see degrees of freedom
domains, 479, 484
dot product, 11, 12, 51, 64, 65
Dow Jones Industrial Average, 378, 386
dual confirmation trading system, 408
dynamic allocation of memory, 24
dynamic binding, 24, 139
dynamics
adaptive, 74
nonadaptive, 74

E

EMA see exponential moving average
encapsulate, 29
encapsulation, 21, 22
encode, 375
encoding, 7, 81, 220
algorithm, 94, 96
correlation-product, 220
phase, 94
thermometer, 380
time, 120
energy, 422
function, 119
level, 113, 422
surface, 177
Engineering, 34
epoch, 125
Ercal, 514
error signal, 103
error surface, 113
error tolerance, 136
Euclidean distance, 13, 280, 296
excitation, 94, 98, 276, 303
excitatory connections, 244, 272
exclusive or, 83
exemplar, 181-183, 201
class in BAM network, 186
pairs, 135, 177
patterns, 135, 177
exemplar pattern, 16, 64
exemplars, 64, 65, 74, 75, 115, 181
Expert systems, 48, 75, 217
exponential moving average, 399
extended (database) model, 486
extended-delta-bar-delta, 406

F

factorial, 420
FAM see Fuzzy Associative Memories
Fast Fourier Transform, 374
fault tolerance, 374
feature detector, 328
feature extraction, 7, 513
Federal Reserve, 388, 388
feedback, 4, 5, 93
connections, 123, 179
feed forward
Backpropagation, 81, 92, 112, 115, 123, 384,
network , 145, 406, 409, 511,
architecture, 124,
layout, 124,
network, 10
operation, 185
type, 2
field, 82
filter, 322
financial forecasting, 377
fire, 3, 71, 87, 89
first derivative, 380
fiscal policy, 36
fit values, 32, 217
fit vector, 32, 217, 221
floating point calculations, 519
compilation for, 519
F1 layer, 245
calculations, 247
Fogel, 76, 77
forecasting, 102
model, 378
T-Bill yields, 405
T-Note yields, 405
forward, 93, 104
Fourier transform, 380
Frank, 515
Freeman, 246, 248
frequency
component, 380
signature, 374
frequency spikes, 380
friend class, 23, 66, 68
F2 layer, 245
calculations, 247
Fukushima, 92, 106
function
constructor function, 28
energy function, 119
evaluation, 109
fuzzy step function, 101
hyperbolic tangent function, 100
linear function, 99, 102
logistic function, 86, 100
Lyapunov function, 118, 119
member function, 28, 308
objective, 417
overloading, 25, 139
ramp function, 99, 101, 102
reference function, 493
sigmoid
function, 99, 100, 126, 129, 133, 164, 177, 395
logistic function, 100
step function, 99, 101
threshold function, 52, 95, 99, 101
XOR function, 83-85, 87
fuzzifier, 35, 36, 47
program, 50
fuzziness, 48, 50
fuzzy adaptive system, 49
fuzzy ARTMAP, 517
fuzzy association, 217
Fuzzy Associative Memories, 49, 50, 81, 92, 104, 115, 117, 217, 218, 473
encoding, 219, 220
Fuzzy Cognitive Map, 48, 49
fuzzy conditional expectations, 490, 509
fuzzy control , 497, 509
system, 47, 48, 473
fuzzy controller, 47
fuzzy database, 473, 475, 509
fuzzy expected value, 490
fuzzy equivalence relation, 481
fuzzy events, 488, 509
conditional probability of, 491
probability of , 490
fuzzy inputs, 47, 73
fuzzy logic, 31, 34, 50, 473
controller, 473, 497, 509
fuzzy matrix, 217
fuzzy means, 488, 490, 509
fuzzy numbers, 493, 496
triangular, 496
fuzzy OR method, 505
fuzzy outputs, 47, 74
fuzzy quantification, 488
fuzzy queries, 483, 488
fuzzy relations, 479, 509
matrix representation, 479
fuzzy rule base, 502-504
fuzzy rules, 47, 50
fuzzy set, 32, 50, 218, 477, 488
complement, 218
height, 218
normal, 218
operations, 32, 217
fuzzy systems, 50, 217
fuzzy-valued, 34
fuzzy values, 477
fuzzy variances, 488, 490, 509
fzneuron class, 221

G

Gader, 513
gain , 107
constant, 273
parameter, 429, 467
gain control, 243, 248
unit, 244
Gallant, 117
Ganesh, 514
Gaussian density function, 458, 459, 524
generalization, 121, 320, 382
ability, 121, 336
generalized delta rule, 112, 176
genetic algorithms, 75, 76, 385
global
minimum, 113, 177
variable, 28
Glover, 471
gradient, 112, 113
grandmother cells, 117
gray scale, 305, 306, 322, 374
grid, 214, 305
Grossberg, 19, 92, 93, 107, 117, 243, 269
Grossberg layer, 9, 19, 82, 92, 106, 302

H

Hamming distance, 13, 201, 202
handwriting analysis, 98
handwriting recognition, 102
handwritten characters, 92
heap, 144
Hebb, 110
Hebbian
conditioned learning, 302
learning, 105, 110
Hebb's rule, 110, 111
Hecht-Nielsen, 93, 106, 302
Herrick Payoff Index, 401
heteroassociation, 7, 8, 82, 92, 102, 104, 180, 181
heteroassociative network, 7, 97
hidden layer, 2, 4, 75, 86, 89
hierarchical neural network, 407
hierarchies of classes, 27, 29
Hinton, 114
Hoff, 102, 112
holographic neural network, 408
Honig, 515
Hopfield, 422, 427, 429
memory, 73, 115, 117, 181
Hopfield network, 9, 11-14, 16, 19, 51, 79, 81, 82, 93, 111, 119, 120, 181, 472
Hotelling transform, 384
Housing Starts, 387
hybrid models, 75
hyperbolic tangent function, 429
hypercube, 218
unit, 218
hyperplane, 84
hypersphere, 273

I

image, 106, 302
compression, 374
processing, 98, 102
five-way transform, 516
recognition, 375
resolution, 322
implementation of functions, 67
ineuron class, 66
inference engine, 47
inheritance, 21, 25, 26, 138
multiple inheritance, 26
inhibition, 9, 94, 98
global, 428, 456
lateral, 272, 276
inhibitory connection, 272
initialization of
bottom-up weights, 250
parameters, 246
top-down weights, 250
weights, 94
inline, 165
input, 98
binary input, 98
bipolar input, 98
layer, 2, 10
nature of , 73
number of , 74
patterns, 51, 65
signals, 65
space, 124
vector, 53, 71, 272, 112
input/output, 71
inqreset function, 251
instar, 93
interactions, 94
interconnections, 7
interest rate, 387
internal activation , 3
intersection, 32, 33
inverse mapping, 62, 182
Investor's Business Daily, 388
iostream, 54, 71
istream, 58
iterative process, 78

J

Jagota, 514
January effect, 380
Jurik, 381, 384, 392

K

Karhunen-Loev transform, 384
Katz, 377
Kimoto, 408
kohonen.dat file, 275, 298, 300, 317
Kohonen, 19, 116, 117, 245, 271, 303, 456
Kohonen feature map, 16, 271, 273, 303, 305, 323
conscience factor, 302
neighborhood size, 280, 299, 300
training law, 273
Kohonen layer, 9, 19, 82, 92, 106, 298, 302, 322
class, 276
Kohonen network, 275, 276, 280, 300, 303, 322
applications of, 302,
Kohonen output layer, 275
Kohonen Self-Organizing Map, 115, 456, 471, 472
Kosaka, 409,
Kosko, 49, 50, 104, 215, 242, 506
Kostenius, 408, 409
Kronecker delta function 428, 524

L

lambda, 136, 433
late binding, 24
lateral, 93
lateral competition, 303
laterally connected, 65
lateral connections, 93, 97, 107, 272, 276
lateral inhibition, 272, 276
layer, 2, 81
C layer, 106
comparison, 244
complex layer, 106
F1, 244
F2, 244
Grossberg layer, 82, 92, 302
hidden layer, 75, 81, 86, 89
input layer, 2, 3, 82
Kohonen layer, 82, 92, 302, 322
middle layer, 329, 372
output layer, 2, 82
recognition, 244
S layer, 106
simple layer, 106
layout, 52, 86, 124
ART1, 244
BAM , 180
Brain-State-in-a-Box, 105
FAM, 219
Hopfield network, 11
for TSP, 427
LVQ, 117
Madaline model, 103
LBS Capital Management, 377
learning, 4, 74, 98, 109, 110, 117, 118
algorithm, 61, 79, 102, 118
cycle, 103
Hebbian, 105, 110
one-shot, 117
probabilistic, 113
rate(parameter), 111, 112, 123, 125, 127, 136, 175
supervised learning, 5, 110, 112, 117, 121
time, 120
unsupervised
competitive learning, 271
learning, 5, 110, 117, 121
Learning Vector Quantizer, 115-117, 302
least mean squared error, 111, 119, 123, 419
rule, 111
Le Cun, 375
Lee, 512
Levenberg-Marquardt optimization, 373
Lewis, 377
Lin, 512
linear function, 99, 102
linear possibility regression model, 493, 496, 509
linear programming, 417
integer, 417
linearly separable, 83- 85
LMS see least mean squared error rule
local minimum, 113, 177, 325
logic
boolean logic, 50
fuzzy logic, 31, 34, 50, 473
logical operations, 31
AND, 64
logistic function, 86, 100
Long-term memory, 6, 77- 79, 118, 472
traces of, 243
look-up
memory, 5
table, 106
LTM see Long-term memory
LVQ see Learning Vector Quantizer
Lyapunov Function, 118, 119

M

MACD see moving average convergence divergence
Madaline, 102, 103
main diagonal, 63, 480
malignant melanoma, 514
malloc, 24
Mandelman, 378
MAPE see mean absolute percentage error
mapping, 123, 180
binary to bipolar, 62, 63
inverse, 62, 182
nonlinear, 109
real to binary, 180
mapping surface, 109
Markowitz, 470
Marquez, 406
Mason, 516
matrix, 97, 521
addition, 521
correlation matrix, 9
fuzzy, 217
multiplication, 11
product, 104, 522
transpose, 11
weight matrix, 97
max_cycles, 326
maximum, 33, 219
max-min composition, 220, 221
McClelland, 103
McCulloch, 6
McNeill, 508
mean absolute percentage error, 406
mean squared error, 111, 410
Mears, 212, 214
membership, 32
functions, 50
triangular, 499, 506
rules, 49
memorization, 121, 320, 336, 382, 397
memorizing inputs, 273, 320
memory, 98
adaptive, 471
fuzzy associative, 218, 221
long-term memory, 6, 77-79, 118, 472
recency based, 471
short-term memory, 6, 77, 78, 79, 107, 118, 471
Mendelsohn, 407, 408
methods, 22, 145
metric, 5, 103
mexican hat function, 272, 273, 274, 276
middle layer, 329, 372
choosing size 372
minimum, 33, 219
global, 113, 177
local, 113, 177, 325
Minsky, 83
model
ART1, 245
continuous, 98
discrete, 98
Perceptron model, 65, 68, 81, 83
modulo, 423
Mohamed, 513
momentum, 325, 330, 337, 372, 400
implementing, 331
parameter, 119, 123, 134, 384
term, 330, 375
Morse, 514
Moss, 514
moving average convergence divergence, 401
moving averages, 380, 399
simple, 399
weighted, 399
multilayered, 92
network, 106
multilayer feed-forward neural network, 7
multilayer networks, 123
multiple inheritance, 26
Munro, 374

N

NF see noise factor
NP-complete, 419, 427, 457
NYSE see New York Stock Exchange
Naim, 457
neighborhood, 276, 303, 457
size, 274, 280, 299, 300
neighbors, 272
Neocognitron, 81, 92, 106
Nellis, 516
NETTalk, 374
network
adaptive, 77
architecture, 77, 384, 388
autoassociative network, 97
backpropagation network, 329
bi-directional associative memory network, 104,, 88
Brain-State-in-a-Box network, 105
class, 53, 66
heteroassociative networks, 97
Hopfield network, 9, 11-14, 16, 19, 51, 79, 81, 82, 93, 111, 119, 120, 181, 472
layout, 86
modeling, 73
multilayer network, 123
nodes, 65
Perceptron network, 65, 66, 68, 79
radial basis function networks, 112, 114, 115
RBF networks see radial basis function networks
self-organizing , 269
statistical trained networks, 112
NeuFuz, 49
neural network, 1, 2
algorithms, 176
artificial neural network, 1
autoassociative, 375
biological, 272
counterpropagation, 302
fault tolerance of, 374
FAM, 218
hierarchical, 407
holographic, 408
Kohonen, 271, 322
multilayer, 123
Perceptron, 65
plug-in boards, 325
self-organizing, 107, 269
Tabu, 471
two-layer, 92
neural-trained fuzzy systems, 49
neuromimes 2
neuron, 1, 3
input neurons, 82
output neuron, 99
new, 24, 144
Newton's method, 373
New York Stock Exchange, 387
new highs, 389
new lows, 389
noise, 5, 330, 336, 337, 372, 375
random, 375
noise factor, 336
noise-saturation dilemma, 79
noise tolerance, 105
noisy data, 320
nonlinear function, 120
nonlinear mapping, 109
nonlinear scaling function, 10
nonlinear optimization, 417, 422, 472
nontraining mode, 135
Normal distribution, 524
normal fuzzy set, 218
normalization of a vector, 272
normalized inputs, 271, 381
normalized weights, 280
notation, 132
nprm parameter, 433
number bins, 43
NYSE see New York Stock Exchange

O

object, 22
objective function, 417
object-orientation, 21
object-oriented programming language, 21
Object-Oriented Analysis and Design, 21
on-balance volume, 402
on center, off surround, 97, 98
one-shot learning, 117
oneuron class, 66
Operations Research, 34
operator overloading, 25, 139
optical character
recognition, 245
recognizer, 514
optimization, 102, 109, 417
nonlinear, 417, 422, 472
stock portfolio, 470
ordered pair, 32
organization of layers for backpropagation program, 144
orienting subsystem, 107, 243
orthogonal, 11, 12, 51, 64, 98
bit patterns, 64
input vectors, 299
set, 65
ostream, 58
output, 99
layer, 2, 10
nature of , 74
number of , 74
space, 124
stream, 58
outstar, 93, 106
overfitting of data, 383
overlap composition method, 506
overloading, 21, 24
function overloading, 25, 139
operator overloading, 25, 139
overtrained network, 329

P

Papert, 83
Parker, 103
partitioning, 87, 88
past_deltas, 331
Patil, 406
pattern
association, 16
binary pattern, 11, 97
bit pattern, 99
character, 17
classification, 8, 98, 109
completion, 105, 109
matching, 8, 98
recognition, 16, 34, 102, 108, 305, 322
studies, 380
system, 121
spatial, 99, 214
Pavlovian, 5
Perceptron, 3, 4, 66, 73, 82, 87, 90, 93, 102, 112
model, 65, 68, 81, 83
multilayer Perceptron, 85, 88
network, 65, 66, 68, 79
single-layer Perceptron, 85
permutations, 420
Perry, 484
perturbation , 5, 113
phoneme, 303, 374
phonetic typewriter, 303
Pimmel, 513
Pitts, 6
pixel, 16, 322, 329, 374
values, 214, 305
plastic, 107
plasticity, 78, 79
plasticity-stability dilemma, 243
Pletta, 515
polymorphic function, 24, 28
polymorphism, 21, 24, 27, 138
Pomerleau, 374
portfolio selection, 470, 472
possibility distributions, 486, 487, 509
relational model, 486, 487
postprocess, 35
postprocessing, 50
filter , 50
potlpair class
in BAM network, 186
preprocess, 35, 50
preprocessing , 87, 379, 399
data, 389
filter, 50
fuzzifier, 35
Price is Right, 3, 65
principal component analysis, 384
private, 23, 26
probabilities, 31, 419
probability, 31, 43
distributions, 113
processing
additive, 75
hybrid, 75
multiplicative, 75
PROJECT operation, 485
proposition, 31
protected, 23, 26, 54, 143
public, 23, 26, 53, 54

Q

quadratic form, 119, 120, 418
quadratic programming problem, 418
quantification, 473, 475
quantization levels, 322
queries, 475, 476, 488
fuzzy, 483, 488

R

radial basis function networks, 112, 114, 115
ramp function, 99, 101,, 102
Ramsay, 515
random number generator, 37
range, 394
normalized, 394, 395
rate of change, 392, 400, 404
function, 392
indicator, 393, 394
real-time recurrent learning algorithm, 515
recall, 7, 81, 184, 220
BAM , 184
FAM, 220, 221
recency based memory, 471
recognition layer, 244
recurrent, 13, 179
recurrent connections, 82, 107, 179
reference
activation level, 456
function, 493
regression analysis, 406
risk-adjusted return, 410
relations, 476
antisymmetric, 480, 481
reflexive, 480, 481
resemblance, 509
similarity, 481, 509
symmetric, 480, 481
transitive, 480, 481
relative strength index, 400, 404
remainder, 423
reset, 243, 247, 251, 262
reset node, 244
reset unit, 244
resonance, 104, 107, 117, 118, 181, 215, 243, 269
responsive exploration, 471
Ressler, 512
restrmax function, 251
return type, 23
reuse, 26
Robotics, 34
ROC see rate of change
Rosenberg, 374
Ross, 517
row vector, 97, 104
RSI see relative strength index
rule
delta, 110, 111
generalized delta, 112
Hebbian, 111
Hebb's, 110
rules
fuzzy rules, 50
Rumbaugh, 21
Rummelhart, 103

S

S&P 500 Index see Standard and Poor's 500 Index
Sathyanarayan Rao, 515
saturate, 381
scalar, 61
scalar multiplier, 64
second derivative, 380
Sejnowski, 114, 374
self-adaptation, 115
self-connected, 53
self-organization, 5, 6, 74, 115, 116
self-organizing feature map, 116
Self-Organizing Map, 245, 271
self-organizing neural networks, 107, 117, 121
self-supervised backpropagation, 375
sensitivity analysis, 384
separable, 84, 86, 88
linearly separable, 83, 84
subsets, 87
separability, 84, 86
separating
line, 86
plane, 85
Sethuraman, 515
set membership, 32
Sharda, 406
shift operator, 25
Short Term Memory, 6, 77, 78, 79, 107, 118, 471
traces of, 243
Sigma Pi neural network , 75
sigmoid
activation function, 381, 387
function, 77, 99, 126, 129, 133, 164, 177, 395
squashing, 381
signal filtering, 102
signals
analog, 98
similarity, 486
class, 481, 509
level, 486
relation, 481
simple cells, 106
simple moving average, 399
simulated annealing, 113, 114
simulator, 372, 396
controls, 173
mode, 138
Skapura, 246, 248
Slater, 515
S layer, 106
SMA see simple moving average
SOM see Self-Organizing Map
sonar target recognition, 374
spatial
pattern, 99, 214
temporal pattern, 105,
speech
recognition, 303,
synthesizer, 374,
spike, 380
squared error, 103
squashing
function, 384, 458, 459
stable, 79, 107
stability 78, 79, 118
and plasticity, 77
stability-plasticity dilemma, 79, 107,, 269
STM see Short Term Memory
Standard and Poor's 500 Index, 377, 378
forecasting, 386
standard I/O routines, 519
state energy, 118
state machine, 48
static binding, 139
Steele, 514
steepest descent, 112, 113, 177, 373
step function, 99, 101
stochastics, 402, 404
Stoecker, 514
Stonham, 516
string
binary, 62
bipolar, 62
structure, 7, 7
subsample, 322
subset, 221
subsystem
attentional, 107, 243
orienting, 107, 243
Sudjianto, 516
summand, 422
summation symbol, 422
supervised , 109
learning, 5, 110, 112, 115, 117, 121
training 94, 110, 115, 121, 125
Sweeney , 516
symbolic approach, 6

T

TSR see Terminate and Stay Resident
Tabu , 471
active, 471
neural network, 471
search, 471, 472
Tank, 422, 427, 429
target 378, 395
outputs, 110, 115
patterns, 105
scaled, 395
tau, 433
technical analysis, 399
temperature, 118
Temporal Associative Memory, 92
Terano, 496
Terminate and Stay Resident programs, 519
terminating value, 298
termination criterion, 322
test.dat file, 327, 328
test mode, 135, 137, 138, 164, 173, 327, 396
Thirty-year Treasury Bond Rate, 387
Three-month Treasury Bill Rate, 387
threshold
function, 2, 3, 12, 17, 19, 52, 95, 99, 101, 125, 183
value, 16, 52, 66, 77, 86, 87, 90, 101, 128, 456
thresholding, 87, 185
function, 133, 177, 182, 184, 214
Thro, 508
Tic-Tac-Toe, 76, 79
time lag, 380
time series forecasting, 406, 410
time shifting, 395
timeframe, 378
tolerance, 119, 125, 173, 245, 318, 322, 328, 329, 372
level, 78, 123
value, 119
top-down
connection weight , 248
connections, 107, 244
top-down inputs, 247
topology, 7
Topology Preserving Maps, 116
tour, 420
traces, 243
of STM, 243
of LTM, 243
trading
commodities, 405,
system, 378
dual confirmation, 408
training, 4, 74, 75, 98, 109, 110, 119, 181, 396
fast, 107
law, 272, 273, 274, 330, 333
mode, 135, 137, 138, 164, 173, 396
supervised, 94, 110, 115
slow, 107
time, 329
unsupervised, 107, 110
transpose
of a matrix, 11, 179, 181, 183
of a vector, 11, 63, 97, 181
traveling salesperson(salesman) problem, 118, 119, 419
hand calculation, 423
Hopfield network solution-Hopfield, 427
Hopfield network solution-Anzai, 456
Kohonen network solution, 456
triple, 217
truth value, 31
tsneuron class, 430
TS see Tabu search
TSP see traveling salesperson problem
turning point predictor, 409
turning points, 407
two-layer networks, 92
two-thirds rule, 107, 244, 245, 269

U

Umano, 486
Unemployment Rate, 387
undertrained network, 329
uniform distribution, 77
union, 32
Unipolar Binary Bi-directional Associative Memory, 212
unit
circle, 299
hypercube, 218
unit length, 273
universal set, 33
universe of discourse, 498, 499
unsupervised , 107
competitive learning, 271
learning, 5, 110, 115, 117, 121
training, 107, 110

V

value
fit value, 32
threshold value, 16, 52, 66, 77, 86, 87, 90, 101, 128, 456
variable
external, 28
global, 28
vector, 17
codebook vectors, 116
column vector, 97, 104, 181
fit vector, 32, 33
heterassociated, 181
input vector, 53, 71, 272, 112
normalization of, 272
potentially associated, 181
quantization, 302
row vector, 97, 181
weight vector, 9, 96
vector pairs, 181
vertex, 88
vertices, 88
vigilance parameter, 107, 243, 245, 247, 262
virtual, 24, 139
trading, 377
visibility, 26
visible, 53

W

walk-forward methodology, 408
Wall Street Journal, 388
Wasserman, 516
weight matrix, 9, 17, 51, 65, 97, 181, 183
weight sharing, 375
weight surface, 113
weight update, 276
weight vector, 9, 96
quantizing, 307
weighted sum, 2, 3, 19, 271
weights , 4, 181
bottom-up, 250
connection , 89, 98
top-down, 250
Werbos, 103
Wetherell, 514
Widrow, 102, 112
winner indexes, 298
winner, 98
neuron, 323
winner-take-all, 97, 98, 115, 116, 243, 271, 274
World Wide Web, 388
Wu, 515

X

XOR function, 83-85, 87

Y

Yan, 473, 497
Yu, 212, 214
Yuret, 410

Z

zero pattern, 65
zero-one programming problem, 471
Zipser, 374

Table of Contents


C++ Neural Networks and Fuzzy Logic

 [Spisak knjiga]
C++ Neural Networks and Fuzzy Logic
Author: Valluru B. Rao

Preface
Dedication

Chapter 1—Introduction to Neural Networks
Neural Processing
Neural Network
Output of a Neuron
Cash Register Game
Weights
Training
Feedback
Supervised or Unsupervised Learning
Noise
Memory
Capsule of History
Neural Network Construction
Sample Applications
Qualifying for a Mortgage
Cooperation and Competition
Example—A Feed-Forward Network
Example—A Hopfield Network
Hamming Distance
Asynchronous Update
Binary and Bipolar Inputs
Bias
Another Example for the Hopfield Network
Summary
Chapter 2—C++ and Object Orientation
Introduction to C++
Encapsulation
Data Hiding
Constructors and Destructors as Special Functions of C++
Dynamic Memory Allocation
Overloading
Polymorphism and Polymorphic Functions
Overloading Operators
Inheritance
Derived Classes
Reuse of Code
C++ Compilers
Writing C++ Programs
Summary
Chapter 3—A Look at Fuzzy Logic
Crisp or Fuzzy Logic?
Fuzzy Sets
Fuzzy Set Operations
Union of Fuzzy Sets
Intersection and Complement of Two Fuzzy Sets
Applications of Fuzzy Logic
Examples of Fuzzy Logic
Commercial Applications
Fuzziness in Neural Networks
Code for the Fuzzifier
Fuzzy Control Systems
Fuzziness in Neural Networks
Neural-Trained Fuzzy Systems
Summary
Chapter 4—Constructing a Neural Network
First Example for C++ Implementation
Classes in C++ Implementation
C++ Program for a Hopfield Network
Header File for C++ Program for Hopfield Network
Notes on the Header File Hop.h
Source Code for the Hopfield Network
Comments on the C++ Program for Hopfield Network
Output from the C++ Program for Hopfield Network
Further Comments on the Program and Its Output
A New Weight Matrix to Recall More Patterns
Weight Determination
Binary to Bipolar Mapping
Pattern’s Contribution to Weight
Autoassociative Network
Orthogonal Bit Patterns
Network Nodes and Input Patterns
Second Example for C++ Implementation
C++ Implementation of Perceptron Network
Header File
Implementation of Functions
Source Code for Perceptron Network
Comments on Your C++ Program
Input/Output for percept.cpp
Network Modeling
Tic-Tac-Toe Anyone?
Stability and Plasticity
Stability for a Neural Network
Plasticity for a Neural Network
Short-Term Memory and Long-Term Memory
Summary
Chapter 5—A Survey of Neural Network Models
Neural Network Models
Layers in a Neural Network
Single-Layer Network
XOR Function and the Perceptron
Linear Separability
A Second Look at the XOR Function: Multilayer Perceptron
Example of the Cube Revisited
Strategy
Details
Performance of the Perceptron
Other Two-layer Networks
Many Layer Networks
Connections Between Layers
Instar and Outstar
Weights on Connections
Initialization of Weights
A Small Example
Initializing Weights for Autoassociative Networks
Weight Initialization for Heteroassociative Networks
On Center, Off Surround
Inputs
Outputs
The Threshold Function
The Sigmoid Function
The Step Function
The Ramp Function
Linear Function
Applications
Some Neural Network Models
Adaline and Madaline
Backpropagation
Figure for Backpropagation Network
Bidirectional Associative Memory
Temporal Associative Memory
Brain-State-in-a-Box
Counterpropagation
Neocognitron
Adaptive Resonance Theory
Summary
Chapter 6—Learning and Training
Objective of Learning
Learning and Training
Hebb’s Rule
Delta Rule
Supervised Learning
Generalized Delta Rule
Statistical Training and Simulated Annealing
Radial Basis-Function Networks
Unsupervised Networks
Self-Organization
Learning Vector Quantizer
Associative Memory Models and One-Shot Learning
Learning and Resonance
Learning and Stability
Training and Convergence
Lyapunov Function
Other Training Issues
Adaptation
Generalization Ability
Summary
Chapter 7—Backpropagation
Feedforward Backpropagation Network
Mapping
Layout
Training
Illustration: Adjustment of Weights of Connections from a Neuron in the Hidden Layer
Illustration: Adjustment of Weights of Connections from a Neuron in the Input Layer
Adjustments to Threshold Values or Biases
Another Example of Backpropagation Calculations
Notation and Equations
Notation
Equations
C++ Implementation of a Backpropagation Simulator
A Brief Tour of How to Use the Simulator
C++ Classes and Class Hierarchy
Summary
Chapter 8—BAM: Bidirectional Associative Memory
Introduction
Inputs and Outputs
Weights and Training
Example
Recall of Vectors
Continuation of Example
Special Case—Complements
C++ Implementation
Program Details and Flow
Program Example for BAM
Header File
Source File
Program Output
Additional Issues
Unipolar Binary Bidirectional Associative Memory
Summary
Chapter 9—FAM: Fuzzy Associative Memory
Introduction
Association
FAM Neural Network
Encoding
Example of Encoding
Recall
C++ Implementation
Program details
Header File
Source File
Output
Summary
Chapter 10—Adaptive Resonance Theory (ART)
Introduction
The Network for ART1
A Simplified Diagram of Network Layout
Processing in ART1
Special Features of the ART1 Model
Notation for ART1 Calculations
Algorithm for ART1 Calculations
Initialization of Parameters
Equations for ART1 Computations
Other Models
C++ Implementation
A Header File for the C++ Program for the ART1 Model Network
A Source File for C++ Program for an ART1 Model Network
Program Output
Summary
Chapter 11—The Kohonen Self-Organizing Map
Introduction
Competitive Learning
Normalization of a Vector
Lateral Inhibition
The Mexican Hat Function
Training Law for the Kohonen Map
Significance of the Training Law
The Neighborhood Size and Alpha
C++ Code for Implementing a Kohonen Map
The Kohonen Network
Modeling Lateral Inhibition and Excitation
Classes to be Used
Revisiting the Layer Class
A New Layer Class for a Kohonen Layer
Implementation of the Kohonen Layer and Kohonen Network
Flow of the Program and the main() Function
Flow of the Program
Results from Running the Kohonen Program
A Simple First Example
Orthogonal Input Vectors Example
Variations and Applications of Kohonen Networks
Using a Conscience
LVQ: Learning Vector Quantizer
Counterpropagation Network
Application to Speech Recognition
Summary
Chapter 12—Application to Pattern Recognition
Using the Kohonen Feature Map
An Example Problem: Character Recognition
C++ Code Development
Changes to the Kohonen Program
Testing the Program
Generalization versus Memorization
Adding Characters
Other Experiments to Try
Summary
Chapter 13—Backpropagation II
Enhancing the Simulator
Another Example of Using Backpropagation
Adding the Momentum Term
Code Changes
Adding Noise During Training
One Other Change—Starting Training from a Saved Weight File
Trying the Noise and Momentum Features
Variations of the Backpropagation Algorithm
Applications
Summary
Chapter 14—Application to Financial Forecasting
Introduction
Who Trades with Neural Networks?
Developing a Forecasting Model
The Target and the Timeframe
Domain Expertise
Gather the Data
Pre processing the Data for the Network
Reduce Dimensionality
Eliminate Correlated Inputs Where Possible
Design a Network Architecture
The Train/Test/Redesign Loop
Forecasting the S&P 500
Choosing the Right Outputs and Objective
Choosing the Right Inputs
Choosing a Network Architecture
Preprocessing Data
A View of the Raw Data
Highlight Features in the Data
Normalizing the Range
The Target
Storing Data in Different Files
Training and Testing
Using the Simulator to Calculate Error
Only the Beginning
What’s Next?
Technical Analysis and Neural Network Preprocessing
Moving Averages
Momentum and Rate of Change
Relative Strength Index
Percentage R
Herrick Payoff Index
MACD
“Stochastics”
On-Balance Volume
Accumulation-Distribution
What Others Have Reported
Can a Three-Year-Old Trade Commodities?
Forecasting Treasury Bill and Treasury Note Yields
Neural Nets versus Box-Jenkins Time-Series Forecasting
Neural Nets versus Regression Analysis
Hierarchical Neural Network
The Walk-Forward Methodology of Market Prediction
Dual Confirmation Trading System
A Turning Point Predictor
The S&P 500 and Sunspot Predictions
A Critique of Neural Network Time-Series Forecasting for Trading
Resource Guide for Neural Networks and Fuzzy Logic in Finance
Magazines
Books
Book Vendors
Consultants
Historical Financial Data Vendors
Preprocessing Tools for Neural Network Development
Genetic Algorithms Tool Vendors
Fuzzy Logic Tool Vendors
Neural Network Development Tool Vendors
Summary
Chapter 15—Application to Nonlinear Optimization
Introduction
Neural Networks for Optimization Problems
Traveling Salesperson Problem
The TSP in a Nutshell
Solution via Neural Network
Example of a Traveling Salesperson Problem for Hand Calculation
Neural Network for Traveling Salesperson Problem
Network Choice and Layout
Inputs
Activations, Outputs, and Their Updating
Performance of the Hopfield Network
C++ Implementation of the Hopfield Network for the Traveling Salesperson Problem
Source File for Hopfield Network for Traveling Salesperson Problem
Output from Your C++ Program for the Traveling Salesperson Problem
Other Approaches to Solve the Traveling Salesperson Problem
Optimizing a Stock Portfolio
Tabu Neural Network
Summary
Chapter 16—Applications of Fuzzy Logic
Introduction
A Fuzzy Universe of Applications
Section I: A Look at Fuzzy Databases and Quantification
Databases and Queries
Relations in Databases
Fuzzy Scenarios
Fuzzy Sets Revisited
Fuzzy Relations
Matrix Representation of a Fuzzy Relation
Properties of Fuzzy Relations
Similarity Relations
Resemblance Relations
Fuzzy Partial Order
Fuzzy Queries
Extending Database Models
Example
Possibility Distributions
Example
Queries
Fuzzy Events, Means and Variances
Example: XYZ Company Takeover Price
Probability of a Fuzzy Event
Fuzzy Mean and Fuzzy Variance
Conditional Probability of a Fuzzy Event
Conditional Fuzzy Mean and Fuzzy Variance
Linear Regression a la Possibilities
Fuzzy Numbers
Triangular Fuzzy Number
Linear Possibility Regression Model
Section II: Fuzzy Control
Designing a Fuzzy Logic Controller
Step One: Defining Inputs and Outputs for the FLC
Step Two: Fuzzify the Inputs
Step Three: Set Up Fuzzy Membership Functions for the Output(s)
Step Four: Create a Fuzzy Rule Base
Step Five: Defuzzify the Outputs
Advantages and Disadvantages of Fuzzy Logic Controllers
Summary
Chapter 17—Further Applications
Introduction
Computer Virus Detector
Mobile Robot Navigation
A Classifier
A Two-Stage Network for Radar Pattern Classification
Crisp and Fuzzy Neural Networks for Handwritten Character Recognition
Noise Removal with a Discrete Hopfield Network
Object Identification by Shape
Detecting Skin Cancer
EEG Diagnosis
Time Series Prediction with Recurrent and Nonrecurrent Networks
Security Alarms
Circuit Board Faults
Warranty Claims
Writing Style Recognition
Commercial Optical Character Recognition
ART-EMAP and Object Recognition
Summary
References
Appendix A
Appendix B
Glossary
Index


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao
Table of Contents

Glossary

A

Activation
The weighted sum of the inputs to a neuron in a neural network.
Adaline
Adaptive linear element machine.
Adaptive Resonance Theory
Theory developed by Grossberg and Carpenter for categorization of patterns, and to address the stability–plasticity dilemma.
Algorithm
A step-by-step procedure to solve a problem.
Annealing
A process for preventing a network from being drawn into a local minimum.
ART
(Adaptive Resonance Theory) ART1 is the result of the initial development of this theory for binary inputs. Further developments led to ART2 for analog inputs. ART3 is the latest.
Artificial neuron
The primary object in an artificial neural network to mimic the neuron activity of the brain. The artificial neuron is a processing element of a neural network.
Associative memory
Activity of associating one pattern or object with itself or another.
Autoassociative
Making a correspondence of one pattern or object with itself.

B

Backpropagation
A neural network training algorithm for feedforward networks where the errors at the output layer are propagated back to the layer before in learning. If the previous layer is not the input layer, then the errors at this hidden layer are propagated back to the layer before.
BAM
Bidirectional Associative Memory network model.
Bias
A value added to the activation of a neuron.
Binary digit
A value of 0 or 1.
Bipolar value
A value of –1 or +1.
Boltzmann machine
A neural network in which the outputs are determined with probability distributions. Trained and operated using simulated annealing.
Brain-State-in-a-Box
Anderson's single-layer, laterally connected neural network model. It can work with inputs that have noise in them or are incomplete.

C

Cauchy machine
Similar to the Boltzmann machine, except that a Cauchy distribution is used for probabilities.
Cognitron
The forerunner to the Neocognitron. A network developed to recognize characters.
Competition
A process in which a winner is selected from a layer of neurons by some criterion. Competition suggests inhibition reflected in some connection weights being assigned a negative value.
Connection
A means of passing inputs from one neuron to another.
Connection weight
A numerical label associated with a connection and used in a weighted sum of inputs.
Constraint
A condition expressed as an equation or inequality, which has to be satisfied by the variables.
Convergence
Termination of a process with a final result.
Crisp
The opposite of fuzzy—usually a specific numerical quantity or value for an entity.

D

Delta rule
A rule for modification of connection weights, using both the output and the error obtained. It is also called the LMS rule.

E

Energy function
A function of outputs and weights in a neural network to determine the state of the system, e.g., Lyapunov function.
Excitation
Providing positive weights on connections to enable outputs that cause a neuron to fire.
Exemplar
An example of a pattern or object used in training a neural network.
Expert system
A set of formalized rules that enable a system to perform like an expert.

F

FAM
Fuzzy Associative Memory network. Makes associations between fuzzy sets.
Feedback
The process of relaying information in the opposite direction to the original.
Fit vector
A vector of values of degree of membership of elements of a fuzzy set.
Fully connected network
A neural network in which every neuron has connections to all other neurons.
Fuzzy
As related to a variable, the opposite of crisp. A fuzzy quantity represents a range of value as opposed to a single numeric value, e.g., "hot" vs. 89.4°.
Fuzziness
Different concepts having an overlap to some extent. For example, descriptions of fair and cool temperatures may have an overlap of a small interval of temperatures.
Fuzzy Associative Memory
A neural network model to make association between fuzzy sets.
Fuzzy equivalence relation
A fuzzy relation (relationship between fuzzy variables) that is reflexive, symmetric, and transitive.
Fuzzy partial order
A fuzzy relation (relationship between fuzzy variables) that is reflexive, antisymmetric, and transitive.

G

Gain
Sometimes a numerical factor to enhance the activation. Sometimes a connection for the same purpose.
Generalized Delta rule
A rule used in training of networks such as backpropagation training where hidden layer weights are modified with backpropagated error.
Global minimum
A point where the value of a function is no greater than the value at any other point in the domain of the function.

H

Hamming distance
The number of places in which two binary vectors differ from each other.
Hebbian learning
A learning algorithm in which Hebb's rule is used. The change in connection weight between two neurons is taken as a constant times the product of their outputs.
Heteroassociative
Making an association between two distinct patterns or objects.
Hidden layer
An array of neurons positioned in between the input and output layers.
Hopfield network
A single layer, fully connected, autoassociative neural network.

I

Inhibition
The attempt by one neuron to diminish the chances of firing by another neuron.
Input layer
An array of neurons to which an external input or signal is presented.
Instar
A neuron that has no connections going from it to other neurons.

L

Lateral connection
A connection between two neurons that belong to the same layer.
Layer
An array of neurons positioned similarly in a network for its operation.
Learning
The process of finding an appropriate set of connection weights to achieve the goal of the network operation.
Linearly separable
Two subsets of a linear set having a linear barrier (hyperplane) between the two of them.
LMS rule
Least mean squared error rule, with the aim of minimizing the average of the squared error. Same as the Delta rule.
Local minimum
A point where the value of the function is no greater than the value at any other point in its neighborhood.
Long-term memory (LTM)
Encoded information that is retained for an extended period.
Lyapunov function
A function that is bounded below and represents the state of a system that decreases with every change in the state of the system.

M

Madaline
A neural network in which the input layer has units that are Adalines. It is a multiple-Adaline.
Mapping
A correspondence between elements of two sets.

N

Neural network
A collection of processing elements arranged in layers, and a collection of connection edges between pairs of neurons. Input is received at one layer, and output is produced at the same or at a different layer.
Noise
Distortion of an input.
Nonlinear optimization
Finding the best solution for a problem that has a nonlinear function in its objective or in a constraint.

O

On center off surround
Assignment of excitatory weights to connections to nearby neurons and inhibitory weights to connections to distant neurons.
Orthogonal vectors
Vectors whose dot product is 0.
Outstar
A neuron that has no incoming connections.

P

Perceptron
A neural network for linear pattern matching.
Plasticity
Ability to be stimulated by new inputs and learn new mappings or modify existing ones.

R

Resonance
The responsiveness of two neurons in different layers in categorizing an input. An equilibrium in two directions.

S

Saturation
A condition of limitation on the frequency with which a neuron can fire.
Self-organization
A process of partitioning the output layer neurons to correspond to individual patterns or categories, also called unsupervised learning or clustering.
Short-term memory (STM)
The storage of information that does not endure long after removal of the corresponding input.
Simulated annealing
An algorithm by which changes are made to decrease energy or temperature or cost.
Stability
Convergence of a network operation to a steady-state solution.
Supervised learning
A learning process in which the exemplar set consists of pairs of inputs and desired outputs.

T

Threshold value
A value used to compare the activation of a neuron to determine if the neuron fires or not. Sometimes a bias value is added to the activation of a neuron to allow the threshold value to be zero in determining the neuron's output.
Training
The process of helping in a neural network to learn either by providing input/output stimuli (supervised training) or by providing input stimuli (unsupervised training), and allowing weight change updates to occur.

U

Unsupervised learning
Learning in the absence of external information on outputs, also called self-organization or clustering.

V

Vigilance parameter
A parameter used in Adaptive Resonance Theory. It is used to selectively prevent the activation of a subsystem in the network.

W

Weight
A number associated with a neuron or with a connection between two neurons, which is used in aggregating the outputs to determine the activation of a neuron.

Table of Contents


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 1
Introduction to Neural Networks

Neural Processing

How do you recognize a face in a crowd? How does an economist predict the direction of interest rates? Faced with problems like these, the human brain uses a web of interconnected processing elements called neurons to process information. Each neuron is autonomous and independent; it does its work asynchronously, that is, without any synchronization to other events taking place. The two problems posed, namely recognizing a face and forecasting interest rates, have two important characteristics that distinguish them from other problems: First, the problems are complex, that is, you can't devise a simple step-by-step algorithm or precise formula to give you an answer; and second, the data provided to resolve the problems is equally complex and may be noisy or incomplete. You could have forgotten your glasses when you're trying to recognize that face. The economist may have at his or her disposal thousands of pieces of data that may or may not be relevant to his or her forecast on the economy and on interest rates.

The vast processing power inherent in biological neural structures has inspired the study of the structure itself for hints on organizing human-made computing structures. Artificial neural networks, the subject of this book, covers the way to organize synthetic neurons to solve the same kind of difficult, complex problems in a similar manner as we think the human brain may. This chapter will give you a sampling of the terms and nomenclature used to talk about neural networks. These terms will be covered in more depth in the chapters to follow.

Neural Network

A neural network is a computational structure inspired by the study of biological neural processing. There are many different types of neural networks, from relatively simple to very complex, just as there are many theories on how biological neural processing works. We will begin with a discussion of a layered feed-forward type of neural network and branch out to other paradigms later in this chapter and in other chapters.

A layered feed-forward neural network has layers, or subgroups of processing elements. A layer of processing elements makes independent computations on data that it receives and passes the results to another layer. The next layer may in turn make its independent computations and pass on the results to yet another layer. Finally, a subgroup of one or more processing elements determines the output from the network. Each processing element makes its computation based upon a weighted sum of its inputs. The first layer is the input layer and the last the output layer. The layers that are placed between the first and the last layers are the hidden layers. The processing elements are seen as units that are similar to the neurons in a human brain, and hence, they are referred to as cells, neuromimes, or artificial neurons. A threshold function is sometimes used to qualify the output of a neuron in the output layer. Even though our subject matter deals with artificial neurons, we will simply refer to them as neurons. Synapses between neurons are referred to as connections, which are represented by edges of a directed graph in which the nodes are the artificial neurons.

Figure 1.1 is a layered feed-forward neural network. The circular nodes represent neurons. Here there are three layers, an input layer, a hidden layer, and an output layer. The directed graph mentioned shows the connections from nodes from a given layer to other nodes in other layers. Throughout this book you will see many variations on the number and types of layers.

Figure 1.1 A typical neural network.

Output of a Neuron

Basically, the internal activation or raw output of a neuron in a neural network is a weighted sum of its inputs, but a threshold function is also used to determine the final value, or the output. When the output is 1, the neuron is said to fire, and when it is 0, the neuron is considered not to have fired. When a threshold function is used, different results of activations, all in the same interval of values, can cause the same final output value. This situation helps in the sense that, if precise input causes an activation of 9 and noisy input causes an activation of 10, then the output works out the same as if noise is filtered out.

To put the description of a neural network in a simple and familiar setting, let us describe an example about a daytime game show on television, The Price is Right.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Cash Register Game

A contestant in The Price is Right is sometimes asked to play the Cash Register Game. A few products are described, their prices are unknown to the contestant, and the contestant has to declare how many units of each item he or she would like to (pretend to) buy. If the total purchase does not exceed the amount specified, the contestant wins a special prize. After the contestant announces how many items of a particular product he or she wants, the price of that product is revealed, and it is rung up on the cash register. The contestant must be careful, in this case, that the total does not exceed some nominal value, to earn the associated prize. We can now cast the whole operation of this game, in terms of a neural network, called a Perceptron, as follows.

Consider each product on the shelf to be a neuron in the input layer, with its input being the unit price of that product. The cash register is the single neuron in the output layer. The only connections in the network are between each of the neurons (products displayed on the shelf) in the input layer and the output neuron (the cash register). This arrangement is usually referred to as a neuron, the cash register in this case, being an instar in neural network terminology. The contestant actually determines these connections, because when the contestant says he or she wants, say five, of a specific product, the contestant is thereby assigning a weight of 5 to the connection between that product and the cash register. The total bill for the purchases by the contestant is nothing but the weighted sum of the unit prices of the different products offered. For those items the contestant does not choose to purchase, the implicit weight assigned is 0. The application of the dollar limit to the bill is just the application of a threshold, except that the threshold value should not be exceeded for the outcome from this network to favor the contestant, winning him or her a good prize. In a Perceptron, the way the threshold works is that an output neuron is supposed to fire if its activation value exceeds the threshold value.

Weights

The weights used on the connections between different layers have much significance in the working of the neural network and the characterization of a network. The following actions are possible in a neural network:

1. Start with one set of weights and run the network. (NO TRAINING)
2. Start with one set of weights, run the network, and modify some or all the weights, and run the network again with the new set of weights. Repeat this process until some predetermined goal is met. (TRAINING)

Training

Since the output(s) may not be what is expected, the weights may need to be altered. Some rule then needs to be used to determine how to alter the weights. There should also be a criterion to specify when the process of successive modification of weights ceases. This process of changing the weights, or rather, updating the weights, is called training. A network in which learning is employed is said to be subjected to training. Training is an external process or regimen. Learning is the desired process that takes place internal to the network.

Feedback

If you wish to train a network so it can recognize or identify some predetermined patterns, or evaluate some function values for given arguments, it would be important to have information fed back from the output neurons to neurons in some layer before that, to enable further processing and adjustment of weights on the connections. Such feedback can be to the input layer or a layer between the input layer and the output layer, sometimes labeled the hidden layer. What is fed back is usually the error in the output, modified appropriately according to some useful paradigm. The process of feedback continues through the subsequent cycles of operation of the neural network and ceases when the training is completed.

Supervised or Unsupervised Learning

A network can be subject to supervised or unsupervised learning. The learning would be supervised if external criteria are used and matched by the network output, and if not, the learning is unsupervised. This is one broad way to divide different neural network approaches. Unsupervised approaches are also termed self-organizing. There is more interaction between neurons, typically with feedback and intralayer connections between neurons promoting self-organization.

Supervised networks are a little more straightforward to conceptualize than unsupervised networks. You apply the inputs to the supervised network along with an expected response, much like the Pavlovian conditioned stimulus and response regimen. You mold the network with stimulus-response pairs. A stock market forecaster may present economic data (the stimulus) along with metrics of stock market performance (the response) to the neural network to the present and attempt to predict the future once training is complete.

You provide unsupervised networks with only stimulus. You may, for example, want an unsupervised network to correctly classify parts from a conveyor belt into part numbers, providing an image of each part to do the classification (the stimulus). The unsupervised network in this case would act like a look-up memory that is indexed by its contents, or a Content-Addressable-Memory (CAM).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Noise

Noise is perturbation, or a deviation from the actual. A data set used to train a neural network may have inherent noise in it, or an image may have random speckles in it, for example. The response of the neural network to noise is an important factor in determining its suitability to a given application. In the process of training, you may apply a metric to your neural network to see how well the network has learned your training data. In cases where the metric stabilizes to some meaningful value, whether the value is acceptable to you or not, you say that the network converges. You may wish to introduce noise intentionally in training to find out if the network can learn in the presence of noise, and if the network can converge on noisy data.

Memory

Once you train a network on a set of data, suppose you continue training the network with new data. Will the network forget the intended training on the original set or will it remember? This is another angle that is approached by some researchers who are interested in preserving a network's long-term memory (LTM) as well as its short-term memory (STM). Long-term memory is memory associated with learning that persists for the long term. Short-term memory is memory associated with a neural network that decays in some time interval.

Capsule of History

You marvel at the capabilities of the human brain and find its ways of processing information unknown to a large extent. You find it awesome that very complex situations are discerned at a far greater speed than what a computer can do.

Warren McCulloch and Walter Pitts formulated in 1943 a model for a nerve cell, a neuron, during their attempt to build a theory of self-organizing systems. Later, Frank Rosenblatt constructed a Perceptron, an arrangement of processing elements representing the nerve cells into a network. His network could recognize simple shapes. It was the advent of different models for different applications.

Those working in the field of artificial intelligence (AI) tried to hypothesize that you can model thought processes using some symbols and some rules with which you can transform the symbols.

A limitation to the symbolic approach is related to how knowledge is representable. A piece of information is localized, that is, available at one location, perhaps. It is not distributed over many locations. You can easily see that distributed knowledge leads to a faster and greater inferential process. Information is less prone to be damaged or lost when it is distributed than when it is localized. Distributed information processing can be fault tolerant to some degree, because there are multiple sources of knowledge to apply to a given problem. Even if one source is cut off or destroyed, other sources may still permit solution to a problem. Further, with subsequent learning, a solution may be remapped into a new organization of distributed processing elements that exclude a faulty processing element.

In neural networks, information may impact the activity of more than one neuron. Knowledge is distributed and lends itself easily to parallel computation. Indeed there are many research activities in the field of hardware design of neural network processing engines that exploit the parallelism of the neural network paradigm. Carver Mead, a pioneer in the field, has suggested analog VLSI (very large scale integration) circuit implementations of neural networks.

Neural Network Construction

There are three aspects to the construction of a neural network:

1. Structure—the architecture and topology of the neural network
2. Encoding—the method of changing weights
3. Recall—the method and capacity to retrieve information

Let's cover the first one—structure. This relates to how many layers the network should contain, and what their functions are, such as for input, for output, or for feature extraction. Structure also encompasses how interconnections are made between neurons in the network, and what their functions are.

The second aspect is encoding. Encoding refers to the paradigm used for the determination of and changing of weights on the connections between neurons. In the case of the multilayer feed-forward neural network, you initially can define weights by randomization. Subsequently, in the process of training, you can use the backpropagation algorithm, which is a means of updating weights starting from the output backwards. When you have finished training the multilayer feed-forward neural network, you are finished with encoding since weights do not change after training is completed.

Finally, recall is also an important aspect of a neural network. Recall refers to getting an expected output for a given input. If the same input as before is presented to the network, the same corresponding output as before should result. The type of recall can characterize the network as being autoassociative or heteroassociative. Autoassociation is the phenomenon of associating an input vector with itself as the output, whereas heteroassociation is that of recalling a related vector given an input vector. You have a fuzzy remembrance of a phone number. Luckily, you stored it in an autoassociative neural network. When you apply the fuzzy remembrance, you retrieve the actual phone number. This is a use of autoassociation. Now if you want the individual's name associated with a given phone number, that would require heteroassociation. Recall is closely related to the concepts of STM and LTM introduced earlier.

The three aspects to the construction of a neural network mentioned above essentially distinguish between different neural networks and are part of their design process.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Sample Applications

One application for a neural network is pattern classification, or pattern matching. The patterns can be represented by binary digits in the discrete cases, or real numbers representing analog signals in continuous cases. Pattern classification is a form of establishing an autoassociation or heteroassociation. Recall that associating different patterns is building the type of association called heteroassociation. If you input a corrupted or modified pattern A to the neural network, and receive the true pattern A, this is termed autoassociation. What use does this provide? Remember the example given at the beginning of this chapter. In the human brain example, say you want to recall a face in a crowd and you have a hazy remembrance (input). What you want is the actual image. Autoassociation, then, is useful in recognizing or retrieving patterns with possibly incomplete information as input. What about heteroassociation? Here you associate A with B. Given A, you get B and sometimes vice versa. You could store the face of a person and retrieve it with the person's name, for example. It's quite common in real circumstances to do the opposite, and sometimes not so well. You recall the face of a person, but can't place the name.

Qualifying for a Mortgage

Another sample application, which is in fact in the works by a U.S. government agency, is to devise a neural network to produce a quick response credit rating of an individual trying to qualify for a mortgage. The problem to date with the application process for a mortgage has been the staggering amount of paperwork and filing details required for each application. Once information is gathered, the response time for knowing whether or not your mortgage is approved has typically taken several weeks. All of this will change. The proposed neural network system will allow the complete application and approval process to take three hours, with approval coming in five minutes of entering all of the information required. You enter in the applicant's employment history, salary information, credit information, and other factors and apply these to a trained neural network. The neural network, based on prior training on thousands of case histories, looks for patterns in the applicant's profile and then produces a yes or no rating of worthiness to carry a particular mortgage. Let's now continue our discussion of factors that distinguish neural network models from each other.

Cooperation and Competition

We will now discuss cooperation and competition. Again we start with an example feed forward neural network. If the network consists of a single input layer and an output layer consisting of a single neuron, then the set of weights for the connections between the input layer neurons and the output neuron are given in a weight vector. For three inputs and one output, this could be W = {w1, w2, w3 }. When the output layer has more than one neuron, the output is not just one value but is also a vector. In such a situation each neuron in one layer is connected to each neuron in the next layer, with weights assigned to these interconnections. Then the weights can all be given together in a two-dimensional weight matrix, which is also sometimes called a correlation matrix. When there are in-between layers such as a hidden layer or a so-called Kohonen layer or a Grossberg layer, the interconnections are made between each neuron in one layer and every neuron in the next layer, and there will be a corresponding correlation matrix. Cooperation or competition or both can be imparted between network neurons in the same layer, through the choice of the right sign of weights for the connections. Cooperation is the attempt between neurons in one neuron aiding the prospect of firing by another. Competition is the attempt between neurons to individually excel with higher output. Inhibition, a mechanism used in competition, is the attempt between neurons in one neuron decreasing the prospect of another neuron's firing. As already stated, the vehicle for these phenomena is the connection weight. For example, a positive weight is assigned for a connection between one node and a cooperating node in that layer, while a negative weight is assigned to inhibit a competitor.

To take this idea to the connections between neurons in consecutive layers, we would assign a positive weight to the connection between one node in one layer and its nearest neighbor node in the next layer, whereas the connections with distant nodes in the other layer will get negative weights. The negative weights would indicate competition in some cases and inhibition in others. To make at least some of the discussion and the concepts a bit clearer, we preview two example neural networks (there will be more discussion of these networks in the chapters that follow): the feed-forward network and the Hopfield network.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Example—A Feed-Forward Network

A sample feed-forward network, as shown in Figure 1.2, has five neurons arranged in three layers: two neurons (labeled x1 and x2) in layer 1, two neurons (labeled x3 and x4) in layer 2, and one neuron (labeled x5) in layer 3. There are arrows connecting the neurons together. This is the direction of information flow. A feed-forward network has information flowing forward only. Each arrow that connects neurons has a weight associated with it (like, w31 for example). You calculate the state, x, of each neuron by summing the weighted values that flow into a neuron. The state of the neuron is the output value of the neuron and remains the same until the neuron receives new information on its inputs.

Figure 1.2 A feed-forward neural network with topology 2-2-1.

For example, for x3 and x5:

x3 = w23 x2 + w13 x1

x5 = w35 x3 + w45 x4

We will formalize the equations in Chapter 7, which details one of the training algorithms for the feed-forward network called Backpropagation.

Note that you present information to this network at the leftmost nodes (layer 1) called the input layer. You can take information from any other layer in the network, but in most cases do so from the rightmost node(s), which make up the output layer. Weights are usually determined by a supervised training algorithm, where you present examples to the network and adjust weights appropriately to achieve a desired response. Once you have completed training, you can use the network without changing weights, and note the response for inputs that you apply. Note that a detail not yet shown is a nonlinear scaling function that limits the range of the weighted sum. This scaling function has the effect of clipping very large values in positive and negative directions for each neuron so that the cumulative summing that occurs across the network stays within reasonable bounds. Typical real number ranges for neuron inputs and outputs are –1 to +1 or 0 to +1. You will see more about this network and applications for it in Chapter 7. Now let us contrast this neural network with a completely different type of neural network, the Hopfield network, and present some simple applications for the Hopfield network.

Example—A Hopfield Network

The neural network we present is a Hopfield network, with a single layer. We place, in this layer, four neurons, each connected to the rest, as shown in Figure 1.3. Some of the connections have a positive weight, and the rest have a negative weight. The network will be presented with two input patterns, one at a time, and it is supposed to recall them. The inputs would be binary patterns having in each component a 0 or 1. If two patterns of equal length are given and are treated as vectors, their dot product is obtained by first multiplying corresponding components together and then adding these products. Two vectors are said to be orthogonal, if their dot product is 0. The mathematics involved in computations done for neural networks include matrix multiplication, transpose of a matrix, and transpose of a vector. Also see Appendix B. The inputs (which are stable, stored patterns) to be given should be orthogonal to one another.

Figure 1.3 Layout of a Hopfield network.

The two patterns we want the network to recall are A = (1, 0, 1, 0) and B = (0, 1, 0, 1), which you can verify to be orthogonal. Recall that two vectors A and B are orthogonal if their dot product is equal to zero. This is true in this case since

     A1B1 + A2 B2 + A3B3 + A4B4 = (1x0 + 0x1 + 1x0 + 0x1) = 0

The following matrix W gives the weights on the connections in the network.

      0   -3    3   -3

     -3    0   -3    3

W =   3   -3    0   -3

     -3    3   -3    0

We need a threshold function also, and we define it as follows. The threshold value [theta] is 0.

            1  if t >= [theta]

f(t) = {

            0  if t < [theta]


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

We have four neurons in the only layer in this network. We need to compute the activation of each neuron as the weighted sum of its inputs. The activation at the first node is the dot product of the input vector and the first column of the weight matrix (0 -3 3 -3). We get the activation at the other nodes similarly. The output of a neuron is then calculated by evaluating the threshold function at the activation of the neuron. So if we present the input vector A, the dot product works out to 3 and f(3) = 1. Similarly, we get the dot products of the second, third, and fourth nodes to be –6, 3, and –6, respectively. The corresponding outputs therefore are 0, 1, and 0. This means that the output of the network is the vector (1, 0, 1, 0), same as the input pattern. The network has recalled the pattern as presented, or we can say that pattern A is stable, since the output is equal to the input. When B is presented, the dot product obtained at the first node is –6 and the output is 0. The outputs for the rest of the nodes taken together with the output of the first node gives (0, 1, 0, 1), which means that the network has stable recall for B also.

NOTE: In Chapter 4, a method of determining the weight matrix for the Hopfield network given a set of input vectors is presented.

So far we have presented easy cases to the network—vectors that the Hopfield network was specifically designed (through the choice of the weight matrix) to recall. What will the network give as output if we present a pattern different from both A and B? Let C = (0, 1, 0, 0) be presented to the network. The activations would be –3, 0, –3, 3, making the outputs 0, 1, 0, 1, which means that B achieves stable recall. This is quite interesting. Suppose we did intend to input B and we made a slight error and ended up presenting C, instead. The network did what we wanted and recalled B. But why not A? To answer this, let us ask is C closer to A or B? How do we compare? We use the distance formula for two four-dimensional points. If (a, b, c, d) and (e, f, g, h) are two four-dimensional points, the distance between them is:

[radic][(a – e)2 + (b – f)2 + (c – g)2 + (d – h)2 ]

The distance between A and C is [radic]3, whereas the distance between B and C is just 1. So since B is closer in this sense, B was recalled rather than A. You may verify that if we do the same exercise with D = (0, 0, 1, 0), we will see that the network recalls A, which is closer than B to D.

Hamming Distance

When we talk about closeness of a bit pattern to another bit pattern, the Euclidean distance need not be considered. Instead, the Hamming distance can be used, which is much easier to determine, since it is the number of bit positions in which the two patterns being compared differ. Patterns being strings, the Hamming distance is more appropriate than the Euclidean distance.

NOTE: The weight matrix W we gave in this example is not the only weight matrix that would enable the network to recall the patterns A and B correctly. You can see that if we replace each of 3 and –3 in the matrix by say, 2 and –2, respectively, the resulting matrix would also facilitate the same performance from the network. For more details, consult Chapter 4.

Asynchronous Update

The Hopfield network is a recurrent network. This means that outputs from the network are fed back as inputs. This is not apparent from Figure 1.3, but is clearly seen from Figure 1.4.

Figure 1.4 Feedback in the Hopfield network.

The Hopfield network always stabilizes to a fixed point. There is a very important detail regarding the Hopfield network to achieve this stability. In the examples thus far, we have not had a problem getting a stable output from the network, so we have not presented this detail of network operation. This detail is the need to update the network asynchronously. This means that changes do not occur simultaneously to outputs that are fed back as inputs, but rather occur for one vector component at a time. The true operation of the Hopfield network follows the procedure below for input vector Invec and output vector Outvec:

1. Apply an input, Invec, to the network, and initialize Outvec = Invec
2. Start with i = 1
3. Calculate Valuei = DotProduct ( Inveci, Columni of Weight matrix)
4. Calculate Outveci = f(Valuei) where f is the threshold function discussed previously
5. Update the input to the network with component Outveci
6. Increment i, and repeat steps 3, 4, 5, and 6 until Invec = Outvec (note that when i reaches its maximum value, it is then next reset to 1 for the cycle to continue)

Now let's see how to apply this procedure. Building on the last example, we now input E = (1, 0, 0, 1), which is at an equal distance from A and B. Without applying the asynchronous procedure above, but instead using the shortcut procedure we've been using so far, you would get an output F = (0, 1, 1, 0). This vector, F, as subsequent input would result in E as the output. This is incorrect since the network oscillates between two states. We have updated the entire input vector synchronously.

Now let's apply asynchronous update. For input E, (1,0,0,1) we arrive at the following results detailed for each update step, in Table 1.1.

Table 1.1 Example of Asynchronous Update for the Hopfield Network
Step | i | Invec | Column of Weight vector | Value | Outvec | notes
0 |  | 1001 |  |  | 1001 | initialization : set Outvec = Invec = Input pattern
1 | 1 | 1001 | 0 -3 3 -3 | -3 | 0001 | column 1 of Outvec changed to 0
2 | 2 | 0001 | -3 0 -3 3 | 3 | 0101 | column 2 of Outvec changed to 1
3 | 3 | 0101 | 3 -3 0 -3 | -6 | 0101 | column 3 of Outvec stays as 0
4 | 4 | 0101 | -3 3 -3 0 | 3 | 0101 | column 4 of Outvec stays as 1
5 | 1 | 0101 | 0 -3 3 -3 | -6 | 0101 | column 1 stable as 0
6 | 2 | 0101 | -3 0 -3 3 | 3 | 0101 | column 2 stable as 1
7 | 3 | 0101 | 3 -3 0 -3 | -6 | 0101 | column 3 stable as 0
8 | 4 | 0101 | -3 3 -3 0 | 3 | 0101 | column 4 stable as 1; stable recalled pattern = 0101


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Binary and Bipolar Inputs

Two types of inputs that are used in neural networks are binary and bipolar inputs. We have already seen examples of binary input. Bipolar inputs have one of two values, 1 and –1. There is clearly a one-to-one mapping or correspondence between them, namely having -1 of bipolar correspond to a 0 of binary. In determining the weight matrix in some situations where binary strings are the inputs, this mapping is used, and when the output appears in bipolar values, the inverse transformation is applied to get the corresponding binary string. A simple example would be that the binary string 1 0 0 1 is mapped onto the bipolar string 1 –1 –1 1; while using the inverse transformation on the bipolar string –1 1 –1 –1, we get the binary string 0 1 0 0.

Bias

The use of threshold value can take two forms. One we showed in the example. The activation is compared to the threshold value, and the neuron fires if the threshold value is attained or exceeded. The other way is to add a value to the activation itself, in which case it is called the bias, and then determining the output of the neuron. We will encounter bias and gain later.

Another Example for the Hopfield Network

You will see in Chapter 12 an application of Kohonen's feature map for pattern recognition. Here we give an example of pattern association using a Hopfield network. The patterns are some characters. A pattern representing a character becomes an input to a Hopfield network through a bipolar vector. This bipolar vector is generated from the pixel (picture element) grid for the character, with an assignment of a 1 to a black pixel and a -1 to a pixel that is white. A grid size such as 5x7 or higher is usually employed in these approaches. The number of pixels involved will then be 35 or more, which determines the dimension of a bipolar vector for the character pattern.

We will use, for simplicity, a 3x3 grid for character patterns in our example. This means the Hopfield network has 9 neurons in the only layer in the network. Again for simplicity, we use two exemplar patterns, or reference patterns, which are given in Figure 1.5. Consider the pattern on the left as a representation of the character "plus", +, and the one on the right that of "minus", - .

Figure 1.5 The "plus" pattern and "minus" pattern.

The bipolar vectors that represent the characters in the figure, reading the character pixel patterns row by row, left to right, and top to bottom, with a 1 for black and -1 for white pixels, are C+ = (-1, 1, -1, 1, 1, 1, -1, 1, -1), and C- = (-1, -1, -1, 1, 1, 1, -1, -1, -1). The weight matrix W is:

      0  0  2 -2 -2 -2  2  0  2

      0  0  0  0  0  0  0  2  0

      2  0  0 -2 -2 -2  2  0  2

      2  0 -2  0  2  2 -2  0 -2

W=    2  0 -2  2  0  2 -2  0 -2

      2  0 -2  2  2  0 -2  0 -2

      2  0  2 -2 -2 -2  0  0  2

      0  2  0  0  0  0  0  0  0

      2  0  2 -2 -2 -2  2  0  0

The activations with input C+ are given by the vector (-12, 2, -12, 12, 12, 12, -12, 2, -12). With input C-, the activations vector is (-12, -2, -12, 12, 12, 12, -12, -2, -12).

When this Hopfield network uses the threshold function

             1  if x >= 0

f(x) = {

            -1  if x [le] 0

the corresponding outputs will be C+ and C-, respectively, showing the stable recall of the exemplar vectors, and establishing an autoassociation for them. When the output vectors are used to construct the corresponding characters, you get the original character patterns.

Let us now input the character pattern in Figure 1.6.

Figure 1.6 Corrupted "minus" pattern.

We will call the corresponding bipolar vector A = (1, -1, -1, 1, 1, 1, -1, -1, -1). You get the activation vector (-12, -2, -8, 4, 4, 4, -8, -2, -8) giving the output vector, C- = (-1, -1, -1, 1, 1, 1, -1, -1, -1). In other words, the character -, corrupted slightly, is recalled as the character - by the Hopfield network. The intended pattern is recognized.

We now input a bipolar vector that is different from the vectors corresponding to the exemplars, and see whether the network can store the corresponding pattern. The vector we choose is B = (1, -1, 1, -1, -1, -1, 1, -1, 1). The corresponding neuron activations are given by the vector (12, -2, 12, -4, -4, -4, 12, -2, 12) which causes the output to be the vector (1, -1, 1, -1, -1, -1, 1, -1, 1), same as B. An additional pattern, which is a 3x3 grid with only the corner pixels black, as shown in Figure 1.7, is also recalled since it is autoassociated, by this Hopfield network.

Figure 1.7 Pattern result.

If we omit part of the pattern in Figure 1.7, leaving only the top corners black, as in Figure 1.8, we get the bipolar vector D = (1, -1, 1, -1, -1, -1, -1, -1, -1). You can consider this also as an incomplete or corrupted version of the pattern in Figure 1.7. The network activations turn out to be (4, -2, 4, -4, -4, -4, 8, -2, 8) and give the output (1, -1, 1, -1, -1, -1, 1, -1, 1), which is B.

Figure 1.8 A partly lost Pattern of Figure 1.7.

Summary

In this chapter we introduced a neural network as a collection of processing elements distributed over a finite number of layers and interconnected with positive or negative weights, depending on whether cooperation or competition (or inhibition) is intended. The activation of a neuron is basically a weighted sum of its inputs. A threshold function determines the output of the network. There may be layers of neurons in between the input layer and the output layer, and some such middle layers are referred to as hidden layers, others by names such as Grossberg or Kohonen layers, named after the researchers Stephen Grossberg and Teuvo Kohonen, who proposed them and their function. Modification of the weights is the process of training the network, and a network subject to this process is said to be learning during that phase of the operation of the network. In some network operations, a feedback operation is used in which the current output is treated as modified input to the same network.

You have seen a couple of examples of a Hopfield network, one of them for pattern recognition.

Neural networks can be used for problems that can't be solved with a known formula and for problems with incomplete or noisy data. Neural networks seem to have the capacity to recognize patterns in the data presented to it, and are thus useful in many types of pattern recognition problems.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 2
C++ and Object Orientation

Introduction to C++

C++ is an object-oriented programming language built on the base of the C language. This chapter gives you a very brief introduction to C++, touching on many important aspects of C++, so you would be able to follow our presentations of the C++ implementations of neural network models and write your own C++ programs.

The C++ language is a superset of the C language. You could write C++ programs like C programs (a few of the programs in this book are like that), or you could take advantage of the object-oriented features of C++ to write object-oriented programs (like the backpropagation simulator of Chapter 7). What makes a programming language or programming methodology object oriented? Well, there are several indisputable pillars of object orientation. These features stand out more than any other as far as object orientation goes. They are encapsulation, data hiding, overloading, polymorphism, and the grand-daddy of them all: inheritance. Each of the pillars of object-orientation will be discussed in the coming sections, but before we tackle these, we need to answer the question, What does all this object-oriented stuff buy me ? By using the object-oriented features of C++, in conjunction with Object-Oriented Analysis and Design(OOAD), which is a methodology that fully utilizes object orientation, you can have well-packaged, reusable, extensible, and reliable programs and program segments. It's beyond the scope of this book to discuss OOAD, but it's recommended you read Booch or Rumbaugh to get more details on OOAD and how and why to change your programming style forever! See the reference section in the back of this book for more information on these readings. Now let's get back to discussing the great object-oriented features of C++.

Encapsulation

In C++ you have the facility to encapsulate data and the operations that manipulate that data, in an appropriate object. This enables the use of these collections of data and function, called objects , in programs other than the program for which they were originally created. With objects, just as with the traditional concept of subroutines, you make functional blocks of code. You still have language-supported abstractions such as scope and separate compilation available. This is a rudimentary form of encapsulation. Objects carry encapsulation a step further. With objects, you define not only the way a function operates, or its implementation, but also the way an object can be accessed, or its interface. You can specify access differently for different entities. For example, you could make function do_operation() contained inside Object A accessible to Object B but not to Object C. This access qualification can also be used for data members inside an object. The encapsulation of data and the intended operations on them prevents the data from being subjected to operations not meant for them. This is what really makes objects reusable and portable! The operations are usually given in the form of functions operating upon the data items. Such functions are also called methods in some object-oriented programming languages. The data items and the functions that manipulate them are combined into a structure called a class. A class is an abstract data type. When you make an instance of a class, you have an object. This is no different than when you instantiate an integer type to create variables i and j. For example, you design a class called ElectronicBook, with a data element called ArrayofPages. When you instantiate your class you make objects of type ElectronicBook. Suppose that you create two of these called EB_Geography and EB_History. Every object that is instantiated has its own data member inside it, referred to by ArrayOfPages.

Data Hiding

Related to the idea of encapsulation is the concept of data hiding. Encapsulation hides the data from other classes and functions in other classes. Going back to the ElectronicBook class, you could define functions like GetNextPage, GetPreviousPage, and GetCurrentPage as the only means of accessing information in the ArrayofPages data member, by functions that access the ElectronicBook object. Although there may be a hundred and one other attributes and data elements in the class ElectronicBook, these are all hidden from view. This makes programs more reliable, since publishing a specific interface to an object prevents inadvertent access to data in ways that were not designed or accounted for. In C++, the access to an object, and its encapsulated data and functions is treated very carefully, by the use of keywords private, protected, and public. One has the opportunity to make access specifications for data objects and functions as being private, or protected, or public while defining a class. Only when the declaration is made as public do other functions and objects have access to the object and its components without question. On the other hand, if the declaration happens to be as private, there is no possibility of such access. When the declaration given is as protected, then the access to data and functions in a class by others is not as free as when it is public, nor as restricted as when it is private. You can declare one class as derived from another class, which will be discussed shortly. So-called derived classes and the declaring class do get the access to the components of the object that are declared protected. One class that is not a derived class of a second class can get access to data items and functions of the second class if it is declared as a friend class in the second. The three types of declarations of access specification can be different for different components of an object. For example, some of the data items could be declared public, some private, and the others protected. The same situation can occur with the functions in an object. When no explicit declaration is made, the default specification is as private.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Constructors and Destructors as Special Functions of C++

Constructors and destructors are special functions in C++. They define how an object is created and destroyed. You cannot have a class defined in a C++ program without declaring and defining at least one constructor for it. You may omit declaring and then defining a destructor only because the compiler you use will create a default destructor. More than one constructor, but only one destructor, can be declared for a class.

Constructors are for the creation of an object of a class and for initializing it. C++ requires that every function has a return type. The only exceptions are constructors and destructors. A constructor is given the same name as the class for which it is a constructor. It may take arguments or it may not need them. Different constructors for the same class differ in the number and types of arguments they take. It is a good idea to provide for each class at least a default constructor that does not take any arguments and does not do anything except create an object of that class type. A constructor is called at the time an object of its class is needed to be created.

A destructor also is given the same name as the class for which it is a destructor, but with the tilde (~) preceding the name. Typically, what is done in a destructor is to have statements that ask the system to delete the various data structures created for the class. This helps to free-up allocated memory for those data structures. A destructor is called when the object created earlier is no longer needed in the program.

Dynamic Memory Allocation

C++ has keywords new and delete, which are used as a pair in that order, though separated by other statements of the program. They are for making dynamic allocation of memory at the time of creation of a class object and for freeing-up such allocated memory when it is no longer needed. You create space on the heap with the use of new. This obviates the need in C++ for malloc, which is the function for dynamic memory allocation used in C.

Overloading

Encapsulation of data and functions would also allow you to use the same function name in two different objects. The use of a name for a function more than once does not have to be only in different object declarations. Within the same object one can use the same name for functions with different functionality, if they can be distinguished in terms of either their return type or in terms of their argument types and number. This feature is called overloading. For example, if two different types of variables are data items in an object, a commonly named function can be the addition, one for each of the two types of variables—thus taking advantage of overloading. Then the function addition is said to be overloaded. But remember that the function main is just about the only function that cannot be overloaded.

Polymorphism and Polymorphic Functions

A polymorphic function is a function whose name is used in different ways in a program. It can be also declared virtual, if the intention is late binding. This enables it to be bound at run time. Late binding is also referred to as dynamic binding. An advantage in declaring a function in an object as virtual is that, if the program that uses this object calls that function only conditionally, there is no need to bind the function early, during the compilation of the program. It will be bound only if the condition is met and the call of the function takes place. For example, you could have a polymorphic function called draw() that is associated with different graphical objects, like for rectangle, circle, and sphere. The details or methods of the functions are different, but the name draw() is common. If you now have a collection of these objects and pick up an arbitrary object without knowing exactly what it is (via a pointer, for example), you can still invoke the draw function for the object and be assured that the right draw function will be bound to the object and called.

Overloading Operators

You can overload operators in addition to overloading functions. As a matter of fact, the system defined left shift operator << is also overloaded in C++ when used with cout, the C++ variation of the C language printf function. There is a similar situation with the right shift operator >> in C++ when used with cin, the C++ variation of the C language scanf function. You can take any operator and overload it. But you want to be cautious and not overdo it, and also you do not create confusion when you overload an operator. The guiding principle in this regard is the creation of a code-saving and time-saving facility while maintaining simplicity and clarity. Operator overloading is especially useful for doing normal arithmetic on nonstandard data types. You could overload the multiplication symbol to work with complex numbers, for example.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Inheritance

The primary distinction for C++ from C is that C++ has classes. Objects are defined in classes. Classes themselves can be data items in other classes, in which case one class would be an element of another class. Of course, then one class is a member, which brings with it its own data and functions, in the second class. This type of relationship is referred to as a "has-a" relationship: Object A has an Object B inside it.

A relationship between classes can be established not only by making one class a member of another but also by the process of deriving one class from another. One class can be derived from another class, which becomes its base class. Then a hierarchy of classes is established, and a sort of parent–child relationship between classes is established. The derived class inherits, from the base class, some of the data members and functions. This type of relationship is referred to as an "is-a" relationship. You could have class Rectangle be derived from class Shape, since Rectangle is a Shape. Naturally, if a class A is derived from a class B, and if B itself is derived from a class C, then A inherits from both B and C. A class can be derived from more than one class. This is how multiple inheritance occurs. Inheritance is a powerful mechanism for creating base functionality that is passed onto next generations for further enhancement or modification.

Derived Classes

When one class has some members declared in it as protected, then such members would be hidden from other classes, but not from the derived classes. In other words, deriving one class from another is a way of accessing the protected members of the parent class by the derived class. We then say that the derived class is inheriting from the parent class those members in the parent class that are declared as protected or public.

In declaring a derived class from another class, access or visibility specification can be made, meaning that such derivation can be public or the default case, private. Table 2.1 shows the consequences of such specification when deriving one class from another.

Table 2.1 Visibility of Base Class Members in Derived Class
Derivation Specification | Base Class Specification | Derived Class Access
private(default) | private | none
 | protected | full access, private in derived class
 | public | full access, public in derived class
public | private | none
 | protected | full access, protected in derived class
 | public | full access, public in derived class

Reuse of Code

C++ is also attractive for the extendibility of the programs written in it and for the reuse opportunity, thanks to the features in C++ such as inheritance and polymorphism mentioned earlier. A new programming project cannot only reuse classes that are created for some other program, if they are appropriate, but can extend another program with additional classes and functions as deemed necessary. You could inherit from an existing class hierarchy and only change functionality where you need to.

C++ Compilers

All of the programs in this book have been compiled and tested with Turbo C++, Borland C++, Microsoft C/C++, and Microsoft Visual C++. These are a few of the popular commercial C++ compilers available. You should be able to use most other commercial C++ compilers also. All of the programs should also port easily to other operating systems like Unix and the Mac, because they are all character-based programs.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Writing C++ Programs

Before one starts writing a C++ program for a particular problem, one has to have a clear picture of the various parameters and variables that would be part of the problem definition and/or its solution. In addition, it should be clear as to what manipulations need to be performed during the solution process. Then one carefully determines what classes are needed and what relationships they have to each other in a hierarchy of classes. Think about is-a and has-a relationships to see where classes need to defined, and which classes could be derived from others. It would be far more clear to the programmer at this point in the program plan what the data and function access specifications should be and so on. The typical compilation error messages a programmer to C++ may encounter are stating that a particular function or data is not a member of a particular class, or that it is not accessible to a class, or that a constructor was not available for a particular class. When function arguments at declaration and at the place the function is called do not match, either for number or for types or both, the compiler thinks of them as two different functions. The compiler does not find the definition and/or declaration of one of the two and has reason to complain. This type of error in one line of code may cause the compiler to alert you that several other errors are also present, perhaps some in terms of improper punctuation. In that case, remedying the fundamental error that was pointed out would straighten many of the other argued matters.

The following list contains a few additional particulars you need to keep in mind when writing C++ programs.

• A member x of an object A is referred to with A.x just as done with structure elements in C.
• If you declare a class B, then the constructor function is also named B. B has no return type. If this constructor takes, say, one argument of type integer, you define the constructor using the syntax: B::B(int){whatever the function does};
• If you declare a member function C of class B, where return type of C is, say, float and C takes two arguments, one of type float, and the other int, then you define C with the syntax: float B::C(float,int){whatever the function does};
• If you declare a member function D of class B, where D does not return any value and takes no arguments, you define D using the syntax: void B::D( ){whatever the function does};
• If G is a class derived from, say, class B previously mentioned, you declare G using the syntax: class G:B. The constructor for G is defined using the syntax: G::G(arguments of G):B(int){whatever the function does}. If, on the other hand, G is derived from, say, class B as well as class T, then you declare G using the syntax: class G:B,T.
• If one class is declared as derived from more than one other class, that is, if there are more than one base class for it, the derivations specification can be different or the same. Thus the class may be derived from one class publicly and at the same time from another class privately.
• If you have declared a global variable y external to a class B, and if you also have a data member y in the class B, you can use the external y with the reference symbol ::. Thus ::y refers to the global variable, whereas y, within a member function of B, or B.y refers to the data member of B. This way polymorphic functions can also be distinguished from each other.

This is by no means a comprehensive list of features, but more a brief list of important constructs in C++. You will see examples of C++ usage in later chapters.

Summary

A few highlights of the C++ language are presented.

• C++ is an object-oriented language with full compatibility with the C language.
• You create classes in C++ that encapsulate data and functions that operate on the data, and hiding data where a public interface is not needed.
• You can create hierarchies of classes with the facility of inheritance. Polymorphism is a feature that allows you to apply a function to a task according to the object the function is operating on.
• Another feature in C++ is overloading of operators, which allows you to create new functionality for existing operators in a different context.
• Overall, C++ is a powerful language fitting the object-oriented paradigm that enables software reuse and enhanced reliability.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 3
A Look at Fuzzy Logic

Crisp or Fuzzy Logic?

Logic deals with true and false. A proposition can be true on one occasion and false on another. "Apple is a red fruit" is such a proposition. If you are holding a Granny Smith apple that is green, the proposition that apple is a red fruit is false. On the other hand, if your apple is of a red delicious variety, it is a red fruit and the proposition in reference is true. If a proposition is true, it has a truth value of 1; if it is false, its truth value is 0. These are the only possible truth values. Propositions can be combined to generate other propositions, by means of logical operations.

When you say it will rain today or that you will have an outdoor picnic today, you are making statements with certainty. Of course your statements in this case can be either true or false. The truth values of your statements can be only 1, or 0. Your statements then can be said to be crisp.

On the other hand, there are statements you cannot make with such certainty. You may be saying that you think it will rain today. If pressed further, you may be able to say with a degree of certainty in your statement that it will rain today. Your level of certainty, however, is about 0.8, rather than 1. This type of situation is what fuzzy logic was developed to model. Fuzzy logic deals with propositions that can be true to a certain degree—somewhere from 0 to 1. Therefore, a proposition's truth value indicates the degree of certainty about which the proposition is true. The degree of certainity sounds like a probability (perhaps subjective probability), but it is not quite the same. Probabilities for mutually exclusive events cannot add up to more than 1, but their fuzzy values may. Suppose that the probability of a cup of coffee being hot is 0.8 and the probability of the cup of coffee being cold is 0.2. These probabilities must add up to 1.0. Fuzzy values do not need to add up to 1.0. The truth value of a proposition that a cup of coffee is hot is 0.8. The truth value of a proposition that the cup of coffee is cold can be 0.5. There is no restriction on what these truth values must add up to.

Fuzzy Sets

Fuzzy logic is best understood in the context of set membership. Suppose you are assembling a set of rainy days. Would you put today in the set? When you deal only with crisp statements that are either true or false, your inclusion of today in the set of rainy days is based on certainty. When dealing with fuzzy logic, you would include today in the set of rainy days via an ordered pair, such as (today, 0.8). The first member in such an ordered pair is a candidate for inclusion in the set, and the second member is a value between 0 and 1, inclusive, called the degree of membership in the set. The inclusion of the degree of membership in the set makes it convenient for developers to come up with a set theory based on fuzzy logic, just as regular set theory is developed. Fuzzy sets are sets in which members are presented as ordered pairs that include information on degree of membership. A traditional set of, say, k elements, is a special case of a fuzzy set, where each of those k elements has 1 for the degree of membership, and every other element in the universal set has a degree of membership 0, for which reason you don't bother to list it.

Fuzzy Set Operations

The usual operations you can perform on ordinary sets are union, in which you take all the elements that are in one set or the other; and intersection, in which you take the elements that are in both sets. In the case of fuzzy sets, taking a union is finding the degree of membership that an element should have in the new fuzzy set, which is the union of two fuzzy sets.

If a, b, c, and d are such that their degrees of membership in the fuzzy set A are 0.9, 0.4, 0.5, and 0, respectively, then the fuzzy set A is given by the fit vector (0.9, 0.4, 0.5, 0). The components of this fit vector are called fit values of a, b, c, and d.

Union of Fuzzy Sets

Consider a union of two traditional sets and an element that belongs to only one of those sets. Earlier you saw that if you treat these sets as fuzzy sets, this element has a degree of membership of 1 in one case and 0 in the other since it belongs to one set and not the other. Yet you are going to put this element in the union. The criterion you use in this action has to do with degrees of membership. You need to look at the two degrees of membership, namely, 0 and 1, and pick the higher value of the two, namely, 1. In other words, what you want for the degree of membership of an element when listed in the union of two fuzzy sets, is the maximum value of its degrees of membership within the two fuzzy sets forming a union.

If a, b, c, and d have the respective degrees of membership in fuzzy sets A, B as A = (0.9, 0.4, 0.5, 0) and B = (0.7, 0.6, 0.3, 0.8), then A [cup] B = (0.9, 0.6, 0.5, 0.8).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Intersection and Complement of Two Fuzzy Sets

Analogously, the degree of membership of an element in the intersection of two fuzzy sets is the minimum, or the smaller value of its degree of membership individually in the two sets forming the intersection. For example, if today has 0.8 for degree of membership in the set of rainy days and 0.5 for degree of membership in the set of days of work completion, then today belongs to the set of rainy days on which work is completed to a degree of 0.5, the smaller of 0.5 and 0.8.

Recall the fuzzy sets A and B in the previous example. A = (0.9, 0.4, 0.5, 0) and B = (0.7, 0.6, 0.3, 0.8). A[cap]B, which is the intersection of the fuzzy sets A and B, is obtained by taking, in each component, the smaller of the values found in that component in A and in B. Thus A[cap]B = (0.7, 0.4, 0.3, 0).

The idea of a universal set is implicit in dealing with traditional sets. For example, if you talk of the set of married persons, the universal set is the set of all persons. Every other set you consider in that context is a subset of the universal set. We bring up this matter of universal set because when you make the complement of a traditional set A, you need to put in every element in the universal set that is not in A. The complement of a fuzzy set, however, is obtained as follows. In the case of fuzzy sets, if the degree of membership is 0.8 for a member, then that member is not in that set to a degree of 1.0 – 0.8 = 0.2. So you can set the degree of membership in the complement fuzzy set to the complement with respect to 1. If we return to the scenario of having a degree of 0.8 in the set of rainy days, then today has to have 0.2 membership degree in the set of nonrainy or clear days.

Continuing with our example of fuzzy sets A and B, and denoting the complement of A by A', we have A' = (0.1, 0.6, 0.5, 1) and B' = (0.3, 0.4, 0.7, 0.2). Note that A' [cup] B' = (0.3, 0.6, 0.7, 1), which is also the complement of A [cap] B. You can similarly verify that the complement of A [cup] B is the same as A' [cap] B'. Furthermore, A [cup] A' = (0.9, 0.6, 0.5, 1) and A [cap] A' = (0.1, 0.4, 0.5, 0), which is not a vector of zeros only, as would be the case in conventional sets. In fact, A and A' will be equal in the sense that their fit vectors are the same, if each component in the fit vector is equal to 0.5.

Applications of Fuzzy Logic

Applications of fuzzy sets and fuzzy logic are found in many fields, including artificial intelligence, engineering, computer science, operations research, robotics, and pattern recognition. These fields are also ripe for applications for neural networks. So it seems natural that fuzziness should be introduced in neural networks themselves. Any area where humans need to indulge in making decisions, fuzzy sets can find a place, since information on which decisions are to be based may not always be complete and the reliability of the supposed values of the underlying parameters is not always certain.

Examples of Fuzzy Logic

Let us say five tasks have to be performed in a given period of time, and each task requires one person dedicated to it. Suppose there are six people capable of doing these tasks. As you have more than enough people, there is no problem in scheduling this work and getting it done. Of course who gets assigned to which task depends on some criterion, such as total time for completion, on which some optimization can be done. But suppose these six people are not necessarily available during the particular period of time in question. Suddenly, the equation is seen in less than crisp terms. The availability of the people is fuzzy-valued. Here is an example of an assignment problem where fuzzy sets can be used.

Commercial Applications

Many commercial uses of fuzzy logic exist today. A few examples are listed here:

• A subway in Sendai, Japan uses a fuzzy controller to control a subway car. This controller has outperformed human and conventional controllers in giving a smooth ride to passengers in all terrain and external conditions.
• Cameras and camcorders use fuzzy logic to adjust autofocus mechanisms and to cancel the jitter caused by a shaking hand.
• Some automobiles use fuzzy logic for different control applications. Nissan has patents on fuzzy logic braking systems, transmission controls, and fuel injectors. GM uses a fuzzy transmission system in its Saturn vehicles.
• FuziWare has developed and patented a fuzzy spreadsheet called FuziCalc that allows users to incorporate fuzziness in their data.
• Software applications to search and match images for certain pixel regions of interest have been developed. Avian Systems has a software package called FullPixelSearch.
• A stock market charting and research tool called SuperCharts from Omega Research, uses fuzzy logic in one of its modules to determine whether the market is bullish, bearish, or neutral.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Fuzziness in Neural Networks

There are a number of ways fuzzy logic can be used with neural networks. Perhaps the simplest way is to use a fuzzifier function to preprocess or post-process data for a neural network. This is shown in Figure 3.1, where a neural network has a preprocessing fuzzifier that converts data into fuzzy data for application to a neural network.

Figure 3.1 A neural network with fuzzy preprocessor.

Let us build a simple fuzzifier based on an application to predict the direction of the stock market. Suppose that you wish to fuzzify one set of data used in the network, the Federal Reserve's fiscal policy, in one of four fuzzy categories: very accommodative, accommodative, tight, or very tight. Let us suppose that the raw data that we need to fuzzify is the discount rate and the interest rate that the Federal Reserve controls to set the fiscal policy. Now, a low discount rate usually indicates a loose fiscal policy, but this depends not only on the observer, but also on the political climate. There is a probability, for a given discount rate that you will find two people who offer different categories for the Fed fiscal policy. Hence, it is appropriate to fuzzify the data, so that the data we present to the neural network is like what an observer would see.

Figure 3.2 shows the fuzzy categories for different interest rates. Note that the category tight has the largest range. At any given interest rate level, you could have one possible category or several. If only one interest rate is present on the graph, this indicates that membership in that fuzzy set is 1.0. If you have three possible fuzzy sets, there is a requirement that membership add up to 1.0. For an interest rate of 8%, you have some chance of finding this in the tight category or the accommodative category. To find out the percentage probability from the graph, take the height of each curve at a given interest rate and normalize this to a one-unit length. At 8%, the tight category is about 0.8 unit in height, and accommodative is about 0.3 unit in height. The total is about 1.1 units, and the probability of the value being tight is then 0.8/1.1 = 0.73, while the probability of the value being accommodative is 0.27.

Figure 3.2 Fuzzy categories for Federal Reserve policy based on the Fed discount rate.

Code for the Fuzzifier

Let's develop C++ code to create a simple fuzzifier. A class called category is defined in Listing 3.1. This class encapsulates the data that we need to define, the categories in Figure 3.2. There are three private data members called lowval, midval, and highval. These represent the values on the graph that define the category triangle. In the tight category, the lowval is 5.0, the midval is 8.5, and the highval is 12.0. The category class allows you to instantiate a category object and assign parameters to it to define it. Also, there is a string called name that identifies the category, e.g. "tight." Various member functions are used to interface to the private data members. There is setval(), for example, which lets you set the value of the three parameters, while gethighval() returns the value of the parameter highval. The function getshare() returns the relative value of membership in a category given an input. In the example discussed earlier, with the number 8.0 as the Fed discount rate and the category tight defined according to the graph in Figure 3.2, getshare() would return 0.8. Note that this is not yet normalized. Following this example, the getshare() value from the accommodative category would also be used to determine the membership weights. These weights define a probability in a given category. A random number generator is used to define a value that is used to select a fuzzy category based on the probabilities defined.

Listing 3.1 fuzzfier.h

// fuzzfier.h V. Rao, H. Rao

// program to fuzzify data

class category

{

private:

     char name[30];

     float   lowval,highval,midval;

public:

     category(){};

     void setname(char *);

     char * getname();

     void setval(float&,float&,float&);

     float getlowval();

     float getmidval();

     float gethighval();

     float getshare(const float&);

     ~category(){};

};

int randnum(int);


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Let's look at the implementation file in Listing 3.2.

Listing 3.2 fuzzfier.cpp

// fuzzfier.cpp      V. Rao, H. Rao

// program to fuzzify data

#include <iostream.h>

#include <stdlib.h>

#include <time.h>

#include <string.h>

#include <fuzzfier.h>

void category::setname(char *n)

{

strcpy(name,n);

}

char * category::getname()

{

return name;

}

void category::setval(float &h, float &m, float &l)

{

highval=h;

midval=m;

lowval=l;

}

float category::getlowval()

{

return lowval;

}

float category::getmidval()

{

return midval;

}

float category::gethighval()

{

return highval;

}

float category::getshare(const float & input)

{

// this member function returns the relative membership

// of an input in a category, with a maximum of 1.0

float output;

float midlow, highmid;

midlow=midval-lowval;

highmid=highval-midval;

// if outside the range, then output=0

if ((input <= lowval) || (input >= highval))

     output=0;

else

     {

     if (input > midval)

          output=(highval-input)/highmid;

     else

     if (input==midval)

          output=1.0;

     else

          output=(input-lowval)/midlow;

     }

return output;

}

int randomnum(int maxval)

{

// random number generator

// will return an integer up to maxval

srand ((unsigned)time(NULL));

return rand() % maxval;

}

void main()

{

// a fuzzifier program that takes category information:

// lowval, midval and highval and category name

// and fuzzifies an input based on

// the total number of categories and the membership

// in each category

int i=0,j=0,numcat=0,randnum;

float l,m,h, inval=1.0;

char input[30]="               ";

category * ptr[10];

float relprob[10];

float total=0, runtotal=0;

//input the category information; terminate with `done';

while (1)

     {

     cout << "\nPlease type in a category name, e.g. Cool\n";

     cout << "Enter one word without spaces\n";

     cout << "When you are done, type `done' :\n\n";

     ptr[i]= new category;

     cin >> input;

     if ((input[0]=='d' && input[1]=='o' &&

             input[2]=='n' && input[3]=='e')) break;

     ptr[i]->setname(input);

     cout << "\nType in the lowval, midval and highval\n";

     cout << "for each category, separated by spaces\n";

     cout << " e.g. 1.0 3.0 5.0 :\n\n";

     cin >> l >> m >> h;

     ptr[i]->setval(h,m,l);

     i++;

     }

numcat=i; // number of categories

// Categories set up: Now input the data to fuzzify

cout <<"\n\n";

cout << "===================================\n";

cout << "==Fuzzifier is ready for data==\n";

cout << "===================================\n";

while (1)

     {

     cout << "\ninput a data value, type 0 to terminate\n";

     cin >> inval;

     if (inval == 0) break;

     // calculate relative probabilities of

     //   input being in each category

     total=0;

     for (j=0;j<numcat;j++)

          {

          relprob[j]=100*ptr[j]->getshare(inval);

          total+=relprob[j];

          }

     if (total==0)

          {

          cout << "data out of range\n";

          exit(1);

          }

     randnum=randomnum((int)total);

     j=0;

     runtotal=relprob[0];

     while ((runtotal<randnum)&&(j<numcat))

          {

          j++;

          runtotal += relprob[j];

          }

     cout << "\nOutput fuzzy category is ==> " <<

          ptr[j]->getname()<<"<== \n";

          cout <<"category\t"<<"membership\n";

          cout <<"---------------\n";

     for (j=0;j<numcat;j++)

          {

          cout << ptr[j]->getname()<<"\t\t"<<

               (relprob[j]/total) <<"\n";

          }

     }

cout << "\n\nAll done. Have a fuzzy day !\n";

}

This program first sets up all the categories you define. These could be for the example we choose or any example you can think of. After the categories are defined, you can start entering data to be fuzzified. As you enter data you see the probability aspect come into play. If you enter the same value twice, you may end up with different categories! You will see sample output shortly, but first a technical note on how the weighted probabilities are set up. The best way to explain it is with an example. Suppose that you have defined three categories, A, B, and C. Suppose that category A has a relative membership of 0.8, category B of 0.4, and category C of 0.2. In the program, these numbers are first multiplied by 100, so you end up with A=80, B=40, and C=20. Now these are stored in a vector with an index j initialized to point to the first category. Let's say that these three numbers represent three adjacent number bins that are joined together. Now pick a random number to index into the bin that has its maximum value of (80+40+20). If the number is 100, then it is greater than 80 and less than (80+40), you end up in the second bin that represents B. Does this scheme give you weighted probabilities? Yes it does, since the size of the bin (given a uniform distribution of random indexes into it) determines the probability of falling into the bin. Therefore, the probability of falling into bin A is 80/(80+40+20).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Sample output from the program is shown below. Our input is in italic; computer output is not. The categories defined by the graph in Figure 3.2 are entered in this example. Once the categories are set up, the first data entry of 4.0 gets fuzzified to the accommodative category. Note that the memberships are also presented in each category. The same value is entered again, and this time it gets fuzzified to the very accommodative category. For the last data entry of 12.5, you see that only the very tight category holds membership for this value. In all cases you will note that the memberships add up to 1.0.

 fuzzfier

Please type in a category name, e.g. Cool

Enter one word without spaces

When you are done, type `done' :

v.accommodative

Type in the lowval, midval and highval

for each category, separated by spaces

 e.g. 1.0 3.0 5.0 :

0 3 6

Please type in a category name, e.g. Cool

Enter one word without spaces

When you are done, type `done' :

accommodative

Type in the lowval, midval and highval

for each category, separated by spaces

 e.g. 1.0 3.0 5.0 :

3 6 9

Please type in a category name, e.g. Cool

Enter one word without spaces

When you are done, type `done' :

tight

Type in the lowval, midval and highval

for each category, separated by spaces

 e.g. 1.0 3.0 5.0 :

5 8.5 12

Please type in a category name, e.g. Cool

Enter one word without spaces

When you are done, type `done' :

v.tight

Type in the lowval, midval and highval

for each category, separated by spaces

 e.g. 1.0 3.0 5.0 :

10 12 14

Please type in a category name, e.g. Cool

Enter one word without spaces

When you are done, type `done' :

done

===================================

==Fuzzifier is ready for data==

===================================

input a data value, type 0 to terminate

4.0

Output fuzzy category is ==> accommodative<==

category   membership

-----------------------------

v.accommodative      0.666667

accommodative        0.333333

tight        0

v.tight      0

input a data value, type 0 to terminate

4.0

Output fuzzy category is ==> v.accommodative<==

category   membership

-----------------------------

v.accommodative      0.666667

accommodative        0.333333

tight        0

v.tight      0

input a data value, type 0 to terminate

7.5

Output fuzzy category is ==> accommodative<==

category   membership

-----------------------------

v.accommodative      0

accommodative        0.411765

tight        0.588235

v.tight      0

input a data value, type 0 to terminate

11.0

       Output fuzzy category is ==> tight<==

category   membership

-----------------------------

v.accommodative      0

accommodative        0

tight        0.363636

v.tight      0.636364

input a data value, type 0 to terminate

12.5

Output fuzzy category is ==> v.tight<==

category   membership

-----------------------------

v.accommodative      0

accommodative        0

tight        0

v.tight      1

input a data value, type 0 to terminate

0

All done. Have a fuzzy day !

Fuzzy Control Systems

The most widespread use of fuzzy logic today is in fuzzy control applications. You can use fuzzy logic to make your air conditioner cool your room. Or you can design a subway system to use fuzzy logic to control the braking system for smooth and accurate stops. A control system is a closed-loop system that typically controls a machine to achieve a particular desired response, given a number of environmental inputs. A fuzzy control system is a closed-loop system that uses the process of fuzzification, as shown in the Federal Reserve policy program example, to generate fuzzy inputs to an inference engine, which is a knowledge base of actions to take. The inverse process, called defuzzification, is also used in a fuzzy control system to create crisp, real values to apply to the machine or process under control. In Japan, fuzzy controllers have been used to control many machines, including washing machines and camcorders.

Figure 3.3 shows a diagram of a fuzzy control system. The major parts of this closed-loop system are:

Figure 3.3 Diagram of a fuzzy control system.

• machine under control—this is the machine or process that you are controlling, for example, a washing machine
• outputs—these are the measured response behaviors of your machine, for example, the temperature of the water
• fuzzy outputs—these are the same outputs passed through a fuzzifier, for example, hot or very cold
• inference engine/fuzzy rule base—an inference engine converts fuzzy outputs to actions to take by accessing fuzzy rules in a fuzzy rule base. An example of a fuzzy rule: IF the output is very cold, THEN increase the water temperature setting by a very large amount
• fuzzy inputs—these are the fuzzy actions to perform, such as increase the water temperature setting by a very large amount
• inputs—these are the (crisp) dials on the machine to control its behavior, for example, water temperature setting = 3.423, converted from fuzzy inputs with a defuzzifier

The key to development of a fuzzy control system is to iteratively construct a fuzzy rule base that yields the desired response from your machine. You construct these fuzzy rules from knowledge about the problem. In many cases this is very intuitive and gives you a robust control system in a very short amount of time.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Fuzziness in Neural Networks

Fuzziness can enter neural networks to define the weights from fuzzy sets. A comparison between expert systems and fuzzy systems is important to understand in the context of neural networks. Expert systems are based on crisp rules. Such crisp rules may not always be available. Expert systems have to consider an exhaustive set of possibilities. Such sets may not be known beforehand. When crisp rules are not possible, and when it is not known if the possibilities are exhaustive, the expert systems approach is not a good one.

Some neural networks, through the features of training and learning, can function in the presence of unexpected situations. Therein neural networks have an advantage over expert systems, and they can manage with far less information than expert systems need.

One form of fuzziness in neural networks is called a fuzzy cognitive map. A fuzzy cognitive map is like a dynamic state machine with fuzzy states. A traditional state machine is a machine with defined states and outputs associated with each state. Transitions from state to state take place according to input events or stimuli. A fuzzy cognitive map looks like a state machine but has fuzzy states (not just 1 or 0). You have a set of weights along each transition path, and these weights can be learned from a set of training data.

Our treatment of fuzziness in neural networks is with the discussion of the fuzzy associative memory, abbreviated as FAM, which, like the fuzzy cognitive map, was developed by Bart Kosko. The FAM and the C++ implementation are discussed in Chapter 9.

Neural-Trained Fuzzy Systems

So far we have considered how fuzzy logic plays a role in neural networks. The converse relationship, neural networks in fuzzy systems, is also an active area of research. In order to build a fuzzy system, you must have a set of membership rules for fuzzy categories. It is sometimes difficult to deduce these membership rules with a given set of complex data. Why not use a neural network to define the fuzzy rules for you? A neural network is good at discovering relationships and patterns in data and can be used to preprocess data in a fuzzy system. Further, a neural network that can learn new relationships with new input data can be used to refine fuzzy rules to create a fuzzy adaptive system. Neural trained fuzzy systems are being used in many commercial applications, especially in Japan:

• The Laboratory for International Fuzzy Engineering Research (LIFE) in Yokohama, Japan has a backpropagation neural network that derives fuzzy rules and membership functions. The LIFE system has been successfully applied to a foreign-exchange trade support system with approximately 5000 fuzzy rules.
• Ford Motor Company has developed trainable fuzzy systems for automobile idle-speed control.
• National Semiconductor Corporation has a software product called NeuFuz that supports the generation of fuzzy rules with a neural network for control applications.
• A number of Japanese consumer and industrial products use neural networks with fuzzy systems, including vacuum cleaners, rice cookers, washing machines, and photocopying machines.
• AEG Corporation of Germany uses a neural-network-trained fuzzy control system for its water- and energy-conserving washing machine. After the machine is loaded with laundry, it measures the water level with a pressure sensor and infers the amount of laundry in the machine by the speed and volume of water. A total of 157 rules were generated by a neural network that was trained on data correlating the amount of laundry with the measurement of water level on the sensor.

Summary

In this chapter, you read about fuzzy logic, fuzzy sets, and simple operations on fuzzy sets. Fuzzy logic, unlike Boolean logic, has more than two on or off categories to describe behavior of systems. You use membership values for data in fuzzy categories, which may overlap. In this chapter, you also developed a fuzzifier program in C++ that takes crisp values and converts them to fuzzy values, based on categories and memberships that you define. For use with neural networks, fuzzy logic can serve as a post-processing or pre-processing filter. Kosko developed neural networks that use fuzziness and called them fuzzy associative memories, which will be discussed in later chapters. You also read about how neural networks can be used in fuzzy systems to define membership functions and fuzzy rules.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 4
Constructing a Neural Network

First Example for C++ Implementation

The neural network we presented in Chapter 1 is an example of a Hopfield network with a single layer. Now we present a C++ implementation of this network. Suppose we place four neurons, all connected to one another on this layer, as shown in Figure 4.1. Some of these connections have a positive weight and the rest have a negative weight. You may recall from the earlier presentation of this example, that we used two input patterns to determine the weight matrix. The network recalls them when the inputs are presented to the network, one at a time. These inputs are binary and orthogonal so that their stable recall is assured. Each component of a binary input pattern is either a 0 or a 1. Two vectors are orthogonal when their dot product—the sum of the products of their corresponding components—is zero. An example of a binary input pattern is 1 0 1 0 0. An example of a pair of orthogonal vectors is (0, 1, 0, 0, 1) and (1, 0, 0, 1, 0). An example of a pair of vectors that are not orthogonal is (0, 1, 0, 0, 1) and (1, 1, 0, 1, 0). These last two vectors have a dot product of 1, different from 0.

Figure 4.1 Layout of a Hopfield Network

The two patterns we want the network to have stable recall for are A = (1, 0, 1, 0) and B = (0, 1, 0, 1). The weight matrix W is given as follows:

               0    -3     3     -3

     W     =  -3     0    -3      3

               3    -3     0     -3

              -3     3    -3      0

NOTE: The positive links (values with positive signs) tend to encourage agreement in a stable configuration, whereas negative links (values with negative signs) tend to discourage agreement in a stable configuration.

We need a threshold function also, and we define it using a threshold value, [theta], as follows:

                  1  if t >= [theta]

     f(t) =  {

                  0  if t < [theta]

The threshold value [theta] is used as a cut-off value for the activation of a neuron to enable it to fire. The activation should equal or exceed the threshold value for the neuron to fire, meaning to have output 1. For our Hopfield network, [theta] is taken as 0. There are four neurons in the only layer in this network. The first node's output is the output of the threshold function. The argument for the threshold function is the activation of the node. And the activation of the node is the dot product of the input vector and the first column of the weight matrix. So if the input vector is A, the dot product becomes 3, and f(3) = 1. And the dot products of the second, third, and fourth nodes become –6, 3, and –6, respectively. The corresponding outputs therefore are 0, 1, and 0. This means that the output of the network is the vector (1, 0, 1, 0), which is the same as the input pattern. Therefore, the network has recalled the pattern as presented. When B is presented, the dot product obtained at the first node is –6 and the output is 0. The activations of all the four nodes together with the threshold function give (0, 1, 0, 1) as output from the network, which means that the network recalled B as well. The weight matrix worked well with both input patterns, and we do not need to modify it.

Classes in C++ Implementation

In our C++ implementation of this network, there are the following classes: a network class, and a neuron class. In our implementation, we create the network with four neurons, and these four neurons are all connected to one another. A neuron is not self-connected, though. That is, there is no edge in the directed graph representing the network, where the edge is from one node to itself. But for simplicity, we could pretend that such a connection exists carrying a weight of 0, so that the weight matrix has 0's in its principal diagonal.

The functions that determine the neuron activations and the network output are declared public. Therefore they are visible and accessible without restriction. The activations of the neurons are calculated with functions defined in the neuron class. When there are more than one layer in a neural network, the outputs of neurons in one layer become the inputs for neurons in the next layer. In order to facilitate passing the outputs from one layer as inputs to another layer, our C++ implementations compute the neuron outputs in the network class. For this reason the threshold function is made a member of the network class. We do this for the Hopfield network as well. To see if the network has achieved correct recall, you make comparisons between the presented pattern and the network output, component by component.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

C++ Program for a Hopfield Network

For convenience every C++ program has two components: One is the header file with all of the class declarations and lists of include library files; the other is the source file that includes the header file and the detailed descriptions of the member functions of the classes declared in the header file. You also put the function main in the source file. Most of the computations are done by class member functions, when class objects are created in the function main, and calls are made to the appropriate functions. The header file has an .h (or .hpp) extension, as you know, and the source file has a .cpp extension, to indicate that it is a C++ code file. It is possible to have the contents of the header file written at the beginning of the .cpp file and work with one file only, but separating the declarations and implementations into two files allows you to change the implementation of a class(.cpp) without changing the interface to the class (.h).

Header File for C++ Program for Hopfield Network

Listing 4.1 contains Hop.h, the header file for the C++ program for the Hopfield network. The include files listed in it are the stdio.h, iostream.h, and math.h. The iostream.h file contains the declarations and details of the C++ streams for input and output. A network class and a neuron class, are declared in Hop.h. The data members and member functions are declared within each class, and their accessibility is specified by the keywords protected or public.

Listing 4.1 Header file for C++ program for Hopfield network.

//Hop.h      V. Rao, H. Rao

//Single layer Hopfield Network with 4 neurons

#include <stdio.h>

#include <iostream.h>

#include <math.h>

class neuron

{

protected:

     int activation;

     friend class network;

public:

     int weightv[4];

     neuron() {};

     neuron(int *j) ;

     int act(int, int*);

};

class network

{

public:

     neuron   nrn[4];

     int output[4];

     int threshld(int) ;

     void activation(int j[4]);

     network(int*,int*,int*,int*);

};

Notes on the Header File Hop.h

Notice that the data item activation in the neuron class is declared as protected. In order to make the member activation of the neuron class accessible to the network class, the network is declared a friend class in the class neuron. Also, there are two constructors for the class neuron. One of them creates the object neuron without initializing any data members. The other creates the object neuron and initializes the connection weights.

Source Code for the Hopfield Network

Listing 4.2 contains the source code for the C++ program for a Hopfield network in the file Hop.cpp. The member functions of the classes declared in Hop.h are implemented here. The function main contains the input patterns, values to initialize the weight matrix, and calls to the constructor of network class and other member functions of the network class.

Listing 4.2 Source code for C++ program for Hopfield network.

//Hop.cpp   V. Rao, H. Rao

//Single layer Hopfield Network with 4 neurons

#include "hop.h"

neuron::neuron(int *j)

{

int i;

for(i=0;i<4;i++)

     {

     weightv[i]= *(j+i);

     }

}

int neuron::act(int m, int *x)

{

int i;

int a=0;

for(i=0;i<m;i++)

     {

     a += x[i]*weightv[i];

     }

return a;

}

int network::threshld(int k)

{

if(k>=0)

     return (1);

else

     return (0);

}

network::network(int a[4],int b[4],int c[4],int d[4])

{

nrn[0] = neuron(a) ;

nrn[1] = neuron(b) ;

nrn[2] = neuron(c) ;

nrn[3] = neuron(d) ;

}

void network::activation(int *patrn)

{

int i,j;

for(i=0;i<4;i++)

     {

     for(j=0;j<4;j++)

          {

          cout<<"\n nrn["<<i<<"].weightv["<<j<<"] is "

               <<nrn[i].weightv[j];

          }

     nrn[i].activation = nrn[i].act(4,patrn);

     cout<<"\nactivation is "<<nrn[i].activation;

     output[i]=threshld(nrn[i].activation);

     cout<<"\noutput value is  "<<output[i]<<"\n";

     }

}

void main ()

{

int patrn1[]= {1,0,1,0},i;

int wt1[]= {0,-3,3,-3};

int wt2[]= {-3,0,-3,3};

int wt3[]= {3,-3,0,-3};

int wt4[]= {-3,3,-3,0};

cout<<"\nTHIS PROGRAM IS FOR A HOPFIELD NETWORK WITH A SINGLE LAYER OF";

cout<<"\n4 FULLY INTERCONNECTED NEURONS. THE NETWORK SHOULD RECALL THE";

cout<<"\nPATTERNS 1010 AND 0101 CORRECTLY.\n";

//create the network by calling its constructor.

// the constructor calls neuron constructor as many times as the number of

// neurons in the network.

network h1(wt1,wt2,wt3,wt4);

//present a pattern to the network and get the activations of the neurons

h1.activation(patrn1);

//check if the pattern given is correctly recalled and give message

for(i=0;i<4;i++)

     {

     if (h1.output[i] == patrn1[i])

          cout<<"\n pattern= "<<patrn1[i]<<

          "  output = "<<h1.output[i]<<"  component matches";

     else

          cout<<"\n pattern= "<<patrn1[i]<<

          "  output = "<<h1.output[i]<<

          "  discrepancy occurred";

     }

cout<<"\n\n";

int patrn2[]= {0,1,0,1};

h1.activation(patrn2);

for(i=0;i<4;i++)

     {

     if (h1.output[i] == patrn2[i])

          cout<<"\n pattern= "<<patrn2[i]<<

          "  output = "<<h1.output[i]<<"  component matches";

     else

          cout<<"\n pattern= "<<patrn2[i]<<

          "  output = "<<h1.output[i]<<

          "  discrepancy occurred";

       }

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Comments on the C++ Program for Hopfield Network

Note the use of the output stream operator cout<< to output text strings or numerical output. C++ has istream and ostream classes from which the iostream class is derived. The standard input and output streams are cin and cout, respectively, used, correspondingly, with the operators >> and <<. Use of cout for the output stream is much simpler than the use of the C function printf. As you can see, there is no formatting suggested for output. However, there is a provision that allows you to format the output, while using cout.

Also note the way comments are introduced in the program. The line with comments should start with a double slash //. Unlike C, the comment does not have to end with a double slash. Of course, if the comments extend to subsequent lines, each such line should have a double slash at the start. You can still use the pair, /* at the beginning with */ at the end of lines of comments, as you do in C. If the comment continues through many lines, the C facility will be handier to delimit the comments.

The neurons in the network are members of the network class and are identified by the abbreviation nrn. The two patterns, 1010 and 0101, are presented to the network one at a time in the program.

Output from the C++ Program for Hopfield Network

The output from this program is as follows and is self-explanatory. When you run this program, you're likely to see a lot of output whiz by, so in order to leisurely look at the output, use redirection. Type Hop > filename, and your output will be stored in a file, which you can edit with any text editor or list by using the type filename | more command.

THIS PROGRAM IS FOR A HOPFIELD NETWORK WITH A SINGLE LAYER OF 4 FULLY

INTERCONNECTED NEURONS. THE NETWORK SHOULD RECALL THE PATTERNS 1010 AND

0101 CORRECTLY.

 nrn[0].weightv[0] is  0

 nrn[0].weightv[1] is  -3

 nrn[0].weightv[2] is  3

 nrn[0].weightv[3] is  -3

activation is 3

output value is  1

 nrn[1].weightv[0] is  -3

 nrn[1].weightv[1] is  0

 nrn[1].weightv[2] is  -3

 nrn[1].weightv[3] is  3

activation is -6

output value is  0

 nrn[2].weightv[0] is  3

 nrn[2].weightv[1] is  -3

 nrn[2].weightv[2] is  0

 nrn[2].weightv[3] is  -3

activation is 3

output value is  1

 nrn[3].weightv[0] is  -3

 nrn[3].weightv[1] is  3

 nrn[3].weightv[2] is  -3

 nrn[3].weightv[3] is  0

activation is -6

output value is  0

 pattern= 1  output = 1  component matches

 pattern= 0  output = 0  component matches

 pattern= 1  output = 1  component matches

 pattern= 0  output = 0  component matches

 nrn[0].weightv[0] is  0

 nrn[0].weightv[1] is  -3

 nrn[0].weightv[2] is  3

 nrn[0].weightv[3] is  -3

activation is -6

output value is  0

 nrn[1].weightv[0] is  -3

 nrn[1].weightv[1] is  0

 nrn[1].weightv[2] is  -3

 nrn[1].weightv[3] is  3

activation is 3

output value is  1

 nrn[2].weightv[0] is  3

 nrn[2].weightv[1] is  -3

 nrn[2].weightv[2] is  0

 nrn[2].weightv[3] is  -3

activation is -6

output value is  0

 nrn[3].weightv[0] is  -3

 nrn[3].weightv[1] is  3

 nrn[3].weightv[2] is  -3

 nrn[3].weightv[3] is  0

activation is 3

output value is  1

 pattern= 0  output = 0  component matches

 pattern= 1  output = 1  component matches

 pattern= 0  output = 0  component matches

 pattern= 1  output = 1  component matches

Further Comments on the Program and Its Output

Let us recall our previous discussion of this example in Chapter 1. What does the network give as output if we present a pattern different from both A and B? If C = (0, 1, 0, 0) is the input pattern, the activation (dot products) would be –3, 0, –3, 3 making the outputs (next state) of the neurons 0,1,0,1, so that B would be recalled. This is quite interesting, because if we intended to input B, and we made a slight error and ended up presenting C instead, the network would recall B. You can run the program by changing the pattern to 0, 1, 0, 0 and compiling again, to see that the B pattern is recalled.

Another element about the example in Chapter 1 is that the weight matrix W is not the only weight matrix that would enable the network to recall the patterns A and B correctly. If we replace the 3 and –3 in the matrix with 2 and –2, respectively, the resulting matrix would facilitate the same performance from the network. One way for you to check this is to change the wt1, wt2, wt3, wt4 given in the program accordingly, and compile and run the program again. The reason why both of the weight matrices work is that they are closely related. In fact, one is a scalar (constant) multiple of the other, that is, if you multiply each element in the matrix by the same scalar, namely 2/3, you get the corresponding matrix in cases where 3 and –3 are replaced with 2 and –2, respectively.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

A New Weight Matrix to Recall More Patterns

Let's continue to discuss this example. Suppose we are interested in having the patterns E = (1, 0, 0, 1) and F = (0, 1, 1, 0) also recalled correctly, in addition to the patterns A and B. In this case we would need to train the network and come up with a learning algorithm, which we will discuss in more detail later in the book. We come up with the matrix W1, which follows.

             0    -5     4    4

     W1 =   -5     0     4    4

             4     4     0   -5

             4     4    -5    0

Try to use this modification of the weight matrix in the source program, and then compile and run the program to see that the network successfully recalls all four patterns A, B, E, and F.

NOTE: The C++ implementation shown does not include the asynchronous update feature mentioned in Chapter 1, which is not necessary for the patterns presented. The coding of this feature is left as an exercise for the reader.

Weight Determination

You may be wondering about how these weight matrices were developed in the previous example, since so far we've only discussed how the network does its job, and how to implement the model. You have learned that the choice of weight matrix is not necessarily unique. But you want to be assured that there is some established way besides trial and error, in which to construct a weight matrix. You can go about this in the following way.

Binary to Bipolar Mapping

Let's look at the previous example. You have seen that by replacing each 0 in a binary string with a –1, you get the corresponding bipolar string. If you keep all 1's the same and replace each 0 with a –1, you will have a formula for the above option. You can apply the following function to each bit in the string:

     f(x) = 2x – 1

NOTE: When you give the binary bit x, you get the corresponding bipolar character f(x)

For inverse mapping, which turns a bipolar string into a binary string, you use the following function:

     f(x) =  (x + 1) / 2

NOTE: When you give the bipolar character x, you get the corresponding binary bit f(x)

Pattern's Contribution to Weight

Next, we work with the bipolar versions of the input patterns. You take each pattern to be recalled, one at a time, and determine its contribution to the weight matrix of the network. The contribution of each pattern is itself a matrix. The size of such a matrix is the same as the weight matrix of the network. Then add these contributions, in the way matrices are added, and you end up with the weight matrix for the network, which is also referred to as the correlation matrix. Let us find the contribution of the pattern A = (1, 0, 1, 0):

First, we notice that the binary to bipolar mapping of A = (1, 0, 1, 0) gives the vector (1, –1, 1, –1).

Then we take the transpose, and multiply, the way matrices are multiplied, and we see the following:

     1  [1   -1   1   -1]       1   -1   1   -1

     1                     =   -1    1  -1    1

     1                          1   -1   1   -1

     1                         -1    1  -1    1

Now subtract 1 from each element in the main diagonal (that runs from top left to bottom right). This operation gives the same result as subtracting the identity matrix from the given matrix, obtaining 0's in the main diagonal. The resulting matrix, which is given next, is the contribution of the pattern (1, 0, 1, 0) to the weight matrix.

      0      -1      1     -1

     -1       0     -1      1

      1      -1      0     -1

     -1       1     -1      0

Similarly, we can calculate the contribution from the pattern B = (0, 1, 0, 1) by verifying that pattern B's contribution is the same matrix as pattern A's contribution. Therefore, the matrix of weights for this exercise is the matrix W shown here.

           0     -2      2      -2

  W  =    -2      0     -2       2

           2     -2      0      -2

          -2      2     -2       0

You can now optionally apply an arbitrary scalar multiplier to all the entries of the matrix if you wish. This is how we had previously obtained the +/- 3 values instead of +/- 2 values shown above.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Autoassociative Network

The Hopfield network just shown has the feature that the network associates an input pattern with itself in recall. This makes the network an autoassociative network. The patterns used for determining the proper weight matrix are also the ones that are autoassociatively recalled. These patterns are called the exemplars. A pattern other than an exemplar may or may not be recalled by the network. Of course, when you present the pattern 0 0 0 0, it is stable, even though it is not an exemplar pattern.

Orthogonal Bit Patterns

You may be wondering how many patterns the network with four nodes is able to recall. Let us first consider how many different bit patterns are orthogonal to a given bit pattern. This question really refers to bit patterns in which at least one bit is equal to 1. A little reflection tells us that if two bit patterns are to be orthogonal, they cannot both have 1's in the same position, since the dot product would need to be 0. In other words, a bitwise logical AND operation of the two bit patterns has to result in a 0. This suggests the following. If a pattern P has k, less than 4, bit positions with 0 (and so 4-k bit positions with 1), and if pattern Q is to be orthogonal to P, then Q can have 0 or 1 in those k positions, but it must have only 0 in the rest 4-k positions. Since there are two choices for each of the k positions, there are 2k possible patterns orthogonal to P. This number 2k of patterns includes the pattern with all zeroes. So there really are 2k–1 non-zero patterns orthogonal to P. Some of these 2k–1 patterns are not orthogonal to each other. As an example, P can be the pattern 0 1 0 0, which has k = 3 positions with 0. There are 23–1=7 nonzero patterns orthogonal to 0 1 0 0. Among these are patterns 1 0 1 0 and 1 0 0 1, which are not orthogonal to each other, since their dot product is 1 and not 0.

Network Nodes and Input Patterns

Since our network has four neurons in it, it also has four nodes in the directed graph that represents the network. These are laterally connected because connections are established from node to node. They are lateral because the nodes are all in the same layer. We started with the patterns A = (1, 0, 1, 0) and B = (0, 1, 0, 1) as the exemplars. If we take any other nonzero pattern that is orthogonal to A, it will have a 1 in a position where B also has a 1. So the new pattern will not be orthogonal to B. Therefore, the orthogonal set of patterns that contains A and B can have only those two as its elements. If you remove B from the set, you can get (at most) two others to join A to form an orthogonal set. They are the patterns (0, 1, 0, 0) and (0, 0, 0, 1).

If you follow the procedure described earlier to get the correlation matrix, you will get the following weight matrix:

             0     -1      3      -1

     W  =   -1      0     -1      -1

             3     -1      0      -1

            -1     -1     -1       0

With this matrix, pattern A is recalled, but the zero pattern (0, 0, 0, 0) is obtained for the two patterns (0, 1, 0, 0) and (0, 0, 0, 1). Once the zero pattern is obtained, its own recall will be stable.

Second Example for C++ Implementation

Recall the cash register game from the show The Price is Right, used as one of the examples in Chapter 1. This example led to the description of the Perceptron neural network. We will now resume our discussion of the Perceptron model and follow up with its C++ implementation. Keep the cash register game example in mind as you read the following C++ implementation of the Perceptron model. Also note that the input signals in this example are not necessarily binary, but they may be real numbers. It is because the prices of the items the contestant has to choose are real numbers (dollars and cents). A Perceptron has one layer of input neurons and one layer of output neurons. Each input layer neuron is connected to each neuron in the output layer.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

C++ Implementation of Perceptron Network

In our C++ implementation of this network, we have the following classes: we have separate classes for input neurons and output neurons. The ineuron class is for the input neurons. This class has weight and activation as data members. The oneuron class is similar and is for the output neuron. It is declared as a friend class in the ineuron class. The output neuron class has also a data member called output. There is a network class, which is a friend class in the oneuron class. An instance of the network class is created with four input neurons. These four neurons are all connected with one output neuron.

The member functions of the ineuron class are: (1) a default constructor, (2) a second constructor that takes a real number as an argument, and (3) a function that calculates the output of the input neuron. The constructor taking one argument uses that argument to set the value of the weight on the connection between the input neuron and the output neuron. The functions that determine the neuron activations and the network output are declared public. The activations of the neurons are calculated with functions defined in the neuron classes. A threshold value is used by a member function of the output neuron to determine if the neuron's activation is large enough for it to fire, giving an output of 1.

Header File

Listing 4.3 contains percept.h, the header file for the C++ program for the Perceptron network. percept.h contains the declarations for three classes, one for input neurons, one for output neurons, and one for network.

Listing 4.3 The percept.h header file.

//percept.h        V. Rao, H. Rao

// Perceptron model

#include <stdio.h>

#include <iostream.h>

#include <math.h>

class ineuron

{

protected:

     float weight;

     float activation;

     friend class oneuron;

public:

     ineuron() {};

     ineuron(float j) ;

     float act(float x);

};

class oneuron

{

protected:

     int output;

     float activation;

     friend class network;

public:

     oneuron() { };

     void actvtion(float x[4], ineuron *nrn);

     int outvalue(float j) ;

};

class network

{

public:

     ineuron   nrn[4];

     oneuron   onrn;

     network(float,float,float,float);

};

Implementation of Functions

The network is designed to have four neurons in the input layer. Each of them is an object of class ineuron, and these are member classes in the class network. There is one explicitly defined output neuron of the class oneuron. The network constructor also invokes the neuron constructor for each input layer neuron in the network by providing it with the initial weight for its connection to the neuron in the output layer. The constructor for the output neuron is also invoked by the network constructor, at the same time initializing the output and activation data members of the output neuron each to zero. To make sure there is access to needed information and functions, the output neuron is declared a friend class in the class ineuron. The network is declared as a friend class in the class oneuron.

Source Code for Perceptron Network

Listing 4.4 contains the source code in percept.cpp for the C++ implementation of the Perceptron model previously discussed.

Listing 4.4 Source code for Perceptron model.

//percept.cpp   V. Rao, H. Rao

//Perceptron model

#include "percept.h"

#include "stdio.h"

#include "stdlib.h"

ineuron::ineuron(float j)

{

weight= j;

}

float ineuron::act(float x)

{

float a;

a = x*weight;

return a;

}

void oneuron::actvtion(float *inputv, ineuron *nrn)

{

int i;

activation = 0;

for(i=0;i<4;i++)

     {

     cout<<"\nweight for neuron "<<i+1<<" is       "<<nrn[i].weight;

     nrn[i].activation = nrn[i].act(inputv[i]);

     cout<<"           activation is      "<<nrn[i].activation;

     activation += nrn[i].activation;

     }

cout<<"\n\nactivation is  "<<activation<<"\n";

}

int oneuron::outvalue(float j)

{

if(activation>=j)

     {

     cout<<"\nthe output neuron activation \

exceeds the threshold value of "<<j<<"\n";

     output = 1;

     }

else

     {

     cout<<"\nthe output neuron activation \

is smaller than the threshold value of "<<j<<"\n";

     output = 0;

     }

cout<<" output value is "<< output;

return (output);

}

network::network(float a,float b,float c,float d)

{

nrn[0] = ineuron(a) ;

nrn[1] = ineuron(b) ;

nrn[2] = ineuron(c) ;

nrn[3] = ineuron(d) ;

onrn = oneuron();

onrn.activation = 0;

onrn.output = 0;

}

void main (int argc, char * argv[])

{

float inputv1[]= {1.95,0.27,0.69,1.25};

float wtv1[]= {2,3,3,2}, wtv2[]= {3,0,6,2};

FILE * wfile, * infile;

int num=0, vecnum=0, i;

float threshold = 7.0;

if (argc < 2)

     {

     cerr << "Usage: percept Weightfile Inputfile";

     exit(1);

     }

// open  files

wfile= fopen(argv[1], "r");

infile= fopen(argv[2], "r");

if ((wfile == NULL) || (infile == NULL))

     {

     cout << " Can't open a file\n";

     exit(1);

     }

cout<<"\nTHIS PROGRAM IS FOR A PERCEPTRON NETWORK WITH AN INPUT LAYER OF";

cout<<"\n4 NEURONS, EACH CONNECTED TO THE OUTPUT NEURON.\n";

cout<<"\nTHIS EXAMPLE TAKES REAL NUMBERS AS INPUT SIGNALS\n";

//create the network by calling its constructor.

//the constructor calls neuron constructor as many times as the number of

//neurons in input layer of the network.

cout<<"please enter the number of weights/vectors \n";

cin >> vecnum;

for (i=1;i<=vecnum;i++)

     {

     fscanf(wfile,"%f %f %f %f\n", &wtv1[0],&wtv1[1],&wtv1[2],&wtv1[3]);

     network h1(wtv1[0],wtv1[1],wtv1[2],wtv1[3]);

     fscanf(infile,"%f %f %f %f \n",

     &inputv1[0],&inputv1[1],&inputv1[2],&inputv1[3]);

     cout<<"this is vector # " << i << "\n";

     cout << "please enter a threshold value, eg 7.0\n";

     cin >> threshold;

     h1.onrn.actvtion(inputv1, h1.nrn);

     h1.onrn.outvalue(threshold);

     cout<<"\n\n";

     }

fclose(wfile);

fclose(infile);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Comments on Your C++ Program

Notice the use of input stream operator cin>> in the C++ program, instead of the C function scanf in several places. The iostream class in C++ was discussed earlier in this chapter. The program works like this:

First, the network input neurons are given their connection weights, and then an input vector is presented to the input layer. A threshold value is specified, and the output neuron does the weighted sum of its inputs, which are the outputs of the input layer neurons. This weighted sum is the activation of the output neuron, and it is compared with the threshold value, and the output neuron fires (output is 1) if the threshold value is not greater than its activation. It does not fire (output is 0) if its activation is smaller than the threshold value. In this implementation, neither supervised nor unsupervised training is incorporated.

Input/Output for percept.cpp

There are two data files used in this program. One is for setting up the weights, and the other for setting up the input vectors. On the command line, you enter the program name followed by the weight file name and the input file name. For this discussion (also on the accompanying disk for this book) create a file called weight.dat, which contains the following data:

  2.0 3.0 3.0 2.0

  3.0 0.0 6.0 2.0

These are two weight vectors. Create also an input file called input.dat with the two data vectors below:

  1.95 0.27 0.69 1.25

  0.30 1.05 0.75 0.19

During the execution of the program, you are first prompted for the number of vectors that are used (in this case, 2), then for a threshold value for the input/weight vectors (use 7.0 in both cases). You will then see the following output. Note that the user input is in italic.

  percept weight.dat input.dat

THIS PROGRAM IS FOR A PERCEPTRON NETWORK WITH AN INPUT LAYER OF 4

NEURONS, EACH CONNECTED TO THE OUTPUT NEURON.

THIS EXAMPLE TAKES REAL NUMBERS AS INPUT SIGNALS

please enter the number of weights/vectors

2

this is vector # 1

please enter a threshold value, eg 7.0

7.0

weight for neuron 1 is  2           activation is 3.9

weight for neuron 2 is  3           activation is 0.81

weight for neuron 3 is  3           activation is 2.07

weight for neuron 4 is  2           activation is 2.5

activation is  9.28

the output neuron activation exceeds the threshold value of 7

 output value is 1

this is vector # 2

please enter a threshold value, eg 7.0

7.0

weight for neuron 1 is  3           activation is 0.9

weight for neuron 2 is  0           activation is 0

weight for neuron 3 is  6           activation is 4.5

weight for neuron 4 is  2           activation is 0.38

activation is  5.78

the output neuron activation is smaller than the threshold value of 7

output value is 0

Finally, try adding a data vector of (1.4, 0.6, 0.35, 0.99) to the data file. Add a weight vector of ( 2, 6, 8, 3) to the weight file and use a threshold value of 8.25 to see the result. You can use other values to experiment also.

Network Modeling

So far, we have considered the construction of two networks, the Hopfield memory and the Perceptron. What are other considerations (which will be discussed in more depth in the chapters to follow) that you should keep in mind ?

Some of the considerations that go into the modeling of a neural network for an application are:

nature of inputs

       fuzzy

                 binary

                 analog

       crisp

                 binary

                 analog

number of inputs

nature of outputs

       fuzzy

                 binary

                 analog

       crisp

                 binary

                 analog

number of outputs

nature of the application

       to complete patterns (recognize corrupted patterns)

       to classify patterns

       to do an optimization

       to do approximation

       to perform data clustering

       to compute functions

dynamics

    adaptive

              learning

                           training

                                           with exemplars

                                           without exemplars

              self-organizing

    nonadaptive

              learning

                           training

                                           with exemplars

                                           without exemplars

              self-organizing

hidden layers

    number

              fixed

              variable

    sizes

              fixed

              variable

processing

       additive

       multiplicative

       hybrid

                 additive and multiplicative

                 combining other approaches

                              expert systems

                              genetic algorithms

Hybrid models, as indicated above, could be of the variety of combining neural network approach with expert system methods or of combining additive and multiplicative processing paradigms.

Decision support systems are amenable to approaches that combine neural networks with expert systems. An example of a hybrid model that combines different modes of processing by neurons is the Sigma Pi neural network, wherein one layer of neurons uses summation in aggregation and the next layer of neurons uses multiplicative processing.

A hidden layer, if only one, in a neural network is a layer of neurons that operates in between the input layer and the output layer of the network. Neurons in this layer receive inputs from those in the input layer and supply their outputs as the inputs to the neurons in the output layer. When a hidden layer comes in between other hidden layers, it receives input and supplies input to the respective hidden layers.

In modeling a network, it is often not easy to determine how many, if any, hidden layers, and of what sizes, are needed in the model. Some approaches, like genetic algorithms—which are paradigms competing with neural network approaches in many situations but nevertheless can be cooperative, as here—are at times used to make a determination on the needed or optimum, as the case may be, numbers of hidden layers and/or the neurons in those hidden layers. In what follows, we outline one such application.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Tic-Tac-Toe Anyone?

David Fogel describes evolutionary general problem solving and uses the familiar game of Tic-Tac-Toe as an example. The idea is to come up with optimal strategies in playing this game. The first player's marker is an X, and the second player's marker is an O. Whoever gets three of his or her markers in a row or a column or a diagonal before the other player does, wins. Shrewd players manage a draw position, if their equally shrewd opponent thwarts their attempts to win. A draw position is one where neither player has three of his or her markers in a row, or a column, or a diagonal.

The board can be described by a vector of nine components, each of which is a three-valued number. Imagine the squares of the board for the game as taken in sequence row by row from top to bottom. Allow a 1 to show the presence of an X in that square, a 0 to indicate a blank there, and a -1 to correspond to an O. This is an example of a coding for the status of the board. For example, (-1, 0, 1, 0, -1, 0, 1, 1, -1) is a winning position for the second player, because it corresponds to the board looking as below.

 O       X

     O

 X   X   O

A neural network for this problem will have an input layer with nine neurons, as each input pattern has nine components. There would be some hidden layers. But the example is with one hidden layer. The output layer also contains nine neurons, so that one cycle of operation of the network shows what the best configuration of the board is to be, given a particular input. Of course, during this cycle of operation, all that needs to be determined is which blank space, indicated by a 0 in the input, should be changed to 1, if strategy is being worked for player 1. None of the 1's and -1's is to be changed.

In this particular example, the neural network architecture itself is dynamic. The network expands or contracts according to some rules, which are described next.

Fogel describes the network as an evolving network in the sense that the number of neurons in the hidden layer changed with a probability of 0.5. A node was equally likely to be added or deleted. Since the number of unmarked squares dwindles after each play, this kind of approach with varying numbers of neurons in the network seems to be reasonable, and interesting.

The initial set of weights is random values between -0.5 and 0.5, inclusive, according to a uniform distribution. Bias and threshold values also come from this distribution. The sigmoid function:

1/(1 + e-x)

is used also, to determine the outputs.

Weights and biases were changed during the network operation training cycles. Thus, the network had a learning phase. (You will read more on learning in Chapter 6.) This network is adaptive, since it changes its architecture. Other forms of adaptation in neural networks are in changing parameter values for a fixed architecture. (See Chapter 6.) The results of the experiment Fogel describes show that you need nine neurons in the hidden layer also, for your network to be the best for this problem. They also purged any strategy that was likely to lose.

Fogel's emphasis is on the evolutionary aspect of an adaptive process or experiment. Our interest in this example is primarily due to the fact that an adaptive neural network is used.

The choice of Tic-Tac-Toe, while being a simple and all too familiar game, is in the genre of much more complicated games. These games ask a player to place a marker in some position in a given array, and as players take turns doing so, some criterion determines if it is a draw, or who won. Unlike in Tic-Tac-Toe, the criterion by which one wins may not be known to the players.

Stability and Plasticity

We discuss now a few other considerations in neural network modeling by introducing short-term memory and long-term memory concepts. Neural network training is usually done in an iterative way, meaning that the procedure is repeated a certain number of times. These iterations are referred to as cycles. After each cycle, the input used may remain the same or change, or the weights may remain the same or change. Such change is based on the output of a completed cycle. If the number of cycles is not preset, and the network is allowed to go through cycles until some other criterion is met, the question of whether or not the termination of the iterative process occurs eventually, arises naturally.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Stability for a Neural Network

Stability refers to such convergence that facilitates an end to the iterative process. For example, if any two consecutive cycles result in the same output for the network, then there may be no need to do more iterations. In this case, convergence has occurred, and the network has stabilized in its operation. If weights are being modified after each cycle, then convergence of weights would constitute stability for the network.

In some situations, it takes many more iterations than you desire, to have output in two consecutive cycles to be the same. Then a tolerance level on the convergence criterion can be used. With a tolerance level, you accomplish early but satisfactory termination of the operation of the network.

Plasticity for a Neural Network

Suppose a network is trained to learn some patterns, and in this process the weights are adjusted according to an algorithm. After learning these patterns and encountering a new pattern, the network may modify the weights in order to learn the new pattern. But what if the new weight structure is not responsive to the new pattern? Then the network does not possess plasticity—the ability to deal satisfactorily with new short-term memory (STM) while retaining long-term memory (LTM). Attempts to endow a network with plasticity may have some adverse effects on the stability of your network.

Short-Term Memory and Long-Term Memory

We alluded to short-term memory (STM) and long-term memory (LTM) in the previous paragraph. STM is basically the information that is currently and perhaps temporarily being processed. It is manifested in the patterns that the network encounters. LTM, on the other hand, is information that is already stored and is not being currently processed. In a neural network, STM is usually characterized by patterns and LTM is characterized by the connections' weights. The weights determine how an input is processed in the network to yield output. During the cycles of operation of a network, the weights may change. After convergence, they represent LTM, as the weight levels achieved are stable.

Summary

You saw in this chapter, the C++ implementations of a simple Hopfield network and of a simple Perceptron network. What have not been included in them is an automatic iteration and a learning algorithm. They were not necessary for the examples that were used in this chapter to show C++ implementation, the emphasis was on the method of implementation. In a later chapter, you will read about the learning algorithms and examples of how to implement some of them.

Considerations in modeling a neural network are presented in this chapter along with an outline of how Tic-Tac-Toe is used as an example of an adaptive neural network model.

You also were introduced to the following concepts: stability, plasticity, short-term memory, and long-term memory (discussed further in later chapters). Much more can be said about them, in terms of the so-called noise-saturation dilemma, or stability–plasticity dilemma and what research has developed to address them (for further reading, see References).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 5
A Survey of Neural Network Models

Neural Network Models

You were introduced in the preceding pages to the Perceptron model, the Feedforward network, and the Hopfield network. You learned that the differences between the models lie in their architecture, encoding, and recall. We aim now to give you a comprehensive picture of these and other neural network models. We will show details and implementations of some networks in later chapters.

The models we briefly review in this chapter are the Perceptron, Hopfield, Adaline, Feed-Forward Backpropagation, Bidirectional Associative Memory, Brain-State-in-a-Box, Neocognitron, Fuzzy Associative Memory, ART1, and ART2. C++ implementations of some of these and the role of fuzzy logic in some will be treated in the subsequent chapters. For now, our discussion will be about the distinguishing characteristics of a neural network. We will follow it with the description of some of the models.

Layers in a Neural Network

A neural network has its neurons divided into subgroups, or fields, and elements in each subgroup are placed in a row, or a column, in the diagram depicting the network. Each subgroup is then referred to as a layer of neurons in the network. A great many models of neural networks have two layers, quite a few have one layer, and some have three or more layers. A number of additional, so-called hidden layers are possible in some networks, such as the Feed-forward backpropagation network. When the network has a single layer, the input signals are received at that layer, processing is done by its neurons, and output is generated at that layer. When more than one layer is present, the first field is for the neurons that supply the input signals for the neurons in the next layer.

Every network has a layer of input neurons, but in most of the networks, the sole purpose of these neurons is to feed the input to the next layer of neurons. However, there are feedback connections, or recurrent connections in some networks, so that the neurons in the input layer may also do some processing. In the Hopfield network you saw earlier, the input and output layers are the same. If any layer is present between the input and output layers, it may be referred to as a hidden layer in general, or as a layer with a special name taken after the researcher who proposed its inclusion to achieve certain performance from the network. Examples are the Grossberg and the Kohonen layers. The number of hidden layers is not limited except by the scope of the problem being addressed by the neural network.

A layer is also referred to as a field. Then the different layers can be designated as field A, field B, and so on, or shortly, FA, FB.

Single-Layer Network

A neural network with a single layer is also capable of processing for some important applications, such as integrated circuit implementations or assembly line control. The most common capability of the different models of neural networks is pattern recognition. But one network, called the Brain-State-in-a-Box, which is a single-layer neural network, can do pattern completion. Adaline is a network with A and B fields of neurons, but aggregation or processing of input signals is done only by the field B neurons.

The Hopfield network is a single-layer neural network. The Hopfield network makes an association between different patterns (heteroassociation) or associates a pattern with itself (autoassociation). You may characterize this as being able to recognize a given pattern. The idea of viewing it as a case of pattern recognition becomes more relevant if a pattern is presented with some noise, meaning that there is some slight deformation in the pattern, and if the network is able to relate it to the correct pattern.

The Perceptron technically has two layers, but has only one group of weights. We therefore still refer to it as a single-layer network. The second layer consists solely of the output neuron, and the first layer consists of the neurons that receive input(s). Also, the neurons in the same layer, the input layer in this case, are not interconnected, that is, no connections are made between two neurons in that same layer. On the other hand, in the Hopfield network, there is no separate output layer, and hence, it is strictly a single-layer network. In addition, the neurons are all fully connected with one another.

Let us spend more time on the single-layer Perceptron model and discuss its limitations, and thereby motivate the study of multilayer networks.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

XOR Function and the Perceptron

The ability of a Perceptron in evaluating functions was brought into question when Minsky and Papert proved that a simple function like XOR (the logical function exclusive or) could not be correctly evaluated by a Perceptron. The XOR logical function, f(A,B), is as follows:

A | B | f(A,B)= XOR(A,B)
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0

To summarize the behavior of the XOR, if both inputs are the same value, the output is 0, otherwise the output is 1.

Minsky and Papert showed that it is impossible to come up with the proper set of weights for the neurons in the single layer of a simple Perceptron to evaluate the XOR function. The reason for this is that such a Perceptron, one with a single layer of neurons, requires the function to be evaluated, to be linearly separable by means of the function values. The concept of linear separability is explained next. But let us show you first why the simple perceptron fails to compute this function.

Since there are two arguments for the XOR function, there would be two neurons in the input layer, and since the function's value is one number, there would be one output neuron. Therefore, you need two weights w1 and w2 ,and a threshold value θ. Let us now look at the conditions to be satisfied by the w's and the θ so that the outputs corresponding to given inputs would be as for the XOR function.

First the output should be 0 if inputs are 0 and 0. The activation works out as 0. To get an output of 0, you need 0 < θ. This is your first condition. Table 5.1 shows this and two other conditions you need, and why.

Table 5.1 Conditions on Weights
Input | Activation | Output | Needed Condition
0, 0 | 0 | 0 | 0 < θ
1, 0 | w1 | 1 | w1 > θ
0, 1 | w2 | 1 | w2 > θ
1, 1 | w1 + w2 | 0 | w1 + w2< θ

From the first three conditions, you can deduce that the sum of the two weights has to be greater than θ, which has to be positive itself. Line 4 is inconsistent with lines 1, 2, and 3, since line 4 requires the sum of the two weights to be less than θ. This affirms the contention that it is not possible to compute the XOR function with a simple perceptron.

Geometrically, the reason for this failure is that the inputs (0, 1) and (1, 0) with which you want output 1, are situated diagonally opposite each other, when plotted as points in the plane, as shown below in a diagram of the output (1=T, 0=F):

 F     T

 T     F

You can't separate the T's and the F's with a straight line. This means that you cannot draw a line in the plane in such a way that neither (1, 1) ->F nor (0, 0)->F is on the same side of the line as (0, 1) ->T and (1, 0)-> T.

Linear Separability

What linearly separable means is, that a type of a linear barrier or a separator—a line in the plane, or a plane in the three-dimensional space, or a hyperplane in higher dimensions—should exist, so that the set of inputs that give rise to one value for the function all lie on one side of this barrier, while on the other side lie the inputs that do not yield that value for the function. A hyperplane is a surface in a higher dimension, but with a linear equation defining it much the same way a line in the plane and a plane in the three-dimensional space are defined.

To make the concept a little bit clearer, consider a problem that is similar but, let us emphasize, not the same as the XOR problem.

Imagine a cube of 1-unit length for each of its edges and lying in the positive octant in a xyz-rectangular coordinate system with one corner at the origin. The other corners or vertices are at points with coordinates (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), and (1, 1, 1). Call the origin O, and the seven points listed as A, B, C, D, E, F, and G, respectively. Then any two faces opposite to each other are linearly separable because you can define the separating plane as the plane halfway between these two faces and also parallel to these two faces.

For example, consider the faces defined by the set of points O, A, B, and C and by the set of points D, E, F, and G. They are parallel and 1 unit apart, as you can see in Figure 5.1. The separating plane for these two faces can be seen to be one of many possible planes—any plane in between them and parallel to them. One example, for simplicity, is the plane that passes through the points (1/2, 0, 0), (1/2, 0, 1), (1/2, 1, 0), and (1/2, 1, 1). Of course, you need only specify three of those four points because a plane is uniquely determined by three points that are not all on the same line. So if the first set of points corresponds to a value of say, +1 for the function, and the second set to a value of –1, then a single-layer Perceptron can determine, through some training algorithm, the correct weights for the connections, even if you start with the weights being initially all 0.

Figure 5.1 Separating plane.

Consider the set of points O, A, F, and G. This set of points cannot be linearly separated from the other vertices of the cube. In this case, it would be impossible for the single-layer Perceptron to determine the proper weights for the neurons in evaluating the type of function we have been discussing.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

A Second Look at the XOR Function: Multilayer Perceptron

By introducing a set of cascaded Perceptrons, you have a Perceptron network, with an input layer, middle or hidden layer, and an output layer. You will see that the multilayer Perceptron can evaluate the XOR function as well as other logic functions (AND, OR, MAJORITY, etc.). The absence of the separability that we talked about earlier is overcome by having a second stage, so to speak, of connection weights.

You need two neurons in the input layer and one in the output layer. Let us put a hidden layer with two neurons. Let w11, w12, w21, and w22, be the weights on connections from the input neurons to the hidden layer neurons. Let v1, v2 , be the weights on the connections from the hidden layer neurons to the outout neuron.

We will select the w's (weights) and the threshold values θ1 , and θ2 at the hidden layer neurons, so that the input (0, 0) generates the output vector (0, 0), and the input vector (1, 1) generates (1, 1), while the inputs (1, 0) and (0, 1) generate (0, 1) as the hidden layer output. The inputs to the output layer neurons would be from the set {(0, 0), (1, 1), (0, 1)}. These three vectors are separable, with (0, 0), and (1, 1) on one side of the separating line, while (0, 1) is on the other side.

We will select the √s (weights) and τ, the threshold value at the output neuron, so as to make the inputs (0, 0) and (1, 1) cause an output of 0 for the network, and an output of 1 is caused by the input (0, 1). The network layout within the labels of weights and threshold values inside the nodes representing hidden layer and output neurons is shown in Figure 5.1a. Table 5.2 gives the results of operation of this network.

Figure 5.1a Example network.

Table 5.2 Results for the Perceptron with One Hidden Layer.
Input | Hidden Layer Activations | Hidden Layer Outputs | Output Neuron activaton | Output of network
(0, 0) | (0, 0) | (0, 0) | 0 | 0
(1, 1) | (0.3, 0.6) | (1, 1) | 0 | 0
(0, 1) | (0.15, 0.3) | (0, 1) | 0.3 | 1
(1, 0) | (0.15, 0.3) | (0, 1) | 0.3 | 1

It is clear from Table 5.2, that the above perceptron with a hidden layer does compute the XOR function successfully.

Note: The activation should exceed the threshold value for a neuron to fire. Where the output of a neuron is shown to be 0, it is because the internal activation of that neuron fell short of its threshold value.

Example of the Cube Revisited

Let us return to the example of the cube with vertices at the origin O, and the points labeled A, B, C, D, E, F, and G. Suppose the set of vertices O, A, F, and G give a value of 1 for the function to be evaluated, and the other vertices give a –1. The two sets are not linearly separable as mentioned before. A simple Perceptron cannot evaluate this function.

Can the addition of another layer of neurons help? The answer is yes. What would be the role of this additional layer? The answer is that it will do the final processing for the problem after the previous layer has done some preprocessing. This can do two separations in the sense that the set of eight vertices can be separated—or partitioned—into three separable subsets. If this partitioning can also help collect within each subset, like vertices, meaning those that map onto the same value for the function, the network will succeed in its task of evaluating the function when the aggregation and thresholding is done at the output neuron.

Strategy

So the strategy is first to consider the set of vertices that give a value of +1 for the function and determine the minimum number of subsets that can be identified to be each separable from the rest of the vertices. It is evident that since the vertices O and A lie on one edge of the cube, they can form one subset that is separable. The other two vertices, viz., F and one for G, which correspond to the value +1 for the function, can form a second subset that is separable, too. We need not bother with the last four vertices from the point of view of further partitioning that subset. It is clear that one new layer of three neurons, one of which fires for the inputs corresponding to the vertices O and A, one for F, and G, and the third for the rest, will then facilitate the correct evaluation of the function at the output neuron.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Details

Table 5.3 lists the vertices and their coordinates, together with a flag that indicates to which subset in the partitioning the vertex belongs. Note that you can think of the action of the Multilayer Perceptron as that of evaluating the intersection and union of linearly separable subsets.

Table 5.3 Partitioning of Vertices of a Cube
Vertex | Coordinates | Subset
O | (0,0,0) | 1
A | (0,0,1) | 1
B | (0,1,0) | 2
C | (0,1,1) | 2
D | (1,0,0) | 2
E | (1,0,1) | 2
F | (1,1,0) | 3
G | (1,1,1) | 3

The network, which is a two-layer Perceptron, meaning two layers of weights, has three neurons in the first layer and one output neuron in the second layer. Remember that we are counting those layers in which the neurons do the aggregation of the signals coming into them using the connection weights. The first layer with the three neurons is what is generally described as the hidden layer, since the second layer is not hidden and is at the extreme right in the layout of the neural network. Table 5.4 gives an example of the weights you can use for the connections between the input neurons and the hidden layer neurons. There are three input neurons, one for each coordinate of the vertex of the cube.

Table 5.4 Weights for Connections Between Input Neurons and Hidden Layer Neurons.
Input Neuron # | Hidden Layer Neuron # | Connection Weight
1 | 1 | 1
1 | 2 | 0.1
1 | 3 | -1
2 | 1 | 1
2 | 2 | -1
2 | 3 | -1
3 | 1 | 0.2
3 | 2 | 0.3
3 | 3 | 0.6

Now we give, in Table 5.5, the weights for the connections between the three hidden-layer neurons and the output neuron.

Table 5.5 Weights for Connection Between the Hidden-Layer Neurons and the Output Neuron
Hidden Layer Neuron # | Connection Weight
1 | 0.6
3 | 0.3
3 | 0.6

It is not apparent whether or not these weights will do the job. To determine the activations of the hidden-layer neurons, you need these weights, and you also need the threshold value at each neuron that does processing. A hidden-layer neuron will fire, that is, will output a 1, if the weighted sum of the signals it receives is greater than the threshold value. If the output neuron fires, the function value is taken as +1, and if it does not fire, the function value is –1. Table 5.6 gives the threshold values. Figure 5.1b shows the neural network with connection weights and threshold values.

Figure 5.1b Neural Network for Cube Example

Table 5.6 Threshold Values
Layer | Neuron | Threshold Value
hidden | 1 | 1.8
hidden | 2 | 0.05
hidden | 3 | -0.2
output | 1 | 0.5


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Performance of the Perceptron

When you input the coordinates of the vertex G, which has 1 for each coordinate, the first hidden-layer neuron aggregates these inputs and gets a value of 2.2. Since 2.2 is more than the threshold value of the first neuron in the hidden layer, that neuron fires, and its output of 1 becomes an input to the output neuron on the connection with weight 0.6. But you need the activations of the other hidden-layer neurons as well. Let us describe the performance with coordinates of G as the inputs to the network. Table 5.7 describes this.

Table 5.7 Results with Coordinates of Vertex G as Input
Vertex/ Coordinates | Hidden Layer | Weighted Sum | Comment | Activation | Contribution to Output | Sum
G:1,1,1 | 1 | 2.2 | >1.8 | 1 | 0.6 |
 | 2 | -0.8 | <0.05 | 0 | 0 |
 | 3 | -1.4 | <-0.2 | 0 | 0 | 0.6

The weighted sum at the output neuron is 0.6, and it is greater than the threshold value 0.5. Therefore, the output neuron fires, and at the vertex G, the function is evaluated to have a value of +1.

Table 5.8 shows the performance of the network with the rest of the vertices of the cube. You will notice that the network computes a value of +1 at the vertices, O, A, F, and G, and a –1 at the rest.

Table 5.8 Results with Other Inputs
 | Hidden Layer Neuron# | Weighted Sum | Comment | Activation | Contribution to Output | Sum
O :0, 0, 0 | 1 | 0 | <1.8 | 0 | 0 |
 | 2 | 0 | <0.05 | 0 | 0 |
 | 3 | 0 | >-0.2 | 1 | 0.6 | 0.6*
A :0, 0, 1 | 1 | 0.2 | <1.8 | 0 | 0 |
 | 2 | 0.3 | >0.05 | 1 | 0.3 |
 | 3 | 0.6 | >-0.2 | 1 | 0.6 | 0.9*
B :0, 1, 0 | 1 | 1 | <1.8 | 0 | 0 |
 | 2 | -1 | <0.05 | 0 | 0 |
 | 3 | -1 | <-0.2 | 0 | 0 | 0
C :0, 1, 1 | 1 | 1.2 | <1.8 | 0 | 0 |
 | 2 | 0.2 | >0.05 | 1 | 0.3 |
 | 3 | -0.4 | <-0.2 | 0 | 0 | 0.3
D :1, 0, 0 | 1 | 1 | <1.8 | 0 | 0 |
 | 2 | .1 | >0.05 | 1 | 0.3 |
 | 3 | -1 | <-0.2 | 0 | 0 | 0.3
E :1, 0, 1 | 1 | 1.2 | <1.8 | 0 | 0 |
 | 2 | 0.4 | >0.05 | 1 | 0.3 |
 | 3 | -0.4 | <-0.2 | 0 | 0 | 0.3
F :1, 1, 0 | 1 | 2 | >1.8 | 1 | 0.6 |
 | 2 | -0.9 | <0.05 | 0 | 0 |
 | 3 | -2 | <-0.2 | 0 | 0 | 0.6*
*The output neuron fires, as this value is greater than 0.5 (the threshold value); the function value is +1.

Other Two-layer Networks

Many important neural network models have two layers. The Feedforward backpropagation network, in its simplest form, is one example. Grossberg and Carpenter's ART1 paradigm uses a two-layer network. The Counterpropagation network has a Kohonen layer followed by a Grossberg layer. Bidirectional Associative Memory, (BAM), Boltzman Machine, Fuzzy Associative Memory, and Temporal Associative Memory are other two-layer networks. For autoassociation, a single-layer network could do the job, but for heteroassociation or other such mappings, you need at least a two-layer network. We will give more details on these models shortly.

Many Layer Networks

Kunihiko Fukushima's Neocognitron, noted for identifying handwritten characters, is an example of a network with several layers. Some previously mentioned networks can also be multilayered from the addition of more hidden layers. It is also possible to combine two or more neural networks into one network by creating appropriate connections between layers of one subnetwork to those of the others. This would certainly create a multilayer network.

Connections Between Layers

You have already seen some difference in the way connections are made between neurons in a neural network. In the Hopfield network, every neuron was connected to every other in the one layer that was present in the network. In the Perceptron, neurons within the same layer were not connected with one another, but the connections were between the neurons in one layer and those in the next layer. In the former case, the connections are described as being lateral. In the latter case, the connections are forward and the signals are fed forward within the network.

Two other possibilities also exist. All the neurons in any layer may have extra connections, with each neuron connected to itself. The second possibility is that there are connections from the neurons in one layer to the neurons in a previous layer, in which case there is both forward and backward signal feeding. This occurs, if feedback is a feature for the network model. The type of layout for the network neurons and the type of connections between the neurons constitute the architecture of the particular model of the neural network.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Instar and Outstar

Outstar and instar are terms defined by Stephen Grossberg for ways of looking at neurons in a network. A neuron in a web of other neurons receives a large number of inputs from outside the neuron's boundaries. This is like an inwardly radiating star, hence, the term instar. Also, a neuron may be sending its output to many other destinations in the network. In this way it is acting as an outstar. Every neuron is thus simultaneously both an instar and an outstar. As an instar it receives stimuli from other parts of the network or from outside the network. Note that the neurons in the input layer of a network primarily have connections away from them to the neurons in the next layer, and thus behave mostly as outstars. Neurons in the output layer have many connections coming to it and thus behave mostly as instars. A neural network performs its work through the constant interaction of instars and outstars.

A layer of instars can constitute a competitive layer in a network. An outstar can also be described as a source node with some associated sink nodes that the source feeds to. Grossberg identifies the source input with a conditioned stimulus and the sink inputs with unconditioned stimuli. Robert Hecht-Nielsen's Counterpropagation network is a model built with instars and outstars.

Weights on Connections

Weight assignments on connections between neurons not only indicate the strength of the signal that is being fed for aggregation but also the type of interaction between the two neurons. The type of interaction is one of cooperation or of competition. The cooperative type is suggested by a positive weight, and the competition by a negative weight, on the connection. The positive weight connection is meant for what is called excitation, while the negative weight connection is termed an inhibition.

Initialization of Weights

Initializing the network weight structure is part of what is called the encoding phase of a network operation. The encoding algorithms are several, differing by model and by application. You may have gotten the impression that the weight matrices used in the examples discussed in detail thus far have been arbitrarily determined; or if there is a method of setting them up, you are not told what it is.

It is possible to start with randomly chosen values for the weights and to let the weights be adjusted appropriately as the network is run through successive iterations. This would make it easier also. For example, under supervised training, if the error between the desired and computed output is used as a criterion in adjusting weights, then one may as well set the initial weights to zero and let the training process take care of the rest. The small example that follows illustrates this point.

A Small Example

Suppose you have a network with two input neurons and one output neuron, with forward connections between the input neurons and the output neuron, as shown in Figure 5.2. The network is required to output a 1 for the input patterns (1, 0) and (1, 1), and the value 0 for (0, 1) and (0, 0). There are only two connection weights w1 and w2.

Figure 5.2 Neural network with forward connections.

Let us set initially both weights to 0, but you need a threshold function also. Let us use the following threshold function, which is slightly different from the one used in a previous example:

 |  |  | 1 if x > 0
f(x) | = | {
 |  |  | 0 if x ≤ 0

The reason for modifying this function is that if f(x) has value 1 when x = 0, then no matter what the weights are, the output will work out to 1 with input (0, 0). This makes it impossible to get a correct computation of any function that takes the value 0 for the arguments (0, 0).

Now we need to know by what procedure we adjust the weights. The procedure we would apply for this example is as follows.

• If the output with input pattern (a, b) is as desired, then do not adjust the weights.
• If the output with input pattern (a, b) is smaller than what it should be, then increment each of w1 and w2 by 1.
• If the output with input pattern (a, b) is greater than what it should be, then subtract 1 from w1 if the product aw1 is smaller than 1, and adjust w2 similarly.

Table 5.9 shows what takes place when we follow these procedures, and at what values the weights settle.

Table 5.9 Adjustment of Weights
step | w1 | w2 | a | b | activation | output | comment
1 | 0 | 0 | 1 | 1 | 0 | 0 | desired output is 1; increment both w's
2 | 1 | 1 | 1 | 1 | 2 | 1 | output is what it should be
3 | 1 | 1 | 1 | 0 | 1 | 1 | output is what it should be
4 | 1 | 1 | 0 | 1 | 1 | 1 | output is 1; it should be 0.
5 |  |  |  |  |  |  | subtract 1 from w2
6 | 1 | 0 | 0 | 1 | 0 | 0 | output is what it should be
7 | 1 | 0 | 0 | 0 | 0 | 0 | output is what it should be
8 | 1 | 0 | 1 | 1 | 1 | 1 | output is what it should be
9 | 1 | 0 | 1 | 0 | 1 | 1 | output is what it should be


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Table 5.9 shows that the network weight vector changed from an initial vector (0, 0) to the final weight vector (1, 0) in eight iterations. This example is not of a network for pattern matching. If you think about it, you will realize that the network is designed to fire if the first digit in the pattern is a 1, and not otherwise. An analogy for this kind of a problem is determining if a given image contains a specific object in a specific part of the image, such as a dot should occur in the letter i.

If the initial weights are chosen somewhat prudently and to make some particular relevance, then the speed of operation can be increased in the sense of convergence being achieved with fewer iterations than otherwise. Thus, encoding algorithms are important. We now present some of the encoding algorithms.

Initializing Weights for Autoassociative Networks

Consider a network that is to associate each input pattern with itself and which gets binary patterns as inputs. Make a bipolar mapping on the input pattern. That is, replace each 0 by –1. Call the mapped pattern the vector x, when written as a column vector. The transpose, the same vector written as a row vector, is xT. You will get a matrix of order the size of x when you form the product xxT. Obtain similar matrices for the other patterns you want the network to store. Add these matrices to give you the matrix of weights to be used initially, as we did in Chapter 4. This process can be described with the following equation:

W = ςixixiT

Weight Initialization for Heteroassociative Networks

Consider a network that is to associate one input pattern with another pattern and that gets binary patterns as inputs. Make a bipolar mapping on the input pattern. That is, replace each 0 by –1. Call the mapped pattern the vector x when written as a column vector. Get a similar bipolar mapping for the corresponding associated pattern. Call it y. You will get a matrix of size x by size y when you form the product xyT. Obtain similar matrices for the other patterns you want the network to store. Add these matrices to give you the matrix of weights to be used initially. The following equation restates this process:

W = ςixiyiT

On Center, Off Surround

In one of the many interesting paradigms you encounter in neural network models and theory, is the strategy winner takes all. Well, if there should be one winner emerging from a crowd of neurons in a particular layer, there needs to be competition. Since everybody is for himself in such a competition, in this case every neuron for itself, it would be necessary to have lateral connections that indicate this circumstance. The lateral connections from any neuron to the others should have a negative weight. Or, the neuron with the highest activation is considered the winner and only its weights are modified in the training process, leaving the weights of others the same. Winner takes all means that only one neuron in that layer fires and the others do not. This can happen in a hidden layer or in the output layer.

In another situation, when a particular category of input is to be identified from among several groups of inputs, there has to be a subset of the neurons that are dedicated to seeing it happen. In this case, inhibition increases for distant neurons, whereas excitation increases for the neighboring ones, as far as such a subset of neurons is concerned. The phrase on center, off surround describes this phenomenon of distant inhibition and near excitation.

Weights also are the prime components in a neural network, as they reflect on the one hand the memory stored by the network, and on the other hand the basis for learning and training.

Inputs

You have seen that mutually orthogonal or almost orthogonal patterns are required as stable stored patterns for the Hopfield network, which we discussed before for pattern matching. Similar restrictions are found also with other neural networks. Sometimes it is not a restriction, but the purpose of the model makes natural a certain type of input. Certainly, in the context of pattern classification, binary input patterns make problem setup simpler. Binary, bipolar, and analog signals are the varieties of inputs. Networks that accept analog signals as inputs are for continuous models, and those that require binary or bipolar inputs are for discrete models. Binary inputs can be fed to networks for continuous models, but analog signals cannot be input to networks for discrete models (unless they are fuzzified). With input possibilities being discrete or analog, and the model possibilities being discrete or continuous, there are potentially four situations, but one of them where analog inputs are considered for a discrete model is untenable.

An example of a continuous model is where a network is to adjust the angle by which the steering wheel of a truck is to be turned to back up the truck into a parking space. If a network is supposed to recognize characters of the alphabet, a means of discretization of a character allows the use of a discrete model.

What are the types of inputs for problems like image processing or handwriting analysis? Remembering that artificial neurons, as processing elements, do aggregation of their inputs by using connection weights, and that the output neuron uses a threshold function, you know that the inputs have to be numerical. A handwritten character can be superimposed on a grid, and the input can consist of the cells in each row of the grid where a part of the character is present. In other words, the input corresponding to one character will be a set of binary or gray-scale sequences containing one sequence for each row of the grid. A 1 in a particular position in the sequence for a row shows that the corresponding pixel is present(black) in that part of the grid, while 0 shows it is not. The size of the grid has to be big enough to accommodate the largest character under study, as well as the most complex features.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Outputs

The output from some neural networks is a spatial pattern that can include a bit pattern, in some a binary function value, and in some others an analog signal. The type of mapping intended for the inputs determines the type of outputs, naturally. The output could be one of classifying the input data, or finding associations between patterns of the same dimension as the input.

The threshold functions do the final mapping of the activations of the output neurons into the network outputs. But the outputs from a single cycle of operation of a neural network may not be the final outputs, since you would iterate the network into further cycles of operation until you see convergence. If convergence seems possible, but is taking an awful lot of time and effort, that is, if it is too slow to learn, you may assign a tolerance level and settle for the network to achieve near convergence.

The Threshold Function

The output of any neuron is the result of thresholding, if any, of its internal activation, which, in turn, is the weighted sum of the neuron's inputs. Thresholding sometimes is done for the sake of scaling down the activation and mapping it into a meaningful output for the problem, and sometimes for adding a bias. Thresholding (scaling) is important for multilayer networks to preserve a meaningful range across each layer's operations. The most often used threshold function is the sigmoid function. A step function or a ramp function or just a linear function can be used, as when you simply add the bias to the activation. The sigmoid function accomplishes mapping the activation into the interval [0, 1]. The equations are given as follows for the different threshold functions just mentioned.

The Sigmoid Function

More than one function goes by the name sigmoid function. They differ in their formulas and in their ranges. They all have a graph similar to a stretched letter s. We give below two such functions. The first is the hyperbolic tangent function with values in (–1, 1). The second is the logistic function and has values between 0 and 1. You therefore choose the one that fits the range you want. The graph of the sigmoid logistic function is given in Fig. 5.3.

1. f(x) = tanh(x) = ( ex - e-x) / (ex + e-x)
2. f(x) = 1 / (1+ e-x)

Note that the first function here, the hyperbolic tangent function, can also be written, as 1 - 2e-x / (ex + e-x ) after adding and also subtracting e-x to the numerator, and then simplifying. If now you multiply in the second term both numerator and denominator by ex, you get 1 - 2/ (e2x + 1). As x approaches -∞, this function goes to -1, and as x approaches +∞, it goes to +1. On the other hand, the second function here, the sigmoid logistic function, goes to 0 as x approaches -∞, and to +1 as x approaches +∞. You can see this if you rewrite 1 / (1+ e-x) as 1 - 1 / (1+ ex), after manipulations similar to those above.

You can think of equation 1 as the bipolar equivalent of binary equation 2. Both functions have the same shape.

Figure 5.3 is the graph of the sigmoid logistic function (number 2 of the preceding list).

Figure 5.3 The sigmoid function.

The Step Function

The step function is also frequently used as a threshold function. The function is 0 to start with and remains so to the left of some threshold value θ. A jump to 1 occurs for the value of the function to the right of θ, and the function then remains at the level 1. In general, a step function can have a finite number of points at which jumps of equal or unequal size occur. When the jumps are equal and at many points, the graph will resemble a staircase. We are interested in a step function that goes from 0 to 1 in one step, as soon as the argument exceeds the threshold value θ. You could also have two values other than 0 and 1 in defining the range of values of such a step function. A graph of the step function follows in Figure 5.4.

Figure 5.4 The step function.

Note: You can think of a sigmoid function as a fuzzy step function.

The Ramp Function

To describe the ramp function simply, first consider a step function that makes a jump from 0 to 1 at some point. Instead of letting it take a sudden jump like that at one point, let it gradually gain in value, along a straight line (looks like a ramp), over a finite interval reaching from an initial 0 to a final 1. Thus, you get a ramp function. You can think of a ramp function as a piecewise linear approximation of a sigmoid. The graph of a ramp function is illustrated in Figure 5.5.

Figure 5.5 Graph of a ramp function.

Linear Function

A linear function is a simple one given by an equation of the form:

f(x) = αx + β

When α = 1, the application of this threshold function amounts to simply adding a bias equal to β to the sum of the inputs.

Applications

As briefly indicated before, the areas of application generally include auto- and heteroassociation, pattern recognition, data compression, data completion, signal filtering, image processing, forecasting, handwriting recognition, and optimization. The type of connections in the network, and the type of learning algorithm used must be chosen appropriate to the application. For example, a network with lateral connections can do autoassociation, while a feed-forward type can do forecasting.

Some Neural Network Models

Adaline and Madaline

Adaline is the acronym for adaptive linear element, due to Bernard Widrow and Marcian Hoff. It is similar to a Perceptron. Inputs are real numbers in the interval [–1,+1], and learning is based on the criterion of minimizing the average squared error. Adaline has a high capacity to store patterns. Madaline stands for many Adalines and is a neural network that is widely used. It is composed of field A and field B neurons, and there is one connection from each field A neuron to each field B neuron. Figure 5.6 shows a diagram of the Madaline.

Figure 5.6 The Madaline model.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Backpropagation

The Backpropagation training algorithm for training feed-forward networks was developed by Paul Werbos, and later by Parker, and Rummelhart and McClelland. This type of network configuration is the most common in use, due to its ease of training. It is estimated that over 80% of all neural network projects in development use backpropagation. In backpropagation, there are two phases in its learning cycle, one to propagate the input pattern through the network and the other to adapt the output, by changing the weights in the network. It is the error signals that are backpropagated in the network operation to the hidden layer(s). The portion of the error signal that a hidden-layer neuron receives in this process is an estimate of the contribution of a particular neuron to the output error. Adjusting on this basis the weights of the connections, the squared error, or some other metric, is reduced in each cycle and finally minimized, if possible.

Figure for Backpropagation Network

You will find in Figure 7.1 in Chapter 7, the layout of the nodes that represent the neurons in a feedforward Backpropagation network and the connections between them. For now, you try your hand at drawing this layout based on the following description, and compare your drawing with Figure 7.1. There are three fields of neurons. The connections are forward and are from each neuron in a layer to every neuron in the next layer. There are no lateral or recurrent connections. Labels on connections indicate weights. Keep in mind that the number of neurons is not necessarily the same in different layers, and this fact should be evident in the notation for the weights.

Bidirectional Associative Memory

Bidirectional Associative Memory, (BAM), and other models described in this section were developed by Bart Kosko. BAM is a network with feedback connections from the output layer to the input layer. It associates a member of the set of input patterns with a member of the set of output patterns that is the closest, and thus it does heteroassociation. The patterns can be with binary or bipolar values. If all possible input patterns are known, the matrix of connection weights can be determined as the sum of matrices obtained by taking the matrix product of an input vector (as a column vector) with its transpose (written as a row vector).

The pattern obtained from the output layer in one cycle of operation is fed back at the input layer at the start of the next cycle. The process continues until the network stabilizes on all the input patterns. The stable state so achieved is described as resonance, a concept used in the Adaptive Resonance Theory.

You will find in Figure 8.1 in Chapter 8, the layout of the nodes that represent the neurons in a BAM network and the connections between them. There are two fields of neurons. The network is fully connected with feedback connections and forward connections. There are no lateral or recurrent connections.

Fuzzy Associative memories are similar to Bidirectional Associative memories, except that association is established between fuzzy patterns. Chapter 9 deals with Fuzzy Associative memories.

Temporal Associative Memory

Another type of associative memory is temporal associative memory. Amari, a pioneer in the field of neural networks, constructed a Temporal Associative Memory model that has feedback connections between the input and output layers. The forte of this model is that it can store and retrieve spatiotemporal patterns. An example of a spatiotemporal pattern is a waveform of a speech segment.

Brain-State-in-a-Box

Introduced by James Anderson and others, this network differs from the single-layer fully connected Hopfield network in that Brain-State-in-a-Box uses what we call recurrent connections as well. Each neuron has a connection to itself. With target patterns available, a modified Hebbian learning rule is used. The adjustment to a connection weight is proportional to the product of the desired output and the error in the computed output. You will see more on Hebbian learning in Chapter 6. This network is adept at noise tolerance, and it can accomplish pattern completion. Figure 5.7 shows a Brain-State-in-a-Box network.

Figure 5.7 A Brain-State-in-a-Box, network.

What's in a Name?

More like what's in the box? Suppose you find the following: there is a square box and its corners are the locations for an entity to be. The entity is not at one of the corners, but is at some point inside the box. The next position for the entity is determined by working out the change in each coordinate of the position, according to a weight matrix, and a squashing function. This process is repeated until the entity settles down at some position. The choice of the weight matrix is such that when the entity reaches a corner of the square box, its position is stable and no more movement takes place. You would perhaps guess that the entity finally settles at the corner nearest to the initial position of it within the box. It is said that this kind of an example is the reason for the name Brain-State-in-a-Box for the model. Its forte is that it represents linear transformations. Some type of association of patterns can be achieved with this model. If an incomplete pattern is associated with a completed pattern, it would be an example of autoassociation.

Counterpropagation

This is a neural network model developed by Robert Hecht-Nielsen, that has one or two additional layers between the input and output layers. If it is one, the middle layer is a Grossberg layer with a bunch of outstars. In the other case, a Kohonen layer, or a self-organizing layer, follows the input layer, and in turn is followed by a Grossberg layer of outstars. The model has the distinction of considerably reducing training time. With this model, you gain a tool that works like a look-up table.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Neocognitron

Compared to all other neural network models, Fukushima's Neocognitron is more complex and ambitious. It demonstrates the advantages of a multilayered network. The Neocognitron is one of the best models for recognizing handwritten symbols. Many pairs of layers called the S layer, for simple layer, and C layer, for complex layer, are used. Within each S layer are several planes containing simple cells. Similarly, there are within each C layer, an equal number of planes containing complex cells. The input layer does not have this arrangement and is like an input layer in any other neural network.

The number of planes of simple cells and of complex cells within a pair of S and C layers being the same, these planes are paired, and the complex plane cells process the outputs of the simple plane cells. The simple cells are trained so that the response of a simple cell corresponds to a specific portion of the input image. If the same part of the image occurs with some distortion, in terms of scaling or rotation, a different set of simple cells responds to it. The complex cells output to indicate that some simple cell they correspond to did fire. While simple cells respond to what is in a contiguous region in the image, complex cells respond on the basis of a larger region. As the process continues to the output layer, the C-layer component of the output layer responds, corresponding to the entire image presented in the beginning at the input layer.

Adaptive Resonance Theory

ART1 is the first model for adaptive resonance theory for neural networks developed by Gail Carpenter and Stephen Grossberg. This theory was developed to address the stability–plasticity dilemma. The network is supposed to be plastic enough to learn an important pattern. But at the same time it should remain stable when, in short-term memory, it encounters some distorted versions of the same pattern.

ART1 model has A and B field neurons, a gain, and a reset as shown in Figure 5.8. There are top-down and bottom-up connections between neurons of fields A and B. The neurons in field B have lateral connections as well as recurrent connections. That is, every neuron in this field is connected to every other neuron in this field, including itself, in addition to the connections to the neurons in field A. The external input (or bottom-up signal), the top-down signal, and the gain constitute three elements of a set, of which at least two should be a +1 for the neuron in the A field to fire. This is what is termed the two-thirds rule. Initially, therefore, the gain would be set to +1. The idea of a single winner is also employed in the B field. The gain would not contribute in the top-down phase; actually, it will inhibit. The two-thirds rule helps move toward stability once resonance, or equilibrium, is obtained. A vigilance parameter ρ is used to determine the parameter reset. Vigilance parameter corresponds to what degree the resonating category can be predicted. The part of the system that contains gain is called the attentional subsystem, whereas the rest, the part that contains reset, is termed the orienting subsystem. The top-down activity corresponds to the orienting subsystem, and the bottom-up activity relates to the attentional subsystem.

Figure 5.8 The ART1 network.

In ART1, classification of an input pattern in relation to stored patterns is attempted, and if unsuccessful, a new stored classification is generated. Training is unsupervised. There are two versions of training: slow and fast. They differ in the extent to which the weights are given the time to reach their eventual values. Slow training is governed by differential equations, and fast training by algebraic equations.

ART2 is the analog counterpart of ART1, which is for discrete cases. These are self-organizing neural networks, as you can surmise from the fact that training is present but unsupervised. The ART3 model is for recognizing a coded pattern through a parallel search, and is developed by Carpenter and Grossberg. It tries to emulate the activities of chemical transmitters in the brain during what can be construed as a parallel search for pattern recognition.

Summary

The basic concepts of neural network layers, connections, weights, inputs, and outputs have been discussed. An example of how adding another layer of neurons in a network can solve a problem that could not be solved without it is given in detail. A number of neural network models are introduced briefly. Learning and training, which form the basis of neural network behavior has not been included here, but will be discussed in the following chapter.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 6
Learning and Training

In the last chapter, we presented an overview of different neural network models. In this chapter, we continue the broad discussion of neural networks with two important topics: Learning and Training. Here are key questions that we would like to answer:

• How do neural networks learn?
• What does it mean for a network to learn ?
• What differences are there between supervised and unsupervised learning ?
• What training regimens are in common use for neural networks?

Objective of Learning

There are many varieties of neural networks. In the final analysis, as we have discussed briefly in Chapter 4 on network modeling, all neural networks do one or more of the following :

• Pattern classification
• Pattern completion
• Optimization
• Data clustering
• Approximation
• Function evaluation

A neural network, in any of the previous tasks, maps a set of inputs to a set of outputs. This nonlinear mapping can be thought of as a multidimensional mapping surface. The objective of learning is to mold the mapping surface according to a desired response, either with or without an explicit training process.

Learning and Training

A network can learn when training is used, or the network can learn also in the absence of training. The difference between supervised and unsupervised training is that, in the former case, external prototypes are used as target outputs for specific inputs, and the network is given a learning algorithm to follow and calculate new connection weights that bring the output closer to the target output. Unsupervised learning is the sort of learning that takes place without a teacher. For example, when you are finding your way out of a labyrinth, no teacher is present. You learn from the responses or events that develop as you try to feel your way through the maze. For neural networks, in the unsupervised case, a learning algorithm may be given but target outputs are not given. In such a case, data input to the network gets clustered together; similar input stimuli cause similar responses.

When a neural network model is developed and an appropriate learning algorithm is proposed, it would be based on the theory supporting the model. Since the dynamics of the operation of the neural network is under study, the learning equations are initially formulated in terms of differential equations. After solving the differential equations, and using any initial conditions that are available, the algorithm could be simplified to consist of an algebraic equation for the changes in the weights. These simple forms of learning equations are available for your neural networks.

At this point of our discussion you need to know what learning algorithms are available, and what they look like. We will now discuss two main rules for learning—Hebbian learning, used with unsupervised learning and the delta rule, used with supervised learning. Adaptations of these by simple modifications to suit a particular context generate many other learning rules in use today. Following the discussion of these two rules, we present variations for each of the two classes of learning: supervised learning and unsupervised learning.

Hebb's Rule

Learning algorithms are usually referred to as learning rules. The foremost such rule is due to Donald Hebb. Hebb's rule is a statement about how the firing of one neuron, which has a role in the determination of the activation of another neuron, affects the first neuron's influence on the activation of the second neuron, especially if it is done in a repetitive manner. As a learning rule, Hebb's observation translates into a formula for the difference in a connection weight between two neurons from one iteration to the next, as a constant [mu] times the product of activations of the two neurons. How a connection weight is to be modified is what the learning rule suggests. In the case of Hebb's rule, it is adding the quantity [mu]aiaj, where ai is the activation of the ith neuron, and aj is the activation of the jth neuron to the connection weight between the ith and jth neurons. The constant [mu] itself is referred to as the learning rate. The following equation using the notation just described, states it succinctly:

[Delta]wij = [mu]aiaj

As you can see, the learning rule derived from Hebb's rule is quite simple and is used in both simple and more involved networks. Some modify this rule by replacing the quantity ai with its deviation from the average of all as and, similarly, replacing aj by a corresponding quantity. Such rule variations can yield rules better suited to different situations.

For example, the output of a neural network being the activations of its output layer neurons, the Hebbian learning rule in the case of a perceptron takes the form of adjusting the weights by adding [mu] times the difference between the output and the target. Sometimes a situation arises where some unlearning is required for some neurons. In this case a reverse Hebbian rule is used in which the quantity [mu]aiaj is subtracted from the connection weight under question, which in effect is employing a negative learning rate.

In the Hopfield network of Chapter 1, there is a single layer with all neurons fully interconnected. Suppose each neuron's output is either a + 1 or a – 1. If we take [mu] = 1 in the Hebbian rule, the resulting modification of the connection weights can be described as follows: add 1 to the weight, if both neuron outputs match, that is, both are +1 or –1. And if they do not match (meaning one of them has output +1 and the other has –1), then subtract 1 from the weight.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Delta Rule

The delta rule is also known as the least mean squared error rule (LMS). You first calculate the square of the errors between the target or desired values and computed values, and then take the average to get the mean squared error. This quantity is to be minimized. For this, realize that it is a function of the weights themselves, since the computation of output uses them. The set of values of weights that minimizes the mean squared error is what is needed for the next cycle of operation of the neural network. Having worked this out mathematically, and having compared the weights thus found with the weights actually used, one determines their difference and gives it in the delta rule, each time weights are to be updated. So the delta rule, which is also the rule used first by Widrow and Hoff, in the context of learning in neural networks, is stated as an equation defining the change in the weights to be affected.

Suppose you fix your attention to the weight on the connection between the ith neuron in one layer and the jth neuron in the next layer. At time t, this weight is wij(t) . After one cycle of operation, this weight becomes wij(t + 1). The difference between the two is wij(t + 1) - wij(t), and is denoted by [Delta]wij . The delta rule then gives [Delta]wij as :

[Delta]wij = 2[mu]xi(desired output value – computed output value)j

Here, [mu] is the learning rate, which is positive and much smaller than 1, and xi is the ith component of the input vector.

Supervised Learning

Supervised neural network paradigms to be discussed include :

• Perceptron
• Adaline
• Feedforward Backpropagation network
• Statistical trained networks (Boltzmann/Cauchy machines)
• Radial basis function networks

The Perceptron and the Adaline use the delta rule; the only difference is that the Perceptron has binary output, while the Adaline has continuous valued output. The Feedforward Backpropagation network uses the generalized delta rule, which is described next.

Generalized Delta Rule

While the delta rule uses local information on error, the generalized delta rule uses error information that is not local. It is designed to minimize the total of the squared errors of the output neurons. In trying to achieve this minimum, the steepest descent method, which uses the gradient of the weight surface, is used. (This is also used in the delta rule.) For the next error calculation, the algorithm looks at the gradient of the error surface, which gives the direction of the largest slope on the error surface. This is used to determine the direction to go to try to minimize the error. The algorithm chooses the negative of this gradient, which is the direction of steepest descent. Imagine a very hilly error surface, with peaks and valleys that have a wide range of magnitude. Imagine starting your search for minimum error at an arbitrary point. By choosing the negative gradient on all iterations, you eventually end up at a valley. You cannot know, however, if this valley is the global minimum or a local minimum. Getting stuck in a local minimum is one well-known potential problem of the steepest descent method. You will see more on the generalized delta rule in the chapter on backpropagation (Chapter 7).

Statistical Training and Simulated Annealing

The Boltzmann machine (and Cauchy machine) uses probabilities and statistical theory, along with an energy function representing temperature. The learning is probabilistic and is called simulated annealing. At different temperature levels, a different number of iterations in processing are used, and this constitutes an annealing schedule. Use of probability distributions is for the goal of reaching a state of global minimum of energy. Boltzmann distribution and Cauchy distribution are probability distributions used in this process. It is obviously desirable to reach a global minimum, rather than settling down at a local minimum.

Figure 6.1 clarifies the distinction between a local minimum and a global minimum. In this figure you find the graph of an energy function and points A and B. These points show that the energy levels there are smaller than the energy levels at any point in their vicinity, so you can say they represent points of minimum energy. The overall or global minimum, as you can see, is at point B, where the energy level is smaller than that even at point A, so A corresponds only to a local minimum. It is desirable to get to B and not get stopped at A itself, in the pursuit of a minimum for the energy function. If point C is reached, one would like the further movement to be toward B and not A. Similarly, if a point near A is reached, the subsequent movement should avoid reaching or settling at A but carry on to B. Perturbation techniques are useful for these considerations.

Figure 6.1 Local and global minima.

Clamping Probabilities

Sometimes in simulated annealing, first a subset of the neurons in the network are associated with some inputs, and another subset of neurons are associated with some outputs, and these are clamped with probabilities, which are not changed in the learning process. Then the rest of the network is subjected to adjustments. Updating is not done for the clamped units in the network. This training procedure of Geoffrey Hinton and Terrence Sejnowski provides an extension of the Boltzmann technique to more general networks.

Radial Basis-Function Networks

Although details of radial basis functions are beyond the scope of this book, it is worthwhile to contrast the learning characteristics for this type of neural network model. Radial basis-function networks in topology look similar to feedforward networks. Each neuron has an output to input characteristic that resembles a radial function (for two inputs, and thus two dimensions). Specifically, the output h(x) is as follows:

h(x)  = exp [ (x - u)2 / 2[sigma]2   ]

Here, x is the input vector, u is the mean, and [sigma] is the standard deviation of the output response curve of the neuron. Radial basis function (RBF) networks have rapid training time (orders of magnitude faster than backpropagation) and do not have problems with local minima as backpropagation does. RBF networks are used with supervised training, and typically only the output layer is trained. Once training is completed, a RBF network may be slower to use than a feedforward Backpropagation network, since more computations are required to arrive at an output.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Unsupervised Networks

Unsupervised neural network paradigms to be discussed include:

• Hopfield Memory
• Bidirectional associative memory
• Fuzzy associative memory
• Learning vector quantizer
• Kohonen self-organizing map
• ART1

Self-Organization

Unsupervised learning and self-organization are closely related. Unsupervised learning was mentioned in Chapter 1, along with supervised learning. Training in supervised learning takes the form of external exemplars being provided. The network has to compute the correct weights for the connections for neurons in some layer or the other. Self-organization implies unsupervised learning. It was described as a characteristic of a neural network model, ART1, based on adaptive resonance theory (to be covered in Chapter 10). With the winner-take-all criterion, each neuron of field B learns a distinct classification. The winning neuron in a layer, in this case the field B, is the one with the largest activation, and it is the only neuron in that layer that is allowed to fire. Hence, the name winner take all.

Self-organization means self-adaptation of a neural network. Without target outputs, the closest possible response to a given input signal is to be generated. Like inputs will cluster together. The connection weights are modified through different iterations of network operation, and the network capable of self-organizing creates on its own the closest possible set of outputs for the given inputs. This happens in the model in Kohonen's self-organizing map.

Kohonen's Linear Vector Quantizer (LVQ) described briefly below is later extended as a self-organizing feature map. Self-organization is also learning, but without supervision; it is a case of self-training. Kohonen's topology preserving maps illustrate self-organization by a neural network. In these cases, certain subsets of output neurons respond to certain subareas of the inputs, so that the firing within one subset of neurons indicates the presence of the corresponding subarea of the input. This is a useful paradigm in applications such as speech recognition. The winner-take-all strategy used in ART1 also facilitates self-organization.

Learning Vector Quantizer

Suppose the goal is the classification of input vectors. Kohonen's Vector Quantization is a method in which you first gather a finite number of vectors of the dimension of your input vector. Kohonen calls these codebook vectors. You then assign groups of these codebook vectors to the different classes under the classification you want to achieve. In other words, you make a correspondence between the codebook vectors and classes, or, partition the set of codebook vectors by classes in your classification.

Now examine each input vector for its distance from each codebook vector, and find the nearest or closest codebook vector to it. You identify the input vector with the class to which the codebook vector belongs.

Codebook vectors are updated during training, according to some algorithm. Such an algorithm strives to achieve two things: (1), a codebook vector closest to the input vector is brought even closer to it; and (two), a codebook vector indicating a different class is made more distant from the input vector.

For example, suppose (2, 6) is an input vector, and (3, 10) and (4, 9) are a pair of codebook vectors assigned to different classes. You identify (2, 6) with the class to which (4, 9) belongs, since (4, 9) with a distance of [radic]13 is closer to it than (3, 10) whose distance from (2, 6) is [radic]17. If you add 1 to each component of (3, 10) and subtract 1 from each component of (4, 9), the new distances of these from (2, 6) are [radic]29 and [radic]5, respectively. This shows that (3, 10) when changed to (4, 11) becomes more distant from your input vector than before the change, and (4, 9) is changed to (3, 8), which is a bit closer to (2, 6) than (4, 9) is.

Training continues until all input vectors are classified. You obtain a stage where the classification for each input vector remains the same as in the previous cycle of training. This is a process of self-organization.

The Learning Vector Quantizer (LVQ) of Kohonen is a self-organizing network. It classifies input vectors on the basis of a set of stored or reference vectors. The B field neurons are also called grandmother cells, each of which represents a specific class in the reference vector set. Either supervised or unsupervised learning can be used with this network. (See Figure 6.2.)

Figure 6.2 Layout for Learning Vector Quantizer.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Associative Memory Models and One-Shot Learning

The Hopfield memory, Bidirectional Associative memory and Fuzzy Associative memory are all unsupervised networks that perform pattern completion, or pattern association. That is, with corrupted or missing information, these memories are able to recall or complete an expected output. Gallant calls the training method used in these networks as one-shot learning, since you determine the weight matrix as a function of the completed patterns you wish to recall just once. An example of this was shown in Chapter 4 with determination of weights for the Hopfield memory.

Learning and Resonance

ART1 is the first neural network model based on adaptive resonance theory of Carpenter and Grossberg. When you have a pair of patterns such that when one of them is input to a neural network the output turns out to be the other pattern in the pair, and if this happens consistently in both directions, then you may describe it as resonance. We discuss in Chapter 8 bidirectional associative memories and resonance. By the time training is completed, and learning is through, many other pattern pairs would have been presented to the network as well. If changes in the short-term memory do not disturb or affect the long-term memory, the network shows adaptive resonance. The ART1 model is designed to maintain it. Note that this discussion relates largely to stability.

Learning and Stability

Learning, convergence, and stability are matters of much interest. As learning is taking place, you want to know if the process is going to halt at some appropriate point, which is a question of convergence. Is what is learned stable, or will the network have to learn all over again, as each new event occurs? These questions have their answers within a mathematical model with differential equations developed to describe a learning algorithm. Proofs showing stability are part of the model inventor's task. One particular tool that aids in the process of showing convergence is the idea of state energy, or cost, to describe whether the direction the process is taking can lead to convergence.

The Lyapunov function, discussed later in this chapter, is found to provide the right energy function, which can be minimized during the operation of the neural network. This function has the property that the value gets smaller with every change in the state of the system, thus assuring that a minimum will be reached eventually. The Lyapunov function is discussed further because of its significant utility for neural network models, but briefly because of the high level of mathematics involved. Fortunately, simple forms are derived and put into learning algorithms for neural networks. The high-level mathematics is used in making the proofs to show the viability of the models.

Alternatively, temperature relationships can be used, as in the case of the Boltzmann machine, or any other well-suited cost function such as a function of distances used in the formulation of the Traveling Salesman Problem, in which the total distance for the tour of the traveling salesman is to be minimized, can be employed. The Traveling Salesman Problem is important and well-known. A set of cities is to be visited by the salesman, each only once, and the aim is to devise a tour that minimizes the total distance traveled. The search continues for an efficient algorithm for this problem. Some algorithms solve the problem in a large number but not all of the situations. A neural network formulation can also work for the Traveling Salesman Problem. You will see more about this in Chapter 15.

Training and Convergence

Suppose you have a criterion such as energy to be minimized or cost to be decreased, and you know the optimum level for this criterion. If the network achieves the optimum value in a finite number of steps, then you have convergence for the operation of the network. Or, if you are making pairwise associations of patterns, there is the prospect of convergence if after each cycle of the network operation, the number of errors is decreasing.

It is also possible that convergence is slow, so much so that it may seem to take forever to achieve the convergence state. In that case, you should specify a tolerance value and require that the criterion be achieved within that tolerance, avoiding a lot of computing time. You may also introduce a momentum parameter to further change the weight and thereby speed up the convergence. One technique used is to add a portion of the previous change in weight.

Instead of converging, the operation may result in oscillations. The weight structure may keep changing back and forth; learning will never cease. Learning algorithms need to be analyzed in terms of convergence as being an essential algorithm property.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Lyapunov Function

Neural networks are dynamic systems in the learning and training phase of their operation, and convergence is an essential feature, so it was necessary for the researchers developing the models and their learning algorithms to find a provable criterion for convergence in a dynamic system. The Lyapunov function, mentioned previously, turned out to be the most convenient and appropriate function. It is also referred to as the energy function. The function decreases as the system states change. Such a function needs to be found and watched as the network operation continues from cycle to cycle. Usually it involves a quadratic form. The least mean squared error is an example of such a function. Lyapunov function usage assures a system stability that cannot occur without convergence. It is convenient to have one value, that of the Lyapunov function specifying the system behavior. For example, in the Hopfield network, the energy function is a constant times the sum of products of outputs of different neurons and the connection weight between them. Since pairs of neuron outputs are multiplied in each term, the entire expression is a quadratic form.

Other Training Issues

Besides the applications for which a neural network is intended, and depending on these applications, you need to know certain aspects of the model. The length of encoding time and the length of learning time are among the important considerations. These times could be long but should not be prohibitive. It is important to understand how the network behaves with new inputs; some networks may need to be trained all over again, but some tolerance for distortion in input patterns is desirable, where relevant. Restrictions on the format of inputs should be known.

An advantage of neural networks is that they can deal with nonlinear functions better than traditional algorithms can. The ability to store a number of patterns, or needing more and more neurons in the output field with an increasing number of input patterns are the kind of aspects addressing the capabilities of a network and also its limitations.

Adaptation

Sometimes neural networks are used as adaptive filters, the motivation for such an architecture being selectivity. You want the neural network to classify each input pattern into its appropriate category. Adaptive models involve changing of connection weights during all their operations, while nonadaptive ones do not alter the weights after the phase of learning with exemplars. The Hopfield network is often used in modeling a neural network for optimization problems, and the Backpropagation model is a popular choice in most other applications. Neural network models are distinguishable sometimes by their architecture, sometimes by their adaptive methods, and sometimes both. Methods for adaptation, where adaptation is incorporated, assume great significance in the description and utility of a neural network model.

For adaptation, you can modify parameters in an architecture during training, such as the learning rate in the backpropagation training method for example. A more radical approach is to modify the architecture itself during training. New neural network paradigms change the number or layers and the number of neurons in a layer during training. These node adding or pruning algorithms are termed constructive algorithms. (See Gallant for more details.)

Generalization Ability

The analogy for a neural network presented at the beginning of the chapter was that of a multidimensional mapping surface that maps inputs to outputs. For each unseen input with respect to a training set, the generalization ability of a network determines how well the mapping surface renders the new input in the output space. A stock market forecaster must generalize well, otherwise you lose money in unseen market conditions. The opposite of generalization is memorization. A pattern recognition system for images of handwriting, should be able to generalize a letter A that is handwritten in several different ways by different people. If the system memorizes, then you will not recognize the letter A in all cases, but instead will categorize each letter A variation separately. The trick to achieve generalization is in network architecture, design, and training methodology. You do not want to overtrain your neural network on expected outcomes, but rather should accept a slightly worse than minimum error on your training set data. You will learn more about generalization in Chapter 14.

Summary

Learning and training are important issues in applying neural networks. Two broad categories of network learning are supervised and unsupervised learning. Supervised learning provides example outputs to compare to while unsupervised learning does not. During supervised training, external prototypes are used as target outputs and the network is given a learning algorithm to follow and calculate new connection weights that bring the output closer to the target output. You can refer to networks using unsupervised learning as self-organizing networks, since no external information or guidance is used in learning. Several neural network paradigms were presented in this chapter along with their learning and training characteristics.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 7
Backpropagation

Feedforward Backpropagation Network

The feedforward backpropagation network is a very popular model in neural networks. It does not have feedback connections, but errors are backpropagated during training. Least mean squared error is used. Many applications can be formulated for using a feedforward backpropagation network, and the methodology has been a model for most multilayer neural networks. Errors in the output determine measures of hidden layer output errors, which are used as a basis for adjustment of connection weights between the input and hidden layers. Adjusting the two sets of weights between the pairs of layers and recalculating the outputs is an iterative process that is carried on until the errors fall below a tolerance level. Learning rate parameters scale the adjustments to weights. A momentum parameter can also be used in scaling the adjustments from a previous iteration and adding to the adjustments in the current iteration.

Mapping

The feedforward backpropagation network maps the input vectors to output vectors. Pairs of input and output vectors are chosen to train the network first. Once training is completed, the weights are set and the network can be used to find outputs for new inputs. The dimension of the input vector determines the number of neurons in the input layer, and the number of neurons in the output layer is determined by the dimension of the outputs. If there are k neurons in the input layer and m neurons in the output layer, then this network can make a mapping from k-dimensional space to an m-dimensional space. Of course, what that mapping is depends on what pair of patterns or vectors are used as exemplars to train the network, which determine the network weights. Once trained, the network gives you the image of a new input vector under this mapping. Knowing what mapping you want the feedforward backpropagation network to be trained for implies the dimensions of the input space and the output space, so that you can determine the numbers of neurons to have in the input and output layers.

Layout

The architecture of a feedforward backpropagation network is shown in Figure 7.1. While there can be many hidden layers, we will illustrate this network with only one hidden layer. Also, the number of neurons in the input layer and that in the output layer are determined by the dimensions of the input and output patterns, respectively. It is not easy to determine how many neurons are needed for the hidden layer. In order to avoid cluttering the figure, we will show the layout in Figure 7.1 with five input neurons, three neurons in the hidden layer, and four output neurons, with a few representative connections.

Figure 7.1 Layout of a feedforward backpropagation network.

The network has three fields of neurons: one for input neurons, one for hidden processing elements, and one for the output neurons. As already stated, connections are for feed forward activity. There are connections from every neuron in field A to every one in field B, and, in turn, from every neuron in field B to every neuron in field C. Thus, there are two sets of weights, those figuring in the activations of hidden layer neurons, and those that help determine the output neuron activations. In training, all of these weights are adjusted by considering what can be called a cost function in terms of the error in the computed output pattern and the desired output pattern.

Training

The feedforward backpropagation network undergoes supervised training, with a finite number of pattern pairs consisting of an input pattern and a desired or target output pattern. An input pattern is presented at the input layer. The neurons here pass the pattern activations to the next layer neurons, which are in a hidden layer. The outputs of the hidden layer neurons are obtained by using perhaps a bias, and also a threshold function with the activations determined by the weights and the inputs. These hidden layer outputs become inputs to the output neurons, which process the inputs using an optional bias and a threshold function. The final output of the network is determined by the activations from the output layer.

The computed pattern and the input pattern are compared, a function of this error for each component of the pattern is determined, and adjustment to weights of connections between the hidden layer and the output layer is computed. A similar computation, still based on the error in the output, is made for the connection weights between the input and hidden layers. The procedure is repeated with each pattern pair assigned for training the network. Each pass through all the training patterns is called a cycle or an epoch. The process is then repeated as many cycles as needed until the error is within a prescribed tolerance.

There can be more than one learning rate parameter used in training in a feedforward backpropagation network. You can use one with each set of weights between consecutive layers.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Illustration: Adjustment of Weights of Connections from a Neuron in the Hidden Layer

We will be as specific as is needed to make the computations clear. First recall that the activation of a neuron in a layer other than the input layer is the sum of products of its inputs and the weights corresponding to the connections that bring in those inputs. Let us discuss the jth neuron in the hidden layer. Let us be specific and say j = 2. Suppose that the input pattern is (1.1, 2.4, 3.2, 5.1, 3.9) and the target output pattern is (0.52, 0.25, 0.75, 0.97). Let the weights be given for the second hidden layer neuron by the vector (–0.33, 0.07, –0.45, 0.13, 0.37). The activation will be the quantity:

     (-0.33 * 1.1) + (0.07 * 2.4) + (-0.45 * 3.2) + (0.13 * 5.1)

     + (0.37 * 3.9) = 0.471

Now add to this an optional bias of, say, 0.679, to give 1.15. If we use the sigmoid function given by:

     1 / ( 1+ exp(-x) ),

with x = 1.15, we get the output of this hidden layer neuron as 0.7595.

We are taking values to a few decimal places only for illustration, unlike the precision that can be obtained on a computer.

We need the computed output pattern also. Let us say it turns out to be actual =(0.61, 0.41, 0.57, 0.53), while the desired pattern is desired =(0.52, 0.25, 0.75, 0.97). Obviously, there is a discrepancy between what is desired and what is computed. The component-wise differences are given in the vector, desired - actual = (-0.09, -0.16, 0.18, 0.44). We use these to form another vector where each component is a product of the error component, corresponding computed pattern component, and the complement of the latter with respect to 1. For example, for the first component, error is –0.09, computed pattern component is 0.61, and its complement is 0.39. Multiplying these together (0.61*0.39*-0.09), we get -0.02. Calculating the other components similarly, we get the vector (–0.02, –0.04, 0.04, 0.11). The desired–actual vector, which is the error vector multiplied by the actual output vector, gives you a value of error reflected back at the output of the hidden layer. This is scaled by a value of (1-output vector), which is the first derivative of the output activation function for numerical stability). You will see the formulas for this process later in this chapter.

The backpropagation of errors needs to be carried further. We need now the weights on the connections between the second neuron in the hidden layer that we are concentrating on, and the different output neurons. Let us say these weights are given by the vector (0.85, 0.62, –0.10, 0.21). The error of the second neuron in the hidden layer is now calculated as below, using its output.

     error = 0.7595 * (1 - 0.7595) * ( (0.85 * -0.02) + (0.62 * -0.04)

     + ( -0.10 * 0.04) + (0.21 * 0.11)) = -0.0041.

Again, here we multiply the error (e.g., -0.02) from the output of the current layer, by the output value (0.7595) and the value (1-0.7595). We use the weights on the connections between neurons to work backwards through the network.

Next, we need the learning rate parameter for this layer; let us set it as 0.2. We multiply this by the output of the second neuron in the hidden layer, to get 0.1519. Each of the components of the vector (–0.02, –0.04, 0.04, 0.11) is multiplied now by 0.1519, which our latest computation gave. The result is a vector that gives the adjustments to the weights on the connections that go from the second neuron in the hidden layer to the output neurons. These values are given in the vector (–0.003, –0.006, 0.006, 0.017). After these adjustments are added, the weights to be used in the next cycle on the connections between the second neuron in the hidden layer and the output neurons become those in the vector (0.847, 0.614, –0.094, 0.227).

Illustration: Adjustment of Weights of Connections from a Neuron in the Input Layer

Let us look at how adjustments are calculated for the weights on connections going from the ith neuron in the input layer to neurons in the hidden layer. Let us take specifically i = 3, for illustration.

Much of the information we need is already obtained in the previous discussion for the second hidden layer neuron. We have the errors in the computed output as the vector (–0.09, –0.16, 0.18, 0.44), and we obtained the error for the second neuron in the hidden layer as –0.0041, which was not used above. Just as the error in the output is propagated back to assign errors for the neurons in the hidden layer, those errors can be propagated to the input layer neurons.

To determine the adjustments for the weights on connections between the input and hidden layers, we need the errors determined for the outputs of hidden layer neurons, a learning rate parameter, and the activations of the input neurons, which are just the input values for the input layer. Let us take the learning rate parameter to be 0.15. Then the weight adjustments for the connections from the third input neuron to the hidden layer neurons are obtained by multiplying the particular hidden layer neuron's output error by the learning rate parameter and by the input component from the input neuron. The adjustment for the weight on the connection from the third input neuron to the second hidden layer neuron is 0.15 * 3.2 * –0.0041, which works out to –0.002.

If the weight on this connection is, say, –0.45, then adding the adjustment of -0.002, we get the modified weight of –0.452, to be used in the next iteration of the network operation. Similar calculations are made to modify all other weights as well.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Adjustments to Threshold Values or Biases

The bias or the threshold value we added to the activation, before applying the threshold function to get the output of a neuron, will also be adjusted based on the error being propagated back. The needed values for this are in the previous discussion.

The adjustment for the threshold value of a neuron in the output layer is obtained by multiplying the calculated error (not just the difference) in the output at the output neuron and the learning rate parameter used in the adjustment calculation for weights at this layer. In our previous example, we have the learning rate parameter as 0.2, and the error vector as (–0.02, –0.04, 0.04, 0.11), so the adjustments to the threshold values of the four output neurons are given by the vector (–0.004, –0.008, 0.008, 0.022). These adjustments are added to the current levels of threshold values at the output neurons.

The adjustment to the threshold value of a neuron in the hidden layer is obtained similarly by multiplying the learning rate with the computed error in the output of the hidden layer neuron. Therefore, for the second neuron in the hidden layer, the adjustment to its threshold value is calculated as 0.15 * –0.0041, which is –0.0006. Add this to the current threshold value of 0.679 to get 0.6784, which is to be used for this neuron in the next training pattern for the neural network.

Another Example of Backpropagation Calculations

You have seen, in the preceding sections, the details of calculations for one particular neuron in the hidden layer in a feedforward backpropagation network with five input neurons and four neurons in the output layer, and two neurons in the hidden layer.

You are going to see all the calculations in the C++ implementation later in this chapter. Right now, though, we present another example and give the complete picture of the calculations done in one completed iteration or cycle of backpropagation.

Consider a feedforward backpropagation network with three input neurons, two neurons in the hidden layer, and three output neurons. The weights on connections from the input neurons to the neurons in the hidden layer are given in Matrix M-1, and those from the neurons in the hidden layer to output neurons are given in Matrix M-2.

We calculate the output of each neuron in the hidden and output layers as follows. We add a bias or threshold value to the activation of a neuron (call this result x) and use the sigmoid function below to get the output.

     f(x) = 1/ (1 + e-x )

Learning parameters used are 0.2 for the connections between the hidden layer neurons and output neurons and 0.15 for the connections between the input neurons and the neurons in the hidden layer. These values as you recall are the same as in the previous illustration, to make it easy for you to follow the calculations by comparing them with similar calculations in the preceding sections.

The input pattern is ( 0.52, 0.75, 0.97 ), and the desired output pattern is ( 0.24, 0.17, 0.65). The initial weight matrices are as follows:

M-1 Matrix of weights from input layer to hidden layer

       0.6     - 0.4

       0.2       0.8

     - 0.5       0.3

M-2 Matrix of weights from hidden layer to output layer

     -0.90       0.43       0.25

      0.11     - 0.67     - 0.75

The threshold values (or bias) for neurons in the hidden layer are 0.2 and 0.3, while those for the output neurons are 0.15, 0.25, and 0.05, respectively.

Table 7.1 presents all the results of calculations done in the first iteration. You will see modified or new weight matrices and threshold values. You will use these and the original input vector and the desired output vector to carry out the next iteration.

Table 7.1 Backpropagation Calculations
Item | I-1 | I-2 | I-3 | H-1 | H-2 | O-1 | O-2 | O-3
Input | 0.52 | 0.75 | 0.97 |  |  |  |  |
Desired Output |  |  |  |  |  | 0.24 | 0.17 | 0.65
M-1 Row 1 |  |  |  | 0.6 | - 0.4 |  |  |
M-1 Row 2 |  |  |  | 0.2 | 0.8 |  |  |
M-1 Row 3 |  |  |  | - 0.5 | 0.3 |  |  |
M-2 Row 1 |  |  |  |  |  | - 0.90 | 0.43 | 0.25
M-2 Row 2 |  |  |  |  |  | 0.11 | - 0.67 | - 0.75
Threshold |  |  |  | 0.2 | 0.3 | 0.15 | 0.25 | 0.05
Activation - H |  |  |  | - 0.023 | 0.683 |  |  |
Activation + Threshold -H |  |  |  | 0.177 | 0.983 |  |  |
Output -H |  |  |  | 0.544 | 0.728 |  |  |
Complement |  |  |  | 0.456 | 0.272 |  |  |
Activation -O |  |  |  |  |  | - 0.410 | - 0.254 | - 0.410
Activation + Threshold -O |  |  |  |  |  | - 0.260 | - 0.004 | - 0.360
Output -O |  |  |  |  |  | 0.435 | 0.499 | 0.411
Complement |  |  |  |  |  | 0.565 | 0.501 | 0.589
Diff. from Target |  |  |  |  |  | - 0.195 | - 0.329 | 0.239
Computed Error -O |  |  |  |  |  | - 0.048 | - 0.082 | 0.058
Computed Error -H |  |  |  | 0.0056 | 0.0012 |  |  |
Adjustment to Threshold |  |  |  | 0.0008 | 0.0002 | -0.0096 | -0.0164 | 0.0116
Adjustment to M-2 Column 1 |  |  |  | -0.0005 | -0.0070 |  |  |
Adjustment to M-2 Column 2 |  |  |  | 0.0007 | 0.0008 |  |  |
Adjustment to M-2 Column 3 |  |  |  | 0.0008 | 0.0011 |  |  |
New Matrix M-2 Row 1 |  |  |  |  |  | - 0.91 | 0.412 | 0.262
New Matrix M-2 Row 2 |  |  |  |  |  | 0.096 | - 0.694 | - 0.734
New Threshold Values -O |  |  |  |  |  | 0.1404 | 0.2336 | 0.0616
Adjustment to M-1 Row 1 |  |  |  | 0.0004 | -0.0001 |  |  |
Adjustment to M-1 Row 2 |  |  |  | 0.0006 | 0.0001 |  |  |
Adjustment to M-1 Row 3 |  |  |  | 0.0008 | 0.0002 |  |  |
New Matrix M-1 Row 1 |  |  |  | 0.6004 | - 0.4 |  |  |
New Matrix M-1 Row 2 |  |  |  | 0.2006 | 0.8001 |  |  |
New Matrix M-1 Row 3 |  |  |  | -0.4992 | 0.3002 |  |  |
New Threshold Values -H |  |  |  | 0.2008 | 0.3002 |  |  |


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The top row in the table gives headings for the columns. They are, Item, I-1, I-2, I-3 (I-k being for input layer neuron k); H-1, H-2 (for hidden layer neurons); and O-1, O-2, O-3 (for output layer neurons).

In the first column of the table, M-1 and M-2 refer to weight matrices as above. Where an entry is appended with -H, like in Output -H, the information refers to the hidden layer. Similarly, -O refers to the output layer, as in Activation + threshold -O.

The next iteration uses the following information from the previous iteration, which you can identify from Table 7.1. The input pattern is ( 0.52, 0.75, 0.97 ), and the desired output pattern is ( 0.24, 0.17, 0.65). The current weight matrices are as follows:

M-1 Matrix of weights from input layer to hidden layer:

             0.6004   - 0.4

             0.2006     0.8001

           - 0.4992     0.3002

M-2 Matrix of weights from hidden layer to output layer:

           -0.910       0.412      0.262

            0.096      -0.694     -0.734

The threshold values (or bias) for neurons in the hidden layer are 0.2008 and 0.3002, while those for the output neurons are 0.1404, 0.2336, and 0.0616, respectively.

You can keep the learning parameters as 0.15 for connections between input and hidden layer neurons, and 0.2 for connections between the hidden layer neurons and output neurons, or you can slightly modify them. Whether or not to change these two parameters is a decision that can be made perhaps at a later iteration, having obtained a sense of how the process is converging.

If you are satisfied with the rate at which the computed output pattern is getting close to the target output pattern, you would not change these learning rates. If you feel the convergence is much slower than you would like, then the learning rate parameters can be adjusted slightly upwards. It is a subjective decision both in terms of when (if at all) and to what new levels these parameters need to be revised.

Notation and Equations

You have just seen an example of the process of training in the feedforward backpropagation network, described in relation to one hidden layer neuron and one input neuron. There were a few vectors that were shown and used, but perhaps not made easily identifiable. We therefore introduce some notation and describe the equations that were implicitly used in the example.

Notation

Let us talk about two matrices whose elements are the weights on connections. One matrix refers to the interface between the input and hidden layers, and the second refers to that between the hidden layer and the output layer. Since connections exist from each neuron in one layer to every neuron in the next layer, there is a vector of weights on the connections going out from any one neuron. Putting this vector into a row of the matrix, we get as many rows as there are neurons from which connections are established.

Let M1 and M2 be these matrices of weights. Then what does M1[i][j] represent? It is the weight on the connection from the ith input neuron to the jth neuron in the hidden layer. Similarly, M2[i][j] denotes the weight on the connection from the ith neuron in the hidden layer and the jth output neuron.

Next, we will use x, y, z for the outputs of neurons in the input layer, hidden layer, and output layer, respectively, with a subscript attached to denote which neuron in a given layer we are referring to. Let P denote the desired output pattern, with pi as the components. Let m be the number of input neurons, so that according to our notation, (x1, x2, …, xm) will denote the input pattern. If P has, say, r components, the output layer needs r neurons. Let the number of hidden layer neurons be n. Let βh be the learning rate parameter for the hidden layer, and βo′, that for the output layer. Let θ with the appropriate subscript represent the threshold value or bias for a hidden layer neuron, and τ with an appropriate subscript refer to the threshold value of an output neuron.

Let the errors in output at the output layer be denoted by ejs and those at the hidden layer by ti's. If we use a Δ prefix of any parameter, then we are looking at the change in or adjustment to that parameter. Also, the thresholding function we would use is the sigmoid function, f(x) = 1 / (1 + exp(–x)).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Equations

Output of jth hidden layer neuron:

yj = f( (Σi xiM1[ i ][ j ] ) + θj )

(7.1)

Output of jth output layer neuron:

zj = f( (Σi yiM2[ i ][ j ] ) + τj )

(7.2)

Ith component of vector of output differences:

desired value - computed value = Pi – zi

Ith component of output error at the output layer:

ei = ( Pi - zi)

(7.3)

Ith component of output error at the hidden layer:

ti = yi (1 - yi ) (Σj M2[ i ][ j ] ej)

(7.4)

Adjustment for weight between ith neuron in hidden layer and jth output neuron:

ΔM2[ i ][ j ] = βo yiej

(7.5)

Adjustment for weight between ith input neuron and jth neuron in hidden layer:

M1[ i ][ j ] = βhxitj

(7.6)

Adjustment to the threshold value or bias for the jth output neuron:

Δ θj = βo ej

Adjustment to the threshold value or bias for the jth hidden layer neuron:

δθj = βh ej

For use of momentum parameter α (more on this parameter in Chapter 13), instead of equations 7.5 and 7.6, use:

ΔM2[ i ][ j ] ( t ) = βo yiej + αΔM2[ i ][ j ] ( t - 1 )

(7.7)

and

ΔM1[ i ][ j ] ( t ) = βh xitj + αΔM1[ i ][ j ] (t - 1)

(7.8)

C++ Implementation of a Backpropagation Simulator

The backpropagation simulator of this chapter has the following design objectives:

1. Allow the user to specify the number and size of all layers.
2. Allow the use of one or more hidden layers.
3. Be able to save and restore the state of the network.
4. Run from an arbitrarily large training data set or test data set.
5. Query the user for key network and simulation parameters.
6. Display key information at the end of the simulation.
7. Demonstrate the use of some C++ features.

A Brief Tour of How to Use the Simulator

In order to understand the C++ code, let us have an overview of the functioning of the program.

There are two modes of operation in the simulator. The user is queried first for which mode of operation is desired. The modes are Training mode and Nontraining mode (Test mode).

Training Mode

Here, the user provides a training file in the current directory called training.dat. This file contains exemplar pairs, or patterns. Each pattern has a set of inputs followed by a set of outputs. Each value is separated by one or more spaces. As a convention, you can use a few extra spaces to separate the inputs from the outputs. Here is an example of a training.dat file that contains two patterns:

     0.4 0.5 0.89           -0.4 -0.8

     0.23 0.8 -0.3          0.6 0.34

In this example, the first pattern has inputs 0.4, 0.5, and 0.89, with an expected output of –0.4 and –0.8. The second pattern has inputs of 0.23, 0.8, and –0.3 and outputs of 0.6 and 0.34. Since there are three inputs and two outputs, the input layer size for the network must be three neurons and the output layer size must be two neurons. Another file that is used in training is the weights file. Once the simulator reaches the error tolerance that was specified by the user, or the maximum number of iterations, the simulator saves the state of the network, by saving all of its weights in a file called weights.dat. This file can then be used subsequently in another run of the simulator in Nontraining mode. To provide some idea of how the network has done, information about the total and average error is presented at the end of the simulation. In addition, the output generated by the network for the last pattern vector is provided in an output file called output.dat.

Nontraining Mode (Test Mode)

In this mode, the user provides test data to the simulator in a file called test.dat. This file contains only input patterns. When this file is applied to an already trained network, an output.dat file is generated, which contains the outputs from the network for all of the input patterns. The network goes through one cycle of operation in this mode, covering all the patterns in the test data file. To start up the network, the weights file, weights.dat is read to initialize the state of the network. The user must provide the same network size parameters used to train the network.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Operation

The first thing to do with your simulator is to train a network with an architecture you choose. You can select the number of layers and the number of hidden layers for your network. Keep in mind that the input and output layer sizes are dictated by the input patterns you are presenting to the network and the outputs you seek from the network. Once you decide on an architecture, perhaps a simple three-layer network with one hidden layer, you prepare training data for it and save the data in the training.dat file. After this you are ready to train. You provide the simulator with the following information:

• The mode (select 1 for training)
• The values for the error tolerance and the learning rate parameter, lambda or beta
• The maximum number of cycles, or passes through the training data you'd like to try
• The number of layers (between three and five, three implies one hidden layer, while five implies three hidden layers)
• The size for each layer, from the input to the output

The simulator then begins training and reports the current cycle number and the average error for each cycle. You should watch the error to see that it is on the whole decreasing with time. If it is not, you should restart the simulation, because this will start with a brand new set of random weights and give you another, possibly better, solution. Note that there will be legitimate periods where the error may increase for some time. Once the simulation is done you will see information about the number of cycles and patterns used, and the total and average error that resulted. The weights are saved in the weights.dat file. You can rename this file to use this particular state of the network later. You can infer the size and number of layers from the information in this file, as will be shown in the next section for the weights.dat file format. You can have a peek at the output.dat file to see the kind of training result you have achieved. To get a full-blown accounting of each pattern and the match to that pattern, copy the training file to the test file and delete the output information from it. You can then run Test mode to get a full list of all the input stimuli and responses in the output.dat file.

Summary of Files Used in the Backpropagation Simulator

Here is a list of the files for your reference, as well as what they are used for.

• weights.dat You can look at this file to see the weights for the network. It shows the layer number followed by the weights that feed into the layer. The first layer, or input layer, layer zero, does not have any weights associated with it. An example of the weights.dat file is shown as follows for a network with three layers of sizes 3, 5, and 2. Note that the row width for layer n matches the column length for layer n + 1:

1 -0.199660 -0.859660 -0.339660 -0.25966 0.520340

1  0.292860 -0.487140 0.212860 -0.967140 -0.427140

1  0.542106 -0.177894 0.322106 -0.977894 0.562106

2 -0.175350 -0.835350

2 -0.330167 -0.250167

2  0.503317 0.283317

2 -0.477158 0.222842

2 -0.928322 -0.388322

In this weights file the row width for layer 1 is 5, corresponding to the output of that (middle) layer. The input for the layer is the column length, which is 3, just as specified. For layer 2, the output size is the row width, which is 2, and the input size is the column length, 5, which is the same as the output for the middle layer. You can read the weights file to find out how things look.
• training.dat This file contains the input patterns for training. You can have as large a file as you'd like without degrading the performance of the simulator. The simulator caches data in memory for processing. This is to improve the speed of the simulation since disk accesses are expensive in time. A data buffer, which has a maximum size specified in a #define statement in the program, is filled with data from the training.dat file whenever data is needed. The format for the training.dat file has been shown in the Training mode section.
• test.dat The test.dat file is just like the training.dat file but without expected outputs. You use this file with a trained neural network in Test mode to see what responses you get for untrained data.
• output.dat The output.dat file contains the results of the simulation. In Test mode, the input and output vectors are shown for all pattern vectors. In the Simulator mode, the expected output is also shown, but only the last vector in the training set is presented, since the training set is usually quite large.
Shown here is an example of an output file in Training mode:

for input vector:

0.400000  -0.400000

output vector is:

0.880095

expected output vector is:

0.900000


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

C++ Classes and Class Hierarchy

So far, you have learned how we address most of the objectives outlined for this program. The only objective left involves the demonstration of some C++ features. In this program we use a class hierarchy with the inheritance feature. Also, we use polymorphism with dynamic binding and function overloading with static binding.

First let us look at the class hierarchy used for this program (see Figure 7.2). An abstract class is a class that is never meant to be instantiated as an object, but serves as a base class from which others can inherit functionality and interface definitions. The layer class is such a class. You will see shortly that one of its functions is set = zero, which indicates that this class is an abstract base class. From the layer class are two branches. One is the input_layer class, and the other is the output_layer class. The middle layer class is very much like the output layer in function and so inherits from the output_layer class.

Figure 7.2 Class hierarchy used in the backpropagation simulator.

Function overloading can be seen in the definition of the calc_error() function. It is used in the input_layer with no parameters, while it is used in the output_layer (which the input_layer inherits from) with one parameter. Using the same function name is not a problem, and this is referred to as overloading. Besides function overloading, you may also have operator overloading, which is using an operator that performs some familiar function like + for addition, for another function, say, vector addition.

When you have overloading with the same parameters and the keyword virtual, then you have the potential for dynamic binding, which means that you determine which overloaded function to execute at run time and not at compile time. Compile time binding is referred to as static binding. If you put a bunch of C++ objects in an array of pointers to the base class, and then go through a loop that indexes each pointer and executes an overloaded virtual function that pointer is pointing to, then you will be using dynamic binding. This is exactly the case in the function calc_out(), which is declared with the virtual keyword in the layer base class. Each descendant of layer can provide a version of calc_out(), which differs in functionality from the base class, and the correct function will be selected at run time based on the object's identity. In this case calc_out(), which is a function to calculate the outputs for each layer, is different for the input layer than for the other two types of layers.

Let's look at some details in the header file in Listing 7.1:

Listing 7.1 Header file for the backpropagation simulator

// layer.h            V.Rao, H. Rao

// header file for the layer class hierarchy and

// the network class

#define MAX_LAYERS    5

#define MAX_VECTORS   100

class network;

class layer

{

protected:

       int num_inputs;

       int num_outputs;

       float *outputs;// pointer to array of outputs

       float *inputs; // pointer to array of inputs, which

                      // are outputs of some other layer

       friend network;

public:

       virtual void calc_out()=0;

};

class input_layer: public layer

{

private:

public:

       input_layer(int, int);

       ~input_layer();

       virtual void calc_out();

};

class middle_layer;

class output_layer:   public layer

{

protected:

       float * weights;

       float * output_errors;    // array of errors at output

       float * back_errors;      // array of errors back-propagated

       float * expected_values;  // to inputs

   friend network;

public:

       output_layer(int, int);

       ~output_layer();

       virtual void calc_out();

       void calc_error(float &);

       void randomize_weights();

       void update_weights(const float);

       void list_weights();

       void write_weights(int, FILE *);

       void read_weights(int, FILE *);

       void list_errors();

       void list_outputs();

};

class middle_layer:   public output_layer

{

private:

public:

    middle_layer(int, int);

    ~middle_layer();

       void calc_error();

};

class network

{

private:

    layer *layer_ptr[MAX_LAYERS];

    int number_of_layers;

    int layer_size[MAX_LAYERS];

    float *buffer;

    fpos_t position;

    unsigned training;

public:

    network();

    ~network();

       void set_training(const unsigned &);

       unsigned get_training_value();

       void get_layer_info();

       void set_up_network();

       void randomize_weights();

       void update_weights(const float);

       void write_weights(FILE *);

       void read_weights(FILE *);

       void list_weights();

       void write_outputs(FILE *);

       void list_outputs();

       void list_errors();

       void forward_prop();

       void backward_prop(float &);

       int fill_IObuffer(FILE *);

       void set_up_pattern(int);

};


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Details of the Backpropagation Header File

At the top of the file, there are two #define statements, which are used to set the maximum number of layers that can be used, currently five, and the maximum number of training or test vectors that can be read into an I/O buffer. This is currently 100. You can increase the size of the buffer for better speed at the cost of increased memory usage.

The following are definitions in the layer base class. Note that the number of inputs and outputs are protected data members, which means that they can be accessed freely by descendants of the class.

int num_inputs;

int num_outputs;

float *outputs;      // pointer to array of outputs

float *inputs;       // pointer to array of inputs, which

                     // are outputs of some other layer

friend network;

There are also two pointers to arrays of floats in this class. They are the pointers to the outputs in a given layer and the inputs to a given layer. To get a better idea of what a layer encompasses, Figure 7.3 shows you a small feedforward backpropagation network, with a dotted line that shows you the three layers for that network. A layer contains neurons and weights. The layer is responsible for calculating its output (calc_out()), stored in the float * outputs array, and errors (calc_error()) for each of its respective neurons. The errors are stored in another array called float * output_errors defined in the output class. Note that the input class does not have any weights associated with it and therefore is a special case. It does not need to provide any data members or function members related to errors or backpropagation. The only purpose of the input layer is to store data to be forward propagated to the next layer.

Figure 7.3 Organization of layers for backpropagation program.

With the output layer, there are a few more arrays present. First, for storing backpropagated errors, there is an array called float * back_errors. There is a weights array called float * weights, and finally, for storing the expected values that initiate the error calculation process, there is an array called float * expected_values. Note that the middle layer needs almost all of these arrays and inherits them by being a derived class of the output_layer class.

There is one other class besides the layer class and its descendants defined in this header file, and that is the network class, which is used to set up communication channels between layers and to feed and remove data from the network. The network class performs the interconnection of layers by setting the pointer of an input array of a given layer to the output array of a previous layer.

This is a fairly extensible scheme that can be used to create variations on the feedforward backpropagation network with feedback connections, for instance.

Another connection that the network class is responsible for is setting the pointer of an output_error array to the back_error array of the next layer (remember, errors flow in reverse, and the back_error array is the output error of the layer reflected at its inputs).

The network class stores an array of pointers to layers and an array of layer sizes for all the layers defined. These layer objects and arrays are dynamically allocated on the heap with the New and Delete functions in C++. There is some minimal error checking for file I/O and memory allocation, which can be enhanced, if desired.

As you can see, the feedforward backpropagation network can quickly become a memory and CPU hog, with large networks and large training sets. The size and topology of the network, or architecture, will largely dictate both these characteristics.

Details of the Backpropagation Implementation File

The implementation of the classes and methods is the next topic. Let's look at the layer.cpp file in Listing 7.2.

Listing 7.2 layer.cpp implementation file for the backpropagation simulator

// layer.cpp           V.Rao, H.Rao

// compile for floating point hardware if available

#include <stdio.h>

#include <iostream.h>

#include <stdlib.h>

#include <math.h>

#include <time.h>

#include "layer.h"

inline float squash(float input)

// squashing function

// use sigmoid -- can customize to something

// else if desired; can add a bias term too

//

{

if (input < -50)

       return 0.0;

else   if (input > 50)

              return 1.0;

       else return (float)(1/(1+exp(-(double)input)));

}

inline float randomweight(unsigned init)

{

int num;

// random number generator

// will return a floating point

// value between -1 and 1

if (init==1)   // seed the generator

       srand ((unsigned)time(NULL));

num=rand() % 100;

return 2*(float(num/100.00))-1;

}

// the next function is needed for Turbo C++

// and Borland C++ to link in the appropriate

// functions for fscanf floating point formats:

static void force_fpf()

{

       float x, *y;

       y=&x;

       x=*y;

}

// --------------------

//                            input layer

//---------------------

input_layer::input_layer(int i, int o)

{

num_inputs=i;

num_outputs=o;

outputs = new float[num_outputs];

if (outputs==0)

       {

       cout << "not enough memory\n";

       cout << "choose a smaller architecture\n";

       exit(1);

       }

}

input_layer::~input_layer()

{

delete [num_outputs] outputs;

}

void input_layer::calc_out()

{

//nothing to do, yet

}

// --------------------

//                            output layer

//---------------------

output_layer::output_layer(int i, int o)

{

num_inputs         =i;

num_outputs        =o;

weights            = new float[num_inputs*num_outputs];

output_errors      = new float[num_outputs];

back_errors        = new float[num_inputs];

outputs            = new float[num_outputs];

expected_values    = new float[num_outputs];

if ((weights==0)||(output_errors==0)||(back_errors==0)

       ||(outputs==0)||(expected_values==0))

       {

       cout << "not enough memory\n";

       cout << "choose a smaller architecture\n";

       exit(1);

       }

}

output_layer::~output_layer()

{

// some compilers may require the array

// size in the delete statement; those

// conforming to Ansi C++ will not

delete [num_outputs*num_inputs] weights;

delete [num_outputs] output_errors;

delete [num_inputs] back_errors;

delete [num_outputs] outputs;

}

void output_layer::calc_out()

{

int i,j,k;

float accumulator=0.0;

for (j=0; j<num_outputs; j++)

       {

       for (i=0; i<num_inputs; i++)

              {

              k=i*num_outputs;

              if (weights[k+j]*weights[k+j] > 1000000.0)

                     {

                     cout << "weights are blowing up\n";

                     cout << "try a smaller learning constant\n";

                     cout << "e.g. beta=0.02    aborting...\n";

                     exit(1);

                     }

              outputs[j]=weights[k+j]*(*(inputs+i));

              accumulator+=outputs[j];

              }

       // use the sigmoid squash function

       outputs[j]=squash(accumulator);

       accumulator=0;

       }

}

void output_layer::calc_error(float & error)

{

int i, j, k;

float accumulator=0;

float total_error=0;

for (j=0; j<num_outputs; j++)

    {

       output_errors[j] = expected_values[j]-outputs[j];

       total_error+=output_errors[j];

       }

error=total_error;

for (i=0; i<num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j<num_outputs; j++)

              {

              back_errors[i]=

                     weights[k+j]*output_errors[j];

              accumulator+=back_errors[i];

              }

       back_errors[i]=accumulator;

       accumulator=0;

       // now multiply by derivative of

       // sigmoid squashing function, which is

       // just the input*(1-input)

       back_errors[i]*=(*(inputs+i))*(1-(*(inputs+i)));

       }

}

void output_layer::randomize_weights()

{

int i, j, k;

const unsigned first_time=1;

const unsigned not_first_time=0;

float discard;

discard=randomweight(first_time);

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              weights[k+j]=randomweight(not_first_time);

       }

}

void output_layer::update_weights(const float beta)

{

int i, j, k;

// learning law: weight_change =

//            beta*output_error*input

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              weights[k+j] +=

                     beta*output_errors[i]*(*(inputs+i));

    }

}

void output_layer::list_weights()

{

int i, j, k;

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              cout << "weight["<<i<<","<<

                     j<<"] is: "<<weights[k+j];

       }

}

void output_layer::list_errors()

{

int i, j;

for (i=0; i< num_inputs; i++)

       cout << "backerror["<<i<<

              "] is : "<<back_errors[i]<<"\n";

for (j=0; j< num_outputs; j++)

       cout << "outputerrors["<<j<<

                     "] is: "<<output_errors[j]<<"\n";

}

void output_layer::write_weights(int layer_no,

              FILE * weights_file_ptr)

{

int i, j, k;

// assume file is already open and ready for

// writing

// prepend the layer_no to all lines of data

// format:

//            layer_no   weight[0,0] weight[0,1] ...

//            layer_no   weight[1,0] weight[1,1] ...

//            ...

for (i=0; i< num_inputs; i++)

       {

       fprintf(weights_file_ptr,"%i ",layer_no);

       k=i*num_outputs;

    for (j=0; j< num_outputs; j++)

       {

       fprintf(weights_file_ptr,"%f",

                     weights[k+j]);

       }

    fprintf(weights_file_ptr,"\n");

    }

}

void output_layer::read_weights(int layer_no,

              FILE * weights_file_ptr)

{

int i, j, k;

// assume file is already open and ready for

// reading

// look for the prepended layer_no

// format:

//            layer_no       weight[0,0] weight[0,1] ...

//            layer_no       weight[1,0] weight[1,1] ...

//            ...

while (1)

       {

       fscanf(weights_file_ptr,"%i",&j);

       if ((j==layer_no)|| (feof(weights_file_ptr)))

              break;

       else

              {

              while (fgetc(weights_file_ptr) != `\n')

                     {;}// get rest of line

              }

       }

if (!(feof(weights_file_ptr)))

       {

       // continue getting first line

       i=0;

       for (j=0; j< num_outputs; j++)

                         {

                         fscanf(weights_file_ptr,"%f",

                              &weights[j]); // i*num_outputs = 0

                         }

       fscanf(weights_file_ptr,"\n");

       // now get the other lines

       for (i=1; i< num_inputs; i++)

              {

              fscanf(weights_file_ptr,"%i",&layer_no);

              k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              {

              fscanf(weights_file_ptr,"%f",

                     &weights[k+j]);

              }

              }

    fscanf(weights_file_ptr,"\n");

    }

else cout << "end of file reached\n";

}

void output_layer::list_outputs()

{

int j;

for (j=0; j< num_outputs; j++)

       {

       cout << "outputs["<<j

              <<"] is: "<<outputs[j]<<"\n";

       }

}

// ---------------------

//                            middle layer

//----------------------

middle_layer::middle_layer(int i, int o):

       output_layer(i,o)

{

}

middle_layer::~middle_layer()

{

delete [num_outputs*num_inputs] weights;

delete [num_outputs] output_errors;

delete [num_inputs] back_errors;

delete [num_outputs] outputs;

}

void middle_layer::calc_error()

{

int i, j, k;

float accumulator=0;

for (i=0; i<num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j<num_outputs; j++)

              {

              back_errors[i]=

                     weights[k+j]*(*(output_errors+j));

              accumulator+=back_errors[i];

              }

       back_errors[i]=accumulator;

       accumulator=0;

       // now multiply by derivative of

       // sigmoid squashing function, which is

       // just the input*(1-input)

       back_errors[i]*=(*(inputs+i))*(1-(*(inputs+i)));

       }

}

network::network()

{

position=0L;

}

network::~network()

{

int i,j,k;

i=layer_ptr[0]->num_outputs;// inputs

j=layer_ptr[number_of_layers-1]->num_outputs; //outputs

k=MAX_VECTORS;

delete [(i+j)*k]buffer;

}

void network::set_training(const unsigned & value)

{

training=value;

}

unsigned network::get_training_value()

{

return training;

}

void network::get_layer_info()

{

int i;

//---------------------

//

//     Get layer sizes for the network

//

// --------------------

cout << " Please enter in the number of layers for your network.\n";

cout << " You can have a minimum of 3 to a maximum of 5. \n";

cout << " 3 implies 1 hidden layer; 5 implies 3 hidden layers : \n\n";

cin >> number_of_layers;

cout << " Enter in the layer sizes separated by spaces.\n";

cout << " For a network with 3 neurons in the input layer,\n";

cout << " 2 neurons in a hidden layer, and 4 neurons in the\n";

cout << " output layer, you would enter: 3 2 4 .\n";

cout << " You can have up to 3 hidden layers,for five maximum entries

:\n\n";

for (i=0; i<number_of_layers; i++)

       {

       cin >> layer_size[i];

       }

// --------------------------

// size of layers:

//            input_layer       layer_size[0]

//            output_layer      layer_size[number_of_layers-1]

//            middle_layers     layer_size[1]

//                              optional: layer_size[number_of_layers-3]

//                              optional: layer_size[number_of_layers-2]

//---------------------------

}

void network::set_up_network()

{

int i,j,k;

//---------------------------

// Construct the layers

//

//---------------------------

layer_ptr[0] = new input_layer(0,layer_size[0]);

for (i=0;i<(number_of_layers-1);i++)

       {

       layer_ptr[i+1y] =

       new middle_layer(layer_size[i],layer_size[i+1]);

       }

layer_ptr[number_of_layers-1] = new

output_layer(layer_size[number_of_layers-2],layer_size[number_of_layers-

1]);

for (i=0;i<(number_of_layers-1);i++)

       {

       if (layer_ptr[i] == 0)

              {

              cout << "insufficient memory\n";

              cout << "use a smaller architecture\n";

              exit(1);

              }

       }

//--------------------------

// Connect the layers

//

//--------------------------

// set inputs to previous layer outputs for all layers,

//     except the input layer

for (i=1; i< number_of_layers; i++)

        layer_ptr[i]->inputs = layer_ptr[i-1y]->outputs;

// for back_propagation, set output_errors to next layer

//            back_errors for all layers except the output

//            layer and input layer

for (i=1; i< number_of_layers -1; i++)

       ((output_layer *)layer_ptr[i])->output_errors =

              ((output_layer *)layer_ptr[i+1])->back_errors;

// define the IObuffer that caches data from

// the datafile

i=layer_ptr[0]->num_outputs;// inputs

j=layer_ptr[number_of_layers-1]->num_outputs; //outputs

k=MAX_VECTORS;

buffer=new

       float[(i+j)*k];

if (buffer==0)

       cout << "insufficient memory for buffer\n";

}

void network::randomize_weights()

{

int i;

for (i=1; i<number_of_layers; i++)

       ((output_layer *)layer_ptr[i])

              ->randomize_weights();

}

void network::update_weights(const float beta)

{

int i;

for (i=1; i<number_of_layers; i++)

       ((output_layer *)layer_ptr[i])

              ->update_weights(beta);

}

void network::write_weights(FILE * weights_file_ptr)

{

int i;

for (i=1; i<number_of_layers; i++)

       ((output_layer *)layer_ptr[i])

              ->write_weights(i,weights_file_ptr);

}

void network::read_weights(FILE * weights_file_ptr)

{

int i;

for (i=1; i<number_of_layers; i++)

       ((output_layer *)layer_ptr[i])

              ->read_weights(i,weights_file_ptr);

}

void network::list_weights()

{

int i;

for (i=1; i<number_of_layers; i++)

       {

       cout << "layer number : " <<i<< "\n";

       ((output_layer *)layer_ptr[i])

              ->list_weights();

       }

}

void network::list_outputs()

{

int i;

for (i=1; i<number_of_layers; i++)

       {

       cout << "layer number : " <<i<< "\n";

       ((output_layer *)layer_ptr[i])

              ->list_outputs();

       }

}

void network::write_outputs(FILE *outfile)

{

int i, ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

float temp;

fprintf(outfile,"for input vector:\n");

for (i=0; i<ins; i++)

       {

       temp=layer_ptr[0]->outputs[i];

       fprintf(outfile,"%f  ",temp);

       }

fprintf(outfile,"\noutput vector is:\n");

for (i=0; i<outs; i++)

       {

       temp=layer_ptr[number_of_layers-1]->

       outputs[i];

       fprintf(outfile,"%f  ",temp);

       }

if (training==1)

{

fprintf(outfile,"\nexpected output vector is:\n");

for (i=0; i<outs; i++)

       {

       temp=((output_layer *)(layer_ptr[number_of_layers-1]))->

       expected_values[i];

       fprintf(outfile,"%f  ",temp);

       }

}

fprintf(outfile,"\n----------\n");

}

void network::list_errors()

{

int i;

for (i=1; i<number_of_layers; i++)

       {

       cout << "layer number : " <<i<< "\n";

       ((output_layer *)layer_ptr[i])

              ->list_errors();

       }

}

int network::fill_IObuffer(FILE * inputfile)

{

// this routine fills memory with

// an array of input, output vectors

// up to a maximum capacity of

// MAX_INPUT_VECTORS_IN_ARRAY

// the return value is the number of read

// vectors

int i, k, count, veclength;

int ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

if (training==1)

       veclength=ins+outs;

else

       veclength=ins;

count=0;

while  ((count<MAX_VECTORS)&&

              (!feof(inputfile)))

       {

       k=count*(veclength);

       for (i=0; i<veclength; i++)

              {

              fscanf(inputfile,"%f",&buffer[k+i]);

              }

       fscanf(inputfile,"\n");

       count++;

       }

if (!(ferror(inputfile)))

       return count;

else return -1; // error condition

}

void network::set_up_pattern(int buffer_index)

{

// read one vector into the network

int i, k;

int ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

if (training==1)

       k=buffer_index*(ins+outs);

else

       k=buffer_index*ins;

for (i=0; i<ins; i++)

       layer_ptr[0]->outputs[i]=buffer[k+i];

if (training==1)

{

       for (i=0; i<outs; i++)

              ((output_layer *)layer_ptr[number_of_layers-1])->

                     expected_values[i]=buffer[k+i+ins];

}

}

void network::forward_prop()

{

int i;

for (i=0; i<number_of_layers; i++)

       {

       layer_ptr[i]->calc_out(); //polymorphic

                                 // function

       }

}

void network::backward_prop(float & toterror)

{

int i;

// error for the output layer

((output_layer*)layer_ptr[number_of_layers-1])->

                      calc_error(toterror);

// error for the middle layer(s)

for (i=number_of_layers-2; i>0; i--)

       {

       ((middle_layer*)layer_ptr[i])->

                     calc_error();

       }

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

A Look at the Functions in the layer.cpp File

The following is a listing of the functions in the layer.cpp file along with a brief statement of each one's purpose.

• void set_training(const unsigned &) Sets the value of the private data member, training; use 1 for training mode, and 0 for test mode.
• unsigned get_training_value() Gets the value of the training constant that gives the mode in use.
• void get_layer_info() Gets information about the number of layers and layer sizes from the user.
• void set_up_network() This routine sets up the connections between layers by assigning pointers appropriately.
• void randomize_weights() At the beginning of the training process, this routine is used to randomize all of the weights in the network.
• void update_weights(const float) As part of training, weights are updated according to the learning law used in backpropagation.
• void write_weights(FILE *) This routine is used to write weights to a file.
• void read_weights(FILE *) This routine is used to read weights into the network from a file.
• void list_weights() This routine can be used to list weights while a simulation is in progress.
• void write_outputs(FILE *) This routine writes the outputs of the network to a file.
• void list_outputs() This routine can be used to list the outputs of the network while a simulation is in progress.
• void list_errors() Lists errors for all layers while a simulation is in progress.
• void forward_prop() Performs the forward propagation.
• void backward_prop(float &) Performs the backward error propagation.
• int fill_IObuffer(FILE *) This routine fills the internal IO buffer with data from the training or test data sets.
• void set_up_pattern(int) This routine is used to set up one pattern from the IO buffer for training.
• inline float squash(float input) This function performs the sigmoid function.
• inline float randomweight (unsigned unit) This routine returns a random weight between –1 and 1; use 1 to initialize the generator, and 0 for all subsequent calls.

Note that the functions squash(float) and randomweight(unsigned) are declared inline. This means that the function's source code is inserted wherever it appears. This increases code size, but also increases speed because a function call, which is expensive, is avoided.

The final file to look at is the backprop.cpp file presented in Listing 7.3.

Listing 7.3 The backprop.cpp file for the backpropagation simulator

// backprop.cpp         V. Rao, H. Rao

#include "layer.cpp"

#define TRAINING_FILE   "training.dat"

#define WEIGHTS_FILE    "weights.dat"

#define OUTPUT_FILE     "output.dat"

#define TEST_FILE       "test.dat"

void main()

{

float error_tolerance        =0.1;

float total_error            =0.0;

float avg_error_per_cycle    =0.0;

float error_last_cycle       =0.0;

float avgerr_per_pattern     =0.0; // for the latest cycle

float error_last_pattern     =0.0;

float learning_parameter     =0.02;

unsigned temp, startup;

long int vectors_in_buffer;

long int max_cycles;

long int patterns_per_cycle  =0;

long int total_cycles, total_patterns;

int i;

// create a network object

network backp;

FILE * training_file_ptr, * weights_file_ptr, * output_file_ptr;

FILE * test_file_ptr, * data_file_ptr;

// open output file for writing

if ((output_file_ptr=fopen(OUTPUT_FILE,"w"))==NULL)

               {

               cout << "problem opening output file\n";

               exit(1);

               }

// enter the training mode : 1=training on     0=training off

cout << "------------------------\n";

cout << " C++ Neural Networks and Fuzzy Logic \n";

cout << "      Backpropagation simulator \n";

cout << "             version 1 \n";

cout << "------------------------\n";

cout << "Please enter 1 for TRAINING on, or 0 for off: \n\n";

cout << "Use training to change weights according to your\n";

cout << "expected outputs. Your training.dat file should contain\n";

cout << "a set of inputs and expected outputs. The number of\n";

cout << "inputs determines the size of the first (input) layer\n";

cout << "while the number of outputs determines the size of the\n";

       cout << "last (output) layer :\n\n";

cin >> temp;

backp.set_training(temp);

if (backp.get_training_value() == 1)

       {

       cout << "--> Training mode is *ON*. weights will be saved\n";

       cout << "in the file weights.dat at the end of the\n";

       cout << "current set of input (training) data\n";

       }

else

       {

       cout << "--> Training mode is *OFF*. weights will be loaded\n";

       cout << "from the file weights.dat and the current\n";

       cout << "(test) data set will be used. For the test\n";

       cout << "data set, the test.dat file should contain\n";

       cout << "only inputs, and no expected outputs.\n";

}

if (backp.get_training_value()==1)

       {

       // --------------------

       //     Read in values for the error_tolerance,

       //     and the learning_parameter

       // --------------------

       cout << " Please enter in the error_tolerance\n";

       cout << " --- between 0.001 to 100.0, try 0.1 to start \n";

       cout << "\n";

       cout << "and the learning_parameter, beta\n";

       cout << " --- between 0.01 to 1.0, try 0.5 to start -- \n\n";

       cout << " separate entries by a space\n";

       cout << " example: 0.1 0.5 sets defaults mentioned :\n\n";

       cin >> error_tolerance >> learning_parameter;

       //---------------------

       // open training file for reading

       //--------------------

       if ((training_file_ptr=fopen(TRAINING_FILE,"r"))==NULL)

              {

              cout << "problem opening training file\n";

              exit(1);

              }

       data_file_ptr=training_file_ptr; // training on

       // Read in the maximum number of cycles

       // each pass through the input data file is a cycle

       cout << "Please enter the maximum cycles for the simula-\

       tion\n";

       cout << "A cycle is one pass through the data set.\n";

       cout << "Try a value of 10 to start with\n";

       cin >> max_cycles;

       }

else

       {

       if ((test_file_ptr=fopen(TEST_FILE,"r"))==NULL)

              {

              cout << "problem opening test file\n";

              exit(1);

              }

       data_file_ptr=test_file_ptr; // training off

       }

//

// training: continue looping until the total error is less than

//            the tolerance specified, or the maximum number of

//            cycles is exceeded; use both the forward signal propaga

tion

//            and the backward error propagation phases. If the error

//            tolerance criteria is satisfied, save the weights in a

file.

// no training: just proceed through the input data set once in the

//            forward signal propagation phase only. Read the starting

//            weights from a file.

// in both cases report the outputs on the screen

// initialize counters

total_cycles=0; // a cycle is once through all the input data

total_patterns=0; // a pattern is one entry in the input data

// get layer information

backp.get_layer_info();

// set up the network connections

backp.set_up_network();

// initialize the weights

if (backp.get_training_value()==1)

       {

       // randomize weights for all layers; there is no

       // weight matrix associated with the input layer

       // weight file will be written after processing

       // so open for writing

       if ((weights_file_ptr=fopen(WEIGHTS_FILE,"w"))

                     ==NULL)

              {

              cout << "problem opening weights file\n";

              exit(1);

              }

       backp.randomize_weights();

       }

else

       {

       // read in the weight matrix defined by a

       // prior run of the backpropagation simulator

       // with training on

       if ((weights_file_ptr=fopen(WEIGHTS_FILE,"r"))

                     ==NULL)

              {

              cout << "problem opening weights file\n";

              exit(1);

              }

       backp.read_weights(weights_file_ptr);

       }

// main loop

// if training is on, keep going through the input data

//             until the error is acceptable or the maximum number of

//     cycles

//             is exceeded.

// if training is off, go through the input data once. report // outputs

// with inputs to file output.dat

startup=1;

vectors_in_buffer = MAX_VECTORS; // startup condition

total_error = 0;

while (              ((backp.get_training_value()==1)

                     && (avgerr_per_pattern

                                   > error_tolerance)

                     && (total_cycles < max_cycles)

                     && (vectors_in_buffer !=0))

                     || ((backp.get_training_value()==0)

                     && (total_cycles < 1))

                     || ((backp.get_training_value()==1)

                     && (startup==1))

                     )

{

startup=0;

error_last_cycle=0; // reset for each cycle

patterns_per_cycle=0;

// process all the vectors in the datafile

// going through one buffer at a time

// pattern by pattern

while ((vectors_in_buffer==MAX_VECTORS))

       {

       vectors_in_buffer=

              backp.fill_IObuffer(data_file_ptr); // fill buffer

              if (vectors_in_buffer < 0)

                     {

                     cout << "error in reading in vectors, aborting\n";

                     cout << "check that there are no extra

linefeeds\n";

                     cout << "in your data file, and that the

number\n";

                     cout << "of layers and size of layers match

the\n";

                     cout << "the parameters provided.\n";

                     exit(1);

                     }

              // process vectors

              for (i=0; i<vectors_in_buffer; i++)

                     {

                     // get next pattern

                     backp.set_up_pattern(i);

                     total_patterns++;

                     patterns_per_cycle++;

                     // forward propagate

                     backp.forward_prop();

                     if (backp.get_training_value()==0)

                             backp.write_outputs(output_file_ptr);

                     // back_propagate, if appropriate

                     if (backp.get_training_value()==1)

                             {

                             backp.backward_prop(error_last_pattern);

                             error_last_cycle += error_last_pattern

z                            *error_last_pattern;

                             backp.update_weights(learning_parameter);

                             // backp.list_weights();

                             // can

                             // see change in weights by

                             // using list_weights before and

                             // after back_propagation

                             }

                     }

       error_last_pattern = 0;

       }

avgerr_per_pattern=((float)sqrt((double)error_last_cycle

/patterns_per_cycle));

total_error += error_last_cycle;

total_cycles++;

// most character displays are 26 lines

// user will see a corner display of the cycle count

// as it changes

cout << "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n";

cout << total_cycles << "\t" << avgerr_per_pattern << "\n";

fseek(data_file_ptr, 0L, SEEK_SET); // reset the file pointer

                              // to the beginning of

                              // the file

vectors_in_buffer = MAX_VECTORS; // reset

} // end main loop

cout << "\n\n\n\n\n\n\n\n\n\n\n";

cout << "------------------------\n";

cout << "    done:   results in file output.dat\n";

cout << "            training: last vector only\n";

cout << "            not training: full cycle\n\n";

if (backp.get_training_value()==1)

       {

       backp.write_weights(weights_file_ptr);

       backp.write_outputs(output_file_ptr);

       avg_error_per_cycle = (float)sqrt((double)total_error/

total_cycles);

   error_last_cycle = (float)sqrt((double)error_last_cycle);

cout << "      weights saved in file weights.dat\n";

cout << "\n";

cout << "-->average error per cycle = " <<

avg_error_per_cycle << " <-\n";

cout << "-->error last cycle= " << error_last_cycle << " <-\n";

cout << "->error last cycle per pattern= " << avgerr_per_pattern << " <-

\n";

       }

cout << "------>total cycles = " << total_cycles << " <--\n";

cout << "------>total patterns = " << total_patterns << " <---\n";

cout << "-------------------------\n";

// close all files

fclose(data_file_ptr);

fclose(weights_file_ptr);

fclose(output_file_ptr);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The backprop.cpp file implements the simulator controls. First, data is accepted from the user for network parameters. Assuming Training mode is used, the training file is opened and data is read from the file to fill the IO buffer. Then the main loop is executed where the network processes pattern by pattern to complete a cycle, which is one pass through the entire training data set. (The IO buffer is refilled as required during this process.) After executing one cycle, the file pointer is reset to the beginning of the file and another cycle begins. The simulator continues with cycles until one of the two fundamental criteria is met:

1. The maximum cycle count specified by the user is exceeded.
2. The average error per pattern for the latest cycle is less than the error tolerance specified by the user.

When either of these occurs, the simulator stops and reports out the error achieved, and saves weights in the weights.dat file and one output vector in the output.dat file.

In Test mode, exactly one cycle is processed by the network and outputs are written to the output.dat file. At the beginning of the simulation in Test mode, the network is set up with weights from the weights.dat file. To simplify the program, the user is requested to enter the number of layers and size of layers, although you could have the program figure this out from the weights file.

Compiling and Running the Backpropagation Simulator

Compiling the backprop.cpp file will compile the simulator since layer.cpp is included in backprop.cpp. To run the simulator, once you have created an executable (using 80X87 floating point hardware if available), you type in backprop and see the following screen (user input in italic):

C++ Neural Networks and Fuzzy Logic

       Backpropagation simulator

               version 1

Please enter 1 for TRAINING on, or 0 for off:

Use training to change weights according to your

expected outputs. Your training.dat file should contain

a set of inputs and expected outputs. The number of

inputs determines the size of the first (input) layer

while the number of outputs determines the size of the

last (output) layer :

1

-> Training mode is *ON*. weights will be saved

in the file weights.dat at the end of the

current set of input (training) data

 Please enter in the error_tolerance

 -- between 0.001 to 100.0, try 0.1 to start --

and the learning_parameter, beta

 -- between 0.01 to 1.0, try 0.5 to start --

 separate entries by a space

 example: 0.1 0.5 sets defaults mentioned :

0.2 0.25

Please enter the maximum cycles for the simulation

A cycle is one pass through the data set.

Try a value of 10 to start with

Please enter in the number of layers for your network.

You can have a minimum of three to a maximum of five.

three implies one hidden layer; five implies three hidden layers:

3

Enter in the layer sizes separated by spaces.

For a network with three neurons in the input layer,

two neurons in a hidden layer, and four neurons in the

output layer, you would enter: 3 2 4.

You can have up to three hidden layers for five maximum entries :

2 2 1

1        0.353248

2        0.352684

3        0.352113

4        0.351536

5        0.350954

...

299      0.0582381

300      0.0577085

------------------------

         done:   results in file output.dat

                 training: last vector only

                 not training: full cycle

                 weights saved in file weights.dat

-->average error per cycle = 0.20268 <--

-->error last cycle = 0.0577085 <--

->error last cycle per pattern= 0.0577085 <--

------>total cycles = 300 <--

------>total patterns = 300 <--

The cycle number and the average error per pattern is displayed as the simulation progresses (not all values shown). You can monitor this to make sure the simulator is converging on a solution. If the error does not seem to decrease beyond a certain point, but instead drifts or blows up, then you should start the simulator again with a new starting point defined by the random weights initializer. Also, you could try decreasing the size of the learning rate parameter. Learning may be slower, but this may allow a better minimum to be found.

This example shows just one pattern in the training set with two inputs and one output. The results along with the (one) last pattern are shown as follows from the file output.dat:

for input vector:

0.400000  -0.400000

output vector is:

0.842291

expected output vector is:

0.900000

The match is pretty good, as can be expected, since the optimization is easy for the network; there is only one pattern to worry about. Let's look at the final set of weights for this simulation in weights.dat. These weights were obtained by updating the weights for 300 cycles with the learning law:

     1 0.175039 0.435039

     1 -1.319244 -0.559244

     2 0.358281

     2 2.421172

We'll leave the backpropagation simulator for now and return to it in a later chapter for further exploration. You can experiment a number of different ways with the simulator:

• Try a different number of layers and layer sizes for a given problem.
• Try different learning rate parameters and see its effect on convergence and training time.
• Try a very large learning rate parameter (should be between 0 and 1); try a number over 1 and note the result.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Summary

In this chapter, you learned about one of the most powerful neural network algorithms called backpropagation. Without having feedback connections, propagating only errors appropriately to the hidden layer and input layer connections, the algorithm uses the so-called generalized delta rule and trains the network with exemplar pairs of patterns. It is difficult to determine how many hidden-layer neurons are to be provided for. The number of hidden layers could be more than one. In general, the size of the hidden layer(s) is related to the features or distinguishing characteristics that should be discerned from the data. Our example in this chapter relates to a simple case where there is a single hidden layer. The outputs of the output neurons, and therefore of the network, are vectors with components between 0 and 1, since the thresholding function is the sigmoid function. These values can be scaled, if necessary, to get values in another interval.

Our example does not relate to any particular function to be computed by the network, but inputs and outputs were randomly chosen. What this can tell you is that, if you do not know the functional equation between two sets of vectors, the feedback backpropagation network can find the mapping for any vector in the domain, even if the functional equation is not found. For all we know, that function could be nonlinear as well.

There is one important fact you need to remember about the backpropagation algorithm. Its steepest descent procedure in training does not guarantee finding a global or overall minimum, it can find only a local minimum of the energy surface.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 8
BAM: Bidirectional Associative Memory

Introduction

The Bidirectional Associative Memory (BAM) model has a neural network of two layers and is fully connected from each layer to the other. That is, there are feedback connections from the output layer to the input layer. However, the weights on the connections between any two given neurons from different layers are the same. You may even consider it to be a single bidirectional connection with a single weight. The matrix of weights for the connections from the output layer to the input layer is simply the transpose of the matrix of weights for the connections between the input and output layer. If we denote the matrix for forward connection weights by W, then W T is the matrix of weights for the output layer to input layer connections. As you recall, the transpose of a matrix is obtained simply by interchanging the rows and the columns of the matrix.

There are two layers of neurons, an input layer and an output layer. There are no lateral connections, that is, no two neurons within the same layer are connected. Recurrent connections, which are feedback connections to a neuron from itself, may or may not be present. The architecture is quite simple. Figure 8.1 shows the layout for this neural network model, using only three input neurons and two output neurons. There are feedback connections from Field A to Field B and vice-versa. This figure also indicates the presence of inputs and outputs at each of the two fields for the bidirectional associative memory network. Connection weights are also shown as labels on only a few connections in this figure, to avoid cluttering. The general case is analogous.

Figure 8.1 Layout of a BAM network

Inputs and Outputs

The input to a BAM network is a vector of real numbers, usually in the set { –1, +1 }. The output is also a vector of real numbers, usually in the set { –1, +1 }, with the same or different dimension from that of the input. These vectors can be considered patterns, and the network makes heteroassociation of patterns. If the output is required to be the same as input, then you are asking the network to make autoassociation, which it does, and it becomes a special case of the general activity of this type of neural network.

For inputs and outputs that are not outside the set containing just –1 and +1, try following this next procedure. You can first make a mapping into binary numbers, and then a mapping of each binary digit into a bipolar digit. For example, if your inputs are first names of people, each character in the name can be replaced by its ASCII code, which in turn can be changed to a binary number, and then each binary digit 0 can be replaced by –1. For example, the ASCII code for the letter R is 82, which is 1010010, as a binary number. This is mapped onto the bipolar string 1 –1 1 –1 –1 1 –1. If a name consists of three characters, their ASCII codes in binary can be concatenated or juxtaposed and the corresponding bipolar string obtained. This bipolar string can also be looked upon as a vector of bipolar characters.

Weights and Training

BAM does not modify weights during its operation, and as mentioned in Chapter 6, like the Hopfield network, uses one-shot training. The adaptive variety of BAM, called the Adaptive Bidirectional Associative Memory, (ABAM) undergoes supervised iterative training. BAM needs some exemplar pairs of vectors. The pairs used as exemplars are those that require heteroassociation. The weight matrix, there are two, but one is just the transpose of the other as already mentioned, is constructed in terms of the exemplar vector pairs.

The use of exemplar vectors is a one-shot learning—to determine what the weights should be. Once weights are so determined, and an input vector is presented, a potentially associated vector is output. It is taken as input in the opposite direction, and its potentially associated vector is obtained back at the input layer. If the last vector found is the same as what is originally input, then there is resonance. Suppose the vector B is obtained at one end, as a result of C being input at the other end. If B in turn is input during the next cycle of operation at the end where it was obtained, and produces C at the opposite end, then you have a pair of heteroassociated vectors. This is what is basically happening in a BAM neural network.

NOTE: The BAM and Hopfield memories are closely related. You can think of the Hopfield memory as a special case of the BAM.

What follow are the equations for the determination of the weight matrix, when the k pairs of exemplar vectors are denoted by ( Xi, Yi), i ranging from 1 to k. Note that T in the superscript of a matrix stands for the transpose of the matrix. While you interchange the rows and columns to get the transpose of a matrix, you write a column vector as a row vector, and vice versa to get the transpose of a vector. The following equations refer to the vector pairs after their components are changed to bipolar values, only for obtaining the weight matrix W. Once W is obtained, further use of these exemplar vectors is made in their original form.

     W =  X1T Y1 + ... + XkT Yk

and

     WT =  Y1TX1 + ... + YkT Xk

Example

Suppose you choose two pairs of vectors as possible exemplars. Let them be:

     X1 = (1, 0, 0, 1), Y1= (0, 1, 1)

and

     X2 = (0, 1, 1, 0), Y2 = (1, 0, 1)

These you change into bipolar components and get, respectively, (1, –1, –1, 1), (–1, 1, 1), (–1, 1, 1, –1), and (1, –1, 1).

      1 [-1 1 1]   -1 [1 -1 1]   -1  1  1     -1  1 -1    -2  2 0

 W = -1           + 1          =  1 -1 -1  +   1 -1  1  =  2 -2 0

     -1             1             1 -1 -1      1 -1  1     2 -2 0

      1            -1            -1  1  1     -1  1 -1    -2  2 0

and

       -2    2    2    -2

WT =    2   -2   -2     2

        0    0    0     0


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

You may think that the last column of W being all zeros presents a problem in that when the input is X1 and whatever the output, when this output is presented back in the backward direction, it does not produce X1, which does not have a zero in the last component. There is a thresholding function needed here. The thresholding function is transparent when the activations are all either +1 or –1. It then just looks like you do an inverse mapping from bipolar values to binary, which is done by replacing each –1 by a 0. When the activation is zero, you simply leave the output of that neuron as it was in the previous cycle or iteration. We now present the thresholding function for BAM outputs.

     1  if  yj > 0   1   if  xi > 0

bj|t+1 =  bj|t    if  yj = 0   and   ai|t+1 =   ai|t  if  xi = 0

    0   if  yj < 0                                     0   if  xi < 0

where xi and yj are the activations of neurons i and j in the input layer and output layer, respectively, and bj|t refers to the output of the jth neuron in the output layer in the cycle t, while ai|t refers to the output of the ith neuron in the input layer in the cycle t. Note that at the start, the ai and bj values are the same as the corresponding components in the exemplar pair being used.

If X1 = (1, 0, 0, 1) is presented to the input neurons, their activations are given by the vector (–4, 4, 0). The output vector, after using the threshold function just described is (0, 1, 1). The last component here is supposed to be the same as the output in the previous cycle, since the corresponding activation value is 0. Since X1 and Y1 are one exemplar pair, the third component of Y1 is what we need as the third component of the output in the current cycle of operation; therefore, we fed X1 and received Y1. If we feed Y1 at the other end (B field) , the activations in the A field will be (2, –2, –2, 2), and the output vector will be (1, 0, 0, 1), which is X1.

With A field input X2 you get B field activations (4, -4, 0), giving the output vector as (1, 0, 1), which is Y2. Thus X2 and Y2 are heteroassociated with each other.

Let us modify our X1 to be (1, 0, 1, 1). Then the weight matrix W becomes

        1  [-1  1  1]       -1  [1  -1  1]       -2    2    0

 W =   -1              + 1               =    2   -2    0

        1                    1                    0    0    2

        1                   -1                   -2    2    0

and

                  -2     2     0    -2

WT =    2    -2     0     2

                   0     0     2     0

Now this is a different set of two exemplar vector pairs. The pairs are X1 = (1, 0, 1, 1), 1 = (0, 1, 1), and X2 = (0, 1, 1, 0), Y2 = (1, 0, 1). Naturally, the weight matrix is different, as is its transpose, correspondingly. As stated before, the weights do not change during the operation of the network with whatever inputs presented to it. The results are as shown in Table 8.1 below.

Table 8.1 Results for the Example
Input vector | activation | output vector
X1 = (1, 0, 1, 1) | (-4, 4, 2) | (0, 1, 1) = Y1
X2 = (0, 1, 1, 0) | (2, -2, 2) | (1, 0, 1) = Y2
Y1 = (0, 1, 1) | (2, -2, 2, 2) | (1, 0, 1, 1) = X1
Y2 = (1, 0, 1) | (-2, 2, 2, -2) | (0, 1, 1,0) = X2

You may think that you will encounter a problem when you input a new vector and one of the neurons has activation 0. In the original example, you did find this situation when you got the third output neuron's activation as 0. The thresholding function asked you to use the same output for this neuron as existed in the earlier time cycle. So you took it to be 1, the third component in (0, 1, 1). But if your input vector is a new X vector for which you are trying to find an associated Y vector, then you do not have a Y component to fall back on when the activation turns out to be 0. How then can you use the thresholding function as stated? What guidance do you have in this situation? If you keep track of the inputs used and outputs received thus far, you realize that the Field B (where you get your Y vector) neurons are in some state, meaning that they had some outputs perhaps with some training vector. If you use that output component as the one existing in the previous cycle, you have no problem in using the thresholding function.

As an example, consider the input vector X3 = ( 1, 0, 0, 0), with which the activations of neurons in Field B would be (-2, 2, 0). The first component of the output vector is clearly 0, and the second clearly 1. The third component is what is in doubt. Considering the last row of the table where Y2 gives the state of the neurons in Field B, you can accept 1, the last component of Y2, as the value you get from the thresholding function corresponding to the activation value 0. So the output would be the vector (0, 1, 1), which is Y1. But Y1 is heteroassociated with X1. Well, it means that X3 = ( 1, 0, 0, 0) is not heteroassociated with any X vector.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Recall of Vectors

When X1 is presented at the input layer, the activation at the output layer will give ( –4, 4, 2) to which we apply the thresholding function, which replaces a positive value by 1, and a negative value by 0.

This then gives us the vector (0, 1, 1) as the output, which is the same as our Y1. Now Y1 is passed back to the input layer through the feedback connections, and the activation of the input layer becomes the vector (2, –2, 2, 2), which after thresholding gives the output vector (1, 0, 1, 1), same as X1. When X2 is presented at the input layer, the activation at the output layer will give (2, –2, 2) to which the thresholding function, which replaces a positive value by 1 and a negative value by 0, is applied. This then gives the vector (1, 0, 1) as the output, which is the same as Y2. Now Y2 is passed back to the input layer through the feedback connections to get the activation of the input layer as (–2, 2, 2, –2), which after thresholding gives the output vector (0, 1, 1, 0), which is X2.

The two vector pairs chosen here for encoding worked out fine, and the BAM network with four neurons in Field A and three neurons in Field B is all set for finding a vector under heteroassociation with a given input vector.

Continuation of Example

Let us now use the vector X3 = (1, 1, 1, 1). The vector Y3 = (0, 1, 1) is obtained at the output layer. But the next step in which we present Y3 in the backward direction does not produce X3, instead it gives an X1 = (1, 0, 1, 1). We already have X1 associated with Y1. This means that X3 is not associated with any vector in the output space. On the other hand, if instead of getting X1 we obtained a different X4 vector, and if this in the feed forward operation produced a different Y vector, then we repeat the operation of the network until no changes occur at either end. Then we will have possibly a new pair of vectors under the heteroassociation established by this BAM network.

Special Case—Complements

If a pair of (distinct) patterns X and Y are found to be heteroassociated by BAM, and if you input the complement of X, complement being obtained by interchanging the 0's and 1's in X, BAM will show that the complement of Y is the pattern associated with the complement of X. An example will be seen in the illustrative run of the program for C++ implementation of BAM, which follows.

C++ Implementation

In our C++ implementation of a discrete bidirectional associative memory network, we create classes for neuron and network. Other classes created are called exemplar, assocpair, potlpair, for the exemplar pair of vectors, associated pair of vectors, and potential pairs of vectors, respectively, for finding heteroassociation between them. We could have made one class of pairvect for a pair of vectors and derived the exemplar and so on from it. The network class is declared as a friend class in these other classes. Now we present the header and source files, called bamntwrk.h and bamntwrk.cpp. Since we reused our previous code from the Hopfield network of Chapter 4, there are a few data members of classes that we did not put to explicit use in the program. We call the neuron class bmneuron to remind us of BAM.

Program Details and Flow

A neuron in the first layer is referred to as anrn, and the number of neurons in this layer is referred to as anmbr. We give the namebnrn to the array of neurons in the second layer, and bnmbr denotes the size of that array. The sequence of operations in the program is as follows:

• We ask the user to input the exemplar vectors, and we transform them into their bipolar versions. The trnsfrm ( ) function in the exemplar class is for this purpose.
• We give the network the X vector, in its bipolar version, in one exemplar pair. We find the activations of the elements of bnrn array and get corresponding output vector as a binary pattern. If this is the Y in the exemplar pair, the network has made a desired association in one direction, and we go on to the next.step. Otherwise we have a potential associated pair, one of which is X and the other is what we just got as the output vector in the opposite layer. We say potential associated pair because we have the next step to confirm the association.
• We run the bnrn array through the transpose of the weight matrix and calculate the outputs of the anrn array elements. If , as a result, we get the vector X as the anrn array, we found an associated pair, (X, Y). Otherwise, we repeat the two steps just described until we find an associated pair.
• We now work with the next pair of exemplar vectors in the same manner as above, to find an associated pair.
• We assign serial numbers, denoted by the variable idn, to the associated pairs so we can print them all together at the end of the program. The pair is called (X, Y) where X produces Y through the weight matrix W, and Y produces X through the weight matrix which is the transpose of W.
• A flag is used to have value 0 until confirmation of association is obtained, when the value of the flag changes to 1.
• Functions compr1 and compr2 in the network class verify if the potential pair is indeed an associated pair and set the proper value of the flag mentioned above.
• Functions comput1 and comput2 in the network class carry out the calculations to get the activations and then find the output vector, in the proper directions of the bidirectional associative memory network.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Program Example for BAM

For our illustration run, we provided for six neurons in the input layer and five in the output layer. We used three pairs of exemplars for encoding. We used two additional input vectors, one of which is the complement of the X of an exemplar pair, after the encoding is done, to see what association will be established in these cases, or what recall will be made by the BAM network.

Header File

As expected, the complement of the Y of the exemplar is found to be associated with the complement of the X of that pair. When the second input vector is presented, however, a new pair of associated vectors is found. After the code is presented, we list the computer output also.

Listing 8.1 bamntwrk.h

//bamntwrk.h   V. Rao,  H. Rao

//Header file for BAM network program

#include <iostream.h>

#include <math.h>

#include <stdlib.h>

#define MXSIZ 10  // determines the maximum size of the network

class bmneuron

{

 protected:

       int nnbr;

       int inn,outn;

       int output;

       int activation;

       int outwt[MXSIZ];

       char *name;

       friend class network;

public:

    bmneuron() { };

    void getnrn(int,int,int,char *);

};

class exemplar

{

protected:

       int xdim,ydim;

       int v1[MXSIZ],v2[MXSIZ];

       int u1[MXSIZ],u2[MXSIZ];

       friend class network;

       friend class mtrx;

public:

       exemplar() { };

       void getexmplr(int,int,int *,int *);

       void prexmplr();

       void trnsfrm();

       void prtrnsfrm();

};

class asscpair

{

protected:

       int xdim,ydim,idn;

       int v1[MXSIZ],v2[MXSIZ];

       friend class network;

public:

       asscpair() { };

       void getasscpair(int,int,int);

       void prasscpair();

};

class potlpair

{

protected:

       int xdim,ydim;

       int v1[MXSIZ],v2[MXSIZ];

       friend class network;

public:

       potlpair() { };

       void getpotlpair(int,int);

       void prpotlpair();

};

class network

{

public:

       int  anmbr,bnmbr,flag,nexmplr,nasspr,ninpt;

       bmneuron (anrn)[MXSIZ],(bnrn)[MXSIZ];

       exemplar (e)[MXSIZ];

       asscpair (as)[MXSIZ];

       potlpair (pp)[MXSIZ];

       int outs1[MXSIZ],outs2[MXSIZ];

       int mtrx1[MXSIZ][MXSIZ],mtrx2[MXSIZ][MXSIZ];

       network() { };

       void getnwk(int,int,int,int [][6],int [][5]);

       void compr1(int,int);

       void compr2(int,int);

       void prwts();

       void iterate();

       void findassc(int *);

       void asgninpt(int *);

       void asgnvect(int,int *,int *);

       void comput1();

       void comput2();

       void prstatus();

};

Source File

The program source file is presented as follows.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Listing 8.2 bamntwrk.cpp

//bamntwrk.cpp   V. Rao, H. Rao

//Source file for BAM network program

#include "bamntwrk.h"

void bmneuron::getnrn(int m1,int m2,int m3,char *y)

{

int i;

name = y;

nnbr = m1;

outn = m2;

inn  = m3;

for(i=0;i<outn;++i){

outwt[i] = 0 ;

}

output = 0;

activation = 0;

}

void exemplar::getexmplr(int k,int l,int *b1,int *b2)

{

int i2;

xdim = k;

ydim = l;

for(i2=0;i2<xdim;++i2){

v1[i2] = b1[i2]; }

for(i2=0;i2<ydim;++i2){

v2[i2] = b2[i2]; }

}

void exemplar::prexmplr()

{

int i;

cout<<"\nX vector you gave is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector you gave is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void exemplar::trnsfrm()

{

int i;

for(i=0;i<xdim;++i){

       u1[i] = 2*v1[i] -1;}

for(i=0;i<ydim;++i){

       u2[i] = 2*v2[i] - 1;}

}

void exemplar::prtrnsfrm()

{

int i;

cout<<"\nbipolar version of X vector you gave is:\n";

for(i=0;i<xdim;++i){

       cout<<u1[i]<<"  ";}

cout<<"\nbipolar version of Y vector you gave is:\n";

for(i=0;i<ydim;++i){

       cout<<u2[i]<<"  ";}

cout<<"\n";

}

void asscpair::getasscpair(int i,int j,int k)

{

idn = i;

xdim = j;

ydim = k;

}

void asscpair::prasscpair()

{

int i;

cout<<"\nX vector in the associated pair no. "<<idn<<"   is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector in the associated pair no. "<<idn<<"   is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void potlpair::getpotlpair(int k,int j)

{

xdim = k;

ydim = j;

}

void potlpair::prpotlpair()

{

int i;

cout<<"\nX vector in possible associated pair is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector in possible associated pair is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void network::getnwk(int k,int l,int k1,int b1[][6],int

       b2[][5])

{

anmbr = k;

bnmbr = l;

nexmplr = k1;

nasspr = 0;

ninpt = 0;

int i,j,i2;

flag =0;

char *y1="ANEURON", *y2="BNEURON" ;

for(i=0;i<nexmplr;++i){

       e[i].getexmplr(anmbr,bnmbr,b1[i],b2[i]);

       e[i].prexmplr();

       e[i].trnsfrm();

       e[i].prtrnsfrm();

       }

for(i=0;i<anmbr;++i){

       anrn[i].bmneuron::getnrn(i,bnmbr,0,y1);}

for(i=0;i<bnmbr;++i){

       bnrn[i].bmneuron::getnrn(i,0,anmbr,y2);}

for(i=0;i<anmbr;++i){

       for(j=0;j<bnmbr;++j){

              mtrx1[i][j]  = 0;

       for(i2=0;i2<nexmplr;++i2){

              mtrx1[i][j]  += e[i2].u1[i]*e[i2].u2[j];}

       mtrx2[j][i] = mtrx1[i][j];

       anrn[i].outwt[j] = mtrx1[i][j];

       bnrn[j].outwt[i] = mtrx2[j][i];

    }

}

prwts();

cout<<"\n";

}

void network::asgninpt(int *b)

{

int i;

cout<<"\n";

for(i=0;i<anmbr;++i){

       anrn[i].output = b[i];

       outs1[i] = b[i];

       }

}

void network::compr1(int j,int k)

{

int i;

for(i=0;i<anmbr;++i){

       if(pp[j].v1[i] != pp[k].v1[i]) flag = 1;

       break;

       }

}

void network::compr2(int j,int k)

{

int i;

for(i=0;i<anmbr;++i){

       if(pp[j].v2[i] != pp[k].v2[i]) flag = 1;

       break;}

}

void network::comput1()

{

int j;

for(j=0;j<bnmbr;++j){

       int ii1;

       int c1 =0,d1;

       cout<<"\n";

       for(ii1=0;ii1<anmbr;++ii1){

              d1 = outs1[ii1] * mtrx1[ii1][j];

              c1 += d1;

              }

       bnrn[j].activation = c1;

       cout<<"\n output layer neuron         "<<j<<" activation is"

              <<c1<<"\n";

if(bnrn[j].activation <0) {

       bnrn[j].output = 0;

       outs2[j] = 0;}

else

       if(bnrn[j].activation>0) {

               bnrn[j].output = 1;

               outs2[j] = 1;}

               else

               {cout<<"\n A 0 is obtained, use previous output

value \n";

               if(ninpt<=nexmplr){

                         bnrn[j].output = e[ninpt-1].v2[j];}

               else

                         { bnrn[j].output = pp[0].v2[j];}

                         outs2[j] = bnrn[j].output; }

       cout<<"\n output layer neuron         "<<j<<" output is"

               <<bnrn[j].output<<"\n";

       }

}

void network::comput2()

{

int i;

for(i=0;i<anmbr;++i){

       int ii1;

       int c1=0;

       for(ii1=0;ii1<bnmbr;++ii1){

              c1 += outs2[ii1] * mtrx2[ii1][i];  }

       anrn[i].activation = c1;

       cout<<"\ninput layer neuron        "<<i<<"activation is "

              <<c1<<"\n";

       if(anrn[i].activation <0 ){

              anrn[i].output = 0;

              outs1[i] = 0;}

       else

              if(anrn[i].activation >0 ) {

                     anrn[i].output = 1;

                     outs1[i] = 1;

                     }

              else

              { cout<<"\n A 0 is obtained, use previous value if available\n";

              if(ninpt<=nexmplr){

                        anrn[i].output = e[ninpt-1].v1[i];}

              else

                        {anrn[i].output = pp[0].v1[i];}

              outs1[i] = anrn[i].output;}

              cout<<"\n input layer neuron

              "<<i<<" output is "

                  <<anrn[i].output<<"\n";

              }

}

void network::asgnvect(int j1,int *b1,int *b2)

{

int  j2;

for(j2=0;j2<j1;++j2){

       b2[j2] = b1[j2];}

}

void network::prwts()

{

int i3,i4;

cout<<"\n  weights—  input layer to output layer: \n\n";

for(i3=0;i3<anmbr;++i3){

        for(i4=0;i4<bnmbr;++i4){

                cout<<anrn[i3].outwt[i4]<<"                  ";}

        cout<<"\n"; }

cout<<"\n";

cout<<"\nweights—  output layer to input layer: \n\n";

for(i3=0;i3<bnmbr;++i3){

        for(i4=0;i4<anmbr;++i4){

                cout<<bnrn[i3].outwt[i4]<<"                  ";}

        cout<<"\n";  }

cout<<"\n";

}

void network::iterate()

{

int i1;

for(i1=0;i1<nexmplr;++i1){

        findassc(e[i1].v1);

        }

}

void network::findassc(int *b)

{

int j;

flag = 0;

        asgninpt(b);

ninpt ++;

cout<<"\nInput vector is:\n" ;

for(j=0;j<6;++j){

       cout<<b[j]<<" ";}

cout<<"\n";

pp[0].getpotlpair(anmbr,bnmbr);

asgnvect(anmbr,outs1,pp[0].v1);

comput1();

if(flag>=0){

           asgnvect(bnmbr,outs2,pp[0].v2);

           cout<<"\n";

           pp[0].prpotlpair();

           cout<<"\n";

           comput2(); }

for(j=1;j<MXSIZ;++j){

       pp[j].getpotlpair(anmbr,bnmbr);

       asgnvect(anmbr,outs1,pp[j].v1);

       comput1();

       asgnvect(bnmbr,outs2,pp[j].v2);

       pp[j].prpotlpair();

       cout<<"\n";

       compr1(j,j-1);

       compr2(j,j-1);

       if(flag == 0) {

               int j2;

               nasspr += 1;

               j2 = nasspr;

               as[j2].getasscpair(j2,anmbr,bnmbr);

               asgnvect(anmbr,pp[j].v1,as[j2].v1);

               asgnvect(bnmbr,pp[j].v2,as[j2].v2);

               cout<<"\nPATTERNS ASSOCIATED:\n";

               as[j2].prasscpair();

               j = MXSIZ ;

      }

      else

               if(flag == 1)

                       {

                       flag = 0;

                       comput1();

                       }

     }

}

void network::prstatus()

{

int j;

cout<<"\nTHE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY BAM\n\n";

for(j=1;j<=nasspr;++j){

       as[j].prasscpair();

       cout<<"\n";}

}

void main()

{

int ar = 6, br = 5, nex = 3;

int inptv[][6]={1,0,1,0,1,0,1,1,1,0,0,0,0,1,1,0,0,0,0,1,0,1,0,1,

   1,1,1,1,1,1};

int outv[][5]={1,1,0,0,1,0,1,0,1,1,1,0,0,1,0};

cout<<"\n\nTHIS PROGRAM IS FOR A BIDIRECTIONAL ASSOCIATIVE MEMORY NETWORK.\n";

cout<<" THE NETWORK ISSET UP FOR ILLUSTRATION WITH "<<ar<<

         " INPUT NEURONS, AND "<<br;

cout<<" OUTPUT NEURONS.\n"<<nex

         <<" exemplars are used to encode \n";

static network bamn;

bamn.getnwk(ar,br,nex,inptv,outv) ;

bamn.iterate();

bamn.findassc(inptv[3]);

bamn.findassc(inptv[4]);

bamn.prstatus();

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Program Output

The output from an illustrative run of the program is listed next. You will notice that we provided for a lot of information to be output as the program is executed. The fourth vector we input is not from an exemplar, but it is the complement of the X of the first exemplar pair. The network found the complement of the Y of this exemplar to be associated with this input. The fifth vector we input is (1, 1, 1, 1, 1, 1). But BAM recalled in this case the complement pair for the third exemplar pair, which is X = (1, 0, 0, 1, 1, 1) and Y = (0, 1, 1, 0, 1). You notice that the Hamming distance of this input pattern from the X's of the three exemplars are 3, 3, and 4, respectively. The hamming distance from the complement of the X of the first exemplar pair is also 3. But the Hamming distance of (1, 1, 1, 1) from the X of the complement of the X of the third exemplar pair is only 2. It would be instructive if you would use the input pattern (1, 1, 1, 1, 1, 0) to see to what associated pair it leads. Now the output:

THIS PROGRAM IS FOR A BIDIRECTIONAL ASSOCIATIVE

    MEMORY NETWORK.

THE NETWORK IS SET UP FOR ILLUSTRATION WITH SIX INPUT

    NEURONS AND FIVE OUTPUT NEURONS.

Three exemplars are used to encode

X vector you gave is:

1  0  1  0  1  0

Y vector you gave is:

1  1  0  0  1

bipolar version of X vector you gave is:

1  -1  1  -1  1  -1

bipolar version of Y vector you gave is:

1  1  -1  -1  1

X vector you gave is:

1  1  1  0  0  0

Y vector you gave is:

0  1  0  1  1

bipolar version of X vector you gave is:

1  1  1  -1  -1  -1

bipolar version of Y vector you gave is:

-1  1  -1  1  1

X vector you gave is:

0  1  1  0  0  0

Y vector you gave is:

1  0  0  1  0

bipolar version of X vector you gave is:

-1  1  1  -1  -1  -1

bipolar version of Y vector you gave is:

1  -1  -1  1  -1

  weights—  input layer to output layer:

           -1        3      -1     -1     3

           -1        1      -1      3    -1

            1        1      -3      1     1

           -1       -1       3     -1    -1

            1        1       1     -3     1

           -1       -1       3     -1    -1

weights—  output layer to input layer:

           -1       -1      1      -1       1    -1

            3       -1      1      -1       1    -1

           -1       -1     -3       3       1     3

           -1        3      1      -1      -3    -1

            3       -1      1      -1       1    -1

Input vector is:

1 0 1 0 1 0

 output layer neuron  0 activation is 1

 output layer neuron  0 output is 1

 output layer neuron  1 activation is 5

 output layer neuron  1 output is 1

 output layer neuron  2 activation is -3

 output layer neuron  2 output is 0

 output layer neuron  3 activation is -3

 output layer neuron  3 output is 0

 output layer neuron  4 activation is 5

 output layer neuron  4 output is 1

 X vector in possible associated pair is:

 1  0  1  0  1  0

 Y vector in possible associated pair is:

 1  1  0  0  1

input layer neuron 0 activation is 5

 input layer neuron  0 output is 1

input layer neuron 1 activation is -3

 input layer neuron  1 output is 0

input layer neuron 2 activation is 3

 input layer neuron  2 output is 1

input layer neuron 3 activation is -3

 input layer neuron  3 output is 0

input layer neuron 4 activation is 3

 input layer neuron  4 output is 1

input layer neuron 5 activation is -3

 input layer neuron  5 output is 0

 output layer neuron  0 activation is 1

 output layer neuron  0 output is 1

 output layer neuron  1 activation is 5

 output layer neuron  1 output is 1

 output layer neuron  2 activation is -3

 output layer neuron  2 output is 0

 output layer neuron  3 activation is -3

 output layer neuron  3 output is 0

 output layer neuron  4 activation is 5

 output layer neuron  4 output is 1

X vector in possible associated pair is:

1  0  1  0  1  0

Y vector in possible associated pair is:

1  1  0  0  1

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

1  0  1  0  1  0

Y vector in the associated pair no. 1 is:

1  1  0  0  1

Input vector is:

1 1 1 0 0 0

// We get here more of the detailed output as in the previous case. We will simply not present it here.

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

1  1  1  0  0  0

Y vector in the associated pair no. 1 is:

0  1  0  1  1

Input vector is:

0 1 1 0 0 0

output layer neuron  0 activation is 0

A 0 is obtained, use previous output value

output layer neuron  0 output is 1

output layer neuron  1 activation is 0

A 0 is obtained, use previous output value

output layer neuron  1 output is 0

output layer neuron  2 activation is -4

output layer neuron  2 output is 0

output layer neuron  3 activation is 4

output layer neuron  3 output is 1

output layer neuron  4 activation is 0

A 0 is obtained, use previous output value

output layer neuron  4 output is 0

X vector in possible associated pair is:

0  1  1  0  0  0

Y vector in possible associated pair is:

1  0  0  1  0

// We get here more of the detailed output as in the previous case. We will simply not present it here.

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

0  1  1  0  0  0

Y vector in the associated pair no. 1 is:

1  0  0  1  0

Input vector is:

0 1 0 1 0 1

// We get here more of the detailed output as in the previous case. We will simply not present it here.

X vector in possible associated pair is:

0  1  0  1  0  1

Y vector in possible associated pair is:

0  0  1  1  0

// We get here more of the detailed output as in the previous case. We will simply not present it here.

X vector in possible associated pair is:

0  1  0  1  0  1

Y vector in possible associated pair is:

0  0  1  1  0

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

0  1  0  1  0  1

Y vector in the associated pair no. 1 is:

0  0  1  1  0

Input vector is:

1 1 1 1 1 1

output layer neuron  0 activation is -2

output layer neuron  0 output is 0

output layer neuron  1 activation is 2

output layer neuron  1 output is 1

output layer neuron  2 activation is 2

output layer neuron  2 output is 1

output layer neuron  3 activation is -2

output layer neuron  3 output is 0

output layer neuron  4 activation is 2

output layer neuron  4 output is 1

X vector in possible associated pair is:

1  1  1  1  1  1

Y vector in possible associated pair is:

0  1  1  0  1

input layer neuron 0 activation is 5

input layer neuron  0 output is 1

input layer neuron 1 activation is -3

input layer neuron  1 output is 0

input layer neuron 2 activation is -1

input layer neuron  2 output is 0

input layer neuron 3 activation is 1

input layer neuron  3 output is 1

input layer neuron 4 activation is 3

input layer neuron  4 output is 1

input layer neuron 5 activation is 1

input layer neuron  5 output is 1

output layer neuron  0 activation is -2

output layer neuron  0 output is 0

output layer neuron  1 activation is 2

output layer neuron  1 output is 1

output layer neuron  2 activation is 6

output layer neuron  2 output is 1

output layer neuron  3 activation is -6

output layer neuron  3 output is 0

output layer neuron  4 activation is 2

output layer neuron  4 output is 1

X vector in possible associated pair is:

1  0  0  1  1  1

Y vector in possible associated pair is:

0  1  1  0  1

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

1  0  0  1  1  1

Y vector in the associated pair no. 1 is:

0  1  1  0  1

THE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY BAM

X vector in the associated pair no. 1 is:   //first exemplar pair

1  0  1  0  1  0

Y vector in the associated pair no. 1 is:

1  1  0  0  1

X vector in the associated pair no. 2 is:   //second exemplar pair

1  1  1  0  0  0

Y vector in the associated pair no. 2 is:

0  1  0  1  1

X vector in the associated pair no. 3 is:   //third exemplar pair

0  1  1  0  0  0

Y vector in the associated pair no. 3 is:

1  0  0  1  0

X vector in the associated pair no. 4 is:   //complement of X of the

0  1  0  1  0  1               first exemplar pair

Y vector in the associated pair no. 4 is:    //complement of Y of the

0  0  1  1  0                    first exemplar pair

X vector in the associated pair no. 5 is:  //input was X = (1, 1, 1,

1  0  0  1  1  1                            1, 1) but result was complement

                                             of third exemplar pair

Y vector in the associated pair no. 5 is:  with X of which Hamming

0  1  1  0  1                 distance is the  least.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Additional Issues

If you desire to use vectors with real number components, instead of binary numbers, you can do so. Your model is then called a Continuous Bidirectional Associative Memory. A matrix W and its transpose are used for the connection weights. However, the matrix W is not formulated as we described so far. The matrix is arbitrarily chosen, and kept constant. The thresholding function is also chosen as a continuous function, unlike the one used before. The changes in activations of neurons during training are according to extensions the Cohen-Grossberg paradigm.

Michael A. Cohen and Stephen Grossberg developed their model for autoassociation, with a symmetric matrix of weights. Stability is ensured by the Cohen-Grossberg theorem; there is no learning.

If, however, you desire the network to learn new pattern pairs, by modifying the weights so as to find association between new pairs, you are designing what is called an Adaptive Bidirectional Associative Memory (ABAM).

The law that governs the weight changes determines the type of ABAM you have, namely, the Competitive ABAM, or the Differential Hebbian ABAM, or the Differential Competitive ABAM. Unlike in the ABAM model, which is additive type, some products of outputs from the two layers, or the derivatives of the threshold functions are used in the other models.

Here we present a brief description of a model, which is a variation of the BAM. It is called UBBAM (Unipolar Binary Bidirectional Associative Memory).

Unipolar Binary Bidirectional Associative Memory

T. C. B. Yu and R. J. Mears describe a design of unipolar binary bidirectional associative memory, and its implementation with a Smart Advanced Spatial Light Modulator (SASLM). The SASLM device is a ferroelectric liquid crystal spatial light modulator. It is driven by a silicon CMOS backplane. We use their paper to present some features of a unipolar binary bidirectional associative memory, and ignore the hardware implementation of it.

Recall the procedure by which you determine the weight matrix W for a BAM network, as described in the previous pages. As a first step, you convert each vector in the exemplar pairs into its bipolar version. If X and Y are an exemplar pair (in bipolar versions), you take the product XT Y and add it to similar products from other exemplar pairs, to get a weight matrix W. Some of the elements of the matrix W may be negative numbers. In the unipolar context you do not have negative values and positive values at the same time. Only one of them is allowed. Suppose you do not want any negative numbers; then one way of remedying the situation is by adding a large enough constant to each element of the matrix. You cannot choose to add to only the negative numbers that show up in the matrix. Let us look at an example.

Suppose you choose two pairs of vectors as possible exemplars. Let them be,

X1 = (1, 0, 0, 1), Y1= (0, 1, 1)

and

X2 = (0, 1, 1, 0), Y2 = (1, 0, 1)

These you change into bipolar components and get, respectively, (1, –1, –1, 1), (–1, 1, 1), (–1, 1, 1, –1), and (1, –1, 1). The calculation of W for the BAM was done as follows.

     1 [-1 1 1]   -1 [1 -1 1]  -1  1  1    -1  1 -1   -2  2 0

W = -1       + 1         =  1 -1 -1+1 -1  1 =  2 -2 0

    -1             1            1 -1 -1     1 -1  1    2 -2 0

     1            -1           -1  1  1    -1  1 -1   -2  2 0

and

                  -2    2    2   -2

WT =    2   -2   -2    2

                   0    0    0    0

You see some negative numbers in the matrix W. If you add the constant m, m = 2, to all the elements in the matrix, you get the following matrix, denoted by W~.

                   0    4    2

W~ =    4    0    2

                   4    0    2

                   0    4    2

You need a modification to the thresholding function as well. Earlier you had the following function.

      1  if yj > 0   1   if  xi > 0

      bj|t+1 = bj|t    if  yj = 0   and      ai|t+1 = ai|t     if  xi = 0

      0  if  yj < 0  0   if  xi < 0

Now you need to change the right-hand sides of the conditions from 0 to the product of m and sum of the inputs of the neurons, that is to m times the sum of the inputs. For brevity, let us use Si for the sum of the inputs. This is not a constant, and its value depends on what values you get for neuron inputs in any one direction. Then the thresholding function can be given as follows:

      1   if yj > m Si         1  if  xi > m Si

bj|t+1 = bj|t      if  yj = m Si       and      ai|t+1 = ai|t     if  xi = m Si

       0  if  yj < m Si                         0  if  xi < m Si

For example, the vector X1 = (1, 0, 0, 1) has the activity vector (0, 8, 4) and the corresponding output vector is (0, 1, 1), which is, Y1 = (0, 1, 1). The value of Si is 2 = 1 + 0 + 0 + 1. Therefore, mSi = 4. The first component of the activation vector is 0, which is smaller than 4, and so the first component of the output vector is 0. The second component of the activation vector is 8, which is larger than 4, and so the second component of the output vector is 1. The last component of the activity vector is 4, which is equal to mSi, and so you use the third component of the vector Y1 = (0, 1, 1), namely 1, as the third component of the output.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The example of Yu and Mears, consists of eight 8-component vectors in both directions of the Bidirectional Associative Memory model. You can take these vectors, as they do, to define the pixel values in a 8x8 grid of pixels. What is being accomplished is the association of two spatial patterns. In terms of the binary numbers that show the pixel values, the patterns are shown in Figure 8.2. Call them Pattern I and Pattern II.

Figure 8.2 Two patterns, Pattern I and Pattern II, given by pixel values: 1 for black, 0 for white.

Instead of Pattern I, they used a corrupted form of it as given in Figure 8.3. There was no problem in the network finding the associated pair (Pattern I, Pattern II).

Figure 8.3 Pattern I and corrupted Pattern I.

In Figure 8.3, the corrupted version of Pattern I differs from Pattern I, in 10 of the 64 places, a corruption of 15.6%. This corruption percentage is cited as the limit below which, Yu and Mears state, heteroassociative recall is obtained. Thus noise elimination to a certain extent is possible with this model. As we have seen with the Hopfield memory, an application of associative memory is pattern completion, where you are presented a corrupted version of a pattern and you recall the true pattern. This is autoassociation. In the case of BAM, you have heteroassociative re call with a corrupted input.

Summary

In this chapter, bidirectional associative memories are presented. The development of these memories is largely due to Kosko. They share with Adaptive Resonance Theory the feature of resonance between the two layers in the network. The bidirectional associative memories (BAM) network finds heteroassociation between binary patterns, and these are converted to bipolar values to determine the connection weight matrix. Even though there are connections in both directions between neurons in the two layers, essentially only one weight matrix is involved. You use the transpose of this weight matrix for the connections in the opposite direction. When one input at one end leads to some output at the other end, which in turn leads to output that is the same as the previous input, resonance is reached and an associated pair is found.

The continuous bidirectional associative memory extends the binary model to the continuous case. Adaptive bidirectional memories of different flavors are the result of incorporating different learning paradigms. A unipolar binary version of BAM is also presented as an application of the BAM for pattern completion.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 9
FAM: Fuzzy Associative Memory

Introduction

In this chapter, you we will learn about fuzzy sets and their elements, both for input and output of an associative neural network. Every element of a fuzzy set has a degree of membership in the set. Unless this degree of membership is 1, an element does not belong to the set (in the sense of elements of an ordinary set belonging to the set). In a neural network of a fuzzy system the inputs, the outputs, and the connection weights all belong fuzzily to the spaces that define them. The weight matrix will be a fuzzy matrix, and the activations of the neurons in such a network have to be determined by rules of fuzzy logic and fuzzy set operations.

An expert system uses what are called crisp rules and applies them sequentially. The advantage in casting the same problem in a fuzzy system is that the rules you work with do not have to be crisp, and the processing is done in parallel. What the fuzzy systems can determine is a fuzzy association. These associations can be modified, and the underlying phenomena better understood, as experience is gained. That is one of the reasons for their growing popularity in applications. When we try to relate two things through a process of trial and error, we will be implicitly and intuitively establishing an association that is gradually modified and perhaps bettered in some sense. Several fuzzy variables may be present in such an exercise. That we did not have full knowledge at the beginning is not a hindrance; there is some difference in using probabilities and using fuzzy logic as well. The degree of membership assigned for an element of a set does not have to be as firm as the assignment of a probability.

The degree of membership is, like a probability, a real number between 0 and 1. The closer it is to 1, the less ambiguous is the membership of the element in the set concerned. Suppose you have a set that may or may not contain three elements, say, a, b, and c. Then the fuzzy set representation of it would be by the ordered triple (ma, mb, mc), which is called the fit vector, and its components are called fit values. For example, the triple (0.5, 0.5, 0.5) shows that each of a, b, and c, have a membership equal to only one-half. This triple itself will describe the fuzzy set. It can also be thought of as a point in the three-dimensional space. None of such points will be outside the unit cube. When the number of elements is higher, the corresponding points will be on or inside the unit hypercube.

It is interesting to note that this fuzzy set, given by the triple (0.5, 0.5, 0.5), is its own complement, something that does not happen with regular sets. The complement is the set that shows the degrees of nonmembership.

The height of a fuzzy set is the maximum of its fit values, and the fuzzy set is said to be normal, if its height is 1. The fuzzy set with fit vector (0.3, 0.7, 0.4) has height 0.7, and it is not a normal fuzzy set. However, by introducing an additional dummy component with fit value 1, we can extend it into a normal fuzzy set. The desirability of normalcy of a fuzzy set will become apparent when we talk about recall in fuzzy associative memories.

The subset relationship is also different for fuzzy sets from the way it is defined for regular sets. For example, if you have a fuzzy set given by the triple (0.3, 0.7, 0.4), then any fuzzy set with a triple (a, b, c) such that a ≤ 0.3, b ≤ 0.7, and c ≤ 0.4, is its fuzzy subset. For example, the fuzzy set given by the triple (0.1, 0.7, 0) is a subset of the fuzzy set (0.3, 0.7, 0.4).

Association

Consider two fuzzy sets, one perhaps referring to the popularity of (or interest in) an exhibition and the other to the price of admission. The popularity could be very high, high, fair, low, or very low. Accordingly, the fit vector will have five components. The price of admission could be high, modest, or low. Its fit vector has three components. A fuzzy associative memory system then has the association (popularity, price), and the fuzzy set pair encodes this association.

We describe the encoding process and the recall in the following sections. Once encoding is completed, associations between subsets of the first fuzzy set and subsets of the second fuzzy set are also established by the same fuzzy associative memory system.

FAM Neural Network

The neural network for a fuzzy associative memory has an input and an output layer, with full connections in both directions, just as in a BAM neural network. Figure 9.1 shows the layout. To avoid cluttering, the figure shows the network with three neurons in field A and two neurons in field B, and only some of the connection weights. The general case is analogous.

Figure 9.1 Fuzzy associative memory neural network.

This figure is the same as Figure 8.1 in Chapter 8, since the model is the same, except that fuzzy vectors are used with the current model.

Encoding

Encoding for fuzzy associative memory systems is similar in concept to the encoding process for bidirectional associative memories, with some differences in the operations involved. In the BAM encoding, bipolar versions of binary vectors are used just for encoding. Matrices of the type XiT Yi are added together to get the connection weight matrix. There are two basic operations with the elements of these vectors. They are multiplication and addition of products. There is no conversion of the fit values before the encoding by the fuzzy sets. The multiplication of elements is replaced by the operation of taking the minimum, and addition is replaced by the operation of taking the maximum.

There are two methods for encoding. The method just described is what is called max–min composition. It is used to get the connection weight matrix and also to get the outputs of neurons in the fuzzy associative memory neural network. The second method is called correlation–product encoding. It is obtained the same way as a BAM connection weight matrix is obtained. Max–min composition is what is frequently used in practice, and we will confine our attention to this method.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Example of Encoding

Suppose the two fuzzy sets we use to encode have the fit vectors ( 0.3, 0.7, 0.4, 0.2) and (0.4, 0.3, 0.9). Then the matrix W is obtained by using max–min composition as follows.

   0.3 [0.4 0.3 0.9] min(0.3,0.4) min(0.3,0.3) min(0.3,0.9) 0.3 0.3 0.3

 W=0.7              =min(0.7,0.4) min(0.7,0.3) min(0.7,0.9)=0.4 0.3 0.7

   0.4               min(0.4,0.4) min(0.4,0.3) min(0.4,0.9) 0.4 0.3 0.4

   0.2               min(0.2,0.4) min(0.2,0.3) min(0.2,0.9) 0.2 0.2 0.2

Recall for the Example

If we input the fit vector (0.3, 0.7, 0.4, 0.2), the output (b1, b2, b3) is determined as follows, using bj = max( min(a1, w1j), …, min(am, wmj), where m is the dimension of the 'a' fit vector, and wij is the ith row, jth column element of the matrix W.

     b1 = max(min(0.3, 0.3), min(0.7, 0.4), min(0.4, 0.4),

          min(0.2, 0.2)) =  max(0.3, 0.4, 0.4, 0.2) = 0.4

     b2 = max(min(0.3, 0.3), min(0.7, 0.3), min(0.4, 0.3),

          min(0.2, 0.2)) = max( 0.3, 0.3, 0.3, 0.2 ) = 0.3

     b3 = max(min(0.3, 0.3), min(0.7, 0.7), min(0.4, 0.4),

          min(0.2, 0.2)) = max (0.3, 0.7, 0.4, 0.2) = 0.7

The output vector (0.4, 0.3, 0.7) is not the same as the second fit vector used, namely (0.4, 0.3, 0.9), but it is a subset of it, so the recall is not perfect. If you input the vector (0.4, 0.3, 0.7) in the opposite direction, using the transpose of the matrix W, the output is (0.3, 0.7, 0.4, 0.2), showing resonance. If on the other hand you input (0.4, 0.3, 0.9) at that end, the output vector is (0.3, 0.7, 0.4, 0.2), which in turn causes in the other direction an output of (0.4, 0.3, 0.7) at which time there is resonance. Can we foresee these results? The following section explains this further.

Recall

Let us use the operator o to denote max–min composition. Perfect recall occurs when the weight matrix is obtained using the max–min composition of fit vectors U and V as follows:

(i) U o W = V if and only if height (U) ≥ height (V).
(ii) V o WT = U if and only if height (V) ≥ height (U).

Also note that if X and Y are arbitrary fit vectors with the same dimensions as U and V, then:

(iii) X o W ∴ V.
(iv) Y o WT ∴ U.

A ∴ B is the notation to say A is a subset of B.

In the previous example, height of (0.3, 0.7, 0.4, 0.2) is 0.7, and height of (0.4, 0.3, 0.9) is 0.9. Therefore (0.4, 0.3, 0.9) as input, produced (0.3, 0.7, 0.4, 0.2) as output, but (0.3, 0.7, 0.4, 0.2) as input, produced only a subset of (0.4, 0.3, 0.9). That both (0.4, 0.3, 0.7) and (0.4, 0.3, 0.9) gave the same output, (0.3, 0.7, 0.4, 0.2) is in accordance with the corollary to the above, which states that if (X, Y) is a fuzzy associated memory, and if X is a subset of X', then (X', Y) is also a fuzzy associated memory.

C++ Implementation

We use the classes we created for BAM implementation in C++, except that we call the neuron class fzneuron, and we do not need some of the methods or functions in the network class. The header file, the source file, and the output from an illustrative run of the program are given in the following. The header file is called fuzzyam.hpp, and the source file is called fuzzyam.cpp.

Program details

The program details are analogous to the program details given in Chapter 8. The computations are done with fuzzy logic. Unlike in the nonfuzzy version, a single exemplar fuzzy vector pair is used here. There are no transformations to bipolar versions, since the vectors are fuzzy and not binary and crisp.

A neuron in the first layer is referred to as anrn, and the number of neurons in this layer is referred to as anmbr. bnrn is the name we give to the array of neurons in the second layer and bnmbr denotes the size of that array. The sequence of operations in the program are as follows:

• We ask the user to input the exemplar vector pair.
• We give the network the X vector, in the exemplar pair. We find the activations of the elements of bnrn array and get corresponding output vector as a binary pattern. If this is the Y in the exemplar pair, the network has made a desired association in one direction, and we go on to the next.step. Otherwise we have a potential associated pair, one of which is X and the other is what we just got as the output vector in the opposite layer. We say potential associated pair because we have the next step to confirm the association.
• We run the bnrn array through the transpose of the weight matrix and calculate the outputs of the anrn array elements. If, as a result, we get the vector X as the anrn array, we found an associated pair, (X, Y). Otherwise, we repeat the two steps just described until we find an associated pair.
• We now work with the next pair of exemplar vectors in the same manner as above, to find an associated pair.
• We assign serial numbers, denoted by the variable idn, to the associated pairs so we can print them all together at the end of the program. The pair is called (X, Y) where X produces Y through the weight matrix W, and Y produces X through the weight matrix, which is the transpose of W.
• A flag is used to have value 0 until confirmation of association is obtained, when the value of the flag changes to 1.
• Functions compr1 and compr2 in the network class verify if the potential pair is indeed an associated pair and set the proper value of the flag mentioned above.
• Functions comput1 and comput2 in the network class carry out the calculations to get the activations and then find the output vector, in the respective directions of the fuzzy associative memory network.

A lot of the code from the bidirectional associative memory (BAM) is used for the FAM. Here are the listings, with comments added where there are differences between this code and the code for the BAM of Chapter 8.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Header File

Listing 9.1 fuzzyam.h

//fuzzyam.h   V. Rao, H. Rao

#include <iostream.h>

#define MXSIZ 10

class fzneuron

{

protected:

       int nnbr;

       int inn,outn;

       float output;

       float activation;

       float outwt[MXSIZ];

       char *name;

       friend class network;

public:

       fzneuron() { };

       void getnrn(int,int,int,char *);

};

class exemplar

{

protected:

       int xdim,ydim;

       float v1[MXSIZ],v2[MXSIZ];   // this is different from BAM

       friend class network;

public:

       exemplar() { };

       void getexmplr(int,int,float *,float *);

       void prexmplr();

};

class asscpair

{

protected:

       int xdim,ydim,idn;

       float v1[MXSIZ],v2[MXSIZ];

       friend class network;

public:

       asscpair() { };

       void getasscpair(int,int,int);

       void prasscpair();

};

class potlpair

{

protected:

       int xdim,ydim;

       float v1[MXSIZ],v2[MXSIZ];

       friend class network;

public:

       potlpair() { };

       void getpotlpair(int,int);

       void prpotlpair();

};

class network

{

public:

       int  anmbr,bnmbr,flag,nexmplr,nasspr,ninpt;

       fzneuron (anrn)[MXSIZ],(bnrn)[MXSIZ];

       exemplar (e)[MXSIZ];

       asscpair (as)[MXSIZ];

       potlpair (pp)[MXSIZ];

       float outs1[MXSIZ],outs2[MXSIZ];   // change from BAM to floats

       double mtrx1[MXSIZ][MXSIZ],mtrx2[MXSIZ][MXSIZ]; // change from

BAM to doubles

       network() { };

       void getnwk(int,int,int,float [][6],float [][4]);

       void compr1(int,int);

       void compr2(int,int);

       void prwts();

       void iterate();

       void findassc(float *);

       void asgninpt(float *);

       void asgnvect(int,float *,float *);

       void comput1();

       void comput2();

       void prstatus();

};

Source File

Listing 9.2 fuzzyam.cpp

//fuzzyam.cpp   V. Rao, H. Rao

#include "fuzzyam.h"

float max(float x,float y)   //new for FAM

{

float u;

u = ((x>y) ? x : y );

return u;

}

float min(float x,float y)      // new for FAM

{

float u;

u =( (x>y) ? y : x) ;

return u;

}

void fzneuron::getnrn(int m1,int m2,int m3,char *y)

{

int i;

name = y;

nnbr = m1;

outn = m2;

inn  = m3;

for(i=0;i<outn;++i){

       outwt[i] = 0 ;

       }

output = 0;

activation = 0;

}

void exemplar::getexmplr(int k,int l,float *b1,float *b2)    // changed

from BAM

{

int i2;

xdim = k;

ydim = l;

for(i2=0;i2<xdim;++i2){

       v1[i2] = b1[i2]; }

for(i2=0;i2<ydim;++i2){

       v2[i2] = b2[i2]; }

}

void exemplar::prexmplr()

{

int i;

cout<<"\nX vector you gave is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector you gave is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void asscpair::getasscpair(int i,int j,int k)

{

idn = i;

xdim = j;

ydim = k;

}

void asscpair::prasscpair()

{

int i;

cout<<"\nX vector in the associated pair no. "<<idn<<"is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector in the associated pair no. "<<idn<<"is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void potlpair::getpotlpair(int k,int j)

{

xdim = k;

ydim = j;

}

void potlpair::prpotlpair()

{

int i;

cout<<"\nX vector in possible associated pair is:\n";

for(i=0;i<xdim;++i){

       cout<<v1[i]<<"  ";}

cout<<"\nY vector in possible associated pair is:\n";

for(i=0;i<ydim;++i){

       cout<<v2[i]<<"  ";}

cout<<"\n";

}

void network::getnwk(int k,int l,int k1,float b1[][6],float

       b2[][4])

{

anmbr = k;

bnmbr = l;

nexmplr = k1;

nasspr = 0;

ninpt = 0;

int i,j,i2;

float tmp1,tmp2;

flag =0;

char *y1="ANEURON", *y2="BNEURON" ;

for(i=0;i<nexmplr;++i){

       e[i].getexmplr(anmbr,bnmbr,b1[i],b2[i]);

       e[i].prexmplr();

       cout<<"\n";

       }

for(i=0;i<anmbr;++i){

       anrn[i].fzneuron::getnrn(i,bnmbr,0,y1);}

for(i=0;i<bnmbr;++i){

       bnrn[i].fzneuron::getnrn(i,0,anmbr,y2);}

for(i=0;i<anmbr;++i){

       for(j=0;j<bnmbr;++j){

              tmp1 = 0.0;

              for(i2=0;i2<nexmplr;++i2){

                      tmp2 = min(e[i2].v1[i],e[i2].v2[j]);

                      tmp1 = max(tmp1,tmp2);

              }

              mtrx1[i][j] = tmp1;

              mtrx2[j][i] = mtrx1[i][j];

              anrn[i].outwt[j] = mtrx1[i][j];

              bnrn[j].outwt[i] = mtrx2[j][i];

              }

       }

prwts();

cout<<"\n";

}

void network::asgninpt(float *b)

{

int i,j;

cout<<"\n";

for(i=0;i<anmbr;++i){

       anrn[i].output = b[i];

       outs1[i] = b[i];

       }

}

void network::compr1(int j,int k)

{

int i;

for(i=0;i<anmbr;++i){

       if(pp[j].v1[i] != pp[k].v1[i]) flag = 1;

       break;

       }

}

void network::compr2(int j,int k)

{

int i;

for(i=0;i<anmbr;++i){

       if(pp[j].v2[i] != pp[k].v2[i]) flag = 1;

       break;}

}

void network::comput1()   //changed from BAM

{

int j;

for(j=0;j<bnmbr;++j){

       int ii1;

       float c1 =0.0,d1;

       cout<<"\n";

       for(ii1=0;ii1<anmbr;++ii1){

              d1 = min(outs1[ii1],mtrx1[ii1][j]);

              c1 = max(c1,d1);

              }

       bnrn[j].activation = c1;

       cout<<"\n output layer neuron  "<<j<<" activation is "

              <<c1<<"\n";

       bnrn[j].output = bnrn[j].activation;

       outs2[j] = bnrn[j].output;

       cout<<"\n output layer neuron  "<<j<<" output is "

              <<bnrn[j].output<<"\n";

       }

}

void network::comput2()             //changed from BAM

{

int i;

for(i=0;i<anmbr;++i){

       int ii1;

       float c1=0.0,d1;

       for(ii1=0;ii1<bnmbr;++ii1){

              d1 = min(outs2[ii1],mtrx2[ii1][i]);

              c1 = max(c1,d1);}

       anrn[i].activation = c1;

       cout<<"\ninput layer neuron "<<i<<"activation is "

              <<c1<<"\n";

       anrn[i].output = anrn[i].activation;

       outs1[i] = anrn[i].output;

       cout<<"\n input layer neuron  "<<i<<"output is "

              <<anrn[i].output<<"\n";

       }

}

void network::asgnvect(int j1,float *b1,float *b2)

{

int  j2;

for(j2=0;j2<j1;++j2){

       b2[j2] = b1[j2];}

}

void network::prwts()

{

int i3,i4;

cout<<"\n  weights--  input layer to output layer: \n\n";

for(i3=0;i3<anmbr;++i3){

       for(i4=0;i4<bnmbr;++i4){

              cout<<anrn[i3].outwt[i4]<<"  ";}

       cout<<"\n"; }

cout<<"\n";

cout<<"\nweights--  output layer to input layer: \n\n";

for(i3=0;i3<bnmbr;++i3){

       for(i4=0;i4<anmbr;++i4){

              cout<<bnrn[i3].outwt[i4]<<"  ";}

       cout<<"\n";  }

cout<<"\n";

}

void network::iterate()

{

int i1;

for(i1=0;i1<nexmplr;++i1){

       findassc(e[i1].v1);

       }

}

void network::findassc(float *b)

{

int j;

flag = 0;

asgninpt(b);

       ninpt ++;

       cout<<"\nInput vector is:\n" ;

       for(j=0;j<6;++j){

              cout<<b[j]<<" ";};

       cout<<"\n";

       pp[0].getpotlpair(anmbr,bnmbr);

asgnvect(anmbr,outs1,pp[0].v1);

comput1();

if(flag>=0){

       asgnvect(bnmbr,outs2,pp[0].v2);

       cout<<"\n";

       pp[0].prpotlpair();

       cout<<"\n";

       comput2(); }

for(j=1;j<MXSIZ;++j){

       pp[j].getpotlpair(anmbr,bnmbr);

       asgnvect(anmbr,outs1,pp[j].v1);

       comput1();

       asgnvect(bnmbr,outs2,pp[j].v2);

       pp[j].prpotlpair();

       cout<<"\n";

       compr1(j,j-1);

       compr2(j,j-1);

       if(flag == 0) {

              int j2;

              nasspr += 1;

              j2 = nasspr;

              as[j2].getasscpair(j2,anmbr,bnmbr);

              asgnvect(anmbr,pp[j].v1,as[j2].v1);

              asgnvect(bnmbr,pp[j].v2,as[j2].v2);

              cout<<"\nPATTERNS ASSOCIATED:\n";

              as[j2].prasscpair();

              j = MXSIZ ;

              }

       else

              if(flag == 1)

                     {

                     flag = 0;

                     comput1();

                     }

              }

}

void network::prstatus()

{

int j;

cout<<"\nTHE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY FUZZY AM\n\n";

for(j=1;j<=nasspr;++j){

       as[j].prasscpair();

       cout<<"\n";}

}

void main()

{

int ar = 6, br = 4, nex = 1;

float inptv[][6]={0.1,0.3,0.2,0.0,0.7,0.5,0.6,0.0,0.3,0.4,0.1,0.2};

float outv[][4]={0.4,0.2,0.1,0.0};

cout<<"\n\nTHIS PROGRAM IS FOR A FUZZY ASSOCIATIVE MEMORY NETWORK. THE

NETWORK \n";

cout<<"IS SET UP FOR ILLUSTRATION WITH "<<ar<<" INPUT NEURONS, AND "<<br;

cout<<" OUTPUT NEURONS.\n"<<nex<<" exemplars are used to encode \n";

static network famn;

famn.getnwk(ar,br,nex,inptv,outv);

famn.iterate();

famn.findassc(inptv[1]);

famn.prstatus();

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Output

The illustrative run of the previous program uses the fuzzy sets with fit vectors (0.1, 0.3, 0.2, 0.0, 0.7, 0.5) and (0.4, 0.2, 0.1, 0.0). As you can expect according to the discussion earlier, recall is not perfect in the reverse direction and the fuzzy associated memory consists of the pairs (0.1, 0.3, 0.2, 0.0, 0.4, 0.4) with (0.4, 0.2, 0.1, 0.0) and (0.1, 0.2, 0.2, 0, 0.2, 0.2) with (0.2, 0.2, 0.1, 0). The computer output is in such detail as to be self-explanatory.

THIS PROGRAM IS FOR A FUZZY ASSOCIATIVE MEMORY NETWORK. THE NETWORK IS

SET UP FOR ILLUSTRATION WITH SIX INPUT NEURONS, AND FOUR OUTPUT NEURONS.

1 exemplars are used to encode

X vector you gave is:

0.1  0.3  0.2  0  0.7  0.5

Y vector you gave is:

0.4  0.2  0.1  0

  weights--input layer to output layer:

0.1  0.1  0.1  0

0.3  0.2  0.1  0

0.2  0.2  0.1  0

0    0    0    0

0.4  0.2  0.1  0

0.4  0.2  0.1  0

weights--output layer to input layer:

0.1  0.3  0.2  0  0.4  0.4

0.1  0.2  0.2  0  0.2  0.2

0.1  0.1  0.1  0  0.1  0.1

0    0    0    0  0    0

Input vector is:

0.1 0.3 0.2 0 0.7 0.5

output layer neuron  0 activation is 0.4

output layer neuron  0 output is 0.4

output layer neuron  1 activation is 0.2

output layer neuron  1 output is 0.2

output layer neuron  2 activation is 0.1

output layer neuron  2 output is 0.1

output layer neuron  3 activation is 0

 output layer neuron  3 output is 0

X vector in possible associated pair is:

0.1  0.3  0.2  0  0.7  0.5

Y vector in possible associated pair is:

0.4  0.2  0.1  0

input layer neuron 0 activation is 0.1

input layer neuron  0 output is 0.1

input layer neuron 1 activation is 0.3

input layer neuron  1 output is 0.3

input layer neuron 2 activation is 0.2

input layer neuron  2 output is 0.2

input layer neuron 3 activation is 0

input layer neuron  3 output is 0

input layer neuron 4 activation is 0.4

input layer neuron  4 output is 0.4

input layer neuron 5 activation is 0.4

input layer neuron  5 output is 0.4

output layer neuron  0 activation is 0.4

output layer neuron  0 output is 0.4

output layer neuron  1 activation is 0.2

output layer neuron  1 output is 0.2

output layer neuron  2 activation is 0.1

output layer neuron  2 output is 0.1

output layer neuron  3 activation is 0

output layer neuron  3 output is 0

X vector in possible associated pair is:

0.1  0.3  0.2  0  0.4  0.4

Y vector in possible associated pair is:

0.4  0.2  0.1  0

PATTERNS ASSOCIATED:

X vector in the associated pair no. 1 is:

0.1  0.3  0.2  0  0.4  0.4

Y vector in the associated pair no. 1 is:

0.4  0.2  0.1  0

Input vector is:

0.6 0 0.3 0.4 0.1 0.2

 output layer neuron  0 activation is 0.2

 output layer neuron  0 output is 0.2

 output layer neuron  1 activation is 0.2

 output layer neuron  1 output is 0.2

 output layer neuron  2 activation is 0.1

 output layer neuron  2 output is 0.1

 output layer neuron  3 activation is 0

 output layer neuron  3 output is 0

X vector in possible associated pair is:

0.6  0  0.3  0.4  0.1  0.2

Y vector in possible associated pair is:

0.2  0.2  0.1  0

input layer neuron 0 activation is 0.1

 input layer neuron  0 output is 0.1

input layer neuron 1 activation is 0.2

 input layer neuron  1 output is 0.2

input layer neuron 2 activation is 0.2

 input layer neuron  2 output is 0.2

input layer neuron 3 activation is 0

 input layer neuron  3 output is 0

input layer neuron 4 activation is 0.2

 input layer neuron  4 output is 0.2

input layer neuron 5 activation is 0.2

 input layer neuron  5 output is 0.2

 output layer neuron  0 activation is 0.2

 output layer neuron  0 output is 0.2

 output layer neuron  1 activation is 0.2

 output layer neuron  1 output is 0.2

 output layer neuron  2 activation is 0.1

 output layer neuron  2 output is 0.1

 output layer neuron  3 activation is 0

 output layer neuron  3 output is 0

X vector in possible associated pair is:

0.1  0.2  0.2  0  0.2  0.2

Y vector in possible associated pair is:

0.2  0.2  0.1  0

 output layer neuron  0 activation is 0.2

 output layer neuron  0 output is 0.2

 output layer neuron  1 activation is 0.2

 output layer neuron  1 output is 0.2

 output layer neuron  2 activation is 0.1

 output layer neuron  2 output is 0.1

 output layer neuron  3 activation is 0

 output layer neuron  3 output is 0

 output layer neuron  0 activation is 0.2

 output layer neuron  0 output is 0.2

 output layer neuron  1 activation is 0.2

 output layer neuron  1 output is 0.2

 output layer neuron  2 activation is 0.1

 output layer neuron  2 output is 0.1

 output layer neuron  3 activation is 0

 output layer neuron  3 output is 0

X vector in possible associated pair is:

0.1  0.2  0.2  0  0.2  0.2

Y vector in possible associated pair is:

0.2  0.2  0.1  0

PATTERNS ASSOCIATED:

X vector in the associated pair no. 2 is:

0.1  0.2  0.2  0  0.2  0.2

Y vector in the associated pair no. 2 is:

0.2  0.2  0.1  0

THE FOLLOWING ASSOCIATED PAIRS WERE FOUND BY FUZZY AM

X vector in the associated pair no. 1 is:

0.1  0.3  0.2  0  0.4  0.4

Y vector in the associated pair no. 1 is:

0.4  0.2  0.1  0

X vector in the associated pair no. 2 is:

0.1  0.2  0.2  0  0.2  0.2

Y vector in the associated pair no. 2 is:

0.2  0.2  0.1  0

Summary

In this chapter, bidirectional associative memories are presented for fuzzy subsets. The development of these is largely due to Kosko. They share the feature of resonance between the two layers in the network with Adaptive Resonance theory. Even though there are connections in both directions between neurons in the two layers, only one weight matrix is involved. You use the transpose of this weight matrix for the connections in the opposite direction. When one input at one end leads to some output at the other, which in turn leads to output same as the previous input, resonance is reached and an associated pair is found. In the case of bidirectional fuzzy associative memories, one pair of fuzzy sets determines one fuzzy associative memory system. Fit vectors are used in max–min composition. Perfect recall in both directions is not the case unless the heights of both fit vectors are equal. Fuzzy associative memories can improve the performance of an expert system by allowing fuzzy rules.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 10
Adaptive Resonance Theory (ART)

Introduction

Grossberg's Adaptive Resonance Theory, developed further by Grossberg and Carpenter, is for the categorization of patterns using the competitive learning paradigm. It introduces a gain control and a reset to make certain that learned categories are retained even while new categories are learned and thereby addresses the plasticity–stability dilemma.

Adaptive Resonance Theory makes much use of a competitive learning paradigm. A criterion is developed to facilitate the occurrence of winner-take-all phenomenon. A single node with the largest value for the set criterion is declared the winner within its layer, and it is said to classify a pattern class. If there is a tie for the winning neuron in a layer, then an arbitrary rule, such as the first of them in a serial order, can be taken as the winner.

The neural network developed for this theory establishes a system that is made up of two subsystems, one being the attentional subsystem, and this contains the unit for gain control. The other is an orienting subsystem, and this contains the unit for reset. During the operation of the network modeled for this theory, patterns emerge in the attentional subsystem and are called traces of STM (short-term memory). Traces of LTM (long-term memory) are in the connection weights between the input layer and output layer.

The network uses processing with feedback between its two layers, until resonance occurs. Resonance occurs when the output in the first layer after feedback from the second layer matches the original pattern used as input for the first layer in that processing cycle. A match of this type does not have to be perfect. What is required is that the degree of match, measured suitably, exceeds a predetermined level, termed vigilance parameter. Just as a photograph matches the likeness of the subject to a greater degree when the granularity is higher, the pattern match gets finer when the vigilance parameter is closer to 1.

The Network for ART1

The neural network for the adaptive resonance theory or ART1 model consists of the following:

• A layer of neurons, called the F1 layer (input layer or comparison layer)
• A node for each layer as a gain control unit
• A layer of neurons, called the F2 layer (output layer or recognition layer)
• A node as a reset unit
• Bottom-up connections from F1 layer to F2 layer
• Top-down connections from F2 layer to F1 layer
• Inhibitory connection (negative weight) form F2 layer to gain control
• Excitatory connection (positive weight) from gain control to a layer
• Inhibitory connection from F1 layer to reset node
• Excitatory connection from reset node to F2 layer

A Simplified Diagram of Network Layout

Figure 10.1 simplified diagram of the neural network for an ART1 model.

Processing in ART1

The ART1 paradigm, just like the Kohonen Self-Organizing Map to be introduced in Chapter 11, performs data clustering on input data; like inputs are clustered together into a category. As an example, you can use a data clustering algorithm such as ART1 for Optical Character Recognition (OCR), where you try to match different samples of a letter to its ASCII equivalent. Particular attention is made in the ART1 paradigm to ensure that old information is not thrown away while new information is assimilated.

An input vector, when applied to an ART1 system, is first compared to existing patterns in the system. If there is a close enough match within a specified tolerance (as indicated by a vigilance parameter), then that stored pattern is made to resemble the input pattern further and the classification operation is complete. If the input pattern does not resemble any of the stored patterns in the system, then a new category is created with a new stored pattern that resembles the input pattern.

Special Features of the ART1 Model

One special feature of an ART1 model is that a two-thirds rule is necessary to determine the activity of neurons in the F1 layer. There are three input sources to each neuron in layer F1. They are the external input, the output of gain control, and the outputs of F2 layer neurons. The F1neurons will not fire unless at least two of the three inputs are active. The gain control unit and the two-thirds rule together ensure proper response from the input layer neurons. A second feature is that a vigilance parameter is used to determine the activity of the reset unit, which is activated whenever there is no match found among existing patterns during classification.

Notation for ART1 Calculations

Let us list the various symbols we will use to describe the operation of a neural network for an ART1 model:

wij | Weight on the connection from the ith neuron in the F1 layer to the jth neuron in the F2 layer
vji | Weight on the connection from the jth neuron in the F2 layer to the ith neuron on the F1 layer
ai | Activation of the ith neuron in the F1 layer
bj | Activation of the jth neuron in the F2 layer
xi | Output of the ith neuron in the F1 layer
yj | Output of the jth neuron in the F2 layer
zi | Input to the ith neuron in F1 layer from F2 layer
ρ | Vigilance parameter, positive and no greater than 1 (0<ρ ≤ 1)
m | Number of neurons in the F1 layer
n | Number of neurons in the F2 layer
I | Input vector
Si | Sum of the components of the input vector
Sx | Sum of the outputs of neurons in the F1 layer
A, C, D | Parameters with positive values or zero
L | Parameter with value greater than 1
B | Parameter with value less than D + 1 but at least as large as either D or 1
r | Index of winner of competition in the F2 layer


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Algorithm for ART1 Calculations

The ART1 equations are not easy to follow. We follow the description of the algorithm found in James A. Freeman and David M. Skapura. The following equations, taken in the order given, describe the steps in the algorithm. Note that binary input patterns are used in ART1.

Initialization of Parameters

wij | should be positive and less than L / ( m - 1 + L)
vji | should be greater than ( B - 1 ) / D
ai = -B / ( 1 + C )

Equations for ART1 Computations

When you read below the equations for ART1 computations, keep in mind the following considerations. If a subscript i appears on the left-hand side of the equation, it means that there are m such equations, as the subscript i varies from 1 to m. Similarly, if instead a subscript j occurs, then there are n such equations as j ranges from 1 to n. The equations are used in the order they are given. They give a step-by-step description of the following algorithm. All the variables, you recall, are defined in the earlier section on notation. For example, I is the input vector.

F1 layer calculations:

       ai = Ii / ( 1 + A ( Ii + B ) + C )

       xi = 1 if ai > 0

            0 if ai ≤ 0

F2 layer calculations:

       bj = Σ wij xi, the summation being on i from 1 to m

       yj = 1 if jth neuron has the largest activation value in the F2

            layer

          = 0 if jth neuron is not the winner in F2 layer

Top-down inputs:

       zi = Σvjiyj, the summation being on j from 1 to n (You will

       notice that exactly one term is nonzero)

F1 layer calculations:

       ai = ( Ii + D zi - B ) / ( 1 + A ( Ii + D zi ) + C )

       xi = 1 if ai > 0

          = 0 if ai ≤ 0

Checking with vigilance parameter:

If ( Sx / SI ) <Σ, set yj = 0 for all j, including the winner r in F2 layer, and consider the jth neuron inactive (this step is reset, skip remaining steps).

If ( Sx / SI ) ≥ Σ, then continue.

Modifying top-down and bottom-up connection weight for winner r:

       vir  = ( L / ( Sx + L -1 ) if xi = 1

            = 0 if xi = 0

       wri  = 1 if xi = 1

            = 0 if xi = 0

Having finished with the current input pattern, we repeat these steps with a new input pattern. We lose the index r given to one neuron as a winner and treat all neurons in the F2 layer with their original indices (subscripts).

The above presentation of the algorithm is hoped to make all the steps as clear as possible. The process is rather involved. To recapitulate, first an input vector is presented to the F1 layer neurons, their activations are determined, and then the threshold function is used. The outputs of the F1 layer neurons constitute the inputs to the F2 layer neurons, from which a winner is designated on the basis of the largest activation. The winner only is allowed to be active, meaning that the output is 1 for the winner and 0 for all the rest. The equations implicitly incorporate the use of the 2/3 rule that we mentioned earlier, and they also incorporate the way the gain control is used. The gain control is designed to have a value 1 in the phase of determining the activations of the neurons in the F2 layer and 0 if either there is no input vector or output from the F2 layer is propagated to the F1 layer.

Other Models

Extensions of an ART1 model, which is for binary patterns, are ART2 and ART3. Of these, ART2 model categorizes and stores analog-valued patterns, as well as binary patterns, while ART3 addresses computational problems of hierarchies.

C++ Implementation

Again, the algorithm for ART1 processing as given in Freeman and Skapura is followed for our C++ implementation. Our objective in programming ART1 is to provide a feel for the workings of this paradigm with a very simple program implementation. For more details on the inner workings of ART1, you are encouraged to consult Freeman and Skapura, or other references listed at the back of the book.

A Header File for the C++ Program for the ART1 Model Network

The header file for the C++ program for the ART1 model network is art1net.hpp. It contains the declarations for two classes, an artneuron class for neurons in the ART1 model, and a network class, which is declared as a friend class in the artneuron class. Functions declared in the network class include one to do the iterations for the network operation, finding the winner in a given iteration, and one to inquire if reset is needed.

//art1net.h   V. Rao,  H. Rao

//Header file for ART1 model network program

#include <iostream.h>

#define MXSIZ 10

class artneuron

{

protected:

       int nnbr;

       int inn,outn;

       int output;

       double activation;

       double outwt[MXSIZ];

       char *name;

       friend class network;

public:

       artneuron() { };

       void getnrn(int,int,int,char *);

};

class network

{

public:

       int  anmbr,bnmbr,flag,ninpt,sj,so,winr;

       float ai,be,ci,di,el,rho;

       artneuron (anrn)[MXSIZ],(bnrn)[MXSIZ];

       int outs1[MXSIZ],outs2[MXSIZ];

       int lrndptrn[MXSIZ][MXSIZ];

       double acts1[MXSIZ],acts2[MXSIZ];

       double mtrx1[MXSIZ][MXSIZ],mtrx2[MXSIZ][MXSIZ];

       network() { };

       void getnwk(int,int,float,float,float,float,float);

       void prwts1();

       void prwts2();

       int winner(int k,double *v,int);

       void practs1();

       void practs2();

       void prouts1();

       void prouts2();

       void iterate(int *,float,int);

       void asgninpt(int *);

       void comput1(int);

       void comput2(int *);

       void prlrndp();

       void inqreset(int);

       void adjwts1();

       void adjwts2();

};


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

A Source File for C++ Program for an ART1 Model Network

The implementations of the functions declared in the header file are contained in the source file for the C++ program for an ART1 model network. It also has the main function, which contains specifications of the number of neurons in the two layers of the network, the values of the vigilance and other parameters, and the input vectors. Note that if there are n neurons in a layer, they are numbered serially from 0 to n–1, and not from 1 to n in the C++ program. The source file is called art1net.cpp. It is set up with six neurons in the F1 layer and seven neurons in the F2 layer. The main function also contains the parameters needed in the algorithm.

To initialize the bottom-up weights, we set each weight to be –0.1 + L/(m – 1 + L) so that it is greater than 0 and less than L/(m – 1 + L), as suggested before. Similarly, the top-down weights are initialized by setting each of them to 0.2 + (B – 1)/D so it would be greater than (B – 1)/D. Initial activations of the F1 layer neurons are each set to –B/(1 + C), as suggested earlier.

A restrmax function is defined to compute the maximum in an array when one of the array elements is not desired to be a candidate for the maximum. This facilitates the removal of the current winner from competition when reset is needed. Reset is needed when the degree of match is of a smaller magnitude than the vigilance parameter.

The function iterate is a member function of the network class and does the processing for the network. The inqreset function of the network class compares the vigilance parameter with the degree of match.

//art1net.cpp  V. Rao, H. Rao

//Source file for ART1 network program

#include "art1net.h"

int restrmax(int j,double *b,int k)

       {

       int i,tmp;

       for(i=0;i<j;i++){

              if(i !=k)

              {tmp = i;

              i = j;}

              }

       for(i=0;i<j;i++){

       if( (i != tmp)&&(i != k))

         {if(b[i]>b[tmp]) tmp = i;}}

       return tmp;

       }

void artneuron::getnrn(int m1,int m2,int m3, char *y)

{

int i;

name = y;

nnbr = m1;

outn = m2;

inn  = m3;

for(i=0;i<outn;++i){

       outwt[i] = 0 ;

       }

output = 0;

activation = 0.0;

}

       void network::getnwk(int k,int l,float aa,float bb,float

       cc,float dd,float ll)

{

anmbr = k;

bnmbr = l;

ninpt = 0;

ai = aa;

be = bb;

ci = cc;

di = dd;

el = ll;

int i,j;

flag = 0;

char *y1="ANEURON", *y2="BNEURON" ;

for(i=0;i<anmbr;++i){

       anrn[i].artneuron::getnrn(i,bnmbr,0,y1);}

for(i=0;i<bnmbr;++i){

       bnrn[i].artneuron::getnrn(i,0,anmbr,y2);}

float tmp1,tmp2,tmp3;

tmp1 = 0.2 +(be - 1.0)/di;

tmp2 = -0.1 + el/(anmbr - 1.0 +el);

tmp3 = - be/(1.0 + ci);

for(i=0;i<anmbr;++i){

       anrn[i].activation = tmp3;

       acts1[i] = tmp3;

       for(j=0;j<bnmbr;++j){

              mtrx1[i][j]  = tmp1;

              mtrx2[j][i] = tmp2;

              anrn[i].outwt[j] = mtrx1[i][j];

              bnrn[j].outwt[i] = mtrx2[j][i];

              }

       }

prwts1();

prwts2();

practs1();

cout<<"\n";

}

int network::winner(int k,double *v,int kk){

int t1;

t1 = restrmax(k,v,kk);

return t1;

}

void network::prwts1()

{

int i3,i4;

cout<<"\nweights for F1 layer neurons: \n";

for(i3=0;i3<anmbr;++i3){

       for(i4=0;i4<bnmbr;++i4){

              cout<<anrn[i3].outwt[i4]<<"  ";}

       cout<<"\n"; }

cout<<"\n";

}

void network::prwts2()

{

int i3,i4;

cout<<"\nweights for F2 layer neurons: \n";

for(i3=0;i3<bnmbr;++i3){

       for(i4=0;i4<anmbr;++i4){

              cout<<bnrn[i3].outwt[i4]<<"  ";};

       cout<<"\n";  }

cout<<"\n";

}

void network::practs1()

{

int j;

cout<<"\nactivations of F1 layer neurons: \n";

for(j=0;j<anmbr;++j){

       cout<<acts1[j]<<"   ";}

cout<<"\n";

}

void network::practs2()

{

int j;

cout<<"\nactivations of F2 layer neurons: \n";

for(j=0;j<bnmbr;++j){

       cout<<acts2[j]<<"   ";}

cout<<"\n";

}

void network::prouts1()

{

int j;

cout<<"\noutputs of F1 layer neurons: \n";

for(j=0;j<anmbr;++j){

       cout<<outs1[j]<<"   ";}

cout<<"\n";

}

void network::prouts2()

{

int j;

cout<<"\noutputs of F2 layer neurons: \n";

for(j=0;j<bnmbr;++j){

       cout<<outs2[j]<<"   ";}

cout<<"\n";

}

void network::asgninpt(int *b)

{

int j;

sj = so = 0;

cout<<"\nInput vector is:\n" ;

for(j=0;j<anmbr;++j){

       cout<<b[j]<<" ";}

cout<<"\n";

for(j=0;j<anmbr;++j){

       sj += b[j];

       anrn[j].activation = b[j]/(1.0 +ci +ai*(b[j]+be));

       acts1[j] = anrn[j].activation;

       if(anrn[j].activation > 0) anrn[j].output = 1;

       else

              anrn[j].output = 0;

       outs1[j] = anrn[j].output;

       so += anrn[j].output;

       }

practs1();

prouts1();

}

void network::inqreset(int t1)

{

int jj;

flag = 0;

jj = so/sj;

cout<<"\ndegree of match: "<<jj<<" vigilance:  "<<rho<<"\n";

if( jj > rho ) flag = 1;

       else

       {cout<<"winner is "<<t1;

       cout<<" reset required \n";}

}

void network::comput1(int k)

{

int j;

for(j=0;j<bnmbr;++j){

       int ii1;

       double c1 = 0.0;

       cout<<"\n";

       for(ii1=0;ii1<anmbr;++ii1){

              c1 += outs1[ii1] * mtrx2[j][ii1];

              }

       bnrn[j].activation = c1;

       acts2[j] = c1;};

winr = winner(bnmbr,acts2,k);

cout<<"winner is "<<winr;

for(j=0;j<bnmbr;++j){

       if(j == winr) bnrn[j].output = 1;

       else bnrn[j].output =  0;

       outs2[j] = bnrn[j].output;

       }

practs2();

prouts2();

}

void network::comput2(int *b)

{

double db[MXSIZ];

double tmp;

so = 0;

int i,j;

for(j=0;j<anmbr;++j){

       db[j] =0.0;

       for(i=0;i<bnmbr;++i){

              db[j] += mtrx1[j][i]*outs2[i];};

       tmp = b[j] + di*db[j];

       acts1[j] = (tmp - be)/(ci +1.0 +ai*tmp);

       anrn[j].activation = acts1[j];

       if(anrn[j].activation > 0) anrn[j].output = 1;

       else anrn[j].output = 0;

       outs1[j] = anrn[j].output;

       so += anrn[j].output;

       }

cout<<"\n";

practs1();

prouts1();

}

void network::adjwts1()

{

int i;

for(i=0;i<anmbr;++i){

       if(outs1[i] >0) {mtrx1[i][winr]  = 1.0;}

       else

              {mtrx1[i][winr] = 0.0;}

       anrn[i].outwt[winr] = mtrx1[i][winr];}

prwts1();

}

void network::adjwts2()

{

int i;

cout<<"\nwinner is "<<winr<<"\n";

for(i=0;i<anmbr;++i){

       if(outs1[i] > 0) {mtrx2[winr][i] = el/(so + el -1);}

       else

              {mtrx2[winr][i] = 0.0;}

       bnrn[winr].outwt[i]  = mtrx2[winr][i];}

prwts2();

}

void network::iterate(int *b,float rr,int kk)

{

int j;

rho = rr;

flag = 0;

asgninpt(b);

comput1(kk);

comput2(b);

inqreset(winr);

if(flag == 1){

       ninpt ++;

       adjwts1();

       adjwts2();

       int j3;

       for(j3=0;j3<anmbr;++j3){

              lrndptrn[ninpt][j3] = b[j3];}

       prlrndp();

       }

else

       {

       for(j=0;j<bnmbr;++j){

              outs2[j] = 0;

              bnrn[j].output = 0;}

       iterate(b,rr,winr);

       }

}

void network::prlrndp()

{

int j;

cout<<"\nlearned vector # "<<ninpt<<"  :\n";

for(j=0;j<anmbr;++j){

       cout<<lrndptrn[ninpt][j]<<"  ";}

cout<<"\n";

}

void main()

{

int ar = 6, br = 7, rs = 8;

float aa = 2.0,bb = 2.5,cc = 6.0,dd = 0.85,ll = 4.0,rr =

       0.95;

int inptv[][6]={0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,\

       1,0};

cout<<"\n\nTHIS PROGRAM IS FOR AN -ADAPTIVE RESONANCE THEORY\

       1 - NETWORK.\n";

cout<<"THE NETWORK IS SET UP FOR ILLUSTRATION WITH "<<ar<<" \

       INPUT NEURONS,\n";

cout<<" AND "<<br<<" OUTPUT NEURONS.\n";

static network bpn;

bpn.getnwk(ar,br,aa,bb,cc,dd,ll) ;

bpn.iterate(inptv[0],rr,rs);

bpn.iterate(inptv[1],rr,rs);

bpn.iterate(inptv[2],rr,rs);

bpn.iterate(inptv[3],rr,rs);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Program Output

Four input vectors are used in the trial run of the program, and these are specified in the main function. The output is self-explanatory. We have included only in this text some comments regarding the output. These comments are enclosed within strings of asterisks. They are not actually part of the program output. Table 10.1 shows a summarization of the categorization of the inputs done by the network. Keep in mind that the numbering of the neurons in any layer, which has n neurons, is from 0 to n – 1, and not from 1 to n.

Table 10.1 Categorization of Inputs
input | winner in F2 layer
0 1 0 0 0 0 | 0, no reset
1 0 1 0 1 0 | 1, no reset
0 0 0 0 1 0 | 1, after reset 2
1 0 1 0 1 0 | 1, after reset 3

The input pattern 0 0 0 0 1 0 is considered a subset of the pattern 1 0 1 0 1 0 in the sense that in whatever position the first pattern has a 1, the second pattern also has a 1. Of course, the second pattern has 1's in other positions as well. At the same time, the pattern 1 0 1 0 1 0 is considered a superset of the pattern 0 0 0 0 1 0. The reason that the pattern 1 0 1 0 1 0 is repeated as input after the pattern 0 0 0 0 1 0 is processed, is to see what happens with this superset. In both cases, the degree of match falls short of the vigilance parameter, and a reset is needed.

Here's the output of the program:

THIS PROGRAM IS FOR AN ADAPTIVE RESONANCE THEORY

1-NETWORK. THE NETWORK IS SET UP FOR ILLUSTRATION WITH SIX INPUT NEURONS

AND SEVEN OUTPUT NEURONS.

*************************************************************

Initialization of connection weights and F1 layer activations. F1 layer

connection weights are all chosen to be equal to a random value subject

to the conditions given in the algorithm. Similarly, F2 layer connection

weights are all chosen to be equal to a random value subject to the

conditions given in the algorithm.

*************************************************************

weights for F1 layer neurons:

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1.964706  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

weights for F2 layer neurons:

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

activations of F1 layer neurons:

-0.357143 -0.357143 -0.357143 -0.357143 -0.357143 -0.357143

*************************************************************

A new input vector and a new iteration

*************************************************************

Input vector is:

0 1 0 0 0 0

activations of F1 layer neurons:

0   0.071429   0   0   0   0

outputs of F1 layer neurons:

0   1   0   0   0   0

winner is 0

activations of F2 layer neurons:

0.344444   0.344444   0.344444   0.344444   0.344444   0.344444   0.344444

outputs of F2 layer neurons:

1   0   0   0   0   0   0

activations of F1 layer neurons:

-0.080271   0.013776   -0.080271   -0.080271   -0.080271   -0.080271

outputs of F1 layer neurons:

0   1   0   0   0   0

*************************************************************

Top-down and bottom-up outputs at F1 layer match, showing resonance.

*************************************************************

degree of match: 1 vigilance:  0.95

weights for F1 layer neurons:

0  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

1  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

0  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

0  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

0  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

0  1.964706  1.964706  1.964706  1.964706  1.964706  1.964706

winner is 0

weights for F2 layer neurons:

0  1  0  0  0  0

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

learned vector # 1  :

0  1  0  0  0  0

*************************************************************

A new input vector and a new iteration

*************************************************************

Input vector is:

1 0 1 0 1 0

activations of F1 layer neurons:

0.071429   0   0.071429   0   0.071429   0

outputs of F1 layer neurons:

1   0   1   0   1   0

winner is 1

activations of F2 layer neurons:

0   1.033333   1.033333   1.033333   1.033333   1.033333   1.033333

outputs of F2 layer neurons:

0   1   0   0   0   0   0

activations of F1 layer neurons:

0.013776   -0.080271   0.013776   -0.080271   0.013776   -0.080271

outputs of F1 layer neurons:

1   0   1   0   1   0

*************************************************************

Top-down and bottom-up outputs at F1 layer match,

showing resonance.

*************************************************************

degree of match: 1 vigilance:  0.95

weights for F1 layer neurons:

0  1  1.964706  1.964706  1.964706  1.964706  1.964706

1  0  1.964706  1.964706  1.964706  1.964706  1.964706

0  1  1.964706  1.964706  1.964706  1.964706  1.964706

0  0  1.964706  1.964706  1.964706  1.964706  1.964706

0  1  1.964706  1.964706  1.964706  1.964706  1.964706

0  0  1.964706  1.964706  1.964706  1.964706  1.964706

winner is 1

weights for F2 layer neurons:

0  1  0  0  0  0

0.666667  0  0.666667  0  0.666667  0

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

learned vector # 2  :

1  0  1  0  1  0

*************************************************************

A new input vector and a new iteration

*************************************************************

Input vector is:

0 0 0 0 1 0

activations of F1 layer neurons:

0   0   0   0   0.071429   0

outputs of F1 layer neurons:

0   0   0   0   1   0

winner is 1

activations of F2 layer neurons:

0   0.666667   0.344444   0.344444   0.344444   0.344444   0.344444

outputs of F2 layer neurons:

0   1   0   0   0   0   0

activations of F1 layer neurons:

-0.189655   -0.357143   -0.189655   -0.357143   -0.060748   -0.357143

outputs of F1 layer neurons:

0   0   0   0   0   0

degree of match: 0 vigilance:  0.95

winner is 1 reset required

*************************************************************

Input vector repeated after reset, and a new iteration

*************************************************************

Input vector is:

0 0 0 0 1 0

activations of F1 layer neurons:

0   0   0   0   0.071429   0

outputs of F1 layer neurons:

0   0   0   0   1   0

winner is 2

activations of F2 layer neurons:

0   0.666667   0.344444   0.344444   0.344444   0.344444   0.344444

outputs of F2 layer neurons:

0   0   1   0   0   0   0

      activations of F1 layer neurons:

-0.080271   -0.080271   -0.080271   -0.080271   0.013776   -0.080271

outputs of F1 layer neurons:

0   0   0   0   1   0

*************************************************************

Top-down and bottom-up outputs at F1 layer match, showing resonance.

*************************************************************

degree of match: 1 vigilance:  0.95

weights for F1 layer neurons:

0  1  0  1.964706  1.964706  1.964706  1.964706

1  0  0  1.964706  1.964706  1.964706  1.964706

0  1  0  1.964706  1.964706  1.964706  1.964706

0  0  0  1.964706  1.964706  1.964706  1.964706

0  1  1  1.964706  1.964706  1.964706  1.964706

0  0  0  1.964706  1.964706  1.964706  1.964706

winner is 2

weights for F2 layer neurons:

0  1  0  0  0  0

0.666667  0  0.666667  0  0.666667  0

0  0  0  0  1  0

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

learned vector # 3  :

0  0  0  0  1  0

*************************************************************

An old (actually the second above) input vector is retried after trying a

subset vector, and a new iteration

*************************************************************

Input vector is:

1 0 1 0 1 0

activations of F1 layer neurons:

0.071429   0   0.071429   0   0.071429   0

outputs of F1 layer neurons:

1   0   1   0   1   0

winner is 1

activations of F2 layer neurons:

0   2   1   1.033333   1.033333   1.033333   1.03333

outputs of F2 layer neurons:

0   1   0   0   0   0   0

activations of F1 layer neurons:

-0.060748   -0.357143   -0.060748   -0.357143   -0.060748   -0.357143

outputs of F1 layer neurons:

0   0   0   0   0   0

degree of match: 0 vigilance:  0.95

winner is 1 reset required

*************************************************************

Input vector repeated after reset, and a new iteration

*************************************************************

Input vector is:

1 0 1 0 1 0

activations of F1 layer neurons:

0.071429   0   0.071429   0   0.071429   0

outputs of F1 layer neurons:

1   0   1   0   1   0

winner is 3

activations of F2 layer neurons:

0   2   1   1.033333   1.033333   1.033333   1.033333

outputs of F2 layer neurons:

0   0   0   1   0   0   0

activations of F1 layer neurons:

0.013776   -0.080271   0.013776   -0.080271   0.013776   -0.080271

outputs of F1 layer neurons:

1   0   1   0   1   0

*************************************************************

Top-down and Bottom-up outputs at F1layer match, showing resonance.

*************************************************************

degree of match: 1 vigilance:  0.95

weights for F1 layer neurons:

0  1  0  1  1.964706  1.964706  1.964706

1  0  0  0  1.964706  1.964706  1.964706

0  1  0  1  1.964706  1.964706  1.964706

0  0  0  0  1.964706  1.964706  1.964706

0  1  1  1  1.964706  1.964706  1.964706

0  0  0  0  1.964706  1.964706  1.964706

winner is 3

weights for F2 layer neurons:

0  1  0  0  0  0

0.666667  0  0.666667  0  0.666667  0

0  0  0  0  1  0

0.666667  0  0.666667  0  0.666667  0

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

0.344444  0.344444  0.344444  0.344444  0.344444  0.344444

learned vector # 4  :

1  0  1  0  1  0


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Summary

This chapter presented the basics of the Adaptive Resonance Theory of Grossberg and Carpenter and a C++ implementation of the neural network modeled for this theory. It is an elegant theory that addresses the stability–plasticity dilemma. The network relies on resonance. It is a self-organizing network and does categorization by associating individual neurons of the F2 layer with individual patterns. By employing a so-called 2/3 rule, it ensures stability in learning patterns.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 11
The Kohonen Self-Organizing Map

Introduction

This chapter discusses one type of unsupervised competitive learning, the Kohonen feature map, or self-organizing map (SOM). As you recall, in unsupervised learning there are no expected outputs presented to a neural network, as in a supervised training algorithm such as backpropagation. Instead, a network, by its self-organizing properties, is able to infer relationships and learn more as more inputs are presented to it. One advantage to this scheme is that you can expect the system to change with changing conditions and inputs. The system constantly learns. The Kohonen SOM is a neural network system developed by Teuvo Kohonen of Helsinki University of Technology and is often used to classify inputs into different categories. Applications for feature maps can be traced to many areas, including speech recognition and robot motor control.

Competitive Learning

A Kohonen feature map may be used by itself or as a layer of another neural network. A Kohonen layer is composed of neurons that compete with each other. Like in Adaptive Resonance Theory, the Kohonen SOM is another case of using a winner-take-all strategy. Inputs are fed into each of the neurons in the Kohonen layer (from the input layer). Each neuron determines its output according to a weighted sum formula:

Output = Σ wij xi

The weights and the inputs are usually normalized, which means that the magnitude of the weight and input vectors are set equal to one. The neuron with the largest output is the winner. This neuron has a final output of 1. All other neurons in the layer have an output of zero. Differing input patterns end up firing different winner neurons. Similar or identical input patterns classify to the same output neuron. You get like inputs clustered together. In Chapter 12, you will see the use of a Kohonen network in pattern classification.

Normalization of a Vector

Consider a vector, A = ax + by + cz. The normalized vector A' is obtained by dividing each component of A by the square root of the sum of squares of all the components. In other words each component is multiplied by 1/ [radic](a2 + b2 + c2). Both the weight vector and the input vector are normalized during the operation of the Kohonen feature map. The reason for this is the training law uses subtraction of the weight vector from the input vector. Using normalization of the values in the subtraction reduces both vectors to a unit-less status, and hence, makes the subtraction of like quantities possible. You will learn more about the training law shortly.

Lateral Inhibition

Lateral inhibition is a process that takes place in some biological neural networks. Lateral connections of neurons in a given layer are formed, and squash distant neighbors. The strength of connections is inversely related to distance. The positive, supportive connections are termed as excitatory while the negative, squashing connections are termed inhibitory.

A biological example of lateral inhibition occurs in the human vision system.

The Mexican Hat Function

Figure 11.1 shows a function, called the mexican hat function, which shows the relationship between the connection strength and the distance from the winning neuron. The effect of this function is to set up a competitive environment for learning. Only winning neurons and their neighbors participate in learning for a given input pattern.

Figure 11.1 The mexican hat function showing lateral inhibition.

Training Law for the Kohonen Map

The training law for the Kohonen feature map is straightforward. The change in weight vector for a given output neuron is a gain constant, alpha, multiplied by the difference between the input vector and the old weight vector:

Wnew = Wold + alpha * (Input -Wold)

Both the old weight vector and the input vector are normalized to unit length. Alpha is a gain constant between 0 and 1.

Significance of the Training Law

Let us consider the case of a two-dimensional input vector. If you look at a unit circle, as shown in Figure 11.2, the effect of the training law is to try to align the weight vector and the input vector. Each pattern attempts to nudge the weight vector closer by a fraction determined by alpha. For three dimensions the surface becomes a unit sphere instead of a circle. For higher dimensions you term the surface a hypersphere. It is not necessarily ideal to have perfect alignment of the input and weight vectors. You use neural networks for their ability to recognize patterns, but also to generalize input data sets. By aligning all input vectors to the corresponding winner weight vectors, you are essentially memorizing the input data set classes. It may be more desirable to come close, so that noisy or incomplete inputs may still trigger the correct classification.

Figure 11.2 The training law for the Kohonen map as shown on a unit circle.

The Neighborhood Size and Alpha

In the Kohonen map, a parameter called the neighborhood size is used to model the effect of the mexican hat function. Those neurons that are within the distance specified by the neighborhood size participate in training and weight vector changes; those that are outside this distance do not participate in learning. The neighborhood size typically is started as an initial value and is decreased as the input pattern cycles continue. This process tends to support the winner-take-all strategy by eventually singling out a winner neuron for a given pattern.

Figure 11.3 shows a linear arrangement of neurons with a neighborhood size of 2. The hashed central neuron is the winner. The darkened adjacent neurons are those that will participate in training.

Figure 11.3 Winner neuron with a neighborhood size of 2 for a Kohonen map.

Besides the neighborhood size, alpha typically is also reduced during simulation. You will see these features when we develop a Kohonen map program.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

C++ Code for Implementing a Kohonen Map

The C++ code for the Kohonen map draws on much of the code developed for the backpropagation simulator. The Kohonen map is a much simpler program and may not rely on as large a data set for input. The Kohonen map program uses only two files, an input file and an output file. In order to use the program, you must create an input data set and save this in a file called input.dat. The output file is called kohonen.dat and is saved in your current working directory. You will get more details shortly on the formats of these files.

The Kohonen Network

The Kohonen network has two layers, an input layer and a Kohonen output layer. (See Figure 11.4). The input layer is a size determined by the user and must match the size of each row (pattern) in the input data file.

Figure 11.4 A Kohonen network.

Modeling Lateral Inhibition and Excitation

The mexican hat function shows positive values for an immediate neighborhood around the neuron and negative values for distant neurons. A true method of modeling would incorporate mutual excitation or support for neurons that are within the neighborhood (with this excitation increasing for nearer neurons) and inhibition for distant neurons outside the neighborhood. For the sake of computational efficiency, we model lateral inhibition and excitation by looking at the maximum output for the output neurons and making that output belong to a winner neuron. Other outputs are inhibited by setting their outputs to zero. Training, or weight update, is performed on all outputs that are within a neighborhood size distance from the winner neuron. Neurons outside the neighborhood do not participate in training. The true way of modeling lateral inhibition would be too expensive since the number of lateral connections is quite large. You will find that this approximation will lead to a network with many if not all of the properties of a true modeling approach of a Kohonen network.

Classes to be Used

We use many of the classes from the backpropagation simulator. We require only two layers, the input layer and the Kohonen layer. We make a new layer class called the Kohonen layer class, and a new network class called the Kohonen_network.

Revisiting the Layer Class

The layer class needs to be slightly modified, as shown in Listing 11.1.

Listing 11.1 Modification of layer.h

// layer.h             V.Rao, H. Rao

// header file for the layer class hierarchy and

// the network class

#define MAX_LAYERS    5

#define MAX_VECTORS   100

class network;

class Kohonen_network;

class layer

{

protected:

       int num_inputs;

       int num_outputs;

       float *outputs;// pointer to array of outputs

       float *inputs; // pointer to array of inputs, which

                                     // are outputs of some other layer

       friend network;

       friend Kohonen_network; // update for Kohonen model

public:

       virtual void calc_out()=0;

};

...

Here the changes are indicated in italic. You notice that the Kohonen_network is made a friend to the layer class, so that the Kohonen_network can have access to the data of a layer.

A New Layer Class for a Kohonen Layer

The next step to take is to create a Kohonen_layer class and a Kohonen_network class. This is shown in Listing 11.2.

Listing 11.2 The Kohonen_layer class and Kohonen_network class in layerk.h

// layerk.h           V.Rao, H. Rao

// header file for the Kohonen layer and

// the Kohonen network

class Kohonen_network;

class Kohonen_layer: public layer

{

protected:

       float * weights;

       int winner_index;

       float win_distance;

       int neighborhood_size;

       friend Kohonen_network;

public:

        Kohonen_layer(int, int, int);

        ~Kohonen_layer();

        virtual void calc_out();

        void randomize_weights();

        void update_neigh_size(int);

        void update_weights(const float);

        void list_weights();

        void list_outputs();

        float get_win_dist();

};

class Kohonen_network

{

private:

        layer *layer_ptr[2];

        int layer_size[2];

        int neighborhood_size;

public:

        Kohonen_network();

        ~Kohonen_network();

        void get_layer_info();

        void set_up_network(int);

        void randomize_weights();

        void update_neigh_size(int);

        void update_weights(const float);

        void list_weights();

        void list_outputs();

        void get_next_vector(FILE *);

        void process_next_pattern();

        float get_win_dist();

        int get_win_index();

};

The Kohonen_layer is derived from the layer class, so it has pointers inherited that point to a set of outputs and a set of inputs. Let's look at some of the functions and member variables.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Kohonen_layer:

• float * weights Pointer to the weights matrix.
• int winner_index Index value of the output, which is the winner.
• float win_distance The Euclidean distance of the winner weight vector from the input vector.
• int neighborhood_size The size of the neighborhood.
• Kohonen_layer(int, int, int) Constructor for the layer: inputs, outputs, and the neighborhood size.
• ~Kohonen_layer() Destructor.
• virtual void calc_out() The function to calculate the outputs; for the Kohonen layer this models lateral competition.
• void randomize_weights() A function to initialize weights with random normal values.
• void update_neigh_size(nt) This function updates the neighborhood size with a new value.
• void update_weights(const float) This function updates the weights according to the training law using the passed parameter, alpha.
• void list_weights() This function can be used to list the weight matrix.
• void list_outputs() This function is used to write outputs to the output file.
• float get_win_dist() Returns the Euclidean distance between the winner weight vector and the input vector.

Kohonen_network:

• layer *layer_ptr[2] Pointer array; element 0 points to the input layer, element 1 points to the Kohonen layer.
• int layer_size[2] Array of layer sizes for the two layers.
• int neighborhood_size The current neighborhood size.
• Kohonen_network() Constructor.
• ~Kohonen_network() Destructor.
• void get_layer_info() Gets information about the layer sizes.
• void set_up_network(int) Connects layers and sets up the Kohonen map.
• void randomize_weights() Creates random normalized weights.
• void update_neigh_size(int) Changes the neighborhood size.
• void update_weights(const float) Performs weight update according to the training law.
• void list_weights() Can be used to list the weight matrix.
• void list_outputs() Can be used to list outputs.
• void get_next_vector(FILE *) Function gets another input vector from the input file.
• void process_next_pattern() Applies pattern to the Kohonen map.
• float get_win_dist() Returns the winner's distance from the input vector.
• int get_win_index() Returns the index of the winner.

Implementation of the Kohonen Layer and Kohonen Network

Listing 11.3 shows the layerk.cpp file, which has the implementation of the functions outlined.

Listing 11.3 Implementation file for the Kohonen layer and Kohonen network :layerk.cpp

// layerk.cpp          V.Rao, H.Rao

// compile for floating point hardware if available

#include "layer.cpp"

#include "layerk.h"

// ————————————————————-

//                          Kohonen layer

//—————————————————————

Kohonen_layer::Kohonen_layer(int i, int o, int

       init_neigh_size)

{

num_inputs=i;

num_outputs=o;

neighborhood_size=init_neigh_size;

weights = new float[num_inputs*num_outputs];

outputs = new float[num_outputs];

}

Kohonen_layer::~Kohonen_layer()

{

delete [num_outputs*num_inputs] weights;

delete [num_outputs] outputs;

}

void Kohonen_layer::calc_out()

{

// implement lateral competition

// choose the output with the largest

// value as the winner; neighboring

// outputs participate in next weight

// update. Winner's output is 1 while

// all other outputs are zero

int i,j,k;

float accumulator=0.0;

float maxval;

winner_index=0;

maxval=-1000000;

for (j=0; j<num_outputs; j++)

        {

        for (i=0; i<num_inputs; i++)

                {

                k=i*num_outputs;

                if (weights[k+j]*weights[k+j] > 1000000.0)

                       {

                       cout << "weights are blowing up\n";

                       cout << "try a smaller learning constant\n";

                       cout << "e.g. beta=0.02

                       aborting...\n";

                       exit(1);

                       }

                outputs[j]=weights[k+j]*(*(inputs+i));

                accumulator+=outputs[j];

                }

        // no squash function

        outputs[j]=accumulator;

        if (outputs[j] > maxval)

                {

                maxval=outputs[j];

                winner_index=j;

                }

                accumulator=0;

                }

        // set winner output to 1

        outputs[winner_index]=1.0;

        // now zero out all other outputs

        for (j=0; j< winner_index; j++)

                outputs[j]=0;

        for (j=num_outputs-1; j>winner_index; j—)

                outputs[j]=0;

}

void Kohonen_layer::randomize_weights()

{

int i, j, k;

const unsigned first_time=1;

const unsigned not_first_time=0;

float discard;

float norm;

discard=randomweight(first_time);

for (i=0; i< num_inputs; i++)

        {

        k=i*num_outputs;

        for (j=0; j< num_outputs; j++)

                {

                weights[k+j]=randomweight(not_first_time);

                }

        }

// now need to normalize the weight vectors

// to unit length

// a weight vector is the set of weights for

// a given output

for (j=0; j< num_outputs; j++)

    {

    norm=0;

          for (i=0; i< num_inputs; i++)

                {

                k=i*num_outputs;

                norm+=weights[k+j]*weights[k+j];

}

        norm = 1/((float)sqrt((double)norm));

for (i=0; i< num_inputs; i++)

        {

        k=i*num_outputs;

        weights[k+j]*=norm;

        }

    }

}

void Kohonen_layer::update_neigh_size(int new_neigh_size)

{

neighborhood_size=new_neigh_size;

}

void Kohonen_layer::update_weights(const float alpha)

{

int i, j, k;

int start_index, stop_index;

// learning law: weight_change =

//             alpha*(input-weight)

//      zero change if input and weight

// vectors are aligned

// only update those outputs that

// are within a neighborhood's distance

// from the last winner

start_index = winner_index -

                      neighborhood_size;

if (start_index < 0)

       start_index =0;

stop_index = winner_index +

                      neighborhood_size;

if (stop_index > num_outputs-1)

       stop_index = num_outputs-1;

for (i=0; i< num_inputs; i++)

        {

        k=i*num_outputs;

        for (j=start_index; j<=stop_index; j++)

               weights[k+j] +=

                      alpha*((*(inputs+i))-weights[k+j]);

    }

}

void Kohonen_layer::list_weights()

{

int i, j, k;

for (i=0; i< num_inputs; i++)

        {

        k=i*num_outputs;

        for (j=0; j< num_outputs; j++)

                cout << "weight["<<i<<","<<

                        j<<"] is: "<<weights[k+j];

        }

}

void Kohonen_layer::list_outputs()

{

int i;

for (i=0; i< num_outputs; i++)

       {

       cout << "outputs["<<i<<

                      "] is: "<<outputs[i];

       }

}

float Kohonen_layer::get_win_dist()

{

int i, j, k;

j=winner_index;

float accumulator=0;

float * win_dist_vec = new float [num_inputs];

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       win_dist_vec[i]=(*(inputs+i))-weights[k+j];

    accumulator+=win_dist_vec[i]*win_dist_vec[i];

    }

win_distance =(float)sqrt((double)accumulator);

delete [num_inputs]win_dist_vec;

return win_distance;

}

Kohonen_network::Kohonen_network()

{

}

Kohonen_network::~Kohonen_network()

{

}

void Kohonen_network::get_layer_info()

{

int i;

//—————————————————————

//

//    Get layer sizes for the Kohonen network

//

// ————————————————————-

cout << " Enter in the layer sizes separated by spaces.\n";

cout << " A Kohonen network has an input layer \n";

cout << " followed by a Kohonen (output) layer \n";

for (i=0; i<2; i++)

        {

        cin >> layer_size[i];

        }

// ———————————————————————————

// size of layers:

//             input_layer       layer_size[0]

//             Kohonen_layer      layer_size[1]

//———————————————————————————-

}

void Kohonen_network::set_up_network(int nsz)

{

int i;

// set up neighborhood size

neighborhood_size = nsz;

//———————————————————————————-

// Construct the layers

//

//———————————————————————————-

layer_ptr[0] = new input_layer(0,layer_size[0]);

layer_ptr[1] =

       new Kohonen_layer(layer_size[0],

                      layer_size[1],neighborhood_size);

for (i=0;i<2;i++)

        {

        if (layer_ptr[i] == 0)

                {

                cout << "insufficient memory\n";

                cout << "use a smaller architecture\n";

                exit(1);

                }

        }

//———————————————————————————-

// Connect the layers

//

//———————————————————————————-

// set inputs to previous layer outputs for the Kohonen layer

layer_ptr[1]->inputs = layer_ptr[0]->outputs;

}

void Kohonen_network::randomize_weights()

{

((Kohonen_layer *)layer_ptr[1])

               ->randomize_weights();

}

void Kohonen_network::update_neigh_size(int n)

{

((Kohonen_layer *)layer_ptr[1])

               ->update_neigh_size(n);

}

void Kohonen_network::update_weights(const float a)

{

((Kohonen_layer *)layer_ptr[1])

               ->update_weights(a);

}

void Kohonen_network::list_weights()

{

((Kohonen_layer *)layer_ptr[1])

               ->list_weights();

}

void Kohonen_network::list_outputs()

{

((Kohonen_layer *)layer_ptr[1])

               ->list_outputs();

}

void Kohonen_network::get_next_vector(FILE * ifile)

{

int i;

float normlength=0;

int num_inputs=layer_ptr[1]->num_inputs;

float *in = layer_ptr[1]->inputs;

// get a vector and normalize it

for (i=0; i<num_inputs; i++)

        {

        fscanf(ifile,"%f",(in+i));

        normlength += (*(in+i))*(*(in+i));

        }

fscanf(ifile,"\n");

normlength = 1/(float)sqrt((double)normlength);

for (i=0; i< num_inputs; i++)

        {

        (*(in+i)) *= normlength;

        }

}

void Kohonen_network::process_next_pattern()

{

        layer_ptr[1]->calc_out();

}

float Kohonen_network::get_win_dist()

{

float  retval;

retval=((Kohonen_layer *)layer_ptr[1])

               ->get_win_dist();

return retval;

}

int Kohonen_network::get_win_index()

{

return ((Kohonen_layer *)layer_ptr[1])

               ->winner_index;

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Flow of the Program and the main() Function

The main() function is contained in a file called kohonen.cpp, which is shown in Listing 11.4. To compile this program, you need only compile and make this main file, kohonen.cpp. Other files are included in this.

Listing 11.4 The main implementation file, kohonen.cpp for the Kohonen Map program

// kohonen.cpp        V. Rao, H. Rao

// Program to simulate a Kohonen map

#include "layerk.cpp"

#define INPUT_FILE "input.dat"

#define OUTPUT_FILE "kohonen.dat"

#define dist_tol   0.05

void main()

{

int neighborhood_size, period;

float avg_dist_per_cycle=0.0;

float dist_last_cycle=0.0;

float avg_dist_per_pattern=100.0; // for the latest cycle

float dist_last_pattern=0.0;

float total_dist;

float alpha;

unsigned startup;

int max_cycles;

int patterns_per_cycle=0;

int total_cycles, total_patterns;

// create a network object

Kohonen_network knet;

FILE * input_file_ptr, * output_file_ptr;

// open input file for reading

if ((input_file_ptr=fopen(INPUT_FILE,"r"))==NULL)

               {

               cout << "problem opening input file\n";

               exit(1);

               }

// open writing file for writing

if ((output_file_ptr=fopen(OUTPUT_FILE,"w"))==NULL)

               {

               cout << "problem opening output file\n";

               exit(1);

               }

// ————————————————————-

//     Read in an initial values for alpha, and the

//  neighborhood size.

//  Both of these parameters are decreased with

//  time. The number of cycles to execute before

//  decreasing the value of these parameters is

//            called the period. Read in a value for the

//            period.

// ————————————————————-

              cout << " Please enter initial values for:\n";

              cout << "alpha (0.01-1.0),\n";

              cout << "and the neighborhood size (integer between 0

              and

              50)\n";

              cout << "separated by spaces, e.g. 0.3 5 \n ";

              cin >> alpha >> neighborhood_size ;

              cout << "\nNow enter the period, which is the\n";

              cout << "number of cycles after which the values\n";

              cout << "for alpha the neighborhood size are

                      decremented\n";

              cout << "choose an integer between 1 and 500 , e.g. 50 \n";

              cin >> period;

              // Read in the maximum number of cycles

              // each pass through the input data file is a cycle

              cout << "\nPlease enter the maximum cycles for the

                      simulation\n";

              cout << "A cycle is one pass through the data set.\n";

              cout << "Try a value of 500 to start with\n\n";

              cin >> max_cycles;

// the main loop

//

//    continue looping until the average distance is less

//            than the tolerance specified at the top of this file

//            , or the maximum number of

//            cycles is exceeded;

// initialize counters

total_cycles=0; // a cycle is once through all the input data

total_patterns=0; // a pattern is one entry in the input data

// get layer information

knet.get_layer_info();

// set up the network connections

knet.set_up_network(neighborhood_size);

// initialize the weights

// randomize weights for the Kohonen layer

// note that the randomize function for the

// Kohonen simulator generates

// weights that are normalized to length = 1

knet.randomize_weights();

// write header to output file

fprintf(output_file_ptr,

       "cycle\tpattern\twin index\tneigh_size\tavg_dist_per_pa

               tern\n");

fprintf(output_file_ptr,

       "———————————————————————————\n");

// main loop

startup=1;

total_dist=0;

while (

                      (avg_dist_per_pattern > dist_tol)

                      && (total_cycles < max_cycles)

                      || (startup==1)

                      )

{

startup=0;

dist_last_cycle=0; // reset for each cycle

patterns_per_cycle=0;

// process all the vectors in the datafile

while (!feof(input_file_ptr))

        {

        knet.get_next_vector(input_file_ptr);

        // now apply it to the Kohonen network

        knet.process_next_pattern();

dist_last_pattern=knet.get_win_dist();

// print result to output file

fprintf(output_file_ptr,"%i\t%i\t%i\t\t%i\t\t%f\n",

     total_cycles,total_patterns,knet.get_win_index(),

     neighborhood_size,avg_dist_per_pattern);

     total_patterns++;

     // gradually reduce the neighborhood size

     // and the gain, alpha

     if (((total_cycles+1) % period) == 0)

             {

             if (neighborhood_size > 0)

                    neighborhood_size —;

             knet.update_neigh_size(neighborhood_size);

             if (alpha>0.1)

                    alpha -= (float)0.1;

             }

     patterns_per_cycle++;

     dist_last_cycle += dist_last_pattern;

     knet.update_weights(alpha);

     dist_last_pattern = 0;

  }

avg_dist_per_pattern= dist_last_cycle/patterns_per_cycle;

total_dist += dist_last_cycle;

total_cycles++;

fseek(input_file_ptr, 0L, SEEK_SET); // reset the file

                              pointer

                              // to the beginning of

                              // the file

} // end main loop

cout << "\n\n\n\n\n\n\n\n\n\n\n";

cout << "—————————————————————————-\n";

cout << "    done \n";

avg_dist_per_cycle= total_dist/total_cycles;

cout << "\n";

cout << "——>average dist per cycle = " << avg_dist_per_cycle

        << " <—-\n";

cout << "——>dist last cycle = " << dist_last_cycle << " <—-

        \n";

cout << "->dist last cycle per pattern= " <<

        avg_dist_per_pattern << " <—-\n";

cout << "——————>total cycles = " << total_cycles << " <—-\n";

cout << "——————>total patterns = " << total_patterns << " <—-

        \n";

cout << "—————————————————————————-\n";

// close the input file

fclose(input_file_ptr);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Flow of the Program

The flow of the program is very similar to the backpropagation simulator. The criterion for ending the simulation in the Kohonen program is the average winner distance. This is a Euclidean distance measure between the input vector and the winner's weight vector. This distance is the square root of the sum of the squares of the differences between individual vector components between the two vectors.

Results from Running the Kohonen Program

Once you compile the program, you need to create an input file to try it. We will first use a very simple input file and examine the results.

A Simple First Example

Let us create an input file, input.dat, which contains only two arbitrary vectors:

0.4 0.98 0.1 0.2
0.5 0.22 0.8 0.9

The file contains two four-dimensional vectors. We expect to see output that contains a different winner neuron for each of these patterns. If this is the case, then the Kohonen map has assigned different categories for each of the input vectors, and, in the future, you can expect to get the same winner classification for vectors that are close to or equal to these vectors.

By running the Kohonen map program, you will see the following output (user input is italic):

Please enter initial values for:

alpha (0.01-1.0),

and the neighborhood size (integer between 0 and 50)

separated by spaces, e.g. 0.3 5

0.3 5

Now enter the period, which is the

number of cycles after which the values

for alpha the neighborhood size are decremented

choose an integer between 1 and 500 , e.g. 50

50

Please enter the maximum cycles for the simulation

A cycle is one pass through the data set.

Try a value of 500 to start with

500

 Enter in the layer sizes separated by spaces.

 A Kohonen network has an input layer

 followed by a Kohonen (output) layer

4 10

——————————————————————————

       done

——>average dist per cycle = 0.544275 <——-

——>dist last cycle = 0.0827523 <——-

->dist last cycle per pattern= 0.0413762 <——-

——————>total cycles = 11 <——-

——————>total patterns = 22 <——-

——————————————————————————

The layer sizes are given as 4 for the input layer and 10 for the Kohonen layer. You should choose the size of the Kohonen layer to be larger than the number of distinct patterns that you think are in the input data set. One of the outputs reported on the screen is the distance for the last cycle per pattern. This value is listed as 0.04, which is less than the terminating value set at the top of the kohonen.cpp file of 0.05. The map converged on a solution. Let us look at the file, kohonen.dat, the output file, to see the mapping to winner indexes:

cycle     pattern     win index     neigh_size    avg_dist_per_pattern

——————————————————————————————————————————————————————————————————————

0         0           1             5             100.000000

0         1           3             5             100.000000

1         2           1             5             0.304285

1         3           3             5             0.304285

2         4           1             5             0.568255

2         5           3             5             0.568255

3         6           1             5             0.542793

3         7           8             5             0.542793

4         8           1             5             0.502416

4         9           8             5             0.502416

5         10          1             5             0.351692

5         11          8             5             0.351692

6         12          1             5             0.246184

6         13          8             5             0.246184

7         14          1             5             0.172329

7         15          8             5             0.172329

8         16          1             5             0.120630

8         17          8             5             0.120630

9         18          1             5             0.084441

9         19          8             5             0.084441

10        20          1             5             0.059109

10        21          8             5             0.059109

In this example, the neighborhood size stays at its initial value of 5. In the first column you see the cycle number, and in the second the pattern number. Since there are two patterns per cycle, you see the cycle number repeated twice for each cycle.

The Kohonen map was able to find two distinct winner neurons for each of the patterns. One has winner index 1 and the other index 8.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Orthogonal Input Vectors Example

For a second example, look at Figure 11.5, where we choose input vectors on a two-dimensional unit circle that are 90° apart. The input.dat file should look like the following:

 1  0

 0  1

-1  0

 0 -1

Figure 11.5 Orthogonal input vectors.

Using the same parameters for the Kohonen network, but with layer sizes of 2 and 10, what result would you expect? The output file, kohonen.dat, follows:

cycle     pattern     win index    neigh_size      avg_dist_per_pattern

———————————————————————————————————————————————————————————————————————

0         0           4            5               100.000000

0         1           0            5               100.000000

0         2           9            5               100.000000

0         3           3            5               100.000000

1         4           4            5               0.444558

1         5           0            5               0.444558

497       1991        6            0               0.707107

498       1992        0            0               0.707107

498       1993        0            0               0.707107

498       1994        6            0               0.707107

498       1995        6            0               0.707107

499       1996        0            0               0.707107

499       1997        0            0               0.707107

499       1998        6            0               0.707107

499       1999        6            0               0.707107

You can see that this example doesn't quite work. Even though the neighborhood size gradually got reduced to zero, the four inputs did not get categorized to different outputs. The winner distance became stuck at the value of 0.707, which is the distance from a vector at 45°. In other words, the map generalizes a little too much, arriving at the middle value for all of the input vectors.

You can fix this problem by starting with a smaller neighborhood size, which provides for less generalization. By using the same parameters and a neighborhood size of 2, the following output is obtained.

cycle     pattern     win index    neigh_size    avg_dist_per_pattern

—————————————————————————————————————————————————————————————————————

0         0           5            2             100.000000

0         1           6            2             100.000000

0         2           4            2             100.000000

0         3           9            2             100.000000

1         4           0            2             0.431695

1         5           6            2             0.431695

1         6           3            2             0.431695

1         7           9            2             0.431695

2         8           0            2             0.504728

2         9           6            2             0.504728

2         10          3            2             0.504728

2         11          9            2             0.504728

3         12          0            2             0.353309

3         13          6            2             0.353309

3         14          3            2             0.353309

3         15          9            2             0.353309

4         16          0            2             0.247317

4         17          6            2             0.247317

4         18          3            2             0.247317

4         19          9            2             0.247317

5         20          0            2             0.173122

5         21          6            2             0.173122

5         22          3            2             0.173122

5         23          9            2             0.173122

6         24          0            2             0.121185

6         25          6            2             0.121185

6         26          3            2             0.121185

6         27          9            2             0.121185

7         28          0            2             0.084830

7         29          6            2             0.084830

7         30          3            2             0.084830

7         31          9            2             0.084830

8         32          0            2             0.059381

8         33          6            2             0.059381

8         34          3            2             0.059381

8         35          9            2             0.059381

For this case, the network quickly converges on a unique winner for each of the four input patterns, and the distance criterion is below the set criterion within eight cycles. You can experiment with other input data sets and combinations of Kohonen network parameters.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Variations and Applications of Kohonen Networks

There are many variations of the Kohonen network. Some of these will be briefly discussed in this section.

Using a Conscience

DeSieno has used a conscience factor in a Kohonen network. For a winning neuron, if the neuron is winning more than a fair share of the time (roughly more than 1/n, where n is the number of neurons), then this neuron has a threshold that is applied temporarily to allow other neurons the chance to win. The purpose of this modification is to allow more uniform weight distribution while learning is taking place.

LVQ: Learning Vector Quantizer

You have read about LVQ (Learning Vector Quantizer) in previous chapters. In light of the Kohonen map, it should be pointed out that the LVQ is simply a supervised version of the Kohonen network. Inputs and expected output categories are presented to the network for training. You get data clustered, just as a Kohonen network, according to the similarity to other data inputs.

Counterpropagation Network

A neural network topology, called a counterpropagation network, is a combination of a Kohonen layer with a Grossberg layer. This network was developed by Robert Hecht-Nielsen and is useful for prototyping of systems, with a fairly rapid training time compared to backpropagation. The Kohonen layer provides for categorization, while the Grossberg layer allows for Hebbian conditioned learning. Counterpropagation has been used successfully in data compression applications for images. Compression ratios of 10:1 to 100:1 have been obtained, using a lossy compression scheme that codes the image with a technique called vector quantization, where the image is broken up into representative subimage vectors. The statistics of these vectors is such that you find that a large part of the image can be adequately represented by a subset of all the vectors. The vectors with the highest frequency of occurrence are coded with the shortest bit strings, hence you achieve data compression.

Application to Speech Recognition

Kohonen created a phonetic typewriter by classifying speech waveforms of different phonemes of Finnish speech into different categories using a Kohonen SOM. The Kohonen phoneme map used 50 samples of each phoneme for calibration. These samples caused excitation in a neighborhood of cells more strongly than in other cells. A neighborhood was labeled with the particular phoneme that caused excitation. For an utterance of speech made to the network, the exact neighborhoods that were active during the utterance were noted, and for how long, and in what sequence. Short excitations were taken as transitory sounds. The information obtained from the network was then pieced together to find out the words in the utterance made to the network.

Summary

In this chapter, you have learned about one of the important types of competitive learning called Kohonen feature map. The most significant points of this discussion are outlined as follows:

• The Kohonen feature map is an example of an unsupervised neural network that is mainly used as a classifier system or data clustering system. As more inputs are presented to this network, the network improves its learning and is able to adapt to changing inputs.
• The training law for the Kohonen network tries to align the weight vectors along the same direction as input vectors.
• The Kohonen network models lateral competition as a form of self-organization. One winner neuron is derived for each input pattern to categorize that input.
• Only neurons within a certain distance (neighborhood) from the winner are allowed to participate in training for a given input pattern.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 12
Application to Pattern Recognition

Using the Kohonen Feature Map

In this chapter, you will use the Kohonen program developed in Chapter 11 to recognize patterns. You will modify the Kohonen program for the display of patterns.

An Example Problem: Character Recognition

The problem that is presented in this chapter is to recognize or categorize alphabetic characters. You will input various alphabetic characters to a Kohonen map and train the network to recognize these as separate categories. This program can be used to try other experiments that will be discussed at the end of this chapter.

Representing Characters

Each character is represented by a 5×7 grid of pixels. We use the graphical printing characters of the IBM extended ASCII character set to show a grayscale output for each pixel. To represent the letter A, for example, you could use the pattern shown in Figure 12.1. Here the blackened boxes represent value 1, while empty boxes represent a zero. You can represent all characters this way, with a binary map of 35 pixel values.

Figure 12.1 Representation of the letter A with a 5×7 pattern.

The letter A is represented by the values:

0 0 1 0 0

0 1 0 1 0

1 0 0 0 1

1 0 0 0 1

1 1 1 1 1

1 0 0 0 1

1 0 0 0 1

For use in the Kohonen program, we need to serialize the rows, so that all entries appear on one line.

For the characters A and X you would end up with the following entries in the input file, input.dat:

0 0 1 0 0  0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

 << the letter A

1 0 0 0 1  0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0  1 0 0 0 1

 << the letter X

Monitoring the Weights

We will present the Kohonen map with many such characters and find the response in output. You will be able to watch the Kohonen map as it goes through its cycles and learns the input patterns. At the same time, you should be able to watch the weight vectors for the winner neurons to see the pattern that is developing in the weights. Remember that for a Kohonen map the weight vectors tend to become aligned with the input vectors. So after a while, you will notice that the weight vector for the input will resemble the input pattern that you are categorizing.

Representing the Weight Vector

Although on and off values are fine for the input vectors mentioned, you need to see grayscale values for the weight vector. This can be accomplished by quantizing the weight vector into four bins, each represented by a different ASCII graphic character, as shown in Table 12.1.

Table 12.1 Quantizing the Weight Vector
<= 0 | White rectangle (space)
0 < weight <= 0.25 | Light-dotted rectangle
0.25 < weight <= 0.50 | Medium-dotted rectangle
0.50 < weight <= 0.75 | Dark-dotted rectangle
weight > 0.75 | Black rectangle

The ASCII values for the graphics characters to be used are listed in Table 12.2.

Table 12.2 ASCII Values for Rectangle Graphic Characters
White rectangle | 255
Light-dotted rectangle | 176
Medium-dotted rectangle | 177
Dark-dotted rectangle | 178
Black rectangle | 219

C++ Code Development

The changes to the Kohonen program are relatively minor. The following listing indicates these changes.

Changes to the Kohonen Program

The first change to make is to the Kohonen_network class definition. This is in the file, layerk.h, shown in Listing 12.1.

Listing 12.1 Updated layerk.h file

class Kohonen_network

{

private:

       layer *layer_ptr[2];

       int layer_size[2];

       int neighborhood_size;

public:

       Kohonen_network();

       ~Kohonen_network();

       void get_layer_info();

       void set_up_network(int);

       void randomize_weights();

       void update_neigh_size(int);

       void update_weights(const float);

       void list_weights();

       void list_outputs();

       void get_next_vector(FILE *);

       void process_next_pattern();

       float get_win_dist();

       int get_win_index();

       void display_input_char();

       void display_winner_weights();

};


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The new member functions are shown in italic. The functions display_input_char() and display_winner_weights() are used to display the input and weight maps on the screen to watch weight character map converge to the input map.

The implementation of these functions is in the file, layerk.cpp. The portion of this file containing these functions is shown in Listing 12.2.

Listing 12.2 Additions to the layerk.cpp implementation file

void Kohonen_network::display_input_char()

{

int i, num_inputs;

unsigned char ch;

float temp;

int col=0;

float * inputptr;

num_inputs=layer_ptr[1]->num_inputs;

inputptr = layer_ptr[1]->inputs;

// we've got a 5x7 character to display

for (i=0; i<num_inputs; i++)

       {

       temp = *(inputptr);

       if (temp <= 0)

              ch=255;// blank

       else if ((temp > 0) && (temp <= 0.25))

              ch=176; // dotted rectangle -light

       else if ((temp > 0.25) && (temp <= 0.50))

              ch=177; // dotted rectangle -medium

       else if ((temp >0.50) && (temp <= 0.75))

              ch=178; // dotted rectangle -dark

       else if (temp > 0.75)

              ch=219; // filled rectangle

       printf("%c",ch); //fill a row

       col++;

       if ((col % 5)==0)

              printf("\n"); // new row

       inputptr++;

       }

printf("\n\n\n");

}

void Kohonen_network::display_winner_weights()

{

int i, k;

unsigned char ch;

float temp;

float * wmat;

int col=0;

int win_index;

int num_inputs, num_outputs;

num_inputs= layer_ptr[1]->num_inputs;

wmat = ((Kohonen_layer*)layer_ptr[1])

              ->weights;

win_index=((Kohonen_layer*)layer_ptr[1])

              ->winner_index;

num_outputs=layer_ptr[1]->num_outputs;

// we've got a 5x7 character to display

for (i=0; i<num_inputs; i++)

       {

       k= i*num_outputs;

       temp = wmat[k+win_index];

       if (temp <= 0)

              ch=255;// blank

       else if ((temp > 0) && (temp <= 0.25))

              ch=176; // dotted rectangle -light

       else if ((temp > 0.25) && (temp <= 0.50))

              ch=177; // dotted rectangle -medium

       else if ((temp > 0.50) && (temp <= 0.75))

              ch=178; // dotted rectangle -dark

       else if (temp > 0.75)

              ch=219; // filled rectangle

       printf("%c",ch); //fill a row

       col++;

       if ((col % 5)==0)

              printf("\n"); // new row

       }

printf("\n\n");

printf("—————————-\n");

}

The final change to make is to the kohonen.cpp file. The new file is called pattern.cpp and is shown in Listing 12.3.

Listing 12.3 The implementation file pattern.cpp

// pattern.cpp      V. Rao, H. Rao

// Kohonen map for pattern recognition

#include "layerk.cpp"

#define INPUT_FILE "input.dat"

#define OUTPUT_FILE "kohonen.dat"

#define dist_tol      0.001

#define wait_cycles   10000 // creates a pause to

                      // view the character maps

void main()

{

int neighborhood_size, period;

float avg_dist_per_cycle=0.0;

float dist_last_cycle=0.0;

float avg_dist_per_pattern=100.0; // for the latest cycle

float dist_last_pattern=0.0;

float total_dist;

float alpha;

unsigned startup;

int max_cycles;

int patterns_per_cycle=0;

int total_cycles, total_patterns;

int i;

// create a network object

Kohonen_network knet;

FILE * input_file_ptr, * output_file_ptr;

// open input file for reading

if ((input_file_ptr=fopen(INPUT_FILE,"r"))==NULL)

              {

              cout << "problem opening input file\n";

              exit(1);

              }

// open writing file for writing

if ((output_file_ptr=fopen(OUTPUT_FILE,"w"))==NULL)

              {

              cout << "problem opening output file\n";

              exit(1);

              }

// ————————————————————-

//     Read in an initial values for alpha, and the

//  neighborhood size.

//  Both of these parameters are decreased with

//  time. The number of cycles to execute before

//  decreasing the value of these parameters is

//            called the period. Read in a value for the

//            period.

// ————————————————————-

              cout << " Please enter initial values for:\n";

              cout << "alpha (0.01-1.0),\n";

              cout << "and the neighborhood size (integer between 0\

                     and 50)\n";

              cout << "separated by spaces, e.g. 0.3 5 \n ";

              cin >> alpha >> neighborhood_size ;

              cout << "\nNow enter the period, which is the\n";

              cout << "number of cycles after which the values\n";

              cout << "for alpha the neighborhood size are \

                     decremented\n";

              cout << "choose an integer between 1 and 500 , e.g. \ 50 \n";

              cin >> period;

       //     Read in the maximum number of cycles

       //     each pass through the input data file is a cycle

              cout << "\nPlease enter the maximum cycles for the

              simulation\n";

              cout << "A cycle is one pass through the data set.\n";

              cout << "Try a value of 500 to start with\n\n";

              cin >> max_cycles;

// the main loop

//

//     continue looping until the average distance is less than

//            the tolerance specified at the top of this file

//            , or the maximum number of

//            cycles is exceeded;

// initialize counters

total_cycles=0; // a cycle is once through all the input data

total_patterns=0; // a pattern is one entry in the input data

// get layer information

knet.get_layer_info();

// set up the network connections

knet.set_up_network(neighborhood_size);

// initialize the weights

// randomize weights for the Kohonen layer

// note that the randomize function for the

// Kohonen simulator generates

// weights that are normalized to length = 1

knet.randomize_weights();

// write header to output file

fprintf(output_file_ptr,

       "cycle\tpattern\twin index\tneigh_size\\

              tavg_dist_per_pattern\n");

fprintf(output_file_ptr,

       "————————————————————————\n");

startup=1;

total_dist=0;

while (

                     (avg_dist_per_pattern > dist_tol)

                     && (total_cycles < max_cycles)

                     || (startup==1)

                     )

{

startup=0;

dist_last_cycle=0; // reset for each cycle

patterns_per_cycle=0;

// process all the vectors in the datafile

while (!feof(input_file_ptr))

       {

       knet.get_next_vector(input_file_ptr);

       // now apply it to the Kohonen network

       knet.process_next_pattern();

  dist_last_pattern=knet.get_win_dist();

  // print result to output file

  fprintf(output_file_ptr,"%i\t%i\t%i\t\t%i\t\t%f\n",

       total_cycles,total_patterns,knet.get_win_index(),

       neighborhood_size,avg_dist_per_pattern);

       // display the input character and the

       // weights for the winner to see match

       knet.display_input_char();

       knet.display_winner_weights();

       // pause for a while to view the

       // character maps

       for (i=0; i<wait_cycles; i++)

              {;}

       total_patterns++;

       // gradually reduce the neighborhood size

       // and the gain, alpha

       if (((total_cycles+1) % period) == 0)

              {

              if (neighborhood_size > 0)

                     neighborhood_size —;

              knet.update_neigh_size(neighborhood_size);

              if (alpha>0.1)

                     alpha -= (float)0.1;

              }

       patterns_per_cycle++;

       dist_last_cycle += dist_last_pattern;

       knet.update_weights(alpha);

       dist_last_pattern = 0;

    }

avg_dist_per_pattern= dist_last_cycle/patterns_per_cycle;

total_dist += dist_last_cycle;

total_cycles++;

fseek(input_file_ptr, 0L, SEEK_SET); // reset the file

       pointer

                            // to the beginning of

                            // the file

} // end main loop

cout << "\n\n\n\n\n\n\n\n\n\n\n";

cout << "———————————————————————\n";

cout << "    done \n";

avg_dist_per_cycle= total_dist/total_cycles;

cout << "\n";

cout << "——>average dist per cycle = " << avg_dist_per_cycle << " <—-\n";

cout << ">dist last cycle = " << dist_last_cycle << " <   \n";

cout << "->dist last cycle per pattern= " <<

       avg_dist_per_pattern << " <—-\n";

cout << "——->total cycles = " << total_cycles << " <—-\n";

cout << "——————>total patterns = " <<

       total_patterns << " <—-\n";

cout << "————————————————————————\n";

// close the input file

fclose(input_file_ptr);

}

Changes to the program are indicated in italic. Compile this program by compiling and making the pattern.cpp file, after modifying the layerk.cpp and layerk.h files, as indicated previously.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Testing the Program

Let us run the example that we have created an input file for. We have an input.dat file with the characters A and X defined. A run of the program with these inputs is shown as follows:

Please enter initial values for:

alpha (0.01-1.0),

and the neighborhood size (integer between 0 and 50)

separated by spaces, e.g., 0.3 5

0.3 5

Now enter the period, which is the

number of cycles after which the values

for alpha the neighborhood size are decremented

choose an integer between 1 and 500, e.g., 50

50

Please enter the maximum cycles for the simulation

A cycle is one pass through the data set.

Try a value of 500 to start with

500

Enter in the layer sizes separated by spaces.

A Kohonen network has an input layer

followed by a Kohonen (output) layer

35 100

The output of the program is contained in file kohonen.dat as usual. This shows the following result.

cycle   pattern    win index   neigh_size    avg_dist_per_pattern

———————————————————————————————————————————————————————————————————

0       0          42          5             100.000000

0       1          47          5             100.000000

1       2          42          5             0.508321

1       3          47          5             0.508321

2       4          40          5             0.742254

2       5          47          5             0.742254

3       6          40          5             0.560121

3       7          47          5             0.560121

4       8          40          5             0.392084

4       9          47          5             0.392084

5       10         40          5             0.274459

5       11         47          5             0.274459

6       12         40          5             0.192121

6       13         47          5             0.192121

7       14         40          5             0.134485

7       15         47          5             0.134485

8       16         40          5             0.094139

8       17         47          5             0.094139

9       18         40          5             0.065898

9       19         47          5             0.065898

10      20         40          5             0.046128

10      21         47          5             0.046128

11      22         40          5             0.032290

11      23         47          5             0.032290

12      24         40          5             0.022603

12      25         47          5             0.022603

13      26         40          5             0.015822

13      27         47          5             0.015822

14      28         40          5             0.011075

14      29         47          5             0.011075

15      30         40          5             0.007753

15      31         47          5             0.007753

16      32         40          5             0.005427

16      33         47          5             0.005427

17      34         40          5             0.003799

17      35         47          5             0.003799

18      36         40          5             0.002659

18      37         47          5             0.002659

19      38         40          5             0.001861

19      39         47          5             0.001861

20      40         40          5             0.001303

20      41         47          5             0.001303

The tolerance for the distance was set to be 0.001 for this program, and the program was able to converge to this value. Both of the inputs were successfully classified into two different winning output neurons. In Figures 12.2 and 12.3 you see two snapshots of the input and weight vectors that you will find with this program. The weight vector resembles the input as you can see, but it is not an exact replication.

Figure 12.2 Sample screen output of the letter A from the input and weight vectors.

Figure 12.3 Sample screen output of the letter X from the input and weight vectors.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Generalization versus Memorization

As mentioned in Chapter 11, you actually don't desire the exact replication of the input pattern for the weight vector. This would amount to memorizing of the input patterns with no capacity for generalization.

For example, a typical use of this alphabet classifier system would be to use it to process noisy data, like handwritten characters. In such a case, you would need a great deal of latitude in scoping a class for a letter A.

Adding Characters

The next step of the program is to add characters and see what categories they end up in. There are many alphabetic characters that look alike, such as H and B for example. You can expect the Kohonen classifier to group these like characters into the same class.

We now modify the input.dat file to add the characters H, B, and I. The new input.dat file is shown as follows.

0 0 1 0 0   0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1

 1 0 0 0 1

1 0 0 0 1   0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0

 1 0 0 0 1

1 0 0 0 1   1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

 1 0 0 0 1

1 1 1 1 1   1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

 1 1 1 1 1

0 0 1 0 0   0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

 0 0 1 0 0

The output using this input file is shown as follows.

—————————————————————————-

       done

——>average dist per cycle = 0.732607 <—-

——>dist last cycle = 0.00360096 <—-

->dist last cycle per pattern= 0.000720192 <—-

——————>total cycles = 37 <—-

——————>total patterns = 185 <—-

—————————————————————————-

The file kohonen.dat with the output values is now shown as follows.

cycle   pattern    win index   neigh_size    avg_dist_per_pattern

—————————————————————————————————————————————————————————————————

0       0          69          5             100.000000

0       1          93          5             100.000000

0       2          18          5             100.000000

0       3          18          5             100.000000

0       4          78          5             100.000000

1       5          69          5             0.806743

1       6          93          5             0.806743

1       7          18          5             0.806743

1       8          18          5             0.806743

1       9          78          5             0.806743

2       10         69          5             0.669678

2       11         93          5             0.669678

2       12         18          5             0.669678

2       13         18          5             0.669678

2       14         78          5             0.669678

3       15         69          5             0.469631

3       16         93          5             0.469631

3       17         18          5             0.469631

3       18         18          5             0.469631

3       19         78          5             0.469631

4       20         69          5             0.354791

4       21         93          5             0.354791

4       22         18          5             0.354791

4       23         18          5             0.354791

4       24         78          5             0.354791

5       25         69          5             0.282990

5       26         93          5             0.282990

5       27         18          5             0.282990

...

35      179        78          5             0.001470

36      180        69          5             0.001029

36      181        93          5             0.001029

36      182        13          5             0.001029

36      183        19          5             0.001029

36      184        78          5             0.001029

Again, the network does not find a problem in classifying these vectors.

Until cycle 21, both the H and the B were classified as output neuron 18. The ability to distinguish these vectors is largely due to the small tolerance we have assigned as a termination criterion.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Other Experiments to Try

You can try other experiments with the program. For example, you can repeat the input file but with the order of the entries changed. In other words, you can present the same inputs a number of times in different order. This actually helps the Kohonen network train faster. You can try applying garbled versions of the characters to see if the network distinguishes them. Just as in the backpropagation program, you can save the weights in a weight file to freeze the state of training, and then apply new inputs. You can enter all of the characters from A to Z and see the classification that results. Do you need to train on all of the characters or a subset? You can change the size of the Kohonen layer. How many neurons do you need to recognize the complete alphabet?

There is no restriction on using digital inputs of 1 and 0 as we had used. You can apply grayscale analog values. The program will display the input pattern according to the quantization levels that were set. This set can be expanded, and you can use a graphics interface to display more levels. You can then try pattern recognition of arbitrary images, but remember that processing time will increase rapidly with the number of neurons used. The number of input neurons you choose is dictated by the image resolution, unless you filter and/or subsample the image before presenting it to the network. Filtering is the process of using a type of averaging function applied to groups of pixels. Subsampling is the process of choosing a lower-output image resolution by selecting fewer pixels than a source image. If you start with an image that is 100 × 100 pixels, you can subsample this image 2:1 in each direction to obtain an image that is one-fourth the size, or 50 × 50 pixels. Whether you throw away every other pixel to get this output resolution or apply a filter is up to you. You could average every two pixels to get one output pixel as an example of a very simple filter.

Summary

The following list highlights the important Kohonen program features which you learned in this chapter.

• This chapter presented a simple character recognition program using a Kohonen feature map.
• The input vectors and the weight vectors were displayed to show convergence and note similarity between the two vectors.
• As training progresses, the weight vector for the winner neuron resembles the input character map.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 13
Backpropagation II

Enhancing the Simulator

In Chapter 7, you developed a backpropagation simulator. In this chapter, you will put it to use with examples and also add some new features to the simulator: a term called momentum, and the capability of adding noise to the inputs during simulation. There are many variations of the algorithm that try to alleviate two problems with backpropagation. First, like other neural networks, there is a strong possibility that the solution found with backpropagation is not a global error minimum, but a local one. You may need to shake the weights a little by some means to get out of the local minimum, and possibly arrive at a lower minimum. The second problem with backpropagation is speed. The algorithm is very slow at learning. There are many proposals for speeding up the search process. Neural networks are inherently parallel processing architectures and are suited for simulation on parallel processing hardware. While there are a few plug-in neural net or digital signal processing boards available in the market, the low-cost simulation platform of choice remains the personal computer. Speed enhancements to the training algorithm are therefore very buffernecessary.

Another Example of Using Backpropagation

Before modifying the simulator to add features, let's look at the same problem we used the Kohonen map to analyze in Chapter 12. As you recall, we would like to be able to distinguish alphabetic characters by assigning them to different bins. For backpropagation, we would apply the inputs and train the network with anticipated responses. Here is the input file that we used for distinguishing five different characters, A, X, H, B, and I:

0 0 1 0 0  0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1

1 0 0 0 1  0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0

1 0 0 0 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

1 1 1 1 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

1 0 0 0 1

1 0 0 0 1

1 0 0 0 1

1 1 1 1 1

0 0 1 0 0

Each line has a 5x7 dot representation of each character. Now we need to name each of the output categories. We can assign a simple 3-bit representation as follows:

A | 000
X | 010
H | 100
B | 101
I | 111

Let's train the network to recognize these characters. The training.dat file looks like the following.

0 0 1 0 0  0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1

1 0 0 0 1  0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0

1 0 0 0 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

1 1 1 1 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

1 0 0 0 1  0 0 0

1 0 0 0 1  0 1 0

1 0 0 0 1  1 0 0

1 1 1 1 1  1 0 1

0 0 1 0 0  1 1 1

Now you can start the simulator. Using the parameters (beta = 0.1, tolerance = 0.001, and max_cycles = 1000) and with three layers of size 35 (input), 5 (middle), and 3 (output), you will get a typical result like the following.

-------------------------- -

         done:   results in file output.dat

                        training: last vector only

                        not training: full cycle

                        weights saved in file weights.dat

-->average error per cycle = 0.035713<--

-->error last cycle = 0.008223 <--

->error last cycle per pattern= 0.00164455 <--

------>total cycles = 1000 <--

------>total patterns = 5000 <--

---------------------------

The simulator stopped at the 1000 maximum cycles specified in this case. Your results will be different since the weights start at a random point. Note that the tolerance specified was nearly met. Let us see how close the output came to what we wanted. Look at the output.dat file. You can see the match for the last pattern as follows:

for input vector:

0.000000  0.000000  1.000000  0.000000  0.000000  0.000000  0.000000

1.000000  0.000000  0.000000  0.000000  0.000000  1.000000  0.000000

0.000000  0.000000  0.000000  1.000000  0.000000  0.000000  0.000000

0.000000  1.000000  0.000000  0.000000  0.000000  0.000000  1.000000

0.000000  0.000000  0.000000  0.000000  1.000000  0.000000  0.000000

output vector is:

0.999637  0.998721  0.999330

expected output vector is:

1.000000  1.000000  1.000000

-----------


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

To see the outputs of all the patterns, we need to copy the training.dat file to the test.dat file and rerun the simulator in Test mode. Remember to delete the expected output field once you copy the file.

Running the simulator in Test mode (0) shows the following result in the output.dat file:

for input vector:

0.000000  0.000000  1.000000  0.000000  0.000000  0.000000  1.000000

0.000000  1.000000  0.000000  1.000000  0.000000  0.000000  0.000000

1.000000  1.000000  0.000000  0.000000  0.000000  1.000000  1.000000

1.000000  1.000000  1.000000  1.000000  1.000000  0.000000  0.000000

0.000000  1.000000  1.000000  0.000000  0.000000  0.000000  1.000000

output vector is:

0.005010  0.002405  0.000141

-----------

for input vector:

1.000000  0.000000  0.000000  0.000000  1.000000  0.000000  1.000000

0.000000  1.000000  0.000000  0.000000  0.000000  1.000000  0.000000

0.000000  0.000000  0.000000  1.000000  0.000000  0.000000  0.000000

0.000000  1.000000  0.000000  0.000000  0.000000  1.000000  0.000000

1.000000  0.000000  1.000000  0.000000  0.000000  0.000000  1.000000

output vector is:

0.001230  0.997844  0.000663

-----------

for input vector:

1.000000  0.000000  0.000000  0.000000  1.000000  1.000000  0.000000

0.000000  0.000000  1.000000  1.000000  0.000000  0.000000  0.000000

1.000000  1.000000  1.000000  1.000000  1.000000  1.000000  1.000000

0.000000  0.000000  0.000000  1.000000  1.000000  0.000000  0.000000

0.000000  1.000000  1.000000  0.000000  0.000000  0.000000  1.000000

output vector is:

0.995348  0.000253  0.002677

-----------

for input vector:

1.000000  1.000000  1.000000  1.000000  1.000000  1.000000  0.000000

0.000000  0.000000  1.000000  1.000000  0.000000  0.000000  0.000000

1.000000  1.000000  1.000000  1.000000  1.000000  1.000000  1.000000

0.000000  0.000000  0.000000  1.000000  1.000000  0.000000  0.000000

0.000000  1.000000  1.000000  1.000000  1.000000  1.000000  1.000000

output vector is:

0.999966  0.000982  0.997594

-----------

for input vector:

0.000000  0.000000  1.000000  0.000000  0.000000  0.000000  0.000000

1.000000  0.000000  0.000000  0.000000  0.000000  1.000000  0.000000

0.000000  0.000000  0.000000  1.000000  0.000000  0.000000  0.000000

0.000000  1.000000  0.000000  0.000000  0.000000  0.000000  1.000000

0.000000  0.000000  0.000000  0.000000  1.000000  0.000000  0.000000

output vector is:

0.999637  0.998721  0.999330

-----------

The training patterns are learned very well. If a smaller tolerance is used, it would be possible to complete the learning in fewer cycles. What happens if we present a foreign character to the network? Let us create a new test.dat file with two entries for the letters M and J, as follows:

1 0 0 0 1  1 1 0 1 1  1 0 1 0 1  1 0 0 0 1  1 0 0 0 1  1 0 0 0 1

0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

1 0 0 0 1

0 1 1 1 1

The results should show each foreign character in the category closest to it. The middle layer of the network acts as a feature detector. Since we specified five neurons, we have given the network the freedom to define five features in the input training set to use to categorize inputs. The results in the output.dat file are shown as follows.

for input vector:

1.000000  0.000000  0.000000  0.000000  1.000000  1.000000  1.000000

0.000000  1.000000  1.000000  1.000000  0.000000  1.000000  0.000000

1.000000  1.000000  0.000000  0.000000  0.000000  1.000000  1.000000

0.000000  0.000000  0.000000  1.000000  1.000000  0.000000  0.000000

0.000000  1.000000  1.000000  0.000000  0.000000  0.000000  1.000000

output vector is:

0.963513  0.000800  0.001231

-----------

for input vector:

0.000000  0.000000  1.000000  0.000000  0.000000  0.000000  0.000000

1.000000  0.000000  0.000000  0.000000  0.000000  1.000000  0.000000

0.000000  0.000000  0.000000  1.000000  0.000000  0.000000  0.000000

0.000000  1.000000  0.000000  0.000000  0.000000  0.000000  1.000000

0.000000  0.000000  0.000000  1.000000  1.000000  1.000000  1.000000

output vector is:

0.999469  0.996339  0.999157

-----------

In the first pattern, an M is categorized as an H, whereas in the second pattern, a J is categorized as an I, as expected. The case of the first pattern seems reasonable since the H and M share many pixels in common.

Other Experiments to Try

There are many other experiments you could try in order to get a better feel for how to train and use a backpropagation neural network.

• You could use the ASCII 8-bit code to represent each character, and try to train the network. You could also code all of the alphabetic characters and see if it's possible to distinguish all of them.
• You can garble a character, to see if you still get the correct output.
• You could try changing the size of the middle layer, and see the effect on training time and generalization ability.
• You could change the tolerance setting to see the difference between an overtrained and undertrained network in generalization capability. That is, given a foreign pattern, is the network able to find the closest match and use that particular category, or does it arrive at a new category altogether?

We will return to the same example after enhancing the simulator with momentum and noise addition capability.

Adding the Momentum Term

A simple change to the training law that sometimes results in much faster training is the addition of a momentum term. The training law for backpropagation as implemented in the simulator is:

Weight change = Beta * output_error * input

Now we add a term to the weight change equation as follows:

Weight change = Beta * output_error * input +

                Alpha*previous_weight_change

The second term in this equation is the momentum term. The weight change, in the absence of error, would be a constant multiple by the previous weight change. In other words, the weight change continues in the direction it was heading. The momentum term is an attempt to try to keep the weight change process moving, and thereby not get stuck in local minimas.

Code Changes

The effected files to implement this change are the layer.cpp file, to modify the update_weights() member function of the output_layer class, and the main backprop.cpp file to read in the value for alpha and pass it to the member function. There is some additional storage needed for storing previous weight changes, and this affects the layer.h file. The momentum term could be implemented in two ways:

1. Using the weight change for the previous pattern.
2. Using the weight change accumulated over the previous cycle.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Although both of these implementations are valid, the second is particularly useful, since it adds a term that is significant for all patterns, and hence would contribute to global error reduction. We implement the second choice by accumulating the value of the current cycle weight changes in a vector called cum_deltas. The past cycle weight changes are stored in a vector called past_deltas. These are shown as follows in a portion of the layer.h file.

class output_layer:   public layer

{

protected:

       float * weights;

       float * output_errors; // array of errors at output

       float * back_errors; // array of errors back-propagated

       float * expected_values;      // to inputs

       float * cum_deltas;   // for momentum

       float * past_deltas;  // for momentum

  friend network;

...

Changes to the layer.cpp File

The implementation file for the layer class changes in the output_layer::update_weights() routine and the constructor and destructor for output_layer. First, here is the constructor for output_layer. Changes are highlighted in italic.

output_layer::output_layer(int ins, int outs)

{

int i, j, k;

num_inputs=ins;

num_outputs=outs;

weights = new float[num_inputs*num_outputs];

output_errors = new float[num_outputs];

back_errors = new float[num_inputs];

outputs = new float[num_outputs];

expected_values = new float[num_outputs];

cum_deltas = new float[num_inputs*num_outputs];

past_deltas = new float[num_inputs*num_outputs];

if ((weights==0)||(output_errors==0)||(back_errors==0)

       ||(outputs==0)||(expected_values==0)

       ||(past_deltas==0)||(cum_deltas==0))

       {

       cout << "not enough memory\n";

       cout << "choose a smaller architecture\n";

       exit(1);

       }

// zero cum_deltas and past_deltas matrix

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              {

              cum_deltas[k+j]=0;

              past_deltas[k+j=0;

              }

       }

}

The destructor simply deletes the new vectors:

output_layer::~output_layer()

{

// some compilers may require the array

// size in the delete statement; those

// conforming to Ansi C++ will not

delete [num_outputs*num_inputs] weights;

delete [num_outputs] output_errors;

delete [num_inputs] back_errors;

delete [num_outputs] outputs;

delete [num_outputs*num_inputs] past_deltas;

delete [num_outputs*num_inputs] cum_deltas;

}

Now let's look at the update_weights() routine changes:

void output_layer::update_weights(const float beta,

                                     const float alpha)

{

int i, j, k;

float delta;

// learning law: weight_change =

//             beta*output_error*input + alpha*past_delta

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              {

              delta=beta*output_errors[j]*(*(inputs+i))

              +alpha*past_deltas[k+j];

              weights[k+j] += delta;

              cum_deltas[k+j]+=delta; // current cycle

              }

       }

}

The change to the training law amounts to calculating a delta and adding it to the cumulative total of weight changes in cum_deltas. At some point (at the start of a new cycle) you need to set the past_deltas vector to the cum_delta vector. Where does this occur? Since the layer has no concept of cycle, this must be done at the network level. There is a network level function called update_momentum at the beginning of each cycle that in turns calls a layer level function of the same name. The layer level function swaps the past_deltas vector and the cum_deltas vector, and reinitializes the cum_deltas vector to zero. We need to return to the layer.h file to see changes that are needed to define the two functions mentioned.

class output_layer:   public layer

{

protected:

       float * weights;

       float * output_errors; // array of errors at output

       float * back_errors; // array of errors back-propagated

       float * expected_values;     // to inputs

       float * cum_deltas;   // for momentum

       float * past_deltas;   // for momentum

  friend network;

public:

       output_layer(int, int);

       ~output_layer();

       virtual void calc_out();

       void calc_error(float &);

       void randomize_weights();

       void update_weights(const float, const float);

       void update_momentum();

       void list_weights();

       void write_weights(int, FILE *);

       void read_weights(int, FILE *);

       void list_errors();

       void list_outputs();

};

class network

{

private:

layer *layer_ptr[MAX_LAYERS];

    int number_of_layers;

    int layer_size[MAX_LAYERS];

    float *buffer;

    fpos_t position;

    unsigned training;

public:

  network();

    ~network();

               void set_training(const unsigned &);

               unsigned get_training_value();

               void get_layer_info();

               void set_up_network();

               void randomize_weights();

               void update_weights(const float, const float);

               void update_momentum();

               ...

At both the network and output_layer class levels the function prototype for the update_momentum member functions are highlighted. The implementation for these functions are shown as follows from the layer.cpp class.

void output_layer::update_momentum()

{

// This function is called when a

// new cycle begins; the past_deltas

// pointer is swapped with the

// cum_deltas pointer. Then the contents

// pointed to by the cum_deltas pointer

// is zeroed out.

int i, j, k;

float * temp;

// swap

temp = past_deltas;

past_deltas=cum_deltas;

cum_deltas=temp;

// zero cum_deltas matrix

// for new cycle

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              cum_deltas[k+j]=0;

       }

}

void network::update_momentum()

{

int i;

for (i=1; i<number_of_layers; i++)

       ((output_layer *)layer_ptr[i])

               ->update_momentum();

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Adding Noise During Training

Another approach to breaking out of local minima as well as to enhance generalization ability is to introduce some noise in the inputs during training. A random number is added to each input component of the input vector as it is applied to the network. This is scaled by an overall noise factor, NF, which has a 0 to 1 range. You can add as much noise to the simulation as you want, or not any at all, by choosing NF = 0. When you are close to a solution and have reached a satisfactory minimum, you don't want noise at that time to interfere with convergence to the minimum. We implement a noise factor that decreases with the number of cycles, as shown in the following excerpt from the backprop.cpp file.

// update NF

// gradually reduce noise to zero

if (total_cycles>0.7*max_cycles)

                     new_NF = 0;

else if (total_cycles>0.5*max_cycles)

                     new_NF = 0.25*NF;

else if (total_cycles>0.3*max_cycles)

                     new_NF = 0.50*NF;

else if (total_cycles>0.1*max_cycles)

                     new_NF = 0.75*NF;

backp.set_NF(new_NF);

The noise factor is reduced at regular intervals. The new noise factor is updated with the network class function called set_NF(float). There is a member variable in the network class called NF that holds the current value for the noise factor. The noise is added to the inputs in the input_layer member function calc_out().

Another reason for using noise is to prevent memorization by the network. You are effectively presenting a different input pattern with each cycle so it becomes hard for the network to memorize patterns.

One Other Change—Starting Training from a Saved Weight File

Shortly, we will look at the complete listings for the backpropagation simulator. There is one other enhancement to discuss. It is often useful in long simulations to be able to start from a known point, which is from an already saved set of weights. This is a simple change in the backprop.cpp program, which is well worth the effort. As a side benefit, this feature will allow you to run a simulation with a large beta value for, say, 500 cycles, save the weights, and then start a new simulation with a smaller beta value for another 500 or more cycles. You can take preset breaks in long simulations, which you will encounter in Chapter 14. At this point, let's look at the complete listings for the updated layer.h and layer.cpp files in Listings 13.1 and 13.2:

Listing 13.1 layer.h file updated to include noise and momentum

// layer.h            V.Rao, H. Rao

// header file for the layer class hierarchy and

// the network class

 // added noise and momentum

#define MAX_LAYERS    5

#define MAX_VECTORS   100

class network;

class Kohonen_network;

class layer

{

protected:

       int num_inputs;

       int num_outputs;

       float *outputs; // pointer to array of outputs

       float *inputs;  // pointer to array of inputs, which

                                      // are outputs of some other layer

       friend network;

       friend Kohonen_network;  // update for Kohonen model

public:

       virtual void calc_out()=0;

};

class input_layer: public layer

{

private:

float noise_factor;

float * orig_outputs;

public:

       input_layer(int, int);

       ~input_layer();

       virtual void calc_out();

       void set_NF(float);

       friend network;

};

class middle_layer;

class output_layer:  public layer

{

protected:

       float * weights;

       float * output_errors; // array of errors at output

       float * back_errors;   // array of errors back-propagated

       float * expected_values;        // to inputs

       float * cum_deltas;    // for momentum

       float * past_deltas;   // for momentum

       friend network;

public:

       output_layer(int, int);

       ~output_layer();

       virtual void calc_out();

       void calc_error(float &);

       void randomize_weights();

       void update_weights(const float, const float);

       void update_momentum();

       void list_weights();

       void write_weights(int, FILE *);

       void read_weights(int, FILE *);

       void list_errors();

       void list_outputs();

};

class middle_layer:   public output_layer

{

private:

public:

    middle_layer(int, int);

    ~middle_layer();

       void calc_error();

};

class network

{

private:

layer *layer_ptr[MAX_LAYERS];

    int number_of_layers;

    int layer_size[MAX_LAYERS];

    float *buffer;

    fpos_t position;

    unsigned training;

public:

    network();

    ~network();

               void set_training(const unsigned &);

               unsigned get_training_value();

               void get_layer_info();

               void set_up_network();

               void randomize_weights();

               void update_weights(const float, const float);

               void update_momentum();

               void write_weights(FILE *);

               void read_weights(FILE *);

               void list_weights();

               void write_outputs(FILE *);

               void list_outputs();

               void list_errors();

               void forward_prop();

               void backward_prop(float &);

               int fill_IObuffer(FILE *);

               void set_up_pattern(int);

               void set_NF(float);

};


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Listing 13.2 Layer.cpp file updated to include noise and momentum

// layer.cpp          V.Rao, H.Rao

// added momentum and noise

// compile for floating point hardware if available

#include <stdio.h>

#include <iostream.h>

#include <stdlib.h>

#include <math.h>

#include <time.h>

#include "layer.h"

inline float squash(float input)

// squashing function

// use sigmoid — can customize to something

// else if desired; can add a bias term too

//

{

if (input < -50)

       return 0.0;

else   if (input > 50)

              return 1.0;

       else return (float)(1/(1+exp(-(double)input)));

}

inline float randomweight(unsigned init)

{

int num;

// random number generator

// will return a floating point

// value between -1 and 1

if (init==1)  // seed the generator

       srand ((unsigned)time(NULL));

num=rand() % 100;

return 2*(float(num/100.00))-1;

}

// the next function is needed for Turbo C++

// and Borland C++ to link in the appropriate

// functions for fscanf floating point formats:

static void force_fpf()

{

       float x, *y;

       y=&x;

       x=*y;

}

// ---------------------

//                            input layer

//---------------------

input_layer::input_layer(int i, int o)

{

num_inputs=i;

num_outputs=o;

outputs = new float[num_outputs];

orig_outputs = new float[num_outputs];

if ((outputs==0)||(orig_outputs==0))

        {

        cout << "not enough memory\n";

        cout << "choose a smaller architecture\n";

        exit(1);

        }

noise_factor=0;

}

input_layer::~input_layer()

{

delete [num_outputs] outputs;

delete [num_outputs] orig_outputs;

}

void input_layer::calc_out()

{

//add noise to inputs

// randomweight returns a random number

// between -1 and 1

int i;

for (i=0; i<num_outputs; i++)

       outputs[i] =orig_outputs[i]*

              (1+noise_factor*randomweight(0));

}

void input_layer::set_NF(float noise_fact)

{

noise_factor=noise_fact;

}

// ---------------------

//                            output layer

//---------------------

output_layer::output_layer(int ins, int outs)

{

int i, j, k;

num_inputs=ins;

num_outputs=outs;

weights = new float[num_inputs*num_outputs];

output_errors = new float[num_outputs];

back_errors = new float[num_inputs];

outputs = new float[num_outputs];

expected_values = new float[num_outputs];

cum_deltas = new float[num_inputs*num_outputs];

past_deltas = new float[num_inputs*num_outputs];

if ((weights==0)||(output_errors==0)||(back_errors==0)

       ||(outputs==0)||(expected_values==0)

       ||(past_deltas==0)||(cum_deltas==0))

       {

       cout << "not enough memory\n";

       cout << "choose a smaller architecture\n";

       exit(1);

       }

// zero cum_deltas and past_deltas matrix

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              {

              cum_deltas[k+j]=0;

              past_deltas[k+j]=0;

              }

       }

}

output_layer::~output_layer()

{

// some compilers may require the array

// size in the delete statement; those

// conforming to Ansi C++ will not

delete [num_outputs*num_inputs] weights;

delete [num_outputs] output_errors;

delete [num_inputs] back_errors;

delete [num_outputs] outputs;

delete [num_outputs*num_inputs] past_deltas;

delete [num_outputs*num_inputs] cum_deltas;

}

void output_layer::calc_out()

{

int i,j,k;

float accumulator=0.0;

for (j=0; j<num_outputs; j++)

       {

       for (i=0; i<num_inputs; i++)

              {

              k=i*num_outputs;

              if (weights[k+j]*weights[k+j] > 1000000.0)

                     {

                      cout << "weights are blowing up\n";

                      cout << "try a smaller learning constant\n";

                      cout << "e.g. beta=0.02    aborting...\n";

                      exit(1);

                     }

              outputs[j]=weights[k+j]*(*(inputs+i));

              accumulator+=outputs[j];

              }

       // use the sigmoid squash function

       outputs[j]=squash(accumulator);

       accumulator=0;

       }

}

void output_layer::calc_error(float & error)

{

int i, j, k;

float accumulator=0;

float total_error=0;

for (j=0; j<num_outputs; j++)

    {

                  output_errors[j] = expected_values[j]-outputs[j];

                  total_error+=output_errors[j];

                  }

error=total_error;

for (i=0; i<num_inputs; i++)

{

k=i*num_outputs;

for (j=0; j<num_outputs; j++)

       {

               back_errors[i]=

                      weights[k+j]*output_errors[j];

               accumulator+=back_errors[i];

               }

       back_errors[i]=accumulator;

       accumulator=0;

       // now multiply by derivative of

       // sigmoid squashing function, which is

       // just the input*(1-input)

       back_errors[i]*=(*(inputs+i))*(1-(*(inputs+i)));

       }

}

void output_layer::randomize_weights()

{

int i, j, k;

const unsigned first_time=1;

const unsigned not_first_time=0;

float discard;

discard=randomweight(first_time);

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              weights[k+j]=randomweight(not_first_time);

       }

}

void output_layer::update_weights(const float beta,

                                     const float alpha)

{

int i, j, k;

float delta;

// learning law: weight_change =

//             beta*output_error*input + alpha*past_delta

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              {

              delta=beta*output_errors[j]*(*(inputs+i))

                     +alpha*past_deltas[k+j];

              weights[k+j] += delta;

              cum_deltas[k+j]+=delta; // current cycle

              }

       }

}

void output_layer::update_momentum()

{

// This function is called when a

// new cycle begins; the past_deltas

// pointer is swapped with the

// cum_deltas pointer. Then the contents

// pointed to by the cum_deltas pointer

// is zeroed out.

int i, j, k;

float * temp;

// swap

temp = past_deltas;

past_deltas=cum_deltas;

cum_deltas=temp;

// zero cum_deltas matrix

// for new cycle

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              cum_deltas[k+j]=0;

       }

}

void output_layer::list_weights()

{

int i, j, k;

for (i=0; i< num_inputs; i++)

       {

       k=i*num_outputs;

       for (j=0; j< num_outputs; j++)

              cout << "weight["<<i<<","<<

                         j<<"] is: "<<weights[k+j];

       }

}

void output_layer::list_errors()

{

int i, j;

for (i=0; i< num_inputs; i++)

       cout << "backerror["<<i<<

                  "] is : "<<back_errors[i]<<"\n";

for (j=0; j< num_outputs; j++)

       cout << "outputerrors["<<j<<

                            "] is: "<<output_errors[j]<<"\n";

}

void output_layer::write_weights(int layer_no,

               FILE * weights_file_ptr)

{

int i, j, k;

// assume file is already open and ready for

// writing

// prepend the layer_no to all lines of data

// format:

//             layer_no   weight[0,0] weight[0,1] ...

//             layer_no   weight[1,0] weight[1,1] ...

//             ...

for (i=0; i< num_inputs; i++)

       {

       fprintf(weights_file_ptr,"%i ",layer_no);

       k=i*num_outputs;

    for (j=0; j< num_outputs; j++)

       {

       fprintf(weights_file_ptr,"%f ",

                      weights[k+j]);

       }

    fprintf(weights_file_ptr,"\n");

    }

}

void output_layer::read_weights(int layer_no,

               FILE * weights_file_ptr)

{

int i, j, k;

// assume file is already open and ready for

// reading

// look for the prepended layer_no

// format:

//             layer_no       weight[0,0] weight[0,1] ...

//             layer_no       weight[1,0] weight[1,1] ...

//             ...

while (1)

        {

        fscanf(weights_file_ptr,"%i";,&j);

        if ((j==layer_no)|| (feof(weights_file_ptr)))

               break;

        else

               {

               while (fgetc(weights_file_ptr) != `\n')

                      {;}// get rest of line

               }

        }

if (!(feof(weights_file_ptr)))

        {

        // continue getting first line

        i=0;

        for (j=0; j< num_outputs; j++)

                          {

                          fscanf(weights_file_ptr,"%f",

                                         &weights[j]); // i*num_outputs

                                                          = 0

     }

        fscanf(weights_file_ptr,"\n");

        // now get the other lines

        for (i=1; i< num_inputs; i++)

               {

               fscanf(weights_file_ptr,

               "%i",&layer_no);

        k=i*num_outputs;

        for (j=0; j< num_outputs; j++)

        {

        fscanf(weights_file_ptr,"%f",

               &weights[k+j]);

               }

    }

    fscanf(weights_file_ptr,"\n");

    }

else cout << "end of file reached\n";

}

void output_layer::list_outputs()

{

int j;

for (j=0; j< num_outputs; j++)

        {

        cout << "outputs["<<j

               <<"] is: "<<outputs[j]<<"\n";

        }

}

// ————————————————————-

//                           middle layer

//—————————————————————

middle_layer::middle_layer(int i, int o):

        output_layer(i,o)

{

}

middle_layer::~middle_layer()

{

delete [num_outputs*num_inputs] weights;

delete [num_outputs] output_errors;

delete [num_inputs] back_errors;

delete [num_outputs] outputs;

}

void middle_layer::calc_error()

{

int i, j, k;

float accumulator=0;

for (i=0; i<num_inputs; i++)

        {

        k=i*num_outputs;

        for (j=0; j<num_outputs; j++)

               {

               back_errors[i]=

                      weights[k+j]*(*(output_errors+j));

               accumulator+=back_errors[i];

               }

        back_errors[i]=accumulator;

        accumulator=0;

        // now multiply by derivative of

        // sigmoid squashing function, which is

        // just the input*(1-input)

        back_errors[i]*=(*(inputs+i))*(1-(*(inputs+i)));

        }

}

network::network()

{

position=0L;

}

network::~network()

{

int i,j,k;

i=layer_ptr[0]->num_outputs;// inputs

j=layer_ptr[number_of_layers-1]->num_outputs; //outputs

k=MAX_VECTORS;

delete [(i+j)*k]buffer;

}

void network::set_training(const unsigned & value)

{

training=value;

}

unsigned network::get_training_value()

{

return training;

}

void network::get_layer_info()

{

int i;

//—————————————————————

//

//      Get layer sizes for the network

//

// ————————————————————-

cout << " Please enter in the number of layers for your net work.\n";

cout << " You can have a minimum of 3 to a maximum of 5. \n";

cout << " 3 implies 1 hidden layer; 5 implies 3 hidden layers : \n\n";

cin >> number_of_layers;

cout << " Enter in the layer sizes separated by spaces.\n";

cout << " For a network with 3 neurons in the input layer,\n";

cout << " 2 neurons in a hidden layer, and 4 neurons in the\n";

cout << " output layer, you would enter: 3 2 4 .\n";

cout << " You can have up to 3 hidden layers,for five maximum entries

:\n\n";

for (i=0; i<number_of_layers; i++)

        {

        cin >> layer_size[i];

        }

// ———————————————————————————

// size of layers:

//    input_layer            layer_size[0]

//    output_layer           layer_size[number_of_layers-1]

//    middle_layers          layer_size[1]

//    optional: layer_size[number_of_layers-3]

//    optional: layer_size[number_of_layers-2]

//———————————————————————————-

}

void network::set_up_network()

{

int i,j,k;

//———————————————————————————-

// Construct the layers

//

//———————————————————————————-

layer_ptr[0] = new input_layer(0,layer_size[0]);

for (i=0;i<(number_of_layers-1);i++)

        {

        layer_ptr[i+1] =

        new middle_layer(layer_size[i],layer_size[i+1]);

        }

layer_ptr[number_of_layers-1] = new

output_layer(layer_size[number_of_layers-2], layer_size[number_of_

layers-1]);

for (i=0;i<(number_of_layers-1);i++)

        {

        if (layer_ptr[i] == 0)

               {

               cout << "insufficient memory\n";

               cout << "use a smaller architecture\n";

               exit(1);

               }

        }

//———————————————————————————-

// Connect the layers

//

//———————————————————————————-

// set inputs to previous layer outputs for all layers,

//             except the input layer

for (i=1; i< number_of_layers; i++)

        layer_ptr[i]->inputs = layer_ptr[i-1]->outputs;

// for back_propagation, set output_errors to next layer

//             back_errors for all layers except the output

//             layer and input layer

for (i=1; i< number_of_layers -1; i++)

        ((output_layer *)layer_ptr[i])->output_errors =

               ((output_layer *)layer_ptr[i+1])->back_errors;

// define the IObuffer that caches data from

// the datafile

i=layer_ptr[0]->num_outputs;// inputs

j=layer_ptr[number_of_layers-1]->num_outputs; //outputs

k=MAX_VECTORS;

buffer=new

        float[(i+j)*k];

if (buffer==0)

        {

        cout << "insufficient memory for buffer\n";

        exit(1);

        }

}

void network::randomize_weights()

{

int i;

for (i=1; i<number_of_layers; i++)

        ((output_layer *)layer_ptr[i])

                ->randomize_weights();

}

void network::update_weights(const float beta, const float alpha)

{

int i;

for (i=1; i<number_of_layers; i++)

        ((output_layer *)layer_ptr[i])

               ->update_weights(beta,alpha);

}

void network::update_momentum()

{

int i;

for (i=1; i<number_of_layers; i++)

        ((output_layer *)layer_ptr[i])

               ->update_momentum();

}

void network::write_weights(FILE * weights_file_ptr)

{

int i;

for (i=1; i<number_of_layers; i++)

        ((output_layer *)layer_ptr[i])

               ->write_weights(i,weights_file_ptr);

}

void network::read_weights(FILE * weights_file_ptr)

{

int i;

for (i=1; i<number_of_layers; i++)

        ((output_layer *)layer_ptr[i])

               ->read_weights(i,weights_file_ptr);

}

void network::list_weights()

{

int i;

for (i=1; i<number_of_layers; i++)

        {

        cout << "layer number : " <<i<< "\n";

        ((output_layer *)layer_ptr[i])

               ->list_weights();

        }

}

void network::list_outputs()

{

int i;

for (i=1; i<number_of_layers; i++)

        {

        cout << "layer number : " <<i<< "\n";

        ((output_layer *)layer_ptr[i])

               ->list_outputs();

        }

}

void network::write_outputs(FILE *outfile)

{

int i, ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

float temp;

fprintf(outfile,"for input vector:\n");

for (i=0; i<ins; i++)

        {

        temp=layer_ptr[0]->outputs[i];

        fprintf(outfile,"%f  ",temp);

        }

fprintf(outfile,"\noutput vector is:\n");

for (i=0; i<outs; i++)

        {

        temp=layer_ptr[number_of_layers-1]->

        outputs[i];

        fprintf(outfile,"%f  ",temp);

        }

if (training==1)

{

fprintf(outfile,"\nexpected output vector is:\n");

for (i=0; i<outs; i++)

        {

        temp=((output_layer *)(layer_ptr[number_of_layers-1]))->

        expected_values[i];

        fprintf(outfile,"%f  ",temp);

        }

}

fprintf(outfile,"\n———————————\n");

}

void network::list_errors()

{

int i;

for (i=1; i<number_of_layers; i++)

        {

        cout << "layer number : " <<i<< "\n";

        ((output_layer *)layer_ptr[i])

               ->list_errors();

        }

}

int network::fill_IObuffer(FILE * inputfile)

{

// this routine fills memory with

// an array of input, output vectors

// up to a maximum capacity of

// MAX_INPUT_VECTORS_IN_ARRAY

// the return value is the number of read

// vectors

int i, k, count, veclength;

int ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

if (training==1)

        veclength=ins+outs;

else

        veclength=ins;

count=0;

while  ((count<MAX_VECTORS)&&

               (!feof(inputfile)))

        {

        k=count*(veclength);

        for (i=0; i<veclength; i++)

               {

               fscanf(inputfile,"%f",&buffer[k+i]);

               }

        fscanf(inputfile,"\n");

        count++;

        }

if (!(ferror(inputfile)))

        return count;

else return -1; // error condition

}

void network::set_up_pattern(int buffer_index)

{

// read one vector into the network

int i, k;

int ins, outs;

ins=layer_ptr[0]->num_outputs;

outs=layer_ptr[number_of_layers-1]->num_outputs;

if (training==1)

        k=buffer_index*(ins+outs);

else

        k=buffer_index*ins;

for (i=0; i<ins; i++)

        ((input_layer*)layer_ptr[0])

                      ->orig_outputs[i]=buffer[k+i];

if (training==1)

{

        for (i=0; i<outs; i++)

               ((output_layer *)layer_ptr[number_of_layers-1])->

                      expected_values[i]=buffer[k+i+ins];

}

}

void network::forward_prop()

{

int i;

for (i=0; i<number_of_layers; i++)

        {

        layer_ptr[i]->calc_out(); //polymorphic

                               // function

        }

}

void network::backward_prop(float & toterror)

{

int i;

// error for the output layer

((output_layer*)layer_ptr[number_of_layers-1])->

                      calc_error(toterror);

// error for the middle layer(s)

for (i=number_of_layers-2; i>0; i—)

        {

        ((middle_layer*)layer_ptr[i])->

                      calc_error();

        }

}

void network::set_NF(float noise_fact)

{

((input_layer*)layer_ptr[0])->set_NF(noise_fact);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The New and Final backprop.cpp File

The last file to present is the backprop.cpp file. This is shown in Listing 13.3.

Listing 13.3 Implementation file for the backpropagation simulator, with noise and momentum backprop.cpp

// backprop.cpp      V. Rao, H. Rao

#include "layer.cpp"

#define TRAINING_FILE   "training.dat"

#define WEIGHTS_FILE "weights.dat"

#define OUTPUT_FILE   "output.dat"

#define TEST_FILE   "test.dat"

void main()

{

float error_tolerance=0.1;

float total_error=0.0;

float avg_error_per_cycle=0.0;

float error_last_cycle=0.0;

float avgerr_per_pattern=0.0; // for the latest cycle

float error_last_pattern=0.0;

float learning_parameter=0.02;

float alpha; // momentum parameter

float NF; // noise factor

float new_NF;

unsigned temp, startup, start_weights;

long int vectors_in_buffer;

long int max_cycles;

long int patterns_per_cycle=0;

long int total_cycles, total_patterns;

int i;

// create a network object

network backp;

FILE * training_file_ptr, * weights_file_ptr, * output_file_ptr;

FILE * test_file_ptr, * data_file_ptr;

// open output file for writing

if ((output_file_ptr=fopen(OUTPUT_FILE,"w"))==NULL)

               {

               cout << "problem opening output file\n";

               exit(1);

               }

// enter the training mode : 1=training on     0=training off

cout << "—————————————————————————-\n";

cout << " C++ Neural Networks and Fuzzy Logic \n";

cout << "    Backpropagation simulator \n";

cout << "      version 2 \n";

cout << "—————————————————————————-\n";

cout << "Please enter 1 for TRAINING on, or 0 for off: \n\n";

cout << "Use training to change weights according to your\n";

cout << "expected outputs. Your training.dat file should contain\n";

cout << "a set of inputs and expected outputs. The number of\n";

cout << "inputs determines the size of the first (input) layer\n";

cout << "while the number of outputs determines the size of the\n";

cout << "last (output) layer :\n\n";

cin >> temp;

backp.set_training(temp);

if (backp.get_training_value() == 1)

        {

        cout << "—> Training mode is *ON*. weights will be saved\n";

        cout << "in the file weights.dat at the end of the\n";

        cout << "current set of input (training) data\n";

        }

else

        {

        cout << "—> Training mode is *OFF*. weights will be loaded\n";

        cout << "from the file weights.dat and the current\n";

        cout << "(test) data set will be used. For the test\n";

        cout << "data set, the test.dat file should contain\n";

        cout << "only inputs, and no expected outputs.\n";

        }

if (backp.get_training_value()==1)

        {

        // ————————————————————-

        //    Read in values for the error_tolerance,

        //    and the learning_parameter

        // ————————————————————-

        cout << " Please enter in the error_tolerance\n";

        cout << " —- between 0.001 to 100.0, try 0.1 to start - \n";

        cout << "\n";

        cout << "and the learning_parameter, beta\n";

        cout << " —- between 0.01 to 1.0, try 0.5 to start - \n\n";

        cout << " separate entries by a space\n";

        cout << " example: 0.1 0.5 sets defaults mentioned :\n\n";

        cin >> error_tolerance >> learning_parameter;

        // ————————————————————-

        //    Read in values for the momentum

        //    parameter, alpha (0-1.0)

        //    and the noise factor, NF (0-1.0)

        // ————————————————————-

        cout << "Enter values now for the momentum \n";

        cout << "parameter, alpha(0-1.0)\n";

        cout << " and the noise factor, NF (0-1.0)\n";

        cout << "You may enter zero for either of these\n";

        cout << "parameters, to turn off the momentum or\n";

        cout << "noise features.\n";

        cout << "If the noise feature is used, a random\n";

        cout << "component of noise is added to the inputs\n";

        cout << "This is decreased to 0 over the maximum\n";

        cout << "number of cycles specified.\n";

        cout << "enter alpha followed by NF, e.g., 0.3 0.5\n";

        cin >> alpha >> NF;

        //—————————————————————-

        // open training file for reading

        //—————————————————————-

        if ((training_file_ptr=fopen(TRAINING_FILE,"r"))==NULL)

               {

               cout << "problem opening training file\n";

               exit(1);

               }

        data_file_ptr=training_file_ptr; // training on

        // Read in the maximum number of cycles

        // each pass through the input data file is a cycle

        cout << "Please enter the maximum cycles for the simulation\n";

        cout << "A cycle is one pass through the data set.\n";

        cout << "Try a value of 10 to start with\n";

        cin >> max_cycles;

        cout << "Do you want to read weights from weights.dat to

start?\n";

        cout << "Type 1 to read from file, 0 to randomize starting

weights\n";

        cin >> start_weights;

        }

else

        {

        if ((test_file_ptr=fopen(TEST_FILE,"r"))==NULL)

               {

               cout << "problem opening test file\n";

               exit(1);

               }

        data_file_ptr=test_file_ptr; // training off

        }

// training: continue looping until the total error is less than

//             the tolerance specified, or the maximum number of

//             cycles is exceeded; use both the forward signal

               propagation

//             and the backward error propagation phases. If the error

//             tolerance criteria is satisfied, save the weights in a

               file.

// no training: just proceed through the input data set once in the

//             forward signal propagation phase only. Read the starting

//             weights from a file.

// in both cases report the outputs on the screen

// initialize counters

total_cycles=0; // a cycle is once through all the input data

total_patterns=0; // a pattern is one entry in the input data

new_NF=NF;

// get layer information

backp.get_layer_info();

// set up the network connections

backp.set_up_network();

// initialize the weights

if ((backp.get_training_value()==1)&&(start_weights!=1))

        {

        // randomize weights for all layers; there is no

        // weight matrix associated with the input layer

        // weight file will be written after processing

        backp.randomize_weights();

        // set up the noise factor value

        backp.set_NF(new_NF);

        }

else

        {

        // read in the weight matrix defined by a

        // prior run of the backpropagation simulator

        // with training on

        if ((weights_file_ptr=fopen(WEIGHTS_FILE,"r"))

                       ==NULL)

               {

               cout << "problem opening weights file\n";

               exit(1);

               }

        backp.read_weights(weights_file_ptr);

        fclose(weights_file_ptr);

        }

// main loop

// if training is on, keep going through the input data

//             until the error is acceptable or the maximum number of

               cycles

//             is exceeded.

// if training is off, go through the input data once. report outputs

// with inputs to file output.dat

startup=1;

vectors_in_buffer = MAX_VECTORS; // startup condition

total_error = 0;

while (   ((backp.get_training_value()==1)

                         && (avgerr_per_pattern

                                      > error_tolerance)

                         && (total_cycles < max_cycles)

                         && (vectors_in_buffer !=0))

                         || ((backp.get_training_value()==0)

                         && (total_cycles < 1))

                         || ((backp.get_training_value()==1)

                         && (startup==1))

                         )

{

startup=0;

error_last_cycle=0; // reset for each cycle

patterns_per_cycle=0;

backp.update_momentum(); // added to reset

                       // momentum matrices

                       // each cycle

// process all the vectors in the datafile

// going through one buffer at a time

// pattern by pattern

while ((vectors_in_buffer==MAX_VECTORS))

        {

        vectors_in_buffer=

               backp.fill_IObuffer(data_file_ptr); // fill buffer

               if (vectors_in_buffer < 0)

                      {

                      cout << "error in reading in vectors, aborting\n";

                      cout << "check that there are no extra linefeeds\n";

                      cout << "in your data file, and that the number\n";

                      cout << "of layers and size of layers match the\n";

                      cout << "the parameters provided.\n";

                      exit(1);

                      }

               // process vectors

               for (i=0; i<vectors_in_buffer; i++)

                      {

                      // get next pattern

                      backp.set_up_pattern(i);

                      total_patterns++;

                      patterns_per_cycle++;

                      // forward propagate

                      backp.forward_prop();

                      if (backp.get_training_value()==0)

                             backp.write_outputs(output_file_ptr);

                      // back_propagate, if appropriate

                      if (backp.get_training_value()==1)

                             {

                             backp.backward_prop(error_last_pattern);

                             error_last_cycle +=

                                    error_last_pattern*error_last_pattern;

                             avgerr_per_pattern=

               ((float)sqrt((double)error_last_cycle/patterns_per_cycle));

                             // if it's not the last cycle, update weights

                           if ((avgerr_per_pattern

                                    > error_tolerance)

                                    && (total_cycles+1 < max_cycles))

                                    backp.update_weights(learning_

                                           parameter, alpha);

                             // backp.list_weights(); // can

                             // see change in weights by

                             // using list_weights before and

                             // after back_propagation

                             }

                      }

       error_last_pattern = 0;

       }

total_error += error_last_cycle;

total_cycles++;

// update NF

// gradually reduce noise to zero

if (total_cycles>0.7*max_cycles)

               new_NF = 0;

else   if (total_cycles>0.5*max_cycles)

                      new_NF = 0.25*NF;

               else   if (total_cycles>0.3*max_cycles)

                                    new_NF = 0.50*NF;

                             else   if (total_cycles>0.1*max_cycles)

                                           new_NF = 0.75*NF;

backp.set_NF(new_NF);

// most character displays are 25 lines

// user will see a corner display of the cycle count

// as it changes

cout << "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n";

cout << total_cycles << "\t" << avgerr_per_pattern << "\n";

fseek(data_file_ptr, 0L, SEEK_SET); // reset the file pointer

                             // to the beginning of

                             // the file

vectors_in_buffer = MAX_VECTORS; // reset

} // end main loop

if (backp.get_training_value()==1)

        {

        if ((weights_file_ptr=fopen(WEIGHTS_FILE,"w"))

                      ==NULL)

               {

               cout << "problem opening weights file\n";

               exit(1);

               }

        }

cout << "\n\n\n\n\n\n\n\n\n\n\n";

cout << "————————————————————————\n";

cout << "    done:   results in file output.dat\n";

cout << "            training: last vector only\n";

cout << "            not training: full cycle\n\n";

if (backp.get_training_value()==1)

        {

        backp.write_weights(weights_file_ptr);

        backp.write_outputs(output_file_ptr);

        avg_error_per_cycle=(float)sqrt((double)total_error/

        total_cycles);

        error_last_cycle=(float)sqrt((double)error_last_cycle);

        fclose(weights_file_ptr);

cout << "              weights saved in file

weights.dat\n";

cout << "\n";

cout << "——>average error per cycle =

" << avg_error_per_cycle << "

<—-\n";

cout << "——>error last cycle = "

<< error_last_cycle << " <—-\n";

???cout << "->error last cycle per

pattern="<<avgerr_per_pattern

<<"<—-\n";

        }

cout << "——————>total

cycles = " << total_cycles << "

<—-\n";

cout << "——————>total

patterns = " << total_patterns

<< " <—-\n";

cout <<

"——————————

;——————————————\n";

// close all files

fclose(data_file_ptr);

fclose(output_file_ptr);

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Trying the Noise and Momentum Features

You can test out the version 2 simulator, which you just compiled with the example that you saw at the beginning of the chapter. You will find that there is a lot of trial and error in finding optimum values for alpha, the noise factor, and beta. This is true also for the middle layer size and the number of middle layers. For some problems, the addition of momentum makes convergence much faster. For other problems, you may not find any noticeable difference. An example run of the five-character recognition problem discussed at the beginning of this chapter resulted in the following results with beta = 0.1, tolerance = 0.001, alpha = 0.25, NF = 0.1, and the layer sizes kept at 35 5 3.

—————————————————————————-

        done:   results in file output.dat

                training: last vector only

                not training: full cycle

                weights saved in file weights.dat

——>average error per cycle = 0.02993<—-

——>error last cycle = 0.00498<—-

->error last cycle per pattern= 0.000996 <—-

——————>total cycles = 242 <—-

——————>total patterns = 1210 <—-

—————————————————————————-

The network was able to converge on a better solution (in terms of error measurement) in one-fourth the number of cycles. You can try varying alpha and NF to see the effect on overall simulation time. You can now start from the same initial starting weights by specifying a value of 1 for the starting weights question. For large values of alpha and beta, the network usually will not converge, and the weights will get unacceptably large (you will receive a message to that effect).

Variations of the Backpropagation Algorithm

Backpropagation is a versatile neural network algorithm that very often leads to success. Its Achilles heel is the slowness at which it converges for certain problems. Many variations of the algorithm exist in the literature to try to improve convergence speed and robustness. Variations have been proposed in the following portions of the algorithm:

• Adaptive parameters. You can set rules that modify alpha, the momentum parameter, and beta, the learning parameter, as the simulation progresses. For example, you can reduce beta whenever a weight change does not reduce the error. You can consider undoing the particular weight change, setting alpha to zero and redoing the weight change with the new value of beta.
• Use other minimum search routines besides steepest descent. For example, you could use Newton's method for finding a minimum, although this would be a fairly slow process. Other examples include the use of conjugate gradient methods or Levenberg-Marquardt optimization, both of which would result in very rapid training.
• Use different cost functions. Instead of calculating the error (as expected—actual output), you could determine another cost function that you want to minimize.
• Modify the architecture. You could use partially connected layers instead of fully connected layers. Also, you can use a recurrent network, that is, one in which some outputs feed back as inputs.

Applications

Backpropagation remains the king of neural network architectures because of its ease of use and wide applicability. A few of the notable applications in the literature will be cited as examples.

• NETTalk. In 1987, Sejnowski and Rosenberg developed a network connected to a speech synthesizer that was able to utter English words, being trained to produce phonemes from English text. The architecture consisted of an input layer window of seven characters. The characters were part of English text that was scrolled by. The network was trained to pronounce the letter at the center of the window. The middle layer had 80 neurons, while the output layer consisted of 26 neurons. With 1024 training patterns and 10 cycles, the network started making intelligible speech, similar to the process of a child learning to talk. After 50 cycles, the network was about 95% accurate. You could purposely damage the network with the removal of neurons, but this did not cause performance to drop off a cliff; instead, the performance degraded gracefully. There was rapid recovery with retraining using fewer neurons also. This shows the fault tolerance of neural networks.
• Sonar target recognition. Neural nets using backpropagation have been used to identify different types of targets using the frequency signature (with a Fast Fourier transform) of the reflected signal.
• Car navigation. Pomerleau developed a neural network that is able to navigate a car based on images obtained from a camera mounted on the car's roof, and a range finder that coded distances in grayscale. The 30×32 pixel image and the 8×32 range finder image were fed into a hidden layer of size 29 feeding an output layer of 45 neurons. The output neurons were arranged in a straight line with each side representing a turn to a particular direction (right or left), while the center neurons represented "drive straight ahead." After 1200 road images were trained on the network, the neural network driver was able to negotiate a part of the Carnegie-Mellon campus at a speed of about 3 miles per hour, limited only by the speed of the real-time calculations done on a trained network in the Sun-3 computer in the car.
• Image compression. G.W. Cottrell, P. Munro, and D. Zipser used backpropagation to compress images with the result of an 8:1 compression ratio. They used standard backpropagation with 64 input neurons (8×8 pixels), 16 hidden neurons, and 64 output neurons equal to the inputs. This is called self-supervised backpropagation and represents an autoassociative network. The compressed signal is taken from the hidden layer. The input to hidden layer comprised the compressor, while the hidden to output layer forms a decompressor.
• Image recognition. Le Cun reported a backpropagation network with three hidden layers that could recognize handwritten postal zip codes. He used a 16×16 array of pixel to represent each handwritten digit and needed to encode 10 outputs, each of which represented a digit from 0 to 9. One interesting aspect of this work is that the hidden layers were not fully connected. The network was set up with blocks of neurons in the first two hidden layers set up as feature detectors for different parts of the previous layer. All the neurons in the block were set up to have the same weights as those from the previous layer. This is called weight sharing. Each block would sample a different part of the previous layer's image. The first hidden layer had 12 blocks of 8×8 neurons, whereas the second hidden layer had 12 blocks of 4×4 neurons. The third hidden layer was fully connected and consisted of 30 neurons. There were 1256 neurons. The network was trained on 7300 examples and tested on 2000 cases with error rates of 1% on training set and 5% on the test set.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Summary

You explored further the backpropagation algorithm in this chapter, continuing the discussion in Chapter 7.

• A momentum term was added to the training law and was shown to result in much faster convergence in some cases.
• A noise term was added to inputs to allow training to take place with random noise applied. This noise was made to decrease with the number of cycles, so that final stage learning could be done in a noise-free environment.
• The final version of the backpropagation simulator was constructed and used on the example from Chapter 12. Further application of the simulator will be made in Chapter 14.
• Several applications with the backpropagation algorithm were outlined, showing the wide applicability of this algorithm.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 14
Application to Financial Forecasting

Introduction

In Chapters 7 and 13, the backpropagation simulator was developed. In this chapter, you will use the simulator to tackle a complex problem in financial forecasting. The application of neural networks to financial forecasting and modeling has been very popular over the last few years. Financial journals and magazines frequently mention the use of neural networks, and commercial tools and simulators are quite widespread.

This chapter gives you an overview of typical steps used in creating financial forecasting models. Many of the steps will be simplified, and so the results will not, unfortunately, be good enough for real life application. However, this chapter will hopefully serve as an introduction to the field with some added pointers for further reading and resources for those who want more detailed information.

Who Trades with Neural Networks?

There has been a great amount of interest on Wall Street for neural networks. Bradford Lewis runs two Fidelity funds in part with the use of neural networks. Also, LBS Capital Management (Peoria, Illinois) manages part of its portfolio with neural networks. According to Barron's (February 27, 1995), LBS's $150 million fund beat the averages by three percentage points a year since 1992. Each weekend, neural networks are retrained with the latest technical and fundamental data including P/E ratios, earnings results and interest rates. Another of LBS's models has done worse than the S&P 500 for the past five years however. In the book Virtual Trading, Jeffrey Katz states that most of the successful neural network systems are proprietary and not publicly heard of. Clients who use neural networks usually don't want anyone else to know what they are doing, for fear of losing their competitive edge. Firms put in many person-years of engineering design with a lot of CPU cycles to achieve practical and profitable results. Let's look at the process:

Developing a Forecasting Model

There are many steps in building a forecasting model, as listed below.

1. Decide on what your target is and develop a neural network (following these steps) for each target.
2. Determine the time frame that you wish to forecast.
3. Gather information about the problem domain.
4. Gather the needed data and get a feel for each inputs relationship to the target.
5. Process the data to highlight features for the network to discern.
6. Transform the data as appropriate.
7. Scale and bias the data for the network, as needed.
8. Reduce the dimensionality of the input data as much as possible.
9. Design a network architecture (topology, # layers, size of layers, parameters, learning paradigm).
10. Go through the train/test/redesign loop for a network.
11. Eliminate correlated inputs as much as possible, while in step 10.
12. Deploy your network on new data and test it and refine it as necessary.

Once you develop a forecasting model, you then must integrate this into your trading system. A neural network can be designed to predict direction, or magnitude, or maybe just turning points in a particular market or something else. Avner Mandelman of Cereus Investments (Los Altos Hills, California) uses a long-range trained neural network to tell him when the market is making a top or bottom (Barron's, December 14, 1992).

Now let's expand on the twelve aspects of model building:

The Target and the Timeframe

What should the output of your neural network forecast? Let's say you want to predict the stock market. Do you want to predict the S&P 500? Or, do you want to predict the direction of the S&P 500 perhaps? You could predict the volatility of the S&P 500 too (maybe if you're an options player). Further, like Mr. Mandelman, you could only want to predict tops and bottoms, say, for the Dow Jones Industrial Average. You need to decide on the market or markets and also on your specific objectives.

Another crucial decision is the timeframe you want to predict forward. It is easier to create neural network models for longer term predictions than it is for shorter term predictions. You can see a lot of market noise, or seemingly random, chaotic variations at smaller and smaller timescale resolutions that might explain this. Another reason is that the macroeconomic forces that fundamentally move market over long periods, move slowly. The U.S. dollar makes multiyear trends, shaped by economic policy of governments around the world. For a given error tolerance, a one-year forecast, or one-month forecast will take less effort with a neural network than a one-day forecast will.

Domain Expertise

So far we've talked about the target and the timeframe. Now one other important aspect of model building is knowledge of the domain. If you want to create an effective predictive model of the weather, then you need to know or be able to guess about the factors that influence weather. The same holds true for the stock market or other financial market. In order to create a real tradable Treasury bond trading system, you need to have a good idea of what really drives the market and works— i.e., talk to a Tbond trader and encapsulate his domain expertise!

Gather the Data

Once you know the factors that influence the target output, you can gather raw data. If you are predicting the S&P 500, then you may consider Treasury yields, 3-month T-bill yields, and earnings as some of the factors. Once you have the data, you can do scatter plots to see if there is some correlation between the input and the target output. If you are not satisfied with the plot, you may consider a different input in its place.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Pre processing the Data for the Network

Surprising as it may sound, you are most likely going to spend about 90% of your time, as a neural network developer, in massaging and transforming data into meaningful form for training your network. We actually defined three substeps in this area of preprocessing in our master list:

• Highlight features
• Transform
• Scale and bias

Highlighting Features in the Input Data

You should present the neural network, as much as possible, with an easy way to find patterns in your data. For time series data, like stock market prices over time, you may consider presenting quantities like rate of change and acceleration (the first and second derivatives of your input) as examples. Other ways to highlight data is to magnify certain occurrences. For example, if you consider Central bank intervention as an important qualifier to foreign exchange rates, then you may include as an input to your network, a value of 1 or 0, to indicate the presence or lack of presence of Central bank intervention. Now if you further consider the activity of the U.S. Federal Reserve bank to be important by itself, then you may wish to highlight that, by separating it out as another 1/0 input. Using 1/0 coding to separate composite effects is called thermometer encoding.

There is a whole body of study of market behavior called Technical Analysis from which you may also wish to present technical studies on your data. There is a wide assortment of mathematical technical studies that you perform on your data (see references), such as moving averages to smooth data as an example. There are also pattern recognition studies you can use, like the "double-top" formation, which purportedly results in a high probability of significant decline. To be able to recognize such a pattern, you may wish to present a mathematical function that aids in the identification of the double-top.

You may want to de-emphasize unwanted noise in your input data. If you see a spike in your data, you can lessen its effect, by passing it through a moving average filter for example. You should be careful about introducing excessive lag in the resulting data though.

Transform the Data If Appropriate

For time series data, you may consider using a Fourier transform to move to the frequency-phase plane. This will uncover periodic cyclic information if it exists. The Fourier transform will decompose the input discrete data series into a series of frequency spikes that measure the relevance of each frequency component. If the stock market indeed follows the so-called January effect, where prices typically make a run up, then you would expect a strong yearly component in the frequency spectrum. Mark Jurik suggests sampling data with intervals that catch different cycle periods, in his paper on neural network data preparation (see references ).

You can use other signal processing techniques such as filtering. Besides the frequency domain, you can also consider moving to other spaces, such as with using the wavelet transform. You may also analyze the chaotic component of the data with chaos measures. It's beyond the scope of this book to discuss these techniques. (Refer to the Resources section of this chapter for more information.) If you are developing short-term trading neural network systems, these techniques may play a significant role in your preprocessing effort. All of these techniques will provide new ways of looking at your data, for possible features to detect in other domains.

Scale Your Data

Neurons like to see data in a particular input range to be most effective. If you present data like the S&P 500 that varies from 200 to 550 (as the S&P 500 has over the years) will not be useful, since the middle layer of neurons have a Sigmoid Activation function that squashes large inputs to either 0 or +1. In other words, you should choose data that fit a range that does not saturate, or overwhelm the network neurons. Choosing inputs from –1 to 1 or 0 to 1 is a good idea. By the same token, you should normalize the expected values for the outputs to the 0 to 1 sigmoidal range.

It is important to pay attention to the number of input values in the data set that are close to zero. Since the weight change law is proportional to the input value, then a close to zero input will mean that that weight will not participate in learning! To avoid such situations, you can add a constant bias to your data to move the data closer to 0.5, where the neurons respond very well.

Reduce Dimensionality

You should try to eliminate inputs wherever possible. This will reduce the dimensionality of the problem and make it easier for your neural network to generalize. Suppose that you have three inputs, x, y and z and one output, o. Now suppose that you find that all of your inputs are restricted only to one plane. You could redefine axes such that you have x' and y' for the new plane and map your inputs to the new coordinates. This changes the number of inputs to your problem to 2 instead of 3, without any loss of information. This is illustrated in Figure 14.1.

Figure 14.1 Reducing dimensionality from three to two dimensions.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Generalization versus Memorization

If your overall goal is beyond pattern classification, you need to track your network's ability to generalize. Not only should you look at the overall error with the training set that you define, but you should set aside some training examples as part of a test set (and do not train with them), with which you can see whether or not the network is able to correctly predict. If the network responds poorly to your test set, you know that you have overtrained, or you can say the network "memorized" the training patterns. If you look at the arbitrary curve-fitting analogy in Figure 14.2, you see curves for a generalized fit, labeled G, and an overfit, labeled O. In the case of the overfit, any data point outside of the training data results in highly erroneous prediction. Your test data will certainly show you large error in the case of an overfitted model.

Figure 14.2 General (G) versus over fitting (0) of data.

Another way to consider this issue is in terms of Degrees Of Freedom (DOF). For the polynomial:

y= a0 + a1x + a2x2 + anxn...

the DOF equals the number of coefficients a0, a1 ... an, which is N + 1. So for the equation of a line (y=a0 + a1x), the DOF would be 2. For a parabola, this would be 3 and so on. The objective to not overfit data can be restated as an objective to obtain the function with the least DOF that fits the data adequately. For neural network models, the larger the number of trainable weights (which is a function of the number of inputs and the architecture), the larger the DOF. Be careful with having too many (unimportant) inputs. You may find terrific results with your training data, but extremely poor results with your test data.

Eliminate Correlated Inputs Where Possible

You have seen that getting to the minimum number of inputs for a given problem is important in terms of minimizing DOF and simplifying your model. Another way to reduce dimensionality is to look for correlated inputs and to carefully eliminate redundancy. For example, you may find that the Swiss franc and German mark are highly correlated over a certain time period of interest. You may wish to eliminate one of these inputs to reduce dimensionality. You have to be careful in this process though. You may find that a seemingly redundant piece of information is actually very important. Mark Jurik, of Jurik Consulting, in his paper on data preprocessing, suggests that one of the best ways to determine if an input is really needed is to construct neural network models with and without the input and choose the model with the best error on training and test data. Although very iterative, you can try eliminating as many inputs as possible this way and be assured that you haven't eliminated a variable that really made a difference.

Another approach is sensitivity analysis, where you vary one input a little, while holding all others constant and note the effect on the output. If the effect is small you eliminate that input. This approach is flawed because in the real world, all the inputs are not constant. Jurik's approach is more time consuming but will lead to a better model.

The process of decorrelation, or eliminating correlated inputs, can also utilize a linear algebra technique called principal component analysis. The result of principal component analysis is a minimum set of variables that contain the maximum information. For further information on principal component analysis, you should consult a statistics reference or research two methods of principal component analysis: the Karhunen-Loev transform and the Hotelling transform.

Design a Network Architecture

Now it's time to actually design the neural network. For the backpropagation feed-forward neural network we have designed, this means making the following choices:

1. The number of hidden layers.
2. The size of hidden layers.
3. The learning constant, beta([beta]).
4. The momentum parameter, alpha([alpha]).
5. The form of the squashing function (does not have to be the sigmoid).
6. The starting point, that is, initial weight matrix.
7. The addition of noise.

Some of the parameters listed can be made to vary with the number of cycles executed, similar to the current implementation of noise. For example, you can start with a learning constant [beta] that is large and reduce this constant as learning progresses. This allows rapid initial learning in the beginning of the process and may speed up the overall simulation time.

The Train/Test/Redesign Loop

Much of the process of determining the best parameters for a given application is trial and error. You need to spend a great deal of time evaluating different options to find the best fit for your problem. You may literally create hundreds if not thousands of networks either manually or automatically to search for the best solution. Many commercial neural network programs use genetic algorithms to help to automatically arrive at an optimum network. A genetic algorithm makes up possible solutions to a problem from a set of starting genes. Analogous to biological evolution, the algorithm combines genetic solutions with a predefined set of operators to create new generations of solutions, who survive or perish depending on their ability to solve the problem. The key benefit of genetic algorithms (GA) is the ability to traverse an enormous search space for a possibly optimum solution. You would program a GA to search for the number of hidden layers and other network parameters, and gradually evolve a neural network solution. Some vendors use a GA only to assign a starting set of weights to the network, instead of randomizing the weights to start you off near a good solution.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Now let's review the steps:

1. Split your data. First, divide you data set into three pieces, a training set, a test set and a blind test set. Use about 80% of your data records for your training set, 10% for your test set and 10% for your blind test set.
2. Train and test. Next, start with a network topology and train your network on your training set data. When you have reached a satisfactory minimum error, save your weights and apply your trained network to the test data and note the error. Now restart the process with the same network topology for a different set of initial weights and see if you can achieve a better error on training and test sets. Reasoning: you may have found a local minimum on your first attempt and randomizing the initial weights will start you off to a different, maybe better solution.
3. Eliminate correlated inputs. You may optionally try at this point to see if you can eliminate correlated inputs, as mentioned before, by iteratively removing each input and noting the best error you can achieve on the training and test sets for each of these cases. Choose the case that leads to the best error and eliminate the input (if any) that achieved it. You can repeat this whole process again to try to eliminate another input variable.
4. Iteratively train and test. Now you can try other network parameters and repeat the train and test process to achieve a better result.
5. Deploy your network. You now can use the blind test data set to see how your optimized network performs. If the error is not satisfactory, then you need to re-enter the design phase or the train and test phase.
6. Revisit your network design when conditions change. You need to retrain your network when you have reason to think that you have new information relevant to the problem you are modeling. If you have a neural network that tries to predict the weekly change in the S&P 500, then you likely will need to retrain your network at least once a month, if not once a week. If you find that the network no longer generalizes well with the new information, you need to re-enter the design phase.

If this sounds like a lot of work, it is! Now, let's try our luck at forecasting by going through a subset of the steps outlined:

Forecasting the S&P 500

The S&P 500 index is a widely followed stock average, like the Dow Jones Industrial Average (DJIA). It has a broader representation of the stock market since this average is based on 500 stocks, whereas the DJIA is based on only 30. The problem to be approached in this chapter is to predict the S&P 500 index, given a variety of indicators and data for prior weeks.

Choosing the Right Outputs and Objective

Our objective is to forecast the S&P 500 ten weeks from now. Whereas the objective may be to predict the level of the S&P 500, it is important to simplify the job of the network by asking for a change in the level rather than for the absolute level of the index. What you want to do is give the network the ability to fit the problem at hand conveniently in the output space of the output layer. Practically speaking, you know that the output from the network cannot be outside of the 0 to 1 range, since we have used a sigmoid activation function. You could take the S&P 500 index and scale this absolute price level to this range, for example. However, you will likely end up with very small numbers that have a small range of variability. The difference from week to week, on the other hand, has a much smaller overall range, and when these differences are scaled to the 0 to 1 range, you have much more variability.

The output we choose is the change in the S&P 500 from the current week to 10 weeks from now as a percentage of the current week's value.

Choosing the Right Inputs

The inputs to the network need to be weekly changes of indicators that have some relevance to the S&P 500 index. This is a complex forecasting problem, and we can only guess at some of the relationships. This is one of the inherent strengths of using neural nets for forecasting; if a relationship is weak, the network will learn to ignore it automatically. Be cognizant that you do want to minimize the DOF as mentioned before though. In this example, we choose a data set that represents the state of the financial markets and the economy. The inputs chosen are listed as:

• Previous price action in the S&P 500 index, including the close or final value of the index
• Breadth indicators for the stock market, including the number of advancing issues and declining issues for the stocks in the New York Stock Exchange (NYSE)
• Other technical indicators, including the number of new highs and new lows achieved in the week for the NYSE market. This gives some indication about the strength of an uptrend or downtrend.
• Interest rates, including short-term interest rates in the Three-Month Treasury Bill Yield, and long-term rates in the 30-Year Treasury Bond Yield.

Other possible inputs could have been government statistics like the Consumer Price Index, Housing starts, and the Unemployment Rate. These were not chosen because long- and short-term interest rates tend to encompass this data already.

You are encouraged to experiment with other inputs and ideas. All of the data mentioned can be obtained in the public domain, such as from financial publications (Barron's, Investor's Business Daily, Wall Street Journal) and from ftp sites on the Internet for the Department of Commerce and the Federal Reserve, as well as from commercial vendors (see the Resource Guide at the end of the chapter). There are new sources cropping up on the Internet all the time. A sampling of World Wide Web addresses for commercial and noncommercial sources include:

• FINWeb, http://riskweb.bus.utexas.edu/finweb.html
• Chicago Mercantile Exchange, http://www.cme.com/cme
• SEC Edgar Database, http://town.hall.org/edgar/edgar.html
• CTSNET Business & Finance Center, http://www.cts.com/cts/biz/
• QuoteCom, http://www.quote.com
• Philadelphia Fed, http://compstat.wharton.upenn.edu:8001/~siler/ fedpage.html
• Ohio state Financial Data Finder, http://cob.ohio-state.edu/dept/fin/ osudata.html


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Choosing a Network Architecture

The input and output layers are fixed by the number of inputs and outputs we are using. In our case, the output is a single number, the expected change in the S&P 500 index 10 weeks from now. The input layer size will be dictated by the number of inputs we have after preprocessing. You will see more on this soon. The middle layers can be either 1 or 2. It is best to choose the smallest number of neurons possible for a given problem to allow for generalization. If there are too many neurons, you will tend to get memorization of patterns. We will use one hidden layer. The size of the first hidden layer is generally recommended as between one-half to three times the size of the input layer. If a second hidden layer is present, you may have between three and ten times the number of output neurons. The best way to determine optimum size is by trial and error.

NOTE: You should try to make sure that there are enough training examples for your trainable weights. In other words, your architecture may be dictated by the number of input training examples, or facts, you have. In an ideal world, you would want to have about 10 or more facts for each weight. For a 10-10-1 architecture, there are (10X10 + 10X1 = 110 weights), so you should aim for about 1100 facts. The smaller the ratio of facts to weights, the more likely you will be undertraining your network, which will lead to very poor generalization capability.

Preprocessing Data

We now begin the preprocessing effort. As mentioned before, this will likely be where you, the neural network designer, will spend most of your time.

A View of the Raw Data

Let's look at the raw data for the problem we want to solve. There are a couple of ways we can start preprocessing the data to reduce the number of inputs and enhance the variability of the data:

• Use Advances/Declines ratio instead of each value separately.
• Use New Highs/New Lows ratio instead of each value separately.

We are left with the following indicators:

1. Three-Month Treasury Bill Yield
2. 30-Year Treasury Bond Yield
3. NYSE Advancing/Declining issues
4. NYSE New Highs/New Lows
5. S&P 500 closing price

Raw data for the period from January 4, 1980 to October 28, 1983 is taken as the training period, for a total of 200 weeks of data. The following 50 weeks are kept on reserve for a test period to see if the predictions are valid outside of the training interval. The last date of this period is October 19, 1984. Let's look at the raw data now. (You get data on the disk available with this book that covers the period from January, 1980 to December, 1992.) In Figures 14.3 through 14.5, you will see a number of these indicators plotted over the training plus test intervals:

• Figure 14.3 shows you the S&P 500 stock index.

Figure 14.3 The S&P 500 Index for the period of interest.

• Figure 14.4 shows long-term bonds and short-term 3-month T-bill interest rates.

Figure 14.4 Long-term and short-term interest rates.

• Figure 14.5 shows some breadth indicators on the NYSE, the number of advancing stocks/number of declining stocks, as well as the ratio of new highs to new lows on the NYSE

Figure 14.5 Breadth indicators on the NYSE: Advancing/Declining issues and New Highs/New Lows.

A sample of a few lines looks like the following data in Table 14.1. Note that the order of parameters is the same as listed above.

Table 14.1 Raw Data
Date | 3Mo TBills | 30YrTBonds | NYSE-Adv/Dec | NYSE-NewH/NewL | SP-Close
1/4/80 | 12.11 | 9.64 | 4.209459 | 2.764706 | 106.52
1/11/80 | 11.94 | 9.73 | 1.649573 | 21.28571 | 109.92
1/18/80 | 11.9 | 9.8 | 0.881335 | 4.210526 | 111.07
1/25/80 | 12.19 | 9.93 | 0.793269 | 3.606061 | 113.61
2/1/80 | 12.04 | 10.2 | 1.16293 | 2.088235 | 115.12
2/8/80 | 12.09 | 10.48 | 1.338415 | 2.936508 | 117.95
2/15/80 | 12.31 | 10.96 | 0.338053 | 0.134615 | 115.41
2/22/80 | 13.16 | 11.25 | 0.32381 | 0.109091 | 115.04
2/29/80 | 13.7 | 12.14 | 1.676895 | 0.179245 | 113.66
3/7/80 | 15.14 | 12.1 | 0.282591 | 0 | 106.9
3/14/80 | 15.38 | 12.01 | 0.690286 | 0.011628 | 105.43
3/21/80 | 15.05 | 11.73 | 0.486267 | 0.027933 | 102.31
3/28/80 | 16.53 | 11.67 | 5.247191 | 0.011628 | 100.68
4/3/80 | 15.04 | 12.06 | 0.983562 | 0.117647 | 102.15
4/11/80 | 14.42 | 11.81 | 1.565854 | 0.310345 | 103.79
4/18/80 | 13.82 | 11.23 | 1.113287 | 0.146341 | 100.55
4/25/80 | 12.73 | 10.59 | 0.849807 | 0.473684 | 105.16
5/2/80 | 10.79 | 10.42 | 1.147465 | 1.857143 | 105.58
5/9/80 | 9.73 | 10.15 | 0.513052 | 0.473684 | 104.72
5/16/80 | 8.6 | 9.7 | 1.342444 | 6.75 | 107.35
5/23/80 | 8.95 | 9.87 | 3.110825 | 26 | 110.62

Highlight Features in the Data

For each of the five inputs, we want use a function to highlight rate of change types of features. We will use the following function (as originally proposed by Jurik) for this purpose.

ROC(n) = (input(t) - BA(t - n)) / (input(t)+ BA(t - n))

where: input(t) is the input's current value and BA(t - n) is a five unit block average of adjacent values centered around the value n periods ago.

Now we need to decide how many of these features we need. Since we are making a prediction 10 weeks into the future, we will take data as far back as 10 weeks also. This will be ROC(10). We will also use one other rate of change, ROC(3). We have now added 5*2 = 10 inputs to our network, for a total of 15. All of the preprocessing can be done with a spreadsheet.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Here's what we get (Table 14.2) after doing the block averages. Example : BA3MoBills for 1/18/80 = (3MoBills(1/4/80) + 3MoBills(1/11/80) + 3MoBills(1/18/80) + 3MoBills(1/25/80) + 3MoBills(2/1/80))/5.

Table 14.2 Data after Doing Block Averages
Date | 3MoBills | LngBonds | NYSE-
1/4/80 | 12.11 | 9.64 | 4.209459
1/11/80 | 11.94 | 9.73 | 1.649573
1/18/80 | 11.9 | 9.8 | 0.881335
1/25/80 | 12.19 | 9.93 | 0.793269
2/1/80 | 12.04 | 10.2 | 1.16293
2/8/80 | 12.09 | 10.48 | 1.338415
2/15/80 | 12.31 | 10.96 | 0.338053
2/22/80 | 13.16 | 11.25 | 0.32381
2/29/80 | 13.7 | 12.14 | 1.676895
3/7/80 | 15.14 | 12.1 | 0.282591
3/14/80 | 15.38 | 12.01 | 0.690286
3/21/80 | 15.05 | 11.73 | 0.486267
3/28/80 | 16.53 | 11.67 | 5.247191
4/3/80 | 15.04 | 12.06 | 0.983562
4/11/80 | 14.42 | 11.81 | 1.565854
4/18/80 | 13.82 | 11.23 | 1.113287
4/25/80 | 12.73 | 10.59 | 0.849807
5/2/80 | 10.79 | 10.42 | 1.147465
5/9/80 | 9.73 | 10.15 | 0.513052
5/16/80 | 8.6 | 9.7 | 1.342444
5/23/80 | 8.95 | 9.87 | 3.110825
NYSE-Adv/Dec | SP-Close NewH/NewL | BA3MoB | BALngBnd
2.764706 | 106.52 |  |
21.28571 | 109.92 |  |
4.210526 | 111.07 | 12.036 | 9.86
3.606061 | 113.61 | 12.032 | 10.028
2.088235 | 115.12 | 12.106 | 10.274
2.936508 | 117.95 | 12.358 | 10.564
0.134615 | 115.41 | 12.66 | 11.006
0.109091 | 115.04 | 13.28 | 11.386
0.179245 | 113.66 | 13.938 | 11.692
0 | 106.9 | 14.486 | 11.846
0.011628 | 105.43 | 15.16 | 11.93
0.027933 | 102.31 | 15.428 | 11.914
0.011628 | 100.68 | 15.284 | 11.856
0.117647 | 102.15 | 14.972 | 11.7
0.310345 | 103.79 | 14.508 | 11.472
0.146341 | 100.55 | 13.36 | 11.222
0.473684 | 105.16 | 12.298 | 10.84
1.857143 | 105.58 | 11.134 | 10.418
0.473684 | 104.72 | 10.16 | 10.146
6.75 | 107.35 | 7.614 | 8.028
26 | 110.62 | 5.456 | 5.944
BAA/D | BAH/L | BAClose |
1.739313 | 6.791048 | 111.248 |
1.165104 | 6.825408 | 113.534 |
0.9028 | 2.595189 | 114.632 |
0.791295 | 1.774902 | 115.426 |
0.968021 | 1.089539 | 115.436 |
0.791953 | 0.671892 | 113.792 |
0.662327 | 0.086916 | 111.288 |
0.69197 | 0.065579 | 108.668 |
1.676646 | 0.046087 | 105.796 |
1.537979 | 0.033767 | 103.494 |
1.794632 | 0.095836 | 102.872 |
1.879232 | 0.122779 | 101.896
1.95194 | 0.211929 | 102.466 |
1.131995 | 0.581032 | 103.446 |
1.037893 | 0.652239 | 103.96 |
0.993211 | 1.94017 | 104.672 |
1.392719 | 7.110902 | 106.686 |
1.222757 | 7.016165 | 85.654 |
0.993264 | 6.644737 | 64.538 |


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Now let's look at the rest of this table, which is made up of the new 10 values of ROC indicators (Table 14.3).

Table 14.3 Added Rate of Change (ROC) Indicators
Date | ROC3_3Mo | ROC3_Bond | ROC10_AD | ROC3_H/L | ROC3_SPC
1/4/80 |  |  |  |
1/11/80 |  |  |  |  |
1/18/80 |  |  |  |  |
1/25/80 |  |  |  |  |
2/1/80 |  |  |  |  |
2/8/80 | 0.002238 | 0.030482 | -0.13026 | -0.39625 | 0.029241
2/15/80 | 0.011421 | 0.044406 | -0.55021 | -0.96132 | 0.008194
2/22/80 | 0.041716 | 0.045345 | -0.47202 | -0.91932 | 0.001776
2/29/80 | 0.0515 | 0.069415 | 0.358805 | -0.81655 | -0.00771
3/7/80 | 0.089209 | 0.047347 | -0.54808 | -1 | -0.03839
3/14/80 | 0.073273 | 0.026671 | -0.06859 | -0.96598 | -0.03814
3/21/80 | 0.038361 | 0.001622 | -0.15328 | -0.51357 | -0.04203
3/28/80 | 0.065901 | -0.00748 | 0.766981 | -0.69879 | -0.03816
4/3/80 | -0.00397 | 0.005419 | -0.26054 | 0.437052 | -0.01753
4/11/80 | -0.03377 | -0.00438 | 0.008981 | 0.437052 | -0.01753
4/18/80 | -0.0503 | -0.02712 | -0.23431 | 0.803743 | 0.001428
4/25/80 | -0.08093 | -0.0498 | -0.37721 | 0.58831 | 0.015764
5/2/80 | -0.14697 | -0.04805 | -0.25956 | 0.795146 | 0.014968
5/9/80 | -0.15721 | -0.05016 | -0.37625 | -0.10178 | 0.00612
5/16/80 | -0.17695 | -0.0555 | 0.127944 | 0.823772 | 0.016043
5/23/80 | -0.10874 | -0.02701 | 0.515983 | 0.86112 | 0.027628
ROC10_3Mo | ROC10_Bnd | ROC10_AD | ROC10_HL | ROC10_SP
0.15732 | 0.084069 | 0.502093 | -0.99658 | -0.04987
0.111111 | 0.091996 | -0.08449 | -0.96611 | -0.05278
0.087235 | 0.069553 | 0.268589 | -0.78638 | -0.04964
0.055848 | 0.030559 | 0.169062 | -0.84766 | -0.06888
0.002757 | -0.01926 | -0.06503 | -0.39396 | -0.04658
-0.10345 | -0.0443 | 0.183309 | 0.468658 | -0.03743
-0.17779 | -0.0706 | -0.127 | 0.689919 | -0.03041
-0.25496 | -0.0996 | 0.319735 | 0.980756 | -0.0061
-0.25757 | -0.0945 | 0.299569 | 0.996461 | 0.02229

NOTE: Note that you don't get completed rows until 3/28/90, since we have a ROC indicator dependent on a Block Average value 10 weeks before it. The first block average value is generated 1/1/80, two weeks after the start of the data set. All of this indicates that you will need to discard the first 12 values in the dataset to get complete rows, also called complete facts.

Normalizing the Range

We now have values in the original five data columns that have a very large range. We would like to reduce the range by some method. We use the following function:

new value = (old value - Mean)/ (Maximum Range)

This relates the distance from the mean for a value in a column as a fraction of the Maximum range for that column. You should note the value of the Maximum range and Mean, so that you can un-normalize the data when you get a result.

The Target

We've taken care of all our inputs, which number 15. The final piece of information is the target. The objective as stated at the beginning of this exercise is to predict the percentage change 10 weeks into the future. We need to time shift the S&P 500 close 10 weeks back, and then calculate the value as a percentage change as follows:

Result = 100 X ((S&P 10 weeks ahead) - (S&P this week))/(S&P this week).

This gives us a value that varies between -14.8 to and + 33.7. This is not in the form we need yet. As you recall, the output comes from a sigmoid function that is restricted to 0 to +1. We will first add 14.8 to all values and then scale them by a factor of 0.02. This will result in a scaled target that varies from 0 to 1.

scaled target = (result + 14.8) X 0.02

The final data file with the scaled target shown along with the scaled original six columns of data is shown in Table 14.4.

Table 14.4 Normalized Ranges for Original Columns and Scaled Target
Date | S_3MOBill | S_LngBnd | S_A/D
3/28/80 | 0.534853 | -0.01616 | 0.765273
4/3/80 | 0.391308 | 0.055271 | -0.06356
4/11/80 | 0.331578 | 0.009483 | 0.049635
4/18/80 | 0.273774 | -0.09674 | -0.03834
4/25/80 | 0.168765 | -0.21396 | -0.08956
5/2/80 | -0.01813 | -0.2451 | -0.0317
5/9/80 | -0.12025 | -0.29455 | -0.15503
5/16/80 | -0.22912 | -0.37696 | 0.006205
5/23/80 | -0.1954 | -0.34583 | 0.349971
S_H/L | S_SPC | Result | Scaled Target
-0.07089 | -0.51328 | 12.43544 | 0.544709
-0.07046 | -0.49236 | 12.88302 | 0.55366
-0.06969 | -0.46901 | 9.89498 | 0.4939
-0.07035 | -0.51513 | 15.36549 | 0.60331
-0.06903 | -0.44951 | 11.71548 | 0.53031
-0.06345 | -0.44353 | 11.61205 | 0.528241
-0.06903 | -0.45577 | 16.53934 | 0.626787
-0.04372 | -0.41833 | 12.51048 | 0.54621
0.033901 | -0.37179 | 9.573314 | 0.487466


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Storing Data in Different Files

You need to place the first 200 lines in a training.dat file (provided for you in the accompanying diskette) and the subsequent 40 lines of data in the another test.dat file for use in testing. You will read more about this shortly. There is also more data than this provided on this diskette in raw form for you to do further experiments.

Training and Testing

With the training data available, we set up a simulation. The number of inputs are 15, and the number of outputs is 1. A total of three layers are used with a middle layer of size 5. This number should be made as small as possible with acceptable results. The optimum sizes and number of layers can only be found by much trial and error. After each run, you can look at the error from the training set and from the test set.

Using the Simulator to Calculate Error

You obtain the error for the test set by running the simulator in Training mode (you need to temporarily copy the test data with expected outputs to the training.dat file) for one cycle with weights loaded from the weights file. Since this is the last and only cycle, weights do not get modified, and you can get a reading of the average error. Refer to Chapter 13 for more information on the simulator's Test and Training modes. This approach has been taken with five runs of the simulator for 500 cycles each. Table 14.5 summarizes the results along with the parameters used. The error gets better and better with each run up to run # 4. For run #5, the training set error decreases, but the test set error increases, indicating the onset of memorization. Run # 4 is used for the final network results, showing test set RMS error of 13.9% and training set error of 6.9%.

Table 14.5 Results of Training the Backpropagation Simulator for Predicting the Percentage Change in the S&P 500 Index
Run# | Tolerance | Beta | Alpha | NF | max cycles | cycles run | training set error | test set error
1 | 0.001 | 0.5 | 0.001 | 0.0005 | 500 | 500 | 0.150938 | 0.25429
2 | 0.001 | 0.4 | 0.001 | 0.0005 | 500 | 500 | 0.114948 | 0.185828
3 | 0.001 | 0.3 | 0 | 0 | 500 | 500 | 0.0936422 | 0.148541
4 | 0.001 | 0.2 | 0 | 0 | 500 | 500 | 0.068976 | 0.139230
5 | 0.001 | 0.1 | 0 | 0 | 500 | 500 | 0.0621412 | 0.143430

NOTE: If you find the test set error does not decrease much, whereas the training set error continues to make substantial progress, then this means that memorization is starting to set in (run#5 in example). It is important to monitor the test set(s) that you are using while you are training to make sure that good, generalized learning is occurring versus memorizing or overfitting the data. In the case shown, the test set error continued to improve until run#5, where the test set error degraded. You need to revisit the 12-step process to forecasting model design to make any further improvements beyond what was achieved.

To see the exact correlation, you can copy any period you'd like, with the expected value output fields deleted, to the test.dat file. Then you run the simulator in Test mode and get the output value from the simulator for each input vector. You can then compare this with the expected value in your training set or test set.

Now that you're done, you need to un-normalize the data back to get the answer in terms of the change in the S&P 500 index. What you've accomplished is a way in which you can get data from a financial newspaper, like Barron's or Investor's Business Daily, and feed the current week's data into your trained neural network to get a prediction of what the S&P 500 index is likely to do ten weeks from now.

Here are the steps to un-normalize:

1. Take the predicted scaled target value and calculate, the result value as Result = (Scaled target/0.02) - 14.8
2. Take the result from step 1 (which is the percentage change 10 weeks from now) and calculate the projected value, Projected S&P 10 weeks from now = (This week's S&P value)(1+ Result/100)

Only the Beginning

This is only a very brief illustration (not meant for trading !) of what you can do with neural networks in financial forecasting. You need to further analyze the data, provide more predictive indicators, and optimize/redesign your neural network architecture to get better generalization and lower error. You need to present many, many more test cases representing different market conditions to have a robust predictor that can be traded with. A graph of the expected and predicted output for the test set and the training set is shown in Figure 14.6. Here, the normalized values are used for the output. Note that the error is about 13.9% on average over the test set and 6.9% over the training set. You can see that the test set did well in the beginning, but showed great divergence in the last few weeks.

Figure 14.6 Comparison of predicted versus actual for the training and test data sets.

The preprocessing steps shown in this chapter should serve as one example of the kinds of steps you can use. There are a vast variety of analysis and statistical methods that can be used in preprocessing. For applying fuzzy data, you can use a program like the fuzzifier program that was developed in Chapter 3 to preprocess some of the data.

What's Next?

There are many other experiments you can do from here on. The example chosen was in the field of financial forecasting. But you could certainly try the simulator on other problems like sales forecasting or perhaps even weather forecasting. The key to all of the applications though, is how you present and enhance data, and working through parameter selection by trial and error. Before concluding this chapter, we will cover more topics in preprocessing and present some case studies in financial forecasting. You should consider the suggestions made in the 12-step approach to forecasting model design and research some of the resources listed at the end of this chapter if you have more interest in this area.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Technical Analysis and Neural Network Preprocessing

We cannot overstate the importance of preprocessing in developing a forecasting model. There is a large body of information related to the study of financial market behavior called Technical Analysis. You can use the mathematical studies defined by Technical Analysis to preprocess your input data to reveal predictive features. We will present a sampling of Technical Analysis studies that can be used, with formulae and graphs.

Moving Averages

Moving averages are used very widely to capture the underlying trend of a price move. Moving averages are simple filters that average data over a moving window. Popular moving averages include 5-, 10-, and 20 period moving averages. The formula is shown below for a simple moving average, SMA:

     SMAt = ( Pt + Pt-1 + ...  Pt-n)/ n

     where  n = the number of time periods back

        P-n= price at n time periods back

An exponential moving average is a weighted moving average that places more weight on the most recent data. The formula for this indicator, EMA is as follows:

     EMAt = (1 - a)Pt + a ( EMAt-1)

     where  a = smoothing constant  (typical 0.10)

        Pt= price at time t

Momentum and Rate of Change

Momentum is really velocity, or rate of price change with time. The formula for this is

     Mt =  ( Pt  -  Pt-a  )

     where  a = lookback parameter

     for a 5-day momentum value, a = 5

The Rate of Change indicator is actually a ratio. It is the current price divided by the price some interval, a, ago divided by a constant. Specifically,

     ROC = Pt / Pt-a  x 1000

Relative Strength Index

The Relative Strength Index (RSI) is the strength of up closes versus down closes over a certain time interval. It is calculated over a time interval T as :

     RSI = 100 - [ 100 / (1 + RS )]

     where  RS = average of x days' up closes/ average of x days' down

  closes

A typical time interval, T, is 14 days. The assumption with the use of RSI is that higher values up closes relative to down closes indicates a strong market, and the opposite indicates weak markets.

Percentage R

This indicator measures where in a recent range of prices today's price falls. The indicator assumes that prices regress to their mean. A low %R indicates that prices are hitting the ceiling of a range, while a high %R indicates that prices are at their low in a range. The formula is:

     %R = 100 x (HighX - P)/(HighX - LowX)

     where  HighX is the highest price over the price interval of interest

        LowX is the lowest price over the price interval of interest

        P is the current price

Herrick Payoff Index

This indicator makes use of other market data that is available besides price information. It uses the volume of the security, which, for a stock, is the number of shares traded for a stock during a particular interval. It also uses the open interest, which is the value of the total number of open trades at a particular time. For a commodity future, this is the number of open short and long positions. This study attempts to measure the flow of money in and out of a market. The formula for this is as follows (note that a tick is the smallest permissible move in a given market) :

     Let MP = mean price over a particular interval

     OI = the larger of yesterday's or today's open interest

then

     K =[ (MPt - MPt-1 ) x dollar value of 1 tick move x volume ]

       x [1 +/-  2/OI]

     HPIt = HPIt-1  + [ 0.1 x (K -      HPIt-1 )] / 100,000

MACD

The MACD (moving average convergence divergence) indicator is the difference between two moving averages, and it tells you when short-term overbought or oversold conditions exist in the market. The formula is as follows:

     Let OSC = EMA1 - EMA2,

     where  EMA1 is for one smoothing constant and time period, for example

     0.15 and 12   weeks

          EMA2 is for another smoothing constant and time period,

           for example

     0.075 and 26   weeks

then

    MACDt = MACDt-1  + K x ( OSCt - MACDt-1 )

    where K is a smoothing constant, for example, 0.2

The final formula effectively does another exponential smoothing on the difference of the two moving averages, for example, over a 9-week period.

"Stochastics"

This indicator has absolutely nothing to do with stochastic processes. The reason for the name is a mystery, but the indicator is composed of two parts: %K and %D, which is a moving average of %K. The crossover of these lines indicates overbought and oversold areas. The formulas follow:

     Raw %K = 100 x (P - LowX )/(HighX - LowX)

     %Kt = [( %Kt-1   x 2  ) + Raw      %Kt ] /3

     %Dt = [( %Dt-1   x 2  ) + %Kt ] /3


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

On-Balance Volume

The on-balance volume (OBV) indicator was created to try to uncover accumulation and distribution patterns of large player in the stock market. This is a cumulative sum of volume data, specified as follows:

If today's close is greater than yesterday's close

OBVt = OBVt-1 + 1

If today's close is less than yesterday's close

OBVt = OBVt-1 - 1

The absolute value of the index is not important; attention is given only to the direction and trend.

Accumulation-Distribution

This indicator does for price what OBV does for volume.

If today's close is greater than yesterday's close:

ADt = ADt-1 + (Closet - Lowt)

If today's close is less than yesterday's close

ADt = ADt-1 + (Hight - Closet)

Now let's examine how these indicators look. Figure 14.7 shows a bar chart, which is a chart of price data versus time, along with the following indicators:

• Ten-unit moving average
• Ten-unit exponential moving average
• Momentum
• MACD
• Percent R

Figure 14.7 Five minute bar chart of the S&P 500 Sept 95 Futures contract with several technical indicators displayed.

The time period shown is 5 minute bars for the S&P 500 September 1995 Futures contract. The top of each bar indicates the highest value ("high") for that time interval, the bottom indicates the lowest value("low"), and the horizontal lines on the bar indicate the initial ("open") and final ("close") values for the time interval.

Figure 14.8 shows another bar chart for Intel Corporation stock for the period from December 1994 to July 1995, with each bar representing a day of activity. The following indicators are displayed also.

• Rate of Change
• Relative Strength
• Stochastics
• Accumulation-Distribution

Figure 14.8 Daily bar chart of Intel Corporation with several technical indicators displayed.

You have seen a few of the hundreds of technical indicators that have been invented to date. New indicators are being created rapidly as the field of Technical Analysis gains popularity and following. There are also pattern recognition studies, such as formations that resemble flags or pennants as well as more exotic types of studies, like Elliot wave counts. You can refer to books on Technical Analysis (e.g., Murphy) for more information about these and other studies.

Neural preprocessing with Technical Analysis tools as well as with traditional engineering analysis tools such as Fourier series, Wavelets, and Fractals can be very useful in finding predictive patterns for forecasting.

What Others Have Reported

In this final section of the chapter, we outline some case studies documented in periodicals and books, to give you an idea of the successes or failures to date with neural networks in financial forecasting. Keep in mind that the very best (= most profitable) results are usually never reported (so as not to lose a competitive edge) ! Also, remember that the market inefficiencies exploited yesterday may no longer be the same to exploit today.

Can a Three-Year-Old Trade Commodities?

Well, Hillary Clinton can certainly trade commodities, but a three-year-old, too? In his paper, "Commodity Trading with a Three Year Old," J. E. Collard describes a neural network with the supposed intelligence of a three-year-old. The application used a feedforward backpropagation network with a 37-30-1 architecture. The network was trained to buy ("go long") or sell ("go short") in the live cattle commodity futures market. The training set consisted of 789 facts for trading days in 1988, 1989, 1990, and 1991. Each input vector consisted of 18 fundamental indicators and six market technical variables (Open, High, Low, Close, Open Interest, Volume). The network could be trained for the correct output on all but 11 of the 789 facts.

The fully trained network was used on 178 subsequent trading days in 1991. The cumulative profit increased from $0 to $1547.50 over this period by trading one live cattle contract. The largest loss in a trade was $601.74 and the largest gain in a trade was $648.30.

Forecasting Treasury Bill and Treasury Note Yields

Milam Aiken designed a feedforward backpropagation network that predicted Treasury Bill Rates and compared the forecast he obtained with forecasts made by top U.S. economists. The results showed the neural network, given the same data, made better predictions (.18 versus .71 absolute error). Aiken used 250 economic data series to see correlation to T-Bills and used only the series that showed leading correlation: Dept. of Commerce Index of Leading Economic Indicators, the Center for International Business Cycle Research (CIBCR) Short Leading Composite Index, and the CIBCR Long Leading Composite Index. Prior data for these three indicators for the past four years (total 12 inputs) was used to predict the average annual T-Bill rate (one output) for the current year.

Guido Deboeck and Masud Cader designed profitable trading systems for two-year and 10-year treasury securities. They used feedforward neural networks with a learning algorithm called extended-delta-bar-delta (EDBD), which is a variant of backpropagation. Training samples composed of 100 facts were selected from 1120 trading days spanning from July 1 1989 to June 30, 1992. The test period consisted of more than 150 trading days from July 1, 1992 to December 30, 1992. Performance on the test set was monitored every N thousand training cycles, and the training procedure was stopped when performance degraded on the test set. (This is the same procedure we used when developing a model for the S&P 500.)

A criterion used to judge model performance was the ratio of the average profit divided by the maximum drawdown, which is the largest unrealized loss seen during the trading period. A portfolio of separate designed trading systems for two-year and 10-year securities gave the following performance: Over a period of 4.5 years, the portfolio had 133 total trades with 65% profitable trades and the maximum drawdown of 64 basis points, or thousands of units for bond yields. The total gain was 677 basis points over that period with a maximum gain in one trade of 52 basis points and maximum loss in one trade of 47 basis points.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The stability and robustness of this system was checked by using over 1000 moving time windows of 3-month, 6-month, and 12-month duration over the 4.5-year interval and noting the standard deviations in profits and maximum drawdown. The maximum drawdown varied from 30 to 48 basis points.

Neural Nets versus Box-Jenkins Time-Series Forecasting

Ramesh Sharda and Rajendra Patil used a standard 12-12-1 feedforward backpropagation network and compared the results with Box-Jenkins methodology for time-series forecasting. Box-Jenkins forecasting is traditional time-series forecasting technique. The authors used 75 different time series for evaluation. The results showed that neural networks achieved better MAPE (mean absolute percentage error) with a mean over all 75 time series MAPEs of 14.67 versus 15.94 for the Box-Jenkins approach.

Neural Nets versus Regression Analysis

Leorey Marquez et al. compared neural network modeling with standard regression analysis. The authors used a feedforward backpropagation network with a structure of 1-6-1. They used three functional forms found in regression analysis:

1. Y = B0 + B1 X + e
2. Y = B0 + B1 log(X) + e
3. Y = B0 + B1/X + e

For each of these forms, 100 pairs of (x,y) data were generated for this "true" model.

Now the neural network was trained on these 100 pairs of data. An additional 100 data points were generated by the network to test the forecasting ability of the network. The results showed that the neural network achieved a MAPE within 0.6% of the true model, which is a very good result. The neural network model approximated the linear model best. An experiment was also done with intentional mis-specification of some data points. The neural network model did well in these cases also, but comparatively worse for the reciprocal model case.

Hierarchical Neural Network

Mendelsohn developed a multilevel neural network as shown in Figure 14.9. Here five neural networks are arranged such that four network outputs feed that final network. The four networks are trained to produce the High, Low, short-term trend, and medium-term trend for a particular financial instrument. The final network takes these four outputs as input and produces a turning point indicator.

Figure 14.9 Hierarchical neural network system to predict turning points.

Each network was trained and tested with 1200 fact days spanning 1988 to 1992 (33% used for testing). Preprocessing was accomplished by using differences of the inputs and with some technical analysis studies:

• Moving averages
• Exponential moving averages
• Stochastic indicators

For the network that produces a predicted High value, the average error ranged between 7.04% and 7.65% for various financial markets over the test period, including Treasury Bonds, Eurodollar, Japanese Yen, and S&P 500 futures contracts.

The Walk-Forward Methodology of Market Prediction

A methodology that is sometimes used in neural network design is walk-forward training and testing. This means that you choose an interval of time (e.g., six months) over which you train a neural network and test the network over a subsequent interval. You then move the training window and testing window forward one month, for example, and repeat the exercise. You do this for the time period of interest to see your forecasting results. The advantage of this approach is that you maximize the network's ability to model the recent past in making a prediction. The disadvantage is that the network forget characteristics of the market that happened prior to the training window.

Takashi Kimoto et al. used the walk forward methodology in designing a trading system for Fujitsu and Nikko Securities. They also, like Mendelsohn, use a hierarchical neural network composed of individual feedforward neural networks. Prediction of the TOPIX, which is the Japanese equivalent of the Dow Jones Industrial Average, was performed for 33 months from January 1987 to September 1980. Four networks were used in the first level of the hierarchy trained on price data and economic data. The results were fed to a final network that generated buy and sell signals. The performance of the trading system achieved a result that is 20% better than a buy and hold strategy for the TOPIX.

Dual Confirmation Trading System

Jeremy Konstenius, discusses a trading system for the S&P 400 index with a holographic neural network, which is unlike the feedforward backpropagation neural network. The holographic network uses complex numbers for data input and output from neurons, which are mathematically more complex than feedforward network neurons. The author uses two trained networks to forecast the next day's direction based on data for the past 10 days. Each network uses input data that is detrended, by subtracting a moving average from the data. Network 1 uses detrended closing values. Network 2 uses detrended High values. If both networks agree, or confirm each other, then a trade is made. There is no trade otherwise.

Network 1 showed an accuracy of 61.9% for the five-month test period (the training period spanned two years prior to the test period), while Network 2 also showed an accuracy of 61.9%. Using the two networks together, Konstenius achieved an accuracy of 65.82%.

A Turning Point Predictor

This neural network approach is discussed by Michitaka Kosaka et al. (1991).

They discuss applying the feedforward backpropagation network to develop buy/sell signals for securities. You would gather time-series data on stock prices, and want to find trends in the data so that changes in the direction of the trend provide you the turning points, which you interpret as signals to buy or sell.

You will need to list these factors that you think have any influence on the price of a security you are studying. You need to also determine how you measure these factors. You then formulate a nonlinear function combining the factors on your list and the past however many prices of your security (your time series data).

The function has the form, as Michitaka Kosaka, et al. (1991) put it,

     p(t + h) = F(x(t), x(t -1), ... , f1, f2, ... )

     where

     f1, f2, represent factors on your list,

     x(t) is the price of your stock at time t,

     p(t + h) is the turning point of security price at time t + h, and

     p(t + h) = -1 for a turn from downward to upward,

     p(t + h) = +1 for a turn from upward to downward,

     p(t + h) = 0 for no change and therefore no turn

Here you vary h through the values 1, 2, etc. as you move into the future one day (time period) at a time. Note that the detailed form of the function F is not given. This is for you to set up as you see fit.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

You can set up a similar function for x(t + h), the stock price at time t + h, and have a separate network computing it using the backpropagation paradigm. You will then be generating future prices of the stock and the future buy/sell signals hand in hand, but parallel.

Michitaka Kosaka, et al. (1991) report that they used time-series data over five years to identify the network model, and time-series data over one year to evaluate the model's forecasting performance, with a success rate of 65% for turning points.

The S&P 500 and Sunspot Predictions

Michael Azoff in his book on time-series forecasting with neural networks (see references) creates neural network systems for predicting the S&P 500 index as well as for predicting chaotic time series, such as sunspot occurrences. Azoff uses feedforward backpropagation networks, with a training algorithm called adaptive steepest descent, a variation of the standard algorithm. For the sunspot time series, and an architecture of 6-5-1, and a ratio of training vectors to trainable weights of 5.1, he achieves training set error of 12.9% and test set error of 21.4%. This series was composed of yearly sunspot numbers for the years 1706 to 1914. Six years of consecutive annual data were input to the network.

One network Azoff used to forecast the S&P 500 index was a 17-7-1 network. The training vectors to trainable weights ratio was 6.1. The achieved training set error was 3.29%, and on the test set error was 4.67%. Inputs to this network included price data, a volatility indicator, which is a function of the range of price movement, and a random walk indicator, a technical analysis study.

A Critique of Neural Network Time-Series Forecasting for Trading

Michael de la Maza and Deniz Yuret, managers for the Redfire Capital Management Group, suggest that risk-adjusted return, and not mean-squared error should be the metric to optimize in a neural network application for trading. They also point out that with neural networks, like with statistical methods such as linear regression, data facts that seem unexplainable can't be ignored even if you want them to be. There is no equivalent for a "don't care," condition for the output of a neural network. This type of condition may be an important option for trading environments that have no "discoverable regularity" as the authors put it, and therefore are really not tradable. Some solutions to the two problems posed are given as follows:

• Use an algorithm other than backpropagation, which allows for maximization of risk-adjusted return, such as simulated annealing or genetic algorithms.
• Transform the data input to the network so that minimizing mean-squared error becomes equivalent to maximizing risk-adjusted return.
• Use a hierarchy (see hierarchical neural network earlier in this section) of neural networks, with each network responsible for detecting features or regularities from one component of the data.

Resource Guide for Neural Networks and Fuzzy Logic in Finance

Here is a sampling of resources compiled from trade literature:

NOTE: We do not take responsibility for any errors or omissions.

Magazines

Technical Analysis of Stocks and Commodities
Technical Analysis, Inc., 3517 S.W. Alaska St., Seattle, WA 98146.
Futures
Futures Magazine, 219 Parkade, Cedar Falls, IA 50613.
AI in Finance
Miller Freeman Inc, 600 Harrison St., San Francisco, CA 94107
NeuroVest Journal
P.O. Box 764, Haymarket, VA 22069
IEEE Transactions on Neural Networks
IEEE Service Center, 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855

Particularly worthwhile is an excellent series of articles by consultant Murray Ruggiero Jr., in Futures magazine on neural network design and trading system design in issues spanning July '94 through June '95.

Books

Azoff, Michael, Neural Network Time Series Forecasting of Financial Markets, John Wiley and Sons, New York, 1994.
Lederman, Jess, Virtual Trading, Probus Publishing, 1995.
Trippi, Robert, Neural Networks in Finance and Investing, Probus Publishing, 1993.

Book Vendors

Traders Press, Inc. (800) 927-8222
P.O. Box 6206, Greenville, SC 29606
Traders' Library (800) 272-2855
9051 Red Branch Rd., Suite M, Columbia, MD 21045


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Consultants

Mark Jurik
Jurik Research
P.O. Box 2379, Aptos, CA 95001
Hy Rao
Via Software Inc, v: (609) 275-4786, fax: (609) 799-7863
BEI Suite 480, 660 Plainsboro Rd., Plainsboro, NJ 08536
ViaSW@aol.com
Mendelsohn Enterprises Inc.
25941 Apple Blossom Lane
Wesley Chapel, FL 33544
Murray Ruggiero Jr.
Ruggiero Associates,
East Haven, CT
The Schwartz Associates (800) 965-4561
801 West El Camino Real, Suite 150, Mountain View, CA 94040

Historical Financial Data Vendors

CSI (800) CSI-4727
200 W. Palmetto Park Rd., Boca Raton, FL 33432
Free Financial Network, New York (212) 838-6324
Genesis Financial Data Services (800) 808-DATA
411 Woodmen, Colorado Springs, CO 80991
Pinnacle Data Corp. (800) 724-4903
460 Trailwood Ct., Webster, NY 14580
Stock Data Corp. (410) 280-5533
905 Bywater Rd., Annapolis, MD 21401
Technical Tools Inc. (800) 231-8005
334 State St., Suite 201, Los Altos, CA 94022
Tick Data Inc. (800) 822-8425
720 Kipling St., Suite 115, Lakewood, CO 80215
Worden Bros., Inc. (800) 776-4940
4905 Pine Cone Dr., Suite 12, Durham, NC 27707

Preprocessing Tools for Neural Network Development

NeuralApp Preprocessor for Windows
Via Software Inc., v: (609) 275-4786 fax: (609) 799-7863
BEI Suite 480, 660 Plainsboro Rd., Plainsboro, NJ 08536
ViaSW@aol.com
Stock Prophet
Future Wave Software (310) 540-5373
1330 S. Gertruda Ave., Redondo Beach, CA 90277
Wavesamp & Data Decorrelator & Reducer
TSA (800) 965-4561
801 W. El Camino Real, #150, Mountain. View, CA 94040

Genetic Algorithms Tool Vendors

C Darwin
ITANIS International Inc.
1737 Holly Lane, Pittsburgh, PA 15216
EOS
Man Machine Interfaces Inc. (516) 249-4700
555 Broad Hollow Rd., Melville, NY 11747
Evolver
Axcelis, Inc. (206) 632-0885
4668 Eastern Ave. N., Seattle, WA 98103

Fuzzy Logic Tool Vendors

CubiCalc
HyperLogic Corp. (619) 746-2765
1855 East Valley Pkwy., Suite 210, Escondido, CA 92027
TILSHELL
Togai InfraLogic Inc.
5 Vanderbilt, Irvine, CA 92718

Neural Network Development Tool Vendors

Braincel
Promised Land Technologies (203) 562-7335
195 Church St., 8th Floor, New Haven, CT 06510
BrainMaker
California Scientific Software (916) 478-9040
10024 Newtown Rd., Nevada City, CA 95959
ForecastAgent for Windows, ForecastAgent for Windows 95
Via Software Inc., v: (609) 275-4786 fax: (609) 799-7863
BEI Suite 480, 660 Plainsboro Rd., Plainsboro, NJ 08536
ViaSW@aol.com
InvestN 32
RaceCom, Inc. (800) 638-8088
555 West Granada Blvd., Suite E-10, Ormond Beach, FL 32714
NetCaster, DataCaster
Maui Analysis & Synthesis Technologies (808) 875-2516
590 Lipoa Pkwy., Suite 226, Kihei, HI 96753
NeuroForecaster
NIBS Pte. Ltd. (65) 344-2357
62 Fowlie Rd., Republic of Singapore 1542
NeuroShell
Ward Systems Group (301)662-7950
Executive Park West, 5 Hillscrest Dr., Frederick, MD 21702
NeuralWorks Predict
NeuralWare Inc. (412) 787-8222
202 Park West Dr., Pittsburgh, PA 15276
N-Train
Scientific Consultant Services (516) 696-3333
20 Stagecoach Rd., Selden, NY 11784

Summary

This chapter presented a neural network application in financial forecasting. As an example of the steps needed to develop a neural network forecasting model, the change in the Standard & Poor's 500 stock index was predicted 10 weeks out based on weekly data for five indicators. Some examples of preprocessing of data for the network were shown as well as issues in training.

At the end of the training period, it was seen that memorization was taking place, since the error in the test data degraded, whereas the error in the training set improved. It is important to monitor the error in the test data (without weight changes) while you are training to ensure that generalization ability is maintained. The final network resulted in average RMS error of 6.9 % over the training set and 13.9% error over the test set.

This chapter's example in forecasting highlights the ease of use and wide applicability of the backpropagation algorithm for large, complex problems and data sets. Several examples of research in financial forecasting were presented with a number of ideas and real-life methodologies presented.

Technical Analysis was briefly discussed with examples of studies that can be useful in preprocessing data for neural networks.

A Resource guide was presented for further information on financial applications of neural networks.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 15
Application to Nonlinear Optimization

Introduction

Nonlinear optimization is an area of operations research, and efficient algorithms for some of the problems in this area are hard to find. In this chapter, we describe the traveling salesperson problem and discuss how this problem is formulated as a nonlinear optimization problem in order to use neural networks (Hopfield and Kohonen) to find an optimum solution. We start with an explanation of the concepts of linear, integer linear and nonlinear optimization.

An optimization problem has an objective function and a set of constraints on the variables. The problem is to find the values of the variables that lead to an optimum value for the objective function, while satisfying all the constraints. The objective function may be a linear function in the variables, or it may be a nonlinear function. For example, it could be a function expressing the total cost of a particular production plan, or a function giving the net profit from a group of products that share a given set of resources. The objective may be to find the minimum value for the objective function, if, for example, it represents cost, or to find the maximum value of a profit function. The resources shared by the products in their manufacturing are usually in limited supply or have some other restrictions on their availability. This consideration leads to the specification of the constraints for the problem.

Each constraint is usually in the form of an equation or an inequality. The left side of such an equation or inequality is an expression in the variables for the problem, and the right-hand side is a constant. The constraints are said to be linear or nonlinear depending on whether the expression on the left-hand side is a linear function or nonlinear function of the variables. A linear programming problem is an optimization problem with a linear objective function as well as a set of linear constraints. An integer linear programming problem is a linear programming problem where the variables are required to have integer values. A nonlinear optimization problem has one or more of the constraints nonlinear and/or the objective function is nonlinear.

Here are some examples of statements that specify objective functions and constraints:

• Linear objective function: Maximize Z = 3X1 + 4X2 + 5.7X3
• Linear equality constraint: 13X1 - 4.5X2 + 7X3 = 22
• Linear inequality constraint: 3.6X1 + 8.4X2 - 1.7X3 ≤ 10.9
• Nonlinear objective function: Minimize Z = 5X2 + 7XY + Y2
• Nonlinear equality constraint: 4X + 3XY + 7Y + 2Y2 = 37.6
• Nonlinear inequality constraint: 4.8X + 5.3XY + 6.2Y2 ≥ 34.56

An example of a linear programming problem is the blending problem. An example of a blending problem is that of making different flavors of ice cream blending different ingredients, such as sugar, a variety of nuts, and so on, to produce different amounts of ice cream of many flavors. The objective in the problem is to find the amounts of individual flavors of ice cream to produce with given supplies of all the ingredients, so the total profit is maximized.

A nonlinear optimization problem example is the quadratic programming problem. The constraints are all linear but the objective function is a quadratic form. A quadratic form is an expression of two variables with 2 for the sum of the exponents of the two variables in each term.

An example of a quadratic programming problem, is a simple investment strategy problem that can be stated as follows. You want to invest a certain amount in a growth stock and in a speculative stock, achieving at least 25% return. You want to limit your investment in the speculative stock to no more than 40% of the total investment. You figure that the expected return on the growth stock is 18%, while that on the speculative stock is 38%. Suppose G and S represent the proportion of your investment in the growth stock and the speculative stock, respectively. So far you have specified the following constraints. These are linear constraints:

G + S = 1 | This says the proportions add up to 1.
S ≤ 0.4 | This says the proportion invested in speculative stock is no more than 40%.
1.18G + 1.38S ≥ 1.25 | This says the expected return from these investments should be at least 25%.

Now the objective function needs to be specified. You have specified already the expected return you want to achieve. Suppose that you are a conservative investor and want to minimize the variance of the return. The variance works out as a quadratic form. Suppose it is determined to be:

    2G2  + 3S2 - GS

This quadratic form, which is a function of G and S, is your objective function that you want to minimize subject to the (linear) constraints previously stated.

Neural Networks for Optimization Problems

It is possible to construct a neural network to find the values of the variables that correspond to an optimum value of the objective function of a problem. For example, the neural networks that use the Widrow-Hoff learning rule find the minimum value of the error function using the least mean squared error. Neural networks such as the feedforward backpropagation network use the steepest descent method for this purpose and find a local minimum of the error, if not the global minimum. On the other hand, the Boltzmann machine or the Cauchy machine uses statistical methods and probabilities and achieves success in finding the global minimum of an error function. So we have an idea of how to go about using a neural network to find an optimum value of a function. The question remains as to how the constraints of an optimization problem should be treated in a neural network operation. A good example in answer to this question is the traveling salesperson problem. Let's discuss this example next.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Traveling Salesperson Problem

The traveling salesperson problem is well-known in optimization. Its mathematical formulation is simple, and one can state a simple solution strategy also. Such a strategy is often impractical, and as yet, there is no efficient algorithm for this problem that consistently works in all instances. The traveling salesperson problem is one among the so- called NP-complete problems, about which you will read more in what follows. That means that any algorithm for this problem is going to be impractical with certain examples. The neural network approach tends to give solutions with less computing time than other available algorithms for use on a digital computer. The problem is defined as follows. A traveling salesperson has a number of cities to visit. The sequence in which the salesperson visits different cities is called a tour. A tour should be such that every city on the list is visited once and only once, except that he returns to the city from which he starts. The goal is to find a tour that minimizes the total distance the salesperson travels, among all the tours that satisfy this criterion.

A simple strategy for this problem is to enumerate all feasible tours—a tour is feasible if it satisfies the criterion that every city is visited but once—to calculate the total distance for each tour, and to pick the tour with the smallest total distance. This simple strategy becomes impractical if the number of cities is large. For example, if there are 10 cities for the traveling salesperson to visit (not counting home), there are 10! = 3,628,800 possible tours, where 10! denotes the factorial of 10—the product of all the integers from 1 to 10—and is the number of distinct permutations of the 10 cities. This number grows to over 6.2 billion with only 13 cities in the tour, and to over a trillion with 15 cities.

The TSP in a Nutshell

For n cities to visit, let Xij be the variable that has value 1 if the salesperson goes from city i to city j and value 0 if the salesperson does not go from city i to city j. Let dij be the distance from city i to city j. The traveling salesperson problem (TSP) is stated as follows:

Minimize the linear objective function:

subject to:

ɷXij for 1 for all i and j (integer constraint)

This is a 0-1 integer linear programming problem.

Solution via Neural Network

This section shows how the linear and integer constraints of the TSP are absorbed into an objective function that is nonlinear for solution via Neural network.

The first consideration in the formulation of an optimization problem is the identification of the underlying variables and the type of values they can have. In a traveling salesperson problem, each city has to be visited once and only once, except the city started from. Suppose you take it for granted that the last leg of the tour is the travel between the last city visited and the city from which the tour starts, so that this part of the tour need not be explicitly included in the formulation. Then with n cities to be visited, the only information needed for any city is the position of that city in the order of visiting cities in the tour. This suggests that an ordered n-tuple is associated with each city with some element equal to 1, and the rest of the n – 1 elements equal to 0. In a neural network representation, this requires n neurons associated with one city. Only one of these n neurons corresponding to the position of the city, in the order of cities in the tour, fires or has output 1. Since there are n cities to be visited, you need n2 neurons in the network. If these neurons are all arranged in a square array, you need a single 1 in each row and in each column of this array to indicate that each city is visited but only once.

Let xij be the variable to denote the fact that city i is the jth city visited in a tour. Then xij is the output of the jth neuron in the array of neurons corresponding to the ith city. You have n2 such variables, and their values are binary, 0 or 1. In addition, only n of these variables should have value 1 in the solution. Furthermore, exactly one of the x's with the same first subscript (value of i) should have value 1. It is because a given city can occupy only one position in the order of the tour. Similarly, exactly one of the x's with the same second subscript (value of j) should have value 1. It is because a given position in the tour can be only occupied by one city. These are the constraints in the problem. How do you then describe the tour? We take as the starting city for the tour to be city 1 in the array of cities. A tour can be given by the sequence 1, a, b, c, …, q, indicating that the cities visited in the tour in order starting at 1 are, a, b, c, …, q and back to 1. Note that the sequence of subscripts a, b, …, q is a permutation of 2, 3, … n – 1, xa1=1, xb2=1, etc.

Having frozen city 1 as the first city of the tour, and noting that distances are symmetric, the distinct number of tours that satisfy the constraints is not n!, when there are n cities in the tour as given earlier. It is much less, namely, n!/2n. Thus when n is 10, the number of distinct feasible tours is 10!/20, which is 181,440. If n is 15, it is still over 43 billion, and it exceeds a trillion with 17 cities in the tour. Yet for practical purposes there is not much comfort knowing that for the case of 13 cities, 13! is over 6.2 billion and 13!/26 is only 239.5 million—it is still a tough combinatorial problem.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The cost of the tour is the total distance traveled, and it is to be minimized. The total distance traveled in the tour is the sum of the distances from each city to the next. The objective function has one term that corresponds to the total distance traveled in the tour. The other terms, one for each constraint, in the objective function are expressions, each attaining a minimum value if and only if the corresponding constraint is satisfied. The objective function then takes the following form. Hopfield and Tank formulated the problem as one of minimizing energy. Thus it is, customary to refer to the value of the objective function of this problem, while using Hopfield-like neural network for its solution, as the energy level, E, of the network. The goal is to minimize this energy level.

In formulating the equation for E, one uses constant parameters A1, A2, A3, and A4 as coefficients in different terms of the expression on the right-hand side of the equation. The equation that defines E is given as follows. Note that the notation in this equation includes dij for the distance from city i to city j.

E = A1 ΣiΣk Σj≠kxikxij + A2 Σi Σk xiΣk Σj≠kxkixji + A3[( ΣiΣkxik) - n]2 + A4Σk Σk Σj≠k Σidkjxki(xj,i+1 + xj,i-1)

Our first observation at this point is that E is a nonlinear function of the x's, as you have quadratic terms in it. So this formulation of the traveling salesperson problem renders it a nonlinear optimization problem.

All the summations indicated by the occurrences of the summation symbol Σ, range from 1 to n for the values of their respective indices. This means that the same summand such as x12x33 also as x33x12, appears twice with only the factors interchanged in their order of occurrence in the summand. For this reason, many authors use an additional factor of 1/2 for each term in the expression for E. However, when you minimize a quantity z with a given set of values for the variables in the expression for z, the same values for these variables minimize any whole or fractional multiple of z, as well.

The third summation in the first term is over the index j, from 1 to n, but excluding whatever value k has. This prevents you from using something like x12x12. Thus, the first term is an abbreviation for the sum of n2(n – 1) terms with no two factors in a summand equal. This term is included to correspond to the constraint that no more than one neuron in the same row can output a 1. Thus, you get 0 for this term with a valid solution. This is also true for the second term in the right-hand side of the equation for E. Note that for any value of the index i, xii has value 0, since you are not making a move like, from city i to the same city i in any of the tours you consider as a solution to this problem. The third term in the expression for E has a minimum value of 0, which is attained if and only if exactly n of the n2 x's have value 1 and the rest 0.

The last term expresses the goal of finding a tour with the least total distance traveled, indicating the shortest tour among all possible tours for the traveling salesperson. Another important issue about the values of the subscripts on the right-hand side of the equation for E is, what happens to i + 1, for example, when i is already equal to n, and to i–1, when i is equal to 1. The i + 1 and i – 1 seem like impossible values, being out of their allowed range from 1 to n. The trick is to replace these values with their moduli with respect to n. This means, that the value n + 1 is replaced with 1, and the value 0 is replaced with n in the situations just described.

Modular values are obtained as follows. If we want, say 13 modulo 5, we subtract 5 as many times as possible from 13, until the remainder is a value between 0 and 4, 4 being 5 – 1. Since we can subtract 5 twice from 13 to get a remainder of 3, which is between 0 and 4, 3 is the value of 13 modulo 5. Thus (n + 3) modulo n is 3, as previously noted. Another way of looking at these results is that 3 is 13 modulo 5 because, if you subtract 3 from 13, you get a number divisible by 5, or which has 5 as a factor. Subtracting 3 from n + 3 gives you n, which has n as a factor. So 3 is (n + 3) modulo n. In the case of –1, by subtracting (n – 1) from it, we get -n, which can be divided by n getting –1. So (n – 1) is the value of (–1) modulo n.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Example of a Traveling Salesperson Problem for Hand Calculation

Suppose there are four cities in the tour. Call these cities, C1, C2, C3, and C4. Let the matrix of distances be the following matrix D.

           0   10   14    7

     D =  10    0    6   12

          14    6    0    9

           7   12    9    0

From our earlier discussion on the number of valid and distinct tours, we infer that there are just three such tours. Since it is such a small number, we can afford to enumerate the three tours, find the energy values associated with them, and pick the tour that has the smallest energy level among the three. The three tours are:

• Tour 1. 1 - 2 - 3 - 4 - 1 In this tour city 2 is visited first, followed by city 3, from where the salesperson goes to city 4, and then returns to city 1. For city 2, the corresponding ordered array is (1, 0, 0, 0), because city 2 is the first in this permutation of cities. Then x21 = 1, x22 = 0, x23 = 0, x24 = 0. Also (0, 1, 0, 0), (0, 0, 1, 0), and (0, 0, 0, 1) correspond to cities 3, 4, and 1, respectively. The total distance of the tour is d12 + d23 + d34 + d41= 10 + 6 + 9 + 7 = 32.
• Tour 2. 1 - 3 - 4 - 2 - 1
• Tour 3. 1 - 4 - 2 - 3 - 1

There seems to be some discrepancy here. If there is one, we need an explanation. The discrepancy is that we can find many more tours that should be valid because no city is visited more than once. You may say they are distinct from the three previously listed. Some of these additional tours are:

• Tour 4. 1 - 2 - 4 - 3 - 1
• Tour 5. 3 - 2 - 4 - 1 - 3
• Tour 6. 2 - 1 - 4 - 3 - 2

There is no doubt that these three tours are distinct from the first set of three tours. And in each of these three tours, every city is visited exactly once, as required in the problem. So they are valid tours as well. Why did our formula give us 3 for the value of the number of possible valid tours, while we are able to find 6?

The answer lies in the fact that if two valid tours are symmetric and have the same energy level, because they have the same value for the total distance traveled in the tour, one of them is in a sense redundant, or one of them can be considered degenerate, using the terminology common to this context. As long as they are valid and give the same total distance, the two tours are not individually interesting, and any one of the two is enough to have. By simple inspection, you find the total distances for the six listed tours. They are 32 for tour 1, 32 also for tour 6, 45 for each of tours 2 and 4, and 39 for each of tours 3 and 5. Notice also that tour 6 is not very different from tour 1. Instead of starting at city 1 as in tour 1, if you start at city 2 and follow tour 1 from there in reverse order of visiting cities, you get tour 6. Therefore, the distance covered is the same for both these tours. You can find similar relationships for other pairs of tours that have the same total distance for the tour. Either by reversing the order of cities in a tour, or by making a circular permutation of the order of the cities in a tour, you get another tour with the same total distance. This way you can find tours. Thus, only three distinct total distances are possible, and among them 32 is the lowest. The tour 1 - 2 - 3 - 4 - 1 is the shortest and is an optimum solution for this traveling salesperson problem. There is an alternative optimum solution, namely tour 6, with 32 for the total length of the tour. The problem is to find an optimal tour, and not to find all optimal tours if more than one exist. That is the reason only three distinct tours are suggested by the formula for the number of distinct valid tours in this case.

The formula we used to get the number of valid and distinct tours to be 3 is based on the elimination of such symmetry. To clarify all this discussion, you should determine the energy levels of all your six tours identified earlier, hoping to find pairs of tours with identical energy levels.

Note that the first term on the right-hand side of the equation results in 0 for a valid tour, as this term is to ensure there is no more than a single 1 in each row. That is, in any summand in the first term, at least one of the factors, xik or xij, where k ≠ j has to be 0 for a valid tour. So all those summands are 0, making the first term itself 0. Similarly, the second term is 0 for a valid tour, because in any summand at least one of the factors, xki or xji, where k ≠ j has to be 0 for a valid tour. In all, exactly 4 of the 16 x's are each 1, making the total of the x's 4. This causes the third term to be 0 for a valid tour. These observations make it clear that it does not matter for valid tours, what values are assigned to the parameters A1, A2, and A3. Assigning large values for these parameters would cause the energy levels, for tours that are not valid, to be much larger than the energy levels for valid tours. Thereby, these tours become unattractive for the solution of the traveling salesperson problem. Let us use the value 1/2 for the parameter A4.

Let us demonstrate the calculation of the value for the last term in the equation for E, in the case of tour 1. Recall that the needed equation is

E = A1 ΣiΣk Σj≠k xikxij + A2 ΣiΣk Σj≠kxkixji + A3[( ΣiΣkxjk) - n]2 + A4Σk Σj≠kΣidkjxki(xj,i+1 + xj,i-1)

The last term expands as given in the following calculation:

    A4{ d12[x12(x23 +x21) + x13(x24 + x22) + x14(x21 + x23)] +

    d13[x12(x33 + x31) + x13(x34 + x32) + x14(x31 + x33)] +

    d14[x12(x43 + x41) + x13(x44 + x42) + x14(x41 + x43)] +

    d21[x21(x12 + x14) + x23(x14 + x12) + x24(x11 + x13)] +

    d23[x21(x32 + x34) + x23(x34 + x32) + x24(x31 + x33)] +

    d24[x21(x42 + x44) + x23(x44 + x42) + x24(x41 + x43)] +

    d31[x31(x12 + x14) + x32(x13 + x11) + x34(x11 + x13)] +

    d32[x31(x22 + x24) + x32(x23 + x21) + x34(x21 + x23)] +

    d34[x31(x42 + x44) + x32(x43 + x41) + x34(x41 + x43)] +

    d41[x41(x12 + x14) + x42(x13 + x11) + x43(x14 + x12)] +

    d42[x41(x22 + x24) + x42(x23 + x21) + x43(x24 + x22)] +

    d43[x41(x32 + x34) + x42(x33 + x31) + x43(x34 + x32)] }

When the respective values are substituted for the tour 1 - 2 - 3 - 4 - 1, the previous calculation becomes:

    1/2{10[0(0 + 1) + 0(0 + 0) + 1(1 + 0)] + 14[0(0 + 0) + 0(0 + 1) +

    1(0 + 0)] +

    7[0(1 + 0) + 0(0 + 0) + 1(0 + 1)] + 10[1(0 + 1) + 0(1 + 0) +

    0(0 + 0)] +

    6[1(1 + 0) + 0(0 + 1) + 0(0 + 0)] + 12[1(0 + 0) + 0(0 + 0) +

    0(0 + 1)] +

    14[0(0 + 1) + 1(0 + 0) + 0(0 + 0)] + 6[0(0 + 0) + 1(0 + 1) +

    0(1 + 0)] +

    9[0(0 + 0) + 1(1 + 0) + 0(0 + 1)] + 7[0(0 + 1) + 0(0 + 0) +

    1(1 + 0)] +

    12[0(0 + 0) + 0(0 + 1) + 1(0 + 0)] + 9[0(1 + 0) + 0(0 + 0) +

    1(0 + 1)]}

    = 1/2( 10 + 0 + 7 + 10 + 6 + 0 + 0 + 6 + 9 + 7 + 0 + 9)

    = 1/2(64)

    = 32

Table 15.1 contains the values we get for the fourth term on the right-hand side of the equation, and for E, with the six listed tours.

Table 15.1 Energy Levels for Six of the Valid Tours
Tour # | Non-Zero x's | Value for the Last Term | Energy Level | Comment
1 | x14, x21, x32, x43 | 32 | 32 | 1 - 2 - 3 - 4 - 1 tour
2 | x14, x23, x31, x42 | 45 | 45 | 1 - 3 - 4 - 2 - 1 tour
3 | x14, x22, x33, x41 | 39 | 39 | 1 - 4 - 2 - 3 - 1 tour
4 | x14, x21, x33, x42 | 45 | 45 | 1 - 2 - 4 - 3 - 1 tour
5 | x13, x21, x34, x42 | 39 | 39 | 3 - 2 - 4 - 1 - 3 tour
6 | x11, x24, x33, x42 | 32 | 32 | 2 - 1 - 4 - 3 - 2 tour


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Neural Network for Traveling Salesperson Problem

Hopfield and Tank used a neural network to solve a traveling salesperson problem. The solution you get may not be optimal in certain instances. But by and large you may get a solution close to an optimal solution. One cannot find for the traveling salesperson problem a consistently efficient method of solution, as it has been proved to belong to a set of problems called NP-complete problems. NP-complete problems have the distinction that there is no known algorithm that is efficient and practical, and there is little likelihood that such an algorithm will be developed in the future. This is a caveat to keep in mind when using a neural network to solve a traveling salesperson problem.

Network Choice and Layout

We will describe the use of a Hopfield network to attempt to solve the traveling salesperson problem. There are n2 neurons in the network arranged in a two-dimensional array of n neurons per row and n per column. The network is fully connected. The connections in the network in each row and in each column are lateral connections. The layout of the neurons in the network with their connections is shown in Figure 15.1 for the case of three cities, for illustration. To avoid cluttering, the connections between diagonally opposite neurons are not shown.

Figure 15.1 Layout of a Hopfield network for the traveling salesperson problem.

The most important task on hand then is finding an appropriate connection weight matrix. It is constructed taking into account that nonvalid tours should be prevented and valid tours should be preferred. A consideration in this regard is, for example, no two neurons in the same row (column) should fire in the same cycle of operation of the network. As a consequence, the lateral connections should be for inhibition and not for excitation.

In this context, the Kronecker delta function is used to facilitate simple notation. The Kronecker delta function has two arguments, which are given usually as subscripts of the symbol δ. By definition δik has value 1 if i = k, and 0 if i ≠ k. That is, the two subscripts should agree for the Kronecker delta to have value 1. Otherwise, its value is 0.

We refer to the neurons with two subscripts, one for the city it refers to, and the other for the order of that city in the tour. Therefore, an element of the weight matrix for a connection between two neurons needs to have four subscripts, with a comma after two of the subscripts. An example is wik,lj referring to the weight on the connection between the (ik) neuron and the (lj) neuron. The value of this weight is set as follows:

Wik,lj = -A1δil(1-δkj)-A2δkj(1-δkj(1-δil)-A3-A4 dil(δj, k+1 + δj,k-1)

Here the negative signs indicate inhibition through the lateral connections in a row or column. The -A3 is a term for global inhibition.

Inputs

The inputs to the network are chosen arbitrarily. If as a consequence of the choice of the inputs, the activations work out to give outputs that add up to the number of cities, an initial solution for the problem, a legal tour, will result. A problem may also arise that the network will get stuck at a local minimum. To avoid such an occurrence, random noise can be added. Usually you take as the input at each neuron a constant times the number of cities and adjust this adding a random number, which can differ for different neurons.

Activations, Outputs, and Their Updating

We denote the activation of the neuron in the ith row and jth column by aij, and the output is denoted by xij. A time constant τ, and a gain λ are used as well. A constant m is another parameter used. Also, Δt denotes the increment in time, from one cycle to the next. Keep in mind that the index for the summation Σ ranges from 1 to n, the number of cities. Excluded values of the index are shown by the use of the symbol ≠.

The change in the activation is then given by Δaij, where:

Δaij = Δt (Term1 + Term2 + Term3 + Term4 + Term5)
Term1 = - aij/τ
Term2 = - A1 k≠jxik
Term3 = - A2
Σk≠ixkj
Term4 = - A3(Σi Σkxik - m)
Term5 = - A4 Σk≠idik(xk,j+1 + xk,j-1)

To update the activation of the neuron in the ith row and jth column, you take:

aijnew = aijold + Δaij

The output of a neuron in the ith row and jth column is calculated from:

xin = (1 + tanh(λaij))/2

NOTE:

The function used here is the hyperbolic tangent function. The gain parameter mentioned earlier λ is. The output of each neuron is calculated after updating the activation. Ideally, you want to get the outputs as 0's and 1's, preferably a single one for each row and each column, to represent a tour that satisfies the conditions of the problem. But the hyperbolic tangent function gives a real number, and you have to settle for a close enough value to 1 or 0. You may get, for example, 0.96 instead of 1, or 0.07 instead of 0. The solution is to be obtained from such values by rounding up or down so that 1 or 0 will be used, as the case may be


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Performance of the Hopfield Network

Let us now look at Hopfield and Tank's approach at solving the TSP.

Hopfield and Tank Example

The Hopfield network's use in solving the traveling salesperson problem is a pioneering effort in the use of the neural network approach for this problem. Hopfield and Tank's example is for a problem with 10 cities. The parameters used were, A1= 500, A2 = 500, A3 = 200, A4 = 500, τ = 1, λ = 50, and m = 15. A good solution corresponding to a local minimum for E is the expected, if not the best, solution (global minimum). An annealing process could be considered to move out of any local minimum. As was mentioned before, the traveling salesperson problem is one of those problems for which a single approach cannot be found that will be successful in all cases. There isn't very much guidance as to how to choose the parameters in general for the use of the Hopfield network to solve the traveling salesperson problem.

C++ Implementation of the Hopfield Network for the Traveling Salesperson Problem

We present a C++ program for the Hopfield network operation for the traveling salesperson problem. The header file is in the Listing 15.1, and the source file is in the Listing 15.2. A tsneuron class is declared for a neuron and a network class for the network. The network class is declared a friend in the tsneuron class. The program follows the procedure described for setting inputs, connection weights, and updating.

Program Details

The following is a listing of the characteristics of the C++ program along with definitions and/or functions.

• The number of cities, the number of iterations, and the distances between the cities are solicited from the user.
• The distances are taken as integer values. If you want to use real numbers as distances, the type for distance matrix needs to be changed to float, and corresponding changes are needed for calcdist ( ) function, etc.
• tourcity and tourorder arrays keep track of the cities that have to be covered and the order in which it is to be done.
• A neuron corresponds to each combination of a city and its order in the tour. The ith city visited in the order j , is the neuron corresponding to the element j + i*n, in the array for neurons. Here n is the number of cities. The i and the j vary from 0 to n – 1. There are n2 neurons.
• mtrx is the matrix giving the weights on the connections between the neurons. It is a square matrix of order n2.
• An input vector is generated at random in the function main ( ), and is later referred to as ip
• asgninpt ( ) function presents the input vector ip to the network and determines the initial activations of the neurons.
• getacts ( ) function updates the activations of the neurons after each iteration.
• getouts ( ) function computes the outputs after each iteration. la is used as abbreviation for lambda in its argument.
• iterate ( ) function carries out the number of iterations desired.
• findtour ( ) function determines the tour orders of cities to be visited using the outputs of the neurons. When used at the end of the iterations, it gives the solution obtained for the traveling salesperson problem.
• calcdist ( ) function calculates the distance of the tour in the solution.

Listing 15.1 Header file for the C++ program for the Hopfield network for the traveling salesperson problem

//trvslsmn.h  V. Rao,  H. Rao

#include <iostream.h>

#include <stdlib.h>

#include <math.h>

#include <stdio.h>

#define MXSIZ 11

class tsneuron

        {

        protected:

        int cit,ord;

        float output;

        float activation;

        friend class network;

        public:

        tsneuron() { };

        void getnrn(int,int);

        };

class network

        {

        public:

        int  citnbr;

        float pra,prb,prc,prd,totout,distnce;

        tsneuron (tnrn)[MXSIZ][MXSIZ];

        int dist[MXSIZ][MXSIZ];

        int tourcity[MXSIZ];

        int tourorder[MXSIZ];

        float outs[MXSIZ][MXSIZ];

        float acts[MXSIZ][MXSIZ];

        float mtrx[MXSIZ][MXSIZ];

        float citouts[MXSIZ];

        float ordouts[MXSIZ];

        network() { };

        void getnwk(int,float,float,float,float);

        void getdist(int);

        void findtour();

        void asgninpt(float *);

        void calcdist();

        void iterate(int,int,float,float,float);

        void getacts(int,float,float);

        void getouts(float);

//print functions

        void prdist();

        void prmtrx(int);

        void prtour();

        void practs();

        void prouts();

        };


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Source File for Hopfield Network for Traveling Salesperson Problem

The following listing gives the source code for the C++ program for the Hopfield network for traveling salesperson problem. The user is prompted to input the number of cities and the maximum number of iterations for the operation of the network.

The parameters a, b, c, d declared in the function main correspond to A1, A2, A3, and A4, respectively. These and the parameters tau, lambda, and nprm are all given values in the declaration statements in the function main. If you change these parameter values or change the number of cities in the traveling salesperson problem, the program will compile but may not work as you'd like.

Listing 15.2 Source file for the C++ program for the Hopfield network for the traveling salesperson problem

//trvslsmn.cpp V. Rao,  H. Rao

#include "trvslsmn.h"

#include <stdlib.h>

#include <time.h>

//generate random noise

int randomnum(int maxval)

{

// random number generator

// will return an integer up to maxval

return rand() % maxval;

}

//Kronecker delta function

int krondelt(int i,int j)

       {

       int k;

       k= ((i == j) ? (1):(0));

       return k;

       };

void tsneuron::getnrn(int i,int j)

       {

       cit = i;

       ord = j;

       output = 0.0;

       activation = 0.0;

       };

//distances between cities

void network::getdist(int k)

       {

       citnbr = k;

       int i,j;

       cout<<"\n";

for(i=0;i<citnbr;++i)

       {

       dist[i][i]=0;

       for(j=i+1;j<citnbr;++j)

              {

              cout<<"\ntype distance (integer) from city "<<

              i<<" to city "<<j<<"\n";

              cin>>dist[i][j];

              }

       cout<<"\n";

       }

for(i=0;i<citnbr;++i)

       {

       for(j=0;j<i;++j)

              {

              dist[i][j] = dist[j][i];

              }

       }

prdist();

cout<<"\n";

}

//print distance matrix

void network::prdist()

       {

       int i,j;

       cout<<"\n Distance Matrix\n";

       for(i=0;i<citnbr;++i)

              {

              for(j=0;j<citnbr;++j)

                     {

                     cout<<dist[i][j]<<"  ";

                     }

       cout<<"\n";

       }

}

//set up network

void network::getnwk(int citynum,float a,float b,float c,float d)

        {

        int i,j,k,l,t1,t2,t3,t4,t5,t6;

        int p,q;

        citnbr = citynum;

        pra = a;

        prb = b;

        prc = c;

        prd = d;

        getdist(citnbr);

        for(i=0;i<citnbr;++i)

                {

                for(j=0;j<citnbr;++j)

                       {

                       tnrn[i][j].getnrn(i,j);

                       }

                }

        //find weight matrix

        for(i=0;i<citnbr;++i)

                 {

                 for(j=0;j<citnbr;++j)

                        {

                        p = ((j == citnbr-1) ? (0) : (j+1));

                        q = ((j == 0) ? (citnbr-1) : (j-1));

                        t1 = j + i*citnbr;

                        for(k=0;k<citnbr;++k)

                                {

                 for(l=0;l<citnbr;++l)

                        {

                        t2 = l + k*citnbr;

                        t3 = krondelt(i,k);

                        t4 = krondelt(j,l);

                        t5 = krondelt(l,p);

                        t6 = krondelt(l,q);

                        mtrx[t1][t2] =

                        -a*t3*(1-t4) -b*t4*(1-t3)

                        -c -d*dist[i][k]*(t5+t6);

            }

                 }

            }

        }

        prmtrx(citnbr);

}

//print weight matrix

void network::prmtrx(int k)

         {

         int i,j,nbrsq;

         nbrsq = k*k;

         cout<<"\nWeight Matrix\n";

         for(i=0;i<nbrsq;++i)

                {

                for(j=0;j<nbrsq;++j)

                       {

                       if(j%k == 0)

                              {

                              cout<<"\n";

                              }

                       cout<<mtrx[i][j]<<"  ";

                       }

                cout<<"\n";

                }

         }

//present input to network

void network::asgninpt(float *ip)

       {

       int i,j,k,l,t1,t2;

       for(i=0;i<citnbr;++i)

             {

             for(j =0;j<citnbr;++j)

                    {

                    acts[i][j] = 0.0;

                    }

             }

       //find initial activations

       for(i=0;i<citnbr;++i)

             {

             for(j =0;j<citnbr;++j)

                    {

                    t1 = j + i*citnbr;

                    for(k=0;k<citnbr;++k)

                           {

                           for(l=0;l<citnbr;++l)

                                  {

                                  t2 = l + k*citnbr;

                                  acts[i][j] +=

                                  mtrx[t1][t2]*ip[t1];

                                  }

                           }

                    }

             }

       //print activations

       cout<<"\ninitial activations\n";

       practs();

       }

//find activations

void network::getacts(int nprm,float dlt,float tau)

       {

       int i,j,k,p,q;

       float r1, r2, r3, r4,r5;

       r3 = totout - nprm ;

       for(i=0;i<citnbr;++i)

             {

             r4 = 0.0;

             p = ((i == citnbr-1) ? (0) : (i+1));

             q = ((i == 0) ? (citnbr-1) : (i-1));

             for(j=0;j<citnbr;++j)

                    {

                    r1 = citouts[i] - outs[i][j];

                    r2 = ordouts[i] - outs[i][j];

                    for(k=0;k<citnbr;++k)

                           {

                           r4 += dist[i][k] *

                           (outs[k][p] + outs[k][q]);

                           }

                    r5 = dlt*(-acts[i][j]/tau -

                    pra*r1 -prb*r2 -prc*r3 -prd*r4);

                    acts[i][j] += r5;

                    }

             }

       }

//find outputs and totals for rows and columns

void network::getouts(float la)

       {

       float b1,b2,b3,b4;

       int i,j;

       totout = 0.0;

       for(i=0;i<citnbr;++i)

             {

             citouts[i] = 0.0;

             for(j=0;j<citnbr;++j)

             {

             b1 = la*acts[i][j];

             b4 = b1/500.0;

             b2 = exp(b4);

             b3 = exp(-b4);

             outs[i][j] = (1.0+(b2-b3)/(b2+b3))/2.0;

             citouts[i] += outs[i][j];};

             totout += citouts[i];

             }

       for(j=0;j<citnbr;++j)

             {

             ordouts[j]  = 0.0;

             for(i=0;i<citnbr;++i)

                    {

                    ordouts[j] += outs[i][j];

                    }

             }

       }

//find tour

void network::findtour()

       {

       int i,j,k,tag[MXSIZ][MXSIZ];

       float tmp;

       for (i=0;i<citnbr;++i)

              {

              for(j=0;j<citnbr;++j)

                     {

                     tag[i][j] = 0;

                     }

              }

              for (i=0;i<citnbr;++i)

                     {

                     tmp = -10.0;

                     for(j=0;j<citnbr;++j)

                           {

                           for(k=0;k<citnbr;++k)

                                 {

                                 if((outs[i][k] >=tmp)&&

                                 (tag[i][k] ==0))

                                        tmp = outs[i][k];

                                 }

                           if((outs[i][j] ==tmp)&&

                           (tag[i][j]==0))

                                 {

                                 tourcity[i] =j;

                                 tourorder[j] = i;

                                 cout<<"\ntourcity "<<i

                                 <<" tour order "<<j<<"\n";

                                 for(k=0;k<citnbr;++k)

                                        {

                                        tag[i][k] = 1;

                                        tag[k][j] = 1;

                                        }

                                 }

                           }

                     }

              }

//print outputs

void network::prouts()

       {

       int i,j;

       cout<<"\nthe outputs\n";

       for(i=0;i<citnbr;++i)

              {

              for(j=0;j<citnbr;++j)

                     {

                     cout<<outs[i][j]<<"  ";

                     }

              cout<<"\n";

              }

       }

//calculate total distance for tour

void network::calcdist()

       {

       int i, k, l;

       distnce = 0.0;

       for(i=0;i<citnbr;++i)

              {

              k = tourorder[i];

              l = ((i == citnbr-1 ) ? (tourorder[0]):(tourorder[i+1]));

              distnce += dist[k][l];

              }

       cout<<"\n distance of tour is : "<<distnce<<"\n";

       }

// print tour

void network::prtour()

       {

       int i;

       cout<<"\n the tour :\n";

       for(i=0;i<citnbr;++i)

              {

              cout<<tourorder[i]<<"  ";

              cout<<"\n";

              }

       }

//print activations

void network::practs()

       {

       int i,j;

       cout<<"\n the activations:\n";

       for(i=0;i<citnbr;++i)

              {

              for(j=0;j<citnbr;++j)

                     {

                     cout<<acts[i][j]<<"  ";

                     }

              cout<<"\n";

              }

       }

//iterate specified number of times

void network::iterate(int nit,int nprm,float dlt,float tau,float la)

       {

       int k;

       for(k=1;k<=nit;++k)

              {

              getacts(nprm,dlt,tau);

              getouts(la);

              }

       cout<<"\n" <<nit<<" iterations completed\n";

       practs();

       cout<<"\n";

       prouts();

       cout<<"\n";

       }

void main()

{

//numit = #iterations; n = #cities; u=intial input; nprm - parameter n'

//dt = delta t;

// —————————————————-

// parameters to be tuned are here:

       int u=1;

       int nprm=10;

       float a=40.0;

       float b=40.0;

       float c=30.0;

       float d=60.0;

       float dt=0.01;

       float tau=1.0;

       float lambda=3.0;

//——————————————————-

       int i,n2;

       int numit=100;

       int n=4;

       float input_vector[MXSIZ*MXSIZ];

       srand ((unsigned)time(NULL));

       cout<<"\nPlease type number of cities, number of iterations\n";

       cin>>n>>numit;

       cout<<"\n";

       if (n>MXSIZ)

              {

              cout << "choose a smaller n value\n";

              exit(1);

              }

       n2 = n*n;

       for(i=0;i<n2;++i)

              {

              if(i%n == 0)cout<<"\n";

              input_vector[i] =(u + (float)(randomnum(100)/100.0))/20.0;

              cout<<input_vector[i]<<" ";

              }

//create network and operate

       network *tntwk = new network;

       if (tntwk==0)

              {

              cout << "not enough memory\n";

              exit(1);

              }

       tntwk->getnwk(n,a,b,c,d);

       tntwk->asgninpt(input_vector);

       tntwk->getouts(lambda);

       tntwk->prouts();

       tntwk->iterate(numit,nprm,dt,tau,lambda);

       tntwk->findtour();

       tntwk->prtour();

       tntwk->calcdist();

       cout<<"\n";

}


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Output from Your C++ Program for the Traveling Salesperson Problem

A three-city tour problem is trivial, since there is just one value for the total distance no matter how you permute the cities for the order of the tour. In this case the natural order is itself an optimal solution. The program is run for two cases, for illustration. The first run is for a problem with four cities. The second one is for a five-city problem. By the way, the cities are numbered from 0 to n – 1. The same parameter values are used in the two runs. The number of cities, and consequently, the matrix of distances were different. In the first run, the number of iterations asked for is 30, and in the second run it is 40.

The solution you get for the four-city problem is not the one in natural order. The total distance of the tour is 32. The tour in the solution is 1 - 0 - 3 - 2 - 1. This tour is equivalent to the tour in natural order, as you can see by starting at 0 and reading cyclically right to left in the previous sequence of cities.

The solution you get for the five-city problem is the tour 1 - 2 - 0 - 4 - 3 - 1. This reads, starting from 0 as either 0 - 4 - 3 - 1 - 2 - 0, or as 0 - 2 - 1 - 3 - 4 - 0. It is different from the tour in natural order. It has a total distance of 73, as compared to the distance of 84 with the natural order. It is not optimal though, as a tour of shortest distance is 0 - 4 - 2 - 1 - 3 - 0 with total distance 50.

Can the program give you the shorter tour? Yes, the solution can be different if you run the program again because each time you run the program, the input vector is different, as it is chosen at random. The parameter values given in the program are by guess.

Note that the seed for the random number generator is given in the statement

srand ((unsigned)time(NULL));

The program gives you the order in the tour for each city. For example, if it says tourcity 1 tour order 2, that means the second (tour) city is the city visited third (in tour order). Your tour orders are also with values from 0 to n – 1, like the cities.

The user input is in italic, and computer output is normal, as you have seen before.

Output for a Four-City Problem

Please type number of cities, number of iterations

4 30

0.097 0.0585 0.053 0.078                //input vector—there

are 16 neurons in the network

0.0725 0.0535 0.0585 0.0985

0.0505 0.061 0.0735 0.057

0.0555 0.075 0.0885 0.0925

type distance (integer) from city 0 to city 1

10

type distance (integer) from city 0 to city 2

14

type distance (integer) from city 0 to city 3

7

type distance (integer) from city 1 to city 2

6

type distance (integer) from city 1 to city 3

12

type distance (integer) from city 2 to city 3

9

 Distance Matrix

0  10  14  7

10  0  6  12

14  6  0  9

7  12  9  0

 Weight Matrix              //16x16 matrix of weights. There are

16 neurons in the network.

-30  -70   -70  -70

-70  -630  -30  -630

-70  -870  -30  -70

-30  -70   -70  -630

-70   -30  -70  -70

-630  -70  -630 -30

-870  -70  -870 -70

-70   -30  -70  -30

-70  -70   -30  -70

-30  -630  -70  -630

-30  -870  -70  -70

-70  -70   -30  -630

-70   -70  -70   -30

-630  -30  -630  -70

-870  -30  -870  -70

-630  -30  -630  -30

-70  -630  -30  -630

-30  -70   -70  -70

-70  -390  -30  -630

-70  -630  -30  -70

-630  -70  -630  -30

-70   -30  -70   -70

-390  -70  -390  -30

-630  -70  -630  -70

-30  -630  -70  -630

-70  -70   -30  -70

-30  -390  -70  -630

-30  -630  -70  -70

-630  -30  -630  -70

-70   -70  -70   -30

-390  -30  -390  -70

-870  -30  -870  -70

-70  -870  -30  -870

-70  -390  -30  -390

-30  -70   -70  -870

-70  -870  -30  -390

-870  -70  -870  -30

-390  -70  -390  -30

-70   -30  -70   -30

-870  -70  -870  -30

-30  -870  -70  -870

-30  -390  -70  -390

-70  -70   -30  -870

-30  -870  -70  -390

-870  -30  -870  -70

-390  -30  -390  -70

-70   -70  -70   -70

-450  -30  -450  -70

-70  -450  -30  -450

-70  -750  -30  -750

-70  -570  -30  -450

-70  -450  -30  -750

-450  -70  -450  -30

-750  -70  -750  -30

-570  -70  -570  -30

-450  -70  -450  -30

-30  -450  -70  -450

-30  -750  -70  -750

-30  -570  -70  -450

-30  -450  -70  -750

-450  -30  -450  -70

-750  -30  -750  -70

-570  -30  -570  -70

-70   -70  -70   -30

initial activations

 the activations:

-333.680054  -215.280014  -182.320023  -371.280029

-255.199997  -207.580002  -205.920013  -425.519989

-258.560028  -290.360016  -376.320007  -228.000031

-278.609985  -363  -444.27005  -377.400024

the outputs

0.017913  0.070217  0.100848  0.011483

0.044685  0.076494  0.077913  0.006022

0.042995  0.029762  0.010816  0.060882

0.034115  0.012667  0.004815  0.010678

 30 iterations completed

 the activations:

-222.586884  -176.979172  -195.530823  -380.166107

-164.0271    -171.654053  -214.053177  -421.249023

-158.297867  -218.833755  -319.384827  -245.097473

-194.550751  -317.505554  -437.527283  -447.651581

the outputs

0.064704  0.10681  0.087355  0.010333

0.122569  0.113061  0.071184  0.006337

0.130157  0.067483  0.021194  0.050156

0.088297  0.021667  0.005218  0.004624

tourcity 0 tour order 1

tourcity 1 tour order 0

tourcity 2 tour order 3

tourcity 3 tour order 2

 the tour :

1

0

3

2

 distance of tour is : 32


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Other Approaches to Solve the Traveling Salesperson Problem

The following describes a few other methods for solving the traveling salesperson problem.

Anzai's Presentation

Yuichiro Anzai describes the Hopfield network for the traveling salesperson problem in a slightly different way. For one thing, a global inhibition term is not used. A threshold value is associated with each neuron, added to the activation, and taken as the average of A1 and A2, using our earlier notation. The energy function is formulated slightly differently, as follows:

E = A1 Σ1(Σk xik-1)2 + A2 Σi(Σk xki-1)2 + A4 Σk Σj≠k Σi≠k dkj xki(xj,i+1 + xj,i-1)

The first term is 0 if the sum of the outputs is 1 in each column. The same is true for the second term with regard to rows.

The output is calculated using a parameter λ, here called the reference activation level, as:

xij = (1 + tan tanh(aij/λ))/2

The parameters used are A1 = 1/2, A2 = 1/2, A4 = 1/2, Δt = 1, τ = 1, and λ = 1. An attempt is made to solve the problem for a tour of 10 cities. The solution obtained is not crisp, in the sense that exactly one 1 occurs in each row and each column, there are values of varying magnitude with one dominating value in each row and column. The prominent value is considered to be part of the solution.

Kohonen's Approach for the Traveling Salesperson Problem

Kohonen's self-organizing maps can be used for the traveling salesperson problem. We summarize the discussion of this approach described in Eric Davalo's and Patrick Naim's work. Each city considered for the tour is referenced by its x and y coordinates. To each city there corresponds a neuron. The neurons are placed in a single array, unlike the two-dimensional array used in the Hopfield approach. The first and the last neurons in the array are considered to be neighbors.

There is a weight vector for each neuron, and it also has two components. The weight vector of a neuron is the image of the neuron in the map, which is sought to self-organize. There are as many input vectors as there are cities, and the coordinate pair of a city constitutes an input vector. A neuron with a weight vector closest to the input vector is selected. The weights of neurons in a neighborhood of the selected neuron are modified, others are not. A gradually reducing scale factor is also used for the modification of weights.

One neuron is created first, and its weight vector has 0 for its components. Other neurons are created one at a time, at each iteration of learning. Neurons may also be destroyed. The creation of the neuron and destruction of the neuron suggest adding a city provisionally to the final list in the tour and dropping a city also provisionally from that list. Thus the possibility of assigning any neuron to two inputs or two cities is prevented. The same is true about assigning two neurons to the same input.

As the input vectors are presented to the network, if an unselected neuron falls in the neighborhood of the closest twice, it is created. If a created neuron is not selected in three consecutive iterations for modification of weights, along with others being modified, it is destroyed.

That a tour of shortest distance results from this network operation is apparent from the fact that the closest neurons are selected. It is reported that experimental results are very promising. The computation time is small, and solutions somewhat close to the optimal are obtained, if not the optimal solution itself. As was before about the traveling salesperson problem, this is an NP-complete problem and near efficient (leading to suboptimal solutions, but faster) approaches to it should be accepted.

Algorithm for Kohonen's Approach

A gain parameter λ and a scale factor q are used while modifying the weights. A value between 0.02 and 0.2 was tried in previous examples for q. A distance of a neuron from the selected neuron is defined to be an integer between 0 and n – 1, where n is the number of cities for the tour. This means that these distances are not necessarily the actual distances between the cities. They could be made representative of the actual distances in some way. One such attempt is described in the following discussion on C++ implementation. This distance is denoted by dj for neuron j. A squashing function similar to the Gaussian density function is also used.

The details of the algorithm are in a paper referenced in Davalo. The steps of the algorithm to the extent given by Davalo are:

• Find the weight vector for which the distance from the input vector is the smallest
• Modify the weights using
wjnew = wjold + (lnew - wjold)g(λ,dj), where g(λ,dj) = exp(- dj2/λ) /√
• Reset λ as λ (1 - q)


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

C++ Implementation of Kohonen's Approach

Our C++ implementation of this algorithm (described above) is with small modifications. We create but do not destroy neurons explicitly. That is, we do not count the number of consecutive iterations in which a neuron is not selected for modification of weights. This is a consequence of our not defining a neighborhood of a neuron. Our example is for a problem with five neurons, for illustration, and because of the small number of neurons involved, the entire set is considered a neighborhood of each neuron.

When all but one neuron are created, the remaining neuron is created without any more work with the algorithm, and it is assigned to the input, which isn't corresponded yet to a neuron. After creating n – 1 neurons, only one unassigned input should remain.

In our C++ implementation, the distance matrix for the distances between neurons, in our example, is given as follows, following the stipulation in the algorithm that these values should be integers between 0 and n – 1.

         0 1 2 3 4

         1 0 1 2 3

  d =    2 1 0 1 2

         3 2 1 0 1

         4 3 2 1 0

We also ran the program by replacing the previous matrix with the following matrix and obtained the same solution. The actual distances between the cities are about four times the corresponding values in this matrix, more or less. We have not included the output from this second run of the program.

        0 1 3 3 2

        1 0 3 2 1

  d =   3 3 0 4 2

        3 2 4 0 1

        2 1 2 1 0

In our implementation, we picked a function similar to the Gaussian density function as the squashing function. The squashing function used is:

f(d,λ) = exp( -d2/λ) / √ (2 )

Header File for C++ Program for Kohonen's Approach

Listing 15.3 contains the header file for this program, and listing 15.4 contains the corresponding source file:

Listing 15.3 Header file for C++ program for Kohonen's approach

//tsp_kohn.h V.Rao, H.Rao

#include<iostream.h>

#include<math.h>

#define MXSIZ 10

#define pi 3.141592654

class city_neuron

       {

       protected:

              double x,y;

              int mark,order,count;

              double weight[2];

              friend class tspnetwork;

       public:

              city_neuron(){};

              void get_neuron(double,double);

       };

class tspnetwork

       {

       protected:

              int chosen_city,order[MXSIZ];

              double gain,input[MXSIZ][2];

              int citycount,index,d[MXSIZ][MXSIZ];

              double gain_factor,diffsq[MXSIZ];

              city_neuron (cnrn)[MXSIZ];

       public:

              tspnetwork(int,double,double,double,double*,double*);

              void get_input(double*,double*);

              void get_d();

              void find_tour();

              void associate_city();

              void modify_weights(int,int);

              double wtchange(int,int,double,double);

              void print_d();

              void print_input();

              void print_weights();

              void print_tour();

       };

Source File Listing

The following is the source file listing for the Kohonen approach to the traveling salesperson problem.

Listing 15.4 Source file for C++ program for Kohonen's approach

//tsp_kohn.cpp  V.Rao, H.Rao

#include "tsp_kohn.h"

void city_neuron::get_neuron(double a,double b)

       {

       x = a;

       y = b;

       mark = 0;

       count = 0;

       weight[0] = 0.0;

       weight[1] = 0.0;

       };

tspnetwork::tspnetwork(int k,double f,double q,double h,

double *ip0,double *ip1)

       {

       int i;

       gain = h;

       gain_factor = f;

       citycount = k;

       // distances between neurons as integers between 0 and n-1

       get_d();

       print_d();

       cout<<"\n";

       // input vectors

       get_input(ip0,ip1);

       print_input();

       // neurons in the network

       for(i=0;i<citycount;++i)

              {

              order[i] = citycount+1;

              diffsq[i] = q;

              cnrn[i].get_neuron(ip0[i],ip1[i]);

              cnrn[i].order = citycount +1;

              }

       }

void tspnetwork::associate_city()

       {

       int i,k,j,u;

       double r,s;

       for(u=0;u<citycount;u++)

              {

              //start a new iteration with the input vectors

              for(j=0;j<citycount;j++)

                       {

                       for(i=0;i<citycount;++i)

                              {

                              if(cnrn[i].mark==0)

                                      {

                                      k = i;

                                      i =citycount;

                                      }

                              }

                       //find the closest neuron

                       for(i=0;i<citycount;++i)

                              {

                              r = input[j][0] - cnrn[i].weight[0];

                              s = input[j][1] - cnrn[i].weight[1];

                              diffsq[i] = r*r +s*s;

                              if(diffsq[i]<diffsq[k]) k=i;

                              }

                       chosen_city = k;

                       cnrn[k].count++;

                       if((cnrn[k].mark<1)&&(cnrn[k].count==2))

                              {

                              //associate a neuron with a position

                              cnrn[k].mark = 1;

                              cnrn[k].order = u;

                              order[u] = chosen_city;

                              index = j;

                              gain *= gain_factor;

                              //modify weights

                              modify_weights(k,index);

                              print_weights();

                              j = citycount;

                              }

                       }

              }

       }

void tspnetwork::find_tour()

       {

       int i;

       for(i=0;i<citycount;++i)

              {

              associate_city();

              }

              //associate the last neuron with remaining position in

       // tour

              for(i=0;i<citycount;++i)

                       {

                       if( cnrn[i].mark ==0)

                              {

                              cnrn[i].order = citycount-1;

                              order[citycount-1] = i;

                              cnrn[i].mark = 1;

                              }

                       }

                       //print out the tour.

                       //First the neurons in the tour order

                       //Next cities in the tour

                       //order with their x,y coordinates

                       print_tour();

              }

void tspnetwork::get_input(double *p,double *q)

       {

       int i;

       for(i=0;i<citycount;++i)

              {

              input[i][0] = p[i];

              input[i][1] = q[i];

              }

       }

//function to compute distances (between 0 and n-1) between

//neurons

void tspnetwork::get_d()

       {

       int i,j;

       for(i=0;i<citycount;++i)

              {

              for(j=0;j<citycount;++j)

                      {

                      d[i][j] = (j-i);

                      if(d[i][j]<0) d[i][j] = d[j][i];

                      }

              }

       }

//function to find the change in weight component

double tspnetwork::wtchange(int m,int l,double g,double h)

       {

       double r;

       r = exp(-d[m][l]*d[m][l]/gain);

       r *= (g-h)/sqrt(2*pi);

       return r;

       }

//function to determine new weights

void tspnetwork::modify_weights(int jj,int j)

       {

       int i;

       double t;

       double w[2];

       for(i=0;i<citycount;++i)

              {

              w[0] = cnrn[i].weight[0];

              w[1] = cnrn[i].weight[1];

              //determine new first component of weight

              t = wtchange(jj,i,input[j][0],w[0]);

              w[0] = cnrn[i].weight[0] +t;

              cnrn[i].weight[0] = w[0];

              //determine new second component of weight

              t = wtchange(jj,i,input[j][1],w[1]);

              w[1] = cnrn[i].weight[1] +t;

              cnrn[i].weight[1] = w[1];

              }

       }

//different print routines

void tspnetwork::print_d()

       {

       int i,j;

       cout<<"\n";

       for(i=0;i<citycount;i++)

              {

              cout<<" d: ";

              for(j=0;j<citycount;j++)

                      {

                      cout<<d[i][j]<<"   ";

                      }

              cout<<"\n";

              }

       }

void tspnetwork::print_input()

       {

       int i,j;

       for(i=0;i<citycount;i++)

              {

              cout<<"input : ";

              for(j=0;j<2;j++)

                      {

                      cout<<input [i][j]<<"   ";

                      }

              cout<<"\n";

              }

       }

void tspnetwork::print_weights()

       {

       int i,j;

       cout<<"\n";

       for(i=0;i<citycount;i++)

              {

              cout<<" weight: ";

              for(j=0;j<2;j++)

                      {

                      cout<<cnrn[i].weight[j]<<"   ";

                      }

              cout<<"\n";

              }

       }

void tspnetwork::print_tour()

       {

       int i,j;

       cout<<"\n tour : ";

       for(i=0;i<citycount;++i)

              {

              cout<<order[i]<<" —> ";

              }

       cout<<order[0]<<"\n\n";

       for(i=0;i<citycount;++i)

              {

              j = order[i];

              cout<<"("<<cnrn[j].x<<", "<<cnrn[j].y<<") —> ";

              }

       j= order[0];

       cout<<"("<<cnrn[j].x<<", "<<cnrn[j].y<<")\n\n";

       }

void main()

       {

       int nc= 5;//nc = number of cities

       double q= 0.05,h= 1.0,p= 1000.0;

       double input2[][5]= {7.0,4.0,14.0,0.0,5.0,3.0,6.0,13.0,12.0,10.0};

       tspnetwork tspn2(nc,q,p,h,input2[0],input2[1]);

       tspn2.find_tour();

       }


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Output from a Sample Program Run

The program, as mentioned, is created for the Kohonen approach to the traveling salesperson problem for five cities. There is no user input from the keyboard. All parameter values are given to the program with appropriate statements in the function main. A scale factor of 0.05 is given to apply to the gain parameter, which is given as 1. Initially, the distance of each neuron weight vector from an input vector is set at 1000, to facilitate finding the closest for the first time. The cities with coordinates (7,3), (4,6), (14,13), (0,12), (5,10) are specified for input vectors.

The tour found is not the one in natural order, namely 0 → 1 → 2 → 3 → 4 → 0, with a distance of 43.16. The tour found has the order 0 → 3 → 1 → 4 → 2 → 0, which covers a distance of 44.43, which is slightly higher, as shown in Figure 15.2. The best tour, 0 → 2 → 4 → 3 → 1 → 0 has a total distance of 38.54.

Figure 15.2 City placement and tour found for TSP.

Table 15.2 gives for the five-city example, the 12 (5!/10) distinct tour distances and corresponding representative tours. These are not generated by the program, but by enumeration and calculation by hand. This table is provided here for you to see the different solutions for this five-city example of the traveling salesperson problem.

Table 15.2 Distances and Representative Tours for Five-City Example
Distance | Tour | Comment
49.05 | 0-3-2-1-4-0 | worst case
47.59 | 0-3-1-2-4-0 |
45.33 | 0-2-1-4-3-0 |
44.86 | 0-2-3-1-4-0 |
44.43 | 0-3-1-4-2-0 | tour given by the program
44.30 | 0-2-1-3-4-0 |
43.29 | 0-1-4-2-3-0 |
43.16 | 0-1-2-3-4-0 |
42.73 | 0-1-2-4-3-0 |
42.26 | 0-1-3-2-4-0 |
40.00 | 0-1-4-3-2-0 |
38.54 | 0-2-4-3-1-0 | optimal tour

There are 12 different distances you can get for tours with these cities by hand calculation, and four of these are higher and seven are lower than the one you find from this program. The worst case tour (0 [rarr] 3 [rarr] 2 [rarr] 1 [rarr] 4 [rarr] 0) gives a distance of 49.05, and the best, as you saw above, 38.54. The solution from the program is at about the middle of the best and worst, in terms of total distance traveled.

The output of the program being all computer generated is given below as follows:

d: 0   1   2   3   4

d: 1   0   1   2   3

d: 2   1   0   1   2

d: 3   2   1   0   1

d: 4   3   2   1   0

input : 7   3

input : 4   6

input : 14  13

input : 0   12

input : 5   10

weight: 1.595769   2.393654

weight: 3.289125e-09   4.933688e-09

weight: 2.880126e-35   4.320189e-35

weight: 1.071429e-78   1.607143e-78

weight: 1.693308e-139  2.539961e-139

weight: 1.595769   2.393654

weight: 5.585192   5.18625

weight: 2.880126e-35   4.320189e-35

weight: 1.071429e-78   1.607143e-78

weight: 1.693308e-139  2.539961e-139

weight: 1.595769   2.393654

weight: 5.585192   5.18625

weight: 5.585192   5.18625

weight: 1.071429e-78   1.607143e-78

weight: 1.693308e-139  2.539961e-139

weight: 1.595769   2.393654

weight: 5.585192   5.18625

weight: 5.585192   5.18625

weight: 5.585192   5.18625

weight: 1.693308e-139   2.539961e-139

weight: 1.595769   2.393654

weight: 5.585192   5.18625

weight: 5.585192   5.18625

weight: 5.585192   5.18625

weight: 5.585192   5.18625

tour : 0 –> 3–> 1 –> 4 –> 2–> 0

(7, 3) –> (0, 12) –> (4, 6) –> (5, 10) –> (14, 13) –> (7, 3)

Optimizing a Stock Portfolio

Development of a neural network approach to a stock selection process in securities trading is similar to the application of neural networks to nonlinear optimization problems. The seminal work of Markowitz in making a mathematical formulation of an objective function in the context of portfolio selection forms a basis for such a development. There is risk to be minimized or put a cap on, and there are profits to be maximized. Investment capital is a limited resource naturally.

The objective function is formulated in such a way that the optimal portfolio minimizes the objective function. There would be a term in the objective function involving the product of each pair of stock prices. The covariance of that pair of prices is also used in the objective function. A product renders the objective function a quadratic. There would of course be some linear terms as well, and they represent the individual stock prices with the stock's average return as coefficient in each such term. You already get the idea that this optimization problem falls into the category of quadratic programming problems, which result in real number values for the variables in the optimal solution. Some other terms would also be included in the objective function to make sure that the constraints of the problem are satisfied.

A practical consideration is that a real number value for the amount of a stock may be unrealistic, as fractional numbers of stocks may not be purchased. It makes more sense to ask that the variables be taking 0 or 1 only. The implication then is that either you buy a stock, in which case you include it in the portfolio, or you do not buy at all. This is what is usually called a zero-one programming problem. You also identify it as a combinatorial problem.

You already saw a combinatorial optimization problem in the traveling salesperson problem. The constraints were incorporated into special terms in the objective function, so that the only function to be computed is the objective function.

Deeming the objective function as giving the energy of a network in a given state, the simulated annealing paradigm and the Hopfield network can be used to solve the problem. You then have a neural network in which each neuron represents a stock, and the size of the layer is determined by the number of stocks in the pool from which you want to build your stock portfolio. The paradigm suggested here strives to minimize the energy of the machine. The objective function needs therefore to be stated for minimization to get the best portfolio possible.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Tabu Neural Network

Tabu search, popularized by Fred Glover with his contributions, is a paradigm that has been used successfully in many optimization problems. It is a method that can steer a search procedure from a limited domain to an extended domain, so as to seek a solution that is better than a local minimum or a local maximum.

Tabu search (TS), suggests that an adaptive memory and a responsive exploration need to be part of an algorithm. Responsive exploration exploits the information derivable from a selected strategy. Such information may be more substantial, even if the selected strategy is in some sense a bad strategy, than what you can get even in a good strategy that is based on randomness. It is because there is an opportunity provided by such information to intelligently modify the strategy. You can get some clues as to how you can modify the strategy.

When you have a paradigm that incorporates adaptive memory, you see the relevance of associating a neural network:. a TANN is a Tabu neural network. Tabu search and Kohonen's self-organizing map have a common approach in that they work with "neighborhoods." As a new neighborhood is determined, TS prohibits some of the earlier solutions, as it classifies them as tabu. Such solutions contain attributes that are identified as tabu active.

Tabu search, has STM and LTM components as well. The short-term memory is sometimes called recency-based memory. While this may prove adequate to find good solutions, the inclusion of long-term memory makes the search method much more potent. It also does not necessitate longer runs of the search process.

Some of the examples of applications using Tabu search are:

• Training neural nets with the reactive Tabu search
• Tabu Learning: a neural network search method for solving nonconvex optimization problems
• Massively parallel Tabu search for the quadratic assignment problem
• Connection machine implementation of a Tabu search algorithm for the traveling salesman problem
• A Tabu search procedure for multicommodity location/allocation with balancing requirements

Summary

The traveling salesperson problem is presented in this chapter as an example of nonlinear optimization with neural networks. Details of formulation are given of the energy function and its evaluation. The approaches to the solution of the traveling salesperson problem using a Hopfield network and using a Kohonen self-organizing map are presented. C++ programs are included for both approaches.

The output with the C++ program for the Hopfield network refers to examples of four- and five-city tours. The output with the C++ program for the Kohonen approach is given for a tour of five cities, for illustration. The solution obtained is good, if not optimal. The problem with the Hopfield approach lies in the selection of appropriate values for the parameters. Hopfield's choices are given for his 10-city tour problem. The same values for the parameters may not work for the case of a different number of cities. The version of this approach given by Anzai is also discussed briefly.

Use of neural networks for nonlinear optimization as applied to portfolio selection is also presented in this chapter. You are introduced to Tabu search and its use in optimization with neural computing.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 16
Applications of Fuzzy Logic

Introduction

Up until now, we have discussed how fuzzy logic could be used in conjunction with neural networks: We looked at a fuzzifier in Chapter 3 that takes crisp input data and creates fuzzy outputs, which then could be used as inputs to a neural network. In chapter 9, we used fuzzy logic to create a special type of associative memory called a FAM (fuzzy associative memory). In this chapter, we focus on applications of fuzzy logic by itself. This chapter starts with an overview of the different types of application areas for fuzzy logic. We then present two application domains of fuzzy logic: fuzzy control systems, and fuzzy databases and quantification. In these sections, we also introduce some more concepts in fuzzy logic theory.

A Fuzzy Universe of Applications

Fuzzy logic is being applied to a wide variety of problems. The most pervasive field of influence is in control systems, with the rapid acceptance of fuzzy logic controllers (FLCs) for machine and process control. There are a number of other areas where fuzzy logic is being applied. Here is a brief list adapted from Yan, et al., with examples in each area:

• Biological and Medical Sciences Fuzzy logic based diagnosis systems, cancer research, fuzzy logic based manipulation of prosthetic devices, fuzzy logic based analysis of movement disorders, etc.
• Management and Decision Support Fuzzy logic based factory site selection, fuzzy logic aided military decision making (sounds scary, but remember that the fuzzy in fuzzy logic applies to the imprecision in the data and not in the logic), fuzzy logic based decision making for marketing strategies, etc.
• Economics and Finance Fuzzy modeling of complex marketing systems, fuzzy logic based trading systems, fuzzy logic based cost-benefit analysis, fuzzy logic based investment evaluation, etc.
• Environmental Science Fuzzy logic based weather prediction, fuzzy logic based water quality control, etc.
• Engineering and Computer Science Fuzzy database systems, fuzzy logic based prediction of earthquakes, fuzzy logic based automation of nuclear plant control, fuzzy logic based computer network design, fuzzy logic based evaluation of architectural design, fuzzy logic control systems, etc.
• Operations Research Fuzzy logic based scheduling and modeling, fuzzy logic based allocation of resources, etc.
• Pattern Recognition and Classification Fuzzy logic based speech recognition, fuzzy logic based handwriting recognition, fuzzy logic based facial characteristic analysis, fuzzy logic based military command analysis, fuzzy image search, etc.
• Psychology Fuzzy logic based analysis of human behavior, criminal investigation and prevention based on fuzzy logic reasoning, etc.
• Reliability and Quality Control Fuzzy logic based failure diagnosis, production line monitoring and inspection, etc.

We will now move to one of the two application domains that we will discuss in depth, Fuzzy Databases. Later in the chapter, we examine the second application domain, Fuzzy Control Systems.

Section I: A Look at Fuzzy Databases and Quantification

In this section, we want to look at some ways in which fuzzy logic may be applied to databases and operations with databases. Standard databases have crisp data sets, and you create unambiguous relations over the data sets. You also make queries that are specific and that do not have any ambiguity. Introducing ambiguity in one or more of these aspects of standard databases leads to ideas of how fuzzy logic can be applied to databases. Such application of fuzzy logic could mean that you get databases that are easier to query and easier to interface to. A fuzzy search, where search criteria are not precisely bounded, may be more appropriate than a crisp search. You can recall any number of occasions when you tend to make ambiguous queries, since you are not certain of what you need. You also tend to make ambiguous queries when a "ball park" value is sufficient for your purposes.

In this section, you will also learn some more concepts in fuzzy logic. You will see these concepts introduced where they arise in the discussion of ideas relating to fuzzy databases. You may at times see somewhat of a digression in the middle of the fuzzy database discussion to fuzzy logic topics. You may skip to the area where the fuzzy database discussion is resumed and refer back to the skipped areas whenever you feel the need to get clarification of a concept.

We will start with an example of a standard database, relations and queries. We then point out some of the ways in which fuzziness can be introduced.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Databases and Queries

Imagine that you are interested in the travel business. You may be trying to design special tours in different countries with your own team of tour guides, etc. , and you want to identify suitable persons for these positions. Initially, let us say, you are interested in their own experiences in traveling, and the knowledge they possess, in terms of geography, customs, language, and special occasions, etc. The information you want to keep in your database may be something like, who the person is, the person's citizenship, to where the person traveled, when such travel occurred, the length of stay at that destination, the person's languages, the languages the person understands, the number of trips the person made to each place of travel, etc. Let us use some abbreviations:

cov—country visited
lov—length of visit (days)
nov—number of visits including previous visits
ctz—citizenship
yov—year of visit
lps—language (other than mother tongue) with proficiency to speak
lpu—language with only proficiency to understand
hs—history was studied (1—yes, 0—no)

Typical entries may appear as noted in Table 16.1.

Table 16.1 Example Database
Name | age | ctz | cov | lov | nov | yov | lps | lpu | hs
John Smith | 35 | U.S. | India | 4 | 1 | 1994 |  | Hindi | 1
John Smith | 35 | U.S. | Italy | 7 | 2 | 1991 | Italian |  | 1
John Smith | 35 | U.S. | Japan | 3 | 1 | 1993 |  |  | 0

When a query is made to list persons that visited India or Japan after 1992 for 3 or more days, John Smith's two entries will be included. The conditions stated for this query are straightforward, with lov [ge] 3 and yov > 1992 and (cov = India or cov = Japan).

Relations in Databases

A relation from this database may be the set of quintuples, (name, age, cov, lov, yov). Another may be the set of triples, (name, ctz, lps). The quintuple (John Smith, 35, India, 4, 1994) belongs to the former relation, and the triple (John Smith, U.S., Italian) belongs to the latter. You can define other relations, as well.

Fuzzy Scenarios

Now the query part may be made fuzzy by asking to list young persons who recently visited Japan or India for a few days. John Smith's entries may or may not be included this time since it is not clear if John Smith is considered young, or whether 1993 is considered recent, or if 3 days would qualify as a few days for the query. This modification of the query illustrates one of three scenarios in which fuzziness can be introduced into databases and their use.

This is the case where the database and relations are standard, but the queries may be fuzzy. The other cases are: one where the database is fuzzy, but the queries are standard with no ambiguity; and one where you have both a fuzzy database and some fuzzy queries.

Fuzzy Sets Revisited

We will illustrate the concept of fuzziness in the case where the database and the queries have fuzziness in them. Our discussion is guided by the reference Terano, Asai, and Sugeno. First, let us review and recast the concept of a fuzzy set in a slightly different notation.

If a, b, c, and d are in the set A with 0.9, 0.4, 0.5, 0, respectively, as degrees of membership, and in B with 0.9, 0.6, 0.3, 0.8, respectively, we give these fuzzy sets A and B as A = { 0.9/a, 0.4/b, 0.5/c} and B = {0.9/a, 0.6/b, 0.3/c, 0.8/d}. Now A[cup]B = {0.9/a, 0.6/b, 0.5/c, 0.8/d} since you take the larger of the degrees of membership in A and B for each element. Also, A[cap]B = {0.9/a, 0.4/b, 0.3/c} since you now take the smaller of the degrees of membership in A and B for each element. Since d has 0 as degree of membership in A (it is therefore not listed in A), it is not listed in A[cap]B.

Let us impart fuzzy values (FV) to each of the attributes, age, lov, nov, yov, and hs by defining the sets in Table 16.2.

Table 16.2 Fuzzy Values for Example Sets
Fuzzy Value | Set
FV(age) | { very young, young, somewhat old, old }
FV(nov) | { never, rarely, quite a few, often, very often }
FV(lov) | { barely few days, few days, quite a few days, many days }
FV(yov) | {distant past, recent past, recent }
FV(hs) | { barely, adequately, quite a bit, extensively }

The attributes of name, citizenship, country of visit are clearly not candidates for having fuzzy values. The attributes of lps, and lpu, which stand for language in which speaking proficiency and language in which understanding ability exist, can be coupled into another attribute called flp (foreign language proficiency) with fuzzy values. We could have introduced in the original list an attribute called lpr ( language with proficiency to read) along with lps and lpu. As you can see, these three can be taken together into the fuzzy-valued attribute of foreign language proficiency. We give below the fuzzy values of flp.

   FV(flp) = {not proficient, barely proficient, adequate,

             proficient, very proficient }

Note that each fuzzy value of each attribute gives rise to a fuzzy set, which depends on the elements you consider for the set and their degrees of membership.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Now let us determine the fuzzy sets that have John Smith as an element, besides possibly others. We need the values of degrees of membership for John Smith (actually his attribute values) in various fuzzy sets. Let us pick them as follows:

——————————————————————————————————————————————————————————————————————————

 Age:   mvery young(35) = 0 (degree of membership of 35 in very young is 0.

                          We will employ this notation from now on).

        myoung(35) = 0.75

        msomewhat old(35) = 0.3

        mold(35) = 0

——————————————————————————————————————————————————————————————————————————

Assume that similar values are assigned to the degrees of membership of values of John Smith's attributes in other fuzzy sets. Just as John Smith's age does not belong in the fuzzy sets young and old, some of his other attribute values do not belong in some of the other fuzzy sets. The following is a list of fuzzy sets in which John Smith appears:

age_young = {0.75/35, ...}

age_somewhat old = {0.3/35, ... }

A similar statement attempted for the number of visits may prompt you to list nov_rarely = {0.7/1, 0.2/2}, and nov_quite a few = {0.3/2, .6/3, ...}. But you readily realize that the number of visits by itself does not mean much unless it is referenced with the country of visit. A person may visit one country very often, but another only rarely. This suggests the notion of a fuzzy relation, which is also a fuzzy set.

NOTE: What follows is an explanation of relations and discussion of fuzzy relations. If you want to skip this part for now, you may go to the "Fuzzy Queries" section a few pages later in this chapter.

Fuzzy Relations

A standard relation from set A to set B is a subset of the Cartesian product of A and B, written as A×B. The elements of A×B are ordered pairs (a, b) where a is an element of A and b is an element of B. For example, the ordered pair (Joe, Paul) is an element of the Cartesian product of the set of fathers, which includes Joe and the set of sons which includes Paul. Or, you can consider it as an element of the Cartesian product of the set of men with itself. In this case, the ordered pair (Joe, Paul) is in the subset which contains (a, b, if a is the father of b. This subset is a relation on the set of men. You can call this relation "father."

A fuzzy relation is similar to a standard relation, except that the resulting sets are fuzzy sets. An example of such a relation is 'much_more_educated'. This fuzzy set may look something like,

much_more_educated = { ..., 0.2/(Jeff, Steve), 0.7/(Jeff, Mike), ... }

Matrix Representation of a Fuzzy Relation

A fuzzy relation can be given as a matrix also when the underlying sets, call them domains, are finite. For example, let the set of men be S = { Jeff, Steve, Mike }, and let us use the same relation, much_more_educated. For each element of the Cartesian product S×S, we need the degree of membership in this relation. We already have two such values, mmuch_more_educated(Jeff, Steve) = 0.2, and mmuch_more_educated(Jeff, Mike) = 0.7. What degree of membership in the set should we assign for the pair (Jeff, Jeff)? It seems reasonable to assign a 0. We will assign a 0 whenever the two members of the ordered pair are the same. Our relation much_more_educated is given by a matrix that may look like the following:

                      0/(Jeff, Jeff)   0.2/(Jeff, Steve)  0.7/(Jeff, Mike)

 much_more_educated = 0.4/(Steve, Jeff)  0/(Steve, Steve) 0.3/(Steve, Mike)

                      0.1/(Mike, Jeff) 0.6/(Mike, Steve)    0/(Mike, Mike)

NOTE: Note that the first row corresponds to ordered pairs with Jeff as the first member, second column corresponds to those with Steve as the second member, and so on. The main diagonal has ordered pairs where the first and second members are the same.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Properties of Fuzzy Relations

A relation on a set, that is a subset of a Cartesian product of some set with itself, may have some interesting properties. It may be reflexive. For this you need to have 1 for the degree of membership of each main diagonal entry. Our example here is evidently not reflexive.

A relation may be symmetric. For this you need the degrees of membership of each pair of entries symmetrically situated to the main diagonal to be the same value. For example (Jeff, Mike) and (Mike, Jeff) should have the same degree of membership. Here they do not, so our example of a relation is not symmetric.

A relation may be antisymmetric. This requires that if a is different from b and the degree of membership of the ordered pair (a, b) is not 0, then its mirror image, the ordered pair (b, a), should have 0 for degree of membership. In our example, both (Steve, Mike) and (Mike, Steve) have positive values for degree of membership; therefore, the relation much_more_educated over the set {Jeff, Steve, Mike} is not antisymmetric also.

A relation may be transitive. For transitivity of a relation, you need the following condition, illustrated with our set {Jeff, Steve, Mike}. For brevity, let us use r in place of much_more_educated, the name of the relation:

     min (mr(Jeff, Steve) , mr(Steve, Mike) )[le]mr(Jeff, Mike)

     min (mr(Jeff, Mike) , mr(Mike, Steve) )[le]mr(Jeff, Steve)

     min (mr(Steve, Jeff) , mr(Jeff, Mike) )[le]mr(Steve, Mike)

     min (mr(Steve, Mike) , mr(Mike, Jeff) )[le]mr(Steve, Jeff)

     min (mr(Mike, Jeff) , mr(Jeff, Steve) )[le]mr(Mike, Steve)

     min (mr(Mike, Steve) , mr(Steve, Jeff) )[le]mr(Mike, Jeff)

In the above listings, the ordered pairs on the left-hand side of an occurrence of [le] are such that the second member of the first ordered pair matches the first member of the second ordered pair, and also the right-hand side ordered pair is made up of the two nonmatching elements, in the same order.

In our example,

     min (mr(Jeff, Steve) , mr(Steve, Mike) ) = min (0.2, 0.3) = 0.2

     mr(Jeff, Mike) = 0.7 > 0.2

For this instance, the required condition is met. But in the following:

     min (mr(Jeff, Mike), mr(Mike, Steve) ) = min (0.7, 0.6) = 0.6

     mr(Jeff, Steve) = 0.2 < 0.6

The required condition is violated, so the relation much_more_educated is not transitive.

NOTE: If a condition defining a property of a relation is not met even in one instance, the relation does not possess that property. Therefore, the relation in our example is not reflexive, not symmetric, not even antisymmetric, and not transitive.

If you think about it, it should be clear that when a relation on a set of more than one element is symmetric, it cannot be antisymmetric also, and vice versa. But a relation can be both not symmetric and not antisymmetric at the same time, as in our example.

An example of reflexive, symmetric, and transitive relation is given by the following matrix:

          1     0.4   0.8

          0.4   1     0.4

          0.8   0.4   1

Similarity Relations

A reflexive, symmetric, and transitive fuzzy relation is said to be a fuzzy equivalence relation. Such a relation is also called a similarity relation. When you have a similarity relation s, you can define the similarity class of an element x of the domain as the fuzzy set in which the degree of membership of y in the domain is ms(x, y). The similarity class of x with the relation s can be denoted by [x]s.

Resemblance Relations

Do you think similarity and resemblance are one and the same? If x is similar to y, does it mean that x resembles y? Or does the answer depend on what sense is used to talk of similarity or of resemblance? In everyday jargon, Bill may be similar to George in the sense of holding high office, but does Bill resemble George in financial terms? Does this prompt us to look at a 'resemblance relation' and distinguish it from the 'similarity relation'? Of course.

Recall that a fuzzy relation that is reflexive, symmetric, and also transitive is called similarity relation. It helps you to create similarity classes. If the relation lacks any one of the three properties, it is not a similarity relation. But if it is only not transitive, meaning it is both reflexive and symmetric, it is still not a similarity relation, but it is a resemblance relation. An example of a resemblance relation, call it t, is given by the following matrix.

Let the domain have elements a, b, and c:

               1     0.4   0.8

          t =  0.4   1     0.5

               0.8   0.5   1

This fuzzy relation is clearly reflexive, and symmetric, but it is not transitive. For example:

      min (mt(a, c) , mt(c, b) ) = min (0.8, 0.5) = 0.5 ,

but the following:

      mt(a, b) = 0.4 < 0.5 ,

is a violation of the condition for transitivity. Therefore, t is not a similarity relation, but it certainly is a resemblance relation.

Fuzzy Partial Order

One last definition is that of a fuzzy partial order. A fuzzy relation that is reflexive, antisymmetric, and transitive is a fuzzy partial order. It differs from a similarity relation by requiring antisymmetry instead of symmetry. In the context of crisp sets, an equivalence relation that helps to generate equivalence classes is also a reflexive, symmetric, and transitive relation. But those equivalence classes are disjoint, unlike similarity classes with fuzzy relations. With crisp sets, you can define a partial order, and it serves as a basis for making comparison of elements in the domain with one another.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Fuzzy Queries

At this point, our digression from the discussion of fuzzy data bases is finished. Let us now recall, for immediate reference, the entries in the definitions we listed earlier in Table 16.3:

Table 16.3 Fuzzy Values for Example Sets
Fuzzy Value | Set
FV(age) | { very young, young, somewhat old, old }
FV(nov) | { never, rarely, quite a few, often, very often }
FV(lov) | { barely few days, few days, quite a few days, many days }
FV(yov) | {distant past, recent past, recent }
FV(hs) | { barely, adequately, quite a bit, extensively }
FV(flp) | {not proficient, barely proficient, adequate, proficient, very proficient }

If you refer to Tables 16.1 and 16.3, you will see that a fuzzy query defines a fuzzy set. For example, suppose you ask for a list of young people with enough knowledge of Italy, and who know Italian almost like a native. You want the intersection of the fuzzy sets, age_young, hs_adequately, and flp_very proficient. John Smith is in the fuzzy set age_young with a 0.75 degree of membership, as noted before. Suppose he is in hs_adequately with a 0.2 degree of membership, and in flp_very proficient with a 0.9 degree of membership. We then put him in the fuzzy set for this query. The degree of membership for John Smith is the smallest value among 0.75, 0.2, and 0.9, since we are taking the intersection of fuzzy sets. So 0.2/John Smith will be part of the enumeration of the query fuzzy set.

Note that you can use unknown in place of the value of an attribute for an item in the database. If you do not know John Smith's age, you can enter unknown in that field of John Smith's record. You may say his age is unknown, even though you have a rough idea that he is about 35. You would then be able to assign some reasonable degrees of membership of John Smith in fuzzy sets like age_young or age_somewhat old.

Extending Database Models

One way of extending a model into a fuzzy model, as far as databases are concerned, is to make use of similarity relations and to extend the operations with them as Buckles and Perry do, such as PROJECT. First there are the domains, for the database. In our example relating to travel and tour guides above, the domains are:

D1 = { John Smith, ... }, the set of persons included in the database,
D2 = {India, Italy, Japan, ... }, the set of countries of visit,
D3 = {Hindi, Italian, Japanese, ... }, the set of foreign languages,
D4 = {U.S., ... }, the set of countries of citizenship,
D5 = set of ages,
D6 = set of years,
D7 = set of number of visits,
D8 = set of length of visits.

Note that the enumerated sets are shown with '...' in the sets, to indicate that there may be more entries listed, but we are not giving a complete list. In practice, you will make a complete enumeration unlike in our example. The domains D5, D6, D7, and D8 can also be given with all their elements listed, since in practice, these sets are finite. Conceptually though, they are infinite sets; for example, D6 is the set of positive integers.

Next, you have the similarity relations that you define. And then there are the operations.

Example

Consider a standard database given below. It can be called also the relation R1, from set D1 = {Georgette, Darrell, Ernie , Grace } to set D2 = {Spanish, Italian, French, Japanese, Chinese, Russian} as shown in Table 16.4:

Table 16.4 Example for Relation R1, with Domains D1 and D2
D1 | D2
Georgette | Spanish
Georgette | Italian
Darrell | French
Darrell | Spanish
Darrell | Japanese
Ernie | Spanish
Ernie | Russian
Grace | Chinese
Grace | French

Let's say a fuzzy database has D1 and D2 as domains. Suppose you have a similarity relation S1 on domain D1 given by the matrix:

          1     0.4   0.8   0.7

          0.4   1     0.5   0.3

          0.8   0.5   1     0.4

          0.7   0.3   0.4   1

Recall that the entries in this matrix represent the degrees of membership of ordered pairs in the relation S1. For example, the first row third column element, which is 0.8, in the matrix refers to the degree of membership of the pair (Georgette, Ernie), since the rows and columns refer to Georgette, Darrell, Ernie and Grace in that order.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

The result of the operation:

PROJECT (R1 over D1) with LEVEL(D1) = 0.6

is the relation R2 given in Table 16.5.

Table 16.5 Relation R2, which is the Result of PROJECT Operation
D1
{Georgette, Ernie}
{Georgette, Grace}
Darrell

This projection operation with a condition on the level works as follows. First, the column for D1 is to be picked, of the two columns that R1 shows. Repetitions are removed. The condition says that if two elements of D1 have a similarity of 0.6 or higher, they should be treated as the same. Even though similarity levels of pairs (Georgette, Ernie) and (Georgette, Grace) are both greater than 0.6, the pair (Ernie, Grace) has similarity of 0.4 only so that we do not treat the three as the same.

This type of table is not constructed in the standard database model, so this is part of an extension of the standard model.

Suppose you recast the information in the relation R1 and call it R3, shown in Table 16.6.

Table 16.6 Example for Relation R3, with Domains D1 and D2
D1D2 |
Georgette | {Spanish, Italian}
Darrell | {French, Spanish, Japanese}
Grace | {Chinese, French}
Ernie | {Russian, Spanish}

This kind of a table also is not found in standard databases (where there are groups with more than one element used in the relation), and is an example of an extended model.

Possibility Distributions

As an alternative to using similarity relations for introducing fuzziness into a database model, you can, following Umano, et al., use a possibility distribution-relational model. The possibility distributions represent the fuzzy values of attributes in the data. An example of a possibility distribution is the fuzzy set you saw before, nov_rarely = {0.7/1, 0.2/2}, where nov_rarely stands for number of visits considered to be "rarely."

Example

An example of a database on the lines of this model is shown in Table 16.7:

Table 16.7 Example of Possibility Distribution Relational Model
Name | Number of Visits Outside the U.S. | Citizenship | Name of Companion on Latest Visit
Peter | 3 | U.S. | Barbara
Roberto | {10, 11}p | Spain | Anne
Andre | 2 | unknown | Carol
Raj | 14 | {India, U.S.}p | Uma
Alan | unknown | U.S. | undefined
James | many | U.K. | null

A standard database cannot look like this. Entries like many and {10, 11}p clearly suggest fuzziness. The entry {10, 11}p, is a possibility distribution, suggesting that the number of visits outside the United States made by Roberto is either 10 or 11. Similarly, Raj's citizenship is India or United States, but not dual citizenship in both. Andre's citizenship and Alan's number of visits outside the United States are not known, and they can have any values. The possibilities cannot be narrowed down as in the case of Raj's citizenship and Roberto's frequency of visits outside the United States The entry undefined is used for Alan because he always traveled alone, he never took a companion.

James' number of visits is fuzzy. He traveled many times. A fuzzy set for many will provide the possibility distribution. It can be defined, for example, as:

    many = {0.2/6, 0.5/7, 0.8/8, 1/9, 1/10, ...}

The name of the companion on James' latest visit outside the United States is entered as null because we do not know on the one hand whether he never took a companion, in which case we could have used undefined as in Alan's case, and on the other whom he took as a companion if he did take one, in which case we could have used unknown. Simply put, we use null when we do not know enough to use either unknown or undefined.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Queries

Let us turn our attention now to how queries are answered with this type of a database model. Suppose you want a list of U.S. citizens in your database. Peter and Alan clearly satisfy this condition on citizenship. Andre and Raj can only be said to possibly satisfy this condition. But Roberto and James clearly do not satisfy the given condition. This query itself is crisp and not fuzzy (either you belong to the list of U.S. citizens or you don't). The answer therefore should be a crisp set, meaning that unless the degree of membership is 1, you will not list an element. So you get the set containing Peter and Alan only.

A second query could be for people who made more than a few visits outside the United States Here the query is fuzzy. James with many visits outside United States seems to clearly satisfy the given condition. It is perhaps reasonable to assume that each element in the fuzzy set for many appears in the fuzzy set more than a few with a degree of membership 1. Andre's 2 may be a number that merits 0 degree of membership in more than a few. The other numbers in the database are such that they possibly satisfy the given condition to different degrees. You can see that the answer to this fuzzy query is a fuzzy set. Now, we switch gears a little, to talk more on fuzzy theory. This will help with material to follow.

Fuzzy Events, Means and Variances

Let us introduce you to fuzzy events, fuzzy means, and fuzzy variances. These concepts are basic to make a study of fuzzy quantification theories. You will see how a variable's probability distribution and its fuzzy set are used together. We will use an example to show how fuzzy means and fuzzy variances are calculated.

Example: XYZ Company Takeover Price

Suppose you are a shareholder of company XYZ and that you read in the papers that its takeover is a prospect. Currently the company shares are selling at $40 a share. You read about a hostile takeover by a group prepared to pay $100 a share for XYZ. Another company whose business is related to XYZ's business and has been on friendly terms with XYZ is offering $85 a share. The employees of XYZ are concerned about their job security and have organized themselves in preparing to buy the company collectively for $60 a share. The buyout price of the company shares is a variable with these three possible values, viz., 100, 85, and 60. The board of directors of XYZ have to make the ultimate decision regarding whom they would sell the company. The probabilities are 0.3, 0.5, and 0.2 respectively, that the board decides to sell at $100 to the first group, to sell at $85 to the second, and to let the employees buy out at $60.

Thus, you get the probability distribution of the takeover price to be as follows:

price | 100 | 85 | 60
probability | 0.3 | 0.5 | 0.2

From standard probability theory, this distribution gives a mean(or expected price) of:

     100 x 0.3 + 85 x 0.5 + 60 x 0.2 = 84.5

and a variance of :

     (100-84.5)2 x 0.3 +(85-84.5)2 x 0.5 + (60-84.5)2 x 0.2 = 124.825

Suppose now that a security analyst specializing in takeover situations feels that the board hates a hostile takeover but to some extent they cannot resist the price being offered. The analyst also thinks that the board likes to keep some control over the company, which is possible if they sell the company to their friendly associate company. The analyst recognizes that the Board is sensitive to the fears and apprehensions of their loyal employees with whom they built a healthy relationship over the years, and are going to consider the offer from the employees.

The analyst's feelings are reflected in the following fuzzy set:

     {0.7/100, 1/85, 0.5/60}

You recall that this notation says, the degree of membership of 100 in this set is 0.7, that of 85 is 1, and the degree of membership is 0.5 for the value 60.

The fuzzy set obtained in this manner is also called a fuzzy event. A different analyst may define a different fuzzy event with the takeover price values. You, as a shareholder, may have your own intuition that suggests a different fuzzy event. Let us stay with the previous fuzzy set that we got from a security analyst, and give it the name A.

Probability of a Fuzzy Event

At this point, we can calculate the probability for the fuzzy event A by using the takeover prices as the basis to correspond the probabilities in the probability distribution and the degrees of membership in A. In other words, the degrees of membership, 0.7, 1, and 0.5 are treated as having the probabilities 0.3, 0.5, and 0.2, respectively. But we want the probability of the fuzzy event A, which we calculate as the expected degree of membership under the probability distribution we are using.

Our calculation gives the following:

     0.7 x 0.3 + 1 x 0.5 + 0.5 x 0.2 = 0.21 + 0.5 + 0.1 = 0.81


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Fuzzy Mean and Fuzzy Variance

Our next step is to calculate the fuzzy mean of the takeover price. Let us call it A_fuzzy_mean, to make reference to the fuzzy event used.

The calculation is as follows:

     A_fuzzy_mean = (1/0.81) x (100 x 0.7 x 0.3 + 85 x 1 x 0.5 + 60 x 0.5 x

0.2) = 85.8

To get the fuzzy variance of the takeover price, you need to use values like (100-85.8)2 , which is the square of the deviation of the takeover price 100 from the fuzzy mean. A simpler way to calculate, which is mathematically equivalent, is to first take the fuzzy expected value of the square of the takeover price and then to subtract the square of the fuzzy mean. To make it easier, let us use p as the variable that represents the takeover price.

The calculations are as below:

     A_fuzzy_expected p2 = 1/(0.81) x (1002x 0.7 x 0.3 + 852x 1 x 0.5 + 602x

                           0.5 x 0.2) = 7496.91

     A_fuzzy_variance = 7496.91 - 85.82 = 7496.91 - 7361.64 = 135.27

Fuzzy logic is thus introduced into the realm of probability concepts such as events, and statistical concepts such as mean and variance. Further, you can talk of fuzzy conditional expectations and fuzzy posterior probabilities, etc. enabling you to use fuzziness in Bayesian concepts, regression analysis, and so on. You will then be delving into the field of fuzzy quantification theories. In what follows, we continue our discussion with fuzzy conditional expectations.

Conditional Probability of a Fuzzy Event

Suppose you, as a shareholder of the XYZ company in the previous example come up with the fuzzy set, we will call the fuzzy event:

     B = {0.8/100, 0.4/85, 0.7/60}

The probability for your fuzzy event is as follows:

     0.8 x 0.3 + 0.4 x 0.5 + 0.7 x 0.2 = 0.58

B_fuzzy_mean and B_fuzzy_variance of the takeover price of XYZ stock work out as 85.17 and 244.35, respectively. But you want to see how these values change if at all when you take A, the analyst's fuzzy event, as a given. You are then asking to determine the conditional probability of your fuzzy event, and your conditional fuzzy mean and fuzzy variance as well.

The conditional probability of your fuzzy event is calculated as follows:

     (1/0.81) x (0.8 x 0.7 x 0.3 + 0.4 x 1 x 0.5 + 0.7 x 0.5 x 0.2) = 0.54

This value is smaller than the probability you got before when you did not take the analyst's fuzzy event as given. The a priori probability of fuzzy event B is 0.58, while the a posteriori probability of fuzzy event B given the fuzzy event A is 0.54.

Conditional Fuzzy Mean and Fuzzy Variance

The conditional B_fuzzy_mean of the takeover price with fuzzy event A as given works out as:

     (1/0.54) x (100 x 0.8 x 0.7 x 0.3 + 85 x 0.4 x 1 x 0.5 + 60 x 0.7 x

0.5 x 0.2) = 70.37

and the conditional B_fuzzy_variance of the takeover price with fuzzy event A, as given, amounts to 1301.76, which is over five times as large as when you did not take the analyst's fuzzy event as given.

Linear Regression a la Possibilities

When you see the definitions of fuzzy means and fuzzy variances, you may think that regression analysis can also be dealt with in the realm of fuzzy logic. In this section we discuss what approach is being taken in this regard.

First, recall what regression analysis usually means. You have a set of x- values and a corresponding set of y values, constituting a number of sample observations on variables X and Y. In determining a linear regression of Y on X, you are taking Y as the dependent variable, and X as the independent variable, and the linear regression of Y on X is a linear equation expressing Y in terms of X. This equation gives you the line 'closest' to the sample points (the scatter diagram) in some sense. You determine the coefficients in the equation as those values that minimize the sum of squares of deviations of the actual y values from the y values from the line. Once the coefficients are determined, you can use the equation to estimate the value of Y for any given value of X. People use regression equations for forecasting.

Sometimes you want to consider more than one independent variable, because you feel that there are more than one variable which collectively can explain the variations in the value of the dependent variable. This is your multiple regression model. Choosing your independent variables is where you show your modeling expertise when you want to explain what happens to Y, as X varies.

In any case, you realize that it is an optimization problem as well, since the minimization of the sum of squares of deviations is involved. Calculus is used to do this for Linear Regression. Use of calculus methods requires certain continuity properties. When such properties are not present, then some other method has to be used for the optimization problem.

The problem can be formulated as a linear programming problem, and techniques for solving linear programming problems can be used. You take this route for solving a linear regression problem with fuzzy logic.

In a previous section, you learned about possibility distributions. The linear regression problem with fuzzy logic is referred to as a linear possibility regression problem. The model, following the description of it by Tarano, Asai, and Sugeno, depends upon a reference function L, and fuzzy numbers in the form of ordered pairs (a, b). We will present fuzzy numbers in the next section and then return to continue our discussion of the linear possibility regression model.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Fuzzy Numbers

A fuzzy number is an ordered pair of numbers (a, b) with respect to a reference function L, which gives you the membership function. Here b has to be a positive number.

Here are the properties of L, the reference function:

1. It is a function of one variable and is symmetric about 0. That is, L(x) = L(-x).
2. It has a value of 1 at x = 0. In other words, L(0) = 1.
3. It is generally decreasing, when x is 0 or a positive number, meaning that its value drops as the value of its argument is increased. For example, L(2) < L(1). Thus L(x) has its values less than 1 for positive values of x. It does not make sense to have negative numbers as values of L, and so you ignore the values of x that cause such values for L.
4. The maximum value for the function is 1, at x = 0. It has a sort of a bell-shaped curve.

If A is a fuzzy number with the ordered pair (a, b) with b > 0, and if the reference function is L, you do the following:

You write the fuzzy number as A = (a, b)L. You get the membership of any element x, by taking the quantity (x - a) / b (which reminds you of how you get a z-score), and evaluate the reference function L at this argument. That is:

     mA(x) = L( (x-a) / b)

Examples of a reference function L are:

Example 1: L(x) = max ( 0, 1- x2)

You obtain the following shown in Table 16.8.

Table 16.8 Reference Function L
x | L(x) |
-2 |  | 0
-1 |  | 0
-0.5 |  | 0.75
0 |  | 1
0.5 |  | 0.75
1 |  | 0
2 |  | 0

This function is not strictly decreasing though. It remains a constant at 0 for x > 1.

Example 2: L(x) = 1/ (1 + x2 )

You get the values shown in Table 16.9.

Table 16.9 Reference Function L
x | L(x) |
-7 |  | 0.02
-2 |  | 0.2
-1 |  | 0.5
-0.5 |  | 0.8
0 |  | 1
0.5 |  | 0.8
1 |  | 0.5
2 |  | 0.2
7 |  | 0.02

Let us now determine the membership of 3 in the fuzzy number A = (4, 10)L, where L is the function in the second example above, viz., 1/ (1 + x2).

First, you get (x- 4) / 10 = (3 - 4) / 10 = - 0.1. Use this as the argument of the reference function L. 1/ (1 + (- 0.1)2 ) gives 0.99. This is expressed as follows:

     mA(3) = L( (3-4) / 10) = 1/ (1 + (- 0.1)2 ) = 0.99

You can verify the values, mA(0) = 0.862, and mA(10) = 0.735.

Triangular Fuzzy Number

With the right choice of a reference function, you can get a symmetrical fuzzy number A, such that when you plot the membership function in A, you get a triangle containing the pairs (x, mA(x)), with mA(x) > 0. An example is A = (5, 8)L, where L = max(1 - |x|, 0).

The numbers x that have positive values for mA(x) are in the interval ( -3, 13 ). Also, mA( -3 ) = 0, and mA(13) = 0. However, mA(x) has its maximum value at x = 5. Now, if x is less than -3 or greater than 13, the value of L is zero, and you do not consider such a number for membership in A. So all the elements for which membership in A is nonzero are in the triangle.

This triangular fuzzy number is shown in Figure 16.1. The height of the triangle is 1, and the width is 16, twice the number 8, midpoint of the base is at 5. The pair of numbers 5 and 8 are the ones defining the symmetrical fuzzy number A. The vertical axis gives the membership, so the range for this is from 0 to 1.

Figure 16.1 Triangular membership function.

Linear Possibility Regression Model

Assume that you have (n + 1)-tuples of values of x1, ... xn, and y. That is, for each i, i ranging from 1 to k, you have (x1, ... , xn, y), which are k sample observations on X1, ... , Xn, and Y. The linear possibility regression model is formulated differently depending upon whether the data collected is crisp or fuzzy.

Let us give such a model below, for the case with crisp data. Then the fuzziness lies in the coefficients in the model. You use symmetrical fuzzy numbers, Aj = (aj, bj)L. The linear possibility regression model is formulated as:

     Yj = A0 + A1Xj1 + ... + AnXjn

The value of Y from the model is a fuzzy number, since it is a function of the fuzzy coefficients. The fuzzy coefficients Ajare chosen as those that minimize the width (the base of the triangle) of the fuzzy number Yj. But Aj is also determined by how big the membership of observed yj is to be, in the fuzzy number Yj . This last observation provides a constraint for the linear programming problem which needs to be solved to find the linear possibility regression. You select a value d, and ask that mY(y) [ge] d.

We close this section by observing that linear possibility regression gives triangular fuzzy numbers for Y, the dependent variable. It is like doing interval estimation, or getting a regression band. Readers who are seriously interested in this topic should refer to Terano, et al. (see references).


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Section II: Fuzzy Control

This section discusses the fuzzy logic controller (FLC), its application and design. Fuzzy control is used in a variety of machines and processes today, with widespread application especially in Japan. A few of the applications in use today are in the list in Table 16.10, adapted from Yan, et al.

Table 16.10 Applications of Fuzzy Logic Controllers (FLCs) and Functions Performed
Application | FLC function(s)
Video camcorder | Determine best focusing and lighting when there is movement in the picture
Washing machine | Adjust washing cycle by judging the dirt, size of the load, and type of fabric
Television | Adjust brightness, color, and contrast of picture to please viewers
Motor control | Improve the accuracy and range of motion control under unexpected conditions
Subway train | Increase the stable drive and enhance the stop accuracy by evaluating the passenger traffic conditions. Provide a smooth start and smooth stop.
Vacuum cleaner | Adjust the vacuum cleaner motor power by judging the amount of dust and dirt and the floor characteristics
Hot water heater | Adjust the heating element power according to the temperature and the quantity of water being used
Helicopter control | Determine the best operation actions by judging human instructions and the flying conditions including wind speed and direction

Designing a Fuzzy Logic Controller

A fuzzy logic controller diagram was shown in Chapter 3. Let us redraw it now and discuss a design example. Refer to Figure 16.2. For the purpose of discussion, let us assume that this FLC controls a hot water heater. The hot water heater has a knob, HeatKnob(0-10) on it to control the heating element power, the higher the value, the hotter it gets, with a value of 0 indicating the heating element is turned off. There are two sensors in the hot water heater, one to tell you the temperature of the water (TempSense), which varies from 0 to 125° C, and the other to tell you the level of the water in the tank (LevelSense), which varies from 0 = empty to 10 = full. Assume that there is an automatic flow control that determines how much cold water (at temperature 10° C) flows into the tank from the main water supply; whenever the level of the water gets below 40, the flow control turns on, and turns off when the level of the water gets above 95.

Figure 16.2 Fuzzy control of a water heater.

The design objective can be stated as:

Keep the water temperature as close to 80° C as possible, in spite of changes in the water flowing out of the tank, and cold water flowing into the tank.

Step One: Defining Inputs and Outputs for the FLC

The range of values that inputs and outputs may take is called the universe of discourse. We need to define the universe of discourse for all of the inputs and outputs of the FLC, which are all crisp values. Table 16.11 shows the ranges:

Table 16.11 Universe of Discourse for Inputs and Outputs for FLC
Name | Input/Output | Minimum value | Maximum value
LevelSense | I | 0 | 10
HeatKnob | O | 0 | 10
TempSense | I | 0 | 125

Step Two: Fuzzify the Inputs

The inputs to the FLC are the LevelSense and the TempSense. We can use triangular membership functions to fuzzify the inputs, just as we did in Chapter 3, when we constructed the fuzzifier program. There are some general guidelines you can keep in mind when you determine the range of the fuzzy variables as related to the crisp inputs (adapted from Yan, et al.):

1. Symmetrically distribute the fuzzified values across the universe of discourse.
2. Use an odd number of fuzzy sets for each variable so that some set is assured to be in the middle. The use of 5 to 7 sets is fairly typical.
3. Overlap adjacent sets (by 15% to 25% typically) .

Both the input variables LevelSense and TempSense are restricted to positive values. We use the following fuzzy sets to describe them:

XSmall, Small, Medium, Large, XLarge

In Table 16.12 and Figure 16.3, we show the assignment of ranges and triangular fuzzy membership functions for LevelSense. Similarly, we assign ranges and triangular fuzzy membership functions for TempSense in Table 16.13 and Figure 16.4. The optimization of these assignments is often done through trial and error for achieving optimum performance of the FLC.

Table 16.12 Fuzzy Variable Ranges for LevelSense
Crisp Input Range | Fuzzy Variable
0–2 | XSmall
1.5–4 | Small
3–7 | Medium
6–8.5 | Large
7.5–10 | XLarge

Figure 16.3 Fuzzy membership functions for LevelSense.

Table 16.13 Fuzzy Variable Ranges for TempSense
Crisp Input Range | Fuzzy Variable
0–20 | XSmall
10–35 | Small
30–75 | Medium
60–95 | Large
85–125 | XLarge

Figure 16.4 Fuzzy membership functions for TempSense.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Step Three: Set Up Fuzzy Membership Functions for the Output(s)

In our example, we have just one output, which is the HeatKnob. We need to assign fuzzy memberships to this variable just as we did for the inputs. This is shown in Table 16.14 and Figure 16.5. We use different variable names to make the example clearer later on.

Table 16.14 Fuzzy Variable Ranges for HeatKnob
Crisp Input Range | Fuzzy Variable
0–2 | VeryLittle
1.5–4 | ALittle
3–7 | AGoodAmount
6–8.5 | ALot
7.5–10 | AWholeLot

Figure 16.5 Fuzzy membership functions for the output HeatKnob.

Step Four: Create a Fuzzy Rule Base

Now that you have the inputs and the output defined in terms of fuzzy variables, you need only specify what actions to take under what conditions; that is, you need to construct a set of rules that describe the operation of the FLC. These rules usually take the form of IF–THEN rules and can be obtained from a human expert (heuristics), or can be supplied from a neural network that infers the rules from behavior of the system. We mentioned this idea in Chapter 3.

Let us construct a rule base for our design example. For the two inputs, we define the matrix shown in Table 16.15. Our heuristic guidelines in determining this matrix are the following statements and their converses:

1. When the temperature is low, the HeatKnob should be set higher than when the temperature is high.
2. When the volume of water is low, the HeatKnob does not need to be as high as when the volume of water is high.

NOTE: In FLCs, we do not need to specify all the boxes in the matrix. That is perfectly fine. No entry signifies that no action is taken, for example, in the column for SenseTemp=XL, no action is required since the temperature is already at or above the target temperature.

Table 16.15 Fuzzy Rule Base for the Example Design, Output HeatKnob
SenseTemp-> Sense Level | XS | S | M | L | XL
\/ |  |  |  |  |
XS | AGoodAmount | ALittle | VeryLittle |  |
S | ALot | AGoodAmount | VeryLittle | VeryLittle |
M | AWholeLot | ALot | AGoodAmount | VeryLittle |
L | AWholeLot | ALot | ALot | ALittle |
XL | AWholeLot | ALot | ALot | AGoodAmount

Let us examine a couple of typical entries in the table: For SenseLevel = Medium (M) and SenseTemp = XSmall (XS), the output is HeatKnob = AWholeLot. Now for the same temperature, as the water level rises, the setting on HeatKnob also should rise to compensate for the added volume of water. You can see that for SenseLevel = Large(L), and SenseTemp = XSmall(XS), the output is HeatKnob = AWholeLot. You can verify that the rest of the table is created by similar reasoning.

Creating IF–THEN Rules

We can now translate the table entries into IF - THEN rules. We take these directly from Table 16.15:

1. IF SenseTemp IS XSmall AND SenseLevel IS XSmall THEN SET HeatKnob TO AGoodAmount
2. IF SenseTemp IS XSmall AND SenseLevel IS Small THEN SET HeatKnob TO ALot
3. IF SenseTemp IS XSmall AND SenseLevel IS Medium THEN SET HeatKnob TO AWholeLot
4. IF SenseTemp IS XSmall AND SenseLevel IS Large THEN SET HeatKnob TO AWholeLot
5. IF SenseTemp IS XSmall AND SenseLevel IS XLarge THEN SET HeatKnob TO AWholeLot
6. IF SenseTemp IS Small AND SenseLevel IS XSmall THEN SET HeatKnob TO ALittle
7. IF SenseTemp IS Small AND SenseLevel IS Small THEN SET HeatKnob TO AGoodAmount
8. IF SenseTemp IS Small AND SenseLevel IS Medium THEN SET HeatKnob TO ALot
9. IF SenseTemp IS Small AND SenseLevel IS Large THEN SET HeatKnob TO ALot
10. IF SenseTemp IS Small AND SenseLevel IS XLarge THEN SET HeatKnob TO ALot
11. IF SenseTemp IS Medium AND SenseLevel IS XSmall THEN SET HeatKnob TO VeryLittle
12. IF SenseTemp IS Medium AND SenseLevel IS Small THEN SET HeatKnob TO VeryLittle
13. IF SenseTemp IS Medium AND SenseLevel IS Medium THEN SET HeatKnob TO AGoodAmount
14. IF SenseTemp IS Medium AND SenseLevel IS Large THEN SET HeatKnob TO ALot
15. IF SenseTemp IS Medium AND SenseLevel IS XLarge THEN SET HeatKnob TO ALot
16. IF SenseTemp IS Large AND SenseLevel IS Small THEN SET HeatKnob TO VeryLittle
17. IF SenseTemp IS Large AND SenseLevel IS Medium THEN SET HeatKnob TO VeryLittle
18. IF SenseTemp IS Large AND SenseLevel IS Large THEN SET HeatKnob TO ALittle
19. IF SenseTemp IS Large AND SenseLevel IS XLarge THEN SET HeatKnob TO AGoodAmount

Remember that the output and inputs to the fuzzy rule base are fuzzy variables. For any given crisp input value, there may be fuzzy membership in several fuzzy input variables (determined by the fuzzification step). And each of these fuzzy input variable activations will cause different fuzzy output cells to fire, or be activated. This brings us to the final step, defuzzification of the output into a crisp value.

Step Five: Defuzzify the Outputs

In order to control the HeatKnob, we need to obtain a crisp dial setting. So far, we have several of the IF–THEN rules of the fuzzy rule base firing at once, because the inputs have been fuzzified. How do we arrive at a single crisp output number ? There are actually several different strategies for this; we will consider two of the most common, the center of area (COA) or centroid method, and the fuzzy Or method. The easiest way to understand the process is with an example.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Assume that at a particular point in time, LevelSense = 7.0 and TempSense = 65. These are the crisp inputs directly from the sensors. With fuzzification (refer to Chapter 3 for a review), assume that you get the following fuzzy memberships:

crisp input — LevelSense = 7.0
fuzzy outputs with membership values -
Medium: 0.4
Large: 0.6
all others : 0.0
crisp input — TempSense=65
fuzzy outputs with membership values -
Medium: 0.75
Large: 0.25
all others: 0.0

This results in four rules firing:

1. TempSense = Medium (0.75) AND LevelSense = Medium (0.4)
2. TempSense = Large (0.25) AND LevelSense = Medium (0.4)
3. TempSense = Medium (0.75) AND LevelSense = Large (0.6)
4. TempSense = Large (0.25) AND LevelSense = Large (0.6)

First you must determine, for each of the AND clauses in the IF–THEN rules, what the output should be. This is done by the conjunction or minimum operator. So for each of these rules you have the following firing strengths:

1. (0.75) ⊥ (0.4) = 0.4
2. (0.25) ⊥ (0.4) = 0.25
3. (0.75) ⊥ (0.6) = 0.6
4. (0.25) ⊥ (0.6) = 0.25

By using the fuzzy rule base and the strengths assigned previously, we find the rules recommend the following output values (with strengths) for HeatKnob:

1. AGoodAmount (0.4)
2. VeryLittle (0.25)
3. ALot (0.6)
4. ALittle (0.25)

Now we must combine the recommendations to arrive at a single crisp value. First, we will use the fuzzy Or method of defuzzification. Here we use a disjunction or maximum operator to combine the values. We obtain the following:

     (0.4) ⊦ (0.25) ⊦ (0.6) ⊦ (0.25) = 0.6

The crisp output value for HeatKnob would then be this membership value multiplied by the range of the output variable, or (0.6) (10-0) = 6.0.

Another way of combining the outputs is with the centroid method. With the centroid method, there are two variations, the overlap composition method and the additive composition method. To review, we have the following output values and strengths.

1. AGoodAmount (0.4)
2. VeryLittle (0.25)
3. ALot (0.6)
4. ALittle (0.25)

We use the strength value and fill in the particular triangular membership function to that strength level. For example, for the first rule, we fill the triangular membership function, AGoodAmount to the 0.4 level. We then cut off the top of the triangle (above 0.4). Next we do the same for the other rules. Finally we align the truncated figures as shown in Figure 16.6 and combine them according to the overlap composition method or the additive composition method (Kosko). You can see the difference in these two composition methods in Figure 16.6. The overlap method simply superimposes all of the truncated triangles onto the same area. You can lose information with this method. The additive method adds the geometrical figures on top of each other.

Figure 16.6 Defuzzification with the Centroid approach: Overlap and Additive composition.

In order to get a crisp value, you take the centroid or center of gravity, of the resulting geometrical figure. Let us do this for the overlap method figure. The centroid is a straight edge that can be placed through the figure to have it perfectly balanced; there is equal area of the figure on either side of the straight edge, as shown in Figure 16.7. Splitting up the geometry into pieces and summing all area contributions on either side of the centroid, we get a value of 5.2 for this example. This is already in terms of the crisp output value range:

     HeatKnob = 5.2

Figure 16.7 Finding the centroid.

This completes the design for the simple example we chose. We conclude with a list of the advantages and disadvantages of FLCs.

Advantages and Disadvantages of Fuzzy Logic Controllers

The following list adapted from McNeill and Thro shows the advantages and disadvantages to FLCs for control systems as compared to more traditional control systems.

Advantages:

• Relates input to output in linguistic terms, easily understood
• Allows for rapid prototyping because the system designer doesn't need to know everything about the system before starting
• Cheaper because they are easier to design
• Increased robustness
• Simplify knowledge acquisition and representation
• A few rules encompass great complexity
• Can achieve less overshoot and oscillation
• Can achieve steady state in a shorter time interval

Disadvantages:

• Hard to develop a model from a fuzzy system
• Require more fine tuning and simulation before operational
• Have a stigma associated with the word fuzzy (at least in the Western world); engineers and most other people are used to crispness and shy away from fuzzy control and fuzzy decision making

Summary

Fuzzy logic applications are many and varied. You got an overview of the different applications areas that exist for fuzzy logic, from the control of washing machines to fuzzy logic based cost benefit analysis. Further you got details on two application domains: fuzzy databases and fuzzy control.

This chapter dealt with extending database models to accommodate fuzziness in data attribute values and in queries as well. You saw fuzzy relations; in particular, similarity and resemblance relations, and similarity classes were reviewed. You found how possibility distributions help define fuzzy databases. You also learned what fuzzy events are and how to calculate fuzzy means, fuzzy variances, and fuzzy conditional expectations. Concepts related to linear possibility regression model were presented.

The chapter presented the design of a simple fuzzy logic control (FLC) system to regulate the temperature of water in a hot water heater. The components of the FLC were discussed along with design procedures. The advantages of FLC design include rapid prototyping ability and the capability to solve very nonlinear control problems without knowing details of the nonlinearities.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Chapter 17
Further Applications

Introduction

In this chapter, we present the outlines of some applications of neural networks and fuzzy logic. Most of the applications fall into a few main categories according to the paradigms they are based on. We offer a sampling of topics of research as found in the current literature, but there are literally thousands of applications of neural networks and fuzzy logic in science, technology and business with more and more applications added as time goes on.

Some applications of neural networks are for adaptive control. Many such applications benefit from adding fuzziness also. Steering a car or backing up a truck with a fuzzy controller is an example. A large number of applications are based on the backpropagation training model. Another category of applications deals with classification. Some applications based on expert systems are augmented with a neural network approach. Decision support systems are sometimes designed this way. Another category is made up of optimizers, whose purpose is to find the maximum or the minimum of a function.

NOTE: You will find other neural network applications related to finance presented toward the end of Chapter 14.

Computer Virus Detector

IBM Corporation has applied neural networks to the problem of detecting and correcting computer viruses. IBM's AntiVirus program detects and eradicates new viruses automatically. It works on boot-sector types of viruses and keys off of the stereotypical behaviors that viruses usually exhibit. The feedforward backpropagation neural network was used in this application. New viruses discovered are used in the training set for later versions of the program to make them "smarter." The system was modeled after knowledge about the human immune system: IBM uses a decoy program to "attract" a potential virus, rather than have the virus attack the user's files. These decoy programs are then immediately tested for infection. If the behavior of the decoy program seems like the program was infected, then the virus is detected on that program and removed wherever it's found.

Mobile Robot Navigation

C. Lin and C. Lee apply a multivalued Boltzmann machine, modeled by them, using an artificial magnetic field approach. They define attractive and repulsive magnetic fields, corresponding to goal position and obstacle, respectively. The weights on the connections in the Boltzmann machine are none other than the magnetic fields.

They divide a two-dimensional traverse map into small grid cells. Given the goal cell and obstacle cells, the problem is to navigate the two-dimensional mobile robot from an unobstructed cell to the goal quickly, without colliding with any obstacle. An attracting artificial magnetic field is built for the goal location. They also build a repulsive artificial magnetic field around the boundary of each obstacle. Each neuron, a grid cell, will point to one of its eight neighbors, showing the direction for the movement of the robot. In other words, the Boltzmann machine is adapted to become a compass for the mobile robot.

A Classifier

James Ressler and Marijke Augusteijn study the use of neural networks to the problem of weapon to target assignment. The neural network is used as a filter to remove unfeasible assignments, where feasibility is determined in terms of the weapon's ability to hit a given target if fired at a specific instant. The large number of weapons and threats along with the limitation on the amount of time lend significance to the need for reducing the number of assignments to consider.

The network's role here is classifier, as it needs to separate the infeasible assignments from the feasible ones. Learning has to be quick, and so Ressler and Augusteijn prefer to use an architecture called the cascade-correlation learning architecture, over backpropagation learning. Their network is dynamic in that the number of hidden layer neurons is determined during the training phase. This is part of a class of algorithms that change the architecture of the network during training.

A Two-Stage Network for Radar Pattern Classification

Mohammad Ahmadian and Russell Pimmel find it convenient to use a multistage neural network configuration, a two-stage network in particular, for classifying patterns. The patterns they study are geometrical features of simulated radar targets.

Feature extraction is done in the first stage, while classification is done in the second. Moreover, the first stage is made up of several networks, each for extracting a different estimable feature. Backpropagation is used for learning in the first stage. They use a single network in the second stage. The effect of noise is also studied.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Crisp and Fuzzy Neural Networks for Handwritten Character Recognition

Paul Gader, Magdi Mohamed, and Jung-Hsien Chiang combine a fuzzy neural network and a crisp neural network for the recognition of handwritten alphabetic characters. They use backpropagation for the crisp neural network and a clustering algorithm called K-nearest neighbor for the fuzzy network. Their consideration of a fuzzy network in this study is prompted by their belief that if some ambiguity is possible in deciphering a character, such ambiguity should be accurately represented in the output. For example, a handwritten "u" could look like a "v" or "u." If present, the authors feel that this ambiguity should be translated to the classifier output.

Feature extraction was accomplished as follows: character images of size 24x16 pixels were used. The first stage of processing extracted eight feature images from the input image, two for each direction (north, northeast, northwest, and east). Each feature image uses an integer at each location that represents the length of the longest bar that fits at that point in that direction. These are referred to as bar features. Next 8x8 overlapping zones are used on the feature images to derive feature vectors. These are made by taking the summed values of the values in a zone and dividing this by the maximum possible value in the zone. Each feature image results in a 15,120 element feature vectors.

Data was obtained from the U.S. Postal Office, consisting of 250 characters. Results showed 97.5% and 95.6% classification rates on training and test sets, respectively, for the neural network. The fuzzy network resulted in 94.7% and 93.8% classification rates, where the desired output for many characters was set to ambiguous.

Noise Removal with a Discrete Hopfield Network

Arun Jagota applies what is called a HcN, a special case of a discrete Hopfield network, to the problem of recognizing a degraded printed word. HcN is used to process the output of an Optical Character Recognizer, by attempting to remove noise. A dictionary of words is stored in the HcN and searched.

Object Identification by Shape

C. Ganesh, D. Morse, E. Wetherell, and J. Steele used a neural network approach to an object identification system, based on the shape of an object and independent of its size. A two-dimensional grid of ultrasonic data represents the height profile of an object. The data grid is compressed into a smaller set that retains the essential features. Backpropagation is used. Recognition on the order of approximately 70% is achieved.

Detecting Skin Cancer

F. Ercal, A. Chawla, W. Stoecker, and R. Moss study a neural network approach to the diagnosis of malignant melanoma. They strive to discriminate tumor images as malignant or benign. There are as many as three categories of benign tumors to be distinguished from malignant melanoma. Color images of skin tumors are used in the study. Digital images of tumors are classified. Backpropagation is used. Two approaches are taken to reduce training time. The first approach involves using fewer hidden layers, and the second involves randomization of the order of presentation of the training set.

EEG Diagnosis

Fred Wu, Jeremy Slater, R. Eugene Ramsay, and Lawrence Honig use a feedforward backpropagation neural network as a classifier in EEG diagnosis. They compare the performance of the neural network classifier to that of a nearest neighbor classifier. The neural network classifier shows a classifier accuracy of 75% for Multiple Sclerosis patients versus 65% for the nearest neighbor algorithm.

Time Series Prediction with Recurrent and Nonrecurrent Networks

Sathyanarayan Rao and Sriram Sethuraman take a recurrent neural network and a feedforward network and train then in parallel. A recurrent neural network has feedback connections from the output neurons back to input neurons to model the storage of temporal information. A modified backpropagation algorithm is used for training the recurrent network, called the real-time recurrent learning algorithm. They have the recurrent neural network store past information, and the feedforward network do the learning of nonlinear dependencies on the current samples. They use this scheme because the recurrent network takes more than one time period to evaluate its output, whereas the feedforward network does not. This hybrid scheme overcomes the latency problem for the recurrent network, providing immediate nonlinear evaluation from input to output.

Security Alarms

Deborah Frank and J. Bryan Pletta study the application of neural networks for alarm classification based on their operation under varying weather conditions. Performance degradation of a security system when the environment changes is a cause for losing confidence in the system itself. This problem is more acute with portable security systems.

They investigated the problem using several networks, ranging from backpropagation to learning vector quantization. Data was collected using many scenarios, with and without the coming of an intruder, which can be a vehicle or a human.

They found a 98% probability of detection and 9% nuisance alarm rate over all weather conditions.


C++ Neural Networks and Fuzzy Logic

 C++ Neural Networks and Fuzzy Logic - Valluru B. Rao

Circuit Board Faults

Anthony Mason, Nicholas Sweeney, and Ronald Baer studied the neural network approach in two laboratory experiments and one field experiment, in diagnosing faults in circuit boards.

Test point readings were expressed as one vector. A fault vector was also defined with elements representing possible faults. The two vectors became a training pair. Backpropagation was used.

Warranty Claims

Gary Wasserman and Agus Sudjianto model the prediction of warranty claims with neural networks. The nonlinearity in the data prompted this approach.

The motivation for the study comes from the need to assess warranty costs for a company that offers extended warranties for its products. This is another application that uses backpropagation. The architecture used was 2-10-1.

Writing Style Recognition

J. Nellis and T. Stonham developed a neural network character recognition system that adapts dynamically to a writing style.

They use a hybrid neural network for hand-printed character recognition, that integrates image processing and neural network architectures. The neural network uses random access memory (RAM) to model the functionality of an individual neuron. The authors use a transform called the five-way image processing transform on the input image, which is of size 32x32 pixels. The transform converts the high spatial frequency data in a character into four low frequency representations. What they achieve by this are position invariance, and a ratio of black to white pixels approaching 1, rotation invariance, and capability to detect and correct breaks within characters. The transformed data are input to the neural network that is used as a classifier and is called a discriminator.

A particular writing style that has less variability and therefore fewer subclasses is needed to classify the style. Network size will also reduce confusion, and conflicts lessen.

Commercial Optical Character Recognition

Optical character recognition (OCR) is one of the most successful commercial applications of neural networks. Caere Corporation brought out its neural network product in 1992, after studying more than 100,000 examples of fax documents. Caere's AnyFax technology combines neural networks with expert systems to extract character information from Fax or scanned images. Calera, another OCR vendor, started using neural networks in 1984 and also benefited from using a very large (more than a million variations of alphanumeric characters) training set.

ART-EMAP and Object Recognition

A neural network architecture called ART-EMAP (Gail Carpenter and William Ross) integrates Adaptive Resonance Theory (ART) with spatial and temporal evidence integration for predictive mapping (EMAP). The result is a system capable of complex 3-D object recognition. A vision system that samples two-dimensional perspectives of a three-dimensional object is created that results in 98% correct recognition with an average of 9.2 views presented on noiseless test data, and 92% recognition with an average of 11.2 views presented on noisy test data. The ART-EMAP system is an extension of ARTMAP, which is a neural network architecture that performs supervised learning of recognition categories and multidimensional maps in response to input vectors. A fuzzy logic extension of ARTMAP is called Fuzzy ARTMAP, which incorporates two fuzzy modules in the ART system.

Summary

A sampling of current research and commercial applications with neural networks and fuzzy logic technology is presented. Neural networks are applied toward a wide variety of problems, from aiding medical diagnosis to detecting circuit faults in printed circuit board manufacturing. Some of the problem areas where neural networks and fuzzy logic have been successfully applied are:

• Filtering
• Image processing
• Intelligent control
• Machine vision
• Motion analysis
• Optimization
• Pattern recognition
• Prediction
• Time series analysis
• Speech synthesis
• Machine learning and robotics
• Decision support systems
• Classification
• Data compression
• Functional approximation

The use of fuzzy logic and neural networks in software and hardware systems can only increase!


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Afterword

When writing about data compression, I am haunted by the idea that many of the techniques discussed in this book have been patented by their inventors or others. The knowledge that a data compression algorithm can effectively be taken out of the hands of programmers through the use of so-called "intellectual property" law seems contrary to the basic principles that led me and many others into this profession.

I have yet to see any evidence that applying patents to software advances that art or protects the rights of inventors. Several companies continue to collect royalties on patents long after their inventors have moved onto bigger and better thing with other companies. Have the patent-holders done anything notable other than collect royalties? Have they advanced the art of computer science?

Making a software product into a commercial success requires innovation, good design, high-quality documentation, and listening to customers. These are things that nobody can steal from you. On the other hand, a mountain of patents can't keep you from letting these things slip away through inattention or complacency. This lesson seems to be lost on those who traffic in intellectual property "portfolios."

What can you do? First, don't patent your own work, and discourage your peers from doing so. Work on improving your products, not erecting legal obstacles to competition. Secondly, lobby for change. This means change within your company, those you do business with, and most importantly, within the federal government. Write to your congressman and your senator. Write to the ACM. Write to the House Subcommittee on Intellectual Property. And finally, you can join me by becoming a member of the League for Programming Freedom. Write for more information:

League For Programming Freedom
1 Kendall Square #143
P.O. Box 9171
Cambridge, MA 02139

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

I concluded, we kinotropists must be numbered among Britain's most adept programmers of Enginery of any sort, and virtually all advances on the compression of data have originated as kinotropic applications.

At this point, he interrupted again, asking if I had indeed said "the compression of data," and was I familiar with the term "algorithmic compression"? I assured him I was.

The Difference Engine

William Gibson and Bruce Sterling

Why This Book Is For You

If you want to learn how programs like PKZIP and LHarc work, this book is for you. The compression techniques used in these programs are described in detail, accompanied by working code. After reading this book, even the novice C programmer will be able to write a complete compression/archiving program that can be ported to virtually any operating system or hardware platform.

If you want to include data compression in other programs you write, this book will become an invaluable tool. It contains dozens of working programs with C code that can easily be added to your applications. In-depth discussions of various compression methods will help you make intelligent decisions when creating programs that use data compression.

If you want to learn why lossy compression of graphics is the key factor in enabling the multimedia revolution, you need this book. DCT-based compression like that used by the JPEG algorithm is described in detail. The cutting edge technology of fractal compression is explained in useful terms, instead of the purly theoretical. Working programs let you experiment with these fascinating new technologies.

The Data Compression Book provides you with a comprehensive reference to this important field. No other book available has the detailed description of compression algorithms or working C implementations for those algorithms. If you are planning to work in this field, The Data Compression Book is indispensable.

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Appendix A
Statistics for Compression Programs

This appendix gives statistics for some of the compression programs found in this book. The data sets used to test the compression where identical to the ones I used when judging the 1991 Dr. Dobb's Journal Data Compression Contest. The results of that contest can be found in the November 1991 issue of Dr. Dobb's Journal.

The compression and expansion speeds given here should be taken with a grain of salt. First of all, no attempt was made to optimize these programs. Secondly, some variation will be seen depending on what compiler was used to build the executable. Most of the executables were built using Borland C++, but in a few cases, expanded memory requirements let me to use Zortech C++ with either the 286 or 386 DOS Extender.

 |  | Compression Ratios
 | Graphics | Executables | Text Files | Overall
HUFF.C Chapter 3
Huffman Coding | 27.22 | 24.79 | 40.38 | 31.04
AHUFF.C Chapter4
Adaptive Huffman | 32.59 | 26.69 | 40.72 | 33.27
ARITH.C Chapter 5
Arithmetic Coding | 27.78 | 25.25 | 40.81 | 31.51
ARITHN.C Chapter 6
Order = 1 | 61.66 | 44.74 | 59.60 | 54.49
ARITHN.C Chapter 6 (1)
Order = 2 | 59.85 | 50.47 | 68.43 | 59.37
ARITHN.C Chapter 6 (1)
Order = 3 | 58.51 | 51.55 | 71.86 | 60.67
LZSS.C Chapter 8
Index/Length = 12/4 | 43.45 | 41.44 | 58.83 | 48.22
LZSS.C Chapter 8
Index/Length = 13/4 | 44.57 | 42.56 | 60.91 | 49.69
LZSS.C Chapter 8 (3)
Index/Length = 14/4 | 44.71 | 42.32 | 61.10 | 49.70
LZSS.C Chapter 8
Index/Length = 12/3 | 42.83 | 40.38 | 57.27 | 47.10
LZSS.C Chapter 8
Index/Length = 12/5 | 43.91 | 42.31 | 59.21 | 48.81
LZSS.C Chapter 8
Index/Length = 12/8 | 40.60 | 39.60 | 54.67 | 45.29
LZW15V.C Chapter 9
15 bit variable LZW | 48.44 | 36.15 | 58.28 | 47.31
LZW15V.C Chapter 9
14 bit variable LZW | 48.23 | 36.27 | 57.76 | 47.11
LZW15V.C Chapter 9
13 bit variable LZW | 47.76 | 36.34 | 56.71 | 46.65
LZW15V.C Chapter 9
12 bit variable LZW | 46.78 | 36.61 | 54.82 | 45.81
LZW12.C Chapter 9
12 bit fixed LZW | 20.61 | 15.07 | 50.32 | 29.20
HUFF.C Chapter 3
Huffman Coding | 15273 | 13306 | 13353 | 13835
AHUFF.C Chapter 4
Adaptive Huffman | 7553 | 6200 | 7523 | 7028
ARITH.C Chapter 5
Arithmetic Coding | 4616 | 4667 | 4474 | 4584
ARITHN.C Chapter 6
Order = 1 | 804 | 395 | 953 | 702
ARITHN.C Chapter 6 (1)
Order = 2 | 929 | 462 | 1158 | 834
ARITHN.C Chapter 6 (1)
Order = 3 | 975 | 483 | 1206 | 871
LZSS.C Chapter 8
Index/Length = 12/4 | 3657 | 4159 | 3165 | 3671
LZSS.C Chapter 8
Index/Length = 13/4 | 3268 | 3853 | 2839 | 3336
LZSS.C Chapter 8 (3)
Index/Length = 14/4 | 3082 | 3832 | 2563 | 3180
LZSS.C Chapter 8
Index/Length = 12/3 | 4093 | 4392 | 3594 | 4027
LZSS.C Chapter 8
Index/Length = 12/5 | 2942 | 3764 | 2773 | 3194
LZSS.C Chapter 8
Index/Length = 12/8 | 1831 | 1460 | 2411 | 1899
LZW15V.C Chapter 9
15 bit variable LZW | 14913 | 11744 | 13332 | 13140
LZW15V.C Chapter 9
14 bit variable LZW | 14850 | 11332 | 12783 | 12769
LZW15V.C Chapter 9
13 bit variable LZW | 14256 | 10904 | 12241 | 12257
LZW15V.C Chapter 9
12 bit variable LZW | 13669 | 10320 | 11432 | 11591
LZW12.C Chapter 9
12 bit fixed LZW | 13106 | 11057 | 14379 | 12786

 |  | Expansion Rates
 | Graphics | Executables | Text Files | Overall
HUFF.C Chapter 3
Huffman Coding | 12428 | 10892 | 12560 | 11891
AHUFF.C Chapter 4
Adaptive Huffman | 7366 | 6141 | 7755 | 7041
ARITH.C Chapter 5
Arithmetic Coding | 2320 | 2205 | 2074 | 2188
ARITHN.C Chapter 6
Order = 1 | 793 | 405 | 944 | 700
ARITHN.C Chapter 6 (1)
Order = 2 | 905 | 472 | 1136 | 824
ARITHN.C Chapter 6 (1)
Order = 3 | 946 | 492 | 1173 | 855
LZSS.C Chapter 8
Index/Length = 12/4 | 18622 | 16526 | 17424 | 17394
LZSS.C Chapter 8
Index/Length = 13/4 | 18899 | 16716 | 17798 | 17673
LZSS.C Chapter 8 (3)
Index/Length = 14/4 | 30334 | 25120 | 27120 | 27196
LZSS.C Chapter 8
Index/Length = 12/3 | 18248 | 16572 | 16756 | 17074
LZSS.C Chapter 8
Index/Length = 12/5 | 18680 | 16674 | 17266 | 17409
LZSS.C Chapter 8
Index/Length = 12/8 | 18113 | 16230 | 16673 | 16879
LZW15V.C Chapter 9
15 bit variable LZW | 15167 | 11986 | 13077 | 13206
LZW15V.C Chapter 9
14 bit variable LZW | 14942 | 11804 | 12911 | 13018
LZW15V.C Chapter 9
13 bit variable LZW | 14445 | 11462 | 12495 | 12609
LZW15V.C Chapter 9
12 bit variable LZW | 13567 | 10893 | 11608 | 11845
LZW12.C Chapter 9
12 bit fixed LZW | 15326 | 13978 | 15705 | 14950

Notes:
(1) Built with Zortech's 286 DOS Extender, so as to access all available extended memory. Higher order models can use megabytes of memory.

(2) Built with Zortech's 386 DOS Extender. One of the arrays in the program was larger than 64K, and this was an easy way to rebuild the code without using MS-DOS based "huge" pointers.

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Appendix B
Test Programs

One of the problems I noticed when testing data compression programs for the Dr. Dobb's programming contest of 1991 was that of inadequate testing. Many of the programs I was given failed on several of the test files in our compression database.

After good design, the best weapon to use against these kind of errors is exhaustive testing. I have a test program I use under MS-DOS, named CHURN, which can apply a compression program to every file on a disk volume, performing both compression, decompression, then compare the input to the output. This has proven very helpful to me when testing the software in this book.

The MS-DOS version of the program is given below. Instructions on how to use this program are found in the program listing.

/**************************START OF CHURN.C****************************

*

*  This is a utility program used to test compression/decompression

*  programs for accuracy, speed, and compression ratios.  CHURN is

*  called with three arguments.  The first is a root directory.  CHURN

*  will attempt to compress  and then decompress every file in and under

*  the specified root directory.  The next parameter on the command

*  line is the compression command.  CHURN needs to compress the input

*  file to a file called TEST.CMP.  The compression command tells CHURN

*  how to do this.  CHURN will execute the compression command by

*  passing the command line to DOS using the system() function call.

*  It attempts to insert the file name into the compression command by

*  calling sprintf(), with the file name as an argument.  This means that

*  if the compression command has a %s anywhere in it, the name of the

*  input file should be substituted for it.  Finally, the third argument

*  on the command line should be the command  CHURN needs to spawn to

*  decompress TEST.CMP to TEST.OUT.

*

*  An example of how this works using programs created in this book

*  would look like this:

*

*  CHURN C:\ "LZSS-C %%s test.cmp" "LZSS-E test.cmp test.out"

*

* The doubled up % symbols are there to defeat variable substitution

* under some command-line interpreters, such as 4DOS.

*

*  A more complicated example testing PKZIP might look like this:

*

*  CHURN C:\ "TEST %%S"  "PKUNZIP TEST.CMP"

*

*  where TEST.BAT had two lines that look like this:

*

*  COPY %1 TEST.OUT

*  PKZIP -M TEST.CMP TEST.OUT

*

*  CHURN stores a summary of compression in a file called CHURN.LOG.  This

*  file could be used for further analysis by other programs.

*

*  To abort this program while it is running, don't start pounding away

*  on the BREAK or CTRL-C keys.  They will just get absorbed by the

*  compression program.  Instead, hit a single key, which will be detected

*  by CHURN, and used as an abort signal.

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <time.h>

#include <process.h>

#include <conio.h>

#include <dos.h>

/*

*  The findfirst and findnext functions operate nearly identically

*  under TurboC and MSC.  The only difference is that the functions

*  names, structures, and structure elements all have different names.

*  I just create macros for these things and redefine them appropriately

*  here.

*/

#ifdef__TURBOC__

#include <dir.h>

#define FILE_INFO                     struct ffblk

#define FIND_FIRST( name, info ) findfirst( ( name ), (info ), FA_DIREC )

#define FIND_NEXT( info )             findnext( ( info ) )

#define FILE_IS_DIR( info )           ( ( info ).ff_attrib & FA_DIREC )

#define FILE_NAME( info )             ( ( info ).ff_name )

#else

#define MSDOS 1

#define FILE_INFO                     struct find_t

#define FIND_FIRST( name, info )      _dos_findfirst( ( name ), _A_SUBDIR,

            ( info ) )

#define FIND_NEXT( info )             _dos_findnext( ( info ) )

#define FILE_IS_DIR( info )           ( ( info ).attrib & _A_SUBDIR )

#define FILE_NAME( info )             ( ( info ).name )

#endif

/*

*  Some global variables.

*/

int total_files:

int total_passed;

int total_failed;

char *compress_command;

char *expand_command;

FILE *input;

FILE *output;

FILE *compressed;

FILE *log_file;

/*

*  Declarations for global routines.

*/

void churn_files( char *path );

int file_is_already_compressed( char *name );

void close_all_the_files( void );

int compress( char *file_name );

void usage_exit( void );

/*

*  main() doesn't have to do a whole lot in this program.  It

*  reads in the command line to determine what the root directory

*  to start looking at is, then it initializes the total byte counts

*  and the start time.  It can then call churn_files(), which does all

*  the work, then report on the statistics resulting from churn_files.

*/

void main( int argc, char * argv[] )

{

  time_t start_time;

  time_t stop_time;

  char root_dir[ 81 ];

  if ( argc != 4 )

     usage_exit();

  strcpy( root_dir, argv [  1 ];

  if ( root_dir[ strlen( root_dir ) - 1 ]  != `\\' )

     strcat( root_dir. "\\" );

  compress_command = argv[ 2 ];

  expand_command = argv[ 3 ];

  setbuf( stdout, NULL );

  setbuf( stderr, NULL );

  total_files = 0;

  total_passed = 0;

  total_failed = 0;

  log_file = fopen( "CHURN.LOG", "w" );

  if ( log_file == NULL ) {

    printf( "Couldn't open the log file!\n" );

    exit( 1 );

  }

  fprintf( log_file, "                     "

       "Original Packed\n" );

  fprintf( log_file, "          File Name   "

       " Size Size Ratio  Result\n" );

  fprintf( log_file, "_________________    "

       " _____ _____ ___ ____\n" );

  time( &start_time );

  churn_files( root_dir );

  time( &stop_time );

  fprintf( log_file, "\nTotal elapsed time: %f seconds\n",

       difftime( stop_time, start_time ) );

  fprintf( log_file, "Total files: %d\n", total_files );

  fprintf( log_file, "Total passed: %d\n", total_passed );

  fprintf( log_file, "Total failed: %d\n", total_failed );

}

/*

* churn_files() is a routine that sits in a loop looking at

* files in the directory specified by its single argument, "path".

* As each file is looked at, one of three things happens.  If it

* is a normal file, and has a compressed extension name, like ".ZIP",

* the file is ignored.  If it is a normal file, and doesn't have a

* compressed extension name, it is compressed and decompressed by

* another routine.  Finally, if the file is a subdirectory,

* churn_files() is called recursively with the file name as its

* path argument.  This is one of those rare routines where recursion

* provides a way to truly simplify the task at hand.

*/

void churn_files( char *path )

{

  FILE_INFO file_info;

  int result;

  char full_name[ 81 ];

  strcpy( full_name, path );

  strcat( full_name, "*.*" );

  result = FIND_FIRST( full_name, &file_info );

  while ( result == 0 ) {

    if ( kbhit() ) {

        getch();

        exit(0);

    }

    if ( FILE_IS_DIR( file_info ) ) {

      if ( FILE-NAME( file_info )[ 0 ] != '.' ) {

        strcpy( full_name, path );

        strcat( full_name, FILE_NAME( file-info) );

        strcat( full_name, "\\" );

        churn_files( full_name );

      }

    } else {

      strcpy( full_name, path );

      strcat( full_name, FILE_NAME( file_info ) );

      if ( !file_is_already_compressed( full_name ) ) {

        fprintf( stderr, "Testing %s\n", full_name );

        if ( !compress( full_name ) )

          fprintf( stderr, "Comparison failed!\n );

      }

    }

    result = FIND_NEXT( &file_info );

  }

}

/*

* The job of this routine is simply to check on the file

* whose name is passed as an argument.  The file extension is compared

* against a list of standard extensions that are commonly used on

* compressed files.  If it matches one of these names, we assume it is

* compressed and return a TRUE, otherwise FALSE is returned.

*

* Note that when checking a compression routine for accuracy, it is

* probably a good idea to stub out this routine.  Trying to compress

* "uncompressible" files is a very good exercise for a compression

* program.  It is probably not a good idea when checking compression

* ratios, however.

*/

int file_is_already_compressed( char *name )

{

  char *extension;

  static char *matches[]={ "ZIP", "ICE", "LZH", "ARC", "GIF", "PAK",

                           "ARJ", NULL };

  int i;

  extension=strchr( name, '.' );

  if ( extension++ == NULL )

    return( 0 );

  i = 0;

  while ( matches[ i ] != NULL )

       if ( strcmp( extension, matches[ i++ ] ) == 0 )

         return( 1 );

  return( 0 );

}

/*

* This is the routine that does the majority of the work for

* this program.  It takes a file whose name is passed here.  It first

* compresses, then decompresses that file.  It then compares the file

* to the decompressed output, and reports on the results.

*/

int compress( char *file_name )

{

  long new_size;

  long old_size;

  int c;

  char command[ 132 ];

  printf( "%s\n", file_name );

  fprintf( log_file, "%-40s ", file_name );

  sprintf( command, compress_command, file_name );

  system( command );

  system( command, expand_command, file_name );

  system( command );

  input = fopen( file_name, "rb" );

  output = fopen( "TEST.OUT", "rb" );

  compressed = fopen( "TEST.CMP", "rb" );

  total_files++;

  if ( input == NULL || output == NULL || compressed == NULL ) {

    total_failed++;

    close_all_the_files();

    fprintf( log_file, "Failed, couldn't open file!\n" );

    return( 0 );

  }

  fseek( input, OL, SEEK_END );

  old_size = ftell( input );

  fseek( input, OL, SEEK_SET );

  fseek( compressed, OL, SEEK_END );

  new_size = ftell( compressed );

  fprintf( log_file, "%8ld %8ld ", old_size, new_size );

  if ( old_size == 0L )

    old_size = 1L;

  fprintf( log_file, "%41d%% ",

    100L - ( ( 100L * new_size ) / old_size ) );

  do {

    c = getc( input );

    if ( getc( output ) != c ) {

      fprintf( log_file, "Failed\n" );

      total_failed++;

      close_all_the_files();

      return( 0 );

    }

  }

  while ( c != EOF );

  fprintf( log_file, "Passed\n" );

  close_all_the_files();

  total_passed++;

  return( 1 );

}

void close_all_the_files()

{

  if ( input != NULL )

    fclose( input );

  if ( output != NULL )

    fclose( output );

  if ( compressed != NULL )

    fclose( compressed );

}

/*

* This routine is used to print out basic instructions for the use

* of CHURN, and then exit.

*/

void usage_exit( void )

{

  char *usage = "CHURN 1.0. Usage: CHURN root-dir \"compress "

                "command\" \"expand command\n"

                "\n"

                "CHURN is used to test compression programs. "

                "It does this by compressing\n"

                "then expanding all of the files in and under "

                "the specified root dir.\n"

                "\n"

                "For each file it finds, CHURN first executes "

                "the compress command to create a\n"

                "compressed file called TEST.CMP.  It then "

                "executes the expand command to\n"

                "create a file called TEST.OUT.  CHURN then "

                "compares the two file to make sure\n"

                "the compression cycle worked properly.\n"

                "\n"

                "The file name to be compressed will be "

                "inserted into the compress command\n"

                "using sprintf, with any %%s argument being "

                "substituted with the name of the\n"

                "file being compressed.  Note that the "

                "compress and expand commands should be\n"

                "enclosed in double quotes so that multiple "

                "words can be included in the\n"

                "printf( " commands.\n"

                "\n"

                "Note that you may have to double the %% "

                "character on your command line to get\n"

                "around argument substitution under some "

                "command processors.  Finally, note that\n"

                "CHURN executes the compression program "

                "using a system() function call, so\n"

                "batch files can be used to execute complex\n"

                "compression sequences.\n"

                "\n"

                "Example:  CHURN C:\\ \"LZSS-C %%%%s TEST.CMP\"

                "\"LZSS-C TEST.CMP TEST.OUT\"";

  puts( usage );

  exit( 1 );

}

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Bibliography

Barnsley, M.F., Fractals Everywhere, 2nd edition, Academic Press, San Diego, 1993.

Bell, Timothy C., Cleary, John G., Witten, Ian H,(1990) Text Compression, Prentice Hall, Englewood Cliffs NJ.

Embree, Paul M., and Kimble, Bruce, (1991), C Language Algorithms for Digital Signal Processing, Prentice Hall, Englewood Cliffs, NJ.

Fiala, Edward R., and Greene, Daniel H., "Data compression with finite windows," Communications of the ACM, Volume 32, Number 4, April 1989, pages 490-505.

Fisher, Y. et al., Fractal Image Compression: Theory and Application, Springer Verlag, New York, 1995.

Huffman, D.A., "A method for the construction of minimum-redundancy codes," Proceedings of the IRE, Volume 40, Number 9, September 1952, pages 1098-1101.

Hutchinson, J.E., "Fractals and self-similarity", Indiana University Mathematics Journal, Volume 3, Number 5, 1981, pages 713-747.

Jacquin, A., "Fractal image coding based on a theory of iterated contractive image transformations", Proc. SPIE Visual Communications and Image Processing, 1990, pages 227-239.

Jain, Anil K., Farrelle, Paul M., and Algazi, V. Ralph, (1984) Image Data Compression, Digital Image Processing Techniques, Academic Press, pages 171-226

Karl, John H., An Introduction to Digital Signal Processing, Academic Press, 1989.

Mandelbrot, B.B, The Fractal Geometry of Nature, W.H. Freeman and company, New York, 1983.

Fisher, Y. et al., Fractal Image Compression: Theory and Application, Springer Verlag, New York, 1995.

Monsour, Robert A., and Whiting, Douglas L., "Data Compression Breaks Through to Disk Memory Technology," Computer Technology Review, Volume 11, Number 6, May 1991, pp 39-45

Nelson, Mark, "LZW Data Compression," Dr. Dobb's Journal, Volume 14, Number 10, October 1989, pp 29-37

Ramabadran, Tenkasi V., and Gaitonde, Sunil S., "A Tutorial on CRC Computations,' IEEE Micro, August 1988, pp. 62-75.

Storer, J.A., (1988) Data Compression, Computer Science Press, Rockville, MD

Wallace, Gregory K., "The JPEG Still Picture Compression Standard," Communications of the ACM, Volume 34, Number 4, April 199, pp 31-44.

Welch, Terry, "A Technique for High-Performance Data Compression," IEEE Computer, Volume 17, Number 6, June 1984, pages 8-19.

Witten, Ian H., Neal, Radford M., and Cleary, John G., "Arithmetic Coding for Data Compression," Communications of the ACM, Volume 30, Number 6, June 1987, pages 520-540.

Ziv, J., and Lempel, A., "A universal algorithm for sequential data compression," IEEE Transactions on Information Theory, Volume 23, Number 3, May 1977, pages 337-343.

Ziv, J., and Lempel, "A., Compression of individual sequences via variable-rate coding," IEEE Transactions on Information Theory, Volume 24, Number 5, September 1978, pages 530-536.

Other Resources

An excellent resource for programmers wanting to learn more about Data Compression is the PC Programming Forum on Compuserve. To open a Compuserve account, call their customer service department at 1-800-848-8990. Some files of interest presently on in the Data Compression Library of that forum include:

AR002.EXE | A straightforward compression/archive program, with commented C source.
ARJ241.EXE UNAR24.EXE | The ARJ compression program. ARJ is free for non-commercial use, and includes UNARJ.C, which can be freely distributed and incorporated in other products.
CMPPAS.EXE | LZSS and LZARI compression programs in Turbo Pascal.
COMPR.TXT | A paper by Haruhiko Okumura discussing the state of the art in Japanese shareware compression programs, notably LHarc.
LHA213.EXE LHASRC.EXE | The LHarc compression program, freeware, with C and assembly source.
LZCOMP.EXE | C source for LZSS, LZHUF, and LZSS algorithms.
PAK251.EXE | Shareware compresson program, supports multiple formats.
PKZ204G.EXE | The well known PKZIP shareware archive/compression package.
ZOO210.EXE | The UNIX archiving package. Public Domain.

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Index

A

ACT (Archive Comparison Test), 25
adaptive compression, 76-77, 527. See also adaptive Huffman coding
dictionary-based compression, 203-207
example (QIC-122), 204-207
with graphic files, 325-326
Adaptive Differential Pulse Code Modulation (ADPCM), 319, 527
adaptive Huffman coding, 77-112
algorithm for, 82
code for, 88-89, 101-112
compression program, 91, 101-112
EncodeSymbol routine, 92-95
escape code in, 83-84
expand program, 91-92, 101
incoming symbols, 100-101
initialization of array, 89-90
overflow problem with, 84-87
rescaling, benefit of, 87-88
swapping in, 81, 96-97
symbols not used, coding space for, 82-83
updating tree in, 77-80, 95-100
adaptive modeling, 18-20, 155-162, 527. See also
adaptive compression; adaptive Huffman coding
compression in, 19
decompression in, 19
ADC. See analog-to-digital converter
AddFileListToArchive() routine (CARMAN), 399-400
Add files command (CARMAN), 382-383
AddString() routine (LZSS), 235-238
Adobe, 213
ADPCM (Adaptive Differential Pulse Code Modulation), 319, 527
affine functions, 462-463
affine maps, 476-477
AHUFF.C, 101-112
statistics for, 512-514
alphabets, 527
American National Standards Institute (ANSI), 3
analog-to-digital converter (ADC), 292-295, 311, 312
ANSI (American National Standards Institute), 3
ANSI C, 3-6
prototyping in, 43-44
Apple, 213-214, 298
ARC (compression program), 24, 209, 210, 528
Archive Comparison Test (ACT), 25
archives, 527
ARITH.C, 123, 136-152
statistics for, 512-514
ARITH-1.C., 156-162
code for, 156-158
escape code as fallback with, 159-161
improvements to, 161-162
ARITH-N.C, 163-200
code for, 172-200
escape possibilities, 164-165
flush code for, 170
memory and implementation of, 170
order(-1) table for, 169
order(-2) table for, 169-170
potential improvements to, 171
redesign of data structures for, 166-169
scoreboarding with, 165-166
statistics for, 512-514
updating ARITH-1.C. for, 163-164
arithmetic coding, 16-17, 114-152, 528
code for, 123, 136-152
compression program, 124-125, 136-152
decoding process in, 116-117, 121-122, 133-136
encoding process in, 114-116, 120-121, 130-133
expansion program, 125-126
floating-point math with, 118
flushing encoder in, 133
Huffman coding, advantages over, 113-114, 122-123
initialization
of encoder, 130
of model, 126-130
integer registers, using, 118-120
output from, 114
totals[] array, building, 130
underflow, handling, 120-121
ARJ (compression program), 24, 209, 210, 299-302, 381, 528
Association for Shareware Professionals (ASP), 534-535
AT&T, 290
audio, digital. See digital audio
average_boundaries() function (FRAC.C), 477

B

Barnsley, Michael, 458-459, 462
BITIO.C, 35-43
BITIO.H, 36-38
black cube, 298
block coding, 528
"blocky" images, 469
Boss, Roger, 458, 510
BuildFileList() routine (CARMAN), 393-394
build_model() routine (arithmetic coding), 124, 127-128
build_totals() routine (arithmetic coding), 130
build_tree() (HUFF.C), 58

C

C (programming language)
issues in writing portable, 3-7
reasons for using, 2-3
versions of, 3
CAR files, 382
structure of, 384
CARMAN, 381-455
archive files, opening, 396-398
CAR files in, 382, 384
code, 408-455
command-line processing in, 390-392
command set, 382-383
file list, generating, 392-396
headers, 384-390
CRC, header, 388-390
storing, 385-388
main processing loop of, 399-408
extraction, file, 406-408
input file, skipping/copying, 404-405
insertion, file, 405-406
workspace cleanup, 408
CCITT (Consultive Committee for International Telegraph and Telephone), 22, 326, 528
Central Point Software, 210
CHEETAH.GS, 340, 375-379
CHURN.C, 517-526
classify_domains() function (FRAC.C), 474-475
Cleary, John, 162
codecs, 312-314
codes, 528
coding, 15-17, 528
adaptive, 76-77
modeling vs., 12-13
Collage Theorem, 462
color images, 344-345
fractal compression of, 461
COMPACT (compression program), 23, 529
COMPAND.C, 314-318
companding, 310-318
code, 314-318
codecs with, 312-314
and linear conversion, 311-312
and minimum resolution, 310-311
compilers, 4
COMPRESS (compression program), 23, 208, 529
and LZW, 262
compressed archive files, 381-382
compress_init() function (FRAC.C), 473-474
compression, lossy vs. lossless, 11-12
compression algorithms
adaptive Huffman coding, 91, 101-112
arithmetic coding, 124-125, 136-152
DCT.C, 346-348
fractal compression, 461-467, 472
LZ77, 216-218
LZSS, 227, 231-242
AddString() routine, 235-238
DeleteString() routine, 238-240
exit code, 235
initialization, 232-233
main loop, 233-235
support routines, binary tree, 240-242
LZW, 262-263, 270-271
compression ratios, 7, 529
compress_range() function (FRAC.C), 475-476
Compton's, 213
Compuserve Information Service, 24, 210
constants, in LZSS, 228-229
Consultive Committee for International Telegraph and Telephone (CCITT), 22, 326, 528
Contractive Mapping theorem, 460
ContractNode() routine (LZSS), 240-241
convert_int_to_symbol() routine
arithmetic coding, 124-125, 131, 135-136
statistical modeling, 160-161
Cosine Transform Matrix, 333-335
count_bytes() routine
arithmetic coding, 128
HUFF.C, 56
CPT (compression program), 25
CPU, and arithmetic coding, 16, 17
Cyclical Redundancy Check (CRC), 529

D

DAC. See digital-to-analog converter
data structures, in highest-order modeling, 166-169
DCLZ, 211
DCT. See Discrete Cosine Transform
DCT.C, 345-370
code for, 357-370
compression module, 346-348
expansion algorithm, 353-354
forward DCT routine, 349-350
initialization of, 348-349
input codes, 355-356
OutputCode() routine, 351-353
ReadDCTData() routine, 354-355
WriteDCTData() routine, 350-351
decode_string() routine (LZW), 272-273
DecodeSymbol routine (adaptive Huffman coding), 91
decompression algorithms
adaptive Huffman coding, 91-92, 101
arithmetic coding, 125-126
DCT.C, 353-354
fractal images, 467-469, 477
LZ77, 218-219
LZSS, 242-244
potential improvements to, 243-244
LZW, 264-270, 271-273
implementation, 267
tree maintenance and navigation, 268-270
undefined code, problem with, 266-267
Delete files command (CARMAN), 383
DeleteString() routine (LZSS), 238-240
dictionaries, 530
dictionary-based compression, 20, 201-214
adaptive methods with, 203-207
example (QIC-122), 204-207
ARC, 209
COMPRESS (program), 208
example of, 201-202
general-purpose programs, 210-211
with graphics files, 322-323
hardware-specific programs, 211-212
history of, 207-208
patented software, 212-214
static dictionaries, 202-203
differentiation modulation, 324-325
digital audio
low-pass filters in, 293-294
PC-based sound, 297-299
and range of hearing, 290
replay of sound, 293
and sampling, 291-292, 294-297
waveforms, 290-291
Digital Signal Processor (DSP), 22
digital-to-analog converter (DAC), 293, 295, 297, 311-312
Discrete Cosine Transform (DCT), 22-23, 327-331, 529
code, 357-370
and entropy encoding, 343-344
implementing, 332-333
matrix multiplication, 333-335
inverse, 330, 356-357
output of, 335-336
and quantization, 337-340
sample program, utilizing, 345-370
compression module, 346-348
expansion algorithm, 353-354
forward DCT routine, 349-350
initialization, 348-349
input DCT codes, 355-356
input format, 346
Inverse DCT, 356-357
OutputCode() routine, 351-352
ReadDCTData() routine, 354-355
WriteDCTData() routine, 350-351
two-dimensional, 330
usefulness of, 331-332
and zig-zag sequence, 341-343
Discrete Fourier Transform, 335
Disk Doubler, 211
DOS, compression programs in, 23-24
Dr. Dobb's Journal, 7
DSP. See Digital Signal Processor

E

8-bit characters, 4
encode_ symbol() routine
adaptive Huffman coding, 91-95
arithmetic coding, 131
encoding, 528
entropy, 13-14, 530
entropy encoding, 343-344
ERRHAND.C, 53-54
ERRHAND.H, 53
errors, quantization, 295
escape codes, 530
adaptive Huffman coding, 83-84
ARITH-1.C., 159-161
ARITH-N.C, 164-165
expansion algorithms. See decompression algorithms
Extract() routine (CARMAN), 405-408

F

Fano, R. M., 27
Fast Fourier Transform (FFT), 22, 327-329
Fibonacci function, 84-85
FIF (Fractal Image Format), 459
filters, low-pass, 293-294
find_map() function (FRAC.C), 476-477
FindNextNode() routine (LZSS), 241-242
finite-context modeling, 154
first pass, 155
Fisher, Yuval, 458, 510
floating-point math, with arithmetic coding, 118
flush_arithmetic_encoder() routine (arithmetic coding), 133
flushing, with highest-order modeling, 170
Forward DCT() routine, 349-350
FRAC.C, 471-505
classify_domains() function of, 474-475
code, 477-505
compress_init() function, 473-474
compression results, 506-509
compress_range() function of, 475-476
find_map() function of, 476-477
main compression module, 472-473
traverse_image() function of, 477
fractal image compression, 457-458
decoding, 467-469
history of, 457-459
iterated function systems, 459-463
image compression with, 461-467
mathematics of, 460-461
partitioned iterated function systems, 463-467
patent concerns, 509-510
and resolution independence, 469-471
sample program, 471-509
code, 477-505
decompression module, 477
domain classification in, 474-475
image partitioning in, 475-476
initialization function, 473-474
main compression module, 472-473
optimal affine maps, finding, 476-477
results, compression, 506-509
Fractal Image Format (FIF), 459
FractInt program, 458
freeware, 298-299, 530
function body, 6-7
function declarations, parameters in, 43-44
function prototypes, 4-6

G

G.721 algorithm, 318-319
ghost buffers, 244
Gibb's phenomenon, 457
GIF format, 210, 212-213
Gilchrist, Jeff, 25
global variables, in LZSS, 229-231
Graphical User Interface (GUI), 321
graphics compression
fractal compression. See Fractal image compression
lossy. See Lossy graphics compression
Group 3 FAX, 530
GS.C, 346
code, 370-372
GSDIFF.C, 372-375, 466-467
GUI (Graphical User Interface), 321

H

hard disk, and dictionary-based compression, 211
hashing function (LZW), 268-269
header CRC, 388-390
headers, in CARMAN, 384-390
hearing, range of, 290
Hewlett-Packard, 211
higher-order modeling, 153-154
highest-order modeling, 162-170
escape possibilities, 164-165
flush code for, 170
memory and implementation of, 170
order(-1) and (-2) tables for, 169-170
and redesign of data structures, 166-169
scoreboarding with, 165-166
updating ARITH-1.C. code for, 163-164
HUFF.C
code, 60-73
command lines for, 74
statistics for, 512-514
HUFF-E, command lines for, 74
Huffman, D. A., 15, 16, 531
Huffman coding, 15, 54, 530-531. See also Adaptive Huffman coding
advantages of arithmetic coding over, 122-123
arithmetic coding vs., 16-17
BITIO.C as illustration of, 35-43
difficulties with, 113-114
model of, 12-13
probability table in, 75-76
Shannon-Fano coding vs., 34-35
Huffman table, 18
Huffman tree, 18
algorithm for building, 31-35
building, 55-58
counts, saving, 56-57
symbols, counting, 56
using, 59-60
Hutchinson, J., 458

I

IBM, 213-214, 297
IDCT (Inverse DCT), 329-330, 356-357
IFS. See Iterated Function Systems
information theory, 13-15, 27, 531
initialization
adaptive Huffman coding, 89-90
arithmetic coding, 126-130
DCT.C, 348-349
Discrete Cosine Transform, 348-349
fractal image compression, 473-474
LZSS, 232-233
initialize_model() routine (adaptive Huffman coding), 76
InitializeTree(tree) (adaptive Huffman coding), 90
InitTree() (LZSS), 233
input_counts() (HUFF.C), 57
integer registers, with arithmetic coding, 118-120
Intel, 298
International Standards Organization (ISO), 22, 326, 338, 531
International Telegraph and Telephone Consultative Committee (CCITT), 528
Internet, 25, 298
Inverse DCT (IDCT), 329-330, 356-357
ISO. See International Standards Organization
Iterated Function Systems (IFS), 459-467. See also Partitioned Iterated Function Systems
image compression with, 461-467
mathematics of, 460-461
Iterated Systems, Inc., 458, 509-510

J

Jacobs, Bill, 458, 510
Jacquin, Arnaud, 458
Joint Photographic Experts Group (JPEG), 22, 23, 326, 531. See also JPEG standard
JPEG. See Joint Photographic Experts Group
JPEG standard, 326-345
coding, 340-345
compression algorithm, 327
Discrete Cosine Transform with, 327-340
drawbacks of, 457
Jung, Robert, 209

K

Katz, Phil, 24
K&R C, 3-7

L

Lau, Raymond, 25
League for Programming Freedom, 213
Lehman, Bruce, 214
Lempel, Abraham, 21, 207, 532, 536
LHArc (compression program), 24, 209, 210, 531
library function calls, 3-4
LIFS (Local Iterated Function Systems), 459
Linear Predictive Coding (LPC), 319, 532
LISAW.GS, 464-471
List files command (CARMAN), 383
Local Iterated Function Systems (LIFS), 459
lossless compression, 11-12, 532
of sound, 299-302
lossy compression (generally), 11, 22-23, 532
of sound, 302-303
lossy graphics compression, 321-379
and adaptive coding, 325-326
and differential modulation, 324-325
JPEG standard for, 326-345
coding, 340-345
compression algorithm, 327
Discrete Cosine Transform with, 327-340
and quantization, 337-340
results, analysis of, 375-379
sample program, 345-370
code, 357-370
compression module, 346-348
expansion algorithm, 353-354
forward DCT routine, 349-350
initialization, 348-349
input DCT codes, 355-356
input format, 346
Inverse DCT, 356-357
OutputCode() routine, 351-352
ReadDCTData() routine, 354-355
WriteDCTData() routine, 350-351
statistical and dictionary compression vs., 322-323
support programs, 370
GS.C, 370-372
GSDIFF.C, 372-375
low-pass filters, 293-294
LPC (Linear Predictive Coding), 319, 532
LZ77, 21, 207-208, 209, 215-221, 532
compression algorithm, 216-218
decompression algorithm, 218-219
as "greedy" algorithm, 226
inefficiency of, 221
limitations of, 255
LZ78 vs., 256-257
LZSS vs., 262
problems with, 220-221
and QIC-122, 205
text window in, 215-216
LZ78, 21-22, 207-208, 209, 255-262, 287-288, 532. See also LZW
functioning of, 257-260
implementation of, 260-262
LZ77 vs., 256-257
tree in, 260-261
LZSS, 221-253, 532
and ARJ 2.10, 300-301
balanced trees, maintaining, 225-226
binary tree in, 230-231
and CARMAN, 383, 405
code for, 244-253
compression program, 227, 231-242
AddString() routine, 235-238
DeleteString() routine (LZSS), 238-240
exit code, 235
initialization, 232-233
main loop, 233-235
support routines, binary tree, 240-242
constants in, 228
data structures in, 222-225
expansion program, 242-244
potential improvements to, 243-244
global variables in, 229-231
as "greedy" algorithm, 226-226
LZ77 vs., 262
macros in, 228-229
text window in, 221-222
LZSS.C, 244-253
statistics for, 512-514
LZW, 208, 210, 212, 213, 262-288, 532
code, 273-278
compression program, 262-263, 270-271
decompression program, 264-270, 271-273
and LZW implementation, 267
tree maintenance and navigation, 268-270
undefined code, problem with, 266-267
LZW15V.C as improvement of, 279
patent concerns, 287-288
LZW12.C
code, 273-278
decompression routine in, 271-273
hashing function of, 268-269
"special" nodes in, 270
statistics for, 512-514
LZW15V.C
code, 280-287
as improvement over LZW, 279
statistics for, 512-514

M

Macintosh, compression programs for, 24-25
macros, in LZSS, 228-229
MAIN-C.C, 45-50
MAIN-E.C, 50-53
MAIN.H, 45-46
Mandelbrot, Benoit, 457, 458
Mandelbrot set, 457-458
MassageMSDOSFileName() routine (CARMAN), 394-396
measurement of compression, 7
Microcom Corp., 210
Microsoft, 211, 213
MNP-5, 210
modeling, 17-18. See also adaptive modeling; highest-order modeling; Statistical modeling
coding vs., 12-13
finite-context, 154
higher-order, 153-154
models, 532
static, 535
modulation, differentiation, 324-325
Moving Pictures Experts Group (MPEG), 22, 23, 532-533
MPEG-II, 22
MPEG (Moving Pictures Experts Group), 22, 23, 532-533
MS-DOS, CARMAN with, 392-396

N

NeXT Computer, 298
nodes[] array, 88-89
Nyquist, Harry, 295
Nyquist Rate, 295, 533

O

on-line services, 298
OpenArchiveFiles() command, 397-398
Oracle, 213
order-0 modeling, 154
order(-1) table (ARITH-N.C), 169
order(-2) table (ARITH-N.C), 169
order (of model), 533
OutputBits routine (adaptive Huffman coding), 93
OutputCode() routine (DCT.C), 351-353
output_counts() routine
arithmetic coding, 129
HUFF.C, 57
overflow, in adaptive Huffman coding, 84-87

P

p x 64, 533-534
P6 processor (Intel), 298
parameters, in function declarations, 43-44
ParseArguments() routine (CARMAN), 390-391
Partitioned Iterated Function Systems (PIFS), 458, 467, 509
compression with, 463-467
patent concerns
dictionary-based compression programs, 212-214
fractal image compression, 509-510
LZW algorithm, 287-288
PC Backup, 210
PC-based sound, 297-299
PCX format, 322
phrases, 533
PIFS. See Partitioned Iterated Function Systems
PKWare, 24
PKZ204.EXE, 24
PKZIP, 24, 209, 210, 381, 533
PNG format, 209, 213
Print files command (CARMAN), 383
probability table (Huffman coding), 75-76
prototypes, 43-44
pseudocode, 2

Q

QIC-122, 205-207, 211, 212
quality factor, 345
quantization, 337-340
errors in, 295
matrix for, 338-340

R

Radford, Neal, 161
ReadDCTData() routine, 354-355
redundancy, 13
redundant information, 13
refine_image() function (FRAC.C), 477
Replace files command (CARMAN), 383
rescaling, in adaptive Huffman coding, 87-88
resolution independence, 469-471
RLE (Run-Length Encoding), 341, 534
root mean square (RMS), 346, 466-467, 507
Run-Length Encoding (RLE), 341, 534

S

SAMPLE-3.RAW., 301
sampling, 291-292
of variables, 294-297
scale_counts() routine (arithmetic coding), 128-129
scoreboarding, with highest-order modeling, 165-166
second pass, 155
Shannon, Claude, 13, 14, 27, 202, 534
Shannon-Fano coding, 15, 27-29, 534
Huffman coding vs., 34-35
Shannon-Fano tree, 28-29, 32
algorithm for building, 29-31
shareware, 298-299, 534-535
SILENCE.C, 305-309
silence compression, 303-310
code, 305-309
effectiveness of, 310
16-bit machines, 4
sliding window compression. See LZ77; LZSS
Sloan, Alan, 458-459
Sound Blaster, 298
speech compression, 289-319
ADPCM algorithm, 319
companding, 310-318
code, 314-318
codecs with, 312-314
linear conversion with, 311-312
and minimum resolution, 310-311
and digital audio, 289-299
low-pass filters, 293-294
PC-based sound, 297-299
range of hearing, 290
replay, 293
sampling, 291-292
variables, sampling, 294-297
waveforms, 290-291
G.721 algorithm, 318-319
Linear Predictive Coding, 319
lossless compression, 299-302
lossy compression, 302-303
silence compression, 303-310
code, 305-309
effectiveness of, 310
SQ program, 23, 535
Stac Electronics, 204-205, 210, 211, 213
Stacker, 211
static dictionary-based compression, 20, 202-203
static models, 535
statistical modeling, 14, 18-20, 153-200
and adaptive compression, 155-162
escape code as fallback, using, 159-161
example (ARITH-1.C.), 156-158
improvements to, 161-162
and finite-context modeling, 154
with graphics files, 322
and higher-order modeling, 153-154
highest-order modeling (ARITH-N.C), 162-170
code, 172-200
data structures, redesign of, 166-169
escape probabilities, 164-165
flush code for, 170
implementation, memory and, 170
order(-1) table for, 169
order(-2) table for, 169-170
scoreboarding with, 165-166
updating ARITH-1.C for, 163-164
potential improvements to, 171
statistics, compression, 7, 511-515
__STDC__ macro, 74
StuffIt (compression program), 25
swap_nodes() routine (adaptive Huffman coding), 82
swapping, in adaptive Huffman coding, 81, 96-97
switching equipment, 290
symbols, 535

T

TAR, 24, 535-536
Test files command (CARMAN), 383
test program (CHURN.C), 517-526
text windows
LZ77, 215-216
LZSS, 221-222
32-bit machines, 4
time domain representation, 328
tokens, 536
traverse_image() function (FRAC.C), 477
tree(s). See also Huffman tree
in LZ78, 260-261
in LZSS, 225-226, 230-231
in LZW, 268-270
updating, in adaptive Huffman coding, 77-80, 95-100

U

underflow, 120-121, 132-133
Unisys, 209, 212, 287
UNIX, 4
CARMAN with, 392-393
compression programs in, 23
dictionary-based compression with, 208, 209
update exclusion, 163-164
UpdateModel() routine (adaptive Huffman coding), 76, 77, 91-92
U.S. Patent Office, 213-214
utility programs, 298-299

V

V.4bis, 212
V.42bis, 211
variables, sampling, 294-297
vector quantization, 457, 463-467
VGA display, 321, 346
VOC-HDR.EXE, 298

W

waveforms, 290-291
Welch, Terry, 208, 209, 212, 287, 536
WinZIP, 24
Witten, Ian H., 161
World-Wide Web, 298
WriteDCTData() routine, 350-351
WriteFileHeader() routine (CARMAN), 388

X

Xtract files command (CARMAN), 383

Y

Yoshizaki, Haruyasu, 24, 209

Z

zig-zag sequence, 341-343
ZIP files, 24, 25
ZipIt V1.2, 25
Ziv, Jacob, 21, 207, 532, 536

Table of Contents


Data Compression

 [Spisak knjiga]
The Data Compression Book
Author: Mark Nelson

Afterword
Why This Book Is For You

Chapter 1—Introduction to Data Compression
The Audience
Why C?
Which C?
Issues in Writing Portable C
Keeping Score
The Structure
Chapter 2—The Data-Compression Lexicon, with a History
The Two Kingdoms
Data Compression = Modeling + Coding
The Dawn Age
Coding
An Improvement
Modeling
Statistical Modeling
Dictionary Schemes
Ziv and Lempel
LZ77
LZ78
Lossy Compression
Programs to Know
Chapter 3—The Dawn Age: Minimum Redundancy Coding
The Shannon-Fano Algorithm
The Huffman Algorithm
Huffman in C
BITIO.C
A Reminder about Prototypes
MAIN-C.C AND MAIN-E.C
MAIN-C.C
ERRHAND.C
Into the Huffman Code
Counting the Symbols
Saving the Counts
Building the Tree
Using the Tree
The Compression Code
Putting It All Together
Performance
Chapter 4—A Significant Improvement: Adaptive Huffman Coding
Adaptive Coding
Updating the Huffman Tree
What Swapping Does
The Algorithm
An Enhancement
The Escape Code
The Overflow Problem
A Rescaling Bonus
The Code
Initialization of the Array
The Compress Main Program
The Expand Main Program
Encoding the Symbol
Updating the Tree
Decoding the Symbol
The Code
Chapter 5—Huffman One Better: Arithmetic Coding
Difficulties
Arithmetic Coding: A Step Forward
Practical Matters
A Complication
Decoding
Where’s the Beef?
The Code
The Compression Program
The Expansion Program
Initializing the Model
Reading the Model
Initializing the Encoder
The Encoding Process
Flushing the Encoder
The Decoding Process
Summary
Code
Chapter 6—Statistical Modeling
Higher-Order Modeling
Finite Context Modeling
Adaptive Modeling
A Simple Example
Using the Escape Code as a Fallback
Improvements
Highest-Order Modeling
Updating the Model
Escape Probabilities
Scoreboarding
Data Structures
The Finishing Touches: Tables –1 and –2
Model Flushing
Implementation
Conclusions
Enhancement
ARITH-N Listing
Chapter 7—Dictionary-Based Compression
An Example
Static vs. Adaptive
Adaptive Methods
A Representative Example
Israeli Roots
History
ARC: The Father of MS-DOS Dictionary Compression
Dictionary Compression: Where It Shows Up
Danger Ahead—Patents
Conclusion
Chapter 8—Sliding Window Compression
The Algorithm
Problems with LZ77
An Encoding Problem
LZSS Compression
Data Structures
A Balancing Act
Greedy vs. Best Possible
The Code
Constants and Macros
Global Variables
The Compression Code
Initialization
The Main Loop
The Exit Code
AddString()
DeleteString()
Binary Tree Support Routines
The Expansion Routine
Improvements
The Code
Chapter 9—LZ78 Compression
Can LZ77 Improve?
Enter LZ78
LZ78 Details
LZ78 Implementation
An Effective Variant
Decompression
The Catch
LZW Implementation
Tree Maintenance and Navigation
Compression
Decompression
The Code
Improvements
Patents
Chapter 10—Speech Compression
Digital Audio Concepts
Fundamentals
Sampling Variables
PC-Based Sound
Lossless Compression of Sound
Problems and Results
Lossy Compression
Silence Compression
Companding
Other Techniques
Chapter 11—Lossy Graphics Compression
Enter Compression
Statistical and Dictionary Compression Methods
Lossy Compression
Differential Modulation
Adaptive Coding
A Standard That Works: JPEG
JPEG Compression
The Discrete Cosine Transform
DCT Specifics
Why Bother?
Implementing the DCT
Matrix Multiplication
Continued Improvements
Output of the DCT
Quantization
Selecting a Quantization Matrix
Coding
The Zig-Zag Sequence
Entropy Encoding
What About Color?
The Sample Program
Input Format
The Code
Initialization
The Forward DCT Routine
WriteDCTData()
OutputCode()
File Expansion
ReadDCTData()
Input DCT Codes
The Inverse DCT
The Complete Code Listing
Support Programs
Some Compression Results
Chapter 12—An Archiving Package
CAR and CARMAN
The CARMAN Command Set
The CAR File
The Header
Storing the Header
The Header CRC
Command-Line Processing
Generating the File List
Opening the Archive Files
The Main Processing Loop
Skipping/Copying Input File
File Insertion
File Extraction
Cleanup
The Code
Chapter 13—Fractal Image Compression
A brief history of fractal image compression
What is an Iterated Function System?
Basic IFS mathematics
Image compression with Iterated Function Systems
Image compression with Partitioned Iterated Function Systems
Fractal image decoding
Resolution independence
The sample program
The main compression module
Initialization
Domain classification
Image partitioning
Finding optimal affine maps
The decompression module
The complete code listing
Some Compression Results
Patents
Bibliography
Appendix A
Appendix B
Glossary
Index


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly
Table of Contents

Glossary

Adaptive compression, Adaptive modeling
Data compression techniques that use a model can either use a fixed model for the entire stream they are processing, or modify the model as the stream is processed. Techniques that modify the model as it is processed are said to use adaptive modeling. An example of an adaptive compression technique would be LZW compression.
ADPCM
Adaptive Differential Pulse Code Modulation. Standard PCM encoding is a common technique for encoding audio data. Telephone conversations and audio CDs both use conventional PCM. PCM samples a waveform at uniform steps and encodes the level of the waveform. DPCM is Differential Pulse Code Modulation. DPCM doesn't encode the level, it instead encodes the difference from the last sample. ADPCM takes that a step further, and modifies the coding of the difference depending on the state of the waveform. PCM encoding in telephone systems uses 64 K bits per second. ADPCM can reduce that rate to 32 or even 16 K bits per second with relatively little reduction in voice quality.
Alphabet
An Alphabet is the set of all of the possible symbols that may appear in a message. For example, when compressing ASCII text files, the Alphabet would probably consist of characters 0x00 through 0x7f.
Archive
An archive is a volume or file containing one or more files that may or may not have been compressed. An archive is typically used as a convenient way to store or transport files. Programs such as ARC and PKZip compress files before placing them into archives.
Arithmetic coding
Traditional coding techniques such as ASCII or Huffman coding encode symbols into unique patterns of bits. Arithmetic coding instead takes an entire text and encodes it as a single floating point number less than 1 and greater than or equal to 0. Arithmetic coding can more efficiently encode texts by eliminating the quantization effects of other coding techniques.
ARC, MS-DOS program
ARC is a commercial archiving program created by System Enhancement Associates, of Wayne, N.J. ARC was one of the earliest compression/archive utilities to achieve wide popularity in the desktop computing world, beginning in the mid-1980s.
ARJ, MS-DOS program
ARJ is a commercial archiving program created by Robert Jung. ARJ is free of charge for individual use, but commercial users must pay a license fee. ARJ is also supplied with ANSI C source code for extracting files from ARJ archives that may be distributed without restrictions.
Block Coding
Compression of images is frequently done by coding smaller blocks of the image independent of one another. For example, the JPEG algorithm uses an 8-by-8 block size when compressing graphics.
CCITT
CCITT is the International Telegraph and Telephone Consultative Committee. This standards organization is responsible for the sanctioning of many compression and transmission methods in use today, including several PCM and ADPCM techniques, FAX transmission, and the evolving JPEG and MPEG standards.
Codes, (en)coding
Symbols that are to be stored or manipulated by a computer are converted to codes. This process is referred to as coding. ASCII and EBCDIC are two of the most common methods of coding written text. Data compression can occur if more efficient methods of coding, such as Huffman coding, are used.
COMPACT, UNIX program
COMPACT is a UNIX program that used Dynamic Huffman coding to compress files. It generally fell out of use in favor of the COMPRESS program.
COMPRESS, UNIX program
COMPRESS is a UNIX program that uses an LZW implementation to compress files. COMPRESS has found widespread use in the UNIX community, and is available in the public domain. It has recently been thought that COMPRESS may infringe on a Unisys patent, which may curtail its use and distribution.
Compression ratios
Compression ratios are used to describe the difference between a file and a compressed copy of itself. There are several different ways of expressing this number. One common method is a ratio between input and output, as in "a 4:1 compression ratio." Another popular method is to express the difference between the files as a percentage ranging from 0% to 100% (or greater, if the compression failed to actually reduce the size of the file). Some people invert this scale, using 100% as the "best" compression ratio. Occasionally, you still see the ratio of compressed to plain files expressed as "bits per byte."
CRC, Cyclical Redundancy Check
A CRC is a number generated by applying a formula to a block of data, generally for use as a checksum. A good CRC formula should generate a different number for as many different error conditions as possible. The CRC formula referred to in this book is the commonly used 32-bit CCITT-specified formula.
Discrete Cosine Transform
The DCT is used in the JPEG image compression method. The DCT is similar to the Fast Fourier Transform, in that it transforms a set of data from the the spatial domain to the frequency domain and back again. Once a photographic image is transformed by the DCT into a set of frequency information, it can be effectively compressed using "lossy" techniques. Expanding the same image involves converting the frequency information back to spatial information.
Dictionary, adaptive/static
Macro substitution methods use a dictionary to compress data. A string of symbols is encoded as a pointer into a dictionary. An adaptive method, such as LZ77, is continually modifying its dictionary. A static dictionary will compress an entire stream using the same dictionary.
Entropy, Information content
Entropy is a measure of the amount of information in an object. The concept of "absolute entropy" remains elusive. In general, entropy is calculated with respect to a given model. Entropy can be expressed in bits. In this form it is generally referred to as "information content."
Escape code
An Escape code is a special symbol used to 'escape' from the current context. In data compression, escape codes are frequently used when a symbol, not found in the current dictionary of symbols, needs to be encoded. The Escape code tells the decoder to change to a different context, where the symbol can be properly coded.
Freeware
Freeware is a term applied to software that is distributed without charge and may be used freely by anyone. It is distinguished from Public Domain software by virtue of the fact that the owner of the software retains the copyright to the work. Retaining the copyright allows the owner to restrain or control any modification or redistribution of the package. Software distributed by the Free Software Foundation, such as the EMACS editor, is generally referred to as freeware.
Group 3 FAX
Group 3 FAX is a CCITT standard for transmission of facsimile data. It can compress black and white images using a combination of differential, run length, and Huffman coding.
Huffman coding
Huffman is a method of encoding symbols that varies the length of the symbol in proportion to its information content. Symbols with a low probability of appearance are encoded with a code using may bits. Symbols with a high probability of appearance are represented with a code using fewer bits. Huffman codes can be properly decoded because they obey the prefix property, which means no code can be a prefix of another code. Huffman coding was first described in a seminal paper by D.A. Huffman in 1952.
Information theory
Information theory is the study of the storage, processing, and transmission of information. This branch of science is generally acknowledged as having been created by Claude Shannon at Bell Labs shortly after World War II.
ISO
ISO is the International Standards Organization. ISO is one of the bodies (along with the CCITT) involved in the JPEG and MPEG standardization efforts.
JPEG
JPEG stands for the Joint Photographic Experts Group. It is referred as a "joint" group because this committee is sanctioned by the CCITT and the ISO, two prominent international standards groups. JPEG refers both to the committee and their work—a compression standard that defines a method for compressing photographic images. Images compressed with the JPEG algorithm undergo a "lossy" compression. The amount of compression can be varied, with a resulting loss or gain in resolution. Most of the JPEG-like algorithms in use today rely on powerful dedicated processors to perform compression and decompression. Software-only techniques on general purpose desktop processors are still fairly slow. JPEG compression can achieve impressive compression ratios, reducing the storage required by images to less than 10% of the size of the original with only very slight loss of resolution. By sacrificing more resolution, you can compress images to 95% or more using JPEG.
LHarc
LHarc is a freeware compression program authored by Haruyasu Yoshizaki. It uses the LZSS variant of LZ77 compression, followed by a Dynamic Huffman postprocessing stage. The freeware status of this program, combined with the availability of source code, have made this a popular program.
Linear Predictive Coding
LPC is a coding technique that transmits voice data using a model of the vocal tract.
Lossless
Lossless compression is used to compress a text stream so that it can be expanded into an identical copy of the stream. This type of compression is normally required for data files.
Lossy
Lossy compression refers to a compression technique where the compressed stream cannot be expanded into an exact copy of the input. Lossy compression can be used on digitally stored representations of analog phenomena, such as graphics images and stored audio samples. The ability to sacrifice small amounts of resolution allows lossy algorithms to compress files to significantly smaller ratios. Lossy compression is sometimes referred to as "noisy" compression.
LZW, LZSS, LZ-77, LZ-78
Jacob Ziv and Abraham Lempel published a pair of papers in 1977 and 1978 that described two different dictionary-based compression techniques. LZ77 substituted strings from a fixed-size window into previously seen text. LZ78 builds up a phrase dictionary from previously seen text, with no limit on how far back a phrase may have appeared. These papers spurred a flurry of activity by other researchers who refined these techniques, resulting in compression algorithms that were superior to earlier statistical-based Huffman coding. Dictionary methods are widely-used today in V.42bis modems, in software such as LHarc, ARJ, and PKZIP, and in QIC magnetic tape drives.
Model
Compression algorithms generally maintain a "model," which is a set of accumulated statistics describing the state of the encoder. For example, in a simple compression program, the model may be a count of the frequency of every symbol in an input file.
MPEG
MPEG stands for Moving Pictures Experts Group. Like JPEG, MPEG refers to both a group and the standard that the group is developing. MPEG is a committee sanctioned by the ISO to work on the digital transmission of broadcast quality full motion video and sound. The goal of the MPEG group is to be able to send a high-quality picture and stereo soundtrack through a 1.5 Mbps channel. The MPEG has drafted a standard which is presently under review.
Nyquist Rate
The rate that analog signal of frequency f has to be sampled at in order to be accurately reproduced. Harry Nyquist determined that this rate has to be greater than 2*f.
Order (in re. model)
The order of a model refers to how many previous symbols are taken into consideration when encoding. For example, an order-0 model ignores all previous symbols when determining what code to use for a given symbol. So even if the previous character in a file was 'q', the probability of a 'u' in an order-0 model will not go up. An order-1 model would take note of the 'q' and greatly increase the probability for the 'u'.
Phrase
A phrase is a string of symbols of an arbitrary length. When programming in C, the term "string" can usually be substituted for phrase.
PKZIP
PKZIP is a popular desktop compression/archiving program. PKZIP is distributed via shareware channels by PKWare, Glendale, WI. This program uses several different dictionary-based compression algorithms to compress input files. It has achieved enough popularity in the MS-DOS world to be accepted as a "standard," although the source code for the program remains proprietary.
p X 64
p x 64 refers to the CCITT standard regarding digital transmission of audiovisual information, commonly referred to as videoconferencing. ISDN communications networks allocate bandwidth in 64K bit/second "chunks." The 'p' in p X 64 refers the notion that a video transmission channel will be allocated a certain number of 64 Kbps channels. The quality of transmission will be affected by the magnitude of p. Videoconferencing can be done with p as low as 1 or 2, but transmission of motion is severely restricted. As p grows, higher quality transmission becomes possible.
Run Length Encoding
Run Length Encoding, or RLE, is a simple technique used to compress runs of identical symbols in a data stream. Typically RLE encodes a run of symbols as a symbol and a count. The simplicity of RLE and ease of implementation result in its frequent appearance in compression programs, although its performance is relatively poor.
Shannon Fano coding
Shannon Fano codings is a coding technique developed in the early 1950s which attempted to minimize the number of bits used in a message when the probabilities of symbols in the message were known. Shannon Fano coding has generally been superseded by Huffman coding, which produces provably optimum code sets, resulting in marginally better performance than Shannon Fano codes.
Shannon, Claude
Claude Shannon is known as the father of information theory for his work done in the 1940s and 1950s. Shannon defined the concept of "information content" and entropy as it relates to data. Shannon is also an ardent student of the art of juggling, both by man and machine.
Shareware
Shareware refers to an increasingly common method of software distribution. Shareware is based on a "try before you buy" concept. The user of a shareware program is authorized by the creator to try the program for a limited amount of time. After the evaluation period is over, the user is expected to "register" the program if it will continue being used. Registration generally consists of a payment in return for improved documentation, support, and other considerations.
While shareware is a decidedly ad hoc concept, some standardization in the industry is being attempted by the Association for Shareware Professionals. The ASP has established a set of guidelines for authors of shareware defining various standards for distribution and support.
While not part of the shareware definition, most shareware programs also have a significantly lower price than their commercial counterparts. Many consumers mistakenly feel that shareware is "free," an idea being battled by the ASP.
lSQ, MS-DOS compression program
One of the earliest desktop compression programs was SQ, and its counterpart USQ. These two programs implemented a simple order 0 Huffman compression algorithm. They were first developed for the CP/M operating system, and were ported to MS-DOS soon after its release. SQ did not perform file archiving, so it was frequently combined with LU, a program that combined files into a library.
The appearance of ARC as a shareware program soon drove SQ into obscurity. ARC not only offered superior compression performance with its LZW algorithm, it also combined the compression and librarian functions into a single program. This let users compress groups of files and move them into a single archive in one operation.
lStatic model
A static model is one that does not change while a stream of symbols is compressed. An example of this would be a simple order-0 Huffman code compression program. The classic implementation of this program counts all of the different characters in a file during a first pass. This data is used to then build a Huffman coding tree, which constitutes the static model. A second pass is then made over the data to perform the actual compression.
Symbols
In data compression terminology, a symbol is an atomic unit of information. General purpose compression programs frequently compress streams of bytes, where the byte is the same thing as the symbol. However, a symbol could just as easily be a floating point number, or a spoken word, etc.
TAR, UNIX program
TAR is a standard UNIX program used to create archives. It takes a group of files and combines them into a single file or volume. TAR does not perform any compression of the files. UNIX archives are frequently created by using TAR to pack a group of files together, then using COMPRESS to perform the data compression. The resulting file is referred to as being in "TAR.Z" format.
Token
An arbitrary object used to encode something else. In dictionary-based compression, a token is an object that can be used to decode a phrase.
Welch, Terry
Terry Welch took the LZ78 data compression algorithm and refined it into a "Technique for High-Performance Data Compression." His patent on the LZW algorithm is now controlled by Unisys.
Ziv & Lempel
Jacob Ziv and Abraham Lempel are two Israeli information theorists who published the seminal papers on dictionary-based data compression in 1977 and 1978.

Table of Contents


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 1
Introduction to Data Compression

The primary purpose of this book is to explain various data-compression techniques using the C programming language. Data compression seeks to reduce the number of bits used to store or transmit information. It encompasses a wide variety of software and hardware compression techniques which can be so unlike one another that they have little in common except that they compress data. The LZW algorithm used in the Compuserve GIF specification, for example, has virtually nothing in common with the CCITT G.721 specification used to compress digitized voice over phone lines.

This book will not take a comprehensive look at every variety of data compression. The field has grown in the last 25 years to a point where this is simply not possible. What this book will cover are the various types of data compression commonly used on personal and midsized computers, including compression of binary programs, data, sound, and graphics.

Furthermore, this book will either ignore or only lightly cover data-compression techniques that rely on hardware for practical use or that require hardware applications. Many of today's voice-compression schemes were designed for the worldwide fixed-bandwidth digital telecommunications networks. These compression schemes are intellectually interesting, but they require a specific type of hardware tuned to the fixed bandwidth of the communications channel. Different algorithms that don't have to meet this requirement are used to compress digitized voice on a PC, and these algorithms generally offer better performance.

Some of the most interesting areas in data compression today, however, do concern compression techniques just becoming possible with new and more powerful hardware. Lossy image compression, like that used in multimedia systems, for example, can now be implemented on standard desktop platforms. This book will cover practical ways to both experiment with and implement some of the algorithms used in these techniques.

The Audience

You will need basic programming skills to adequately discuss data-compression code. The ability to follow block-structured code, such as C or Pascal, is a requirement. In addition, understanding computer architecture well enough to follow bit-oriented operations, such as shifting, logical ORing and ANDing, and so on, will be essential.

This does not mean that you need to be a C guru for this book to be worthwhile. You don't even have to be a programmer. But the ability to follow code will be essential, because the concepts discussed here will be illustrated with portable C programs. The C code in this book has been written with an eye toward simplicity in the hopes that C novices will still be able to follow the programs. We will avoid the more esoteric constructs of C, but the code will be working tested C—no pseudocode or English.

Why C?

The use of C to illustrate data-compression algorithms may raise some hackles, although less so these days than when the first edition of this book came out. A more traditional way to write this book would have been to use pseudocode to sketch out the algorithms. But the lack of rigor in a pseudocode "program" often leads to hazy or incomplete definitions full of lines like "PROCESS FILE UNTIL OUT OF DATA." The result is that pseudocode is easy to read, but not so easy to translate into a working program.

If pseudocode is unsatisfactory, the next best choice is to use a conventional programming language. Though hundreds of choices are available, C seems the best choice for this type of book for several good reasons. First, in many respects C has become the lingua franca of programmers. That C compilers support computers ranging from a lowly 8051 microcontroller to supercomputers capable of 100 million instructions per second (MIPS) has had much to do with this. It doesn't mean that C is the language of choice for all programmers. What it does mean is that most programmers should have a C compiler available for their machines, and most are probably regularly exposed to C code. Because of this, many programmers who use other languages can still manage to code in C, and even more can at least read C.

A second reason for using C is that it is a language without too many surprises. The few constructs it uses as basic language elements are easily translated to other languages. So a data-compression program that is illustrated using C can be converted to a working Pascal program through a relatively straightforward translation procedure. Even assembly-language programmers should find the process relatively painless.

Perhaps the most important reason for using C is simply one of efficiency. C is often thought of as a high-level assembly language, since it allows programmers to get close to the hardware. Despite the increasing optimization found in recent C compilers, it is not likely that C will ever exceed the speed or size possible in hand-coded assembly language. That flaw is offset, however, by the ability to easily port C code to other machines. So for a book of this type, C is probably the most efficient choice.

Which C?

Despite being advertised as a "portable" language, a C program that compiles and executes on a given machine is not guaranteed to run on any other. It may not even compile using a different compiler on the same machine. The important thing to remember is not that C is portable, but that it can be portable. The code for this book has been written to be portable, and it compiles and runs cleanly using several compilers and environments. The compilers/environments used here include:

• Microsoft Visual C++ 1.5, MS-DOS 5.0/6.22
• Borland C++ 4.0-4.5, MS-DOS 5.0/6.22
• Symantec C++ 6.0-7.0, MS-DOS 5.0/6.22
• Interactive Unix System 3.2 with the portable C compiler
• Solaris 2.4 with SunSoft compiler
• Linux 1.1 with the GNU C compiler


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Issues in Writing Portable C

One important portability issue is library function calls. Though the C programming language was fairly well defined by the original K&R book (Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language [Englewood Cliffs, NJ.: Prentice-Hall, 1978]), the run-time library implementation was left totally up to the whims of the implementor. Fortunately, the American National Standards Institute was able to complete the C language specification in 1990, and the result was published as ANSI standard XJ11.34. This standard not only expanded and pinned down the original K&R language specification, but it also took on the definition of a standard C run-time library. This makes it much easier to write code that works the same way from machine to machine. The code in this book will be written with the intention of using only ANSI C library calls. Compiler-dependent extensions to either the language or the library will be avoided wherever possible.

Given the standardization of the libraries, the remaining portability issues center around two things: sizes of the basic data types and dealing with noncompliant compilers. The majority of data-type conflicts arise when switching between 16- and 32-bit machines.

Fortunately, it is fairly easy to manage the change between 16- and 32-bit machines. Though the basic integer data type switches between 16- and 32-bits, both machines have a 16-bit "short int" data type. Once again, a "long int" is generally 32 bits on both machines. So in cases where the size of an integer clearly matters, it can be pinned down to either 16-or 32-bits with the appropriate declaration.

On the vast majority of machines used in the world today, the C compiler implementation of the "char" data type is 8 bits wide. In this book, we will gloss over the possibility that any other size exists and stick with 8-bit characters. In general, porting a program shown here to a machine with an unusual char size is not too difficult, but spending too much time on it will obscure the important point of the programs here, which is data compression.

The final issue to deal with when writing portable code is the problem of noncompliant compilers. In the MS-DOS world, most C compilers undergo major releases and upgrades every two years or so. This means that most compiler vendors have been able to release new versions of their compilers that now conform closely to the ANSI C standard. But this is not the case for users of many other operating systems. In particular, UNIX users will frequently be using a C compiler which came with their system and which conforms to the older K&R language definition. While the ANSI C committee went to great lengths to make ANSI C upwardly compatible from K&R C, we need to watch out for a few problems.

The first problem lies in the use of function prototypes. Under K&R C, function prototypes were generally used only when necessary. The compiler assumed that any unseen function returned an integer, and it accepted this without complaint. If a function returned something unusual—a pointer or a long, for instance—the programmer would write a function prototype to inform the compiler.

long locate_string();

Here, the prototype told the compiler to generate code that assumes that the function returned a long instead of an int. Function prototypes didn't have much more use than that. Because of this, many C programmers working under a K&R regime made little or no use of function prototypes, and their appearance in a program was something of an oddity.

While the ANSI C committee tried not to alter the basic nature of C, they were unable to pass up the potential improvements to the language that were possible through the expansion of the prototyping facility. Under ANSI C, a function prototype defines not only the return type of a function, but also the type of all the arguments as well. The function shown earlier, for example, might have the following prototype with an ANSI C compiler:

long locate_string( FILE *input_file, char *string );


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

This lets the compiler generate the correct code for the return type and check for the correct type and number of arguments as well. Since passing the wrong type or number of arguments to a function is a major source of programmer error in C, the committee correctly assumed that allowing this form of type checking constituted a step forward for C.

Under many ANSI C compilers, use of full ANSI function prototypes is strongly encouraged. In fact, many compilers will generate warning messages when a function is used without previously encountering a prototype. This is well and good, but the same function prototypes will not work on a trusty portable C compiler under UNIX.

The solution to this dilemma is not pretty, but it works. Under ANSI C, the predefined macro ___STDC___ is always defined to indicate that the code is being compiled through a presumably ANSI-compliant compiler. We can let the preprocessor turn certain sections of our header files on or off, depending on whether we are using a noncompliant compiler or not. A header file containing the prototypes for a bit-oriented package, for example, might look something like this:

#ifdef ___STDC___

FILE *open_bitstream( char *file_name, char *mode );

void close_bitstream( FILE *bitstream );

int read_bit( FILE*bitstream );

int write_bit( FILE *bitstream, int bit );

#else

FILE *open_bitstream();

void close_bitstream();

int read_bit();

int write_bit();

#endif

The preprocessor directives don't contribute much to the look of the code, but they are a necessary part of writing portable programs. Since the programs in this book are supposed to be compiled with the compiler set to its maximum possible warning level, a few "#ifdef" statements will be part of the package.

A second problem with the K&R family of C compilers lies in the actual function body. Under K&R C, a particular function might have a definition like the one below.

int foo( c )

char c;

{

/* Function body */

}

The same function written using an ANSI C function body would look like this:

int foo( char c )

{

/* Function body */

}

These two functions may look the same, but ANSI C rules require that they be treated differently. The K&R function body will have the compiler "promote" the character argument to an integer before using it in the function body, but the ANSI C function body will leave it as a character. Promoting one integral type to another lets lots of sneaky problems slip into seemingly well-written code, and the stricter compilers will issue warnings when they detect a problem of this nature.

Since K&R compilers will not accept the second form of a function body, be careful when defining character arguments to functions. Unfortunately, the solutions are once again either to not use character arguments or to resort to more of the ugly "#ifdef" preprocessor baggage.

Keeping Score

Throughout this book, there will be references to "compression ratios" and compression statistics. To keep the various forms of compression on a level playing field, compression statistics will always be in relationship to the sample compression files used in the February 1991 Dr. Dobb's Journal compression contest. These files consist of about 6 megabytes of data broken down into three roughly equal categories. The first category is text, consisting of manuscripts, programs, memos, and other readable files. The second category consists of binary data, including database files, executable files, and spreadsheet data. The third category consists of graphics files stored in raw screen-dump formats.

The programs created and discussed in this book will be judged by three rough measures of performance. The first will be the amount of memory consumed by the program during compression; this number will be approximated as well as it can be. The second will be the amount of time the program takes to compress the entire Dr. Dobb's dataset. The third will be the compression ratio of the entire set.

Different people use different formulas to calculate compression ratios. Some prefer bits/bytes. Other use ratios, such as 2:1 or 3:1 (advertising people seem to like this format). In this book, we will use a simple compression-percentage formula:

( 1 - ( compressed_size / raw_size ) ) * 100

This means that a file that doesn't change at all when compressed will have a compression ratio of 0 percent. A file compressed down to one-third of its original size will have a compression ratio of 67 percent. A file that shrinks down to 0 bytes (!) will have a compression ratio of 100 percent.

This way of measuring compression may not be perfect, but it shows perfection at 100 percent and total failure at 0 percent. In fact, a file that goes through a compression program and comes out larger will show a negative compression ratio.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Structure

This book consists of thirteen chapters and a floppy disk. The organization roughly parallels the historical progression of data compression, starting in the "dawn age" around 1950 and working up to the present.

Chapter 2 is a reference chapter which attempts to establish the fundamental data-compression lexicon. It discusses the birth of information theory, and it introduces a series of concepts, terms, buzzwords, and theories used over and over in the rest of the book. Even if you are a data-compression novice, mastery of chapter 2 will bring you up to the "cocktail party" level of information, meaning that you will be able to carry on an intelligent-sounding conversation about data compression even if you don't fully understand its intricacies.

Chapter 3 discusses the birth of data compression, starting with variable-length bit coding. The development of Shannon-Fano coding and Huffman coding represented the birth of both data compression and information theory. These coding methods are still in wide use today. In addition, chapter 3 discusses the difference between modeling and coding—the two faces of the data-compression coin.

Standard Huffman coding suffers from a significant problem when used for high-performance data compression. The compression program has to pass a complete copy of the Huffman coding statistics to the expansion program. As the compression program collects more statistics and tries to increase its compression ratio, the statistics take up more space and work against the increased compression. Chapter 4 discusses a way to solve this dilemma: adaptive Huffman coding. This is a relatively recent innovation, due to CPU and memory requirements. Adaptive coding greatly expands the horizons of Huffman coding, leading to vastly improved compression ratios.

Huffman coding has to use an integral number of bits for each code, which is usually slightly less than optimal. A more recent innovation, arithmetic coding, uses a fractional number of bits per code, allowing it to incrementally improve compression performance. Chapter 5 explains how this recent innovation works, and it shows how to integrate an arithmetic coder with a statistical model.

Chapter 6 discusses statistical modeling. Whether using Huffman coding, adaptive Huffman coding, or arithmetic coding, it is still necessary to have a statistical model to drive the coder. This chapter shows some of the interesting techniques used to implement powerful models using limited memory resources.

Dictionary compression methods take a completely different approach to compression from the techniques discussed in the previous four chapters. Chapter 7 provides an overview of these compression methods, which represent strings of characters with single codes. Dictionary methods have become the de facto standard for general-purpose data compression on small computers due to their high-performance compression combined with reasonable memory requirements.

The fathers of dictionary-based compression, Ziv and Lempel published a paper in 1977 proposing a sliding dictionary methods of data compression which has become very popular. Chapter 8 looks at recent adaptations of LZ77 compression used in popular archiving programs such as PKZIP.

Chapter 9 takes detailed look at one of the first widely popular dictionary-based compression methods: LZW compression. LZW is the compression method used in the UNIX COMPRESS program and in earlier versions of the MS-DOS ARC program. This chapter also takes a look at the foundation of LZW compression, published in 1978 by Ziv and Lempel.

All of the compression techniques discussed through chapter 9 are "lossless." Lossy methods can be used on speech and graphics, and they are capable of achieving dramatically higher compression ratios. Chapter 10 shows how lossy compression can be used on digitized sound data which techniques like linear predictive coding and adaptive PCM.

Chapter 11 discusses lossy compression techniques applied to computer graphics. The industry is standardizing rapidly on the JPEG standard for compressing graphical images. The techniques used in the JPEG standard will be presented in this chapter.

Chapter 12 describes how to put it all together into an archive program. A general-purpose archiving program should be able to compress and decompress files while keeping track of files names, dates, attributes, compression ratios, and compression methods. An archive format should ideally be portable to different types of computers. A sample archive program is developed, which applies the techniques used in previous chapters to put together a complete program.

Chapter 13 is a detailed look at fractal compression techniques. The world of fractal compression offers some exciting methods of achieving maximum compression for your data.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 2
The Data-Compression Lexicon, with a History

Like any other scientific or engineering discipline, data compression has a vocabulary that at first seem overwhelmingly strange to an outsider. Terms like Lempel-Ziv compression, arithmetic coding, and statistical modeling get tossed around with reckless abandon.

While the list of buzzwords is long enough to merit a glossary, mastering them is not as daunting a project as it may first seem. With a bit of study and a few notes, any programmer should hold his or her own at a cocktail-party argument over data-compression techniques.

The Two Kingdoms

Data-compression techniques can be divided into two major families; lossy and lossless. Lossy data compression concedes a certain loss of accuracy in exchange for greatly increased compression. Lossy compression proves effective when applied to graphics images and digitized voice. By their very nature, these digitized representations of analog phenomena are not perfect to begin with, so the idea of output and input not matching exactly is a little more acceptable. Most lossy compression techniques can be adjusted to different quality levels, gaining higher accuracy in exchange for less effective compression. Until recently, lossy compression has been primarily implemented using dedicated hardware. In the past few years, powerful lossy-compression programs have been moved to desktop CPUs, but even so the field is still dominated by hardware implementations.

Lossless compression consists of those techniques guaranteed to generate an exact duplicate of the input data stream after a compress/expand cycle. This is the type of compression used when storing database records, spreadsheets, or word processing files. In these applications, the loss of even a single bit could be catastrophic. Most techniques discussed in this book will be lossless.

Data Compression = Modeling + Coding

In general, data compression consists of taking a stream of symbols and transforming them into codes. If the compression is effective, the resulting stream of codes will be smaller than the original symbols. The decision to output a certain code for a certain symbol or set of symbols is based on a model. The model is simply a collection of data and rules used to process input symbols and determine which code(s) to output. A program uses the model to accurately define the probabilities for each symbol and the coder to produce an appropriate code based on those probabilities.

Modeling and coding are two distinctly different things. People frequently use the term coding to refer to the entire data-compression process instead of just a single component of that process. You will hear the phrases "Huffman coding" or "Run-Length Encoding," for example, to describe a data-compression technique, when in fact they are just coding methods used in conjunction with a model to compress data.

Using the example of Huffman coding, a breakdown of the compression process looks something like this:

Figure 2.1 A Statistical Model with a Huffman Encoder.

In the case of Huffman coding, the actual output of the encoder is determined by a set of probabilities. When using this type of coding, a symbol that has a very high probability of occurrence generates a code with very few bits. A symbol with a low probability generates a code with a larger number of bits.

We think of the model and the program's coding process as different because of the countless ways to model data, all of which can use the same coding process to produce their output. A simple program using Huffman coding, for example, would use a model that gave the raw probability of each symbol occurring anywhere in the input stream. A more sophisticated program might calculate the probability based on the last 10 symbols in the input stream. Even though both programs use Huffman coding to produce their output, their compression ratios would probably be radically different.

So when the topic of coding methods comes up at your next cocktail party, be alert for statements like "Huffman coding in general doesn't produce very good compression ratios." This would be your perfect opportunity to respond with "That's like saying Converse sneakers don't go very fast. I always thought the leg power of the runner had a lot to do with it." If the conversation has already dropped to the point where you are discussing data compression, this might even go over as a real demonstration of wit.

The Dawn Age

Data compression is perhaps the fundamental expression of Information Theory. Information Theory is a branch of mathematics that had its genesis in the late 1940s with the work of Claude Shannon at Bell Labs. It concerns itself with various questions about information, including different ways of storing and communicating messages.

Data compression enters into the field of Information Theory because of its concern with redundancy. Redundant information in a message takes extra bit to encode, and if we can get rid of that extra information, we will have reduced the size of the message.

Information Theory uses the term entropy as a measure of how much information is encoded in a message. The word entropy was borrowed from thermodynamics, and it has a similar meaning. The higher the entropy of a message, the more information it contains. The entropy of a symbol is defined as the negative logarithm of its probability. To determine the information content of a message in bits, we express the entropy using the base 2 logarithm:

Number of bits = - Log base 2 (probability)

The entropy of an entire message is simply the sum of the entropy of all individual symbols.

Entropy fits with data compression in its determination of how many bits of information are actually present in a message. If the probability of the character 'e' appearing in this manuscript is 1/16, for example, the information content of the character is four bits. So the character string "eeeee" has a total content of 20 bits. If we are using standard 8-bit ASCII characters to encode this message, we are actually using 40 bits. The difference between the 20 bits of entropy and the 40 bits used to encode the message is where the potential for data compression arises.

One important fact to note about entropy is that, unlike the thermodynamic measure of entropy, we can use no absolute number for the information content of a given message. The problem is that when we calculate entropy, we use a number that gives us the probability of a given symbol. The probability figure we use is actually the probability for a given model, not an absolute number. If we change the model, the probability will change with it.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

How probabilities change can be seen clearly when using different orders with a statistical model. A statistical model tracks the probability of a symbol based on what symbols appeared previously in the input stream. The order of the model determines how many previous symbols are taken into account. An order-0 model, for example, won't look at previous characters. An order-1 model looks at the one previous character, and so on.

The different order models can yield drastically different probabilities for a character. The letter 'u' under an order-0 model, for example, may have only a 1 percent probability of occurrence. But under an order-1 model, if the previous character was 'q,' the 'u' may have a 95 percent probability.

This seemingly unstable notion of a character's probability proves troublesome for many people. They prefer that a character have a fixed "true" probability that told what the chances of its "really" occurring are. Claude Shannon attempted to determine the true information content of the English language with a "party game" experiment. He would uncover a message concealed from his audience a single character at a time. The audience guessed what the next character would be, one guess at a time, until they got it right. Shannon could then determine the entropy of the message as a whole by taking the logarithm of the guess count. Other researchers have done more experiments using similar techniques.

While these experiments are useful, they don't circumvent the notion that a symbol's probability depends on the model. The difference with these experiments is that the model is the one kept inside the human brain. This may be one of the best models available, but it is still a model, not an absolute truth.

In order to compress data well, we need to select models that predict symbols with high probabilities. A symbol that has a high probability has a low information content and will need fewer bits to encode. Once the model is producing high probabilities, the next step is to encode the symbols using an appropriate number of bits.

Coding

Once Information Theory had advanced to where the number of bits of information in a symbol could be determined, the next step was to develop new methods for encoding information. To compress data, we need to encode symbols with exactly the number of bits of information the symbol contains. If the character 'e' only gives us four bits of information, then it should be coded with exactly four bits. If 'x' contains twelve bits, it should be coded with twelve bits.

By encoding characters using EBCDIC or ASCII, we clearly aren't going to be very close to an optimum method. Since every character is encoded using the same number of bits, we introduce lots of error in both directions, with most of the codes in a message being too long and some being too short.

Solving this coding problem in a reasonable manner was one of the first problems tackled by practitioners of Information Theory. Two approaches that worked well were Shannon-Fano coding and Huffman coding—two different ways of generating variable-length codes when given a probability table for a given set of symbols.

Huffman coding, named for its inventor D.A. Huffman, achieves the minimum amount of redundancy possible in a fixed set of variable-length codes. This doesn't mean that Huffman coding is an optimal coding method. It means that it provides the best approximation for coding symbols when using fixed-width codes.

The problem with Huffman or Shannon-Fano coding is that they use an integral number of bits in each code. If the entropy of a given character is 2.5 bits, the Huffman code for that character must be either 2 or 3 bits, not 2.5. Because of this, Huffman coding can't be considered an optimal coding method, but it is the best approximation that uses fixed codes with an integral number of bits. Here is a sample of Huffman codes:

Symbol | Huffman Code
E | 100
T | 101
A | 1100
I | 11010
… |
X | 01101111
Q | 01101110001
Z | 01101110000

An Improvement

Though Huffman coding is inefficient due to using an integral number of bits per code, it is relatively easy to implement and very economical for both coding and decoding. Huffman first published his paper on coding in 1952, and it instantly became the most-cited paper in Information Theory. It probably still is. Huffman's original work spawned numerous minor variations, and it dominated the coding world till the early 1980s.

As the cost of CPU cycles went down, new possibilities for more efficient coding techniques emerged. One in particular, arithmetic coding, is a viable successor to Huffman coding.

Arithmetic coding is somewhat more complicated in both concept and implementation than standard variable-width codes. It does not produce a single code for each symbol. Instead, it produces a code for an entire message. Each symbol added to the message incrementally modifies the output code. This is an improvement because the net effect of each input symbol on the output code can be a fractional number of bits instead of an integral number. So if the entropy for character 'e' is 2.5 bits, it is possible to add exactly 2.5 bits to the output code.

An example of why this can be more effective is shown in the following table, the analysis of an imaginary message. In it, Huffman coding would yield a total message length of 89 bits, but arithmetic coding would approach the true information content of the message, or 83.56 bits. The difference in the two messages works out to approximately 6 percent. Here are some sample message probabilities:

Symbol | Number of Occurrences | Information Content | Huffman Code Bit Count | Total Bits Huffman Coding | Total Bits Arithmetic Coding
E | 20 | 1.26 bits | 1 bits | 20 | 25.2
A | 20 | 1.26 bits | 2 bits | 40 | 25.2
X | 3 | 4.00 bits | 3 bits | 9 | 12.0
Y | 3 | 4.00 bits | 4 bits | 12 | 12.0
Z | 2 | 4.58 bits | 4 bits | 8 | 9.16
 |  |  |  | 89 | 83.56


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The problem with Huffman coding in the above message is that it can't create codes with the exact information content required. In most cases it is a little above or a little below, leading to deviations from the optimum. But arithmetic coding gets to within a fraction of a percent of the actual information content, resulting in more accurate coding.

Arithmetic coding requires more CPU power than was available until recently. Even now it will generally suffer from a significant speed disadvantage when compared to older coding methods. But the gains from switching to this method are significant enough to ensure that arithmetic coding will be the coding method of choice when the cost of storing or sending information is high enough.

Modeling

If we use a an automotive metaphor for data compression, coding would be the wheels, but modeling would be the engine. Regardless of the efficiency of the coder, if it doesn't have a model feeding it good probabilities, it won't compress data.

Lossless data compression is generally implemented using one of two different types of modeling: statistical or dictionary-based. Statistical modeling reads in and encodes a single symbol at a time using the probability of that character's appearance. Dictionary-based modeling uses a single code to replace strings of symbols. In dictionary-based modeling, the coding problem is reduced in significance, leaving the model supremely important.

Statistical Modeling

The simplest forms of statistical modeling use a static table of probabilities. In the earliest days of information theory, the CPU cost of analyzing data and building a Huffman tree was considered significant, so it wasn't frequently performed. Instead, representative blocks of data were analyzed once, giving a table of character-frequency counts. Huffman encoding/decoding trees were then built and stored. Compression programs had access to this static model and would compress data using it.

But using a universal static model has limitations. If an input stream doesn't match well with the previously accumulated statistics, the compression ratio will be degraded—possibly to the point where the output stream becomes larger than the input stream. The next obvious enhancement is to build a statistics table for every unique input stream.

Building a static Huffman table for each file to be compressed has its advantages. The table is uniquely adapted to that particular file, so it should give better compression than a universal table. But there is additional overhead since the table (or the statistics used to build the table) has to be passed to the decoder ahead of the compressed code stream.

For an order-0 compression table, the actual statistics used to create the table may take up as little as 256 bytes—not a very large amount of overhead. But trying to achieve better compression through use of a higher order table will make the statistics that need to be passed to the decoder grow at an alarming rate. Just moving to an order 1 model can boost the statistics table from 256 to 65,536 bytes. Though compression ratios will undoubtedly improve when moving to order-1, the overhead of passing the statistics table will probably wipe out any gains.

For this reason, compression research in the last 10 years has concentrated on adaptive models. When using an adaptive model, data does not have to be scanned once before coding in order to generate statistics. Instead, the statistics are continually modified as new characters are read in and coded. The general flow of a program using an adaptive model looks something like that shown in Figures 2.2 and 2.3.

Figure 2.2 General Adaptive Compression.

Figure 2.3 General Adaptive Decompression.

The important point in making this system work is that the box labeled "Update Model" has to work exactly the same way for both the compression and decompression programs. After each character (or group of characters) is read in, it is encoded or decoded. Only after the encoding or decoding is complete can the model be updated to take into account the most recent symbol or group of symbols.

One problem with adaptive models is that they start knowing essentially nothing about the data. So when the program first starts, it doesn't do a very good job of compression. Most adaptive algorithms tend to adjust quickly to the data stream and will begin turning in respectable compression ratios after only a few thousand bytes. Likewise, it doesn't take long for the compression-ratio curve to flatten out so that reading in more data doesn't improve the compression ratio.

One advantage that adaptive models have over static models is the ability to adapt to local conditions. When compressing executable files, for example, the character of the input data may change drastically as the program file changes from binary program code to binary data. A well-written adaptive program will weight the most recent data higher than old data, so it will modify its statistics to better suit changed data.

Dictionary Schemes

Statistical models generally encode a single symbol at a time— reading it in, calculating a probability, then outputting a single code. A dictionary-based compression scheme uses a different concept. It reads in input data and looks for groups of symbols that appear in a dictionary. If a string match is found, a pointer or index into the dictionary can be output instead of the code for the symbol. The longer the match, the better the compression ratio.

This method of encoding changes the focus of dictionary compression. Simple coding methods are generally used, and the focus of the program is on the modeling. In LZW compression, for example, simple codes of uniform width are used for all substitutions.

A static dictionary is used like the list of references in an academic paper. Through the text of a paper, the author may simply substitute a number that points to a list of references instead of writing out the full title of a referenced work. The dictionary is static because it is built up and transmitted with the text of work—the reader does not have to build it on the fly. The first time I see a number in the text like this—[2]—I know it points to the static dictionary.

The problem with a static dictionary is identical to the problem the user of a statistical model faces: The dictionary needs to be transmitted along with the text, resulting in a certain amount of overhead added to the compressed text. An adaptive dictionary scheme helps avoid this problem.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Mentally, we are used to a type of adaptive dictionary when performing acronym replacements in technical literature. The standard way to use this adaptive dictionary is to spell out the acronym, then put its abbreviated substitution in parentheses. So the first time I mention the Massachusetts Institute of Technology (MIT), I define both the dictionary string and its substitution. From then on, referring to MIT in the text should automatically invoke a mental substitution.

Ziv and Lempel

Until 1980, most general-compression schemes used statistical modeling. But in 1977 and 1978, Jacob Ziv and Abraham Lempel described a pair of compression methods using an adaptive dictionary. These two algorithms sparked a flood of new techniques that used dictionary-based methods to achieve impressive new compression ratios.

LZ77

The first compression algorithm described by Ziv and Lempel is commonly referred to as LZ77. It is relatively simple. The dictionary consists of all the strings in a window into the previously read input stream. A file-compression program, for example, could use a 4K-byte window as a dictionary. While new groups of symbols are being read in, the algorithm looks for matches with strings found in the previous 4K bytes of data already read in. Any matches are encoded as pointers sent to the output stream.

LZ77 and its variants make attractive compression algorithms. Maintaining the model is simple; encoding the output is simple; and programs that work very quickly can be written using LZ77. Popular programs such as PKZIP and LHarc use variants of the LZ77 algorithm, and they have proven very popular.

LZ78

The LZ78 program takes a different approach to building and maintaining the dictionary. Instead of having a limited-size window into the preceding text, LZ78 builds its dictionary out of all of the previously seen symbols in the input text. But instead of having carte blanche access to all the symbol strings in the preceding text, a dictionary of strings is built a single character at a time. The first time the string "Mark" is seen, for example, the string "Ma" is added to the dictionary. The next time, "Mar" is added. If "Mark" is seen again, it is added to the dictionary.

This incremental procedure works very well at isolating frequently used strings and adding them to the table. Unlike LZ77 methods, strings in LZ78 can be extremely long, which allows for high-compression ratios. LZ78 was the first of the two Ziv-Lempel algorithms to achieve popular success, due to the LZW adaptation by Terry Welch, which forms the core of the UNIX compress program.

Lossy Compression

Until recently, lossy compression has been primarily performed on special-purpose hardware. The advent of inexpensive Digital Signal Processor (DSP) chips began lossy compression's move off the circuit board and onto the desktop. CPU prices have now dropped to where it is becoming practical to perform lossy compression on general-purpose desktop PCs.

Lossy compression is fundamentally different from lossless compression in one respect: it accepts a slight loss of data to facilitate compression. Lossy compression is generally done on analog data stored digitally, with the primary applications being graphics and sound files.

This type of compression frequently makes two passes. A first pass over the data performs a high-level, signal-processing function. This frequently consists of transforming the data into the frequency domain, using algorithms similar to the well-known Fast Fourier Transform (FFT). Once the data has been transformed, it is "smoothed," rounding off high and low points. Loss of signal occurs here. Finally, the frequency points are compressed using conventional lossless techniques.

The smoothing function that operates on the frequency-domain data generally has a "quality factor" built into it that determines just how much smoothing occurs. The more the data is massaged, the greater the signal loss—and more compression will occur.

In the small systems world, a tremendous amount of work is being done on graphical image compression, both for still and moving pictures. The International Standards Organization (ISO) and the Consultive Committee for International Telegraph and Telephone (CCITT) have banded together to form two committees: The Joint Photographic Experts Group (JPEG) and the Moving Pictures Expert Group (MPEG). The JPEG committee has published its compression standard, and many vendors are now shipping hardware and software that are JPEG compliant. The MPEG committee completed an intial moving picture compression standard, and is finalizing a second, MPEG-II.

The JPEG standard uses the Discrete Cosine Transform (DCT) algorithm to convert a graphics image to the frequency domain. The DCT algorithm has been used for graphics transforms for many years, so efficient implementations are readily available. JPEG specifies a quality factor of 0 to 100, and it lets the compressor determine what factor to select.

Using the JPEG algorithm on images can result in dramatic compression ratios. With little or no degradation, compression ratios of 90–95 percent are routine. Accepting minor degradation achieves ratios as high as 98–99 percent.

Software implementations of the JPEG and MPEG algorithms are still struggling to achieve real-time performance. Most multimedia development software that uses this type of compression still depends on the use of a coprocessor board to make the compression take place in a reasonable amount of time. We are probably only a few years away from software-only real-time compression capabilities.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Programs to Know

General-purpose data-compression programs have been available only for the past ten years or so. It wasn't until around 1980 that machines with the power to do the analysis needed for effective compression started to become commonplace.

In the Unix world, one of the first general-purpose compression programs was COMPACT. COMPACT is a relatively straightforward implementation of an order-0 compression program that uses adaptive Huffman coding. COMPACT produced good enough compression to make it useful, but it was slow. COMPACT was also a proprietary product, so it was not available to all Unix users.

Compress, a somewhat improved program, became available to Unix users a few years later. It is a straightforward implementation of the LZW dictionary-based compression scheme. compress gave significantly better compression than COMPACT, and it ran faster. Even better, the source code to a compress was readily available as a public-domain program, and it proved quite portable. compress is still in wide use among UNIX users, though its continued use is questionable due to the LZW patent held by Unisys.

In the early 1980s, desktop users of CP/M and MS-DOS systems were first exposed to data compression through the SQ program. SQ performed order-0 compression using a static Huffman tree passed in the file. SQ gave compression comparable to that of the COMPACT program, and it was widely used by early pioneers in desktop telecommunications.

As in the Unix world, Huffman coding soon gave way to LZW compression with the advent of ARC. ARC is a general-purpose program that performs both file compression and archiving, two features that often go hand in hand. (Unix users typically archive files first using TAR, then they compress the entire archive.) ARC could originally compress files using run-length encoding, order-0 static Huffman coding, or LZW compression. The original LZW code for ARC appears to be a derivative of the Unix compress code.

Due to the rapid distribution possible using shareware and telecommunications, ARC quickly became a de facto standard and began spawning imitators right and left. ARC underwent many revisions but has faded in popularity in recent years. Today, if there is a compression standard in the DOS world, it is the shareware program PKZIP, written by Phil Katz.

PKZIP is a relatively inexpensive program that offers both superior compression ratios and compression speed. At this writing, the current shareware version is PKZip V2.04g and can be found on many bulletin boards and online forums. Katz's company, PKWare, also sells a commercial version. Note that V2.04g of PKZIP can create ZIP files that are not backward compatible with previous versions. On Compuserve, many forums have switched to the new format for files kept in the forum libraries. Usually, a copy of the distribution PKZ204.EXE is also found in the forum library. For example, you can find this file on 23 different forums on Compuserve. Because Phil Katz has placed the file format in the public domain, there are many other archiving/compression utilities that support the ZIP format. A search on Compuserve, using the File Finder facility on the keyword "PKZIP" resulted in 580 files found, most of which were utilities rather than data files. Programs like WinZIP, that integrate with the Windows File Manager, provide a modern interface to a venerable file format.

In DOS, two strong alternatives to PKZIP are LHArc and ARJ. LHARC comes from Japan, and has several advantages over other archiving/compression programs. First, the source to LHArc is freely available and has been ported to numerous operating systems and hardware platforms. Second, the author of LHarc, Haruyasu Yoshizaki (Yoshi), has explicitly granted the right to use his program for any purpose, personal or commercial.

ARJ is a program written by Robert Jung (robjung@world.std.com) and is free for non-commercial use. It has managed to achieve compression ratios slightly better than the best LHArc can offer. It is available for DOS, Windows, Amiga, MAC, OS/2, and includes source code.

On the Macintosh platform, there are also many archiving/compression programs which support file formats found on DOS and Unix. In addition to LHArc and ARJ, there are programs like ZipIt V1.2 lets you work with ZIP files. However, the predominant archiving/compression program is StuffIt, a shareware program written by Raymond Lau. On bulletin boards and online services that are geared to Macintosh users, you will find more SIT files (StuffIt files) than any other format. Another popular Macintosh format is CPT (created by Compact-Pro program) but it is not as widespread as StuffIt.

In general, the trend is toward greater interoperability among platforms and formats. Jeff Gilchrist (jeffg@mi.net) distributes a monthly Archive Comparison Test (ACT) that compares sixty different DOS programs for speed and efficiency, working on a variety of files (text, binary executables, graphics). If you have Internet access, you can view the current copy of ACT by fingering: s0b8@jupiter.sun.csd.unb.ca. You can also view ACT using the World-Wide Web at http://www.mi.net/act/act.html. At this writing, one promising new archiver on Gilchrist's ACT list is X1, written by Stig Valentini (sv@id.dtu.dk). The current version is 0.90, still in beta stage. This program supports thirteen different archive formats, include: ZIP, LHA, ARJ, HA, PUT, TAR+GZIP(TGZ), and ZOO.

As mentioned earlier, you can find archive programs on Compuserve, America Online and other online services and bulletin boards. On the Internet, there are several ftp repositories. One is at oak.oakland.edu (in the directory /SimTel/msdos/archiver). Another is garbo.uwasa.fi, in the directory /pc/arcers.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 3
The Dawn Age: Minimum Redundancy Coding

In the late 1940s, the early years of Information Theory, the idea of developing efficient new coding techniques was just starting to be fleshed out. Researchers were exploring the ideas of entropy, information content, and redundancy. One popular notion held that if the probability of symbols in a message were known, there ought to be a way to code the symbols so that the message would take up less space.

Remarkably, this early work in data compression was being done before the advent of the modern digital computer. Today it seems natural that information theory goes hand in hand with computer programming, but just after World War II, for all practical purposes, there were no digital computers. So the idea of developing algorithms using base 2 arithmetic for coding symbols was really a great leap forward.

The first well-known method for effectively coding symbols is now known as Shannon-Fano coding. Claude Shannon at Bell Labs and R.M. Fano at MIT developed this method nearly simultaneously. It depended on simply knowing the probability of each symbol's appearance in a message. Given the probabilities, a table of codes could be constructed that has several important properties:

• Different codes have different numbers of bits.
• Codes for symbols with low probabilities have more bits, and codes for symbols with high probabilities have fewer bits.
• Though the codes are of different bit lengths, they can be uniquely decoded.

The first two properties go hand in hand. Developing codes that vary in length according to the probability of the symbol they are encoding makes data compression possible. And arranging the codes as a binary tree solves the problem of decoding these variable-length codes.

An example of the type of decoding tree used in Shannon-Fano coding is shown below. Decoding an incoming code consists of starting at the root, then turning left or right at each node after reading an incoming bit from the data stream. Eventually a leaf of the tree is reached, and the appropriate symbol is decoded.

Figure 3.1 is a Shannon-Fano tree designed to encode or decode a simple five-symbol alphabet consisting of the letters A through E. Walking through the tree yields the code table:

Symbol | Code
A | 00
B | 01
C | 10
D | 110
E | 111

Figure 3.1 A simple Shannon-Fano tree.

The tree structure shows how codes are uniquely defined though they have different numbers of bits. The tree structure seems designed for computer implementations, but it is also well suited for machines made of relays and switches, like the teletype machines of the 1950s.

While the table shows one of the three properties discussed earlier, that of having variable numbers of bits, more information is needed to talk about the other two properties. After all, code trees look interesting, but do they actually perform a valuable service?

The Shannon-Fano Algorithm

A Shannon-Fano tree is built according to a specification designed to define an effective code table. The actual algorithm is simple:

1. For a given list of symbols, develop a corresponding list of probabilities or frequency counts so that each symbol's relative frequency of occurrence is known.
2. Sort the lists of symbols according to frequency, with the most frequently occuring symbols at the top and the least common at the bottom.
3. Divide the list into two parts, with the total frequency counts of the upper half being as close to the total of the bottom half as possible.
4. The upper half of the list is assigned the binary digit 0, and the lower half is assigned the digit 1. This means that the codes for the symbols in the first half will all start with 0, and the codes in the second half will all start with 1.
5. Recursively apply the steps 3 and 4 to each of the two halves, subdividing groups and adding bits to the codes until each symbol has become a corresponding code leaf on the tree.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Shannon-Fano tree shown in Figure 3.1 was developed from the table of symbol frequencies shown next.

Symbol | Count
A | 15
B | 7
C | 6
D | 6
E | 5

Putting the dividing line between symbols B and C assigns a count of 22 to the upper group and 17 to the lower, the closest to exactly half. This means that A and B will each have a code that starts with a 0 bit, and C, D, and E are all going to start with a 1 as shown:

Symbol | Count |
A | 15 | 0
B | 7 | 0
 |  | First division
C | 6 | 1
D | 6 | 1
E | 5 | 1

Subsequently, the upper half of the table gets a new division between A and B, which puts A on a leaf with code 00 and B on a leaf with code 01. After four division procedures, a table of codes results. In the final table, the three symbols with the highest frequencies have all been assigned 2-bit codes, and two symbols with lower counts have 3-bit codes as shown next.

Symbol | Count |  |  |
A | 15 | 0 | 0 |
 |  |  | Second division
B | 7 | 0 | 1 |
 |  | First division |  |
C | 6 | 1 | 0 |
 |  |  | Third division
D | 6 | 1 | 1 | 0
 |  |  |  | Fourth division
E | 5 | 1 | 1 | 1

That symbols with the higher probability of occurence have fewer bits in their codes indicates we are on the right track. The formula for information content for a given symbol is the negative of the base two logarithm of the symbol's probability. For our theoretical message, the information content of each symbol, along with the total number of bits for that symbol in the message, are found in the following table.

Symbol | Count | Info Cont. | Info Bits
A | 15 | 1.38 | 20.68
B | 7 | 2.48 | 17.35
C | 6 | 2.70 | 16.20
D | 6 | 2.70 | 16.20
E | 5 | 2.96 | 14.82

The information for this message adds up to about 85.25 bits. If we code the characters using 8-bit ASCII characters, we would use 39 × 8 bits, or 312 bits. Obviously there is room for improvement.

When we encode the same data using Shannon-Fano codes, we come up with some pretty good numbers, as shown below.

Symbol | Count | Info Cont. | Info Bits | SF Size | SF Bits
A | 15 | 1.38 | 20.68 | 2 | 30
B | 7 | 2.48 | 17.35 | 2 | 14
C | 6 | 2.70 | 16.20 | 2 | 12
D | 6 | 2.70 | 16.20 | 3 | 18
E | 5 | 2.96 | 14.82 | 3 | 15

With the Shannon-Fano coding system, it takes only 89 bits to encode 85.25 bits of information. Clearly we have come a long way in our quest for efficient coding methods. And while Shannon-Fano coding was a great leap forward, it had the unfortunate luck to be quickly superseded by an even more efficient coding system: Huffman coding.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Huffman Algorithm

Huffman coding shares most characteristics of Shannon-Fano coding. It creates variable-length codes that are an integral number of bits. Symbols with higher probabilities get shorter codes. Huffman codes have the unique prefix attribute, which means they can be correctly decoded despite being variable length. Decoding a stream of Huffman codes is generally done by following a binary decoder tree.

Building the Huffman decoding tree is done using a completely different algorithm from that of the Shannon-Fano method. The Shannon-Fano tree is built from the top down, starting by assigning the most significant bits to each code and working down the tree until finished. Huffman codes are built from the bottom up, starting with the leaves of the tree and working progressively closer to the root.

The procedure for building the tree is simple and elegant. The individual symbols are laid out as a string of leaf nodes that are going to be connected by a binary tree. Each node has a weight, which is simply the frequency or probability of the symbol's appearance. The tree is then built with the following steps:

• The two free nodes with the lowest weights are located.
• A parent node for these two nodes is created. It is assigned a weight equal to the sum of the two child nodes.
• The parent node is added to the list of free nodes, and the two child nodes are removed from the list.
• One of the child nodes is designated as the path taken from the parent node when decoding a 0 bit. The other is arbitrarily set to the 1 bit.
• The previous steps are repeated until only one free node is left. This free node is designated the root of the tree.

This algorithm can be applied to the symbols used in the previous example. The five symbols in our message are laid out, along with their frequencies, as shown:

15 | 7 | 6 | 6 | 5
A | B | C | D | E

These five nodes are going to end up as the leaves of the decoding tree. When the process first starts, they make up the entire list of free nodes.

The first pass through the tree identifies the two free nodes with the lowest weights: D and E, with weights of 6 and 5. (The tie between C and D was broken arbitrarily. While the way that ties are broken affects the final value of the codes, it will not affect the compression ratio achieved.) These two nodes are joined to a parent node, which is assigned a weight of 11. Nodes D and E are then removed from the free list.

Once this step is complete, we know what the least significant bits in the codes for D and E are going to be. D is assigned to the 0 branch of the parent node, and E is assigned to the 1 branch. These two bits will be the LSBs of the resulting codes.

On the next pass through the list of free nodes, the B and C nodes are picked as the two with the lowest weight. These are then attached to a new parent node. The parent node is assigned a weight of 13, and B and C are removed from the free node list. At this point, the tree looks like that shown in Figure 3.2.

Figure 3.2 The Huffman tree after two passes.

On the next pass, the two nodes with the lowest weights are the parent nodes for the B/C and D/E pairs. These are tied together with a new parent node, which is assigned a weight of 24, and the children are removed from the free list. At this point, we have assigned two bits each to the Huffman codes for B, C, D, and E, and we have yet to assign a single bit to the code for A.

Finally, on the last pass, only two free nodes are left. The parent with a weight of 24 is tied with the A node to create a new parent with a weight of 39. After removing the two child nodes from the free list, we are left with just one parent, meaning the tree is complete. The final result looks like that shown in Figure 3.3.

Figure 3.3 The Huffman tree.

To determine the code for a given symbol, we have to walk from the leaf node to the root of the Huffman tree, accumulating new bits as we pass through each parent node. Unfortunately, the bits are returned to us in the reverse order that we want them, which means we have to push the bits onto a stack, then pop them off to generate the code. This strategy gives our message the code structure shown in the following table.

The Huffman Code Table
A | 0
B | 100
C | 101
D | 110
E | 111

As you can see, the codes have the unique prefix property. Since no code is a prefix to another code, Huffman codes can be unambiguously decoded as they arrive in a stream. The symbol with the highest probability, A, has been assigned the fewest bits, and the symbol with the lowest probability, E, has been assigned the most bits.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Note, however, that the Huffman codes differ in length from Shannon-Fano codes. The code length for A is only a single bit, instead of two, and the B and C symbols have 3-bit codes instead of two bits. The following table shows what effect this has on the total number of bits produced by the message.

Symbol | Count | Shannon-Fano Size | Shannon-Fano Bits | Huffman Size | Huffman Bits
A | 15 | 2 | 30 | 1 | 15
B | 7 | 2 | 14 | 3 | 21
C | 6 | 2 | 12 | 3 | 18
D | 6 | 3 | 18 | 3 | 18
E | 5 | 3 | 15 | 3 | 15

This adjustment in code size adds 13 bits to the number needed to encode the B and C symbols, but it saves 15 bits when coding the A symbol, for a net savings of 2 bits. Thus, for a message with an information content of 85.25 bits, Shannon-Fano coding requires 89 bits, but Huffman coding requires only 87.

In general, Shannon-Fano and Huffman coding are close in performance. But Huffman coding will always at least equal the efficiency of Shannon-Fano coding, so it has become the predominant coding method of its type. Since both algorithms take a similar amount of processing power, it seems sensible to take the one that gives slightly better performance. And Huffman was able to prove that this coding method cannot be improved on with any other integral bit-width coding stream.

Since D. A. Huffman first published his 1952 paper, "A Method for the Construction of Minimum Redundancy Codes," his coding algorithm has been the subject of an overwhelming amount of additional research. Information theory journals to this day carry numerous papers on the implementation of various esoteric flavors of Huffman codes, searching for ever better ways to use this coding method. Huffman coding is used in commercial compression programs, FAX machines, and even the JPEG algorithm. The next logical step in this book is to outline the C code needed to implement the Huffman coding scheme.

Huffman in C

A Huffman coding tree is built as a binary tree, from the leaf nodes up. Huffman may or may not have had digital computers in mind when he developed his code, but programmers use the tree data structure all the time.

Two programs used here illustrate Huffman coding. The compressor, HUFF-C, implements a simple order-0 model and a single Huffman tree to encode it. HUFF-E expands files compressed using HUFF-C. Both programs use a few pieces of utility code that will be seen throughout this book. Before we go on the actual Huffman code, here is a quick overview of what some of the utility modules do.

BITIO.C

Data-compression programs perform lots of input/output (I/O) that does reads or writes of unconventional numbers of bits. Huffman coding, for example, reads and writes bits one at a time. LZW programs read and write codes that can range in size from 9 to 16 bits. The standard C I/O library defined in STDIO.H only accommodates I/O on even byte boundaries. Routines like putc() and getc() read and write single bytes, while fread() and fwrite() read and write whole blocks of bytes at a time. The library offers no help for programmers needing a routine to write a single bit at a time.

To support this conventional I/O in a conventional way, bit-oriented I/O routines are confined to a single source module, BITIO.C. Access to these routines is provided via a header file called BITIO.H, which contains a structure definition and several function prototypes.

Two routines open files for bit I/O, one for input and one for output. As defined in BITIO.H, they are

BIT_FILE *OpenInputBitFile( char *name );

BIT_FILE *OpenOutputBitFile ( char *name );

These two routines return a pointer to a new structure, BIT_FILE. BIT_FILE is also defined in BITIO.H as shown:

typedef struct bit_file {

     FILE *file;

     unsigned char mask;

     int rack;

     int pacifier_counter;

} BIT_FILE:

OpenInputBitFile() or OpenOutputBitFile() perform a conventional fopen() call and store the returned FILE structure pointer in the BIT_FILE structure. The other two structure elements are initialized to their startup values, and a pointer to the resulting BIT_FILE structure is returned.

In BITIO.H, rack contains the current byte of data either read in from the file or waiting to be written out to the file. mask contains a single bit mask used either to set or clear the current output bit or to mask in the current input bit.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The two new structure elements, rack and mask, manage the bit-oriented aspect of a most significant bit in the I/O byte gets or returns the first bit, and the least significant bit in the I/O byte gets or returns the last bit. This means that the mask element of the structure is initialized to 0x80 when the BIT_FILE is first opened. During output, the first write to the BIT_FILE will set or clear that bit, then the mask element will shift to the next. Once the mask has shifted to the point at which all the bits in the output rack have been set or cleared, the rack is written out to the file, and a new rack byte is started.

Performing input from a BIT_FILE is done in a similar fashion. The mask is first set to 0x80, and a single byte from the file is read into the rack element. Each call to read a bit from the file masks in a new bit, then shifts the mask over to the next lower significant bit. Eventually, all bits in the input rack have been returned, and the input routine can read in a new byte from the input file.

Two types of I/O routines are defined in BITIO.C. The first two routines read or write a single bit at a time. The second two read or write multiple bits, up to the size of an unsigned long. These four routines have the following ANSI prototype in BITIO.H:

void            OutputBit( BIT_FILE *bit_file, int bit );

void            OutputBits( BIT_FILE *bit_file,

                            unsigned long code, int count);

int             InputBit( BIT_FILE *bit_file );

unsigned long   InputBits( BIT_FILE *bit_file, int bit_count );

Specialized routines open a BIT_FILE, and two specialized routines close a BIT_FILE. The output routine makes sure that the last byte gets written out to the file. Both the input and output routines need to close their files, then free up the BIT_FILE structure allocated when the file was opened. The BIT_FILE routines used to close a file are defined in BITIO.H with these ANSI prototypes:

void    CloseInputBitFile( BIT_FILE *bit_file );

void    CloseOutputBitFile( BIT_FILE *bit_file );

The input and output routines in BITIO.H also have a pacifier feature that can be useful in testing compression code. Every BIT_FILE structure has a pacifier_counter that gets incremented every time a new byte is read in or written out to the corresponding file. Once every 2,048 bytes, a single character is written to stdout. This helps assure the impatient user that real work is being done. On MS-DOS systems, it also helps ensure that the user can break out of the program if it does not appear to be working correctly.

The header file and code for BITIO.H is shown next:.

/******************** Start of BITIO.H **********************/

#ifndef _BITIO_H

#define _BITIO_H

#include <stdio.h>

typedef struct bit_file {

     FILE *file;

     unsigned char mask;

     int rack;

     int pacifier_counter;

} BIT_FILE;

#ifdef __STDC__

BIT_FILE*     OpenInputBitFile( char *name );

BIT_FILE*     OpenOutputBitFile( char *name );

void          OutputBit( BIT_FILE *bit_file, int bit );

void          OutputBits( BIT_FILE *bit_file,

                          unsigned long code, int count );

int           InputBit( BIT_FILE *bit_file );

unsigned long InputBits( BIT_FILE *bit_file, int bit_count );

void          CloseInputBitFile( BIT_FILE *bit_file );

void          CloseOutputBitFile( BIT_FILE *bit_file );

void          FilePrintBinary( FILE *file, unsigned int code, int bits);

#else /* __STDC__ */

BIT_FILE*     OpenInputBitFile();

BIT_FILE*     OpenOutputBitFile();

void          OutputBit();

void          OutputBits();

int           InputBit();

unsigned long InputBits();

void          CloseInputBitFile();

void          CloseOutputBitFile();

void          FilePrintBinary();

#endif /* __STDC__ */

#endif /* _BITIO_H */

/********************** End of BITIO.H *********************/

/******************** Start of BITIO.C ********************/

/*

* This utility file contains all of the routines needed to implement

* bit oriented routines under either ANSI or K&R C.  It needs to be

* linked with every program used in the book.

*/

#include <stdio.h>

#include <stdlib.h>

#include "bitio.h"

#include "errhand.h"

BIT_FILE *OpenOutputBitFile( name )

char *name;

{

     BIT_FILE *bit_file;

     bit_file = (BIT_FILE *) calloc( 1, sizeof( BIT_FILE ) );

     if ( bit_file == NULL )

          return( bit_file );

     bit_file->file = fopen( name, "rb" );

     bit_file->rack = 0;

     bit_file->mask = 0x80;

     bit_file->pacifier_counter = 0;

     return( bit_file );

}

BIT_FILE *OpenInputBitFile( name )

char *name;

{

     BIT_FILE *bit_file;

     bit_file = (BIT_FILE *) calloc( 1, sizeof( BIT_FILE ) );

     if ( bit_file == NULL )

          return( bit_file );

     bit_file->file = fopen( name, "rb" );

     bit_file->rack = 0;

     bit_file->mask = 0x80;

     bit_file->pacifier_counter = 0;

     return( bit_file );

}

void CloseOutputBitFile( bit_file )

BIT_FILE *bit_file;

{

     if ( bit_file->mask != 0x80 )

      if ( putc( bit_file->rack, bit_file->file ) != bit_file->rack )

           fatal_error( "Fatal error in CloseBitFile!\n" );

     fclose( bit_file->file );

     free( (char *) bit_file );

}

void CloseInputBitFile( bit_file )

BIT_FILE *bit_file;

{

     fclose( bit_file->file );

     free( (char*) bit_file );

}

void OutputBit( bit_file, bit )

BIT_FILE *bit_file;

int bit;

{

   if ( bit )

        bit_file->rack | = bit_file->mask;

   bit_file->mask >>= 1;

   if ( bit_file->mask == 0 ) {

   if ( putc( bit_file->rack, bit_file->file ) != bit_file->rack )

          fatal_error( "Fatal error in OutputBit!\n" );

     else

          if ( ( bit_file->pacifier_counter++ & 4095 ) == 0 )

               putc( '.', stdout );

     bit_file->rack = 0;

     bit_file->mask = 0x80;

    }

}

void OutputBits( bit_file, code, count )

BIT_FILE *bit_file;

unsigned long code;

int count;

{

  unsigned long mask;

  mask = 1L << ( count - 1 );

  while ( mask != 0) {

       if ( mask & code )

            bit_file->rack | = bit_file->mask;

  bit_file->mask >>= 1;

  if ( bit_file->mask == 0 ) {

  if ( putc( bit_file->rack, bit_file->file ) != bit_file->rack )

        fatal_error( "Fatal error in OutputBit!\n" );

   else if ( ( bit_file->pacifier_counter++ & 2047 ) == 0 )

          putc( '.', stdout );

          bit_file->rack = 0;

          bit_file->mask = 0x80;

     }

     mask >>= 1;

   }

}

int InputBit( bit_file )

BIT_FILE *bit_file;

{

     int value;

     if ( bit_file->mask == 0x80 ) {

          bit_file->rack = getc( bit_file->file );

          if ( bit_file->rack == EOF )

               fatal_error( "Fatal error in InputBit!\n" );

          if ( ( bit_file->pacifier_counter++ & 2047 ) == 0 )

                 putc( '.', stdout );

     }

     value = bit_file->rack & bit_file->mask;

     bit_file->mask >>= 1;

     if ( bit_file->mask = 0 )

          bit_file->mask = 0x80;

     return ( value ? 1 : 0 );

     }

unsigned long InputBits( bit_file, bit_count )

BIT_FILE *bit_file;

int bit_count;

{

     unsigned long mask;

     unsigned long return_value;

     mask = 1L << ( bit_count - 1 );

     return_value = 0;

     while ( mask != 0) {

          if ( bit_file->mask == 0x80 ) {

               bit_file->rack = getc( bit_file->file );

               if ( bit_file->rack == EOF )

                    fatal_error( "Fatal error in InputBit!\n" );

          if ( ( bit_file->pacifier_counter++ & 2047 ) == 0 )

               putc( '.', stdout );

          }

          if ( bit_file->rack & bit_file->mask )

               return_value |=mask;

          mask >>= 1;

          bit_file->mask >>= 1;

          if ( bit_file->mask = 00 )

               bit_file->mask = 0x80;

     }

     return( return_value );

}

void FilePrintBinary( file, code, bits )

FILE *file;

unsigned int code;

int bits;

{

     unsigned int mask;

     mask = 1 << ( bits - 1 ):

     while ( mask != 0 ){

         if ( code & mask )

             fputc( '1', file );

         else

             fputc( '0', file);

         mask >>= 1;

     }

}

/********************** End of BITIO.C **********************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

A Reminder about Prototypes

The code in this book works on both Unix K&R and the more modern MS-DOS compilers. This affects the code in this book mainly in the area of function parameters in both prototypes and the function body itself. For the function body, all code in this book will use old-fashioned parameter specifications like this:

int main( argc, argv )

int argc;

char *argv[];

{

...

This is the only method of parameter declaration acceptable to K&R compilers, and as such it has the blessing of the ANSI standard. A few compilers (Microsoft C 6.0 at Warning Level 4, for example) will issue a warning when it encounters this type of function declaration, so be prepared to ignore those warnings. Declaring function parameters in this method will generally have no effect on code reliability or readability, so using the K&R style should be considered a benign anachronism.

Parameters in function declarations present a little more of a problem. The ANSI C specification will accept old style K&R function declarations (such as int main();), but there are good reasons to specify all function arguments in the declaration. When using full prototyping—as in int main( int argc, char *argv[] );—the compiler checks for correct parameter passing when it encounters a call to a function. This helps avoid one of the most commonplace C coding mistakes; incorrect parameter types.

To use this prototyping, and at the same time to stay compatible with K&R compilers, all function prototypes are given in two forms: a K&R-compatible prototype and a full ANSI C prototype. The ANSI C prototypes are selected through a check for __STDC__, a predefined macro defined when a compiler conforms to the ANSI C standard. So the prototype for a set of functions in a header file will look something like this:

#ifdef __STDC__

int main( int argc, char *argv[] );

FOO *open_foo( char *name );

#else /* __STDC__ */

int main();

FOO *open_foo();

#endif /* __STDC__ */

This compromise approach definitely hurts readability, and it is probably not the way to go during code development. But once a set of routines is working properly and not likely to be changed, this type of header file will work fine.

ANSI C compiler users will find that a problem with this header file crops up with numerous MS-DOS compilers. Compilers such as Microsoft C or Borland C++ are ANSI C compilers, but by default they include a number of language extensions, such as far pointers, alternate calling conventions, and so on. When these language extensions are enabled (as they are by default), __STDC__ is not defined, since the compiler is not operating strictly as an ANSI C compiler. This means that the correct function prototypes will not be invoked.

The solution to this problem is to compile the code in this book with the compiler in ANSI C mode. Put the compiler in this mode generally by disabling extensions. Microsoft C accomplishes this from the command line with the /Za switch. Borland C++ uses the -A switch to disable C extensions.

To adapt this code for a specific use on a specific compiler, you may want to eliminate the "#ifdef __STDC__" lines in the header file and code. As more and more compilers use ANSI C prototypes and parameter definitions, this portability machinery will become less and less useful.

MAIN-C.C AND MAIN-E.C

Another piece of utility code used throughout this book is the "main()" program for the compression and expansion programs. Any piece of compression code needs to be plugged into a main program that accepts command-line arguments, opens files, calls the compression routines, then closes the files. For simplicity, I have created two versions of this code: one for the compression program (MAIN-C.C) and one for the expansion program (MAIN-E.C).

Both MAIN-C.C and MAIN-E.C expect to find a compression or expansion routine in another file, a help routine to explain command-line parameters, and an external string with the name of the compression technique being used. The declarations for the functions and name are found in MAIN.H. MAIN.H should be included in the compression module to ensure that the routines are properly typed. MAIN.H is shown next.

The idea behind these two routines is that the infrastructure of a compression test program should not have to be rewritten every time a new compression module is coded. A new routine should just have to interface with the existing compression code.

/********************** Start of MAIN.H ***********************/

#ifndef _MAIN_H

#define _MAIN_H

#ifdef _STDC_

void CompressFile( FILE *input, BIT_FILE *output, int argc, char *argv[] );

void ExpandFile( BIT_FILE *input, FILE *output, int argc, char *argv[] );

#else /* __STDC__ */

void CompressFile();

void ExpandFile();

#endif /* __STDC__ */

extern char *Usage;

extern char *CompressionName;

#endif /* _MAIN_H */

/************************* End of MAIN.H ************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

In MAIN-C.C, a compression module supplies three things: a Usage string, which can print out a list of parameters, etc.; a CompressionName string, which lets the MAIN-C.C program print out the compression method; and a CompressFile() routine, which actually compresses the file. In this chapter, these routines are in a file called HUFF.C, which implements an order 0 model with a Huffman coder. MAIN-C.C is shown below.

/*********************** Start of MAIN-C.C **********************/

/*

* This is the driver program used when testing compression algorithms.

* In order to cut back on repetitive code, this version of main is

* used with all of the compression routines.  In order to turn it into

* a real program, it needs to have another module that supplies one

* routine and two strings, namely:

*

*    void CompressFile( FILE *input, BIT_FILE *output,

*                        int argc, char *argv );

*    char *Usage;

*    char *CompressionName;

*

* The main() routine supplied here has the job of checking for valid

* input and output files, opening them, and then calling the

* compression routine.  If the files are not present, or no arguments

* are supplied, it prints out an error message, which includes the

* Usage string supplied by the compression module.  All of the

* routines and strings needed by this routine are defined in the

* main.h header file.

*

* After this is built into a compression program of any sort, the

* program can be called like this:

*

*    main-c infile outfile [ options ]

*

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include "bitio.h"

#include "errhand.h"

#include "main.h"

#ifdef ___STDC___

void usage_exit( char *prog_name );

void print_ratios( char *input, char *output );

long file_size( char *name );

#else

void usage_exit();

void print_ratios();

long file_size();

#endif

int main( argc, argv )

int argc;

char *argv[];

{

     BIT_FILE *ouput;

     FILE *input;

     setbuf( stdout, NULL );

     if ( argc < 3 )

          usage_exit( argv[ 0 ] );

     input = fopen(argv [ 1 ], "rb" );

     if ( input == NULL )

          fatal_error( "Error opening %s for input/n", argv[ 1 ] );

     output = OpenOutputBitFile( argv[ 2 ] );

     if ( output == NULL )

          fatal error( "Error opening %s for input/n", argv[ 2 ] );

     printf( "\nCompressing %s to %s\n", argv[ 1 ], argv[ 2 ] );

     printf( "Using %s\n, CompressionName );

     argc -= 3;

     argv += 3;

     CompressFile( input, output, argc, argv );

     CloseOutputBitFile( output );

     fclose( input );

     print_ratios( argv[ 1 ], argv[ 2 ] );

     return( 0 );

}

/*

* This routine just wants to print out the usage message that is

* called for when the program is run with no parameters.  The first

* part of the Usage statement is supposed to be just the program

* name. argv[ 0 ] generally holds the fully qualified path name

* of the program being run.  I make a half-hearted attempt to strip

* out that path info and file extension before printing it.  It should

* get the general idea across.

*/

void usage_exit( prog_name )

char *prog_name;

{

     char *short_name;

     char *extension;

     short_name = strrchr( prog_name, '\\' );

     if (short_name == NULL )

          short_name = strrchr( prog_name, '/' );

     if (short_name == NULL )

          short_name = strrchr( prog_name, ':' );

     if (short_name != NULL )

          short_name++;

     else

          short_name = prog_name;

     extension = strrchr( short_name, '.' );

     if ( extension != NULL )

          *extension = '\0';

     printf( "\nUsage: %s %s\n", short_name, Usage );

     exit( 0 );

}

/*

* This routine is used by main to get the size of a file after it has

* been closed.  It does all the work, and returns a long.  The main

* program gets the file size for the plain text, and the size of the

* compressed file, and prints the ratio.

*/

#ifndef SEEK_END

#define SEEK_END 2

#endif

long file_size( name )

char *name;

{

     long eof ftell;

     FILE *file;

     file = fopen( name, "r");

     if ( file == NULL )

          return( OL );

     fseek( file, OL, SEEK_END );

     eof_ftell = ftell( file );

     fclose( file );

     return( eof_ftell );

}

/*

* This routine prints out the compression ratios after the input and

* output files have been closed.

*/

void print_ratios( input, output )

char *input;

char *output;

{

     long input_size;

     long output_size;

     int ratio;

     input_size = file_size( input );

     if ( input_size == 0 )

         input_size = 1;

     output_size = file_size * 100L / input_size );

     ratio = 100 - (int) ( output_size * 100L / input_size );

     printf( "\nInput bytes:        %ld\n", input_size );

     printf( "Output bytes:      %ld/n", output_size );

     if ( output_size == 0 )

         output_size = 1;

     printf( "Compression ratio:  %d%%\n", ratio );

/*********************** End of MAIN-C.C *************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

MAIN-C.C

There are a few expectations about how MAIN-C.C will run. MAIN-C.C is called to compress an input file supplied on the command line and send the compressed output to another file, also supplied on the command line. Thus, the basic command-line invocation of MAIN-C.C is MAIN-C input-file output-file. If the user invokes MAIN-C.C without any arguments, a simple usage statement prints out. The usage statement includes the usage string supplied by the compression module.

If two likely looking file names are on the command line, MAIN-C.C tries to open them both. The input file is opened as a standard file specified in STDIO.H, using fopen(). The output file is opened as a BIT_FILE, as defined in BITIO.H. If either file doesn't open, an error message is printed out and the program exits. If both files open, the next step is to call the compression routine.

MAIN-C.C expects the compression routine to be named CompressFile(). This routine is called with four arguments. The first two are pointers to the file structure for the input file and a pointer to the BIT_FILE structure for the output file. Finally, the updated values for argc and argv are passed to the compression routine. The values for argc and argv will have been adjusted to go past argv[0], which should be the program name, as well as argv[1] and argv[2], the names of the input and output files. The compression program can then scan the remaining arguments for any arguments specific to that particular compression routine. After the compression routine has finished, it returns to MAIN-C.C, which closes down the files and exits.

MAIN-E.C is the converse program to MAIN-C.C. It takes two arguments as well, but this time the input file is the compressed file and the output file is destined to be the uncompressed clear text file. Just like MAIN-C.C, it checks to be sure there are at least two arguments, then tries to open the two files. If there aren't two arguments, a usage message is printed. If either of the files fails to open, an error message is printed. MAIN-E.C is listed below.

/***********************Start of MAIN-E.C***********************/

* This driver program tests compression algorithms.  To cut back on

* repetitive code, this version of main is used with all the expansion

* routines.  The main() routine supplied here checks for valid input and

* output files, opens them, then calls the compression routine.

*

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include "bitio.h"

#include "errhand.h"

#include "main.h"

#ifdef__STDC__

void usage_exit( char *prog_name );

#else

void usage_exit();

#endif

int main( argc, argv )

int argc;

char *argv[];

{

     FILE *output;

     BIT_FILE *input;

     setbuf( stdout, NULL );

     if  ( argc < 3 )

        usage_exit( argv[ 0 ] );

     input = OpenInputBitFile( argv[ 1 ] );

     if ( input == NULL )

          fatal_error( "Error opening %s for input\n", argv[ 1 ]

     output = fopen( argv[ 2 ], "wb" );

     if ( output == NULL )

          fatal_error( "Error opening %s for output\n", argv[ 2 ]

     printf( "\nExpanding %s to %s for output\n", argv[ 2 ] );

     printf( "Using %\n", CompressionName );

     argc -= 3;

     argv += 3;

     ExpandFile( input, output, argc, argv );

     CloseInputBitFile( input );

     fclose( output );

     putc( '\n', stdout );

     return( 0 );

}

/*

* This routine wants to print out the usage message called for when the

* program is run with no parameters.  The first part of the Usage state

* ment is supposed to be just the programname. argv[ 0 ] generally holds

* the fully qualified path name of the program being run.

*/

void usage_exit( prog_name )

char *prog_name;

}

     char *short_name;

     char *extension;

     short_name = strrchr( prog_name, '\\' );

     if ( short_name = = NULL )

          short_name = = strrchr( prog_name, '/' );

     if ( short_name = = NULL )

          short_name = strrchr( prog_name, ':' );

     if ( short_name != NULL )

         short_name++;

     else

        short_name = prog_name;

     extension = strrchr( short_name, '.' );

     if ( extension != NULL )

          *extension = '\0';

     printf( "\nUsage: %s %s\n", short_name, Usage );

     exit( 0 );

}

/********************** End of MAIN-E.C**************************/

ERRHAND.C

One additional routine helps simplify the code. A production version of a program generally needs a somewhat sophisticated error-handling mechanism. In particular, it is important to let the end user know what is happening, clean up any files that may have been only partially processed, and restore any system settings that may have been changed.

In this book, our interest in compression concentrates on testing for accuracy, speed, and compression ratios. Because of this, we have created a simple universal fatal-error handler. The error handler is defined in ERRHAND.H:

/********************** Start of ERRHAND.H **********************/

     #ifndef _ERRHAND_H

     #define _ERRHAND_H

     #ifdef ___STDC___

     void fatal_error( char *fmt, ... );

     #else /* ___STDC___ */

     void fatal_error();

     #endif /* ___STDC___ */

     #endif /* _ERRHAND_H */

/********************** End of ERRHAND.H *************************/

The fatal-error handler is called when an unrecoverable error occurs in the program. It has the same syntax as printf, which means it can be passed a format string and arguments to format and print out.

/************************ Start of ERRHAND.C ***********************/

#include <stdio.h>

#include <stdlib.h>

#include <stdarg.h>

#include "errhand.h"

#ifdef __STDC__

void fatal_error( char *fmt, ... )

#else

#ifdef __UNIX__

void fatal_error( fmt )

char *fmt;

va_dcl

#else

void fatal_error( fmt )

#endif

#endif

{

     va_list argptr;

     va_start( argptr, fmt );

     printf( "Fatal error: " );

     vprintf( fmt, argptr  );

     va_end( argptr );

     exit( -1 );

}

/************************ End of ERRHAND.C ***********************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Into the Huffman Code

With the infrastructure code in place, all we need to do to create a program that demonstrates Huffman coding is to write two routines, CompressFile() and ExpandFile(), and a couple of strings that describe the name of the compression method and program usage. The code for this is found in HUFF.C.

To build the Huffman decoding tree, we need to create a data structure that models the tree on the computer. In our previous examples, each node on the tree had several pieces of information: first, the weight associated with it; second, pointers to two child nodes, one associated with the 0 bit and one associated with the 1 bit. Finally, leaf nodes had the value of the symbol associated with the leaf.

The data structure used in this program to model the Huffman tree was built around the node structure:

typedef struct tree_node {

     unsigned int count;

     unsigned int saved_count;

     int child_0;

     int child_1;

} NODE;

The first thing to notice about this structure is that there is no information about the value of a leaf node. This is because the node structures are allocated as an array of 514 nodes. The lower nodes are all assigned to be leaf nodes, and the upper nodes become internal nodes. The information about the value of a leaf is encoded based on the position of the node in the array.

Instead of having 256 symbols in our alphabet for this program, we actually have 257. Values 0 through 255 are reserved for the normal range of bytes that fit into a character. The remaining symbol value of 256 is reserved for the end-of-stream indicator. This is the last code written out to the stream, and it indicates that no more data will be arriving. Because of the bit-oriented nature of compressed data, it is not ordinarily a simple matter to determine when you have reached an end-of-file state. Handling it with a special code for end-of-stream is one method for getting around this. Another would be to encode the length of the file as a prefix to the compressed data.

With 257 symbols to deal with, we know in advance the largest possible size of the Huffman tree. If all 257 symbols are in use, we will have 256 internal nodes, meaning that we have to allocate an array of 513 node structures. In the program, I actually allocate 514 and use the last one as a dummy value for comparisons when building the tree.

Counting the Symbols

To build the tree, I first calculate the relative frequencies of the symbols. In HUFF.C, I set up an array of 256 longs and count the occurrences of every character in the file, from the start to the end. The position of the file input pointer is saved when the count starts and is restored when it is done. All this takes place in function count_bytes().

Though I start with 32-bit unsigned long counts, I scale the counts back significantly in module scale_counts. Scale_counts() finds the maximum count for any symbol in the file, then develops a scaling factor to make that count and all the rest of the counts fit in a single unsigned character. These counts are then copied into the weight elements of the first 257 node elements.

There are several good reasons for scaling back the counts. First, by limiting any symbol's weight to an 8-bit unsigned character, I can confine all of the math I perform when building the tree to 16-bit unsigned integers. This helps the program run a little faster, and it cuts back on the amount of storage required for the node array. It also limits the maximum size of a Huffman code as well, ensuring that it will fit in a 16-bit unsigned integer.

Saving the Counts

For the expansion program to correctly expand the Huffman encoded bit stream it will be receiving, it needs a copy of the Huffman tree identical to the one used by the encoder. This means that the tree, or its equivalent, must be passed as a header to the file so the expander can read it in before it starts to read Huffman codes.

The easiest way for the expansion program to get this data would probably be to store the entire node array as a preamble to the compressed data. This would work well and would not be too hard for the compressor to do. An alternative method that occupies far less space in the compressed file, however, is to transmit the symbol counts to the expander. Since the Huffman tree is built up in an unambiguous manner from the symbol counts, it stands to reason that the expansion program doesn't need more to do its job. And since the scaled count array will be only 256 bytes, compared to the Huffman tree's 4K bytes, there is good reason to choose this.

I elected to try to cut down on the amount of data to be passed even further. Under many circumstances, the number of counts that stay at zero is considerable. With ASCII text files, such as program listings, there will generally be only around 100 symbols in use out of the possible 256. It seems a waste to transmit all those zero counts when they aren't necessary. To make this happen, I use a slightly more complicated format for the header.

The header used in HUFF.C that contains the symbol counts consists of a series of "count run" definitions, followed by a 0 terminator. A count-run definition consists of the value of the first symbol in the run, followed by the value of the last symbol in the run, followed by the counts for all of the symbols in the run from first to last. This is repeated until each run has been stored in the output file. When there is no more data to store, a first value of zero is written out to the file. Note that a value of zero for the very first run is not treated as an end of data.

For a typical ASCII file, the start of the compressed file might look something like Figure 3.4.

Figure 3.4 The start of a typical compressed ASCII file.

This symbol count format takes a fair amount of work to generate, performed in output_counts() in HUFF.C. Reading in the symbols counts is much simpler, since the work has been done in advance. Reading the counts in from the compressed file during expansion is done in the input_counts() routine.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Building the Tree

Whether compressing or expanding, once the counts have been loaded, it is time to build the Huffman tree. In HUFF.C, this is done in a function called build_tree(). Because some care was taken when creating the data structure, the actual process of creating the tree is the simple matter of sitting in a loop and combining the two free nodes with the lowest weight into a new internal node with the combined weight of the nodes. Once only one free node is left, the tree is done, and the free node is the root of the tree.

The logic of the build_tree() routine is fairly simple. When the routine is first entered, all nodes below 257 have a count value set to their frequency in the file. A nonzero value here means that this is an active node.

build_tree() also sets up a special node used as a straw man for comparison purposes. Node 513, which will never be used, is set to have a count value of 65535, which no normal node can ever exceed. When searching for the two minimum nodes, I will start by setting the minimum node to 513, knowing that any valid active node will fall below its value.

Finally, before the comparisons start, an index to the next free node's initialized. The node array is in use from 0 to 256, so the next free node will be at 257.

After things have been set up, build_tree() goes into an infinite loop. On each pass through the loop, build_tree tries to find the two active nodes with the lowest weights. If only one node is found, the tree is complete and the loop is exited. If there are two good minimum values, a new node to the tree can be created. This new node is set up using the next_free node index. Its two child pointers are set to point to the two minimum nodes found before, and its weight is their sum. The two minimum nodes are now marked as being inactive by setting their weights to 0. Nodes with a weight of 0 are considered to be unused and will never again be selected to represent a minimum.

One piece of inefficient code is deliberately left in build_tree(). There is an extra member in the node structure called saved_count. When a node is taken off the active list by having its count set to zero, the previous count is stored in saved_count. Later, if the user has selected the -d option in order to print out the model, the saved_count can be printed. This helps when debugging the program and when trying to understand how the tree works.

Using the Tree

During the expansion phase, it is easy to see how to use the Huffman tree. Starting at the root node, a single bit at a time is read in by the decoder. If the bit is a 0, the next node is the one pointed to by the child_0 index. If the bit is a 1, the next node is the one pointed to by the child_1 index. If the new node is 256 or less, we have reached a leaf of the tree and can output the corresponding symbol. If the symbol was the special end-of-stream symbol, we can exit instead of sending it out. This is what is done in the expand_node() function. It is just a few lines of code, and it decodes a compressed Huffman code file with relative ease.

Compressing the same file is a bit harder. Essentially, we want to work down the tree, outputting a 1 or a 0 bit at each node, till we get to the appropriate leaf node. Unfortunately, the tree structure makes this impossible. When we start at the root node, we have no idea whether to take the 0 or the 1 branch to arrive at a particular symbol.

One way to solve this problem when building the tree would be to add a parent member to the node structure. When combining the two minimum nodes to form a new internal node, each minimum node would have its parent structure set to point to the new node. With this new node, we could start at the leaf node and work our way up through the tree toward the root. The only problem with this procedure is that we would accumulate bits in reverse order as we went up the tree. We would have to rack them up till we reached the root node, then put them out in reverse order.

Fortunately, there is a better way to do this. Rather than trying to use the tree to code our symbols when compressing a file, we could build a code table by recursively traversing the entire tree one time only. This creates a table of codes, one for each symbol, along with the length of each code. Once the table is built, the file can be encoded by simply outputting the appropriate code for every character in the input file.

The code to convert the tree data structures into a table of codes is very simple, thanks to a recursive algorithm. We start at the root node of the tree with a zero. Then we begin working down the individual branches of the tree, adding a one or a zero to the code each time we travel down a branch. Whenever we reach a leaf, we store the code values for that leaf in the code array and back up to the previous node, where we can start searching down the other side of the tree.

The code to accomplish this is in function convert_tree_to_code(). This routine takes a fair amount of work to create the code table, but once it is done the actual file compression is very easy.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Compression Code

The code for Huffman compression and decompression is shown in the listing below. This single file, HUFF.C, is about 500 lines long, of which probably 30 percent is comments. So we are able to implement a static dictionary Huffman compressor in only about 300 lines of code. The actual amount of code could easily be crunched down to a number much less than that. The small code and storage requirements make Huffman coding ideal for applications where both memory and CPU storage are at a premium.

/********************** Start of HUFF.C  *************************/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#include "bitio.h"

#include "errhand.h"

#include "main.h"

/*

* The NODE structure is a node in the Huffman decoding tree.  It has a

* count, which is its weight in the tree, and the node numbers of its

* two children.  The saved_count member of the structure is only

* there for debugging purposes, and can be safely taken out at any

* time.  It just holds the intial count for each of the symbols, since

* the count member is continually being modified as the tree grows.

*/

typedef struct tree_node {

     unsigned int count;

     unsigned int saved_count;

     int child_0;

     int child_1

} NODE;

/*

* A Huffman tree is set up for decoding, not encoding.  When encoding,

* I first walk through the tree and build up a table of codes for

* each symbol.  The codes are stored in this CODE structure.

/*

typedef struct code {

     unsigned int code;

     int code_bits;

} CODE;

/*

* The special EOS symbol is 256, the first available symbol after all

* of the possible bytes.  When decoding, reading this symbol

* indicates that all of the data has been read in.

*/

#define END_OF_STREAM 256

/*

* Local function prototypes, defined with or without ANSI prototypes.

*/

#ifdef  __STDC__

void count_bytes( FILE *input, unsigned long *long_counts );

void scale_counts( unsigned long *long_counts, NODE *nodes );

int build_tree( NODE *nodes );

void convert_tree_to_code( NODE *nodes,

                           CODE *codes,

                           unsigned int code_so_far,

                           int bits,

                           int node );

void output_counts( BIT_FILE *output, NODE *nodes );

void input_counts( BIT_FILE *input, NODE *nodes );

void print_model( NODE *nodes, CODE *codes );

void compress_data( FILE *input, BIT_FILE *output, CODE *codes );

void expand_data( BIT_FIle *input, FILE *output, NODE *nodes,

                  int root_node );

void print_char( int c );

#else /* __STDC__ */

void count_bytes();

void scale_counts();

int build_tree();

void convert_tree_to_code();

void output_counts();

void input_counts();

void print_model();

void compress_data();

void expand_data();

void print_char();

#endif /* __STDC__ */

/*

* These two strings are used by MAIN-C.C and MAIN-E.C to print

* messages of importance to the use of the program.

*/

char *CompressionName = "static order 0 model with Huffman coding";

char *Usage =

"infile outfile [-d]\n\n\ Specifying -d will dump the modeling\

   data\n";

/*

* CompressFile is the compression routine called by MAIN-C.C.  It

* looks for a single additional argument to be passed to it from

* the command line:  "-d".  If a "-d" is present, it means the

* user wants to see the model data dumped out for debugging

* purposes.

*

* This routine works in a fairly straightforward manner.  First,

* it has to allocate storage for three different arrays of data.

* Next, it counts all the bytes in the input file.  The counts

* are all stored in long int, so the next step is to scale them down

* to single byte counts in the NODE array.  After the counts are

* scaled, the Huffman decoding tree is built on top of the NODE

* array.  Another routine walks through the tree to build a table

* of codes, one per symbol. Finally, when the codes are all ready,

* compressing the file is a simple matter.  After the file is

* compressed, the storage is freed up, and the routine returns.

*

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

     unsigned long *counts;

     NODE *nodes;

     CODE *codes;

     int root_node;

     counts = ( unsigned long *)

                   calloc( 256, sizeof( unsigned long ) );

     if ( counts == NULL )

       fatal_error( "Error allocating counts array\n" );

     if ( ( nodes = (NODE *)

                     calloc( 514, sizeof( NODE ) ) ) == NULL )

       fatal_error( "Error allocating nodes array\n" );

     if ( ( codes = (CODE *)

                     calloc( 257, sizeof( CODE ) ) ) == NULL )

       fatal_error( "Error allocating codes array\n" );

     count_bytes( input, counts );

     scale_counts( counts, nodes );

     output_counts( output, nodes );

     root_node = build_tree( nodes );

     convert_tree_to_code( nodes, codes, 0, 0, root_node );

     if ( argc > 0 && strcmp( argv[ 0 ], "-d" ) == 0 )

       print_model( nodes, codes );

     compress_data( input, output, codes );

     free( (char *) counts );

     free( (char *) nodes );

     free( (char *) codes );

}

/*

* ExpandFile is the routine called by MAIN-E.C to expand a file that

* has been compressed with order 0 Huffman coding.  This routine has

* a simpler job than that of the Compression routine.  All it has to

* do is read in the counts that have been stored in the compressed

* file, then build the Huffman tree.  The data can then be expanded

* by reading in a bit at a time from the compressed file.  Finally,

* the node array is freed and the routine returns.

*

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

     NODE *nodes;

     int root_node;

     if ( ( nodes = (NODE *)

                    calloc( 514, sizeof( NODE ) ) ) == NULL )

       fatal_error( "Error allocating nodes array\n" );

     input_counts( input, nodes );

     root_node = build_tree( nodes );

     if ( argc > 0 && strcmp( argv[ 0 ], "-d" ) == 0 )

          print_model( nodes, 0 );

     expand_data( input, output, nodes, root_node );

     free( (char *) nodes );

}

/*

* In order for the compressor to build the same model, I have to

* store the symbol counts in the compressed file so the expander can

* read them in. In order to save space, I don't save all 256 symbols

* unconditionally. The format used to store counts looks like this:

*

* start, stop, counts, start, stop, counts, ... 0

*

* This means that I store runs of counts, until all the non-zero

* counts have been stored. At this time the list is terminated by

* storing a start value of 0. Note that at least 1 run of counts has

* to be stored, so even if the first start value is 0, I read it in.

* It also means that even in an empty file that has no counts, I have

* to pass at least one count, which will have a value of 0.

*

* In order to efficiently use this format, I have to identify runs of

* non-zero counts. Because of the format used, I don't want to stop a

* run because of just one or two zeros in the count stream. So I have

* to sit in a loop looking for strings of three or more zero values

* in a row.

*

* This is simple in concept, but it ends up being one of the most

* complicated routines in the whole program. A routine that just

* writes out 256 values without attempting to optimize would be much

* simpler, but would hurt compression quite a bit on small files.

*

*/

void output_counts ( output, nodes )

BIT_FILE *output;

NODE *nodes;

{

     int first;

     int last;

     int next;

     int i;

     first = 0;

     while ( first < 255 && nodes[ first ].count == 0 )

          first++;

/*

* Each time I hit the start of the loop,  I assume that first is the

* start of a run of non-zero values.  The rest of the loop is

* concerned with finding the value for last, which is the end of the

* run, and the value of next, which is the start of the next run.

* At the end of the loop, I assign next to first, so it starts in on

* the next run.

*/

     for ( ; first < 256 ; first = next) {

          last = first + 1;

          for ( ; ; ) {

              for ( ; last < 256 ; last ++ )

                   if ( nodes[ last ].count == 0 )

                        break;

              last--;

              for ( next = last + 1; next < 256 ; next++ )

                   if ( nodes[ next ]. count ! = 0 )

                        break;

              if ( next > 255 )

                   break;

              if ( ( next - last ) > 3 )

                   break;

              last = next;

          };

/*

* Here is where I output first, last, and all the counts in between.

*/

             if ( putc( first, output->file ) != first)

                  fatal_error( "Error writing byte counts\n" );

             if ( putc( last, output->file ) != last)

                  fatal_error( "Error writing byte counts\n" );

             for ( i = first ; i <= last ; i++ ) {

                 if ( putc( nodes[ i ]. count, output->file ) !=

                      (int) nodes[ i ]. count)

                      fatal_error( "Error writing byte counts\n" );

             }

       }

       if ( putc( 0, output->file ) != 0

              fatal_error( "Error writing byte counts\n" );

}

/*

* When expanding, I have to read in the same set of counts.  This is

* quite a bit easier that the process of writing them out, since no

* decision making needs to be done.  All I do is read in first, check

* to see if I am all done, and if not, read in last and a string of

* counts.

*/

void input_counts( input, nodes)

BIT_FILE *input;

NODE *nodes;

{

     int first;

     int last;

     int i;

     int c;

     for ( i = 0 ; i < 256 ; i++ )

         nodes[ i ]. count = 0;

     if ( ( first = getc( input->file ) ) == EOF)

          fatal_error( "Error reading byte counts\n" );

     if ( ( last = getc( input->file ) ) == EOF )

          fatal_error( "Error reading byte counts\n);

     for ( ; ; ) {

         for ( i = first ; i <= last ; i++ )

             if ( ( c = getc( input->file ) ) == EOF)

                  fatal_error( "Error reading byte counts\n" );

             else

                  nodes[ i ]. count = (unsigned int) c;

         if ( ( first = getc( input->file ) ) == EOF )

              fatal_error( "Error reading byte counts\n" );

         if ( first == 0)

              break;

         if ( ( last = getc( input->file ) ) == EOF )

              fatal_error( "Error reading byte counts\n" );

     }

     nodes[ END_OF_STREAM ].count = 1;

}

/*

* This routine counts the frequency of occurence of every byte in

* the input file.  It marks the place in the input stream where it

* started, counts up all the bytes, then returns to the place where

* it started.  In most C implementations, the length of a file

* cannot exceed an unsigned long, so this routine should always

* work.

*/

#ifndef SEEK_SET

#define SEEK_SET 0

#endif

void count_bytes( input, counts)

FILE *input;

unsigned long *counts;

{

     long input_marker;

     int c;

     input_marker = ftell( input );

     while ( ( c = getc( input ) ) != EOF )

         counts[ c ]++;

     fseek( input, input_marker, SEEK_SET );

}

/*

* In order to limit the size of my Huffman codes to 16 bits, I scale

* my counts down so they fit in an unsigned char, and then store them

* all as initial weights in my NODE array.  The only thing to be

* careful of is to make sure that a node with a non-zero count doesn't

* get scaled down to 0.  Nodes with values of 0 don't get codes.

*/

void scale_counts( counts, nodes )

unsigned long *counts;

NODE *nodes;

{

     unsigned long max_count;

     int i;

     max_count = 0;

     for ( i = 0 ; i < 256 ; i++ )

         if ( counts[ i ] > max_count )

              max_count = counts[ i ] ;

     if ( max_count == 0 ) {

          counts[ 0 ] = 1;

          max_count = 1;

     }

     max_count = max_count / 255;

     max_count = max_count + 1;

     for ( i = 0 ; i < 256 ; i++ ) {

          nodes[ i ].count = (unsigned int)

                                 ( counts[ i ] / max_count );

          if ( nodes[ i ].count == 0 && counts[ i ] !=0 );

               nodes[ i ].count = 1;

     }

     nodes[ END_OF_STREAM ]. count = 1;

}

/*

* Building the Huffman tree is fairly simple.  All of the active nodes

* are scanned in order to locate the two nodes with the minimum

* weights.  These two weights are added together and assigned to a new

* node.  The new node makes the two minimum nodes into its 0 child

* and 1 child.  The two minimum nodes are then marked as inactive.

* This process repeats until there is only one node left, which is

* the root node.  The tree is done, and the root node is passed back

* to the calling routine.

*

* Node 513 is used here to arbitratily provide a node with a guaran

* teed maximum value.  It starts off being min_1 and min_2.  After all

* active nodes have been scanned, I can tell if there is only one

* active node left by checking to see if min_1 is still 513.

*/

int build_tree( nodes )

NODE *nodes;

{

     int next_free;

     int i;

     int min_1;

     int min_2;

     nodes[ 513 ].count = Oxffff;

     for ( next_free = END_OF_STREAM + 1 ; ; next_free++ ) {

          min_1 = 513;

          min_2 = 513;

          for ( i = 0 ; i < next_free; i++ )

              if ( nodes[ i ].count != 0) {

                  if ( nodes[ i ].count < nodes[ min_1 ].count ) {

                   min_2 + min_1;

                   min_1 = i;

                  } else if ( nodes[ i ].count

                              < nodes[ min_2 ].count)

                    min_2 = i;

              }

          if ( min_2 == 513 )

               break;

          nodes[ next_free ].count = nodes[ min_1 ].count

                                      + nodes[ min_2 ].count;

          nodes[ min_1 ].saved_count = nodes[ min_1 ].count;

          nodes[ min_1 ].count = 0;

          nodes[ min_2 ].saved_count = nodes[ min_2 ].count;

          nodes[ min_2 ].count = 0;

          nodes[ next_free ].child_0 = min_1;

          nodes[ next_free ].child_1 = min_2;

          }

          next_free--;

          nodes[ next_free ].saved_count = nodes[ next_free ].count;

          return( next_free );

}

/*

* Since the Huffman tree is built as a decoding tree, there is

* no simple way to get the encoding values for each symbol out of

* it.  This routine recursively walks through the tree, adding the

* child bits to each code until it gets to a leaf.  When it gets

* to a leaf, it stores the code value in the CODE element, and

* returns.

*/

void convert_tree_to_code( nodes, codes, code_so_far, bits, node )

NODE *nodes;

CODE *codes;

unsigned int code_so_far;

int bits;

int node;

{

     if ( node <= END_OF_STREAM ) {

          codes[ node ].code = code_so_far;

          codes[ node ].code_bits = bits;

          return;

     }

     code_so_far <<= 1;

     bits++;

     convert_tree_to_code( nodes, codes, code_so_far, bits,

                           nodes[ node ]. child_0 );

     convert_tree_to_code( nodes, codes, code_so_far | 1,

                           bits, nodes[ node ].child_1 );

}

/*

* If the -d command line option is specified, this routine is called

* to print out some of the model information after the tree is built.

* Note that this is the only place that the saved_count NODE element

* is used for anything at all, and  in this case it is just for

* diagnostic information.  By the time I get here, and the tree has

* been built, every active element will have 0 in its count.

*/

void print_mode1( nodes, codes )

NODE *nodes;

CODE *codes;

{

     int i;

     for ( i = 0 ; i < 513 ; i++ ) {

       if ( nodes[ i ].saved_count != 0 ) {

            printf( "node=" );

            print_char( i );

            printf( " count=%3d", nodes[ i ].saved_count );

            printf( " child_0=" );

            print_char( nodes[ i ]. child_0 );

            printf( " child_1=" );

            print_char( nodes[ i ].child_1 );

            if ( codes && i <= END_OF_STREAM ) {

                printf( " Huffman code=" );

                FilePrintBinary( stdout, codes[ i ].code,

                                 codes[ i ].code_bits );

            }

            printf( "\n" );

       }

     }

}

/*

* The print_model routine uses this function to print out node num

* bers.  The catch is if it is a printable character, it gets printed

* out as a character.  This makes the debug output a little easier to

* read.

*/

void print_char( c )

int c;

{

     if ( c >= 0x20 && c < 127 )

          printf( "`%c'", c );

     else

          printf( "%3d", c );

}

/*

* Once the tree gets built, and the CODE table is built, compressing

* the data is a breeze.  Each byte is read in, and its corresponding

* Huffman code is sent out.

*/

void compress_data( input, output, codes )

FILE *input;

BIT_FILE *output;

CODE *codes;

{

     int c;

     while ( ( c = getc( input ) ) != EOF )

          OutputBits( output, (unsigned long) codes[ c ].code,

                      codes[ c ].code_bits );

     OutputBits( output, (unsigned long) codes[ END_OF_STREAM ].code,

                          codes[ END_OF_STREAM ].code_bits );

}

/*

* Expanding compressed data is a little harder than the compression

* phase.  As each new symbol is decoded, the tree is traversed,

* starting at the root node, reading a bit in, and taking either the

* child_0 or child_1 path.  Eventually, the tree winds down to a

* leaf node, and the corresponding symbol is output.  If the symbol

* is the END_OF_STREAM symbol, it doesn't get written out, and

* instead the whole process terminates.

*/

void expand_data( input, output, nodes, root_node )

BIT_FILE *input;

FILE *output;

NODE *nodes;

int root_node;

{

     int node;

     for ( ; ; ) {

         node = root_node;

         do {

              if ( InputBit( input ) )

                   node = nodes[ node ].child_1;

              else

                   node = nodes[ node ].child_0;

         } while ( node . END_OF_STREAM );

         if ( node == END_OF_STREAM )

              break;

         if ( ( putc( node, output ) ) != node )

              fatal_error( "Error trying to write byte to output" );

     }

}

/******************************End of HUFF.C***************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Putting It All Together

The actual commands to build the compression and expansion programs will differ depending on which compiler and operating system you are using. Assuming you name the compression program HUFF-C and the expansion program HUFF-E, here are the command lines to compile the programs with various compilers:

     Microsoft C:  cl /W3 /Za /FeHUFF-C MAIN-C.C HUFF.C BITIO.C ERRHAND.C

                   cl /W3 /Za /FeHUFF-E MAIN-E.C HUFF.C BITIO.C ERRHAND.C

     Borland C++:  bcc -Ax -w -eHUFF-C MAIN-C.C HUFF.C BITIO.C ERRHAND.C

                   bcc -Ax -w -eHUFF-E MAIN-E.C HUFF.C BITIO.C ERRHAND.C

     UNIX pcc:  cc -ohuff-c main-c.c huff.c bitio.c errhand.c

                cc -ohuff-e main-e.c huff.c bitio.c errhand.c

Remember that ANSI-compatible C compilers must have their extensions turned off on the command line to enable the __STDC__ macro. The __STDC__macro is necessary to turn on the ANSI prototypes. If you don't want to continually have to add this unfamiliar command-line switch when you compile, simply strip out the "#ifdef __STDC__" line and always pull in the ANSI C prototypes. The only reason for doing this is to have code that will compile cleanly on K&R compilers. If you aren't using a K&R compiler, keeping in the K&R prototypes is of dubious value.

The module ERRHAND.C needs the __UNIX__ definition in order to use old-style variable arguments. Fully compliant ANSI C compilers may not have to turn this option on. If you are going to only be using your source code on your UNIX system, it would probably be simpler to put a "#define__UNIX__" in your ERRHAND.C file.

Performance

Order 0 Huffman coding is not going to take any prizes for compression ratios. But it does fairly well in terms of program size, memory requirements, and processing speed. To see how HUFF.C does overall, see the scorecards in Appendix A.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 4
A Significant Improvement: Adaptive Huffman Coding

In Chapter 3, we saw how Huffman coding could perform effective data compression by reducing the amount of redundancy in the coding of symbols. Huffman coding does not in itself tell how to reduce the information content of each symbol by developing an accurate model. But any model that can calculate the probability of a symbol with any accuracy should be able to use Huffman coding to compress data.

The examples in Chapter 3 all used order 0 models, which are essentially context free. This means that the probability of a given character is calculated without taking into account the characters that preceded it in a message. The programs used in Chapter 3 just analyzed the entire input file and created a table of probabilities for each symbol. As long as these probabilities deviated from a purely uniform distribution, we were able to compress the data.

A minor drawback to Huffman coding programs is the requirement that they transmit a copy of the probability table with the compressed data. The expansion program would have no way of correctly decoding the data without the probability table. The table requires at most the addition of an extra 250 or so bytes to the output table, and consequently it usually doesn't make much difference in the compression ratio. Even small files won't be greatly affected, since the probability table should also be small for these files.

The problem with this "minor drawback" is that as we attempt to improve the compression ability of our program, the penalty becomes more and more significant. If we move from order-0 to order-1 modeling, for example, we now have to transmit 257 probability tables instead of just one. So by using a technique that enables us to predict characters more accurately, we incur a penalty in terms of added overhead. Unless the files we are going to compress are very large, this added penalty will frequently wipe out any improvements made by increasing the order.

Adaptive Coding

This seems to lead to an impasse. To compress better, we need to accumulate more statistics. When we get more statistics, we achieve better compression but we wipe out any gains by having to send more modeling data.

Fortunately, there is a way out of this dilemma. Adaptive coding lets us use higher-order modeling without paying any penalty for added statistics. It does this by adjusting the Huffman tree on the fly, based on data previously seen and having no knowledge about future statistics.

Adaptive coding is not something that can just be used with Huffman coding. In principle, almost any form of coding can be converted to use an adaptive method. The high-level C program required to do adaptive compression is shown below.

initialize_model();

do {

     c = getc( input );

     encode( c, output );

     update_model( c );

} while ( c !=  EOF );

The decompressor works in a nearly identical fashion, as shown here:

initialize_model();

while ( ( c = decode( input ) ) ! = EOF ) {

    putc( c, output );

    update_model( c );

}

Adaptive coding works since two of the routines used in these two algorithms are identical: initialize_model() and update_model(). If these routines differed even slightly between the compression and decompression programs, the whole system would fall apart.

This sort of coding is fairly simple. The compressor and decompressor start off with identical models to encode and decode. So when the compressor puts out its very first encoded symbol, the decompressor will be able to interpret it.

After the compressor emits the first symbol, it proceeds to the update_model() function. This is where the adaptive nature of the program begins. The update model takes into account the character that has just been seen and updates the frequency and encoding data used to encode that character. In a Huffman tree, it means incrementing the count for the particular symbol, then updating the Huffman coding tree.

Updating the Huffman Tree

The algorithm for constructing a Huffman coding tree is fairly simple, but it is not something we would want to do after every character is encoded. It would be relatively simple to implement adaptive Huffman coding with the following update function:

update_model( int c )

{

    counts[ c ]++;

    construct_tree( counts );

}

Unfortunately, what we would end up with would probably be the world's slowest data-compression program. Building the tree takes too much work to reasonably expect to do it after every character.

Fortunately, there is a way to take an existing Huffman coding tree and modify it to account for a new character. All it takes is a slightly different approach to building the tree in the first place. This approach introduces a concept known as the sibling property. A Huffman tree is simply a binary tree that has a weight assigned to every node, whether an internal node or a leaf node. Each node (except for the root) has a sibling, the other node that shares the same parent. The tree exhibits the sibling property if the nodes can be listed in order of increasing weight and if every node appears adjacent to its sibling in the list.

A binary tree is a Huffman tree if and only if it obeys the sibling property. Figure 4.1 shows a Huffman tree that illustrates how this works. In this tree, the nodes have been assigned numbers, with the numbers assigned from left to right starting at the lowest row of nodes and working up. This tree was created using a conventional Huffman algorithm given the weights A=1, B=2, C=2, D=2, and E=10.

Figure 4.1 A Huffman tree.

In Figure 4.1, the A, B, C, and D nodes at the bottom of the tree are numbered in increasing order starting at 1. Nodes 5 and 6 are the first two internal nodes, with weights of 3 and 4. The node numbers work their way up to node 9, the root. This arrangement shows that this tree obeys the sibling property. The nodes have been numbered in order of increasing weight, and each node is adjacent to its sibling in the list.

The sibling property is important in adaptive Huffman coding since it helps show what we need to do to a Huffman tree when it is time to update the counts. Maintaining the sibling property during the update assures that we have a Huffman tree before and after the counts are adjusted.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Updating the tree consists of two basic types of operations. The first, incrementing the count, is easy to follow conceptually. To increment the count for symbol 'c,' start at the leaf node for the symbol and increment the count for the leaf node. Then move up to the parent node. Since the weight of the parent node is the sum of the weight of its children, incrementing its weight by one will adjust it to its correct value. This process continues all the way up the tree till we reach the root node.

Figure 4.2 shows how the increment operation affects the tree. Starting at the leaf, the increment works its way up the tree till it reaches the parent node. Implementing this portion of the code is relatively simple. Be sure that each node has a parent pointer and that an index points to the leaf node for each symbol. This can be done using conventional data structures at a low cost. The average number of increment operations required will correspond to the average number of bits needed to encode a symbol.

Figure 4.2 The increment process.

The second operation required in the update procedure arises when the node increment causes a violation of the sibling property. This occurs when the node being incremented has the same weight as the next highest node in the list. If the increment were to proceed as normal, we would no longer have a Huffman tree.

When we have an increment that violates the sibling property, we need to move the affected node to a higher point in the list. This means that the node is detached from its present position in the tree and swapped with a node farther up the list.

Figure 4.3 shows the same Huffman tree from Figure 4.2 after the A node has been incremented again, then switched with the D node. How was the D node selected as the one to be switched? To minimize the amount of work during the shuffle, we want to swap just two nodes. If the newly incremented node has a weight of W + 1, the next higher node will have a weight of W. There may be more nodes after the next higher one that have a value of W as well. The swap procedure moves up the node list till it finds the last node with a weight of W. That node is swapped with the node with weight W + 1. The new node list will then have a string of 1 or more weight W nodes, followed by the newly incremented node with weight W + 1.

Figure 4.3 After a node switch (only the A node has been incremented).

In Figure 4.3, the A node was incremented from a weight of 2 to 3. Since the next node in the list, the B node, had a weight of 2, the tree no longer obeyed the sibling property. This meant it was time to swap. We worked our way up the list of nodes till we found the last node with a weight of 2, the D node. The A and D nodes were then swapped, yielding a correctly ordered tree.

After the swap is completed, the update can continue. The next node to be incremented will be the new parent of the incremented node. In Figure 4.3, this would be internal node #6. As each node is incremented, a check is performed for correct ordering. A swap is performed if necessary.

What Swapping Does

The swap shown in Figure 4.3 doesn't have a noticeable effect on the coding of the symbols. The A and D nodes were swapped, but the length of their codes did not change. They were both three bits long before the swap and three bits long after.

Figure 4.4 shows what happens to the three after the A symbol has been incremented two more times. After the second increment, the A node has increased enough to swap positions with an internal node on a higher level of the tree. This reshapes the tree, impacting the length of the codes. When A had a count of two like three other symbols, it was encoded using three bits. Now, when its count has increased to five, it is encoded using only 2 bits. Symbols C is still encoded using 3 bits, but B and D have slipped down to 4 bits.

Figure 4.4 After another node switch.

The Algorithm

In summary, the algorithm for incrementing the count of a node goes something like what's shown below:

for ( ; ; ) {

     increment nodes[ node ].count;

     if ( node == ROOT )

         break;

     if ( nodes[ node ].count > nodes[ node + 1 ].count )

         swap_nodes();

     node = nodes[ node ].parent;

}

The swap_nodes() routine has to move up through the list of nodes until it finds the right node to swap with. It then performs the swap. This routine looks something like that shown below:

swap_node = node + 1;

while ( nodes[ swap_node + 1 ].count < nodes[ node ].count )

     swap_node++;

temp = nodes[ swap_node ].parent;

nodes[ swap_node ].parent = nodes[ node ].parent;

nodes[ node ].parent = temp;

An Enhancement

One way to make coding more efficient is to make sure your coder doesn't waste coding space for symbols not used in the message. With the standard Huffman coding in the previous chapter, this was easy. Since we made a pass over the data to collect statistics before building the tree, we knew in advance which symbols weren't used. So when we built the Huffman tree we didn't have to include symbols with a count of 0.

With an adaptive process, we don't know in advance which symbols will show up in the message. The simplest way to handle this problem is to initialize the Huffman tree to have all 256 possible bytes (for conventional 8-bit data messages) predefined with a count of 1. When the encoding first starts, each message will have a length of eight bits. As statistics accumulate, frequently seen characters will start to use fewer and fewer bits.

This method of encoding works, but in many cases it wastes coding capacity. Particularly in shorter messages, the extra unused codes tend to blunt the effect of compression by skewing the statistics of the message.

A better way to handle this aspect of coding is to start the encoding process with an empty table and add symbols only as they are seen in the incoming message. But this presents us with a seeming contradiction. The first time a symbol appears, it can't be encoded since it doesn't appear in the table. So how do we get around this problem?


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Escape Code

The answer to this puzzle is the escape code. The escape code is a special symbol sent out of the encoder to signify that we are going to `escape' from the current context. The decoder know that the next symbol will be encoded in a different context. We can use this mechanism to encode symbols that don't appear in the currently defined Huffman tree.

In the example program in this chapter, the escape code signifies that the next symbol to be encoded will be sent as a plain 8-bit character. The symbol is added to the table, and regular encoding resumes. The C code to implement the encoder for this algorithm looks something like this:

encode( char c )

{

     if ( in_tree( c ) )

          transmit_huffman_code( c, out_file );

     else {

          transmit_huffman_code( ESCAPE, out_file );

          putc( c, out_file );

          add_code_to_tree( c );

     }

     update_tree( c );

}

This example shows that the escape code is transmitted like any other symbol from the Huffman tree, so it has to appear in the Huffman tree to be properly transmitted. When the encoder first starts up, it needs to be initialized with the escape code already present.

In the implementation used in the example code for this chapter, the Huffman tree is actually initialized with two values: the escape code and the end of file code. Since both will appear in the file, we start off with them in a very small Huffman tree:

Figure 4.5 A Huffman tree initialized with two values.

As the encoding process goes on, the table fills up and the tree fleshes out. The end of file code will always have a weight of one, and in this implementation, so will the escape code. As the tree grows, these two codes will always be stuck down at the remotest branches of the tree and have the longest codes.

The Overflow Problem

As the compression program progresses, the counts in the table increase. At some point, the counts become large enough to cause trouble for the program. There are two possible areas of concern. The first occurs when the weight of the root exceeds the capacity of the counters in the tree. For most of the programs used here, that will be 16 bits.

Another possible problem can occur even sooner. It happens when the length of the longest possible Huffman code exceeds the range of the integer used to transmit it. The decoding process doesn't care how long a code is, since it works its way down through the tree a bit at a time. The transmitter has a different problem though. It has to start at the leaf node of the tree and work up towards the root. It accumulates bits to be transmitted in reverse order, so it has to stack them up. This is conventionally done in an integer variable, so this means that when a Huffman code exceeds the size of that integer, there is a problem.

The maximum length of a Huffman code is related to the maximum count via a Fibonacci sequence. A Fibonacci function is defined as follows:

int fib( int n )

{

  if ( n <= 1 )

      return( 1 );

  else

    return( fib( n - 1 ) + fib( n -2 ) );

}

The sequence of Fibonacci numbers looks something like this: 1, 1, 2, 3, 5, 8, 13, 21, 34, etc. These numbers show up in the worst-case, most lopsided Huffman tree:

Figure 4.6 A lopsided Huffman tree produced through a sequence of Fibonacci numbers.

From this we can deduce that if the weight at the root node of a Huffman tree equals fib(i), then the longest code for that tree is i - 1. This means that if the integers used with our Huffman codes are only 16 bits long, a root value of 4181 could potentially introduce an overflow. (This low value is frequently overlooked in simple Huffman implementations. Setting up a file with Fibonacci counts up to fib[18] is a good way to test a Huffman program). When we update the tree, we ought to check for a maximum value. Once we reach that value, we need to rescale all the counts, typically dividing them by a fixed factor, often two.

One problem with dividing all the counts by two is that it can possibly reshape the tree. Since we are dealing with integers, dividing by two truncates the fractional part of the result, which can lead to imbalances. Consider the Huffman tree shown in Figure 4.7.

Figure 4.7 A Huffman tree created for four symbols.

This is a tree created for four symbols: A, B, C, and D, with weights of 3, 3, 6, and 6. The nodes of the tree are numbered in this diagram, and the diagram clearly shows that the tree is a Huffman tree, since it obeys the sibling property. The problem with this tree occurs if we try a rescaling operation. The simple version of the rescaling algorithm would go through the tree, dividing every leaf node weight by two, then rebuilding upwards from the leaf nodes. The resulting tree would look like what follows.

Figure 4.8 The rescaling problem after the nodes are divided by two.

The problem with the resulting tree is that it is no longer a Huffman tree. Because of the vagaries of truncation that follow integer division, we need to end up with a tree that has a slightly different shape:

Figure 4.9 What the tree should look like after integer division.

The properly organized Huffman tree has a drastically different shape from what it had before being rescaled. This happens because rescaling loses some of the precision in our statistical base, introducing errors sometimes reflected in the rescaled tree. As time goes on, we accumulate more statistics, and the effect of the errors gradually fades away.

Unfortunately, there is no simple way to compensate for the necessary reshaping the tree after rescaling. The sample code in this chapter merely does a brute-force rebuilding after rescaling. Rebuilding the entire tree is a relatively expensive operation, but since the rescaling operation is rare, we can live with the cost.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

A Rescaling Bonus

An interesting side effect comes out of rescaling our tree at periodic intervals. Though we lose accuracy by scaling our counts, testing reveals that rescaling generally results in better compression ratios than if rescaling is postponed. This occurs because data streams frequently have a "decaying recency" effect, or the statistics for recently seen symbols are generally more valid than those accumulated farther back in the data stream. To put it simply, current symbols are more like recent symbols than older symbols.

The rescaling operation tends to discount the effect of older symbols, while increasing the importance of recent symbols. Though difficult to quantify, this seems to have a good effect on compression. Experimenting with various rescaling points will yield different results at differing values, but it doesn't seem possible to pin down an optimal strategy. There may be an optimal value for rescaling, but it moves around with different types of data streams.

The Code

The sample code for this chapter is a simple order-0 adaptive Huffman compression routine. It is linked with the standard I/O and user interface routines from the previous chapter to create a standalone compression program and a decompression program.

The key to understanding how this sample code operates lies in understanding the data structures in the program. The data structure that describes the tree is shown next.

struct tree {

     int leaf[ SYMBOL_COUNT ];

     int next_free_node;

     struct node {

          unsigned int weight;

          int parent;

          int child_is_leaf;

          int child;

      } nodes[ NODE_TABLE_COUNT ];

} Tree;

Two arrays describe the tree. The tree itself is entirely represented by the nodes[] array. This array is a set of structures with the following elements:

unsigned int weight: | This weight element is the weight of individual node, just as it has been described previously in this chapter.
int parent: | This int is the index of the parent node. The parent node information is necessary both when encoding a symbol, and when updating the model.
int child_is_leaf: | The child of a given node can either be a leaf or a pair of nodes. This flag is used to indicate which it is.
int child: | If the child is a leaf, this int holds the value of the symbol encoded at the leaf. If the child is a pair of nodes, this value is the index to the first node. Because of the sibling property, the two nodes will always be adjacent to one another, so we know the first node will be child, and the second node will be child+1.

As described earlier in the chapter, every node in the tree is kept in a number list. When discussing the list before, we had the nodes with the lowest weight starting at 1 and working up to higher numbers until reaching the root. The implementation in this program is backwards from that, though the same principles apply. The list of nodes is the nodes[] array, with the highest number on the list appearing at nodes[0]. As we work our way down through the lower weights, we go to higher indices in the nodes list.

When the tree is first initialized, nodes[0] is the root node, nodes [1] is set to the end-of-stream symbol, and nodes[2] is set to the escape symbol. The next_free_node element in the tree is then set to 3, and the next time a character is added to the tree, it will be placed in nodes[3].

The leaf[] array in the tree data structure is used to find the leaf node for a particular symbol. To encode a symbol, start at the leaf node and work up to the root node of the tree, accumulating bits on the way (in reverse order). Without a leaf[] array to keep track of the leaf nodes, we would have to do a search through the entire tree every time we wanted to encode a character.

Initialization of the Array

Regardless of whether we perform compression or expansion, we initialize the Huffman tree using the same routine. When performing adaptive compression, it is extremely important to use an identical algorithm for both initialization and updating of the compression model. In this case, we use the same code to ensure that it happens.

The initialization routine, InitializeTree(tree), is the first thing called by both the compression and expansion code. It uses the following code to initialize nodes 0, 1, and 2:

tree->nodes[ ROOT_NODE].child                = ROOT_NODE + 1;

tree->nodes[ ROOT_NODE].child_is_leaf        = FALSE;

tree->nodes[ ROOT_NODE].weight               = 2;

tree->nodes[ ROOT_NODE].parent               = -1;

tree->nodes[ ROOT_NODE + 1 ].child           = END_OF_STREAM;

tree->nodes[ ROOT_NODE + 1 ].child_is_leaf   = TRUE;

tree->nodes[ ROOT_NODE + 1 ].weight          = 1;

tree->nodes[ ROOT_NODE + 1 ].parent          = ROOT_NODE;

tree->leaf[ END_OF_STREAM ]                  = ROOT_NODE + 1;

tree->nodes[ ROOT_NODE + 2 ].child           = ESCAPE;

tree->nodes[ ROOT_NODE + 2 ].child_is_leaf   = TRUE;

tree->nodes[ ROOT_NODE + 2 ].weight          = 1;

tree->nodes[ ROOT_NODE + 2 ].parent          = ROOT_NODE;

tree->leaf[ ESCAPE ]                         = ROOT_NODE + 2;

tree->next_free_node                         = ROOT_NODE + 3;

for ( i = 0 ; i < END_OF_STREAM ; i++ )

    tree->leaf[ i ] = -1;

The initialization of the tree->nodes[] elements is fairly direct. We assign the escape and end-of-stream nodes a weight of 1, which gives the root a weight of 2. The escape and end-of stream elements in the tree->leaf[] array are initialized to point to the appropriate nodes, and the parent and child pointers for each of the three nodes are initialized.

The final details required during initialization are to set up the tree->next_free_node index and to initialize the remaining elements of the tree->leaf[] array. Since none of the leaf[] elements for our conventional symbols have been initialized, they are all set to values of -1. During the encoding process, we will compare the tree->leaf[] value for a given symbol to -1 to see if it has already been defined.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Compress Main Program

The code for the compression program is short:

InitializeTree( &Tree );

while ( ( c = getc( input ) ) != EOF ) {

  EncodeSymbol( &Tree, c, output );

  UpdateModel( &Tree, c );

}

EncodeSymbol(  &Tree, END_OF_STREAM, output );

Once the tree has been initialized, the program sits in a loop encoding characters and updating the model. When there are no more characters left to encode, it encodes the end-of-stream symbol, and it is done.

Complexities are hidden in these functions. The EncodeSymbol function needs to see if the symbol is already defined. If it isn't, EncodeSymbol needs to output the escape code and the unencoded symbol. EncodeSymbol then needs to add the symbol to the tree, with a count of 0.

The UpdateModel function also hides some complexity. It performs the update discussed previously in the chapter, which is fairly complex. Before doing the update, it checks to see if the root node has reached the maximum allowable weight. If it has, the tree is scaled by a factor of two and rebuilt.

The Expand Main Program

Like the compress main program, the expand program is short and to the point. After initializing the tree, it reads in symbols via the DecodeSymbol routine, then writes them to the output file. After each symbol is decoded, it is written to the output file, and the model is updated.

As in the compress program, a certain amount of complexity is concealed in the higher-level functions. The DecodeSymbol routine has to see if the symbol it decodes is an escape code. If it is, DecodeSymbol throws away the escape code and reads in an "unencoded" 8-bit symbol. The symbol is then added to the Huffman tree, with an initial count of 0.

As previously seen, the UpdateModel() routine has to see if the root node has reached the maximum allowable count. If it has, the Huffman tree is rebuilt. After that, the normal increment/test/swap routine ensues.

InitializeTree( &Tree );

while ( ( c = DecodeSymbol( &Tree, input ) ) !=END_OF_STREAM ) {

  if ( putc( c, output ) == EOF )

    fatal_error( "Error writing character" );

  UpdateModel( &Tree, c );

}

Encoding the Symbol

After initializing the tree, the compress routine repeatedly calls the EncodeSymbol routine. The EncodeSymbol routine (shown below) first identifies the leaf node for the symbol to be encoded. If the leaf table returns a-1, it means that this symbol is not presently found in the Huffman tree. In that case, the symbol to be encoded is switched to the escape code, and its root node is located.

The encoding process for a Huffman tree works by starting at the leaf node and moving up through the parent nodes one at a time, until the root is reached. In a conventional Huffman tree, there will usually be two child nodes for each parent, one that encodes a 0 bit and another that encodes a 1 bit. The data structures used in this program take advantage of the sibling property by always grouping the two children of a parent node next to one another in the node list.

Grouping children together saves space by requiring only a single child pointer instead of two. It also means that a child automatically knows whether it is the child that encodes a 1 or 0 by whether it is odd or even. In this program, the odd child is arbitrarily designated the 1, and the even is always the 0.

Given this information, the encoding process is accomplished without too many lines of C code. Starting at the leaf node, each bit is added to the cumulative Huffman code. Whether the bit is a 1 or a 0 determines whether the bit is odd or even. As each bit is encoded, the current_bit mask is shifted by one so the next bit encoded will appear in the next most significant position. A counter is also incremented that keeps track of how many bits have been accumulated in the output word so far.

code = 0;

current_bit = 1;

code_size = 0;

current_node = tree->leaf[ c ];

if ( current_node == -1 )

    current_node = tree->leaf[ ESCAPE ];

while ( current_node != ROOT_NODE  ) {

    if ( ( current_node & 1 ) == 0 )

       code | = current_bit;

    current_bit <<= 1;

    code_size++;

    current_node = tree->nodes[ current_node ].parent;

};

OutputBits( output, code, code_size );

if ( tree->leaf[ c ] == -1 ) {

    OutputBits( output, c, 8 );

    add_new_node( tree, c );

}

After the bits of the Huffman code have been accumulated in the code word, the utility routine OutputBits (found in BITIO.C) is called to send out the code. There is still one piece of work left to do, however, before returning. If the code that was just output was the escape code, we have to handle the special condition created when a previously unreferenced symbol is encountered.

The first step taken after the escape code is sent is simple. The new symbol is output in an unencoded fashion, just as it was read in from the file. This lets the decoder know what symbol to add to the table. The second step is to add the new node to the Huffman tree. When the new node is added to the tree, it is inserted with a weight of 0. The 0-weight node will be incremented later when the model is updated, so it will not be 0 for long. The advantage to adding a node with a weight of zero is that it can be done without having to worry about updating any other nodes or, worse, changing the shape of the tree.

Using the sibling property definitions, we know that if the new node has a weight of 0, it will be the very first node in the list, since nodes are ranked in order of weight. We add the node to the table by finding the presently lowest-weight node and break it out into two new nodes. The old lowest-weight node will be one of the two new nodes, and the new 0-weight node will be the other one.

Figure 4.10 illustrates how this process modifies a working tree. The Huffman tree has already had the A, B, C, and D nodes defined with nonzero weights. The A node has the lowest count and consequently is at the start of the list (remember that the list in this program has the highest weights at 0). When symbol E is going to be added to the tree, we first identify A as the node at the end of the list. The position A used to occupy is converted to an internal node, and two new nodes are created. Since E is guaranteed to have the lowest weight in the tree, it is set to be the first node in the list, and A is set to be the second node.

Figure 4.10 The Huffman tree before and after addition of a zero weight node.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The code needed to add the node is listed below. The first step is to find the lowest-weight node, which in the example was A. The new_node variable is the node which A will occupy, and the 0_weight_node is where E will go. Since these are two new nodes, they are set to be the next_free_node and next_free_node+1. After this is done, next_free_node is incremented by two so it will be ready for the next operation.

lightest_node = tree->next_free_node - 1;

new_node = tree->next_free_node;

zero_weight_node = tree->next_free_node + 1;

tree->next_free_node += 2;

tree->nodes[ new_node ] = tree->nodes[ lightest_node ];

tree->nodes[ new_node ].parent = lightest_node;

tree->leaf[ tree->nodes[ new_node ].child] = new_node;

tree->nodes[ lightest_node ].child = new_node;

tree->nodes[ lightest_node ].child_is_leaf = FALSE;

tree->nodes[ zero_weight_node ].child          = c;

tree->nodes[ zero_weight_node ].child_is_leaf  = TRUE;

tree->nodes[ zero_weight_node ].weight         = 0;

tree->nodes[ zero_weight_node ].parent         = lightest_node;

tree->leaf[ c ] = zero_weight_node;

After the two new nodes are created, a few more bookkeeping details are needed to link the two new children to their parent, to make sure their weights are correct, and to make sure the leaf array points to the correct nodes. After that, the routine is done, and the tree is ready to be updated.

Updating the Tree

The most complicated part of the Adaptive Huffman program is in the routines called to update the tree. Updating the model is simply a matter of incrementing a symbol weight by one and taking care of all the side effects of that action. Taking care of the side effects, however, involves some strenuous effort.

if ( tree->nodes[ ROOT_NODE].weight == MAX_WEIGHT )

   RebuildTree( tree );

current_node = tree->leaf[ c ];

while ( current_node != -1 ) {

   tree->nodes[ current_node ].weight++;

   for ( new_node = current_node ; new_node > ROOT_NODE ;

          new_node—– )

       if ( tree->nodes[ new_node - 1 ].weight >=

             tree->nodes[ current_node ].weight )

            break;

   if ( current_node != new_node ) {

      swap_nodes( tree, current_node, new_node );

      current_node = new_node;

   }

   current_node = tree->nodes[ current_node ].parent;

}

This code performs all the work needed to update the tree. The first thing it checks for is to see if the tree has reached its maximum weight. If it has, the routine invokes the RebuildTree routine to scale down all the counts.

After getting past the tree-rebuilding step, the loop that increments the node weights is entered. The first node to be incremented is the leaf node associated with the symbol. The loop increments the weight of that symbol, then moves up to the parent to increment that node. This process continues till the root is reached, at which point the update is done. The single symbol added to statistical base forming the tree has been accounted for.

There is one additional step inside the loop, however, that takes place after every node has its weight incremented. The loop immediately following the increment operation works its way back through the list of nodes to see if the increased weight of the current node means it has to move up in the list. After the loop exits, new_node has the proper new location for the current node in the node list. If new_node is the same as current_node, the incremented node is fine where it is, and we can move on to the parent node. But if new_node and current_node aren't the same, they have to be swapped.

The process of swapping nodes involves lifting two entire subtrees out of their present positions in the list and exchanging them. The use of a tree data structure makes this easier than it may first appear. The swapping process is straightforward, complicated only by the fact that each node being swapped has links to both its parent and child, and the parent and child each have links to the node. This takes no great conceptual leaps. The swapping code is illustrated:

struct node temp;

     if ( tree->nodes[ i ].child_is_leaf )

        tree->leaf[ tree->nodes [ i ].child ] = j;

     else {

        tree->nodes[ tree->nodes [ i ].child ].parent = j;

        tree->nodes[ tree->nodes [ i ].child + 1 ].parent = j;

     }

     if ( tree->nodes[ j ].child_is_leaf )

        tree->leaf[ tree->nodes [ j ].child ] = i;

     else {

        tree->nodes[ tree->nodes[ j ].child ].parent = i;

        tree->nodes[ tree->nodes[ j ].child + 1 ].parent = i;

     }

     temp = tree->nodes[ i ];

     tree->nodes[ i ] = tree->nodes[ j ];

     tree->nodes[ i ].parent = temp.parent;

     temp.parent = tree->nodes[ j ].parent;

     tree->nodes[ j ] = temp;

An update can also force the rebuilding of the tree. Rebuilding takes a fair amount of work, unfortunately, since it amounts to building a new Huffman tree.

The rebuilding process proceeds in three discrete steps. The first step is to collect all the leaf nodes, throw away all the internal nodes, and divide the leaf-node weights by 2. The node list is compacted so that the new leaf nodes are all at the start of the list.

j = tree->next_free_node - 1;

for ( i = j ; i >= ROOT_NODE ; i–– ) {

    if ( tree->nodes[ i ].child_is_leaf ) {

      tree->nodes[ j ] = tree->nodes[ i ];

      tree->nodes[ j ].weight = (tree->nodes[ j ].weight + 1 ) / 2;

      j––;

    }

}

The code to do this is shown above. Note that in this implementation, none of the nodes scale down to zero. This is accomplished by adding 1 to the node before dividing it by two. It may be desirable to throw away infrequently seen symbols by reducing their counts to zero and deleting them from the list, but we don't do that here.

What we end up with in the above code is a list of leaf nodes that are at the start of the list, terminating at the next_free_node index. The internal nodes which start at 0 and end at the current value of j, will now be rebuilt in the next step.

In Chapter 3, we built a Huffman tree by repeatedly scanning the node list and finding the two nodes with the lowest weight. Those two nodes would be combined to form a new internal node. The tree-rebuilding phase here takes a different approach based on the sibling property.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The loop that creates internal nodes starts with j pointing to the next node that needs to be defined; i is an index that points to the next pair of nodes to be combined into an internal node. The loop progressively steps through the nodes, combining and adding new internal nodes until reaching the root node at location 0.

The process of creating the new internal node is simple. The new node, located at index j, is assigned a weight. The weight is simply the sum of the two nodes at location i, as would be expected. The hard work comes next. After node j is created, we have to decide where it belongs in the node list. The decision on where the new node belongs is made based on the weights of the nodes in the list. Recall that the sibling property says that the nodes have to be listed based on increasing weight. We have to step through the list till we find the first node that is less that the new node j, then place the new node immediately adjacent to that node in the list. Here is the procedure for the correct location for j:

for ( i = tree->next_free_node - 2 ; j >= ROOT_NODE ;

     i -= 2, j–– ) {

   k = i + 1;

   tree->nodes[ j ].weight = tree->nodes[ i ].weight +

                              tree->nodes[ k ].weight;

   weight = tree->nodes[ j ].weight;

   tree->nodes[ j ].child_is_leaf = FALSE;

       for ( k = j + 1 ; weight < tree->nodes[ k ].weight ; k++)

          ;

   k––;

  memmove( &tree->nodes[ j ]. &tree->nodes[ j + 1 ],

             ( k - j ) * sizeof( struct node ) );

   tree->nodes[ k ].weight = weight;

   tree->nodes[ k ].child = i;

   tree->nodes[ k ].child_is_leaf = FALSE;

}

The variable 'weight' is assigned the weight of the new internal node. The routine then loops back through the list until it finds the first node with a weight less than or equal to the weight of j. Node j will be placed immediately after that node in the list. Before that node can be positioned, we need to make room in the list by moving all the nodes that have higher weights up by one position. This is done with the memmove() function. After that, the new node has its child and weight assigned, and the loop continues.

This process can be seen in the short list of nodes about to be rebuilt in Figure 4.11. After having been rescaled, symbols A, B, C, and D have weights of 1, 3, 5, and 7 respectively. After the internal nodes have been stripped out, the list of nodes looks like Figure 4.11.

Figure 4.11 The list of nodes after the internal nodes have been stripped.

The first two nodes to be combined will be B and A, creating a new node, j, at location 2 in the table. By stepping back through the list from location 2, we see that the resulting internal node belongs between locations 4 and 5, right before B. After the memory move function is executed and the node is connected, the partial Huffman tree looks like this:

Figure 4.12 A partial Huffman tree after the memory move function is executed.

Because we are moving nodes around so freely, we do not assign parent pointers during this process. Once a node has been assigned as a child to another node, it is not going to change position in the list. But parent nodes that have yet to be combined together as children of another internal node may be moved farther up in the list as other nodes are combined. If we assigned parent nodes earlier in the process, every time we moved a node we would have to go through a backtracking procedure to locate its children and update their parent indices. We bypass this costly procedure by deferring parent node building to the third and final step in the Rebuild procedure. Assigning parent nodes is fairly simple. We start at the root node, and locate the children of each node in the tree. The children then have their parent node index set. If the child is a leaf instead of another node, the leaf[] array node index is set.

for ( i = tree->next_free_node - 1 ; i >= ROOT_NODE ; i-- ) {

  if ( tree->nodes[ i ].child_is_leaf ) {

    k = tree->nodes[ i ].child;

    tree->leaf[ k ] = i;

  } else {

    k = tree->nodes[ i ].child;

    tree->nodes[ k ].parent = tree->nodes[ k + 1 ].parent = i;

  }

}

Decoding the Symbol

The final high-level procedure to be discussed here is the routine used to decode an incoming symbol. Like the symbol encoder, this routine is short and simple. It starts at the root node of the tree and reads in a single bit at a time. As each bit is read in, one of the two children of the node is selected based on whether the input bit is a one or a zero. Eventually, this leads the routine to a leaf node.

When the leaf node is reached, we have decoded a symbol. The only possible complication at this point is if the decoded symbol is an escape code. If so, the symbol encoded by the encoder did not yet appear in the Huffman tree. This means that the next eight bits in the stream will contain an unencoded version of the symbol. If this is the case, the input routine is called to get the plain-text version of the symbol.

current_node = ROOT_NODE;

while ( !tree->nodes[ current_node ].child_is_leaf ) {

  current_node = tree->nodes[ current_node ].child;

  current_node += InputBit( input );

}

c = tree->nodes[ current_node ].child;

if ( c == ESCAPE ) {

  c = (int) InputBits( input, 8 );

  add_new_node( tree, c );

}

return( c );

Either the decoded symbol or the escaped plain version of the symbol is passed back to the calling program where it can be placed in the output file. The hard work will come after the symbol is decoded, when the tree has to be updated to reflect the newly arrived symbol.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Code

The code for the Adaptive Huffman compressor is listed next. This single module is contained on the source disk in the file AHUFF.C. Building the compression routine requires that you compile this file and link it with the utility routines discussed in the last chapter: BITIO.C, ERRHAND.C, and MAIN-C.C. To build the decompression routine you substitute MAIN-E.C. The two arguments for both programs are simply an input file followed by an output file.

/*************************** Start of AHUFF.C *************************/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#include "bitio.h"

#include "errhand.h"

char *CompressionName = "Adaptive Huffman coding, with escape codes";

char *Usage           = "infile outfile";

#define END_OF_STREAM      256

#define ESCAPE             257

#define SYMBOL_COUNT       258

#define NODE_TABLE_COUNT   ( ( SYMBOL_COUNT * 2 ) - 1 )

#define ROOT_NODE          0

#define MAX_WEIGHT         0X8000

#define TRUE               1

#define FALSE              0

*/

* This data structure is all that is needed to maintain an adaptive

* Huffman tree for both encoding and decoding.  The leaf array is a

* set of indices into the nodes that indicate which node is the

* parent of a symbol.  For example, to encode 'A', we would find the

* leaf node by way of leaf[ 'A' ].  The next_free_node index is used

* to tell which node is the next one in the array that can be used.

* Since nodes are allocated when characters are read in for the first

* time, this pointer keeps track of where we are in the node array.

* Finally, the array of nodes is the actual Huffman tree.  The child

* index is either an index pointing to a pair of children, or an

* actual symbol value, depending on whether 'child_is_leaf' is true

* or false.

*/

typedef struct tree {

  int leaf[ SYMBOL_COUNT ];

  int next_free_node;

  struct node {

    unsigned int weight;

    int parent;

    int child_is_leaf;

    int child;

  } nodes [ NODE_TABLE_COUNT ];

} TREE;

/*

* The Tree used in this program is a global structure.  Under other

* circumstances it could just as well be a dynamically allocated

* structure built when needed, since all routines here take a TREE

* pointer as an argument.

*/

TREE Tree;

/*

* Function prototypes for both ANSI C compilers and their K&R breth-

* ren.

*/

#ifdef __STDC__

void CompressFile( FILE *input, BIT_FILE *output, int argc,

                   char *argv[] );

void ExpandFile( BIT_FILE *input, FILE *output, int argc,

                 char *argv[] );

void InitializeTree( TREE *tree );

void EncodeSymbol( TREE *tree, unsigned int c, BIT_FILE *output );

int DecodeSymbol( TREE *tree, BIT_FILE *input );

void UpdateModel( TREE *tree, int c );

void RebuildTree( TREE *tree );

void swap_nodes( TREE *tree, int i, int j );

void add_new_node( TREE *tree, int c );

void PrintTree( TREE *tree );

void print_codes( TREE *tree );

void print_code( TREE *tree, int c );

void calculate_rows( TREE *tree, int node, int level );

int calculate_columns( TREE *tree, int node, int starting_guess );

int find_minimum_column( TREE *tree, int node, int max_row );

void rescale_columns( int factor );

#else

void CompressFile();

void ExpandFile();

void InitializeTree();

void EncodeSymbol();

int DecodeSymbol();

void UpdateModel();

void RebuildTree();

void swap_nodes();

void add_new_node();

#endif

/*

* The high level view of the compression routine is very simple.

* First, we initialize the Huffman tree, with just the ESCAPE and

* END_OF_STREAM symbols.  Then, we sit in a loop, encoding symbols,

* and adding them to the model.  When there are no more characters

* to send, the special END_OF_STREAM symbol is encoded.  The decoder

* will later be able to use this symbol to know when to quit.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

  int c;

  InitializeTree( &Tree );

  while ( ( c = getc( input ) ) != EOF ) {

    EncodeSymbol( &Tree, c, output );

    UpdateModel( &Tree, c );

  }

  EncodeSymbol( &Tree, END_OF_STREAM, output );

  while ( argc— > 0 ) {

    printf( "Unused argument:  %s\n", *argv ); argv++;

   }

}

/*

* The Expansion routine looks very much like the compression routine.

* It first initializes the Huffman tree, using the same routine as

* the compressor did.  It then sits in a loop, decoding characters and

* updating the model until it reads in an END_OF_STREAM symbol.  At

* that point, it is time to quit.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

     int c;

     while ( argc–– > 0 )

       printf( "Unused argument:  %s\n", *argv++ );

     InitializeTree( &Tree );

     while ( ( c = DecodeSymbol( &Tree, input ) ) != END OF STREAM ) {

       if ( putc( c, output ) == EOF )

         fatal_error( "Error writing character" );

       UpdateModel( & Tree, c );

     }

}

/*

* When performing adaptive compression, the Huffman tree starts out

* very nearly empty.  The only two symbols present initially are the

* ESCAPE symbol and the END_OF_STREAM symbol.  The ESCAPE symbol has to

* be included so we can tell the expansion program that we are

* transmitting a previously unseen symbol.  The END_OF_STREAM symbol

* is here because it is greater than eight bits, and our ESCAPE

* sequence only allows for eight bit symbols following the ESCAPE

* code.

*

* In addition to setting up the root node and its two children, this

* routine also initializes the leaf array.  The ESCAPE and

* END_OF_STREAM leaves are the only ones initially defined, the rest

* of the leaf elements are set to -1 to show that they aren't present

* in the Huffman tree yet.

*/

void InitializeTree( tree )

TREE *tree;

{

     int i;

     tree->nodes[ ROOT_NODE ].child           = ROOT_NODE + 1;

     tree->nodes[ ROOT_NODE ].child_is_leaf   = FALSE;

     tree->nodes[ ROOT_NODE ].weight          = 2;

     tree->nodes[ ROOT_NODE ].parent          = -1;

     tree->nodes[ ROOT_NODE + 1 ].child           = END_OF_STREAM;

     tree->nodes[ ROOT_NODE + 1 ].child_is_leaf   = TRUE;

     tree->nodes[ ROOT_NODE + 1 ].weight          = 1;

     tree->nodes[ ROOT_NODE + 1 ].parent          = ROOT_NODE;

     tree->leaf[ END_OF_STREAM ]                      = ROOT_NODE + 1;

     tree->nodes[ ROOT_NODE + 2 ].child           = ESCAPE;

     tree->nodes[ ROOT_NODE + 2 ].child_is_leaf   = TRUE;

     tree->nodes[ ROOT_NODE + 2 ].weight          = 1;

     tree->nodes[ ROOT_NODE + 2 ].parent          = ROOT_NODE;

     tree->leaf[ ESCAPE ]                         = ROOT_NODE + 2;

     tree->next_free_node                         = ROOT_NODE + 3;

     for ( i = 0 ; i < END_OF_STREAM ; i++ )

         tree->leaf[ i ] = -1; }

/*

* This routine is responsible for taking a symbol, and converting

* it into the sequence of bits dictated by the Huffman tree.  The

* only complication is that we are working our way up from the leaf

* to the root, and hence are getting the bits in reverse order.  This

* means we have to rack up the bits in an integer and then send them

* out after they are all accumulated.  In this version of the program,

* we keep our codes in a long integer, so the maximum count is set

* to an arbitrary limit of 0x8000.  It could be set as high as 65535

* if desired.

*/

void EncodeSymbol( tree, c, output )

TREE *tree;

unsigned int c;

BIT_FILE *output;

{

     unsigned long code;

     unsigned long current_bit;

     int code_size;

     int current_node;

     code = 0;

     current_bit = 1;

     code_size = 0;

     current node = tree->leaf[ c ];

     if ( current node == -1 )

       current_node = tree->leaf[ ESCAPE ];

     while ( current_node != ROOT_NODE ) {

       if ( ( current_node & 1 ) == 0 )

         code | = current_bit;

       current_bit <<= 1;

       code_size++;

       current_node = tree->nodes[ current_node ].parent;

     };

     OutputBits( output, code, code_size );

     if ( tree->leaf[ c ] == -1 ) {

       OutputBits( output, (unsigned long) c, 8 );

       add_new_node( tree, c );

     }

}

/*

* Decoding symbols is easy.  We start at the root node, then go down

* the tree until we reach a leaf.  At each node, we decide which

* child to take based on the next input bit.  After getting to the

* leaf, we check to see if we read in the ESCAPE code.  If we did,

* it means that the next symbol is going to come through in the next

* eight bits, unencoded.  If that is the case, we read it in here,

* and add the new symbol to the table.

*/

int DecodeSymbol( tree, input )

TREE *tree;

BIT_FILE *input;

{

     int current_node;

     int c;

     current_node = ROOT_NODE;

     while ( !tree->nodes[ current_node ].child_is_leaf ) {

       current_node = tree->nodes[ current_node ].child;

       current_node += InputBit( input );

     }

     c = tree->nodes[ current_node ].child;

     if ( c == ESCAPE ) {

       c = (int) InputBits( input, 8 );

       add_new_node( tree, c );

     }

     return( c );

}

/*

* UpdateModel is called to increment the count for a given symbol.

* After incrementing the symbol, this code has to work its way up

* through the parent nodes, incrementing each one of them.  That is

* the easy part.  The hard part is that after incrementing each

* parent node, we have to check to see if it is now out of the proper

* order.  If it is, it has to be moved up the tree into its proper

* place.

*/

void UpdateModel( tree, c )

TREE *tree;

int c;

{

     int current_node;

     int new_node;

     if ( tree->nodes[ ROOT_NODE].weight == MAX_WEIGHT )

       RebuildTree( tree );

     current_node = tree->leaf[ c ];

     while ( current_node != -1 ) {

       tree->nodes[ current_node ].weight++;

       for ( new_node = current_node ; new_node > ROOT_NODE ;

                         new_node–– )

         if ( tree->nodes[ new_node - 1 ].weight >=

            tree->nodes[ current_node ].weight )

            break;

       if ( current_node != new_node ) {

         swap_nodes( tree, current_node, new_node );

         current_node = new_node;

       }

       current_node = tree->nodes[ current_node ].parent;

     }

}

/*

* Rebuilding the tree takes place when the counts have gone too

* high.  From a simple point of view, rebuilding the tree just means

* that we divide every count by two.  Unfortunately, due to truncation

* effects, this means that the tree's shape might change.  Some nodes

* might move up due to cumulative increases, while others may move

* down.

*/

void RebuildTree( tree )

TREE *tree;

{

     int i;

     int j;

     int k;

     unsigned int weight;

/*

* To start rebuilding the table, I collect all the leaves of the

* Huffman tree and put them in the end of the tree.  While I am doing

* that, I scale the counts down by a factor of 2.

*/

     printf( "R" );

     j = tree->next_free_node - 1;

     for ( i = j ; i >= ROOT_NODE ; i–– ) {

       if ( tree->nodes[ i ].child_is_leaf ) {

         tree->nodes[ j ] = tree->nodes[ i ];

         tree->nodes[ j ].weight = ( tree->nodes[ j ].weight + 1 ) / 2;

         j––;

       }

     }

/*

* At this point, j points to the first free node.  I now have all the

* leaves defined, and need to start building the higher nodes on the

* tree.  I will start adding the new internal nodes at j.  Every time

* I add a new internal node to the top of the tree, I have to check

* to see where it really belongs in the tree.  It might stay at the

* top, but there is a good chance I might have to move it back down.

* If it does have to go down, I use the memmove() function to scoot

* everyone bigger up by one node.

*/

     for ( i = tree->next_free_node - 2 ; j >= ROOT_NODE;

           i -= 2, j–– ) {

       k = i + 1;

       tree->nodes[ j ].weight = tree->nodes[ i ].weight +

                                 tree->nodes[ k ].weight;

       weight = tree->nodes[ j ].weight;

       tree->nodes[ j ].child_is_leaf = FALSE;

       for ( k = j + 1 ; weight < tree->nodes[ k ].weight ; k++ )

         ;

       k––;

       memmove( &tree->nodes[ j ], &tree->nodes[ j + 1 ],

              ( k - j ) * sizeof( struct node ) );

       tree->nodes[ k ].weight = weight;

       tree->nodes[ k ].child = i;

       tree->nodes[ k ].child_is_leaf = FALSE;

     }

/*

* The final step in tree reconstruction is to go through and set up

* all of the leaf and parent members.  This can be safely done now

* that every node is in its final position in the tree.

*/

   for ( i = tree->next_free_node - 1 ; i >= ROOT_NODE ; i–– ) {

     if ( tree->nodes[ i ].child_is_leaf ) {

       k = tree->nodes[ i ].child;

       tree->leaf[ k ] = i;

     } else {

       k = tree->nodes[ i ].child;

       tree->nodes[ k ].parent = tree->nodes[ k + 1 ].parent = i;

     }

   }

}

/*

* Swapping nodes takes place when a node has grown too big for its

* spot in the tree.  When swapping nodes i and j, we rearrange the

* tree by exchanging the children under i with the children under j.

void swap_nodes( tree, i, j )

TREE *tree;

int i;

int j;

{

     struct node temp;

     if ( tree->nodes [ i ].child_is_leaf )

          tree->leaf[ tree->nodes[ i ].child ] = j;

     else {

          tree->nodes[ tree->nodes[ i ].child ].parent = j;

          tree->nodes[ tree->nodes[ i ].child + 1 ].parent = j;

     }

     if ( tree->nodes[ j ].child_is_leaf )

          tree->leaf[ tree->nodes[ j ].child ] = i;

     else {

          tree->nodes[ tree->nodes[ j ].child ].parent = i;

          tree->nodes[ tree->nodes[ j ].child + 1 ].parent = i;

     }

     temp = tree->nodes[ i ];

     tree->nodes[ i ] = tree->nodes[ j ];

     tree->nodes[ i ].parent = temp.parent;

     temp.parent = tree->nodes[ j ].parent;

     tree->nodes[ j ] = temp;

}

/*

* Adding a new node to the tree is pretty simple.  It is just a matter

* of splitting the lightest-weight node in the tree, which is the

* highest valued node.  We split it off into two new nodes, one of

* which is the one being added to the tree.  We assign the new node a

* weight of 0, so the tree doesn't have to be adjusted.  It will be

* updated later when the normal update process occurs.  Note that this

* code assumes that the lightest node has a leaf as a child.  If this

* is not the case, the tree would be broken.

*/

void add_new_node( tree, c )

TREE *tree;

int c;

{

   int lightest_node;

   int new_node;

   int zero_weight_node;

   lightest_node = tree->next_free_node - 1;

   new_node = tree->next_free_node;

   zero_weight_node = tree->next_free_node + 1;

   tree->next_free_node += 2;

   tree->nodes[ new_node ] = tree->nodes[ lightest_node ];

   tree->nodes[ new_node ].parent = lightest_node;

   tree->leaf[ tree->nodes[ new_node ].child ]    = new_node;

   tree->nodes[ lightest_node ].child             = new_node;

   tree->nodes[ lightest_node ].child_is_leaf     = FALSE;

   tree->nodes[ zero_weight_node ].child           = c;

   tree->nodes[ zero_weight_node ].child_is_leaf   = TRUE;

   tree->nodes[ zero_weight_node ].weight          = 0;

   tree->nodes[ zero_weight_node ].parent          = lightest_node;

/************************** End of AHUFF.C *****************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 5
Huffman One Better: Arithmetic Coding

The last two chapters show that Huffman coding uses knowledge about information content to efficiently encode symbols. If the probability of a symbol's appearance in a message is known, Huffman techniques can encode that symbol using a minimal number of bits. Huffman coding has been proven the best fixed-length coding method available.

Difficulties

Huffman codes have to be an integral number of bits long, and this can sometimes be a problem. If the probability of a character is 1/3, for example, the optimum number of bits to code that character is around 1.6 bits. Huffman coding has to assign either one or two bits to the code, and either choice leads to a longer compressed message than is theoretically possible.

This non optimal coding becomes a noticeable problem when the probability of a character is very high. If a statistical method could assign a 90 percent probability to a given character, the optimal code size would be 0.15 bits. The Huffman coding system would probably assign a 1-bit code to the symbol, which is six times longer than necessary.

This would be a problem when compressing two-color images, like those coming from a fax machine. Since there are only two colors, an ordinary coding method would assign the 1 bit to one color and the 0 bit to the other. Since both codes have only a single bit, Huffman coding is not going to be able to compress this data at all. No matter how high the probability of one of the bits, we are still going to have to encode it using one bit.

The conventional solution to this problem is to group the bits into packets and apply Huffman coding. But this weakness prevents Huffman coding from being a universal compressor.

Arithmetic Coding: A Step Forward

Only in the last fifteen years has a respectable candidate to replace Huffman coding been successfully demonstrated: arithmetic coding. Arithmetic coding bypasses the idea of replacing an input symbol with a specific code. It replaces a stream of input symbols with a single floating-point output number. More bits are needed in the output number for longer, complex messages. This concept has been known for some time, but only recently were practical methods found to implement arithmetic coding on computers with fixed-sized registers.

The output from an arithmetic coding process is a single number less than 1 and greater than or equal to 0. This single number can be uniquely decoded to create the exact stream of symbols that went into its construction. To construct the output number, the symbols are assigned a set probabilities. The message "BILL GATES," for example, would have a probability distribution like this:

Character | Probability
SPACE | 1/10
A | 1/10
B | 1/10
E | 1/10
G | 1/10
I | 1/10
L | 2/10
S | 1/10
T | 1/10

Once character probabilities are known, individual symbols need to be assigned a range along a "probability line," nominally 0 to 1. It doesn't matter which characters are assigned which segment of the range, as long as it is done in the same manner by both the encoder and the decoder. The nine-character symbol set used here would look like the following:

Character | Probability | Range
SPACE | 1/10 | 0.00 [gte] r > 0.10
A | 1/10 | 0.10 [gte] r > 0.20
B | 1/10 | 0.20 [gte] r > 0.30
E | 1/10 | 0.30 [gte] r > 0.40
G | 1/10 | 0.40 [gte] r > 0.50
I | 1/10 | 0.50 [gte] r > 0.60
L | 2/10 | 0.60 [gte] r > 0.80
S | 1/10 | 0.80 [gte] r > 0.90
T | 1/10 | 0.90 [gte] r > 1.00


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Each character is assigned the portion of the 0 to 1 range that corresponds to its probability of appearance. Note that the character "owns" everything up to, but not including, the higher number. So the letter T in fact has the range .90 to .9999…

The most significant portion of an arithmetic-coded message belongs to the first symbols—or B, in the message "BILL GATES." To decode the first character properly, the final coded message has to be a number greater than or equal to .20 and less than .30. To encode this number, track the range it could fall in. After the first character is encoded, the low end for this range is .20 and the high end is .30.

During the rest of the encoding process, each new symbol will further restrict the possible range of the output number. The next character to be encoded, the letter I, owns the range .50 to .60 in the new subrange of .2 to .3. So the new encoded number will fall somewhere in the 50th to 60th percentile of the currently established range. Applying this logic will further restrict our number to .25 to .26. The algorithm to accomplish this for a message of any length is shown here:

low = 0.0;

high = 1.0;

while ( ( c = getc( input ) ) != EOF ) {

    range = high - low;

    high = low + range * high_range( c );

    low = low + range * low_range( c );

}

output ( low );

Following this process to its natural conclusion with our message results in the following table:

New Character | Low value | High Value
 | 0.0 | 1.0
B | 0.2 | 0.3
I | 0.25 | 0.26
L | 0.256 | 0.258
L | 0.2572 | 0.2576
SPACE | 0.25720 | 0.25724
G | 0.257216 | 0.257220
A | 0.2572164 | 0.2572168
T | 0.25721676 | 0.2572168
E | 0.257216772 | 0.257216776
S | 0.2572167752 | 0.2572167756

So the final low value, .2572167752, will uniquely encode the message "BILL GATES" using our present coding scheme.

Given this encoding scheme, it is relatively easy to see how the decoding process operates. Find the first symbol in the message by seeing which symbol owns the space our encoded message falls in. Since .2572167752 falls between .2 and .3, the first character must be B. Then remove B from the encoded number. Since we know the low and high ranges of B, remove their effects by reversing the process that put them in. First, subtract the low value of B, giving .0572167752. Then divide by the width of the range of B, or .1. This gives a value of .572167752. Then calculate where that lands, which is in the range of the next letter, I. The algorithm for decoding the incoming number is shown next:

number = input_code();

for ( ; ; ) {

    symbol = find_symbol_straddling_this_range( number );

    putc( symbol );

    range = high_range( symbol ) - low_range( symbol );

    number = number - low_range( symbol );

    number = number / range;

}

I have conveniently ignored the problem of how to decide when there are no more symbols left to decode. This can be handled either by encoding a special end-of-file symbol or by carrying the stream length with the encoded message. In the example in this chapter, as elsewhere in the book, I carry along a special end-of-stream symbol that tells the decoder when it is out of symbols. The decoding algorithm for the "BILL GATES" message will proceed as shown:

Encoded Number | Output Symbol | Low | High | Range
0.2572167752 | B | 0.2 | 0.3 | 0.1
0.572167752 | I | 0.5 | 0.6 | 0.1
0.72167752 | L | 0.6 | 0.8 | 0.2
0.6083876 | L | 0.6 | 0.8 | 0.2
0.041938 | SPACE | 0.0 | .1 | 0.1
0.41938 | G | 0.4 | 0.5 | 0.1
0.1938 | A | 0.2 | 0.3 | 0.1
0.938 | T | 0.9 | 1.0 | 0.1
0.38 | E | 0.3 | 0.4 | 0.1
0.8 | S | 0.8 | 0.9 | 0.1
0.0 |  |  |  |

In summary, the encoding process is simply one of narrowing the range of possible numbers with every new symbol. The new range is proportional to the predefined probability attached to that symbol. Decoding is the inverse procedure, in which the range is expanded in proportion to the probability of each symbol as it is extracted.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Practical Matters

Encoding and decoding a stream of symbols using arithmetic coding is not too complicated. But at first glance it seems completely impractical. Most computers support floating-point numbers of around 80 bits. So do you have to start over every time you encode ten or fifteen symbols? Do you need a floating-point processor? Can machines with different floating-point formats communicate through arithmetic coding?

As it turns out, arithmetic coding is best accomplished using standard 16-bit and 32-bit integer math. Floating-point math is neither required nor helpful. What is required is an incremental transmission scheme in which fixed-size integer state variables receive new bits at the low end and shift them out at the high end, forming a single number that can be as long as necessary, conceivably millions or billions of bits.

Earlier, we saw that the algorithm works by keeping track of a high and low number that brackets the range of the possible output number. When the algorithm first starts, the low number is set to 0 and the high number is set to 1. The first simplification made to work with integer math is to change the 1 to .999 …, or .111… in binary. Mathematicians agree that .111… binary is exactly the same as 1 binary, and we take their assurance at face value. It simplifies encoding and decoding.

To store these numbers in integer registers, first justify them so the implied decimal point is on the left side of the word. Then load as much of the initial high and low values as will fit into the word size we are working with. My implementation uses 16-bit unsigned math, so the initial value of high is 0xFFFF, and low is 0. We know that the high value continues with Fs forever, and the low continues with zeros forever; so we can shift those extra bits in with impunity when needed.

If you imagine our "BILL GATES" example in a five-decimal digit register (I use decimal digits in this example for clarity), the decimal equivalent of our setup would look like what follows on the next page.

HIGH:  99999 implied digits => 999999999...

LOW:  00000 implied digits => 000000000...

To find the new range of numbers, apply the encoding algorithm from the previous section. First, calculate the range between the low and high values. The difference between the two registers will be 100000, not 99999. This is because we assume the high register has an infinite number of 9s added to it, so we need to increment the calculated difference. We then compute the new high value using the formula from the previous section:

high = low + high_range(symbol)

In this case, the high range was .30, which gives a new value for high of 30000. Before storing the new value of high, we need to decrement it, once again because of the implied digits appended to the integer value. So the new value of high is 29999. The calculation of low follows the same procedure, with a resulting new value of 20000. So now high and low look like this:

high:  29999 (999...)

low:  20000 (000...)

At this point, the most significant digits of high and low match. Due to the nature of our algorithm, high and low can continue to grow closer together without quite ever matching. So once they match in the most significant digit, that digit will never change. We can now output that digit as the first digit of our encoded number. This is done by shifting both high and low left by one digit and shifting in a 9 in the least significant digit of high. The equivalent operations are performed in binary in the C implementation of this algorithm.

As this process continues, high and low continually grow closer together, shifting digits out into the coded word. The process for our "BILL GATES" message is shown next.

 | High | Low | Range | Cumulative Output
Initial state | 99999 | 00000 | 100000 |
Encode B (0.2—0.3) | 29999 | 20000 |  |
Shift out 2 | 99999 | 00000 | 10000 | .2
Encode I (0.5—0.6) | 59999 | 50000 |  | .2
Shift out 5 | 99999 | 00000 | 100000 | .25
Encode L (0.6—0.8) | 79999 | 60000 | 20000 | .25
Encode L (0.6—0.8) | 75999 | 72000 |  | .25
Shift out 7 | 59999 | 20000 | 40000 | .257
Encode SPACE (0.0—0.1) | 23999 | 20000 |  | .257
Shift out 2 | 39999 | 00000 | 40000 | .2572
Encode G (0.4—0.5) | 19999 | 16000 |  | .2572
Shift out 1 | 99999 | 60000 | 40000 | .25721
Encode A (0.1—0.2) | 67999 | 64000 |  | .25721
Shift out 6 | 79999 | 40000 | 40000 | .257216
Encode T (0.9—1.0) | 79999 | 76000 |  | .257216
Shift out 7 | 99999 | 60000 | 40000 | .2572167
Encode E (0.3—0.4) | 75999 | 72000 |  | .2572167
Shift out 7 | 59999 | 20000 | 40000 | .25721677
Encode S (0.8—0.9) | 55999 | 52000 |  | .25721677
Shift out 5 | 59999 | 20000 |  | .257216775
Shift out 2 |  |  |  | .2572167752
Shift out 0 |  |  |  | .25721677520

After all the letters are accounted for, two extra digits need to be shifted out of either the high or low value to finish the output word. This is so the decoder can properly track the input data. Part of the information about the data stream is still in the high and low registers, and we need to shift that information to the file for the decoder to use later.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

A Complication

This scheme works well for incrementally encoding a message. Enough accuracy is retained during the double-precision integer calculations to ensure that the message is accurately encoded. But there is potential for a loss of precision under certain circumstances.

If the encoded word has a string of 0s or 9s in it, the high and low values will slowly converge on a value, but they may not see their most significant digits match immediately. High may be 700004, and low may be 699995. At this point, the calculated range will be only a single digit long, which means the output word will not have enough precision to be accurately encoded. Worse, after a few more iterations, high could be 70000, and low could be 69999.

At this point, the values are permanently stuck. The range between high and low has become so small that any iteration through another symbol will leave high and low at their same values. But since the most significant digits of both words are not equal, the algorithm can't output the digit and shift. It seems to have reached an impasse.

You can defeat this underflow problem by preventing things from ever getting that bad. The original algorithm said something like, "If the most significant digit of high and low match, shift it out." If the two digits don't match, but are now on adjacent numbers, a second test needs to be applied. If high and low are one apart, we test to see if the second most significant digit in high is a 0 and if the second digit in low is a 9. If so, it means we are on the road to underflow and need to take action.

Head off underflow with a slightly different shift operation. Instead of shifting the most significant digit out of the word, delete the second digits from high and low and shift the rest of the digits left to fill the space. The most significant digit stays in place. Then set an underflow counter to remember that we threw away a digit and aren't quite sure whether it was going to be a 0 or a 9. This process is shown here:

 | Before | After
High: | 40344 | 43449
Low: | 39810 | 38100
Underflow: | 0 | 1

After every recalculation, check for underflow digits again if the most significant digit don't match. If underflow digits are present, we shift them out and increment the counter.

When the most significant digits do finally converge to a single value, output that value. Then output the underflow digits previously discarded. The underflow digits will all be 9s or 0s, depending on whether high and low converged on the higher or lower value. In the C implementation of this algorithm, the underflow counter keeps track of how many ones or zeros to output.

Decoding

In the "ideal" decoding process, we had the entire input number to work with, and the algorithm had to do things like "divide the encoded number by the symbol probability." In practice, we can't perform an operation like that on a number that could be billions of bytes long. As in the encoding process, however, the decoder can operate using 16- and 32-bit integers for calculations.

Instead of using just two numbers, high and low, the decoder has to use three numbers. The first two, high and low, correspond exactly to the high and low values maintained by the encoder. The third number, code, contains the current bits being read in from the input bit stream. The code value always falls between the high and low values. As they come closer and closer to it, new shift operations will take place, and high and low will move back away from code.

The high and low values in the decoder will be updated after every symbol, just as they were in the encoder, and they should have exactly the same values. By performing the same comparison test on the upper digit of high and low, the decoder knows when it is time to shift a new digit into the incoming code. The same underflow tests are performed as well.

In the ideal algorithm, it was possible to determine what the current encoded symbol was just by finding the symbol whose probabilities enclosed the present value of the code. In the integer math algorithm, things are somewhat more complicated. In this case, the probability scale is determined by the difference between high and low. So instead of the range being between .0 and 1.0, the range will be between two positive 16-bit integer counts. Where the present code value falls along that range determines current probability. Divide (value - low) by (high - low + 1) to get the actual probability for the present symbol.

Where's the Beef?

It is not immediately obvious why this encoding process is an improvement over Huffman coding. It becomes clear when we examine a case in which the probabilities are a little different. If we have to encode the stream "AAAAAAA," and the probability of A is known to be .9, there is a 90 percent chance that any incoming character will be the letter A. We set up our probability table so that A occupies the .0 to . 9 range, and the end-of-message symbol occupies the .9 to 1 range. The encoding process is shown next:

New Character | Low value | High value
 | 0.0 | 1.0
A | 0.0 | 0.9
A | 0.0 | 0.81
A | 0.0 | 0.729
A | 0.0 | 0.6561
A | 0.0 | 0.59049
A | 0.0 | 0.531441
A | 0.0 | 0.4782969
END | 0.43046721 | 0.4782969

Now that we know the range of high and low values, all that remains is to pick a number to encode this message. The number .45 will make this message uniquely decode to "AAAAAAA." Those two decimal digits take slightly less than seven bits to specify, which means we have encoded eight symbols in less than eight bits! An optimal Huffman message would have taken a minimum of nine bits.

To take this point to an even further extreme, I set up a test that had only two symbols. In it, 0 had a 16382/16383 probability, and an end-of-file symbol had a 1/16383 probability. I then created a file filled with 100,000 0s. After compression using arithmetic coding, the output file was only three bytes long! The minimum size using Huffman coding would have been 12,501 bytes. This is obviously a contrived example, but it shows that arithmetic coding compresses data at rates much better than one bit per byte when the symbol probabilities are right.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Code

The code supplied with this chapter in ARITH.C is a simple module that performs arithmetic compression and decompression using a simple order 0 model. It works exactly like the non-adaptive Huffman coding program in Chapter 3. It first makes a single pass over the data, counting the symbols. The data is then scaled down to make the counts fit into a single, unsigned character. The scaled counts are saved to the output file for the decompressor to get at later, then the arithmetic coding table is built. Finally, the compressor passes through the data, compressing each symbol as it appears. When done, the end-of-stream character is sent out, the arithmetic coder is flushed, and the program exits.

The Compression Program

The compression portion of this program is shown shortly. The main module is called by the utility version of MAIN-E.C., which will have already taken care of opening files, parsing arguments, etc. Once we get to the compression phase of the program, things are ready to go.

The compressor code breaks down neatly into three sections. The first two lines initialize the model and the encoder. The while loop consists of two lines, which together with the line following the loop perform the compression, and the last three lines shut things down.

build_model( input, output->file );

initialize_arithmetic_encoder();

while ( ( c = getc( input ) ) ! = EOF ) {

  convert_int_to_symbol( c, &s );

  encode_symbol( output, &s );

}

convert_int_to_symbol( END_OF_STREAM, &s );

encode_symbol( output, &s );

flush_arithmetic_encoder( output );

OutputBits( output, OL, 16 );

The build_model() routine has several responsibilities. It makes the first pass over the input data to count all the characters. It scales down the counts to fit in unsigned characters, then it takes those counts and builds the range table used by the coder. Finally, it writes the counts to the output file so the decompressor will have access to them later.

The initialize arithmetic encoder routine is fairly simple. It just sets up the high- and low-integer variables used during the encoding. The encoding loop calls two different routines to encode the symbol. The first, convert_int_to_symbol(), takes the character read in from the file and looks up the range for the given symbol. The range is then stored in the symbol object, which has the structure shown:

typedef struct {

  unsigned short int low_count;

  unsigned short int high_count;

  unsigned short int scale;

} SYMBOL;

These three values are all that are needed for the symbol to be encoded using the arithmetic encoder. The low-count and high-count values uniquely define where on the 0 to 1 range the symbol lies, and the scale value tells what the total span of the 0 to 1 scale is. If 1,000 characters had been counted in a text file, for example, the low_count and high_count for A might be 178 and 199, and the scale would be 1,000. This would indicate that on the 0 to 1 number scale, A would own the range .178 to .199.

Once the symbol object has been defined, it can be passed to the encoder. The arithmetic encoder needs only those three items to process the symbol and update the output number. It has the high- and low-range values and the underflow count stored internally as static variables, and it doesn't need anything else.

The way we detached the modeling data from the coding data gives us a convenient mechanism for experimenting with new ways of modeling. We just have to come up with the numbers that get plugged into the symbol. The encoder doesn't care how we got those numbers as long as they were derived consistently so we can decode the file later.

When we reach the end of the input file, we encode and send the end-of-stream symbol. The decompression program will use it to determine when it is done. To finish, call a routine to flush the arithmetic encoder, which takes care of any underflow bits. Finally, we have to output an extra sixteen bits. The reason for this is simple. When decoding symbols from the input bit stream, the effective bits are in the most significant bit position of the input bit registers. To get the bits there, we have to load other bits into the lower positions and shift them over. At least 15 insignificant bits are needed to decode the last symbol. Outputting 16 bits at the end of the program ensures that the decoder won't get a premature end of file while decoding the input file.

The Expansion Program

The main part of the expansion program follows the same pattern. First, the model is set up and the arithmetic coder is initialized. In this program, initializing the model means reading in the counts from the input file where the compressor put them. Initializing the arithmetic decoder means loading the low and high registers with 0000 and FFFF, then reading the first 16 bits from the input file into the current code.

input_counts( input->file );

initialize_arithmetic_decoder( input );

for ( ; ; ) {

    get_symbol_scale( &s );

    count = get_current_count( &s );

    c = convert_symbol_to_int( count, &s );

    if ( c == END_OF_STREAM )

       break;

    remove_symbol_from_stream( input, &s );

    putc( (char) c, output );

}

The decoding loop is a little more complicated in this routine to keep the modeling and decoding separate. First, get the scale for the current model to pass back to the arithmetic decoder. The decoder then converts its current input code into a count in the routine get_current_count. With the count, we can determine which symbol is the correct one to decode. This is done in the routine convert_symbol_to_int().

Though it seems strange, we don't remove the encoded symbol from the input bit stream till after we actually decode it. The process of removing it involves standard modifications of high and low and, if necessary, shifting in new bits from the input stream. Finally, the decoded character is written to the output file.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Initializing the Model

The model for a program using arithmetic coding needs to have three pieces of information for each symbol: the low end of its range, the high end of its range, and the scale of the entire alphabet's range (this is the same for all symbols in the alphabet). Since the top of a given symbol's range is the bottom of the next, we only need to keep track of N + 1 numbers for N symbols in the alphabet.

An example of how the range information would be created is shown for symbols A, B, C, D, and E. These symbols have the counts 5, 2, 3, 3, and 8 respectively. The numbers can be arranged in any order along the range, if done consistently.

Range | : | 21
E | : | 13
D | : | 10
C | : | 7
B | : | 5
A | : | 0

In this case, the alphabet has five symbols, and the number of counts to keep track of is six. The array is formed by creating the cumulative total at each symbol, starting at zero and working up to the range.

Given a structure like this, it is simple to derive the three items of information that define the symbol for the arithmetic encoder. For symbol x in the array, the low count can be found at totals[ x ], the high count at totals[ x + 1 ], and the range of scale at totals[ N ], N being the number of symbols in the alphabet.

The routines that do compression create this array. In this program, the array is named totals[], and it has 258 elements. The number of symbols in the alphabet is 257, the normal 256 plus one for the end-of-stream symbol.

One additional constraint is placed on these counts. Two things determine the highest possible count: the number of bits available in the high and low registers and the number of bits in the code values. Since floating-point calculations are performed in fixed-length registers, we have to minimize the amount of precision in our calculations so errors do not occur.

As it happens, there are two restrictions on the number of bits used in arithmetic coding: (1) the number of bits in the frequency values must be at least two less than the number of bits in the high and low registers; and (2) the number of bits in the frequency values plus the bits used in either the high and low registers must not exceed the number of bits used in arithmetic calculations during the coding process.

During calculations on the arithmetic registers, the code in this chapter uses unsigned long values, which give 32 bits to work with. Since our high and low registers and our frequency counts are limited to 16 bits in unsigned short ints, we meet restriction 2 implicitly. Restriction 1, however, requires more modifications. Since we are only using 16-bit registers for our high and low values, we have to restrict the highest cumulative counts in the totals[] array to no more than 14 bits, or 16,384.

The code to make sure that our total count of symbols is less than 16,384 is in the build_model routine called on initialization of the model. It takes the process a step further, scaling the counts down so they all fit in a single byte. This is done so that the count data stored in the output file takes a minimal amount of space.

During the compression phase of the program, the build_model() routine is called to perform all the necessary chores associated with the order-0 modeling used in this program. The four lines of code from build_model() are shown here:

count_bytes( input, counts );

scale_counts( counts, scaled_counts );

output_counts( output, scaled_counts );

build_totals( scaled_counts );

As you can see above, the build_model routine does no work on its own. It calls a series of four worker routines to handle the data for it. The first routine is count_bytes(). It does just that, counting all the occurrences of each symbol in the file and maintaining the total in an array, like so:

input_marker = ftell( input );

while ( ( c = getc( input )) != EOF )

  counts[ c ]++;

fseek( input, input_marker, SEEK_SET );


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The code for count_bytes scans the entire input file, then returns the input pointer to where it was when called. We assume that the number of counts of a given symbol in the file cannot exceed the span of an unsigned long. If this is not true, other measures will need to be taken to avoid overflow of the counts[] array elements.

After the array has been counted, the counts have to be scaled down. This is done in the scale_counts() routine. The first step here is to scale down the unsigned long counts[] array to fit in an array of unsigned characters. This stores the counts in less space in the output file. The code for this is listed here.

max_count = 0;

for ( i = 0 ; i < 256 ; i++ )

  if (counts[ i ] > max_count )

    max_count = counts[ i ];

scale = max_count / 256;

scale = scale + 1;

for ( i = 0 ; i < 256 ; i++ ) {

    scaled_counts[ i ] = (unsigned char ) ( counts[ i ] /scale );

    if ( scaled_counts[ i ] == 0 && counts[ i ] != 0 )

      scaled_counts[ i ] = 1;

}

After this is complete, one more scaling may need to be done. As part of the limitations on performing arithmetic coding using fixed-length registers, we have to restrict the total of our counts to less than 16,384, or fourteen bits. The second part of scale_counts does this with brute force, checking the total, then dividing by either two or four to get the correct results. An additional count has to be added because of the single count used by the end-of-stream character.

total = 1;

for ( i = 0 ; i < 256 ; i++ )

  total += scaled_counts[ i ];

if ( total > ( 32767 - 256 ) )

  scale = 4;

else if ( total > 16383 )

  scale = 2;

else

  return;

for ( i = 0 ; i < 256 ; i ++ )

   scaled_counts[ i ] /= scale;

There is certainly room in the scale_counts() routine for more sophistication. Every time we lose some of the accuracy in our counts by scaling, we also lose a certain amount of compression. An ideal scaling routine would scale down the counts to fit in the correct range while doing as little scaling as possible. The routines listed here don't work very hard at doing that.

Once the counts have been scaled down to fit in the unsigned character array, the output_counts() routine is called to save them to the output file. This program employs the same method to store the counts as used in Chapter 3 for the Huffman coding example. Instead of storing the entire array of counts, we only store runs on nonzero values. For details on this, refer to Chapter 3.

The last step in building the model is to set up the cumulative totals array in totals[]. This is the array used when actually performing the arithmetic coding. The code shown below builds that array. Remember that after the totals[] array has been built, the range for symbol x is found in totals[ x ] and totals[ x + 1 ]. The range used for the entire alphabet is found in totals[ END_OF_STREAM +1 ].

totals[ 0 ] = 0;

for ( i = 0 ; i < END_OF_STREAM ; i++ )

  totals[ i + 1 ] = totals[ i ] + scaled_counts[ i ];

totals[ END_OF_STREAM + 1 ] = totals[ END_OF_STREAM ] + 1;

Reading the Model

For expansion, the code needs to build the same model array in totals[] that was used in the compression routine. Since the original file is not available to scan for counts, the program reads in the scaled_counts[] array stored in the compressed file. The code that accomplishes this is identical to the Huffman expansion code in Chapter 3. Refer to Chapter 3 for details on how this code works.

After the scaled_counts[] array has been read in, the same routine used by the compression code can be invoked to build the totals[] array. Calling build_totals() in both the compression and expansion routines helps ensure that we are working with the same array.

Initializing the Encoder

Before compression can begin, we have to initialize the variables that constitute the arithmetic encoder. Three 16-bit variables define the arithmetic encoder: low, high, and underflow_bits. When the encoder first starts to run, the range of the output floating-point number is anywhere between 0 and 1. The low variable is initialized to 0 and the high to 0xFFFF. These two variables have an implied decimal point just ahead of the most significant bit and an infinite trail of ones or zeros. The ones will be shifted into the high variable and the zeros into the low.

low = 0;

high = 0xffff;

underflow_bits = 0;


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Encoding Process

At this point, the program is ready to begin the actual encoding process. This consists of looping through the entire file, reading in a character, determining its range variables, then encoding it. After the file has been scanned, the final step is to encode the end-of-stream symbol.

while ( ( c = getc( input ) ) !=EOF ) {

  convert_int_to_symbol( c, &s );

  encode_symbol( output, &s );

}

convert_int_to_symbol( END_OF_STREAM, &s );

encode_symbol( output, &s );

Two routines encode a symbol. The convert_int_to_symbol() routine looks up the modeling information for the symbol and retrieves the numbers needed to perform the arithmetic coding. This consists of stuffing numbers into the three elements of the symbol's structure, as shown here:

s->scale = totals[ END_OF_STREAM + 1 ];

s->low_count = totals[ c ];

s->high_count = totals[ c + 1 ];

After the symbol information is present, we are ready to encode the symbol using the arithmetic encoding routine. The C code to do this, listed in encode_symbol(), has two distinct steps. The first is to adjust the high and low variables based on the symbol data passed to the encoder.

range = (long) ( high-low ) + 1;

high = low + (unsigned short int)

       (( range * s->high_count ) / s->scale - 1 );

low = low + (unsigned short int)

       (( range * s->low_count ) / s->scale );

The code shown below restricts the range of high and low by the amount indicated in the symbol information. The range of the symbol is defined by s->low_count and s->high_count. These two counts are measured relative to the s->scale variable. After the adjustments are made, low and high will both be greater than or equal to their previous values. The range, or the distance between the two, will be less than or equal to its previous value.

for ( ; ; ) {

  if ( ( high & 0x8000 ) == ( low & 0x8000 ) ) {

     OutputBit( stream, high & 0x8000 );

     while ( underflow_bits > 0 ) {

       OutputBit( stream, ~high & 0x8000 );

       underflow_bits--;

     }

  } else if ( ( low & 0x4000 ) && !( high & 0x4000 ) ) {

    underflow_bits += 1;

    low &= 0x3fff;

    high |= 0x4000;

  } else

    return

  low <<= 1;

  high <<= 1;

  high |= 1;

}

After high and low have been adjusted, the routine needs to shift out any bits available for shifting. After a given arithmetic adjustment, it is never certain how many bits will need to be shifted out. If the encoded symbol has a very high probability, the number of bits will be low. If the encoded symbol has a low probability, the number of bits may be high.

Since the number isn't known in advance, the encoding routine sits in a loop shifting bits until there are no more shifts possible. The routine tests for two conditions to see if shifting is necessary. The first occurs when the most significant bits of the low and high word are the same. Because of the math being used, once the two bits are identical, they will never change. When this occurs, the bit is sent to the output file, and the high and low values are shifted.

Before shifting out the bit found when the most MSBs match, however, we have to transmit any underflow bits previously saved up. The underflow bits will be a sequence of bits set to the opposite value of the MSB. When we have an underflow, we have a binary sequence that ends up looking like that shown above. The number of underflow bits is found in the underflow_bits variable.

high = .100000...

low  = .011111...

Which leads to the second condition under which high and low variables require shifting: underflow. This occurs when the high and low words are growing dangerously close together but have not yet had their most significant bits match, a situation similar to that shown above.

When words begin growing close together like this, the dynamic range becomes dangerously low. Test for this condition after determining that the two most significant bits don't match. If they don't, check to see if the next bit is 0 in the high word and 1 in the low word. If they are, perform an underflow shift.

The underflow shift operation consists of throwing away the underflow bit (the one next to the most significant digit), shifting the remaining bits over one by one to fill the gap, and incrementing the underflow counter. The code to do this is somewhat opaque, but it performs this operation.

The underflow code takes advantage of the fact that when in danger of underflow, we know two things. First, we know that the most significant bit in the high word is 1 and in the low word 0. Second, the bit we throw away from the high word is 0 and from the low word 1.

Since we know the value of the highest two bits, we can simplify the shift operation. The code used in this chapter toggles the second most significant bit in both the high and low registers, then performs the normal shift operation. It looks as though the lower 14 bits were shifted left and the MSB was left alone.

If we check for both possible shift conditions and don't flag either one, we are done shifting bits out and can end the encoding operation. If either of the tests passed, the actual shift operation can take place. Both the high and low words are shifted left one bit position. The high word has a 1 shifted in to the LSB, and the low word has a 0 shifted in. The loop then repeats, outputting and shifting additional bits as necessary.

Flushing the Encoder

After encoding, it is necessary to flush the arithmetic encoder. The code for this is in the flush_arithmetic_encoder() routine. It outputs two bits and any additional underflow bits added along the way.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Decoding Process

Before arithmetic decoding can start, we need to initialize the arithmetic decoder variables. While encoding, we had just a high and low variable. Both are maintained by the decoder with a code variable, which contains the current bit stream read in from the input file.

During arithmetic decoding, the high and low variables should track exactly the same values they had during the encoding process, and the code variable should reflect the bit stream as it is read in from the input file. The only exception to this is that the code variable has underflow bits taken from it and thrown away, as with the high and low variables.

code = 0;

for ( i = 0 ; i < 16 ; i++ ) {

   code <<= 1;

   code += InputBit( stream );

}

low = 0;

high = Oxffff;

This implementation of the arithmetic decoding process requires four separate steps to decode each character. The first is to get the current scale for the symbol. This is simply a matter of looking in the current model data. In this implementation, the scale is found at totals[ END_OF_STREAM + 1 ]. The reason for breaking this out as a separate routine rather than coding it in-line is that future expansions to the basic compression program may use different modeling techniques. If a different model is used, determining the scale of the model could end up being more complex. This happens in the program used in the next chapter.

Once the current scale for the model is known, a second call is made to get the count for the current arithmetic code. This involves translating the decoders range, expressed by the high and low variables, into the range used by the model, which is in the scale variable.

range = (long) ( high - low ) + 1;

count = (short int)

     ((((long) ( code - low ) + 1 ) * s->scale-1 ) / range );

return( count );

The count returned to the expansion program is in essence a simple translation of the code variable from one scale to another. Once the count has been determined, we can determine what symbol has been encoded. Since we know the low and high range of the count for every symbol in the alphabet, determining which symbol goes with which count is just a matter of looking through the counts listed in the totals[] array.

for ( c = END_OF_STREAM ; count < totals[ c ] ; c–– )

   ;

s->high_count = totals[ c + 1 ];

s->low_count = totals[ c ];

return( c );

The implementation of the convert_symbol_to_int() used here determines the correct symbol with brute force. It simply starts looking at the top of the totals[] array for a count that fits with the current symbol, and it works down until it finds one that does. This is not optimal, since it could take 257 comparisons to locate the correct symbol.

An improved method of decoding would keep the symbols sorted so that the most frequently used symbols would be checked first. This would reduce the average time required to locate a symbol, though with random data we would not see much improvement. For simplicity, this was not the method used here.

Once convert_symbol_to_int() locates the correct symbol in the totals[] array, it takes the high and low counts and stores them in the symbol variable. They will be used in the next step of the decoding process. The correct value of the symbol is then returned to the calling program as an int.

After the correct symbol value is set up in the symbol structure, remove_symbol_from_stream() is called. Arithmetic coding is unusual in that it determines what the symbol is before it removes the bits associated with it. Then it calls the routine that actually removes those bits from the code and sets up the input code for the next symbol.

range = (long)( high - low ) + 1;

high = low + (unsigned short int)

       (( range * s->high_count ) / s->scale - 1 );

low = low + (unsigned short int)

       (( range * s->low_count ) / s->scale );

for ( ; ; ) {

  if ( ( high & 0x8000 ) == ( low & 0x8000 ) ) {

  }  else if ((low & 0x4000) == 0x4000 && (high & 0x4000) == 0 ) {

     code ^= 0x4000;

     low &= 0x3fff;

     high |= 0x4000;

  }  else

     return;

  low <<= 1;

  high <<= 1;

  high |= 1;

  code <<= 1;

  code += InputBit( stream );

}

The code that removes the symbol from the stream is listed above. It operates almost as a mirror image of the encoding routine. It first rescales the high and low variables to their new ranges as dictated by the range of the symbol being removed. Then the shifting process begins.

As before, there are two possible reasons to shift in new bits. First, if high and low have the same most significant bit, they will be discarded and a new bit will be shifted in as a replacement. Second, if high and low don't have the same MSB, and the second most significant bits are threatening underflow, we will discard the second most significant bits and shift in new bits.

If neither of the possible shift criteria are met, we can return, since the effects of the symbol have been entirely removed from the input stream. Otherwise, we shift high, low, and code. The lowest bit of high gets a 1, the lowest bit of low gets a 0, and the lowest bit of code gets a new bit from the input stream. This process continues indefinitely until all shifting is complete, at which point we return to the calling routine.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Summary

Arithmetic coding seems more complicated than Huffman coding, but the size of the program required to implement it is not significantly different. Runtime performance is significantly slower than Huffman coding, however, due to the computational burden imposed on the encoder and decoder. If squeezing the last bit of compression capability out of the coder is important, arithmetic coding will always do as good a job or better, than Huffman coding. But careful optimization is needed to get performance up to acceptable levels.

Code

/************************** Start of ARITH.C **************************/

#include <stdio.h>

#include <stdlib.h>

#include "errhand.h"

#include "bitio.h"

/*

* The SYMBOL structure is what is used to define a symbol in

* arithmetic coding terms. A symbol is defined as a range between

* 0 and 1.  Since we are using integer math, instead of using 0 and 1

* as our end points, we have an integer scale.  The low_count and

* high_count define where the symbol falls in the range.

*/

typedef struct {

  unsigned short int low_count;

  unsigned short int high_count;

  unsigned short int scale;

} SYMBOL;

/*

* Internal function prototypes, with or without ANSI prototypes.

*/

#ifdef __STDC__

void build_model( FILE *input, FILE *output );

void scale_counts( unsigned long counts[],

                   unsigned char scaled_counts[] );

void build_totals( unsigned char scaled_counts[] );

void count_bytes( FILE *input, unsigned long counts[] );

void output_counts( FILE *output, unsigned char scaled_counts[] );

void input_counts( FILE *stream );

void convert_int_to_symbol( int symbol, SYMBOL *s );

void get_symbol_scale( SYMBOL *s );

int convert_symbol_to_int( int count, SYMBOL *s );

void initialize_arithmetic_encoder( void );

void encode_symbol( BIT_FILE *stream, SYMBOL *s );

void flush_arithmetic_encoder( BIT_FILE *stream );

short int get_curret_count( SYMBOL *s );

void initialize_arithmetic_decoder( BIT_FILE *stream ):

void remove_symbol_from_stream( BIT_FILE *stream, SYMBOL * s );

#else

void build_model();

void scale_counts();

void build_totals();

void count_bytes();

void output_counts();

void input_counts();

void convert_int_to_symbol();

void get_symbol_scale();

int convert_symbol_to_int();

void initialize_arithmetic_encoder();

void encode_symbol();

void flush_arithmetic_encoder();

short int get_current_count();

void initialize_arithmetic_decoder();

void remove_symbol_from_stream();

#endif

#define END_OF_STREAM 256

short int totals[ 258 ];       /* The cumulative totals */

char *CompressionName = "Adaptive order 0 model with arithmetic coding";

char *Usage           = "in-file out-file\n\n\";

/*

* This compress file routine is a fairly orthodox compress routine.

* It first gathers statistics, and initializes the arithmetic

* encoder.  It then encodes all the characters in the file, followed

* by the EOF character.  The output stream is then flushed, and we

* exit.  Note that an extra two bytes are output.  When decoding an

* arithmetic stream, we have to read in extra bits.  The decoding process

* takes place in the msb of the low and high range ints, so when we are

* decoding our last bit we will still have to have at least 15 junk

* bits loaded into the registers.  The extra two bytes account for

* that.

*/

void CompressFile( input, output, argc, argv )

FILE * input;

BIT_FILE *output;

int argc;

char *argv[];

{

     int c;

     SYMBOL s;

     build_model( input, output->file );

     initialize_arithmetic_encoder();

     while ( ( c = getc( input ) ) != EOF ) {

        convert_int_to_symbol( c, &s );

        encode_symbol( output, &s );

     }

     convert_int_to_symbol( END_OF_STREAM, &s );

     encode_symbol( output, &s );

     flush_arithmetic_encoder( output );

     OutputBits( output, 0L, 16 );

     while ( argc–– > 0 ) {

       printf( "Unused argument: %s\n", *argv );

       argv++;

     }

}

/*

* This expand routine is also very conventional.  It reads in the

* model, initializes the decoder, then sits in a loop reading in

* characters.  When we decode an END_OF_STREAM, it means we can close

* up the files and exit.  Note decoding a single character is a three

* step process: first we determine what the scale is for the current

* symbol by looking at the difference between the high and low values.

* We then see where the current input values fall in that range.

* Finally, we look in our totals array to find out what symbol is

* a match. After that is done, we still have to remove that symbol

* from the decoder.  Lots of work.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

     SYMBOL s;

     int c;

     int count;

     input_counts( input->file );

     initialize_arithmetic_decoder( input );

     for ( ; ; ) {

       get_symbol_scale( &s );

       count = get_current_count( &s );

       c = convert_symbol_to_int( count, &s );

       if ( c == END_OF_STREAM )

         break;

       remove_symbol_from_stream( input, &s );

       putc( (char) c, output ):

     }

     while ( argc–– > 0 ) {

       printf( "Unused argument:  %s\n", *argv );

       argv++;

     }

}

/*

* This is the routine that is called to scan the input file, scale

* the counts, build the totals array, the output the scaled counts

* to the output file.

*/

void build_model( input, output )

FILE *input;

FILE *output;

{

     unsigned long counts[ 256 ];

     unsigned char scaled_counts[ 256 ];

     count_bytes( input, counts );

     scale_counts( counts, scaled_counts );

     output_counts( output, scaled_counts );

     build_totals( scaled_counts );

}

/*

* This routine runs through the file and counts the appearances of

* each character.

*/

#ifndef SEEK_SET

#define SEEK_SET 0

#endif

void count_bytes( input, counts )

FILE *input;

unsigned long counts[];

{

     long input_maker;

     int i;

     int c;

     for ( i = 0 ; i < 256; i++ )

       counts[ i ] = 0;

     input_marker = ftell( input );

     while ( ( c = getc( input ) ) != EOF )

       counts[ c ]++;

     fseek( input, input_marker, SEEK_SET );

}

/*

* This routine is called to scale the counts down. There are two

* types of scaling that must be done. First, the counts need to be

* scaled down so that the individual counts fit into a single unsigned

* char.  Then, the counts need to be rescaled so that the total of all

* counts is less than 16384.

*/

void scale_counts( counts, scaled_counts )

unsigned long counts[];

unsigned char scaled_counts[];

{

     int i;

     unsigned long max_count;

     unsigned int total;

     unsigned long scale;

/*

* The first section of code makes sure each count fits into a single

* byte.

*/

     max_count = 0;

     for ( i = 0 ; i < 256 ; i++ )

       if ( counts[ i ] > max_count )

         max_count = counts[ i ];

     scale = max_count / 256;

     scale = scale + 1;

     for ( i = 0 ; i < 256 ; i++ ) {

        scaled_counts[ i ] = (unsigned char ) ( counts[ i ] / scale );

        if ( scaled_counts[ i ] == 0 && counts[ i ] != 0 )

           scaled_counts[ i ] = 1;

     }

/*

* This next section makes sure the total is less than 16384.

* I initialize the total to 1 instead of 0 because there will be an

* additional 1 added in for the END_OF_STREAM symbol;

*/

     total = 1;

     for ( i = 0 ; i < 256 ; i++ )

        total += scaled_counts[ i ];

     if ( total > ( 32767 - 256 ) )

        scale = 4;

     else if ( total > 16383 )

        scale = 2;

     else

       return;

     for ( i = 0 ; i < 256 ; i++ )

       scaled_counts[ i ] /= scale;

}

/*

* This routine is used by both the encoder and decoder to build the

* table of cumulative totals.  The counts for the characters in the

* file are in the counts array, and we know that there will be a

* single instance of the EOF symbol.

*/

void build_totals( scaled_counts )

unsigned char scaled_counts[];

{

     int i;

     totals[ 0 ] = 0;

     for ( i = 0 ; i < END_OF_STREAM ; i++ )

       totals[ i + 1 ] = totals[ i ] + scaled_counts[ i ];

     totals[ END_OF_STREAM + 1 ] = totals[ END_OF_STREAM ] + 1;

}

/*

* In order for the compressor to build the same model, I have to

* store the symbol counts in the compressed file so the expander can

* read them in.  In order to save space, I don't save all 256 symbols

* unconditionally.  The format used to store counts looks like this:

*

* start, stop, counts, start, stop, counts, … 0

*

* This means that I store runs of counts, until all the non-zero

* counts have been stored.  At this time the list is terminated by

* storing a start value of 0.  Note that at least 1 run of counts has

* to be stored, so even if the first start value is 0, I read it in.

* It also means that even in an empty file that has no counts, I have

* to pass at least one count.

*

* In order to efficiently use this format, I have to identify runs of

* non-zero counts.  Because of the format used, I don't want to stop a

* run because of just one or two zeros in the count stream.  So I have

* to sit in a loop looking for strings of three or more zero values

* in a row.

*

* This is simple in concept, but it ends up being one of the most

* complicated routines in the whole program.  A routine that just

* writes out 256 values without attempting to optimize would be much

* simpler, but would hurt compression quite a bit on small files.

*

*/

void output_counts( output, scaled_counts )

FILE *output;

unsigned char scaled_counts[];

{

     int first;

     int last;

     int next;

     int i;

     first = 0;

     while ( first < 255 && scaled_counts[ first ] == 0 )

               first++;

/*

* Each time I hit the start of the loop, I assume that first is the

* number for a run of non-zero values.  The rest of the loop is

* concerned with finding the value for last, which is the end of the

* run, and the value of next, which is the start of the next run.

* At the end of the loop, I assign next to first, so it starts in on

* the next run.

*/

     for ( ; first < 256 ; first = next ) {

       last = first + 1;

       for ( ; ; ) {

         for ( ; last < 256 ; last++ )

           if ( scaled_counts[ last ] == 0 )

             break;

         last ––;

         for ( next = last + 1; next < 256 ; next++ )

           if ( scaled_counts[ next ] != 0 )

             break;

         if ( next > 255 )

           break;

         if ( ( next - last ) > 3 )

           break;

         last = next;

     };

/*

* Here is where I output first, last, and all the counts in between.

*/

        if ( putc( first, output ) != first )

          fatal_error( "Error writing byte counts\n" );

        if ( putc( last, output ) != last )

          fatal_error( "Error writing byte counts\n" );

        for ( i = first ; i <= last ; i++ ) {

          if ( putc( scaled_counts[ i ], output ) !=

            (int) scaled_counts[ i ] )

            fatal_error( "Error writing byte counts\n" );

        }

     }

     if ( putc( 0, output ) != 0 )

          fatal_error( "Error writing byte counts\n" );

}

/*

* When expanding, I have to read in the same set of counts.  This is

* quite a bit easier that the process of writing them out, since no

* decision making needs to be done.  All I do is read in first, check

* to see if I am all done, and if not, read in last and a string of

* counts.

*/

void input_counts( input )

FILE *input;

{

     int first;

     int last;

     int i;

     int c;

     unsigned char scaled_counts[ 256 ];

     for ( i = 0 ; i < 256 ; i++ )

       scaled_counts[ i ] = 0;

     if ( ( first = getc( input ) ) == EOF )

       fatal_error( "Error reading byte counts\n" );

     if ( ( last = getc( input ) ) == EOF )

       fatal_error( "Error reading byte counts\n" );

     for ( ; ; ) {

       for ( i = first ; i <= last ; i++ )

         if ( ( c = getc( input ) ) == EOF )

           fatal_error( "Error reading byte counts\n" );

         else

            scaled_counts[ i ] = (unsigned int) c;

       if ( ( first = getc( input ) ) == EOF )

         fatal_error( "Error reading byte counts\n" );

       if ( first == 0 )

         break;

       if ( ( last = getc( input ) ) == EOF )

         fatal_error( "Error reading byte counts\n" );

     }

     build_totals( scaled_counts );

}

/*

* Everything from here down defines the arithmetic coder section

* of the program.

*/

/*

* These four variables define the current state of the arithmetic

* coder/decoder.  They are assumed to be 16 bits long.  Note that

* by declaring them as short ints, they will actually be 16 bits

* on most 80X86 and 680X0 machines, as well as VAXen.

*/

static unsigned short int code;/* The present input code value     */

static unsigned short int low; /* Start of the current code range  */

static unsigned short int high;/* End of the current code range    */

long underflow_bits:           /* Number of underflow bits pending */

/*

* This routine must be called to initialize the encoding process.

* The high register is initialized to all 1s, and it is assumed that

* it has an infinite string of 1s to be shifted into the lower bit

* positions when needed.

*/

void initialize_arithmetic_encoder() {

     low = 0;

     high = 0xffff;

     underflow_bits = 0;

}

/*

* At the end of the encoding process, there are still significant

* bits left in the high and low registers.  We output two bits,

* plus as many underflow bits as are necessary.

*/

void flush_arithmetic_encoder( stream )

BIT_FILE *stream;

{

     OutputBit( stream, low & 0x4000 );

     underflow_bits++;

     while ( underflow_bits–– > 0 )

       OutputBit( stream, ~low & 0x4000 );

}

/*

* Finding the low count, high count, and scale for a symbol

* is really easy, because of the way the totals are stored.

* This is the one redeeming feature of the data structure used

* in this implementation.

*/

void convert_int_to_symbol( c, s )

int c;

SYMBOL *s;

}

     s->scale = totals[ END_OF_STREAM + 1];

     s->low_count = totals[ c ];

     s->high_count = totals[ c + 1 ];

}

/*

* Getting the scale for the current context is easy.

*/

void get_symbol_scale( s )

SYMBOL *s;

{

     s->scale = totals[ END_OF_STREAM + 1 ];

}

/*

* During decompression, we have to search through the table until

* we find the symbol that straddles the "count" parameter.  When

* it is found, it is returned.  The reason for also setting the

* high count and low count is so that symbol can be properly removed

* from the arithmetic coded input.

*/

int convert_symbol_to_int( count, s )

int count;

SYMBOL *s;

{

     int c;

     for ( c = END_OF_STREAM ; count < totals[ c ] ; c–– )

        ;

     s->high_count = totals[ c + 1 ];

     s->low_count = totals[ c ];

     return( c );

}

/*

* This routine is called to encode a symbol.  The symbol is passed

* in the SYMBOL structure as a low count, a high count, and a range,

* instead of the more conventional probability ranges.  The encoding

* process takes two steps.  First, the values of high and low are

* updated to take into account the range restriction created by the

* new symbol.  Then, as many bits as possible are shifted out to

* the output stream.  Finally, high and low are stable again and

* the routine returns.

*/

void encode_symbol( stream, s )

BIT_FILE *stream;

SYMBOL *s;

{

  long range;

/*

* These three lines rescale high and low for the new symbol.

*/

     range = (long) ( high-low ) + 1;

     high = low + (unsigned short int)

                   (( range * s->high_count ) / s->scale - 1 );

     low = low + (unsigned short int)

                   (( range * s->low_count ) / s->scale );

/*

* This loop turns out new bits until high and low are far enough

* apart to have stabilized.

*/

     for ( ; ; ) {

/*

* If this test passes, it means that the MSDigits match, and can

* be sent to the output stream.

*/

     if ( ( high & 0x8000 ) == (  low & 0x8000 ) ) {

       OutputBit( stream, high & 0x8000 );

       while ( underflow_bits > 0 ) {

         OutputBit( stream, ~high & 0x8000 );

         underflow_bits––;

       }

     }

/*

* If this test passes, the numbers are in danger of underflow, because

* the MSDigits don't match, and the 2nd digits are just one apart.

*/

     else if ( ( low & 0x4000 ) && !( high & 0x4000 )) {

       underflow_bits += 1;

       low &= 0x3fff;

       high |= 0x4000;

     } else

       return ;

     low <<= 1;

     high <<= 1;

     high |= 1;

   }

}

/*

* When decoding, this routine is called to figure out which symbol

* is presently waiting to be decoded.  This routine expects to get

* the current model scale in the s->scale parameter, and it returns

* a count that corresponds to the present floating point code;

*

* code = count / s->scale

*/

short int get_current_count( s )

SYMBOL *s;

{

     long range;

     short int count;

     range = (long) ( high - low ) + l;

     count = (short int)

             ((((long) ( code - low ) + 1 ) * s->scale-1 ) / range ) ;

     return( count );

}

/*

* This routine is called to initialize the state of the arithmetic

* decoder.  This involves initializing the high and low registers

* to their conventional starting values, plus reading the first

* 16 bits from the input stream into the code value.

*/

void initialize_arithmetic_decoder( stream )

BIT_FILE *stream;

{

     int i;

     code = 0;

     for ( i = 0 ; i < 16 ; i++ ) {

       code <<= 1;

       code += InputBit( stream );

     }

     low = 0;

     high = 0xffff;

}

/*

* Just figuring out what the present symbol is doesn't remove

* it from the input bit stream.  After the character has been

* decoded, this routine has to be called to remove it from the

* input stream.

*/

void remove_symbol_from_stream( stream, s )

BIT_FILE *stream;

SYMBOL *s;

{

     long range;

/*

* First, the range is expanded to account for the symbol removal.

*/

     range = (long)( high - low ) + l;

     high = low + (unsigned short int)

             (( range * s->high_count ) / s->scale - 1);

     low = low + (unsigned short int)

             (( range * s->low_count ) / s->scale );

*/

* Next, any possible bits are shipped out.

*/

     for ( ; ; ) {

/*

* If the MSDigits match, the bits will be shifted out.

*/

     if ( ( high & 0x8000 ) == ( low & 0x8000 ) ) {

     }

/*

* Else, if underflow is threatening, shift out the 2nd MSDigit.

*/

     else if ((low & 0x4000) == 0x4000  && (high & 0x4000) == 0 ) {

       code ^= 0x4000;

       low &= 0x3ffff;

       high |= 0x4000;

     } else

/*

* Otherwise, nothing can be shifted out, so I return.

*/

          return;

     low <<= 1;

     high <<= 1;

     high |= 1;

     code <<= 1;

     code += InputBit( stream );

   }

}

/*************************** End of ARITH.C ****************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 6
Statistical Modeling

The previous three chapters have shown several coding techniques used to compress data. The two coding methods disscussed, Huffman and arithmetic coding, can be implemented using either the fixed or adaptive approaches, but in all cases a statistical model needs to drive them. The chapters which discuss these coding techniques all used a simple order-0 model, which provides fairly good compression. This chapter discusses how to combine more sophisticated modeling techniques with standard coding methods to acheive better compression.

Higher-Order Modeling

To compress data using arithmetic or Huffman coding, we need a model of the data stream. The model needs to do two things to achieve compression: (1) it needs to accurately predict the frequency/probability of symbols in the input data stream, and (2) the symbol probabilities generated by the model need to deviate from a uniform distribution.

Accurately predicting the probability of symbols in the input data is an inherent need in arithmetic or Huffman coding. This type of coding reduces the number of bits needed to encode a character as its probability of appearance increases. If the letter E represents 25 percent of the input data, it should take only two bits to code. If the letter Z represents only .1 percent of the input data, it might take ten bits to code. If the model is not generating probabilities accurately, it might take ten bits to code E and two bits to code Z, causing data expansion instead of compression.

The model also needs to make predictions that deviate from a uniform distribution. The better the model is at making these predictions, the better the compression ratios will be. A model could be created, for example, that assigned all 256 possible symbols a uniform probability of 1/256. This model would create an output file exactly the same size as the input file, since every symbol would take exactly eight bits to encode. Only by correctly finding probabilities that deviate from a normal distribution can the number of bits be reduced, leading to compression. The increased probabilities have to accurately reflect reality, of course, as prescribed by the first condition.

It may seem that the probability of a given symbol occurring in a data stream is fixed, but this is not quite true. The probability of a character can change quite a bit, depending on the model. When compressing a C program, for example, the probability of a new-line character in the text might be 1/40, a probability determined by scanning the entire text and dividing the number of occurrences of the character by the total number of characters. But if a modeling technique looks at a single previous character, the probabilities change. In that case, if the previous character were a '}', the probability of a new-line character goes up to 1/2. This improved modeling technique leads to better compression, though both models were generating accurate probabilities.

Finite Context Modeling

The modeling discussed in this chapter is called "finite-context" modeling. It is based on a simple idea; calculate the probabilities for each incoming symbol based on the context in which the symbol appears. In all of the examples shown here, the context consists of nothing more than symbols previously encountered. The "order" of the model refers to the number of previous symbols that make up the context.

The simplest finite-context model would be an order-0 model that calculates the probability of each symbol independently of any previous symbols. To implement this model, all that is needed is a single table containing the frequency counts for each symbol that might be encountered in the input stream. An order-1 model keeps track of 256 different tables of frequencies, since it needs a separate set of counts for each possible context. Similarly, an order-2 model needs to handle 65,536 different tables of contexts.

The models used in chapters 3, 4, and 5 were all order-0 models. They didn't take up much storage space, and they were simple to manipulate. By confining ourselves to order-0 modeling, however, we ensured that our data-compression ratios were relatively modest.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Adaptive Modeling

It seems logical that as the order of the model increases, compression ratios ought to improve as well. The probability of the letter u appearing in the text of this book may only be 5 percent, for example, but if the previous context character is q, the probability goes up to 95 percent. Predicting characters with high probability lowers the number of bits needed, and larger contexts ought to let us make better predictions.

Unfortunately, as the order of the model increases linearly, the memory consumed by the model increases exponentially. With an order 0 model, the space consumed by the statistics could be as small as 256 bytes. Once the order of the model increases to 2 or 3, even the most cleverly designed models will consume hundreds of kilobytes.

The conventional way of compressing data is to make a pass over the symbols to gather statistics for the model. Then a second pass is made to actually encode the data. The statistics are usually carried with the compressed data so the decoder will have a copy. This approach obviously has serious problems if the statistics for the model take more space than the data to be compressed.

Adaptive compression is the solution to this problem. In adaptive data compression, both the compressor and the decompressor start with the same model. The compressor encodes a symbol using the existing model, then it updates the model to account for the new symbol using the existing model, then it updates the model to account for the new symbol. The decompressor likewise decodes a symbol using the existing model, then it updates the model. As long as the algorithm to update the model operates identically for the compressor and the decompressor, the process can operate perfectly without needing to pass a statistics table from the compressor to the decompressor.

Adaptive data compression has a slight disadvantage in that it starts compressing with less than optimal statistics. By subtracting the cost of transmitting the statistics with the compressed data, however, an adaptive algorithm will usually perform better than a fixed statistical model.

Adaptive compression also suffers in the cost of updating the model. When updating the count for a particular symbol using arithmetic coding, for example, the update code has the potential cost of updating the cumulative counts for all other symbols as well, leading to code that on the average performs 128 arithmetic operations for every symbol encoded or decoded, using the modeling techniques needed for arithmetic coding.

Because of high cost in both memory and CPU time, higher-order adaptive models have only become practical in perhaps the last ten years. It is ironic that as the cost of disk space and memory goes down, the cost of compressing the data stored there also goes down. As these costs continue to decline, we will be able to implement even more effective programs than are practical today.

A Simple Example

The sample program in Chapter 4 used Huffman coding to demonstrate adaptive compression. In this chapter, the sample program will use adaptive arithmetic coding. When performing finite-context modeling, we need a data structure to describe each context used while compressing the data. If we move up from an order to an order-1, for example, we will use the previous symbol as a context for encoding the current symbol.

An array of 256 context arrays is probably the simplest way to create the data structures for an order-1 model. As we saw in the last chapter, a simple context model for an arithmetic encoder can be created using an array of cumulative counts for each symbol. If we have 256 symbols in our alphabet, an array of pointers to 256 different context arrays can be created like this:

int *totals[ 256 ];

void initialize_model()

{

     int context;

     int i;

     for (context= 0 ; context < END_OF_STREAM ; context++ ) {

       totals[ context ] = (int *) calloc( END_OF_STREAM + 2,

                       sizeof( int ) );

       if ( totals[ context ] == NULL )

          fatal_error( "Failure allocating context %d", context );

       for ( i = 0 ; i <= ( END_OF_STREAM + 1 ) ; i++ )

          totals[ context ][ i ] = 1;

     }

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

This code not only creates the 256 context arrays, it also initializes each symbol's count to 1. At this point, we can begin encoding symbols as they come in. The loop for encoding the symbols looks similar to the one used for other adaptive programs. Here is an order 1 arithmetic compression loop:

for ( ; ; ) {

  c = getc( input );

  if (c == EOF )

    c = END_OF_STREAM;

  convert_int_to_symbol( c, context, &s );

  encode_symbol( output, &s );

  if ( c == END_OF_STREAM )

    break;

  update_model( c, context );

  context = c;

}

This works fairly simply. Instead of just having a single context table, like the code in chapter 5, we now have a set of 256 context tables. Every symbol is encoded using the context table from the previously seen symbol, and only the statistics for the selected context get updated after the symbol is seen. This means we can now more accurately predict the probability of a character's appearance.

The decoding process for this order 1 code is also very simple, and it looks similar to the decoding example from chapter 5. Here is the order 1 expansion loop:

for ( ; ; ) {

   get_symbol_scale( context, &s );

   count = get_current_count( &s );

   c = convert_symbol_to_int( count, context, &s );

   remove_symbol_from_stream( input, &s );

   if (c == END_OF_STREAM )

     break;

   putc( (char) c, output );

   update_model( c, context );

   context = c;

}

The only difference between this and conventional order-0 code is the addition of the context variable, both within the loop and as a parameter to other functions. The remaining routines that differ from the code in Chapter 5 are are shown next. The C source for this module is included on the program disk.

void update_model( int symbol, int context )

   int i;

   for ( i = symbol + 1 ; i <= ( END_OF_STREAM + 1 ) ; i++ )

      totals[ context ][ i ]++;

   if ( totals[ context ][ END_OF_STREAM + 1 ] < MAXIMUM_SCALE )

       return;

   for ( i = 1 ; i <= ( END_OF_STREAM + 1 ) ; i++ ) {

      totals[ context ][ i ] /= 2;

      if ( totals[ context ][ i ] <= totals[ context ][ i - 1 ] )

       totals[ context ][ i ] = totals[ context ][ i - 1 ] + 1;

   }

}

void convert_int_to_symbol( int c, int context, SYMBOL *s )

{

  s->scale = totals[ context ][ END_OF_STREAM + ];

  s->low_count = totals[ context ][ c ];

  s->high_count = totals[ context ][ c + 1 ];

}

void get_symbol_scale( int context, SYMBOL *s )

{

  s->scale = totals[ context][ END_OF_STREAM + 1 ];

}

int convert_symbol_to_int( int count, int context, SYMBOL *s)

{

  int c;

   for ( c = 0; count >= totals[ context ][ c + 1 ] ; c++ )

     ;

   s->high_count = totals[ context ][ c + 1 ];

   s->low_count = totals[ context ][ c ];

   return( c );

}

Using the Escape Code as a Fallback

The simple order-1 program does in fact do a creditable job of compression, but it has a couple of problems to address. First, the model for this program makes it a slow starter. Every context starts off with 257 symbols initialized to a single count, meaning every symbol starts off being encoded in roughly eight bits. As new symbols are added to the table, they will gradually begin to be encoded in fewer bits. This process, however, will not happen very quickly.

For the context table for the letter q, for example, we will probably see a a very high number of u symbols. The very first u will have a probability of 1/257, and will accordingly be encoded in eight bits. The second u will have a probability of 2/258, but will still require over seven bits to encode. In fact, it will take sixteen consecutive u symbols with no other appearances before the entropy of the symbol is reduced to even four bits.

The reason for this slow reduction in bit count is obvious. The probability of the u symbol is being weighted down by the other 256 symbols in the table. Though they may never appear in the message, they need a nonzero count. If their count were reduced to zero, we would not be able to encode them if and when they appeared in the message.

There is a solution to this problem, however, and it is relatively painless. Instead of having every symbol appear automatically in every table, start off with a nearly empty table and add symbols to the table only as they appear. The q table would have zero counts for all the other symbols, giving the first u that appears a low bit count.

But there is a catch here. If a symbol doesn't appear in a context table, how will it be encoded when it appears in a message? The easiest way to accomplish this is to use an escape code. The escape code is a special symbol (much like the end-of-stream symbol) that indicates we need to "escape" from the current context.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

When a context issues an escape symbol, we generally fall back to a lower-order context. In our next sample program, we escape to the escape context, a context that never gets updated. It contains 258 symbols, each of which has a count of 1. This guarantees that any symbol encountered in the message can be encoded by outputting an escape code from the current context and by encoding the symbol using the escape context.

How does this affect the example used for the letter u? As it turns out, it makes an enormous difference. The first u symbol that took eight bits in the previous example will take about eight bits here as well. The escape code takes no bits to encode, and in the escape context the u has a 1/257 probability. After that, however, the u is added to the table and given a count of 1. The next appearance of u will require only one bit to encode, since it has a probability of 1/2. By the time 16 u's have appeared, and while the previous model is still taking four bits to encode it, the escape-driven model will take .06 bits!

The escape code frees us from burdening our models with characters that may never appear. This lets the model adjust rapidly to changing probabilities and quickly reduces the number of bits needed to encode high- probability symbols.

The encoding process for this particular implementation of a multi-order model requires only a few modifications to the previous program. The convert_int_to_symbol() routine now has to check whether a symbol is present in the given context. If not, the escape code is encoded instead, and the function returns the appropriate result to the main encoding loop, as shown:

context = 0;

initialize_model();

initialize_arithmetic_encoder();

for ( ; ; ) {

  c = getc( input );

  if ( c == EOF )

   c = END_OF_STREAM;

  escaped = convert_int_to_symbol( c, context, &s );

  encode_symbol( output, &s );

  if ( escaped ) {

    convert_int_to_symbol( c, ESCAPE, &s );

    encode_symbol( output, &s );

  }

  if ( c == END_OF_STREAM )

    break;

  update_model( c, context );

  context = c;

}

In the main compression loop shown, the compressor first tries to send the original symbol. If the convert_int_to_symbol() routine returns a true, the symbol did not appear in the current context, and the routine resends the symbol using the escape context. We update just the current context model with the symbol just sent, not the escape model.

The decompression loop for this program follows a similar pattern. The code shown next makes one or two possible passes through the loop, depending on whether an escape code is detected. The program for this order-1 context-switching program is on the program diskette that accompanies this book.

context = 0;

initialize_model();

initialize_arithmetic_decoder( input );

for ( ; ; ) {

  last_context = context;

  do {

   get_symbol_scale( context, &s );

    count = get_current_count( &s );

    c = convert_symbol_to_int( count, context, &s );

    remove_symbol_from_stream( input, &s );

    context = c;

  } while ( c == ESCAPE );

  if ( c == END_OF_STREAM )

    break;

  putc( (char) c, output );

  update_model( c, last_context );

  context = c;

}

Improvements

Some problems with the method of encoding in ARITH-1.C are the high-cost operations associated with the model. Each time we update the counts for symbol c, every count in totals[context][] from c up to 256 has to be incremented. An average of 128 increment operations have to be performed for every character encoded or decoded. For a simple demonstration program like the one shown here, this may not be a major problem, but a production program should be modified to be more efficient.

One way to reduce the number of increment operations is to move the counts for the most frequently accessed symbols to the top of the array. This makes the model keep track of each symbol's position in the totals[context] array, but it reduces the number of increment operations by an order of magnitude. This is a relatively simple enhancement to make to this program. A very good example of a program that uses this technique has been published as part of the paper by Ian H. Witten, Neal Radford, and John Cleary, "Arithmetic Coding for Data Compression," Communications of the ACM (June 1987). This paper is an excellent source of information regarding arithmetic coding, with some sample C source code illustrating the text.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Highest-Order Modeling

The previous sample program used order-1 statistics to compress data. It seems logical that if we move to higher orders, we should achieve better compression. The importance of the escape code becomes even more clear here. When using an order-3 model, we potentially have 16 million context tables to work with (actually 256*256*256, or 16,777,216). We would have to read in an incredible amount of text before those 16 million tables would have enough statistics to start compressing data, and many of those 16 million tables will never be used—which means they take up space in our computer's memory for no good reason. When compressing English text, for example, it does no good to allocate space for the table QQW. It will never appear.

The solution to this is, again, to set the initial probabilities of all of the symbols to 0 for a given context and to fall back to a different context when a previously unseen symbol occurs. So the obvious question is: what do we use as the fallback context after emitting an escape code? In the last example, we fell back to a default context called the escape context. The escape context was never updated, which meant that using it generally would not provide any compression.

In the higher-order models, there is a better way to compress than just automatically falling back to a default context. If an existing context can't encode a symbol, fall back to the next smaller-order context. If our existing context was REQ, for example, and U needs to be encoded for the first time, an escape code will be generated. Following that, we drop back to an order-2 model to try to encode the character U using the context EQ. This continues down through the order-0 context. If the escape code is still generated at order-0, we fall back to a special order(-1) context that is never updated and is set up at initialization to have a count of 1 for every possible symbol—so it is guaranteed to encode every symbol.

Using this escape-code technique means only a slight modification to the driver program. The program (see the code found in ARITH-N.C) now sits in a loop trying to encode its characters, as shown here:

do  {

    escaped = convert_int_to_symbol( c, &s );

    encode_symbol( compressed_file, &s );

} while ( escaped );

The modeling code keeps track of what the current order is, decrementing it whenever an escape is emitted. Even more complicated is the modeling module's job of keeping track of which context table needs to be used for the current order.

Updating the Model

ARITH1E.C does a good job of compressing data. But quite a few improvements can still be made to this simple statistical method without changing the fundamental nature of its algorithm. The rest of this chapter is devoted to discussing those improvements, along with a look at a sample compression module, ARITH-N.C, that implements most of them.

Using the highest-order modeling algorithm requires that instead of keeping just one set of context tables for the highest order, we keep a full set of context tables for every order up to the highest order. If we are doing order-2 modeling, for example, there will be a single order-0 table, 256 order-1 tables, and 65,536 order-2 tables. When a new character is encoded or decoded, the modeling code updates one of these tables for each order. In the example of U following REQ, the modeling code would update the U counters in the order-3 REQ table, the order-2 EQ table, the order-1 Q table, and the order-0 table. The code to update all of these tables is shown next:

for ( order = 0 ; order <= max_order ; order++ )

   update_model( order, symbol );

A slight modification to this algorithm results in both faster updates and better compression. Instead of updating all the different order models for the current context, we can instead update only those models actually used to encode the symbol. This is called "update exclusion," since it excludes unused lower-order models from being updated. It will generally give a small but noticeable improvement in the compression ratio. Update exclusion works since symbols showing up frequently in the higher-order models won't be seen as often in the lower-order models, which means we shouldn't increment the counters in the lower-order models. The modified code for update exclusion will look like this:

for ( order = encoding_order ; order <= max_order ; order++ )

   update_model( order, symbol );


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Escape Probabilities

When the program first starts encoding a text stream, it will emit quite a few escape codes. The number of bits used to encode escape characters will probably have a large effect on the compression ratio, particularly in small files. In our first attempts to use escape codes, we set the escape count to 1 and left it there, regardless of the state of the rest of the context table. Bell, Cleary, and Witten call this "Method A." Method B sets the count for the escape character at the number of symbols presently defined for the context table. If eleven different characters have been seen so far, for example, the escape symbol count will be set at eleven, regardless of what the counts are.

Both methods seem to work fairly well. The code in our previous program can easily be modified to support either one. Probably the best thing about methods A and B is that they are not computationally intensive. Adding the escape symbol to the Method A table can be done so that it takes almost no more work to update the table with the symbol than without it.

The next sample program, ARITH-N.C, implements a slightly more complicated escape-count calculation algorithm. It tries to take into account three different factors when calculating the escape probability. First, as the number of symbol defined in the context table increases, the escape probability naturally decreases. This reaches its minimum when the table has all 256 symbols defined, making the escape probability 0.

Second, it takes into account a measure of randomness in the table. It calculates this by dividing the maximum count in the table by the average count. The higher the ratio, the less random the table. The REQ table, for example, may have only three symbols defined: U, with a count of 50; u, with a count of 10; and e, with a count of 3. The ratio of U's count, 50, to the average, 21, is fairly high. The U is thus predicted with a relatively high amount of accuracy, and the escape probability ought to be lower. In a table where the high count was 10 and the average was 8, things would seem a little more random, and the escape probability should be higher.

The third factor taken into account is simply the raw count of how many symbols have been seen for the particular table. As the number of symbols increases, the predictability of the table should go up, making the escape probability go down.

The formula I use for calculating the number of counts for the escape symbol is below.

count = (256 - number of symbols seen)*number of symbols seen

count = count /(256 * the highest symbol count)

if count is less than 1

  count = 1

The missing variable in this equation is the raw symbol count. This is implicit in the calculation, because the escape probability is the escape count divided by the raw count. The raw count will automatically scale the escape count to a probability.

Scoreboarding

When using highest-order modeling techniques, an additional enhancement, scoreboarding, can improve compression efficiency. When we first try to compress a symbol, we can generate either the code for the symbol or an escape code. If we generate an escape code, the symbol had not previously occurred in that context, so we had a count of 0. But we do gain some information about the symbol just by generating an escape. We can now generate a list of symbols that did not match the symbol to be encoded. These symbols can temporarily have their counts set to 0 when we calculate the probabilities for lower-order models. The counts will be reset back to their permanent values after the encoding for the particular character is complete. This process is called scoreboarding.

An example of this is shown below. If the present context is HAC and the next symbol is K, we will use the tables shown next to encode the K. Without scoreboarding, the HAC context generates an escape with a probability of 1/6. The AC context generates an escape with a probability of 1/8. The C context generates an escape with a probability of 1/40, and the "" context finally generates a K with a probability of 1/73.

"" | "C" | "AC" | "HAC"
ESC 1 | ESC 1 | ESC 1 | ESC 1
'K' 1 | 'H' 20 | 'C' 5 | 'E' 3
'E 40 | 'T' 11 | 'H' 2 | 'L' 1
'I' 22 | 'L' 5 |  | 'C' 1
'A 9 | 'A' 3 |  |

If we use scoreboarding to exclude counts of previously seen characters, we can make a significant improvement in these probabilities. The first encoding of an escape from HAC isn't affected, since no characters were seen before. But the AC escape code eliminates the C from its calculations, resulting in a probability of 1/3. The C escape code excludes the H and the A counts, increasing the probability from 1/40 to 1/17. And finally, the "" context excludes the E and A counts, reducing that probability from 1/73 to 1/24. This reduces the number of bits required to encode the symbol from 14.9 to 12.9, a significant savings.

Keeping a symbol scoreboard will almost always result in some improvement in compression, and it will never make things worse. The major problem with scoreboarding is that the probability tables for all of the lower-order contexts have to be recalculated every time the table is accessed. This results in a big increase in the CPU time required to encode text. Scoreboarding is left in ARITH-N.C. to demonstrate the gains possible when compressing text using it.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Data Structures

All improvements to the basic statistical modeling assume that higher-order modeling can actually be accomplished on the target computer. The problem with increasing the order is one of memory. The cumulative totals table in the order-0 model in Chapter 5 occupied 516 bytes of memory. If we used the same data structures for an order-1 model, the memory used would shoot up to 133K, which is still probably acceptable. But going to order-2 will increase the RAM requirements for the statistics unit to thirty-four megabytes! Since we would like to try orders even higher than 2, we need to redesign the data structures that hold the counts.

To save memory space, we have to redesign the context statistics tables. In Chapter 5, the table is about as simple as it can be, with each symbol being used as an index directly into a pair of counts. In the order-1 model, the appropriate context tables would be found by indexing once into an array of context tables, then indexing again to the symbol in question, a procedure like that shown here:

low_count = totals[ last_char ][ current_char ];

high_count = totals[ last_char ][ current_char + 1 ];

range = totals[ last_char ][ 256 ];

This is convenient, but enormously wasteful. Full context space is allocated even for unused tables, and within the tables space is allocated for all symbols, seen or not. Both factors waste enormous amounts of memory in higher-order models.

The solution to the first problem, reserving space for unused contexts, is to organize the context tables as a tree. Place the order-0 context table at a known location and use it to contain pointers to order-1 context tables. The order-1 context tables will hold their own statistics and pointers to order-2 context tables. This continues until reaching the "leaves" of the context tree, which contain order_n tables but no pointers to higher orders. Using a tree structure can keep the unused pointer nodes set to null pointers until a context is seen. Once the context is seen, a table is created and added to the parent node of the tree.

The second problem is creating a table of 256 counts every time a new context is created. In reality, the highest-order contexts will frequently have only a few symbols, so we can save a lot of space by only allocating space for symbols seen for a particular context.

After implementing these changes, we have a set of data structures that look like this:

     typedef struct {

       unsigned char symbol;

       unsigned char counts;

     } STATS;

     typedef struct {

       struct context *next;

     } LINKS;

     typedef struct context {

       int max_index;

       STATS *stats;

       LINKS *links;

       struct context *lesser_context;

     } CONTEXT;

The new context structure has four major elements. The first is the counter, max_index, which tells how many symbols are presently defined for this particular context table. When a table is first created, it has no defined symbols, and max_index is -1. A completely defined table will have a max_index of 255. The max_index variable tells how many elements are allocated for the arrays pointed to by stats and links. Stats is an array of structures, each containing a symbol and a count for that symbol. If the context table is not one of the highest-order tables, it will also have a links array. Each symbol defined in the stats array will have a pointer to the next higher-order context table in the links table.

A sample of the context table tree is shown in figure 6.1. The table shown is one that will have just been created after the input text "ABCABDABE" when keeping maximum order-3 statistics. Just nine input symbols have already generated a fairly complicated data structure, but it is orders of magnitude smaller than one consisting of arrays of arrays.

Figure 6.1 A context table tree: "ABCABDABE."

One element in this structure that hasn't been explained is the lesser_context pointer. This pointer is needed when using higher-order models. If the modeling code is trying to locate an order-3 context table, it first has to scan through the order-0 symbol list looking for the first symbol, the match, the order-1 symbol list, and so on. If the symbol lists in the lower orders are relatively full, this can be a lengthy process. Even worse, every time an escape is generated, the process has to be repeated when looking up the lower-order context. These searches can consume an inordinate amount of CPU time.

The solution to this is to maintain a pointer for each table that points to the table for the context one order less. The context table ABC should have its back pointer point to BC, for example, which should have a back pointer to C, which should have a pointer to "", the null table. Then the modeling code only needs to keep a pointer to the current highest order context. Given that, finding the order-1 context table is simply a matter of performing (n-1) pointer operations.

With the table shown in Figure 6.1, for example, suppose the next incoming text symbol is X and the current context is ABE. Without the benefit of the lesser context pointers, I have to check the order-3, 2, 1, and 0 tables for X. This takes 15 symbol comparisons and three table lookups. Using reverse pointers eliminates all the symbols comparisons and performs just three table lookups.

To update the figure 6.1 context tree to contain an X after ABE, the modeling code has to perform a single set of lookups for each order/context. This code is shown in ARITH-N.C in the add_character_to_model() routine. Every time a new table is created, it needs to have its back pointer created properly at the same time, which requires a certain amount of care in the design of update procedures.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Finishing Touches: Tables –1 and –2

The final touch to the context tree in ARITH-N.C is the addition of two special tables. The order(-1) table has been discussed previously. This is a table with a fixed probability for every symbol. If a symbol is not found in any of the higher-order models, it will show up in the order(-1) model. This is the table of last resort. Since it guarantees that it will always provide a code for every symbol in the alphabet, we don't update this table, which means it uses a fixed probability for every symbol.

ARITH-N.C also has a special order(-2) table used for passing control information from the encoder to the decoder. The encoder can pass a-1 to the decoder to indicate end-of-file. Since normal symbols are always defined as unsigned values ranging from 0 to 255, the modeling code recognizes a negative number as a special symbol that will generate escapes all the way back to the order(-2) table. The modeling code can also determine that since -1 is a negative number, the symbol should just be ignored when the update_model() code is called.

Model Flushing

The creation of the order(-2) model allows us to pass a second control code from the encoder to the expander—the flush code, which tells the decoder to flush statistics out of the model. The compressor does this when the performance of the model starts to slip. The ratio is adjustable and is set in this implementation to 10 percent. When compression falls belows this ratio, the model is "flushed" by dividing all counts by two. This gives more weight to newer statistics, which should improve the compression.

In reality the model should probably be flushed whenever the input symbol stream drastically changes character. If the program is compressing an executable file, for example, the statistics accumulated during the compression of the executable code are probably of no value when compressing the program's data. Unfortunately, it isn't easy to define an algorithm that detects a "change in the nature" of the input.

Implementation

Even with the Byzantine data structures used here, the compression and expansion programs built around ARITH-N.C have prodigious memory requirements. When running on DOS machines limited to 640K, these programs have to be limited to order-1, or perhaps order-2 for text that has a higher redundancy ratio.

To examine compression ratios for higher orders on binary files, there are a couple of choices for these programs. First, they can be built using a DOS Extender, such as Rational Systems/16M. Or they can be built on a machine that has either a larger address space or support for virtual memory, such as Windows 95, VMS, or UNIX. The code distributed here was written in an attempt to be portable across all these options.

Testing shows that with an extra megabyte of extended memory and a DOS Extender, virtually any ASCII file can be compressed on a PC using order -3 compression. Some binary files require more memory. A UNIX system had no problem with order -3 compression and turned in the best performance overall in terms of speed.

Conclusions

Compression-ratio test show that statistical modeling can perform at least as well as dictionary-based methods. But these programs are at present somewhat impractical because of their high resource requirements. ARITH-N is fairly slow, compressing data with speeds in the range of 1K per second and needing huge amounts of memory to use higher-order modeling. As memory becomes cheaper and processors become more powerful, however, schemes such as the ones shown here may become practical. They could be applied today to circumstances in which either storage or transmission costs are extremely high.

Order-0 adaptive modeling using arithmetic coding could be useful today in situations requiring extremely low consumption of memory. The compression ratios might not be as good as those gained with sophisticated models, but memory consumption is minimized.

Enhancement

The performance of these algorithms could be improved significantly beyond the implementation discussed here. The first area for improvement would be in memory management. Right now, when the programs run out of memory, they abort. A more sensible approach would be to have the programs start with fixed amounts of memory available for statistics. When the statistics fill the space, the program should then stop updating the tables and just use what it had. This would mean implementing internal memory-management routines instead of using the C run-time library routines.

Another potential improvement could come in the tree structure for the context tables. Locating tables through the use of hashing could be quite a bit faster and might require less memory. The context tables themselves could also be improved. When a table has over 50 percent of the potential symbols defined for it, an alternate data structure could be used with the counts stored in a linear array. This would allow for faster indexing and would reduce memory requirements.

Finally, it might be worth trying ways to adaptively modify the order of the model being used. When compressing using order-3 statistics, early portions of the input text generate a lot of escapes while the statistics tables fill up. It ought to be possible to start encoding using order-0 statistics while keeping order-3 statistics. As the table fills up, the order used for encoding could be incremented until it reaches the maximum.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

ARITH-N Listing

/************************* Start of ARITH-N.C **************************/

* * This is the order-n arithmetic coding module used in the final

* part of chapter 6.

*

* Compile with BITIO.C. ERRHAND.C, and either MAIN-C.C OR MAIN-E.C. This

* program should be compiled in large model.  An even better alternative

* is a DOS extender.

*

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include "errhand.h"

#include "bitio.h"

/*

* The SYMBOL structure is what is used to defined a symbol in

* arithmetic coding terms.  A symbol is defined as a range between

* 0 and 1.  Since we are using integer math, instead of using 0 and 1

* as our end points, we have an integer scale.  The low_count and

* high_count define where the symbol falls in the range.

*/

typedef struct {

     unsigned short int low_count;

     unsigned short int high_count;

     unsigned short int scale;

} SYMBOL;

#define MAXIMUM_SCALE 16383  /* Maximum allowed frequency count */

#define ESCAPE   256         /* The escape symbol               */

#define DONE (-1)            /* The output stream empty  symbol */

#define FLUSH  (-2)          /* The symbol to flush the model   */

/*

*  Function prototypes.

*/

#ifdef __STDC__

void initialize_options( int argc, char **argv );

int check_compression( FILE *input, BIT_FILE *output );

void initialize_model( void );

void update_model( int symbol );

int convert_int_to_symbol( int symbol, SYMBOL *s );

void get_symbol_scale( SYMBOL *s );

int convert_symbol_to_int( int count, SYMBOL *s );

void add_character_to_model( int c );

void flush_model( void );

void initialize_arithmetic_decoder( BIT_FILE *stream ) ;

void remove_symbol_from_stream( BIT_FILE *stream, SYMBOL *s );

void initialize_arithmetic_encoder( void );

void encode_symbol( BIT_FILE *stream, SYMBOL *s );

void flush_arithmetic_encoder( BIT_FILE *stream );

short int get_current_count( SYMBOL *s );

#else

void initialize_options();

int check_compression();

void initialize_model();

void update_model();

int convert_int_to_symbol();

void get_symbol_scale();

int convert_symbol_to_int();

void add_character_to_model();

void flush_model();

void initialize_arithmetic_decoder();

void remove_symbol_from_stream();

void initialize_arithmetic_encoder();

void encode_symbol();

void flush_arithmetic_encoder();

short int get_current_count();

#endif

char *CompressionName = "Adaptive order n model with arithmetic coding";

char *Usage           = "in-file out-file [ -o order ]\n\n";

int max_order         = 3;

/*

* The main procedure is similar to the main found in ARITH1E.C.  It has

* to initialize the coder and the model.  It then sits in a loop reading

* input symbols and encoding them.  One difference is that every 256

* symbols a compression check is performed.  If the compression ratio

* falls below 10%, a flush character is encoded.  This flushes the encod

* ing model, and will cause the decoder to flush its model when the

* file is being expanded.  The second difference is that each symbol is

* repeatedly encoded until a successful encoding occurs.  When trying to

* encode a character in a particular order, the model may have to

* transmit an ESCAPE character.  If this is the case, the character has

* to be retransmitted using a lower order.  This process repeats until a

* successful match is found of the symbol in a particular context.

* Usually this means going down no further than the order -1 model.

* However, the FLUSH and DONE symbols drop back to the order -2 model.

*

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

     SYMBOL s;

     int c;

     int escaped;

     int flush = 0;

     long int text_count = 0;

     initialize_options( argc, argv );

     initialize_model();

     initialize-arithmetic_encoder();

     for ( ; ; ) {

       if ( ( ++text_count & 0x0ff ) == 0 )

         flush = check_compression( input, output );

       if ( !flush )

         c = getc( input );

       else

         c = FLUSH;

       if ( c == EOF )

         c = DONE;

       do {

         escaped = convert_int_to_symbol( c, &s);

         encode_symbol( output, &s );

       } while ( escaped );

       if ( c == DONE )

         break;

       if ( c == FLUSH ) {

         flush_model();

         flush = 0;

       }

       update_model( c );

       add_character_to_model( c );

     }

     flush_arithmetic_encoder( output );

}

/*

* The main loop for expansion is very similar to the expansion

* routine used in the simpler compression program, ARITH1E.C.  The

* routine first has to initialize the the arithmetic coder and the

* model.  The decompression loop differs in a couple of respect.

* First of all, it handles the special ESCAPE character, by

* removing them from the input bit stream but just throwing them

* away otherwise.  Secondly, it handles the special FLUSH character.

* Once the main decoding loop is done, the cleanup code is called,

* and the program exits.

*

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

     SYMBOL s;

     int c;

     int count;

     initialize_options( argc, argv );

     initialize_model();

     initialize_arithmetic_decoder( input );

     for ( ; ; ) {

       do {

         get_symbol_scale( &s );

         count = get_current_count( &s );

         c = convert_symbol_to_int( count, &s );

         remove_symbol_from_stream( input, &s );

       } while ( c == ESCAPE );

       if ( c == DONE )

         break;

       if ( c != FLUSH )

         putc( (char) c, output );

       else

         flush_model();

       update_model( c );

       add_character_to_model( c );

     }

}

/*

* This routine checks for command line options.  At present, the only

* option being passed on the command line is the order.

*/

void initialize_options( argc, argv )

int argc;

char *argv[];

{

     while ( argc–– > 0 ) {

       if ( strcmp( *argv, "-o" ) == 0 ) {

         argc––;

         max_order = atoi( *++argv );

       } else

         printf( "Uknown argument on command line: %s\n", *argv );

       argc––;

       argv++;

     }

}

/*

* This routine is called once every 256 input symbols.  Its job is to

* check to see if the compression ratio falls below 10%.  If the

* output size is 90% of the input size, it means not much compression

* is taking place, so we probably ought to flush the statistics in the

* model to allow for more current statistics to have greater impact.

* This heuristic approach does seem to have some effect.

*/

int check_compression( input, output )

FILE *input;

BIT_FILE *output;

{

     static long local_input_marker = 0L;

     static long local_output_marker = 0L;

     long total_input_bytes;

     long total_output_bytes;

     int local_ratio;

     total_input_bytes = ftell( input ) - local_input_marker;

     total_output_bytes = ftell( output->file );

     total_output_bytes -= local_output_marker;

     if ( total_output_bytes == 0 )

       total_output_bytes = 1;

     local_ratio = (int)( ( total_output_bytes * 100 ) /

                                   total_input_bytes );

     local_input_marker = ftell( input );

     local_output_marker = ftell( output->file );

     return( local_ratio > 90 );

}

/*

*

* The next few routines contain all of the code and data used to

* perform modeling for this program.  This modeling unit keeps track

* of all contexts from 0 up to max_order, which defaults to 3.  In

* addition, there is a special context -1 which is a fixed model used

* to encode previously unseen characters, and a context -2 which is

* used to encode EOF and FLUSH messages.

*

* Each context is stored in a special CONTEXT structure, which is

* documented below.  Context tables are not created until the context

* is seen, and they are never destroyed.

*

*/

/*

* A context table contains a list of the counts for all symbols

* that have been seen in the defined context.  For example, a

* context of "Zor" might have only had 2 different characters

* appear.  't' might have appeared 10 times, and '1' might have

* appeared once.  These two counts are stored in the context

* table.  The counts are stored in the STATS structure.  All of

* the counts for a given context are stored in and array of STATS.

* As new characters are added to a particular contexts, the STATS

* array will grow.  Sometimes the STATS array will shrink

* after flushing the model.

*/

typedef struct {

     unsigned char symbol;

     unsigned char counts;

} STATS;

/*

* Each context has to have links to higher order contexts.  These

* links are used to navigate through the context tables.  For example,

* to find the context table for "ABC", I start at the order 0 table,

* then find the pointer to the "A" context table by looking through

* the LINKS array.  At that table, we find the "B" link and go to

* that table.  The process continues until the destination table is

* found.  The table pointed to by the LINKS array corresponds to the

* symbol found at the same offset in the STATS table.  The reason that

* LINKS is in a separate structure instead of being combined with

* STATS is to save space.  All of the leaf context nodes don't need

* next pointers, since they are in the highest order context.  In the

* leaf nodes, the LINKS array is a NULL pointer.

*/

typedef struct {

     struct context *next;

} LINKS;

/*

* The CONTEXT structure holds all of the known information about

* a particular context.  The links and stats pointers are discussed

* immediately above here.  The max_index element gives the maximum

* index that can be applied to the stats or link array.  When the

* table is first created, and stats is set to NULL, max_index is set

* to -1.  As soon as single element is added to stats, max_index is

* incremented to 0.

*

* The lesser context pointer is a navigational aid.  It points to

* the context that is one less than the current order.  For example,

* if the current context is "ABC", the lesser_context pointer will

* point to "BC".  The reason for maintaining this pointer is that

* this particular bit of table searching is done frequently, but

* the pointer only needs to be built once, when the context is

* created.

*/

typedef struct context {

     int max_index;

     LINKS *links;

     STATS *stats;

     struct context *lesser_context;

} CONTEXT;

/*

* *contexts[] is an array of current contexts.  If I want to find

* the order 0 context for the current state of the model.  I just

* look at contexts[0].  This array of context pointers is set up

* every time the model is updated.

*/

CONTEXT **contexts;

/*

* current_order contains the current order of the model.  It starts

* at max_order, and is decremented every time an ESCAPE is sent.  It

* will only go down to -1 for normal symbols, but can go to -2 for

* EOF and FLUSH.

*/

int current_order;

/*

* This table contains the cumulative totals for the current context.

* Because this program is using exclusion, totals has to be calculated

* every time a context is used.  The scoreboard array keeps track of

* symbols that have appeared in higher order models, so that they

* can be excluded from lower order context total calculations.

*/

short int totals[ 258 ];

char scoreboard[ 256 ];

/*

* Local procedure declarations for modeling routines.

*/

#ifdef __STDC__

void update_table( CONTEXT *table, int symbol );

void rescale_table( CONTEXT *table );

void totalize_table( CONTEXT *table );

CONTEXT *shift_to_next_context( CONTEXT *table, int c, int order );

CONTEXT *allocate_next_order_table( CONTEXT *table,

                                    int symbol,

                                    CONTEXT *lesser_context );

void recursive_flush( CONTEXT *table );

#else

void update_table();

void rescale_table();

void totalize_table();

CONTEXT *shift_to_next_context();

CONTEXT *allocate_next_order_table();

void recursive_flush();

#endif

/*

* This routine has to get everything set up properly so that

* the model can be maintained properly.  The first step is to create

* the *contexts[] array used later to find current context tables.

* The *contexts[] array indices go from -2 up to max_order, so

* the table needs to be fiddled with a little.  This routine then

* has to create the special order -2 and order -1 tables by hand,

* since they aren't quite like other tables.  Then the current

* context is set to \0, \0, \0, ... and the appropriate tables

* are built to support that context.  The current order is set

* to max_order, the scoreboard is cleared, and the system is

* ready to go.

*/

void initialize_model()

{

     int i;

     CONTEXT *null_table;

     CONTEXT *control_table;

     current_order = max_order;

     contexts = (CONTEXT **) calloc( sizeof( CONTEXT * ), 10 );

     if ( contexts == NULL )

       fatal_error( "Failure #1:  allocating context table!" );

     context += 2;

     null_table = (CONTEXT *) calloc( sizeof( CONTEXT ), 1 );

     if ( null_table == NULL )

       fatal_error( "Failure #2:  allocating null table!" );

     null_table->max_index = -1;

     contexts[ -1 ] = null_table;

     for ( i = 0 ; i <= max_order ; 1++ )

        contexts[ i ] = allocate_next_order_table( contexts[ i-1 ],

                                                   0,

                                                   contexts[ i-1 ] );

     free( (char *) null_table->stats );

     null_table->stats =

       (STATS &) calloc( sizeof( STATS ), 256 );

     if ( null_table->stats == NULL )

       fatal_error( "Failure #3:  allocating null table!" );

     null_table->max_index = 255;

     for ( i=0 ; i < 256 ; i++ ) {

       null_table->stats[ i ].symbol = (unsigned char) i;

       null_table->stats[ i ].counts = 1;

     }

     control_table = (CONTEXT *) calloc( sizeof(CONTEXT), 1 );

     if ( control_table == NULL )

       fatal_error( "Failure #4:  allocating null table!" );

     control_table->stats =

       (STATS *) calloc( sizeof( STATS ), 2 );

     if ( control_table->stats == NULL )

       fatal_error( "Failure #5:  allocating null table!" );

     contexts[ -2 ] = control_table;

     control_table->max_index = 1;

     control_table->stats[ 0 ].symbol = -FLUSH;

     control_table->stats[ 0 ].counts = 1;

     control_table->stats[ 1 ].symbol =– DONE;

     control_table->stats[ 1 ].counts = 1;

     for ( i = 0 ; i < 256 ; i++ )

       scoreboard[ i ] = 0;

}

/*

* This is a utility routine used to create new tables when a new

* context is created.  It gets a pointer to the current context,

* and gets the symbol that needs to be added to it.  It also needs

* a pointer to the lesser context for the table that is to be

* created.  For example, if the current context was "ABC", and the

* symbol 'D' was read in, add_character_to_model would need to

* create the new context "BCD".  This routine would get called

* with a pointer to "BC", the symbol 'D', and a pointer to context

* "CD".  This routine then creates a new table for "BCD", adds it

* to the link table for "BC", and gives "BCD" a back pointer to

* "CD".  Note that finding the lesser context is a difficult

* task, and isn't done here.  This routine mainly worries about

* modifying the stats and links fields in the current context.

*/

CONTEXT *allocate_next_order_table( table, symbol, lesser_context )

CONTEXT *table;

int symbol;

CONTEXT *lesser_context;

{

     CONTEXT *new_table;

     int i;

     unsigned int new_size;

     for ( i = 0 ; i <= table->max_index ; i++ )

       if (table->stats[ i ].symbol == (unsigned char) symbol )

         break;

       if ( i > table->max_index ) {

         table->max_index++;

         new_size = sizeof( LINKS );

         new_size *= table->max_index + 1;

         if ( table->links == NULL )

           table->links = (LINKS *) calloc( new_size, 1 );

         else

           table->links = (LINKS *)

             realloc( (char *) table->links, new_size );

         new_size = sizeof( STATS );

         new_size *= table->max_index + 1;

         if ( table->stats == NULL )

           table->stats = (STATS *) calloc( new_size, 1 );

         else

           table->stats = (STATS *)

             realloc( (char *) table->stats, new_size );

         if ( table->links == NULL )

           fatal_error( "Failure #6:  allocating new table" );

         if ( table->stats == NULL )

           fatal_error( "Failure #7:  allocating new table" );

         table->stats[ i ].symbol = (unsigned char) symbol;

         table->stats[ i ].counts = 0;

     }

     new_table = (CONTEXT *) calloc(sizeof( CONTEXT ), 1 );

     if ( new_table == NULL )

       fatal_error( "Failure #8:  allocating new table" );

     new_table->max_index = -1;

     table->links[ i ].next = new_table;

     new_table->lesser_context = lesser_context;

     return( new_table );

}

/*

* This routine is called to increment the counts for the current

* contexts.  It is called after a character has been encoded or

* decoded.  All it does is call update_table for each of the

* current contexts, which does the work of incrementing the count.

* This particular version of update_model() practices update exclusion,

* which means that if lower order models weren't used to encode

* or decode the character, they don't get their counts updated.

* This seems to improve compression performance quite a bit.

* To disable update exclusion, the loop would be changed to run

* from 0 to max_order, instead of current_order to max_order.

*/

void update_model( symbol )

int symbol;

{

     int i;

     int local_order;

     if ( current_order < 0 )

       local_order = 0;

     else

       local_order = current_order;

     if ( symbol >= 0 ) {

       while ( local_order <= max_order ) {

         if ( symbol >= 0 )

           update_table( contexts[ local_order ], symbol );

         local_order++;

        }

     }

     current_order = max_order;

     for ( i = 0 ; i < 256 ; i++ )

       scoreboard[ i ] = 0;

}

/*

* This routine is called to update the count for a particular symbol

* in a particular table.  The table is one of the current contexts,

* and the symbol is the last symbol encoded or decoded.  In principle

* this is a fairly simple routine, but a couple of complications make

* things a little messier.  First of all, the given table may not

* already have the symbol defined in its statistics table.  If it

* doesn't, the stats table has to grow and have the new guy added

* to it.  Secondly, the symbols are kept in sorted order by count

* in the table so that the table can be trimmed during the flush

* operation.  When this symbol is incremented, it might have to be moved

* up to reflect its new rank.  Finally, since the counters are only

* bytes, if the count reaches 255, the table absolutely must be rescaled

* to get the counts back down to a reasonable level.

*/

void update_table( table, symbol )

CONTEXT *table;

int symbol;

{

     int i;

     int index;

     unsigned char temp;

     CONTEXT *temp_ptr;

     unsigned int new_size;

/*

* First, find the symbol in the appropriate context table.  The first

* symbol in the table is the most active, so start there.

*/

     index = 0;

     while ( index <= table->max_index &&

          table->stats[index].symbol != (unsigned char) symbol )

       index++;

     if ( index > table->max_index ) {

       table->max_index++;

       new_size = sizeof( LINKS );

       new_size *= table->max_index + 1;

       if ( current_order < max_order ) {

         if ( table->max_index == 0 )

           table->links - (LINKS *) calloc( new_size, 1 );

         else

           table->links = (LINKS *)

             realloc( (char *) table->links, new_size );

         if (  table->links == NULL )

           fatal_error( "Error #9:  reallocating table space!" );

         table->links[ index ].next = NULL;

     }

     new_size = sizeof( STATS );

     new_size *= table->max_index + 1;

     if (table->max_index==0)

       table->stats = (STATS *) calloc( new_size, 1 );

     else

       table->stats = (STATS *)

         realloc( (char *) table->stats, new_size );

     if ( table->stats == NULL )

       fatal_error( "Error #10:  reallocating table space!" );

     table->stats[ index ].symbol = (unsigned char) symbol;

     table->stats[ index ].counts = 0;

   }

/*

* Now I move the symbol to the front of its list.

*/

     i = index;

     while ( i > 0 &&

       table->stats[ index ]. counts == table->stats[ i-1 ].counts )

       i--;

     if ( i != index ) {

       temp = table->stats[ index ].symbol;

       table->stats[ index ].symbol = table->stats[ i ].symbol;

       table->stats[ i ].symbol = temp;

       if ( table->links != NULL ) {

         temp_ptr = table->links[ index ].next;

         table->links[ index ].next = table->links[ i ].next;

         table->links[ i ].next = temp_ptr;

       }

       index = 1;

  }

/*

* The switch has been performed, now I can update the counts

*/

  table->stats[ index ].counts++;

  if ( table->stats[ index ].counts == 255 )

    rescale_table( table );

}

/*

* This routine is called when a given symbol needs to be encoded.

* It is the job of this routine to find the symbol in the context

* table associated with the current table, and return the low and

* high counts associated with that symbol, as well as the scale.

* Finding the table is simple.  Unfortunately, once I find the table,

* I have to build the table of cumulative counts, which is

* expensive, and is done elsewhere.  If the symbol is found in the

* table, the appropriate counts are returned.  If the symbol is

* not found, the ESCAPE symbol probabilities are returned, and

* the current order is reduced.  Note also the kludge to support

* the order -2 character set, which consists of negative numbers

* instead of unsigned chars.  This insures that no match will ever

* be found for the EOF or FLUSH symbols in  any "normal" table.

*/

int convert_int_to_symbol( c, s )

int c;

SYMBOL *s;

{

     int i;

     CONTEXT *table;

     table = contexts[ current_order ];

     totalize_table( table );

     s->scale = totals[ 0 ];

     if ( current_order == –2 )

       c = -c;

     for ( i = 0 ; i <= table->max_index ; i++ ) {

       if ( c == (int) table->stats[ i ].symbol ) {

         if ( table->stats[ i ].counts == 0 )

           break;

         s->low_count = totals[ i+2 ];

         s->high_count = totals[ i+1 ];

         return( 0 );

       }

     }

     s->low_count = totals[ 1 ];

     s->high-count = totals[ 0 ];

     current_order––;

     return( 1 );

}

/*

* This routine is called when decoding an arithmetic number.  In

* order to decode the present symbol, the current scale in the

* model must be determined.  This requires looking up the current

* table, then building the totals table.  Once that is done, the

* cumulative total table has the symbol scale at element 0.

*/

void get_symbol_scale( s)

SYMBOL *s;

{

     CONTEXT *table;

     table = contexts[ current_order ];

     totalize_table( table );

     s->scale = totals[ 0 ];

}

/*

* This routine is called during decoding.  It is given a count that

* came out of the arithmetic decoder, and has to find the symbol that

* matches the count.  The cumulative totals are already stored in the

* totals[] table, from the call to get_symbol-scale, so this routine

* just has to look through that table.  Once the match is found,

* the appropriate character is returned to the caller.  Two possible

* complications.  First, the character might be the ESCAPE character,

* in which case the current_order has to be decremented.  The other

* complication.  First, the character might be the ESCAPE character,

* in which case the current_order has to be decremented.  The other

* complication is that the order might be -2, in which case we return

* the negative of the symbol so it isn't confused with a normal

* symbol.

*/

int convert_symbol_to_int( count, s )

int count;

SYMBOL *s;

{

     int c;

     CONTEXT *table;

     table - contexts[ current_order ];

     for ( c = 0; count < totals[ c ] ; c++ )

       ;

     s->high_count = totals[ c - 1 ];

     s->low_count = totals[ c ]:

     if ( c == 1 ) {

       current_order––;

       return( ESCAPE );

     }

     if ( current_order < -1 )

       return( (int) -table->stats[ c-2 ].symbol );

     else

       return( table->stats[ c-2 ].symbol );

}

/*

* After the model has been updated for a new character, this routine

* is called to "shift" into the new context.  For example, if the

* last context was "ABC", and the symbol 'D' had just been processed,

* this routine would want to update the context pointers to that

* context[1]=="D", contexts[2]=="CD" and contexts[3]=="BCD".  The

* potential problem is that some of these tables may not exist.

* The way this is handled is by the shift_to_next_context routine.

* It is passed a pointer to the "ABC" context, along with the symbol

* 'D', and its job is to return a pointer to "BCD".  Once we have

* "BCD", we can follow the lesser context pointers in order to get

* the pointers to "CD" and "C".  The hard work was done in

* shift_to_context().

*/

void add_character_to_model( c )

int c;

{

     int i;

     if ( max_order < 0 || c < 0 )

       return;

     contexts[ max_order ] =

       shift_to_next_context( contexts[ max_order ], c, max_order );

     for ( i = max_order-1 ; i > 0 ; i–– )

       contexts[ i ] = contexts[ i+1 ]->lesser_context;

}

/*

* This routine is called when adding a new character to the model.  From

* the previous example, if the current context was "ABC", and the new

* symbol was 'D', this routine would get called with a pointer to

* context table "ABC", and symbol 'D', with order max_order.  What this

* routine needs to do then is to find the context table "BCD".  This

* should be an easy job, and it is if the table already exists.  All

* we have to in that case is follow the back pointer from "ABC" to "BC".

* We then search the link table of "BC" until we find the link to "D".

* That link points to "BCD", and that value is then returned to the

* caller.  The problem crops up when "BC" doesn't have a pointer to

* "BCD".  This generally means that the "BCD" context has not appeared

* yet.  When this happens, it means a new table has to be created and

* added to the "BC" table.  That can be done with a single call to

* the allocate_new_table routine.  The only problem is that the

* allocate_new_table routine wants to know what the lesser context for

* the new table is going to be.  In other words, when I create "BCD",

* I need to know where "CD" is located.  In order to find "CD", I

* have to recursively call shift_to_next_context, passing it a pointer

* to context "C" and the symbol 'D'.  It then returns a pointer to

* "CD", which I use to create the "BCD" table.  The recursion is

* guaranteed to end if it ever gets to order -1, because the null table

* is guaranteed to have a link for every symbol to the order 0 table.

* This is the most complicated part of the modeling program, but it is

* necessary for performance reasons.

*/

CONTEXT *shift_to_next_context( table, c, order )

CONTEXT *table;

int c;

int order;

{

     int i;

     CONTEXT *new_lesser;

/*

* First, try to find the new context by backing up to the lesser

* context and searching its link table.  If I find the link, we take

* a quick and easy exit, returning the link.  Note that there is a

* special kludge for context order 0.  We know for a fact that

* the lesser context pointer at order 0 points to the null table,

* order -1, and we know that the -1 table only has a single link

* pointer, which points back to the order 0 table.

*/

     table = table->lesser_context;

     if ( order == 0 )

       return( table->links[ 0 ].next );

     for ( i = 0 ; i <= table->max_index ; i++ )

       if ( table->stats[ i ].symbol == (unsigned char) c )

         if ( table->links[ i ].next != NULL)

           return( table->links[ i ].next );

         else

           break;

/*

* If I get here, it means the new context did not exist.  I have to

* create the new context, add a link to it here, and add the backwards

* link to *his* previous context.  Creating the table and adding it to

* this table is pretty easy, but adding the back pointer isn't.  Since

* creating the new back pointer isn't easy, I duck my responsibility

* and recurse to myself in order to pick it up.

*/

  new_lesser = shift_to_next_context( table, c, order-1 );

/*

* Now that I have the back pointer for this table, I can make a call

* to a utility to allocate the new table.

*/

  table = allocate_next_order_table( table, c, new_lesser );

  return( table );

}

/*

* Rescaling the table needs to be done for one of three reasons.

* First, if the maximum count for the table has exceeded 16383, it

* means that arithmetic coding using 16 and 32 bit registers might

* no longer work.  Secondly, if an individual symbol count has

* reached 255, it will no longer fit in a byte.  Third, if the

* current model isn't compressing well, the compressor program may

* want to rescale all tables in order to give more weight to newer

* statistics.  All this routine does is divide each count by 2.

* If any counts drop to 0, the counters can be removed from the

* stats table, but only if this is a leaf context.  Otherwise, we

* might cut a link to a higher order table.

*/

void rescale_table( table )

CONTEXT *table;

{

     int i;

     if ( table->max_index == -1 )

       return;

     for ( i = 0 ; i <= table->max_index ; i ++ )

       table->stats[ i ].counts /= 2;

     if ( table->stats[ table]>max_index ].counts == 0 &&

         table->links == NULL ) {

         while ( table->stats[ table->max_index ].counts == 0 &&

              table->max_index >= 0 )

           table->max_index––;

         if ( table->max_index == -1 ) {

           free( (char *) table->stats );

           table->stats = NULL;

         } else {

           table->stats = (STATS *)

             realloc( (char *) table->stats,

                     sizeof( STATS ) * ( table->max_index + 1 ) );

           if ( table->stats == NULL )

             fatal_error( "Error #11: reallocating stats space!" );

     }

   }

}

/*

* This routine has the job of creating a cumulative totals table for

* a given context.  The cumulative low and high for symbol c are going to

* be stored in totals[c+2] and totals[c+1].  Locations 0 and 1 are

* reserved for the special ESCAPE symbol.  The ESCAPE symbol

* count is calculated dynamically, and changes based on what the

* current context looks like.  Note also that this routine ignores

* any counts for symbols that have already shown up in the scoreboard,

* and it adds all new symbols found here to the scoreboard.  This

* allows us to exclude counts of symbols that have already appeared in

*  higher order contexts, improving compression quite a bit.

*/

void totalize_table( table )

CONTEXT *table;

{

     int i;

     unsigned char max;

     for ( ; ; ) {

       max = 0;

       i = table->max_index + 2;

       totals[ i ] = 0;

       for ( ; i > 1 ; i- ) {

         totals[ i-1 ] = totals[ i ];

         if ( table->stats[ i-2 ].counts )

           if ( ( current_order == -2 ) ||

             scoreboard[ table->stats[ i-2 ].symbol ] == 0 )

             totals[ i-1 ] += table->stats[ i-2].counts;

         if ( table->stats[ i-2 ].counts > max )

           max = table->stats[ i-2 ].counts;

       }

/*

* Here is where the escape calculation needs to take place.

*/

     if ( max == 0 )

       totals[ 0 ] = 1;

     else {

       totals[ 0 ] = (short int) ( 256 - table->max_index );

       totals[ 0 ] *= table->max_index;

       totals[ 0 ] /= 256;

       totals[ 0 ] /= max;

       totals[ 0 ]++;

       totals[ 0 ] += totals[ 1 ];

     }

     if ( totals[ 0 ] < MAXIMUM_SCALE )

       break;

     rescale_table( table );

     }

     for ( i = 0 ; i < table->max_index ; i++ )

       if (table->stats[i].counts != 0)

         scoreboard[ table->stats[ i ].symbol ] = 1;

}

/*

* This routine is called when the entire model is to be flushed.

* This is done in an attempt to improve the compression ratio by

* giving greater weight to upcoming statistics.  This routine

* starts at the given table, and recursively calls itself to

* rescale every table in its list of links.  The table itself

* is then rescaled.

*/

void recursive_flush( table )

CONTEXT *table;

{

     int i;

     if ( table->links != NULL )

       for ( i = 0 ; i <= table->max_index ; i++ )

         if ( table->links[ i ].next != NULL )

           recursive_flush( table->links[ i ].next );

     rescale_table( table );

}

/*

* This routine is called to flush the whole table, which it does

* by calling the recursive flush routine starting at the order 0

* table.

*/

void flush_model()

{

  putc( 'F', stdout );

  recursive_flush( contexts[ 0 ] );

}

/*

* Everything from here down define the arithmetic coder section

* of the program.

*/

*/

* These four variables define the current state of the arithmetic

* coder/decoder.  They are assumed to be 16 bits long.  Note that

* by declaring them as short ints, they will actually be 16 bits

* on most 80X86 and 680X0 machines, as well as VAXen.

*/

static unsigned short int code;/* The present input code value      */

static unsigned short int low; /* Start of the current code range   */

static unsigned short int high;/* End of the current code range     */

long underflow_bits;           /* Number of underflow bits pending

*/

/*

* This routine must be called to initialize the encoding process.

* The high register is initialized to all 1s, and it is assumed that

* it has an infinite string of 1s to be shifted into the lower bit

* positions when needed.

*/

void initialize_arithmetic_encoder()

{

     low = 0;

     high = 0xffff;

     underflow_bits = 0;

}

/*

* At the end of the encoding process, there are still significant

* bits left in the high and low registers.  We output two bits,

* plus as many underflow bits as are necessary.

*/

void flush_arithmetic_encoder( stream )

BIT_FILE *stream;

{

      OutputBit( stream, low & 0x4000 );

      underflow_bits++;

      while ( underflow_bits-- > 0 )

        OutputBit( stream, ~low & 0X4000 );

      OutputBits( stream, 0L, 16 );

}

/*

* This routine is called to encode a symbol.  The symbol is passed

* in the SYMBOL structure as a low count, a high count, and a range,

* instead of the more conventional probability ranges.  The encoding

* process takes two steps.  First, the values of high and low are

* updated to take into account the range restriction created by the

* new symbol.  Then, as many bits as possible are shifted out to

* the output stream.  Finally, high and low are stable again and

* the routine returns.

*/

void encode_symbol( stream, s )

BIT_FILE *stream;

SYMBOL *s;

{

  long range;

/*

*  These three lines rescale high and low for the new symbol.

*/

     range = (long) ( high-low ) + 1;

     high = low + (unsigned short int)

                  (( range * s->high_count ) / s->scale -1 );

     low = low + (unsigned short int)

                  (( range * s->low_count ) / s->scale );

/*

* This loop turns out new bits until high and low are far enough

* apart to have stabilized.

*/

  for ( ; ; ) {

/*

* If this test passes, it means that the MSDigits match, and can

* be sent to the output stream.

*/

     if ( ( high & 0x8000 ) == ( low & 0x8000 ) ) {

       OutputBit( stream, high & 0x8000 );

       while ( underflow_bits > 0 ) {

         OutputBit( stream, ~high & 0x8000 );

         underflow_bits--;

       }

     }

/*

* If this test passes, the numbers are in danger of underflow, because

* the MSDigits don't match, and the 2nd digits are just one apart.

*/

     else if ( ( low & 0x4000 ) && !( high & 0x4000 )) {

       underflow_bits += 1;

       low &= 0x3fff;

       high |= 0x4000;

     } else

       return ;

     low <<= 1;

     high <<= 1;

     high  |= 1;

   }

}

/*

* When decoding, this routine is called to figure out which symbol

* is presently waiting to be decoded.  This routine expects to get

* the current model scale in the s->scale parameter, and it returns

* a count that corresponds to the present floating point code;

*

* code = count / s->scale

*/

short int get_current_count( s )

SYMBOL *s;

{

     long range;

     short int count;

     range = (long) ( high - low ) + 1;

     count = (short int)

          ((((long) ( code - low ) + 1 ) * s->scale-1 ) / range );

     return( count );

}

/*

* This routine is called to initialize the state of the arithmetic

* decoder.  This involves initializing the high and low registers

* to their conventional starting values, plus reading the first

* 16 bits from the input stream into the code value.

*/

void initialize_arithmetic_decoder( stream )

BIT_FILE *stream;

{

     int i;

     code = 0;

     for ( i = 0 ; i < 16 ; i++ ) {

        code <<= 1;

        code += InputBit( stream );

     }

     low = 0;

     high = 0xffff;

}

/*

* Just figuring out what the present symbol is doesn't remove

* it from the input bit stream.  After the character has been

* decoded, this routine has to be called to remove it from the

* input stream.

*/

void remove_symbol_from_stream( stream, s )

BIT_FILE *stream;

SYMBOL *s;

{

  long range;

/*

* First, the range is expanded to account for the symbol removal.

*/

     range = (long)( high - low ) + 1;

     high = low + (unsigned short int)

                  (( range * s->high_count ) / s->scale -1 );

     low = low + (unsigned short int)

                  (( range * s->low_count ) / s->scale );

/*

* Next, any possible bits are shipped out.

*/

     for ( ; ; ) {

/*

* If the MSDigits match, the bits will be shifted out.

*/

     if ( ( high & 0x8000 ) == ( low & 0x8000 ) ) {

     }

/*

* Else, if underflow is threatening, shift out the 2nd MSDigit.

*/

     else if ((low & 0x4000) == 0x4000  && (high & 0x4000) == 0 ) {

       code ^= 0x4000;

       low &= 0x3fff;

       high |= 0x4000;

     } else

/*

* Otherwise, nothing can be shifted out, so I return.

*/

        return;

     low <<= 1;

     high <<= 1;

     high |= 1;

     code <<= 1;

     code += InputBit( stream );

   }

}

/************************** End of ARITH-N.C ***************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 7
Dictionary-Based Compression

So far, the compression methods we have looked at used a statistical model to encode single symbols. They achieve compression by encoding symbols into bit strings that use fewer bits than the original symbols. The quality of the compression goes up or down depending on how good the program is at developing a model. The model not only has to accurately predict the probabilities of symbols, it also has to predict probabilities that deviate from the mean. More deviation achieves better compression.

But dictionary-based compression algorithms use a completely different method to compress data. This family of algorithms does not encode single symbols as variable-length bit strings; it encodes variable-length strings of symbols as single tokens. The tokens form an index to a phrase dictionary. If the tokens are smaller than the phrases they replace, compression occurs.

In many respects, dictionary-based compression is easier for people to understand. It represents a strategy that programmers are familiar with—using indexes into databases to retrieve large amounts of storage. In everyday life, we use phone numbers, Dewey Decimal numbers, and postal codes to encode larger strings of text. This is essentially what a dictionary-based encoder does.

An Example

A good example of how dictionary based compression works can be created by using a standard dictionary. For this example, I will use the Random House Dictionary of the English Language, Second Edition, Unabridged. Using this dictionary's system as a key for encoding messages, I can achieve a reasonable amount of compression. Using my proprietary scheme, the first eight words of the first sentence in this paragraph would read:

1/1 822/3 674/4 1343/60 928/75 550/32 173/46 421/2

This dictionary-based encoding scheme consists of a simple lookup table. The first number gives the page of the dictionary, and the second number tells the number of the word on that page. The dictionary has 2,200 pages with less than 256 entries on each page. Thus, 1/1 encodes the first word on the first page, which is "A." 822/3 encodes the third word on the 822nd page, which is "good."

To see how much space this scheme would save, look at the number of bits actually used to encode a word. Since a word can land on any of 2,200 pages, we need 12 bits to encode the page number. Each page has fewer than 256 entries, so the number of the entry will take just 8 bits to encode. This gives a total of 20 bits to encode any word in the dictionary, or 2.5 bytes per word.

The ASCII representation of the eight words in our encoded message takes 43 bytes. The encoded message takes 2.5 × 8 bytes, or 20 bytes. Thus, we compressed our text to 50 percent of its original size using dictionary encoding.

In theory, a different encoding method can probably improve on this. The dictionary has about 315,000 words. Shannon's formula for information content tells us that any one of the words in the dictionary can be encoded using just a little over eighteen bits. We used the page number/entry number scheme to make it easier to look up the encoded word, a general theme in dictionary-based compression.

Static vs. Adaptive

In general, dictionary-based compression replaces phrases with tokens. If the number of bits in the token is less than the number of bits in the phrase, compression will occur. But this definition of dictionary-based compression still leaves enormous room for variation. Consider, for example, the methods for building and maintaining a dictionary.

In some cases, it is advantageous to use a predefined dictionary to encode text. If the text to be encoded is a database containing all motor-vehicle registrations for Texas, we could develop a dictionary with only a few thousand entries that concentrated on words like "General Motors," "Smith," "Main," and "1977." Once this dictionary were compiled, it could be kept on-line and used by both the encoder and decoder as needed.

A dictionary like this is called a static dictionary. It is built up before compression occurs, and it does not change while the data is being compressed. It has advantages and disadvantages. One of the biggest advantages is that a static dictionary can be "tuned" to fit the data it is compressing. With the motor-vehicle registration database, for example, Huffman encoding could allocate fewer bits to strings such as "Ford" and more bits to "Yugo." Of course, we could use different bit strings depending on which field is being compressed.

Adaptive compression schemes can't tune their dictionaries in advance, which in principle would seem a major disadvantage. But static dictionary schemes have to deal with the problem of how to pass the dictionary from the encoder to the decoder. Chapters 3 and 5 showed that passing statistics along with compressed data can significantly harm compression, particularly on small files.

But this doesn't have to be a disadvantage in every case. In many situations, a static dictionary could remain the same over long periods of time and be kept on line, available to both the compressor and the decompressor. The motor-vehicle database dictionary could be calculated once, for example, then kept on hand. In the case of an exceptionally large amount of data, the compression ratio may not be significantly degraded if the dictionary is passed with the compressed text.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Adaptive Methods

At present, dictionary-based compression schemes using static dictionaries are mostly ad hoc, implementation dependent, and not general purpose. Most well-known dictionary algorithms are adaptive. Instead of having a completely defined dictionary when compression begins, adaptive schemes start out either with no dictionary or with a default baseline dictionary. As compression proceeds, the algorithms add new phrases to be used later as encoded tokens.

The basic principle behind adaptive dictionary programs is relatively easy to follow. Imagine a section of code that compressed text using an algorithm that looked something like this:

for ( ; ; ) {

  word = read_word( input_file );

  dictionary_index = look_up( word, dictionary );

  if ( dictionary_index < 0 ) {

    output( word, output_file );

    add_to_dictionary( word, dictionary );

  } else

    output( dictionary_index, output_file );

}

If the dictionary index used here could be encoded as an integer index into a table, we would achieve respectable compression with what is actually a very simple algorithm. This code is a specialized one set up to apply to written documents, but the principle behind it is similar to that behind many more sophisticated algorithms. It illustrates the basic components of an adaptive dictionary compression algorithm;

1. To parse the input text stream into fragments tested against the dictionary.
2. To test the input fragments against the dictionary; it may or may not be desirable to report on partial matches.
3. To add new phrases to the dictionary.
4. To encode dictionary indices and plain text so that they are distinguishable.

The corresponding decompression program has a slightly different set of requirements. It no longer has to parse the input text stream into fragments, and it doesn't have to test fragments against the dictionary. Instead, it has the following requirements: (1) to decode the input stream into either dictionary indices or plain text; (2) to add new phrases to the dictionary; (3) to convert dictionary indices into phrases; and (4) to output phrases as plain text. The ability to accomplish these tasks with relatively low costs in system resources made dictionary-based programs popular over the last ten years.

A Representative Example

Compressing data when sending it to magnetic tape has several nice side effects. First, it reduces the use of magnetic tape. Though magnetic tape is not particulary expensive, some applications make prodigous use of it. Second, the effective transfer rate to and from the tape is increased. Improvements in transfer speed through hardware are generally expensive, but compression through software is in a sense "free." Finally, in some cases, the overall CPU time involved may actually be reduced. If the CPU cost of writing a byte to magnetic tape is sufficiently high, writing half as many compressed bytes may save enough cycles to pay for the compression.

While the benefits of compressing data before sending it to magnetic tape have been clear, only sporadic methods were used until the late 1980s. In 1989, however, Stac Electronics successfully implemented a dictionary-based compression algorithm on a chip. This algorithm was quickly embraced as an industry standard and is now widely used by tape-drive manufacturers worldwide.

This compression method is generally referred to by the standard which defines it: QIC-122. (QIC refers to the Quarter Inch Cartridge industry group, a trade association of tape-drive manufacturers.) As you may know, Stac Electronics expanded the scope of this algorithm beyond tape drives to the consumer hard disk utility market in the form of its successful Stacker program (discused later in this chapter).

QIC-122 provides a good example of how a sliding-window, dictionary-based compression algorithm actually works. It is based on the LZ77 sliding-window concept. As symbols are read in by the encoder, they are added to the end of a 2K window that forms the phrase dictionary. To encode a symbol, the encoder checks to see if it is part of a phrase already in the dictionary. If it is, it creates a token that defines the location of the phrase and its length. If it is not, the symbol is passed through unencoded.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The output of a QIC-122 encoder consists of a stream of data, which, in turn, consists of tokens and symbols freely intermixed. Each token or symbol is prefixed by a single bit flag that indicates whether the following data is a dictionary reference or a plain symbol. The definitions for these two sequences are: (1) plaintext: <1><eight-bit-symbol>; (2) dictionary reference: <0><window-offset><phrase-length>.

The QIC-122 encoder complicates things by further encoding the window-offset and phrase-length codes. Window offsets of less than 128 bytes are encoded in seven bits. Offsets between 128 bytes and 2,047 bytes are encoded in eleven bits. The phrase length uses a variable-bit coding scheme which favors short phrases over long. This explanation will gloss over these as "implementation details." The glossed-over version of the C code for this algorithm is shown here.

while ( !out_of_symbols ) {

  length = find_longest_match(&offset);

  if ( length > 1 ) {

    output_bit( 0 );

    length = find_longest_match( &offset );

    output_bits( offset );

    output_bits( length );

    shift_input_buffer( length );

  } else {

    output_bit( 1 );

    output_byte( buffer[ 0 ] );

    shift_input_buffer( 1 );

  }

}

Following is an example of what this sliding window looks like when used to encode some C code, in this case the phrase "output_byte." The previously encoded text, which ends with the phrase "output_bit( 1 );\r," is at the end of the window. The find_longest_match routine will return a value of 8, since the first eight characters of "output_byte" match the first eight characters of "output_bit." The encoder will then output a 0 bit to indicate that a dictionary reference is following. Next it will output a 15 to indicate that the start of the phrase is fifteen characters back into the window ('\r' is a single symbol). Finally, it will output an 8 to indicate that there are eight matching symbols from the phrase.

Figure 7.1 A sliding window used to encode some C code.

Using QIC-122 encoding, this will take exactly sixteen bits to encode, which means it encodes 8 bytes of data with only 2 bytes. This is clearly a respectable compression ratio, typical of how QIC-122 works under the best circumstances as shown here:

Figure 7.2 Encoding 8 bytes of data using only 2 bytes.

After the dictionary reference is output, the input stream over eight characters, with the last symbol encoded becoming the last symbol in the window. The next three symbols will not match anything in the window, so they will have to be individually encoded.

This example of QIC-122 gives a brief look at how a dictionary-based compression scheme might work. Chapter 8 will take a more extensive look at LZ77 and its derivatives.

Israeli Roots

Dig beneath the surface of virtually any dictionary-based compression program, and you will find the work of Jacob Ziv and Abraham Lempel. For all practical purposes, these two Israeli researchers gave birth to this branch of information theory in the late 1970s.

Research in data compression up to 1977 included work on entropy, character and word frequencies, and various other facets of statistical modeling. There were minor forays into other esoteric areas of interest, such as finite state machines and linguistic models, but research went mainly into new and improved methodologies for driving Huffman coders.

All this changed in 1977 with the publication of Jacob Ziv's and Abraham Lempel's "A Universal Algorithm for Sequential Data Compression" in IEEE Transactions on Information Theory. This paper, with its 1978 sequel "Compression of Individual Sequences via Variable-Rate Coding," triggered a flood of dictionary-based compression research, algorithms, and programs.

The two compression techniques developed in these papers are called LZ77 and LZ78 (the transposition of the author's initials is apparently an innocent historical accident, but one that is here to stay). LZ77 is a "sliding window" technique in which the dictionary consists of a set of fixed-length phrases found in a "window" into the previously processed text. The size of the window is generally somewhere between 2K and 16K bytes, with the maximum phrase length ranging from perhaps 16 to 64 bytes. LZ78 takes a completely different approach to building a dictionary. Instead of using fixed-length phrases from a window into the text, LZ78 builds phrases up one symbol at a time, adding a new symbol to an existing phrase when a match occurs.

It is easy to think that these two compression methods are closely related, particularly since people will casually speak of "Lempel Ziv Compression" as if it were just one thing. These are, however, two very different techniques. They have different implementation problems and solutions, different strengths and weaknesses. Since they have had such a large impact on the world of data compression, the next two chapters of this book will take a detailed look at an LZ77 and an LZ78 implementation.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

History

While the publication of the two papers in 1977 and 1978 may have had an immediate impact in the world of information theory, it was some time before programmers noticed the effects. In fact, it took the publication of another paper in 1984 to really get things moving.

The June 1984 issue of IEEE Computer had an article entitled "A Technique for High-Performance Data Compression" by Terry Welch. Welch described work performed at Sperry Research Center (now part of Unisys). His paper was a practical description of this implementation of the LZ78 algorithm, which he called LZW. It discussed the LZW compression algorithm in reference to its possible use in disk and tape-drive controllers, but it was clear that the same algorithm could easily be built into a general-purpose compression program.

Almost immediately after the article appeared, work began on the Unix compress program. compress is a C program developed initially on the DEC's VAX. Ports to other machines, including the IBM PC, followed shortly. The public release of compress became available almost exactly a year after the publication of the IEEE article.

Compress was a very influential program for a number of reasons. The program was well written. It performed well, and it had a reasonable level of documentation. Many UNIX installations began actively using compress soon after its release. Manual pages distributed with UNIX systems are now routinely shipped in compressed form, and they are not decompressed until accessed for the first time by the man program. The code was in the public domain from its initial release, which made for wide distribution and study. Perhaps most importantly, the authors went out of their way to ensure that the code was portable so that it could be used on a wide variety of systems with no modifications.

While compress was becoming a standard in the UNIX community, desktop software was still struggling along with a rather inefficient order-0 Huffman coding program known as SQ. But in 1985, desktop power was increasing; more and more people were using modems to communicate; and hard-disk space was still relatively expensive. Conditions were ripe for an improvement in compression, and dictionary-based coding stepped in.

ARC: The Father of MS-DOS Dictionary Compression

In 1985, System Enhancement Associates released a general-purpose compression and cataloging program called ARC. ARC quickly took the MS-DOS desktop world by storm, becoming a de facto standard for PC users in a matter of months. Several factors helped ARC gain this position. First, it ordinarily used a close derivative of compress to compress files. At the time, this provided state-of-the-art compression and was essentially without peer. Second, ARC provided a cataloging or archiving function as an integral part of the program. UNIX users were accustomed to using the "tar" program to combine groups of files into a single archive, but PC users did not have a similar function as part of their operating system. ARC added that capability, vital for transferring groups of files by modem or even floppy diskette. Finally, ARC was distributed as shareware, which helped saturate the user base in a short time.

With compress reigning supreme in the UNIX world and ARC ruling the MS-DOS world, it seemed LZ78 would be the dominant compression method for years. Imitators such as PKWare's PKARC only strengthened LZ78's hold by providing performance improvements in both speed and compression ratios. But oddly enough, in recent years the field has taken a step back, if you consider moving from LZ78 to LZ77 a step backwards.

ARC lost its dominance of the desktop world to new contenders, most notably PKZIP, by PKWare; but also LHarc, by Haruyasu Yoshizaki; and ARJ, by Robert Jung. These programs are built on an LZ77 algorithm which uses a dictionary based on a sliding window that moves through the text. LZ77 was not a practical algorithm to implement until refinements were made in the mid 1980s. Now LZ77 has a legitimate position alongside LZ78 as co-ruler of the general-purpose compression world.

Most recently, a patent dispute between Unisys, which owns the patent for LZ78-derived algorithms (Terry Welch's work), versus the rest of the computer industry, has resulted in a definite shift over to LZ77-derived algorithms. For example, the recently designed PNG format (discussed later in this book) is being promulgated as a replacement to Compuserve's GIF format, in order to sidestep Unisys' patent claims.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Dictionary Compression: Where It Shows Up

Dictionary-based compression has found more and more homes in the last ten years as both hardware and software improvements make it practical. We can subdivide applications for dictionary-based compression into two areas: general-purpose programs and hardware-specific code.

As shown, dictionary-based coding took over desktop general-purpose compression. In the MS-DOS world, programs such as PKZIP, ARC, ARJ, and LHarc all use dictionary-based algorithms to compress and archive files in a general-purpose manner. Most of these programs have ports to at least one or two other platforms, UNIX being the most popular.

Dictionary-based compression is also used in some special-purpose desktop programs. Most backup programs, for example, use some form of compression to make their operation faster and more efficient. PC Backup, developed by Central Point Software developed (a company later acquired by symantic), uses a dictionary-based algorithm from Stac Electronics, the company that produces the Stacker disk compression utility and which initiated the QIC-122 compression standard.

Compuserve Information Service developed a dictionary-based compression scheme used to encode bit-mapped graphical images. The GIF format uses an LZW variant to compress repeated sequences in screen images. Compression is clearly needed when using these type of images. Computer images take up lots of storage space. As video resolutions improve, the size of the saved images grows dramatically. Compuserve users also typically use modems to upload or download these images. When running on older 2400 baud modems, compressing images becomes even more crucial. Even at 14400 bps and faster speeds, the exploding use of the World-Wide Web on the Internet means increased demand for speedy transfer of graphics, and increased reliance on the GIF format for non-photographic images. (Image-oriented formats will be discussed in later chapters.)

Compressing files before transmitting them saves telecommunications bandwidth. But this requires compatible compression software on both ends. A more convenient method of conserving bandwidth's to build data compression directly into the modem. Microcom Corp. originally developed this idea, which used Huffman coding to compress data before it was transmitted by its modems. Microcom's compression algorithm, MNP-5, uses a dynamic Huffman coding scheme that performs well as a general-purpose compressor on most data streams.

In recent years, the international telecommunications industry has widely ratified and adopted a new compression algorithm used by modem manufacturers: V.42bis, a dictionary-based compression scheme which offers better compression ratios than MNP-5. With the adoption of an international standard, modem builders can now implement data compression in their modems and have confidence that they can communicate with modems from other manufacturers.

As mentioned previously, tape drive manufacturers have adopted an industry-standard compression algorithm: QIC-122. QIC-122 is generally implemented on the tape drive itself or on the tape controller using a dedicated microcontroller or the Stac Electronics compression engine chip. Hewlett-Packard has proposed an alternative compression standard known as DCLZ. DCLZ uses an LZ78-type algorithm, which supposedly offers better compression performance than programs based on QIC-122.

The hard disk has been an active battleground for dictionary-based compression. In recent years, the utility programs for archiving and compressing files have been supplemented by programs that work at the device-driver level to transparently compress data stored on disk. The most visible of these utilities is Stacker, from Stac Electronics. There are others, including Disk Doubler on the Macintosh platform. In version 6 of MS-DOS, Microsoft added Stacker-like compression to its operating system, and then was forced to remove it after a well-publicized lawsuit between Microsoft and Stac Electronics. When operating at the device-driver level, these programs add a level of convenience to compression that is hard to beat.

One of the primary difficulties with compressing data on the hard disk is that most of today's dictionary schemes are adaptive. A general-purpose algorithm would be needed to operate on a disk drive or controller, making a static dictionary difficult to implement. Most adaptive algorithms do not perform very well until they have built up some statistical information, which may take several K of input data.

Unfortunately, disk drives are used in a random-access mode, which means a program can begin reading data at any point in the file. If the file were compressed using a conventional adaptive method, we might have to go back to the start and begin reading there in order to properly decompress the file. This may not be much of a problem in a 16 Kbyte text file, but imagine the performance problems in a 16-Mbyte database!

At present, manufactures of software- and hardware-based disk compression drivers avoid these problems by compressing at the sector level. Since the device driver typically reads in a sector or more at a time, the adaptive algorithm will restart at the begining of each sector boundary. Even better, if the device driver controls the size of the sector, it can be set to a somewhat larger value than might normally be used, giving the adaptive algorithm a chance to improve its compression.

With algorithms such as QIC-122, increasing the sector size past a certain point will not likely improve matters, since the dictionary is only composed of the previous 2K bytes of data. But more powerful compression algorithms that take advantage of older information will frequently want to increase the sector size.

In practice, many users of disk-compression programs that work at the device-driver level find performance to be less of an issue than one might expect. This is due to the fact that, while compression/decompression does consume additional CPU cycles, this effort is compensated for by the reduced amount of data that needs to be transferred between memory and the hard disk. When a mechanical procedure (physically moving the hard disk arm across the platter) competes with an electronic one (decompressing data in a CPU cache), the electronic process usually wins out.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Danger Ahead—Patents

The fact that most work on dictionary-based compression has been done over the last ten or fifteen years has a potentially dangerous side effect. Until the early 1980s, it was generally not possible to patent software. But during the past ten years, increasingly large numbers of patents were awarded for what are clearly algorithms.

One of the first data-compression patents was granted to Sperry Corp. (now Unisys) for the improvements to LZ78 developed by Terry Welch at the Sperry Research Center. In fact, this patent became a point of contention during the standardization process for the V.42bis data-communications standard. Since V.42bis is based on the LZW algorithm, Unisys claimed the right to collect royalties on implementations which use V.42bis. There was some concern in the CCITT about the effect of basing a standard on a patented technique. Unisys dampened concern while protecting its patent rights by publicly offering to license the algorithm to any modem manufacturer for a onetime $25,000 fee.

After a hiatus of several years, Unisys recently woke from slumber and began patent-related legal maneuvers, seeking to get licensing fees from CompuServe and other users of the popular GIF format. Some in the industry viewed this request as unreasonable, given the lengthy delay between the introduction of the GIF format, and the unexpected demands for licensing fees once the format had gained wide acceptance. The response to Unisys' effort was creative rather than contentious. Developers came together and in short order produced the PNG effort, a license-free format that contains many improvements over the existing GIF format.

As research in dictionary-based compression continues, patents are being filed at a relatively rapid pace. Since patent filings are not a matter of public record, it is not possible to know if and when certain techniques will be freely available. At present, the most prudent course for potential data-compression users would be to conduct a patent search and to contact the inventors of any techniques they intend to use.

Fortunately, manufacturers can generally come to terms on patent royalties for relatively modest terms. The danger comes when the owner of the patent is competing for the same market as a potential licensee. Unisys was only too happy to license the LZW algorithm to modem manufacturers, but it may have adopted an entirely different strategy in the online market—that of benign neglect followed by hasty demands.

In recent years, the US Patent Office has come under severe criticism from some sectors in the industry by granting patents that some consider inappropriate or undeserved. The most visible instance was the patent granted to Compton's for simple and widely-used techniques found in its multimedia encyclopedia product. This patent was later invalidated by the Patent Office, after much debate and criticism. Another well-publicized case was the lawsuit between Microsoft and Stac Electronics, which revolved around a number of issues, including certain patents on data compression.

Regarding the general subject of software patents, the computer industry has been split along at least two points of view. One view holds that patents are inappropriate in a dynamic and innovative field such as software, in which a computer program is built using dozens if not hundreds of techniques, algorithms, and data structures (as opposed to a physical/mechanical device or product, whose design centers around a much smaller array of techniques). In the case of software, patents can stifle innovation and economic growth that benefits us all. Large, well-established companies in the industry that generally adhere to this view include Adobe and Oracle, in addition to smaller non-profit organizations such as the League for Programming Freedom (see "Afterword" at the end of this book).

Another point of view is that the patent process, while in some instances flawed and in need of overhaul, is basically well-conceived and should be preserved. Companies like IBM and Apple seem to adhere to this view. Regardless of their view, today most companies have stepped up their patent-related efforts, either as an aggressive business strategy or a precautionary self-defensive maneuver. Microsoft, for example, historically indifferent to the patent process, has bulked up for future skirmishes by hiring a phalanx of intellectual-property lawyers.

After a series of public hearings on the matter of software patents, the US Patent Office, under Commissioner Bruce Lehman, has made improvements, added resources, and modified its instructions to patent examiners so that software patents are more easily granted. Whether these changes will have a positive effect on the computer industry remains to be seen. The only certainty is that patents will continue to play a central role in the field of data compression.

Conclusion

Dictionary-based compression techniques are presently the most popular forms of compression in the lossless arena. Almost without exception, these techniques can trace their origins back to the original work published by Ziv and Lempel in 1977 and 1978. Refinements on these algorithms yield better performance at lower cost, but both types of improvements are evolutionary, not revolutionary.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 8
Sliding Window Compression

The genesis of modern dictionary-based compression can be traced to the 1977 Ziv and Lempel paper, " A Universal Algorithm for Sequential Data Compression," published in IEEE Transactions on Information Theory. In retrospect, this algorithm (referred to hereafter as LZ77) does not seem particularly remarkable. It is simple enough that it could have easily been described thirty or forty years earlier, and there is no doubt that it could have been implemented at least as a "proof of principle" program well before 1977.

However, as was discussed in the previous chapter, till the late 1970s most data compression work concentrated on improved ways to drive Huffman coders and perhaps on more exotic studies of digrams or other statistical topics. So the LZ77 paper truly broke new ground.

The Algorithm

LZ77 compression uses previously seen text as a dictionary. It replaces variable-length phrases in the input text with fixed-size pointers into the dictionary to achieve compression. The amount of compression depends on how long the dictionary phrases are, how large the window into previously seen text is, and the entropy of the source text with respect to the LZ77 model.

The main data structure in LZ77 is a text window, divided into two parts. The first consists of a large block of recently decoded text. The second, normally much smaller, is a look-ahead buffer. The look-ahead buffer has characters read in from the input stream but not yet encoded.

The normal size of the text window is several thousand characters. The look-ahead buffer is generally much smaller, maybe ten to one hundred characters. The algorithm tries to match the contents of the look-ahead buffer to a string in the dictionary. A simplistic example of a text window is shown in Figure 8.1.

Figure 8.1 A text window in use.

Figure 8.1 shows a snippet of C code being compressed. The text window has a total width of 64 characters, with 16 of those characters used by the look-ahead buffer. The LZ77 algorithm, as originally conceived, issued sequences of tokens. Each token consists of three different data items which defined a phrase of variable length in the current look-ahead buffer. The three items in the token are: (1) an offset to a phrase in the text window; (2) the length of the phrase; and (3) the first symbol in the look-ahead buffer that follows the phrase.

In the above example, the look-ahead buffer contains the phrase "<MAX;j++)\r." By searching through the buffer, we find that "<MAX" is located at position 14 in the text window. It matches the look-ahead buffer for the first four symbols. The first symbol not present in the look-ahead buffer is the space character. So this token is encoded as: 14, 4, ' '.

The compression program that implements the LZ77 algorithm first emits the token, then shifts the text window over by five characters, which is the width of the phrase just encoded. Five new symbols are then read into the look-ahead buffer, and the process repeats.

Figure 8.2 The window after encoding 14, 4, ' '

The next token issued by the compression algorithm would encode the phrase ";j+" as "40, 2,'+'." The syntax of this token allows for phrases that have no match of any length in the window. If the look-ahead buffer shown above had no match, for example, it could be encoded a single character at a time using a phrase length of zero: "0, 0, ';'." This method is not efficient, but it ensures that the algorithm can encode any message.

The code to implement this compression algorithm should be fairly simple. It merely has to look through the entire text window for the longest match, encode it, then shift. A brute force application of this algorithm might look something like this:

int window_cmp( char *w, int i, int j, int length )

{

 int count = 0;

 while ( length-- ) {

  if ( w[ i++ ] == w[ j++ ] )

   count++;

  else

   return( count );

 }

 return( count );

}

    .

    .

    .

 match_position = 0;

 match_length = 0;

 for ( i = 0 ; i < ( WINDOW_SIZE - LOOK_AHEAD_SIZE ); i++ ) {

  length = window_cmp( window, i, LOOK_AHEAD, LOOK_AHEAD_SIZE );

  if ( length > match_length ) {

   match_position = i;

   match_length = length;

  }

 }

 encode( match_position, match_length,

         window[ LOOK_AHEAD+match_length ] );

 memmove( window, window+match_length+1, WINDOW_SIZE - match_length );

 for ( i = 0 ; i < match_length+1 ; i++ )

  window[ WINDOW_SIZE - match_length + i ] = getc( input );

   .

   .

   .


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The decompression algorithm for LZ77 is even simpler, since it doesn't have to do comparisons. It reads in a token, outputs the indicated phrase, outputs the following character, shifts, and repeats. It maintains the window, but it does not work with string comparisons. A decompression program that used the output of the previous program might have a loop like this:

decode( &match_position, &match_length, &character );

fwrite( window+match_position, 1, match_length, output );

putc( character, output );

for ( i = 0 ; i < match_length ; i++ )

window[ LOOK_AHEAD+i ] = window[ match_position+i ];

window[ LOOK_AHEAD+i ] = character;

memmove( window, window+match_length+1, WINDOW_SIZE - match_length );

One interesting side effect of this decompression method is that it can use phrases that have not yet been encoded to encode existing phrases. In a file that had one hundred consecutive 'A' characters, for example, we would encode the first A as (0, 0, 'A'). Our window would then look like that shown in Figure 8.3.

Figure 8.3 Coding for one hundred consecutive A characters.

We could then encode the next nine A characters as (38, 9, 'A'). It may seem odd to use a nine-character phrase here. Though we can see the eight characters in the phrase presently in the look-ahead buffer, the decoder won't be able to. When the decoder receives the (38, 9, 'A') token, its buffer will look like Figure 8.4.

Figure 8.4 The buffer for the decoder when it receives the (38, 9, 'A') token.

But by examining the decompression algorithm, you can see how the decompression routine manages this trick. It sits in a loop, copying from the match position to the look-ahead buffer. After the first character has been copies, the buffer looks like Figure 8.5.

Figure 8.5 What the buffer looks like after it copies the first character.

The next time through the loop the second A character will be available to be copied though it was not in the window when the decoding started. After the entire copy is complete, along with the single character store, the buffer is ready to shift, as shown in Figure 8.6.

Figure 8.6 The buffer, when ready to shift.

This illustrates a powerful feature of LZ77 compression: rapid adaptation to the character of the input stream. In this example, it encoded a sequence of ten characters when its "dictionary" had only been loaded with a single character.

Problems with LZ77

The implementation of LZ77 shown here is deliberately crude. It also has to be considered a somewhat liberal interpretation of the algorithm. The authors presented little discussion of implementation details when they presented their method.

There is clearly a major performance bottleneck in the LZ77 approach. When encoding, it has to perform string comparisons against the look-ahead buffer for every position in the text window. As it tries to improve compression performance by increasing the size of the window, and thus the dictionary, this performance bottleneck only gets worse. On the bright side, however, the decompression portion of this algorithm does not have to suffer through this bottleneck. Since it only copies the phrases, it can operate at a much higher rate. Even better, the LZ77 decompressor will not be severely affected by increases in either the size of the text window or the look-ahead buffer.

A second performance problem occurs with the way the sliding window is managed. For conceptual convenience, the discussion here treated the sliding window as though it were truly sliding "across" the text, progressing from the end of the buffer to the front as the encoding process was executed.

This may be conceptually superior, but it is certainly not the best way to code an LZ77 program. In fact, it is much better to have a sliding index or pointer into a fixed buffer. Instead of moving the phrases toward the front of the window, a sliding pointer would keep the next in the same place in the window and move the start and end pointers along the buffer as text is encoded.

Using sliding pointers does create a few problems. For one, we can't use a straightforward string-compare function like strncmp() to look for longest phrases because a phrase may land across the end of the physical window, with the first character at window[ WINDOW_SIZE - 1 ] and the second at window[ 0 ]. This means that as we do string comparisons we need to use a modulo index into the window instead of a normal index. A recoded version of strncmp() that would work properly under these revised circumstances might look like this:

int window_cmp( char *w, int i, int j, int length )

{

 int count = 0;

 while ( length-- ) {

  if ( w[ i ] == w[ j ] )

   count++;

  else

   return( count );

  i = ++i % WINDOW_SIZE;

  j = ++j % WINDOW_SIZE;

 }

 return( count );

}

This routine is slightly more complicated, but it will pay for itself in savings on calls to memmove(). Keeping the buffer in one place is a big savings in CPU cycles. In addition, the routine can be made even more efficient if WINDOW_SIZE is an integral power of 2. The modulus operator can then be replaced by a logical AND, saving even more time.

An Encoding Problem

Besides the CPU cost problems, the LZ77 algorithm has a major efficiency problem. When encoding phrases, LZ77 achieves good compression rapidly. Even if the phrases being substituted for input text are short, they will still generally cause very effective compression to take place.

The problem occurs when matching phrases are not found in the dictionary. When this is the case, the compression program still has to use the same three component tokens to encode a single character. To realize the cost of this, imagine encoding a single character when using a 4,096-byte window and a sixteen-byte look-ahead buffer. This would take twelve bits to encode a window position and another four bits to encode a phrase length. Using this system, encoding the (0, 0, c) token would take twenty-four bits, all to encode a single eight-bit symbol. This is a very high price to pay, and there ought to be a way to improve it.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

LZSS Compression

LZSS compression seeks to avoid some of the bottlenecks and performance problems in the original LZ77 algorithm. It makes two major changes to the way the algorithm works. The first is in the way the text window is maintained. Under LZ77, the phrases in the text window were stored as a single contiguous block of text, with no other organization on top of it. LZSS still stores text in contiguous windows, but it creates an additional data structure that improves on the organization of the phrases.

As each phrase passes out of the look-ahead buffer and into the encoded portion of the text windows, LZSS adds the phrase to a tree structure. In the implementation that will be used in this chapter, the tree is a binary search tree. By sorting the phrases into a tree such as this, the time required to find the longest matching phrase in the tree will no longer be proportional to the product of the window size and the phrase length. Instead, it will be proportional to the base 2 logarithm of the window size multiplied by the phrase length.

The savings created by using the tree not only makes the compression side of the algorithm much more efficient, it also encourages experimentation with longer window sizes. Doubling the size of the text window now might only cause a small increase in the compression time, whereas before it would have doubled it.

The second change lies in the actual tokens output by the compression algorithm. Recall that LZ77 output tokens consisted of a phrase offset, a match length, and the character that followed the phrase. This meant that LZ77 was compelled to alternate pointers with plain characters, regardless of the nature of the input text.

LZSS instead allows pointers and characters to be freely intermixed. When first starting up, for example, the compression algorithm may not find any phrase matches to output for the first dozen or so input symbols. Under the LZ77 system, the encoder would still have to output a dummy match position with a length of zero for every symbol it output.

LZSS instead uses a single bit as a prefix to every output token to indicate whether it is an offset/length pair or a single symbol for output. When outputting several consecutive single characters, this method reduces the overhead from possibly several bytes per character down to a single byte per character.

Once the data is well characterized, the compressor may efficiently match up pointers every time it loads new data into the look-ahead buffer. LZ77 had some inefficiency here as well, since every offset/length pair had to be accompanied by a single character. This is not as bad as the previous type of inefficiency, but it still reduces the compression ratio.

Data Structures

Two important data structures are used in the implementation of LZSS shown in this chapter. They are the text window, which contains the previously encoded text buffer, and the look-ahead buffer. The text buffer is a simple character buffer declared and used as might normally be expected in a C program:

unsigned char window[ WINDOW_SIZE ];

As discussed previously, while the idea of a sliding window might imply that the text should "slide" through the window, this would actually be an inefficient way to implement it. Instead, the look-ahead buffer moves through the array and tracks its index as it goes along. This means that once a phrase is stored in the array, it stays there until it is overwritten after WINDOW_SIZE characters have been encoded.

This method of working with the text window also means that all string operations, copies, etc. performed on the window have to be done using modulo WINDOW_SIZE arithmetic. Computing (i+1) mod WINDOW_SIZE is usually done most efficiently if WINDOW_SIZE is an integral power of 2, and this implementation of LZSS expects that to be the case. Having WINDOW_SIZE be an integral power of 2 also leads to the most efficient way of encoding the window indices, so this is almost always the method used in sliding-window data compression.

The second data structure in this program is the binary tree used to store the phrases currently in the text window. The tree is defined by the tree structure shown here:

struct {

 int parent;

 int smaller_child;

 int larger_child;

} tree[ WINDOW_SIZE + 1 ];


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

For every phrase in the window, a corresponding structure element defines the position that phrase occupies in the tree. Each phrase has a parent and up to two children. Since this is a binary tree, the two child nodes are defined as "smaller" and "larger" children. Every phrase that resides under the smaller_child node must be smaller than the phrase defined by the current node, and every phrase under the larger_ child node must be larger. The terms "larger" and "smaller" refer to where the phrases fall in the collating sequence used by the compression program. In this particular program, one phrase is "larger" or "smaller" in the same sense as that used by the standard library strcmp() function.

The tree used in this program has a couple of unusual features that need to be explained. First, though only WINDOW_SIZE phrases are in the window, we have defined the tree to have WINDOW_SIZE + 1 elements. In the implementation used here, tree[ WINDOW_SIZE ] is the special node used to locate the tree's root. This element doesn't have a phrase of its own, as do all other nodes in the tree. It also doesn't have smaller and greater children like the other nodes. Instead, it has the index of a larger_child only, and this index points to the root node of the tree.

Pointing to the root node in this fashion saves processing time and simplifies the code. When working our way up through the tree during an insertion or deletion, therefore, we don't have to check to see if a parent node points to the root or to another node. Instead, we can assume that a node's parent node is always a valid tree element. When deleting node i from the tree, for example, you will typically have a section of code that looks something like this: tree[ tree[ i ].parent ].child = tree[ i ].child. Because the pointer to the root node is stored in the same tree, we don't have to perform any special checks to see if i is the root node. Even if i is the root node, tree[ i ].parent still points to a valid node in the tree.

The second unusual feature is the use of another node for special purposes. Like the other programs in this book, LZSS uses a special code to indicate when the end of the compressed data is reached. In this case, a window index of zero indicates an end-of-stream condition.

Since index 0 has a special purpose, it can never be used as a valid phrase. So the code to insert a new phrase into the tree automatically returns without even trying to insert the phrase at index 0. Since phrase 0 is not used, we can achieve even more code savings by using node 0 as the special UNUSED index. This becomes useful when writing code to maintain the tree. A typical operation performed when deleting node i from the tree, for example, is to reassign a new parent node to i's children. Code to perform this might look like what follows.

if ( tree[ i ].smaller_child != UNUSED )

 tree[ tree[ i ].smaller_child ].parent = tree[ i ].parent;

if ( tree[ i ].larger_child != UNUSED )

 tree[ tree[ i ].larger_child ].parent = tree[ i ].parent;

But if the UNUSED index actually points to a legitimate storage area, the test for validity can be bypassed, with the resulting code looking like the following:

tree[ tree[ i ].smaller_child ].parent = tree[ i ].parent;

tree[ tree[ i ].larger_child ].parent = tree[ i ].parent;

If either of the children in this example are UNUSED, no harm is done—the parent node for tree[ 0 ] will merely be modified. Since tree [ 0 ] is never used for any tree navigation, no harm is done, and significant CPU time is saved during tree updates.

A Balancing Act

Saving phrases in a binary tree can simplify the search for the best match. But a binary tree can deteriorate when given data that is ordered in some fashion. In the worst case, a binary tree can turn into nothing more than a linked list. Imagine a file that had the string "ABCDEFGHIJKLMNOP" in it. Since the phrases in that string would have to be added to the tree in order, the structure in Figure 8.7 would evolve.

Figure 8.7 The structure that would evolve from the sequence "ABCDEFGHIJKLMNOP."

This structure may have a pleasing pattern, but it is not well built for locating strings. Given that data compressed from computer files will frequently have patterns of increasing or decreasing phrases, what can we do to avoid this problem?

Of course we can do a lot to help maintain a balanced tree. Many well-known algorithms are built expressly to keep nicely built trees from turning into cycle-stealing unbalanced lists.

In the case of sliding-window data compression, however, it is relatively safe to ignore the problem. Severely unbalanced trees may develop as data is compressed, but the nature of the sliding window almost mandates that unbalanced situations quickly converge to more balanced states. Since old phrases are pulled out of the tree as rapidly as new ones are put in, the effects of an ordered sequence quickly disappear.

As a result, tree balancing is usually not built into sliding-window programs. Probably the only time it would be considered would be in a production version of a compression program that was under severe constraints in terms of CPU cost allowed per byte compressed.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Greedy vs. Best Possible

Both LZ77 and LZSS are called "greedy" algorithms: They don't look ahead into the input stream to analyze it for the best combination of indices and characters. Consider a dictionary-based encoding scheme that used nine bits to encode a single character and twenty-five bits to encode a combined index/offset pair. This scheme would have a break-even point somewhere between two and three characters, which means it would encode a match of two characters as two individual symbols and a match of three symbols as in index/offset token.

Consider now how we would go about encoding the phrase "Go To Statement Considered Harmful" if the contents of the phrase dictionary contained the following fragments: "Go T" "o S" "tat" "Stat." A greedy encoder would naturally encode the "Go T" phrase of four characters length first, followed by the "o S" phrase of three characters length, then the "tat" phrase of three characters length. The output of the encoder up to this point would look like this:

Offset/Length of "Go T" | : | 25 bits
Offset/Length of "o S" | : | 25 bits
Offset/Length of "tat" | : | 25 bits
 |  | ————
 |  | 75 bits

The encoder looks like it was doing what makes sense, trying to build phrases up instead of characters. But an optimal encoder would encode the fragment as shown:

Offset/Length of "Go " | : | 25 bits
Character 'T' | : | 9 bits
Character 'o' | : | 9 bits
Offset/Length of "Stat" | : | 25 bits
 |  | ———-
 |  | 68 bits

These figures clearly show that the greedy encoder did not do as well as the optimal encoder. But it should also be noted that even in this contrived example, the difference between the two is only about 10 percent. When using dictionary coding, it is difficult to find examples of optimal encoders outperforming greedy encoders by more than a few percent. The largest differences occur when only short phrases are in the dictionary, and there is a real possibility that encoding single symbols will take less space than a phrase.

The problem with optimal coding is simply one of payback. Implementing an optimal encoder generally means that encoding speed will be drastically reduced. While optimizing algorithms are available, they tend to be CPU intensive, and the profit derived is generally small. In the world of data compression, a few good heuristics are often more respected than a provably superior algorithm. The greedy heuristic in this case is definitely the choice of most compression programmers.

The Code

The C implementation of LZSS shown here is relatively simple. A production program would probably want to take advantage of numerous potential improvements, which will be discussed at the end of the chapter.

By the very nature of LZSS compression, the compression program will be considerably more complicated than the decoder. The decoder does not have to worry about maintaining the tree or searching for matches. Those two activities are what the encoder spends most of its time doing.

Constants and Macros

All of the constants and global data used in this program are shown following. The parameters of the text window are initially defined by deciding how many bits to allocate to the two fields used to define a pointer or index into the text window. In this example. INDEX_BIT_COUNT is set to twelve: It will use twelve bits to define an index into the text window. The LENGTH_BIT_COUNT macro is set to four bits, which means it will use a four-bit field to encode the length of a matching phrase.

After determining the size of the two bit fields, other macros can be given values derived from them. First, the WINDOW_SIZE is directly determined by the size of the INDEX_BIT_COUNT. In this case, our text window will consist of 4,096 bytes, or 1 << 12. Since we have allocated four bits for the length parameter used to encode a phrase, we will be able to encode a length of up to sixteen bytes, or 1 << 4. This is defined as the RAW_LOOK_AHEAD_SIZE.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The next macro defined, BREAK_EVEN, determines whether it is better to encode a phrase as an index/length pair or as single characters. In this program, the BREAK_EVEN point is determined by adding up the INDEX_BIT_COUNT and the LENGTH_BIT_COUNT plus 1. These add up to seventeen: it take seventeen bits to encode an index/length pair. Because of this, we set our BREAK_EVEN point to one character. This means that in the program, any matching phrase that is one character or fewer will be encoded as single characters instead of as a phrase.

#define INDEX_BIT_COUNT        12

#define LENGTH_BIT_COUNT       4

#define WINDOW_SIZE            ( 1 << INDEX_BIT_COUNT )

#define RAW_LOOK_AHEAD_SIZE    ( 1 << LENGTH_BIT_COUNT )

#define BREAK_EVEN    ( ( 1 + INDEX_BIT_COUNT + LENGTH_BIT_COUNT ) / 9 )

#define LOOK_AHEAD_SIZE        ( RAW_LOOK_AHEAD_SIZE + BREAK_EVEN )

#define TREE_ROOT              WINDOW_SIZE

#define END-OF_STREAM          0

#define UNUSED                 0

#define MOD_WINDOW( a )        ( ( a ) & ( WINDOW_SIZE - 1 ) )

The BREAK_EVEN point adjusts our LOOK_AHEAD_SIZE. Since we aren't going to code any phrases with lengths 0 or 1, we can adjust our LOOK_AHEAD_SIZE upward by two. So when we want to encode a phrase length, instead of outputting the length, we output the length - BREAK_EVEN - 1. This means that the length numbers 0 through 15 will actually correspond to phrases of length 2 through 17.

The TREE_ROOT macro defines the node that points to the binary tree root. Since TREE_ROOT is defined as index WINDOW_SIZE, it is a special node that actually lies outside the binary tree. Whenever the program searches the binary tree, it looks at the child of the node at TREE_ROOT to find the root of the tree.

The END_OF_STREAM constant defined the special index used to place an end-of-file indicator in the output stream. In this implementation, END_OF_STREAM is set to zero, specifically because the UNUSED node index is also set to zero. Having the UNUSED node index set to zero leads to a slight improvement in the program's performance. By using static initialization or be creating the tree with calloc(), we will automatically have a three with every node pointer set to UNUSED—which means we don't have to have a specific initialization step.

The final macro used in the program in MOD_WINDOW. Since input strings can wrap around the end of the tree and head back to the front, we need to perform all of our arithmetic on window indices modulo the tree size. The MOD_WINDOW macro provides a convenient way to do that.

Global Variables

The data structures that hold both the text window and the binary tree are defined as global variables in this program. They could just as easily be dynamically allocated and passed to the encoder and decoder as arguments, but here we cut a few corners by using globals.

The window[ WINDOW_SIZE ] variable holds the last 4,096 characters read in from the input file. The last seventeen of those characters will have been read in from the file but not yet encoded. These constitute the look-ahead buffer.

For comparison purposes, we also consider those 4,096 characters to be 4,096 strings, with each character being the first character in an 17-byte string. The code in this program also universally refers to a string by the index of its first character in the text window. So when a piece of code does a comparison on string 'r', it is looking at the seventeen-byte string starting at position 'r' in the text window.

unsigned char window[ WINDOW_SIZE ];

struct {

 int parent;

 int smaller_child;

 int larger_child;

} tree[ WINDOW_SIZE + 1 ];

The binary tree in this program is the data structure tree, which consists of an array of unnamed structures. Each tree node has only three elements: a parent index, a smaller_child index, and a larger_child index. Each of these indices are single numbers referring to the string at that position in the text window. An example of how this tree might look after reading in twenty-five characters follows. Remember that position 0 in the text window is used for other purposes, so strings there don't get added to the tree.

Figure 8.8 The binary tree after reading in 25 characters.

One nice feature of this binary tree is that we know in advance how many total nodes will be in it, so we can allocate the space in advance instead of while building the tree. This saves the code needed for allocating and freeing nodes, and we know that the node for the string at position 'i' will be in the tree structure at position 'i.'

The Compression Code

The compression code follows. Like all previous programs in this book, it is called with pointers to an open input file and to a special BIT_FILE structure that takes the compressed output. The BIT_FILE structure lets us use the bit-oriented I/O routines in BITIO.H. The compression routine breaks down into three different sections of code: an initialization section, the main compression loop, and a termination section.

void CompressFile( FILE *input. BIT_FILE *output, int argc,

                            char *argv[] )

{

  int i;

  int c;

  int look_ahead_bytes;

  int current_position;

  int replace_count;

  int match_length;

  int match_position;

  current_position = 1;

  for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

   if ( ( c = getc( input ) ) == EOF )

     break;

   window[ current_position + i ] = c;

  }

  look_ahead_bytes = i;

  InitTree( current_position );

  match_length = 0;

  while ( look_ahead_bytes > 0 ) {

   if ( match_length > look_ahead_bytes )

     match_length = look_ahead_bytes;

   if ( match_length <= BREAK_EVEN ) {

    replace_count = 1;

    OutputBit( output, 1 );

    OutputBits( output, window[ current_position ], 8 );

  }  else {

    OutputBit( output, 0 );

    OutputBits( output, match_position, INDEX_BIT_COUNT );

    OutputBits( output, match_length - ( BREAK_EVEN + 1 ),

                LENGTH_BIT_COUNT );

    replace_count = match_length;

   }

   for ( i = 0 ; i < replace_count ; i++ ) {

    DeleteString( MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) );

    if ( ( c = getc( input ) ) == EOF )

     look_ahead_bytes--;

    else

     window[ MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) ] = c;

    current_position = MOD_WINDOW( current_position + 1 );

    if ( look_ahead_bytes )

     match_length = AddString( current_position, &match_position );

   }

 }

 OutputBit( output, 0 );

 OutputBits( output, END_OF_STREAM, INDEX_BIT_COUNT );

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Initialization

The compression loop needs a steady state before it can start. Two things need to be done: first, the look-ahead buffer needs to be loaded; and second, the tree needs to be initialized.

The code to load the look-ahead buffer is shown next. It tries to load up to seventeen bytes into the buffer. After the loading is complete, two local variables are set up so that the main loop can begin executing. First, current_position is set to one. This means that the look-ahead buffer now starts at position 1. Second, look-ahead_bytes is set to the number of bytes left to be encoded in the look-ahead buffer.

current_position = 1;

for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

  if ( ( c = getc( input ) ) == EOF )

    break;

  window[ current_position + i ] = c;

}

look_ahead_bytes = i;

The look-ahead_bytes variable will be set to seventeen most of the time the main loop executes. Usually seventeen characters are left in the look-ahead buffer to encode. Once the program approaches the end of the file, that number will start to drop.

The next step in the initialization program calls InitTree(). InitTree() establishes a root node for the tree. The first node put into the tree will be at the current position, position 1. The code in InitTree() executes a standard insertion algorithm, establishing the child of the root pointer node and setting up the parents and children of position 1.

void InitTree( int r )

{

    tree[ TREE_ROOT ].larger_child = r;

    tree[ r ].parent = TREE_ROOT;

    tree[ r ].larger_child = UNUSED;

    tree[ r ].smaller_child = UNUSED;

}

The final step in the initialization of CompressFile sets up a match_length of one. This forces the encoding loop to output the first character of the look-ahead buffer as a single character instead of as a phrase. It would not be possible at this point even to search for a match to the string at position 1, since it is the only string in the tree.

The Main Loop

The main loop runs as long as characters are left in the look-ahead buffer to encode. It does three things in the loop: (1) It encodes the current phrase in the look-ahead buffer; (2) it reads new characters into the look-ahead buffer while deleting the oldest from the tree; and (3) it inserts the new strings defined by the new characters into the binary tree while the new characters are being loaded into the look-ahead buffer.

while ( look_ahead_bytes > 0 ) {

 if ( match_length > look_ahead_bytes )

  match_length = look_ahead_bytes;

 if ( match_length <= BREAK_EVEN ) {

  replace_count = 1;

  OutputBit( output, 1 );

  OutputBits( output, window[ current_position ], 8 );

 } else {

  OutputBit( output, 0 );

  OutputBits( output, match_position, INDEX_BIT_COUNT );

  OutputBits( output, match_length - ( BREAK_EVEN + 1 ),

              LENGTH_BIT_COUNT );

  replace_count = match_length;

 }

 for ( i = 0 ; i < replace_count ; i++ ) {

  DeleteString( MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) );

  if ( ( c = getc( input ) ) == EOF )

   look_ahead_bytes--;

  else

   window[ MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) ] = c;

  current_position = MOD_WINDOW( current_position + 1 );

  if ( look_ahead_bytes )

   match_length = AddString( current_position, &match_position );

 }

}

The main loop assumes that at the top of the loop, the best match length and position are stored in variables match_length and match_position. The AddString() operation normally does this at the bottom of the loop, but the first time through the loop the initialization code set match_length to zero.

Since the match_length is known, the code just has to decide whether to encode the current phrase in the look-ahead buffer as an index/length pair or whether to output a single character. All that is necessary here is a simple comparison against BREAK_EVEN. If the current phrase match length is more than BREAK_EVEN, it makes sense to encode it as a phrase. Otherwise it is encoded as a single character.

The encoding process is straightforward. The output sequence for a solo character is output as a single 1-bit, followed by the eight bits in the character. For an index/position token, the encoder outputs a 0-bit, followed by the twelve-bit position and the four-bit length. The length is encoded as a number from zero to fifteen that corresponds to a length of two to seventeen.

After encoding, the look-ahead buffer has to be loaded with new characters to replace the ones that have been output. If a phrase was encoded, the variable replace_count is set to the length of the phrase, otherwise, replace_count is set to one to indicate that a single character needs to be replace.

The replacement loop code is shown in the following code excerpt.. New characters read into the look-ahead buffer land on top of the oldest phrases in the text window. Accordingly, before each character is read in, the DeleteString() routine deletes the older phrase.

for ( i = 0 ; i < replace_count ; i++ ) {

 DeleteString( MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) );

 if ( ( c = getc( input ) ) == EOF )

  look_ahead_bytes--;

 else

  window[ MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) ] = c;

 current_position = MOD_WINDOW( current_position + 1 );

 if ( look_ahead_bytes )

  match_length = AddString( current_position, &match_position );

}

After the new character is read in, the current_position is updated, and the AddString() routine adds a new phrase to the tree. AddString() also returns the position and length of the best match for the inserted string. These variables will then be used at the top of the loop to encode the current phrase in the look-ahead buffer.

The Exit Code

The exit code for the compression routine is very simple to implement in this program. All that needs to be done is to encode the special END_OF_STREAM position code so that the decoder will know that there is no more data to pull out of the compressed stream. Its job is completed, and it can then return.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

AddString()

The bulk of the work done by the compression routine takes place in AddString(). This routine does two jobs. First, it adds a new string to the binary tree. Second, it tracks the string currently in the tree that best matches the one being inserted.

The process of locating the node for inspection of the new string uses standard techniques for traversing a binary tree. AddString first checks to see if the new string is the END_OF_STREAM node. If it is, it shouldn't be inserted into the tree, so it takes immediate return with a match_length of zero. This forces the encoder to output a single character instead of trying to encode a phrase at index 0.

After checking for a bad node, the test_node and initial match_length are set up. Throughout the main loop, test_node will point to the current node that will be compared to the new_node. The match_length variable will contain the current longest match found during traversal of the tree.

int AddString( int new_node, int *match_position )

{

  int i;

  int test_node;

  int delta;

  int match_length;

  int *child;

  if ( new_node == END_OF_STREAM )

   return( 0 );

  test_node = tree[ TREE_ROOT ].larger_child;

  match_length = 0;

  for ( ; ; ) {

    for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

     delta = window[ MOD_WINDOW( new_node + i ) ] -

       window[ MOD_WINDOW( test_node + i ) ];

     if ( delta != 0 )

      break;

  }

  if ( i >= match_length ){

    match_length = i;

    *match_position = test_node;

       if ( match_length >= LOOK_AHEAD_SIZE ) {

         ReplaceNode( test_node, new_node );

         return( match_length );

       }

    }

    if ( delta >= 0 )

      child = &tree[ test_node ].larger_child;

    else

      child = &tree[ test_node ].smaller_child;

    if ( *child == UNUSED ) {

      *child = new_node;

      tree[ new_node ].parent = test_node;

      tree[ new_node ].larger_child = UNUSED;

      tree[ new_node ].smaller_child = UNUSED;

      return( match_length );

    }

    test_node = *child;

  }

}

At this point, the main comparison loop is entered. The first section executes a comparison of the test_node and the new_node. Two pieces of information are available after falling out of the loop. The first is the value of delta. The delta variable will be less than one if the string at new_node is less than the test_node, zero if they are the same, and one if the new_node is greater. The second, found in the loop variable i, tells how many characters in the two strings were identical, or the match_length for a particular string.

for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

  delta = window[ MOD_WINDOW( new_node + i ) ] -

          window[ MOD_WINDOW( test_node + i ) ];

  if ( delta != 0 )

    break;

}

After the comparison code completes, the main loop tests whether the match for this phrase is the longest one recorded so far. If it is, the match_length variable is updated, and the test_node position is saved.

if ( i >= match_length ) {

  match_length = i;

  *match_position = test_node;

  if ( match_length >= LOOK_AHEAD_SIZE ) {

    ReplaceNode( test_node, new_node );

    return( match_length );

  }

}

Frequently, the phrase in the look-ahead buffer is an exact match for the test_node. When this is the case, two things happen. First, since the longest match is found, the code will exit the AddString() routine. But before exiting, it performs a node replacement by deleting the test_node and replacing it with the new_node. It could just add new_node to the binary tree, but there is really no point to it, test_node will be redundant data taking up time in the search path if it just uses the normal insertion code. Instead, a special routine replaces test_node with new_node and returns. This leaves a sparser tree that can be searched more quickly. And, since test_node would have been deleted before new_node, it doesn't sacrifice any compression by doing this.

The final section of the main test loop is the tree navigation step. The delta variable tells whether to follow the larger_child or smaller_child branches from the test_node. If the child we are supposed to follow is UNUSED, we have gone as far as we can in the tree. At this point, the code inserts new_node into the binary tree at the correct child and returns. Otherwise, it moves to the new test_node and goes back to the start of the test loop.

if ( delta >= 0 )

  child = &tree[ test_node ].larger_child;

else

  child = &tree[ test_node ].smaller_child;

if ( *child == UNUSED ) {

  *child = new_node;

  tree[ new_node ].parent = test_node;

  tree[ new_node ].larger_child = UNUSED;

  tree[ new_node ].smaller_child = UNUSED;

  return( match_length );

}

test_node = *child;


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

DeleteString()

DeleteString() is called from the main compression loop every time a new character is read into the look-ahead buffer. It uses a standard binary tree deletion algorithm to delete a phrase from the text window.

DeleteString() first determines whether the node is really in the tree. It is possible for the AddString() routine to have already deleted a string because it was a duplicate. If this is the case, the work has been done, and the routine can return.

void DeleteString( int p )

{

 int replacement;

 if ( tree[ p ].parent == UNUSED )

  return;

 if ( tree[ p ].larger_child == UNUSED )

  ContractNode( p. tree[ p ].smaller_child );

 else if ( tree[ p ].smaller_child == UNUSED)

  ContractNode( p, tree[ p ].larger_child );

 else {

  replacement = FindNextNode( p );

  DeleteString( replacement );

  ReplaceNode( p, replacement );

 }

}

If the string is presently in the tree, there are two possibilities for a deletion strategy. If either of the node's children are unused, deleting the node is just a matter of closing the link between the current node's parent and the child in use, effectively pulling the node out of the tree. This is done by a routine called ContractNode().

Figure 8.9 Tree before contraction of node p.

Figure 8.10 Tree after contraction of node p.

The situation is a little more complicated if the node to be deleted has children on both the larger_child and smaller_child nodes. When this is the case, the alternate deletion algorithm has to be used. The way to delete node p when both children are used to find the node in the tree either directly before or indirectly after node p in the ordered list of nodes. In this program, we find the next smaller node. This is done in the FindNextNode() routine and is accomplished by taking the first smaller_child branch, then following the larger_child branches until an UNUSED smaller_child is found. This next smaller node in the list is the replacement node.

The replacement node is then deleted from the tree, with a recursive call to DeleteString(). Out of control recursion is not a worry at this point, since the replacement node by definition has at least one UNUSED child. This means we will never go more than one level deep in our recursion.

After the replacement node has been deleted, it is used to replace the original deleted node. This is done by a routine called ReplaceNode() which simply inserts it in the tree in the same position as the original node.

Binary Tree Support Routines

The support routines used by AddString() and DeleteString() are ContractNode(), ReplaceNode(), and FindNextNode(). ContractNode() deletes a node when one of the children is UNUSED. To do this, the used child is linked with the parent, effectively pulling the node out of the tree. The deleted node has its parent node set to UNUSED, which is what is used internally to determine if a node is in use.

void ContractNode( int old_node, int new_node )

{

 tree[ new_node ].parent = tree[ old_node ].parent;

 if ( tree[ tree[ old_node ].parent ].larger_child == old_node )

  tree[ tree[ old_node ].parent ].larger_child = new_node;

 else

  tree[ tree[ old_node ].parent ].smaller_child = new_node;

 tree[ old_node ].parent = UNUSED;

}

ReplaceNode() is used during the deletion process when a new_node is going to be dropped into the tree on top of the old_node. It is assumed that the new_node is not currently linked to the tree. When the operation completes, the old_node will have been removed, and this is indicated by setting the parent to UNUSED.

void ReplaceNode( int old_node, int new_node )

{

 int parent;

 parent = tree[ old_node ].parent;

 if ( tree[ parent ].smaller_child == old_node )

  tree[ parent ].smaller_child = new_node;

 else

  tree[ parent ].larger_child = new_node;

 tree[ new_node ] = tree[ old_node ];

 tree[ tree[ new_node ].smaller_child ].parent = new_node;

 tree[ tree[ new_node ].larger_child ].parent = new_node;

 tree[ old_node ].parent = UNUSED;

}

FindNextNode() is called when the DeleteString() routine needs to find the next smaller node in the sorted list. It first takes the smaller branch from the starting node, then follows the larger branches until an UNUSED child is located. The node with the UNUSED larger_child is the next highest in the list. This routine assumes that the node has a next smallest node, meaning it has to have a smaller_child branch.

int FindNextNode( int node )

{

 int next;

 next = tree[ node ].smaller_child;

 while ( tree[ next ].larger_child != UNUSED )

  next = tree[ next ].larger_child;

 return( next );

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Expansion Routine

The LZSS compression algorithm is highly asymmetrical. The compression routine is fairly complicated, and it does quite a bit of work for every character or phrase that is compressed. In comparison, the expansion code is extremely simple. It has very little work to do, and in fact it can operate nearly as fast as an ordinary copy routine. This makes LZSS an excellent choice for data that needs to be compressed once and expanded many times.

void ExpandFile( BIT_FILE *input, FILE *output, int argc, char *argv[] )

{

 int i;

 int current_position;

 int c;

 int match_length;

 int match_position;

 current_position = 1;

 for ( ; ; ) {

  if ( InputBit( input ) ) {

   c = InputBits( input, 8 );

   putc( c, output );

   window[ current_position ] = c;

   current_position = MOD_WINDOW( current_position + 1 );

  } else {

   match_position = InputBits( input, INDEX_BIT_COUNT );

   if ( match_position == END_OF_STREAM )

    break;

   match_length = InputBits( input, LENGTH_BIT_COUNT );

   match_length += BREAK_EVEN;

   for ( i = 0 ; i <= match_length ; i++ ) {

    c = window[MOD_WINDOW( match_position + i ) ];

    putc( c, output );

    window[ current_position ] = c;

    current_position = MOD_WINDOW( current_position + 1 );

   }

  }

 }

}

Virtually all the time in the expansion routine is spent in the main loop. The first step reads in a single bit. If this is a zero, the next byte will contain an unencoded character. The character is read in, output to the file, and put in the text window at the current position.

If the input bit was a one instead of a zero, the expansion routine reads in a match_position and length instead of a character. It checks to see if the match_position is actually the encoded END_OF_STREAM message. If so, the program is done, and it can exit. Otherwise, the match_length is read in and adjusted to range 2 through 17.

Once the match_position and length are known, a short loop executes. In this loop, the character from the match string is pulled out of the text window, stored in the file, and put in the window at the current position, which is then updated.

That is the entire expansion routine. It is easy to see why expansion takes place so quickly.

Improvements

While LZSS makes for a fairly good compression algorithm, improvements can be made to it. For instance, LZSS compresses poorly when it starts since it does not have any data in the text window for matches. It is fairly simple to preload the window with WINDOW_SIZE-LOOK_AHEAD_SIZE characters, then add all the appropriate strings to the binary tree.

The trouble with preloading the window is deciding what to preload it with, since we have no idea what type of data will come up in the input stream. Probably the easiest thing to do is to insert 256 strings that contain 16 consecutive occurrences of all of the possible symbols in the alphabet. Unfortunately, runs of repeated characters are the types of initial data that will be helped the least by preloading, but there will be some improvement.

Much experimentation can be done with the number of bits used in the index and the length codes. It is possible to achieve better compression by increasing the number of bits used for each of these, but there are at least two negative effects. First, the compression speed will suffer as the window grows, due to the extra work required to navigate and maintain a larger binary tree. Second, compression of smaller files will suffer due to the increased time needed to build a full dictionary. Some of this startup overhead can be reduced by starting the compression code with a smaller code size and working up to the larger sizes as the dictionary fills up.

One way the programs used here can be sped up dramatically is by using blocked I/O. The tokens output by the compression program here are all exactly either one or two bytes. It is possible to buffer these bytes up while accumulating the flag bits. When eight flag bits have been output, the flag byte can be sent, followed by between 8 and 16 bytes of byte-oriented data. Since the eight to sixteen bytes can be written using conventional byte-oriented code, the compression routines will run considerably faster.

Another technique which speeds up both compression and expansion is to create a "ghost buffer" at the end of the text window. In the case of this program the ghost buffer would hold seventeen characters, which would be identical to the characters in the first seventeen locations of the window. By maintaining a copy of the first seventeen bytes in the ghost buffer, the comparison routines can run without using modulo arithmetic on the indices. Since the compression program spends so much time on string comparison, this results in big time savings.

One final bit of inefficiency found in LZSS relates to the handling of duplicate strings. We remove duplicate strings from the binary tree, but we leave them in the text window, where they take up valuable space. It is possible to free up the space used by these duplicate strings in the text window, allowing for expansion of the dictionary. However, the side effect of this is that the decompression program has to keep track of duplicate strings, which will result in a significant cutback in expansion speed.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Code

/*********************** Start of LZSS.C ***********************

*

* This is the LZSS module, which implements an LZ77 style compression

* algorithm. As implemented here it uses a 12 bit index into the sliding

* window, and a 4 bit length, which is adjusted to reflect phrase

* lengths of between 2 and 17 bytes.

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#include "bitio.h"

/*

* Various constants used to define the compression parameters. The

* INDEX_BIT_COUNT tells how many bits we allocate to indices into the

* text window. This directly determines the WINDOW_SIZE. The

* LENGTH_BIT_COUNT tells how many bits we allocate for the length of

* an encode phrase. This determines the size of the look-ahead buffer.

* The TREE_ROOT is a special node in the tree that always points to

* the root node of the binary phrase tree. END_OF_STREAM is a special

* index used to flag the fact that the file has been completely

* encoded, and there is no more data. UNUSED is the null index for

* the tree. MOD_WINDOW() is a macro used to perform arithmetic on tree

* indices.

*

*/

#define INDEX_BIT_COUNT       12

#define LENGTH_BIT_COUNT      4

#define WINDOW_SIZE           ( 1 << INDEX_BIT_COUNT )

#define RAW_LOOK_AHEAD_SIZE   ( 1 << LENGTH_BIT_COUNT )

#define BREAK_EVEN            ( ( 1 + INDEX_BIT_COUNT +

                              LENGTH_BIT_COUNT ) / 9 )

#define LOOK_AHEAD_SIZE       (RAW_LOOK_AHEAD_SIZE + BREAK_EVEN)

#define TREE_ROOT             WINDOW_SIZE

#define END_OF_STREAM         0

#define UNUSED                0

#define MOD_WINDOW( a )       ( ( a ) & ( WINDOW_SIZE - 1 ) )

char *CompressionName = "LZSS Encoder";

char *Usage      = "in-file out-file\n\n";

/*

* These are the two global data structures used in this program.

* The window[] array is exactly that, the window of previously seen

* text, as well as the current look-ahead text. The tree[] structure

* contains the binary tree of all of the strings in the window sorted

* in order.

*/

unsigned char window WINDOW_SIZE ];

struct {

 int parent;

 int smaller_child;

 int larger_child;

} tree WINDOW_SIZE + 1 ];

/*

* Function prototypes for both ANSI C compilers and their K&R brethren.

*/

#ifdef __STDC__

void InitTree( int r );

void ContractNode( int old_node, int new_node );

void ReplaceNode( int old_node, int new_node );

int FindNextNode( int node );

void DeleteString( int p );

int AddString( int new_node, int *match_position );

void CompressFile( FILE *input, BIT_FILE *output,

                   int argc, char *argv[] );

void ExpandFile( BIT_FILE *input, FILE *output, int argc, char *argv[] );

#else

void InitTree();

void ContractNode();

void ReplaceNode();

int FindNextNode();

void DeleteString();

int AddString();

void CompressFile();

void ExpandFile();

#endif

/*

* Since the tree is static data, it comes up with every node

* initialized to 0, which is good, since 0 is the UNUSED code.

* However, to make the tree really usable, a single phrase has to be

* added to the tree so it has a root node. That is done right here.

*/

void InitTree( r )

int r;

{

 tree[ TREE_ROOT ].larger_child = r;

 tree[ r ].parent = TREE_ROOT;

 tree[ r ].larger_child = UNUSED;

 tree[ r ].smaller_child = UNUSED;

}

/*

* This routine is used when a node is being deleted. The link to

* its descendant is broken by pulling the descendant in to overlay

* the existing link.

*/

void ContractNode( old_node, new_node )

int old_node;

int new_node;

{

 tree[ new_node ].parent = tree[ old_node ]. parent;

 if ( tree[ tree[ old_node ].parent ].larger_child == old_node )

  tree[ tree[ old_node ].parent ].larger_child = new_node;

 else

  tree[ tree[ old_node ].parent ].smaller_child = new_node;

 tree[ old_node ].parent = UNUSED;

}

/*

* This routine is also used when a node is being deleted. However,

* in this case, it is being replaced by a node that was not previously

* in the tree.

*/

void ReplaceNode( old_node, new_node )

int old_node;

int new_node;

{

 int parent;

 parent = tree[ old_node ].parent;

 if ( tree[ parent ].smaller_child == old_node )

  tree[ parent ].smaller_child = new_node;

 else

  tree[ parent ].larger_child = new_node;

 tree[ new_node ] = tree[ old_node ];

 tree[ tree[ new_node ].smaller_child ].parent = new_node;

 tree[ tree[ new_node ].larger_child ].parent = new_node;

 tree[ old_node ].parent = UNUSED;

}

/*

* This routine is used to find the next smallest node after the node

* argument. It assumes that the node has a smaller child. We find

* the next smallest child by going to the smaller_child node, then

* going to the end of the larger_child descendant chain.

*/

int FindNextNode( node )

int node;

{

 int next;

 next = tree[ node ].smaller_child;

 while ( tree[ next ].larger_child != UNUSED )

  next = tree[ next ].larger_child;

 return( next );

}

/*

* This routine performs the classic binary tree deletion algorithm.

* If the node to be deleted has a null link in either direction, we

* just pull the non-null link up one to replace the existing link.

* If both links exist, we instead delete the next link in order, which

* is guaranteed to have a null link, then replace the node to be deleted

* with the next link.

*/

void DeleteString( p )

int p;

{

 int replacement;

 if ( tree[ p ].parent == UNUSED )

  return;

 if ( tree[ p ].larger_child == UNUSED )

  ContractNode( p, tree[ p ].smaller_child );

 else if ( tree[ p ].smaller_child == UNUSED )

  ContractNode( p, tree[ p ].larger_child );

 else {

  replacement = FindNextNode( p );

  DeleteString( replacement );

  ReplaceNode( p, replacement );

 }

}

/*

* This where most of the work done by the encoder takes place. This

* routine is responsible for adding the new node to the binary tree.

* It also has to find the best match among all the existing nodes in

* the tree, and return that to the calling routine. To make matters

* even more complicated, if the new_node has a duplicate in the tree,

* the old_node is deleted, for reasons of efficiency.

*/

int AddString( new_node, match_position )

int mew_node;

int *match_position;

{

 int i;

 int test_node;

 int delta;

 int match_length;

 int *child;

 if ( new_node == END_OF_STREAM )

  return( 0 );

 test_node = tree[ TREE_ROOT ].larger_child;

 match_length = 0;

 for ( ; ; ) {

  for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

     delta = window[ MOD_WINDOW( new_node + i ) ] -

         window[ MOD_WINDOW( test_node + i ) ];

     if ( delta != 0 )

         break;

  }

  if ( i >= match_length ) {

   match_length = i;

   *match_position = test_node;

   if ( match_length >= LOOK_AHEAD_SIZE ) {

    ReplaceNode( test_node, new_node );

    return( match_length );

   }

  }

  if ( delta >= 0 )

   child = &tree[ test_node ]. larger_child;

  else

   child = &tree[ test_node ].smaller-child;

  if ( *child == UNUSED ) {

   *child = new_node;

   tree[ new_node ].parent = test_node;

   tree[ new_node ].larger_child = UNUSED;

   tree[ new_node ].smaller_child = UNUSED;

   return( match_length );

  }

  test_node = *child;

 }

}

/*

* This is the compression routine. It has to first load up the look

* ahead buffer, then go into the main compression loop. The main loop

* decides whether to output a single character or an index/length

* token that defines a phrase. Once the character or phrase has been

* sent out, another loop has to run. The second loop reads in new

* characters, deletes the strings that are overwritten by the new

* character, then adds the strings that are created by the new

* character.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

 int i;

 int c;

 int look_ahead_bytes;

 int current_position;

 int replace_count;

 int match_length;

 int match_position;

 current_position = 1;

 for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

  if ( ( c = getc( input ) ) == EOF )

   break;

  window[ current_position + i ] = (unsigned char) c;

 }

 look_ahead_bytes = 1;

 InitTree( current_position );

 match_length = 0;

 match_position = 0;

 while ( look_ahead_bytes > 0 ) {

  if ( match_length > look_ahead_bytes )

   match_length = look_ahead_bytes;

  if ( match_length <= BREAK_EVEN ) {

   replace_count = 1;

   OutputBit( output, 1 );

   OutputBits( output,

        (unsigned long)window[ current_position ], 8 );

  } else {

   OutputBit( output, 0 );

   OutputBits( output,

        (unsigned long) match_position, INDEX_BIT_COUNT );

   OutputBits( output,

        (unsigned long) ( match_length - ( BREAK_EVEN + 1 ) ),

        LENGTH_BIT_COUNT );

   replace_count = match_length;

  }

  for ( i = 0 ; i < replace_count ; i++ ) {

   DeleteString( MOD_WINDOW( current_position + LOOK_AHEAD-SIZE ) );

   if ( ( c = getc( input ) ) == EOF )

    look_ahead_bytes--;

   else

    window[ MOD_WINDOW ( current_position + LOOK_AHEAD_SIZE ) ]

              = (unsigned char) c;

   current position = MOD_WINDOW( current_position + 1 );

   if ( look ahead_bytes )

    match_length = AddString( current_position, &match_position );

  }

 }

 OutputBit( output, 0 );

 OutputBits( output,

         (unsigned long) END_OF_STREAM, INDEX_BIT_COUNT);

 while ( argc-- > 0 )

  printf( "Unknown argument: %s\n", *argv++ );

}

/*

* This is the expansion routine for the LZSS algorithm. All it has

* to do is read in flag bits, decide whether to read in a character or

* a index/length pair, and take the appropriate action.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

  int i;

  int current_position;

  int c;

  int match_length;

  int match_ position;

  current_position = 1;

  for ( ; ; ) {

   if ( InputBit( input ) ) {

    c = (int) InputBits( input, 8 );

    putc( c, output );

    window[ current_position ] = (unsigned char) c;

    current_position = MOD_WINDOW( current_position + 1 );

   } else {

    match_position = (int) InputBits( input, INDEX_BIT_COUNT );

    if ( match_position == END_OF_STREAM )

     break;

    match_length = (int) InputBits( input, LENGTH_BIT_COUNT );

    match_length += BREAK_EVEN;

   for ( i = 0 ; i <= match_length ; i++ ) {

    c = window[ MOD_WINDOW( match_position + i ) ];

    putc( c, output );

    window[ current_position ] = (unsigned char) c;

    current_position = MOD_WINDOW( current_position + 1 );

   }

  }

 }

 while ( argc-- > 0 )

  printf( "Unknown argument: %s\n", *argv++ );

}

/************************* End of LZSS.C *************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 9
LZ78 Compression

The LZ77 algorithms have a deficiency that must have troubled their creators, Jacob Ziv and Abraham Lempel. These algorithms use only a small window into previously seen text, which means they continuously throw away valuable phrases because they slide out of the dictionary. The phrase "Jacob Ziv," for example, may have appeared in the text in Chapter 7. If this book were being compressed using an LZ77 algorithm, it almost certainly would not appear in the sliding window by the time we arrived at this chapter. But the book would compress better if "Jacob Ziv" did appear in the dictionary, since it shows up four or five times in scattered locations throughout the text.

The sliding window makes LZ77 algorithms biased toward exploiting recency in the text. Many, but not all, streams of data tend to "look" more like what has been seen recently than what was seen long ago. Encoding a telephone book is a good example of this. After encoding the first "Jones" entry, it would seem that the word "Jones" was showing up everywhere. But the word "Adams," seen long ago, would probably not show up at all. As we moved through the phone book, recency effects would be patently obvious, and LZ77 would take advantage of them.

While compressing the phone book, if we looked carefully, we would also notice that some data was more or less immune to recency effects. The street address associated with each listing, for example, would only show the effect faintly, when two listings with the same last name lived at the same address. This would result in fewer matches to text in the window, leading to less effective compression.

A second deficiency in LZ77 compression is the limited size of a phrase that can be matched. The longest match is approximately the size of the look-ahead buffer, which in the previous chapter was just 17 bytes. Many of the seventeen-byte matches found when compressing using that algorithm may actually be much longer.

Can LZ77 Improve?

One obvious way to tackle these problems is simply to start tinkering with the size of the window and the size of the look-ahead buffer. Instead of using a 4K window and a seventeen-byte buffer, for example, why not use a 64K text window and a 1K look-ahead buffer? Wouldn't that address both problems effectively?

While raising the size of both parameters does seem to address these problems, the scheme has two major drawbacks. First, when we increase the buffer size from 4K to 64K, we now need sixteen bits to encode an index location instead of twelve. And instead of needing four bits to encode a phrase length, we now need ten. So the cost of encoding a phrase rises from seventeen bits to twenty-seven.

This 50 percent increase in the bit size of an index/length token can have a severely negative impact on the compression algorithm. For one thing, it will change the BREAK_EVEN point in the program from just under two characters to three characters. This means that matches of three or fewer characters will no longer be effectively coded in index/length tokens and will instead have to be encoded as single characters. Encoding data as single characters is actually less efficient than plain text, since it needs an additional bit to indicate that a normal character is coming.

An even more distressing effect is that changing these parameters will drastically increase the amount of CPU time needed to perform compression. Under LZ77, just changing the text window size from 4K to 64K will result in the average search taking sixteen times longer, since every string in the window is compared to the look-ahead buffer. The situation is somewhat better under LZSS, since the strings are kept in a binary tree. In this case, the runtime cost of the window size is proportional to the logarithm of the window size. But this still means over a 30 percent increase in runtime.

The real penalty comes when the size of the look-ahead buffer is increased. Since our string comparisons between the text window phrases and the look-ahead buffer proceed sequentially, the runtime here will increase in direct proportion to the length of the look-ahead buffer. Going from sixteen to 1,024 characters means this portion of the program is going to run sixty-four times more slowly—a costly penalty indeed.

These effects combine to effectively cancel out any gains from increasing either of these parameters in an LZ77 algorithm. And even with a 64K text window, we are still effectively tied to an algorithm that depends on recency to perform adequate compression.

Enter LZ78

To effectively sidestep these problems, Ziv and Lempel developed a different form of dictionary-based compression. This algorithm, popularly referred to as LZ78, was published in "Compression of Individual Sequences via Variable-Rate Coding" in IEEE Transactions on Information Theory (September 1978).

LZ78 abandons the concept of a text window. Under LZ77 the dictionary of phrases was defined by a fixed window of previously seen text. Under LZ78, the dictionary is a potentially unlimited list of previously seen phrases.

LZ78 is similar to LZ77 in some ways. LZ77 outputs a series of tokens. Each token has three components: a phrase location, the phrase length, and a character that follows the phrase. LZ78 also outputs a series of tokens with essentially the same meanings. Each LZ78 token consists of a code that selects a given phrase and a single character that follows the phrase. Unlike LZ77, the phrase length is not passed since the decoder knows it.

Unlike LZ77, LZ78 does not have a ready-made window full of text to use as a dictionary. It creates a new phrase each time a token is output, and it adds that phrase to the dictionary. After the phrase is added, it will be available to the encoder at any time in the future, not just for the next few thousand characters.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

LZ78 Details

When using the LZ78 algorithm, both encoder and the decoder start off with a nearly empty dictionary. By definition, the dictionary has a single encoded string—the null string. As each character is read in, it is added to the current string. As long as the current string matches some phrase in the dictionary, this process continues.

But eventually the string will no longer have a corresponding phrase in the dictionary. This is when LZ78 outputs a token and a character. Remember that the string did have a match in the dictionary until the last character was read in. The current string, therefore, is defined as that last match with one new character added on. This is what LZ77 outputs: the index for the previous match and the character that broke that match.

But at this point, LZ78 takes an additional step. The new phrase, consisting of the dictionary match and the new character; is added to the dictionary. The next time that phrase appears, it can be used to build an even longer phrase.

A code fragment to implement this algorithm is shown next. Some of the detail has been glossed over, but this is a fairly faithful representation of the algorithm.

  for ( ; ; ) {

   current_match = 1;

   current_length = 0;

   memset( test_string, '\0', MAX_STRING );

   for ( ; ; ) {

   test_string[ current_length++ ] = getc( input );

   new_match = find_match( test_string );

   if ( new_match == -1 )

     break;

   current_match = new_match;

 }

  output_code( current_match );

  output_char( test_string[ current_length - 1 ] );

  add_string_to_dictionary( test_string );

  }

By definition, the empty string will always match string 0, the null node in the dictionary. Thus, when we encounter a character for the first time, it is encoded as phrase 0 plus the new character. The next time that character appears, it will be encoded as part of a phrase.

An example of the encoder output follows. The input text is a sequence of words from the dictionary of a spelling checker. The LZ78 encoder starts encoding with no phrases in the dictionary; therefore, the first character it reads in from the input text, 'D', creates a string that has no match in the dictionary. The encoders will then output a phrase/character pair, in this case 0 and 'D'. Remember that the dictionary starts up with zero defined as the empty phrase.

Input text: "DAD DADA DADDY DADO..."

Output Phrase | Output Character | Encoded String
0 | 'D' | "D"
0 | 'A' | "A"
1 | ' ' | "D "
1 | 'A' | "DA"
4 | ' ' | "DA "
4 | 'D' | "DAD"
1 | 'Y' | "DY"
0 | ' ' | " "
6 | 'O' | "DADO"

The first two characters to come through the encoder, 'D' and 'A,' have not been seen before. Each will have to be encoded as a phrase, 0+ character pair. "D" is added to the dictionary as phrase 1, and "A" is added as phrase 2.

When the third character, 'D,' is read in, it matches an existing dictionary phrase. The ' ' character, the next character read in, creates a new phrase with no match in the dictionary. LZ78 will output code 1 for the previous match (the D string), then the " " character.

As the encoding continues, the dictionary quickly builds up fairly long phrases. Of course, since these entries are from a dictionary sorted in alphabetical order, we probably build up phrases much faster than would normally be the case. After just nineteen characters have been read in and encoded, the dictionary looks like the one following.

0 | ""
1 | "D"
2 | "A"
3 | "D"
4 | "DA"
5 | "DA"
6 | "DAD"
7 | "DY"
8 | ""
9 | "DADO"

LZ78 Implementation

Like LZ77, LZ78 can arbitrarily set the size of the phrase dictionary. And like LZ77, in LZ78 we have to worry about the effects of this in two ways. First, we have to consider the number of bits allocated in the output token for the phrase code. Second, and more importantly, we have to consider how much CPU time managing the dictionary will take.

In theory, LZ78 should compress better and better as the size of the dictionary increases. But this only holds true as the length of the input text tends towards infinity. In practice, smaller files will quickly begin to suffer as the code size grows larger.

The real difficulty with LZ78 actually comes in managing the dictionary. If we use a sixteen-bit code for the phrase index, for example, we can accommodate 65,536 phrases, including the null code. The phrases can vary tremendously in length, including the improbable possibility of 65,536 different versions of a phrase composed of runs of a single, repeated character.

These phrases are conventionally stored in a multiway tree. The tree starts at a root node, 0, the null string. Each possible character that can be added to the null string is a new branch of the tree, with each phrase created that way getting a new node number.

Figure 9.1 An LZ78 Dictionary Tree.

The dictionary tree shown here would be created after the previous nineteen-character phrase was encoded. The major difficulty with managing a tree such as this is the potentially large number of branches off of each node. When compressing binary files with an eight-bit alphabet, 256 branches off of each node are possible. We could simply allocate an array of indices or pointers at each node that was large enough to accommodate all 256 possible descendants. But since most nodes will not have that many descendants, it would be incredibly wasteful to allocate so much storage. Instead, descendant nodes are usually managed as a list of indices no longer than the number of descendant nodes that actually exits. This technique makes better use of available memory, but it is also significantly slower. It is essentially the same technique used in chapter 6 to perform higher-order modeling of data streams.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

With a tree like this, comparing an existing string to the dictionary is simple. It is just a matter of walking through the tree, traversing a single node of the tree for every character in the phrase. If the phrase terminates at a particular node, we have a match. If there are more phrases but we have reached a leaf node, there is not a match. After the symbol has been encoded, adding it to the leaf node is also simple—just a matter of adding space to the descendant list, then inserting a new descendant node at the node last matched.

One negative side effect of LZ78 not found in LZ77 is that the decoder has to maintain this tree as well. With LZ77, a dictionary index was just a pointer or index to a previous position in the data stream. But with LZ78, the index is the number of a node in the dictionary tree. The decoder, therefore, has to keep up the tree in exactly the same fashion as the encoder, or a disastrous mismatch will occur.

Another issue ignored so far is that of the dictionary filling up. Regardless of how big the dictionary space is, it is going to fill up sooner or later. If we are using a sixteen-bit code, the dictionary will fill up after it has 65,535 phrases defined in it.

There are several alternative choices regarding a full dictionary. Probably the safest default choice is to stop adding new phrases to the dictionary after it is full. This only requires an extra line or two of code in the add_phrase_to_dictionary() routine.

But just leaving the dictionary alone may not be the best choice. When compressing large streams of data, we may see significant changes in the character of the incoming data. When compressing a program's binary image (such as an EXE file), for example, we would expect to see a major shift in the statistical model of the data as we move out of the code section of the file and into the data section.

If we keep using our existing phrase dictionary, we may be stuck with an out-of-date dictionary that isn't compressing very well. At the same time, we have to be careful not to throw away a dictionary that is compressing well.

The UNIX compress program, which uses an LZ78 variant, manages the full dictionary problem by monitoring the compression ratio of the file. If the compression ratio ever starts to deteriorate, the dictionary is deleted and the program starts over from scratch. Otherwise, the existing dictionary continues to be used, though no new phrases are added to it.

An Effective Variant

As with LZ77, LZ78 was first published in a research journal and discussed in a very technical and abstract fashion. It wasn't until 1984 that a variant of LZ78 made headway in the programming world. This was when Terry Welch published "A Technique for High-Performance Data Compression" in IEEE Computer.

Work on the UNIX compress program began almost immediately after Terry Welch's article appeared. The technique Welch described, and the implementation in compress, are referred to as LZW compression.

LZSS improved on LZ77 compression by eliminating the requirement that each token output a phrase and a character. LZW makes the same improvement on LZ78. In fact, under LZW, the compressor never outputs single characters, only phrases.

To do this, the major change in LZW is to preload the phrase dictionary with single-symbol phrases equal to the number of symbols in the alphabet. Thus, there is no symbol that cannot be immediately encoded even if it has not already appeared in the input stream.

The LZW compression algorithm in its simplest form follows. A quick examination of the algorithm shows that LZW always tries to output codes for strings that are already known. And each time a new code is output, a new string is added to the string table.

  old_string[ 0 ] = getc(input);

  old_string[ 1 ] = '\0';

  while ( !feof( input ) ) {

    character = getc( input );

    strcpy( new_string, old_string );

    strncat( new_string, &character, 1 );

    if ( in_dictionary( new_string ) )

      strcpy( old_string, new_string );

    else {

      code = look_up_dictionary( old_string );

      output code( code );

      add_to_dictionary( new_string );

      old_string[ 0 ] = character;

      old_string[ 1 ] = '\0';

    }

    code = look_up_dictionary( old string );

    output_code( code );

A sample string used to demonstrate the algorithm is shown next. The input string is a set of English words from a spelling dictionary, separated by the ' ' character. On the first pass through the loop, a check is performed to see if the string " W" is in the table. Since it isn't, the code for " " is output, and the string " W" is added to the table. Since the dictionary has codes 0-255 already defined as the 256 possible character values, the first string definition is assigned to code 256. After the third letter,'E', has been read in, the second string code, "WE", is added to the table, and the code for letter 'W' is output. In the second word, the characters ' ' and 'W' are read in, matching string number 256. Code 256 is then output, and a three-character string is added to the string table. The process continues until the string is exhausted and all codes have been output.

Input String: " WED WE WEE WEB WET "

Characters Input | Code Output | New code value and associated string
" W" | ' ' | 256 = " W"
"E" | 'W' | 257 = "WE"
"D" | 'E' | 258 = "ED"
" " | "D" | 259 = "D "
"WE" | 256 | 260 = "WE"
" " | 'E' | 261 = "E"
"WEE" | 260 | 262 = " WEE"
" W" | 261 | 263 = "E W"
"EB" | 257 | 264 = "WEB"
" " | B | 265 = "B"
"WET" | 260 | 266 = " WET"
<EOF> | T


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The sample output for the string is shown with the resulting string table. The string table fills up rapidly, since a new string is added each time a code is output. In this highly redundant input, five code substitutions were output, along with seven characters. If we were using nine-bit codes for output, the nineteen-character input string would be reduced to a 13.5-byte output string. Of course, this example was carefully chosen to demonstrate code substitution. In real world examples, compression usually doesn't begin until a sizable table has been built, usually after at least one hundred or so bytes have been read in.

Decompression

The companion algorithm for compression is the decompression algorithm. It takes the stream of codes output from the compression algorithm and uses them to recreate the exact input stream. One reason for the efficiency of the LZW algorithm is that it does not need to pass the dictionary to the decompressor. The table can be built exactly as it was during compression, using the input stream as data. This is possible because the compression algorithm always outputs the phrase and character components of a code before it uses it in the output stream, so the compressed data is not burdened with carrying a large dictionary.

    old_string[ 0 ] = input_bits();

    old_string[ 1 ] = '\0';

    putc( old_string[ 0 ], output )

    while ( ( new_code = input_bits() ) != EOF )

       new_string = dictionary_lookup( new_code );

       fputs( new_string, output );

       append_char_to_string( old_string, new_string[ 0 ] );

       add_to_dictionary( old_string );

       strcpy( old_string, new_string );

    }

Preceding is a rough C implementation. Like the compression algorithm, it adds a new string to the string table each time it reads in a new code. In addition, it translates each incoming code into a string and sends it to the output.

Following is the output of the algorithm given the input created by the earlier compression. Note that the string table ends up looking exactly like the table built during compression. The output string is identical to the input string from the compression algorithm. Note also that the first 256 codes are already defined to translate to single-character strings, as in the compression code.

Input Codes: "WED<256>E<260><261><257>B<260>T"

Input/ NEW_CODE | OLD_CODE | STRING/ Output | CHARACTER | New table entry
' ' | ' ' | " "
'W' | ' ' | "W" | 'W' | 256 = " W"
'E' | 'W' | "E" | 'E' | 257 = "WE"
'D' | 'E' | "D" | 'D' | 258 = "ED"
256 | 'D' | " W" | ' ' | 259 = "D"
'E' | 256 | "E" | 'E' | 260 = " WE"
260 | 'E' | " WE" | ' ' | 261 = "E"
261 | 260 | "E " | 'E' | 262 = " WEE"
257 | 261 | "WE" | 'W' | 263 = "E W"
'B' | 257 | "B" | 'B' | 264 = "WEB"
260 | 'B' | " WE" | ' ' | 265 = "B"
'T' | 260 | "T" | 'T' | 266 = " WET"

The Catch

Unfortunately, the decompression algorithm shown is just a little too simple. A single exception in the LZW compression algorithm causes some trouble in decompression. Each time the compressor adds a new string to the phrase table, it does so before the entire phrase has actually been output to the file. If for some reason the compressor used that phrase as its next code, the expansion code would have a problem. It would be expected to decode a string that was not yet in its table.

Unfortunately, there is a way this can occur. If there is a phrase already in the table composed of a CHARACTER, STRING pair, and the input stream then sees a sequence of CHARACTER, STRING, CHARACTER, STRING, CHARACTER, the compression algorithm will output a code before the decompressor defines it.

A simple example will illustrate the point. Imagine the string IWOMBAT is defined in the table as code 300. Later, the sequence IWOMBATIWOMBATI occurs in the table. The compression output will look like the following:

Input String: IWOMBAT.......IWOMBATIWOMBATIXXX

<Problem section>
Character Input | New code value and associated string | Code Output
...I |  |
WOMBATA | 300 = IWOMBAT | 288 (IWOMBA)
. | . | .
. | . | .
...I | . | .
WOMBATI | 400 = IWOMBATI | 300 (IWOMBAT)
WOMBATIX | 401 = IWOMBATIX | 400 (IWOMBATI)

When the decompression algorithm sees this input stream, it first decodes code 300 and outputs the IWOMBATI string. It will then add the definition for code 399 to the table, whatever that may be. It then reads the next input code, 400, and finds that it is not in the table.

Fortunately, this is the only time when the decompression algorithm will encounter an undefined code. Since it is the only time, we can add an exception handler to the algorithm. The modified algorithm just looks for the special case of an undefined code and handles it. In the sample, the decompression routine sees a code of 400. Since 400 is undefined the program goes back to the previous code/string, which was "IWOMBAT", or code 300. It then appends the first character of the string to the end of the string, yielding "IWOMBATI," the correct value for code 400. Processing then proceeds as normal.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The exception handler takes advantage of the knowledge that this problem can happen only in the special circumstances of CHARACTER+ STRING+CHARACTER+STRING+CHARACTER. Given that, any time an unknown code occurs, it can determine what the unknown code is given knowledge of the previous string from the input.

   old_string[ 0 ] = input_bits();

   old_string[ 1 ] = '\0';

   putc( old_string[ 0 ], output )

   while ( ( new_code = input_bits() ) != EOF ) {

     new_string = dictionary_lookup( new_code );

     if ( new_string == NULL ) {

       strcpy( new_string, old_string );

       append_character_to_string( new_string, new_string[ 0 ] );

     }

     fputs( new_string, output );

     append_character_to_string( old_string, new_string[ 0 ] );

     add_to_dictionary( old_string );

     strcpy( old_string, new_string );

 }

LZW Implementation

The concepts in the compression algorithm are so simple that the whole algorithm can be expressed in a dozen lines. Implementation of this algorithm is somewhat more complicated, mainly due to management of the dictionary. A short example program that uses twelve-bit codes is in LZW12.C, and it will illustrate some of the techniques used here.

Tree Maintenance and Navigation

As in the LZ78 algorithm, the LZW dictionary is maintained as a multiway tree. But in the case of LZW, the way the data is stored doesn't look much like a tree. A little analysis, however will reveal a multiway tree hidden behind the dictionary data structures.

   struct dictionary {

    int code_value;

    int parent_code;

    char character;

   } dict[ TABLE_SIZE ];

The structure shown in the preceding figure holds the entire dictionary tree. Each element in the data structure represents a single node. The node is defined by three items: (1) Code_value. This number is the actual code for the string that terminates at this node and is what the compression program emits when it wants to encode the string; (2) Parent_code. Under LZ78-style compression, every string in the dictionary has a parent string one character shorter than it. This integer is the code for that parent string; (3) Character. This is the character for this particular node. If the string encoded by the parent of a node were "GREENLEA," and the character value was "F," this node would encode "GREENLEAF."

Something that immediately becomes noticeable as a problem here is that each dictionary node does not have a pointer or pointers to its child nodes. As we navigate the tree, how are we supposed to find the children of each node if there are no pointers to children?

The answer is that this tree maintains the dictionary pointers through a hashed array of nodes. To find the child of a particular node, we apply a hashing function to see where that puts us in the list. The hashing function used in LZW12.C is shown next.

unsigned int find_child_node( parent_code, child_character )

int parent_code;

int child_character;

{

 int index;

 int offset;

 index = ( child_character << ( BITS - 8 ) ) ^ parent_code;

  if ( index == 0 )

   offset = 1;

  else

   offset = TABLE_SIZE - index;

  for ( ; ; ) {

   if ( dict[ index ].code_value == UNUSED )

     return( index );

   if ( dict[ index ].parent_code == parent_code &&

        dict[ index ].character == (char) child_character )

   return( index );

   index -= offset;

   if ( index < 0 )

     index += TABLE_SIZE;

  }

}

This hashing function is essentially the same one used in the UNIX compress program. It combines the numeric values of the parent_code and the child_character to form a sixteen-bit offset into the list of nodes. After finding the target node, it checks for collisions, since that node may be in use by some other element in the tree. Eventually, one of two things happens. Either this function finds a node already defined as belonging to the parent and child, or it finds an empty node that can be used that way.

This hashing function performs fairly well. The collision avoidance mechanism depends on having TABLE_SIZE be a prime number, and performance depends on it being at least 20 percent larger than two raised to the BITS power. In LZW12.C, TABLE_SIZE needs to be larger than 4,096. The number actually used was 5,021.

With the hashing function in place, we can now effectively navigate down through the tree. The data structures used to maintain the dictionary during compression don't help us move up the tree, but during compression we don't need to move up the tree, only down.

During decompression, the hashing function is no longer used. Instead, each node in the tree has its parent code and character value stored at the array offset defined by its own code. This allows for quick lookup of dictionary values, which lets us move up the tree quickly. We need to move up the tree during decompression to determine the entire contents of a string, and this different storage method makes this possible. We never need to move down the tree during decompression, so the hashing function is no longer needed.

One additional feature of the dictionary tree used in LZW12.C needs explanation. The first 256 nodes are considered "special" nodes by the program. Each of them represents the one character string that corresponds with its node value. In other words, code 65 will always represent the character "A," and it will automatically be assumed not to have a parent. These nodes are all predefined when the program is first initialized.

Compression

Armed just with the hashing function and the data structure, the compression program can be written fairly easily. The program goes through a short initialization phase, then sits in an encoding loop reading characters in and sending codes out. Finally, it does a small amount of cleanup work, then exits.

next_code = FIRST CODE;

for ( i = 0 ; i < TABLE_SIZE ; i++ )

  dict[ i ].code_value = UNUSED;

if ( ( string_code = getc( input ) ) == EOF )

  string_code = END_OF_STREAM;

while ( ( character = getc( input ) ) != EOF ) {

  index = find_child_node( string_code, character );

  if ( dict[ index ].code_value != - 1)

    string_code = dict[ index ].code_value;

  else {

    if ( next_code <= MAX_CODE ) {

      dict[ index ].code_value = next_code++;

      dict[ index ].parent-code = string_code;

      dict[ index ].character = (char) character;

    }

    OutputBits( output, string_code, BITS );

    string_code = character;

  }

}

OutputBits( output, string_code, BITS );

OutputBits( output, END_OF_STREAM, BITS );

This routine first initializes the dictionary array. It does this by marking all nodes in the tree as unused. Remember that the first 256 nodes are special and will be considered used automatically.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The next_code variable is then set to the first available code. This program uses code 256 as an end-of-file marker, so the first code defined will have a value of 257. As new strings are read in and defined, this number grows until it reaches the maximum code value of 4,095.

Finally, the first character is read in and assigned to the loop variable string_code. We can arbitrarily assign the first character to a code value, since it is a special single-character string.

After initialization, the main encoding loop begins. The working variable, string_code, keeps track of which code matches the characters read in so far. When the program first starts, that is just as single-character string, but as the dictionary grows, string_code can represent very long strings.

A single character is read in from the input file, then find_child_node() is called to see if the current string_code has a child node that corresponds to that character. If it does, the child's code is assigned to string_code, and we move back to the top of the loop.

If there is no child node, we have reached the end of our string match. When this occurs, we output the current code, then start over with a new string_code. Finally, we add the new string created by the combination of string_code and character to the dictionary so the next time it occurs we will get a match.

The main loop repeats until an end-of-file is read in. When this occurs, we output the string_code built up so far. Finally, the END_OF_STREAM code is output, which tells the decoder when we are at the end of the data stream.

Decompression

As mentioned previously, maintaining the dictionary is simpler during decompression. We don't ever have to navigate down through the tree. Instead, we read in codes straight from the encoded stream, then work our way up through the tree. As long as the parent nodes are properly defined in the data structure, everything works fine.

The only problem with working up through the tree is that the decoded characters are gathered in reverse order, so they have to be pushed into a stack, popped off in reverse order, and written to the output file. This is done with the decode_string routine, shown next.

decode_string() follows the parent pointers up though the dictionary until it finds a code less than 256, which we have defined as the first character in the string. A count of characters in the decode stack is then returned to the calling program.

    unsigned int decode_string( count, code )

    unsigned int count;

    unsigned int code;

    {

       while ( code > 255 ) {

        decode_stack[ count++ } = dict[ code ].character;

        code = dict[ code].parent_code;

    }

    decode_stack[ count++ ] = (char) code;

    return( count );

   }

Once the decode routine is in place, the decompression routine falls into order fairly easily. The routine has a few lines of initialization code, followed by a main decoding loop.

The initialization section of the decompression routine in LZW12.C sets up the next_code variable. This let it track the code value of each string as it is added to the table. Next, is reads in the first code and copies it to the output file. Once that is done, it can enter the main decoding loop.

   next_code = FIRST_CODE;

   old_code = InputBits( input, BITS );

   if ( old_code == END_OF STREAM )

    return;

   character = old_code;

   putc( old_code, output );

   while ( ( new_code = InputBits( input, BITS ) ) != END_OF_STREAM ) {

    if ( new_code >= next_code ) {

     decode_stack[ 0 ] = (char) character;

     count = decode_string( 1, old_code );

    }

    else

     count = decode_string( 0, new_code );

    character = decode_stack[ count - 1 ];

    while ( count > 0 )

     putc( decode_stack[ --count ], output );

    if ( next_code <= MAX_CODE ) {

      dict[ next_code ].parent_code = old_code;

      dict[ next_code ].character = (char) character;

    next_code++;

   }

   old_code = new_code;

 }

Normally, decoding is a simple matter. The loop reads in a code value, looks up the string, and outputs it. Then it create a new string by adding the old_code and the first character of the current string to the string table. It then goes back to the top of the loop and starts over.

But an additional complication is created when the CHARACTER+ STRING+CHARACTER+STRING+CHARACTER sequence shows up. This creates a code larger than the largest currently defined code. Fortunately, we know what to do in this case. Our new string will be the same as our last string, defined by old_code, with a copy of its first character appended to its end. This is handled by preinitializing the decoding_stack with a single character, then decoding the old_code string into the stack with an offset of one instead of zero.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Code

The source code for a complete twelve-bit version of LZW compression and decompression follows.

/************************* Start of LZW12.C **************************

*

* This is 12 bit LZW program, which is discussed in the first part

* of the chapter. It uses a fixed size code, and does not attempt

* to flush the dictionary after it fills up.

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include "errand.h"

#include "bitio.h"

/*

* Constants used throughout the program. BITS defines how many bits

* will be in a code. TABLE_SIZE defines the size of the dictionary

* table.

*/

#define BITS                    12

#define MAX_CODE                ( ( 1 << BITS ) - 1 )

#define TABLE_SIZE              5021

#define END_OF_STREAM           256

#define FIRST_CODE              257

#define UNUSED                  -1

/*

* Local prototypes.

*/

#ifdef __STDC__

unsigned int find_child_node( int parent_code, int child_character );

unsigned int decode_string( unsigned int offset, unsigned int code );

#else

unsigned int find_child_node ();

unsigned int decode_string ();

#endif

char *CompressionName = "LZW 12 Bit Encoder";

char *Usage  = "in-file out-file\n\n";

/*

* This data structure defines the dictionary. Each entry in the

* dictionary has a code value. This is the code emitted by the

* compressor. Each code is actually made up of two piece: a

* parent_code, and a character. Code values of less than 256 are

* actually plain text codes.

*/

struct dictionary {

 int code_value;

 int parent_code;

 char character;

} dict[TABLE_SIZE ];

char decode_stack[ TABLE_SIZE };

/*

* The compressor is short and simple. It reads in new symbols one

* at a time from the input file. It then checks to see if the

* combination of the current symbol and the current code are already

* defined in the dictionary. If they are not, they are added to the

* dictionary, and we start over with a new one symbol code. If they

* are, the code for the combination of the code and character becomes

* our new code.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

  int next_code;

  int character;

  int string_code;

  unsigned int index;

  unsigned int i;

  next_code = FIRST_CODE;

  for ( i = 0 ; i < TABLE_SIZE ; i++ )

   dict[ i ].code_value = UNUSED;

  if ( ( string_code = getc( input ) )== EOF )

   string_code = END_OF_STREAM;

  while ( ( character = getc( input ) ) != EOF ) {

   index = find_child_node( string_code, character );

   if ( dict[ index ].code_value != - 1)

    string_code = dict[ index ].code_value;

   else {

    if ( next_code <= MAX_CODE ) {

    dict[ index ].code_value = next_code++;

    dict[ index ].parent_code = string_code;

    dict[ index ].character = (char) character;

   }

   OutputBits( output, (unsigned long) string_code, BITS );

   string_code = character;

  }

 }

 OutputBits( output, (unsigned long) string_code, BITS );

 OutputBits( output, (unsigned long) END_OF_STREAM, BITS );

 while ( argc-- > 0 )

  printf( "Unknown argument: %s\n", *argv++ ); }

/*

* The file expander operates much like the encoder. It has to

* read in codes, then convert the codes to a string of characters.

* The only catch in the whole operation occurs when the encoder

* encounters a CHAR+STRING+CHAR+STRING+CHAR sequence. When this

* occurs, the encoder outputs a code that is not presently defined

* in the table. This is handled as an exception.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

  unsigned int next_code;

  unsigned int new_code;

  unsigned int old_code;

  int character;

  unsigned int count;

  next_code = FIRST_CODE;

  old_code = (unsigned int) InputBits( input, BITS );

  if ( old_code == END_OF_STREAM )

   return;

  character = old_code;

  putc( old_code, output );

while ( ( new_code = (unsigned int) InputBits( input, BITS ) )

          != END_OF_STREAM ) {

/*

** This code checks for the CHARACTER+STRING+CHARACTER+STRING+CHARACTER

** case which generates an undefined code. It handles it by decoding

** the last code, and adding a single character to the end of the

** decode string.

*/

  if (new_code >= next_code ) {

   decode_stack[ 0 ] = (char) character;

   count = decode_string( 1, old_code );

  }

  else

   count  = decode_string( 0, new_code );

  character = decode_string[ count - 1 ];

  while ( count > 0 )

   putc( decode_stack[ --count ], output );

  if ( next_code <= MAX_CODE ) {

   dict[ next_code ].parent_code = old_code;

   dict[ next_code ].character = (char) character;

   next_code++;

  }

  old_code = new_code;

  }

  while ( argc-- > 0 )

  print( "Unknown argument: %s\n", *argv++ );

}

/*

* This hashing routine is responsible for finding the table location

* for a string/character combination. The table index is created

* by using an exclusive OR combination of the prefix and character.

* This code also has to check for collisions, and handles them by

* jumping around in the table.

*/

unsigned int find_child_node( parent_code, child_character )

int parent_code;

int child_character;

{

 int index;

 int offset;

 index = ( child_character << ( BITS - 8 ) ) ^ parent_code;

 if ( index == 0 )

  offset = 1;

 else

  offset = TABLE_SIZE - index;

 for ( ; ; ) {

  if ( dict[ index ].code_value == UNUSED )

   return( index );

  if ( dict[ index ].parent_code == parent_code &&

   dict[ index ].character == (char) child_character )

   return( index );

  index -= offset;

  if ( index < 0 )

   index += TABLE_SIZE;

 }

}

/*

* This routine decodes a string from the dictionary, and stores it

* in the decode_stack data structure. It returns a count to the

* calling program of how many characters were placed in the stack.

*/

unsigned int decode_string( count, code )

unsigned int count;

unsigned int code;

{

 while ( code > 255 ) {

  decode_stack[ count++ ] = dict[ code ].character;

  code = dict[ code ].parent_code;

  }

  decode_stack[ count++ ] = (char) code;

  return( count );

}

/*************************** End of LZW12.C ****************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Improvements

A second version of the LZW program, LZW15V.C, follows. It contains several enhancements, most of which are also found in the UNIX compress program.

LZW can be improved by increasing the size of the dictionary. As it becomes possible to store more and longer phrases, the program compresses to higher ratios. In this case, LZW15V.C uses a maximum code size of fifteen bits, which allows for a 32K phrase dictionary. While there is enough memory available on most MS-DOS machines to accommodate sixteen-bit code sizes, the program has to convert most of the indices used during compression from unsigned int values to long ints. This usually exacts a fairly heavy performance penalty on a sixteen-bit machine, so this program stayed with fifteen-bit values.

Unfortunately, moving to a larger code size actually retards compression when the file to be compressed is small. Since phrases are initially found and added to the dictionary at the same pace, whether the code is nine bits or fifteen bits long, the nine bit code will actually produce a smaller file.

There is a simple solution to this problem, however. Instead of always outputting codes using fifteen bits, LZW15V.C starts out using a nine-bit code, and it doesn't advance to ten bits until the dictionary has added 256 new entries. It progresses through ten, eleven, twelve, etc, until it starts using fifteen-bit codes. This puts it on an equal footing with compressors using a smaller code size.

To let the decompression program know when the bit size of the output code is going to change, a special BUMP_CODE is used. This code tells the decompression program to increase the bit size immediately. While it is possible to synchronize the compressor and decompressor so that they don't need to explicitly use a code, the BUMP_CODE was employed for purposes of clarity.

One final enhancement in LZW15V.C is the FLUSH_CODE. This tells the decompressor to throw away all phrases currently in the dictionary and to start over with a blank slate. When compressing files several hundred K bytes long, the dictionary will ordinarily fill up with phrases. At this point, the UNIX compress program starts to monitor the compression ration, looking for signs of decay. This type of algorithm could be employed in LZW15V.C, but a simpler heuristic was chosen: when the dictionary fills up, it is discarded. In practice this method generally gives results comparable to those of the compress algorithm.

/************************** Start of LZW15V.C ***************************

*

* This is the LZW module which implements a more powerful version

* of the algorithm. This version of the program has three major

* improvements over LZW12.C. First, it expands the maximum code size

* to 15 bits. Second, it starts encoding with 9 bit codes, working

* its way up in bit size only as necessary. Finally, it flushes the

* dictionary when done.

*

* Note that under MS-DOS this program needs to be built using the

* Compact or Large memory model.

*

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include "errhand.h"

#include "bitio.h"

/*

* Constants used throughout the program. BITS defines the maximum

* number of bits that can be used in the output code. TABLE_SIZE defines

* the size of the dictionary table. TABLE_BANKS are the number of

* 256 element dictionary pages needed. The code defines should be

* self-explanatory.

*/

#define BITS                15

#define MAX_CODE            ( ( 1 << BITS ) - 1 )

#define TABLE_SIZE          35023L

#define TABLE_BANKS         ( ( TABLE_SIZE >> 8 ) + 1 )

#define END_OF_STREAM       256

#define BUMP_CODE           257

#define FLUSH_CODE          258

#define FIRST_CODE          259

#define UNUSED             -1

/*

* Local prototypes.

*/

#ifdef__STDC__

unsigned int find_child_node( int parent_code, int child_character );

unsigned int decode_string( unsigned int offset, unsigned int code );

#else

unsigned int find_child_node();

unsigned int decode_string();

#endif

char *CompressionName = "LZW 15 Bit Variable Rate Encoder";

char *Usage  = "in-file out-file\n\n";

/*

* This data structure defines the dictionary. Each entry in the

* dictionary has a code value. This is the code emitted by the

* compressor. Each code is actually made up of two pieces: a

* parent_code, and a character. Code values of less than 256 are

* actually plain text codes.

*

* Note that in order to handle 16 bit segmented compilers, such as most

* of the MS-DOS compilers, it was necessary to break up the dictionary

* into a table of smaller dictionary pointers. Every reference to the

* dictionary was replaced by a macro that did a pointer dereference

* first. By breaking up the index along byte boundaries we should be as

* efficient as possible.

*/

struct dictionary

{

 int code_value;

 int parent_code:

 char character;

} *dict[ TABLE_BANKS ]:

#define DICT( i ) dict[ i >> 8 ][ i & Oxff ]

/*

* Other global data structures. The decode_stack is used to reverse

* strings that come out of the tree during decoding. next_code is the

* next code to be added to the dictionary, both during compression and

* decompression. current_code_bits defines how many bits are currently

* being used for output, and next_bump_code defines the code that will

* trigger the next jump in word size.

*/

char decode_stack[ TABLE_SIZE ];

unsigned int next_code;

int current_code_bits;

unsigned int next_bump_code;

/*

* This routine is used to initialize the dictionary, both when the

* compressor or decompressor first starts up, and also when a flush

* code comes in. Note that even though the decompressor sets all

* the code_value elements to UNUSED, it doesn't really need to.

*/

void InitializeDictionary( void )

{

 unsigned int i;

 for ( i = 0 ; i < TABLE_SIZE ; i++ )

  DICT( i ).code_value = UNUSED;

 next_code = FIRST_CODE;

 putc( 'F', stdout );

 current_code_bits = 9;

 next_bump_code = 511;

}

/*

* This routine allocates the dictionary. Since the total size of the

* dictionary is much larger than 64k, it can't be allocated as a single

* object. Instead, it is allocated as a set of pointers to smaller

* dictionary objects. The special DICT() macro is used to translate

* indices into pairs of references.

*/

void InitializeStorage( void )

{

 int i;

 for ( i = 0 ; i < TABLE_BANKS ; i++ ) {

  dict[ i ] = ( struct dictionary *)

                calloc( 256, sizeof ( struct dictionary ) );

  if ( dict[ i ] == NULL )

    fatal_error( "Error allocating dictionary space" );

  }

}

/*

* The compressor is short and simple. It reads in new symbols one

* at a time from the input file. It then checks to see if the

* combination of the current symbol and the current code are already

* defined in the dictionary. If they are not, they are added to the

* dictionary, and we start over with a new one symbol code. If they

* are, the code for the combination of the code and character becomes

* our new code. Note that in this enhanced version of LZW, the

* encoder needs to check the codes for boundary conditions.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[]

{

  int character;

  int string_code;

  unsigned int index;

  InitializeStorage();

  InitializeDictionary();

  if ( ( string_code = getc( input ) ) == EOF )

    string_code = END_OF_STREAM;

  while ( ( character = getc( input ) ) != EOF ) {

    index = find_child_node( string_code, character );

    if ( DICT( index ).code_value != -1)

      string_code = DICT( index ).code_value;

    else {

      DICT( index ).code_value = next_code++;

      DICT( index ).parent_code = string_code;

      DICT( index ).character = (char) character;

      OutputBits( output,

                  (unsigned long) string_code, current_code_bits );

      string_code = character;

      if ( next_code > MAX_CODE ) {

        OutputBits( output,

                    (unsigned long) FLUSH_CODE, current_code_bits );

        InitializeDictionary();

      } else if ( next_code > next_bump_code ) {

        OutputBits( output,

                    (unsigned long) BUMP_CODE, current_code_bits );

        current_code_bits++;

        next_bump_code <<= 1;

        next_bump_code |= 1;

        putc( 'B', stdout );

      }

    }

  }

  OutputBits( output, (unsigned long) string_code, current_code_bits );

  OutputBits( output, (unsigned long) END_OF_STREAM, current_code_bits);

  while ( argc— > 0 )

    printf( "Unknown argument: %s\n", *argv++ );

}

/*

* The file expander operates much like the encoder. It has to

* read in codes, then convert the codes to a string of characters.

* The only catch in then whole operation occurs when the encoder

* encounters a CHAR+STRING+CHAR+STRING+CHAR sequence. When this

* occurs, the encoder outputs a code that is not presently defined

* in the table. This is handled as an exception. All of the special

* input codes are handled in various ways.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

  unsigned int new_code;

  unsigned int old_code;

  int character;

  unsigned int count;

  InitializeStorage();

  while ( argc-- > 0 )

    printf( "Unknown argument: %s\n", *argv++ );

  for ( ; ; ) {

    InitializeDictionary();

    old_code = (unsigned int) InputBits( input, current_code_bits );

    if ( old_code == END_OF_STREAM )

      return;

    character = old_code;

    putc( old_code, output );

    for ( ; ; ) {

      new_code = (unsigned int) InputBits( input, current_code_bits );

      if ( new_code == END_OF_STREAM )

        return;

      if ( new_code == FLUSH_CODE )

        break;

      if ( new_code == BUMP_CODE ) {

        current_code_bits++;

        putc( 'B', stdout );

        continue;

      }

      if ( new_code >= next_code ) {

        decode_stack[ 0 ] = (char) character;

        count = decode_string( 1, old_code );

      }

      else

        count = decode_string( 0, new_code );

      character = decode_stack[ count - 1 ];

      while ( count > 0 )

        putc( decode_stack[ --count ], output );

      DICT( next_code ).parent_code = old_code;

      DICT( next_code ).character = (char) character;

      next_code++;

      old_code = new_code;

    }

  }

}

/*

* This hashing routine is responsible for finding the table location

* for a string/character combination. The table index is created

* by using an exclusive OR combination of the prefix and character.

* This code also has to check for collisions, and handles them by

* jumping around in the table.

*/

unsigned int find_child_node( parent_code, child_character )

int parent_code;

int child_character;

{

  unsigned int index;

  int offset;

  index = ( child_character << ( BITS - 8 ) ) ^ parent_code;

  if ( index == 0 )

    offset = 1;

  else

    offset = TABLE_SIZE -index;

  for ( ; ; ) {

    if ( DICT( index ).code_value == UNUSED )

      return( (unsigned int) index );

    if ( DICT( index ).parent_code == parent_code &&

      DICT( index ). character == (char) child_character )

      return( index );

    if ( index >= offset )

      index -= offset;

    else

      index += TABLE_SIZE - offset;

  }

}

/*

* This routine decodes a string from the dictionary, and stores it

* in the decode_stack data structure. It returns a count to the

* calling program of how many characters were placed in the stack.

*/

unsigned int decode_string( count, code )

unsigned int count;

unsigned int code;

{

  while ( code > 255 ) {

    decode_stack[ count++ ] = DICT( code ).character;

    code = DICT( code ).parent_code;

  }

  decode_stack[ count++ ] = (char) code;

  return( count );

}

/**************************** End of LZW15V.C **************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Patents

One note of caution regarding the use of the LZW algorithm. Terry Welch filed for, and was awarded, a U.S. patent covering at least some portions of his algorithm. This patent is presently assigned to Unisys, which has made public its intention to protect its intellectual property rights, and has in recent years begun to do so. LZW compression is defined as part of the CCITT V.42bis specification, and Unisys has defined specific terms under which it will license the algorithm to modem manufacturers. Unisys is now in the process of seeking license payments from software vendors who use the LZW algorithm in programs, most notably in the GIF graphics file format.

Clearly LZW is a derivative work of the LZ78 algorithm, but defining the boundaries of what is covered by the patent and what is not probably requires the assistance of a skilled patent attorney. Over the past ten years, quite a few software copyright battles have been fought in the courts, enough so that software developers can sensibly use some general rules. The same cannot be said for software patents. The U.S. patent office has only begun issuing these patents in a major way since the mid 1980s, and very little significant litigation has made its way through the courts. Programmers and manufacturers would be wise to seek competent counsel before stepping into these waters.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 10
Speech Compression

Manipulation of sound by computers is a relatively new development. It has been possible since the birth of digital computers, but only in the last five years or so has inexpensive hardware brought this to the average user's desktop. Now the ability to play digitized sound is expected to be an integral part of the "multimedia revolution."

The use of multimedia focuses the issue of data compression for most users. Computer graphics in particular quickly take up all available disk space. Digitized audio is far less voracious in its storage requirements, but even so it can quickly swallow up all free space on the average user's hard disk.

Fortunately for computer users, the world of telephony has used digitized audio since the 1960s, and extensive research has been done on effective methods of encoding and compressing audio data. The world's telecommunications companies were intensely aware of the cost of transmission bandwidth and made efforts to reduce expenses in this area. Computer users today benefit from much of this research.

This chapter looks first at some of the basic concepts involved in using digital audio, including the software and hardware in today's generation of computers. Next, it looks at how well conventional lossless compression techniques work on digitized voice. Finally, it explores some lossy techniques.

Digital Audio Concepts

For modern computers to manipulate sound, they first have to convert it to a digital format. The sound samples can then be processed, transmitted, and converted back to analog format, where they can finally be received by the human ear.

Digitization of sound began in earnest in the early 1960s. Like much of our early computer technology, credit for development lies with AT&T, which at that time had a regulated monopoly on long-distance service in the United States. In 1962, AT&T established the first commercial digital telephone link, a T1 interoffice trunk in Chicago.

In the short space of thirty years, we have seen the long-distance network in the United States convert almost entirely from analog to digital transmission. Virtually all new switching equipment installed by telephone companies today is digital. But analog switching is still found in older installations and in the smaller PBX and key systems installed in businesses. Of course, the final subscriber loop between the telephone company and the end user is still persistently analog.

Digital audio is now coming of age in the highly visible consumer electronics arena as well. The digital compact disk has nearly completed its displacement of analog LP records. It remains to be seen whether digital audio tape will do the same thing to analog cassette tape, but it seems likely that some day most recorded music will be distributed in digital format.

Fundamentals

While this book cannot give a complete course in digital signal processing, it certainly has room to cover a few basic concepts involved in digital sound. Figure 10.1 shows a typical audio waveform as it might be displayed on an oscilloscope. The X axis in this diagram represents time. The Y axis represents a voltage measured at an input device, typically a microphone. The microphone attempts to faithfully reproduce changes in air pressure caused by sound waves traveling through it.

Some human ears can hear sounds at frequencies as high as 20,000Hz and nearly as low as DC. The dynamic range of our hearing is so wide that we have to employ a logarithmic scale of measurement, the decibel, to reasonably accommodate it. This presents a unique set of requirements for digitization.

A waveform like the shown in Figure 10.1 is typical of audio sample. It isn't a nice, clean sine wave that has a regular period and can be described as a simple mathematical function. Instead, it is a combination of various frequencies at different amplitudes and phases. When combined, we see something that looks fairly irregular and not easy to characterize.

Figure 10.1 A typical audio waveform.

This particular "snapshot" shows about 5 milliseconds (ms) of output. Notice that the largest recognizable components of the waveform appear to have a period of roughly two milliseconds. This corresponds to a frequency of about 500Hz, a fairly characteristic frequency found in speech or music.

The first step in working with digital audio is "sampling." Sampling consists of taking measurements of the input signal at regular times, converting them to an appropriate scale, and storing them. Figure 10.2 shows the same waveform sampled at an 8KHz rate. This means that 8,000 times per second a measurement is taken of the voltage level of the input signal. The measurement points are marked with an "x" on the waveform.

Figure 10.2 A typical audio waveform being sampled at 8KHz.

In most computer systems, this first step of digitization is done with an analog-to-digital converter (ADC). The ADC takes a given voltage and scales it to an appropriate digital measurement. An eight-bit ADC, for example, might have a "full scale" input voltage of 500 millivolts (mv)—it would output an eight-bit value of 255 if the input voltage were 500mv and zero if the input voltage were zero. A voltage between these values would be scaled to fit in the linear range of zero to 255.

Since audio signals are AC in natured, the ranges are usually adjusted so that a zero voltage signal falls in the middle of the range. For the previous example, the range would be adjusted to between -250mv and +250mv. Outputs from the eight-bit ADC would range from -128 to +127.

The stored sample points then represent a series of voltages that were measured at the input of the ADC. Figure 10.3 shows the representation of those voltages overlaid with the input AC signal. Note that since the sample points in this case are occurring many times more frequently than the period of the waveform, the digital samples themselves trace the analog signal very accurately.

Figure 10.3 Sample voltages overlaid with the input AC signal

Now that the sound has been digitized, it can be stored via computer using any number of technologies, ranging from fast storage, such as main processor RAM, to off-line slow storage on magnetic tape. The actual speed of the storage medium is relatively unimportant with digital sound, since the bandwidth needed to accurately store the sound is relatively slow compared to most digital media.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Eventually, the sound needs to be played back. This is done via another electronic component that is the converse of the ADC: the digital-to-analog converter (DAC). The DAC is responsible for taking a digital value and converting it to a corresponding analog signal. To be effective, the conversion process needs to be the mirror image of that performed when converting the analog signal to digital. While the exact voltages produced at the output of the DAC do not need to be identical to those seen at the input, they do need to be proportional to one another so that one waveform corresponds to the other. In addition, the samples need to be output at exactly the same rate that they were read in. Any deviation here will cause the output frequencies to be shifted up or down from the input, generally not a good thing.

Figure 10.4 shows the output of the DAC when given the same set of samples produced in Figure 10.2. At first glance, it seems that this is a radically different waveform. All the nice, smooth shapes shown in the earlier figures are gone, replaced by this stair-step, rectangular, artificial-looking creation.

Figure 10.4 DAC output

Fortunately, Figure 10.4 is not that far removed from Figure 10.1. Mathematically, the sharp jumps that occur when we move from sample to sample represent high-frequency components in the output signal. These can (and must) be eliminated from the signal by means of a low-pass filter that lies between the output of the DAC and the final amplification stage of the audio output.

A low-pass filter is a network of electrical components designed to let frequencies below a certain value pass through it unhindered, while attenuating frequencies above that point. An ideal low-pass filter used with the samples shown here would completely stop any frequency above 4KHz and let frequencies below 4KHz pass through with no attenuation.

In practice, low-pass filters don't work perfectly, but even a low-budget filter can take Figure 10.4 and create a nearly indistinguishable copy of Figure 10.1. Without the filter, the sound sample will still be intelligible, but it will be filled with distracting high-frequency "noise" that is part of the reproduction process.

Figure 10.5 shows the same figure when the sampling rate has been stepped up to a much higher rate. This increase in sampling rate clearly does a more accurate job of reproducing the signal. The next section discusses how variations in these parameters affect the output signal.

Figure 10.5 Sampling at a much higher rate.

Sampling Variables

When an audio waveform is sampled, two important variables affect the quality of the reproduction: the sample rate and the sample resolution. Both are important factors, but they play different roles in determining the level of distortion produced when a sample is played back.

The sample resolution is simply a measure of how accurately the digital sample can measure the voltage it is recording. When the input range is -500mv to +500mv, for example, an eight-bit ADC can resolve the input signal down to about 4mv. So an input signal of 2mv will either get rounded up to 4mv or down to 0mv. This is called a quantization error.

Figure 10.6 shows the results of quantization error when sampling a waveform. In some cases the sample point has a larger magnitude than the audio signal, but in other places it has less. When the digitized signal is played back through a DAC, the output waveform will closely track the sample points, resulting in a certain amount of distortion.

Figure 10.6 Quantization error when sampling a waveform

It might seem that eight bits should be enough to accurately record audio data, but this may not be the case because of the large dynamic range of audio the human ear can detect. If our 500mv range example were used, we might find that our input signal magnitudes range from 1mv to 500mv in a single recording session. The crash of drums in an orchestra could push the ADC to its limits, while a delicate violin solo may never go outside 5mv. If the minimum digital resolution is only 5mv, a very noticeable level of distortion will be introduced during this part of a recording session.

The sampling rate plays a different role in determining the quality of digital sound reproduction. One classic law in digital signal processing was published by Harry Nyquist in 1993. He determined that to accurately reproduce a signal of frequency f, the sampling rate has to be greater than 2*f. This is commonly called the Nyquist Rate.

The audio signal in Figure 10.7 is being measured at a considerably slower rate than that shown in the previous examples, with noticeably negative consequences. At several places in the waveform it is not even sampled a single time during an excursion above or below the center line.

Figure 10.7 A slower sampling rate.

Figure 10.8 shows the waveform we could expect after playing back the digitized samples stored from Figure 10.7. Clearly, after the digitized output is filtered, the resulting waveform differs quite a bit from that shown in the previous figure. What has happened is that the high-frequency components of the waveform have been lost by the slower sampling rate, letting only the low-frequency parts of the sample through.

Figure 10.8 The waveform after playing back digitized samples.

The human ear hears sound up to 20KHz, which implies that we need to sample audio to 40KHz or better to achieve good reproduction. In fact, the sampling rate used for digital reproduction of music via compact disk or digital audio tape is 44KHz, using sixteen-bit samples. The quality of sound achieved at this sampling rate is generally acknowledged to be superior.

This does not mean that all digital recordings have to be done at 44KHz rates. Virtually every digital phone system in the world uses an 8KHz sampling rate to record human speech, with generally good results. This means that the phone system is unable to pass any signal with a frequency of 4KHz or higher. This clearly does not render the system useless—millions of long-distance calls over digital lines are made every day. The average speech signal is composed of many different frequencies, and even if everything above 4KHz is discarded, most of the speech energy still makes it through the system. Our ears detect this loss as a lower-fidelity signal, but they still understand it quite well.

The ultimate test of all this is how the audio output sounds to our ears. It is difficult to quantify a "quality of sound" measurement is strictly mathematical terms, so when discussing audio output, it is always best to temper judgments with true listener trials.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

PC-Based Sound

Some exotic work in digital signal processing has been going on for years, but it usually involved expensive special-purpose peripherals far out of reach of the average computer installation.

Early desktop computers did not really push the state of the art in sound reproduction. Original IBM and Apple computers both had built-in speakers as standard equipment, but they gave the programmer only a single bit with which to control the speaker. This meant the speaker could generally be used only to emit beeps and buzzes, not true digitized sound.

In the early 1980s, however, many computer manufacturers saw that a true digitized sound capability could be added to their computers at a relatively low cost. Apple was the most prominent manufacturer, adding an eight-bit DAC to the Macintosh, which opened the door to the use of true digitized audio.

Most desktop computers today are IBM compatible ISA computers based on Intel's 80x86 CPU chips. Unfortunately for sound enthusiasts, IBM has not yet elected to add sound capability to the PC, but third-party solutions are relatively inexpensive. The sound samples used in this book have been created and manipulated using the Sound Blaster card, manufactured by Creative Labs. But several other cards are on the market that can play digitized sound samples, and any of these can be used, provided file-format conversion utilities exist.

The next generation of digitized sound on the desktop is now here. Many of today's consumer machines can digitize and playback 44 KHz sixteen-bit CD-quality sound data. Only a few years ago, this capability seemed a bit unusual. The exotic black cube from NeXT Computer seemed to presage the future when it was first introduced, incorporating a digital signal-processor (DSP) chip as a co-processor; the intent was to offload work, such as manipulating digitized audio, from the main CPU. For a while, other manufacturers followed this design, for example, Apple with its AV line of Macintosh computers. Today, the vast majority of PC-compatible machines sold in retail consumer outlets come equipped with sound cards and CD-ROM drives—by one count, over 75%. More recently, it seems that the pendulum may shift back in the other direction, as a new largesse of processing power in the CPU will allow its deployment for audio and video processing, in addition to handling its regular duties. Intel is promulgating such a configuration with its new P6 processor, which has cycles to spare, that can be used for compressing and decompressing audio and video on the fly, even while crunching numbers in a spreadsheet. Regardless of how it's done, the multimedia capabilities of today's machines only highlight the need for data compression, since they fill up a hard disk faster than ever before. The explosive growth of the Internet and the World-Wide Web, which allows multimedia-enriched distributed documents, also increases the need for compression, because the bandwidth of communications links is not increasing as fast as the processing power of the host computers.

The files distributed with this book will be "raw" sound files. These will be pure binary recordings of eight-bit input data. Virtually all sound software on desktop machines today expects more than that for a sound file, but many software packages have utilities to convert raw sound files to a particular format. The Sound Blaster, for example, includes an executable program called VOC-HDR.EXE that prepends a header file to a raw sound file. The sound samples here were all sampled at 11KHz, a commonly used rate for medium-fidelity digital recording.

By supplying sound data only, the code here can concentrate on compression, without worries about additional superfluous data in the file. A full-fledged sound-file compression package by necessity needs to support the dozens of different file formats in existence, but that mostly consists of implementation details.

Some sound capability resources are available for a relatively small investment. Many on-line services, such as Compuserve, America Online, GEnie, and BIX, have active forums for audio manipulation. There are also active forums on the Internet, such as Web sites and Usenet newsgroups, focusing on digital audio. Freeware and shareware utility programs available in these forums do a passable job of playing sound out of the PC speaker. Other programs convert sound files between various formats. It wouldn't be feasible to try to list specific examples here, but it should be relatively simple to find this type of software. In addition, third-party sound cards are available for a relatively low investment.

Lossless Compression of Sound

The original applications for sound compression could not take advantage of lossless data-compression techniques. One characteristic of all the compression techniques discussed so far in this book is that the amount of compression they achieve on a given data set is not known in advance. In some cases, the compression program can actually cause the data to expand, taking up more space than it occupied before.

In the 1960s, telecommunications researchers were trying to find ways to put more conversions on digital trunk lines, particularly on "expensive" lines, such as undersea cables or satellite links. Unlike disk space, which is somewhat flexible, these links have a fixed total bandwidth. A single telephone conversion might be allocated a 64Kbps slot on one of these channels. If it suddenly needed 100Kbps because the compression code hit a rough spot, there would be a major problem.

These early researchers were attempting to divide a 64Kbps channel into two 32Kbps channels to get two for the price of one. This required compression techniques that would consistently compress data by 50 percent, even if it meant losing some resolution.

Today, when trying to compress sound on disk for multimedia applications, we are in a slightly better position. We store and retrieve data from fixed disks, a more flexible medium for our work. If our files are compressed by 95 percent in some cases, and -10 percent in others, it will not really cause any trouble.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Problems and Results

How much can we compress voice files using conventional lossless techniques? To answer this question, a set of six short sound files were created, ranging in length from about one second to about seven seconds. To determine how compressible these files were, they were packed into an archive using ARJ 2.10, a shareware compression program that generally compresses as well or better than any other general-purpose program.

ARJ results showed that voice files did in fact compress relatively well. The six sample raw sound files gave the following results:

Filename | Original | Compressed | Ratio
SAMPLE-1.RAW | 50777 | 33036 | 35%
SAMPLE-2.RAW | 12033 | 8796 | 27%
SAMPLE-3.RAW | 73091 | 59527 | 19%
SAMPLE-4.RAW | 23702 | 9418 | 60%
SAMPLE-5.RAW | 27411 | 19037 | 30%
SAMPLE-6.RAW | 15913 | 12771 | 20%

These compression results look relatively promising. All the files were compressible to some extent, and some were reduced to less than half their original size. This level of compression is undoubtedly useful and may well be enough for some applications.

ARJ.EXE performs two sorts of compression on an input data stream. First, it does an LZSS type of windowed string matching on the string. The output from LZSS is, of course, a stream of tokens referring to either individual characters or matched strings. ARJ, like LHArc, takes LZSS a step further by performing Huffman compression on the output stream. Compressing these sound files using just LZSS compression and simple order-0 Huffman coding might tell us a little bit about what kind of redundancy is in these voice files.

To check the results, the files were compressed again with the LZSS program from Chapter 8 and the HUFF program from chapter 3. The results of these experiments are shown in the following table.

Filename | ARJ Ratio | LZSS Ratio | HUFF Ratio
SAMPLE-1.RAW | 35% | 23% | 26%
SAMPLE-2.RAW | 27% | 5% | 30%
SAMPLE-3.RAW | 19% | 3% | 17%
SAMPLE-4.RAW | 60% | 25% | 27%
SAMPLE-5.RAW | 30% | 15% | 32%
SAMPLE-6.RAW | 20% | 2% | 18%

The table shows that in every case, we perform more compression with simple order-0 Huffman coding than we do LZSS dictionary compression. Since LZSS is normally a much more powerful compression technique, this is a telling result.

What LZSS takes advantage of when compressing is repeated strings of characters in the file. Order-0 Huffman coding just takes advantage of overall frequency differences for individual sequences. What we see in these sounds files is some overall frequency difference between the various codes that make up the files, but not as many repeated strings as we might normally expect.

A look at snapshots of these sound files reveals some of the character of the data we are trying to compress. Figure 10.9 shows a section of about 600 sample points from SAMPLE-3.RAW. In this case, the sound samples are only taking up about 30 percent of the possible range allocated for them by the hardware. While individual samples can range from +127 to -128, in this snapshot they run only from about +30 to -30.

By only using a portion of the available bandwidth, a sound file automatically makes itself a good candidate for Huffman compression. The sample shown in Figure 10.9 can probably be compressed by about 30 percent by just squeezing the samples down from eight bits to six or so bits. This is, in effect, what the Huffman coder does.

Figure 10.9 Sample points from SAMPLE-3.RAW.

Looking for repeated sequences in a sample such as this is less fruitful. We can certainly see a pattern in the waveform, but it is somewhat irregular, and it is not likely to produce many repeated patterns of even length 2. If we keep sampling long enough, random chance dictates that repeated strings will recur, but the compression will be much less than in a data or program file.

Figure 10.10 shows a sound sample that is a much more difficult candidate for compression. Unlike Figure 10.9, this sound sample utilizes nearly the entire dynamic range of the ADC, so an order-0 Huffman encoder will be much less effective. Likewise, the chances of finding repeated patterns with an LZSS algorithm diminish considerably here. This is the type of file that gives us only a few percentage points of compression.

Figure 10.10 A sound sample that is difficult to compress.

Of course, even when looking at a "busy" sample like this, the human eye picks out patterns. The peaks and valleys of the waveform occur at somewhat regular intervals, telling us that sinusoidal waveforms are present in the signal. Unfortunately, our existing compression algorithms aren't equipped to find this type of redundancy in an input waveform. To do better, we need to move to a new frontier: lossy compression.

Lossy Compression

The very word "lossy" implies that when using this type of compression we are going to give up a certain amount of precision. This would certainly not be acceptable when compressing the data or text files we use on our computers. We could probably compress the M&T Books annual financial statement if we rounded all figures off to the nearest million dollars, for example, but the accounting department would definitely have a problem working with the books after that.

By digitizing sound samples, however, we have in effect given up quite a bit of precision. For example, our sound samples used in this chapter were all recorded at 11KHz. This means that we have thrown away the entire portion of every sample greater than 5.5KHz in frequency. We are also using only eight-bit samples, so we are introducing a significant amount of distortion in the form quantization error.

All these factors are taken into account when designing the hardware and software for digitization. Instead of trying to perfectly reproduce analog phenomena, we instead make compromises that give us reproduction that is satisfactory for our purposes.

Likewise, when we look at lossy compression, we once again accept a certain loss in fidelity. The signal we get after going through a compression/expansion cycle will not be identical to the original, but we can adjust the parameters to get better or worse fidelity, and likewise better or worse compression.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Lossy compression is not necessarily an end to itself. We frequently use lossy compression in a two-phase process: a lossy stage followed by a lossless stage. One nice thing about lossy compression is that it frequently smooths out the data, which makes it even more suitable for lossless compression. So we get an extra unexpected benefit from lossy compression, above and beyond the compression itself.

Silence Compression

Silence compression on sound files is the equivalent of run-length encoding on normal data files. In this case, however, the runs we encode are sequences of relative silence in a sound file. This is a lossy technique because we replace the sequences of relative silence with absolute silence.

Figure 10.11 shows a typical sound sample that has a long sequence of silence. The first two thirds of it is composed of silence. Note that though we call it "silence," there are actually very small "blips" in the waveform. These are normal background noise and can be considered inconsequential.

Figure 10.11 A typical sound sample with a long sequence of silence.

A compression program for a sample like this needs to work with a few parameters. First, it needs a threshold value for what can be considered silence. With our eight-bit samples, for example, 80H is considered "pure" silence. We might want to consider any sample value within a range of plus or minus three from 80H to be silence.

Second, it needs a way to encode a run of silence. The sample program that follows creates a special SILENCE_CODE with a value of FF used to encode silence. The SILENCE_CODE is followed by a single byte that indicates how many consecutive silence codes there are.

Third, it needs a parameter that gives a threshold for recognizing the start of a run of silence. We wouldn't want to start encoding silence after seeing just a single byte of silence. It doesn't even become economical until three bytes of silence are seen. We may want to experiment with even higher values than three to see how it affects the fidelity of the recording.

Finally, we need another parameter that indicates how many consecutive non-silence codes need to be seen in the input stream before we declare the silence run to be over. Setting this parameter to a value greater than one filters out anomalous spikes in the input data. This can also cut back on noise in the recording.

The code to implement this silence compression follows. It incorporates a starting threshold of four and a stop threshold of two, so we have to see four consecutive silence codes before we consider a run started.

SILENCE.C by definition spends a lot of time looking ahead at upcoming input data. For example, to see if a silence run has really started the program must look at the next upcoming four input values. To simplify this, the program keeps a look-ahead buffer full of input data. It never directly examines the upcoming data read in via getc(). Instead, it looks at the bytes read into the buffer. This makes it easy to write functions to determine if a silence run has been started or if one is now over.

/************************ Start of SILENCE.C ************************

*

* This is the silence compression coding module used in chapter 10.

* Compile with BITIO.C, ERRHAND.C, and either MAIN-C.C or MAIN-E.C

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#include "bitio.h"

#include "errhand.h"

#include "main.h"

/*

* These two strings are used by MAIN-C.C and MAIN-E.C to print

* messages of importance to the user of the program.

*/

char *CompressionName = "Silence compression";

char *Usage = "infile outfile\n";

/*

* These macros define the parameters used to compress the silent

* sequences. SILENCE_LIMIT is the maximum size of a signal that can

* be considered silent, in terms of offset from the center point.

* START_THRESHOLD gives the number of consecutive silent codes that

* have to be seen before a run is started. STOP_THRESHOLD tells how

* many non-silent codes need to be seen before a run is considered

* to be over. SILENCE_CODE is the special code output to the

* compressed file to indicate that a run has been detected.

* SILENCE_CODE is always followed by a single byte indicating how

* many consecutive silence bytes are to follow.

*/

#define SILENCE_LIMIT   4

#define START_THRESHOL  5

#define STOP_THRESHOL   2

#define SILENCE_CODE    Oxff

#define IS_SILENCE( c ) ( (c) >( 0x7f - SILENCE_LIMIT ) && \

                        (c) < ( 0x80 + SILENCE_LIMIT ) )

/*

* BUFFER_SIZE is the size of the look-ahead buffer. BUFFER_MASK is

* the mask applied to a buffer index when performing index math.

*/

#defined BUFFER_SIZE 8

#defined BUFFER_MASK 7

/*

* Local function prototypes.

*/

#ifdef __STDC__

int silence_run( int buffer [], int index )

int end_of_silence( int buffer[], int index)

#else

int silence_run();

int end_of_silence();

#endif

/*

* The compression routine has the hard job here. It has to detect

* when a silence run has started and when it is over. It does this

* by keeping up-and-coming bytes in a look-ahead buffer. The buffer

* and the current index are passed ahead to routines that check to

* see if a run has started or if it has ended.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

  int look_ahead[ BUFFER_SIZE ];

  int index;

  int i;

  int run_length;

  for ( i = 0 ; i < BUFFER_SIZE ; i++ )

   look_ahead[ i ] = getc( input );

  index = 0;

  for ( ; ; ) {

   if ( look_ahead[ index ] == EOF )

    break;

/*

* If run has started, I handle it here. I sit in the do loop until

* the run is complete, loading new characters all the while.

*/

   if ( silence_run( look_ahead, index ) ) {

    run_length = 0;

    do {

     look_ahead[ index ++ ] = getc( input );

     index &= BUFFER_MASK;

     if ( ++run_length == 255 ) {

      putc( SILENCE_CODE, output->file );

      putc( 255, output->file );

      run_length = 0;

   }

  } while ( !end_of_silence( look_ahead, index ) );

  if ( run_length > 0 ) {

   putc( SILENCE_CODE, output->file );

   putc( run_length, output->file );

  }

 }

/*

* Eventually, any run of silence is over, and I output some plain codes.

* Any code that accidentally matches the silence code gets silently

* changed.

*/

   if ( look_ahead[ index ]== SILENCE_CODE )

    look_ahead[ index ]--;

   putc( look_ahead[ index ], output->file );

   look_ahead[ index++ ] = getc( input );

   index & = BUFFER_MASK;

  }

  while ( argc-- > 0 )

   printf( "Unused argument: %s\n", *argv++ );

}

/*

* The expansion routine used here has a very easy time of it. It just

* has to check for the run code, and when it finds it, pad out the

* output file with some silence bytes.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input;

FILE *output;

int argc;

char argv[];

{

  int c;

  int run_count;

  while ( ( c = getc( input->file ) ) != EOF ) {

   if ( c == SILENCE_CODE ) {

    run_count = getc( input->file );

    while ( run_count-- > 0 )

     putc( 0x80, output );

   } else

    putc( c, output );

  }

  while ( argc-- > 0 )

   printf( "Unused argument: %s\n", *argv++ );

}

/*

* This support routine checks to see if the look-ahead buffer

* contains the start of a run, which by definition is

* START_THRESHOLD consecutive silence characters.

*/

int silence_run( buffer, index )

int buffer[];

int index;

{

  int i;

  for ( i = 0 ; i < START_THRESHOLD ; i++ )

   if ( !IS_SILENCE( buffer[ ( index + i ) & BUFFER_MASK ] ) )

    return( 0 );

  return( 1 );

}

/*

* This support routine is called while we are in the middle of a

* run of silence. It checks to see if we have reached the end of

* the run. By definition this occurs when we see STOP_THRESHOLD

* consecutive non-silence characters.

*/

int end_of silence( buffer, index )

int buffer[];

int index;

{

  int i;

  for ( i = 0 ; i < STOP_THRESHOLD ; i++ )

   if ( IS_SILENCE( buffer[ ( index + i ) & BUFFER_MASK ] ) )

    return( 0 );

  return( 1 );

}

/************************ End of SILENCE.C ************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Just how effective silence compression can be at compressing files is shown in the following table. As expected, files without much silence in them were not greatly affected. But files that contained significant gaps were compressed quite a bit.

File Name | Raw Size | Compressed Size | Compression
SAMPLE-1.RAW | 50777 | 37769 | 26%
SAMPLE-2.RAW | 12033 | 11657 | 3%
SAMPLE-3.RAW | 73091 | 73072 | 0%
SAMPLE-4.RAW | 13852 | 10962 | 21%
SAMPLE-5.RAW | 27411 | 22865 | 17%

The final question to ask about silence detection is how it affects the fidelity of input files. The best way to answer that is to take the sample files, compress them, then expand them into new files. The expanded files should differ only from the originals in that strings of characters near the silence value of 80H should all have been arbitrarily made exactly 80H, changing slightly noisy silence to pure silence.

In most cases, it is not possible to tell the sound samples that have been through a compression/expansion cycle from the originals. By tinkering with the parameters, it is possible to start erasing significant sections of speech, but that obviously means the parameters are not set correctly. All in all, when applied correctly, silence compression provides an excellent way to squeeze redundancy out of sound files.

Companding

Silence compression can be a good way to remove redundant information from sound files, but in some cases it may be ineffective. In the preceding examples, SAMPLE-3.RAW had so few silent samples it was only reduced by a few bytes out of 73K. This situation is somewhat analogous to using run-length encoding on standard text or data files: it will sometimes produce great gains, but it is not particularly reliable.

In the early 1960s, telecommunications researchers were looking for a method of data compression that could always reduce the number of bits in a sound sample. Customer satisfaction tests showed that it took about thirteen bits of resolution in the DAC sampled at 8,000Hz to provide an acceptable voice connection, but it seemed likely that much of that resolution was going to waste.

We need thirteen bits of resolution in a phone conversion because of the large dynamic range of the human voice. To accommodate a loud speaker, the voltage input range of the DAC has to be set at a fairly high level. The problem is that the input voltage from a very soft voice is several orders of magnitude lower than this. If the ADC had eight bits of resolution, it would only detect input signals close to 1 percent of the magnitude of the highest input. This proved unacceptable.

It turns out, however, that the thirteen bits of resolution needed to pick up the voice of the quietest speaker is overkill for resolution of the loudest speaker. If our microphone input for a loud speaker is in the neighborhood of 100mv, we might only need one millivolt of resolution to provide good sound reproduction. The thirteen-bit ADC might be giving 200 microvolt resolution, which turns out to be more than is necessary.

The telecommunications industry solved this using a non-linear matched set of ADCs and DACs. The normal ADC equipment used in desktop computers (and most electronic equipment) uses a linear conversion scheme in which each increase in a code value corresponds to a uniform increase in input/output voltage. This arrangement is shown in Figure 10.12.

Figure 10.12 A linear conversion scheme in which each increase in a code value corresponds to a uniform increase in input/output voltage.

Using a linear conversion scheme such as this, when we go from code 0 to code 1, the output voltage from the DAC might change from 0mv to 1mv. Likewise, going from code 100 to code 101 will change the DAC output voltage from 100mv to 101mv.

The system in our telecommunications equipment today uses a "companding codec"—jargon for "compressing/expanding coder/decoder." The codec is essentially a chip that combines several functions, including those of the DAC, ADC, and input and output filters. We are concerned with the DAC and ADC.

The codec used in virtually all modern digital telephone equipment does not use a standard linear function when converting codes to voltages and voltages to codes. Instead, it uses an exponential function that changes the size of the voltage step between codes as the codes grow larger. Figure 10.13 shows an example of what this curve looks like. The resolution for smaller code values is much finer than at the extremes of the range. For example, the difference between a code of zero and a code of one might be 1mv, while the difference between code 100 and code 101 could be 10mv.

Figure 10.13 An exponential function that changes the size of voltage steps.

The exponential curve defined for telecommunications codecs gives an effective range of thirteen bits out of a codec that only uses eight-bit samples. We can do the same thing with out eight-bit sound files by squeezing eight-bit samples into a smaller number of codes.

Our eight-bit sound files can be considered approximately seven-bit samples with a single sign bit, indicating whether the output voltage is positive or negative. This gives us a range running from zero to 128 to encode for the output of our non-linear compression function.

If we assume that we will have N codes to express the range of zero to 127, we can develop a transfer function for each code using the following equation:

output = 127.0 * ( pow( 2.0, code / N ) - 1.0 )

In other words, we calculate the output by raising 2 to the code/N power. The value of code/N will range from zero for code 0 up to one for code N, resulting in an output range that runs from zero to 127, with a decidedly non-linear look.

An example of how this might work would be found if we used eight samples to encode the range zero to 128. This, in effect, compresses seven bits to three. The output value produced by an input code is shown in the table that follows.

Transforming three bits to seven
Input Code | Output Value
0 | 0
1 | 13
2 | 28
3 | 44
4 | 62
5 | 81
6 | 103
7 | 127


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The output step taken here between codes 0 and 1 is 13, but between 6 and 7 it goes up to 24. This means that we have more resolution in the smaller input ranges, where we need it, and less resolution for the loud sounds, where we don't need it.

This compression algorithm does an excellent job of compressing sound without damaging quality. It also has several nice features. First, both decompression and compression can take place via table lookup, resulting in extremely fast processing. Second, the amount of compression is known in advance, since the amount of compression does not vary based on the input data stream. Finally, the algorithm can be tuned to any desired compression ratio simply by varying the number of codes used in the compressed file.

The code to implement a simple version of this compression scheme follows. File COMPAND.C can be called with an additional parameter that tells how many bits to use per code during the compression process. Experimental results show that compressing eight-bit samples to four bits results in very little degradation. Three bits or fewer starts to generate high-frequency components in the output, which are heard as static, or noise. But it is a tribute to the human ear to note that even samples compressed down to one bit per sample are still frequently intelligible.

/************************ Start of COMPAND.C ************************

*

* This is the companding module used in chapter 10.

* Compile with BITIO.C, ERRHAND.C, and either MAIN-C.C or MAIN-E.C

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#include <math.h>

#include "bitio.h"

#include "errhand.h"

#include "main.h"

/*

* These two strings are used by MAIN-C.C and MAIN-E.C to print

* messages of importance to the user of the program.

*/

char *CompressionName = "Sound sample companding";

char *Usage =

"infile outfile [n]\n\n n optionally sets the bits per sample\n\n";

#ifdef __STDC__

long get_file_length( FILE *file );

#else

long get_file_length();

#endif

#ifndef SEEK_END

#define SEEK_END 2

#endif

#ifndef SEEK_SET

#define SEEK_SET 0

#endif

/*

* The compression routine runs very quickly, since all it does is

* perform a table lookup on each input byte.  The first part of the

* routine builds that table.  After that it is just a matter  of

* reading bytes in and writing out the compressed value.

*

* Unlike all of the other compression routines in the book, this

* routine does not have a special END_OF_STREAM code.  With as few

* as four or eight codes, they seem too precious to give one up.

* Instead, the file length is written out at the very start of the

* compressed file, along with the number of bits used to encode the

* data.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

  int compress[ 256 ];

  int steps;

  int bits;

  int value;

  int i;

  int j;

  int c;

/*

* The first section of code determines the number of bits to use

* for output codes, then writes it to the compressed file.  The

* length of the input file is also written out.

*/

  if ( argc-- > 0 )

    bits = atoi( *argv );

  else

    bits = 4;

  printf( "Compressing using %d bits per sample...\n", bits );

  steps = ( 1 << ( bits - 1 ) );

  OutputBits( output, (unsigned long) bits, 8 );

  OutputBits( output, (unsigned long) get_file_length( input), 32 );

/*

* The compression table is built here.  Each input code maps to

* a single output code.  There are "steps" codes to be used in

* the output space.  This builds an exponential output function.

*/

  for ( i = steps ; i > 0; i-- ) {

    value = 128.0 * ( pow( 2.0, (double) i / steps ) - 1.0 ) + 0.5;

    for ( j = value ; j > 0 ; j-- ) {

      compress[ j + 127 ] = i + steps - 1;

      compress[ 128 - j ] = steps - i;

    }

  }

/*

* The actual compression takes place here.

*/

  while ( ( c = getc( input ) ) != EOF )

    OutputBits( output, (unsigned long) compress[ c ], bits ); }

/*

* The expansion routine gets the number of bits per code from the

* compressed file, then builds an expansion table.  Each of the

* "steps" codes expands to a unique eight-bit code that lies on

* the exponential encoding curve.

*/

void ExpandFile( input, output, argc, argv )

BIT_FILE *input:

FILE *output;

int argc;

char *argv[];

{

  int steps;

  int bits;

  int value;

  int last_value;

  int i;

  int c;

  long count;

  int expand[ 256 ];

/*

* First this routine reads in the number of bits, then it builds

* the expansion table.  Once the table is built, expanding the file

* is simply a matter of performing a table lookup on each code.

*/

  bits = (int) InputBits( input, 8 );

  printf( "Expanding using %d bits per sample...\n", BITS );

  steps = ( 1 << (bits - 1 ) );

  last_value = 0;

  for ( i = 1; i <= steps; i++ ) {

    value = 128.0 * ( pow(2.0, (double) i / steps ) - 1.0 ) + 0.5;

    expand[ steps + i - 1 ] = 128 + ( value + last_value ) / 2;

    expand[ steps - i ] = 127 - ( value + last_value ) / 2;

    last_value = value;

  }

/*

* The actual file size is stored at the start of the compressed

* file. It is read in to determine how many codes need to be

* expanded.  Once that is done, expansion takes place rapidly.

*/

  for ( count = InputBits( input, 32 ); count > 0 ; count-- ) {

    c = (int) InputBits( input, bits );

    putc(expand[ c ], output );

  }

  while ( argc-- > 0 )

    printf( "Unused argument: %s\n", *argv++ ) ;

}

/*

* This utility routine determines the size of the input file.

*/

long get_file_length( file )

FILE *file;

{

  long marker;

  long eof_ftell;

  marker = ftell( file );

  fseek( file, OL, SEEK_END );

  eof_ftell = ftell( file );

  fseek( file, marker, SEEK_SET );

  return( eof_ftell - marker );

}

/************************ End of COMPAND.C ************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

As was mentioned before, lossy compression is frequently used as a front end to a lossless compressor. In the case of COMPAND.C, this is a very effective strategy. After the files have been processed, far fewer codes are present in the output file, which makes string matching more likely, such as that used by LZSS compressors. By compressing a file by 50 percent using the companding strategy, then by applying LZSS compression, we can frequently achieve upwards of 90 percent compression on sound samples.

Other Techniques

This chapter covered some of the simpler techniques used to compress sound samples. As the level of processing power available for processing goes up, far more complicated algorithms are being applied.

One of the most common compression algorithms in use today has been sanctioned by the CCITT in their recommendation G.721. The G.721 algorithm uses Adaptive Differential Pulse Code Modulation (ADPCM) to encode digital signals at 16Kbps or 32Kbps. This algorithm is commonly performed by digital signal processors, and it is generally applied to data that has already been digitized using standard codes.

The ADPCM algorithm combines two techniques. The first, delta pulse code modulation, encodes sound signals by measuring the difference between two consecutive samples, not their absolute values. The quantization level adapts itself to the changing input signals, so the size of the encoded voltage changes as the input signal changes. When the signal moves from a high voltage to a low voltage at a step rate, the encoded step value will be high. If a quiet input signal is being encoded, the step value will be low.

This becomes complicated because the ADPCM algorithm requires that the transmitter predict in advance where the input signal is headed. If this prediction is not made accurately, it is not possible to make good judgments about the size of the step defined by each code. The process of predicting where a waveform is headed occupies most of the processor's time.

To compress sound samples to even lower bit rates, even more sophisticated techniques, such as Linear Predictive Coding (LPC), are used. Human speech can be compressed and replayed in a recognizable state with rates as low as 2,400 bits per second using LPC.

LPC attempts to compress human speech by modeling the vocal tract that produces the speech. Instead of storing thousands of samples per second. LPC instead attempts to determine just a few parameters that model the process used to create the sound. The success or failure of LPC hinges on the ability of the compressor to execute millions of instructions per second during the compression process.

Processes such as LPC and ADPCM represent the type of algorithms that will be used more and more frequently on the desktop. Unfortunately, the complexity of these algorithms are far beyond the scope of a sample program in this chapter.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 11
Lossy Graphics Compression

Desktop computers communicate information primarily via their screens, so graphics are a major concern for computer programmers and designers. Programmers spend enormous amounts of time and effort trying to accommodate the proliferation of the Graphical User Interface (GUI). Millions of man hours and billions of dollars worth of equipment are being allocated just to make improvements in the way programs display data.

The money being spent on computers equipped to perform properly under GUIs such as Microsoft Windows or Motif has created a vast array of computers capable of displaying complex graphical images, with resolution approaching that of conventional media, such as television or magazines. In turn, this capability has spawned new software designed to exploit these capabilities.

Programs using complex graphics are showing up in virtually every area of computing applications: games, education, desktop publishing, and graphical design, just to mention a few. These programs have one factor in common. The images they use consume prodigious amounts of disk storage.

In the IBM world, for example, the VGA display is probably the current lowest common denominator for high-quality color graphics. The VGA can display 256 simultaneous colors selected from a palette of 262,144 colors. This lets the VGA display continuous tone images, such as color photographs, with a reasonable amount of fidelity.

The problem with using images of photographic quality is the amount of storage required to use them in a program. For the previously mentioned VGA, a 256-color screen image has 200 rows of 320 pixels, each consuming a single byte of storage. This means that a single screen image consumes a minimum of 64K! It isn't hard to imagine applications that would require literally hundreds of these images to be accessed. An on-line catalog for a retail sales outlet, for example, could easily have 1,000 images stored for immediate access. The problem is that 1,000 images of this quality would consume 64MB of storage. And this is not an unreasonable number: We are just beginning to see game programs being distributed on CD-ROM, due to the enormous amounts of storage required by screen images.

This chapter discusses the use of lossy compression techniques to achieve very high levels of data compression of continuous tone graphical images, such as digitized images of photographs.

Enter Compression

There has been an explosion of research into graphics storage during the last decade, and many interesting results have been published. In the late 1970s and early 1980s, most graphics compression concentrated on using conventional lossless techniques. Popular PC file formats now use techniques discussed earlier in the book to achieve savings ranging from 10 to 90 percent on graphics images. Well-known formats using compression include the PCX, GIF, and BMP standards.

As the use of stored graphical images increased, file formats such as PCX began to appear inadequate. Cutting file sizes in half certainly is a worthwhile thing to do, but developers and users Ywere filling their storage space up so fast that system requirements for multimedia systems appeared prohibitively expensive. Worse yet, the promise of full motion video on the desktop was simply not possible until some method was developed for radically reducing storage needs. Clearly, compression capabilities needed to improve, perhaps by orders of magnitude.

Statistical and Dictionary Compression Methods

Conventional programs and data on computers respond well to compression based on exploiting statistical variations in the frequency of both individual symbols and strings of symbols or phrases. Dictionary-based systems are in fact just statistical programs in disguise. Unfortunately, these types of compression don't tend to do very well on continuous tone images.

The primary problem these programs have stems from the fact that pixels in photographic images tend to be well spread out over their entire range. If the colors in an image are plotted as a histogram based on frequency, the histogram is not as "spiky" as we would like for statistical compression to succeed. In fact, over the long run, histograms for live images from sources such as television tend to be flat. This means that each pixel code has approximately the same chance of appearing as any other, negating any opportunity for exploiting entropy differences.

Dictionary-based compression programs run into similar problems. Images based on scanned photographs just don't have the right kind of data characteristics to create multiple occurrences of the same phrase. In a rasterized image, for example, a vertical structure such as the side of a house may give similar strings in many consecutive rows of a picture. Unfortunately, because of the vagaries of the real world, the same feature in each row will tend to be slightly different from the one before. Out of a string of twenty pixels, one or two will vary by a single step from the scans before and after. And while these differences are small enough that they are either undetectable or meaningless to the human eye, they throw a monkey wrench into the works of dictionary-based compression. Strings have to match exactly for this compression method to work. Because of minute variations, the length of matching strings tends to be small, which limits the effectiveness of compression.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Lossy Compression

Just like audio data (discussed in Chapter 10), graphical images have an advantage over conventional computer data files: They can be slightly modified during the compression/expansion cycle without affecting the perceived quality on the part of the user. Minor changes in the exact shade of a pixel here and there can easily go completely unnoticed if the modifications are done carefully. Since the graphical images on a computer are generally scanned from real-world sources, they usually represent an already imperfect representation of a photograph or some other printed media. A lossy compression program that doesn't change the basic nature of the image ought to be feasible.

Given that lossy compression for graphical images is possible, how is it implemented? Researchers initially tried some of the same techniques that worked on speech, such as differential coding and adaptive coding. While these techniques helped compress graphics, they did not do as well as hoped. One reason for this lies in the fundamental difference between audio and video data.

Audio data sampled using conventional formats tends to be very repetitive. Sounds, including speech, are made of sine waves that repeat for seconds at a time. Though the input stream at the DAC on a computer may consist of dozens of different frequencies added together, sine waves generally combine to produce repetitive waveforms.

The repetitive nature of audio data naturally lends itself to compression. Techniques such as linear predictive coding and adaptive differential pulse code modulation take advantage of this fact to compress audio streams anywhere from 50 to 95 percent.

When research began on compression of graphics, attempts were made to apply similar techniques to digitized images, with some success. Initially, researchers worked on the compression of streams of rasterized data, such as would be displayed on a television set.

When graphics data is rasterized, it is displayed as a sequential stream of pixels. One row at time is displayed on a screen, working from left to right, then top to bottom. Thus, a thin slice of the picture is painted as each row is completed, until the complete screen is filled. When digitized, pixels can range in size from a single bit to as many as twenty-four bits. Desktop graphics today frequently uses eight bits to define a single pixel.

Differential Modulation

Differential modulation depends on the notion that analog data tends to vary in "smooth" patterns, with radical jumps in the magnitude of a signal being the exception, not the rule. In audio data, this is true as long as the sampling rate of the signal is somewhat higher than its maximum frequency component.

Differential modulation of an audio signal takes advantage of this fact by encoding each sample as the difference from its predecessor. If audio samples are eight bits each, for example, a differential encoding system might encode the difference between samples in four bits, compressing the input data by 50 percent. The lossy part of the compression scheme arises from the fact that an exact difference can't always be encoded using the standard differential method. The signal may be rising faster than the encoding permits, or the encoding may be too coarse to accommodate a small difference. The lossy aspect of differential encoding can be managed well enough to produce a good signal.

Differential modulation has more of a problem when compressing graphical data. For one thing, pixels in a graphical image can't be reliably depended on to vary upward or downward in smooth increments. Sharp dividing lines between different components of an image are the rule. This means that a system that relies on differential encoding needs to accommodate both small and large differences between samples, limiting its effectiveness. Many images will feature long stretches of data where pixels have little or no difference between one another, and these will compress well; however, others will feature many abrupt changes, and these may not compress at all.

In general, differential encoding of graphical images doesn't seem to produce compression that is significantly greater than that of the best lossless algorithms. It certainly doesn't yield the order of magnitude of improvement in compression that is needed.

Adaptive Coding

Adaptive coding (which is often used with differential coding) relies on predicting some information about upcoming pixels based on previously seen pixels. If the last ten pixels in a grey-scale photograph all had values between forty-five and fifty, for example, an adaptive compression system might predict with high probability that the next pixel would be in the same range. An entropy-based encoding scheme, such as Huffman or arithmetic coding, could then assign probabilities to various incoming codes. An alternative would be to use a companding scale, with the finest granularity assigned to the range nearest the predicted guess. Assuming that the prediction method enabled you to make an educated guess about the probabilities of the pixels, you should achieve some data compression.

Most adaptive schemes rely on using just a few of the surrounding pixels as part of the calculation for probabilities of the upcoming pixel. In Figure 11.1, the pixel to be encoded is shown at position 0,0. Pixels that are most commonly used when calculating probabilities are shown at positions A, B, C, and D. Predictions about the upcoming value of the target pixel can be made based on any of several predicting equations:

Figure 11.1 Pixels used for adaptive coding.

Figure 11.2 Pixel predictors.

These techniques use previous data to calculate the most likely value of the target pixel, and they adjust the coding scheme accordingly. While these calculations produce good results, once again they are certainly not the order of magnitude needed to perform effective compression.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

A Standard That Works: JPEG

In the late 1970s and early 1980s, research began on new types of image compression that promised to greatly outperform the more conventional compression techniques discussed earlier. By the late 1980s, this work was beginning to find commercial applications for image processing on desktop systems, mostly in the form of add-on coprocessor cards for UNIX and Macintosh workstations. These cards were able to perform lossy compression on images at ratios of as much as 95 percent without visible degradation of the image quality.

Other forces at this time combined to start development of an international standard that would encompass these new varieties of compression. There are clear advantages to all parties if standards allowed for easy interchange of graphical formats. The main concern regarding early standardization is the possibility that it would constrain further innovation. The two standardization groups involved, the CCITT and the ISO, worked actively to get input from both industry and academic groups concerned with image compression, and they seem to have avoided the potentially negative consequences of their actions.

The standards group created by these two organizations is the Joint Photographic Experts Group (JPEG). The JPEG standard was developed over the curse of several years, and is now firmly entrenched as the leading format for lossy graphics compression.

The JPEG specification consists of several parts, including a specification for both lossless and lossy encoding. The lossless compression uses the predictive/adaptive model described earlier in this chapter, with a Huffman code output stage, which produces good compression of images without the loss of any resolution.

The most interesting part of the JPEG specification is its work on a lossy compression technique. The rest of this chapter discusses the basics of this technique, with sample code to illustrate its components.

JPEG Compression

The JPEG lossy compression algorithm operates in three successive stages, shown in Figure 11.3.

Figure 11.3 JPEG lossy compression.

These three steps combine to form a powerful compressor, capable of compressing continuous tone images to less than 10 percent of their original size, while losing little, if any, of their original fidelity.

The Discrete Cosine Transform

The key to the compression process discussed here is a mathematical transformation known as the Discrete Cosine Transform (DCT). The DCT is in a class of mathematical operations that includes the well-known Fast Fourier Transform (FFT), as well as many others. The basic operation performed by these transforms is to take a signal and transform it from one type of representation to another.

This transformation is done frequently when analyzing digital audio samples using the FFT. When we collect a set of sample points from an incoming audio signal, we end up with the representation of a signal in the time domain. That is, we have a collection of points that show what the voltage level was for the input signal at each point in time. The FFT transforms the set of sample points into a set of frequency values that describes exactly the same signal.

Figure 11.4 shows the classic time domain representation of an analog signal. This particular signal is composed of three different sine waves added together to form a single, slightly more complicated waveform. Each of the sample points represents the relative voltage or amplitude of the signal at a specific point in time.

Figure 11.4 The classic time domain representation of an analog signal.

Figure 11.5 shows what happens to the same set of data points after FFT processing. In the time-domain representation of the signal, each of the points on the X axis represents a different point in time, and each of the points on the Y axis represents a specific magnitude of the signal. After processing the data points with an FFT, the X axis no longer has the same meaning. Now, each point on the X axis represents a specific frequency, and the Y axis represents the magnitude of that frequency.

Figure 11.5 Data points after FFT processing.

Given that interpretation of the output of the FFT, Figure 11.5 makes immediate sense. It says that the signal displayed in the earlier figure can also be represented as the sum of three different frequencies of what appears to be identical magnitude. Given this information, it should be just as easy to construct the signal as it would be with Figure 11.4.

Another important point to make about the this type of transformation function is that the function is reversible. In principle, the same set of points shown in Figure 11.5 can be processed through an inverse FFT function, and the points shown in Figure 11.4 should result. The two transformation cycles are essentially lossless, except for loss of precision resulting from rounding and truncation errors.

The DCT is closely related to the Fourier Transform, and produces a similar result. It takes a set of points from the spatial domain and transforms them into an identical representation in the frequency domain; however, we are going to introduce an additional complication in this particular instance. Instead of a two-dimensional signal plotted on an X and Y axis, the DCT will operate on a three-dimensional signal plotted on an X, Y, and Z axis.

In this case, the "signal" is a graphical image. The X and Y axes are the two dimensions of the screen. The amplitude of the "signal" in this case is simply the value of a pixel at a particular point on the screen. For the examples used in this chapter, that is an eight-bit value used to represent a grey-scale value. So a graphical image displayed on the screen can be thought of as a complex three-dimensional signal, with the value on the Z axis denoted by the color on the screen at a given point. This is the spatial representation of the signal.

The DCT can be used to convert spatial information into "frequency" or "spectral" information, with the X and Y axes representing frequencies of the signal in two different dimensions. And like the FFT, there is an Inverse DCT (IDCT) function that can convert the spectral representation of the signal back to a spatial one.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

DCT Specifics

The actual formula for the two-dimensional DCT is shown in Figure 11.6, with its partner, the IDCT, shown immediately below in Figure 11.7. The DCT is performed on an N x N square matrix of pixel values, and it yields an N x N square matrix of frequency coefficients. The formula looks somewhat intimidating at first glance, but it can be done with a relatively straightforward piece of code.

Figure 11.6 The Discrete Cosine Transform

Figure 11.7 The Inverse DCT

To write code to implement this function, it first becomes clear that simple table lookups can replace many terms of the equation. The two cosine terms that have to be multiplied together only need to be calculated once at the beginning for the program, and they can be stored for later use. Likewise, the C(x) terms that fall outside the summation loops can also be replaced with table lookups. Once that is done, code to compute the N-by-N portion of a display looks somewhat like that shown below:

  for ( i = 0 ; i < N ; i++ )

   for ( j = 0 ; j < N ; j++ ) {

    temp = 0.0;

    for ( x = 0 ; x < N ; x++ )

     for ( y = 0 ; y < N ; y++ ) {

      temp += Cosines[ x ][ i ] *

          Cosines[ y ][ j ] *

          pixel[ x ][ y ];

     }

    temp *= sqrt( 2 * N ) * Coefficients[ i ][ h ];

    DCT[ i ][ j ] = INT_ROUND( temp );

   }

Why Bother?

While this code fragment looks as though it may be somewhat interesting to a mathematician, why anyone would want to use it on a graphical image is not immediately obvious. After we transform the pixels to frequency coefficients, we still have just as many points as before. It doesn't seem as if that is a particularly good way to go about compressing data. It would be much more impressive if the DCT took an N-by-N matrix of data and transformed it to an N/2 by N/2 matrix.

However, Figure 11.5 provides a clue as to what the JPEG committee sees in this algorithm. Figure 11.5 shows that the spectral representation of the audio waveform takes all the information needed to describe the waveform and packs it into the three non-zero points on the graph. So in principle we could describe the 512 points that make up the input waveform with just three points of frequency data.

The DCT accomplishes something similar when it transforms data. In the N-by-N matrix, all the elements in row 0 have a frequency component of zero in one direction of the signal. All the elements in column 0 have a frequency component of zero in the other direction. As the rows and columns move away from origin, the coefficients in the transformed DCT matrix begin to represent higher frequencies, with the highest frequencies found at position N-1 of the matrix.

This is significant because most graphical images on our computer screens are composed of low-frequency information. As it turns out, the components found in row and column 0 (the DC components) carry more useful information about the image than the higher-frequency components. As we move farther away from the DC components in the picture, we find that the coefficients not only tend to have lower values, but they become far less important for describing the picture.

So the DCT transformation identifies pieces of information in the signal that can be effectively "thrown away" without seriously compromising the quality of the image. It is hard to imagine how we would do this with a picture that hadn't been transformed. With the image still described in spatial terms, using pixels, a program would have a difficult time figuring out which pixels are important to the overall look of the picture and which aren't.

After defining the DCT as the transformation to be used, the JPEG committee then tackled the truly difficult work: how to "throw away" the insignificant portions of the picture. Details on that come later in this chapter.

Implementing the DCT

One of the first things that shows up when examining the DCT algorithm is that the calculation time required for each element in the DCT is heavily dependent on the size of the matrix. Since a doubly nested loop is used, the number of calculations is O(N squared): as N goes up, the amount of time required to process each element in the DCT output array will go up dramatically.

One of the consequences of this is that it is virtually impossible to perform a DCT on an entire image. The amount of calculation needed to perform a DCT transformation on even a 256-by-256 grey-scale block is prohibitively large. To get around this, DCT implementations typically break the image down into smaller, more manageable blocks. The JPEG group selected an 8-by-8 block for the size of their DCT calculation.

While increasing the size of the DCT block would probably give better compression, it doesn't take long to reach a point of diminishing returns. Research shows that the connections between pixels tend to diminish quickly, such that pixels even fifteen or twenty positions away are of very little use as predictors. This means that a DCT block of 64-by-64 might not compress much better than if we broke it down into four 16-by-16 blocks. And to make matters worse, the computation time would be much longer.

While there is probably a good argument for using 16-by-16 blocks as the basis for DCT computations, the JPEG committee elected to stick with 8-by-8. Much of this was motivated by a desire to allow for practical implementations that could be built using today's technology. This type of compression is referred to as "block coding."


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Matrix Multiplication

The definition of the DCT shown above is a relatively straightforward, doubly nested loop. The inner element of the loop gets executed N*N times for every DCT element that is calculated. The inner line of the loop has two multiplication operations and a single addition operation.

A considerably more efficient form of the DCT can be calculated using matrix operations. To perform this operation, we first create an N-by-N matrix known as the Cosine Transform matrix, C. This matrix is defined by the equation shown in Figure 11.8.

Figure 11.8 The Cosine Tranform Matrix.

Once the Cosine Transform matrix has been built, we transpose it by rotating it around the main diagonal. This matrix is referred to in code as Ct, the Transposed Cosine Transform matrix. Building this matrix is done only once during program initialization. Both matrices can be built at the same time with a relatively short loop, shown below:

 for ( j = 0 ; j < N ; j++ ) {

    C[ 0 ][ j ] = 1.0 / sqrt( N );

    Ct[ j ][ 0 ] = C[ 0 ][ j ];

 }

 for ( i = 1 ; i < N ; i++ ) {

   for ( j = 0; j < N ; j ++ ) {

      C[ i ][ j ] = sqrt( 2.0 / N ) *

             cos( ( 2 * j + 1 ) * i * pi

              / ( 2.0 * N ) );

      Ct[ j ][ i ] = C[ i ][ j ];

   }

 }

Once these two matrices have been built, we can take advantage of the alternative definition of the DCT function:

DCT = C * Pixels * Ct

In this particular equation, the '*' operator refers to matrix multiplication, not normal arithmetic multiplication. Each factor in the equation is an N-by-N matrix. In the case of the JPEG algorithm and the program used to illustrate this chapter, the matrices are 8 by 8.

When multiplying two square matrices together, the arithmetic cost of each element of the output matrix will be N multiplication operations and N addition operations. Since we perform two matrix multiplications to create the DCT matrix, each element in the transformed DCT matrix was created at the cost of 2N multiplications and additions, a considerable improvement over the nested loop definition of the DCT used earlier.

/* MatrixMultiply( temp, input, Ct ); */

  for ( i = 0 ; i < N ; i++ ) {

   for ( j = 0 ; j < N ; j++ ) {

    temp[ i ][ j ] = 0.0;

    for ( k = 0 ; k < N ; k++ )

     temp[ i ][ j ] += ( pixel[ i ][ k ] ) * Ct[ k ][ j ];

   }

  }

/* MatrixMultiply( output, C, temp ); */

  for ( i = 0 ; i < N ; i++ ) {

   for ( j = 0 ; j < N ; j++ ) {

    temp1 = 0.0;

    for ( k = 0 ; k < N ; k++ )

     temp1 += C[ i ][ k ] * temp[ k ][ j ];

    DCT[ i ][ j ] = temp1;

   }

  }

A sample piece of code that implements the DCT via matrix arithmetic is shown above. Note that the code is essentially nothing more than a set of two triply nested loops. The first set of loops multiplies the transposed Cosine Transform Matrix by the input set of pixels, creating a temporary matrix. The temporary matrix is then multiplied by the Cosine Transform matrix, which results in the output, the DCT matrix.

Continued Improvements

The versions of the DCT presented here perform the same operations as those used in commercial implementations, but without several more optimization steps needed to produce JPEG compressors that operate in something approaching real time.

One improvement that can be made to the algorithm is to develop versions of the algorithm that only use integer arithmetic. To achieve the accuracy needed for faithful reproduction, the versions to the program tested in this chapter all stick with reliable floating point math. It is possible, however, to develop versions of the DCT that use scaled integer math, which is considerably faster on most platforms.

Since the DCT is related to the Discrete Fourier Transform, it shouldn't be surprising that many of the techniques used to speed up the family of Fourier Transforms can also be applied to the DCT. In fact, people all over the world are working full time on applying Digital Signal Processing techniques to the DCT. Every cycle shaved off the time taken to perform the transform can be worth a small fortune, so there is good incentive for these research efforts.

Output of the DCT

Figure 11.9 shows a representative input block from a grey-scale image. As can be seen, the input consists of an 8-by-8 matrix of pixel values which are somewhat randomly spread around the 140 to 175 range. These integer values are fed to the DCT algorithm, creating the output matrix shown below it.

Figure 11.9 The DCT on a Block of Pixels from CHEETAH.GS

The output matrix shows the spectral compression characteristic the DCT is supposed to create. The "DC coefficient" is at position 0,0 in the upper left-hand corner of the matrix. This value represents an average of the overall magnitude of the input matrix, since it represents the DC component in both the X and the Y axis. Note that the DC coefficient is almost an order of magnitude greater than any of the other values in the DCT matrix. In addition, there is a general trend in the DCT matrix. As the elements move farther and farther from the DC coefficient, they tend to become lower and lower in magnitude.

This means that by performing the DCT on the input data, we have concentrated the representation of the image in the upper left coefficients of the output matrix, with the lower right coefficients of the DCT matrix containing less useful information. The next section discusses how this can help compress data.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Quantization

Figure 11.3 shows the JPEG compression process as a three-step procedure, the first step being a DCT transformation. DCT is a lossless transformation that doesn't actually perform compression. It prepares for the "lossy," or quantization, stage of the process.

The DCT output matrix takes more space to store than the original matrix of pixels. The input to the DCT function consists of eight-bit pixel values, but the values that come out can range from a low of -1,024 to a high of 1,023, occupying eleven bits. Something drastic needs to happen before the DCT matrix can take up less space.

The "drastic" action used to reduce the number of bits required for storage of the DCT matrix is referred to as "Quantization." Quantization is simply the process of reducing the number of bits needed to store an integer value by reducing the precision of the integer. Once a DCT image has been compressed, we can generally reduce the precision of the coefficients more and more as we move away from the DC coefficient at the origin. The farther away are from 0,0, the less the element contributes to the graphical image, so the less we care about maintaining rigorous precision in its value.

The JPEG algorithm implements Quantization using a Quantization matrix. For every element position in the DCT matrix, a corresponding value in the quantization matrix gives a quantum value. The quantum value indicates what the step size is going to be for that element in the compressed rendition of the picture, with values ranging from one to 255.

The elements that matter most to the picture will be encoded with a small step size, size 1 offering the most precision. Values can become higher as we move away from the origin. The actual formula for quantization is quite simple:

                         DCT(i,j)

Quantized Value(i,j) = -------------- Rounded to nearest integer

                       Quantum(i,j)

From the formula, it becomes clear that quantization values above twenty-five or perhaps fifty assure that virtually all higher-frequency components will be rounded down to zero. Only if the high-frequency coefficients get up to unusually large values will they be encoded as non-zero values.

During decoding, the dequantization formula operates in reverse:

DCT(i,j) = Quantized Value(i,j) * Quantum(i,j)

Once again, from this we can see that when you use large quantum values, you run the risk of generating large errors in the DCT output during dequantization. Fortunately, errors generated in the high-frequency components during dequantization normally don't have a serious effect on picture quality.

Selecting a Quantization Matrix

Clearly an enormous number of schemes could be used to define values in the quantization matrix. At least two experimental approaches can test different quantization schemes. One measures the mathematical error found between an input and output image after it has been decompressed, trying to determine an acceptable level of error. A second approach tries to judge the effect of decompression on the human eye, which may not always correspond exactly with mathematical differences in error levels.

Since the quantization matrix can obviously be defined a runtime when compression takes place, JPEG allows for the use of any quantization matrix; however, the ISO has developed a standard set of quantization values supplied for use by implementers of JPEG code. These tables are based on extensive testing by members of the JPEG committee, and they provide a good baseline for established levels of compression.

One nice feature about selecting quantization matrices at runtime is that it is quite simple to "dial in" a picture quality value when compressing graphics using the JPEG algorithm. By choosing extraordinarily high step sizes for most DCT coefficients, we get excellent compression ratios and poor picture quality. By choosing cautiously low step sizes, compression ratios will begin to slip to not so impressive levels, but picture quality should be excellent. This allows for a great deal of flexibility for the user of JPEG code, choosing picture quality based on both imaging requirements and storage capacity.

The quantization tables used in the test code supplied with this program are created using a very simple algorithm. To determine the value of the quantum step sizes, the user inputs a single "quality factor" which should range from one to about twenty-five. Values larger than twenty-five would work, but picture quality has degraded far enough at quality level 25 to make going any farther an exercise in futility.

 for ( i = 0 ; i < N ; i++ )

   for ( j = 0 ; j < N ; j++ )

     Quantum[ i ][ j ] = 1 + ( ( 1 + i + j ) * quality );

The quality level sets the difference between adjoining bands of the same quantization level. These bands are oriented on diagonal lines across the matrix, so quantization levels of the same value are all roughly the same distance from the origin. An example of what the quantization matrix looks like with a quality factor of two follows:

Figure 11.10 The matrix at quality factor 2.

As a result of this configuration, the DCT coefficient at 7,7 would have to reach a value of sixteen to be encoded as a value other than zero. This sets the threshold for the value of an element before it is going to contribute any meaningful information to the picture. Any contribution under the value of this threshold is simply thrown out. This is the exact point in the algorithm where the "lossy" effect takes place. The first DCT step is lossless except for minor mathematical precision loss. And the step following quantization is a lossless encoding step. So this is the only place where we get a chance to actually discard data.

Figure 11.11 shows the effects of quantization on a DCT matrix. The quantization matrix shown in the previous figure was applied to this block of DCT, which comes from the first block of test file CHEETAH.GS. The quantization/dequantization cycle has readily apparent effects. The high-frequency portions of the matrix have for the most part been truncated down to zero, eliminating their effect on the decompressed image. The coefficients in the matrix that are close to the DC coefficient may have been modified, but only by small amounts.

Figure 11.11 The effects of quantization.

The interesting thing is that while we appear to be making wholesale changes to the saved image, quality factor 2 makes only minor changes that are barely noticeable. Yet the clearing of so many of the coefficients allows the image to be compressed by 60 percent, even in the very simple compression program used in this chapter.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Coding

The final step in the JPEG process is coding the quantized images. The JPEG coding phase combines three different steps to compress the image. The first changes the DC coefficient at 0,0 from an absolute value to a relative value. Since adjacent blocks in an image exhibit a high degree of correlation, coding the DC element as the difference from the previous DC element typically produces a very small number. Next, the coefficients of the image are arranged in the "zig-zag sequence." Then they are encoded using two different mechanisms. The first is run-length encoding of zero values. The second is what JPEG calls "Entropy Coding." This involves sending out the coefficient codes, using either Huffman codes or arithmetic coding depending on the choice of the implementer.

The Zig-Zag Sequence

One reason the JPEG algorithm compresses so effectively is that a large number of coefficients in the DCT image are truncated to zero values during the coefficient quantization stage. So many values are set to zero that the JPEG committee elected to handle zero values differently from other coefficient values.

Instead of relying on Huffman or arithmetic coding to compress the zero values, they are coded using a Run-Length Encoding (RLE) algorithm. A simple code is developed that gives a count of consecutive zero values in the image. Since over half of the coefficients are quantized to zero in many images, this gives an opportunity for outstanding compression.

One way to increase the length of runs is to reorder the coefficients in the zig-zag sequence. Instead of compressing the coefficient in row-major order, as a programmer would probably do, the JPEG algorithm moves through the block along diagonal paths, selecting what should be the highest value elements first, and working its way toward the values likely to be lowest.

The actual path of the zig-zag sequence is shown in Figure 11.12. In the code used in this chapter, the diagonal sequences of quantization steps follow exactly the same lines, so the zig-zag sequence should be optimal for our purposes.

Figure 11.12 The path of the zig-zag sequence.

Implementing the zig-zag sequence in C is probably done best using a simple lookup table. In our sample code for this chapter, the sequence is coded as part of a structure that can be accessed sequentially to determine which row and column to encode:

struct zigzag {

 int row;

 int col;

} ZigZag[ N * N ] =

{

  {0, 0},

  {0, 1}, {1, 0},

  {2, 0}, {1, 1}, {0, 2},

  {0, 3}, {1, 2}, {2, 1}, {3, 0},

  {4, 0}, {3, 1}, {2, 2}, {1, 3}, {0, 4},

  {0, 5}, {1, 4}, {2, 3}, {3, 2}, {4, 1}, {5, 0},

  {6, 0}, {5, 1}, {4, 2}, {3, 3}, {2, 4}, {1, 5}, {0, 6},

  {0, 7}, {1, 6}, {2, 5}, {3, 4}, {4, 3}, {5, 2}, {6, 1}, {7, 0},

  {7, 1}, {6, 2}, {5, 3}, {4, 4}, {3, 5}, {2, 6}, {1, 7},

  {2, 7}, {3, 6}, {4, 5}, {5, 4}, {6, 3}, {7, 2},

  {7, 3}, {6, 4}, {5, 5}, {4, 6}, {3, 7},

  {4, 7}, {5, 6}, {6, 5}, {7, 4},

  {7, 5}, {6, 6}, {5, 7},

  {6, 7}, {7, 6},

  {7, 7}

};

The C code that sends each of the DCT results to the compressor follows. Note that instead of directly looking up each result, we instead determine which row and column to use next by looking it up in the zig-zag structure. We then encode the element determined by the row and column from the zig-zag structure.

 for ( i = 0 ; i < ( N * N ) ; i ++ ) {

  row = ZigZag[ i ].row;

  col = ZigZag[ i ].col;

  result = DCT[ row ][ col ] / Quantum[ row ][ col ];

  OutputCode( output_file, ROUND( result ) );

 }

Entropy Encoding

After converting the DC element to a difference from the last block, then reordering the DCT block in the zig-zag sequence, the JPEG algorithm outputs the elements using an "entropy encoding" mechanism. The output has RLE built into it as an integral part of the coding mechanism. Basically, the output of the entropy encoder consists of a sequence of three tokens, repeated until the block is complete. The three tokens look like this:

•Run Length: | The number of consecutive zeros that preceded the current element in the DCT output matrix.
•Bit Count: | The number of bits to follow in the amplitude number.
•Amplitude: | The amplitude of the DCT coefficient.

The coding sequence used in this chapter's test program is a combination of Run Length Encoding and variable-length integer coding. The run-length and bit-count values are combined to form a code that is output. The bit count refers to the number of bits used to encode the amplitude as a variable-length integer.

The variable-length integer coding scheme takes advantage of the fact that the DCT output should consist of mostly smaller numbers, which we want to encode with smaller numbers of bits. The bit counts and the amplitudes which the encode follow.

Bit Count | Amplitudes
1 | -1, 1
2 | -3 to -2, 2 to 3
3 | 7 to -4, 4 to 7
4 | -15 to -8, 8 to 15
5 | -31 to -16, 16 to 31
6 | -63 to -32, 32 to 64
7 | -127 to -64, 64 to 127
8 | -255 to -128, 128 to 255
9 | -511 to -256, 256 to 511
10 | -1023 to -512, 512 to 1023

Note that each bit count encodes a symmetrical set of high and low values. The values skipped over in the middle will be encoded with a lower bit count from one in the table.

While this form of variable-bit coding is not quite as efficient as Huffman coding, it works fairly well, particularly if the data performs as expected, which means smaller values dominate and larger values are rare.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

What About Color?

The sample programs in this chapter and most of the text have talked about how to compress images that have only one color component, usually a grey scale. This leaves the question of what to do with color images.

Color images are generally composed of three components, such as the red, green, and blue of RGB, or the luminance and chrominance of YUV. In these cases, JPEG treats the image as if it were actually three separate images. An RGB image would first have its red component compressed, then its green, then its blue. This is essentially just more of the same.

The Sample Program

The sample program used to demonstrate DCT compression in this chapter is in the C source file DCT.C. It scan be compiled and linked with the standard support source files, BITIO.C, ERRHAND.C, and either MAIN-C.C for compression or MAIN-E.C for expansion.

The DCT compression program takes an additional parameter on the command line, the quality factor. A factor of zero through twenty-five can be selected, zero being the best quality and twenty-five being the lowest. As was discussed earlier in this chapter, the quality factor is used to initialize the quantum table with the step sizes for each DCT element.

The command syntax for the compression program is:

DCT-C input-file output-file [quality]

If no quality value is selected, it defaults to a value of three, which is an arbitrarily chosen constant. The quality factor is encoded in the compressed file, so the expansion program doesn't need that parameter on the command line. The syntax for expansion is:

DCT-E input-file output-file

The DCT sample program in this chapter is not an implentation of JPEG compression. It does closely duplicate the first and second stages of the algorithm, however, which are the DCT transformation of the input, followed by the quantization and zig-zag coding steps. The only significant difference from the JPEG algorithm at this point is that the DC coefficient at 0,0 is not encoded as a difference from the last coefficient.

The test program used here departs from being a JPEG implementation in the encoding phase after quantization is complete. DCT.C does not implement Huffman coding on the output, but it does implement a slightly different form of RLE and uses variable-length integer codes for output.

Input Format

Graphics files come in a plethora of formats. Decoding and understanding every format can become a bewildering problem, and the purpose of this book is not to be a treatise on file formats. Thus, the graphics examples used in this chapter are stored in the closest thing possible to a "non-format."

All of the graphics files used in this section are stored in a row-major order, so that all the pixels in each row are stored adjacent to one another. The top of the screen is stored first, with subsequent rows working their way down the screen. Each file is a 320 column by 200 row grey-scale image, with pixels having eight bits, ranging from zero to 255. The grey-scale files have a file suffix of "GS," which identifies them as "non-formatted" grey-scale files.

This format is particularly easy to display on IBM VGA displays, but should be easy to adapt to any system that can display 256 colors. A short program, GS.C, is included to display the files on IBM VGA displays. Since VGA displays can only handle sixty-four grey-scale colors, some of the resolution of the image is lost on display, but the effect is relatively insignificant to the human eye.

In addition to GS.C, which displays GS files on an IBM compatible VGA adaptor, there is a second display program called GSDIFF.C. This tests the differences between an original file and its reproduction after a compression/decompression cycle. First it gives a visual display of the differences between the two files. Then the root mean squared (rms) error is written to the screen. While the rms value is not the best way to assign a quality factor to a compression cycle, it does provide a good way to see how well compression is working.

The Code

A summarized version of the main compression module follows (a complete listing is at the end of the chapter). The main program first calls the initialization module, which sets up the quantization table and the cosine transform matrices. The quality parameter must be passed to this module to have it set up the quantization matrix properly.

The next step is to write out the quality factor to the output file. By outputting this information, we eliminate the possibility of inadvertently trying to decompress using the wrong quantization matrix. This would cause the output to be catastrophically in error if it happened.

   void CompressFile( FILE *input, BIT_FILE *output,

             int argc, char *argv[] )

 {

   int row;

   int col;

   int i;

   unsigned char *input_array[ N ];

   int output_array[ N ][ N ];

   int quality;

   quality = atoi( argv[ 0 ] );

   printf( "Using quality factor of %d\n", quality );

   Initialize( quality );

   OutputBits( output, quality, 8 );

   for ( row = 0 ; row <ROWS ; row += N ) {

    ReadPixelStrip( input, PixelStrip );

    for ( col = 0 ; col < COLS ; col += N ) {

     for ( i = 0 ; i < N ; i++ )

      input_array[ i ] = PixelStrip[ i ] + col;

     ForwardDCT( input_array, output_array );

     WriteDCTData( output, output_array );

   }

  }

  OutputCode( output, 1 );

 }

Finally, the main compression loop is entered. Since the data is stored a single row at a time, we need to read in a block of eight rows together before we can begin building 8-by-8 blocks to compress. This is accomplished in the routine called ReadPixelStrip. It reads an entire strip of pixels 8 rows deep and 320 columns wide.

The next part of the loop sets up the input_array. This actually gets passed to the DCT routine. It consists of a block of eight pointers into the pixel strip. When it is passed to the DCT routine, the input_array can be treated in the code as an 8-by-8 input matrix.

The DCT routine is then called. It is passed an 8-by-8 unsigned character matrix and returns an 8-by-8 integer matrix. The integer matrix is then passed to the WriteDCTData() routine for compression and to be written to the file.

The final step in the program is to call the OutputCode() routine one last time with a dummy non-zero value. The OutputCode() routine tracks consecutive zeros for the run-length encoding portion of the program. If the file ends with several consecutive zeros, they many need to be flushed before the program exits.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Initialization

DCT.C has single initialization routine that is called for both compression and expansion. It first sets up the quantization matrix, using the quality parameter passed to it. This uses the simple formula for defining step sizes discussed earlier.

Once the quantization matrix is set up, the next step is to set up the cosine transform matrix and the transposed cosine transform matrix These matrices are used by the forward DCT and the inverse DCT, so they can be set up in a common routine. Setting them up involves nothing more than a simple translation of the formula shown in Figure 11.8.

The final step in initialization is to initialize the run-length encoding counters used on input and output. These values are used when either outputting or inputting codes, and they track the number of consecutive zero codes that have output or will be input.

 void Initialize( int quality )

 {

  int i;

  int j;

  for ( i = 0 ; i < N ; i++ )

   for (j = 0 ; j < N ; j++ )

    Quantum[ i ][ j ] = 1 + ( ( 1 + i + j ) * quality );

  for ( j = 0 ; j < N ; j++ ) {

   C[ 0 ][ j ] = 1.0 / sqrt( N );

   Ct[ j ][ 0 ] = C[ 0 ][ j ];

  }

  for ( i = 1 ; i < N ; i++ ) {

   for ( j = 0 ; j < N ; j++ ) {

    C[ i ][ j ] = sqrt( 2.0 / N ) *

           cos( ( 2 * j + 1 ) * i * pi / ( 2.0 * N ) ) ;

    Ct[ j ][ i ] = C[ i ][ j ];

   }

  }

  OutputRunLength = 0;

  InputRunLength = 0;

 }

The Forward DCT Routine

Despite the seeming complexity of the DCT, it is accomplished in a very short routine. All it does is first perform a matrix multiplication of the input pixel data matrix by the transposed cosine transform matrix and store the result in a temporary N-by-N matrix. Then the temporary matrix is multiplied by the cosine transform matrix, and the result is stored in the output matrix, which is passed back to the caller.

Note here that all input pixel values are scaled before being multiplied by the transposed cosine transform matrix. After being scaled, they have a range of -128 to 127 instead of the zero to 255 range they had before. This is consistent with the way the JPEG algorithm handles input data.

  void ForwardDCT( input, output )

  unsigned char *input[ N ];

  int output[ N ][ N ];

  {

   double temp[ N ][ N ];

   double temp1;

   int i;

   int j;

   int k;

  /*MatrixMultiply( temp, input, Ct ); */

   for ( i = 0 ; i < N ; i++ ) {

    for ( j = 0 ; j < N ; j++ ) {

     temp[ i ][ j ] = 0.0;

     for ( k = 0 ; k < N ; k++ )

      temp[ i ][ j ] += ( input[ i ][ k ] - 128 ) *

               Ct[ k ] [ j ];

    }

   }

  /*MatrixMultiply( output, C, temp ); */

   for ( i = 0 ; i < N ; i++ ) {

    for ( j = 0 ; j < N ; j++ ) {

     temp1 = 0.0;

     for ( k = 0 ; k < N ; k++ )

      temp1 += C[ i ][ k ] * temp[ k ][ j ];

     output[ i ][ j ] = ROUND( temp1 );

   }

  }

 }

Another point to observe is that we are dealing with several different data types here and a certain amount of care needs to be exercised so as not to cause problems during conversions. The input data coming from the pixel strip is unsigned character converted during the matrix multiplication to integer, then multiplied by a double. The result is stored in a double temporary matrix. Finally, the last matrix multiplication produces double values, which are then rounded to integers for storage in the output matrix. If everything goes as planned, the integers should be in the range of -1,024 to 1,023, and they are ready for quantization.

WriteDCTData()

This routine is responsible for ordering the DCT result matrix into the zigzag pattern, then quantizing the data. Both of these just involve table lookups of values that have been stored either during initialization or at compile time. Then, the quantized value is rounded to the nearest integer and sent to the routine that outputs codes.

  void WriteDCTData( BIT_FILE *output_file, output_data[ N ][ N ] )

  {

   int i;

   int row;

   int col;

   double result;

   for ( i = 0 ; i < ( N * N ) ; i++ ) {

    row = ZigZag[ i ].row;

    col = ZigZag[ i ].col;

    result = output_data[ row ][ col ] / Quantum[ row ][ col ];

    OutputCode( output_file, ROUND( result ) );

   }

 }


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

OutputCode()

This routine is complicated by the fact that it has to handle quite a few different situations in the output data. In general, this routine puts out two numbers every time it is called. The first number is the number of bits used in the output word to follow. The second number is the actual amplitude of the output, encoded using a variable-length word, as in the JPEG algorithm.

The number of bits parameter that is output first can range anywhere from zero to ten. To encode this number using standard binary arithmetic would take four bits for every number. To achieve a minor amount of savings, this routine uses a simple prefix code to output the number of bits, which will result in a small net savings.

  void OutputCode( BIT_FILE *output_file, int code )

  {

   int top_of_range;

   int abs_code;

   int bit_count;

   if ( code == 0 ) {

    OutputRunLength++;

    return;

   }

   if ( OutputRunLength != 0 ) {

     while ( OutputRunLength > 0 ) {

      OutputBits( output_file, 0, 2 );

      if ( OutputRunLength <= 16 ) {

       OutputBits( output_file, OutputRunLength - 1, 4 );

       OutputRunLength = 0;

      } else {

       OutputBits( output_file, 15, 4 );

       OutputRunLength -= 16;

      }

    }

   }

   if ( code < 0 )

    abs_code = -code;

   else

     abs_code = code;

    top_of_range = 1;

    bit_count = 1;

    while ( abs_code > top_of_range ) {

     bit_count++;

     top_of_range = ( ( top_of_range + 1 ) * 2 ) - 1;

    }

    if ( bit_count < 3 )

     OutputBits( output_file, bit_count + 1, 3 ) ;

    else

     OutputBits( output_file, bit_count + 5, 4 );

    if ( code > 0 )

     OutputBits( output_file, code, bit_count );

    else

     OutputBits( output_file, code + top_of_range, bit_count );

  }

Figure 11.13 The coding for number of bits.

As if this prefix code didn't complicate things enough, OutputCode() has an additional thing to worry about; run-length encoding. Since it doesn't make sense to have a number of bits equal to zero, that value is actually used to encode a run of zeros. The number of consecutive zeros is encoded as a four-bit number immediately following a bit count of zero. Note that the four-bit number encodes runs of length 1 to 16, not 0 to 15 as might be first suspected. This is done since there is no reason to waste a code on a run length of zero.

To properly encode runs of zeros, OutputCode() tracks the current run length. Anytime OutputCode() is called to send out a value of zero, the routine actually just increments the run-length counter, then returns.

The routine will finally be able to output the length of a run when one of two things happens. First, the run length can actually reach sixteen. This is the longest run we can encode, which means it will flush the counter with a run output. The other situation is when OutputCode() is called to send a non-zero code, and the run-length counter is greater than zero. This means a run has just concluded, and it is time to output if.

The final complication in this routine is the encoding of normal numbers. As was shown in the earlier figure, these have an unusual format, with each code encoding a range of negative numbers, then a range of positive numbers, with a gap in between.

OutputCode() first determines how many bits are going to be needed to encode the code by sitting in a loop checking to see if the output code falls in the appropriate range. When it finds the correct range, it encodes the number, using a different offset for negative and positive numbers.

File Expansion

Once the file-compression algorithm is understood, file expansion is relatively easy to follow. The expansion routine first reads in the quality number from the file and uses it to initialize the matrix data. It then sits in a loop, reading in 8-by-8 DCT blocks. The routine that reads the DCT data also takes it out of the zig-zag sequence, storing it in row normal fashion, then dequantizing it. At that point, it is run through the inverse DCT procedure, which returns a block of pixel data. Once an entire strip of pixel data has been read in, it is written to the uncompressed output file.

Note that the expansion routine uses an array of pointers to redirect the output of the inverse DCT to the PixelStrip array. This array has to be set up before every inverse DCT is called so the data is directed to the correct point in the pixel strip. The pixel strip is a matrix 8 rows deep and 320 columns wide.

void ExpandFile( BIT_FILE *input, FILE *output,

         int argc, char *argv[] )

{

 int row;

 int col;

 int i;

 int input_array[ N ][ N ];

 unsigned char *output_array[ N ];

 int quality;

 quality = (int) InputBits( input, 8 );

 Initialize( quality );

 for ( row = 0 ; row < ROWS; row += N ) {

  for ( col = 0 ; col < COLS ; col += N ) {

   for ( i = 0 ; i < N ; i++ )

    output_array[ i ] = PixelStrip[ i ] + col;

  ReadDCTData( input, input_array );

  InverseDCT( input_array, output_array );

  }

  WritePixelStrip( output, PixelStrip );

 }

}

ReadDCTData()

This routine reads in DCT codes from the InputCode routine, dequantizes them, then stores them in the correct location. The codes read back in have been stored in the zig-zag sequence, so they have to be redirected to their appropriate locations in the 8-by-8 block. This is accomplished with a simple table lookup.

void ReadDCTData( input_file, input_data )

BIT_FILE *input_file;

int input_data[ N ][ N ];

{

 int i;

 int row;

 int col;

 for ( i = 0 ; i < ( N * N ) ; i++ ) {

  row = ZigZag[ i ].row;

  col = ZigZag[ i ].col;

  input_data[ row ][ col ] = InputCode( input_file ) *

             Quantum[ row ][ col ];

 }

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Input DCT Codes

Reading in the DCT codes is somewhat less complicated than writing them out, but a number of factors still need to be taken into account. First, we read in the first two bits of the bit count code. If the two bits have a value of zero, it means that a run of zeros is being encoded with this value. The zero count is read in using the next four bits and stored in the global run-length indicator.

The global run-length indicator is stored in the InputRunLength variable, and it is checked every time the InputCode routine is called. If the value in this variable is non-zero, we are still returning a run of zeros. When this is the case, the run-length indicator is decremented, and a zero is returned to the calling program.

In the event that the first two bits aren't zero, we are working with a normal bit count code. Either two or three more bits are read in to compose the rest of the code, which yields the correct bit count. We can then read in the encoded amplitude of the DCT variable by reading in that bit count.

Once that value is loaded in, we need to convert it to a normal number from the specially encoded form it is in, which is relatively simple. Finally, the correct number is returned to the calling for dequantization and processing.

int InputCode( input_file )

BIT_FILE *input_file;

{

 int bit_count;

 int result;

 if ( InputRunLength > 0 ) {

  InputRunLength--;

  return( 0 );

 }

 bit_count = (int) InputBits( input_file, 2 );

 if ( bit_count == 0 ) {

  InputRunLength = (int) InputBits( input_file, 4 );

  return( 0 );

 }

 if ( bit_count == 1 )

  bit_count = (int) InputBits( input_file, 1 ) + 1;

 else

  bit_count = (int) InputBits( input_file, 2 ) +

       ( bit_count << 2 ) - 5;

 result = (int) InputBits( input_file, bit_count );

 if ( result & ( 1 << ( bit_count - 1 ) ) )

  return( result );

 return( result - ( 1 << bit_count ) + 1 );

}

The Inverse DCT

The Inverse DCT is performed using the exact reverse of the operations performed in the DCT. First, the DCT values in the N-by-N matrix are multiplied by the cosine transform matrix. The result of this transformation is stored in a temporary N-by-N matrix of doubles. This matrix is then multiplied by the transposed cosine transform matrix. The result of this multiplication is rounded, scaled to the correct unsigned character range of zero to 255, then stored in the output block of pixels.

void InverseDCT( int input[ N ][ N ], unsigned char *output[ N ] )

{

 double temp[ N ][ N ];

 double temp1;

 int i;

 int j;

 int k;

/*MatrixMultiply( temp, input, C ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp[ i ][ j ] = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp[ i ][ j ] += input[ i ][ k ] * C[ k ][ j ];

 }

}

/*MatrixMultiply( output, Ct, temp ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp1 = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp1 += Ct[ i ][ k ] * temp[ k ][ j ];

   temp1 += 128.0;

   if ( temp1 < 0 )

    output[ i ][ j ] = 0;

   else if ( temp1 > 255 )

    output[ i ][ j ] = 255;

   else

    output[ i ][ j ] = ROUND( temp1 );

  }

 }

}

The Complete Code Listing

The complete listing of DCT.C follows.

\*************************** Start of DCT.C *****************************

*

* This is the DCT module, which implements a graphics compression

* program based on the discrete cosine transform. It needs to be

* linked with the standard support routines.

*

*/

#include <stdio. h>

#include <stdlib.h>

#include <math.h>

#include "bitio.h"

#include "errhand.h"

/*

* A few parameters can be adjusted to modify the compression

* algorithm. The first two define the number of rows and columns in

* the grey-scale image. The last one, 'N,' defines the DCT block size.

*/

#define ROWS 200

#define COLS 320

#define N  8

/*

* This macro is used to ensure correct rounding of integer values.

*/

#define ROUND( a )   ( ( (a) < 0 ) ? (int) ( (a) - 0.5 ) : \

                         (int) ( (a) + 0.5 ) )

char *CompressionName = "DCT compression";

char *Usage      = "infile outfile [quality]\nQuality from 0-25";

/*

* Function prototypes for both ANSI and K&R.

*/

#ifdef __STDC__

void Initialize( int quality );

void ReadPixelStrip( FILE *input, unsigned char strip[ N ][ COLS ] );

int InputCode( BIT_FILE *input );

void ReadDCTData( BIT_FILE *input, int input_data[ N ][ N ] );

void OutputCode( BIT_FILE *output_file, int code );

void WriteDCTData( BIT_FILE *output_file, int output_data[ N ][ N ] );

void WritePixelStrip( FILE *output, unsigned char strip[ N ][ COLS ] );

void ForwardDCT( unsigned char *input[ N ], int output[ N ][ N ] );

void InverseDCT( int input[ N ][ N ], unsigned char *output[ N ] );

void CompressFile( FILE *input, BIT_FILE *output,

                   int argc, char *argv[] );

void ExpandFile( BIT_FILE *input, FILE *output, int argc, char *argv[] );

#else

void Initialize();

void ReadPixelStrip();

int InputCode();

void ReadDCTData();

void OutputCode();

void WriteDCTData();

void WritePixelStrip();

void ForwardDCT();

void InverseDCT();

void CompressFile();

void ExpandFile();

#endif

/*

* Global data used at various places in the program.

*/

unsigned char PixelStrip[ N ][ COLS ];

double C[ N ][ N ];

double Ct[ N ][ N ];

int InputRunLength;

int OutputRunLength;

int Quantum[ N ][ N ];

struct zigzag {

 int row;

 int col;

} ZigZag[ N * N ] =

{

{0, 0},

{0, 1}, {1, 0},

{2, 0}, {1, 1}, {0, 2},

{0, 3}, {1, 2}, {2, 1}, {3, 0},

{4, 0}, {3, 1}, {2, 2}, {1, 3}, {0, 4},

{0, 5}, {1, 4}, {2, 3}, {3, 2}, {4, 1}, {5, 0},

{6, 0}, {5, 1}, {4, 2}, {3, 3}, {2, 4}, {1, 5}, {0, 6},

{0, 7}, {1, 6}, {2, 5}, {3, 4}, {4, 3}, {5, 2}, {6, 1}, {7, 0},

{7, 1}, {6, 2}, {5, 3}, {4, 4}, {3, 5}, {2, 6}, {1, 7},

{2, 7}, {3, 6}, {4, 5}, {5, 4}, {6, 3}, {7, 2},

{7, 3}, {6, 4}, {5, 5}, {4, 6}, {3, 7},

{4, 7}, {5, 6}, {6, 5}, {7, 4},

{7, 5}, {6, 6}, {5, 7},

{6, 7}, {7, 6},

{7, 7}

};

/*

* The initialization routine has the job of setting up the cosine

* transform matrix, as well as its transposed value. These two matrices

* are used when calculating both the DCT and its inverse. In addition,

* the quantization matrix is set up based on the quality parameter

* passed to this routine. The two run-length parameters are both

* set to zero.

*/

void Initialize( quality )

int quality;

{

 int i;

 int j;

 double pi = atan( 1.0 ) * 4.0;

 for ( i = 0 ; i < N ; i++ )

  for ( j = 0 ; j < N ; j++ )

   Quantum[ i ][ j ] = 1 + ( ( 1 + i + j ) * quality );

 OutputRunLength = 0;

 InputRunLength = 0;

 for ( j = 0 ; j < N ; j++ ) {

   C[ 0 ][ j ] = 1.0 / sqrt( (double) N );

   Ct[ j ][ 0 ] = C[ 0 ][ j ];

 }

 for ( i = 1 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   C[ i ][ j ] = sqrt( 2.0 / N ) *

          cos( pi * ( 2 * j + 1 ) * i / ( 2.0 * N ) ) ;

   Ct[ j ][ i ] = C[ i ][ j ];

  }

 }

}

/*

* This routine is called when compressing a grey-scale file. It reads

* in a strip that is N (usually eight) rows deep and COLS (usually 320)

* columns wide. This strip is then repeatedly processed, a block at a

* time, by the forward DCT routine.

*/

void ReadPixelStrip( input, strip )

FILE *input;

unsigned char strip[ N ][ COLS ];

{

 int row;

 int col;

  int c;

  for ( row = 0 ; row < N ; row++ )

   for ( col = 0 ; col < COLS ; col++ ) {

    c = getc( input );

    if ( c == EOF )

     fatal_error( "Error reading input grey scale file" );

    strip[ row ][ col ] = (unsigned char) c;

   }

}

/*

* This routine reads in a DCT code from the compressed file. The code

* consists of two components, a bit count, and an encoded value. The

* bit count is encoded as a prefix code with the following binary

* values:

*

*       Number of Bits                 Binary Code

*             0                            00

*             1                            010

*             2                            011

*             3                           1000

*             4                           1001

*             5                           1010

*             6                           1011

*             7                           1100

*             8                           1101

*             9                           1110

*             10                          1111

*

* A bit count of zero is followed by a four-bit number telling how many

* zeros are in the encoded run. A value of one through ten indicates a

* code value follows, which takes up that many bits. The encoding of

* values into this system has the following characteristics:

*

*      Bit Count                         Amplitudes

*         1                                -1, 1

*         2                           3 to -2, 2 to 3

*         3                          -7 to -4, 4 to 7

*         4                         -15 to -8, 8 to 15

*         5                        -31 to -16, 16 to 31

*         6                        -63 to -32, 32 to 64

*         7                       -127 to -64, 64 to 127

*         8                      -255 to -128, 128 to 255

*         9                      -511 to -256, 256 to 511

*        10                     -1023 to -512, 512 to 1023

*

*/

int InputCode( input_file )

BIT_FILE *input_file;

{

 int bit_count;

 int result;

 if ( InputRunLength > 0 ) {

  InputRunLength--;

  return( 0 );

 }

 bit_count = (int) InputBits( input_file, 2 );

 if ( bit_count == 0 ) {

  InputRunLength = (int) InputBits( input_file, 4);

  return( 0 );

 }

 if ( bit_count == 1 )

  bit_count = (int) InputBits( input_file, 1 ) + 1;

 else

  bit_count = (int) InputBits( input_file, 2 ) +

              ( bit_count << 2 ) - 5;

 result = (int) InputBits( input_file, bit_count );

 if ( result & ( 1 << ( bit_count - 1 ) ) )

  return( result );

 return( result - ( 1 << bit_count ) + 1 );

}

/*

* This routine reads in a block of encoded DCT data from a compressed

* file. The routine reorders it in row major format and dequantizes it

* using the quantization matrix.

*/

void ReadDCTData( input_file, input_data )

BIT_FILE *input_file;

int input_data[ N ][ N ];

{

 int i;

 int row;

 int col;

 for ( i = 0 ; i < ( N * N ) ; i++ ) {

  row = ZigZag[ i ].row;

  col = ZigZag[ i ].col;

  input_data[ row ][ col ] = InputCode( input_file ) *

                Quantum[ row ][ col ];

 }

}

/*

* This routine outputs a code to the compressed DCT file. For specs

* on the exact format, see the comments that go with InputCode, shown

* earlier in this file.

*/

void OutputCode( output_file, code ) BIT_FILE *output_file;

int code;

{

 int top_of_range;

 int abs_code;

 int bit_count;

 if ( code == 0 ) {

  OutputRunLength++;

  return;

 }

 if ( OutputRunLength != 0 ) {

  while ( OutputRunLength > 0 ) {

  if ( OutputRunLength <= 16 ) {

    OutputBits( output_file,

         (unsigned long) (OutputRunLength - 1 ), 4 );

    OutputRunLength = 0;

  } else {

   OutputBits( output_file, 15L, 4 );

   OutputRunLength -= 16;

  }

 }

}

if ( code < 0 )

 abs_code = -code;

else

 abs_code = code;

top_of_range = 1;

bit_count = 1;

while ( abs_code > top_of_range ) {

 bit_count++;

 top_of_range = ( ( top_of_range + 1 ) * 2 ) - 1;

 }

 if ( bit_count < 3 )

   OutputBits( output_file, (unsigned long) ( bit_count + 1 ), 3 );

 else

   OutputBits( output_file, (unsigned long) ( bit_count + 5 ), 4 );

 if (code > 0 )

   OutputBits( output_file, (unsigned long) code, bit_count );

 else

   OutputBits( output_file, (unsigned long) ( code + top_of_range ) ,

               bit_count );

}

/*

* This routine takes DCT data, puts it in zigzag order, quantizes

* it, and outputs the code.

*/

void WriteDCTData( output_file, output_data )

BIT_FILE *output_file;

int output_data[ N ][ N ];

{

 int i;

 int row;

 int col;

 double result;

 for ( i = 0 ; i < ( N * N ) ; i++ ) {

  row = ZigZag[ i ].row;

  col = ZigZag[ i ].col;

  result = output_data[ row ][ col ] / Quantum[ row ][ col ];

  OutputCode( output_file, ROUND( result ) ):

 }

}

/*

* This routine writes out a strip of pixel data to a GS format file.

*/

void WritePixelStrip( output, strip )

FILE *output;

unsigned char strip[ N ][ COLS ];

{

 int row;

 int col;

 for ( row = 0 ; row < N ; row++ )

  for ( col = 0 ; col < COLS ; col++ )

   putc( strip[ row ][ col ], output ); }

/*

* The Forward DCT routine implements the matrix function:

*

*       DCT = C * pixels * Ct

*/

void ForwardDCT( input, output )

unsigned char *input[ N ];

int output[ N ][ N ];

{

 double temp[ N ][ N ];

 double temp1;

 int i;

 int j;

 int k;

/* MatrixMultiply( temp, input, Ct ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp[ i ][ j ] = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp[ i ][ j ] += ( (int) input [ i ][ k ] - 128 ) *

               Ct[ k ][ j ];

  }

 }

/* MatrixMultiply( output, C, temp ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp1 = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp1 += C[ i ][ k ] * temp[ k ][ j ];

   output[ i ][ j ] = ROUND( temp1 );

  }

 }

}

/*

* The Inverse DCT routine implements the matrix function:

*

*       pixels = C * DCT * Ct

*/

void InverseDCT( input, output )

int input[ N ][ N ];

unsigned char *output[ N ];

{

 double temp[ N ][ N ];

 double temp1;

 int i;

 int j;

 int k;

/* MatrixMultiply( temp, input, C ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp[ i ][ j ] = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp[ i ][ j ] += input[ i ][ k ] * C[ k ][ j ];

  }

 }

/* MatrixMultiply( output, Ct, temp ); */

 for ( i = 0 ; i < N ; i++ ) {

  for ( j = 0 ; j < N ; j++ ) {

   temp1 = 0.0;

   for ( k = 0 ; k < N ; k++ )

    temp1 += Ct[ i ][ k ] * temp[ k ][ j ];

   temp1 += 128.0;

   if ( temp1 < 0 )

    output[ i ][ j ] = 0;

   else if ( temp1 > 255 )

    output[ i ][ j ] = 255;

   else

    output[ i ][ j ] = (unsigned char) ROUND( temp1 );

  }

 }

}

/*

* This is the main compression routine. By the time it gets called,

* the input and output files have been properly opened, so all it has to

* do is the compression. Note that the compression routine expects an

* additional parameter, the quality value, ranging from 0 to 25.

*/

void CompressFile( input, output, argc, argv )

FILE *input;

BIT_FILE *output;

int argc;

char *argv[];

{

 int row;

 int col;

 int i;

 unsigned char *input_array[ N ];

 int output_array[ N ][ N ];

 int quality;

 if ( argc-- > 0 )

  quality = atoi( argv[ 0 ] );

 else

  quality = 3;

 if ( quality < 0 || quality > 50 )

  fatal_error( "Illegal quality factor of %d\n", quality );

 printf( "Using quality factor of %d\n", quality );

 Initialize( quality );

 OutputBits( output, (unsigned long) quality, 8 );

 for ( row = 0 ; row < ROWS ; row += N ) {

  ReadPixelStrip( input, PixelStrip );

  for ( col = 0 ; col < COLS ; col += N ) {

    for ( i = 0 ; i < N ; i++ )

     input_array[ i ] = PixelStrip[ i ] + col;

    ForwardDCT( input_array, output_array );

    WriteDCTData( output, output_array );

   }

 }

 OutputCode( output, 1);

 while ( argc-- > 0 )

   printf( "Unused argument: %s\n", *argv++ );

}

/* The expansion routine reads in the compressed data from the DCT file,

* then writes out the decompressed grey-scale file.

*/

void ExpandFile( input, output, argc, argv)

BIT_FILE *input;

FILE *output;

int argc;

char *argv[];

{

 int row;

 int col;

 int i;

 int input_array[ N ][ N ];

 unsigned char *output_array[ N ];

 int quality;

 quality = (int) InputBits( input, 8 );

 printf( "\rUsing quality factor of %d\n", quality );

 Initialized( quality );

 for ( row = 0 ; row < ROWS ; row += N ) {

  for ( col = 0 ; col < COLS ; col += N ) {

   for ( i = 0 ; i < N ; i++ )

    output_array[ i ] = PixelStrip[ i ] + col;

   ReadDCTData( input, input_array );

   InverseDCT( input_array, output_array );

  }

  WritePixelStrip( output, PixelStrip );

 }

 while ( argc-- > 0 )

  printf( "Unused argument: %s\n", *argv++ );

}

/**************************** End of DCT.C *****************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Support Programs

The two support programs used in this chapter are GS.C, used to display "non-format" grey-scale files, and GSDIFF.C, used to display the differences between two files and to print the rms value of the error. They follow.

/*************************** Start of GS.C *****************************/

*

* This is the GS program, which displays grey-scale files on the

* IBM VGA adaptor. It assumes that the grey-scale values run from

* zero to 255, and scales them down to a range of zero to sixty-three,

* so they will be displayed properly on the VGA.

*

* This program can be called with a list of files, and will display them

* in consecutive order, which is useful for trying to measure visual

* differences in compressed files.

*

* This program writes directly to video memory, which should work

* properly on most VGA adaptors. In the event that it doesn't, the

* constant USE_BIOS can be turned on, and the code will use BIOS calls

* to write pixels instead. This will be somewhat slower, but should work

* on every VGA adaptor.

*

* Note that the use of far pointers means this program should probably

* be compiled without using the strict ANSI option of your compiler.

*/

#include <stdio.h>

#include <stdlib.h>

#include <dos.h>

#include <conio.h>

main( int argc, char *argv[] )

{

 union REGS rin;

 union REGS rout;

 int i;

 FILE *file;

 char far *video;

 if ( argc < 2 ) {

  printf( "Usage: gs file\n\n" );

  exit( 1 );

 }

 rin.h.ah = 0;

 rin.h.al = 0x13;

 int86( 0x10, &rin, &rout );

 rin.h.ah = 0x10;

 rin.h.al = 0x10;

  for ( i = 0 ; i < 64 ; i++ ) {

   rin.h.dh = (unsigned char) i;

   rin.h.ch = (unsigned char) i;

   rin.h.cl = (unsigned char) i;

   rin.x.bx = i;

   int86( 0x10, &rin, &rout );

  }

  rin.h.ah = 0x10;

  rin.h.al = 0x1b;

  rin.x.cx = 256;

  rin.x.bx = 0;

  int86( 0x10. &rin, &rout );

  argv++;

  while ( —argc > 0 ) {

   file = fopen( *argv++, "rb" );

   if ( file == NULL ) {

     putc( 7, stdout );

     break;

   }

   video = (char far *) 0xA0000000L;

   rin.h.ah = 0x0c;

   rin.h.bh = 0;

   for ( rin.x.dx = 0 ; rin.x.dx < 200 ; rin.x.dx++ ) {

     for ( rin.x.cx = 0 ; rin.x.cx < 320 ; rin.x.cx++ ) {

#ifdef USE_BIOS

        rin.h.al = (unsigned char) ( getc( file ) >> 2);

        int86( 0x10, &rin, &rout );

#else

        *video++ = (char) ( getc( file ) >> 2);

#endif

    }

   }

   fclose( file );

   getch();

  }

  rin.h.ah = 0;

 rin.h.al = 3;

 int86( 0x10. &rin, &rout );

 return 0;

}

/***************************** End of GS.C *****************************/

/***************************** Start of GSDIFF.C ***********************

*

* This is the GSDIFF program, which displays the differences between

* two grey-scale files on the IBM VGA adaptor. It assumes that the

* grey-scale values run from zero to 255, and scales them down to a

* range of zero to sixty-three, so they will be displayed properly on

* the VGA.

*

* This program writes directly to video memory, which should work

* properly on most VGA adaptors. In the event that it doesn't, the

* constant USE_BIOS can be turned on, and the code will use BIOS calls

* to write pixels instead. This will be somewhat slower, but should work

* on every VGA adaptor.

*

* While this program is writing out to the display, it is also keeping a

* running total of the error differences. When the program is

* complete, it prints out the RMS error. If the -B switch is turned

* on, the program operates in batch mode, and doesn't display the

* differences. It just computes and prints the rms error value.

*

* Note that the use of far pointers means this program should probably

* be compiled without using the strict ANSI option of your compiler.

*/

#include <stdio.h>

#include <stdlib.h>

#include <string>

#include <dos.h>

#include <conio.h>

#include <math.h>

main( int argc, char *argv[] )

{

 union REGS rin;

 union REGS rout;

 int i;

 FILE *file1;

 FILE *file2;

 int diff;

 int c1;

 int c2;

 char far *video;

 double error;

 int batch;

 if ( argc < 3 ) {

  printf( "Usage: gsdiff file1 file2 [-B]\n\n" );

  exit( 1 );

 }

 file1 = fopen( argv[ 1 ], "rb" );

 file2 = fopen( argv[ 2 ], "rb" );

 if ( file1 == NULL || file2 == NULL ) {

  printf( "Could not open file!\n" );

  exit( 1 );

 }

 batch = 0;

 if ( argc > 3 )

  if ( strcmp( argv[ 3 ], "-b" ) == 0 ||

   strcmp( argv [ 3 ], "-B" ) == 0 )

   batch = 1;

 if ( !batch ) {

  rin.h.ah = 0;

  rin.h.al = 0x13;

  int86( 0x10, &rin, &rout );

  rin.h.ah = 0x10;

  rin.h.al = 0x10;

  for ( i = 0 ; i < 64 ; i++ ) {

    rin.h.dh = (unsigned char) i;

    rin.h.ch = (unsigned char) i;

    rin.h.cl = (unsigned char) i;

    rin.x.bx = i;

    int86( 0x10, &rin, &rout );

  }

  rin.h.ah = 0x10;

  rin.h.al = 0x1b;

  rin.x.cx = 256;

  rin.x.bx = 0;

  int86( 0x10, &rin, &rout );

 }

 error = 0.0;

 video = (char far *) 0xA0000000L;

 rin.h.ah = 0x0c;

 rin.h.bh = 0;

 for ( rin.x.dx = 0 ; rin.x.dx < 200 ; rin.x.dx++ ) {

   for ( rin.x.cx = 0 ; rin.x.cx < 320 ; rin.x.cx++ ) {

    c1 = getc( file1 );

    c2 = getc( file2 );

    diff = c1 - c2;

    error += diff*diff;

    if ( diff < 0 )

     diff *= -1;

    if ( diff > 63 )

     diff = 63;

    if ( !batch ) {

#ifdef USE_BIOS

     rin.h.al = diff;

     int86( 0x10, &rin, &rout );

#else

     *video++ = diff;

#endif

    }

   }

 }

 fclose( file1 );

 fclose( file2 );

 if ( !batch ) {

  getch();

  rin.h.ah = 0;

  rin.h.al = 3;

  int86( 0x10, &rin, &rout );

 }

 error /= 320.0 * 200.0;

 printf( "RMS error between %s and %s is %lf\n",

  argv[ 1 ], argv[ 2 ], sqrt( error ) );

  return 0;

 }

/*************************** End of GSDIFF.C ***************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Some Compression Results

The disk included with this book contains five grey-scale files to experiment with. Some of the results of compressing these files using the DCT program is shown in Figure 11.14.

Figure 11.14 Compression results

On most of the images, compression quality figures of five or lower produce a slight loss of resolution, but no significant loss of picture quality. Once the quality factor gets above five, visible artifacts of the compression process start to become visible as "blocking" of the image.

Figure 11.15 shows a few images of CHEETAH.GS after going through a compression cycle. The first few images look fairly good. In fact, it is hard to spot much of a difference. Viewing the images one immediately after the other using GS.EXE will clearly show that there have been changes, but they are not glaring differences. At quality 5, close inspection shows a few areas where the compression is clearly starting to cause picture quality to slip. The images with quality factors 10, 15, and 20 show clear degradation.

CHEETAH.GS Original Image

Qaulity = 1

Qaulity = 2

Qaulity = 3

Qaulity = 5

Qaulity = 10

Qaulity = 15

Qaulity = 25

Figure 11.15 CHEETAH.GS after a compression cycle.

The compression results achieved from these experiments are quite impressive. In most cases, images can be compressed up to about 85 percent without losing much picture quality. Better compression than this could be expected from the JPEG algorithm, since it adds a Huffman coding stage which DCT.C lacks.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 12
An Archiving Package

Programmers and users are perhaps most frequently exposed directly to data compression through the use of an archiving program. In the MS-DOS world, the use of archiving packages is ubiquitous, with the distribution of some packages fast approaching the saturation point. Programs such as PKZIP and ARJ that are distributed through non-commercial channels tend to blanket the world of "power users," with new releases getting world-wide distribution in a matter of only days.

Because data compression is a competitive field, these programs tend to have very good performance characteristics, having both high throughput and tight compression ratios. They tend to do their jobs well.

But for the programmer, these data-compression programs are solely lacking in one respect: their handling of source code. While it's nice to be able to invoke PAK or ARC from the MS-DOS command line, that doesn't help the programmer who wants to compress all of the on-line help screens in his new spreadsheet program. It would be somewhat impractical for his program to have to spawn a copy of PKUNZIP every time a new help screen needed to be accessed.

This chapter presents a solution to such dilemmas by showing you how to create a simple, stripped-down version of an archiving program. While space limitations in the book prevent this program from being a match for commercial programs, a good programmer armed with the techniques found in this book should be able to enhance this program to make it as useful as commercial equivalents.

CAR and CARMAN

This chapter deals with two topics: Compressed Archive files and the program used to maintain them. Compressed archive files conventionally have a file extension of ".CAR," and will be referred to as CAR files. The CAR file Manager will be named CARMAN.

CARMAN is a stand alone program designed to manipulate CAR files. It has a fairly simple set of commands, and runs using command-line mode. CARMAN's real strength lies in either its extension with more powerful compression techniques or more detailed file data, or the inclusion of portions of its code into other programs.

The CARMAN Command Set

Running CARMAN with no arguments gives a brief help screen showing the usage of the program, as shown in Figure 12.1.

Figure 12.1 The CARMAN Help Screen

Every CARMAN operation has two basic requirements. First, it must have a single letter command and, second, it must have the name of a CAR file. A brief synopsis of the commands follows.

Add files: | This command is used to add new files to an archive, which may or may not already exist. Wild cards on the command line will be expanded under MS-DOS. Full path names can be used to specify input files, but CAR will strip the path components before storing the files. If the CAR file already exists, and a file to be added already exists in the archive, the new version will replace the old.
Xtract files: | This command extracts files from the archive and stores them in the current directory. If no file names are listed on the command line all files are extracted from the archive.
Replace files: | This command attempts to replace all of the named archive files with a new version from the current directory. If a specified file exists in the archive but not in the current directory, a warning message is printed.
Delete files: | The named files are deleted from the CAR file.
Print files: | The specified files are copied to stdout. If no files are named, all files will be extracted.
Test files: | The specified files are tested to be sure they can be properly extracted, and that the resulting CRC value will be correct.
List files: | The statistics for the specified files are listed on stdout. If no file names are specified, all files are listed. A typical listing is shown next.

Figure 12.2 CARMAN List Command Output

As can be seen from this listing, the compression method employed in CARMAN is LZSS, with the compression code being nearly identical to that shown in Chapter 8. Files that could not be compressed to less than their original size will instead be stored in uncompressed format.

While LZSS does not offer the tightest compression possible, it does provide adequate performance. In addition, it has one unique advantage: its expansion speed will meet or exceed that of nearly any compression program available. So applications that need to decompress frequently may find LZSS to be the algorithm of choice.

The CAR File

The structure of a CAR file is very simple: it simply consists of a sequential list of file header blocks followed by file data. This sequence repeats indefinitely until a special header with a null file name is encountered. An illustration of this structure is shown in Figure 12.3.

Figure 12.3 The structure of a CAR file.

A sequential structure like this has both advantages and disadvantages. The sequential nature of the data means that both searches through and updates of the archive are not done using a random access method. Instead, linear searches and copies are used. Even worse, any time files in the archive are modified, it means the entire archive has to be copied from the original to a new version of the file. These disadvantages are outweighed by the simplicity this technique offers. Good reliable code can easily be written to support this storage method. In fact, most popular archiving programs use a nearly identical format.

The Header

In the CAR format, the header for each file contains everything we know about the file. Thus, selecting what goes in the header and what doesn't is fairly important. CARMAN uses a fairly stripped down set of information in the header file, with a C structure as follows:

   typedef struct header {

    char file_name[ FILENAME_MAX ];

    char compression_method;

    unsigned long original_size;

    unsigned long compressed_size;

    unsigned long original_crc;

    unsigned long header_crc;

   } HEADER;

Most of the information in the header is self explanatory, particularly in terms of how it is used a C program. The place where the header information gets a little confusing is in the process of storing or reading it to/from a CAR file.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Storing the Header

To make CAR files portable across different systems and architectures, you need to take care when writing data to files. Conventionally, if we were to write this structure out to a file, we might use a line of code that looks like this:

   fwrite( header, sizeof( HEADER ), 1, outfile );

This writes a binary image of the header data directly out to the file, so that can be easily read in using an equivalent fread() statement.

There are two potential problems that crop up when writing data out this way. The first relates to the packing of structures. Different C compilers will pack structure elements in different fashions. For example, we have a single char element as the second element of the header array shown above. Since the next element is a long integer, and MS-DOS compiler might put in three bytes of padding so that the long element is aligned on a four-byte boundary. Generally, this is done to generate faster, more efficient code. (Many CPUs tend to work better with data aligned on certain boundaries.) A different compiler on a different machine might not insert any padding, or might use an eight-byte boundary.

When structures are packed differently, we can no longer count on portability between binary files generated using fwrite() calls such as the one shown above. However, it would seem that this would be a relatively easy problem to overcome. Instead of writing out the structure as a single entity, we could just store it one element at a time, which would guarantee that no packing bytes were inadvertently added to our output.

This solution runs afoul of our second portability problem. Unfortunately, we cannot be sure that different computers will store identical data elements using the same structure. For example, a long integer with the hex value 0x12345678L would be stored in the following manner on an Intel 8086 machine:

Address | Value
0000 | 78
0001 | 56
0002 | 34
0003 | 12

The same long integer stored on machine based on the Motorola 68000 architecture would have the bytes stored in exactly the reverse order! These differences result from decisions the hardware designers made long ago, for better or worse, and we all have to live with the consequences. In this case, the consequence is a problem with binary file interchange.

The solution is to take control of the binary file format at the lowest level. Instead of trying to write out short and long integers in one fell swoop, we write them out a byte at a time, using the ordering that we select. This way we should be able to store and retrieve data items so that our CAR files can be ported across various systems without worrying about incompatibilities.

When reading and writing the headers, you would first pack and unpack the short and long integer data in the header file into a character array, using a pair of utility routines. We arbitrarily pack the data with the least significant bytes first, although it could just as easily be done in the other order. The routines that do the packing follow:

   void PackUnsignedData( number_of_bytes, number, buffer )

   int number_of_bytes;

   unsigned long number;

   unsigned char *buffer;

   {

    while ( number_of_bytes-- > 0 ) {

     *buffer++ = ( unsigned char ) number & Oxff;

     number >>= 8;

    }

   }

   unsigned long UnpackUnsignedData( number_of_bytes, buffer

   int number_of_bytes;

   unsigned char *buffer;

   {

    unsigned long result;

    int shift_count;

    result = 0;

    shift_count = 0;

    while ( number_of_bytes-- > 0 ) {

     result |= (unsigned long) *buffer++ << shift_count;

     shift_count += 8;

    }

    return( result );

   }

Given these packing and unpacking routines, reading and storing the header files is simple. The process is accomplished for the file I/O using an intermediate character array. The actual header data is packed and unpacked to and from the array.

void WriteFileHeader()

{

  unsigned char header_data[ 17 ];

  int i;

  for ( i = 0 ; ; ) {

   putc( Header.file_name[ i ], OutputCarFile );

   if ( Header.file_name[ i++ ] == '\O' )

    break;

  }

  Header.header_crc = CalculateBlockCRC32( i, CRC_MASK,

                          Header.file_name );

  PackUnsignedData( 1, (long)

            Header.compression_method, header_data + 0 );

  PackUnsignedData( 4, Header.original_size, header_data + 1 );

  PackUnsignedData( 4, Header.compressed_size, header_data + 5 );

  PackUnsignedData( 4, Header.original_crc, header_data + 9 );

  Header.header_crc = CalculateBlockCRC32( 13, Header.header_crc,

       header_data );

  Header.header_crc ^= CRC_MASK;

  PackUnsignedData( 4, Header.header_crc, header_data + 13 );

  fwrite( header_data, 1, 17, OutputCarFile );

}

The routine to write the file header out to the CAR file is somewhat simpler than the routine to read the same data, since it doesn't have to check for some of the possible error conditions. The first part of a header consists of the name of the compressed file, stored with a null terminator character. No special care needs to be taken when writing out the file name, since eight-bit ASCII characters are portable across all of the systems toward which CARMAN is targeted.

The remaining elements of the header are packed, one by one, into a character array using the PackUnsignedData() routine. Once they have all been properly packed, they can be written out with a call to fwrite(), with everything being in a known state.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Header CRC

One of the header elements written out by WriteFileHeader() is called "header_crc." The header CRC is a 32-bit number generated using the data in the header structure, and used as a checksum. The CRC is generated using the CCITT-32 formula, which is the same formula used by many other archiving programs, such as PKZIP and ARJ. It provides us with a reasonably high probability of detecting errors in the header.

The reason for creating a CRC checksum for the header data is to provide an additional check for validity of a CAR file. If for some reason one of the data elements in the header file was inadvertently modified, it could lead to to disastrous results either during decompression, or later when attempting to use erroneous file data.

int ReadFileHeader()

{

 unsigned char header_data[ 17 ];

 unsigned long header_crc;

 int i;

 int c;

 for ( i = 0 ; ; ) {

  c = getc( InputCarFile );

  Header.file_name[ i ] = (char) c;

  if ( c == '\0' )

   break;

  if ( ++i == FILENAME_MAX )

   FatalError( "File name exceeded maximum in header" );

 }

 if ( i == 0 )

  return( 0 );

  header_crc= CalculateBlockCRC32( i + 1, CRC_MASK, Header.file_name );

  fread( header_data, 1, 17, InputCarFile );

  Header.compression_method= (char)

               UnpackUnsignedData( 1, header_data + 0);

  Header.original_size = UnpackUnsignedData(4, header_data + 1);

  Header.compressed_size = UnpackUnsignedData(4, header_data + 5);

  Header.original_crc = UnpackUnsignedData(4, header_data + 9);

  Header.header_crc = UnpackUnsignedData(4, header_data + 13);

  header_crc = CalculatedBlockCRC32( 13, header_crc, header_data );

  header_crc ^=CRC_MASK;

  if ( Header.header_crc!= header_crc )

    FatalError( "Header checksum error for file %s", Header.file_name );

  return( 1 );

}

Reading the file header is essentially the reverse procedure of writing it out—with a couple of twists. During the process of reading in the file name, we need to check for a couple of different possibilities. First, if this is the last header in a CAR file, it will have a file name length of 0. If this is the case, we immediately return with a failure indication, so the calling routine will know that we have reached the end of the input CAR file.

A second possibility is that the file name may exceed the storage allocated for in the header structure. In that case, a fatal error exit is taken.

After all of the header data has been read in, we perform one last validity check by comparing the calculated CRC for the header file with the CRC that was stored in the CAR file. In case of a mismatch, we once again take the fatal error exit.

Command-Line Processing

Once we have the ability to read in a header from a CAR file, we have the capability to list the archive. A simple loop like this would be enough to do it:

  while ( ReadFileHeader() != 0 ) {

   ListCarFileEntry();

   fseek( input, header.compressed_size, SEEK_CUR );

  }

All that is needed to skip over all of the compressed data for a given file is the fseek() statement, since we know the size of the compressed data. This is the mechanism used to work our way through the CAR file when performing any type of processing. We start with the very first file, and work our way from header to header, processing each file as needed. At no time does CARMAN ever back up through an input archive, or try to seek ahead past the next file.

Now that we have the ability to start doing something with the CAR file, it is time to start putting the other pieces of the program together. The next logical step is to start adding the ability to handle the command line.

There are three components to the CARMAN command line. First, the command is one of seven single letters discussed previously. Second is the name of the CAR file. Finally comes the optional list of file names. An initial call to ParseArguments() checks for the validity of the first command, and performs some checking on the next two.

int ParseArguments( argc, argv )

int argc;

char *argv[];

{

  int command;

  if ( argc < 3 || strlen( argv[ 1 ] ) > 1 )

   UsageExit();

  switch( command = toupper( argv[ 1 ][ 0 ] ) ) {

  case 'X':

   fprintf( stderr, "Extracting files\n");

   break;

  case 'R' :

   fprintf( stderr, "Replacing files\n" );

   break;

  case 'P' :

   fprintf( stderr, "Print files to stdout\n" );

   break;

  case 'T' :

   fprint( stderr, "Testing integrity of files\n" );

   break;

  case 'L' :

   fprintf( stderr, "Listing archive contents\n" );

   break;

  case 'A' :

   if ( argc <= 3)

    UsageExit();

   fprintf( stderr, "Adding/replacing files to archive\n" );

   break;

  case 'D' :

   if ( argc <= 3 )

    UsageExit();

   fprintf( stderr, "Deleting files from archive\n" );

   break;

  default :

   UsageExit();

 };

 return( command );

}

The first step in parsing the command line is to make sure that are at least three arguments on the command line: the command name (CARMAN), a single letter command, and a CAR file name. The next step is to check the command letter for validity, to be sure it is one of the legally defined CARMAN commands. As the command letter is determined, a short message is printed to indicate that CARMAN has acknowledged the command. Finally, for two particular cases, CARMAN insists that specific file names be included on the command line. For most of the CARMAN commands, specifying no file names on the command line is defined as the equivalent of using the wildcard argument "*", (or "*.*", the MS-DOS equivalent). That means that "CARMAN I backup.car" will list all the files in the backup.car archive.

For the Add and Delete commands, this default mode of operation is probably a little too dangerous, so it results in an error message. If the user wants to add every file in the current directory to a CAR file, it will be necessary to specify "*" or "*.*" on the command line, which should not be too much of an inconvenience.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Generating the File List

One of the basic requirements of CARMAN is that it be able to handle lists of files, so that it can perform operations on select groups of files. Every one of the seven CARMAN commands accepts a list of files as an argument, so we need to have a general purpose way to build and manage a list of files. The list function should also be able to accommodate at least some level of wild card pattern matching as well.

Wild card matching needs to be done a little differently under MS-DOS and UNIX. First of all, there are actually two types of wild card matching taking place in CARMAN. File specifications on the command line with the 'Add' command, including wild cards, specify external files that are going to be added to the CAR file. File specifications for all of the other commands refer to files stored inside the CAR file. Thus we have to handle these two types of file lists using slightly different methods.

To complicate further, UNIX and MS-DOS differ significantly in the way their command lines handle wild card file specifications. Under either operating system, if we want to add all the C files in a directory to an archive, we would type a similar command:

carman a c_files.car *.c

Under UNIX, the command interpreter, or shell program, expands the list of wild card file names before the program ever sees it. This means that by the time CARMAN is invoked, it is presented with a command line that might look something like this:

carman a c_files.car test.c io.c foo.c bar.c

This is one of the nice features of UNIX; an application program doesn't have to worry about wild card expansion of files from the command line because the shell takes care of the work.

Under MS-DOS, matters are a little more complicated. Wild card expansion is thought of as being the province of an application program, not the command-line interpreter. So code that builds the file name list has to perform the expansion manually using C run-time library functions. Even worse, the function names and structure definitions used to expand wild card listings have not been standardized among compiler vendors, so that each new compiler needs a slightly different implementation.

In CARMAN, the list of the file names is found in an array called FileList[]. FileList is an array of character pointers which is set up via a routine called BuildFileList(). BuildFileList() is called right after the command line is parsed, and is passed a list of command-line arguments, along with a count.

BuildFileList() normally just copies the arguments passed to it into the FileList[] array. If there are no command-line arguments, FileList[0] is set to '*', so that all file names in the archive will be matched as they are processed. This has the effect of converting a command like "CARMAN L TEST.CAR" to "CARMAN L TEST.CAR *".

BuildFileList() changes its mode of operation if the user has specified that the command is 'A,' to add files, and the operating system is MS-DOS. Under these circumstances, a special routine is called to expand a potential wild-card file specification into a list of file names, with the results all being stored in the FileList[].

void BuildFileList( argc, argv, command )

int argc;

char *argv[];

int command;

{

 int i;

 int count;

 count = 0;

 if ( argc == 0 )

    FileList[ count++ ] = "*";

 else {

    for ( i = 0 ; < argc ; i++ ) {

#ifdef MSDOS

      if ( command == 'A' )

       count = ExpandAndMassageMSDOSFileNames (count, argv[ i ]);

      else

       MassageMSDOSFileName( count++, argv[ i ] );

#endif

#ifndef MSDOS

  FileList[ count ] = malloc( strlen( argv[ i ] ) + 2);

  if ( FileList[ count ] == NULL )

    FatalError( "Ran out of memory storing file names" );

  strcpy( FileList[ count++ ], argv[ i ];

#endif

  if ( count > 99 )

    FatalError( "Too many file names" );

  }

 }

 FileList[ count ] = NULL;

}

In addition, a special routine called MassageMSDOSFileName() is called to normalize all MS-DOS file names. MS-DOS has a couple of complications in its file system. First of all, file names are case insensitive, meaning that "FOO.BAR" and "foo.bar" both refer to the same file, despite the fact that their names are different. Secondly, the 8+3 file naming convention means that "FOO" and "FOO." both refer to the same file, even though one has a trailing '.' character and the other doesn't.

Massage MSDOSFileName() gets around this problem by performing two operations on file names. First of all, uppercase characters in file names on the command line are all converted to lowercase, to avoid ambiguities created by case mismatches. Secondly, any file that doesn't have an extension or a '.' in the name has a '.' character appended to the end of its name. Note that if the file name contains a '*' or '?' character, meaning it is a wild card, the '.' character is not appended to a file name with no extension.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Note that not all MS-DOS compilers will define the manifest constant MSDOS. If your compiler doesn't, you may need to edit the source files for CARMAN.C.

void MassageMSDOSFileName( count, file )

int count;

char *file;

{

 int i;

 char *p;

 FileList[ count ] = malloc( strlen( file ) + 2 );

 if ( FileList[ count ] == NULL )

  FatalError( "Ran out of memory storing file names" );

 strcpy( FileList[ count ], file );

 for ( i = 0 ; FileList[ count ][ i ] != '\0' ; i++ )

  FileList[ count ][ i ] = tolower( FileList[ count ][ i ] );

 if ( strpbrk( FileList[ count ], "*?" ) == NULL ) }

   p = strrchr( FileList[ count ], '\\' );

   if ( p == NULL )

     p = FileList[ count ];

   if ( strrchr( p, '.' ) == NULL )

        strcat( FileList[ count ], "." );

  }

 }

int ExpandAndMassageMSDOSFileNames( count, wild_name )

int count;

char *wild_name;

{

 int done;

 DIR_STRUCT file_info_block

 char *leading_path;

 char *file_name;

 char *p;

 leading_path = malloc( strlen( wild_name ) + 1 );

 file_name = malloc( strlen(wild_name ) + 13 );

 if ( leading_path == NULL || file_name == NULL )

  FatalError( "Ran out of memory storing file names" );

 strcpy( leading_path, wild_name );

    p = strrchr( leading_path, '\\' );

 if ( p != NULL )

    p[ 1 ] = '\0';

 else {

   p = strrchr( leading_path, ':' );

  if ( p != NULL )

    p[ 1 ] = '\0';

  else

    leading_path[ 0 ] = '\0';

 }

 done = FIND_FIRST( wild_name, &file_info_block, 0);

 while ( !done ) {

  strcpy( file_name, leading_path );

  strcat( file_name, file_info_block, DIR_FILE_NAME );

  MassageMSDOSFileName( count++, file_name );

  done = FINDNEXT( &file_info_block );

  if ( count > 99 )

   FatalError( "Too many file names" );

 }

 free( leading_path );

 free( file_name );

 return( count );

}

The code shown here for expanding wild cards words with most popular MS-DOS compilers. Conditional complication is used to define the macros in slightly different ways depending on which compiler is being used. Check the complete listing at the end of this chapter for details on how this is accomplished.

Opening the Archive Files

The final step before you can begin processing the CAR file is opening the input and output CAR files. This routine is complicated by several possible conditions. In general, a routine that modifies the archive will have both an input and an output CAR file. This includes the 'Add,' 'Replace,' and 'Delete' commands. These commands operate by reading in and processing the CAR file while copying files to the output file.

The remaining processing commands don't actually modify the input CAR file, so they don't have to open an OutputFile. In addition, if the command is to 'Add' files to the archive, an input file may not exist, in which case a new one has to be created. The OpenArchiveFiles() command manages all these possibilities.

void OpenArchiveFiles( name, command )

char *name;

int command;

{

 char *s;

 int i;

 strncpy( CarFileName, name, FILENAME_MAX - 1 );

 CarFileName[ FILENAME_MAX - 1 ] = '\0';

 InputCarFile = fopen( CarFileName, "rb" );

 if ( InputCarFile == NULL ) {

#ifdef MSDOS

    s = strrchr( CarFileName, '\\' );

#else /* UNIX */

    s = strrchr( CarFileName, '/' );

#endif

    if ( s == NULL )

     s = CarFileName;

    if ( strrchr( s, '.' ) == NULL )

     if ( strlen( CarFileName ) < ( FILENAME_MAX - 4 ) ) {

      strcat( CarFileName, ".car");

      InputCarFile = fopen( CarFileName, "rb" );

     }

 }

 if ( InputCarFile == NULL && command != 'A' )

  FatalError( "Can't open archive '%s'", CarFileName );

 if ( command == 'A' || command == 'R' || command == 'D' ) {

  strcpy( TempFileName, CarFileName );

  s = strrchr( TempFileName, '.' );

  if ( s == NULL )

   s = TempFileName + strlen( TempFileName );

  for ( i = 0 ; i < 10 ; i++ ) {

   sprintf( s, ".$$%d", i );

   if ( ( OutputCarFile = fopen( TempFileName, "r" ) ) == NULL )

    break;

   fclose( OutputCarFile );

   OutputCarFile = NULL;

   }

   if ( i == 10 )

    FatalError( "Can't open temporary file %s", TempFileName );

   OutputCarFile = fopen( TempFileName, "wb" );

   if ( OutputCarFile == NULL )

    FatalError( "Can't open temporary file %s", TempFileName );

 };

 if ( InputCarFile != NULL )

  setvbuf( InputCarFile, NULL, _IOFBF, 8192 );

 if ( OutputCarFile != NULL )

  setvbuf( OutputCarFile, NULL, _IOFBF, 8192 );

}

When an output file is created, it will be the file that eventually gets a copy of all the selected files from the input archive. Once all that processing is over, the input file can be deleted and the output file can be renamed to have the correct name. However, while CARMAN is still processing, the output file has to have a different name. In this case, we create a temporary file name based on the name of the CAR file. We do a limited amount of checking to try and create a name that isn't already in use.

One additional feature that CARMAN develops in this routine is the ability to automatically attempt to add a file extension to the archive name. If the input file name does not include an extension, OpenArchiveFiles() first tries to open it normally. If the file cannot be opened, OpenArchiveFiles() tries again with the ".CAR" extension appended to the file name. This lets the user type a command like "CARMAN L FILES" when the correct command might really be "CARMAN L FILES.CAR".

Finally, note that both files have big buffers set up using the setvbuf() command. This helps throughput, particularly when performing bulk copies of compressed files from the input archive to the output CAR file.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The Main Processing Loop

With these steps out of the way, CARMAN is ready to begin processing in earnest. The main processing loop is found in the routine called ProcessAllFilesInInputCar(). It sits in a big loop, finding files in the input CAR file, checking to see if they match up with any of the names in the file list, then deciding what to do based on the command.

Before entering the main processing loop, CARMAN checks to see if the command given was to 'Add' files to the CAR file. If it was, these files all need to be inserted into the output CAR file before anything else happens. This is done in a routine called AddFileListToArchive(). This routine attempts to add every file name that was specified in the command line to the output CAR file.

int AddFileListToArchive

()

{

 int i;

 int j;

 int skip;

 char *s;

 FILE *input_text_file;

 for ( i = 0 ; FileList[ i ] != NULL ; i++ ) {

  input_text_file = fopen( FileList[ i ], "rb" );

  if ( input_text_file == NULL )

   FatalError( "Could not open %s to add to CAR file",

                         FileList[ i ] );

#ifdef MSDOS

    s = strrchr( FileList[ i ], '\\' );

    if ( s == NULL )

     s = strrchr( FileList[ i ], ':' );

#endif

#ifndef MSDOS /* Must be UNIX */

    s = strrchr( FileList[ i ], '/');

#endif

    if ( s != NULL )

     s++;

    else

     s = FileList[ i ];

    skip = 0;

    for ( j = 0; j < i ; j++ )

     if ( strcmp( s, FileList[ j ] ) == 0 ) {

       fprintf( stderr, "Duplicate file name: %s", FileList[ i ] );

       fprintf( stderr, " Skipping this file...\n" );

       skip = 1;

       break;

      }

    if ( s != FileList[ i ] {

     for ( j = 0 ; s [ j ] != '\0' ; j++ )

      FileList[ i ][ j ] = s[ j ];

     FileList[ i ][ j ] = '\0';

    }

    if ( !skip ) {

      strcpy( Header.file_name, FileList[ i ];

      Insert( input_text_file, "Adding" );

    } else

      fclose( input_text_file );

   }

  return( i );

}

Adding files to the archive has to take care of several things in order to accomplish its goal. First, it has to use the full path name specified on the command line to try to open the file to be added. Once the file is opened, however, the file name needs to be stripped of its path and drive component. CARMAN stores file names only, not leading drive or path definitions. Once the file name has been stripped down, a search is made to be sure that this is not a duplicate file name. Adding two files with the same name to a CARMAN file would be a bad idea. Finally, if everything went well, the Insert() routine is called to compress the file and place it in the output CAR file.

Once any new files have been added, CARMAN enters the main processing loop, where most of the real work takes place. The main processing loop is fairly simple in structure. It simply reads in file header from the input file, then checks to see if the file appears in the FileList, saving the answer. Next, it enters a switch, and then performs processing on the input file based on the command type. Each command makes a different decision on what to do based on whether or not the file name appeared in the file list.

Two commands used in the main processing loop require a small amount of setup. The 'Test' and 'Print' commands actually act just like the 'Xtract' command, except that they direct their output to the null device and stdout, respectively. These output destinations are set up before the loop is entered.

int ProcessAllFilesInInputCar( command, count )

int command;

int count;

{

 int matched;

 FILE *input_text_file;

 FILE *output_destination;

 if ( command == 'P' )

  output_destination = stdout;

 else if ( command == 'T' )

#ifdef MSDOS

   output_destination = fopen( "NUL", "wb" );

#else

   output_destination = fopen( "/dev/null", "wb" );

#endif

 else

  output_destination = NULL;

/*

* This is the loop where it all happens. I read in the header for

* each file in the input CAR, then see if it matches any of the file

* and wild card specifications in the FileList created earlier. That

* information, combined with the command, tells me what I need to

* know in order to process the file. Note that if the 'Addfiles' command

* is being executed, the InputCarFile will be NULL, so this loop

* can be safety skipped.

*/

 while ( InputCarFile != NULL && ReadFileHeader() != 0 ) {

  matched = SearchFileList( Header.file_name );

  switch ( command ) {

   case 'D' :

    if ( matched ) {

     SkipOverFileFromInputCar();

     count++;

    } else

     CopyFileFromInputCar();

    break;

   case 'A' :

    if ( matched )

     SkipOverFileFromInputCar();

    else

     CopyFileFromInputCar();

    break;

   case 'L' :

    if ( matched ) {

     ListCarFileEntry();

     count++;

    }

    SkipOverFileFromInputCar();

    break;

   case 'P' :

   case 'X' :

   case 'T' :

    if ( matched ) {

     Extract( output_destination );

     count++;

    } else

     SkipOverFileFromInputCar();

    break;

   case 'R' :

    if ( matched ) {

     input_text_file = fopen( Header.file_name, "rb" );

     if ( input_text_file == NULL ) {

      fprintf( stderr, "Could not find %s", Header.file_name );

      fprintf( stderr, " for replacement, skipping\n" );

      CopyFileFromInputCar();

     } else {

      SkipOverFileFromInputCar();

      Insert( input_text_file, Replacing" );

      count++;

      fclose( input_text_file );

     }

    } else

     CopyFileFromInputCar();

    break;

   }

 }

 return( count );

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The processing loop starts off by reading in the next available header file from the input CAR file. If we are at the end of file, this operation returns a 0 and we exit. Otherwise, we call SearchFileList(), which looks for a match of the file name in the FileList[] array, including wild card matches. The result of that search is stored in the match variable, at which point the switch statement is started. The actions taken in the switch depend on the command give on the command line:

Delete: | If match is true, it means the user wants to delete this file from the CAR archive. In this case, the file is skipped over with a call to SkipOverFileFromInputCar(). Otherwise, the file is copied to the output CAR file with a call to CopyFileFromInputCar().
Add: | If match is true, means that one of the files that was added to the output CAR file at the start of the program also appears in input CAR file. When this is the case, we have to skip over the file, since it has been superseded. If no match is found, the file is copied to the output CAR file.
List: | If a match is found, the file statistics are listed on stdout. No output file is being created by this command, so after it is listed, it is automatically skipped.
Print: Test: Xtract: | If a match is found, it means this file has to be extracted to one of the possible destinations. For the Print command, it goes to stdout. For Test, it goes to the null device, and for Xtract, to a file that is created with the appropriate name. If no match was found, the file is skipped.
Replace: | If a match is found, it means we need to replace the version of the file found in the archive with a file of the same name in the current directory. If that file is found, it is Inserted into the output file, and the current input file is skipped. If no match is found, or the file cannot be opened, the file in the input CAR file is copied to the output CAR file.

Once all of these operations are complete, a count of matched files is returned to the calling routine, for display when the program exits.

Skipping/Copying Input File

The main processing loop only has one of three choices to take to go past the current file in the input CAR file. The first two are the skip and copy routines. One of these copies the current file in the input CAR to the output CAR file. The second routine skips over the file and moves on to the next header.

The skip operation was discussed previously, and is quite simple, since we have the exact size in bytes of the compressed file stored in the header. All the program has to do is advance that number of bytes forward in the input file. Once this is done, the file is lost to the output file, so this is only done when the file is to be Deleted or Replaced (including replacement with an Add command).

void SkipOverFileFromInputCar()

{

   fseek( InputCarFile, Header.compressed_size, SEEK_CUR );

}

void CopyfileFromInputCar()

{

 char buffer[ 256 ];

 unsigned int count;

 WriteFileHeader();

 while ( Header.compressed_size != 0 ) {

  if ( Header.compressed_size < 256 )

   count = (int) Header.compressed_size;

  else

   count = 256;

  if ( fread( buffer, 1, count, InputCarFile ) != count )

   FatalError( "Error reading input file %s", Header.file_name );

  Header.compressed_size -= count;

  if ( fwrite( buffer, 1, count, OutputCarFile) != count )

   FatalError( "Error writing to output CAR file" );

 }

}

Copying the file from the input CAR file to the output CAR file is the "normal" mode of operation, where the contents of the input file are not lost. This is only marginally more complicated than the skip routine. All we need to do here is read in the predetermined number of bytes a block at a time, and write them out to the output file, checking for errors along the way.

Once the copy is complete, the input file pointer is left pointing at the next file header in the input CAR file, and the program is ready to start back at the top of the loop.

File Insertion

The Insertion routine is called to insert an external file into the output CAR file. The insertion routine makes a first attempt to compress the file using the LZSS compression routine. If that routine fails, a straight storage routine is called instead. Since we don't know what the size of the compressed file will be until after the compression actually takes place, Insert() has to back up and rewrite the header after the compression is finally successful. In addition, the compression method is stored in the header file as well. A compression method of 1 is used for normal storage, 2 for LZSS compression. Clearly it would be relatively simple to add new forms of compression by adding new numbers to the table. All that would be needed then is additional code in the Extract() routine to support the new compression method.

void Insert( input_text_file, operation )

FILE *input_text_file;

char *operation;

{

 long saved_position_of_header;

 long saved_position_of_file;

 fprintf( stderr, "%s %-20s", operation, Header.file_name );

 saved_position_of_header = ftell( OutputCarFile );

 Header.compression_method = 2;

 WriteFileHeader();

 saved_position_of_file = ftell(OutputCarFile);

 fseek( input_text_file, OL, SEEK_END );

 Header.original_size = ftell( input_text_file );

 fseek( input_text_file, OL, SEEK_SET );

 if ( !LZSSCompress( input_text_file ) ) {

  Header.compression_method = 1;

  fseek( OutputCarFile, saved_position_of_file, SEEK_SET );

  rewind( input_text_file );

  Store( input_text_file );

 }

 fclose( input_text_file );

 fseek( OutputCarFile, saved_position_of_header, SEEK_SET );

 WriteFileHeader();

 fseek( OutputCarFile, OL, SEEK_END );

 printf( " %d%%\n", RatioInPercent( Header.compressed_size,

                     Header.original_size ) );

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

File Extraction

The extraction routine in some ways is simpler than the Insert() routine. It doesn't have to deal with the possibility that the LZSS compression routine failed to compress. Instead, it just calls the appropriate routine based on the compression method stored in the header file. However, it does have a few extra jobs to deal with.

First of all, Extract can be called with a predefined destination FILE pointer. This occurs when the Print or Test commands are being executed. Print just extracts to stdout, and Test extracts to the null device, or the "bit bucket". When this is the case, Extract() doesn't have to open a file to store the output.

In the case where Extract() is being called based on the Xtract command, it has to open the output file, check to make sure that goes okay, then close the file after the expansion takes place.

In all cases, Extract() has to check the CRC of the output file after the expansion routine has completed. When using the Test command, this is the way CARMAN verifies the integrity of the CAR file.

void Extract( destination )

FILE *destination;

{

 FILE *output_text_file;

 unsigned long crc;

  int error;

  fprintf( stderr, "%-20s ", Header.file_name );

  error = 0;

  if ( destination == NULL ) {

   if ( ( output_text_file = fopen(Header.file_name, "wb")

                            ) == NULL ) {

    fprintf( stderr, "Can't open %s\n", Header.file_name );

    fprintf( stderr, "Not extracted\n" );

    SkipOverFileFromInputCar();

    return;

   }

  } else

   output_text_file = destination;

  switch( Header.compression_method ) {

   case 1 :

    crc = Unstore( output_text_file );

    break;

   case 2 :

    crc = LZSSExpand( output_text_file );

    break;

   default :

    fprintf( stderr, "Unknown method: %c\n",

         Header.compression_method );

    SkipOverFileFromInputCar();

    error = 1;

    crc = Header.original_crc;

    break;

  }

  if ( crc != Header.original_crc ) {

    fprintf( stderr, "CRC error reading data\n" );

    error = 1;

  }

  if ( destination == NULL ) {

   fclose( output_text_file );

   if ( error )

  #ifdef __STDC__

        remove( Header.file_name );

  #else

        unlink( Header.file_name );

  #endif

  }

  if ( !error )

   fprintf (stderr, " OK\n" );

}

Cleanup

The final job left to CARMAN after making its way through the main processing loop is to clean up the workspace used by the program. The first step is to write out the special EOF header to the output CAR file. This is done using a dedicated routine called WriteEndOfCarHeader(), which simply writes a zero length file name to the header file.

Next, the output CAR file is closed and checked for errors. At this point, the output CAR file has been completely processed and is ready to replace the input file. In order to do this, the input file is deleted, and the output file is renamed to have the original archive name. This takes slightly different code under UNIX than MS-DOS, but it is relatively straightforward.

The Code

A complete listing of the CARMAN program follows, including sections that have only been lightly touched on in this chapter. The LZSS compression code is nearly identical to that shown earlier in Chapter 8, with a slightly modified I/O system.

Programmers wishing to compile this under MS-DOS are advised to pay close attention to those portions of the code that are surrounded by #ifdef MSDOS sections. These portions may need slight modifications to work with different MSDOS compilers, but the modifications should only consist of renamed functions and structures. The actual flow of control inside the program should be identical.

/**************************** Start of CARMAN.C *************************

*

* This is the main program for the simple Compressed Archive Manager.

* This program can be used to add, delete, extract, or list the files

* in a CAR archive. The code here should run under standard ANSI

* compilers under MS-DOS (with ANSI mode selected) or K&R compilers

* under UNIX. The code uses an LZSS compression algorithm identical to

* that used earlier in the book.

*/

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <ctype.h>

#ifdef __STDC__

#include <stdarg.h>

#else

#include <varargs.h>

#endif

#ifdef __STDC__

 /* All Borland C/C++ versions */

 #ifdef __TURBOC__

   #define MSDOS 1

   #include <io.h>

   #include <dir.h>

   #define DIR_STRUCT struct ffblk

   #define FIND_FIRST(n, d, a ) findfirst( n, d, a )

   #define FIND_NEXT findnext

   #define DIR_FILE_NAME ff_name

#endif

 /*Microsoft, Watcom, Zortech */

#if defined( M__186 ) || defined ( __ZTC__ ) || defined ( __TSC__ )

 #define MSDOS 1

 #include <dos.h>

 #define DIR_STRUCT struct find_t

 #define FIND_FIRST( n, d, a) _dos_findfirst( n, a, d )

 #define FIND_NEXT _dos_findnext

 #define DIR_FILE_NAME name

 #endif

#endif

/*

* A few constants used throughout the program.

*/

#define BASE_HEADER_SIZE 19

#define CRC_MASK 0xFFFFFFFFL

#define CRC32_POLYNOMIAL 0xEDB88320L

/*

* The only data structure used inside the CAR file is the header block.

* Each file is preceded by a header, stored in a portable format.

* The header is read into and out of the structure defined below.

* The CAR file is structured as a series of header/data sequences, with

* the EOF being denoted as a header with a file name length of 0. Note

* that the length of each header will vary depending on the length of

* the file name.

*/

#ifndef FILENAME_MAX

#define FILENAME_MAX 128

#endif

typedef struct header {

 char file_name[ FILENAME_MAX ];

 char compression_method;

 unsigned long original_size;

 unsigned long compressed_size;

 unsigned long original_crc;

 unsigned long header_crc;

} HEADER;

/*

* Local function prototypes

*/

#ifdef __STDC__

void FatalError( char *message, ... );

void BuildCRCTable( void );

unsigned long CalculateBlockCRC32( unsigned int count, unsigned long crc,

                          void *buffer );

unsigned long UpdateCharacterCRC32( unsigned long crc, int c );

int ParseArguments( int argc, char *argv[] );

void UsageExit( void );

void OpenArchiveFiles( char *name, int command );

void BuildFileList( int argc, char *argv[], int command );

int ExpandAndMassageMSDOSFileNames( int count, char *wild_name );

void MassageMSDOSFileName( int count, char *file );

int AddFileListToArchive( void );

int ProcessAllFilesInInputCar( int command, int count );

int SearchFileList( char *file_name );

int WildCardMatch( char *s1, char *s2 );

void SkipOverFileFromInputCar( void );

void CopyFileFromInputCar( void );

void PrintListTitles( void );

void ListCarFileEntry( void );

int RatioInPercent( unsigned long compressed, unsigned long original );

int ReadFileHeader( void );

unsigned long UnpackUnsignedData( int number_of_bytes,

                      unsigned char *buffer );

void WriteFileHeader( void );

void PackUnsignedData( int number_of_bytes, unsigned long number,

                      unsigned char *buffer );

void WriteEndOfCarHeader( void );

void Insert( FILE *input_text_file, char *operation );

void Extract( FILE *destination );

int Store( FILE *input_text_file );

unsigned long Unstore( FILE *destination );

int LZSSCompress( FILE *input_text_file );

unsigned long LZSSExpand( FILE *destination );

#else

void FatalError();

void BuildCRCTable();

unsigned long CalculateBlockCRC32();

unsigned long UpdateCharacterCRC32();

int ParseArguments();

void UsageExit();

void OpenArchiveFiles();

void BuildFileList();

int ExpandAndMassageMSDOSFileNames();

void MassageMSDOSFileName();

int AddFileListToArchive();

int ProcessAllFilesInInputCar();

int SearchFileList();

int WildCardMatch();

void SkipOverFileFromInputCar();

void CopyFileFromInputCar();

void PrintListTitles();

void ListCarFileEntry();

int RatioInPercent();

int ReadFileHeader();

unsigned long UnpackUnsignedData();

void WriteFileHeader();

void PackUnsignedData();

void WriteEndOfCarHeader();

void Insert();

void Extract();

int Store();

unsigned long Unstore();

int LZSSCompress();

unsigned long LZSSExpand();

#endif

/*

* All global variables are defined here.

*/

char *TempFileName[ FILENAME_MAX ]; /* The output archive is first    */

                                    /* opened with a temporary name   */

FILE * InputCarFile;                /* The input CAR file. This file  */

                                    /* may not exist for 'A' commands */

char CarFileName[ FILENAME_MAX ];   /* Name of the CAR file, defined  */

                                    /* on the command line            */

FILE *OutputCarFile;                /* The output CAR, only exists for*/

                                    /* the 'A' and 'R' operations     */

HEADER Header;                      /* The Header block for the file  */

                                    /* presently being operated on    */

char *FileList[ 100 ];              /* The list of file names passed  */

                                    /* on the command line            */

unsigned long Ccitt32Table[ 256 ];  /* This array holds the CRC       */

                                    /* table used to calculate the 32 */

                                    /* bit CRC values                 */

/*

* This is the main routine for processing CAR commands. Most of the

* major work involved here has been delegated to other functions.

* This routine first parses the command line, then opens up the input

* and possibly the output archive. It then builds a list of files

* to be processed by the current command. If the command was 'A', all

* of the files are immediately added to the output archives. Finally,

* the main processing loop is called. It scans through the entire

* archive, taking action on each file as necessary. Once that is

* complete, all that is left to do is optionally delete the input file,

* then rename the output file to have the correct CAR file name.

*/

int main( argc, argv )

int argc;

char *argv[];

{

 int command;

 int count;

 setbuf( stdout, NULL );

 setbuf( stderr, NULL ):

 fprintf( stderr, "CARMAN 1.0 : " );

 BuildCRCTable();

 command = ParseArguments( argc, argv );

 fprintf( stderr, "\n" );

 OpenArchiveFiles( argv[ 2 ], command );

 BuildFileList( argc - 3, argv + 3, command );

 if ( command == 'A' )

  count = AddFileListToArchive();

 else

  count = 0;

 if ( command == 'L' )

  PrintListTitles();

 count = ProcessAllFilesInInputCar( command, count );

 if ( OutputCarFile != NULL && count != 0 ) {

  WriteEndOfCarHeader();

  if ( ferror( OutputCarFile ) || fclose( OutputCarFile ) == EOF )

   FatalError( "Can't write" );

  #ifdef __STDC__

   remove( CarFileName );

   rename(TempFileName, CarFileName );

  #else

   unlink( CarFileName );

   link( TempFileName, CarFileName );

   unlink( TempFileName );

  #endif

 }

 if ( command != 'P' )

  printf( "\n%d file%s\n", count, ( count == 1 ) ? "" : "s" );

  else

  fprintf( stderr, "\n%d file%s\n", count,

                 ( count == 1 ) ? '' : "s" );

  return( 0 );

}

/*

* FatalError provides a short way for us to exit the program when

* something bad happens, as well as printing a diagnostic message.

* If an output CAR file has been opened, it is deleted as well,

* which cleans up most of the traces of our work here. Note that

* K&R compilers handle variable length argument lists differently

* than ANSI compilers, so we have two different entries for the

* routines.

*/

#ifdef __STDC__

void FatalError( char *fmt, ... )

{

 va_list args;

 va_start( args, fmt );

#else

void FatalError( va_alist )

va_dcl

{

 va_list args;

 char *fmt;

 va_start( args );

 fmt = va_arg( args, char * );

#endif

 putc( '\n', stderr );

 vfprintf( stderr, fmt, args );

 putc( '\n', stderr );

 va_end( args );

 if ( OutputCarFile != NULL )

  fclose( OutputCarFile );

#ifdef __STDC__

    remove( TempFileName );

#else

  unlink( TempFileName );

#endif

 exit( 1 );

}

/*

* This routine simply builds the coefficient table used to calculate

* 32-bit CRC values throughout this program. The 256-long word table

* has to be set up once when the program starts. Alternatively, the

* values could be hard-coded in, which would offer a miniscule

* improvement in overall performance of the program.

*/

void BuildCRCTable()

{

 int i;

 int j;

 unsigned long value;

 for ( i = 0; i <= 255 ; i++ ) {

  value = i;

  for ( j = 8 ; j > 0; j-- ) {

   if ( value & 1 )

    value = ( value >> 1 ) ^ CRC32_POLYNOMIAL;

   else

    value >>= 1;

  }

  Ccitt32Table[ i ] = value;

 }

}

/*

* This is the routine used to calculate the 32-bit CRC of a block of

* data. This is done by processing the input buffer using the

* coefficient table that was created when the program was initialized.

* This routine takes an input value as a seed, so that a running

* calculation of the CRC can be used as blocks are read and written.

*/

unsigned long CalculateBlockCRC32( count, crc, buffer )

unsigned int count;

unsigned long crc;

void *buffer;

{

 unsigned char *p = (unsigned char *) buffer;

 unsigned long temp1;

 unsigned long temp2;

 while ( count-- != 0 ) {

  temp1 = ( crc >> 8 ) & 0x00FFFFFFL;

  temp2 = Ccitt32Table[ ( (int) crc ^ *p++ ) & 0xff ];

  crc = temp1 ^ temp2;

 }

 return( crc );

}

/*

* If I/0 is being done on a byte-by-byte basis, as is the case with the

* LZSS code, it is easier to calculate the CRC of a byte at a time

* instead of a block at a time. This routine performs that function,

* once again taking a CRC value as input, so that this can be used to

* perform on the fly calculations. In situations where performance is

* critical, this routine could easily be recorded as a macro.

*/

unsigned long UpdateCharacterCRC32( crc, c )

unsigned long crc;

int c;

{

 unsigned long temp1;

 unsigned long temp2;

 temp1 = ( crc >> 8 ) & 0x00FFFFFFL;

 temp2 = Ccitt32Table[ ( (int) crc ^ c ) & 0xff ];

 crc = temp1 ^ temp2;

 return( crc );

}

/*

* When CARMAN first starts up, it calls this routine to parse the

* command line. We look for several things here. If any of the

* conditions needed to run CARMAN is not met, the routine opts for

* the usage printout exit. The first thing to be sure of is that

* the command line has at least three arguments, which should be

* the "CARMAN", a single character command, and an CAR archive name.

* After that, we check to be sure that the command name is a valid

* letter, and incidentally print out a short message based on it.

* Both the Addfiles and Delete commands require that some file names

* be listed as well, so a check is made for additional arguments when

* each of those arguments is encountered. Finally, the command itself

* is returned to main(), for use later in processing the command.

*/

int ParseArguments( argc, argv )

int argc;

char *argv[];

{

 int command;

 if ( argc < 3 || strlen( argv[ 1 ] ) > 1 )

  UsageExit();

 switch( command = toupper( argv[ 1 ][ 0 ] ) ) {

  case 'X' :

   fprintf( stderr, "Extracting files\n";

   break;

  case 'R' :

   fprintf( stderr, "Replacing files\n" );

   break;

  case 'P' :

   fprintf( stderr, "Print files to stdout\n" );

   break;

  case 'T' :

   fprintf( stderr, "Testing integrity of files\n" );

   break;

  case 'L' :

   fprintf( stderr, "Listing archive contents\n" );

   break;

  case 'A' :

   if ( argc <= 3 )

    UsageExit();

   fprintf( stderr, "Adding/replacing files to archive\n" );

   break;

  case 'D' :

   if ( argc <= 3 )

    UsageExit();

    fprintf( stderr, "Deleting files from archive\n" )'

    break;

   default :

   UsageExit();

  };

  return( command );

}

/*

* UsageExit just provides a universal point of egress for those

* times when there appears to be a problem on the command line.

* This routine prints a short help message then exits back to the OS.

*/

void UsageExit()

{

 fputs( "CARMAN - Compressed ARchive MANager\n", stderr );

 fputs( "Usage: carman command car-file [file ...]\n", stderr );

 fputs( "Commands:\n", stderr );

 fputs( " a: Add files to a CAR archive (replace if present)\n",

                                                    stderr );

 fputs( " x: Extract files from a CAR archive\n", stderr );

 fputs( " r: Replace files in a CAR archive\n", stderr );

 fputs( " d: Delete files from a CAR archive\n", stderr );

 fputs( " p: Print files on standard output\n", stderr );

 fputs( " l: List contents of a CAR archive\n", stderr );

 fputs( " t: Test files in a CAR archive\n", stderr );

 fputs( "\n", stderr );

 exit( 1 );

}

/*

* After the command line has been parsed, main() has enough information

* to intelligently open the input and output CAR archive files. The

* name should have been specified on the command line, and passed to

* this routine by main(). As a convenience to the user, if the CAR

* suffix is left off the archive, this routine will add it on.

* There is one legitimate excuse for not being able to open the input

* file, which is if this is the 'Addfiles' command. There may not be

* an input archive when that command is called, in which case a failure

* is tolerated. Once the input file has been opened, an output file

* may have to be opened as well. The 'Addfiles', 'Delete', and

* 'Replace' commands all modify the CAR archive, which means the input

* CAR file is going to be processed and copied to the output. Initially,

* the output CAR file gets a temporary name. It will be renamed later

* after the input has been processed.

*

* Since we will probably be doing lots of bulk copies from the input

* CAR file to the output CAR file, it makes sense to allocate big

* buffers for the files. This is done with the two calls to setvbuf()

* right before the routine exits.

*

*/

void OpenArchiveFiles( name, command )

char *name;

int command;

{

 char *s;

 int i;

 strncpy( CarFileName, name, FILENAME_MAX - 1 );

 CarFileName[ FILENAME_MAX - 1 ] = '\0';

 InputCarFile = fopen( CarFileName, "rb" );

 if ( InputCarFile == NULL ) {

#ifdef MSDOS

    s = strrchr( CarFileName, '\\' );

#else /* UNIX */

    s = strrchr( CarFileName, '/' );

#endif

   if ( s == NULL )

    s = CarFileName;

   if ( strrchr( s, '.' ) == NULL )

    if ( strlen( CarFileName ) < ( FILENAME_MAX - 4 ) ) {

     strcat( CarFileName, ".car" );

     InputCarFile = fopen( CarFileName, "rb" );

   }

 }

 if ( InputCarFile == NULL && command != 'A' )

  FatalError( "Can't open archive '%s'", CarFileName );

 if ( command == 'A' || command == 'R' || command == 'D' ) {

  strcpy( TempFileName, CarFileName );

  s = strrchr( TempFileName, '.');

  if ( s == NULL )

   s = TempFileName + strlen( TempFileName );

  for ( i = 0 ; i < 10 ; i++ ) {

   sprintf( s, ".$$%d", i );

   if ( ( OutputCarFile = fopen( TempFileName, "r" ) ) == NULL )

    break;

   fclose( OutputCarFile );

   OutputCarFile = NULL;

  }

   if ( i == 10 )

    FatalError( "Can't open temporary file %s", TempFileName );

   OutputCarFile = fopen( TempFileName, "wb" );

   if ( OutputCarFile == NULL )

    FatalError( "Can't open temporary file %s", TempFileName );

  }

  if ( InputCarFile != NULL )

   setvbuf( InputCarFile, NULL, _IOFBF, 8192 );

  if ( OutputCarFile != NULL )

   setvbuf( OutputCarFile, NULL, _IOFBF, 8192 );

}

/*

* Most of the commands given here take one or more file names as

* arguments. The list of files given on the command line needs to be

* processed here and put into a list that can easily be manipulated by

* other parts of the program. That processing is done here. An array

* called FileList is created, which will have a series of pointers to

* file names. If no file names were listed on the command line, which

* could be the case for commands like 'List' or 'Extract', a single

* file name of '*' is put on the start of the list. Since '*' is the

* ultimate wild card, matching everything, we don't have to have special

* processing anywhere else for an empty file list. The file names here

* are also massaged a bit further for MS-DOS file names. Under MS-DOS,

* case is not significant in file names. This means that CARMAN

* shouldn't get confused by thinking 'foo.c' and 'FOO.C' are two

* different files. To avoid this, all MS-DOS file names are converted

* here to lower case. Additionally, any file name without an extension

* is forced to end with a period, for similar reasons. This ensures that

* CARMAN knows 'FOO' and 'FOO.' are the same file. Note that I don't

* want to do this for wild card specifications. Finally, there is the

* problem of MS-DOS wild card file names. When using the 'Add' command,

* wild cards on the command line need to be expanded into real file

* names, then undergo the additional processing mentioned earlier. This

* is done with a call to a function that is MS-DOS specific. None of

* this special processing is done under UNIX, where case is significant,

* and wild cards are expanded by the shell.

*/

void BuildFileList( argc, argv, command )

int argc;

char *argv[];

int command;

{

 int i;

 int count;

 count = 0;

 if ( argc == 0 )

  FileList[ count++ ] = "*";

 else {

  for ( i = 0 ; i < argc ; i++ ) {

#ifdef MSDOS

    if ( command == 'A' )

     count = ExpandAndMassageMSDOSFileNames( count, argv[ i ] );

    else

     MassageMSDOSFileName( count++, argv[ i ] );

#endif

#ifndef__MSDOS__

    FileList [ count ] = malloc( strlen( argv[ i ] ) + 2 );

    if ( FileList[ count ] == NULL )

     FatalError( "Ran out of memory storing file names" );

    strcpy( FileList[ count++ ], argv[ i ] );

#endif

    if ( count > 99 )

     FatalError( "Too many file names" );

  }

 }

 FileList[ count ] = NULL;

}

/*

* Under MS-DOS, wildcards on the command line are not expanded to

* a list of file names, so it is up to application programs to do the

* expansion themselves. This routine takes care of that, by using

* the findfirst and findnext routines. Unfortunately, each MS-DOS

* compiler maker has implemented this function slightly differently, so

* this may need to be modified for your particular compiler. However,

* this routine can be replaced with a call to MassageMSDOSFileName(),

* and the program will work just fine, without the ability to handle

* wild card file names.

*/

#ifdef MSDOS

#include <dos.h>

#include <dir.h>

int ExpandAndMassageMSDOSFileNames( count, wild_name )

int count;

char *wild_name;

{

 int done;

 DIR_STRUCT file_info_block;

 char *leading_path;

 char *file_name;

 char *p;

 leading_path = malloc( strlen( wild_name ) + 1 );

 file_name = malloc( strlen( wild_name ) + 13 );

 if ( leading_path == NULL || file_name == NULL )

  FatalError( "Ran out of memory storing file names" );

 strcpy( leading_path, wild_name );

 p = strrchr( leading_path, '\\' );

 if ( p != NULL )

  p[ 1 ] = '\0';

 else {

  p = strrchr( leading_path, ';' );

  if ( p != NULL )

   p[ 1 ] = '\0';

  else

   leading_path[ 0 ] = '\0';

 }

 done = FIND_FIRST( wild_name, & file_info_block, 0 );

 while ( !done ) {

  strcpy( file_name, leading_path );

  strcat( file_name, file_info_block.DIR_FILE_NAME );

  MassageMSDOSFileName( count++, file_name );

  done = FIND_NEXT( & file_info_block );

  if ( count > 99 )

   FatalError( "Too many file names" );

 }

 free( leading_path );

 free( file_name );

 return( count );

}

/*

* As was discussed earlier, this routine is called to perform a small

* amount of normalization on file names. Under MS_DOS, case is not

* significant in file names. In order to avoid confusion later, we force

* all file names to be all lower case, so we can't accidentally add two

* files with the same name to a CAR archive. Likewise, we need to

* prevent confusion between files that end in a period, and the same

* file without the terminal period. We fix this by always forcing the

* file name to end in a period.

*/

void MassageMSDOSFileName( count, file )

int count;

char *file;

{

 int i;

 char *p;

 FileList[ count ] = malloc( strlen( file ) + 2 );

 if ( FileList[ count ] == NULL )

  FatalError( "Ran out of memory storing file names" );

 strcpy( FileList[ count ], file );

 for ( i = 0 ; FileList[ count ] [ i ] != '\0' ; i++ )

  FileList[ count ][ i ] = (char)

       tolower(FileList[ count ][ i ]);

 if ( strpbrk( FileList [ count ], "*?" ) == NULL ) {

  p = strrchr( FileList[ count ], '\\' );

  if ( p == NULL )

   p = FileList[ count ];

  if ( strrchr( p, '.' ) == NULL )

   strcat( FileList[ count ], "." );

 }

}

#endif

/*

* Once all of the argument processing is done, the main() procedure

* checks to see if the command is 'Addfiles'. If it is, it calls

* this procedure to add all of the listed files to the output buffer

* before any other processing is done. That is taken care of right

* here. This routine basically does three jobs before calling the

* Insert() routine, where the compression actually takes place. First,

* it tries to open the file, which ought to work. Second, it strips the

* leading drive and path information from the file, since we don't keep

* that information in the archive. Finally, it checks to see if the

* resulting name is one that has already been added to the archive.

* If it has, the file is skipped so that we don't end up with an

* invalid archive.

*/

int AddFileListToArchive()

{

 int i;

 int j;

 int skip;

 char *s;

 FILE *input_text_file;

 for ( i = 0 ; FileList[ i ] != NULL ; i++ ) {

  input_text_file = fopen( FileList[ i ], "rb" );

   if ( input_text_file == NULL )

    FatalError( "Could not open %s to add to CAR file",

       FileList[ i ] );

#ifdef MSDOS

   s = strrchr( FileList[ i ], '\\' );

   if ( s == NULL )

    s = strrchr( FileList[ i ], ':' );

#endif

#ifndef MSDOS /* Must be UNIX */

   s = strrchr( FileList[ i ], '/' );

#endif

   if ( s != NULL )

    s++;

   else

    s = FileList[ i ];

   skip = 0;

   for ( j = 0 ; j < i ; j++ )

    if ( strcmp( s, FileList[ j ] ) == 0 ) {

     fprintf( stderr, "Duplicate file name: %s", FileList[ i ] );

     fprintf( stderr, " Skipping this file...\n" );

     skip = 1;

     break;

    }

   if (s != FileList[ i ] ) {

    for ( j = 0 ; s[ j ] != '\0' ; j++ )

     FileList[ i ][ j ] = s[ j ];

    FileList[ i ][ j ] = '\0';

   }

   if ( !skip ) {

    strcpy( Header.file_name, FileList[ i ] );

    Insert( input_text_file, "Adding" );

   } else

    fclose( input_text_file );

 }

 return( i );

}

/*

* This is the main loop where all the serious work done by this

* program takes place. Essentially, this routine starts at the

* beginning of the input CAR file, and processes every file in

* the CAR. Depending on what command is being executed, that might

* mean expanding the file, copying it to standard output,

* adding it to the output CAR, or skipping over it completely.

*/

int ProcessAllFilesInInputCar( command, count )

int command;

int count;

{

 int matched;

 FILE *input_text_file;

 FILE *output_destination;

 if ( command == 'P' )

  output_destination = stdout;

 else if ( command == 'T' )

#ifdef MSDOS

   output_destination = fopen( "NUL", "wb" );

#else

   output_destination = fopen( "/dev/null", "wb" );

#endif

 else

   output_destination = NULL;

/*

* This is the loop where it all happens. I read in the header for

* each file in the input CAR, then see if it matches any of the file

* and wildcard specifications in the FileList created earlier. That

* information, combined with the command, tells me what I need to

* know in order to process the file. Note that if the 'Addfiles' command

* is being executed, the InputCarFile will be NULL, so this loop

* can be safely skipped.

*/

 while ( InputCarFile != NULL && ReadFileHeader() != 0 ) {

 matched = SearchFileList( Header.file_name );

 switch ( command ) {

  case 'D' :

   if ( matched ) {

    SkipOverFileFromInputCar();

    count++;

   } else

    CopyFileFromInputCar();

   break;

  case 'A' :

   if ( matched )

    SkipOverFileFromInputCar();

   else

    CopyFileFromInputCar();

   break;

  case 'L' :

   if ( matched ) {

    ListCarFileEntry();

    count++;

   } else

    SkipOverFileFromInputCar();

   break;

  case 'P' :

  case 'X' :

  case 'T' :

   if ( matched ) {

    Extract( output_destination );

    count++;

   } else

    SkipOverFileFromInputCar();

   break;

  case 'R' :

   if ( matched ) {

    input_text_file = fopen( Header.file_name, "rb" );

    if ( input_text_file == NULL ) {

     fprintf( stderr, "Could not find %s", Header.file_name );

     fprintf( stderr, " for replacement, skipping\n" );

     CopyFileFromInputCar();

    } else {

     SkipOverFileFromInputCar();

     Insert( input_text_file, "Replacing" );

     count++;

     fclose(input_text_file );

    }

   } else

    CopyFileFromInputCar();

   break;

  }

 }

 return( count );

}

/*

* This routine looks through the entire list of arguments to see if

* there is a match with the file name currently in the header. As each

* new file in InputCarFile is encountered in the main processing loop,

* this routine is called to determine if it has an appearance anywhere

* in the FileList[] array. The results is used to in the main loop

* to determine what action to take. For example, if the command were

* the 'Delete' command, the match result would determine whether to

* copy the file form the InputCarFile to the OutputCarFile, or skip

* over it.

*

* The actual work in this routine is really performed by the

* WildCardMatch() routine which checks the file name against one of the

* names in the FileList[] array. Since most of the commands can use

* wild cards to specify file names inside the CAR file, we need a

* special comparison routine.

*/

int SearchFileList( file_name )

char *file_name;

{

  int i;

  for ( i = 0 ; FileList[ i ] != NULL ; i++ ) {

   if ( WildCardMatch( file_name, FileList[ i ] ) )

    return( 1 );

  }

  return( 0 );

}

/*

* WildCardMatch() compares string to wild_string, looking for a match.

* Wild card characters supported are only '*' and '?', where '*'

* represents a string of any length, including 0, and '?' represents any

* single character.

*/

int WildCardMatch( string, wild_string )

char *string;

char *wild_string;

{

 for ( ; ; ) {

  if ( *wild_string == '*' ) {

   wild_string++;

   for ( ; ; ) {

    while ( *string != '\0' && *string != *wild_string )

      string++;

    if ( WildCardMatch( string, wild_string ) )

     return( 1 );

    else if ( *string == '\0' )

     return( 0 );

    else

     string++;

   }

 } else if ( *wild_string == '?' ) {

   wild_string++;

   if ( * string++ == '\0' )

     return( 0 );

   } else {

    if ( * string != *wild_string )

     return( 0 );

    if ( *string == '\0' )

     return( 1 );

    string++;

    wild_string++;

   }

  }

}

/*

* When the main processing loop reads in a header, it checks to see

* if it is going to copy that file either to the OutputCarFile or expand

* it. If neither is going to happen, we need to skip past this file and

* go on to the next header. This can be done by seeking past the

* compressed file. Since the compressed size is stored in the header

* information, it is easy to do. Note that this routine assumes that the

* file pointer has not been modified since the header was read in. This

* means it should be located at the first byte of the compressed data.

*/

void SkipOverFileFromInputCar()

{

   fseek( InputCarFile, Header.compressed_size, SEEK_CUR );

}

/*

* When performing an operation that modifies the input CAR file,

* compressed files will frequently need to be copied from the input CAR

* file to the output CAR file. This routine does that using simple

* repeated block copy operations. Since it is writing directly to the

* output CAR file, the first thing it needs to do is write out the

* current Header so that the CAR file will be properly structured.

* Following that, the compressed file is copied one block at a time to

* the output. When this routine completes, the input file pointer is

* positioned at the next header in the input CAR file, and the output

* file pointer is positioned at the EOF position in the output file.

* This is the proper place for the next record to begin.

*/

void CopyFileFromInputCar()

{

 char buffer[ 256 ];

 int count;

 WriteFileHeader();

 while ( Header.compressed_size != 0 ) {

  if ( Header.compressed_size < 256 )

   count = (int) Header.compressed_size;

  else

   count = 256;

  if ( fread( buffer, 1, count, InputCarFile ) != count )

   FatalError( "Error reading input file %s", Header.file_name );

  Header.compressed_size -= count;

  if ( fwrite( buffer, 1, count, OutputCarfile) != count )

   FatalError( "Error writing to output CAR file" );

 }

}

/*

* When the operation requested by the user is 'List', this routine is

* called to print out the column headers. List output goes to standard

* output, unlike most of the other messages in this program, which go

* to stderr.

*/

void PrintListTitles()

{

 printf( "\n" );

 printf( "         Original Compressed\n" );

 printf( "Filename  Size  Size     Ratio CRC-32 Method\n" );

 printf( "________  ____  ____     _________________\n" );

}

/*

* When the List command is given, the main loop reads in each header

* block, then tests to see if the file name in the header block matches

* one of the file names (including wildcards) in the FileList. If it is,

* this routine is called to print out the information on the file.

*/

void ListCarFileEntry()

{

 static char *methods[] = {

  "Stored",

  "LZSS"

 };

printf( "%-20s %10lu %10lu %4d%% %081x %s\n",

    Header.file_name,

    Header.original_size,

    Header.compressed_size,

    RatioInPercent( Header.compressed_size,

    Header.original_size ),    Header.original_crc,

    methods[ Header.compression_method - 1 ] );

}

/*

* The compression figure used in this book is calculated here. The value

* is scaled so that a file that has just been stored has a compression

* ratio of 0%, while one that has been shrunk down to nothing would have

* a ratio of 100%.

*/

int RatioInPercent( compressed, original )

unsigned long compressed;

unsigned long original;

{

 int result;

 if ( original == 0 )

  return( 0 );

 result = (int) ( ( 100L * compressed ) / original );

 return( 100 - result );

}

/*

* This routine is where all the information about the next file in

* the archive is read in. The data is read into the global Header

* structure. To preserve portability of CAR files across systems,

* the data in each file header is packed into an unsigned char array

* before it is written out to the file. To read this data back in

* to the Header structure, we first read it into another unsigned

* character array, then employ an unpacking routine to convert that

* data into ints and longs. This helps us avoid problems with

* big/little endian conflicts, as well as incompatibilities in structure

* packing, which show up even between different compilers targetted for

* the same architecture.

*

* To avoid causing any additional confusion, the data members for the

* header structure are at least stored in exactly the same order as

* they appear in the structure definition. The primary difference is

* that the entire file name character array is not stored, which would

* waste a lot of space. Instead, we just store the number of characters

* in the name, including the null termination character. The file name

* serves the additional purpose of identifying the end of the CAR file

* with a file name length of 0 bytes.

*/

int ReadFileHeader()

{

 unsigned char header_data[ 17 ];

 unsigned long header_crc;

 int i;

 int c;

 for ( i = 0 ; ; ) {

  c = getc( InputCarFile );

  Header.file_name[ i ] = (char) c;

  if ( c == '\0' )

   break;

  if ( ++i == FILENAME_MAX )

   FatalError ( "File name exceeded maximum in header" );

}

if ( i == 0 )

 return( 0 );

header_crc = CalculateBlockCRC32( i + 1, CRC_MASK, Header.file_name );

fread( header_data, 1, 17, InputCarFile );

Header.compression_method = (char)

              UnpackUnsignedData( 1, header_data + 0 );

Header.original_size    = UnpackUnsignedData( 4, header_data + 1 );

Header.compressed_size  = UnpackUnsignedData( 4, header_data + 5 );

Header.original_crc     = UnpackUnsignedData( 4, header_data + 9 );

Header.header_crc       = UnpackUnsignedData( 4, header_data + 13 );

header_crc = CalculateBlockCRC32(13, header_crc, header_data );

header_crc ^= CRC_MASK;

if ( Header.header_crc != header_crc )

 FatalError( "Header checksum error for file %s", Header.file_name );

return( 1 );

}

/*

* This routine is used to transform packed characters into unsigned

* integers. Its only purpose is to convert packed character data

* into integers and longs.

*/

unsigned long UnpackUnsignedData( number_of_bytes, buffer )

int number_of_bytes;

unsigned char *buffer;

{

 unsigned long result;

 int shift_count;

 result = 0;

 shift_count = 0;

 while ( number_of_bytes-- > ) {

  result |= (unsigned long) * buffer++ << shift_count;

  shift_count += 8;

 }

 return( result );

}

/*

* This routine is called to write out the current Global header block

* to the output CAR file. It employs the same packing mechanism

* discussed earlier. This routine also calculates the CRC of the

* header, which is sometimes not necessary.

*/

void WriteFileHeader()

{

 unsigned char header_data[ 17 ];

 int i;

 for ( i = 0 ; ; ) {

  putc( Header.file_name[ i ], OutputCarFile );

  if ( Header.file_name[ i++ ] == '\0' )

   break;

}

Header.header_crc = CalculateBlockCRC32( i, CRC_MASK,

                     Header.file_name );

PackUnsignedData( 1, (long)

                     Header.compression_method,header_data + 0 );

PackUnsignedData( 4, Header.original_size,    header_data + 1 );

PackUnsignedData( 4, Header.compressed_size,  header_data + 5 );

PackUnsignedData( 4, Header.original_crc,     header_data + 9 );

Header.header_crc = CalculatedBlockCRC32( 13, Header.header_crc,

                                         header_data );

Header.header_crc ^= CRC_MASK;

 PackUnsignedData( 4, Header.header_crc, header_data + 13 );

 fwrite( header_data, 1, 17, OutputCarFile );

}

/*

* This is the routine used to pack integers and longs into a character

* array. The character array is what eventually gets written out to the

* CAR file. The data is always written out with the least significant

* bytes of the integers or long integers going first.

*/

void PackUnsignedData( number_of_bytes, number, buffer )

int number_of_bytes;

unsigned long number;

unsigned char *buffer;

{

 while ( number_of_bytes-- > 0 ) {

  *buffer++ = ( unsigned char ) number & Oxff;

  number >>= 8;

 }

}

/*

* The last header in a CAR file is defined by the fact that it has

* a file name length of zero. Since the file name is the

* first element to be written out, we can create the final header

* by just writing out a null termination character. This technique

* saves a little bit of space.

*/

void WriteEndOfCarHeader()

{

 fputc( 0, OutputCarFile );

}

/*

* This is the routine called by the main processing loop and the

* Addfiles routine. It takes an input file and writes the header and

* file data to the Output CAR file. There are several complications that

* the routine has to deal with. First of all, the header information

* it gets when it first starts is incomplete. For instance, we don't

* know how many bytes the file will take up when it is compressed.

* Because of this, the position of the header is stored, and the

* incomplete copy is written out. After the compression routine finishes,

* the header is now complete. In order to put the correct header into

* the output CAR file, this routine seeks back in the file to the

* original header position and rewrites it.

*

* The second complication lies in the fact that some files are not very

* compressible. In fact, for some files the LZSS algorithm may actually

* cause the file to expand. In these cases, the compression routine

* gives up and passes a failure code back to Insert(). When this

* happens, the routine has to seek back to the start of the file, rewind

* the input file, and store it instead of compressing it. Because of

* this, the starting position of the file in the output CAR file is also

* stored when the routine starts up.

*/

void Insert( input_text_file, operation )

FILE *input_text_file;

char *operation;

{

 long saved_position_of_header;

 long saved_position_of_file;

 fprintf( stderr, "%s %-20s", operation, Header, file_name );

 saved_position_of_header = ftell( OutputCarFile );

 Header.compression_method = 2;

 WriteFileHeader();

 saved_position_of_file = ftell(OutputCarFile);

 fseek( input_text_file, OL, SEEK_END );

 Header.original_size = ftell( input_text_file );

 fseek( input_text_file, OL, SEEK_SET );

 if ( !LZSSCompress( input_text_file ) ) {

  Header.compression_method = 1;

  fseek( OutputCarFile, saved_position_of_file, SEEK_SET );

  rewind( input_text_file );

  Store( input_text_file );

 }

 fclose( input_text_file );

 fseek( OutputCarFile, saved_position_of_header, SEEK_SET );

 WriteFileHeader();

 fseek( OutputCarFile, OL, SEEK_END );

 printf( "%d%%\n", RatioInPercent( Header.compressed_size,

                     Header.original_size ) );

}

/*

* The Extract routine can be called for one of three reasons. If the

* file in the CAR is truly being extracted, Extract() is called with

* no destination specified. In this case, the Extract routine opens the

* file specified in the header and either unstores or decompresses the

* file from the CAR file. If the archive is being tested for veracity,

* the destination file will have been opened up earlier and specified as

* the null device. Finally, the 'Print' option may have been selected,

* in which case the destination file will be extracted to stdout.

*/

void Extract( destination )

FILE *destination;

{

 FILE *output_text_file;

 unsigned long crc;

 int error;

 fprintf( stderr, "%-20s ", Header.file_name );

 error = 0;

 if ( destination == NULL ) {

 if ( ( output_text_file = fopen(Header.file_name, "wb")

                            ) == NULL ) {

  fprintf( stderr, "Can't open %s\n", Header.file_name );

  fprintf( stderr, "Not extracted\n" );

  SkipOverFileFromInputCar();

  return;

 }

} else

 output_text_file = destination;

switch( Header.compression_method ) {

 case 1 :

  crc = Unstore( output_text_file );

  break;

 case 2 :

  crc = LZSSExpand( output_text_file );

  break;

 default :

  fprintf( stderr, "Unknown method: %c\n",

   Header.compression_method );

  SkipOverFileFromInputCar();

  error = 1;

  crc = Header.original_crc;

  break;

 }

 if ( crc != Header.original_crc ) {

  fprintf( stderr, "CRC error reading data\n" );

  error = 1;

 }

 if ( destination == NULL ) {

    fclose( output_text_file );

    if ( error )

#ifdef __STDC__

    remove( Header.file_name );

#else

    unlink( Header.file_name );

#endif

    }

    if ( !error )

    fprintf( stderr, " OK\n" );

}

/*

* The CAR manager program is capable of handling many different forms of

* compression. All the compression program has to do is obey a few

* simple rules. First of all, the compression routine is required

* to calculate the 32-bit CRC of the uncompressed data, and store the

* result in the file Header, so it can be written out by the Insert()

* routine. The expansion routine calculates the CRC of the file it

* creates, and returns it to Extract() for a check against the Header

* value. Second, the compression routine is required to quit if its

* output is going to exceed the length of the input file. It needs to

* quit *before* the output length passes the input, or problems will

* result. The compression routine is required to return a true or false

* value indicating whether or not the compression was a success. And

* finally, the expansion routine is expected to leave the file pointer

* to the Input CAR file positioned at the first byte of the next file

* header. This means it has to read in all the bytes of the compressed

* data, no more or less.

*

* All these things are relatively easy to accomplish for Store() and

* Unstore(), since they do no compression or expansion.

*

*/

int Store( input_text_file )

FILE *input_text_file;

{

 unsigned int n;

 char buffer[ 256 ];

 int pacifier;

 pacifier = 0;

 Header.original_crc = CRC_MASK;

 while ( ( n = fread( buffer, 1, 256, input_text_file ) ) != 0 ) {

  fwrite( buffer, 1, n, OutputCarFile );

  Header.original_crc = CalculateBlockCRC32( n, Header.original_crc,

                                                       buffer );

  if ( ( ++pacifier & 15 ) == 0 )

   putc( '.', stderr );

 }

 Header.compressed_size = Header.original_size;

 Header.original_crc ^= CRC_MASK;

 return( 1 );

}

unsigned long Unstore( destination )

FILE *destination;

{

 unsigned long crc;

 unsigned int count;

 unsigned char buffer[ 256 ];

 int pacifier;

 pacifier = 0;

 crc = CRC_MASK;

 while ( Header.original_size != 0 ) {

  if ( Header.original_size > 256 )

   count = 256;

  else

   count = (int) Header.original_size;

  if ( fread( buffer, 1, count, InputCarFile ) != count )

   FatalError( "Can't read from input CAR file" );

  if ( fwrite( buffer, 1, count, destination ) != count ) {

   fprintf( stderr. "Error writing to output file" );

   return( ~Header.original_crc );

  }

  crc = CalculateBlockCRC32( count, crc, buffer );

  if ( destination != stdout && ( pacifier++ & 15 ) == 0 )

   putc( '.', stderr );

  Header.original_size -= count;

}

 return( crc ^ CRC_MASK );

}

/*

* The second set of compression routines is found here. These

* routines implement LZSS compression and expansion using 12-bit

* index pointers and 4-bit match lengths. These values were

* specifically chosen because they allow for "blocked I/O". Because

* of their values, we can pack match/length pairs into pairs of

* bytes, with characters that don't have matches going into single

* bytes. This helps increase I/O since single bit input and

* output does not have to be employed. Other than this single change,

* this code is identical to the LZSS code used earlier in the book.

*/

/*

* Various constants_used to define the compression parameters. The

* INDEX_BIT_COUNT tells how many bits we allocate to indices into the

* text window. This directly determines the WINDOW_SIZE. The

* LENGTH_BIT_COUNT tells how many bits we allocate for the length of

* an encode phrase. This determines the size of the look ahead buffer.

* The TREE_ROOT is a special node in the tree that always points to

* the root node of the binary phrase tree. END_OF_STREAM is a special

* index used to flag the fact that the file has been completely

* encoded, and there is no more data. UNUSED is the null index for

* the tree. MOD_WINDOW() is a macro used to perform arithmetic on tree

* indices.

*

*/

#define INDEX_BIT_COUNT              12

#define LENGTH_BIT_COUNT             4

#define WINDOW_SIZE                  ( 1 << INDEX_BIT_COUNT )

#define RAW_LOOK_AHEAD_SIZE          ( 1 << LENGTH_BIT_COUNT )

#define BREAK_EVEN         ( ( 1 + INDEX_BIT_COUNT + LENGTH_BIT_COUNT ) \

                                        / 9 )

#define LOOK_AHEAD_SIZE    ( RAW_LOOK_AHEAD_SIZE + BREAK_EVEN )

#define TREE_ROOT                    WINDOW_SIZE

#define END_OF_STREAM                0

#define UNUSED                       0

#define MOD_WINDOW( a )              ( ( a ) & ( WINDOW_SIZE - 1 ) )

/*

* These are the two global data structures used in this program.

* The window[] array is exactly that, the window of previously seen

* text, as well as the current look ahead text. The tree[] structure

* contains the binary tree of all of the strings in the window sorted

* in order.

*/

unsigned char window[ WINDOW_SIZE ];

struct {

 int parent;

 int smaller_child;

 int larger_child;

} tree[ WINDOW_SIZE + 1 ];

/*

* Function prototypes for both ANSI C compilers and their K&R brethren.

*/

#ifdef __STDC__

void InitTree( int r );

void ContractNode( int old_node, int new_node );

void ReplaceNode( int old_node, int new_node );

int FindNextNode( int node );

void DeleteString( int p );

int AddString( int new_node, int *match_position );

void InitOutputBuffer( void );

int FlushOutputBuffer( void );

int OutputChar( int data );

int OutputPair( int position, int length );

void InitInputBuffer( void );

int InputBit( void );

#else

void InitTree();

void ContractNode();

void ReplaceNode();

int FindNextNode();

void DeleteString();

int AddString();

void InitOutputBuffer();

int FlushOutputBuffer();

int OutputChar();

int OutputPair();

void InitInputBuffer();

int InputBit();

#endif

void InitTree( r )

int r;

{

 int i;

 for ( i = 0 ; i < ( WINDOW_SIZE + 1 ) ; i++ ) {

  tree[ i].parent = UNUSED;

  tree[ i].larger_child = UNUSED;

  tree[ i].smaller_child = UNUSED;

 }

 tree[ TREE_ROOT ].larger_child = r;

 tree[ r].parent = TREE_ROOT;

 tree[ r ].larger_child = UNUSED;

 tree[ r ].smaller_child = UNUSED;

}

/*

* This routine is used when a node is being deleted. The link to

* its descendant is broken by pulling the descendant in to overlay

* the existing link.

*/

void ContractNode( old_node, new_node )

int old_node;

int new_node;

{

 tree[ new_node ].parent = tree[ old_node ].parent;

 if ( tree[ tree[ old_node ].parent ].larger_child == old_node )

  tree[ tree[ old_node ].parent ].larger_child = new_node;

 else

  tree[ tree[ old_node ].parent ].smaller_child = new_node;

 tree[ old_node ].parent = UNUSED;

}

/*

* This routine is also used when a node is being deleted. However,

* in this case, it is being replaced by a node that was not previously

* in the tree.

*/

void ReplaceNode( old_node, new_node )

int old_node;

int new_node;

{

 int parent;

 parent = tree[ old_node ].parent;

 if ( tree [ parent ].smaller_child == old_node )

  tree[ parent ].smaller_child = new_node;

 else

  tree[ parent ].larger_child = new_node;

 tree[ new_node ] = tree[ old_node ];

 tree[ tree[ new_node ].smaller_child ].parent = new_node;

 tree[ tree[ new_node ].larger_child ].parent = new_node;

 tree[ old_node ].parent = UNUSED;

}

/*

* This routine is used to find the next smallest node after the node

* argument. It assumes that the node has a smaller child. We find

* the next smallest child by going to the smaller_child node, then

* going to the end of the larger_child descendant chain.

*/

int FindNextNode( node )

int node;

{

 int next;

 next = tree[ node ].smaller_child;

 while ( tree [ next ].larger_child != UNUSED )

  next = tree[ next ].larger_child;

 return( next );

}

/*

* This routine performs the classic binary tree deletion algorithm.

* If the node to be deleted has a null link in either direction, we

* just pull the non-null link up one to replace the existing link.

* If both links exist, we instead delete the next link in order, which

* is guaranteed to have a null link, then replace the node to be deleted

* with the next link.

*/

void DeleteString( p )

int p;

{

 int replacement;

 if ( tree[ p ].parent == UNUSED )

  return;

 if ( tree[ p ].larger_child == UNUSED )

  ContractNode( p, tree [ p ].smaller_child );

 else if (tree[ p ].smaller_child == UNUSED )

  ContractNode( p , tree[ p ].larger_child );

 else {

  replacement = FindNextNode( p );

  DeleteString( replacement );

  ReplaceNode( p , replacement );

 }

}

/*

* This is where most of the work done by the encoder takes place. This

* routine is responsible for adding the new node to the binary tree.

* It also has to find the best match among all the existing nodes in

* the tree, and return that to the calling routine. To make mattes

* even more complicated, if the new_node has a duplicate in the tree,

* the old_node is deleted, for reasons of efficiency.

*/

int AddString( new_node, match_position )

int new_node;

int *match_position;

{

 int i;

 int test_node;

 int delta;

 int match_length;

 int *child;

 if ( new_node == END_OF_STREAM )

  return( 0 ) ;

 test_node = tree[ TREE_ROOT ].larger_child;

 match_length = 0;

 for ( ; ; ) {

  for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

  delta = window[ MOD_WINDOW( new_node + 1 ) ] -

      window[ MOD_WINDOW( test_node + i ) ];

  if ( delta != 0 )

   break;

 }

 if ( i >= match_length ) {

  match_length = i;

  *match_position = test_node;

  if ( match_length >= LOOK_AHEAD_SIZE ) {

   ReplaceNode( test_node, new_node );

   return( match_length );

  }

 }

 if ( delta >= 0 )

  child = &tree[ test_node ].larger_child;

 else

  child = &tree[ test_node ].smaller_child;

 if ( *child == UNUSED ) {

  *child = new_node;

  tree[ new_node].parent = test_node;

  tree[ new_node].larger_child = UNUSED;

  tree[ new_node].smaller_child = UNUSED;

  return( match_length );

  }

  test_node = *child;

 }

}

/*

* This section of code and data makes up the blocked I/O portion of the

* program. Every token output consists of a single flag bit, followed

* by either a single character or a index/length pair. The flag bits

* are stored in the first byte of a buffer array, and the characters

* and index/length pairs are stored sequentially in the remaining

* positions in the array. After every eight output operations, the

* first character of the array is full of flag bits, so the remaining

* bytes stored in the array can be output. This can be done with a

* single fwrite() operation, making for greater efficiency.

*

* All that is needed to implement this is a few routines, plus three

* data objects, which follow below. The buffer has the flag bits

* packed into its first character, with the remainder consisting of

* the characters and index/length pairs, appearing in the order they

* were output. The FlagBitMask is used to indicate where the next

* flag bit will go when packed into DataBuffer[ 0 ]. Finally, the

* BufferOffset is used to indicate where the next token will be stored

* in the buffer.

*/

char DataBuffer[ 17 ];

int FlagBitMask;

int BufferOffset;

/*

* To initialize the output buffer, we set the FlagBitMask to the first

* bit position, can clear DataBuffer[0], which will hold all the

* Flag bits. Finally, the BufferOffset is set to 1, which is where the

* first character or index/length pair will go.

*/

void InitOutputBuffer()

{

 DataBuffer[ 0 ] = 0;

 FlagBitMask = 1;

 BufferOffset = 1;

}

/*

* This routine is called during one of two different situations. First,

* it can potentially be called right after a character or a length/index

* pair is added to the DataBuffer[]. If the position of the bit in the

* FlagBitMask indicates that it is full, the output routine calls this

* routine to flush data into the output file, and reset the output

* variables to their initial state. The other time this routine is

* called is when the compression routine is ready to exit. If there is

* any data in the buffer at that time, it needs to the flushed.

*

* Note that this routine checks carefully to be sure that it doesn't

* ever write out more data than was in the original uncompressed file.

* It returns a 0 if this happens, which filters back to the compression

* program, so that it can abort if this happens.

*

*/

int FlushOutputBuffer()

{

 if ( BufferOffset == 1 )

  return( 1 );

 Header.compressed_size += BufferOffset;

 if ( ( Header.compressed_size ) >= Header.original_size )

  return( 0 );

 if ( fwrite( DataBuffer, 1, BufferOffset, OutputCarFile )

   !=BufferOffset )

   FatalError( "Error writing compressed data to CAR file" );

 InitOutputBuffer();

 return( 1 );

}

/*

* This routine adds a single character to the output buffer. In this

* case, the flag bit is set, indicating that the next character is an

* uncompressed byte. After setting the flag and storing the byte,

* the flag bit is shifted over, and checked. If it turns out that all

* eight bits in the flag bit character are used up, then we have to

* flush the buffer and reinitialize the data. Note that if the

* FlushOutputBuffer() routine detects that the output has grown larger

* than the input, it returns a 0 back to the calling routine.

*/

int OutputChar( data )

int data;

{

 DataBuffer[ BufferOffset++ ] = (char) data;

 DataBuffer[ 0 ] |= FlagBitMask;

 FlagBitMask <<= 1;

 if ( FlagBitMask == 0x100 )

  return( FlushOutputBuffer() );

 else

  return( 1 );

}

/*

* This routine is called to output a 12-bit position pointer and a 4-bit

* length. The 4-bit length is shifted to the top four bits of the first

* of two DataBuffer[] characters. The lower four bits contain the upper

* four bits of the 12-bit position index. The next of the two DataBuffer

* characters gets the lower eight bits of the position index. After

* all that work to store those 16 bits, the FlagBitMask is shifted over,

* and checked to see if we have used up all our bits. If we have,

* the output buffer is flushed, and the output data elements are reset.

* If the FlushOutputBuffer routine detects that the output file has

* grown too large, it passes and error return back via this routine,

* so that it can abort.

*/

int OutputPair( position, length )

int position;

int length;

{

 DataBuffer[ BufferOffset ] = (char) ( length << 4 );

 DataBuffer[ BufferOffest++ ] |= ( position >> 8 );

 DataBuffer[ BufferOffset++ ] = (char) ( position & Oxff );

 FlagBitMask <<= 1;

 if ( FlagBitMask == 0x100 )

  return( FlushOutputBuffer() );

 else

  return( 1 );

}

/*

* The input process uses the same data structures as the blocked output

* routines, but it is somewhat simpler, in that it doesn't actually have

* to read in a whole block of data at once. Instead, it just reads in

* a single character full of flag bits into DataBuffer[0], and passes

* individual bits back to the Expansion program when asked for them.

* The expansion program is left to its own devices for reading in the

* characters, indices, and match lengths. They can be read in

* sequentially using normal file I/0.

*/

void InitInputBuffer()

{

 FlagBitMask = 1;

 DataBuffer[ 0 ] = (char) getc( InputCarFile );

}

/*

* When the Expansion program wants a flag bit, it calls this routine.

* This routine has to keep track of whether or not it has run out of

* flag bits. If it has, it has to go back and reinitialize so as to

* have a fresh set.

*/

int InputBit()

{

 if ( FlagBitMask == 0x00 )

  InitInputBuffer();

 FlagBitMask <<= 1;

 return( DataBuffer[ 0 ] & ( FlagBitMask >> 1 ) );

}

/*

* This is the compression routine. It has to first load up the look

* ahead buffer, then go into the main compression loop. The main loop

* decides whether to output a single character or an index/length

* token that defines a phrase. Once the character or phrase has been

* sent out, another loop has to run. The second loop reads in new

* characters, deletes the strings that are overwritten by the new

* character, then adds the strings that are created by the new

* character. While running it has the additional responsibility of

* creating the checksum of the input data, and checking for when the

* output data grows too large. The program returns a success or failure

* indicator. It also has to update the original_crc and compressed_size

* elements in Header data structure.

*

*/

int LZSSCompress( input_text_file )

FILE *input_text_file;

{

 int i;

 int c;

 int look_ahead_bytes;

 int current_position;

 int replace_count;

 int match_length;

 int match_position;

 Header.compressed_size = 0;

 Header.original_crc = CRC_MASK;

 InitOutputBuffer();

 current_position = 1;

 for ( i = 0 ; i < LOOK_AHEAD_SIZE ; i++ ) {

   if ( ( c = getc( input_text_file ) ) == EOF )

    break;

   window[ current_position + i ] = (unsigned char) c;

   Header.original_crc = UpdateCharacterCRC32( Header.original_crc, c );

 }

 look_ahead_bytes = i;

 InitTree( current_position );

 match_length = 0;

 match_position = 0;

 while ( look_ahead_bytes > 0 ) {

  if ( match_length > look_ahead_bytes )

   match_length = look_ahead_bytes;

  if ( match_length <= BREAK_EVEN ) {

   replace_count = 1;

   if ( ! OutputChar( window[ current_position ] ) )

    return( 0 );

  } else {

   if ( !OutputPair( match_position, match_length -

                     ( BREAK_EVEN + 1 ) ) )

    return( 0 ):

   replace_count = match_length;

  }

  for ( i = 0 ; i < replace_count ; i++ ) {

   DeleteString( MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) );

   if ( ( c = getc( input_text_file ) ) == EOF ) {

    look ahead bytes--;

   } else {

    Header.original_crc =

     UpdateCharacterCRC32( Header.original_crc, c );

    window[ MOD_WINDOW( current_position + LOOK_AHEAD_SIZE ) ] =

       (unsigned char) c;

   }

   current_position = MOD_WINDOW( current_position + 1 );

   if ( current_position == 0 )

    putc( '.', stderr );

   if ( look_ahead_bytes )

    match_length = AddString( current_position, &match_position );

  }

 };

 Header.original_crc ^= CRC_MASK;

 return( FlushOutputBuffer() );

}

/*

* This is the expansion routine for the LZSS algorithm. All it has to do

* is read in flag bits, decide whether to read in a character or a

* index/length pair, and take the appropriate action. It is responsible

* for keeping track of the crc of the output data, and must return it

* to the calling routine, for verification.

*/

unsigned long LZSSExpand( output )

FILE *output;

{

 int i;

 int current_position;

 int c;

 int match_length;

 int match_position;

 unsigned long crc;

 unsigned long output_count;

 output_count = 0;

 crc = CRC_MASK;

 InitInputBuffer();

 current_position = 1;

 while ( output_count < Header.original_size ) {

  if ( InputBit() ) {

   c = getc( InputCarFile );

   putc( c, output );

   output_count++;

   crc = UpdateCharacterCRC32( crc, c );

   window[ current_position ] = (unsigned char) c;

   current_position = MOD_WINDOW( current_position + 1 );

   if ( current_position == 0 && output != stdout )

    putc( '.', stderr );

  } else {

   match_length = getc( InputCarFile );

   match_position = getc( InputCarFile );

   match_position |= (match_length & Oxf ) << 8;

   match_length >>= 4; match_length += BREAK_EVEN;

   output_count += match_length + 1;

   for ( i = 0 ; i <= match_length ; i++ ) {

    c = window[ MOD_WINDOW( match_position + i ) ];

    putc( c, output );

    crc = UpdateCharacterCRC32( crc, c );

    window[ current_position ] = (unsigned char) c;

    current_position = MOD_WINDOW( current_position + 1 );

    if ( current_position == 0 && output != stdout )

     putc( '.', stderr );

   }

  }

 }

 return( crc ^ CRC_MASK );

}

/*************************** End of CARMAN.C ***************************/


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Chapter 13
Fractal Image Compression

DCT-based JPEG compression is quite effective at low or moderate compression ratios, up to ratios of 20 or 25 to 1. Beyond this point, the image becomes very "blocky" as the compression increases and the image quality becomes too poor for practical use. JPEG obtains high compression ratios by cutting off the high frequency components of the image. This can also introduce very visible artifacts, in particular for sharp edges in the image. This is known as Gibb's phenomenon. The practical limit on the compression ratio in turns implies a limit on the number of images that can fit on a hard disk or a CD-ROM. As such, the storage space requirements of graphics applications continue to increase at a very fast rate.

Another drawback of JPEG compression is its resolution dependence. In order to "zoom-in" on a portion of an image and to enlarge it, it is necessary to replicate pixels. The enlarged image will exhibit a certain level of "blockiness" which soon becomes unacceptable as the expansion factor increases. Because of this problem, it is sometimes necessary to store the same image at different resolutions, thus wasting storage space.

So, although JPEG is now a well-established standard for lossy image compression, it has its limits and alternative compression methods must be considered. Wavelet-based methods are gaining popularity. They are similar in spirit to the DCT methods but do not suffer from some of its shortcomings. Methods based on vector quantization (VQ) are also very promising. But, in this chapter we will look in more detail at yet another technique: fractal image compression.

A brief history of fractal image compression

The term fractal was first used by Benoit Mandelbrot to designate objects that are self-similar at different scales. Such objects have details at every scale. A well-known example is the Mandelbrot set, which is described by a very simple equation yet exhibits an infinite variety of detail. This can be viewed as an extreme form of compression: the equation itself can be described with a few bits of information or implemented in a very short program, but the resulting image would need an infinite amount of bits to be represented as a set of pixels. The popular FractInt program can generate very delicate pictures from one-line formulas.

Mandelbrot did not actually consider fractals for compression, but he showed that they could be used for modeling real objects such as clouds, trees or mountains. Such objects reveal more detail whenever you look closer at them. Mandelbrot's book The Fractal Geometry of Nature first published in 1977 attracted a lot of attention and the word fractal became very popular. The images generated by fractal modeling were very realistic looking and these techniques are now commonly used in many applications using computer-generated images.

Michael Barnsley and his coworkers at the Georgia Institute of Technology were the first to recognize the potential interest of fractal methods for image compression. Barnsley developed the theory of Iterated Function Systems (IFS) first introduced by J. Hutchinson in 1981. After the publication of Barnsley's book Fractals Everywhere in 1988, and his paper in the January 1988 issue of BYTE magazine, fractal compression became a very fashionable subject. The interest in this technique was aroused by the fantastic compression ratios claimed by Barnsley, up to 10,000 to 1. Together with Alan Sloan, Barnsley founded Iterated Systems, Inc. and obtained US patent 4,941,193 on image compression using IFS.

Unfortunately, the fantastic compression ratios could be obtained only on specially constructed images, and only with considerable help from a person guiding the compression process. This process is also known as the "graduate student algorithm," consisting of giving a graduate student an office and a graphics workstation, locking the door, waiting until the student has found a good IFS for the image, and opening the door. It was impossible to completely automate the compression process, even with a supercomputer. Thus, IFS-based compression turned out to be impractical.

A breakthrough was made in 1988 by Arnaud Jacquin, one of Barnsley's Ph.D. students. Instead of trying to find an IFS for a complete image, Jacquin had the idea of partitioning the image into non-overlapping ranges, and finding a local IFS for each range. This transformed the problem into a manageable task, which could be automated. For his doctoral thesis, Jacquin developed the theory of Partitioned Iterated Function Systems (PIFS) and implemented a version of his algorithm in software. Yuval Fisher, Roger Boss, and Bill Jacobs were also among the first to make public contributions to the theory of PIFS.

In 1991, Barnsley and Sloan obtained US patent 5,065,447 on this technique. Their company, Iterated Systems, sells software and hardware products using it, but they have not made public the details of their technology. In particular, the FIF (Fractal Image Format) used by the Iterated Systems products has not been publicly described. Possibly because of this, and of the patents attached to the method, fractal image compression is not yet used in practice as much as other techniques. However fractal compression is still a subject of active research, and it has already demonstrated its superiority at least for applications where very high compression ratios are required.

PIFS are also named Local Iterated Function Systems (LIFS), and Barnsley uses the term "Fractal Transform." They all refer to the same technique. Barnsley's terminology may be misleading because the Fractal Transform is not a transform in the same sense as a Fourier Transform or a Discrete Cosine Transform, so we will use the PIFS terminology in the rest of this chapter. All practical implementations of fractal image compression, including the program given later in this chapter, are based on PIFS. To understand better what a PIFS is, let us first come back to the technique which precedeed it, based on IFS.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

What is an Iterated Function System?

Mandelbrot had observed that complex images could be obtained from simple formulas. Given a formula, it is relatively easy to derive the corresponding image. Barnsley had the idea of going in the other direction, from the image to the formula. When this is possible, it can result in fantastic compression ratios. Instead of representing the image as a long sequence of pixel values, the image can be reconstructed from a formula, which can be encoded in a much smaller number of bytes. Let's take an example which is not one of fractal compression but which can convey the right idea.

Assume that you want to represent a black and white image consisting of a black circular disk on a white background. For an image of a given resolution (a given number of pixels in the horizontal and vertical dimensions), you can enumerate all the pixels which are inside the disk. Alternatively, you can give an equation for the disk, specifying it as the set of points (x, y) which satisfy:

(x-a)2+ (y-b)2 < r2

where r is the radius of the disk and (a, b) its center. This equation is sufficient to reconstruct the image of the disk. Moreover, the image can be reconstructed at any resolution—this would not be true for an explicit list of pixels. A similar difference exists between scalable character fonts (TrueType fonts in Microsoft Windows) and fonts that are made of a fixed number of pixels. Each character in a scalable font is described by a formula for drawing the character; a character in a fixed font is just a set of pixels.

Unfortunately, real-world images almost never let themselves be represented as such simple equations. Given an image, it is generally impossible to find a simple formula representing the image exactly. Barnsley's idea was to take advantage of the self-similarity present in an image to find an approximate representation of the image as a fractal, like Mandelbrot's fractals which exhibit self-similarity at different scales. Also, instead of giving an explicit equation satisfied by all points of the image, the fractal is implicitly defined as the fixed-point solution of an Iterated Function System. The theory of IFS is outside the scope of this book, so we will only mention very briefly some of the mathematics involved without giving precise definitions.

Basic IFS mathematics

A mapping from a set to itself is said to be contractive if it reduces the distances: the distance between f(x) and f(y) is smaller than the distance between x and y. For example, the function f(x) = x/2 defined on the set of real numbers is contractive. The Contractive Mapping theorem states, in short, that a contractive mapping has a unique fixed point, that is, a value x such as f(x) = x. Moreover, the fixed point can be obtained by starting from any point x0 and computing the sequence:

x1= f(x0)

x2= f(x1) = f(f(x0))

etc...

The sequence converges to the unique fixed point. For example, starting with the value x0= 1 and applying the function f(x) = x/2 iteratively, we get the sequence 1, 1/2, 1/4, 1/8, etc... which converges to the value 0. The example was given in the set |R of the reals, but the Contractive Mapping theorem also applies to higher dimensions, in particular for two dimensional images.

An Iterated Function System consists of a finite set of contractive mappings w1... wN on the plane |R2. The IFS can be applied to a black and white image as follows. Each (black) point x of the image is mapped to N points w1(x) ... wN(x). The union of all the resulting points forms itself another black and white image. So the IFS transforms an image into another image. Hutchinson proved that in some well defined sense, the IFS is itself a contractive mapping, and thus it has a unique fixed point within the set of all black and white images. So by starting from an arbitrary image and applying the IFS iteratively, the process converges to a unique image which depends only on the IFS and not on the initial image.

This is in essence how fractal decompression works. The decompressor need only know the description of the IFS, and can reconstruct an image from this. The method works regardless of the exact form of the IFS, as long as it is contractive. The IFS can be viewed as a special copy machine which creates N reduced copies of the input image, and pastes them together. The copies are reduced since each wi is contractive. By feeding the output of this copying machine into itself in a feedback loop, the images generated at each step get closer and closer to each other and the process converges to the unique fixed-point image, also called the attractor of the IFS.

The resulting image is a fractal, since it contains reduced copies of itself at every scale. More detail can be seen if we zoom on a portion of the image. Because of this self-similarity property, image compression using IFS deserves the name of fractal image compression.

We have only mentioned black and white images so far, but the theory can be extended to grayscale images. Color images can themselves be encoded as three grayscale images, one for each of the red, green, and blue components (just like the JPEG algorithm does).

Up to now, we have only considered the decompression process. The compression process is to find a good IFS for a given image. This is known in the fractal compression literature as "the inverse problem." This problem is vastly more complicated than the decompression process.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Image compression with Iterated Function Systems

An IFS provides a good approximation of a target image I if the fixed point of the IFS is an image closely resembling I. The goal is to find a set of contractive mappings w1 ... wN so that the union W of all these mappings has a fixed point close to I.

It is hopeless to find W by trying some mappings wj, computing the resulting fixed point, comparing it with I, and starting again with other mappings until a close match is found. We will instead try to find a contractive mapping W such that W(I) is close to I. Barnsley's Collage Theorem states that if W(I) is sufficiently close to I, then the fixed point

is also close to I. The image W(I) is composed of a collage (union) of all the reduced images wj (I). This is how the theorem got its name.

So with the help of the Collage Theorem, the "inverse problem" can be restated as finding a good collage for an image. This is where the real difficulties start. It is generally not feasible, even with enormous computing power, to find a good collage automatically, thus human intervention is required. A person must guide the compression process by segmenting the original image so that each segment looks like a reduced copy of the whole image, and so that the union of all segments covers the original image as best as possible. For example, the person determines that a branch of a tree can be viewed as a reduced copy of the whole tree, possibly distorted.

Once the segments have been defined, the computer can derive the wj mappings and thus an IFS for the image. There is a lot of flexibility in the choice of the mappings, as long as they are contractive, but affine functions are generally used for simplicity. In one dimension, an affine function has the form

f(x) = a * x  +  b

where a and b are constants. In two dimensions, the image of a point X = (x, y) is

f(X) =  A * X + B

where A is a two-dimensional matrix and B a constant vector. The matrix A determines a rotation, skew and scaling for the image, and the vector B determines a translation. The contractivity condition on f can be expressed as conditions on the coefficients of the matrix A: the scaling factor must be less than one.

After all the affine transformations have been selected, the IFS can be represented in a compact form by encoding the coefficients of all the transformations. If a good collage has been found, the total number of affine transformations is much smaller than the total number of pixels in the image, so encoding the coefficients requires far fewer bits than enumerating all the pixel values. This is why encoding an image as an IFS is actually a form of data compression. The compression is lossy because the attractor of the IFS is in the best case close to the original image but not strictly equal to it.

The main difficulty of this compression process is to find within the image reduced copies of the whole image. Real-world images often contain some self-similarity, but only between selected portions of the image. The breakthrough made by Jacquin was to partition the input image, and to find a local IFS for each partition. With this new method, it finally became possible to completely automate the compression process and furthermore to do it in a reasonable amount of time.

Image compression with Partitioned Iterated Function Systems

We have seen in chapter 8 "Sliding Window Compression" that the LZ77 algorithm works by finding redundancy within the input text in the form of common phrases. Similarly, some image compression algorithms such as Vector Quantization and Fractal compression with PIFS work by finding redundancy within the input image in the form of similar image portions. The parallel between the text and image compression methods is not complete since the former are lossless, whereas the latter only look for approximate matching and are thus lossy.

Vector Quantization uses a dictionary (or codebook) of pixel patterns. The input image is partitioned into small pixel blocks, and each block is encoded as a reference to the dictionary pattern which most resembles the block. A block has the same size as the corresponding dictionary pattern, but all the blocks need not have the same size. The decoder must have a copy of the dictionary, and can easily reconstruct an approximation of the original image by assembling the dictionary patterns specified by the encoder.

Fractal compression with PIFS is similar to Vector Quantization, but in this case, there is no external dictionary. The input image acts as its own dictionary. The decoder doesn't have this image initially, but it can reconstruct it gradually by iterating a PIFS. Thus the dictionary is only a "virtual codebook".


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The compressor first partitions the input image into a set of non-overlapping ranges. The ranges are generally squares or rectangles, but good results can also be obtained with other shapes such as triangles. For each range, the compressor looks for a part of the input image called a domain, which is similar to the range. The domain plays the role of the pixel pattern in Vector Quantization, but here it must be larger than the range to ensure that the mapping from the domain to the range is contractive in the spatial dimensions.

Note: Barnsley has reversed the meanings of range and domain, but we prefer keeping the established terminology: a transformation maps from a domain to a range.

In general, the compressor looks for domains that are twice as large as the range, but other ratios are possible. As opposed to ranges, domains may overlap.

The test file LISAW.GS is shown in Figure 13.1. The same file compressed with fractal compression at a compression ratio of 96% is shown in Figure 13.2.

Figure 13.1 The original LISAW.GS (64000 bytes).

Figure 13.2 LISAW with fractal compression (2849 bytes).

In Figure 13.2, two ranges and the two corresponding domains selected by the compression algorithm have been highlighted. The algorithm has found similarity between two unrelated portions of the image: a range below the eye and a domain, twice as large, on the forehead. The compressor has also found self-similarity within one domain, in the chin. In the latter case, the range and the domain overlap. This kind of self-similarity is quite frequent in real-world images, and fractal compression takes advantage of this.

To assess the similarity between a domain D and a range R, the compressor finds the best possible mapping w from the domain to the range, so that the image w(D) is as close as possible to the image R. As we have seen previously, affine transformations are convenient for this purpose, but non-linear transformations could also be used as long as they are contractive. Two-dimensional transformations were used for black and white images. Three dimensions are needed for grey scale images: two for the spatial components and one for the luminance component. An affine map is then composed of a geometric part which maps the domain into the range, and of a luminance part which changes the pixel intensity values.

More precisely, a point (x, y) with luminance z belonging to domain Di is mapped into:

The constants ai,j and di,j specify the geometric part, the constants ci and bi specify the luminance part. ci represents a contrast factor, which must be smaller than one to make sure that the mapping is contractive in the luminance dimension. bi represents a brightness offset applied after the contrast has been reduced.

In practice, the compressor does not have to determine explicitly the constants ai,j and di,j for each domain. They are implicitly defined by the relative size, orientation and position of the domain with respect to the range. In particular, if the compressor only looks for domains that are exactly twice as large as the range, the scaling factor which would normally be derived from the ai,j values is already imposed and is equal to 0.5. Similarly, if domains and ranges are restricted to be squares, there are only 8 possible orientations of the domain relative to the square: 4 rotations and 4 symmetries. Thus 3 bits are sufficient to encode this orientation. Finally, the translation constants di,j are determined by the position of the top left corner of the domain.

The simplifications described above may seem too drastic, but they are actually necessary to reduce the complexity of the "inverse problem" to manageable proportions. In theory, ranges and domains can have potato shapes instead of a square shape, the contractive mappings can be non linear instead of being affine maps, etc... However the total search space would become much too large and as a result the compression would be too slow for practical use. But even with simple affine maps, finding the optimal domain for a given range can still be an expensive operation.

For a given range R, the compressor must examine a number of possible domains. For each such domain D, the compressor must find the optimal affine map from D to R. The best one is the map w which minimizes the distance between the image R and the image w(D), where the distance is taken in the luminance dimension, not the spatial dimensions.

Such a distance can be defined in various ways, but to simplify the computations it is convenient to use the Root Mean Square (RMS) metric (the program GSDIFF.C given in the previous chapter computes such a distance). For a given range and and a given domain, the RMS distance depends only on the contrast factor ci and the brightness offset bi The distance is minimum when the partial derivatives with respect to these two variables are both zero. ci and bi can thus be obtained by solving two simple linear equations, as we will see later in our sample fractal compression program.

Once the RMS distances between the range and all selected domains have been determined, the compressor chooses the domain with the smallest distance, encodes the corresponding affine map, and goes on working on the next range.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Fractal image decoding

The Partitioned Iterated Function System created with the above algorithm consists of a list of affine maps, each map being restricted to a specific range. Since each map is contractive in the luminance dimension (the contrast factor is less than one), we can apply the Contractive Mapping theorem to decode the image. Starting from an arbitrary image, for example a completely black image, the decoder can apply the PIFS iteratively. This process converges to the fixed point of the PIFS. If the compressor has found a good PIFS for the image, that is, if the collage of all the transformed domains is close to the original image, then the fixed point of the PIFS is also close to this image.

To perform one iteration of the PIFS, the decoder takes the list of all affine maps and applies each one in turn. This transforms a set of domains into a set of ranges. Since the ranges have been selected to be non-overlapping and to cover the whole input image, a new complete image emerges as a result. The decoder can then repeat the whole process, until convergence is achieved, that is, until there is very little difference between the input image and the output image. Convergence is generally obtained in 8 to 10 iterations.

Figure 13.3 shows the result of decoding the LISAW image with our sample program after 1, 2, 3 and 8 iterations, starting from an initially black image.

Figure 13.3 Image decoding after 1, 2, 3 and 8 iterations

Even after only one iteration, the image is recognizable. The minimum range size selected by the encoder was 4 pixels wide, and such ranges are clearly visible, giving a "blocky" aspect to the image. However since the affine maps are also contractive in the spatial dimensions (the domains are twice as large as the ranges), more detail is created at each iteration. After the second iteration, the remaining blocks are only two pixels wide. After 8 iterations, convergence has been achieved, and the resulting image is extremely close to the original image. (The compressor was set to best quality for this image.)

We can now better understand why the affine maps were chosen to be contractive in both the spatial and luminance dimensions. The contraction in luminance (reduction of contrast) was essential to ensure convergence of the decoding process. The spatial contraction is useful to create detail in the image at all scales, and thus get a much better approximation of the original image. Without spatial contraction, the decoding process would still converge, but it would converge to a a very "blocky" image, without any contrast inside each block. With spatial contraction the contrast across ranges, initially provided by the brightness offset components of the affine maps, is propagated within each range to smaller and smaller scales after each iteration.

Close inspection of the first image of Figure 13.3 reveals that sometimes detail is visible even within a range. Since the domains of the initial image are uniformly black, each range after the first iteration should have a uniform grey level, given by the brightness offset in the affine map. However to accelerate the convergence of the decoding process, our sample program uses the same buffer for both input and output images, as we will see later. Thus the detail visible within a range after the first iteration is only a side effect of our particular implementation.

Resolution independence

To reconstruct an image, the decoder starts its iterations with an arbitrary image of the same size as the original image. But what happens if the decoder starts with an initial image that is twice as large? The result of the decoding process will be also an image twice as large as the original image. However, since the affine maps used to encode the image do not depend on its resolution, the decoded image will not have the "blocky" aspect that would have been obtained if we had simply replicated the pixels of the original image. It will instead still contain detail at every scale.

Figure 13.4 shows a detail of the original LISAW image, enlarged 8 times. The 8x8 pixel blocks are clearly visible. Figure 13.5 shows the same part of the image, but decoded with scale factor 8.

Figure 13.4 Detail of original LISAW enlarged 8 times.

Figure 13.5 Detail of LISAW decompressed with scale factor 8.

The image obtained through fractal decoding is much more natural-looking. The decoding process has created artificial detail which was not present in the original image, but which looks as if we had really zoomed on the original image. This is a very useful feature, but it does have limits. If we try to zoom at an enormous scale factor, we will not end up seeing the atoms at the surface of LISAW's skin, but rather we will be looking at detail which is completely artificial.

The fantastic compression ratios put forward by some advocates of fractal compression may have to be taken with a grain of salt in some cases. For example, assume that an image of 320x200 pixels, with an original size of 64 KB, has been compressed with a ratio of 32 to 1, resulting in a compressed image of 2 KB. Now, decode this image with a scale factor of 4. This creates an image of 1280x800 pixels, with an uncompressed size of 64x16 = 1024 KB. It would be incorrect to state that we have achieved a compression ratio of 1024 / 2 = 512 to 1. The reason is that the uncompressed image contains artificial detail which was not present in the original image. The original information is still compressed with a ratio of 32 to 1; the rest of the information has been created artificially by the decoding process.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The sample program

The sample program used to demonstrate fractal compression in this chapter is found in the C source file FRAC.C. It must be compiled and linked with the standard support source files, BITIO.C, ERRHAND.C, and either MAIN-C.C for compression or MAIN-E.C for expansion.

The fractal compression program optionally takes additional parameters on the command line:

* The quality value, ranging from 0 to 20. It is used as average tolerated error between the original image and its uncompressed version. Small values result in better quality images, large values result in better compression ratios. The default value has been arbitrarily chosen as 2.
* The domain density factor, ranging from 0 (fastest compression) to 2 (best but very slow compression). This parameter affects the size of the domain pool that will be searched.
* Horizontal and vertical images sizes (default 320 x 200). Both sizes must be multiples of 4 in this implementation.

The command syntax for the compression program is:

FRAC infile outfile [-q quality] [-d density] [-h h_size] [-v v_size]

The image dimensions and the domain density factor are encoded in the compressed file, so the expansion program doesn't need these parameters on the command line. The syntax for expansion is:

FRAC-E infile outfile [-i iterations] [-s scale]

The optional parameters are:

* The number of iterations, ranging from 1 to 15. The image quality does not improve much after 8 to 10 iterations. The default is 8.
* The scale factor (decompressed size divided by original size). The default is 1.

The main compression module

A summarized version of the main compression module is shown below.

void CompressFile (FILE *input, BIT_FILE *output, int argc, char *argv[])

{

    /* Allocate and initialize the image data and cumulative image

       data: */

    compress_init(x_size, y_size, input);

    /* Initialize the domain size information as in the

       decompressor: */

    dominfo_init(x_size, y_size, dom_density);

    /* Classify all domains: */

    for (s = MIN_BITS; s <= MAX_BITS; s++) {

        classify_domains(x_size, y_size, s);

    }

    /* Output the header of the compressed file. */

    OutputBits(frac_file, (uns_long)x_size, 16);

    OutputBits(frac_file, (uns_long)y_size, 16);

    OutputBits(frac_file, (uns_long)dom_density, 2);

    /* Compress the whole image recursively */

    traverse_image(0, 0, x_size, y_size, compress_range);

    /* Free all dynamically allocated memory: */

    compress_cleanup(y_size);

}

We first allocate and initialize all the necessary data structures. Then we classify all the possible domains. This speeds up the compression process considerably, as we will see later. After this, we store the image dimensions and the domain density factor in the compressed output file, since these parameters are needed by the decompressor.

The bulk of the work is done in the call of the traverse_image() function, which partitions the image recursively, and compresses each partition. In the end we free all the dynamically allocated data structures.

Initialization

The initialization function compress_init() first allocates and initializes arrays for the image and domain data. Since the domains are twice as large as the ranges, each pixel in the domain image is the sum of 4 pixels in the range image. We don't average (divide by 4) to keep an integer format without losing precision.

To speed up the compression process, we initialize cumulative tables for the image and domain data. We often have to compute the sum of pixel values or squared pixel values in a range or domain. To avoid repeated computations, we maintain for each pixel the sum of all pixel values strictly above and to the left of the given pixel. Then we can easily obtain the sum of all pixel values within a square region by adding the cumulative values at the top left and bottom right corners, and subtracting the cumulative values at the top right and bottom left corners. This is done in the code by the region_sum() macro.

To reduce the memory requirements, some tables are maintained only for pixels of even coordinates. In particular, domains are restricted to have their top left pixel with even coordinates. To simplify some algorithms, ranges are also restricted to have a size which is a multiple of four, hence the width and height of the image must also be multiple of four in our simple implementation.

In order to get the best possible image quality, the compressor could compare a given range with all possible domains. However this would be extremely slow. For an image of 320x200 pixels with ranges 4 pixels wide and domains 8 pixels wide, there are 4000 ranges and 313x193 = 60409 domains, if the domains can start on any pixel boundary. Thus 241,636,000 domain-range comparisons would have to be made, each comparison involving a least-square regression analysis to find the optimal affine map.

Note: Our sample program does not attempt comparisons in all 8 possible orientations of the range relative to the domain, otherwise this would increase the compression time by yet another factor.

To avoid such a lengthy computation, we reduce the size of the domain pool using the domain density parameter. For a density value of zero, the domains start on a boundary multiple of their size, thus the domains do not overlap. When compression time is not an issue and we are looking for the best possible image quality, the domain step (distance between two consecutive domains) is divided by 2 for a density of 1, or by 4 for a density of 2. The function dominfo_init(), which is common to the compressor and the decompressor, takes this into account.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Domain classification

Despite the reduction of the domain pool described above, compression is still very slow compared to other methods such as DCT based algorithms. Another significant speed improvement can be obtained by avoiding domain-range comparisons which have little probability of providing a good match. We assign a class number to each range and each domain, and we compare only members of the same class. To avoid repeated computations, all domains are classified only once at the beginning of the program, by the function classify_domains().

Several methods are possible for classifying ranges and domains. In our sample program, the class is determined in the function find_class() by the ordering of the image brightness in the four quadrants of the range or domain. There are 24 possible orderings, hence 24 classes. For each quadrant we compute the number of brighter quadrants; this is sufficient to uniquely determine the class. Class 0 has quadrants in order of decreasing brightness; class 23 has quadrants in order of increasing brightness.

For simplicity and speed reasons, a range and a domain are compared only with the same orientation. In addition, the contrast factor of affine maps is constrained to be non-negative. After application of such an affine map, the relative ordering of the image brightness in the four quadrants is unchanged. Thus a domain and a range are likely to match well only if they belong to the same class. The validity of this argument can be confirmed in practice by compiling the program with the option -DCOMPLETE_SEARCH. This removes the class restrictions, dramatically increases the compression time, but improves the image quality only very marginally.

Since ranges and domain are compared only with the same orientation, the affine maps are composed in the spatial dimensions of just a scaling by 0.5 and a translation. There is no rotation or symmetry. Thus we do not have to output in the compressed file 3 bits per affine map indicating the selected orientation.

Image partitioning

The encoder can arbitrarily partition the input image into ranges, as long as the decompressor is able to reconstruct the same partition. For simplicity, our sample program uses only square ranges with sizes that are powers of two. We could have further simplified the algorithm by using only ranges 4 pixels wide, but this would not have taken advantage of flat portions of the image, which can be covered well by larger ranges. On the other hand, using ranges that are all 16 pixels wide or larger would not achieve a good quality for the decoded image. Regions of the image with fine detail generally have to be covered with small ranges.

To obtain a good balance between compression ratio and image quality, the function compress_range() first attempts to find a good match for a range 16 pixels wide. If there is no domain which is close enough to the range, that is, if the RMS distance between the range and any domain is greater than the quality value chosen by the user, then the range is split into 4 ranges, and the process is repeated recursively for each of them. When a range is split, the encoder outputs a single zero bit into the compressed stream, to let the decoder know about the need for a split and thus follow the same partitioning algorithm.

In theory, we could allow range splitting up to ranges that consist of a single pixel. This would ensure a perfect quality for the image if the quality factor is selected as zero. For the affine maps, it would be sufficient to chose a null contrast coefficient, and a brightness offset equal to the original pixel value. This would have the additional advantage that image decoding would converge in a single iteration. Unfortunately, all dreams about compression ratios would be gone, since the original file would be expanded instead of being compressed! Therefore it makes sense to impose a minimum size of 4x4 pixels for each range.

The extreme example given above shows, by the way, that it is always possible to find a Partitioned Iterated Function System for a given image; the real problem is to find a good one, which can encode an image with good quality while still offering decent compression ratios.

Since the original image is generally not a square, the function traverse_image() first encodes the largest square fitting in the image, then the two rectangles respectively on the right and below the square. To simplify the algorithm, the size of the square is constrained to be a power of two. If the square is larger than 16x16 pixels, the square is split recursively. We do not attempt to find good matches for regions larger than 16x16 pixels, since it is generally a waste of time: no domain is close enough.

Finding optimal affine maps

The function find_map() finds the best affine map from a range to a domain. This is done by minimizing the RMS distance between range and mapped domain as a function of the contrast and brightness. Actually, we use the sum of squared errors instead of the RMS value to simplify the algorithm and avoid a square root operation. This error value is:

where c is the contrast factor, b the brightness offset, n the number of pixels in the range, di the pixel values in the domain and ri the pixel values in the range. The optimal values of c and b are obtained when the partial derivatives E(c,b) with respect to c and b are both null. The resulting linear equations are easily solved. The values of c and b obtained in this manner are floating point numbers. To ensure a compact encoding of the affine map, these values are quantized and written to the output file as integers using 4 bits for the contrast and 6 bits for the brightness. Using 5 and 7 bits or more would provide a marginally better image quality at the expense of the compression ratio.

The error value E(c,b) is computed using the quantized values of c and b, since the decoder will only know these and not the original floating point values. The error value must depend on the affine map actually selected, not the theoretical best map. There can be a large difference between the optimal contrast and the actual selected contrast since before the quantization, c is restricted to the interval 0.0—1.0. The lower bound is needed because of our simple domain classification mechanism, the upper bound is needed to ensure that the affine map is contractive in the luminance dimension.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

The decompression module

The decompression code uses the same traverse_image() function as the compressor, to ensure that both use exactly the same partitioning strategy. The only difference is that during the traversal, the decoder only reads the encoded affine maps; it does not actually generates the output image at this step.

Once the affine maps have been read in, the decoder starts with a random initial image and calls refine_image() a number of times selected by the user. For each iteration, refine_image() goes through all the maps and applies them to the current image. The "pure" method would compute a separate new image and then swap the roles of the old and new image. However the convergence towards the final image happens to be quicker if we overwrite the same image while applying the affine maps; for the same quality of reconstructed image we need fewer iterations. Overwriting the same image also reduces the memory requirements since we need only a buffer for one image instead of two.

After the image has been reconstructed, the decoder calls the function average_boundaries() to smooth the transition between adjacent ranges. This is only done for large ranges (8x8 and 16x16), since averaging the small 4x4 ranges can degrade the image quality.

The complete code listing

The complete listing of FRAC.C follows.

/*********************** Start of FRAC.C  ***********************

 *

 * This is the FRAC module, which implements a graphics fractal

   compression

 * program.  It needs to be linked with the standard support routines.

 * Copyright 1995 Jean-loup Gailly

 */

#include <stdio.h>

#include <stdlib.h>

#include <math.h>

#include "bitio.h"

#include "errhand.h"

#ifdef unix

#  define float double /* better accuracy but more memory usage */

#endif

char *CompressionName = "Fractal compression";

char *Usage =

"infile outfile [-q quality] [-d density] [-h h_size] [-v v_size]\n\

   quality from 1..20, domain density from 0..2\n\

Decompression parameters:\n\

    infile outfile [-i iterations] [-s scale]\n";

typedef unsigned char image_data;

typedef unsigned long uns_long;

/*

 * Maximum gray level in an image

 */

#define MAX_GREY 255

/*

 * Number of classes. Each class corresponds to one specific ordering

 * of the image brightness in the four quadrants of a range or domain.

 * There are 4*3*2 = 24 classes.

 */

#define NCLASSES 24

/*

 * Minimum and maximum number of bits for the side of a range. The actual

 * range sizes are between 1<<MIN_BITS and 1<<MAX_BITS. To simplify the

 * implementation and avoid ridiculously small ranges, MIN_BITS must be

   >= 2.

 * This implementation also requires MAX_BITS <= 7.

 */

#define MIN_BITS 2

#define MAX_BITS 4

/*

 * Maximum contrast factor in a range to domain mapping.

 */

#define MAX_CONTRAST 1.0

/*

 * Bit sizes for encodings of contrast and brightness in an affine map.

 * Using smaller sizes increases compression and degrades image quality.

 */

#define CONTRAST_BITS    4

#define BRIGHTNESS_BITS  6

#define MAX_QCONTRAST   ((1<<CONTRAST_BITS)-1)   /* max quantized

                                                    contrast */

#define MAX_QBRIGHTNESS ((1<<BRIGHTNESS_BITS)-1) /* max quantized

                                                    brightness */

/*

 * De-quantize an integer value in the range 0 .. imax to the range

   0.0 .. max

 * while preserving the mapping 0 -> 0.0 and imax -> max.

 */

#define dequantize(value, max, imax) ((double)(value)*(max)/(double)imax)

/*

 * Compute the square of a pixel value and return the result as unsigned

   long

 */

#define square(pixel) (uns_long)(pixel)*(pixel)

/*

 * Range data: range[i][j] is the brightness at row i and column j

 */

image_data **range;

/*

 * Domain data, summed over 4 pixels: domain[i][j] is the sum of the

 * pixel values at (2j, 2i), (2j+1, 2i), (2j, 2i+1) and (2j+1, 2i+1)

 */

unsigned **domain;

/*

 * Cumulative range data, kept only for pixels of even coordinates.

 * cum_range[i][j] is the sum of all pixel values strictly above and to

   the

 * left of pixel (2j, 2i). In particular, cum_range[y_size/2][x_size/2]

   is

 * the sum of all pixel values in the image. This table is also used for

 * the cumulative domain data.

 */

uns_long **cum_range;

/*

 * Cumulative squared range data, kept only for pixels of even

   coordinates.

 * cum_range2[i][j] is the sum of the squares of all pixel values

   strictly

 * above and to the left of pixel (2j, 2i). In particular,

 * cum_range2[y_size/2][x_size/2] is the sum of all the squared pixel

   values

 * in the image.

 */

float **cum_range2;

/*

 * Cumulative squared domain data. cum_domain2[i][j] is the sum of the

   squares

 * of all domain values strictly above and to the left of domain (j,i),

   which

 * corresponds to pixel (2j, 2i). The values in cum_domain2 are scaled

   by

 * a factor of 16.0 to avoid some multiplications.

 */

float **cum_domain2;

/*

 * Domain density: domains of size s*s are located every

   (s>>dom_density)

 * pixels. The density factor can range from 0 to 2 (smallest domains

 * have a size of 8 and must start on even pixels boundaries). Density

 * factors 1 and 2 get better image quality but significantly slow

 * down compression.

 */

int dom_density = 0;

/*

 * Maximum tolerated mean square error between original image and

 * reconstructed uncompressed image.

 */

double max_error2;

/*

 * The fractal (compressed) file

 */

BIT_FILE *frac_file;

/*

 * Information common to all domains of a certain size: info[s]

   describes

 * domains of size 1<<(s+1), corresponding to ranges of size 1<<s

 */

struct domain_info {

    int pos_bits;   /* Number of bits required to encode a domain

                       position */

    int x_domains;  /* Number of domains in x (horizontal) dimension */

} dom_info[MAX_BITS+1];

/*

 * Each domain is described by a `domain_data' structure.

 * domain_head[c][s] is the head of the list of domains of class c

 * and size 1<<(s+1) (corresponding to ranges of size 1<<s).

 */

typedef struct domain_struct {

    int x;                      /* horizontal position */

    int y;                      /* vertical position */

    float d_sum;                /* sum of all values in the domain */

    float d_sum2;               /* sum of all squared values in the

                                   domain */

    struct domain_struct *next; /* next domain in same class */

} domain_data;

domain_data *domain_head[NCLASSES][MAX_BITS+1];

/*

 * Ranges are described by a `range_data' structure. This structure

 * is computed on the fly for each range as it is compressed.

 */

typedef struct range_struct {

    int x;         /* horizontal position */

    int y;         /* vertical position */

    int s_log;     /* log base 2 of the range size */

    double r_sum;  /* sum of all values in the range */

    double r_sum2; /* sum of all squared values in the range */

} range_data;

/*

 * Range to domain mappings are described by an `affine_map' structure.

 */

typedef struct map_struct {

    int    contrast;   /* quantized best contrast between range and

                          domain */

    int    brightness; /* quantized best brightness offset */

    double error2;    /* sum of squared differences between range

                         and domain */

} affine_map;

/*

 * Function prototypes for both ANSI and K&R.

 */

#ifdef __STDC__

#  define OF(args)  args

#else

#  define OF(args)  ()

#endif

/*

 * Functions used for compression

 */

void CompressFile  OF((FILE *input, BIT_FILE *output, int argc, char

*argv[]));

void compress_init OF((int x_size, int y_size, FILE *image_file));

void compress_cleanup OF((int y_size));

void classify_domains OF((int x_size, int y_size, int s));

int  find_class       OF((int x, int y, int size));

void compress_range   OF((int x, int y, int s_log));

void find_map  OF((range_data *rangep, domain_data *dom, affine_map

*map));

/*

 * Functions used for decompression

 */

void ExpandFile OF((BIT_FILE *input, FILE *output, int argc, char

*argv[]));

void decompress_range   OF((int x, int y, int s_log));

void refine_image       OF((void));

void average_boundaries OF((void));

/*

 * Functions common to compression and decompression

 */

typedef void (*process_func) OF((int x, int y, int s_log));

void traverse_image OF((int x, int y, int x_size, int y_size,

                        process_func process));

int  quantize       OF((double value, double max, int imax));

void dominfo_init   OF((int x_size, int y_size, int density));

void *xalloc        OF((unsigned size));

void **allocate     OF((int rows, int columns, int elem_size));

void free_array     OF((void **array, int rows));

int  bitlength      OF((uns_long val));

                /**********************************/

                /* Functions used for compression */

                /**********************************/

/* =====================================================================

 * This is the main compression routine.  By the time it gets called,

 * the input and output files have been properly opened, so all it

   has to

 * do is the compression.  Note that the compression routine optionally

 * accepts additional parameters:

 * - the quality value, ranging from 0 to 20. It is used as average

   tolerated

 *   error between the original image and its uncompressed version.

     (Non

 *   integer values are also accepted.)

 * - the domain density factor, ranging from 0 (fastest compression)

     to 2

 *   (best but very slow compression).

 * - horizontal and vertical images sizes (default 320 x 200). Both

     sizes

 *   must be multiple of 4 in this implementation (this restriction

     could be

 *   removed with slightly more complex code).

 */

void CompressFile(input, output, argc, argv)

    FILE *input;

    BIT_FILE *output;

    int argc;

    char *argv[];

{

    int x_size = 320;     /* horizontal image size */

    int y_size = 200;     /* vertical image size */

    double quality = 2.0; /* quality factor */

    int s;                /* size index for domains; their size is

                             1<<(s+1) */

    /* Check the command line parameters: */

    for ( ; argc != 0; argv++, argc--) {

        if (argv[0][0] != `-' || argc == 1) {

            fatal_error("Incorrect argument: %s\n", *argv);

        }

        switch(argv[0][1]) {

            case `q': quality     = atof(*++argv); argc--; break;

            case `d': dom_density = atoi(*++argv); argc--; break;

            case `h': x_size      = atoi(*++argv); argc--; break;

            case `v': y_size      = atoi(*++argv); argc--; break;

            default:  fatal_error("Incorrect argument: %s\n", *argv);

        }

    }

    if (dom_density < 0 || dom_density > 2) {

        fatal_error("Incorrect domain density.\n");

    }

    if (x_size % 4 != 0 || y_size % 4 != 0) {

        fatal_error ("Image sizes must be multiple of 4\n");

    }

    /* Allocate and initialize the image data and cumulative image

       data: */

    compress_init(x_size, y_size, input);

    /* Initialize the domain size information as in the

       decompressor: */

    dominfo_init(x_size, y_size, dom_density);

    /* Classify all domains: */

    for (s = MIN_BITS; s <= MAX_BITS; s++) {

        classify_domains(x_size, y_size, s);

    }

    /* Output the header of the compressed file. The first byte

       (`F' for

     * fractal') is just for a consistency check in the decompressor.

     */

    frac_file = output;

    OutputBits(frac_file, (uns_long)'F',    8);

    OutputBits(frac_file, (uns_long)x_size, 16);

    OutputBits(frac_file, (uns_long)y_size, 16);

    OutputBits(frac_file, (uns_long)dom_density, 2);

    /* Compress the whole image recursively, stopping when the image

     * quality is good enough:

     */

    max_error2 = quality*quality;

    traverse_image(0, 0, x_size, y_size, compress_range);

    /* Free all dynamically allocated memory: */

    compress_cleanup(y_size);

}

/* =================================================================

 * Allocate and initialize the image data and cumulative image data.

 */

void compress_init(x_size, y_size, image_file)

    int x_size;       /* horizontal image size */

    int y_size;       /* vertical image size */

    FILE *image_file; /* the input image file */

{

    int x, y;             /* horizontal and vertical indices */

    uns_long r_sum;       /* cumulative range and domain data */

    double r_sum2;        /* cumulative squared range data */

    double d_sum2;        /* cumulative squared domain data */

    range =   (image_data**)allocate(y_size,     x_size,

                                     sizeof(image_data));

    domain    = (unsigned**)allocate(y_size/2,   x_size/2,

                                     sizeof(unsigned));

    cum_range = (uns_long**)allocate(y_size/2+1, x_size/2+1,

                                     sizeof(uns_long));

    cum_range2  =  (float**)allocate(y_size/2+1, x_size/2+1,

                                     sizeof(float));

    cum_domain2 =  (float**)allocate(y_size/2+1, x_size/2+1,

                                     sizeof(float));

    /* Read the input image: */

    for (y = 0; y < y_size; y++) {

       if (fread(range[y], sizeof(image_data), x_size, image_file)

       != x_size) {

           fatal_error("error reading the image data\n");

       }

    }

    /* Compute the `domain' image from the `range' image. Each

       pixel in

     * the domain image is the sum of 4 pixels in the range image.

       We

     * don't average (divide by 4) to avoid losing precision.

     */

     for (y=0; y < y_size/2; y++)

     for (x=0; x < x_size/2; x++) {

         domain[y][x] = (unsigned)range[y<<1][x<<1] + range[y<<1]

         [(x<<1)+1] + range[(y<<1)+1][x<<1] + range[(y<<1)+1][(x<<1)+1];

}

    /* Compute the cumulative data, which will avoid repeated

       computations

     * later (see the region_sum() macro below).

     */

    for (x=0; x <= x_size/2; x++) {

        cum_range[0][x] = 0;

        cum_range2[0][x] = cum_domain2[0][x] = 0.0;

    }

    for (y=0; y < y_size/2; y++) {

        d_sum2 = r_sum2 = 0.0;

        r_sum = cum_range[y+1][0] = 0;

        cum_range2[y+1][0] = cum_domain2[y+1][0] = 0.0;

        for (x=0; x < x_size/2; x++) {

            r_sum += domain[y][x];

            cum_range[y+1][x+1] = cum_range[y][x+1] + r_sum;

            d_sum2 += (double)square(domain[y][x]);

            cum_domain2[y+1][x+1] = cum_domain2[y][x+1] + d_sum2;

            r_sum2 += (double) (square(range[y<<1][x<<1])

                    + square(range[y<<1][(x<<1)+1])

                    + square(range[(y<<1)+1][x<<1])

                    + square(range[(y<<1)+1][(x<<1)+1]));

            cum_range2[y+1][x+1] = cum_range2[y][x+1] + r_sum2;

        }

    }

}

/* ================================================================

 * Free all dynamically allocated data structures for compression.

 */

void compress_cleanup(y_size)

    int y_size;              /* vertical image size */

{

    int s;                   /* size index for domains */

    int class;               /* class number */

    domain_data *dom, *next; /* domain pointers */

    free_array((void**)range,       y_size);

    free_array((void**)domain,      y_size/2);

    free_array((void**)cum_range,   y_size/2 + 1);

    free_array((void**)cum_range2,  y_size/2 + 1);

    free_array((void**)cum_domain2, y_size/2 + 1);

    for (s = MIN_BITS; s <= MAX_BITS; s++)

    for (class = 0; class < NCLASSES; class++)

    for (dom = domain_head[class][s]; dom != NULL; dom = next) {

        next = dom->next;

        free(dom);

    }

}

/* ==================================================================

 * Compute the sum of pixel values or squared pixel values in a range

 * or domain from (x,y) to (x+size-1, y+size-1) included.

 * For a domain, the returned value is scaled by 4 or 16.0

   respectively.

 * x, y and size must all be even.

 */

#define region_sum(cum,x,y,size) \

   (cum[((y)+(size))>>1][((x)+(size))>>1] - cum[(y)>>1]

   [((x)+(size))>>1] \

  - cum[((y)+(size))>>1][(x)>>1]          + cum[(y)>>1][(x)>>1])

/* =================================================================

 * Classify all domains of a certain size. This is done only once to

   save

 * computations later. Each domain is inserted in a linked list

    according

 * to its class and size.

 */

void classify_domains(x_size, y_size, s)

    int x_size;  /* horizontal size of the complete image */

    int y_size;  /* vertical size   of the complete image */

    int s;       /* size index of the domains; their size is

                    1<<(s+1) */

{

    domain_data *dom = NULL; /* pointer to new domain */

    int x, y;                /* horizontal and vertical domain

                                position */

    int class;               /* domain class */

    int dom_size = 1<<(s+1); /* domain size */

    int dom_dist = dom_size >> dom_density; /* distance between

                                      domains */

    /* Initialize all domain lists to be empty: */

    for (class = 0; class < NCLASSES; class++) {

        domain_head[class][s] = NULL;

    }

    /* Classify all domains of this size: */

    for (y = 0; y <= y_size - dom_size; y += dom_dist)

    for (x = 0; x <= x_size - dom_size; x += dom_dist) {

       dom = (domain_data *)xalloc(sizeof(domain_data));

       dom->x = x;

       dom->y = y;

       dom->d_sum  = 0.25  *(double)region_sum(cum_range, x, y,

       dom_size);

       dom->d_sum2 = 0.0625*(double)region_sum(cum_domain2, x, y,

       dom_size);

       class = find_class(x, y, dom_size);

       dom->next = domain_head[class][s];

       domain_head[class][s] = dom;

    }

    /* Check that each domain class contains at least one domain.

     * If a class is empty, we do as if it contains the last created

     * domain (which is actually of a different class).

     */

    for (class = 0; class < NCLASSES; class++) {

        if (domain_head[class][s] == NULL) {

            domain_data *dom2 = (domain_data *)

                                 xalloc(sizeof(domain_data));

            *dom2 = *dom;

            dom2->next = NULL;

            domain_head[class][s] = dom2;

        }

    }

}

/* =================================================================

 * Classify a range or domain.  The class is determined by the

 * ordering of the image brightness in the four quadrants of the

   range

 * or domain. For each quadrant we compute the number of brighter

 * quadrants; this is sufficient to uniquely determine the

 * class. class 0 has quadrants in order of decreasing brightness;

 * class 23 has quadrants in order of increasing brightness.

 *

 * IN assertion: x, y and size are all multiple of 4.

 */

int find_class(x, y, size)

    int x, y;  /* horizontal and vertical position of the range or

                  domain */

    int size;  /* size of the range or domain */

{

    int class = 0;               /* the result class */

    int i,j;                     /* quadrant indices */

    uns_long sum[4];             /* sums for each quadrant */

    static delta[3] = {6, 2, 1}; /* table used to compute the class

                                    number */

    int size1 = size >> 1;

    /* Get the cumulative values of each quadrant. By the IN

       assertion,

     * size1, x+size1 and y+size1 are all even.

     */

    sum[0] = region_sum(cum_range, x,       y,       size1);

    sum[1] = region_sum(cum_range, x,       y+size1, size1);

    sum[2] = region_sum(cum_range, x+size1, y+size1, size1);

    sum[3] = region_sum(cum_range, x+size1, y,       size1);

    /* Compute the class from the ordering of these values */

    for (i = 0;   i <= 2; i++)

    for (j = i+1; j <= 3; j++) {

        if (sum[i] < sum[j]) class += delta[i];

    }

    return class;

}

/* =================================================================

 * Compress a range by searching a match with all domains of the same

   class.

 * Split the range if the mean square error with the best domain

   is larger

 * than max_error2.

 * IN assertion: MIN_BITS <= s_log <= MAX_BITS

 */

void compress_range(x, y, s_log)

    int x, y;    /* horizontal and vertical position of the range */

    int s_log;   /* log base 2 of the range size */

{

    int r_size = 1<<s_log; /* size of the range */

    int class;             /* range class */

    domain_data *dom;     /* used to iterate over all domains of this

    class */

    domain_data *best_dom = NULL; /* pointer to the best domain */

    range_data range;      /* range information for this range */

    affine_map map;        /* affine map for current domain  */

    affine_map best_map;   /* best map for this range */

    uns_long dom_number;   /* domain number */

    /* Compute the range class and cumulative sums: */

    class = find_class(x, y, r_size);

    range.r_sum =  (double)region_sum(cum_range,  x, y, r_size);

    range.r_sum2 = (double)region_sum(cum_range2, x, y, r_size);

    range.x = x;

    range.y = y;

    range.s_log = s_log;

    /* Searching all classes can improve image quality but

       significantly

       slows

     * down compression. Compile with -DCOMPLETE_SEARCH if you can

       wait...

     */

#ifdef COMPLETE_SEARCH

    for (class = 0; class < NCLASSES; class++)

#endif

    for (dom = domain_head[class][s_log];  dom != NULL; dom =

    dom->next) {

        /* Find the optimal map from the range to the domain:

         */

        find_map(&range, dom, &map);

        if (best_dom == NULL || map.error2 < best_map.error2) {

                best_map = map;

                best_dom = dom;

        }

    }

    /* Output the best affine map if the mean square error with the

     * best domain is smaller than max_error2, or if it not possible

     * to split the range because it is too small:

     */

    if (s_log == MIN_BITS ||

        best_map.error2 <= max_error2*((long)r_size*r_size)) {

        /* If the range is too small to be split, the decompressor

           knows

         * this, otherwise we must indicate that the range has not

           been

           split:

         */

        if (s_log != MIN_BITS) {

            OutputBit(frac_file, 1);  /* affine map follows */

        }

        OutputBits(frac_file, (uns_long)best_map.contrast,

        CONTRAST_BITS);

        OutputBits(frac_file, (uns_long)best_map.brightness,

        BRIGHTNESS_BITS);

        /* When the contrast is null, the decompressor does not

           need to

           know

         * which domain was selected:

         */

        if (best_map.contrast == 0) return;

        dom_number = (uns_long)best_dom->y * dom_info[s_log]

        .x_domains

                      + (uns_long)best_dom->x;

        /* The distance between two domains is the domain size

           1<<(s_log+1)

         * shifted right by the domain_density, so it is a power of two.

         * The domain x and y positions have (s_log + 1 - dom_density)

           zero

         * bits each, which we don't have to transmit.

         */

        OutputBits(frac_file, dom_number >> (s_log + 1 - dom_density),

                   dom_info[s_log].pos_bits);

    } else {

        /* Tell the decompressor that no affine map follows because

         * this range has been split:

         */

        OutputBit(frac_file, 0);

        /* Split the range into 4 squares and process them

           recursively: */

        compress_range(x,          y,          s_log-1);

        compress_range(x+r_size/2, y,          s_log-1);

        compress_range(x,          y+r_size/2, s_log-1);

        compress_range(x+r_size/2, y+r_size/2, s_log-1);

    }

}

/* ==================================================================

 * Find the best affine mapping from a range to a domain. This is

   done

 * by minimizing the sum of squared errors as a function of the

   contrast

 * and brightness:  sum on all range pixels ri and domain pixels

   di of

 *      square(contrast*domain[di] + brightness - range[ri])

 * and solving the resulting equations to get contrast and

   brightness.

 */

void find_map(rangep, dom, map)

    range_data  *rangep; /* range information (input parameter) */

    domain_data *dom;    /* domain information (input parameter) */

    affine_map  *map;    /* resulting map (output parameter) */

{

    int ry;            /* vertical position inside the range */

    int dy = dom->y >> 1; /* vertical position inside the

                                      domain */

    uns_long rd = 0;   /* sum of range*domain values (scaled

                          by 4) */

    double rd_sum;     /* sum of range*domain values

                         (normalized) */

    double contrast;   /* optimal contrast between range and

                          domain */

    double brightness; /* optimal brightness offset between range

                          and domain */

    double qbrightness;/* brightness after quantization */

    double max_scaled; /* maximum scaled value =

                          contrast*MAX_GREY */

    int r_size = 1 << rangep->s_log;                 /* the range

                                               size */

    double pixels = (double)((long)r_size*r_size); /* total number of

    pixels */

    for (ry = rangep->y; ry < rangep->y + r_size; ry++, dy++) {

        register image_data *r = &range[ry][rangep->x];

        register unsigned   *d = &domain[dy][dom->x >> 1];

        int i = r_size >> 2;

        /* The following loop is the most time consuming part of the

           whole

         * program, so it is unrolled a little. We rely on r_size

           being a

         * multiple of 4 (ranges smaller than 4 don't make sense

           because

         * of the very bad compression). rd cannot overflow with

           unsigned

         * 32-bit arithmetic since MAX_BITS <= 7 implies r_size

           <= 128.

         */

        do {

            rd += (uns_long)(*r++)*(*d++);

            rd += (uns_long)(*r++)*(*d++);

            rd += (uns_long)(*r++)*(*d++);

            rd += (uns_long)(*r++)*(*d++);

        } while (--i != 0);

    }

    rd_sum = 0.25*rd;

    /* Compute and quantize the contrast: */

    contrast = pixels * dom->d_sum2 - dom->d_sum * dom->d_sum;

    if (contrast != 0.0) {

        contrast = (pixels*rd_sum - rangep->r_sum*dom->d_sum)/contrast;

    }

    map->contrast = quantize(contrast, MAX_CONTRAST, MAX_QCONTRAST);

    /* Recompute the contrast as in the decompressor: */

    contrast = dequantize(map->contrast, MAX_CONTRAST, MAX_QCONTRAST);

    /* Compute and quantize the brightness. We actually quantize the

       value

     * (brightness + 255*contrast) to get a positive value:

     *    -contrast*255 <= brightness <= 255

     * so 0 <= brightness + 255*contrast <= 255 + contrast*255

     */

    brightness = (rangep->r_sum - contrast*dom->d_sum)/pixels;

    max_scaled = contrast*MAX_GREY;

    map->brightness = quantize(brightness + max_scaled,

                         max_scaled + MAX_GREY, MAX_QBRIGHTNESS);

    /* Recompute the quantized brightness as in the decompressor: */

    qbrightness = dequantize(map->brightness, max_scaled + MAX_GREY,

                             MAX_QBRIGHTNESS) - max_scaled;

    /* Compute the sum of squared errors, which is the quantity we

       are

     * trying to minimize:

     */

    map->error2 = contrast*(contrast*dom->d_sum2 -

    2.0*rd_sum) +

    rangep->r_sum2 + qbrightness*pixels*(qbrightness - 2.0*brightness);

}

                /************************************/

                /* Functions used for decompression */

                /************************************/

/*

 * Scale factor for decompression (decompressed size divided by

   original size).

 * Only integer values are supported to simplify the implementation.

 */

int image_scale = 1;

/*

 * An affine map is described by a contrast, a brightness offset, a range

 * and a domain. The contrast and brightness are kept as integer values

 * to speed up the decompression on machines with slow floating point.

 */

typedef struct map_info_struct {

    int contrast;   /* contrast scaled by 16384 (to maintain

                       precision) */

    int brightness; /* brightness offset scaled by 128 */

    int x;          /* horizontal position of the range */

    int y;          /* vertical position of the range */

    int size;       /* range size */

    int dom_x;      /* horizontal position of the domain */

    int dom_y;      /* vertical position of the domain */

    struct map_info_struct *next; /* next map */

} map_info;

map_info *map_head = NULL; /* head of the linked list of all affine

                              maps */

/* ==================================================================

 * This is the main decompression routine.  By the time it gets called,

 * the input and output files have been properly opened, so all it has to

 * do is the decompression.  Note that the decompression routine optionally

 * accepts additional parameters:

 * - the number of iterations, ranging from 1 to 15. The image quality

 *   does not improve much after 8 to 10 iterations. The default is 8.

 * - the scale factor (decompressed size divided by original size)

 */

void ExpandFile(input, output, argc, argv)

    BIT_FILE *input;

    FILE *output;

    int argc;

    char *argv[];

{

    int x_size;         /* horizontal image size */

    int y_size;         /* vertical image size */

    int x_dsize;        /* horizontal size of decompressed image */

    int y_dsize;        /* vertical size of decompressed image */

    int iterations = 8; /* number of iterations */

    int y;              /* current row being written to disk */

    /* Check the command line parameters: */

    for ( ; argc != 0; argv++, argc--) {

        if (argv[0][0] != `-' || argc == 1) {

            fatal_error("Incorrect argument: %s\n", *argv);

        }

        switch(argv[0][1]) {

            case `i': iterations  = atoi(*++argv); argc--; break;

            case `s': image_scale = atoi(*++argv); argc--; break;

            default:  fatal_error("Incorrect argument: %s\n", *argv);

        }

    }

    if (image_scale < 1) {

        fatal_error("Incorrect image scale\n");

    }

    /* Read the header of the fractal file: */

    frac_file = input;

    if (InputBits(frac_file, 8) != `F') {

        fatal_error("Bad fractal file format\n");

    }

    x_size = (int)InputBits(frac_file, 16);

    y_size = (int)InputBits(frac_file, 16);

    dom_density = (int)InputBits(frac_file, 2);

    /* Allocate the scaled image: */

    x_dsize = x_size * image_scale;

    y_dsize = y_size * image_scale;

    range = (image_data**)allocate(y_dsize, x_dsize, sizeof

    (image_data));

    /* Initialize the domain information as in the compressor: */

    dominfo_init(x_size, y_size, dom_density);

    /* Read all the affine maps, by using the same recursive

       traversal

     * of the image as the compressor:

     */

    traverse_image(0, 0, x_size, y_size, decompress_range);

    /* Iterate all affine maps over an initially random image.

       Since the

     * affine maps are contractive, this process converges.

     */

    while (iterations-- > 0) refine_image();

    /* Smooth the transition between adjacent ranges: */

    average_boundaries();

    /* Write the uncompressed file: */

    for (y = 0; y < y_dsize; y++) {

        if (fwrite(range[y], sizeof(image_data), x_dsize, output)

        != x_dsize) {

            fatal_error("Error writing uncompressed image\n");

        }

    }

    /* Cleanup: */

    free_array((void**)range, y_dsize);

}

/* =================================================================

 * Read the affine map for a range, or split the range if the

   compressor

 * did so in the function compress_range().

 */

void decompress_range(x, y, s_log)

    int x, y;    /* horizontal and vertical position of the range */

    int s_log;   /* log base 2 of the range size */

{

    int r_size = 1<<s_log; /* range size */

    map_info *map;         /* pointer to affine map information */

    double contrast;       /* contrast between range and domain */

    double brightness;     /* brightness offset between range and

                              domain */

    double max_scaled;     /* maximum scaled value =

                              contrast*MAX_GREY */

    uns_long dom_number;   /* domain number */

    /* Read an affine map if the compressor has written one at this

       point: */

    if (s_log == MIN_BITS || InputBit(frac_file)) {

        map = (map_info *)xalloc(sizeof(map_info));

        map->next = map_head;

        map_head = map;

        map->x = x;

        map->y = y;

        map->size = r_size;

        map->contrast   = (int)InputBits(frac_file, CONTRAST_BITS);

        map->brightness = (int)InputBits(frac_file, BRIGHTNESS_BITS);

        contrast = dequantize(map->contrast, MAX_CONTRAST,

        MAX_QCONTRAST);

        max_scaled = contrast*MAX_GREY;

        brightness = dequantize(map->brightness, max_scaled +

        MAX_GREY,

                                MAX_QBRIGHTNESS) - max_scaled;

        /* Scale the brightness by 128 to maintain precision later,

           while

         * avoiding overflow with 16-bit arithmetic:

         *     -255 <= -contrast*255 <= brightness <= 255

         * so -32767 < brightness*128 < 32767

         */

        map->brightness = (int)(brightness*128.0);

        /* When the contrast is null, the compressor did not encode

           the

         * domain number:

         */

        if (map->contrast != 0) {

            /* Scale the contrast by 16384 to maintain precision

               later.

             *   0.0 <= contrast <= 1.0 so 0 <= contrast*16384

                 <= 16384

             */

            map->contrast = (int)(contrast*16384.0);

            /* Read the domain number, and add the zero bits that

               the

             * compressor did not transmit:

             */

            dom_number = InputBits(frac_file, dom_info[s_log]

            .pos_bits);

            map->dom_x = (int)(dom_number % dom_info[s_log]

            .x_domains)

                          << (s_log + 1 - dom_density);

            map->dom_y = (int)(dom_number / dom_info[s_log]

            .x_domains)

                          << (s_log + 1 - dom_density);

        } else {

            /* For a null contrast, use an arbitrary domain: */

            map->dom_x = map->dom_y = 0;

        }

        /* Scale the range and domain if necessary. This

           implementation

         * uses only an integer scale to make sure that the union

           of all

         * ranges is exactly the scaled image, that ranges never

           overlap,

         * and that all range sizes are even.

         */

        if (image_scale != 1) {

            map->x *= image_scale;

            map->y *= image_scale;

            map->size *= image_scale;

            map->dom_x *= image_scale;

            map->dom_y *= image_scale;

        }

    } else {

        /* Split the range into 4 squares and process them

           recursively

         * as in the compressor:

         */

        decompress_range(x,          y,          s_log-1);

        decompress_range(x+r_size/2, y,          s_log-1);

        decompress_range(x,          y+r_size/2, s_log-1);

        decompress_range(x+r_size/2, y+r_size/2, s_log-1);

    }

}

/* ===================================================================

 * Refine the image by applying one round of all affine maps on

   the

 * image. The "pure" method would compute a separate new image and

   then

 * copy it to the original image. However the convergence towards

   the

 * final image happens to be quicker if we overwrite the same image

 * while applying the affine maps; for the same quality of

   reconstructed

 * image we need fewer iterations. Overwriting the same image also

 * reduces the memory requirements.

 */

void refine_image()

{

    map_info *map;   /* pointer to current affine map */

    long brightness; /* brightness offset of the map, scaled by

                        65536 */

    long val;        /* new pixel value */

    int y;           /* vertical position in range */

    int dom_y;       /* vertical position in domain */

    int j;

    for (map = map_head; map != NULL; map = map->next) {

        /* map->brightness is scaled by 128, so scale it again

           by 512 to

         * get a total scale factor of 65536:

         */

        brightness = (long)map->brightness << 9;

        dom_y = map->dom_y;

        for (y = map->y; y < map->y + map->size; y++) {

            /* The following loop is the most time consuming, so

               we move

             * some address calculations outside the loop:

             */

            image_data *r  = &range[y][map->x];

            image_data *d  = &range[dom_y++][map->dom_x];

            image_data *d1 = &range[dom_y++][map->dom_x];

            j = map->size;

            do {

                val  = *d++ + *d1++;

                val += *d++ + *d1++;

                /* val is now scaled by 4 and map->contrast is

                   scaled by

                   16384,

                 * so val * map->contrast will be scaled by

                   65536.

                 */

                val = val * map->contrast + brightness;

                if (val < 0) val = 0;

                val >>= 16; /* get rid of the 65536 scaling */

                if (val >= MAX_GREY) val = MAX_GREY;

                *r++ = (image_data)val;

            } while (--j != 0);

        }

    }

}

/* =================================================================

 * Go through all ranges to smooth the transition between adjacent

 * ranges, except those of minimal size.

 */

void average_boundaries()

{

    map_info *map;   /* pointer to current affine map */

    unsigned val;    /* sum of pixel value for current and adjacent

                        ranges */

    int x;           /* horizontal position in current range */

    int y;           /* vertical position in current range */

    for (map = map_head; map != NULL; map = map->next) {

        if (map->size == (1<<MIN_BITS)) continue; /* range

                                                    too small */

        if (map->x > 1) {

            /* Smooth the left boundary of the range and the

               right boundary

             * of the adjacent range(s) to the left:

             */

            for (y = map->y; y < map->y + map->size; y++) {

                 val  = range[y][map->x - 1] + range[y][map->x];

                 range[y][map->x - 1] =

                     (image_data)((range[y][map->x - 2] + val)/3);

                 range[y][map->x] =

                     (image_data)((val + range[y][map->x + 1])/3);

            }

        }

        if (map->y > 1)  {

            /* Smooth the top boundary of the range and the bottom

               boundary

             * of the range(s) above:

             */

            for (x = map->x; x < map->x + map->size; x++) {

                 val  = range[map->y - 1][x] + range[map->y][x];

                 range[map->y - 1][x] =

                     (image_data)((range[map->y - 2][x] + val)/3);

                 range[map->y][x] =

                     (image_data)((val + range[map->y + 1][x])/3);

            }

        }

    }

}

        /*****************************************************/

        /* Functions common to compression and decompression */

        /*****************************************************/

/* ===================================================================

 * Split a rectangle sub-image into a square and potentially two

   rectangles,

 * then split the square and rectangles recursively if necessary.

   To simplify

 * the algorithm, the size of the square is chosen as a power of

   two.

 * If the square if small enough as a range, call the appropriate

   compression

 * or decompression function for this range.

 * IN assertions: x, y, x_size and y_size are multiple of 4.

 */

void traverse_image(x, y, x_size, y_size, process)

    int x, y;             /* sub-image horizontal and vertical

                             position */

    int x_size, y_size;   /* sub-image horizontal and vertical

                             sizes */

    process_func process; /* the compression or decompression

                             function */

{

    int s_size;  /* size of the square; s_size = 1<<s_log */

    int s_log;   /* log base 2 of this size */

    s_log = bitlength(x_size < y_size ? (uns_long)x_size :

    (uns_long)y_size)-1;

    s_size = 1 << s_log;

    /* Since x_size and y_size are >= 4, s_log >= MIN_BITS */

    /* Split the square recursively if it is too large for a

       range: */

    if (s_log > MAX_BITS) {

        traverse_image(x,          y,          s_size/2, s_size/2,

        process);

        traverse_image(x+s_size/2, y,          s_size/2, s_size/2,

        process);

        traverse_image(x,          y+s_size/2, s_size/2, s_size/2,

        process);

        traverse_image(x+s_size/2, y+s_size/2, s_size/2, s_size/2,

        process);

    } else {

        /* Compress or decompress the square as a range: */

        (*process)(x, y, s_log);

    }

    /* Traverse the rectangle on the right of the square: */

    if (x_size > s_size) {

        traverse_image(x + s_size, y, x_size - s_size, y_size,

         process);

        /* Since x_size and s_size are multiple of 4, x + s_size and

         * x_size - s_size are also multiple of 4.

         */

    }

    /* Traverse the rectangle below the square: */

    if (y_size > s_size) {

        traverse_image(x, y + s_size, s_size, y_size - s_size,

        process);

    }

}

/* =================================================================

 * Initialize the domain information dom_info. This must be done

   in the

 * same manner in the compressor and the decompressor.

 */

void dominfo_init(x_size, y_size, density)

    int x_size;       /* horizontal size of original image */

    int y_size;       /* vertical size of original image */

    int density;      /* domain density (0 to 2) */

{

    int s;            /* size index for domains; their size is

                         1<<(s+1) */

    for (s = MIN_BITS; s <= MAX_BITS; s++) {

        int y_domains;            /* number of domains vertically */

        int dom_size = 1<<(s+1);  /* domain size */

        /* The distance between two domains is the domain size

           1<<(s+1)

         * shifted right by the domain density, so it is a power

           of two.

         */

        dom_info[s].x_domains = ((x_size - dom_size)>>(s + 1 -

        density)) + 1;

        y_domains             = ((y_size - dom_size)>>(s + 1 -

       density)) + 1;

        /* Number of bits required to encode a domain position: */

        dom_info[s].pos_bits =  bitlength

            ((uns_long)dom_info[s].x_domains * y_domains - 1);

    }

}

/* ==============================================================

 * Quantize a value in the range 0.0 .. max to the range 0..imax

 * ensuring that 0.0 is encoded as 0 and max as imax.

 */

int quantize(value, max, imax)

    double value, max;

    int imax;

{

    int ival = (int) floor((value/max)*(double)(imax+1));

    if (ival < 0) return 0;

    if (ival > imax) return imax;

    return ival;

}

/* ==============================================================

 * Allocate memory and check that the allocation was successful.

 */

void *xalloc(size)

    unsigned size;

{

    void *p = malloc(size);

    if (p == NULL) {

        fatal_error("insufficient memory\n");

    }

    return p;

}

/* ==============================================================

 * Allocate a two dimensional array. For portability to 16-bit

 * architectures with segments limited to 64K, we allocate one

 * array per row, so the two dimensional array is allocated

 * as an array of arrays.

 */

void **allocate(rows, columns, elem_size)

    int rows;      /* number of rows */

    int columns;   /* number of columns */

    int elem_size; /* element size */

{

    int row;

    void **array = (void**)xalloc(rows * sizeof(void *));

    for (row = 0; row < rows; row++) {

        array[row] = (void*)xalloc(columns * elem_size);

    }

    return array;

}

/* ==========================================================

 * Free a two dimensional array allocated as a set of rows.

 */

void free_array(array, rows)

    void **array;  /* the two-dimensional array */

    int rows;      /* number of rows */

{

    int row;

    for (row = 0; row < rows; row++) {

        free(array[row]);

    }

}

/* =============================================================

 * Return the number of bits needed to represent an integer:

 * 0 to 1 -> 1,

 * 2 to 3 -> 2,

 * 3 to 7 -> 3, etc...

 * This function could be made faster with a lookup table.

 */

int  bitlength(val)

    uns_long val;

{

    int bits = 1;

    if (val > 0xffff) bits += 16, val >>= 16;

    if (val > 0xff)   bits += 8,  val >>= 8;

    if (val > 0xf)    bits += 4,  val >>= 4;

    if (val > 0x3)    bits += 2,  val >>= 2;

    if (val > 0x1)    bits += 1;

    return bits;

}


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Some Compression Results

Some results of the fractal program are shown below in Figure 13.6. We have used the same five grey scale images as in Chapter 11. All images have been compressed with a domain density value of 2.

File | Quality | Starting Size | Compressed Size | Ratio | RMS Error
CHEETAH.GS | 1 | 64000 | 12157 | 82% | 9.9
CHEETAH.GS | 2 | 64000 | 12157 | 82% | 9.9
CHEETAH.GS | 3 | 64000 | 12140 | 82% | 9.9
CHEETAH.GS | 4 | 64000 | 12094 | 82% | 9.9
CHEETAH.GS | 5 | 64000 | 11993 | 82% | 9.9
CHEETAH.GS | 10 | 64000 | 10017 | 85% | 10.4
CHEETAH.GS | 15 | 64000 | 6613 | 90% | 12.4
CHEETAH.GS | 20 | 64000 | 3888 | 94% | 15.1
CLOWN.GS | 1 | 64000 | 11961 | 82% | 7.9
CLOWN.GS | 2 | 64000 | 11206 | 83% | 8.0
CLOWN.GS | 3 | 64000 | 10137 | 85% | 8.0
CLOWN.GS | 4 | 64000 | 9571 | 86% | 8.0
CLOWN.GS | 5 | 64000 | 9203 | 86% | 8.1
CLOWN.GS | 10 | 64000 | 6144 | 91% | 9.2
CLOWN.GS | 15 | 64000 | 3937 | 94% | 10.9
CLOWN.GS | 20 | 64000 | 2643 | 96% | 12.7
LISAW.GS | 1 | 64000 | 12136 | 82% | 2.7
LISAW.GS | 2 | 64000 | 11246 | 83% | 2.7
LISAW.GS | 3 | 64000 | 7945 | 88% | 3.0
LISAW.GS | 4 | 64000 | 5208 | 92% | 3.4
LISAW.GS | 5 | 64000 | 3704 | 95% | 3.8
LISAW.GS | 10 | 64000 | 1296 | 98% | 5.7
LISAW.GS | 15 | 64000 | 951 | 99% | 6.9
LISAW.GS | 20 | 64000 | 796 | 99% | 7.4
ROSE.GS | 1 | 64000 | 8707 | 87% | 8.0
ROSE.GS | 2 | 64000 | 8703 | 87% | 8.0
ROSE.GS | 3 | 64000 | 8597 | 87% | 8.0
ROSE.GS | 4 | 64000 | 8349 | 87% | 8.1
ROSE.GS | 5 | 64000 | 7885 | 88% | 8.1
ROSE.GS | 10 | 64000 | 5065 | 93% | 9.0
ROSE.GS | 15 | 64000 | 3369 | 95% | 10.3
ROSE.GS | 20 | 64000 | 2280 | 97% | 12.5
MOUSE.GS | 1 | 64000 | 9580 | 86% | 2.8
MOUSE.GS | 2 | 64000 | 7546 | 89% | 2.8
MOUSE.GS | 3 | 64000 | 5323 | 92% | 2.9
MOUSE.GS | 4 | 64000 | 4411 | 94% | 3.1
MOUSE.GS | 5 | 64000 | 3571 | 95% | 3.3
MOUSE.GS | 10 | 64000 | 1379 | 98% | 5.1
MOUSE.GS | 15 | 64000 | 989 | 99% | 6.3
MOUSE.GS | 20 | 64000 | 798 | 99% | 7.1

Figure 13.6 Compression Results

Figure 13.6 can be compared with Figure 11.13 giving the results of DCT compression with some caution: the RMS error value is not an accurate measure of image quality, and the two programs are simple implementations which can both be improved. Yet from the comparison it is apparent that DCT compression performs better than fractal compression at low compression ratios, and that fractal compression is much better at high compression ratios. This is confirmed by visual inspection of the images.

The fractal compression program can achieve compression ratios that the DCT program cannot reach without extreme degradation of the image. For example, the LISAW image can be compressed to 1752 bytes (97% compression ratio) with fractal compression and still retain a natural aspect. Of course some degradation of the image is unavoidable at such a compression ratio, but the image quality is still acceptable at least for some applications where high-fidelity is not an absolute requirement. With the DCT compression program, LISAW compressed to 4170 bytes (more than twice as large as the fractal version) has an extremely "blocky" aspect which is immediately apparent.

Even a production-quality JPEG compressor cannot reach the quality of our simple fractal compression program at such high compression ratios. This is clearly visible in Figure 13.7, where both compressed files have approximately the same size (1752 bytes for the fractal version, 1741 bytes for the JPEG version). The fractal version was produced with quality factor 8 and domain density 2; the JPEG quality factor was chosen to get an equivalent size for the compressed file.

Fractal compression, 1752 bytes

Figure 13-7 JPEG compression, 1741 bytes

In general, there is little improvement in the image quality between quality factors 2 and 1. This is due to the simple nature of our algorithm, which reaches its limits at high quality factors and low compression ratios. However even with a more complex algorithm, a fractal-based method may not be able to beat DCT-based methods at low compression ratios. Fractal compression really shines for high compression ratios, or when zooming on a portion of an image or enlarging a complete image.

The compression ratios obtained with our simple fractal compression program could be further improved by taking into account possible symmetries and rotations between ranges and domains, by using non-square ranges and domains, by using more bits to encode contrast and brightness in. the affine transformations, and by using Huffman or arithmetic encoding on the parameters of affine transformations.

The fractal compression program is slower than the DCT program, but the decompression is faster even in this simple implementation. The decompression speed could be further improved significantly by taking advantage of the resolution-independence of fractal compression: the image could be decoded at a lower resolution in the first iterations, and decoded at the full image size only for the last two iterations. While working at a lower resolution, the decoder need only work with a fraction of all the pixels of the original image, thus the total number of instructions required for decoding is much reduced.


Data Compression

 Data Compression - Mark Nelson, Jean-Loup Gailly

Patents

The field of data compression is mined with patents, and fractal image compression is no exception. Most patents are owned by Iterated Systems, Inc. US patent 4,941,193 covers the "graduate student algorithm" for fractal compression with IFS, which is not practical as we have seen.

US patent 5,065,447 (continued in 5,347,600) covers fractal compression with the "Fractal Transform", that is, with Partitioned Iterated Function Systems. The sample program given in the chapter is provided only for educational purposes, as it may be affected by these patents. If you wish to use it for other purposes, you must contact Iterated Systems at the following address in the USA:

5550A Peachtree Parkway, Suite 650
Norcross, GA 30092

Iterated Systems also owns US patent 5,384,867 (continued in 5,430,812) on hardware boards performing fractal compression.

US patent 5,416,856, filed by Jacobs, Boss, and Fisher, covers a refinement of fractal compression with PIFS, where at least one of the domain to range transformations is non-contractive. The inventors observed that it is not necessary to impose contractivity on any given transformation, as long as the PIFS consisting of the union of all transformations is eventually contractive after some iterations. The contractive transformations may dominate the expansive ones after two or more iterations. Our sample program does not use this refinement.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Preface

Thank you for buying this book. I hope it provides you the knowledge and insight the subject deserves. This book has proven to be an adventure, leading me to meet Pedro Meyer and E.R. Beardsley. I also became acquainted with Jan Lindenbaum, Olle Nilsson, and Alf Jansson (Swedish Television in Stockholm, Sweden) who assisted with the JAS-Gripen crash images. I thank Ben Dawson for a superlative technical edit, Simon Yates for a helpful development edit, and Betsy Bayha for taking the time to glance at the legal and social issues chapters. Kudos as well to Jim Hill, Beth Kohler, and the numerous folks at Wordware Publishing, Inc. whose gracious efforts have produced this book. The remaining errors are all mine.

Future book editions will also acknowledge contributors, perhaps even yourself—please read the Call to Readers. Good luck in your imaging endeavors and feel free to contact me through Wordware Publishing, Inc.

About This Book

This is an introductory digital imaging book for readers lacking extensive mathematical skills. Consequently, a principal goal is to achieve understanding while eliminating complex mathematical entry barriers. By relying on understandable analogies, the material attempts to avoid impenetrable barriers necessarily contained in traditional imaging discussions. This allows readers to concentrate on understanding theory, processes, and expected outcomes rather than coefficient indefiniteness, Poission distributions, eigenvalues, etc. But, truth be known, academic rigor is lost though intuition is gained.

I layered the material in increasing technical depth to make it accessible to various audiences. The first section deals primarily with light, vision system anatomy, and color perception. This material explains why some imaging aspirations are physically unattainable. Following sections assume readers read this material.

The second section discusses essential imaging peripherals: CCD components found in cameras and scanners, display subsystems, and low-cost commercial printers. It also includes a discussion of device independent color, explaining why it is difficult to reproduce faithful images on different devices. Associated appendices cover display controller and printer (color and black and white) programming considerations essential for an end-to-end imaging system perspective.

The third section painlessly presents a few introductory mathematical concepts and briefly discusses image file formats. It discusses monochromatic and elementary color space, and digital halftone theory and color quantization. I have yet to find this section's material within any other single book.

The fourth section provides listings and detailed explanatory discussions for the previous three sections. This section clearly illustrates, by example, digital image processing programming. While the listings are deliberately lowest common denominator MS-DOS programs, readers can port the processing routines to other operating system environments. While MS-DOS is finally fading, its use here is helpful since it isolates image processing mechanics from numerous other complex programming distractions. The programs also operate with MS-DOS consoles available from Microsoft Windows 95.

The fifth and final section addresses the legal and social issues that digital imaging technology presents.

W. David Schwaderer
Saratoga, California
June 1997

Call To Readers

Books frequently omit relevant material through oversight or pressing schedules. If you are aware of an error, an interesting point, or a technique this book should include but omits, the author would like to hear about it and genuinely seeks your insights. The acknowledgment section below will contain contributing readers' insights when the contributions are incorporated in future editions. Please send written suggestions on paper to the author via Wordware Publishing, Inc. Contributions must be new and accompanying evaluation reference material is useful.

Contributions become the property of the author who reserves suggestion final disposition determination.

Contributing Reader Acknowledgments

Contributor | Suggested Enhancement
Your name here... City, Country | Your edited suggestion here...


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Bibliography

Books

Aukstakalnis, Steve. Silicon Mirage. Berkeley: Peachpit Press, 1992.

Baxes, Gregory A. Digital Image Processing: A Practical Primer. Englewood Cliffs: Prentice-Hall, 1984.

Berry, Richard. Choosing and Using a CCD Camera. Richmond: Willmann-Bell, 1992. Author comment: A splendid and happy book. Check out Richard's WWW site.

__________. Introduction to Astronomical Image Processing. Richmond Willmann-Bell, 1992. Author comment: Another splendid and happy book.

Billmeyer, Fred W., and Max Saltzman. Principles of Color Technology, Second Edition. New York: Wiley-Interscience, 1981. Author comment: A classic.

Binstock, Andrew, et al. HP LaserJet Programming. Reading: Addison Wesley, 1991.

Blatner, David, and Steven F. Roth. Real World Scanning and Halftones. Berkeley: Peachpit Press, 1993.

Buil, Christian. CCD Astronomy. Richmond: Willman-Bell, 1991. Author comment: The reigning classic with lots of detail.

Burrows, William E. Deep Black. New York: Berkeley Books, 1988. Author comment: Incredible spy photos inside.

Chaisson, Eric J. Hubble Wars. New York: Harper Collins, 1990.

Conrac Corporation. Raster Graphics Handbook. New York: Van Nostrand Reinhold, 1985.

Foley, James D., et al. Computer Graphics: Principles and Practice - Second Edition. Reading: Addison Wesley, 1990.

Gonzalez, Raphael C., and Richard E. Woods. Digital Image Processing. Reading: Addison Wesley, 1992.

Green, William B. Digital Image Processing: A Systems Approach. New York: Van Nostrand Reinhold, 1989.

Gregory, Richard L. Eye and Brain. Princeton: Princeton Press, 1990.

Hunt, R.W.G. The Reproduction of Color. New York: Wiley, 1975. Author comment: A classic.

Jackson, Richard, et al. Computer Generated Color. New York: John Wiley, 1994.

Kay, David C., and, John R. Levine. Graphics File Formats. Blue Ridge Summit: Windcrest/McGraw Hill, 1992.

Luse, Marv. Bitmapped Graphics Programming in C++. Reading: Addison Wesley, 1993.

McMahan, Robert. Pixel Photography. Lake Hughes, Calif.: Olive Press, 1993.

Mitchell, William J. The Reconfigured Eye: Visual Truth in the Post-Photographic Era. Cambridge: The MIT Press, 1992. Author comment: An absolutely, incredibly superlative book.

Morrison, Mike. The Magic of Image Processing. Indianapolis: Sams Publishing, 1993.

Nelson, Mark. The Data Compression Book. San Mateo: M&T Books, 1992.

Oliver, Dick, et al. Tricks of the Graphics Gurus. Indianapolis: Sams Publishing, 1993.

Peddie, John. High-Resolution Graphics Display Systems. New York: Windcrest/McGraw Hill, 1994.

__________. Multimedia & Graphics Controllers. New York: Windcrest/McGraw Hill, 1994.

Rimmer, Steve. Windows Bitmapped Graphics. Blue Ridge Summit: Windcrest/McGraw Hill, 1993.

__________. Graphical User Interface Programming. Blue Ridge Summit: Windcrest/McGraw Hill, 1992.

__________. Bitmapped Graphics. Blue Ridge Summit: Windcrest/McGraw Hill, 1990.

__________. SuperVGA Graphics Programming Secrets. Blue Ridge Summit: Windcrest/McGraw Hill, 1993.

__________. Bitmapped Graphics: Second Edition. Blue Ridge Summit: Windcrest/McGraw Hill, 1993.

__________. The Graphics File Toolkit. Reading: Addison Wesley, 1992.

__________. Supercharged Bitmapped Graphics. Blue Ridge Summit: Windcrest/McGraw Hill, 1992.

Ritchin, Fred. In Our Own Image: The Coming Revolution in Photography. New York: Aperture, 1990. Author comment: Years ahead of its time, a fast must-read by a master.

Roth, Steven F. Real World PostScript. Reading: Addison Wesley, 1988.

Southworth, Miles, and Donna Southworth. Color Separation on the Desktop. Livornia, New York: Graphics Arts Publishing, 1993.

Summers, Anthony, Official and Confidential: The Secret Life of J. Edgar Hoover. New York: Putnam, 1993.

Sutty, George, and Steve Blair. Advanced Programmer's Guide to the EGA/VGA. New York: Brady, 1988.

__________. Advanced Programmer's Guide to the SuperVGAs. New York: Brady, 1990.

Wandell, Brian A. Foundations of Vision. Sunderland: Sinauer Associates, 1995. Author comment: Fantastic book.

Wilton, Richard. PC & PS2 Video Systems. Redmond: Microsoft Press, 1987.

Wolf, Robert S., and Larry Yaeger. Visualization of Natural Phenomena. Santa Clara, Calif.: Telos, 1993. Author comment: A fantastic book.

Wyszwcki, Gunter, and W.S. Stiles. Color Science: Concepts and Methods, Quantitative Data and Formulae, Second Edition. New York: John Wiley, 1982. Author comment: Fantastic book with all the math and more.

Newspapers

Betty Crocker: San Jose Mercury News, 9/12/95, Pg. 1A; 9/27/95, Pg. 1F; 3/20/96, Pg. 1C.

Color Blindness: San Jose Mercury News, 3/26/97, Pg. 12A.

Digital Imaging: Time, 8/20/84 Pg. 4, 25; Fall 1989, Pg. 66 ff.

Digital Manipulation: San Jose Mercury News, 8/8/96, Pg. 13A.

Digital Manipulation: Hufvudstadsbladet, 2/3/89, Pg. 1.

Digital Manipulation: Scientific American, February 1994 Cover Story, Pg. 68.

Digital Manipulation: Discover, July 1990, Pg. 54.

Dr. Martin Luther King Jr.: San Jose Mercury News, 10/23/96, Pg. 1A.

Gordon Joyner: The Atlanta Journal/Constitution, 11/5/94, Pg. B4; 12/5/94, Pg. A8; 12/6/94 Viewpoints; 12/17/94, Pg. C1; 12/29/94, Pg. D3; 1/14/95, Pg. B8; 1/19/95, Pg. E1; 1/21/95, Pg. B10; 2/1/95, Pg. 5A.

Gordon Joyner: The Fulton County Daily Report, 1/10/95, Pg. 1.

Image Authenticity: San Jose Mercury News, Letter to Editor, 11/3/96, Pg. 5E.

John W. Warner: The Washington Post, 10/10/96, Pg. A18.

Morphing: San Diego Union Tribune, 4/27/93, photo morph of Boris Yeltsin to Bill Clinton.

O.J. Simpson: San Jose Mercury News, 1/7/97, Pg. 3B; 1/26/97, Pg. 2C; 1/24/97, Pg. 3B; 2/5/97, Pg. 1A.

O. J. Simpson: Time, 4/4/94 Pg. 4; 7/18/94, Pg. 5.

Pocahontas: San Jose Mercury News, 7/9/95, Pg. 21A.

Retinal Imaging: Business Week, 11/11/91, Pg. 162M.

Satellite Imaging:San Jose Mercury News, 3/24/94, Pg. 3B.

Vietnam Images: San Jose Mercury News, 4/13/97, Pg. 27A.

Video Applications: San Jose Mercury News, 5/19/95, Pg. 1B; 11/14/95, Pg. 14E.

Radio Programs

National Public Radio Morning Edition, 6/22/95 Segment #14, "Sculptor Sues Warner Bros. for Big Fat Bat Bucks."

Court Filings

Andrew Leicester v. Warner Bros., United States District Court for the Central District of California.

Gordon Joyner: Civil Action File No. E32255, Superior Court of Fulton County, State of Georgia, 1/9/95.

Rock and Roll Hall of Fame & Museum, Inc. v. Gentile Productions and Charles Gentile, Case No. 1:96CV0899.

Magazines

Popular Science. Various issues

Spy. September 1991, March 1992, February 1993.

Advanced Imaging.

Videography.

Image Blending: Time, Fall 1993.

Standards

American Society of Newspaper Editors, Report of the Ethics Committee, 1990-1991.

Hewlett-Packard Corp., A Guide to the Tag Image File Format Version 5.0, Greeley, Colorado, 8/1/91.

National Press Photographers Association, Inc., Statement of Principle, July 1991.

Video Electronics Standards Association, VESA Super VGA Standard, VBE Version 1.2 10/91.

Articles

Ethics: FineLine: The Newsletter on Journalism Ethics, Louisville, Kentucky, February and July 1990.

Imaging: Image Processing, Byte Magazine Special Section, March 1987, Pg. 140.

Physics: Starkweather, Gary, "Digital Color Printing," Physics Today, December 1992, Pg. 60.

Spy Photos: Jane's Defence Weekly, 11/8/84, Pg. 171. Author comment: The most amazing spy satellite photos ever stolen and published.

Academic Papers and Proceedings

Harris, Christopher R. "Digitization and Manipulation of News Photographs," Journal of Mass Media Ethics, 1991, Pg. 164.

Long, John (past NPPA president). "The Ethics of Digital Manipulation: Areas for Discussion," NPPA Proceedings, 1990.

Long, John. " 'Truth' and the New Technology," News Photographer, August 1990, Pg. 14.

Eastman Kodak Company. "Ethics, Copyright, and the Bottom Line: A Symposium on Digital Technologies and Professional Photography," Center for Creative Imaging, Camden, Maine, 1992.

Floyd, Robert W., and Louis Steinberg. "An Adaptive Algorithm for Spatial Greyscale," SID 75, International Symposium for Digital Technology Papers, 1975, Pg. 36.

Heckbert, Paul. "Color Image Quantization for Frame Buffer Display," Computer Graphics, July 1982, Pg. 297.

Knuth, Donald E. "Digital Halftones by Dor Diffusion," ACM Transactions on Graphics, October 1987, Pg. 245.

Levy, Alvy Ray. "Color Gamut Transform Pairs," Computer Graphics, August 1978, Pg. 12.

Goertzel, Gerald, and Gerhard R. Thompson. "Digital Halftoning on the IBM 4250 Printer," IBM Journal of Research and Development, January 1987, Pg. 2.

Strickland, Robin N., et al. "Digital Color Enhancement Based on the Saturation Component," Optical Engineering, July 1987, Pg. 609.

Society for Imaging Science and Technology, SPSE's 43rd Annual Conference, Advanced Printing of Conference Summaries, Rochester, May 20-25, 1990.

Tomlinson, Don E., and Christopher R. Harris. "Free-Lance Photojournalism in a Digital World: Copyright, Lanham Act, and Droit Moral Considerations Plus a Sui Generis Solution," Federal Communications Law Journal, December 1992, Pg. 1.

Tomlinson, Don E. "Digitization of Eyewitness News: The future of television newspictures as research tools," Southwestern Mass Communication Journal, Vol. 5, No. 1, 1989, Pg. 36.

__________."People Do Read Large Ads: The Law of Advertising from Outer Space," Federal Communications Law Journal, Vol. 5, No. 1, 1989, Pg. 36.

__________."Computer Manipulation and Creation of Images and Sounds: Assessing the Impact," The Annenberg Washington Program, Communication Policy Studies, Northwestern University, 1993.

__________. "Journalism and Entertainment as Intellectual Property on the Information Superhighway: The Challenge of the Digital Domain" and "Beyond the Internet: Settling the Electronic Frontier," Stanford Law and Policy Review, Vol. 6, No. 1, 1994, Pg. 61.

__________. "Ecoalesce or Collide? Ethics, Technology, and TV Journalism in 1991," Journal of Mass Media Ethics, Vol. 47, No. 3, April 1995, Pg. 535.

__________. "Digitexed Television News: The Beginning of the End for Photographic Reality in Photojournalism," Business and Professional Journal, Vol. 11, No. 1.

Company Literature

Agfa: An Introduction to Digital Colour Prepress

Digital Color Prepress, Volume 2

A Guide to Color Separation: Digital Color Prepress Volume Two

An Introduction to Digital Scanning: Digital Color Prepress Volume Four

An Introduction to Digital Color Printing

An Introduction to Digital Photo Imaging

A Guide to Digital Photography

Digital Imaging Group, Westlake Village, Calif.

Lyra Research, Inc.: The Hard Copy Observer, Newton Highlands, Mass., 1993. Author comment: The definitive journal of the printer industry. Call (617) 332-0708 for details.

Hewlett-Packard: DeskJet Printer Family, Camas, 1992.

Photometrics: Charged Coupled Devices for Quantitative Electronic Imaging, Tucson, Ariz., 1992.

Scientific Imaging Technologies, Inc.: An Introduction to Scientific Imaging Charged-Coupled Devices, Beaverton, Ore., 1994 (also available on their WWW site).

Xerox Corp.: Color Encoding Standard, Sunnyvale, Calif., 1990.

Xerox Corp.: Color Printing for Computer Graphics, Maureen C. Stone, Palo Alto, Calif., 1988.

X-Rite, Inc.: Understanding Color Communication, Grandville, Mich., Rev. 8-90.

WWW Sites

Biometrics: http://www.state.il.us/dpa/iscan.htm

http://www.sjb.co.uk/btt.html

http://www.raycosecurity.com/access/EyeDentify.html

http://www.and.nl/id/gen_biom.html

John Filo Kent State photo alteration:

http://www.cris.com/~Mppa/ethics.html

http://zonezero.com/


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer
Q

Index

6x6x6 palette, 85

A

aberration, 9
accommodation,19
achromatic, 10, 110
active pixel sensor, 36
acuity, 13, 20, 24
adaptation, 18
adaptive equalization, 116
adaptive histogram equalization, 116
adaptive palette, 83
additive color, 3, 53, 72
additive light, 72
aliasing, 94
aligning, 131
aperture mask, 50
APS, see active pixel sensor
area operations, 117-127
artifacts, 153
ascending, 61
aspect ratio, 49
averaging, 126

B

back illuminated device, 35
bands, 56
bandwidth, 80
bank switching, 41
bilevel, 147
binning, 36
bit depth, 93
bit planes, 110
blanking amplifier, 46
bleed, 56
blooming, 35
blue-block, 19
brightness, 11
browser safe palette, 85
browsers, 84

C

cathode, 44
cathode ray tube, 43
CCD, 31-37
CCD imaging device, 32
CCD pixels, 32
cell, 54
Centronics, 53
channel stop, 32
charge transfer, 35
charge-coupled devices, 31-37
checkerboard, 112
chroma range, 28
chromatic aberration, 9
chrominance, 25
CIE, 5
CIE luminous efficiency function, 12
CIE XYZ model, 140-145
clocking the gates, 35
clustered dot ordered dither, 151
CMY model, 132-134
CMYK model, 135
color,
additive, 3, 4, 53, 72
complementary, 4
secondary, 5
subtractive, 3, 53, 72, 74
color models, 129-146
color plane, 53
color quantization, 159-164
color ramp, 42
colorant, 5, 53, 56
complementary color, 4
CompuServe, 81
cones, 9, 20, 71
connector, 40
continuous, 55
contrast, 11
control grid, 44, 47
copyright, 187
cornea, 9, 18
critical fusion frequency, 14
CRT, 43
custom palette, 83

D

DAC, 40
DAC width, 42
dark current, 35
data processing mode, 54, 61
daylight vision, 9
decay, 45
dental office, 19
descending, 61
DeskJet, 59
diffusion, 56
digital halftone, 147
digital to analog (DAC), 40
digitization, 93-94
direct color, 41
dither-free palette, 85
dithering, 83, 148
dithers, 149-152
donor, 56
dot mask, 50
dot thermal transfer, 56
DRAM, 40
drop-on-demand, 55
dye, 53
dye diffusion, 57
dye sub, 57
dye sublimation, 57
dynamic range, 8

E

electromagnetic energy, 2
electromagnetic spectrum, 2
electron beam, 44
electron gun, 44
electron stream, 44
electron-hole, 32
electrostatic, 55
EM, see electromagnetic energy
encoding, 32
Endless Summer, 94
equalization, 116
equi-energy light, 4
error diffusion, 83, 153-158
escape character, 58
escape sequence, 58
Eschback, Dr. Reiner, 157
excited phosphor, 45

F

face, 44
fair use, 187
Fechner's paradox, 19
fields, 48
filter kernel, 98
flicker, 28
flickering, 14
flickering hazards, 14
Floyd Steinberg, 154, 156
focus, 45
footroom, 131
fovea, 9, 21
frame buffer, 40, 43
frame transfer, 36
frequency, 2, 87
front illumination, 35
Fujicopian Co., 57
full frame CCD, 36
fully saturated color, 4
fuser, 55
fusion frequency, 14

G

gamma correction, 50, 87, 88-91
gif, 81
GIF, 81, 82-85
Goertzel, Gerald, 157
Graphics Interchange Format, 81
graphics mode, 54
gray balance, 132

H

halftone, 147
headroom, 131
Heckbert, Paul, 159, 162
Hewlett-Packard, 55, 59, 66, 76, 133, 147
high dynamic range, 35
high-pass filter, 120
HIS model, 138-139
histogram, 94, 111-112, 163
histogram equalization, 116-117
histogram slide, 114-115
histogram stretch, 115-116
HLS model, 138-139
horizontal deflection, 46
horizontal synch, 46
HP, see Hewlett-Packard
HSB model, 136-137
HSI model, 139
HSL model, 138-139
HSV model, 136-137
Hubbell telescope, 13
hue discrimination, 28

I

IBM, 54, 59, 157
ideal colorant, 74
illuminant, 10
illumination source compensation, 14, 28
impact printer, 54
infrared, 17
ink, 53
ink gutter, 55
ink jet, 55
integration time, 32
intensity, 10
interlaced, 28, 48
invar, 50
iris, 18
iron, 50
irrelevant information, 95

J

jaggies, 94
Jarvis, Judice, and Ninke, 155
JFIF, 81
Joint Photographic Experts Group, 81
JPEG, 81, 82, 84-85
JPEG File Interchange Format, 81
jpg, 81

L

Laplacian filters, 125
L-cone, 9, 11-12
Leicester, Andrew, vs. Warner Brothers Studio, 182-184
Lempel-Ziv & Welsh, 81
lens, 7, 19
light, 1
additive, 72
equi-energy, 4
subtractive, 5, 74-76
visible, 7
light adaptation, 23
light adapted, 12
line spacing, 61
linearity, 47
look-up table, 88
lossless compression, 81
lossy compression, 81
low-pass filter 122
lumen, 11
luminance, 10
luminous efficiency function, 12
LUT, see look-up table
LZ77, 81
LZW, 79, 81

M

mapping function, 113
mask, 50
masking, 95, 103-106
Maxwell, James Clerk, 71
M-cone, 9, 11-12
media, 53
median, 127
median cut, 159, 162-163
median filter, 127
metamer, 6, 20
midpoint correction, 89
mode, 49, 58
models, 129
monitor, 43
monochromatic, 109
Mount Umunhum, 92

N

nanometers, 2
neural processing cells, 9
nickel, 50
night vision, 9
noise removal, 126
non-impact printer, 54
non-interlaced, 28, 48
non-spectral color, 4
non-vibrated, 55
normalization, 87-88
nozzle, 56

O

optic nerve, 7
ordered dither, 148

P

packed pixel, 41
palette, 40
palette registers, 41
pane, 56
parallel binning, 36
PCX, 77, 78-80
pels, 110
perception, 23
persistence, 45, 48
phosphor, 44
phosphors, 48
photoelectric effect, 32
photon, 2
photopic, 9, 20
photopic vision, 23
photosite, 32
piezoelectric, 55
pixel, 32
pixel shape, 49
pixels, 110
png, 81
PNG, 81
point operations, 113-117
polysilicon gate structures, 32
Portable Network Graphics, 81
primary color, 5
Principle of Refraction, 3
print head, 54
printer, 53
probability, 87, 94
progressive display, 83
Proprinter, 54, 60-63
pupil, 7, 18
Purkinje shift, 24

Q

QE, see quantum efficiency
quantization, 87, 91-93
color, 159-164
quantize, 92
quantum efficiency, 34

R

radiance, 10
random dither, 148
raster display, 43
realistic colorant, 75
receiver media, 57
refraction, 3
refresh, 47
removing noise, 126
retina, 7, 20, 71
retinal action time, 24
RGB model, 129-132
Rock and Roll Hall of Fame & Museum, 184-185
rods, 9, 20
run length encoding, 80

S

sample area, 32
satellites, 56
saturated color, 4
saturation, 35
scanline, 61
S-cone, 9, 11-12
scotopic vision, 23
screen refresh, 47
screen resolution, 41
SCSI, 53, 80
secondary color, 5
separations, 71
serial amplification, 34
serial binning, 36
serial register, 33
serpentine raster, 156
set of weighting coefficients, 98
shadow mask, 50
sharpening, 13, 25, 95, 100-103, 119-121
slew rate, 47, 50
slide, 114
Smith, Alvy Ray, 136
smoothing, 95, 97-100, 122
Snell, 3
softening compensation, 156
source compensation, 14
spatial filter, 117-119
spatial frequency, 95
spectral colors, 3
spectral response function, 11
spectral sensitivity function, 11
square pixel, 49
standard browser palette, 85
Stevenson and Arce, 155
stretch, 115
Stucki, 155
subtractive color, 3, 53, 72, 74
subtractive light, 5, 74-76
Super VGA, 39
sweep voltage, 46
system bus, 40

T

Tagged Information File Format, see TIFF
tap, 117
text mode, 58
thermal, 55
thermal dye diffusion, 57
thermal printer, 53
thermal wax, 56
Thompson, Gerald, 157
threshold modulation, 157
TIFF, 77
Treviranus, 20
tritanopes, 12

U

unbalanced gray, 132
uniform response, 35
Unisys Corp., 79
unsaturated color, 4
unsharp masking, 95, 103-106, 122
USGS, 92

V

variance, 126
VBE, 39
vertical deflection, 46
vertical synch, 46
VESA, 39
vibrated, 55
video adapter, 40
Video BIOS Extension, 39
Video Electronics Standards Association (VESA), 39
video mode, 40
video output signal, 47
Vietnam Women's Memorial, 179-182
visible light, 7
visually lossless, 95
VRAM, 40

W

Warner Brothers Studio, 182-184
watt, 10
wave energy, 95
wave frequency, 94-95
wavelength, 2
Wedge, 94
weight, 117
weighting coefficients, 98
white point, 28
working range, 8, 25
Wu, Xiaolin, 159, 164

Y

Y'CbCr model, 135-136
yoke, 45
Young, Thomas, 10, 11, 27
Young-Helmholtz theory, 11, 27

Z

ZSoft, 77

Other Books from Wordware Publishing, Inc.

Advanced CGI Techniques for Information Processing
Collaborative Computing with Delphi 3
The Complete Communications Handbook (2nd Ed.)
Demystifying ATM/ADSL
Demystifying ISDN
Demystifying TCP/IP (2nd Ed.)
Developer's Guide to HP Printers
Developing Enterprise Applications with PowerBuilder 6.0
Developing Internet Information Services
The HTML Example Book
Illustrated UNIX System V
Internet Publishing with Microsoft Word 98
Learn ACT! 1998 for the Advanced User
Learn ACT! 3.0 for Windows 95
Learn ActiveX Development Using Visual Basic 5.0
Learn ActiveX Development Using Visual C++ 98
Learn ActiveX Scripting with Microsoft Internet Explorer 4.0
Learn Advanced HTML 4.0
Learn Advanced Internet Relay Chat
Learn Advanced JavaScript Programming
Learn Advanced MFC Programming with Visual C++ 98
Learn Archiving and File Compression Programming with Visual C++ 98
Learn AutoCAD LT for Windows 95
Learn Computers in a Day
Learn Graphics File Programming with Delphi 3
Learn Encryption Techniques with Visual Basic 98
Learn Internet Publishing with Microsoft Publisher 97
Learn Internet Relay Chat
Learn Lotus Domino
Learn Microsoft Access 98 Programming for the Advanced User
Learn Microsoft Active Desktop Programming using Visual C++ 98
Learn Microsoft Exchange Server 5.5 Core Technologies
Learn Microsoft FrontPage 97
Learn Microsoft Office 95
Learn Microsoft Office 97
Learn Microsoft Transaction Server Development using Visual C++ 98
Learn Pascal in Three Days (2nd Ed.)
Learn Perl 5 for the Advanced User
Learn P-CAD Master Designer
Learn the MFC C++ Classes
Learn the MFC C++ Classes for Visual C++ 98
Learn Visio 5.0
Learn Visio 6.0 for the Advanced User
Learn Visual Basic 5.0 in Three Days
Lotus Notes Developer's Guide
Nathan Wallace's Delphi 3 Example Book
Networking with Microsoft Windows NT 5.0
Networking with Windows NT
Object-Oriented Sofware Project Management
Practical Guide to Intranet Client-Server Applications Using the Web
Practical Guide to SGML Filters
Practical Guide to SGML/XML Filters
Tomes of Delphi 3: Win 32 Core API
Tomes of Delphi 3: Win 32 Graphical API
The Ultimate Computer Buyer's Guide
Visio 4 for Everyone
The Visual Basic 4.0 Example Book
The WordBasic Example Book
Writing and Publishing with Your PC

The Companion CD

The companion CD-ROM contains source code in C, executable programs, and PCX images. The directory structure is as follows:

Source: Source code in C

Exe: Executable MS-DOS programs. Reboot Windows 95+ in MS-DOS mode for best results.

Images: PCX images. Most images are indexed color (256-color), but those with filenames ending in the letter "m" are 24-bit color.


Digital Imaging in C and the World Wide Web

 [Spisak knjiga]

Digital Imaging in C and the World Wide Web
Author: W. David Schwaderer

Preface

SECTION I
Chapter 1—Visible Light
What This Chapter is About
Some Principles of Light
Electromagnetic Energy
Refraction
Some Color Science Principles
Spectral Light
Additive Color
Standardization of Colors
Subtractive Color
Metamers
Chapter 2—Human Eye Overview
What This Chapter is About
Visible Light
Dynamic Range
Anatomy of the Eye
Human Perception
Definition of Important Terms
Spectral Sensitivity
Light Intensity and Adaptation
Acuity
Visual Input and Image Processing
Illumination Source Compensation
Critical Fusion Frequency
Chapter Summary
Chapter 3—Some Human Vision Detail
What This Chapter is About
Eye Primer
The Cornea
The Pupil
The Lens
The Retina
The Fovea
Chapter 4—Human Perception Overview
What This Chapter is About
Light Intensity Effects
Visual Input
Visual Processing
Perception Summary
Thomas Young’s 1801 Color Experiments
Color Perception Summary
White Point
Flicker
Conclusion
SECTION II
Chapter 5—CCD Image Capture Devices
What This Chapter is About
Imaging Fundamentals
CCD Imaging Basics
CCD Terminology
Conclusion
Chapter 6—Video Display Boards and Controllers
What This Chapter is About
Video Board Architecture
System Bus Connector
Frame Buffer Memory
Digital to Analog Converter (DAC)
Video Monitor Connector
Chapter 7—Monitors
What This Chapter is About
Adjusting Monitors for Proper Viewing
Cathode Ray Tube (CRT)
Displaying Information
Refresh Rate
Phosphors
Aspect Ratio
Color Monitors
Color Image Challenges
Chapter 8—Printers
Printer Technology Overview
General
Dot Matrix Printers
Non-Impact Printers
Electrostatic
Ink Jet
Thermal Transfer
Dye Sublimation
Printing Considerations
Printer Programming
Overview
Sending Characters to Printers
IBM Proprinter II Dot Matrix
Hewlett-Packard Printers
Chapter 9—Device Color Consideration Overview
What This Chapter is About
James Clerk Maxwell’s Color Reproduction Experiments
Additive Light Implications
Subtractive Light Implications
Conclusion
SECTION 10
Chapter 10—Image File Format Overview
What This Chapter is About
Image File Headers
The PCX File Format
PCX Header
PCX Image Compression and Decompression
Image File Size and Transmission Bandwidth
Image Files and Their Image Compression Algorithms
JPEG
GIF
Browser Software
The 6x6x6 Browser Palette
Chapter 11—Mathematics Primer
What This Chapter is About
Normalization
Gamma Correction
Quantization
Digitization
Probability
Wave Frequency
Running Computations
Running Calculation Overview
Smoothing
Sharpening
Unsharp Masking
Summary
Chapter 12—Monochromatic Concepts
What This Chapter is About
Achromatic Image Representation
Single Rectangular Integer Array
Bits Per Pixel
Histograms
Point Operations
Histogram Slide
Histogram Stretch Contrast Adjustment
Histogram Equalization Contrast Adjustment
Area Operations
Spatial Filters
Sharpening
Smoothing
Unsharp Masking
Emphasizing Edges and Line Segments
Shift and Difference Edge Enhancement
Line Segment Detection
Edge Detection Using Gradient Filters
Laplacian Filters
Noise Removal
Variance Calculations
Averaging
Median Filter
Image Processing
Chapter 13—Color Models
What This Chapter is About
RGB Model
CMY Model
CMYK Model
Y'CbCr Model
HSV/HSB Model
HLS/HSL/HIS Model
HSI Model
CIE XYZ
RGB/HSV/HLS/HSI Color Model Conversion Routines
Chapter 14—Bilevel Digital Halftones
What This Chapter is About
Bilevel Devices
Screen Painting and Colorant Aerosols
Ordered and Random Dithers
Clustered Dot Ordered Dither
Printer Lines Per Inch
Error Diffusion Dither
Scan-line Centering
Serpentine Raster
Softening Compensation
Error Diffusion Variations
Error Diffusion Summary
Chapter 15—Color Image Quantization
What This Chapter is About
Uniform Quantization Introduction
The RGB24TO8.C Program
Paul Heckbert’s Median Cut Approach
Xiaolin Wu’s Optimal Color Quantization
SECTION IV
Chapter 16—Miscellaneous Programs
What This Chapter is About
TIFFEXAM.C Program
TIFF2BIN.C Program
ARRAY96.C Program
VESAEXAM.C Program
GAMMA.C Program
CLRSPC.C Program
VESATEST.C Program
Chapter 17—Image Display
What This Chapter is About
VIDEO.C Monochromatic Image Display Program
VESAPCX.C Color Image Display Program
RGB24TO8.C Color Quantization Program
WDS_QNT.C Program and WU_QNT.C Color Quantization Program
Chapter 18—PRINTPCX.C Image Printing Program
SECTION V
Chapter 19—Some Legal Considerations
What This Chapter is About
Vietnam Women’s Memorial
Andrew Leicester vs. Warner Brothers Studios
Rock and Roll Hall of Fame & Museum
Other Considerations
Copyright Primer
Fair Use Primer
Moral Rights
Conclusion
Chapter 20—Social Issues
What This Chapter is About
Level Set
Photographic Myths
Photographic Reality
Digital Image Processing—Reality’s Solvent
Photographic Impact Examples
Press Ethics
Video Futures
U.S. Military Position
Conclusion
Epilogue
Digital Image Processing Importance
Entertainment
Identification
Virtual Reality
Exploration
What If?
Art
Medicine
Counterfeiting
Conclusion
Appendix A—VESA VBE VGA Programming
Video Board Inquiries
Video Hardware Initialization and Programming
Frame Buffer Granule (Bank) Switching
Frame Buffer CPU Visibility Considerations
Window A and Window B
Hardware Window Type Summary
Super VGA Mode Number Overview
VESA Mode Number Format
VESA 256-Color Packed Pixel Mode Numbers
VESA Direct Color Mode Numbers
Extended VGA BIOS Calls
1. VBE Function Return Status Information
2. VBE Function 0x00—Return Super VGA Information
3. VBE Function 0x01—Return Super VGA Mode Information
4. VBE Function 0x02—Set Super VGA Video Mode
5. VBE Function 0x03—Return Current Video Mode
6. VBE Function 0x04—Save/Restore Super VGA Video State
7. VBE Function 0x05—CPU Video Memory Window Control
8. VBE Function 0x06—Set/Get Logical Scan-Line Length
9. VBE Function 0x07—Set/Get Display Start
10. VBE Function 0x08—Set/Get DAC Palette Control
Sample VBE Application
Appendix B—RGB/HSV/HLS/HSI Color Model Conversion Routines
Appendix C—MATH.C Sample Program
Appendix D—VESA.H and PCX.H Headers
VESA.H
PCX.H
Bibliography
Index


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

SECTION I

Chapter 1
Visible Light

What This Chapter is About

We begin our study of digital image processing with a gentle introduction to light and the human visual system. In order to create high impact images, understanding how light works and how humans perceive the effects of light is useful, if not interesting.

Many books exist that examine the physics of light and the biology of the eye in much greater depth. In this book, we condense this information. The goal is to provide you with a conceptual understanding of the factors that affect image processing techniques. As you will see in later chapters, the quality and impact of your digital images can be attributed to human perception of color. The way in which we see a particular color is a direct result of many phenomena, including the light source and effect of reflective surfaces on the light.

When you have finished this chapter, you will have a primer on the principles of light and color science, the structure of the human eye, and the techniques that the eyes and brain use to process visual information later chapters expand. You may skip those detailed chapters if you desire.

Some Principles of Light

An introduction to the principles of light is the best starting point because, as noted earlier, color perception is a direct result of many phenomena. If you have ever been in a new car showroom, you have experienced the effect of light. A car's color often fails to look as dazzling once you move your new auto away from the lighting of the showroom. Why is that? This phenomenon is examined later in the chapter, but let's start with an understanding of what light actually is and how it is measured.

Electromagnetic Energy

Visible light is a form of electromagnetic energy (EM energy). Other EM energy examples include radio waves, radar, microwaves, infrared and ultraviolet rays, X-rays, gamma rays, and cosmic rays. These various energy forms are actually arbitrarily delineated ranges within a continuous electromagnetic spectrum. Visible light occupies a very narrow band between the infrared and ultraviolet ranges.

While not wanting to oversimplify the physical principles of light, visible light behaves simultaneously like waves and particles (photons). In this book, we are most interested in light's wave nature because it is the primary consideration for image processing and human perception.

Each EM wave has a constant frequency meaning that the wave peaks and dips at constant time intervals. In the vacuum of space, all EM energy travels in a straight line at 299,792,458 meters (approximately 3x1010 centimeters, or 186,000 miles) per second. This velocity is called the speed of light. Dividing 3x108 meters per second by a wave frequency calculates the wave's wavelength—the distance required between each peak-to-peak wave position. Wavelength is traditionally expressed in nanometers (abbreviated as [mu] or nm), where one nm equals 10–9 (one-billionth) meters.

For example, any radio wave has a low frequency and a long wavelength. In contrast, any cosmic ray has a high frequency and very short wavelength. These two energy forces are at opposite sides of the electromagnetic spectrum. In the spectrum, photons with a higher frequency (and a shorter wavelength) contain greater energy than waves with a lower frequency (and a longer wavelength).

Visible light frequency ranges from about 380 nm to 760 nm—a span of about 380 nm. Table 1-1 illustrates the various wavelength ranges we associate with various colors.

violet | 380-450
blue | 450-480
blue-green | 480-510
green | 510-550
yellow-green | 550-570
yellow | 570-590
orange | 590-630
red | 630-760

Table 1-1: Visible colors and their associated wavelength ranges

Refraction

In 1621, W. Snell developed the Principle of Refraction. Snell found that when light travels through anything except a vacuum, it exhibits a change in direction when it reaches a material through which it must travel and then resumes its straight line course. This process, where the light direction actually bends as the light enters the material, is called refraction. The level of refraction depends on the density of the material and the frequency of the light.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Some Color Science Principles

Our perception of color results from many phenomena, including the nature of a light source and the effects of reflective surfaces on the light. An important distinction should be made before diving into color theory. Color reproduction can be accomplished by either additive or subtractive means.

Additive color, as we will see in the next section, is the addition of different colored lights to produce a variety of colors. Television uses additive color. Subtractive color, on the other hand, uses colorants (dyes or inks) that absorb some elements (frequencies) of light to reflect a derived color. Art paintings use subtractive color.

Spectral Light

In 1666, Sir Isaac Newton discovered that he could separate sunlight into a color spectrum by using a prism. He also found that he could recombine the color spectrum back into sunlight using a second inverted prism. Colors created from separating sunlight are called spectral colors.

Remember that a wave with a higher frequency and a shorter wavelength contains more energy than a wave with low frequency and long wavelengths. It is also true that a wave with a higher frequency bends more when light passes through a prism. As an example, the red light bends the least because it has a lower frequency.

In contrast, blue light has a high frequency and therefore bends the most. So, if you were looking at a rainbow, the red would bend the least and appears at the top. Blue light bends the most and appears at the bottom of rainbows.

Now you begin to understand the relationship between light and color. So far in this chapter, we have talked about how light is measured and have examined a color separation process. The next step in understanding the relationship between light and color is to classify how colors are created when different concentrations of light frequencies are used.

Because it is difficult, and usually unnecessary, to know the exact wavelength of a photon, the visible spectrum is divided into mutually exclusive and collectively exhaustive 10 nm bands. A source of light that generates all visible light frequencies in equal intensities within each band is called equi-energy white light and appears "white to the eye." However, there is no unique, physical, or perceptual definition of "white light," and many arbitrary light combinations are used as standards for different purposes.

Additive Color

Recall that an inverted prism recombines separated colors into an original light source. Here, light sources blend together to cause a different perceptual stimulation than the unblended light. Color computer display monitors approximate this process by placing different intensities of various primary colors (red, green, and blue) in close proximity, which our visual system subsequently blends to interpret as various colors.

Suppose now that you combined a red and blue light—the same as removing all green light from a separated equi-energy white light. Here, the resulting color sensation would be magenta—an example of non-spectral color. Nonspectral colors do not exist in the rainbow and are an artifact of the properties of the human visual system. The eye detects certain light frequencies and instructs the brain to construct a "close" color interpretation.

In the same way that removing green from an equi-energy light separation creates the illusion of magenta, the reverse is also true. Correct amounts of green (the missing color) and magenta (the perceptually invented color) combine to make white light. Colors that cause a white light sensation when mixed are called complementary colors.

If we removed all of the spectral colors except for a narrow part of the green area, then the green is called a fully saturated color. This means that the color is in its purest state, unsullied by other rainbow colors. If you begin to add in what appears to be magenta light, then the green light gradually becomes an unsaturated color or what is known as a pastel hue. Remember that magenta, in this case, is a complementary color to the green and that adding enough magenta eventually produces white. However, combining two fully saturated colors—in this case green and magenta—eventually produces a white light that may not be equi-energy white light. Passing this light through a prism may reveal that other spectral colors are missing.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Standardization of Colors

Color theory is a very complex problem that color scientists have wrestled with for years. How can you accurately generate and replicate specific colors present in sunlight? How can you possibly be sure that you would get the same results every time?

Like many industries, the color science community turned to the standardization process. To study predictable and reproducible color blending, the International Commission on Illumination, or Commission Internationale de l'Eclairge (CIE), noted the effects of observing blends of three primary colors at specific frequencies: red (700 nm), green (546.1 nm), and blue (435.8 nm). The CIE selected these particular colors and wavelengths because, in various combinations and intensities, they can form many, but not all, colors. Combining equal amounts of these colors in pairs produces three secondary colors: cyan (green + blue), magenta (red + blue), and yellow (red + green).

Combining the designated primary or secondary colors in the correct proportions produces a visual sensation indistinguishable from equi-energy white light. However passing this prism reveals that the light does not contain all spectral colors, only red, blue, and green.

Subtractive Color

Where additive color creates color by adding light of different wavelengths, subtractive color absorbs and reflects back light to create color.

Colorants (paint or ink, for example), when applied to a surface (the media) and viewed using a particular light source, impart hues by absorbing, or subtracting, some light frequencies before reflecting others from the surface. This process of absorbing some frequencies depending on the light conditions is called subtractive light.

Subtractive primary colors are different from additive primary colors. The subtractive light primary colors are cyan, magenta, and yellow. In subtractive light, the secondary colors are red (magenta + yellow), green (cyan + yellow), and blue (cyan + magenta). Notice that the primaries and secondaries for subtractive color are the opposite of those for additive color.

It is for this and other complex reasons that images displayed on computer screens using additive color may not look the same when printed using subtractive color without considerable forethought and intervention.

This returns us to the car showroom example mentioned at the chapter beginning. The paint on that car will never look exactly the same unless you look at it under the exact same light conditions, because different lights have different spectral compositions.

Metamers

Colored objects that match under one light source (sunlight, for example) but differ under another (perhaps a street light) are called metamers. The disappointing results of a car purchase or paint touch-up may be a metameric consequence, and there are many more examples. Makeup mirror manufacturers capitalize on this phenomenon by providing selectable mirror lights labeled "fluorescent," "inside," and "outside." This enables the person applying makeup to select the optimum makeup with respect to the anticipated lighting.

Before moving on to the human eye and how our visual system works, here are the salient things to remember about light. First, the way in which we perceive a color is a direct result of the effects of light on its surface. Things may not always be what they seem and they may not always look identical under different light conditions. Second, additive color is the combination of spectral lights that are added together to create hundreds of colors, pastels, and hues. With subtractive color, the ink or paint absorbs some light frequencies and reflects others back.

Having considered the complications of color sources, you must now consider the human factors involved. The interaction between the eye and the brain is complex; what the eye sees, the brain may not. The next step in building this framework for the problems that you may encounter in digital image processing is to take a closer look at how the eye works. I intend to be brief here because you bought this book to learn about image processing, not human biology!


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 2
Human Eye Overview

What This Chapter is About

Simplistically, the eye does nothing more than gather visual information. However, to create images that catch the eye, understanding how the eye gathers data and the limitations of the eye can be important. To become a skillful color image specialist, you need to know how to cater to human vision complexities.

Visible Light

The previous chapter discussed color wavelengths and colors generated by additive and subtractive light. Red, for example, has a wavelength ranging from 630 to 760 nm. The human eye can sense EM energy within the 380 nm to 760 nm range, so it detects "red." This frequency range is known as visible light. Because infrared waves have a wavelength of approximately 10–3 to 10–5 nm, they fall outside the range of visible light. As a result they cannot be sensed by human visual systems and require thermal imaging techniques that convert infrared energy into visible energy.

The human eye admits measured amounts of light through a small circular opening called the pupil. The pupil expands and contracts depending on the light source intensity. The light then passes through a clear lens, which, under eye muscle control, changes shape to correctly focus the light. Finally, the light strikes the retina, which converts optical power to electrical impulses that the optic nerve transmits to the brain for interpretation.

Dynamic Range

Under the right conditions, unaided eyes can see a candle burning 17 miles away. Conversely, the eye can also discern objects a million times more intense. The ability of the eye to adapt and collect information on objects with such varying levels of intensity is simply amazing.

The total range of light levels over which the eye can work is called its dynamic range. At any given instant, retinal chemical and neural mechanisms, as well as pupil size, adapt the eye to work efficiently over some small portion of the total dynamic range. Within this working range, the eye allows us to differentiate only about 100 to 200 different light intensities. The eye's limited working range means we don't have to have a huge intensity range at any instant (image "tones") to exploit.

Anatomy of the Eye

It is helpful to understand what role each part of the eye plays in visual image processing. Refer to Figure 2-1 for the following discussion.

Figure 2-1 The human eye

The cornea is a dense organ that causes light to refract within the eye. One would think that this took place in the lens, but in fact, as we have said, the lens merely focuses the light at the right spot on the retina.

As we learned earlier, light refracts more for light sources with a higher frequency. When a lens refracts (bends) light to focus it, the higher frequencies (blue colors) bend more and focus in a different position than the lower frequencies (red colors). This chromatic aberration prevents a perfect focus in many instances.

To reduce chromatic aberration, a good quality camera lens contains multiple lenses, each with a different index of refraction. To provide a similar color correction, the human eye uses a lens that varies not only in shape but also in refractive index. Finally, the eye and the brain do not use color information for differentiating fine detail and this might reduce the residual effects of chromatic aberration.

The retina is a layered membrane containing neural processing cells and 130 million interspersed cells called rods and cones. The approximately 120 million rods provide humans with their night, or scotopic, vision. Rods provide achromatic perception at very low intensity light levels —which explains why moonlit scenes appear colorless. At high intensity light levels, rods excel at detecting faint contrast changes and detecting movements at the periphery (edge) of your field of view. Since image processing occurs in relatively high light levels, rods are unimportant in digital image processing.

The eight to ten million cones are spread out over the retina, with another 300,000 concentrated in one spot called the fovea. The fovea is a small, circular area that responds to our direct line of sight and is responsible for daylight, or photopic, vision. The fovea is responsible for seeing fine details and for much of our depth perception.

Most people have three types of cones. One type of cone (the L-cone) is most sensitive to long light wavelengths (yellow-red), a second type (the M-cone) to medium wavelengths (yellow-green), and the third type (the S-cone) is most sensitive to short wavelengths (blues). Together, these three cone types provide the brain color information. This explains in part why the CIE selected three particular primary colors and why most colors can be represented as a mixture of three different wavelengths at various intensities.

From the relative density of cones (color sensors) and rods, you can see that color vision is best in the fovea. The fovea also provides the best detail vision for gray scale (achromatic) intensities, because the cones also compute light intensity. As we go from the fovea towards the field of view periphery, the cone density rapidly decreases and so does visual detail. For this reason, the fovea is perceived as the "center" of our field of view and the brain directs the eye to move so visually important objects project onto the fovea.

In scotopic (night) vision, the cones are not active. This means that the best visual detail at night is achieved by looking slightly away from an object so that the object image falls not on the inactive cones but on nearby rods. In photopic (daytime) vision, the rods are still active but most of our color and gray scale visual information comes from the cones.

Having introduced the role of the eye in collecting visual information, we now move on to the most complex area of this discussion—human perception. It is the brain that interprets all the data and then reconstructs the images we see.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Human Perception

We have covered the principles of light, color creation using additive and subtractive methods, and how the eye captures images. This next section addresses the important topic of how images are constructed and what factors affect color perception. We start with an explanation of important terms, discuss cones and color vision in more detail, and then move on to other elements of human perception that are important for image processing. Finally we review Thomas Young's revolutionary color experiments. A complete treatment of perception is not possible since the science is not completely understood. Therefore, this section is a superficial overview.

Definition of Important Terms

Intensity, or more precisely radiance, is the total EM power that an energy source transmits onto a surface area, measured in watt units. Camera light meters that measure intensity to calculate proper exposures for a given film is a practical light intensity measurement example. Radiometers are a more general measurement instrument.

When the human visual system processes arriving EM energy, it expresses perceptual light intensity differences as luminance differences. Luminance is an objective metric calculated using the light frequency distribution of a source light, called an illuminant, and the human eye's sensitivity to various wavelengths, called the spectral sensitivity function. Luminance is expressed in lumen units, and humans can determine side-by-side luminance differences with an accuracy of about 1 percent. The luminance difference between areas of highest and lowest luminance is called the contrast.

Intensity and luminance have dramatically different meanings as far as the human visual system is concerned. By definition, one lumen of blue seems to have the same strength as one lumen of yellow-green light. However, 1 watt of 555 nm yellow-green radiant power provides 680 lumens and 1 watt of blue light is approximately 70 lumens. This means that it provides only about 3 percent of the sensation strength that a yellow-green light produces. This color sensation strength difference is due to the long and medium wavelength (red and green) cones collectively being much more sensitive than the short wavelength (blue) cones.

Brightness is a subjective visual response to EM energy and depends on the current adaptation state of the eye. For example, a lighted match has a constant radiation level. While it may appear "bright" in dark rooms, it is nearly invisible in "bright" sunlight. Hence, the term brightness is considered subjective and possesses little technical value.

Spectral Sensitivity

In 1801, Thomas Young began experimenting with color, and based on his observations the Young-Helmholtz Theory became a landmark discovery in the field of color science. The Young-Helmholtz Theory determined that the human retina contains three types of cones. Each cone is sensitive to a wavelength range known as its spectral response function. Although each cone has a fairly wide spectral response function, Young discovered that each of the three cone types is particularly sensitive to light at certain wavelengths. This was a significant departure from conventional thinking because it suggested that a few receptor types could distinguish many colors while contemporary conventional wisdom suggested different sensors sensed each color.

One type of cone, the S-cone, is most sensitive to light at 440 nm (blue). Another, the M-cone, favors light at 545 nm (yellow-green), and a third, the L-cone, is most sensitive to light at 580 nm (yellow-red). These spectral response functions are shown in Figure 2-2. Notice that the blue spectral response function is isolated from the red and green which themselves significantly overlap. This significant overlap causes color reproduction problems because it is impossible to stimulate either of the L-cones or M-cones without stimulating the other—causing ambiguous sensing. Collectively, the three cone types experience a peak sensitivity at around 550 nm (green). A plot of the collective sensitivity is called the CIE luminous efficiency function.

Figure 2-2 Spectral response functions

Individuals (mostly males) who are color-blind have fewer than three cone types (they are not tritanopes). A color-blind person lacks the spectral information to distinguish between certain pairs of colors. Restated, color-blind people have more metamers (apparent color matches) than normal people. Because even normal people cannot distinguish between metameric color mixtures, you could say that everyone exhibits some color-blindness. In fact, S-cones appear to be absent in the fovea center. So, tiny blue objects or text characters on monitors are not perceived correctly or only with difficulty.

Looking at Figure 2-2 we can see why the luminance of a blue light is lower than the luminance of an equal intensity yellow-green light—the spectral response functions show that the green and red cones are collectively more sensitive than the blue (short wavelength) cones. Television and image compression schemes exploit this relative insensitivity by reducing the blue spectrum fidelity.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Light Intensity and Adaptation

As light intensity levels change, our eyes become light adapted. As we stated earlier, during daylight hours humans rely on photopic vision to provide vivid color and depth perception. This task is the primary responsibility of the cones. When the sun begins to set, our eyes adapt to the lower light levels and gradually scotopic vision and the rods take over. With scotopic vision, our perception of color and fine intensity detail decreases. This makes sense because our perception of color is a function of the light acting upon the cones that become inactive at low light levels.

Acuity

Another responsibility of cones is acuity. Acuity is the ability of the eye to distinguish detail using the contrast and luminance of the object. Under the right conditions, the human eye can distinguish the right and left turn signals of a car several miles away. The Hubble Telescope can differentiate them from 2,000 miles away. Thus, the Hubble Telescope has a much greater acuity than the human eye.

Acuity, as we said, is provided by the cones. Because the cones are responsible for distinguishing fine details, good acuity requires a relatively intense light level. In scotopic conditions (night) visual acuity decreases. This is because the cones are not active and because the same nerve connects to the outputs of several rods, trading acuity (spatial resolution) for better light detection.

Visual Input and Image Processing

Fully 90 percent of the light that reaches the pupil is lost during transit within the eye and thus, only 10 percent eventually reaches the rods and cones in the retina. This 10 percent arrives as a faint, upside-down image. Depending on the light levels, the eye may continue to recalibrate and collect information before it processes the image and sends it on to the brain for interpretation. The retina must validate all the information it has gathered and also eliminate possible random signal errors that increasingly arise with advancing age.

The retina then amplifies validated signals and decomposes them into luminance and chrominance data. This is the equivalent of encoding and digitizing information in digital image processing. The retina also sharpens the image to make edges more apparent. Sharpening an image to enhance the edges of objects is another example of how digital image processing techniques mirror the human visual system.

All this information is then transmitted along the optic nerve to the six billion cells in the brain responsible for visual processing.

Processing the image once it reaches the brain is complex for a number of reasons. The brain receives only fragmentary information that may or may not accurately reflect the true situation. It measures differences in color and luminosity, sifts through alternatives searching for the closest representation that it can come up with, and finally presents the "best fit" based on its stored knowledge. Since visual input may lack important data, numerous best-fit possibilities may be equally plausible. Humans are therefore vulnerable to illusions based on incomplete accumulated knowledge.

Illumination Source Compensation

The human visual system is not particularly discriminating about what it considers to be white light. For example, if you look at a burning match in the dark, the flame appears white. However, if you look at the match at an afternoon barbecue, the general lighting conditions are dominated by afternoon sunlight and the flame appears yellow.

This phenomenon is called illumination source compensation. Illumination source compensation shifts the white reference points—what our minds know to be white based on accumulated knowledge—depending on the environment in which it is viewed. The human visual system automatically tends to strive for consistency and thereby produces different color balance sensations in different settings.

Critical Fusion Frequency

The brain attempts to maintain a consistent presence sense for objects. This means that the brain will allow for the perception of constant environment when there is actually a rapid movement taking place in the light source. For example, florescent lights flicker on and off 60 times per second, but our brain conveniently presents us with the illusion of continuous lighting. The critical fusion frequency is the threshold at which the flicker becomes detectable to the individual. Once the flickering rate falls below this threshold, people can suffer various reactions from headaches to convulsions. Driving along a road lined with bare trees on a bright, sunny day can cause this reaction in drivers. In December 1997, a Japanese television network yanked a cartoon off the air after one episode reportedly caused "convulsions, spasms, or nausea in hundreds of children" with its flickering images.

Movies exploit this characteristic by playing frames at 72 frames per second, well above critical fusion frequencies. However, in order to reach the 72 frames per second, the camera must use a three-bladed shutter to flash each frame three times before moving on to the next frame. Without the shutter, images would be presented at 24 frames per second, well below the critical fusion frequency of many individuals.

Chapter Summary

This chapter covered the principles of light, how light interacts with colorants to produce subtractive colors, how colored light interacts with other colored lights to create additive colors, and how the human visual system gathers and processes all the information to create an image. So what does all this have to do with digital image processing?

The human visual system gathers and processes information in much the same way as a digital camera captures a scene and image processing hardware digitizes and either stores or reproduces that image in another form. The two systems mirror each other in many ways but also have major differences. Being aware of the complex processes that take place within both systems enables you to produce supercharged digital images.

If you feel you have enough information on the eye and perception to suit your needs, you might want to skip the next two chapters and proceed to Chapter 5 titled "CCD Image Capture Devices." Otherwise, continue with the next chapter to discover why things are not nearly as simple as they seem when it comes to sight and vision.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 3
Some Human Vision Detail

What This Chapter is About

The human eye-brain visual system senses electromagnetic energy from 380 nm to 760 nm and attributes subjective color sensations depending on energy wavelength mix.

Testing shows vision, like taste, varies by individual. For example, in World War II, some Japanese soldiers reportedly saw the infrared lights on early infrared rifle designs American troops used at night and considered invisible. This was an immense personal disappointment for the GIs debugging the new weapons, which endured until the light frequency changed to a longer, truly invisible wavelength.

This chapter discloses considerations skillful color image specialists use daily as they address the complications human visual systems cause for color computer screen, photograph, and printer output object representations.

Eye Primer

Figure 3-1 illustrates the eye as a 10 mm radius sphere. A transparent cornea covers the front, admitting light through a small circular opening, the pupil. Through expansion and contraction, the iris determines the pupil's radius, which can range from 1 mm to 4 mm. Light entering the pupil passes through the nearly transparent crystalline lens which, under pressure from the ciliary muscles, changes shape to focus light correctly. After traversing the transparent viscous vitreous humor, focused light strikes the retina which sends signals to the brain via the optic nerve for interpretation. Simple, isn't it? Hardly....

Figure 3-1 Simplified diagram of the human eye

The Cornea

The cornea bathes in salt-laden tears that imitate primordial oceans and is easily the body's fastest healing organ. Deep scratches can heal and disappear within days as a result of the cornea's incredible cell sloughing rate. The cornea's density causes light's greatest refraction within the eye—not the lens.

The Pupil

The iris contracts and enlarges to modify the pupil radius from its smallest size to as much as four times larger. One obvious effect of this change in pupil size is to change the amount of light entering the eye. This in turn helps adjust the working range of the eye to the lighting conditions. In fact, most of the adaptation of the eye to light is done by retinal chemical and neural mechanisms.

As the pupil size decreases, only light passing through the optically best part of the lens is allowed to go on to the retina. This reduces lens aberrations and produces a sharper image and one with greater depth of field (the distance over which objects are in sharp focus). The effect is just like "stopping down" a lens on a camera—you get a darker image but one that has more detail over a greater depth of field.

When staring at a seemingly "bright" light in a dark room, adding a second, dimmer light located away from the first causes pupil expansion (Fechner's Paradox). Apparently, the eye adjusts to average light levels, not absolute amounts.

The Lens

The crystalline lens is a flexible, fine-focus, light-bending mechanism. The ciliary muscle surrounds the lens and controls its shape (curvature radius). For looking at close objects, the lens is "fatter" in the center (smaller curvature radius). For farther objects, it is "thinner" (larger curvature radius). This object-distance adjustment is part of the eye's accommodation process which also includes eye convergence.

From the previous chapter, we know different light wavelengths refract different amounts. So for lenses, each color has its own focus point. For high-quality cameras, this is a significant problem. Numerous different density glass elements color correct each camera lens, allowing any wavelength color to land where it should on the film.

However, the human lens has variable density to compensate for the chromatic aberrations caused by different wavelengths of light. The eye also has a yellow filter over the fovea (the most sensitive part of the retina) that blocks some blue light. This might help reduce chromatic aberration by reducing the range of wavelengths that reach the retina.

So-called "blue-block" sunglasses also filter out shorter wavelength light (blue). Blue light wavelengths bend more, converging closer to the lens, hence farther away from the cornea, than other frequencies. Eliminating blue light wavelengths allows vision systems to contend with fewer wavelengths more gracefully, though some folks feel such filtering presents a "nuclear winter" visual experience.

Finally, lenses have a slight yellow tint which increases with age as the lens thickens and loses flexibility. The increasing tint makes the visual system less sensitive to blue which may partly explain why older folks sometimes wear clothes with increasing blue tones. One San Francisco Bay dentist's visual system is so blue-insensitive, he has assistants verify customer false-tooth tints. Otherwise, he invariably selects the wrong gray tint (watch those metamers, folks).


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

The Retina

The retina is a complex, incompletely understood processing membrane. Its layered-cell construction contains 300 million highly intertwined cells, some of which comprise a brain surface extension tethered to same-tissue counterparts via the optic nerve. These light-sensitive estranged brain cells perform extensive image processing within the eye. Some image processing techniques discussed later can exploit their processing sensitivities.

Treviranus discovered retinal sensors in 1835 but did not notice they faced away from the lens, like film installed backwards in a camera. Because of this, other dispersion, and absorption elsewhere, retinal sensors receive a meager 10 percent of all light entering the eye.

Light strikes retinal sensors, causing a poorly understood receptor photo-pigment chemical bleaching. This triggers a receptor electrical property change and subsequent nerve stimulation within 0.001 of a second. Before reuse, the sensor must return to its initial state. The retina contains two light sensor types: rods and cones.

  • The retina's 75 to 150 million rods provide our principal night (scotopic) vision that only produces light level intensity differences. Rods provide achromatic gray scale (black and white TV) sensations. Because rods have only one spectral sensitivity function, moon-lit colored objects appear colorless.

    Rod vision is relatively primitive but excellent for detecting contrast changes such as peripheral vision shadow movements. Note that high-contrast stationary objects are invisible at the retina's edge. Because several rods connect to a given nerve end, rods provide general field-of-view perceptions but cannot provide detail information.

  • The retina's eight million cones provide daylight color (photopic) vision. Cones concentrate in the fovea which contains some 300,000 cones packed together at a 150,000 per mm2 density. The retina's edge contains few cones. Hence, peripheral vision lacks color and detail. Because cones connect to unique nerve ends, they provide detail information, called acuity, as well as color.

The Fovea

The fovea is a small, 0.75 mm radius, circular retinal area that senses our "direct line of sight." Though the fovea is the retina's thinnest part, it is the retina's "sweet spot" for color differentiation and detail because it contains the retina's highest cone concentration (300,000 tightly packed cones). Foveae are also important for precise depth perception. As you might expect, hawks have them but cats do not.

We typically focus conscious attention on fovea cone stimulation and process peripheral vision as a background task based on rod sensor input. When the rod "early warning" system signals alert, we pivot the vision system to bathe our foveae with pertinent quanta (glance at the danger) for assessment.

On the retina, both the rod and cone distributions are nonuniform, radially symmetric distributions around the fovea. The fovea contains the greatest cone concentration level which tapers precipitously as measured away from the fovea. The largest rod concentration surrounds the fovea and tapers off to the retina's edge to provide unconscious vision. Because rods are more sensitive to faint intensity levels than cones at night, it is helpful to look slightly away from stars, allowing their faint twinkles to fall on rod-rich areas, not on the cone-rich fovea. Conversely, if you look at intensely lit color objects, turning your head prevents discerning the object's color and detail because rods cannot match cones in these respects.

We now turn our attention to a human perception overview discussion that emphasizes important concepts for digital image processing.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 4
Human Perception Overview

What This Chapter is About

Human perception involves numerous disciplines spanning art, graphics design, optics, physics, philosophy, psychology, physiology, psychophysiology, and biochemistry, among others. The complexity of human perception and our poor understanding of it prevents giving a complete treatment here.

Light Intensity Effects

When light intensity levels change, as with setting suns, our eyes become light adapted. Cones adapt more rapidly, adapting fully in about seven minutes. Rods can take up to ten times longer. At a given adaptation level, the sensors can differentiate from 12 to 24 different intensities. The neural processing circuits in the retina rapidly adjust the sensitivity and output of the cones and rods based on local luminance and contrast. Thus, the resulting working range of the eye is increased to between 100 and 200 intensity levels.

As light decreases, high photopic (cone) vision gives way to scotopic (rod) vision. At the crossover point, it is fair to say we have two different intermingled retina systems since we actually have both sensor types operating simultaneously. At the lowest light levels, only the rods operate and full night vision occurs below 0.01 to 0.005 lumens. While single quanta stimulate individual rods, we cannot see them though we can see as few as five or eight under optimum conditions.

Acuity is the ability to distinguish object details with a given contrast and luminance. Because cones provide acuity, are concentrated in fovea, and require relatively intense light to function, it follows that acuity decreases with decreased light intensity. As illumination levels decrease, all colors fade as the Purkinje shift occurs. Our color vision becomes more sensitive to blue wavelengths that seem brighter and less sensitive to reds. The spectral-response function actually shifts as Figure 4-1 illustrates.

Figure 4-1 The Purkinje shift

Things that were once blue become brighter than things that were once red. But there is little or no color, because this is due to a shift to rods that have only one spectral response function and hence cannot "see" color. Finally, increased luminance increases acuity also but decreases absolute luminosity difference detection.

Decreasing light levels cause the eye to increase its light integration time before firing optic nerve signals. This delay, called the retinal action time, can increase delays as much as one-tenth of a second. While light's finite speed means we always sense historical data, this extra dark adaptation delay can be disastrous for night-time drivers. For example, if a car travels 60 miles per hour, that equals 88 feet per second. The one-tenth of a second delay equals 8.8 feet that may be dearly needed in a stopping emergency. The same applies to digital video game response times. In other words, effective video games likely provide high radiance images.

Adaptation is useful, effortlessly allowing us to visually discriminate under dramatically varying light conditions. For example, viewing photographs may involve surveying hundreds of input intensity levels. Since a given eye adaptation differentiates between 12 to 24 levels for the given level, rapid and unconscious processing allows eyes to shift to different adaptation levels and accommodate significantly more radiant energy levels, thereby increasing the eye's working range. But, on the other hand, adaptation conceals radiant energy level perceptions essential for reliable camera operation.

Finally, note that our ability to distinguish between radiance levels depends on the total light present. For example, one flashlight's intensity may seem noticeably stronger than another's at night, but the intensity is likely inconsequential during the day.

Visual Input

Earlier discussion indicated spectral light has frequency. Our visual system assigns sensations to light frequencies and their combinations. We assign a given sensation a name that allows individuals to converse though they may actually experience different (but consistently different) sensations.

Visual Processing

The retina consists of brain cells that tether via the optic nerve to six billion visual processing cell counterparts located in the brain. Rods and cones gather input and pass it to neurons in the retina which integrate the input over time and across numerous retinal cell interconnections. The retina must eliminate possible random signal errors. It does this by increasing integration time and confirming signals from neighboring receptors.

The neurons in the retina combine, amplify, integrate, and differentiate the signals from the rods and cones. Chrominance (color) information is separated from luminance (black and white) information, although the same cells and fibers may encode both kinds of information with different codes. Retinal neurons also process the luminance information by exaggerating luminosity differences using a process referred to as sharpening which makes object edges more apparent. It then transmits the processed data to the brain's recognition mechanism via the optic nerve, delivering significantly more luminance than chrominance information.

Perception Summary

Perception is the miraculous output from many slow, highly interconnected cells. It usually provides a predictable internal interpretation of an authentic external world we construct using information our brains previously accumulated and cataloged. While contemporary charge-coupled device sensors may rival retinal sensor densities, lack of accumulated knowledge will hinder their recognition capabilities for the foreseeable future.

Our eye-brain perception system is likely a probability engine that:

  • Receives feeble, fragmentary input
  • Eliminates some signals
  • Operates as a differential input device that measures detected differences in color and luminosities off the achromatic axis
  • Sifts through numerous alternatives searching for a best interpretation
  • Presents the most plausible best-fit contender based on stored knowledge


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

It converts retinal luminance and chroma signals into neural activity patterns it uses in parallel pattern-matching activities. This process may produce numerous hypotheses. Some may involve unwarranted possibilities manufactured by creating absent objects to complete nonexistent patterns.

Since visual input may lack sufficient data, numerous contenders may be equally plausible. Therefore, we are vulnerable to visual misperceptions (illusions) based on incomplete accumulated knowledge and perceptual creativity that suggests incorrect interpretations when the truth is improbable. Worse, in ambiguous situations, the process may never converge and the interpretation may unpredictably alternate between equally likely candidates. Research clearly indicates our upbringing affects our susceptibility to illusions.

Thomas Young's 1801 Color Experiments

In 1801, Thomas Young noted the well-known fact that mixing any two spectrum colors produced another color that was visually indistinguishable from another authentic spectrum color. This landmark observation is not intuitive and is unique to visual stimulation. As an example, musicians can typically listen to (sense auditory stimuli) musical chords containing numerous individual notes and decompose the chord into individual constituent components (notes). However, this is impossible to do with color blends.

Young also noted he could select three, but not two, widely spaced spectral colors and mix them in varying amounts to produce a wide variety of other colors including many spectral colors, white, and pastels. The three colors did not have to be unique, though for producing the greatest color variations it was imperative that any one of the three colors could not be produced by mixing the other two. He initially used red, yellow, and blue but later changed to red, green, and blue.

From these observations, Young deduced the first version of what is now referred to as the Young-Helmholtz theory. This theory hypothesizes the human eye contains three cone types which respond to red, green, and blue stimuli and all color sensations result from blending the three cone type signals.

This was a significant departure from conventional thinking because it suggested a few receptor types could distinguish many colors. Because each cone type has different sensitivity characteristics, the frequency change required from one spectral color to the next visually different spectral color varies across the visible spectrum as illustrated by Figure 4-2 on the following page. Humans can typically detect 128 fully saturated visual spectrum colors and can detect 50 percent more saturation levels (23 levels) at the visible spectrum ends than at the center (16 levels).

Figure 4-2 Hue discrimination curve

Color Perception Summary

Humans normally can generally distinguish hundreds of thousands, if not millions, of colors derived from various combinations of 500 different luminances (achromatic gray level intensities), 200 hues (individual colors), and 20 shades for a hue (intensities, sometimes called the chroma range). The eye is more sensitive to hue shifts at the extreme ends of the visible spectrum than the interior spectrum colors.

In other words, human detectability of color errors varies by hue, saturation, and intensity. While we might be proud of our visual perception that stems from three sensor types, consider that one humble type of desert salt brine organism has nearly a dozen different color receptor types. Birds, fish, reptiles, and various insects also possess excellent color vision though dogs and cats have none.

White Point

The human vision system is not particularly discriminating about what it considers white light. The reference white point selection process is somewhat elusive but involves general illumination levels. For example, car headlights may seem white at night but yellow during a summer afternoon. Hence, white reference points shift depending on circumstance. And, as it does, human color vision effortlessly and easily adjusts to compensate—referred to as illumination source compensation. Thus, the human visual system's automatic tendency to strive for consistency produces different color balance sensations in different settings. Among other things, this makes it difficult to select house paint colors. While the paint looks great in the store, it may look hideous on your house.

Flicker

Flicker primarily complicates digital video because CRTs necessarily exhibit some flicker which has historically been linked to electrical power frequency rates. To further complicate matters, Britain and many European countries display 50 television images per second while people in the United States view 60 images per second. In both cases each image remains on its screen portion during the period the next image replaces a previous image on the other screen portion. So, viewers actually simultaneously see two or more interlaced images which interleave line by line. This can complicate digital image processing for television images captured with frame grabbers since the various image portions are time separated by as much as 0.04 second.

Conclusion

Human perception is an enormously complicated subject. The main objective of this chapter is to mention a few points that clearly illustrate perception complexities we regularly take for granted. But understanding how human perception works helps predict or explain the effect of some image processing approaches.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

SECTION II

Chapter 5
CCD Image Capture Devices

What This Chapter is About

Charge-coupled devices (CCDs) are solid-state semiconductor circuits that detect impacting photons ranging in wavelength from the near infrared to X-rays. They were conceived at AT&T Bell Labs in 1970. Depending on their design, CCDs can be extremely sensitive. Some commercially available CCDs reliably detect fewer than ten photons, allowing phenomenal images of extremely faint objects. Since CCDs detect nearly every arriving photon and produce linear output signals (directly proportional to the number of detected photons), CCDs are considered nearly perfect photon detectors.

As the heart of digital imaging devices such as desktop scanners, camcorders, and digital cameras, CCD applications range from astronomy to zoology, A to Z. CCD devices are rapidly becoming ubiquitous. The principles behind CCD design and operation are both interesting and instructive in understanding their strengths and limitations.

Monochrome capture devices require one CCD while color capture devices require color filtering. In one color capture approach, a capture device uses one CCD and requires multiple exposures, one through each filter. Another more expensive approach requires one exposure and uses three CCDs that are positioned behind one of three colored filters. An alternative approach requires one exposure and uses one CCD that is positioned behind a filter with different colored areas. This trades spatial resolution for color information.

In any event, color filtering introduces many previously discussed subtractive color problems. Finally, because color capture reduces to three individual monochromatic image captures, the remaining text only considers monochromatic image capture.

Imaging Fundamentals

To capture a digital image first requires partitioning a desired view into numerous, distinct sample areas. Each sample area is usually rectangular or square. Sample areas collectively encompass the view and must be small enough to capture desired detail. In general any significant detail transition must span at least two sample areas. It follows that highly detailed views require more sample areas for faithful capture operations.

After exposing the collective sample areas to a scene, each sample area result is encoded by assigning an average quantized value to represent the total sample area. Encoding values usually range from 0 (representing black) up to a maximum value (representing white) that is usually convenient for computer storage (1 for bilevel images or 3, 7, 15, 31, 63, 127, 255, 511, 1023, 2047, or 4095, etc. for monochromatic images). These collective values allow subsequent view reconstruction after they are saved in an image file.

CCD Imaging Basics

CCD construction is well suited to digital imaging. In a CCD imaging device, a lens focuses views onto a CCD which contains an M x N photosite (sampling) array. Individual photosites are often called CCD pixels or just pixels. An M x N photosite array may be collectively regarded as M linear registers, each containing N photosites. Channel stop regions isolate each linear register from adjacent linear registers and polysilicon gate structures isolate photosites within a linear register from adjacent photosites. This allows each photosite to sample independently of any other photosite. In Figure 5-1, the physically adjacent M linear registers are vertically aligned.

During image sampling, the photoelectric effect allows CCD photosites to detect arriving photons. An arriving photon strikes a photosite, creating one or more electron-hole pairs within the silicon structure. This creates an electrical charge that accumulates within the photosite with each impacting photon. The channel stop regions normally prevent accumulating charges from leaking into adjacent photosites. The cumulative charge indicates the number of incident photons over the integration time, the sample period.

Figure 5-1 Photosite (sampling) array

After the integration time expires, the M x N accumulated charges must be transferred from the CCD to the imaging circuitry. This requires an additional independent linear register, the serial register, placed next to the M x N photosite array as in Figure 5-1.

To use the serial register, the M linear arrays transfer the weak photosite charges closest to the serial array to corresponding elements in the serial array. The serial array then sequentially feeds each transferred charge into a charge detection output amplifier which produces an amplified voltage. Other logic uses this voltage to produce an appropriate digital pixel sample value for presentation to other components.

During the serial amplification process, the CCD linear arrays carefully ripple their charges to the next pixel position in preparation for the next serial register transfer operation. The resulting digital values represent the view focused on the CCD. Figure 5-2 illustrates the process.

Figure 5-2 Serial amplification process


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

CCD Terminology

As you read CCD literature, you will encounter new terminology. Here are a few important terms.

The quantum efficiency (QE) metric is the ratio of detected electrons divided by the number of electrons expected. The expected count depends on the number of photons and the number of electron-hole pairs the photon should create. Some incident photons may reflect off the CCD surface or may impact outside of any photosite. Others, depending on their frequency, create more than one electron-hole pair. CCDs usually have a QE ranging from 50 percent to 90 percent. In comparison, human eyes have less than 1 percent QE. Photographic film has a 5 percent to 20 percent QE.

The charge transfer process is the process of extracting photosite charges. It is sometimes called "clocking the gates" and can take several seconds to complete before another exposure is possible.

The charge transfer efficiency (CTE) measures the charge transfer efficiency which is the percent of charge transferred from one pixel to the next. Typically, the CTE is greater than 99.999 percent per photosite, where 100 percent is perfect. The CTE is important because on a 1024 x 1024 photosite CCD, a charge may transfer from two to 2,048 times and the loss is cumulative.

With a front-illuminated device, photons pass through gate structures and a protective layer to generate a charge. Unfortunately, some photons may be absorbed in these layers and not contribute to the signal.

With a back-illuminated device, incident photons strike a specially thinned CCD from the rear, bypassing the front surface electrodes and passivation layers. An optional enhancement layer forces photo-generated electrons toward the collection areas, and an optional anti-reflective coating may reduce reflections.

Any detector with high dynamic range can simultaneously detect low and high intensity objects. CCDs have a dynamic range sensitive to object intensities differing by a factor of 10,000. In contrast, photographic film has a dynamic range of about 100.

Dark current is an undesirable charge that accumulates in CCDs whenever they operate at temperatures above absolute zero (always). Dark current contributes sampling errors since it is indistinguishable from photon-generated charges. Dark current is similar to inadvertently exposing film to light. Sufficient CCD cooling (e.g., liquid nitrogen) effectively eliminates the problem.

CCDs have a uniform response characteristic. Because a CCD detector is permanent, it is straightforward to determine the behavior of each photosite, and differences can be compensated for using a "flat field" frame. This is not possible with photographic film which is always variable.

Saturation and blooming occur in all CCDs. If individual photosites comprise an "electron well," saturation occurs when the well becomes full. The maximum charge which a photosite can store is principally determined by its area. As a photosite approaches its maximum sustainable charge, it loses its ability to accommodate additional charge which then blooms (spreads) into adjacent pixels. This causes reporting or cascades the saturation problem. The blooming manifests itself in the image as a white streak or blob. Controlling saturation and blooming depends on the CCD architecture.

Binning combines adjacent pixel charges during the charge transfer process. This trades spatial resolution for improved signals and extends the CCD dynamic range. It also accelerates the charge transfer process. Parallel binning combines two or more rows before serial shifting. Serial binning combines two or more charges in the output amplifier before charge digitization. The CCD binning factor specifies the number of pixels combined.

A full frame CCD uses the parallel register array for exposure, charge integration, and charge transport. A shutter controls exposures and blocks light during the charge transfer process, preventing image smearing.

A frame transfer CCD divides the parallel register into two equal size areas. The image array is located farthest from the serial register and is where the images are focused. The remaining area is covered with an opaque mask and provides temporary storage for image area charges. After exposing the image array, the collected photosite charges rapidly shift to the storage array. This allows the image array to capture another image during the storage array charge transfer. Frame transfer CCDs can operate continuously at high rates without requiring a shutter.

Active Pixel Sensor (APS) is a relatively recent solid-state imaging technology that uses a more prevalent fabrication process. (Where CCDs use NMOS, APS uses CMOS.) This allows building sensors in existing semiconductor fabrication facilities. Advocates stress the following APS advantages:

  • A combined photo detector and dedicated readout amplifier at each photosite
  • A 100x reduction in sensor power requirements over CCDs
  • Popular fabrication processes that produce approximately 90 percent of all modern semiconductors
  • Ability to combine all necessary camera functions onto one chip
  • Single 5-volt (or 3.3-volt) power supply
  • Column and row addressability similar to common DRAM
  • Reduced column streaking due to blooming pixels
  • No smear as with CCDs during the charge transfer process
  • Potential integrated signal processing

However, as attractive as APS appears, CCD technology still has a promising future. CCD scientists are developing features to offset some APS advantages. For the near term, CCDs enjoy significantly higher resolutions because their photosites do not require the large readout amplifier that APS requires at each photosite.

Conclusion

CCDs promise ubiquitous digital imaging and are in perpetual evolution. Daily developments and product announcements clearly indicate we will be increasingly immersed in its output. For more information on CCD technology, see:

  • http://www.site-inc.com
  • http://www.photomet.com
  • http://www.pha.jhu.edu/~cat/seminar.html


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 6
Video Display Boards and Controllers

What This Chapter is About

This chapter discusses video display hardware and relevant portions of the Video Electronics Standards Association (VESA) BIOS Extension (VBE) version 1.2 video controller programming specification. Readers desiring a more complete discussion should obtain a VBE specification package by contacting:

Video Electronics Standards Association
2150 N. First St., Suite 440 | Phone: (408) 435-0333
San Jose, CA 95131-2020 | FAX: (408) 435-8225

For VBE 1.2 compliant display controllers, VBE specifies standardized PC video interrupt (int 10h) extensions software exploits for video capabilities exceeding the original VGA hardware standard that only allowed screen resolutions of up to 640x480 with 16 colors. These Super VGA modes allow extended:

  • Pixel colors
  • Screen resolutions
  • Frame buffer organizations

Video Board Architecture

Figure 6-1 is a high-level view depicting a typical video adapter hardware architecture containing:

  • System bus connector
  • Frame buffer memory
  • Display controller containing:
    • a palette that describes 256 independent colors
    • an optional color ramp
    • a digital to analog converter (DAC)
  • Video monitor connector

Figure 6-1 Typical video adapter hardware architecture

System Bus Connector

The system bus connector connects the video adapter to the system bus, allowing the system CPU to read and write frame buffer data. It also allows the CPU to manipulate display controller I/O registers to set different screen resolutions and frame buffer data interpretations.

Frame Buffer Memory

A CPU places data in frame buffer memory for display controller interpretation. The video display controller interprets frame buffer data depending on the system's current video mode and correspondingly generates the correct video monitor signals based on frame buffer data and the video mode.

Frame buffer sizes vary from 256 KBytes to 4 MBytes (and beyond) of ordinary lower-cost Dynamic Random Access Memory (DRAM) or higher performance Video Random Access Memory (VRAM) that allows CPUs and display controllers to access a memory location simultaneously. Because with MS-DOS, only a maximum of 128 KBytes of the frame buffer memory is visible to the CPU at any one time, the CPU must perform frame buffer bank switching activities to gain access to the full memory space, 128 KBytes at a time. The Appendix A topic titled Frame Buffer Granule (Bank) Switching discusses this further.

Digital to Analog Converter (DAC)

The display controller DAC interprets digital frame buffer data in various ways to generate analog video display signals. This book only discusses two interpretation types:

  • Packed pixel
  • Direct color

In packed pixel interpretations, each display pixel corresponds to one, and only one, specific frame buffer data byte. Frame buffer data byte values are index values into a color list referred to as a palette. The CPU uses standard VGA video interrupts to set desired packed pixel screen resolutions and load palette data into some or all of the display controller's 256 palette registers.

Each of the 256 independent palette registers holds three 1-byte values that completely describe one color that can appear on the screen in a packed pixel mode. It follows that packed pixel interpretation in combination with a specified screen resolution determines a specific video mode and that a CPU can load different values into the palette as required.

Because a byte can only hold one of 256 color values, a maximum of 256 colors from a possible 16,772,216 can simultaneously appear on display screens during packed pixel interpretation. For well-chosen colors, this is not a severe limitation and packed pixel images provide a natural compression advantage over direct color images.

Finally, software loads palette data into video controller hardware palettes using standard VGA int 10h requests. As the sample programs illustrate, pixel color information for each of the 256 colors often consists of 24 bits—8 bits describing a pixel's red contribution, 8 bits describing the blue contribution, and 8 bits describing the green contribution. Due to a cruel original VGA programming legacy, software must reduce each component of each pixel to six bits with two-bit shift right operations before loading the information. This adjusts the 768 color component values (three components for each of 256 values) to the standard VGA's, six-bit DAC width. Software can set some DAC widths to larger values, and VBE provides a way to detect this.

In direct color interpretations, each display pixel corresponds to two or more adjacent frame buffer data bytes. Typically, two, three, or four adjacent frame buffer bytes describe a given pixel. Some bits within the bytes may be reserved. At any instant, remaining bits are allocated in a fixed manner to the individual red, green, and blue color components. The bit allocation scheme and screen resolution uniquely determine direct color video modes which the CPU can set as needed.

Direct color interpretation allows video systems to display more simultaneous colors than packed pixel interpretation, but equivalent screen resolutions require more frame buffer memory (two, three, or four times as much) which may not be installed on a system's particular video adapter. In such cases, while the display controller could theoretically provide a requested mode, the installed memory amount may preclude it.

Finally, display controllers may provide optional color ramp hardware for direct color video modes. Color ramp hardware allows display controllers to adjust display outputs rapidly by manipulating color ramp hardware rather than adjusting individual pixel frame buffer values. After determining the initial red, green, and blue signal strengths for each pixel, the display controller uses the color component signal strengths as an index into a CPU loaded table to determine the final signal strengths. While the table is a global table for the entire screen, it allows independent fine-tuning of the individual color components before screen display. This book does not discuss color ramps further.

Video Monitor Connector

The video monitor connector provides an attachment point for the monitor cable that carries display signals to the video display.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 7
Monitors

What This Chapter is About

There are many types of monitors, but the most popular is the raster display monitor that displays information as illuminated points of varying intensities located on horizontal lines. In this chapter, the word monitor refers to raster display monitors unless otherwise indicated.

A monitor illuminates the essentially rectangular portion of a cathode ray tube (CRT) surface that is visible to viewers. The illuminated area may represent graphical or text data that human eye-brain systems translate to information using spatial integration and perceptual processes. Producing a desired illumination pattern requires exploiting numerous electrical, electromagnetic, chemical, and physical principles.

As illustrated in Figure 6-1, computer systems place monitor data in a special memory area called a frame buffer. The frame buffer can be in the computer's memory or on a special card, or the VGA card memory can be used as a frame buffer (see Chapter 6).

A display controller circuit interprets this data and generates various signals that travel to the monitor over the monitor cable. The monitor receives these signals from the display controller, interprets them, and generates coordinated signals for other monitor components that produce the desired illuminations. To understand the coordination that produces these illuminations requires understanding the individual monitor components.

Adjusting Monitors for Proper Viewing

Whether you have a gray scale or color monitor, it is important to adjust the monitor's brightness and contrast settings to obtain the best visual output from your display subsystem. The easiest method is to:

  1. Display a completely black image.
  2. Turn the contrast control to its minimum setting.
  3. Adjust the brightness control so the black image appears correctly.
  4. Adjust the picture control to the desired intensity level.

An incorrect brightness setting usually is the source of picture reproduction problems though contrast ratio is a major determinant of perceived picture quality.

Cathode Ray Tube (CRT)

A CRT is a bottle-shaped transparent glass tube with a narrow neck and large, relatively flat bottom called the face. In the simplest case, the inside surface of the CRT face is coated with a single special material called a phosphor. In the narrow-neck end, the CRT has a special assembly called the electron gun which emits electrons that bombard the phosphor surface, causing it to glow. To achieve this effect, the electron gun contains a heating element which heats a cathode. When completely assembled, the air pressure within a CRT is reduced to minus 14 pounds per square inch—a near vacuum. Heating the cathode in such a vacuum causes electrons to "boil" or "spray" off the cathode in a relatively chaotic cloud. These electrons appear at a relatively constant rate.

Near the CRT face, a component called the anode produces a strong (10,000 to 30,000+ volts) positive voltage charge which strongly attracts the negatively charged electrons. As the electrons begin accelerating from the cathode towards the anode in a relatively wide electron stream, electron gun components called control grids produce electrostatic fields that modulate the beam intensity (number of electrons allowed to pass by) channel. Under control, the various control grids narrow and focus the streaming electron spray into a focused electron beam. This determines the monitor's spot size. A monitor's spot size is the minimum size area the monitor can illuminate.

Looking at a cross section of a focused beam reveals a near Gaussian distribution of energy within the beam that leaves the beam center with more energy than the outside perimeter. Thus, an illuminated spot exhibits a blur which tends to make the spot size larger than it should be, causing blurry images.

Dynamic focusing varies the focusing depending on intended impact position. Because the beam strikes the sides of the viewing area at an oblique angle, the circular beam produces an elliptical impact area towards the CRT screen edges. To reduce the blurring effect caused by elliptical spots, dynamic focusing can reduce the spot size at the display area edges.

As the electron beam continues its journey, it passes inside a special circular electrical coil made of precision-wound thin wire, called the yoke. The yoke is located outside the CRT where the neck joins the rest of the CRT and consists of two independent parts. The yoke components generate vertical and horizontal magnetic fields, respectively, which divert the straight path of the focused beam to a desired impact point on the CRT face. Varying these magnetic fields varies the impact point with great precision but slightly defocuses the electron beam as it passes through the yoke's magnetic field. Avoiding unnecessary beam distortion requires careful yoke designs and careful alignment procedures.

As electrons approach the phosphor-coated CRT face, their velocity continually increases. When an electron eventually impacts a phosphor molecule, the phosphor molecule absorbs the tremendous energy and enters an excited state. Excited phosphor molecules eventually, but not simultaneously, release their energy in the form of light. As excited phosphors release this energy they are said to fluoresce. A fluorescing phosphor area provides a specific chromaticity (color and saturation) that depends on its chemical composition. The resulting light passes through the CRT glass face and is visible to the viewer. Some CRT faces have anti-glare coatings to reduce reflections and these treatments diffuse (slightly blur) the fluorescing light as well.

Bombarding phosphors with more electrons produces greater excitement and increases light intensity. The excitement level decreases, or decays, at a predicable rate once the electron stream is removed. This decay rate is called the phosphor's persistence—different phosphors have different decay rates as well as different chromaticities. The human eye-brain system's temporal integration characteristic and retinal afterimage make decay rates appear slower to viewers than they really are.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Displaying Information

CRTs depict text and graphical information based on signals produced by the display controller. Two such signals are the horizontal and vertical sync signals. The display controller generates the vertical sync signal at a constant rate ranging between 60 and 100+ times per second (Hz). A vertical deflection generator phase locks on this signal and produces a signal that allows a vertical deflection amplifier to produce a strong synchronous sweep voltage. The sweep voltage varies from zero volts to a peak voltage of between 50 and 200 volts, depending on the monitor design.

Shortly before the vertical sync signal arrives, the peak voltage abruptly discharges to zero volts which causes the yoke to deflect the electron beam to the top of the monitor (taking between 250 to 800 [mu]s). The voltage then remains at zero volts for a brief period before beginning to rise constantly.

As the yoke vertical deflection component receives the increasing voltage signal from the vertical deflection amplifier, it begins to deflect the electron beam uniformly towards the lower part of the CRT when viewing the front of the monitor. The voltage peak deflects the electron beam to the bottom of the viewing area.

Monitors process horizontal sync signals similar to the way they process vertical sync signals. Horizontal sync signals occur hundreds of times more frequently than vertical sync signals and result in a horizontal deflection generator producing a signal that allows a horizontal deflection amplifier to generate a synchronous sweep voltage independent of the vertical sweep voltage. The sweep voltage increases from zero volts to a peak voltage, causing the yoke to deflect the electron beam from left to right when viewing the monitor screen from the front. Shortly before the horizontal sync signal arrives, the peak voltage abruptly discharges to zero volts which causes the yoke to deflect the electron beam to the left side of the monitor (taking between 2 and 20 [mu]s) where it remains for a brief period before beginning another left-to-right pass.

Between vertical sync signals, these combined vertical and horizontal rhythmic deflections allow monitors to deflect the electron beam in a pattern of parallel lines that slope slightly down from left to right before the beam whips back to produce the next line. The monitor generates a third signal internally from the vertical and horizontal sync signals which it passes to the blanking amplifier. The blanking amplifier controls whether the electron gun produces any electrons.

A final display controller signal for our purposes is the video output signal. In monochromatic and gray scale monitors, this signal continuously varies (is modulated), assuming various values from zero volts to a peak voltage of as much as 1 volt. A typical PC VGA video output signal provides 64 different output levels while the original bilevel IBM Monochrome Display controller produces only two.

The video amplifier in the monitor receives this signal and boosts it to a signal that varies between 10 and 60 volts depending on the monitor model. The electron gun control grid receives this variable signal, causing it to repel more or fewer electrons, depending on the voltage level. To accurately render an image, the video amplifier must react very quickly to video output signal changes and accurately amplify low voltage signals. A video amplifier's slew rate indicates the time it takes to alter and stabilize its output signal.

As the deflection amplifiers operate, their effect is to drag the beam horizontally at a uniform speed from the monitor's left edge towards the right. At a precise distance into the sweep, it reaches the point where the current control grid begins allowing electrons to escape the electron gun. When the beam reaches the right edge of the viewing area, the blanking amplifier shuts off the beam in anticipation of the horizontal retrace operation. If a beam sweeps uniformly across all horizontal lines in an identical fashion, the horizontal deflection circuitry is exhibiting linearity.

When the blanking amplifier allows electrons to escape the electron gun assembly and the video output signal is above zero volts, the escaping electron beam excites phosphors. Meanwhile, the vertical deflection amplifier drags the beam from the top of the viewing area until it reaches the bottom of the viewing area. At that point the blanking amplifier shuts off the signal and begins moving the beam towards the screen top. Eventually the deflection circuitry positions the beam at the upper left corner of the monitor and begins another vertical pass.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Refresh Rate

When a monitor bombards all horizontal lines with electrons in a single pass, each vertical pass presents one frame and is called a screen refresh. European television screens refresh at 50 Hz while U.S. television screens refresh at 60 Hz. However, users have sometimes found these refresh rates too slow, particularly in high ambient intensity environments, and detect visual flicker at such speeds.

For monitors, there are typically between 60 and 100+ vertical passes per second, depending on the design of the monitor. More expensive monitors use higher performance components that permit higher refresh rates. For example, assume a monitor refreshes 1,000 lines, each having 1,000 points 50 times per second (50 Hz refresh rate). Thus, 1,000,000 points are refreshed every 20 ms. If each horizontal retrace requires 10 [mu]s, then ignoring time lost to vertical retrace as well as vertical/horizontal blanking time:

20 ms per refresh – 1000 * 10 [mu]s = 1/100th sec for 1,000,000 pixels

So, in this example, the pixel rate is 100 MHz which requires a video amplifier with a high performance slew rate. Higher refresh rates require far better video amplifiers which were not economical in early monitor and television development, causing a search for alternatives. In one approach, called non-interlaced, the monitor refreshes all horizontal lines on the screen. In a lower-cost alternative approach, the interlaced approach, the monitor traces only half the lines. In one screen refresh only the even-numbered lines are traced. In the next pass, only the odd-numbered lines are refreshed. The set of even-numbered lines and the set of odd-numbered lines are respectively called fields.

The display controller produces an interlaced display by decreasing the horizontal retrace frequency (making horizontal passes take twice as long) and producing alternating signals for the alternating fields. This requires DACs to access frame buffer memory differently than when producing non-interlaced displays and beginning the vertical retrace operation halfway through a blanked horizontal line at the bottom of the screen. Since every line is refreshed every other pass, long-persistence phosphors help reduce flicker but have different chromaticities and visual artifacts than the shorter persistence phosphors non-interlaced displays use.

Phosphors

Phosphors are chemical compounds that emit light when bombarded with electrons. They excite rapidly and decay more slowly. There are many types, and the various types differ in their chromaticities, granularity, and persistence. Phosphors with finer granularity can provide more detail than coarser phosphors. A phosphor's persistence is usually defined as the time it takes to decay to 10 percent of its initial fluorescence level.

The human visual system's eye persistence and phosphor decay combine to extend an illumination sensation long after an electron bombardment is removed. However, to avoid flicker, refresh rates and phosphor decay must not descend below a user's critical fusion frequency which varies by environment and individual. To combat flicker, some monitor manufacturers use long-persistence phosphors which can cause smearing and streaking when viewing moving images. Finally, phosphor performance decays with time, particularly with blue phosphors.

Aspect Ratio

Most popular monitor faces are one-third wider than they are tall. So, if a monitor is three units high, it is four units wide—referred to as a 4:3 aspect ratio. This becomes important when displaying images after setting a display controller to a certain mode which causes the monitor to generate a certain number of horizontal lines, each composed of a fixed number of dots.

Suppose a display controller is in a mode that generates 640 dots on each of 480 lines—called 640x480. In this instance, the pixel shape is considered square because a 400x400 dot area appears as a square area on the screen. However, if the display controller is in a mode that only displays 640 dots on each of 400 lines, called 640x400, the previously square area now appears as a rectangle—i.e., the previous square is "stretched downward."

This distortion occurs because monitor electronics fill the display area by varying the dot size. In this example, the monitor conceptually pulls each dot 480/400 or 1.2 times farther down the screen than it would in 640x480 mode. So the primary point is display controller modes distort images if programs do not compensate for the display controller mode's pixel shape.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Color Monitors

The previous discussion considered monochromatic and gray scale monitors—monitors with a single electron gun that generates varying pixel intensities by bombarding single monochromatic phosphor with varying numbers of electrons. Modern color monitors use three electron guns that are offset slightly from one another. These three guns bombard red, green, and blue phosphors, respectively, from behind a perforated metal shadow mask, positioned near the monitor face, that isolates the three beams, preventing the beams from bombarding incorrect phosphor areas. Shadow masks vary in hole size, hole shape, and hole location and are also called dot masks and aperture masks.

Shadow masks are inserted in monitors early in the manufacturing process and, together with high intensity lamps and appropriate optics that simulate electron guns, allow red, green, and blue phosphors to be applied to the monitor face using photographic techniques. In operation, a shadow mask blocks up to 70 percent to 80 percent of the electrons produced by the electron guns and produces beams with reduced size on the monitor face. Shadow masks that pass more electrons produce higher intensity displays, all other things being equal. The electron gun offsets allow the remaining electrons to pass through the shadow mask at slightly different angles and strike the appropriate phosphor areas. Absorbing the electrons can generate heat which can cause shadow mask distortion, producing image chromatic distortions when electron beams strike inappropriate phosphors. To combat this, many popular masks use invar, an alloy composed of 64 percent iron and 36 percent nickel.

Color Image Challenges

High quality color monitors are more difficult to produce than high quality monochromatic and gray scale monitors. Each of the three guns may have different operational characteristics and require individual adjustment such as gamma correction. Moreover, each electron gun has its own video amplifier with its own individual slew rate.

Since the shadow mask absorbs up to 70 percent to 80 percent of all electrons, the electron guns must produce more electrons, generating an aggregate electron beam which is more difficult to focus. Though a display controller may be in a specific mode, the monitor video amplifier slew rates may not be able to support the mode. In addition, the monitor phosphor dot density and the display controller mode may together produce low quality images since an electron beam typically passes through more than one aperture mask hole at a time. Finally, dozens of choices exist for each of the red, green, and blue phosphors. Their chromaticities, persistence, and fading characteristics vary significantly, making the job of authentic color and intensity reproduction a never-ending task.

In the final analysis, making an image attractive on one type of monitor provides no guarantee that it will appear identically on another monitor, particularly with the existence of monitor adjustment controls that are easily accessible to users. However, the various sample programs do attempt to provide some compensation for varying electron gun operational differences.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 8
Printers

Printer Technology Overview

General

Printers deposit colorant (ink or dye) onto media (paper or transparency) surfaces. They usually connect to desktop systems via a standard Centronics parallel port, serial port, SCSI host adapter, or network adapter. Many printers accept numerous data bytes and store them in internal buffers before producing output. The output a printer produces from given data depends on the mode the printer is in, as described below. It follows that printer programming involves both determining what to send to a printer and the process required to send the necessary data.

Printers use subtractive color while color monitors use additive color. So, without color correction intervention, what printers produce is rarely what monitors display. What action must be taken depends on the printer, the media, and its colorants. For color output, printers apply multiple colorants, each in their own color plane. The human eye's spatial integration and pattern recognition processes produce colored object sensations from the individual color planes.

Inexpensive printers produce bilevel output in each color plane while high cost printers provide multiple intensities in each color plane. Because each printed page has an associated cost depending on colorant and media costs, many printers provide various output qualities ranging from trial draft modes to final finished modes that use less or more colorant respectively. Almost all printers require colorant to impact the media (thermal printers that burn dark spots on special thermal paper such as used in many fax machines being an exception).

Impact printers use a mechanical brute force to hammer objects against a ribbon to transfer colorant to media. Typewriter hammers produce fully formed characters because they have fully formed embossed characters on their faces. Dot matrix printers use multiple wires to create dot patterns the human visual system recognizes as characters. Non-impact printers do not require the abrupt mechanical forces impact printers require. Since only ink or colorant strikes paper, non-impact printers are much quieter than their counterparts. They are usually much more reliable as well.

Dot Matrix Printers

Dot matrix printers strike colorant-soaked ribbons with small wires driven by solenoids to produce individual dots. Sweeping the print head back and forth horizontally and advancing the page allows the printer to produce dot patterns we recognize. An excellent example of a dot matrix printer is the IBM Proprinter II. The IBM Proprinter's print mechanism, the print head, uses a vertical column of 9 wires spaced 1/72 inch apart. By carefully controlling the individual wire movement, print head horizontal movement, and vertical spacing, the Proprinter produces various character spacing (5, 6, 8.55, 10, 12, 17.1 characters per inch) and line spacing (1/6, 1/8, 7/72, n/72, n/216 lines per inch). The Proprinter prints 240 characters per second (cps) in a draft mode (called data processing mode) as well as 40 cps in near-letter quality mode.

Like most dot matrix printers, the Proprinter prints characters in cells. With the Proprinter, character cells are 9 vertical pixels by 13 horizontal pixels. Using these cells, the Proprinter prints bold, double-wide, double-high, overscore, underscore, superscript, and subscript characters as well as several Greek/math symbols. Advanced applications could produce any pattern or typeface under program control because the Proprinter allows applications to control each wire and set the line spacing by entering graphics mode.

Because of the Proprinter's phenomenal success, many other manufacturers used the same control sequences for their products. So, learning how to program the Proprinter provides an easy entry to many other printer models. The PRINTPCX.C sample program later in this chapter illustrates using the Proprinter graphics mode to produce digital halftone images on the Proprinter.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Non-Impact Printers

Electrostatic

Electrostatic printers use electrostatic processes found in normal office copiers. First, the surface of a light-sensitive drum (or sometimes a belt) is given an electrical charge. Next, a light shines on portions of the surface, causing the drum to lose its charge where struck by light. The drum surface then passes through a toner which is attracted to the drum areas not struck by the light. The toner is then deposited and bonded to the media by a fuser. Some electrostatic copiers provide four-color toner (C, M, Y, K) cartridges that require four exposure passes to produce color output. Such copiers are very expensive.

Ink Jet

There are two types of ink jet printers—continuous and drop on demand. Continuous ink jet printers produce a continuous stream of electrically charged ink drops which may be of varying size depending on the particular printer. Unwanted ink drops are collected into a print head ink gutter using electrical force and returned to the ink well while the remaining drops impact the media. Continuous ink jet printers are classified as either vibrated (drops produced by vibrations) or non-vibrated.

Drop on demand printers produce individual drops under printer control and are either piezoelectric or thermal. Piezoelectric printers use either liquid colorant or solid colorant (called hot melt) consisting of solid wax blocks, pellets, or powder. The colorant is melted by a heating element and fed to a print element. Supplying electrical pulses to the one or more actuators connected to a print element pumps (similar to a piston) the actuator, forcing colorant out a nozzle and onto the media. The colorant instantly solidifies when it strikes the paper, avoiding ink wicking (seepage or bleeding). Applying heat to the media fuses the colorant to the media which can be any type of paper. Piezoelectric printers produce poor transparency output.

Thermal ink jet printers operate the same way and use either disposable print cartridges (Hewlett-Packard) or permanent print heads (Canon). HP invented thermal ink jet technology in 1979 and developed the first disposable print head in 1982. HP's first ink jet product was the 96 dpi ThinkJet printer announced in 1984. In 1988, HP announced its first 300 dpi DeskJet printer.

HP ink jet printer cartridge print heads use thin-film semiconductor technology suitable for high volume manufacturing. Ink jet cartridges may have one or more color ink wells, each with their own set of print elements. A print element produces ink spots under printer control. It resides on a monocrystalline silicon substrate and consists of a microscopic chamber with an electrically resistive heating element connected to an electrical conductor. One side of the chamber has a small hole, or nozzle, that faces the media.

To emit an ink drop, 5 to 20 microjoules of electrical power are applied to the heating element which heats at 100º C per microsecond and heats to over 500º C before power is removed. After 15 microseconds, an ink drop begins emerging from the hole, propelled by vapor bubble pressure caused by vaporized ink. After 25 microseconds, the ink drop escapes the nozzle, the remaining ink subsequently regroups, and the nozzle meniscus helps keep the chamber filled with ink by sucking ink from the ink well.

Because HP ink jet printers use multiple, fast-response nozzles, it achieves 300 dpi resolutions. However, because ink drops arrive propelled by expanding vapor, small aerosol drops may impact around the primary drop impact area, producing satellites that reduce effective resolution. When ink drops strike normal paper media, paper fibers absorb the ink vertically and horizontally, causing ink bleed as the ink diffuses (spreads over a wider area). It follows that using specially treated media that restricts ink bleed substantially improves actual output resolution by minimizing dot gain. It also eliminates the surface warping caused by soaking paper with ink.

Thermal Transfer

Thermal transfer printers are also known as thermal wax and dot thermal transfer printers. Fujicopian Co. LTD (Japan) developed the technology and holds the fundamental thermal transfer technology patent. Thermal transfer printers use a continuous roll ribbon, sometimes called a donor, with page-sized, alternating CMY sheets (bands). The ribbon cannot be reused and the colorants usually require special smooth paper to avoid spotty colorant adhesion. The paper is usually clay-coated to improve colorant adhesion though Tektronix has a novel approach that uses a special ribbon that applies a special foundation to normal paper.

Thermal transfer printers require one complete pass per color plane, also called a pane, to produce color output. During each pass, the colorant is spot heated, typically at 300 dpi, and pressed into the media (paper or transparencies). Because there is little colorant bleed, thermal printers produce excellent detail and rich, saturated colors. However, the colorant can scratch or crack if the media is folded.

Thermal transfer printers initially used a 5.7 mil ribbon substrate and produced fixed size dots. In 1994, Fujicopian Co. LTD introduced a thinner, 3.5 mil donor ribbon. This allows variable sized dot transfers since printing operations require less heat and provide better control. Combined with a new film-like paper with micropores (that absorbs colorant into paper rather than allowing it to bleed) and a new rectangular pixel print head, thermal printers can now produce 16 intensities per color per dot (4 bits per pixel). One study indicates that under proper conditions, the human eye may perceive this as continuous color output.

Dye Sublimation

Dye sublimation is also known as thermal dye diffusion, dye diffusion, and dye sub. Dye sublimation requires carefully matched colorants and media consisting of special paper or polyester-coated transparencies. Like thermal transfer, the media makes multiple passes. However, with dye sublimation, discrete thermal elements vaporize a sublimable dye(s) on ribbon. The gas vapor diffuses onto the special receiver media and is absorbed by a polyester coating. Controlling the discrete heating elements controls the colorant vapor amount, producing near-photographic quality using 24 bits per color. Dye sublimation is excellent for full page color since diffusion blends the colorants smoothly, though the diffusion reduces detail. Dye sublimation printers and print page costs are very expensive.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Printing Considerations

Under the best of conditions, printing graphics images on high-resolution printers is slow. Thermal transfer print page cost is constant and is considered expensive for text output. Ink jet print page costs depend on how much ink is required to produce acceptable output. Bar graphs that use cyan, magenta, and yellow bars use much less ink than bar graphs using red, green, and blue bars since the latter requires multiple CMY ink coats. Dye sublimation is the most expensive printing method and is only competitive with photography. Because of high colorant and media cost, some dye sublimation printers provide thermal transfer printing for drafts.

Printer Programming

Overview

Most printers power-up in text mode. In text mode, printers print a character "A" when the printer receives a character "A" over its interface. To invoke printer features such as text underlining, superscripts, or graphics requires sending the printer an escape sequence. Unless you are working with a family of printers such as Hewlett-Packard's DeskJet, PaintJet, and LaserJet printers, the escape sequences you send to the printer and when you send them are very dependent on the specific model of printer you are using.

An escape sequence notifies printers to interpret immediately following data differently, perhaps altering their current behavior or mode. Some temporarily place printers in graphics mode so subsequent data is treated as dot patterns to paint rather than characters to render. Others slow the print head's horizontal speed to increase or decrease dot density as the head prints.

Escape sequences begin with the 1-byte escape character (Esc) which has a hexadecimal value of 0x1B (decimal 27). When printers receive an Esc, they know that the next few characters will not be printed but will specify some characteristic. The byte or few bytes following Esc are usually case sensitive and identify the specific sequence to the printer. These bytes may be followed by a few bytes that specify various parameters.

Here are a few IBM Proprinter escape sequences:

Escape Sequence | Hexadecimal Example | Meaning
EscE | '\x1B', 'E' | Start printing all received text in bold.
EscF | '\x1B', 'F' | Stop printing all received text in bold.
EscP1 | '\x1B', 'P', '1' | Start printing text proportionally spaced.
EscP0 | '\x1B', 'P', '0' | Stop printing text proportionally spaced.
Esc3.. | '\x1B', '3', '\x18' | Set line spacing (move head down 24/216" after receiving a line feed in the example)
EscLn1n2.... | 'x1B', 'L', '\x03', '\x01' | Print the following n2*256+n1 (19 in the example) characters as graphics data at 120 dpi. Then revert to character mode.

Here are a few Hewlett-Packard InkJet escape sequences:

Escape Sequence | Hexadecimal Example | Meaning
Esc*†75R | '\x1B', '*', 'r', '7', '5', 'Q' | Print graphics data at 75 dpi
Esc*†300R | '\x1B', '*', 'r', '3', '0', '0', 'Q' | Print graphics data at 300 dpi
Esc*rbC | '\x1B', '*', 'r', 'b', 'C' | Stop printing data as graphics

Because Hewlett-Packard's printer family has many more models with many more sophisticated features than the IBM Proprinter, they have many more escape sequences with many more options. But Hewlett-Packard printer escape sequences all observe many common rules, such as always ending with capital letters, and can be combined under certain circumstances.

To navigate among the many escape sequences and their various options, PRINTPCX.C strictly observes the recommendations found in Hewlett-Packard's DeskJet Printer Family Software Developer's Guide (available directly from Hewlett-Packard). This well-written, essential guide explains Hewlett-Packard escape sequences in detail and specifies the order in which a printer should receive them. Any question not answered by the manual receives quick response from the printer support teams.

Sending Characters to Printers

The PRINTPCX.C program first calculates what initialization escape sequences to transmit to the printer based on user input. The program then uses the PC BIOS interrupt to transmit to the printer, assuming the printer is attached to the PC's LPT1: parallel printer interface. PrintByte() is the routine that performs the transmission and is isolated to simplify adapting the code to serial and network attached printers. Before transmitting any character to a printer, PrintByte() first calls WaitForPrinter() which keeps checking the LPT1: PrinterBusy bit until it indicates the printer can accept a character before returning to PrintByte(). The essential programming logic follows:

#define BIOS_LPT1               0x00

#define BIOS_PRINT_CHAR          0x00

#define BIOS_PRINTER_INT        0x17

#define BIOS_RESET_PRINTER       0x01

#define BIOS_FETCH_PRINTER_STATUS 0x02

int PrintByte(char InChar)

{

    union REGS Regs;

    WaitForPrinter(0);

    Regs.h.al = InChar;

    Regs.h.ah = BIOS_PRINT_CHAR;

    Regs.x.dx = BIOS_LPT1;

    int86(BIOS_PRINTER_INT, &Regs, &Regs);

    return;

}

int WaitForPrinter(int WaitMsgFlag)

{

    union REGS Regs;

    if (WaitMsgFlag)

        printf("\n\nWaitForPrinter: Waiting for printer to come

                      online...\n");

    do {

        Regs.h.ah = BIOS_FETCH_PRINTER_STATUS;

        Regs.x.dx = BIOS_LPT1;

        int86(BIOS_PRINTER_INT, &Regs, &Regs);

    } while(!(Regs.h.ah & 0x0080));  /* test printer busy bit */

    return;

}

After initializing the printer, PRINTPCX.C determines, line by line and color by color, what graphics data to send and transmits the graphics data following the appropriate escape sequence. Both the escape sequence and graphics data can vary by printer.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

IBM Proprinter II Dot Matrix

The IBM Proprinter II was introduced by IBM in 1986 and remains one of the most successful bilevel dot matrix printers ever manufactured. It uses a 9-wire print head with the print wires vertically spaced 1/72 inch (0.3528 mm) apart. This allows a maximum vertical resolution of 72 dpi. To facilitate follow-on printer introductions that used 27 wires for letter quality rendering, Proprinter printing resolution is expressed in 1/216 inch units. So, one print wire consumes 1/72, or 3/216, inch of vertical space.

The Proprinter prints both character text and graphics in a variety of resolutions. User programs can download custom typefaces as well. Characters appear in 9x12 pixel cells and are either designated as ascending (extending upwards such as "L") or descending (extending downwards such as "g") characters. This designation is necessary because the Proprinter uses either the top eight or bottom eight wires exclusively when rendering ascending or descending characters, respectively. Graphics modes use only the top eight wires and can intermingle with text characters under correct, but tricky, programming.

Using the top eight wires to print eight scan lines of a graphics image consumes 24/216 inch vertical space. Nine such 24/216 inch adjacent bands fit within one inch so the correct graphics mode line spacing is nine lines per inch when printing eight scan lines at once as a single band. However, the default line spacing is six lines per inch for text printing and such default text lines consume 36/216 inch to allow for blank space between text lines. Because switching from a character mode to a graphics mode does not automatically alter line spacing to 24/216 inch for graphics bands, when switching to graphics modes it is also necessary to set the line spacing. Otherwise, in this instance, the default 6/216 inch spacing leaves a 12/216 inch blank interval between each graphics band.

It is also possible to produce graphic images using less than eight scan lines per band (eight wires), though performance usually suffers since this usually requires more bands per image. This is accomplished by using as many of the top wires as necessary, ignoring all unneeded lower wires. For example, using just the top four wires produces four scan lines but requires twice as many image bands. Since each four scan-line band takes as long to print as one that uses eight scan wires, printing times double. In this instance, the line spacing must be set to 12/216 inch.

When the Proprinter powers-on or otherwise resets, it enters its Data Processing mode which prints text at 10 characters per inch at 200 characters per second using a coarse typeface. To enter into a graphics mode requires sending the printer an escape sequence. The Proprinter supports four graphics modes which all provide 72 dpi vertical resolution but varying horizontal resolutions and print speeds (see Table 8-1 on the following page).

Escape Sequence | Horizontal Speed | Horizontal DPI | Maximum Columns | Allows Adjacent Horizontal Dots?
EscK | Normal | 60 | 480 | Y
EscL | Half | 120 | 960 | Y
EscY | Normal | 120 | 960 | N
EscZ | Normal | 240 | 1920 | N

Table 8-1: IBM Proprinter graphics modes

The EscL mode simply moves the print head half as fast as the EscK mode while printing, allowing twice as many columns per inch. The EscY and EscZ modes require two passes to print each band whenever two black pels are adjacent. Since the EscL mode provides the same resolution as the EscY mode and the EscZ mode resolution exceeds general dot matrix printing capability because of black pixel imperfections and dot gain, the PRINTPCX.C sample program uses only the EscL mode to produce images.

Though the Proprinter only produces bilevel images with black or white pels, PRINTPCX.C produces output on the Proprinter using gamma corrected color input images by first reducing them to scaled achromatic gray scale images. Scaling is necessary because the horizontal resolutions and vertical printing resolutions are not equal while the horizontal and vertical scanning resolutions for the sample images are equal.

Consider printing a 72x72 pixel square using a 120 dpi horizontal resolution and 72 dpi vertical resolution. Without scaling, rather than producing a square, this produces a rectangle 1 inch (72/72) high and 0.6 inch (72/120) wide. So, to scale desired output intensity values, a scaled output horizontal pixel position is multiplied by 0.6 to determine which two input pixel intensities contribute to it. Then, the input pixel intensities are weighted.

For example, to calculate the intensity of scaled output pixel 22, multiplying 22 by 0.6 produces 13.2 which is between 13 and 14. Thus, input pixel 13 and 14 contribute to the scaled output value. But 13.2 is much closer to 13 than 14 which means input pixel 13's intensity should contribute to the output pixel value much more than input pixel 14's intensity should contribute.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Since 13.2 is 0.8 units distant from 14, adding 0.8 of input pixel 13's intensity to 0.2 of input pixel 14's intensity produces the desired output pixel 22. These scaled output pixel values are then reduced to bilevel pixel values using digital halftoning procedures. PRINTPCX.C finally prints the derived digital halftone images on an IBM Proprinter using the following logic:

#define ESC       '\x1B'

USGC IbmPpNoAutoLf[]            = { ESC, '5',  '0' };

USGC IbmPpGraphicsLineSpacing[] = { ESC, '3',  24  };

USGC IbmPpDualDensityGraphics[] = { ESC, 'L',      };

USGC IbmPpUniDirectionalPrint[] = { ESC, 'U',  '1' };

USGC IbmPpPrintArray[8][120];  /* 120 * 8 == 960 */

int IbmPpPrintBiLevelLine()

{

    static NextLineValue = 0, Entered = 0;

    if (!Entered)

    {

        InitializeIbmPp();

        Entered = 1;

    }

    m em cpy(IbmPpPrintArray[NextLineValue], GrayBilevelLine,

           MaxPrintPelsPerLine / 8);

    NextLineValue = (++NextLineValue % 8);

    if (!NextLineValue)

    {

        USGI OutputByteIndex, RowIndex, TargetByte, Index;

        USGC CharValue, MaskValue, TestMask;

        for (Index = 0; Index < sizeof(IbmPpDualDensityGraphics); Index++)

                             PrintByte(IbmPpDualDensityGraphics[Index]);

        PrintByte( ActualPrintScanLinePels       & 0x00FF);

        PrintByte((ActualPrintScanLinePels >> 8) & 0x00FF);

        for (OutputByteIndex = 0;

             OutputByteIndex < ActualPrintScanLinePels;

             OutputByteIndex++)

        {

            CharValue   = 0x00;

            MaskValue   = 0x80;

            TargetByte  = OutputByteIndex / 8;

            TestMask    = (0x80 >> (OutputByteIndex % 8));

            for (RowIndex = 0; RowIndex < 8; RowIndex++)

            {

                if (TestMask &

                    IbmPpPrintArray[RowIndex][TargetByte])

                      CharValue |= MaskValue;

                MaskValue >>= 1;

            }

            PrintByte(CharValue);

        }

        PrintByte('\x0D'); PrintByte('\x0A');

    }

    return 0;

}

int InitializeIbmPp()

{

    USGI Index;

    ResetPrinter();

    for (Index = 0; Index < sizeof(IbmPpUniDirectionalPrint); Index++)

                         PrintByte(IbmPpUniDirectionalPrint[Index]);

    for (Index = 0; Index < sizeof(IbmPpNoAutoLf); Index++)

                         PrintByte(IbmPpNoAutoLf[Index]);

    for (Index = 0; Index < sizeof(IbmPpGraphicsLineSpacing); Index++)

                         PrintByte(IbmPpGraphicsLineSpacing[Index]);

    return 0;

}

The IbmPpPrintBiLevelLine() receives pixel printing information generated from the derived achromatic image by a digital halftoning routine discussed in Chapter 14. IbmPpPrintBiLevelLine() first checks if it has previously been entered. If not, it invokes the InitializeIbmPP() routine.

InitializeIbmPP() first sends an escape sequence directing the printer to print pels only when moving the head from left to right. This prevents the printer from printing as the print head moves from right to left. Though this wastes time as the print head returns to the left margin to print the next band, it eliminates image distortions that occur when bands print slightly horizontally shifted—causing vertical lines to wiggle down the page. InitializeIbmPP() next directs the printer not to automatically generate a line feed when it receives a carriage return, providing the program greater control over vertical head movements. Finally InitializeIbmPP() sets the line spacing to 24/216 inch for eight scan-line band printing before returning to IbmPpPrintBiLevelLine().

IbmPpPrintBiLevelLine() moves the received pixel scan line into the first available row in the array IbmPpPrintArray[8][120] (in 120 bytes, there are 960 bits for 960 bilevel pels) and checks if it has now accumulated eight image scan lines. If not, IbmPpPrintBiLevelLine() returns. Otherwise, IbmPpPrintBiLevelLine() prints the eight scan-line band, transmits a carriage return-line feed combination, and returns to the caller. When IbmPpPrintBiLevelLine() executes again, it treats the IbmPpPrintArray array as empty and reuses IbmPpPrintArray starting with row zero to accumulate the next band.

To print a band, IbmPpPrintBiLevelLine() first sends the printer the IbmPpDualDensityGraphics escape sequence followed by the number of image columns it will print. The value of the number of bytes to print is byte-reversed (little-endian byte ordering). Moving from column zero to the last valid image column in IbmPpPrintArray, IbmPpPrintBiLevelLine() repeatedly calls PrintByte() to print eight vertical image pels at a time.

For a print wire to generate a black pel, its corresponding bit value in the character should be a binary one. For a print wire to generate a white pixel (not print), its corresponding bit value should be a binary zero. The top (uppermost) wire in the vertical print wire stack corresponds to the high-order bit position in the character to print and the bottom (lowest) wire corresponds as indicated in Table 8-2.

Wire Position | Bit Position
8 (Top) | 10000000 (High Order)
7 | 01000000
6 | 00100000
5 | 00010000
4 | 00001000
3 | 00000100
2 | 00000010
1 (Bottom) | 00000001 (Low Order)

Table 8-2: IBM Proprinter print wire bit positions


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Hewlett-Packard Printers

Hewlett-Packard offers a wide variety of high quality xerographic and ink jet printers with street prices that range from a few hundred dollars to a few thousand dollars, depending on configuration options and design complexity. What is unique about these highly imitated and competitively priced printers is their unusual degree of common programming considerations. Unlike the dot-matrix printer market with highly diverse escape sequences that require distinct programming, most Hewlett-Packard printers use identical escape sequences wherever possible. This greatly simplifies programming Hewlett-Packard printers which, combined with their low cost, comprise excellent devices to conduct color printing experiments.

Hewlett-Packard LaserJet and DeskJet 500 printers produce single-plane bilevel images using xerographic toner and black ink, respectively. The Hewlett-Packard DeskJet 500C produces three-plane CMY bilevel images, and the Hewlett-Packard DeskJet 550C produces four-plane CMYK bilevel images; both printer models use ink jet technology. All of these printers produce 300 dpi output and subsequent DeskJet models produce even higher resolution output using improved inks.

Unchanged, the PRINTPCX.C program produces output on these devices, and slight modifications of the principles illustrated in the program allow users to exploit features on newer Hewlett-Packard printers. So, mastering one printer model allows programmers to apply and expand their skills with the newer models. Because of their combined high quality and low cost, backed by renowned service, it is small wonder Hewlett-Packard holds a well deserved, commanding lead in the printer market, particularly with ink jet printers.

To control Hewlett-Packard printers, the PRINTPCX.C program defines the following escape sequences:

#define ESC       '\x1B'
#define DUMMY_VALUE  0x00

USGC HpReset[]                       = { ESC, 'E'                };
USGC HpSetPortraitOrientation[]      = { ESC, '&', 'l', '0', 'O' };
USGC HpSetPerfSkipOff[]              = { ESC, '&', 'l', '0', 'L' };
USGC HpEndRstrGr[]                   = { ESC, '*', 'r', 'b', 'C' };
USGC HpSet1PlanePerRowBw[]           = { ESC, '*', 'r', '+', '1', 'U' };
USGC HpSet3PlanesPerRowRGB[]         = { ESC, '*', 'r', '+', '3', 'U' };
USGC HpSet3PlanesPerRowCMY[]         = { ESC, '*', 'r', '-', '3', 'U' };
USGC HpSet4PlanesPerRowKCMY[]        = { ESC, '*', 'r', '-', '4', 'U' };
USGC HpSetRstrGrResolution75[]       = { ESC, '*', 't',      '7', '5', 'R' };
USGC HpSetRstrGrResolution100[]      = { ESC, '*', 't', '1', '0', '0', 'R' };
USGC HpSetRstrGrResolution150[]      = { ESC, '*', 't', '1', '5', '0', 'R' };
USGC HpSetRstrGrResolution300[]      = { ESC, '*', 't', '3', '0', '0', 'R' };
USGC HpSetRstrGrWidth[]              = { ESC, '*', 'r', '#', 'S' };
USGC HpSetRstrGrPrintMode[]          = { ESC, '*', 'p', '4', 'N' };
USGC HpSetDepletionLevel[]           = { ESC, '*', 'o', '1', 'D' };
USGC HpSetShinglingLevel[]           = { ESC, '*', 'o', '0', 'Q' };
USGC HpSetRstrGrQualityKeypad[]      = { ESC, '*', 'r', '0', 'Q' };
USGC HpSetRstrGrQualityDraft[]       = { ESC, '*', 'r', '1', 'Q' };
USGC HpSetRstrGrQualityHigh[]        = { ESC, '*', 'r', '2', 'Q' };
USGC HpStartRstrGr[]                 = { ESC, '*', 'r', '0', 'A' };
USGC HpSetRstrGrCompressionMethod[]  = { ESC, '*', 'b', '0', 'M' };
USGC HpTransferRstrGrData[]          = { ESC, '*', 'b', '#', 'V' };
USGC HpTransferRstrGrLastData[]      = { ESC, '*', 'b', '#', 'W' };

Some of these escape sequences, such as HpSetRstrGrWidth, contain a "#". This indicates an integer value is substituted in place of the "#" character. The following routine transmits escape sequences to Hewlett-Packard printers and performs this translation using sprintf():

int XmitHpEscapeSequence(USGC *EscSeqArray, int EscSeqSize, int Value)
{
    USGI EscSeqIndex, FormatIndex;
    USGC FormattingBuffer[10];

    for (EscSeqIndex = 0;
         EscSeqIndex < EscSeqSize;
         EscSeqIndex++)
            if (EscSeqArray[EscSeqIndex] != '#')
                 PrintByte(EscSeqArray[EscSeqIndex]);
            else
            {
                sprintf(FormattingBuffer, "%u", Value);

                for (FormatIndex = 0;
                     FormatIndex < strlen(FormattingBuffer);
                     FormatIndex++)
                         PrintByte(FormattingBuffer[FormatIndex]);
            }

     return 0;
}


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

To print on Hewlett-Packard printers, PRINTPCX.C first determines what printer model is specified from the program input arguments. It then sends the correct printer initialization escape sequences. Table 8-3 illustrates which sequences PRINTPCX.C sends to which Hewlett-Packard printers. The ultimate programming source for this printer family is Hewlett-Packard's 150-page DeskJet Printer Family Software Developer's Guide available directly from Hewlett-Packard. This manual's 35-page Graphics Control section provides extensive escape sequence guidance as well as an essential gray balancing algorithm.

Escape Sequence | LaserJet and DeskJet 500 | DeskJet 500C | DeskJet 550C | Comment
HpReset | Y | Y | Y | Resets printer
HpSetPortraitOrientation |  |  |  | Portrait orientation (11" high, 8.5" wide)
HpSetPerfSkipOff |  |  |  | Start printing at top left of page
HpEndRstrGr | Y | Y | Y | Defensive escape sequence to force defaults
HpSet1PlanePerRowBw | Y |  |  | Print one bilevel plane (black and white)
HpSet3PlanesPerRowRGB |  |  |  | Print three bilevel planes (RGB input)
HpSet3PlanesPerRowCMY |  | Y |  | Print three bilevel planes (CMY input)
HpSet4PlanesPerRowKCMY |  |  | Y | Print three bilevel planes (KCMY input)
HpSetRstrGrResolution75 |  |  |  | Print bilevel output at 75 dpi
HpSetRstrGrResolution100 |  |  |  | Print bilevel output at 100 dpi
HpSetRstrGrResolution150 |  |  |  | Print bilevel output at 150 dpi
HpSetRstrGrResolution300 | Y | Y | Y | Print bilevel output at 300 dpi
HpSetRstrGrWidth | Y | Y | Y | Specifies horizontal scan line width
HpSetRstrGrPrintMode | Y | Y | Y | Print using smart bi-directional movements (default)
HpSetDepletionLevel | Y | Y | Y | Do not remove any dots—print them all.
HpSetShinglingLevel | Y | Y | Y | Create dots in order without interlacing
HpSetRstrGrQualityKeypad | Y | Y | Y | Check user set printer keypad for print quality
HpSetRstrGrQualityDraft |  |  |  | Use draft print quality to conserve ink
HpSetRstrGrQualityHigh |  |  |  | Use high print quality
HpStartRstrGr | Y | Y | Y | Start raster graphics
HpSetRstrGrCompressionMethod | Y | Y | Y | Selects the compression method (none for PRINTPCX.C)
HpTransferRstrGrData |  |  |  | Sends all but the first plane scan lines on an output row
HpTransferRstrGrLastData |  |  |  | Sends the last (or only) plane scan line

Table 8-3: Escape sequences PRINTPCX.C uses with various printers

If the printer is a LaserJet or DeskJet 500, PRINTPCX.C converts the color image into an achromatic gray scale image. Because Hewlett-Packard printer vertical and horizontal images are equal as they are in the scanning resolutions for all sample images, no scaling is required to compensate for unequal horizontal and vertical resolutions. However, because 300 dpi is a relatively high resolution, PRINTPCX.C magnifies images up to MAX_REPLICATION_FACTOR times using horizontal linear interpolation to calculate values falling between input pixel values. These interpolated lines are then replicated vertically the correct number of times to provide appropriate vertical scaling.

Finally, the CMY and CMYK values are gray balanced to provide truer gray shades by reducing cyan ink levels near the RGB cube achromatic diagonal. After adjusting the gray balance, the RGB values are individually error diffused within their color planes to produce their final individual bilevel output scan lines. When printed on Hewlett-Packard Premium InkJet Glossy Paper, the results sometimes have to be seen to be believed. The colors seem to lunge off the paper.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 9
Device Color Consideration Overview

What This Chapter is About

Previous chapters discussed light and color, and how we perceive them. This chapter provides an overview discussing considerations that complicate faithful color image reproduction for both additive and subtractive devices. Usually referred to as device independent color, the end objective is to render identical output regardless of rendering device. Since, say, advertising agencies demand to know exactly how color pictures will appear in various magazines, it is big business.

James Clerk Maxwell's Color Reproduction Experiments

In 1861, James Clerk Maxwell produced the first trichromatic (three-color) color photographic images by photographing scenes using black and white film. Actually, he photographed any given scene three times—once through red glass, once through green glass, and once through blue glass. Maxwell then created positive slides, referred to as separations, for each of the three images and projected them through lantern projectors.

The light from the lantern projector displaying the slide image captured through red glass then passed through red glass. Similarly, light emitted from the projector displaying the slide image captured through green glass passed through green glass, and light emitted from the projector displaying the slide image captured through the blue glass passed through blue glass. When the three now-colored slide projections were superimposed on a white wall, a seemingly colorful image resulted that visually contained many more colors than just red, green, and blue. All modern color imaging processes owe their origin to the principles these experiments demonstrated. As amazing as Maxwell's experiments were in their day, their simplicity belied human perception complexities.

Additive Light Implications

Suppose for a moment Maxwell could use blue glass lantern projector filters that transmitted power at wavelengths within a narrow 1 nm wide range centered at 450 nm and absorbed all other wavelengths. Similarly, suppose he could use an analogous green and red filter centered at 510 nm and 650 nm respectively. Figure 9-1 illustrates the human retinal cone response curves discussed earlier and superimposes these hypothetical red, green, and blue filter ranges. This immediately gives rise to a few problems.

Figure 9-1 Response characteristics of the eye

First, looking at the figure, it is clear that this filter set only allows roughly 1 percent of all visible light to pass (3 nms out of 370 nms). Such low resulting power lengthens exposure times during image capture and darkens images during viewing. Second, light passing through the blue filter essentially only stimulates the blue cones because blue stimulations dominate the red and green cone stimulations at 450 nm. Similarly, light passing through the 650 nm filter essentially stimulates only the red cones. However, light passing through the 510 nm green filter significantly stimulates the M-, L-, and S-cones.

Thus, attempting to reproduce green objects in a scene using this approach introduces unwanted red and blue stimulations which desaturates greens and makes white objects exhibit magenta tones. Adjusting projector light intensities may generate more genuine appearing whites but compromises saturated color fidelity elsewhere. So, additive color reproduction requires tradeoffs when attempting to reproduce a spectrally rich scene using scant spectral resources.

An alternative approach would use filters that transmitted a wider wavelength range, but that exacerbates unwanted stimulation problems because a quick inspection of Figure 9-2 quickly reveals that there is no possible filter set that stimulates only the green cone without stimulating either blue or red cones, and possibly both. The feeble mechanisms you have are simply unable to stimulate the complex nonlinear triangulation computations human vision systems perform. And, it's worse in color printing.

Figure 9-2 Filter sets


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Subtractive Light Implications

Color printing uses cyan, magenta, and yellow filters in the form of colorants. Cyan colorants absorb photons with "red" wavelengths but pass photons with "green" and "blue" wavelengths. Similarly, magenta colorants absorb photons with "green" wavelengths and pass photons with "red" and "blue" wavelengths. Finally, yellow colorants absorb photons with "blue" wavelengths but pass photons with "red" and "green" wavelengths. The following naive relationships are often suggested:

White – Red | = Cyan
White – Green | = Magenta
White – Blue | = Yellow

Hold on a second. What does the statement Cyan colorants absorb photons with "red" wavelengths but pass photons with "green" and "blue" wavelengths actually mean? It simply means cyan colorants (e.g., inks, toners, wax, etc.) tend to absorb photons with wavelengths greater than 580 nm. What percentage of the photons of a specific wavelength a colorant absorbs depends on the particular colorant and its concentration as well as the specific wavelength under discussion. And here is where it gets very ugly, very fast.

The solid line in Figure 9-3 illustrates the effect of an ideal cyan colorant. At any concentration, it correctly allows all wavelengths below 580 nm to pass. The percentage of photons at wavelengths above 580 nm that pass is strictly a linear colorant concentration function. For example, a zero percent concentration (no colorant) passes all red photons. A 50 percent concentration absorbs 50 percent of the photons at all wavelengths above 580 nm. It's all pretty simple but, unfortunately, equally naive.

Figure 9-3 Ideal and realistic cyan colorant behavior

The dotted line in Figure 9-3 illustrates more realistic cyan colorant behavior. Note that the colorant absorbs photons with wavelengths less than 580 nm when it should not be doing so. And, above 580 nm the colorant is sometimes allowing red photons to escape that it should be absorbing and absorbing some that should be escaping. Worse, the percentage of photons absorbed varies by photon wavelength and is anything but linear. For example, increasing some black colorants to areas actually lightens the areas rather than darkens them because the additional colorant develops a sheen (specular reflection) that reflects more light. Combine that with some complications such as:

  • Colorant spectral reflectance metamerisms
  • Varying paper reflectance and colorant absorption
  • Varying ambient lighting spectral power distributions
  • Varying human visual system nonlinear sensitivities to colors and color shifts based on photon wavelengths and saturations

and you have entered the bold world of device independent color challenges. The effect is that attempts to use straightforward but unfortunately naive approaches to print color pictures and faithfully render them on CRTs are doomed. As one sad example, one printer company president futilely spent an entire day trying to adjust his image colors on a CRT so his son's color image would print "as it should" on one of his printers.

Even in the simpler case of CRTs, CRT phosphors vary by manufacturer and model with each phosphor exhibiting a unique spectral power distribution intensity based on use (i.e., fading). And, before viewing important images, be sure to leave your CRT on for a few hours and stabilize the room temperature and lighting. And, of course, don't touch the contrast, brightness, or hue dials capriciously, etc.

In the case of high quality color printers, expect them to have high-speed color look-up tables that specify precise colorant mixes for each possible color. The tables assume specific colorants and reflect painstaking measurements of all possible colorant combinations (don't buy a generic low-cost cartridge). This allows sophisticated imaging programs to generate faithful output images predictably until someone inadvertently breaks the fragile rules.

As a final note, one of this book's demonstration programs shows how to print color pictures on low-cost Hewlett-Packard color ink jet printers. The program uses an algorithm Hewlett-Packard Corp. openly includes in its DeskJet printer documentation that helps correct noticeable characteristic color imbalances occurring when cyan, magenta, and yellow ink amounts are nearly equal (near black, white, and gray values).

This algorithm scales the cyan amount back approximately one-third to produce a more faithful achromatic result. Otherwise, the color is tinged jade green. There is absolutely nothing unusual about this particular situation—it's simply representative of subtractive color technical challenges.

Conclusion

A well known computing industry dignitary from a large Northwestern software company once addressed color scientists at an internationally renowned color research laboratory suggesting they were "making color too difficult." After all, there is red, green, and blue for CRTs and cyan, magenta, and yellow for printers—that's "all" that folks needed to know. For most people, faithful image reproduction and device independent color is difficult if not impossible to achieve, period. But, you be the judge.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

SECTION 10

Chapter 10
Image File Format Overview

What This Chapter is About

Digital images are stored as data files. Because of file size considerations, image data is usually compressed to conserve disk storage space or transmission times. This requires files to contain descriptive information that assists programs to reconstruct the image. This chapter discusses one simple image format in detail, the PCX file format developed by ZSoft Corp. of Marietta, Georgia. It then presents an overview of two other popular World Wide Web image file formats.

Image File Headers

Among other things, images

  • Vary in horizontal and vertical resolution (number of samples)
  • Use different compression algorithms
  • Represent their pixel information differently
  • Vary in the order they provide image scan-line information

Therefore, compressed image data requires separate description information, usually contained in a file header, that allows programming to reconstruct the associated image.

Some of these file formats, such as the Tagged Information File Format (TIFF), are extremely flexible, providing considerations for numerous variations of dozens of different image characteristics. In fact, most folks feel the TIFF file format is too flexible because no single program can possibly decode all potential image variations and must resort to decoding well-defined subsets.

Further complicating matters, the Windcrest/McGraw-Hill book Graphics File Formats by David C. Kay and John R. Levine lists nearly 50 different file formats. So, it becomes obvious why image file conversions have proven profitable for their developers. Rather than becoming distracted by image file format minutia, we select a single simple file format as a basis for detailed discussion on the processing necessary to reconstruct compressed image data faithfully.

The PCX File Format

The PCX file format is one of the oldest, simplest, and easiest to understand image file formats. Developed for its popular PC Paintbrush program, ZSoft extensively documented the PCX file format, eventually allowing it to achieve the status of a leading PC image file format. Moreover, because its several extensions complement PC display adapter evolution, it remains pervasive though it is less popular than it once was.

The PCX file format exhibits the symptoms of an originally well-conceived design that technology eventually made obsolete. For example, the original file format consisted of a fixed-length 128-byte header that described the compressed image data that immediately followed. The file's header reserved 48 bytes for a 16-entry color palette which is now completely useless since today's palettes have 256 entries requiring 768 bytes.

Therefore, in file format version 3.0 format five, the larger palette was appended to the file's end following a signature byte located 769 bytes from the file's end and having a value of 0x0C. This is intellectually clumsy in theory, but works well in practice.

Since the PCX file format version 3.0 also supports direct color files, it is important to check the header's NumberOfColorPlanes field value to determine the number of color planes the image contains. If Number-OfColorPlanes has a value of 0x01, the image is a palette image and the palette is located 768 bytes from the end of the file as described. Otherwise, NumberOfColorPlanes's value must be 0x03 indicating three color planes—one each for red, green, and blue. Programs in this book only deal with file format version 3.0, format five files.

Another significant reason for the decline of PCX file format popularity is due to its compression scheme. To its credit, the compression scheme is very simple and lossless—decompressed data is identical to the original data. However, it is relatively inefficient as compared to popular compression schemes such as LZW that other file formats use. However, LZW is, unfortunately, a patented compression scheme, and software that uses it requires a license from Unisys Corp.

PCX Header

The PCX header describes the file's single compressed image that begins at file offset 128L. It defines the image's horizontal and vertical resolution and whether the image is a direct color image or a palette image. The header's BitsPerPixel field indicates the number of bits per pixel for direct color images. The PCX.H header file contains the header fields which are summarized in Table 10-1:

Offset | Field Type | Field Name | Comment
00 | USGC | ManufacturerFlag | Must have 0x0A for value
01 | USGC | FileFormatVersionNumber | Must be 0x05 for this book
02 | USGC | EncodingMethod | Must have a value of 0x01 for run length encoding
03 | USGC | BitsPerPixelPerPlane | Palette images => 8; Direct color images =>24
04 | USGI | XMin, YMin, XMax, YMax | Image Width = XMax – XMin +1, Height = YMax – YMin + 1
12 | USGI | SuggestedHorizontalDPI | Horizontal scanning resolution
14 | USGI | SuggestedVerticalDPI | Vertical scanning resolution
16 | USGC | SixteenColorPalette[48] | Obsolete
64 | USGC | Reserved |
65 | USGC | NumberOfColorPlanes | 0x01 => Palette image; 0x03 => Direct color image
66 | USGI | BytesPerScanLine | For each color plane, the number of bytes to allocate for scan lines
68 | USGI | PaletteInformation | Ignore for this book
70 | USGI | ScreenHorizontalPixelCount | Horizontal screen size in pixels
72 | USGI | ScreenVerticalPixelCount | Vertical screen size in pixels
74 | USGC | Filler[54] |

Table 10-1: PCX.H header file fields


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

PCX Image Compression and Decompression

The PCX file format uses a simple lossless compression scheme. To decompress PCX scan-line data, first read a single byte. If the two high-order bits are both set to binary one values, then the low-order six bits indicate a replication count for the next byte which ranges from 1 to 63. Otherwise, the value is a pixel value. A replication value of 0x0A indicates the next byte should be repeated ten times—hence the name run length encoding. The DecodePcxScanLine() present in many of this book's programs demonstrates how to decode entire image scan lines for palette or direct color images. For direct color images, the color plane scan lines are individually compressed in the following order:

Scan Line 0: | RRRRRRR... | (Red color component)
 | GGGGGG... | (Green color component)
 | BBBBBBB... | (Blue color component)
Scan Line 1: | RRRRRRR... | (Red color component)
 | GGGGGG... | (Green color component)
 | BBBBBBB... | (Blue color component)

etc.

In summary, the PCX file format is a simple file format that illustrates many of the principles involved in recovering images from their compressed form within files. While it is becoming less popular, to its credit it provides a lossless compression scheme which does not require a license. Thank you, ZSoft.

Image File Size and Transmission Bandwidth

When scanning a photo to create an image file for local or LAN-connected system viewing, transmission bandwidth is often not an issue. So, it's tempting, and not unrealistic, to scan photos in millions of colors (24 bits per pixel) or thousands of colors (15 or 16 bits per pixel) at high scanning rates such as 300 dots per inch (dpi). This approach visually produces continuous-tone, photographic-quality images on monitors. Such high detail and color quality scanning also produces large images— 1 MByte images are not unusual. A 10 MBytes/sec SCSI channel would transport this image in about a tenth of a second and a Pentium processor would move the image into video memory in a heartbeat, using a 132 MBytes/sec PCI system bus.

However, downloading such an image from the World Wide Web using a cocktail-straw 28.8 KBits/sec modem significantly shifts the landscape, usually resulting in frantic quality versus size tradeoffs to drastically reduce download time. Moreover, consider that typical home modems only provide users less than 80 percent data throughput at any transmission speed because they add a start and stop bit to each eight data bits. TCP/IP protocol adds even more overhead and delay. Recall that transmission weight is what counts with international postal correspondence, resulting in tissue paper envelopes that double as stationery on the inside. Similarly, WWW images presently should be approximately 20 Kbytes if at all possible. That's 1/50th the size of a 1 MByte file and introduces us to the necessity of image compression and image file types, not to mention visual quality tradeoffs.

Image Files and Their Image Compression Algorithms

There are two primary image file types used on the WWW. They have radically different image compression approaches and formats, each with its own advantages. As luck would have it, these two types of file formats complement each other nicely for WWW use.

An image file with a jpg extension is a Joint Photographic Experts Group (JPEG) File Interchange Format (JFIF) file and uses JPEG compression. Alternately, an image file with a gif extension is a Graphics Interchange Format (GIF) file developed in 1987 and copyrighted by CompuServe Corp. GIF files use Lempel-Ziv & Welsh (LZW) compression, now patented by Unisys Corp. Hence, developers who distribute GIF creation applications must obtain a license from Unisys. A third image file type which is becoming popular is the Portable Network Graphics (PNG) file format. PNG files have a png extension and use a compression called Lempel-Ziv (LZ77). In contrast to LZW, while LZ77 is similar to LZW, LZ77 is an open, royalty free standard and promises to allow PNG to replace GIF eventually.

Here's the second rub. Images are considered the digital world's Styrofoam—bulky and tough to compress. There are two types of general compression approaches: lossy and lossless. Lossless compression approaches, such as LZW, preserve all image detail. Lossy compression approaches, such as JPEG, first facilitate subsequent compression operations by discarding (hopefully) visually insignificant image detail. This limits the number of times you can edit an image since, like audiotape recordings, the image degrades slightly with each new copy made from a previous copy.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

To create an image file, users usually first create direct color (24-, 16-, or 15-bit color) image data either using a graphics program, a scanner, or both. When the image is suitable for saving, users can usually select the file format type as a menu option during the save operation. The content usually favors one format or the other but here are some guidelines.

JPEG

JPEG exploits the fact that human vision is more sensitive to slow intensity and color transitions than to rapid changes over short distances. Consequently, JPEG does not work well on lettering, line drawings, or art containing sharp edges or flat colors. Rather, JPEG best compresses photographs, natural artwork, and images of otherwise natural scenes.

JPEG compression first divides images into 8x8 pixel groups and compresses each group individually, independent of the others, by decomposing the R, G, and B spatial image data into waves of different frequencies—think of different size egg cartons. Depending on the compression amount (quality setting) higher frequency information is then selectively discarded—much like sanding rough wood. Most importantly, when decompressed, JPEG images always produce slightly degraded 24-bit images.

GIF

There are two popular GIF file versions:

  • GIF87a supports progressive display via interlaced scan lines.
  • GIF89a extends GIF87a and adds transparency, text comments, and graphics animation.

GIF only supports indexed color images, and users can usually specify less than 8 bits per pixel. This naturally helps reduce file size but limits images to 2 (bilevel), 4, 8, 16, 32, 64, 128, or 256 colors maximum. Table 10-2 summarizes how this affected one image (sizes rounded to nearest 100 bytes).

Bit Depth | Number of Colors | Dithered File Size | Nondithered File Size
8 | 256 | 9,500 | 9,500
7 | 128 | 8,000 | 7,600
6 | 64 | 6,100 | 5,500
5 | 32 | 4,400 | 3,800
4 | 16 | 3,900 | 1,900

Table 10-2: Effect of dithering on file size

After specifying the number of colors allowed in an image, selecting the correct colors is the second step. With leading image processing applications such as Adobe's PhotoShop or JASC's PaintShop Pro (a.k.a. PSP) shareware, users can specify a predefined palette or have the image program determine the best colors on an image by image basis. This uses a color selection algorithm that produces image palettes called custom or adaptive palettes.

Regardless, the resulting palette is eventually saved in the image file along with the compressed image data in order to recover the image information later. The program then uses the palette to determine the best color representation for each pixel and optionally aerosols any error to nearby pixels that have not yet been indexed. This optional error diffusion process, also called dithering, often tricks the human visual system into believing there are more colors in an image than there actually are. However, these fuller-gamut images often have a grainy or otherwise pointillist appearance some folks consider undesirable.

After converting image data to index image data, LZW compresses the indexed image row by row, byte by byte, by remembering the previously encountered input data and incrementally extending its memory (that old stuff plus this new byte). This row-oriented behavior best compresses horizontal patterns. In addition to saving scan lines consecutively from top to bottom, GIF file images can be saved as interlaced (every eighth row, then every fourth row, etc.) This allows WWW browsers to display an interlaced image as a blocky, low-resolution image which gradually resolves, called a progressive display, often at the expense of a 10 percent additional file size increase. Because LZW compression is tricky, getting the smallest possible file size is difficult. The most impressive GIF wizard allows users to vary the number of allowed colors and applies several different algorithms to minimize file size. Table 10-3 summarizes the file formats.

Format | Typical Compression Ratio | Description
GIF | 4:1 - 10:1 | Lossless only when images have 256 or fewer colors. Best for images with flat color and sharp edges. Horizontal patterns compress better than vertical patterns.
PNG | 10 to 30 percent smaller than GIFs | Similar to GIFs only often better. Best with flat color art and sharp edges. Compresses horizontally and vertically but solid color blocks often compress best.
JFIF (JPEG) | 10:1 - 20:1 (High quality) | Has little or no image quality loss with continuous tone images. Bilevel images and flat color art with sharp edges degrade. Degradation loss can be cumulative with subsequent image edits.
 | 30:1 - 50:1 (Moderate quality) | Often the best WWW choice.
 | 60:1 - 100:1 (Poorest quality) | Usually suitable only for previews and thumbnails.

Table 10-3: Summary of file formats


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Browser Software

On the WWW, you often you see the statement "Best viewed with Netscape v2.0" or some other similar statement. For digital images (not text), this is largely a content-free statement since both Netscape's and Microsoft's browsers render images similarly at this time and these two browsers command at least 90 percent of all distributed browsers. However, both browsers must contend with representing WWW images in the user's current display system mode, which is the third rub.

If a user system is in a direct color mode, a browser's job is easy. JPEG images naturally decompress to 24-bit images and any necessary conversion to 15- or 16-bit color modes is trivial. With indexed color GIF images, the image file color palette describes each allowed pixel value using 24 bits. So, it is trivial to render any indexed color pixel. Hence, there is no problem with GIF images, though the number of colors are reduced and the image may exhibit dithering that occurred during image compression. The bad news, however, is that it's a rare display system that operates in direct color mode. Hence, the fourth and final rub is that the overwhelming majority of user systems operate in an indexed color mode.

Consider now what happens when rendering a JPEG image in indexed color mode. In this situation, a 24-bit image usually contains thousands of colors not represented in the paltry 256-color palette. This requires a reconciliation procedure. Depending on the user's preferences settings, the browser either dithers the image using the current palette and a computationally expensive (slow) algorithm or uses a closest color match procrustean fit and ignores any chromatic error introduced. The same approach is used for GIF images that have a palette that does not match the display system palette. This means that images that were dithered during compression may be dithered a second time. Hence it is important to understand what the system palette looks like when a browser runs.

The 6x6x6 Browser Palette

When Netscape and Microsoft browsers begin executing, they load 216 entries in the system palette, leaving 40 for other system uses. These 216 entries specify all possible combinations of six different intensities for red, green, and blue—0, 51, 102, 153, 204, and 255. You will see these collective entries referred to as the dither-free, standard browser, or browser safe palette. From the previous discussion, it follows that specifying these 216 colors as the palette for indexed color image compression or artistic graphics development prevents dithering during browser rendering— though with images, dithering may still occur during image compression. So, it follows that it is perhaps best to avoid custom or adaptive palettes during indexed color image compression because that virtually guarantees browser dithering and potential visual surprises from system to system. Avoid browser dithering. This is easily achieved by using a small number of the colors in the browser safe palette.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 11
Mathematics Primer

What This Chapter is About

This chapter reviews all the important mathematical concepts this book requires. The various topics are:

  • Normalization
  • Gamma correction
  • Quantization
  • Probability
  • Wave frequency
  • Running computations

This material attempts to capitalize on intuitive analogies. We opt for the latter because there is no shortage of complex image processing discussions. At least give the first topic a try—you may well be surprised how well you do.

Normalization

Suppose you have an automobile that drives forward at a maximum 100 miles per hour (mph). If we define a forward speed as not going in reverse, this car's allowable forward speeds are zero (stopped but not going in reverse) up to and including 100 mph. Here, the car's forward speeds can vary continuously (can assume any value, say exactly 52.6863452639204 mph) and have an absolute range or dynamic range from zero to 100.

A normalized forward speed is the speed divided by the absolute range's maximum speed (100 mph). Thus, 50 mph becomes 50/100, or 0.50, and 25 mph becomes 25/100, or 0.25. How about 66.3 mph? (Answer: 0.663.) Since allowable absolute speeds range from and include zero up to and including 100, normalized speeds therefore range from and include 0.00 (0 mph divided by 100 mph) up to and including 1.00 (100 mph divided by 100 mph). Pretty simple, right?

Normalization is important when image processing assigns values to light intensities. Sometimes a monitor's absolute intensity range varies from zero (dark) to fifteen (maximum intensity). Other times the absolute range is from zero (dark) to 255 (maximum intensity). In the first case, an intensity with a value of five is 5/15, or one-third the maximum intensity value. In the second case, an intensity with a value of five is 5/255, or 1/51 the maximum intensity value. To have one-third the maximum intensity value requires the absolute value to be 255/3, or 85.

So, to eliminate intensity value confusion, digital imaging discussions use normalized intensity values. When an intensity has a 0.5 value, multiplying it by the maximum intensity value translates it to its output intensity value. For example, 0.5 times 255 produces 127.5 which becomes 127 in integer arithmetic. Finally, normalization is used to understand gamma correction procedures.

Gamma Correction

Using normalized image processing values suggests we can obtain a 50 percent output display intensity by specifying a value of 0.5, regardless of the display (e.g., monitor) characteristics. But things are not that simple. In fact, most display devices do not have a linear (straight line) response to normalized image values. An analogy helps to explain this.

Suppose you have a one-gear automobile that drives forward at a maximum 100 miles per hour (mph). If you set your speed control dial to zero percent, you are at rest (zero mph). Similarly, if you set the speed control dial to 100 percent, you eventually go 100 mph. Normalizing, you can continuously go 0.00 to 1.00, depending on the speed dial setting.

Now, suppose you set the speed dial halfway (0.50) and discover you go less than 50 mph. You eventually reach 50 mph by periodically increasing the speed dial setting and watching your speedometer. When you achieve 50 mph, you leave the speed dial in that position. For other speeds you might do the same. Finally, to automate the process, you might construct a look-up table (LUT), similar to Table 11-1, which lists a speed dial setting for every desired speed.

Speed Look-Up Table
Desired Speed (MPH) | Normalized Desired Value | Required Speed Dial Setting
0 | 0.00 | 0.00
50 | 0.50 | 0.7075
100 | 1.00 | 1.00

Table 11-1: Look-up table

This type of situation exists in image processing. Quantized images may direct monitors to reach a 0.50 normalized intensity level for one or more pixels. However, intensity measurements may reveal the normalized intensity level is only 0.25—far too dark and far from satisfactory. So, image processing software must compensate for this hardware phenomena. One approach creates a LUT for every display intensity and color combination. This requires millions of measurements and several megabytes. And, as phosphors age, the table becomes increasingly invalid.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

A simpler approach uses a function to describe the relationship between the input pixel values and the monitor's intensity value. A function is simply a mathematical rule that relates one set of numbers (pixel values in this case) to another (the monitor's measured intensity). Once the function is known, it is easy to put the values in a look-up table (described above) and use this to correct the monitor output values. For high accuracy, a complicated function could be required. But, it turns out we can do a very good job with the following simple function:

output display intensity = (input pixel value)gamma

where the value gamma is an exponent (the number of times a normalized value is multiplied by itself). For most display monitors we need a gamma in the range of 0.5 to 0.8, and this fractional exponent compresses the range pixel values so that the monitor output is approximately correct. The challenge is picking the gamma value without making numerous measurements and calculations.

A straightforward approach uses midpoint correction. In this approach we pick the gamma value so a 0.50 input pixel value gives a 0.50 output intensity and then apply this value (using the above function) to all pixel values in the input range of 0 to 1.0. To continue the speed dial example, we compute a gamma of 0.5 (square root) using the midpoint correction method.

Gamma correction is a fundamental video system and computer graphics consideration. Referring to Table 11-1, note setting the speed dial to 0.7075 produced a desired normalized 0.50 speed value. Note also that multiplying 0.7075 times 0.7075 approximately produces 0.50. Restated, 0.7075 squared approximately produces 0.50 which means 0.7075 is the approximate square root of 0.50. So, taking the square root of the desired normalized speed provides a quick speed dial setting for the normalized speed. What's going on?

Previous material discussed normalization, a process that transforms a number range into a range from 0.00 to 1.00. This range has an unusual property—multiplying any number in the range by itself produces a number that is smaller than the original number. So for numbers within this range, a number's square root is bigger than the original number. Transforming the normalized square root value back into the original absolute range, therefore, produces a larger absolute number than the original absolute number.

So, apply the process to the above speed example to achieve 64 mph:

Normalize 64 mph: | 64/100 = 0.64
Find the normalized value's square root (0.80): | 0.80 x 0.80 = 0.64
Convert the normalized square root to speed dial: | 0.80 = 8/10

But life is not so simple. As the phosphors slowly age, this rudimentary correction slowly fails as well, requiring another recalculation. So, the problem becomes, What simple process produces a speed dial setting that causes the automobile to achieve a normalized 0.50 speed? Also, note the red, green, and blue electron guns likely have individual root correction values, requiring different independent corrections.

The previous discussion contrived the 0.7075 value so we could use the square root (gamma = 0.50). In other cases, the square root may amplify intensities too much or too little. So, the real process actually consists of determining what root (gamma value) to use. It proceeds by displaying intermixed intensities. Half the intensities are normalized at 0.50. The remaining intensities alternate between scan lines consisting of 0.00 and 1.00 to minimize slew rate problems. Under visual spatial integration these lines should approximate the 0.50 intensity. Programming has users indicate when illuminated areas appear equal, providing different trial root values in the process.

The GAMMA.C program allows users to increase and decrease trial root values for each electron gun in a color monitor by depressing the CURSOR_UP and CURSOR_DOWN keys. When the intensities appear "correct" when viewed from a distance to help blend screen texture differences, users signal this by depressing the ENTER key. Redirecting the program output to a file named gamma.val saves the gamma values for other program use in this book.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Quantization

Suppose you are going to hike in the mountains. To prevent getting lost, you might build a three-dimensional scale clay model of the area you plan to hike in and take the model with you. However, this is likely impractical. In addition to extra weight, such models consume backpack space and do not fold well. So, you compromise and bring a contour map.

Figure 11-1 Sample of a contour map

Contour maps are flat and lightweight, providing a handy terrain representation you must interpret to be useful. Figure 11-1 is a portion of a United States Geological Survey (USGS) map for San Jose, California's famous Mount Umunhum area. As the map indicates, Mount Umunhum is a 3,486-foot peak, nestled in an exquisite public-land hiking area replete with hidden, babbling mountain springs. To determine this scenic hiking area's terrain requires simple map interpretation.

Suppose you had a massive earth-plane tool that shaves earth similar to the way wood planes shave away wood. First, set your earth plane to a 3,500-foot elevation and begin shaving Mount Umunhum by making a horizontal pass from above. Since Mount Umunhum's summit is only 3,486 feet, you do not impact the earth surface, though you do contact a few buildings the map designates with dark rectangles. Lowering your earth plane 40 feet to a 3,460-foot elevation and making another pass shaves off a bit of earth. The peak is now gone and a flat area remains.

The map shows the flat area's outline with a contour line that surrounds the former peak's position—representing where the mountain descends from the new flat area. Lowering your earth plane 40 feet to a 3,420-foot elevation and making another pass shaves off more earth. The flattened area expands as the map indicates with another contour line that surrounds the first. Continuing this operation produces enclosing contour lines until you eventually reach sea level. Now let's interpret the map.

To assist interpretation, the map provides contour line elevation labels every 400 feet starting at sea level. Labeled contour lines are thicker than others. Some thick lines are not labeled and represent 200-foot elevation changes from thick labeled lines. The remaining contour lines represent 40-foot elevation changes. So, if we descended Mount Umunhum's thrilling Loma Almaden Road by bicycle, would our trip consist of wheeling over flat areas interspersed with precipitous 40-foot plunges naive map interpretation might suggest? Hardly! The map is merely an abstraction that quantizes reality, thereby introducing side effects requiring interpretative compensation.

Specifically, this map creates 40-foot elevation bins and assigns all mountain surface points to specific bins by dividing their elevations by 40 (using integer arithmetic) and then multiplying the quotient by 40. Dividing both 2,610 and 2,620 by 40 produces 65 which when multiplied by 40 produces 2,600. Consequently, the 2,600-foot bin owns all mountain points with elevations from 2,600 feet up to, but not including 2,640 feet.

This scheme reduces a three-dimensional object, the mountain surface elevations, to a two-dimensional contour map abstraction. It does this by converting an infinite number of continuously varying measurements (zero to 3,486-foot elevations) into 88 discrete values (zero feet, 40 feet, 80 feet, 120 feet,..., 3,480 feet). Like the bathroom scale, this quantization simplifies reality by discarding information detail. And, once discarded, no process can recover this detail from the quantized abstraction, though estimation (mathematical interpolation) can provide guesses.

Finally, understanding quantization is important in image processing because digitized images represent quantized abstractions of continuous phenomena. Specifically, image digitization quantizes continuously varying light intensity levels into a few discrete values (16, 32, 256, etc.). In color imaging, it is sometimes necessary to represent color scenes with an infinite number of colors using a very few colors. In both cases, quantization can introduce abrupt transitions the human visual system amplifies. This phenomena's name is, you guessed it, contouring—where quantized data exerts its visually detectable rebellion.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Digitization

Image capture devices create images by first superimposing an invisible grid over (usually) continuously varying real-world scenes. This segments scenes into small rectangles that are represented by a single image pixel. Next the devices compute an average value (color intensity) that best represents the contents within each grid rectangle. The devices then quantize the values to give a final representation pixel value.

When quantizing a continuous tone image into a digital image, each pixel sample is assigned an integer value which denotes the pixel's intensity value. Within an image, each pixel value ranges from zero to a maximum value which usually has a value of 2n–1 such as 1 (21–1), 3 (22–1), 7 (23–1), 15 (24–1), 31 (25–1), 64 (26–1), 127 (27–1), 255 (28–1),..., 4095 (212–1), etc. This allows a uniform number of bits, sometimes called the bit depth, to represent all assigned pixel values within an image (1, 2, 3, 4, 5, 6, 7, 8, and 12, respectively).

As with quantization, digitization can introduce artifacts. When we compute the average value within a sample rectangle, details will disappear (average out). For example, in the previous map example, a house might contribute to the average altitude of a topographic map, but we are not sampling at a fine enough grid to know there is a house there.

Another digitization artifact occurs when you don't sample fast enough—the sampling rectangles are not closely spaced and do not average (blur) the image enough. For example, taking every other rectangle in a map may introduce sudden altitude changes that are not real. In the image domain, sampling so infrequently can introduce sudden changes in intensity that appear as edges that are not real or beating patterns (regular lines) in the image. These artifacts are due to aliasing—a phenomenon that also gives straight lines the "jaggies."

Another aspect of forming a digital image is digitizing in time, often called "framing." A digital image is a "snapshot" in time, and any movement during the creation of the image will cause blurring and other artifacts. Movies flicker still images together to give the perception of movement (as mentioned in the human vision chapters). As with digitization, sampling too slowly introduces artifacts (temporal aliasing), such as wheels appearing to spin backwards.

Probability

Suppose you analyze a digital image's intensities by determining their various values and counting how often each occurs. This allows you to graph the various pixel values versus how often they occur. The chart is called a histogram and allows users to predict how macro image adjustments affect the resulting image. However, histograms have limited utility since they do not indicate intensity positions, only their relative numbers. So, a checkerboard has the same histogram as any pattern that leaves half an area white and half black. The VIDEO.C program produces histograms for the original and all derived images.

Wave Frequency

Figure 11-2 Wave pattern

Bruce Brown's famous movie Endless Summer affectionately refers to one of Southern California's most notorious and dangerous body surfing beaches as "the dirty old Wedge." Located at the southern end of Newport Beach's Balboa Peninsula, the Wedge faces due south and catches summer storm swells that travel unimpeded from Antarctica. On a good Wedge day, arriving swells careen off of the Newport Harbor jetty and join with the following wave, forming ominous wedge-shaped liquid pockets that advance straight to shore. The huge lumbering aquatic slabs plummet into shallow water, causing explosive ground convulsions spectators revel. Combined with the violent backwash, it's not unheard of for body surfers to drop 40 feet onto slightly wet sand. As is true in digital imaging, big waves transmit tremendous energy.

Digital imaging refers to wave energy as well. Figure 11-2 illustrates a single wave pattern which is actually a combination of two wave patterns: a relatively large lumbering wave pattern and a smaller choppy wave pattern that rides on top of the larger wave. As at the Wedge, the huge wave pattern has a low spatial frequency and transmits most of the image's energy (information). Conversely, the nuisance choppy wave has a high spatial frequency and provides detail which image processing may remove. In some instances, such as natural scenes, there is no significant visual impairment by removing detail. Such information is referred to as irrelevant information, and procedures that remove it are referred to as visually lossless. However, removing high spatial frequency from scanned text document images usually causes havoc and should be avoided.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Running Computations

This is the chapter's final topic and sets the stage for the following chapter. It provides an introduction to the following imaging concepts:

  • Smoothing
  • Sharpening
  • Unsharp masking

These three concepts share a common concept—running calculations.

Running Calculation Overview

Consider the following number sequence:

0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 8
2, 2, 4, 4, 6, 6, 8, 8, 8, 8
6, 4, 2, 2, 2
8, 8, 8, 2, 2, 2
8, 8, 8, 4, 4, 4
8, 8, 8, 6, 6, 6
8, 6, 4, 2, 0
8, 8, 8, 7, 6, 5, 4, 3, 2, 1, 0

Rather than look at numbers, it's easier to look at a chart that shows the number pattern (Figure 11-3).

 14 |------------------------------------------------------------ | 14
 13 |------------------------------------------------------------ | 13
 12 |------------------------------------------------------------ | 12
 11 |------------------------------------------------------------ | 11
 10 |------------------------------------------------------------ | 10
  9 |------------------------------------------------------------ | 9
  8 |--------***------****-----***---***---***---*----***-------- | 8
  7 |-------****------****-----***---***---***---*----****------- | 7
  6 |------*****----*******----***---***---********---*****------ | 6
  5 |-----******----*******----***---***---********---******----- | 5
  4 |----*******--**********---***---***************--*******---- | 4
  3 |---********--**********---***---***************--********--- | 3
  2 |--**********************************************-*********-- | 2
  1 |-***********************************************-**********- | 1
  0
+ ------------------------------------------------------------  +
0
Input:012345678882244668888642228882228884448886668642088876543210
============================================================

Figure 11-3: Sample number sequence

Now, the number sequence has a shape. From left to right it:

  • Ascends from zero to eight incrementing by one at each step
  • Stays at eight for a while, then plunges to two
  • Ratchets back to eight by incrementing by two every second number
  • Plummets back to two, decrementing by two each step
  • Lunges back to eight and remains there for a while
  • Dives to four and remains there for a while
  • Lunges back to eight and remains there for a while
  • Descends to six and remains there for a while
  • Jumps to eight and remains there until gracefully decrementing to zero at each step

Smoothing

Smoothing removes high frequency variations using averaging techniques. Remember the Wedge's small choppy wave? To remove it, leaving the large lumbering wave essentially intact, use smoothing. Let's see how it works with our number sequence.

First, image processing operations typically receive digital image input number sequences which processes use to derive output sequences. Processing usually leaves the original data intact as it produces derived sequences—important because subsequent calculations often reference the original data as we'll soon see.

Second, if smoothing uses averaging, one salient question is, What kind of averaging? In our first example, we generate an averaging curve by adding three adjacent sequence values and dividing the sum by three using integer arithmetic, generating an arithmetic mean (average). This mean value is output at the middle number's location in the output curve.

For example, the first three numbers, 0, 1, and 2, add up to 3. Dividing by 3 produces 1. Therefore the middle input sequence number with a value of 1 has an output point value of 1. In the case of the sequence numbers 8, 4, and 4, their sum is 16 and dividing by 3 using integer arithmetic produces 5. So, the middle input number with a value of 4 has an output value of 5. If we replaced the 4 with a 5 in the input sequence, the next calculation would incorrectly use the 5 instead of the correct value—4.

This approach works for interior sequence numbers but fails for the starting and ending values, which both have a zero value. In this instance, we use the end points twice in the sum before dividing the sum by 3. This approach is purely a matter of convenience—other strategies work.

Now consider the output curve using this averaging scheme (Figure 11-4 on the following page):

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |------------------------------------------------------------ | 11
  10 |------------------------------------------------------------ | 10
   9 |------------------------------------------------------------ | 9
   8 |---------*--------**-------*-----*-----*----------*--------- | 8
   7 |-------***-------****------*-----*-----**---------***------- | 7
   6 |------*****-----******----***---***---********----****------ | 6
   5 |-----******----*******----***---****-*********---******----- | 5
   4 |----********--*********--*****-****************--*******---- | 4
   3 |---*********-**********--*****-****************-*********--- | 3
   2 |--********************************************************-- | 2
   1 |-**********************************************************- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========-=-+=-=-=-==-====+-=-+=+-=-+=+-=-===-===+-=-========

Figure 11-4: Smoothing 1/3 [+1,+1,+1] output array

The figure's title indicates we use three input points to calculate an output value. We multiply each input point's value by +1 and then add, dividing the sum by 3 using integer arithmetic. The result is the output value for the center sequence number. The notation 1/3 [+1, +1, +1] is referred to as the process's filter kernel or set of weighting coefficients. A filter kernel nearly always uses an odd number of values symmetrically surrounding a center point.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Figure 11-4 lists each input value and graphs its corresponding output value. The indicator below the input value indicates whether the output value:

  • (=) Equals the input sequence number's value
  • (+) Is larger than the input value
  • (–) Is smaller than the input value

Looking at the output curve, only 22 of the 59 output points differ from their original input value so the output curve resembles the input curve. The differences occur at points preceding input curve ascents and descents. These differences make the fluctuation edges more gentle, helping dampen rapid curve oscillations. In other words, the output curve is smoother than the input curve—hence the name smoothing.

Now let's consider another averaging approach. This time, to generate output values, add three adjacent sequence number values to the middle sequence number's value and divide by 4 using integer arithmetic. Here we use the middle sequence number's value twice so we denote the process filter kernel as:

1/4 [+1, +2, +1]

Figure 11-5 shows the output.

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |------------------------------------------------------------ | 11
  10 |------------------------------------------------------------ | 10
   9 |------------------------------------------------------------ | 9
   8 |---------*--------**-------*-----*-----*----------*--------- | 8
   7 |-------***-------****------*-----**---***---*-----***------- | 7
   6 |------*****-----******----***---***---********---*****------ | 6
   5 |-----******----*******----***---****-*********---******----- | 5
   4 |----*******---*********---***---***************--*******---- | 4
   3 |---*********-**********--*****-****************--********--- | 3
   2 |--********************************************************-- | 2
   1 |-**********************************************************- | 1
   0
+------------------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========-=-+=-=-=-==-====+-=-+=+-=-+=+-=-===-===+-=-========

Figure 11-5: Smoothing 1/4 [+1,+2,+1] output array

Here we obtain a curve similar to the previous one but not as smooth. Using the middle sequence number twice and dividing by 4 allowed the middle sequence number's to influence the average as much as the other two numbers combined. This mathematical weighting, therefore, favors the middle sequence number, allowing it to dominate the output value.

Finally, we consider the following filter kernel:

1/2 [+1,0,+1]

Here, the middle sequence number value is not used and the values on both its sides are averaged. Figure 11-6 on the following page depicts the result.

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |------------------------------------------------------------ | 11
  10 |------------------------------------------------------------ | 10
   9 |------------------------------------------------------------ | 9
   8 |---------*--------**-------*-----*-----*----------*--------- | 8
   7 |-------***------*****------*-----*-----***-*------***------- | 7
   6 |------****------******-----*-----***-*********----****------ | 6
   5 |-----*******--********---*****-*****-*********--*-*****----- | 5
   4 |----********--*********--*****-****************-********---- | 4
   3 |---*********************-*****-****************-*********--- | 3
   2 |--********************************************************-- | 2
   1 |-**********************************************************- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========-=-++-+-+-==-==+=+-=-+=+-=-+=+-=-+=+-===+-=-========

Figure 11-6: Smoothing 1/2 [+1,0,+1] output array

Here we get yet another different curve that does resemble the first but differs from the others. So, it is clear that using different algorithms can easily produce different results even though they are all equally valid.

Before moving on, it is important to note one important concept—locality. When calculating an output sequence number, the computation only uses sequence numbers near the current input sequence number. The previous examples only considered adjacent sequence numbers. But filter kernels can include points farther away, introducing more averaging. This de-emphasizes local output sequence number variations around any input sample point.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Sharpening

Smoothing reduces output number sequence variations. In direct contrast, the sharpening process emphasizes them. Sharpening calculates how sequence number values vary around the current input sequence number and combines the variation with the current input sequence number's value to create an output sequence number value. Consider the following filter kernel:

[+1,+1,–1]

At each current input sequence number, the input sequence number's value on the right subtracts from the one on the left and the difference adds to the current input sequence number value. Now consider the following filter kernel:

[–1,+1,+1]

At each current input sequence number, the input sequence number's value on the left subtracts from the one on the right and the difference adds to the current input sequence number value.

With both filter kernels, the three filter kernel multiplier (coefficient) values add to +1. Now consider the following filter kernel:

1/2 [–1,+2,+1]

At each current input sequence number, the input sequence number's value on the right subtracts from the one on the left and the difference adds to twice the current input sequence number value. Here, adding the coefficients produces a sum of 2, so we divide by 2 to obtain a correct weighting that leaves constant input sequence number runs such as... 8, 8, 8, 8, 8, 8, 8, 8, 8,... unchanged other than at the beginning and end points. Thus, sharpening emphasizes local input differences but leaves constant input value runs unchanged. Let's see how the 1/2 [–1,+2,+1] sharpening filter kernel modifies the input sequence numbers (Figure 11-7):

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |-------------------------------------------------*---------- | 12
  11 |--------------------------*-----*----------------*---------- | 11
  10 |--------------------------------------*----------*---------- | 10
   9 |-----------------*--------*-----*-----*----------*---------- | 9
   8 |-------***-------***------**----**----**----*----***-------- | 8
   7 |------****-----******-----**----**----***--**----***-------- | 7
   6 |-----*****-----******-----**----***--****-***----****------- | 6
   5 |----*******--********----****--****--********----*****------ | 5
   4 |---********--*********---****--****-**********---******----- | 4
   3 |--*********-**********---****--****-**********--********---- | 3
   2 |-**********-***********-*****-*****************-*********--- | 2
   1 |-**********-*****************-*****************-**********-- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

=+++++++==--++++++==----=++=--=++=--=++=--=+=---++==-------=

Figure 11-7: Sharpening 1/2 [–1,+2,+1] output array

For the output sequence, this filter kernel increases input sequence number values where the sequence is rising but decreases them where the sequence number is falling. Larger input sequence number variations produce larger output sequence number variations. Now consider the 1/2 [+1,+2,–1] filter kernel (Figure 11-8):

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |----------*-----------------*------------------------------- | 11
  10 |----------*-----------------*-----*------------------------- | 10
   9 |----------*---------*-------*-----*-----*------------------- | 9
   8 |--------***-------****-----**----**----**---**----***------- | 8
   7 |--------***------*****-----**----**----***--**----****------ | 7
   6 |-------****------******----**----***--*****-***---*****----- | 6
   5 |------******---********---****--****--*********---******---- | 5
   4 |-----*******---********---****--*****-**********-********--- | 4
   3 |----********-***********--****--*****-**********-*********-- | 3
   2 |---*********-************-*****-****************-**********- | 2
   1 |--***********************-*****-****************-**********- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

=-------==++------==++++=--=++=--=++=--=++=-=+++--==+++++++=

Figure 11-8: Sharpening 1/2 [+1,+2,–1] output array

For the output sequence, this filter kernel decreases input sequence number values where the sequence is rising but increases them where the sequence number is falling. Larger input sequence number variations produce larger output sequence number variations. It follows that sharpening operations have directionality—individual filter kernels are sensitive to whether input sequences are rising or falling. Now consider the [+1, +1, –1] filter kernel (Figure 11-9):

  14 |----------*-----------------*------------------------------- | 14
  13 |----------*-----------------*------------------------------- | 13
  12 |----------*-----------------*-----*------------------------- | 12
  11 |----------*-----------------*-----*------------------------- | 11
  10 |----------*---------**------*-----*-----*----*-------------- | 10
   9 |----------*---------**------*-----*-----*----*-----**------- | 9
   8 |---------***------*****----***---***---***--***---****------ | 8
   7 |--------****------*****----***---***---***--***---*****----- | 7
   6 |--------****-----******----***---***---****-****--******---- | 6
   5 |-------*****-----******----***---***---****-****--*******--- | 5
   4 |------******---*********---***---****-**********--********-- | 4
   3 |-----*******---*********---***---****-**********--*********- | 3
   2 |----********-************-*****-*****-**********--*********- | 2
   1 |---*********-************-*****-*****-**********--********** | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

---------=++------==++++=--=++=--=++=--=++=-=+++--=+++++++++

Figure 11-9: Sharpening [+1,+1,–1] output array


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

For the output sequence, this filter kernel decreases input sequence number values where the sequence is rising but increases them where the sequence number is falling. However, output sequence variations are significantly more pronounced than with the 1/2 [–1,+2,+1] filter kernel which is equivalent to [–1/2,+1,+1/2]. The equivalent filter kernel only adds half the local input variations rather than all of it and computationally suffers by performing two intermediate value divisions rather than one single one at the end.

Finally, consider the [–1,+1,+1] filter kernel (Figure 11-10):

  14 |--------------------------*-----*----------------*---------- | 14
  13 |--------------------------*-----*----------------*---------- | 13
  12 |--------------------------*-----*-----*----------*---------- | 12
  11 |--------------------------*-----*-----*----------*---------- | 11
  10 |-----------------*--------*-----*-----*----------*---------- | 10
   9 |-------**--------*--------*-----*-----*----------*---------- | 9
   8 |------****-----*****-----***---***---***---**----**--------- | 8
   7 |-----*****-----*****-----***---***---***---**----***-------- | 7
   6 |----******---********----***---***---****-***---****-------- | 6
   5 |---*******---********----***---***---****-***---*****------- | 5
   4 |--********--*********----***---****-*********---******------ | 4
   3 |-*********--*********----***---****-*********---*******----- | 3
   2 |-**********-**********--*****-*****-**********--********---- | 2
   1 |***********-**********--*****-*****-**********--*********--- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

+++++++++=--++++++==----=++=--=++=--=++=--=+=---++=---------

Figure 11-10: Sharpening [–1,+1,+1] output array

For the output sequence, this filter kernel increases input sequence number values where the sequence is rising but decreases them where the sequence number is falling.

Unsharp Masking

Unsharp masking is another process that accentuates differences usually using a two-step process. The first step creates an output sequence using a smoothing filter kernel. The second step creates the final output sequence by:

  1. Subtracting the smoothing sequence numbers from their corresponding input sequence numbers.
  2. Scaling the differences (multiplying or dividing the differences by some constant value).
  3. Adding the scaled differences to the corresponding input sequence numbers.

Unsharp masking subtracts out a fraction of the low spatial frequencies to increase the contribution of the high spatial frequencies. Also, subtracting the input mask [0, 2, 0] from the smoothing mask [1,1,1] you get a "high-pass" filter [–1,1,–1] which accentuates changes in both slopes. This is the Laplacian, or second finite difference, filter. You need to make it [–1,3,–1] to scale correctly and sharpen rather than "edge detect." This shows that unsharp masking is just a variant of the sharpening filter.

People use unsharp masking rather than a sharpening filter because sometimes blurring is physically easier than sharpening (e.g., with photography) or because of computational advantages (reduced chance of overflow). Otherwise, an unoriented sharpening filter is preferred.

As with sharpening, the final output sequence leaves constant-value sequence number runs untouched other than the beginning and end values. However, the accentuation affects both rises and falls, occurring where input sequence numbers begin and stop rising or falling. For example, rather than immediately beginning to rise, the output curve first drops before the rise. Similarly, rather than immediately stopping a rise, the output sequence briefly continues rising before dropping back.

Unsharp masking, therefore, has an initial undershoot and subsequent overshoot effect for input sequence rises. Similarly, unsharp masking has an initial overshoot and subsequent undershoot effect for input sequence falls. The effect is similar to automobile drivers who veer in one direction before turning in the opposite direction. Numerically, the effect exaggerates numeric rises and falls with an anticipatory counter movement. Figure 11-11 illustrates unsharp masking using the 1/3 [+1,+1,+1] filter kernel as the first step's smoothing mask. The second step scales the difference using a coefficient of 1 (leaves the difference unchanged).

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |------------------------------------------------------------ | 11
  10 |----------*---------------*-*---*-*---*-----*----*---------- | 10
   9 |--------*-*------*--*-----*-*---*-*---*-*---*----*-*-------- | 9
   8 |--------***------****-----***---***---***---*----***-------- | 8
   7 |-------****----*-****-----***---***---***---*----****------- | 7
   6 |------*****----*******----***---***---********---*****------ | 6
   5 |-----******--*-*******----***---***---********---******----- | 5
   4 |----*******--**********---***---***-*-*********--*******---- | 4
   3 |---********--**********---***---***************--********--- | 3
   2 |--*********-*************-***-*-****************-*********-- | 2
   1 |-**********-*************-***-*-****************-**********- | 1
   0
+------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========+=+-=+=+=+==+====-+=+-=-+=+-=-+=+===+===-+=+========

Figure 11-11: Unsharp masking 1/3 [+1,+1,+1] output array


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Figure 11-12 illustrates unsharp masking using the 1/4 [+1,+2,+1] filter kernel as the first step's smoothing mask.

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 | ----------------------------------------------------------- | 11
  10 |----------*---------------*-*---*----------------*---------- | 10
   9 |--------*-*------*--*-----*-*---*-*---*-*---*----*-*-------- | 9
   8 |--------***------****-----***---***---***---*----***-------- | 8
   7 |-------****----*-****-----***---***---***---*----****------- | 7
   6 |------*****----*******----***---***---********---*****------ | 6
   5 |-----******--*-*******----***---***---********---******----- | 5
   4 |----*******--**********---***---***-*-*********--*******---- | 4
   3 |---********--**********---***---***************--********--- | 3
   2 |--*********-*************-***-*-****************-*********-- | 2
   1 |-***********************************************-**********- | 1
   0
+-------------------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========+=+-=+=+=+==+====-+=+-=-+=+-=-+=+===+===-+=+========

Figure 11-12: Unsharp masking 1/4 [+1,+2,+1] output array

Finally, since each unsharp masking step references the input sequence, it is possible to perform the complete process as a single step. The logic follows:

FinalValue == InputValue + ScalingValue (InputValue – SmoothingValue) | by definition
FinalValue == InputValue + 1 * (InputValue – SmoothingValue) | substituting
FinalValue == 2 * InputValue – SmoothingValue | combining

Figure 11-13 illustrates the single step process's identical result using the 1/4 [+1,+2,+1] filter kernel as the first step's smoothing mask.

  14 |------------------------------------------------------------ | 14
  13 |------------------------------------------------------------ | 13
  12 |------------------------------------------------------------ | 12
  11 |------------------------------------------------------------ | 11
  10 |----------*---------------*-*---*----------------*---------- | 10
   9 |--------*-*------*--*-----*-*---*-*---*-*---*----*-*-------- | 9
   8 |--------***------****-----***---***---***---*----***-------- | 8
   7 |-------****----*-****-----***---***---***---*----****------- | 7
   6 |------*****----*******----***---***---********---*****------ | 6
   5 |-----******--*-*******----***---***---********---******----- | 5
   4 |----*******--**********---***---***-*-*********--*******---- | 4
   3 |---********--**********---***---***************--********--- | 3
   2 |--*********-*************-***-*-****************-*********-- | 2
   1 |-***********************************************-**********- | 1
   0
+-------------------------------------------------------------------------+
0
Input:012345678882244668888642228882228884448886668642088876543210

========+=+-=+=+=+==+====-+=+-=-+=+-=-+=+===+===-+=+========

Figure 11-13: Direct unsharp masking 1/4 [+1,+2,+1] output array

Summary

This chapter covered smoothing, sharpening, and unsharp masking. These three processes are fundamental digital image processing techniques that de-emphasize or accentuate numeric differences. As the next chapter shows, digital images are simply two-dimensional number collections. Digital image processing applies smoothing, sharpening, and unsharp masking techniques to these number collections using two-dimensional masks which, in degenerate cases, are one-dimensional masks similar to the ones this chapter uses.

With two-dimensional images:

  • Smoothing blurs visual detail
  • Sharpening accentuates visual detail selectively in one direction
  • Unsharp masking accentuates visual detail by indulging the human visual system's natural retinal sharpening process by providing transition exaggerations in all directions.

The MATH.C program in Appendix C provides a means to further study these operations on values that range from zero to 14.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 12
Monochromatic Concepts

What This Chapter is About

Image enhancement alters images to make them subjectively more pleasing or improve their utility by making previously invisible (but mathematically present) content in the physical scene visible. Simply put, image enhancement makes images more visually digestible while retaining original image truth. In contrast, image editing and manipulation involves image alterations such as compositing which fundamentally change original image truths.

This chapter presents image enhancement techniques applicable to achromatic images such as gray scale images created by scanning continuous-tone black and white photographs. As described in Section 1, equi-energy white light is achromatic light that contains equal proportions of all light wavelengths. By varying equi-energy white light intensity from black through intervening grays to a maximum value deemed fully white, all present wavelengths remain in the same proportion and vary equally. Since any light source, whether it contains only one light frequency or multiple frequencies, can also preserve light component proportions as the source intensity varies, these techniques also work for images created with light sources other than true white light.

The VIDEO.C sample program shows implementations and provides output illustrating the effect for most of the discussed techniques.

Achromatic Image Representation

Single Rectangular Integer Array

An achromatic image is typically a rectangular integer array whose element values represent sampled and quantized intensity samples called pixels (picture elements). When the quantized sample values are only zero and one (bilevel images), the individual elements are sometimes referred to as pels. Within image files, image data is usually, but not always, stored as complete top-to-bottom image scan lines. Within a scan line, the data is usually stored from left to right. This gives rise to a Cartesian coordinate system with the Y axis inverted where the first (top) scan line is row (scan line) zero and the first positions (column) within all scan lines is column zero. So, the upper left pixel has (row, column) coordinate values (0, 0) instead of the bottom left pixel.

It is convenient to think of achromatic images as three-dimensional objects. The various pixel coordinate positions occupy two dimensions (horizontal row and vertical column positions) while the pixel value found at any (row, column) position denotes a distance into a third dimension. A useful metaphor considers astronauts gazing down from space as the sun sets on the Himalayan Mountains. They sense no vertical surface elevation differences but know white snow marks higher surface elevations while darker areas represent lower elevations—just as white light signifies higher intensities in black and white movies. Analogously, color images can be rendered using three primary colors. Each of the three primary contributions can be regarded as a three-dimensional object that is similarly viewed from above.

Bits Per Pixel

As previously discussed, long wavelength waves with high peaks and troughs transmit the most image energy and comprise an image's foundation. The higher-order bit positions correspond to such waves because they usually tend to change more slowly across an image and have higher peaks and troughs than the low-order bit positions. Lower-order bit positions ride on and refine the higher-order bit positions, adding increasing image accuracy (image detail)—refinements analogous to ocean-wave chop. Literature sometimes refers to these individual bit positions as bit planes.

Viewed differently, instead of cutting an image up into individual pixels —much like slicing a cake—we can consider digital images to be made up of layers of bits—much like the layers of a cake, each layer being a bit plane. An image with 8-bit pixel values (0 to 255 intensity values) has eight bit planes. The bit plane view of an image is mainly used when adding graphics or special effects to an image. However, most continuous tone images are best regarded as pixel arrays rather than bit planes.

The VIDEO.C sample program illustrates the contribution each bit position individually has on a 6-bit (64 intensity level) achromatic image. Beginning with the high-order bit position, the program first visually displays which bits have a zero or one value at each pixel location. One values have a white visual representation and zero values have a dark visual representation. For the high-order bit, this amounts to a threshold process where pixel values greater than 31 receive an "on" value and pixel values less than 32 have an "off" value.

After a brief pause, the program progresses to the next lower bit position and performs the same visual representation (which is now not a digital halftone threshold operation because the bit position is not the high-order bit position). The process continues with the output becoming increasingly unintelligible because successive bit positions constitute refinements to the collective contributions of higher-order bit positions which are not visually provided. Hence, successive visual output increasingly assumes a random numeric-chatter appearance.

After examining all possible bit positions in isolation, VIDEO.C shows the cumulative effects of bit positions when they progressively accumulate, starting with the high-order bit position and progressing to the low-order. This effectively renders the image as a 1-, 2-, 3-, 4-, and 5-bit image. Note the extensive contouring that gradually disappears as successive bit positions refine the image.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Histograms

An image histogram is a two-dimensional, graphical bar-chart representation of the three-dimensional achromatic image. As you might suspect, something is lost along the way, namely pixel position information.

Histograms present a simplistic accounting that indicates how many pixels assume various pixel values. If an image is an 8-bit image, then there are 256 possible values—zero through 28–1, or 255. The horizontal axis usually specifies the valid pixel values and the vertical axis usually indicates how many pixels assume the values. Because thousands of pixels may assume any allowed value in large, high-resolution images and the actual pixel value distribution is less important than the relative distribution, the vertical axis scale is usually compressed to give a bird's eye view of the distribution that fits on a screen or page.

The VIDEO.C program depends on standard VGA resolution that only supports 64 colors which are set as 64 monochromatic intensities. Thus, VIDEO.C generates histograms for its 6-bit achromatic images by first determining how many pixels assume each of the 26–1 (64) values and coincidentally sets the maximum vertical height for each graphical value to 64. It then determines how many pixels assume each of the possible 64 pixel values. The program calculates its scaling factor by identifying the pixel value that has the most occurrences within the image and dividing this pixel value's occurrence count into the maximum allowable height which is arbitrarily selected as 64. The program then multiplies the occurrence count for each possible pixel value by this calculated scaling factor to determine the graphical output bar height for each possible pixel value.

The use of histograms is limited because their representations forfeit information regarding pixel locations. For example, images of black and white checkerboards would have the same histogram as images consisting of black lines on white backgrounds. The lines could be vertical, horizontal, slanted, thick, or thin. Yet the images are all decidedly different.

Histograms are useful because they show when an image contains poor dynamic range that looks foggy because it exhibits low contrast that provides little visual detail. Remembering that dynamic range is the difference between the highest pixel value and the lowest pixel value, histograms for images having poor dynamic range consist of a few close-proximity vertical bars. Images with improved contrast better exploit the available pixel value range which monitor slew rates and printer mechanics, ink, or paper may constrain from the maximum numerically allowable.

In all cases, one input point generates one output point, but the output image remains faithful to the input image though its appearance is enhanced to some end or subjective purpose. Image enhancement techniques therefore attempt to convert images exhibiting poor dynamic range into images exhibiting expanded dynamic range by "spreading out" pixel values using various techniques which fall under the category of point operations.

Point Operations

Point operations are operations that determine an image pixel output value using some strategy that only examines the corresponding input pixel value, regardless of position or surrounding values. This simplistic approach often allows algorithms to calculate look-up tables in advance, accelerating output generation. Graphically representing the look-up table reveals the specific strategy, or mapping function, which should not be confused with an image histogram.

Figure 12-1 Identity mapping function

Figure 12-2 Threshold strategy

Figure 12-3 Complement mapping function

For example, suppose we trivially wish to generate a 6-bit output image that is identical to a 6-bit input image. Here, a pixel position output value would be identical to the pixel position input value. Figure 12-1 presents the mapping function's graph which is the identity mapping function. Figure 12-2 depicts a threshold strategy where every value below 32 is set to zero and every value above 31 is set to 63. Figure 12-3 presents a complement mapping function which generates a negative image. Here the mapping formula is:

(output pixel value) = (maximum allowable pixel value) – (input pixel value)

Finally, remembering the preceding gamma correction discussion, gamma correction is also a point process that depends only on an input value to generate an output value using a selected gamma value.

Histogram Slide

Sometimes it is possible to improve an image by adding a constant value to each pixel value. When the sum is a value that is greater than the maximum allowable value, the output pixel value is set (or clamped) to the maximum allowable value.

(temporary value) = (input pixel value) + (constant value)

(output pixel value) = minimum of (maximum allowed value, temporary value)

Similarly, it is sometimes possible to improve an image by subtracting a constant value from each pixel value. When the difference is a value that is less than the minimum allowable value, the output pixel value is clamped to the minimum allowable value.

(temporary value) = (input pixel value) – (constant value)

(output pixel value) = maximum of (minimum allowed value, temporary value)

Because these mapping functions alter input values by a constant, the output image histogram is essentially the input image histogram moved to the right or left depending on whether the mapping function is subtracting or adding the constant respectively—hence the name histogram slide. This is similar to adjusting a monitor's brightness control setting to a higher or lower value respectively. Figure 12-4 illustrates the mapping function that adds a constant value of 4 to each pixel value. This shifts the input image histogram to the right.

Figure 12-4 Mapping function that adds a constant value of 4 to each pixel value

Histogram slides are sometimes useful to move pixel value ranges into the region where the eye is most sensitive to discerning detail. They are also useful to position a histogram for a subsequent histogram stretch.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Histogram Stretch Contrast Adjustment

Histogram stretches scale output pixel values by some constant.

(output pixel value) = (constant value) * (input pixel value)

Figure 12-5 Mapping function that slopes up at 45º

Figure 12-6 Mapping function with a constant greater than 1

Figure 12-7 Mapping function with a constant less than 1

If the constant has a value of one, the mapping function is the identity mapping function represented in Figure 12-5 which consists of a line that slopes up at 45º. If the constant is greater than one, then mapping function "spreads out" the histogram and has a graph consisting of a line that slopes up at an angle greater than 45º as illustrated in Figure 12-6. If the constant is a value less than one, then the mapping function is a histogram shrink and has a graph that is a line that slopes up at an angle less than 45º as illustrated in Figure 12-7.

Histogram Equalization Contrast Adjustment

Histogram equalization is another point process that attempts to improve contrast by "spreading out" an input histogram. It attempts to distribute output pixels so that equal-sized pixel value ranges contain the same number of output pixels. It does this by first determining the number of allowable output values based on some criteria such as image bit depth constrained by monitor slew rates. It then generates a histogram and divides the total number of pixels by the number of allowable output values. The division produces the average number of output pixel values to assign to each output value.

The approach then assigns a zero output value to input pixel values zero through the minimum allowable output value and continues to assign sequentially larger input pixel values a zero value until the total number of pixels assigned a zero value exceeds the calculated average pixel count. The approach then advances one or more output values until the cumulative output average drops below the average number of output pixel values. At that point, the process begins assigning output values again until the calculated average is exceeded again. Because images involve quantized data, a pixel input value range sometimes maps to a single output value. Other times, output values have no corresponding input points.

There are several histogram equalization approaches. One, called adaptive histogram equalization, is sometimes compared to traditional darkroom dodging and burning. It consists of tiling an input image into individual non-overlapping, equal-sized areas and performing histogram equalization independently on each area. However, darkroom dodging and burning is actually a point operation over small areas and not an area operation.

Another approach divides an image into overlapping, equal-sized areas and performs histogram equalization independently on each area. Pixel output values for each pixel are then calculated by weighting the various histogram values assigned to each pixel. The weights are assigned by calculating how far the pixel position is from the center of the respective histogram area. The United States Justice Department used this technique to enhance the Rodney King videotape footage.

Area Operations

Point operation approaches sometimes construct algorithms using global image data such as histograms. Thereafter, output value calculations only consider input pixel values. In contrast, area operations consider individual input pixel values and a small area surrounding the input pixel. One example of an area operation is called a spatial filter.

Spatial Filters

Recalling Chapter 11, running computations used a one-dimensional filter kernel which weighted pixel values surrounding calculation input points. So, Chapter 11 computations involved operations using values to the left and right of input values. Since image pixels have two-dimensional (row, column) positions, this allows masks, referred to as spatial filters, that have two dimensions.

Spatial filters are usually represented as two-dimensional, square matrices that have an odd number of rows and columns [size (2n+1)x(2n+1)]. The individual matrix elements are called taps or weights. The matrix of weights is usually horizontally and vertically symmetrical. The odd number of rows and columns provides a central weight which is spatially coincident with the input pixel. Output value calculations occur at every input pixel position by using the spatial filter elements as coefficients to multiply corresponding points surrounding the input pixel point. It is a simple process but difficult to describe. To understand the calculation process, consider the following hypothetical 3x3 spatial filter kernel and input image portions:

Hypothetical Input Image Portion:

... | ... | ... | ... | ... | ...
... | 201 | 202 | 203 | 204 | ...
... | 205 | 206 | 207 | 208 | ...
... | 209 | 210 | 211 | 212 | ...
... | 213 | 214 | ... | ... | ...
... | ... | ... | ... | ... | ...

Hypothetical Spatial Filter:

2 | 4 | 6
8 | 10 | 12
14 | 16 | 18


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

To understand 3x3 spatial filter application, we now apply the hypothetical filter to the hypothetical image fragment assuming the current computation pixel is the boldface value of 206.

Pixel Value | Spatial Mask Coefficient | Product
201 | 2 | 2 x 201 = 402
202 | 4 | 4 x 202 = 808
203 | 6 | 6 x 203 = 1218
205 | 8 | 8 x 205 = 1640
206 | 10 | 10 x 206 = 2060
207 | 12 | 12 x 207 = 2484
209 | 14 | 14 x 209 = 2926
210 | 16 | 16 x 210 = 3360
211 | 18 | 18 x 211 = 3798
 |  | Sum = 18,696

Applying the hypothetical spatial filter to the hypothetical image fragment produces a value of 18,696. This is a nonsense value that arose from the hypothetical spatial filter having nonsense element values, arbitrarily selected for process clarity.

The following material uses appropriate values that produce selected effects. Before discussing these effects, note that spatial filter application requires special treatment along image edges and corners since pixel positions along the edges and corners are not completely surrounded by image pixel values. The VIDEO.C sample application addresses this by replicating edge information and using it appropriately. For example, if the top left corner image fragment appears as:

201 | 202 | 203 | 204 | ...
205 | 206 | 207 | 208 | ...
209 | 210 | 211 | 212 | ...
213 | 214 | ... | ... | ...
... | ... | ... | ... | ...

Then VIDEO.C processes the image as if it were:

201 | 201 | 202 | 203 | 204 | ...
201 | 201 | 202 | 203 | 204 | ...
205 | 205 | 206 | 207 | 208 | ...
209 | 209 | 210 | 211 | 212 | ...
213 | 213 | 214 | ... | ... | ...
... | ... | ... | ... | ... | ...

This adds two extra rows and two extra columns which is acceptable for a 3x3 spatial filter. A 5x5 spatial filter requires four extra rows and columns, and so on.

Sharpening

Sharpening accentuates intensity transitions within images. Unlike histogram stretching and equalization which modifies contrast by redefining dynamic ranges, sharpening achieves its effect by amplifying local intensity differences. To conceptually understand sharpening, consider a sandbox with sand covering each position within the sandbox. In some places, the sand height is the same depth as all nearby surrounding points. So, the sand surface is flat in such instances. Though sand heights may differ from area to area, within an individual area there is no height difference to accentuate. In other places, points are lower or higher than surrounding points.

To accentuate height differences, sharpening processes create a second sandbox based on the first by initially replicating the first sand surface and then removing sand from low points, placing it on surrounding higher points. This preserves original surface feature positions but makes valleys and dips deeper while simultaneously increasing the height of surrounding higher points such as ridges and peaks. The double removal and adding operations amplify differences between the original sand surface features.

Image sharpening uses spatial filters called high-pass filters which accentuate high frequency changes that appear as abrupt intensity changes. Spatial filter calculations involve subtracting the current calculation point from one or more of its neighbors. Adding the filter coefficients often produces a sum value of 1. A sharpening (or any other) filter's coefficients need not add to 1—this just makes it easier because the output need not be scaled. You can regard sharpening as adding an original image to a spatially differenced image because the process is linear. This means you want a positive center value that is at least 1 above the sum of the (negated) surrounding values.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

This is a bit confusing so here is a high-pass sharpening filter that computes output values by subtracting surrounding pixel values from the current calculation point and adding the eight differences to the current calculation point pixel value:

–1 | –1 | –1
–1 | 9 | –1
–1 | –1 | –1

Here is another example that results in less sharpening—often more visually appealing.

–1/4 | –1/4 | –1/4
–1/4 | 12/4 | –1/4
–1/4 | –1/4 | –1/4

High-pass sharpening filters are characterized by taps that have a positive center tap with some or all of the surrounding taps having negative values. Here are a few other typical sharpening filters:

0 | –1 | 0
0 | 3 | 0
0 | –1 | 0

–1 | 0 | 0
0 | 3 | 0
0 | 0 | –1

0 | 0 | –1
0 | 3 | 0
–1 | 0 | 0

0 | –1 | 0
–1 | 5 | –1
0 | –1 | 0

1 | –2 | 1
–2 | 5 | –2
1 | –2 | 1

1/6 | –2/6 | 1/6
–2/6 | 10/6 | –2/6
1/6 | –2/6 | 1/6

The last two filters are similar but the second of the two is scaled by six since the center tap would otherwise have a value of 10, leaving a tap total sum of 6, rather than the required value of 1. This diminishes the sharpening effect. Finally, when images consist of two interlaced fields, one interlaced field should not be sharpened using data from the other interlaced field. Here is a filter that solves the problem:

0 | 0 | 0
–1 | 3 | –1
0 | 0 | 0

Smoothing

Smoothing reduces intensity transitions within images. Using the sandbox analogy, smoothing processes remove sand from ridges and peaks and places it in neighboring lower-elevation areas. This attenuates differences between the original sand surface features. In images, this visually blurs detail, producing a soft "out-of-focus" effect which grows as the spatial filter increases in row and column count.

Image smoothing uses spatial filters called low-pass filters. Low-pass filtering calculations reduce pixel value differences by averaging local intensities. Like sharpening, adding the filter coefficients always produces a sum value of 1. Here is a low-pass smoothing filter:

1/9 | 1/9 | 1/9
1/9 | 1/9 | 1/9
1/9 | 1/9 | 1/9

1/10 | 1/10 | 1/10
1/10 | 1/5 | 1/10
1/10 | 1/10 | 1/10

1/16 | 1/8 | 1/16
1/8 | 1/4 | 1/8
1/16 | 1/8 | 1/16


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Unsharp Masking

Unsharp masking produces a contrast enhancement by first performing an image smoothing operation. Next it computes the pixel-by-pixel difference between the original image and the smoothed image, creating an image consisting primarily of high frequency components. Finally, it boosts the difference (multiplies by a constant) and adds the result to the original image pixel by pixel, producing the final image. Output varies depending on which image smoothing algorithm is used. The following 5x5 filter produces an unsharp mask image in a single pass:

–4/25 | –4/25 | –4/25 | –4/25 | –4/25
–4/25 | –4/25 | –4/25 | –4/25 | –4/25
–4/25 | –4/25 | 121/25 | –4/25 | –4/25
–4/25 | –4/25 | –4/25 | –4/25 | –4/25
–4/25 | –4/25 | –4/25 | –4/25 | –4/25

Emphasizing Edges and Line Segments

Edge detection filters emphasize edge presence in a desired direction that varies depending on the spatial filter applied. Edge detection discards most image information, leaving only existing edges oriented in the selected direction dramatically highlighted.

Shift and Difference Edge Enhancement

Shift and difference edge enhancements are computationally simple, hence fast. Various 3x3 edge detection filters follow.

To emphasize vertical edges:

0 | –1 | 0
0 | 1 | 0
0 | 0 | 0

To detect horizontal edges:

0 | 0 | 0
–1 | 1 | 0
0 | 0 | 0

To emphasize 135º edges:

–1 | 0 | 0
0 | 1 | 0
0 | 0 | 0

To emphasize 45º edges:

0 | 0 | –1
0 | 1 | 0
0 | 0 | 0

Line Segment Detection

Filters that detect line segments are similar to shift and difference filters. Several examples follow.

To emphasize vertical line segments:

–1 | 2 | –1
–1 | 2 | –1
–1 | 2 | –1

To emphasize horizontal line segments:

–1 | –1 | –1
2 | 2 | 2
–1 | –1 | –1

To emphasize 45º line segments:

–1 | –1 | 2
–1 | 2 | –1
2 | –1 | –1

To emphasize 135º line segments:

2 | –1 | –1
–1 | 2 | –1
–1 | –1 | 2

Edge Detection Using Gradient Filters

Gradient filters provide intensive means to detect image edges with more refinement than shift and difference or line segment filters allow. The taps sum to zero.

North:

1 | 1 | 1
1 | –2 | 1
–1 | –1 | –1

Northeast:

1 | 1 | 1
–1 | –2 | 1
–1 | –1 | 1

East:

–1 | 1 | 1
–1 | –2 | 1
–1 | 1 | 1

Southeast:

–1 | –1 | 1
–1 | –2 | 1
1 | 1 | 1

South:

–1 | –1 | –1
1 | –2 | 1
1 | 1 | 1

Southwest:

1 | –1 | –1
1 | –2 | –1
1 | 1 | 1

West:

1 | 1 | –1
1 | –2 | –1
1 | 1 | –1

Northwest:

1 | 1 | 1
1 | –2 | –1
1 | –1 | –1


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Laplacian Filters

Laplacian edge enhancement emphasizes all edges present regardless of orientation. It is a sharpening process variant. Sharpening therefore adds the Laplacian output to the original image. Laplacian filters follow:

–1 | –1 | –1
–1 | 8 | –1
–1 | –1 | –1

1 | –2 | 1
–2 | 4 | –2
1 | –2 | 1

0 | –1 | 0
–1 | 4 | –1
0 | –1 | 0

Noise Removal

Sometimes image data is corrupted and lost, for example, during transmission from satellites to ground stations. The lost data often appears as visually obvious incorrect pixel intensities. In certain instances, it is possible to approximate corrupted pixel values using valid pixel values for pixels surrounding the corrupted pixel value. The basic problem is how to devise algorithms that automatically detect pixels that have corrupted values.

Usually in a pixel-by-pixel operation, an algorithm uses surrounding pixels to calculate a figure of merit which is compared to the pixel value under scrutiny. If the pixel value exceeds the figure of merit by some arbitrary measure, then the output value is computed using the surrounding input pixel values rather than the actual pixel value under scrutiny.

Variance Calculations

For example, at every input pixel position, one approach averages the eight adjacent pixels surrounding input pixels and calculates the standard deviation. This is the equivalent of a spatial low-pass filter with a central weight equal to zero. If the suspect pixel value is more than one standard deviation from the calculated mean, then the output value for that pixel position is the mean. Otherwise, it is the input pixel value.

Averaging

Sometimes it is possible to capture multiple images of a scene before the scene changes or the camera moves. In such cases, it is possible to average valid pixel values point by point and divide by the number of pixel values summed. The corrupting noise varies randomly at each pixel position but the average tends towards the true pixel values. This is because random pixel intensity decreases cancel out to some degree when added to random intensity increases.

Median Filter

The median filter technique sorts each pixel position value with the various pixel values surrounding the point under scrutiny. It then selects the median (middle) value as the output value. The median filter works because noisy pixels tend to have values that are extremely high or low when compared with the neighboring values. The median (mid-range) of the pixel and neighboring values reject extreme values.

Image Processing

The techniques this chapter discusses cannot be used on individual RGB component planes without introducing noticeable color inaccuracies in the final image. To use the techniques on color images first requires separating the intensity information from the chromaticity information and performing the operation on the derived achromatic image.

After performing the enhancement operation on the achromatic pixel information, on a pixel-by-pixel basis, divide the original achromatic intensity value by the derived output intensity value and use the quotient to scale the individual original RGB components. This preserves color information while modifying the intensity values in accordance with the enhancement strategy. The VESAPCX.C sample program in Chapter 17 sharpens 24-bit images using this linear scaling technique.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 13
Color Models

What This Chapter is About

Color models facilitate color specification in a standard, non-subjective, repeatable manner. This chapter discusses various ways to represent the spectral colors and their various tone, tint, and shade mixtures. Because of human trichromatic vision, these models all involve three variables and involve three-dimensional considerations. Monitors primarily use the RGB model, printers the CMY and CMYK models, and televisions use the YCbCr model. The HSV and HLS models are computationally faster than the HSI model, but none of the three models are perceptually uniform models where uniform distance changes within the model cause uniform visual changes. So their utility is diminished as compared to uniform perception models whose mathematics exceed this book's scope.

The VESAPCX.C program illustrates how to convert back and forth between four of these color spaces (RGB, HSV, HLS, and HSI) and the VESATEST.C program in Chapter 16 displays three of the spaces (RGB, HSV, and HSI).

RGB Model

CRTs typically add emitted red, green, and blue phosphors light, hence this model's name RGB. Suppose you have a 700.0 nm red spectral illumination source (light), 546.1 nm green spectral illumination source, and 435.8 nm blue spectral illumination source, though the discussion applies to other spectral wavelength selections and spectral power distributions that observe a special form of linear mathematics. However, expecting two monitors to have these or any other consistent RGB primary combination is naive.

When a light is completely off, its radiance value is 0.0. By illuminating a white surface that reflects all light equally and adjusting each light's radiance independently, we can produce achromatic light that seems to be "white" and has the maximum luminosity the light sources allow. In this state, we assign a value of 1.0 to each light's radiance value. Earlier material discussed various black body illuminants and how the human perception system is not too picky about what it considers white. Finally, we assume any light's radiance value can vary continuously and between 0.0 and 1.0 independent of the other two. This is the RGB color model's foundation.

The RGB model represents all possible additive primary color combinations that a specific monitor can produce using a cube. But different monitors can have different cubes. Remembering high school geometry, a cube is a three-dimensional solid enclosed by six equal-size squares that are perpendicular or parallel to one another. Using three intersecting cube edges as axes and defining their intersection as an origin allows us to identify each point within the cube using Cartesian coordinates as in Figure 13-1. In the figure, one edge is an axis for the red values, the other two are for green and blue, respectively, and the origin is the point common to the red, green, and blue axes.

Figure 13-1 The RGB model

Since any point within the cube has unique coordinates, it identifies the unique radiance value for each of the three primary lights that produce the specific visual blend corresponding to the point. So you might envision the RGB model as a cube filled with continuously varying primary color mixtures. In practice, color palettes and the programs in this book specify RGB component radiance values as digital values, ranging from zero to 255, rather than continuous values, ranging from 0.0 to 1.0. Therefore, RGB model discussions assume you have normalized the allowed radiance value set by dividing each allowed value by the maximum value. So, the value set { 0, 1,..., 255} becomes {0/255, 1/255,..., 255/255}. Also, varying the maximum settings slightly varies the white definition—analogous to selecting a different illuminant. This corresponds to aligning display controller DAC color ramps (loading with slightly different values) to achieve warmer or cooler whites.

Looking at Figure 13-1 which assigns the first coordinate to red, the second to green, and the third to red for (R, G, B) ordering, note the following coordinate points:

(0.0, 0.0, 0.0) represents black

(1.0, 1.0, 1.0) represents white

(1.0, 0.0, 0.0) represents maximum radiance, fully saturated red

(0.0, 1.0, 0.0) represents maximum radiance, fully saturated green

(0.0, 0.0, 1.0) represents maximum radiance, fully saturated blue

(0.0, 1.0, 1.0) represents maximum radiance, fully saturated cyan (red's complement)

(1.0, 0.0, 1.0) represents maximum radiance, fully saturated magenta (green's complement)

(1.0, 1.0, 0.0) represents maximum radiance, fully saturated yellow (blue's complement)

First, note that the printing, film, and video industries often provide footroom and headroom provisions for the black and white points, respectively. This means that, say, (0.05, 0.05, 0.05) is arbitrarily considered black, while (0.95, 0.95, 0.95) is arbitrarily considered white. This allows color ramp fine-tuning adjustments. Second, looking carefully at Figure 13-1, it follows that:

Any point on the line segment connecting the black and white point represents an achromatic mixture with some unique radiant value.

Any point on a coordinate axis provides a unique radiant value for the corresponding spectral color since, by definition, the other two lights have a 0.0 radiance value (are off).

Any cube surface having two axes for edges (i.e., the RG, RB, and GB sides) contains only saturated colors.

Any point inside the cube or not on a saturated surface is unsaturated to some degree since R, G, and B radiance values are greater than zero, generating some achromatic light which blends in with the remaining saturated color.

Now consider any point not outside the cube (inside or on a cube surface) and draw a straight line through it and the black point (0.0, 0.0, 0.0). From geometry, it follows that all points lying on the line have proportional coordinates. As an example, (0.10, 0.20, 0.30) and (0.20, 0.40, 0.60) have proportional coordinates because they lie on the line connecting (0.10, 0.20, 0.30) and (0.0, 0.0, 0.0). In the RGB model, all points lying on such a line represent analogous proportional primary light radiance selections which produce different radiances for the same relative blend. So, the (0.20, 0.40, 0.60) setting is equivalent to using two primary light groups with each group set at (0.10, 0.20, 0.30). But in practice the principle does not hold for individual RGB monitor electron guns. So radiance accuracy, hence color fidelity, can wobble along as the RGB values remain proportional but collectively change.

Finally consider the point (0.10, 0.20, 0.30). This is equivalent to using two primary light groups with one group set at achromatic (0.10, 0.10, 0.10) and the other set at a fully saturated (0.00, 0.10, 0.20) cyan tint.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

CMY Model

Printers typically use cyan, magenta, and yellow colorants for subtractive coloring operations, hence this model's name CMY. Like the RGB model, the CMY model can be viewed as a three-dimensional cube, using three Cartesian axes. Individual coordinate values range from, and include, 0.0 to 1.0—representing how much colorant to deposit on presumably white surfaces. Since this model is subtractive, the origin (0.0, 0.0, 0.0) corresponds to a white value (i.e., paper with no colorant) and the (1.0, 1.0, 1.0) point corresponds to maximum allowed colorant application (Figure 13-2). As Chapter 8 discussed, this point should appear black, but colorant color shifts often render it something else. The same is true for most achromatic values and is a significant problem because the eye is extremely sensitive to color shifts that corrupt achromatic light, producing unbalanced grays.

Figure 13-2 The CMY model

The RGB model and the CMY model are theoretically related. A CMY model point corresponds to a known RGB model point using the following formulas:

C = 1.0 – R
M = 1.0 – G
Y = 1.0 – B

Conversely, an RGB model point corresponds to a known CMY model point using the following formulas:

R = 1.0 – C
G = 1.0 – M
B = 1.0 – Y

However, following these formulas usually produces a printing disaster. So, Hewlett-Packard thoughtfully provides gray balance documentation in its Hewlett-Packard DeskJet Printer Family Software Developer's Guide. This guidance describes a gray balancing technique that compensates for noticeable unwanted absorption when using Hewlett-Packard DeskJet color ink jet printers. The method consists of first determining how close to an achromatic value a point is by computing the difference between the maximum and minimum CMY value and dividing the difference by the maximum value. If fully saturated, the quotient's value is 1.0 point and the point is left alone. If it is achromatic, the quotient is 0.0 and the cyan value is reduced by one-third. In between, the cyan value is proportionally reduced depending on how close to an achromatic value the point is. The sample program PRINTPCX.C has the following programming:

int GrayBalanceCmyTriad(USGI *CyanPtr, USGI *MagentaPtr, USGI *YellowPtr)
{
    USGI CyanValue, MagentaValue, YellowValue, Range;
    USGI MaxPrimaryValue, MinPrimaryValue, MaxMinDelta;

       CyanValue =     *CyanPtr;
    MagentaValue = *MagentaPtr;
     YellowValue = *YellowPtr;

    MaxPrimaryValue = (CyanValue > MagentaValue)  ?
                    CyanValue : MagentaValue;

    MaxPrimaryValue = (YellowValue > MaxPrimaryValue) ?
                    YellowValue : MaxPrimaryValue;

    MinPrimaryValue = (CyanValue < MagentaValue) ?
                    CyanValue : MagentaValue;

    MinPrimaryValue = (YellowValue < MinPrimaryValue) ?
                    YellowValue: MinPrimaryValue;

    if (MaxPrimaryValue == 0)
        *CyanPtr = 0;
    else
    {
        Range = MaxPrimaryValue - MinPrimaryValue;

        *CyanPtr =                    (int)
            ((long)CyanValue * ((long)Range + (2L * (long) MaxPrimaryValue))
                                           /
                               (long) (MaxPrimaryValue * 3));
    }

    *MagentaPtr = MagentaValue;
     *YellowPtr = YellowValue;

    return 0;
}

While specific corrections necessary to address color shift vary from printer manufacturer to manufacturer, this code provides an example of one type of correction needed for color separations. There are many more beyond the scope of this book.

CMYK Model

There is really no substitute for black ink's ability to plunge image areas into darkness. So, the CMYK model, while related to the CMY model, additionally uses black ink to produce genuine achromatic tints, shades, and tones. In the acronym CMYK, K stands for Key. In this model K equals the minimum CMY model value. So:

Kcmyk= minimum (C, M, Y)
Ccmyk= Ccmy– Kcmyk
Mcmyk= Mcmy– Kcmyk
Ycmyk= Ycmy– Kcmyk

This produces an under color removal operation that replaces a fraction of a point's three C, M, Y coloration with a single black ink separation. However, Hewlett-Packard DeskJet CMY color inks and its black ink have different formulations. One is oil based and the other is water based. So, PRINTPCX.C reduces the ink's natural oil-on-water dispersion effect by assigning a portion of the K value to the black ink, leaving the remaining darkening job to a gray balanced CMY combination. Welcome to the world of color printing and its typical considerations.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Y'CbCr Model

The digital video industry uses the Y'CbCr model since it preserves image luminosities and facilitates chroma subsampling (which reduces transmission bandwidth via a color averaging technique) which is not particularly noticeable in most instances. This conserves transmission capacity by minimizing the information necessary for acceptable reception. As the TIFF Revision 6.0 specification indicates, literature and the multimedia industry often incorrectly refer to Y'CbCr, and the U.S. analog television industry's YIQ system, as YUV—be warned.

The Y'CbCr model uses three components—the luma (Y') and two color components Cb and Cr. This model is related to the RGB model by the following linear relationships (CCIR Recommendation Standard 601-1 version titled Encoding Parameters of Digital Television for Studios):

Y' = 0.299*R' + 0.587*G' + 0.114*B'
Cb = 128 + 112*(0.5*(B' – Y')/(1 – 0.114))
Cr = 128 + 112*(0.5*(R' – Y')/(1 – 0.299))

where R', G', and B' are respective gamma corrected values for source R, G, and B values. The corresponding inverse relationship is this:

R' = Cr*(2 – 2*0.299) + Y'
B' = Cb*(2 – 2*0.114) + Y'
G' = (Y' – 0.114*B' – 0.299*R') / 0.587

Television transmissions devote most capacity to faithful luma preservation. If a receiver is a color television, it also presents the color information which is otherwise ignored. Thus, the luma formula illustrates one good way to reduce color images to black and white images—by replacing each color sample by its luma value. Note that look-up tables can replace the numerous multiplications and divisions.

HSV/HSB Model

In 1978, Alvy Ray Smith of the New York Institute of Technology's Computer Graphics Lab adapted the HSV color perception model to computing. The acronym letters stand for hue, saturation, and value (intensity). Literature sometimes refers to this model as the HSB model where B stands for brightness (intensity). See Figure 13-3.

To understand the HSV model, first consider the RGB cube. If you position yourself to look directly at the white point along the achromatic diagonal with one eye, the three visible cube surfaces appear as a colored hexagon because you have no depth perception with one eye. Because the RGB contains an infinite number of subcubes, each containing the origin (0.0, 0.0, 0.0), similarly viewing these various subcubes produces different, various sized hexagons possessing unique tints, shades, and tones. The hexagons are actually geometric projections of cube shapes onto the plane perpendicular to the achromatic diagonal.

Figure 13-3 The HSV hexacone

Remembering the RGB model proportional coordinate discussion, it is clear that any given point on any one hexagon is uniquely related to a single point on any other hexagon. Any two such related points lie on the same line that passes through the points and the RGB cube origin. Hence, any two such related points only differ in their intensity (same proportional primary light mix but with different intensity values). Therefore, we only need to consider one hexagon to select individual proportional primary mixtures, and a luminosity to identify what specific tint, shade, or tone the point represents.

If you could stack all possible hexagons on top of each other with their centers and edges correctly aligned, the result is a three-dimensional, single hexagon solid, or hexacone, that collectively contains all red, green, and blue combinations within its distorted RGB space. The hexacone point corresponds to the RGB model's (0.0, 0.0, 0.0) black point.

Referencing Figure 13-3, the HSV model involves distances as well as angles. The relative angle identifies the unsaturated hue with red conventionally defined at zero degrees. The relative distance from the white center point towards the hexagon edge, where the saturated hues are located, identifies the saturation value. Zero distance means achromatic with zero saturation and undefined hue. Maximum distance to the hexagon edge gives V a 1.0 value. Note this linear distance varies since it involves a hexagon rather than a circle. Finally, note the maximum value of the RGB values provide the subcube edge size which uniquely determines V's value and that saturated hue complements are located directly opposite (180 degrees away) from a given hue.

The HSV model is useful because it decomposes RGB triad values into processes more closely related to human perception that uncouple intensity from hue and saturation color information. However, its simplistic approach provides differing intensities for a given V value. These luminosities are readily evident when running the VESAPCX.C program which contains the RGB/HSV conversion routines listed at the chapter's end. PRINTPCX.C uses these routines extensively to produce its effects. Finally, note RgbToHsv() leaves hue angles expressed in degrees rather than normalized values.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

HLS/HSL/HIS Model

In 1979, the Graphics Standards Planning Committee's core committee adopted the perceptual HIS model which consists of two cylindrical cones joined on their flat surfaces.

The HLS model, which literature sometimes refers to as HSL and HIS (hue, intensity, and saturation), borrows some ideas from the HSV model but uses a double hexacone versus a double cylindrical cone model. In the acronym, HLS stands for hue, lightness, and saturation. It is essentially an HSV space distortion consisting of a double hexacone space created by pulling the white point up off the V = = 1 surface. Unlike the HSV model, fully saturated hues are located along the cross section's perimeter containing L = 0.5.

The HLS model is useful because, like the HSV model, it decomposes RGB triad values into values more closely related to human perception. However, its simplistic approach similarly provides differing luminosities for given L values. Like HSV, this model is useful in midpoint gamma corrections which separate chromaticity from luminosity, allowing luminosity recalculations with subsequent recombination of chromaticity and luminosity that preserve color content.

Figure 13-4 The HLS model

When reading about these models, it is useful to note there are no standards governing HLS, HSL, HIS, etc. acronym use. So, expect to find confusing if not conflicting literature. Finally, the HlsToRgb2 function implements Ken Fishkin's accelerated HLS to RGB transformation found in Graphics Gems Volume 1.

HSI Model

The HSI model is yet another perceptual model, but its mathematical computations are extremely computationally intensive and beyond this book's scope. Like the HSV model, the RGB cube is deformed into another shape using stacked cross sections determined by projective geometry. Unlike the HSV model which has a hexagon cross section, the HSI cross section is an equilateral triangle which provides hue and saturation information, leaving cross section altitude to determine the intensity component which is determined by the equation:

I = (R + G + B)/3

However, accurate color reproduction is impossible using this equation because it is at variance with visual perception processes.

Figure 13-5 The HSI model and solid

CIE XYZ

Recalling Thomas Young's color matching experiments, adjusting the intensity and combining three suitable primary lights matches many selected colors. The three radiance values collectively required to achieve a match are referred to as the sample's tristimulus values and are provided graphically in Figure 13-6.

Adding the three radiances together and dividing each of a sample's tristimulus value by the sum provides the sample's chromaticity coordinates. Thus, the tristimulus values indicate how much of each light's maximum radiance was required to produce the sample match, and the trichromatic coefficients indicate how much fractional radiance each primary light contributed to produce the match. So, if R, G, and B are a sample's tristimulus values, r, g, and b are its normalized trichromatic coefficients, where:

r = R/(R + G + B)

g = G/(R + G + B)

b = B/(R + G + B)

Figure 13-6 Tristimulus value graph

Clearly, the sum of r, g, and b is always 1.0. So, subtracting the sum of r and g from 1.0 produces b's value. Like the proportional coordinate discussion, matches with the same chromaticity coordinates represent the same tints, shades, and tones and only vary in intensity, information the normalization process loses.

However, recall that in certain spectral regions it is impossible to stimulate adjacent red or green cones without stimulating both, sometimes considerably. This stimulation "cross talk" introduced the "negative light" concept required to produce color matches. So, if we tried to create a three-dimensional color solid as coordinates, it would contain some negative values. Suppose we require g's value to always be non-negative. This requires r's values to assume negative values. Figure 13-7 displays the trichromatic coefficients corresponding to the Figure 13-6 tristimulus values.

Figure 13-7 Chromaticity coordinates for the spectral colors using the NPL primaries


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Now consider Figure 13-8 on the following page which provides a different graphical representation of the preceding discussion based on well-accepted spectral color matching experiments. This figure only displays r and g values for the visible spectrum. Any point on the curve corresponds to a spectral frequency, and subtracting the sum of a point's r and g coordinate values from 1.0 provides the corresponding b value. Note that all g values are equal to or greater than zero but that r has negative values.

Figure 13-8 is useful because it predicts the result obtained by blending any two light sources. Simply find their corresponding points on or inside the curve and draw a line between the two points. The result's corresponding point will be located on the line positioned between the two points. The point's distance between the two original lights is a function of the radiance of each light. If three lights are mixed, then the triangle formed by the points corresponding to the source lights encloses all color combinations the three lights can construct—called the color gamut.

Referencing Figure 13-7, the curves for the red, green, and blue lights are useful in predicting what resulting chromaticity would result when shining a specific light source on a specific object. The effort involved multiplying the source's spectral power distribution with the object spectral reflectance curve separately with each of the three chromaticity coordinate curves in Figure 13-7. The area under each of the three curves was calculated (using mathematical integration) and the three values comprised the resulting chromaticity.

Figure 13-8 Spectral horseshoe chromaticity diagram

As confusing as this might sound, it was even more difficult to perform in 1931 using hand calculators because of the existence of negative numbers in the calculations. So, rather than use the NPL primaries, the CIE decided in 1931 to use the imaginary lights indicated in Figure 13-9 referred to as X, Y, and Z. After affine transformation distortion magic, Figure 13-10 illustrates the corresponding chromaticity coordinates and Figure 13-11 associates colors with the coordinates and is referred to as the CIE XYZ chromaticity diagram. Note that all curve values are positive, assisting the tireless calculator operators.

Figure 13-9 Z,Y, and Z diagram

Figure 13-10 Chromaticity coordinates

Figure 13-11 CIE XYZ chromaticity diagram

Like the other previous perceptual spaces, the CIE XYZ space is not a perceptually uniform space. Within its three-dimensional coordinate system, it begins its descent to the xy surface as a gleaming white spot. It assumes a potato shape and descends with a plop, leaving its characteristic horseshoe print on the xy plane. While its exterior is brightly adorned, its xy plane print represents the space's maximum range. Other z values cross sections cover less area. Looking down on the space from above provides the generally accepted color areas Figure 13-11 illustrates.

The CIE XYZ space is useful because it predicts the resulting chromaticity that results from mixing two chromaticities (Figure 13-12). By extension, it also shows the gamut that results from source primary selections (Figure 13-13), intuitively proving that no combination of source primaries can generate all chromaticities. As an example, it correctly indicates the difficulty of producing 500 nm wavelengths from typical primaries. Finally, when the white point is known, it provides an easy way to calculate the dominant wavelength of any chromaticity.

Figure 13-12 Chromaticity resulting from mixing two chromaticities

Figure 13-13 Chromaticity resulting from mixing three colors

RGB/HSV/HLS/HSI Color Model Conversion Routines

Appendix B contains several color space conversion routines from the VESAPCX.C program.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 14
Bilevel Digital Halftones

What This Chapter is About

Digital halftoning is a process that provides a continuous intensity illusion for output rendered using a few intensity levels, usually two. There are three general digital halftoning approaches and variations of each. This chapter discusses these methods and the PRINTPCX.C program provides implementations that allow you to print images on the IBM Proprinter dot matrix printer and most Hewlett-Packard DeskJet printers.

Bilevel Devices

Devices having only two intensity levels are bilevel devices. An example of a bilevel monitor is an IBM Monochrome display driven by a Hercules compatible display controller or a VGA display which is programmed with only two intensity levels (maximum or zero) for every display position. Good examples of bilevel printers are regular facsimile machines and ink jet printers. At every image position, bilevel devices either set the native intensity level of the output medium to a maximum value or leave it unchanged. With subtractive color output, marked areas subtract (remove or darken) reflected light wavelengths.

Color printers use multiple bilevel mechanisms. For example, the Hewlett-Packard DeskJet 550C printer provides four bilevel mechanisms, one each for the cyan, magenta, yellow, and black ink flows. To print a full-color image first requires separating the image into individual color channels and individually printing them so that the various renderings do not interfere with one another by causing moiré patterns or other distortions.

Screen Painting and Colorant Aerosols

Conceptually understanding digital halftoning is easy. Consider screen painting where artists dip a toothbrush into watercolors and drag the wet bristles over common window screen. As the bristles snap across the screen's mesh surface, a colored mist appears and gently drifts down. The mist eventually lands on a surface which the mist colors. Spray painting produces a similar aerosol effect, much to the detriment of many city buildings.

The described aerosol consists of colorant drops which can vary in size, deposit at any surface location, and may overlay or otherwise overlap one another. Digital halftoning also uses an aerosol but differs in the following ways:

  • Individual aerosol drops are usually a uniform size or can assume one of a very few sizes.
  • Aerosol drops can only land at specific, designated points.
  • Potential landing points are small, contiguous, regularly spaced square areas.
  • Only one drop can cover any designated point.
  • Any drop completely covers its associated point but no more.

Increasing the number of landing points per inch, the resolution, requires a finer aerosol but produces a finer rendering. Increasing the number of aerosol drops per area increases the effect until the surface is completely covered. For example, black and white photographs that seemingly produce continuous gray intensities between white and black contain minute, various-size metallic silver clumps that individually reflect or absorb light. So ordinary photographs use a process similar to digital halftoning but on a microscopic scale.

In a nutshell then, digital halftoning algorithms carefully craft and deposit aerosol drops. The result triggers human eye-brain spatial integration and pattern matching capabilities to produce sensations of continuous intensity variations consisting of black, white, and grays. But it is just an illusion, as masters usually discover when showing pictures of themselves to their dog—the canines usually fail to produce heart-warming Rorschach tail wags to their master's ink blot paper.

Ordered and Random Dithers

All halftone approaches share the same principle—every image input pixel produces a corresponding output pixel that has either the maximum or minimum intensity value for the output medium. The various algorithms only differ in their output value assignment methodology and fall into one of two classifications: ordered and random dithers.

With ordered dithers, the input image is tiled with (divided into) small, non-overlapping, usually uniform-sized areas called cells that each contain multiple input points. To simplify the operation, usually the areas are NxN squares with N2 points but sometimes the areas are hexagonal or other rectangular shapes. The output value for any input pixel within a given area:

  • Is calculated independently of any other area
  • Depends on its input pixel intensity and position within its area

For example, suppose an image is divided into 4x4 pixel square areas, each containing 16 pixels, and we wish to display it on a monitor as a digital halftone image using the assigned eight-bit values of zero and 255. With ordered dithers, the algorithm first decides at every pixel position within any square area when to assign the 255 intensity value or zero value. Within every square, we specify pixel (row, column) coordinates as follows:

(0, 0) | (0, 1) | (0, 2) | (0, 3)
(1, 0) | (1, 1) | (1, 2) | (1, 3)
(2, 0) | (2, 1) | (2, 2) | (2, 3)
(3, 0) | (3, 1) | (3, 2) | (3, 3)


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

For a monitor, at every row and column position, the assigned output pixel value is 255 when the input pixel value exceeds a threshold which the algorithm specifies for each pixel position. Otherwise the output value is zero. (For printers, the strategy is the reverse since the output medium is already white and colorant must darken it. So, for printers, the colorant is applied when the input pixel value is less than a threshold to darken the corresponding pixel.) The following is one actual 4x4 square algorithm's thresholds:

Pixel Position | Threshold Value
(0, 0) | 243
(0, 1) | 167
(0, 2) | 218
(0, 3) | 231
(1, 0) | 226
(1, 1) | 47
(1, 2) | 110
(1, 3) | 183
(2, 0) | 210
(2, 1) | 155
(2, 2) | 137
(2, 3) | 221
(3, 0) | 240
(3, 1) | 224
(3, 2) | 198
(3, 3) | 236

Pictorially, the following simple diagram describes the algorithm:

243 | 167 | 218 | 231
226 | 47 | 110 | 183
210 | 155 | 137 | 221
240 | 224 | 198 | 236

So, if all input pixel values within a square have a value of 230, only output pixels (0, 0) and (0, 3) have a value of zero because their corresponding threshold values (243 and 231, respectively) exceed 230. The assigned output values for all the other pixels is 255. The following diagram orders the pixel thresholds—1 has the lowest threshold with 16 having the highest.

16 | 5 | 9 | 13
12 | 1 | 2 | 6
8 | 4 | 3 | 10
15 | 11 | 7 | 14

For uniform intensities across the square, (1, 1) is the first pixel to assume a value of 255, followed by (1, 2), (2, 2), etc., in that order. Over an image, the transition values for pixel positions within an area can vary but usually do not.

The preceding example discussed a 4x4 tile which provides 17 different intensities for the square—all can have zero output values or one to 16 total output values can assume a value of 255. However, remember that determining output values is performed independently on a tile-by-tile basis by pixel position within a tile.

Clustered Dot Ordered Dither

Various ordered dither algorithms illuminate pixels (assign a maximum intensity value) on displays using different strategies. In some instances, isolated pixels are set to the maximum value. Because of monitor slew rate considerations, this can cause problems with low-cost displays which do not have the necessary slew rate to display isolated pixels at high intensity. Applying isolated ink spots also causes problems for many printers. Hence, a popular form of ordered dither is the clustered dot ordered dither while the other forms are known as disbursed dither.

Clustered dot ordered dither algorithms attempt to avoid illuminating or printing isolated pixels by specifying "seed" pixel positions that have the lowest threshold values. The threshold sequence pattern then spirals around the seed position. As in the previous example, consider the following spiral threshold order:

9 | 7 | 8 | 10 | 12 | 11
6 | 1 | 2 | 13 | 18 | 17
5 | 4 | 3 | 14 | 15 | 16
10 | 12 | 11 | 9 | 7 | 8
13 | 18 | 17 | 6 | 1 | 2
14 | 15 | 16 | 5 | 4 | 3

Note that the upper left 3x3 square and the lower right 3x3 square are identical. Note also the upper right and lower left 3x3 squares are also identical. This actually depicts a 3x6 rectangular tiling which is offset by three columns every three input scan-line rows as depicted by the following:

9 | 7 | 8 | 10 | 12 | 11 | 9 | 7 | 8 | 10 | 12 | 11
6 | 1 | 2 | 13 | 18 | 17 | 6 | 1 | 2 | 13 | 18 | 17
5 | 4 | 3 | 14 | 15 | 16 | 5 | 4 | 3 | 14 | 15 | 16
10 | 12 | 11 | 9 | 7 | 8 | 10 | 12 | 11 | 9 | 7 | 8
13 | 18 | 17 | 6 | 1 | 2 | 13 | 18 | 17 | 6 | 1 | 2
14 | 15 | 16 | 5 | 4 | 3 | 14 | 15 | 16 | 5 | 4 | 3
9 | 7 | 8 | 10 | 12 | 11 | 9 | 7 | 8 | 10 | 12 | 11
6 | 1 | 2 | 13 | 18 | 17 | 6 | 1 | 2 | 13 | 18 | 17
5 | 4 | 3 | 14 | 15 | 16 | 5 | 4 | 3 | 14 | 15 | 16
10 | 12 | 11 | 9 | 7 | 8 | 10 | 12 | 11 | 9 | 7 | 8
13 | 18 | 17 | 6 | 1 | 2 | 13 | 18 | 17 | 6 | 1 | 2
14 | 15 | 16 | 5 | 4 | 3 | 14 | 15 | 16 | 5 | 4 | 3

Now consider the example in a different light that reveals the way the offset rectangle patterns interact and the resulting output diamond pattern more fully:

- | - | - | 10 | - | - | -
- | - | 2 | 13 | 18 | - | -
- | 4 | 3 | 14 | 15 | 16 | -
10 | 12 | 11 | 9 | 7 | 8 | 10
- | 18 | 17 | 6 | 1 | 2 | -
- | - | 16 | 5 | 4 | - | -
- | - | - | 10 | - | - | -


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

While normal ordered dither output can present images with a rectangular-lattice pattern across uniform input pixel value areas, this clustered dot strategy builds patterns that represent uniform intensity areas as various sized diamonds. Modern cultures usually consider this more appealing because they are less sensitive to the natural 45 degree patterns that clustered dot approaches generate. Thus, magazines and newspapers frequently use clustered dot algorithms for their pictures but must use different angles for each colorant. These angles require different shaped cells for each color separation and greatly complicate the processing. Hence, this book only provides a printing example for error diffusion which produces excellent output but is difficult to reproduce.

The VIDEO.C sample program illustrates several clustered dot algorithms as well as several ordered dot algorithms. The cluster patterns build horizontal, vertical, slanted, and diamond patterns. The horizontal pattern is particularly suited to facsimile applications because facsimile compression favors the long runs of zeros and binary ones horizontal clustered dot algorithms produce.

Printer Lines Per Inch

Ordered dither algorithms process groups of scan lines together. Suppose a printer has a 300 dpi resolution. If an algorithm processes four scan lines together to produce tiled ordered dither output, then its corresponding output is 300/4, or 75 lines per inch. Similarly, processing six lines together produces 50 lines per inch.

Processing more lines together provides more intensity levels since more input pixels are included in each tile. However, output images begin to present unappealing-appearing textures—coarse and grainy—if too many lines are processed together. Hence, to minimize such visual distractions, collectively called artifacts, ordered dither algorithms provide tradeoffs between output resolution and intensity levels.

While ordered dot dithers compute output values based on thresholds and ignore errors that occur when input values do not equal output values, error diffusion algorithms incorporate error considerations in subsequent calculations.

Error Diffusion Dither

To understand error diffusion, first consider images produced by threshold algorithms. With threshold algorithms, every pixel output value is determined by comparing the pixel's input value to some constant value. If the input value is above the threshold, the pixel output value is set to the maximum value; otherwise it is set to zero. Resulting errors are ignored. The VIDEO.C sample program illustrates two images produced from the same input image using two different respective threshold values. The images are unattractive because they provide draconian contrast and massive contouring as a consequence of the pixel rendering algorithm generating and ignoring significant assignment errors.

Like threshold algorithms, error diffusion dithering also uses a constant threshold to compute pixel-by-pixel output values and generates rendering errors during its assignment operations. Sometimes error diffusion algorithms assign zero output values to input pixels with values greater than zero. Other times the algorithms assign maximum output values to input pixels having less than the maximum input value. However, unlike threshold algorithms, error diffusion algorithms calculate the generated error and distribute varying fractions of it to surrounding image pixels whose output values have not yet been calculated. The particular way the error is distributed determines the particular error diffusion algorithm.

In 1975, Robert W. Floyd and Louis Steinberg published their milestone paper titled An Adaptive Algorithm for Spatial Gray Scale in the Society for Information Display 1975 Symposium Digest of Technical Papers. This landmark paper introduced the first error diffusion algorithm, referred to in the literature as Floyd Steinberg error diffusion. To understand the Floyd Steinberg algorithm, consider the following 8-bit image sample fragment:

... | ... | ... | ... | ...
... | ... | 223 | 200 | ...
... | 200 | 200 | 200 | ...
... | ... | ... | ... | ...

For the algorithm, select a constant value such as a mid-intensity value of 127. In this image fragment, each intensity value will now be raised to a maximum 255 value or reduced to a minimum zero value depending on whether the pixel's value plus its accumulated error is greater than or less than the constant selected.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Assume we are processing the image from the top scan line to the bottom and we are now at the pixel with a value of 223. Since 223 is greater than 127, the output value for this pixel is 255. This assignment generates an output error of (255 – 223) or –32. Since error calculation often produces negative numbers, remember that signed arithmetic is implied. According to the Floyd Steinberg algorithm the generated error is distributed to neighboring pixels as follows:

  • 7/16ths of the error is added to the value immediately to the right of the point being processed
  • 3/16ths of the error is added to the pixel one row down and one column to the left
  • 5/16ths of the error is added to the pixel immediately below the pixel being processed
  • 1/16th of the error is added to the pixel one row down and one column to the left

Pictorially, this is represented as:

 | • | 7/16
3/16 | 5/16 | 1/16

or:

(1/16) | • | 7
3 | 5 | 1

where • represents the current computation pixel. So, after processing this point, the image fragment now looks like:

... | ... | ... | ... | ...
... | ... | 255 | 186 | ...
... | 194 | 190 | 198 | ...
... | ... | ... | ... | ...

Processing continues with the next point to the right which has an example value of 186, and so on. When the row is completely processed, the next row is processed similarly. So, the process begins at the first position (leftmost) on the first scan line and processes each scan line left to right.

After Floyd and Steinberg published their paper, many error diffusions appeared. A few are listed below:

Jarvis, Judice, and Ninke:

 | (1/48) | • | 7 | 5
3 | 5 | 7 | 5 | 3
1 | 3 | 5 | 3 | 1

Stucki:

 | (1/42) | • | 8 | 4
2 | 4 | 8 | 4 | 2
1 | 2 | 4 | 2 | 1

Stevenson and Arce:

 | (1/200) |  | • | 0 | 32 | 0
12 | 0 | 26 | 0 | 30 | 0 | 16
0 | 12 | 0 | 26 | 0 | 12 | 0
5 | 0 | 12 | 0 | 12 | 0 | 5

Scan-line Centering

Note that error diffusion algorithms only alter pixel values that are not yet set to minimum values. These pixels are either ahead on the same line in the current direction of travel (forward is left to right and reverse is right to left) or close by (ahead or behind) on lower scan lines. When calculating pixel values bordering scan-line edges, there is not always a place to diffuse error to, particularly on lower lines. So, it is useful to center image scan lines in arrays that are longer than the scan line to facilitate computations. Floyd Steinberg needs an array two columns wider than image scan lines because errors are diffused to the following scan line one column to the right and one to the left of the current calculation pixel. Stevenson and Arce needs an array six columns wider than the scan line.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Serpentine Raster

It is not necessary to process each line left to right. Programs can process even-numbered lines left to right and odd-numbered lines right to left. This is called a serpentine raster, which eliminates artifacts in some images. With the serpentine raster, the Floyd Steinberg algorithm becomes:

left to right | (1/16) |  | • | 7
 |  | 3 | 5 | 1
right to left | (1/16) |  | 7 | •
 |  | 1 | 5 | 3

A similar algorithm exists when images must be processed from bottom to top.

Softening Compensation

Error diffusion naturally softens images. So, to counteract the smoothing effect, it is useful to sharpen images before processing them. However, note the previously suggested procedure alters pixel values, perhaps several times depending on the algorithm, before they are used. This renders them incapable of being sharpened since sharpening can only occur on original pixel data, not the original pixel data plus the cumulative error. One solution is to maintain a separate error array which is referenced only after calculating the pixel sharpened value.

Another approach discards some of the error which provides a natural contrast enhancement. The larger the error discarded, the more pronounced the contrast. In the extreme case, all error is discarded which is equivalent to a threshold algorithm. One of Bill Adkinson's favorite error diffusion algorithms, also found in Digital Vision, Inc.'s ComputerEyes/ Pro EYES halftone algorithm documentation, is:

(1/8) |  | • | 1 | 1
 | 1 | 1 | 1 | 0
 | 0 | 1 | 0 | 0

which uses 6/8ths of the error, discarding 2/8ths. The algorithm is fast since 1/8th of the error is quickly obtained by shifting the error value right three bits, effectively dividing by 8. However, like all error diffusion algorithms, the repeated memory references place extensive pressure on the processor caches.

A third sharpening technique was developed by Dr. Reiner Eschbach of Xerox's Webster Research Center and was awarded United States Patent 5,045,952. Rather than select a constant threshold value, this approach allows the threshold to vary depending on the original pixel value, hence its name Threshold Modulation in 1-D Error Diffusion. The VIDEO.C program shows how to use this approach, which is applicable with any error diffusion algorithm.

A fourth, but certainly not final, approach uses unsharp masking. Consider the following situation for the current calculation point x:

a | b | c
d | x | f
g | h | i

Rather than use the current calculation point's value in the threshold comparison and error calculation, simply substitute:

3*x – (b + d + f + h)/2

which is one way of generating the point's unsharp mask value.

Error Diffusion Variations

There are numerous error diffusion variations and implementations. IBM researchers Gerhard Thompson and Gerald Goertzel developed an interesting approach which uses randomization to distribute error, in an attempt to eliminate visual artifacts. Let e be the assignment error and z be a random number between 0.25 and 0.75 which is picked for each pixel and not repeated often. Consider the following error diffusion arrangement:

 | • | r
a | b | c

Where:

r = 14ze/16
q = (15 – 7z)e/8
a = 3q/16
b = 5q/16
c = e/16

This approach is more computationally intensive than others.

Error Diffusion Summary

The VIDEO.C algorithm provides numerous error diffusion implementations. It should be noted that the original Floyd Steinberg algorithm generates excellent results, particularly when using serpentine rasters and discarding some of the generated error for contrast. However, error diffusion algorithms generally produce output that is difficult to compress compactly and render on many devices because of the isolated points it produces.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 15
Color Image Quantization

What This Chapter is About

Color image quantization is the process that selects a set of primary color mixtures to represent an image's color gamut. In this book, it is the process that reduces 15-, 16-, and 24-bit direct color images to similar appearing images that usually contain 256 or fewer carefully selected RGB combinations. This chapter discusses two popular statistical pixel distribution analysis techniques that determine the best 256 RGB combinations for individual image palettes.

The first, known as Paul Heckbert's median cut method, is a simplistic approach that depends on an image's pixel statistical distribution and often yields acceptable results. The second approach, developed by Xiaolin Wu of the University of Western Ontario in London, Ontario, Canada, trades computational speed for greater image fidelity using more rigorous mathematical methods. Until Xiaolin Wu developed and published his ingenious computational shortcuts, the general approach was generally considered too computationally intensive for personal computers.

Uniform Quantization Introduction

Direct color images with 15, 16, and 24 bits describing each pixel are increasingly common. However, older, legacy display controllers typically only display 256 RGB combinations at a time. So, it is advantageous to develop strategies and approaches that transform direct color images into palettized images containing 256 RGB combinations or fewer. The process of distilling direct color image pixel information into members of a much smaller, fixed-size palette is referred to as color quantization.

Color quantization usually generates images with 256 (or fewer) colors. This allows 8 (or fewer) bits to identify the RGB combination for every pixel. This pixel information reduction provides natural compression that conserves image storage capacity and transmission bandwidth. It also allows image hue, saturation, and intensity manipulations that involve the few palette RGB combinations rather than each individual pixel within an image, allowing the manipulations to appear virtually instantaneous.

Finally, transmission bandwidth conservation is also important in multimedia video applications which often quantize individual video frame pixel information into 128 RGB combinations. This allows successive frames to alternate between palette entries zero to 127 on one frame and 128 through 255 on the next, and so on. Careful palette manipulation between successive frame display eliminates undesirable color flashing that might occur when pixel palette entries are incorrect for the current or previous frame.

The RGB24TO8.C Program

The RGB24TO8.C program illustrates two naive, straightforward, image-independent, uniform color quantization approaches and their unsatisfactory results. For each palette entry, the first approach assigns palette entry bit positions. For each of the 256 palette entries, the three high-order bits specify the red intensity, the middle three bits specify green intensities, and the two low-order bits specify blue intensities. This allows red and green intensities independently to assume eight different intensities each while blue intensities only assume four intensities which can introduce contouring in blue gradient areas.

The SetR3G3B2Palette() function initializes the palette by distributing the individual primary RGB components' intensities uniformly (red and green use intensity values 0, 32, 64, 96, 128, 160, 192, and 224 while blue uses intensity values 0, 64, 128, and 192) and gamma correcting the resulting palette.

The program could use other schemes rather than uniform distribution, but the selected strategy is expedient. While fast, the arbitrary primary color intensity assignment leaves the palette unable to display white since the RGB (255, 255, 255) triad is absent. To display direct color image pixels, the R3G3B2() function captures the appropriate number of high-order bits for each primary color component and shifts them into a byte in the proper pixel byte position as follows:

if (Red | < 240) | Red | = 16;
if (Green | < 240) | Green | = 16;
if (Blue | < 224) | Blue | += 32;
PixelValue = | ((Red & 0x00E0) |  | ) |
 | ((Green & 0x00E0) | >> | 3) |
 | ((Blue & 0x00C0) | >> | 6);

This produces recognizable palette-adapted images that exhibit noticeable contouring or posterization.

The second approach RGB24TO8.C uses assigns numbers of uniformly distributed primary color intensities rather than palette entry bit positions. The selected approach in function SetMorphsPalette() uses seven uniformly distributed intensities for red, nine for green, and four for blue. This assigns 252 color combinations (7 x 9 x 4), using palette entries zero through 251.

However, the assignment again omits white as well as any gray values since the uniform distributions are all different, only sharing the value zero in common. Because the eye is so sensitive to achromatic deviation errors, the remaining four intensity assignments for palette entries 252 through 255 are (60, 60, 60), (120, 120, 120), (180, 180, 180), and (240, 240, 240), respectively, which provide some achromatic gray values.

From the color space discussion, the RGB cube is a three-dimensional space. To select the appropriate palette representation for each image pixel, the program computes the straight-line three-dimensional distance from each image pixel to each palette entry using the Pythagorean theorem and determines which palette entry is geometrically closest. The program assigns the first pixel it finds with the shortest distance as the representative palette entry.

Despite the improved number of green intensities, displayed images still exhibit detectable contours and the geometric distance calculation dramatically slows image rendering despite heavy look-up table use to avoid repetitive square calculations. An excellent reader exercise would involve calculating the error caused by inexact pixel assignments and distributing the error to neighboring pixels using an error diffusion algorithm. This dramatically improves many image renderings.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Paul Heckbert's Median Cut Approach

Heckbert's median cut approach is a process that selects an optimum palette for direct color images. In contrast to the two previous approaches, median-cut output is image-dependent so that different images usually have very different palettes. The primary objective is democratic in that each palette entry theoretically represents the same number of image pixels. So, for images with 512 x 512 resolution, theoretically each palette entry represents (512 x 512)/256, or 1,024, image pixels. The process repeatedly subdivides the RGB cube into smaller and smaller rectangular spaces based on the equal representation principle.

The median-cut approach generally produces good results despite a few essential oversimplifications that favor performance. It consists of four steps:

  1. Determining the original image's pixel distribution
  2. Selecting an optimum color map (palette) based on the first step
  3. Mapping image pixels to the "nearest" palette entry
  4. Assigning image pixel ranges to individual palette entries with optional error diffusion

To understand the process, first assume that an image has no primary blue intensity contribution present in any pixel so that the image consists of fully saturated colors consisting solely of red and green intensities. This reduces the palette selection to a two-dimensional exercise.

To determine the original image's pixel distribution, plot each point on a chart that allows red and green values to range from zero to the maximum intensity, say 255. This produces the color image's histogram (see Figure 15-1).

To select the optimum color map, subtract the maximum red value from the minimum red value and call the difference DeltaR. Similarly, subtract the maximum green value from the minimum green value and call the difference DeltaG. Suppose now that DeltaR is bigger than DeltaG— indicating the red component range is bigger than the green component range. Now determine the median red value—the value such that half the red values are above this value and half are equal to or less than this value. Drawing a horizontal line at this point splits the square into two rectangles.

Figure 15-1 Color image histogram (assumes blue contribution value is zero for each pixel)

Now, find which of the two rectangles has the largest component range for red or green and split that rectangle at the median point for the color component with the largest range. Now there are three rectangles. Each rectangle subdivision produces two rectangles from one, so each subdivision increases the total number of rectangles by one. Repeat this maximum red-green component range analysis procedure for all existing rectangles and median subdivision process until you have created as many rectangles as you have allotted palette entries.

Now, within each rectangle, average the red value for each pixel the rectangle contains and average the green value for each pixel the rectangle contains. This produces a weighted center-of-gravity pixel value for the rectangle which is a palette entry. Finally, assign each image pixel to one of the palette entries and diffuse the red-green errors to neighboring pixels.

The only process difference for images that contain all three color components is that the subdivision process is three-dimensional, involving rectangular volumes. Selecting what volume to divide into two rectangular volumes involves analyzing the ranges of the three color components for each existing rectangular solid and dividing at the median point. Anton Krueger's thoughtful "Median-Cut Color Quantization" article and C language implementation is in Dr. Dobb's Journal September 1994 issue on page 46.

Xiaolin Wu's Optimal Color Quantization

Xiaolin Wu's optimal color quantization divides the RGB color cube into rectangular solids like Heckbert's median-cut method. However, the division criteria significantly differs. Wu's approach calculates the center of gravity of each volume and the statistical variance of the color components around it. The subdivision occurs so that it minimizes the sum of the statistical variances for the color component with the maximum statistical variance within an existing volume, producing two new subvolumes.

The computational resources to calculate the division point are formidable, exceeded only by the complexity of Xiaolin Wu's mathematics presented in his article titled "Efficient Statistical Computations for Optimal Color Quantization" in Graphics Gems II, page 126. Luckily, Dr. Wu makes a public domain implementation available which first quantizes color components to 32 intensity levels for computational efficiency. His version is for a Sun Microsystems workstation and has been significantly modified to adapt it to the PC.

Dr. Wu's implementation uses several automatic 32x32x32 four-byte word arrays. Each such array requires 128 KBytes of memory. This exceeds available PC-DOS memory and coincidentally destroys the PC's rather anemic stack, so this book's implementation first quantizes color components to 16 intensity levels and uses 16x16x16 four-byte word static arrays. Unfortunately, this reduces the color gamut from Dr. Wu's 32K possible colors to 12K possible colors. To quantize at 32 levels for non-PC-DOS implementations, simply change the COMPONENT_BIT_ DEPTH preprocessor identifier value to 5.

The sample WDS_QNT.C program in Chapter 17 contains the PC-adapted version of Dr. Wu's original program.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

SECTION IV

Chapter 16
Miscellaneous Programs

What This Chapter is About

This chapter provides brief descriptions of programs contained on the CD-ROM accompanying this book. Please read the licensing information each program contains.

TIFFEXAM.C Program

The TIFFEXAM.C program displays TIFF file information for TIFF specification 5.0 format files that Hewlett-Packard ScanJet scanners and other popular computing products produce. The program handles Intel Corp. byte-ordering (referred to as little-endian) as well as Motorola Corp. processor byte-ordering (referred to as big-endian). The TIFF 6.0 specification significantly expands the TIFF 5.0 specification, and the programming extensions are left as an extended reader exercise.

TIFF2BIN.C Program

The TIFF2BIN.C program reads a program argument-specified 8-bit negative image TIFF file and creates a 6-bit positive image file. The program assumes the input file is a carefully produced file with 92 rows x 98 columns and 256 monochromatic intensities. Hewlett-Packard ScanJet IIc scanners using Hewlett-Packard's DeskScan software easily produce such files. The ARRAY96.C program performs a hard-coded seek 734 bytes into the file where Hewlett-Packard software places the start of image data. (Use the TIFFEXAM.C program to confirm this hard-coded assumption.)

The program reads the entire 92x98 byte image into a 92x98 character array using one read operation and then converts the image from an 8-bit negative image to a 6-bit positive image. A gray response curve correction is commented out. The program creates a binary file named 96x96.bin by writing two 96-column rows of binary zeros followed by 92 rows each with 96 columns created by reading 96 bytes from each row at a row offset of one byte. The output file operation ends after writing two final rows of binary data.

ARRAY96.C Program

The ARRAY96.C program reads a program argument-specified binary file created by a program such as the TIFF2BIN.C program. It then creates a text file named image.h which the VIDEO.C program can use as an example include file.

VESAEXAM.C Program

The VESAEXAM.C program determines whether VESA VBE support is available to MS-DOS application programs. (Either the graphics card vendor or SciTech shareware module (available by calling 916-894-8400 in the U.S.) can provide the VBE support). If VBE support is available, the VESAEXAM.C program interrogates the support and reports its findings which are documented in the Video Display Boards and Controllers chapter. By redirecting output to a file, users can review or print the information later.

GAMMA.C Program

The GAMMA.C program allows users to determine visual midpoint gamma corrections for each monitor electron gun. Users run the program, specifying a VESA packed pixel mode their display controller supports. The program first verifies that VESA support is available and then confirms the display controller supports the argument-specified mode.

If the mode is supported, the program creates an initial rectangular monochromatic array of alternating horizontal areas. Some areas contain pixels with (R, G, B) intensity values of (127, 127, 127). The other areas contain alternating (R, G, B) values of (0, 0, 0) and (255, 255, 255). Pressing the cursor-up and cursor-down keys respectively increases or decreases the initial 1.0 gamma correction value. When the areas seem to provide the same intensity when viewed at a distance, the user should press the Enter key. The program then repeats the operation for red, green, and blue in isolation.

Before exiting, the program displays a gamma corrected rectangular monochromatic array of alternating horizontal areas using individually adjusted red, green, and blue gamma correction values and returns to MS-DOS after reporting the user selected gamma correction values in each of the operations. Redirecting the output to a file named gamma.val allows the VESAPCX.C program to provide gamma corrected image displays.

CLRSPC.C Program

The CLRSPC.C program provides a test program that converts from RGB color space into another color space and back. This allows developers to test RGB color space conversions to and from the HLS, HSV, and HSI color spaces. A standard HLS to RGB routine (HlsToRgb()) is given as well as an alternate implementation (HlsToRgb2()) for Ken Fishkin's accelerated HLS to RGB transformation found in Graphics Gems Volume 1.

To operate the program, users specify R, G, and B values as arguments where:

   0 <= R, G, B <= 255

The program converts the specified RGB triad into other color space triads and converts these triads back into RGB space values. The HSV and HSI conversions are commented out, leaving only HLS conversions.

VESATEST.C Program

The VESATEST.C program displays the RGB cube, CMY cube, and an HSI triangular cross section at a user-specified VESA VBE packed pixel mode. To operate the program, users specify a desired VESA packed pixel mode value as a program argument. After determining that both VESA VBE support and the selected direct color mode are available, the program first draws the RGB cube at the selected direct color mode's bit depth.

The program draws the first RGB cube cross section by setting all blue values equal to zero and drawing a 256x256 pixel rectangular array (square array if the selected mode has a 4:3 aspect ratio) that varies red values vertically from 255 to zero and green values horizontally from zero to 255. The next cross section is drawn by starting one pixel lower and one to the left. On this cross section, the blue component has a value of BLUE_INCREMENT. At each successive cross section, the blue component value increases by BLUE_INCREMENT, and so on until it exceeds a value of 255. Note the banding illusion that occurs at a 15-bit depth. This visual illusion decreases at a 16-bit depth and disappears at a 24-bit depth.

The process is similar for the CMY cube. Displaying the HSI triangular cross section requires some geometry and trigonometry calculations which are not discussed here other than to mention that the routine uses mirroring to simplify calculations. In addition, the routines request an intensity value between zero and 1.0, allowing a default of 0.5.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 17
Image Display

What This Chapter is About

This chapter provides brief descriptions of programs contained on the CD-ROM accompanying this book. Please read the licensing information each program contains.

VIDEO.C Monochromatic Image Display Program

The VIDEO.C program demonstrates monochromatic image manipulation techniques using a 96x96, 6-bit-per-pixel monochromatic image. The source image is slightly out of focus and chosen to illustrate sharpening techniques. It was scanned at a horizontal resolution 1.2 times greater than vertical resolution to compensate for a 320x200 graphics screen resolution aspect ratio (a 320x240 screen resolution has a 1:1 aspect ratio and 240 divided by 200 equals 1.2). This original image is included in the program using the image.h file the ARRAY96.C program produces. Users can alternately create their own 96x96 binary image and specify its name as a run-time argument. This allows VIDEO.C to load the specified image for subsequent manipulation operations that include gamma correction, sharpening, histogram stretching, histogram equalization, and a variety of digital halftone algorithms.

VIDEO.C begins operation by first checking whether a run-time argument is specified. If so, the LoadOriginalImage() function loads the specified image into the Image[96][96] array defined in image.h and zeros the high-order two bits of each pixel value to ensure each pixel consists of six significant bits. After presenting the program logo and waiting five seconds, VIDEO.C sets the screen resolution to 320x200— the lowest common denominator VGA screen resolution—by invoking SetVideo320x200().

The 320x200 screen resolution provides a standard 256-color VGA palette. However, VIDEO.C only uses 64 palette entries with each entry providing a different monochromatic screen intensity. While the screen resolution is less than possible on many screens, the program operates on standard VGA systems as well as SVGA systems. Even with 6-bit DACs, a standard VGA palette provides more monochromatic intensity values than 15-bit and 16-bit direct color modes which each can only support 32 monochromatic intensities. Only 24-bit direct modes combined with 8-bit DACs provide more (256) monochromatic intensities.

After setting the screen mode, VIDEO.C invokes GenerateGammaCorrectedGrayScalePalette() which constructs a gamma corrected palette in the PaletteArray[PALETTE_SIZE_64][3] global array by designating equal RGB intensity values. The GAMMA_CORRECTION_VALUE preprocessor identifier determines the gamma correction value, and the intensity normalization factor is 63 since intensities vary from zero to 63. Invoking the normal int 10h BIOS video interrupt loads the palette into the video controller.

VIDEO.C next presents the source image by invoking ZoomIn() which uses a progressive resolution technique. ZoomIn() samples the source image by dividing it into 16x16 square areas and using the upper left pixel to represent the entire 16x16 output image area. After a half-second delay, it repeats the process using progressively smaller 8x8 square areas, then 4x4 square areas, and finally 2x2 square areas.

Eventually, DisplayOriginalImage() executes and presents the entire source image. This routine invokes DisplayScreenHistogram() which calculates a histogram for any 96x96 screen area with the specified upper left corner and displays the histogram at the specified position (below the source image). The continuously varying intensity bar beneath the horizontal provides a visual guide to the pixel intensities immediately above.

When DisplayNegativeImage() executes, it presents the source image negative starting at row position zero, column position 100. This routine invokes DisplayScreenHistogram() to calculate the negative image histogram and display it below the negative image.

VIDEO.C next presents the source image bit planes individually starting with the high-order bit position and moving progressively to the low-order bit position by invoking DisplayBitPlanes(). At every pixel position, if a tested bit has a one value, the operations produce a white dot in that pixel position. Otherwise, the output pixel is black. The first pass pre- sents a threshold operation and subsequent operations present increasing bit-plane refinements. Note the banded appearance of some intermediate bit positions and the final random appearance of the final operation.

VIDEO.C next invokes DisplayContourImages() which presents a sequence of images that illustrate image contouring. Beginning with a threshold operation that presents only high-order bit information, the successive images add the next significant bit value to produce increasing detail. So, the first image presents only bit five information; the next bits five and four; the next bits five, four, and three, etc. The successive images are cumulative refinements, and the posterization effect that occurs when only a few bit levels are used to represent an image is evident.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

VIDEO.C next invokes DisplayFilteredImages() which produces 11 different filtered images using different algorithms. The algorithm FilterImageScanLine() used to filter individual scan lines depends on its input variable TypeFilter. DisplayFilteredImages() illustrates eight Laplacian edge enhanced images followed by three unsharp mask operations. The FilterImageScanLine() routine lists the specific 3x3 masks.

VIDEO.C then invokes DisplayEqualizedImage() to perform a histogram stretch operation. CreateEqualizationArray() creates equalization mapping values which DisplayEqualizedImage() uses to produce its output. CreateEqualizationArray() establishes a minimum intensity value by determining what intensity is greater than 1.5 percent of the total number of pixel values. It also establishes a maximum intensity value by determining what intensity value is greater than 98.5 percent of the total number of pixel values. CreateEqualizationArray() then uses a linear function (straight line drawn through the Cartesian coordinates (minimum, 0) and (maximum, 63)) to determine the values of the remaining 97 percent of the pixels.

VIDEO.C next performs a gamma correction for all palette entries by invoking FiddleWithPalette() which changes all screen pixel intensities nearly instantly. The initial gamma correction value is the current gamma correction value—GAMMA_CORRECTION_VALUE. The gamma correction value decrements to 1.25 and then increments to a value of 2.2 before decrementing down to GAMMA_CORRECTION_VALUE. Glancing at the horizontal bars beneath the histograms provides a visual representation of the effect of these gamma adjustments.

VIDEO.C then executes PerformScanLineProfileAnalysis(). This routine displays the individual pixel values for all pixels on each scan line in the histogram-stretched image by slowly sweeping a horizontal line down the image from top to bottom. The moving graph represents pixel intensities on the vertical axis and uses the horizontal pixel position as the horizontal coordinate.

VIDEO.C next displays numerous halftones using both ordered dither and error diffusion algorithms by invoking GenerateHalftones(). GenerateHalftones first loads the Borland Graphical Interface module EGAVGA.BGI by invoking BorlandGraphicsInit(). This routine switches the screen resolution to 640x480 graphics mode which has a 1:1 aspect ratio.

Using the Borland International graphics library routines allows the various halftone routines to use the Borland International C++ library routine putpixel() to emit a white or dark pixel. However, the aspect ratio change from 1:1.2 to 1:1 requires compensation (vertical stretching) to present the halftone images. The various halftone routines accomplish this by invoking LoadInterpolatedArrayLine() which assumes the source scan lines are expanded horizontally by a factor of four. To calculate the intermediate horizontal pixel intensity values between source image pixels, the routine uses linear interpolation. To calculate the intermediate vertical pixel intensity values, the routine interpolates between source input scan lines. The computations proceed as follows:

A VIDEO.C halftone routine produces an image with 384 horizontal pixels in each scan line (96 times 4) and almost 460 scan lines (96 times 4 times 1.2 for scaling). Before proceeding with a halftone iteration, a halftone routine invokes LoadInterpolatedArrayLine(), specifying what generated scan line should be loaded into its computation array. The scan-line number varies from 1 to almost 460. LoadInterpolatedArrayLine() divides the input number by 460.0 (for scaling) rather than 384 and multiplies the product by 96—the number of actual source image scan lines.

Suppose the product is 49.65, which is between 49 and 50. This indicates source image scan lines 49 and 50 should be used to generate the requested scan line. After horizontally interpolating both scan lines 49 and 50 to produce two interpolated scan lines with 384 pixels, the routine calculates how much each horizontally interpolated scan line contributes in the vertical interpolation step. In this instance, line 50 contributes 65 percent of the value and line 49 contributes 35 percent because the requested scan line is closer to line 50 than source line 49.

The GenerateHalfTones() routine first produces two threshold images using pixel intensities 16 and 18 as respective thresholds. It then invokes GenerateOrderedHalfTones() and GenerateDiffusedHalfTones() to generate a series of ordered and error diffusion images. In order, the 8x8 pixel algorithms ordered halftones are FineFattening, Bayer, VerticalLine, UpperLeftDown (135 degree), HorizontalLine, LowerLeftUp (45 degree), and CoarseFattening. The FineFattening, Bayer, VerticalLine, HorizontalLine, and CoarseFattening algorithms use Hewlett-Packard's threshold arrays listed in Hewlett-Packard printer documentation. The LowerLeftUp and UpperLeftDown algorithms are derivations of the Hewlett-Packard HorizontalLine algorithm. Because the arrays assume pixel intensities from zero to 255, NormalizeArrays() divides each pixel threshold by 4 to generate pixel thresholds from zero to 63.

GenerateDiffusedHalfTones() invokes the following error diffusion routines in order:

  1. Stucki
  2. Adkinson
  3. Stevenson and Arce
  4. Sierra
  5. Jarvis, Judice, and Ninke
  6. Burkes
  7. Floyd Steinberg using integer calculations with no error loss
  8. Floyd Steinberg using integer calculations with 10 percent error loss
  9. IBM Corp.'s Floyd Steinberg with random error distribution
  10. Floyd Steinberg using a serpentine pattern
  11. Eschbach using integer calculations
  12. Floyd Steinberg with a concurrent Laplacian operation using integer calculations
  13. Eschbach using floating point calculations with no error loss
  14. Floyd Steinberg using floating point calculations with no error loss.

The various results are derived from the low-resolution, blurry image. VIDEO.C program ends by setting the resolution to 80x25 text mode and displaying a closing message.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

VESAPCX.C Color Image Display Program

The VESAPCX.C program displays PCX file version images on monitors controlled by display adapters that support VESA VBE version 1.2 or later. The specific monitor mode and PCX file are input arguments. PCX palette images can display on VBE packed pixel modes and 24-bit images can display on VBE direct color modes.

The program begins execution by verifying two input arguments exist. The first specifies the VBE mode the program should use and the second specifies the PCX filename the program should display. After verifying the display adapter provides VBE support, the program verifies the adapter supports the specific VBE mode. If the mode is not supported, the program exits with an error message. Otherwise, the program sets the requested VBE mode.

If the mode is a packed pixel mode, DisplayDefaultPaletteDiagonally() displays the default VGA palette across each line starting with palette entry zero in scan line zero, column zero. Whenever a display line is not completely filled after reaching palette entry 255, the program continues filling the line starting from value zero. When a line is finally filled, the next line is filled starting with a palette value one greater than the line above it started with. This gives the appearance of diagonal colored lines which slope from the upper right to the lower left. Eventually, the initial screen fills with colored diagonal lines. The program then continues displaying more scan lines by invoking SetDisplayStartLine() which performs a vertical scroll. SetDisplayStartLine() therefore provides a working example of memory scrolling which is also useful for horizontal scrolling.

The VESAPCX.C program now attempts to display the selected PCX file by calling DisplayPcxFile(). DisplayPcxFile() opens the file. ValidatePcxFileHeader() validates the file has a valid PCX header and attempts to load RGB gamma correction values from a file named gamma.val. If the file does not have a valid PCX header, DisplayPcxFile() returns. Otherwise, DisplayPcxFile() resets the screen to the desired mode, setting the clear video memory flag so the VBE support clears the screen.

DisplayPcxFile() now tests to see if the image has one display plane (palette). If so, the VESA VBE mode must be packed pixel and an error occurs if not. Alternately, the image may have more than one display plane (three for direct color images), and the VESA VBE mode must be a direct color mode. PresentDirectColorPcxImage() presents direct color images and PresentColorPalettePcxImage() presents palette images.

PresentDirectColorPcxImage() presents the direct color image, correcting the bit depth if necessary. It then displays a sharpened color image by converting the color information into luminance information and sharpening the luminance data. The resulting color information is converted back to color information using a proportional intensity scaling approach.

PresentColorPalettePcxImage() first displays the palette image and then presents the R, G, B, C, M, and Y color information in isolation by successively invoking DisplayRgbPlanes(). DisplayRgbPlanes() displays the image after stripping unwanted color plane information from the original palette. PresentColorPalettePcxImage() last checks to see whether the gamma correction values loaded correctly and performs an image gamma correction if so. PresentColorPalettePcxImage() returns to DisplayPcxFile() which then invokes PerformColorSpaceManipulation().

PerformColorSpaceManipulation() performs numerous color image manipulations in HLS, HSV, and HSI color space by invoking PerformHlsColorSpaceManipulation(), PerformHsvColorSpace Manipulation(), and PerformHsiColorSpaceManipulation(), respectively. Using each of these color spaces, the palette entry hues are first rotated 360 degrees, then the saturation varies from zero to a factor of three of the original value, and the intensity varies from a gamma value of 0.5 to nearly 2. The screen output provides a visual example of many concepts this book discusses, such as non-uniform color space side effects that appear as various image pixels suddenly becoming brighter with hue changes.

PresentColorPalettePcxImage() finally ends after presenting the original image again, followed by the gamma corrected version.

RGB24TO8.C Color Quantization Program

The RGB24TO8.C program displays 24-bit color images using two crude image quantization approaches described in detail in Chapter 15. The first approach constructs an RGB palette using the three high-order bits for the red intensity, the middle three bits for green intensities, and the two low-order bits for blue intensities. SetR3G3B2Palette() initializes the palette by distributing the individual primary RGB components intensities uniformly (red and green use intensity values 0, 32, 64, 96, 128, 160, 192, and 224, while blue uses intensity values 0, 64, 128, and 192) and gamma correcting the resulting palette.

The second approach assigns numbers of uniformly distributed primary color intensities rather than palette entry bit positions. SetMorphsPalette() uses seven uniformly distributed intensities for red, nine for green, and four for blue. This assigns 252 color combinations (7 x 9 x 4), using palette entries zero through 251. The remaining four intensity assignments for palette entries 252 through 255 are (60, 60, 60), (120, 120, 120), (180, 180, 180), and (240, 240, 240), respectively, which provide some monochromatic gray values.

In both approaches RGB24TO8.C attempts to minimize quantization errors by calculating the minimum Euclidean distance within RGB space and discarding any errors. A more refined approach would disperse the RGB component error to nearby pixels' component values, similar to digital halftone error diffusion approaches. This is left as a reader exercise.

WDS_QNT.C Program and WU_QNT.C Color Quantization Program

The WU_QNT.C program is the original implementation of Dr. Xiaolin Wu's color quantization program for Sun Microsystems workstations. The WDS_QNT.C program is an MS-DOS derivative version of the WU_QNT.C program.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 18
PRINTPCX.C Image Printing Program

The PRINTPCX.C program prints specified packed pixel color PCX images on the following printers:

IBM Proprinter II | (black and white dot matrix)
Hewlett-Packard LaserJet Series | (black and white xerographic)
Hewlett-Packard DeskJet 500 | (black and white ink jet)
Hewlett-Packard DeskJet 500C | (color CMY ink jet printer)
Hewlett-Packard DeskJet 550C | (color CMYK ink jet printer)

The program may operate on other Hewlett-Packard printers but is untested and results may vary.

The PRINTPCX.C main() routine expects three input arguments. In numeric order they are: a PCX filename, a printer type, and a gamma correction value. The PCX file must be a valid version 3 palette image or a 24-bit image. ValidatePrinter() verifies the printer argument is one of the following:

Argument | Device | Print Routine
IbmPpII | (IBM Proprinter II) | PrintOnIbmPpII()
HpLj/Dj500 | (Hewlett-Packard achromatic) | PrintOnHpLjAndDj500()
HpDj500C | (Hewlett-Packard CMY) | PrintOnHpDj500C()
HpDj550C | (Hewlett-Packard CMYK) | PrintOnHpDj550C()

main() first creates a gamma translate value look-up table and a random number table for the digital halftone random error distribution operation. Selecting a random error distribution halftone algorithm provides thousands of variations of the printed image that are indistinguishable from a distance. main() calls ValidatePrinter() to verify the printer specification and calculate gamma corrected values. main() then attempts to print the selected PCX file by calling PrintPcxFile(). After PrintPcxFile() returns, main() waits for a user keystroke before calling ResetPrinter() which provides a PC BIOS printer reset. The PC BIOS reset ejects the printed page from Hewlett-Packard printers and helps prevent printers from flushing printer memory before printing the entire image. main() consumes the keystroke and concludes with an ending message.

PrintPcxFile() opens the designated PCX file and calls ValidatePcxFileHeader() to verify the file has a valid PCX header. The file must be a valid PCX version 3 format file with a palette image or a 24-bit image. The PrintDirectColorPcxImageFile() provides a start but each of the print routines must handle converting 15- and 16-bit pixels into an acceptable format.

If the file does not have a valid PCX file, PrintPcxFile() returns without attempting to print anything. Otherwise, PrintPcxFile() calculates the image size and the maximum allowed pixel replication factor to scale the image up to a maximum pixel replication factor of MAX_REPLICATION_FACTOR set to a value of 5.

PrintPcxFile() next displays image dimensions on the monitor before calculating replication factors used during the forward and reverse scan-line print operations. Finally, PrintPcxFile() calls PrintColorPaletteImage() or PrintDirectColorPcxImageFile() which invokes the proper print routine for the designated printer. Each of the routines sends the correct escape sequences to the printer and reduces the image to a monochromatic image if necessary. The Proprinter II support routines also correct for the printer aspect ratio, which is not 1:1 like HP printers provide, while the Hewlett-Packard routines magnify the images up to a factor of five to produce larger images.

All print routines use the Floyd Steinberg halftone algorithm with a serpentine raster and dynamic sharpening operation. The algorithm also randomly distributes error using the IBM developed randomization approach. Because of the array sizes and their number, PrintOnHpDj500C() and PrintOnHpDj550C() use far pointers to arrays allocated on the heap rather than automatic or static arrays. This forces PRINTPCX.C to be a large model program.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

SECTION V

Chapter 19
Some Legal Considerations

What This Chapter is About

Copying digital images, videos, and music clips from the WWW is usually possible with a few mouse clicks and keyboard taps. You can even use images on distant WWW pages on your own WWW pages by referencing their WWW URLs within your own WWW pages. Moreover, an inexpensive scanner quickly produces a digital image from a traditional photograph or magazine print. However, ease of referencing and copying in this—or any other manner—does not impart a right to do so. With digital imaging, owning an image does not bestow a copyright.

The laws regarding copyrights, moral rights thereof, and "fair use" are very complicated and vary globally. You might imagine the surprise of the Silicon Valley company contacted by the descendants of Leonardo da Vinci for using an image of the Mona Lisa in a commercial advertisement. Moreover, digital watermark technology now exists to help identify rightful digital image owners. So, it is important to understand where some of the many problems lie.

To this end, this chapter moves directly into actual incidents which illuminate potential problems regarding image ownership. However, please note this chapter does not constitute legal advice, only comments and observations from an interested layperson. If you need legal advice, consult your attorney.

Vietnam Women's Memorial

Jane Schweitzer, a retired surgery nurse, was visiting the Vietnam Veterans Memorial (a.k.a. The Wall) in Washington, D.C., when she noticed the nearby Vietnam Women's Memorial sculpture. Walking over to the women's memorial, which is located on National Park Service property, she snapped a few photographs of the sculpture.

After returning home, she was genuinely surprised at how well the pictures turned out. So much so, she and her publisher decided to use one on the cover of her forthcoming and highly acclaimed book titled Tears and Rage: The Nursing Crisis in America (ISBN 1-883422-05-1).

After promotional flyers that included a book cover image were circulated at the San Francisco Book Festival, Al Canton, the publisher, received a facsimile on letterhead stationery from the Vietnam Women's Memorial Project, Inc. on November 8, 1995. It read:

Gentlemen:

The Vietnam Women's Memorial Project owns the copyright of the Vietnam Women's Memorial. The image of the Vietnam Women's Memorial may not be used without the written permission of the Vietnam Women's Memorial Project. You have infringed upon copyright law by publishing Tears and Rage: The Nursing Crisis in America with the image of the Vietnam Women's Memorial as its cover.

You must cease the production of this publication that bears the image of Vietnam Women's Memorial on its cover and remove whatever inventory retailers and/or wholesalers have in stock with said image. Again, the Vietnam Women's Memorial image can only be used with the explicit approval of the Project.

We request that you communicate in writing within 10 business days your understanding of our legal copyright interest. If you do not respond within that time, we will refer this matter to our general counsel.

Sincerely,

Diana Hellinger

Apparently, the general counsel referral could not wait. Two days later, Al Canton received a facsimile on letterhead stationery from Mahon & Patusky Law Offices that read:

Dear Mr. Canton:

We are copyright attorneys for Vietnam Women's Memorial Project, Inc. and are writing in connection with your unauthorized reproduction of the Vietnam Women's Memorial sculpture on the cover of Tears and Rage: The Nursing Crisis in America. The Memorial copyright is owned by the Vietnam Women's Memorial Project and your reproduction constitutes a willful copyright infringement. Accordingly, you must immediately cease and desist printing, distributing, promoting and selling the book with the infringing cover in any capacity or we will enjoin you from doing so.

We trust you realize the implications of what is a willful copyright infringement. Not only are you exposed to damages of up to $100,000 for the copyright infringement under [paragraph] 504 of the Copyright Act as amended, but all those individuals who participated in and benefited from the copyright infringement will be personally jointly and severally liable as co-defendants for that conduct in any lawsuit we are required to file. In addition, the wonderful thing about copyright law is that you will be paying all our fees while we pursue you. We're expensive.

Your company should know better than to use an image without checking the copyright ownership of the object depicted. To avoid a lawsuit, we require a written confirmation by you that you will immediately cease and desist printing, distributing, promoting and selling the book bearing the infringing image and that you will recall all unsold books previously distributed by you. In addition, we require a complete accounting of all infringing copies printed and sold by you and all your distributors, including remaining inventory. In order to resolve our claims. If you do not provide us with this information before Tuesday, November 14, 1995, we will have no choice than to file suit against you and those other individuals who knowingly participated in the infringement, seeking copyright damages, costs and our fees. We will also seek an injunction against you and your vendors to prevent any further harm to our client.

We trust you will ensure that this matter is resolved now, while it still can be. In this manner, we may all pursue more interesting endeavors than will be the case otherwise. The alternative is for us to file a suit in which we will most certainly prevail. And, in addition to the judgment, you will be paying for our fees at the prevailing rate.

Sincerely

Paul A. Mahon

From Al Canton's perspective, this maximum surprise-element situation clearly threatened to devastate, if not bankrupt, his publishing company and himself. Resolving the matter was very difficult and Al Canton eventually mounted a public Internet battle for his defense, finally paying $500 for the rights to use the image and issuing a closure statement drafted by Vietnam Women's Memorial Project, Inc. This sum was what Al Canton had offered to pay immediately after receiving the first facsimile.

In retrospect, it never occurred to Jane Schweitzer or Al Canton that a sculpture on public National Park Service property that did not display an obvious copyright notice would be copyrighted. If you are similarly surprised, you are in good company. Regardless, Paul A. Mahon's letter provides one example of how you may be treated by opposing lawyers.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Andrew Leicester vs. Warner Brothers Studios

On June 16, 1995, Gregory B. Wood of the Los Angeles law firm Merchant, Gould, Smith, Edell, Welter, & Schmidt filed a Complaint for Copyright Infringement, Unfair Competition, Dilution, Violation of Cultural Affairs Laws, and Interference with Prospective Business Relations in the United States District Court for the Central District of California.

The 14-page Demand for Jury Trial suggests that Andrew Leicester and his attorneys believe Warner Brothers Studios infringed Andrew Leicester's artistic work. Among many things, the document alleges:

  • Mr. Leicester is an internationally recognized environmental artist.
  • In 1992 he created an original $2 million outdoor artwork titled Zanja Madre.
  • Mr. Leicester retained all copyrights for the artwork.
  • Zanja Madre is publicly displayed in downtown Los Angeles and has won several awards.

The filing also alleges:

  • In mid-1995, Warner Brothers released its Batman Forever feature which takes place in the "newly conceived" futuristic world of "Gotham City."
  • The "newly conceived" Gotham City is a central the [sic] focal point of both the movie itself and the considerable pre-release promotional activities, materials, and literature.
  • In order to create the desired new identity for the new "Gotham City," Warner, without authorization, misappropriated and copied Mr. Leicester's work including making detailed and virtually identical scale model reproductions of Zanja Madre and used those scale model reproductions in the filming.
  • Warner also filmed Zanja Madre for the specific purpose of incorporating Mr. Leicester's work into Batman Forever and using that work as the defining appearance of and for the purpose of identifying the new "Gotham City."
  • Warner filmed multiple scenes of Batman Forever at the Zanja Madre location, and prominently displayed Mr. Leicester's work in those scenes as the new Gotham City.

Warner Brothers would presumably resent anyone infringing their copyrights by copying their copyrighted materials such as the Batman Forever video. One need only read the copyright and associated penalty statement the video contains. So, it is somewhat ironic they find themselves in this situation. And, the stakes are very high since the filing alleges:

  • Scenes of the new "Gotham City" are actually unauthorized reproductions of Zanja Madre.
  • Warner has corrupted Mr. Leicester's work by deliberately portraying Gotham City, including the copied Zanja Madre work, in a "comic book" fashion.
  • The depiction of Mr. Leicester's work in Batman Forever degrades a work intended as an allegorical landscape exploring the relationship of a city to its water resources to a comic book setting which, from the opening scenes and continuing throughout the film, is the focal point for violent crime.

Meanwhile, the filing contends in part:

  • Warner has, furthermore, been unjustly enriched by the copying of Mr. Leicester's work in an amount which is uncertain, but which shall be proven at trial.
  • Mr. Leicester will suffer serious and irreparable damage to his reputation an [sic] the reputation of his artistic creations, if Warner's copyright infringing activities are not enjoined.

The stakes in this matter are very high since an estimated 3,300 theaters were showing the feature a combined total of 92,000 times a week at the feature's release. Over time, Batman Forever enjoyed an estimated 1,000,000+ showings and each showing could be argued as a copyright infringement, individually worth $500 to $100,000 in damages. The math is as simple as it is staggering and left as a reader exercise. If Mr. Leicester prevails, he could possibly own Warner Brothers and the legal department that opposed him.

While the trial proceedings have concluded, the judge has not released his complete findings as of this writing. In the final analysis, this may prove very expensive for Warner Brothers. Neal Showman, professor of copyright law at William Mitchell College of Law, is cited as indicating that having permission from building owners adjacent to Zanja Madre is not sufficient grounds to proceed filming the sculpture. Additional information is available at:

  • http://www.gag.org/gnews_16.html
  • http://www.benedict.com/batman.htm#batman

Finally in February 1998, a federal judge was considering delaying the release of Warner Brothers' Devil's Advocate videotape. The footage reportedly contains images of a copyrighted religious statue which "comes to life and writhes erotically." The sculptor's attorneys are claiming the film is an unfair use of the sculpture. Warner Brothers is citing its first amendment rights.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Rock and Roll Hall of Fame & Museum

On July 16, 1996, the New York Times front page reported that the Rock and Roll Hall of Fame & Museum had sued a photographer, Charles M. Gentile of Cleveland, Ohio, for infringing its trademark. Mr. Gentile was selling posters of the Rock and Roll Hall of Fame & Museum created from a photograph he had taken.

Mr. Gentile had produced many posters of other subjects before without a problem. Unfortunately, the distinctive Rock and Roll Hall of Fame & Museum's architecture was trademarked and the museum owners had taken specific measures to obtain the trademark from the building's designer, I.M. Pei, to avoid similar problems.

Usually, trademarks prevent enterprises from copying designs developed by other enterprises—such as McDonald's golden arches. Owning the building trademark, therefore, allows the museum owners to generate protected revenues from posters, mugs, hats, T-shirts, etc., that display the distinctive building shape. These revenues help pay back the public bonds used to finance the Rock and Roll Hall of Fame & Museum.

Make no mistake about it, the museum had a very strong case. A U.S. District Court issued a preliminary injunction that required Mr. Gentile to collect all unsold posters and deliver them to an opposing lawyer "for destruction," which the photographer subsequently appealed.

Additional information can be found at:

  • http://essential.org/listproc/cpt-ip/0219.html
  • http://132.235.238.184/archives/050296/rock.html
  • http://www.universe.digex.net/~danielob/dan.html
  • http://target.ndgphoenix.com/asmp/newslines.html#200
  • http://www.tmn.com/Artswire/www/current/current.html
  • http://www.tv.cbc.ca/undercurrents/week07/copyright/one.html
  • http://www.arlington.net/news/doc/1047/1:NATIONC/1:NATIONC061596.html

How many photographs do you have of interesting settings? Do any contain trademarked or copyrighted structures? How can you be sure? Where is the line between copyright and trademark infringement? If you are not certain, lawyers will graciously help you find out, at prevailing rates, of course.

Other Considerations

Digital image editing introduces a variety of legal issues. Here are a few more:

  1. From preceding chapters, black and white digital images are quantized analog sample values, arranged as two-dimensional monochromatic intensity value arrays. When transformed into visual spectrum monochromatic intensities, the numeric values collectively produce an achromatic pointillist mosaic that Seurat himself would envy. Human perception pattern matching suggests the mosaic represents something based on a best bet.
    Consider sampling 4-by-5-inch photographs at a vertical and horizontal sampling rate of 100 samples per inch. This produces 200,000 (4 x 100 x 5 x 100) samples, or pixels, total. If each sample has one of 256 (28) values, there are 28*200,000 or 21,600,000 different possible images for this image size at this sampling rate.
    Admittedly, this is a very large number. But, it is finite and while any possible single image is verifiably different from any other, the image may be visually indistinguishable from trillions of other images. Who decides image ownership and which other images are visual plagiarism because they are subjectively indistinguishable from a claimed image? It is a good bet they will be well paid for their services.

    More basically,

    • Who owns the number p? How about "8" and the individual achromatic intensity sample it represents?
    • Who owns sample value sequences constructed from the achromatic intensity value set {0, 1, 2,.... 255}?
    • Finally, who owns achromatic sample value patterns like:
      8 4 6 2 ...
      9 3 5 5 ...
      0 2 4 4 ...
      2 9 5 6 ...
      ... ... ... ... ...
  2. What constitutes fair image use? Suppose you:
    • Obtain a copyrighted digital image someone else created, perhaps from the WWW, and alter it, maybe by changing a single sample value. Who owns the new derivative image? Answer: the original copyright owner; the derivative artist only owns the derivative changes. What if you change two sample values, etc.? Same answer until you get "far enough" away, variously defined. It is important to know that Digimarc, Inc. of Portland, Oregon, provides a nearly invisible way to "fingerprint" digital images, allowing their owners to identify them despite alterations.
    • Obtain a copyrighted digital image someone else created, remove an image fragment, and use the fragment again in the same image. Is this a fair use of the image? No, it is not. Who owns the new derivative image? Answer: the original copyright owner(s).
    • Remove an image fragment from one copyrighted image and use it in another copyrighted image. Who owns the new derivative image? Not you. The other two image creators? Some combination? Yes. Is there a copyright infringement? Yes.
    • Create a composite image by inserting an image fragment of models into a gambling casino image. Can the models sue and win if they abhor gambling and feel you defamed them by suggesting they gamble, thereby portraying them in a false light? Possibly so.

In an actual incident, considered as false light representation by many civil rights leaders, the San Jose Mercury News reported that Phillip Jones, general manager of Dr. Martin Luther King's estate, stated Dr. King's famous "I Have a Dream" speech is copyrighted and requested that a political campaign commercial promoting a ballot measure that threatened affirmative action rights in California "cease and desist" using a video recording of it. Mr. Phillips commented, "We do not allow use of Dr. King's intellectual property in political campaigns." Many folks suspected it was true particularly for that one.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Copyright Primer

A U.S. copyright protects published and unpublished literary, dramatic, musical, artistic, and other works—including the way the creator expresses the work's contents and ideas. Copyrights grant their owners limited exclusive rights to perform or display works publicly, reproduce and distribute copies, and create derivative works from an original.

Any work created, published, or registered after December 31, 1977, enjoys automatic copyright protection. The protection expires 50 years after the author's death. All other works created and published or registered before January 1, 1978, and that were still under copyright protection on that date, are copyrighted for 75 years from their creation, publish, or registration date. When U.S. copyrights expire, the work enters the public domain and anyone may use it without permission. However, in some countries, copyrights do not expire.

To register a work in the U.S., send one or more copies of the work with a completed application and $20 U.S. to:

The Copyright Office
Library of Congress
Washington, D.C. 20559

Fair Use Primer

Fair use allows using portions of copyrighted works without permission for such purposes as research, academic teaching and study, news reports, commentary, and other criticism. Four principal criteria determine whether copying falls within the fair use doctrine:

  1. What is the nature of the original (e.g. published or unpublished)?
  2. How important to the original work is the portion used and what total amount is used?
  3. Is the use intended for commercial or nonprofit purposes?
  4. How does the use impact the original work's value?

    When it comes to digital imaging, many questions are unanswered. For an interesting commentary, see:

    • http://www.ljx.com/public/mailinglists/cni-copyright/threadex.html

Moral Rights

In 1994, Donna A. Demac composed a copyright-issues white paper for the Working Group for the Digital Image at the International Museum of Photography at George Eastman House. In this paper, she noted the emerging U.S. legal trend to protect noncommercial rights of artistic work. Recent U.S. laws increasingly protect the integrity of artistic work, the rights of which are never transferrable to customers or heirs. In contrast, European law has long asserted moral rights that affirm creators have inherent, inalienable rights stemming from their creations. These rights cannot be sold and cannot be inherited by heirs. The international Berne copyright treaty explicitly includes moral rights.

The federal U.S. Copyright Act, Section 106A—also known as the 1990 Visual Artists Rights Act (VARA)—bestows moral rights within the United States. This act focuses on the moral rights of attribution and integrity for works such as paintings, drawings, prints, still photographs, and sculptures created on or after May 31, 1991.

Attribution rights allow artists to assert or disown authorship of a work, and to prevent associating them with a work if it has been subsequently distorted or modified. Integrity rights allow artists to prevent destruction or modification of a work and to receive compensation for destruction when the work enjoys "recognized stature," variously defined. Whether moral rights apply to digitized art presently remains unanswered.

Conclusion

Increasingly, explicit answers to the perplexing questions of image use are found in the contracts required to obtain images. These contracts usually convey sentiment that images are not image-fragment repositories.

However, since digital imaging is so new, many existing movie and image contracts do not contain digital reuse considerations which address such questions. Consequently most multimedia developers create new content, abandoning all hope of using existing cinematic content rather than negotiate the hundreds of contracts necessary for reuse.

Eventually, common sense, cast into copyright law, will answer such questions without having to consult contracts.

E.R. Beardsley is a former dean of the division of fine arts at the University of California, Riverside, and the founder of the university's celebrated California Museum of Photography. He has also been a member of the Working Group for the Digital Image at the International Museum of Photography at George Eastman House. In a private correspondence regarding some of the legal questions entangling contemp- orary copyright and trademark infringement issues he aptly notes:

Until there is better understanding, a more common sense approach to its resolution (which means something other than a strictly "legal" resolution), we stand in danger of complete and utter paralysis. No writer will be able to write. No photographer will be able to photograph. Taken to its logical extreme, nothing can be written or depicted that has not already been written or depicted in some form, and therefore subject to copyright or trademark protection.

... The notion of "fair use" is now much in dispute everywhere, largely because it has been left to the lawyers to define its terms. This is not useful, and it has already put obstacles in the path of the flow of information. Even libraries and museums are unsure about what rights they might or might not own to the images and other materials in their collections. Instead of granting permission to use this or that thing, they more often now simply say "use it at your own risk."

It is difficult to improve upon these succinctly stated views.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Chapter 20
Social Issues

What This Chapter is About

Visual information is essential to all of us because it provides us the majority of our worldly knowledge and understanding. Indeed, visual perception deficiencies can cause considerable property damage and personal injury. In one instance, color blindness precipitated a major 1996 railroad disaster.

By now, it should be painfully obvious that digital images are pixel mosaics that growing numbers of computer users can shuffle and reflow with arbitrary undetectability. Because of this, digital image processing introduces new recombinant realities that the public regards as indistinguishable from photographic "reality."

Motivations for altering images vary. However, in the final analysis you should recognize that image manipulation is considered a particularly heinous practice in any professional or social arena that relies on images to represent truth or fact. Consider that U2 spy plane photographs over Cuba precipitated an international confrontation with Fidel Castro who already, unknown to the United States, possessed a handful of tactical nuclear weapons. Here, image truth was paramount.

There are numerous instances where digital manipulation of photographic images appear to have eroded the credibility of governments, complicated court cases, and weakened public confidence. It would be a shame to allow someone inconvenienced by a Tiananmen Square youth stopping an armored tank by standing in its way to dismiss an authentic image as a fake. David King's The Commissar Vanishes: The Falsification of Photographs and Art in Stalin's Russia indicates how useful Joseph Stalin regarded image alterations.

While your digital images may not possess such portent, it is important to reconsider carefully before manipulating images that you later assert are factual. This chapter discusses a few of the more pressing considerations that are emerging as digital image processing becomes increasingly accessible. Professional photographers have thought about the matter at length. So, it likely serves us well to consider the matters they have grappled with.

Level Set

On February 2, 1989, a top secret Swedish Air Force JAS-39 Gripen fighter aircraft crashed while landing at Lindkoping Airport during its sixth test flight. Most folks would have wanted to witness the dramatic scene, just like the Swedish television camera crew that filmed the action. The sight was unforgettable, with flying dust generated as the plane violently tumbled about before coming to rest. The pilot escaped.

A few hours later, four digital images flashed onto television sets and onto news wires. They were memorable indeed—not just because they accompanied the crash description, but because the images were not from crash footage. They were digital image composites constructed by Alf Jansson from verbal eyewitness reports from the film crew.

Because the Swedish Air Force had temporarily confiscated the crew's film, Alf Jansson created fictional images from earlier test flight footage and a file photo. (If you looked carefully, the sun shone on the side of the wing facing away from the sun.)

To ensure the images would be recognized as human creations, the creators thoughtfully indicated the images were digitally produced and came from computers when they placed them on international news wires. Problem was, in 1989, few folks realized what that meant. So, when the images appeared in several Swedish newspapers in Finland, many readers simply accepted the images as authentic.

Fictional photo illustrations created by Alf Jansson in an age of digital imaging innocence, courtesy of Aktuellt-Sveriges Television AB.

This occurred despite urgent attempts by the television station that created them to get the fictional "photo illustrations" replaced with authentic ones that were available a few hours after the fictional ones were released. The problem was, the fictional images were too plausible and lifelike.

In a nutshell, such situations, unintentional or otherwise, appear to erode photographic credibility, causing problems in various court cases and in numerous other settings. Without a doubt, digital image processing has securely introduced a sharp-edged technology that cuts deeply in many directions. Like all technologies, this sharp edge can be applied in various situations with varying intents. To understand the realities and possibilities for abuse, we begin at photography's inception.

Actual crash images, courtesy of Swedish Television (SVT)


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Photographic Myths

When Louis Daguerre introduced daguerrotype images in 1839, society was thrilled. Gone were the days, it seemed (naively), of fabrication, subjective bias, and spin management (romantic, best-light interpretations) that artists had capriciously imparted in paintings and sketches. Soon, numerous perspectives on the new technology pervaded society. Some were:

  • Seeing is believing.
  • A camera is a mirror with a memory that does not lie. It provides a transparent window on the world, providing supremely accurate reality traces of "fossilized light" that mirrors nature.
  • Photographers, like ancient scribes, are impartial mechanics.
  • There is no human intervention between reality and a photograph. Photography mechanics remove subjectivity, ensuring veracity while banishing error.
  • Photographs are real, objective, and unadorned representations of unvarnished truth and reality.

After all, the new "impartial photon scribes" mechanically captured images without bias. So, it certainly seemed like the "fossilized light" had to mirror reality.

However, as is usually the case, things were never really as they initially seemed because from photography's birth, images were the direct result of human manipulation and intervention.

Photographic Reality

The following is true:

  1. We often, but decreasingly, assume photos are authentic and have not been manipulated. The assumption varies perhaps depending on, say, whether we see images in LIFE or Time magazine versus an image in a tabloid accompanying an article titled "Chocoholic Mother Gives Birth to Sugar-Coated Baby"that, at distance, may seem somewhat unreliable, curiously implausible, or otherwise suspect.
  2. Photographs seem to possess innate veracity and are extraordinarily persuasive because they seem lifelike. Photographic capacity to produce exact likeness holds persuasive power and credibility difficult for written words to rival. Photographs seem to supply incontrovertible proof, enjoying powerful presumptions of, and privileged connections to, reality. Here's one famous example:

    On January 4, 1989, Vernon Walters, United States Ambassador to the United Nations, squelched criticism of the United States downing two Libyan MiG-23 fighters. He used an extremely blurry, virtually unrecognizable photo, supposedly proving the enemy fighters were armed with air-to-air missiles. Here, the mere presence of a photograph, seemingly any photograph, provided immense, though perhaps unwarranted, credibility.

    The power of photography is incontestable and color photographs are often even more persuasive than black and white photographs. Color advertisements, including those with color photographs, can generate 43 percent more sales, prompting the New York Times to introduce permanent color pages in its June 5, 1993, Sunday issue.

  3. Image contents can redefine peoples' perceptions and provide powerful arousal. Here are a few examples:
    • After press coverage largely precipitated adverse Vietnam War public reactions, the U.S. military prevented news agencies from directly photographing Operation Desert Storm operations.
    • Controlling image collections is sometimes politically important. A sizable uproar occurred after a California museum released its archive microfilm copies of the Dead Sea Scrolls to researchers in violation of its agreement—following years of frustrating glacial interpretation efforts by the artifact owners. Some suspected the owners subscribed to Ptolemaic self interest.
    • The O.J. Simpson defense team fought against allowing the jury to see autopsy photographs because they felt the images would coalesce jury anger towards the defendant. Reports indicated nothing prepared viewers for what they saw.
  4. Photographic development and reproduction requires substantial manual intervention. Cameras have lens filters; varying film exposure, focus, and depth of field settings; and films with varying response characteristics. Film development procedures vary, and final print efforts may involve contrast enhancement, cropping, color correction, burning and dodging operations, and subjective tonal manipulation that hardly renders an image faithfully or replicates on print paper "what the photographer saw." Finally, the printing paper and its development procedures vary as well. All these varying characteristics may be, and historically have been, used "unfaithfully" to render false images. Finally, because of subtractive color complications, ink bleed, and adhesion problems, enhancement manipulations for print production is an absolute requirement.
  5. Like all news reporting, photographs usually contain ambiguity that requires significant text to explain—captions, for instance. Consequently, photography has always had a tentative relationship with reality that remains highly interpretative. Many photographs have been staged photo opportunities—mere rehearsed orchestrations.
  6. Photographers have manipulated images since photography's infancy. More aggressive practitioners resorted to airbrushing, slicing out the photograph's center and moving the end pieces together, object removal and addition, and negative flopping (reversal). But early manipulation was usually tedious, noticeable, or otherwise implausible. Moving obstructive microphones off desks previously required significant compositing and airbrushing skill. Finally, adult magazines have vigorously airbrushed centerfold images for decades.

Photographs are not reality. They are simply two-dimensional dye or silver halide arrangements symbolizing a time-space sample which the human perception system interprets with varying success to produce best-guess associations. After all, ink-stained paper does not remotely resemble reality.

It follows that, for decades, it has been ill-advised to accept photos blindly at face value—just like many other things in life. While photography may not lie, liars have indeed used photography to perpetrate deceptions. And, digital image processing provides an even sharper tool to do so.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Digital Image Processing—Reality's Solvent

Traditional photography is experiencing a relentless assault that has already transformed the profession. Like other mass communication technologies, some folks suggest digital imaging's evolution is outstripping society's ability to ensure truth in content. It's not that film photography has not had its share of fraud and forgery. It's just that digital imaging is now so pervasive and provides immense precision to move, eliminate, add, alter, and shuffle image content virtually undetected and without trace.

Since digital imaging often eliminates negatives and paper, there is no archive negative, print trail, fingerprints, or footprints to verify authenticity. And, it's a foregone conclusion that tampering with photographs is now inevitable. Consider the Hollywood term zipper head where a star's face is pasted on an alluring model's body to create stunning movie posters. One famous 1989 TV Guide cover image fabrication reportedly involved placing Oprah Winfrey's head on a very detailed sketch of Ann Margret's body—detected only when Ann Margret's husband identified Ann's unique ring on one finger. Now, the same technology, once available only to Hollywood special effects studios and elite advertising agencies, is pervasive—producing equivalent illusion perfection.

To demonstrate this technology's dramatic potential, Fred Ritchin, a New York University Tisch School of the Arts professor of photography, visited the Scitex America headquarters in Bedford, Massachusetts. Before departing that day, he and the Scitex staff created a convincing composite color image for a November 4, 1984, New York Times image manipulation article by combining images of:

  • the Manhattan East River skyline
  • the Eiffel Tower
  • the Statue of Liberty
  • San Francisco's Transamerica Pyramid

In the process, the team digitally:

  • Increased the Empire State Building's height and moved it several blocks
  • Removed the Chrysler Building
  • Added an East River pier
  • Reversed the Citicorp Building
  • Removed a smokestack
  • Added traffic around the Eiffel Tower
  • Removed buildings wherever desired

This was an academic exercise. On the positive side, digital imaging techniques allow frantic families to construct likely facial appearances of long-kidnapped children for posters. It shows sun-worshipers how the sun may age their skin. But on numerous other fronts, some folks feel technology practitioners are out of control. A quick Mead Data Central NEXIS full text search inquiry requesting references discussing digital imaging and ethics produced dozens of citations discussing the subject. The National Press Photographers Association is mentioned many times because it became concerned in the late 1980s regarding the dangers of fabricated news images. Here are a few examples of what the group is concerned about:

  • Christopher Harris, professor of photojournalism and director of Middle Tennessee State's Digital Imaging Lab, contends the Orange County Register won a Pulitzer Prize with a digitally altered Olympic image that turned the sky blue. This newspaper reportedly has a long history of past "reality enhancements" including closing a young celebrity boy's open zipper in a photograph and removing red and yellow (i.e., smog) wavelengths from the sky—providing sparkling "Register sky blue" skies. The Orange County Register seemingly abides by its slogan "We're on Orange County's side" with little probable opposition from local chambers of commerce.
  • John Long, past NPPA president, reported an NPPA Press Clip contest winnercalled to ask whether he should send the out-of-focus original photograph to the News Photographer Magazine for contest result publication or the digitally sharpened one that won the contest. This situation alerted the NPPA to potential sharpening ethical abuses which never existed until digital image processing introduced them.

There is some real danger here, and technology's advance will soon make it possible to fabricate video footage in real time. Don Tomlinson, associate professor of journalism at Texas A&M, has coined a word for this development—digitexed—where digital images are created and passed off as authentic. He reports that Tom Pettit, NBC News vice president, observed, "...With the highly sophisticated editing of videotape, we can reshape reality with great ease." Walter Porges of ABC News Practices observed, "...I can create pictures and sounds...that never happened. And that creates all kinds of ethical...problems for everyone."

Major television networks deny fabricating video footage. But, in 1989, ABC was embarrassed after airing an electronically manipulated staged photograph of one man passing another a briefcase. The individual receiving the briefcase supposedly appeared somewhat like Felix Block, a diplomat suspected of espionage. After the nature of this image was revealed by others, ABC stated its inadvertent failure to identify the image as a "simulation" was an oversight. Ben Blank, ABC News art director, then reportedly halted routine image manipulation to, for example, fix rumpled jacket creases.

But some digital manipulation motivations are completely transparent. In 1990, Massachusetts gubernatorial candidate John Silber was videotaped railing against working mothers. His political opposition obtained a videotape of the outburst, reportedly manipulated the footage by distorting Silber's image to make him appear more menacing, and used it in negative television campaign ads. Another example of alleged image manipulation occurred in Atlanta, Georgia, where a city council candidate's ethnic features were reportedly accentuated. A libel suit was filed and quietly settled in closed proceedings. In September 1997, a former female student at the Citadel filed a harassment suit against the school alleging, in part, that male cadets digitally altered an image of her and posted the obscene image on the Internet.

Unaltered image of Gordon Joyner, candidate for Atlanta City Council.

Digitally altered image produced by political opponents


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Photographic Impact Examples

Regardless of numerous vulnerabilities, photographs continue to refine peoples' perception of the world. Despite periodic incidents, society continues to consider photographs real and representational, so they enjoy powerful psychological power. Consider a few authentic and forged photographic impact examples:

  • In 1951, during the McCarthy era, a crude photographic forgery that composited a photograph of U.S. Senator Millard Tydings and another of communist leader Earl Browder likely cost Tydings the following election.
  • A more recent example of creative image mutation occurred in the 1996 Virginia U.S. Senate election. There, a campaign image appeared with the Democratic challenger's head pasted onto the body of another person who was shaking hands with the former Virginia governor and President Clinton two years earlier. After discovery, the Republican candidate blamed a media consultant and deleted the image from the commercial.
  • Presidential candidate Gary Hart withdrew from the 1988 presidential campaign after a photograph of Donna Rice sitting on his lap aboard the boat Monkey Business publicly circulated.
  • Presidential candidate H. Ross Perot withdrew from the 1992 presidential campaign, asserting he was doing so in part because he learned political adversaries were going to circulate a compromising image forgery exploiting his daughter's face at her forthcoming wedding, hoping to disrupt the festivities.
  • Chapter 22 of Anthony Summers's Official and Confidential—The Secret Life of J. Edgar Hoover suggests both the CIA and Mafia possessed pictures of J. Edgar Hoover engaged in a compromising activity with his male assistant, Clyde Tolson. The text and the television show Frontline (#1111 aired February 9, 1993) go on to suggest that various interests used these photographs to blackmail J. Edgar Hoover, directly motivating his inept pursuit of the Mafia for decades.
  • The Chinese government denied the Tiananmen Square massacre occurred, but the world disagreed based on photographic and videotape evidence.
  • The Los Angeles Police Department dismissed police brutality complaints for years but were unable to dismiss the 1991 Rodney King videotape. When the responsible officers were not convicted, the Los Angeles riots began.
  • Kurt Waldheim only admitted to being a Nazi officer after being confronted in 1986 with a photograph of him in uniform.

So, it follows that photographs have enjoyed immense persuasive power because they "seem so lifelike." But digital imaging fundamentally alters photographic procedures.In the process, it revises society's relationship to the photograph.

Compositing digital images introduces time and space recombinancy that allows images to achieve textual malleability. Rather than describe world leader meetings, it's now easy to "illustrate" one from image fragments composited into an image. Some people argue there is little difference in doing this since textual descriptions are traditionally based on second-hand accounts subjected to many edits. Now it seems that photography is in the same situation. We have departed the photographic era and have entered a brave new era where alterations abound. Here are a few examples that range from trivial to serious:

  • An unidentified employee at U.S. News and World Report reportedly cleaned Pakistan Prime Minister Benazir Bhutto's fingernails in an image taken after she had been campaigning in dusty Pakistan rural communities.
  • The Christian Science Monitor reportedly once regularly removed cigarettes from subjects' hands, even Winston Churchill's cigar.
  • In July 1994, Time magazine's cover deliberately provided a "photo illustration" of O.J. Simpson starting with his mug shot. Many outraged readers felt the illustration made him appear more sinister than the unaltered mug shot Newsweek published.

Press Ethics

Most print journalists are well aware of the difference between fact or fiction and accurate reporting or fabrication.

  • Janet Cook won a Pulitzer Prize for her tragic Washington Post series on a young ghetto boy. She returned the prize when it was learned the boy was a fictional composite.
  • The press was singularly indignant when it learned White House spokesman Larry Speakes fabricated quotes from Ronald Reagan for public consumption and presidential spin management.

One can conclude the press generally does not like reporting to fool the public. But does it mind fooling the public in images? Aware that clandestine image manipulations were tantamount to fabricating quotes and textual composites, the National Press Photographers Association Board of Directors adopted the following statement of principle on July 2, 1991:

Statement of Principle

As journalists, we believe the guiding principle of our profession is accuracy, therefore, we believe it is wrong to alter the content of a photograph in any way that deceives the public.

As journalists, we have the responsibility to document society and to preserve its images as a matter of historical record. It is clear that the emerging electronic technologies provide new challenges to the integrity of photographic images. This technology enables the manipulation of the content of an image in such a way that the change is virtually undetectable. In light of this we, the National Press Photographers Association, reaffirm the basis of our ethics: accurate representation is the benchmark of our profession.

We believe the photojournalistic guidelines for fair and accurate reporting should be the criteria for judging what may be done electronically to a photograph. Altering the content of a photograph, in any degree, is a breach of the ethical standards recognized by the NPPA.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

This lucid statement was driven in part by moral dilemmas such as those discussed in Fineline, a journalism ethics newsletter:

  • A reporter for a small weekly hockey publication was pressured to remove a manufacturer's name from a hockey stick in an image because a competitor was a prime advertiser and threatened the publisher with canceling the account.
  • When Ron Olshwanger celebrated his Pulitzer Prize with a Diet Coke, the St. Louis Dispatch ironically digitally removed the Diet Coke can from his celebration photo.

Because many periodicals use altered images for non-news items (food sections, etc.), Fineline suggests such images be labeled composite photo, photographic recreation, or photo dramatization. Fuller explanations are acceptable, as well. But photo illustration, feature photo, and cover photo are unacceptable. And the acceptable qualifications need to be in the image caption, not in back-page credit lines. To its credit, the Norwegian Press clearly labels manipulated images with the letter m inside a circle or the word montage. But this simple approach leaves many problems as well.

The NPPA's statement of principle came none too early because the Associated Press was in the process of installing its digital image delivery system, Photo Stream, and editing computers, Leaf Picture Desk, in 60 percent of U.S. daily newspapers. At the time, the technology ship was truly docking. Yet two years later, Time magazine published O.J. Simpson's manipulated image on its cover. This seems truly ironic because Tom Bentkowski, LIFE magazine's director of design, wrote in 1988, "A photograph that represents an actual situation can never be manipulated in good conscience."

Tom Bentkowski had previously been Time magazine's Special Projects manager. His August 20, 1984, issue covered the Olympics. In that issue, John Meyers, Time's publisher, wrote on page 4:

...TIME has premiered a new system for transmitting color photographs that can reduce the time between shutter snap in the Los Angeles Coliseum and finished picture in TIME's U.S. printing plants to as little as 6 1/4 hours. Immediately after an important shot, the film was processed and edited, then flown twelve miles by helicopter to Torrance, Calif., where TIME Inc.'s corporate manufacturing and distribution division had installed a scanner to turn the pictures into digital signals. These were beamed by satellite to New York City, where they were reassembled on a video monitor. After review by TIME's editors, the final pictures were retransmitted to the printing sites. To technical director John Mitchell, the new system is "a significant breakthrough." It also means that TIME readers can see color photographs in their magazines only 36 hours after a news event.

One of the truly impressive images this process produced was of Mary Decker Slaney lying in agony after Zola Budd bumped her, knocking her down and injuring her in the process. A benevolent Olympic official was futilely attempting to assist her up—page 45 of the August 20, 1984, issue has the full image.

This remarkable image later appeared both on the cover of TIME's Fall 1989 special edition, titled 150 Years of Photo Journalism, and inside on page 70—four pages after Richard Lacayo's page 66 observation:

"Now the prospect of computer tinkering threatens to bring a day when the whole enterprise of picture taking will not be so subtly compromised"

(referencing forged authenticity).

A few pages later, the text reads:

"Computer retouching has already become commonplace in the fantasy world of advertising photography. Now many journalists are troubled by the prospect that the practice will creep into the reproduction of news photos. A computer can remake a picture, but only ethical people can draw the ethical line."

This is all true, but the immense irony is that the Mary Decker Slaney image originally published in 1984 had been digitally edited to remove an unsightly, prominent walkie-talkie antenna. So, Time experienced a slice of its worst fears at its own hand at the new process's inauguration.

Finally, there is the problem of exterminating mutant variations of images regardless of the process used. One surprising instance occurred in a May 1995 LIFE magazine feature on the famous Kent State National Guard shooting. LIFE unknowingly used a manipulated version of John Filo's Pulitzer Prize winning photo retrieved from its photo library.

When called to its attention, David Friend, LIFE's director of photography, personally responded, candidly acknowledging the alteration, clearly stating LIFE was completely unaware the image was altered when it used it and that several other publications had used the image for over 20 years before the airbrush manipulation was eventually noticed.

David Friend concluded his response with the statement:

At no time would LIFE's photo, art, or production department intentionally alter a news photograph.

David Friend's full response can be found with an AltaVista search using "John Filo" as the search keyword. At this printing, the text resides at:

http://www.cris.com/~Mppa/ethics.html
http://www.mppa.org/mppa/ethics.html

as well as in an article titled "The Renaissance of Photography"—a Los Angeles SPE Conference keynote address by Pedro Meyer. (Reference http://zonezero.com/ for this article as well as several additional thought-provoking imaging articles.) Clearly, once unleashed, mutant images become truth viruses.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Video Futures

Because video is essentially a series of individual images viewed at high speed, digital imaging also provides the ability to alter, synthesize, and fabricate convincing video footage. Consequently, expect to see:

  • Increasing numbers of videos, such as Jurassic Park, which synthesize elements (e.g., dinosaurs) and insert them in actual, new footage.
  • Current actors inserted in archive footage as was done in Forrest Gump.
  • Long-deceased movie stars revived as digital video fragments as in some Coca-Cola advertisements and rock music videos.
  • Motion estimation software assisted by 360 degree actor image scans to generate ageless actors.
  • Synthespians—totally artificial, digitally created actors for entertainment that surpasses expensive animated productions in quality. Synthespians relieve directors from dealing with temperamental, exorbitantly paid actors who are usually very active profit partners.

We are truly at the start of the digital imaging age as evidenced by various megamergers such as DreamScape. Colorization of black and white movies was simply a fledgling preliminary step to what we will soon experience.

However, note that because digital fabrication is becoming so prevalent, court proceedings are now asking whether videos and photographs are accurate representations, like accurate drawings, of what eyewitnesses saw. Is it possible guilty defendants may escape conviction by dismissing authentic photographic evidence as fabrications?

As an example, Kodak researcher Majid Rabbani led a team at the Sandia National Laboratory Supercomputer Center that digitally enhanced portions of the Rodney King videotape for the federal government. He had to establish the team had not fabricated evidence but had simply visually enhanced what the videotape numerically contained.

Finally, it is worth recalling that Dan Leonard, an O.J. Simpson civil trial defense attorney, stated in his closing arguments, "If you have access to the right equipment, you can fake any photograph." Unfortunately for Mr. Simpson, over two dozen different photos emerged showing him wearing what appeared to be Bruno Magli shoes six weeks after he denied under oath ever wearing shoes like them.

U.S. Military Position

On December 9, 1994, John Deutch, then serving as deputy Secretary of Defense, issued a memo establishing Department of Defense guidelines regarding digital manipulation of official DOD photographic and video images. While generally prohibiting official DOD imagery alteration, the guidelines allow:

  • Dodging, burning, color balancing, spotting, and contrast adjustment to achieve accurate event or object recording
  • Photographic and video image enhancement, exploitation and simulation to support unique cartography, geodesy, intelligence, medical, research and development, scientific, and training requirements, provided these techniques do not misrepresent the original image's subject
  • Obvious photograph masking for specific security or criminal investigation requirements
  • Cropping, editing, or enlarging to isolate, link, or display a photograph or video image part as long as the event or object's facts or circumstances are not misrepresented
  • Digitally converting and/or compressing photographic and video imagery without altering content
  • Post-production enhancement, including animation, digital simulation, graphics, and special effects for education, recruiting, safety and training illustrations, publications, or productions as long as the original image is not misrepresented. It must be clearly and readily apparent from the image's content or accompanying text that the enhanced image is not intended to be an accurate representation of any actual event.

Prohibited manipulation techniques unless clearly marked include:

  • repositioning image elements
  • adding or removing image elements
  • merging multiple image elements into one image element
  • changing image element colors or spatial relationships
  • changing image element size, shape, or physical appearance

Further information can be obtained from:

  • http://www.gn.apc.org/media/manip.html
  • http://www.gn.apc.org/media/j-manip.html


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Conclusion

Digital imaging provides a new and exciting format with far-reaching implications for visually intensive cultures. It provides the potential to blend and blur reality and fiction within creative, historical-revisionist arenas. Sufficient skill impedes determining whether images are accurate renderings, recombinant fantasies, or creative illusions. Because image editing is becoming pervasive, it forces us to recognize some photographic tampering is inevitable. Alterations will range from innocuous enhancements to intentional deceptions.

For convenience and efficiency reasons, darkrooms as we know them are dwindling and may disappear in many cases. So it is difficult to predict the degree to which we can trust images as we do photos that are produced differently. However, it is naive to believe that film-based photography has ever been exempt from manipulation.

Photojournalists and the U.S. military have wrestled with the question of whether altering an image is unethical if it produces a result identical to a different camera angle or time shot—sometimes called a "retroactive photographer repositioning." The military prohibits it but in civilian life, the policy varies by periodical, organization, time, and location.

The post-photographic era dramatically increases image forgery possibilities. Practitioners can easily insert and remove image elements within images, then combine, transform, copy, and distribute the results. Once mutants are distributed, the variants can endure much like a computer virus. Because digital images and their mutations have no negatives, they leave no trail, so their origin and lineage is untraceable.

A natural reaction is to search for defenses to endure the onslaught. One perplexing Columbo television episode depicts a villain who asserts innocence by using a dated and time-stamped police vehicle speed-trap photograph showing him speeding in his automobile. This impartial evidence, directly from the police, seemingly places him miles from the crime scene. Columbo solved the case by noting the driver's nose shadow lay in a different direction than shadows outside the speeding automobile. From this, he deduced a photographic mask of the villain covered the accomplice driver's face.

In this instance a subtle photographic internal inconsistency, the shadow, revealed fraud. Thus, diligent scrutiny is a primary, though diminishing, defense against image manipulation. Another is eternal vigilance, and for this effort, we can thank the photojournalists who are the issue's watchdogs. So in the final analysis, your source of photographic truth remains with the publisher—also your source of written truth.

Like liberty, society can decay incrementally through gradual erosion. But many professional photographers recognize the potentially slippery slope that lies ahead. Trust, truth, liberty, and authenticity are no small matter. Clearly, the bell tolls for all of us as various advocacy groups increasingly question authentic images they consider inconvenient such as those from Nazi death camps.

It is a certainty that many future image fabrications will blatantly promote self-serving agendas designed to engineer perceptions. And, technology advances will make image fabrication increasingly trivial and difficult to detect. Because of its nearly infinite mutability, digital image processing does challenge truth and authenticity. Can you believe what you see? Responsible photojournalists hope so. But, caveat emptor when viewing images of unknown origin or lineage.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Epilogue

Digital Image Processing Importance

Increasingly, sustained employment requires continuous skill expansion. To this end, it is helpful to become familiar with core technologies that are essential to numerous disciplines. Welcome to a premier such technology—digital image processing.

Digital image processing is a remarkable technology that spans numerous application arenas including desktop publishing, video conferencing, multimedia, and countless other emerging applications involving the WWW. Here are a few existing image processing applications that have existed for years:

Entertainment

Movies regularly use digital imaging—witness Titanic, Field of Dreams, Terminator 2, Free Willy, Rising Sun, rock videos, sensational advertisements, etc. Because it is now cost-prohibitive to create extravagant sets used in such movies as Cleopatra and Ben-Hur, directors will eventually construct digital film locations rather than construct elaborate real ones. And, expect artificial actors (Running Man) and actors, real and cartoon, inserted into authentic historic footage (e.g., Roger Rabbit and Forrest Gump). Finally, movie producers can now hire a few extra actors, film them in various sequences or poses, and compose crowd footage at a fraction of previous costs.

Identification

Biometric devices now routinely compare retinal blood vessel patterns to establish positive identity. California's Department of Motor Vehicles local offices digitally capture driver color images. At the end of each week, the local offices mail small computer tapes to the DMV headquarters in Sacramento, which engraves the images on new licenses and uses them to help identify drivers caught by, you guessed it, robot-camera speed traps. Similarly, the Price Costco discount store prints black and white member images directly on membership cards while major credit cards routinely offer color cardholder images for improved user identification.

Virtual Reality

Jet Propulsion Lab's Kevin Hussey obtained Los Angeles basin elevation data from the United States Geological Survey and satellite images from NASA. By combining the elevation and imaging data, he created L.A.—The Movie, a digital video that simulated the view seen flying over the Los Angeles area. He later used the same approach to construct a virtual reality Venus overflight from satellite telemetry data and images from Venus. The same technology has obvious utility for targeting cruise missiles and low-level aerial approach missions.

Exploration

In April and September 1994, space shuttle pilots used a $366 million synthetic-aperture-radar imaging system to map sand-covered ancient stream beds, caravan routes, and Ubar, Oman's 5,000-year-old, octagonal, eight-towered city. Using NASA's Space Radar Laboratory during a single mission, the crew captured 14,000 images at more than 400 locations, mapping 5 percent of the earth's surface. The system can measure elevation changes as well as ocean wave heights. It penetrates clouds, volcanic dust, vegetation, ice, and dry sand—allowing scientists to map dry Sahara stream beds covered for centuries by sand dunes. Meanwhile, researchers used United States Geological Survey data to create a digital earth surface model of North America. By digitally shining a pseudo sun on the surface from an unnatural angle, they located a large, previously undiscovered basin.

What If?

The Westlake, California, Design Imaging Group provides software that allows home owners to preview proposed landscaping and swimming pool appearances before construction. The company's system simply overlays proposed vegetation and pool images on digitized house images.

Styles on Video digitizes customer faces and superimposes different hairstyles on the images. The equipment easily fits on a small push-cart, suitable for locating in mall walkways. Customers can purchase a videotape of several options from hundreds of possibilities.

Finally, amusement park vendors routinely provide attendees with pictures that superimpose their faces on other people's bodies. A few years ago, folks had to settle for inserting their heads in clumsy, cartoon-painted, plywood cut-out face holes. Soon, Styles on Video may seem similarly stilted.

Art

The March 1987 Byte magazine article titled "Digital Image Processing in Art Conservation" (pg. 140) discussed the digital imaging work John F. Asmus performed on a high-resolution, digital scan of Leonardo da Vinci's Mona Lisa. After applying a yellow-green color correction to compensate for aging varnish discoloration, what some art critics thought were green background trees turned out to be blue mountains.

A Bill Gates enterprise is securing digital image distribution rights for artworks. During a television interview, he discussed his plans to provide the ability to customize large screen viewing by artist, style, era, color, type scene, display interval, etc. Naturally, the image selection will suit the people viewing them which will be determined by active badges that transmit the identity of the wearer to computer sensors.

Medicine

Biomedical imaging uses digital image processing to enhance X-rays while MRI and CAT scans employ intensive digital image processing. The University of Colorado's The Visible Human Project has created cross section images of razor-thin, head-to-toe slices of a deceased human male and female. These images are available over the Internet (http://www.ncar.ucar.edu/METASCI/VHP.html). After downloading the images, researchers can assemble three-dimensional images of any body organ or area for study. Hollywood producers have inquired if they can use the images as the basis of future movies.

Counterfeiting

Using digital image capture and elementary software analysis, counterfeiters routinely copy and counterfeit complex textile patterns. Improvements in commercial color copiers are forcing national governments to redesign their paper currencies. Today, store cash register operators routinely test paper money using chemically treated markers.

Conclusion

Clearly, digital image processing is pervasive. It will continue to be an extraordinarily powerful technology since, for better or worse, the visually intensive society we live in is forever immersed in its effects. Even now, some of what we experience is a reality amplification to assist inspection and discovery. In other instances, what we see is specious fiction—a counterfeit recombinant reality produced using a powerful reality solvent in hopes of mesmerizing, cajoling, or deceiving to some end. The determination will usually be left a viewer exercise.


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Appendix A
VESA VBE VGA Programming

After its PS/2 product line introduction, the IBM VGA BIOS and display controller quickly became PC graphics de facto standards. Subsequently, numerous OEM VGA products appeared, each providing IBM VGA BIOS and/or register compatibility by preserving the original VGA I/O mapped registers, CPU address space video buffer location, DAC register I/O address locations and functions, etc. For product differentiation, these products also often implemented VGA extensions, including higher screen resolutions and enhanced color support. However, OEM vendors used different hardware approaches for these extensions and exploiting these various display controller extensions proved formidable.

Typically, only after successful video board identification could software exploit specific VGA extensions. Because no standard mechanism existed to identify video boards, software developers usually resorted to arcane, oblique tests such as reading and testing registers found at unique OEM-specified I/O addresses. Because foolproof testing proved technically difficult, if not impossible, for most application developers, they often abandoned VGA extension exploitation, resorting to lowest-common-denominator VGA services. Consequently, end users purchased Super VGA display controller cards and software that exhibited no capabilities beyond standard VGA capabilities.

VESA's VBE standard directly addresses this situation. It provides a single uniform software interface to Super VGA display controllers even when they have vastly different extension designs. Hiding these hardware differences allows software to exploit varying VGA extensions using a well-defined, single, uniform approach. Meanwhile, for OEM vendors, the VBE preserves maximum standard VGA compatibility and minimizes existing VGA BIOS changes and allows RAM as well as ROM-based implementations.

Specifically, the VBE addresses:

  • Video board inquiries
  • Video hardware initialization and programming

Video Board Inquiries

VBE functions return system level information as well as specific video mode details.

  • VBE Function 0x00, Return Super VGA Information, returns general system level information, including an OEM identification string and a pointer to supported video mode information.
  • VBE Function 0x01, Return Super VGA Mode Information, returns information about each supported video mode.
  • VBE Function 0x03, Return Current Video Mode, returns the current video mode.

Video Hardware Initialization and Programming

VBE provides several functions to exploit varying Super VGA hardware implementations.

  • VBE Function 0x02, Set Super VGA Video Mode, isolates applications from video mode initialization details.
  • VBE Function 0x04, Save/Restore Super VGA Video State, saves and restores Super VGA states without knowing anything of the specific implementation.
  • VBE Function 0x05, CPU Video Memory Window Control, provides a memory mapping hardware interface.

Finally, VBE is fully upwards compatible so specific VBE version features are guaranteed to work in future versions. Hence, VBE provides a much needed stable video application programming interface (API).

Frame Buffer Granule (Bank) Switching

Standard VGA video boards provide up to 256 KBytes of frame buffer memory and well-defined frame buffer access mechanisms. Several VBE modes require additional memory but compatibility necessarily restricts frame buffer memory CPU visibility to the standard VGA address range. Frame buffer bank switching provides a mechanism for CPUs to access extended frame buffer memory within standard VGA address space range constraints.

Specifically, software invokes VBE frame buffer bank switching functions to swap (map) video memory granules into the narrow VGA address range for frame buffer memory read and write operations. Software determines the precise way to accomplish this by examining the information VBE Function 0x01 (Return Super VGA Mode Information) returns. Readers not wanting to understand the precise mechanics should note the sample programs contain functions that accomplish this manipulation.

Frame Buffer CPU Visibility Considerations

The standard CPU memory address range typically allotted to VGA display cards is 64 or 128 Kbytes, beginning at CPU memory segment 0xA000. This designated area, reserved for software VGA adapter frame buffer access operations, is referred to as the CPU video memory window. Mapping VGA frame buffer memory portions into the standard CPU video memory window using standard VGA program calls gives access to the entire standard VGA's frame buffer memory which can be up to 256 KBytes in size.

Standard VGA BIOS functions allow software to specify the frame buffer memory address range to appear in the CPU address space range for subsequent software memory read and write activities. The trick then is to swap (map) a given scan line into the CPU address space subject to display adapter hardware constraints so software can read/write all or part of the scan line as necessary. A VBE display adapter achieves this by dividing up frame buffer memory into fixed size granules. A granule contains multiple display adapter scan lines and, depending on the CPU video memory window size, a window may display more than one granule.

Different VBE display adapters may have different-sized granules, but a given adapter has only one size referred to as the window mapping function granularity. Some adapters have 4 KByte granules while others have up to 64 KByte granules. The essential concept is that VBE Function 0x05, CPU Video Memory Window Control, makes desired frame buffer memory granules appear in the CPU's 64 or 128 KByte VGA adapter address range under software control. This book's sample programs only access one scan line at a time, greatly simplifying the code in exchange for performance penalties. Unfortunately, in some instances, scan lines may span two adjacent granules, requiring software bounds testing for proper scan-line access in those situations where only a scan-line portion is visible in the CPU video memory window.

Window A and Window B

VBE allows adapters to divide the CPU video memory into two separate windows referred to as Window A and Window B. When a display adapter supports two VBE windows, this guarantees two different granules can simultaneously appear within the CPU's video memory window. VBE Function 0x01 (Return Super VGA Mode Information) returns indicators for each window's CPU video memory window starting address and whether CPU software can read or write to each window.

When CPU software can only read one VBE window and write to the other, the software can copy from one window into another without repeated VBE Function 0x05 (CPU Video Memory Window Control) calls. In such instances, software repetitively reads scan-line data from one granule in one VBE window and writes scan-line data into the other VBE window's granule. Otherwise, software would swap in the read granule, read data from it into main memory, swap in the write granule, and copy the data from main memory. This constant swapping and double data movement imposes significant performance penalties.

Finally, both VBE windows always appear in different CPU video memory window areas because each VBE window has its own starting memory segment address. When software swaps a granule into a window, video controller hardware aligns the granule's first byte at the window's starting segment address.

Hardware Window Type Summary

Independent of granule size differences, VBE supports three basic hardware windowing implementation types:

  • Single window systems—provide a single readable and writable VBE window
  • Dual window systems—provide two windows in one of two configurations:

    1. Non-overlapping windows—the video adapter provides two VBE windows that occupy different CPU memory ranges. In this arrangement, CPU software can read and write to each VBE window.
    2. Overlapping windows—video adapter hardware designates a window as read-only and the other as write-only. The window starting addresses are the same and the display adapter determines which window a frame buffer access applies to depending on whether the access is a CPU memory read or a write operation. In this instance, the starting segment addresses for each window may be identical.

VBE Function 0x05 (CPU Video Memory Window Control) provides the information necessary for applications to detect the specific hardware implementation type an adapter provides and the read/write characteristics of each VBE window in dual window systems.

Super VGA Mode Number Overview

Standard VGA mode numbers reserve seven bit positions and range in value from 0x00 to 0x13. OEMs extend video mode range values from 0x14 to 0x7F. Because the standard VGA BIOS Function 0x00 (Set Video Mode) interprets bit 7 as a clear/don't clear video memory flag, values ranging from 0x80 to 0xFF are not available to VBE. Therefore, VBE video mode numbers are 15 bits wide, beginning at 0x0100.

This mode numbering scheme incorporates standard VGA mode numbers and OEM-defined mode numbers as VESA mode-number subsets, allowing programs to initialize regular VGA modes using VBE Function 0x02 (Set Super VGA mode).

VESA Mode Number Format

Bit Range | Meaning | Comment
D8-D0 | VBE Mode Indicator and Mode Value | D8 == 1 indicates this is a VESA defined mode D8 == 0 indicates this is not a VBE defined mode
D15  D14  D13  D12  D11  D10  D9  D8   D7  D6  D5  D4  D3  D2  D1  D0
 X   X    X   X    X    X    X   X . X   X  X   X   X   X  X   X
D15-D9 Reserved Reserved

where D0 is the low-order bit and D15 is the high-order bit as illustrated:

VESA 256-Color Packed Pixel Mode Numbers

VBE defines the following packed pixel modes that allow 256 maximum on-screen colors:

VBE Packed Pixel Mode Number | Resolution (Cols x Rows)
0x100 | 640x400
0x101 | 640x480
0x103 | 800x600
0x105 | 1024x768
0x107 | 1280x1024

VESA Direct Color Mode Numbers

VBE supports five direct color screen resolutions:

  • 320x200
  • 640x480
  • 800x600
  • 1024x768
  • 1280x1024

VBE defines three pixel bit assignments for each of the five resolutions, for a total of 15 different modes. This allows each resolution to potentially provide the following different numbers of simultaneous colors:

  • 32,768 (32x1024, or 32K)
  • 65,536 (64x1024, or 64K)
  • 16,777,216 (16x1024x1024, or 16M)

Display adapters must have enough memory to support the given modes. The pixel bit assignments for each of the three VBE defined direct color modes are:

Total # Colors | # Rsvd Bits | # Red Bits | # Green Bits | # Blue Bits | Bit Assgt. Notation | Total Color Bits | Comment
32K | 1 | 5 | 5 | 5 | (1:5:5:5) | 15 | 15-bit color
64K | 0 | 5 | 6 | 5 | (5:6:5) | 16 | 16-bit color
16M | 0 | 8 | 8 | 8 | (8:8:8) | 24 | 24-bit color

The following table specifies the 15 VBE direct color modes and their resolutions and bit assignments:

VBE Direct Color Mode Number | Resolution (Cols x Rows) | Bit Assignment | Number of Colors
0x10D | 320x200 | (1:5:5:5) | 32K
0x10E | 320x200 | (5:6:5) | 64K
0x10F | 320x200 | (8:8:8) | 16.8M
0x110 | 640x480 | (1:5:5:5) | 32K
0x111 | 640x480 | (5:6:5) | 64K
0x112 | 640x480 | (8:8:8) | 16.8M
0x113 | 800x600 | (1:5:5:5) | 32K
0x114 | 800x600 | (5:6:5) | 64K
0x115 | 800x600 | (8:8:8) | 16.8M
0x116 | 1024x768 | (1:5:5:5) | 32K
0x117 | 1024x768 | (5:6:5) | 64K
0x118 | 1024x768 | (8:8:8) | 16.8M
0x119 | 1280x1024 | (1:5:5:5) | 32K
0x11A | 1280x1024 | (5:6:5) | 64K
0x11B | 1280x1024 | (8:8:8) | 16.8M

Finally, VBE provides a very flexible way for software to determine other direct color operating modes by using the RedMaskSize, RedFieldPosition, GreenMaskSize, GreenFieldPosition, BlueMaskSize, and BlueFieldPosition values VBE Function 0x01 (Return Super VGA Mode Information) returns.

Extended VGA BIOS Calls

VBE defines several new video function calls. For maximum standard VGA BIOS compatibility, these calls are grouped under one function number. The AH register passes this number, 0x4F, to the video int 10h handler. Standard VGA BIOSs do not recognize this value and ignore these calls.

1. VBE Function Return Status Information

The AX register returns status information for every VBE function call. The status word format follows:

Register Value | Value Meaning
AH == 0x00 | Function call successful
AH == 0x01 | Function call failed
AL == 0x4F | VBE function is supported
AL != 0x4F | VBE function is not supported

Treat non-zero AH register values as general failure conditions, rather than testing for the specific 0x01 value.

2. VBE Function 0x00—Return Super VGA Information

This function returns Super VGA capability information by completing a 256-byte structure located at a caller specified address.

Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x00 | Return Super VGA information
ES:DI | ES:DI | Pointer to information structure

Output: | AX: VBE status value
Comment: | Preserves all other registers

The information structure follows:

#define USGI unsigned int
#define USGC unsigned char
#define USGL unsigned long

struct VesaInfoBlock
    {
        USGC  VesaSignature[4]; /* 'VESA'  */
        USGI  VesaVersion;      /* e.g. 0x0102 == 1.2    */
        char far *OemStringPtr; /* who dat?   */
        USGC  Capabilities[4];
        USGI far *VideoModePtr;
        USGI  Total64kMemoryBlocks;
        char  Reserved[256];    /* more than enough space */
    };

After successful completion:

VESASignature contains the characters "V," "E," "S," and "A" for valid blocks.

VESAVersion specifies the VBE support level provided. The high-order byte specifies the major version number, and the low-order byte specifies the minor version number. Thus a 0x0102 value specifies VBE version 1.2.

OEMStringPtr is a far pointer to a null-terminated OEM-provided string. The string may identify the video chip, video board, memory configuration, etc. The string's format has no restrictions.

Capabilities describes what features the video adapter supports. The bit definition follows:

Bit Position | Meaning | Comment
D0 | Is DAC switchable? | D0 == 1 indicates the DAC width is switchable D0 == 0 indicates the DAC width is fixed to 6 bits per primary colorx
D31-D1 | Reserved | Reserved

VideoModePtr points to a Super VGA (VESA-defined and OEM-specific) supported mode list placed within ROM or RAM. A –1 (0xFFFF) terminates the mode number list and each enumerated mode number occupies one word (16 bits). Either the list is a static ROM string, or the list is generated and placed at run time in the RAM information block (see above). Software must verify the display adapter supports given modes using VBE Function 0x01 Return Super VGA Mode Information. Some enumerated modes may not be available because of video board frame buffer memory limitations and monitor characteristics.

TotalMemory returns the display adapter frame buffer size expressed in 64 KByte blocks. Multiplying this value by 65,536 yields the frame buffer size in bytes.

3. VBE Function 0x01—Return Super VGA Mode Information

This function returns information about specific Super VGA video modes, including OEM-unique, non-VESA video modes, returned by VBE Function 0x00 (Return Super VGA Information). This function returns information by completing a 256-byte maximum-size mode information structure located at a caller-specified address. To provide mode access, this function also returns additional mode information. Finally, pre-initialize the 256-byte buffer to zero before invoking Return Super VGA Mode Information.

Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x01 | Return Super VGA mode information
CX | Mode Value | Must be returned by VBE Function 0x00
ES:DI | ES:DI | Pointer to 256-byte structure

Output: | AX: VBE status value
Comment: | Preserves all other registers

The mode information block has the following structure:

#define USGI unsigned int
#define USGC unsigned char
#define USGL unsigned long

struct VesaModeInfoBlock
    {
        USGI ModeAttributes;
        USGC WinAAttributes;
        USGC WinBAttributes;
        USGI WinGranularity;
        USGI WinSize;
        USGI WinASegment;
        USGI WinBSegment;
        void ((far *WinFuncPtr)());
        USGI BytesPerScanLine;
        USGI XResolution;
        USGI YResolution;
        USGC XCharSize;
        USGC YCharSize;
        USGC NumberOfPlanes;
        USGC BitsPerPixel;
        USGC NumberOfBanks;
        USGC MemoryModel;
        USGC BankSize;
        USGC NumberOfImagePages;
        USGC ReservedForPageFunction;
        USGC RedMaskSize;
        USGC RedFieldPosition;
        USGC GreenMaskSize;
        USGC GreenFieldPosition;
        USGC BlueMaskSize;
        USGC BlueFieldPosition;
        USGC RsvdMaskSize;
        USGC RsvdFieldPosition;
        USGC DirectColorModeInfo;

        char Reserved[256]; /* more than enough space */
    };

ModeAttributes describes important video mode characteristics:

Bit Position | Meaning | Values | Comment
D0 | Mode available in present video configuration? | 0= Mode not supported 1=Mode supported | Blocks video mode access when required hardware is not available (frame buffer memory, monitor types, etc.)
D1 | Reserved. | Must be set to 1. | Must be set to 1.
D2 | BIOS output functions availability indicator (TTY output, screen scroll, pixel output). | 0= Output functions not supported 1=Output functions supported | If D2 is 1 then the BIOS supports all standard output functions in this mode
D3 | Monochrome or color mode? | 0= Monochrome mode 1=Color mode |
D4 | Mode type? | 0= Text mode 1=Graphics mode |
D15-D5 | Reserved | Reserved | Reserved

WinAAttributes and WinBAttributes describe the CPU windowing scheme characteristics, indicating whether windows exist and whether they are individually readable and writeable. The field format for both window A and window B follows:

Bit Position | Meaning | Values | Comment
D0 | Window supported? | 0= Window not supported 1=Window is supported | Indicates whether a window (A or B) is available.
D1 | Window readable? | 0= Window not readable 1=Window is readable | Indicates whether a window (A or B) is readable.
D2 | Window writeable? | 0= Window not writeable 1=Window is writable | Indicates whether a window (A or B) is writeable.
D7-D3 | Reserved | Reserved | Reserved

If bit D0 = 0 for both Window A and Window B (windowing not supported), assume the frame buffer memory completely resides at the standard CPU address appropriate for the mode's MemoryModel.

WinGranularity specifies the smallest boundary, in KBytes, on which the window can align in video memory. The field's value is undefined when the appropriate WinAttributes field bit D0 has a zero value.

WinSize specifies the window size in KBytes.

WinASegment and WinBSegment specify CPU memory segment address window origins. When swapped in, granules align on these memory segment addresses.

WinFuncPtr specifies the CPU's video memory windowing function address. VBE Function 0x05 (CPU Video Memory Window Control) or direct function calls can invoke windowing support. Direct calls provide faster hardware paging register access than VBE function 0x05, and are intended for high performance application use. When this field has a zero value (Null), use VBE Function 0x05 to set the memory window when paging is supported.

BytesPerScanline specifies the logical scan-line length in bytes. The logical scan line's length is equal to or larger than the displayed scan line. When the logical scan line is longer than the display's scan line, software can use VBE Function 0x07 (Set/Get Display Start) to pan left and right smoothly.

XResolution and YResolution specify video graphics mode width and height in pixels per visible scan line and total visible scan lines, respectively.

XCharCellSize and YCharCellSize specify character cell dimensions in pixels.

NumberOfPlanes specifies the number of available memory planes for the mode. For standard packed pixel and direct color modes, this is 1.

BitsPerPixel specifies the total number of bits defining one pixel. For example, packed pixel 256-color graphics mode specifies 8.

NumberOfBanks indicates the number of banks holding scan-line groups. When dividing a screen scan-line number by NumberOfBanks, the remainder is the bank number containing the screen scan line and the quotient is the scan-line number within the bank. For modes that don't have scan-line banks, this field should contain one.

MemoryModel specifies the mode's memory organization. MemoryModel values follow:

Value | Value Meaning
0x00 | Text mode
0x01 | CGA graphics
0x02 | Hercules graphics
0x03 | 4-plane planar
0x04 | Packed pixel
0x05 | Non-chain 4, 256 color
0x06 | Direct Color
0x07 | YUV
0x08-0x0F | VESA reserved
0x10-0xFF | OEM defined

Direct color modes should use the ModeInfoBlock MaskSize and FieldPosition fields to describe pixel formats. BitsPerPixelis always defined to be the total pixel memory size expressed in bits.

BankSize specifies the bank size in 1 KByte units. For modes that don't have scan-line banks, this field should contain zero.

NumberOfImagePages specifies the number of complete display images that simultaneously fit into the VGA's memory in this mode. The application may load more than one image into the VGA's memory if this field is non-zero, and rapidly flip the display between them.

Reserved supports future VBE extension features and always contains one in VBE 1.2.

RedMaskSize, GreenMaskSize, BlueMaskSize, and RsvdMaskSize (collectively referred to as MaskSize) define direct color pixel red, green, blue, and reserved component sizes in bits. Simple shift arithmetic using MaskSize field information provides pixel bit masks. MaskSize values for direct color (5:6:5) modes are 5, 6, 5, and 0, for the red, green, blue, and reserved fields, respectively. When MemoryModel contains the YUV value, the Y field uses the green value, the U field uses the blue value, and the V field uses the red value. The MaskSize field values are 0 for MemoryModel modes not having component field pixels.

RedFieldPosition , GreenFieldPosition, BlueFieldPosition , and RsvdFieldPosition (collectively referred to as FieldPosition) define respective color component least-significant-bit positions within direct color or YUV pixels. Color values are properly aligned with pixels by shifting component values left by component FieldPosition values. FieldPosition values for Direct Color (5:6:5) modes would be 11, 5, 0, and 0, for the red, green, blue, and reserved fields, respectively. When MemoryModel contains the YUV value, the Y field uses the green value, the U field uses the blue value, and the V field uses the red value. The FieldPosition field values are 0 for MemoryModel modes not having component field pixels.

DirectColorModeInfo describes direct color (not packed pixel) mode characteristics. Bit D0 specifies whether the DAC color ramp is fixed or programmable. If the color ramp is fixed, then it cannot be changed. If the color ramp is programmable, standard VGA DAC color registers BIOS call (AX=0x1012) loads red, green, and blue look-up tables. Bit D1 specifies whether applications can use direct color pixel Rsvd field bits or whether they are reserved, hence unusable.

Bit Position | Meaning | Values
D0 | Fixed color ramp? | 0 == Color ramp is fixed Color ramp is programmable
D1 | Rsvd field bits usable? | 0= Rsvd field bits not usable 1=Bits in Rsvd field usable
D7-D2 | Reserved | Reserved

4. VBE Function 0x02—Set Super VGA Video Mode

This function initializes the video mode the BX register specifies. If the video BIOS cannot set the mode, the video BIOS leaves the video environment unchanged, returning a failure error code.

Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x02 | Set Super VGA video mode
BX | Mode Value D0-D14=VBE Video mode D15=Clear memory flag | Only modes returned by VBE Function 0x00. If D15==0, frame buffer is cleared to zeros. If D15==1, frame buffer is not cleared.

Output: | AX: VBE status value
Comment: | Preserves all other registers

5. VBE Function 0x03—Return Current Video Mode

This function returns the current video mode in BX.

Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x03 | Return current video mode

Output: | AX: VBE Status Value
 | BX: Current video mode
Comment: | Preserves all other register

6. VBE Function 0x04—Save/Restore Super VGA Video State

These functions save and restore Super VGA video states and are a super-set of three standard VGA BIOS Function 0x1C (Save/Restore Video State) subfunctions. All Super VGA video states (except video memory) should be saveable/restorable by setting the requested state's mask (in the CX register) to 0x000F. Software:

  • Determines state-save buffer space requirements using subfunction zero (DL == 0)
  • Saves states in the buffer using subfunction one (DL == 1)
  • Restores states using subfunction two (DL == 2).

Save State(s) Buffer Requirement Determination Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x04 | Save/Restore Super VGA Video state request
DL | 0x00 | Return Save/Restore state buffer size
CX | Requested State(s) Save Buffer Size | D0= Save video hardware state D1= Save/Restore video BIOS data state D2= Save/Restore video DAC state D3= Save/Restore Super VGA state

Output: | AX: VBE status value
 | BX: Number of 64-byte blocks to hold the state buffer
Comment: | Preserves all other registers

Save State Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x04 | Save/Restore Super VGA Video state(s)
DL | 0x01 | Save state(s)
CX | Requested State(s) To Save | D0= Save video hardware state D1= Save video BIOS data state D2= Save video DAC state D3= Save Super VGA state
ES:BX | Buffer Pointer |

Output: | AX: VBE status value
Comment: | Preserves all other registers

Restore State Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x04 | Save/Restore Super VGA Video state(s)
DL | 0x02 | Restore state(s)
CX | Requested State(s) to Restore | D0= Restore video hardware state D1= Restore video BIOS data state D2= Restore video DAC state D3= Restore Super VGA state
ES:BX | Buffer Pointer |
Output: | AX: VBE status value
Comment: | Preserves all other registers

7. VBE Function 0x05—CPU Video Memory Window Control

This function allows direct access to the hardware paging register and sets or gets the specified window position in video memory. VBE Function 0x01 (Return Super VGA Mode Information) provides window size, location, and granularity information essential to use this function properly.

For improved performance, software can access this function directly using far calls to the ModeInfoBlock WinFuncPtr address VBE Function 0x01 (Return Super VGA Mode Information) returns. When using far calls:

  • Callers receive no status information.
  • The AX and DX registers are destroyed.
  • Applications must set BH, BL, and DX (for set window) input arguments but need not set AH or AL.
  • Obtain the function pointer after each mode set since addresses may vary for different video modes.

Select Window Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x05 | Super VGA video memory window control
BH | 0x00 | Select super VGA video memory window
BL | Window Number | 0= Window A 1= Window B
DX | Video Memory Window Position | Specified in window granularity units

Output: | AX: VBE status value
 | DX: Window position in video memory (in window granularity units)
Comment: | Preserves all other registers

Return Window Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x05 | Super VGA video memory window control
BH | 0x01 | Return super VGA video memory window
BL | Window Number | 0= Window A 1= Window B
DX | Video Memory Window Position | Specified in window granularity units

Output: | AX: VBE status value
 | DX: Window position in video memory (in window granularity units)
Comment: | Preserves all other registers

8. VBE Function 0x06—Set/Get Logical Scan-Line Length

This function sets or gets the current logical scan-line length. Setting a logical video memory buffer wider than the display scan-line length allows applications to smoothly pan horizontally and possibly vertically using VBE Function 0x07 (Set/Get Display Start) which allows applications to set the starting display position at and within logical scan lines.

Set Scan Line Length Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x06 | Logical scan-line length
BL | 0x00 | Set scan-line length
CX | Desired Length | Expressed in pixels

Output: | AX: VBE status value
 | BX: Bytes per scan line
 | CX: Actual pixels per scan line
 | DX: Maximum number of scan lines
Comment: Preserves all other registers

Return Scan-Line Length Input:

Register | Value | Comment
AH | 0x4 | FVBE request indicator
AL | 0x06 | Logical scan-line length
BL | 0x01 | Return scan-line length
Output: | AX: VBE status value
 | BX: Bytes per scan line
 | CX: Actual pixels per scan line
 | DX: Maximum number of scan lines
Comment: | Preserves all other registers

When VGA hardware considerations preclude setting a requested scan-line length, the next larger scan-line length that accommodates the desired scan-line length is provided, and:

  • CX returns the actual scan-line length value.
  • BX returns value indicating how many frame buffer bytes a single scan line consumes. For example, in mode 0x13 this is 320.
  • DX returns the number of logical scan lines available based on:
    1. The new scan-line length BX returns
    2. The total frame buffer memory installed
    3. The total frame buffer memory the display mode uses

Software can use VBE function 0x07 (Set/Get Display Start) to smoothly pan vertically within these available scan lines.

9. VBE Function 0x07—Set/Get Display Start

This function selects the pixel from the current logical page the display presents in its upper left corner. This function allows applications to pan and scroll around logical screens larger than the display screen resolution. This function allows rapid switching between different frame buffer areas for double-buffered animation effects.

Select Display Start Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x07 | Super VGA video memory window control
BH | 0x00 | Reserved and must be 0
BL | 0x00 | Select Display Start
CX | Pixel Number | First displayed frame buffer scan-line pixel
DX | Scan Line Number | First displayed frame buffer scan line

Output: | AX: VBE status value
 | DX: Window position in video memory (in window granularity units)
Comment: | Preserves all other registers

Return Display Start Input:

AH | 0x4 | FVBE request indicator
AL | 0x07 | Super VGA video memory window control
BL | 0x01 | Return Display Start
CX | Pixel Number | First displayed frame buffer scan-line pixel
DX | Scan Line Number | First displayed frame buffer scan line

Output: | AX: VBE status value
 | BH: 0x00 reserved and will be 0
 | CX: First displayed pixel in scan line
 | DX: First displayed scan line
Comment: | Preserves all other registers

10. VBE Function 0x08—Set/Get DAC Palette Control

This function queries and selects the DAC width. The DAC palette width is always the standard VGA 6 bits per primary color after standard VGA or VBE Function 0x02 (Set Super VGA Mode) calls. Some DACs provide 6 bits, 8 bits, or more for primary colors.

Set DAC Width Input:

Register | Value | Comment
AH | 0x4 | FVBE request indicator
AL | 0x08 | Set/Get DAC palette control
BL | 0x00 | Set DAC palette width
BH | DAC Width | Desired number of bits per primary color (Standard VGA = 6)

Output: | AX: VBE status value
 | BH: Current number of bits per primary color (Standard VGA = 6)
Comment: | Preserves all other registers

Get DAC Width Input:

Register | Value | Comment
AH | 0x4F | VBE request indicator
AL | 0x08 | Set/Get DAC palette control
BL | 0x01 | Get DAC palette width

Output: | AX: VBE status value
 | BH: Current number of bits per primary color (Standard VGA = 6)
Comment: | Preserves all other registers

Applications can determine DAC width switching capability by querying the Capabilities field's bit D0 in the VgaInfoBlock structure VBE Function 0x00 (Return Super VGA Information) returns. If DAC switching is supported, the application can then attempt to set the DAC palette width to a desired value. If the DAC cannot support the requested width, the next lower value the DAC supports is selected. BH returns the resulting palette width.

Sample VBE Application

The VESAEXAM.C program provides the logic necessary to query for, and determine the extent of, VBE support. Appendix D lists the VESA.H and PCX.H headers.

 /*
               Copyright (C) 1995, W. David Schwaderer
Some rights reserved - see WdsLogo() for details
   Function: This program interrogates and documents VESA capabilities
             for a single user-specified video mode or all available
             video modes. Redirect the output to a file for hard copy.
             See the text description in:

              Digital Imaging in C and the World Wide Web
                         W. David Schwaderer
                     Wordware Publishing, Inc., 1997

   Compiler: Borland C++ 3.1

   Compile command: bcc -ml VesaExam.C

   Call to programmers...

     If you are interested enough to read this code, your name
     probably should be in it. I will place contributing programmer's
     names in source code when their suggestion(s) results in bug fixes
     or program enhancements. Contributions must be new. Send written
     suggestions on paper to W. David Schwaderer via Wordware Publishing,
     Inc.

     Complete or massive rewrites are not helpful (though possibly
     warranted!). Contributions become the property of W. David
     Schwaderer who reserves suggestion final disposition determination.

     Thanks in advance for the consideration.
                                             W. David Schwaderer

   Version 1.0 3/15/93
       Contributor                         Enhancement
   --------------------------------
----------------------------------------------------------
   Your name here...            Your edited suggestion here...
   Hometown, country            Your edited suggestion here...

*/

#include <mem.h>
#include <dos.h>
#include <stdio.h>
#include <stdlib.h>

#define USGI unsigned int
#define USGC unsigned char
#define USGL unsigned long
#include "vesa.h"

int main(int argc, char *argv[]);
int WdsLogo(void);
int VesaNotPresent(void);
int DisplayVesaInfo(void);
int FillVesaModeBlock(struct VesaModeInfoBlock *VesaModeInfoBlockPtr,
                                         USGI  ModeNumber);
int DisplayVesaModeAttributes(USGI VesaModeAttributes);
int DisplayVesaWindowAttributes(USGC VesaWindowAttributes);
int DisplayVesaModeInfo(USGI ModeNumber, USGI VesaModesOnlyFlag);
int ExitLogo(void);

struct VesaInfoBlock VesaInfo;

USGI   ModeValue;

int main(int argc, char *argv[])
{

    if(VesaNotPresent())
    {

        printf("\nmain: VESA support not detected, exiting...");
        ExitLogo();

    }

    printf("\nmain: VESA support detected...");

    if (argc > 1)
         sscanf(argv[1], "%x", &ModeValue);

    DisplayVesaInfo();

    WdsLogo();

    printf("\nmain: Returning...\n");

    return 0;

}

int WdsLogo(void)

{

    printf("\n");
    printf("\nCopyright (C) 1993 W. David Schwaderer, some rights reserved");
    printf("\nThis program uses source code from W. David Schwaderer's");
    printf("\n       'Digital Imaging in C and the World Wide Web");
    printf("\n            Wordware Publishing, Inc., 1997");
    printf("\nW. David Schwaderer grants an unrestricted executable
                           program");
    printf("\nuse-license for unmodified executable programs he releases.");
    printf("\nNon-commercial source code use-licenses are also granted to:");
    printf("\n - individuals for their non-commercial private use (enjoy!)");
    printf("\n - educational institutions for teaching materials  (learn!)");
    printf("\nprovided the WdsLogo() function is compiled (execution not");
    printf("\nrequired) without modification into derived executable");
    printf("\nmodules. Other source code uses, including 'shareware',");
    printf("\nare encouraged but strictly prohibited without W. David");
    printf("\nSchwaderer's explicit written permission, easily obtained");
    printf("\nwith appropriate book credit. Please contact David via");
    printf("\Wordware Publishing, Inc. for written permission requests.");
    printf("\nDeepest thanks for your cooperation.");
    printf("\n                                       W. David Schwaderer");
    printf("\n");

    return 0;

}

int VesaNotPresent()
{

    struct SREGS SegReg;
    union   REGS OutReg, InReg;

    InReg.h.ah  = VESA_FUNCTION;
    InReg.h.al = VESA_RETURN_INFO;

    SegReg.es   = FP_SEG(&VesaInfo);
    InReg.x.di = FP_OFF(&VesaInfo);

    int86x(VIDEO_INT, &InReg, &OutReg, &SegReg);

    if(OutReg.h.ah != VESA_CALL_SUCCESSFUL)
        return 1;

    if(OutReg.h.al != VESA_FUNCTION_SUPPORTED)
    {

        printf("\nVesaNotPresent: VESA Presence test not supported");
        return 1;

    }

    return 0;

}

int DisplayVesaInfo()
{

    USGI far *FarIntPtr,  *NearIntPtr,  Index1, Index2, Mask;
    char far *FarCharPtr, *NearCharPtr, LocalOemString[256];

         /*------------------- signature ------------------*/

    printf("\n\nDisplayVesaInfo: VesaSignature (hex) ==>");

    for (Index1 = 0;
         Index1 < sizeof(VesaInfo.VesaSignature);
         Index1++)
    {
         printf("%02X", VesaInfo.VesaSignature[Index1]);
         if (Index1 < (sizeof(VesaInfo.VesaSignature) - 1))
             putchar('.');

    }

    printf("<==  ('");

    for (Index1 = 0;
         Index1 < sizeof(VesaInfo.VesaSignature);
         Index1++)
            printf("%c", VesaInfo.VesaSignature[Index1]);

    printf("')");

           /*------------------- version -------------------*/

    printf("\nDisplayVesaInfo: VesaVersion == VBE %u.%u",
            ((VesaInfo.VesaVersion >> 8) & 0x00FF),
             (VesaInfo.VesaVersion)      & 0x00FF);

         /*------------------- OEM string -------------------*/

    printf("\nDisplayVesaInfo: OemString ==>");

    FarCharPtr = VesaInfo.OemStringPtr;

    while(*FarCharPtr)
        putchar(*FarCharPtr++);

    printf("<==");

         /*----------------- capabilities -----------------*/

    printf("\nDisplayVesaInfo: Capabilities ==>");

    for (Index1 = 0;
         Index1 < sizeof(VesaInfo.Capabilities);
         Index1++)

   {

       for ( Index2 = 0, Mask = 0x0080;
             Index2 < 8;
             Index2++,   Mask >>= 1)
           if (VesaInfo.Capabilities[Index1] & Mask)
               putchar('1');
           else
               putchar('0');

       if (Index1 != 3)
           putchar('.');
   }
    printf("<==");
    printf("\n                     - %s DAC width...",
             (VesaInfo.Capabilities[3] & VESA_DAC_WIDTH_TEST_BIT) ?
             "Switchable" : "Fixed (6 bits/primary color)");

         /*------------------- total memory -------------------*/

    printf("\nDisplayVesaInfo: Total64kMemoryBlocks == %u (%uKb total)...",
                       VesaInfo.Total64kMemoryBlocks,
                       VesaInfo.Total64kMemoryBlocks * 64);

         /*------------------- video modes -------------------*/

    FarIntPtr = VesaInfo.VideoModePtr;

    if (!ModeValue)
    {

        while(*FarIntPtr != 0xFFFF)
        {

            if (*FarIntPtr & VESA_VIDEO_MODE_TEST_BIT)
                DisplayVesaModeInfo(*FarIntPtr, !VESA_MODES_ONLY);

            FarIntPtr++;

        }
    } else
        DisplayVesaModeInfo(ModeValue, !VESA_MODES_ONLY);

    return 0;
}

int FillVesaModeBlock(struct VesaModeInfoBlock *VesaModeInfoBlockPtr,
                                         USGI  ModeNumber)

{

    struct SREGS SegReg;
    union   REGS OutReg, InReg;

    InReg.h.ah  = VESA_FUNCTION;
    InReg.h.al = VESA_RETURN_MODE_INFO;

    InReg.x.cx = ModeNumber;

    SegReg.es   = FP_SEG(VesaModeInfoBlockPtr);
    InReg.x.di = FP_OFF(VesaModeInfoBlockPtr);

    int86x(VIDEO_INT, &InReg, &OutReg, &SegReg);

    if(OutReg.h.ah != VESA_CALL_SUCCESSFUL)
        return 0;

    if(OutReg.h.al != VESA_FUNCTION_SUPPORTED)
    {
        printf("\n\nFillVesaModeBlock: VESA mode info request not
                             supported");
        return 0;

    }

    return 1;

}

int DisplayVesaModeAttributes(USGI VesaModeAttributes)
{

    printf("\n                   -Your hardware does%s support it...",
        (VesaModeAttributes & VESA_MODE_AVAILABLE_TEST_BIT) ?
                                       "" : "n't");

    if (!(VesaModeAttributes & VESA_MODE_AVAILABLE_TEST_BIT))
        return 0;

    printf("\n                   -Extended mode info is%s available...",
        (VesaModeAttributes & VESA_MODE_EXTENDED_INFO_AVAIL_TEST_BIT) ?
                                       "" : "n't");

    printf("\n                   -BIOS output functions are%s supported...",
        (VesaModeAttributes & VESA_MODE_OUTPUT_FUN_TEST_BIT) ?
                                       "" : "n't");

    printf("\n                   -%s mode...",
        (VesaModeAttributes & VESA_MODE_MONO_COLOR_TEST_BIT) ?
                                       "Color" : "Monochrome");

    printf("\n                   -%s mode type...",
        (VesaModeAttributes & VESA_MODE_MONO_TYPE_TEST_BIT) ?
                                       "Graphics" : "Text");

    return 0;

}

int DisplayVesaWindowAttributes(USGC VesaWindowAttributes)
{

    printf("\n                   - Window is%s supported...",
        (VesaWindowAttributes & VESA_WINDOW_SUPPORTED_TEST_BIT) ?
                                       "" : "n't");

    if (!(VesaWindowAttributes & VESA_WINDOW_SUPPORTED_TEST_BIT))
        return 0;

    printf("\n                   - Window is%s readable...",
        (VesaWindowAttributes & VESA_WINDOW_READABLE_TEST_BIT) ?
                                       "" : "n't");

    printf("\n                   - Window is%s writable...",
        (VesaWindowAttributes & VESA_WINDOW_WRITABLE_TEST_BIT) ?
                                       "" : "n't");

    return 1;

}

int DisplayVesaModeInfo(USGI ModeNumber, USGI VesaModesOnlyFlag)
{

    struct VesaModeInfoBlock VesaModeInfo;

    memset(&VesaModeInfo, '\x00', sizeof(VesaModeInfo));

    if (VesaModesOnlyFlag)

    {

        if (!(ModeNumber & VESA_VIDEO_MODE_TEST_BIT))

        {

            printf("\nDisplayVesaModeInfo: Mode %04X is not a VESA mode...",
                                            ModeNumber);

            return 1;

        }

    }

    FillVesaModeBlock(&VesaModeInfo, ModeNumber);

    printf("\n\nDisplayVesaModeInfo: *---------- Mode == 0x%04X ----------*",
                                                  ModeNumber);

    DisplayVesaModeAttributes(VesaModeInfo.ModeAttributes);

    printf("\nDisplayVesaModeInfo: Bytes per scan line == %u",
                                         VesaModeInfo.BytesPerScanLine);

           /*------------------ general window information ------------------*/

    printf("\nDisplayVesaModeInfo: Window Size == %uKb, Granularity == %uKb",
                                            VesaModeInfo.WinSize,
                                            VesaModeInfo.WinGranularity);

    printf("\nDisplayVesaModeInfo: Window function pointer == %04X:%04X",
                                            FP_SEG(VesaModeInfo.WinFuncPtr),
                                            FP_OFF(VesaModeInfo.WinFuncPtr));

                  /*----------------- window A -----------------*/

    printf("\nDisplayVesaModeInfo: Window A segment == 0x%04X",
                                             VesaModeInfo.WinASegment);

    DisplayVesaWindowAttributes(VesaModeInfo.WinAAttributes);

                 /*------------------ window B -----------------*/

    printf("\nDisplayVesaModeInfo: Window B segment == 0x%04X",
                                             VesaModeInfo.WinBSegment);

    DisplayVesaWindowAttributes(VesaModeInfo.WinBAttributes);

    /*------------------- extended mode information (preVBE 1.2) ------------------*/

    if (!(VesaModeInfo.ModeAttributes &
          VESA_MODE_EXTENDED_INFO_AVAIL_TEST_BIT))
        return 0;

    printf("\nDisplayVesaModeInfo: Resolutions  X == %u, Y == %u",
                                            VesaModeInfo.XResolution,
                                            VesaModeInfo.YResolution);

    printf("\nDisplayVesaModeInfo: Character cell Width == %u, Height == %u",
                                            VesaModeInfo.XCharSize,
                                            VesaModeInfo.YCharSize);

    printf("\nDisplayVesaModeInfo: Number of planes == %u",
                                            VesaModeInfo.NumberOfPlanes);

    printf("\nDisplayVesaModeInfo: Bits per pixel == %u",
                                      VesaModeInfo.BitsPerPixel);

    printf("\nDisplayVesaModeInfo: Number of banks == %u",
                                      VesaModeInfo.NumberOfBanks);

    printf("\nDisplayVesaModeInfo: Memory model == %u    ",
                                      VesaModeInfo.MemoryModel);

    switch(VesaModeInfo.MemoryModel)
    {

        case VESA_MODE_TEXT :
                                        printf("(Text mode)");
                                        break;
        case VESA_MODE_CGA_GRAPHICS :
                                        printf("(CGA graphics)");
                                        break;
        case VESA_MODE_HERCULES_GRAPHICS :
                                        printf("(Hercules graphics)");
                                        break;
        case VESA_MODE_4_PLANE_PLANAR :
                                        printf("(4-plane planar)");
                                        break;
        case VESA_MODE_PACKED_PIXEL :
                                        printf("(Packed pixel)");
                                        break;
        case VESA_MODE_NON_CHAIN_4_256_COLOR :
                                        printf("(Non-chain 4,  256 color)");
                                        break;
        case VESA_MODE_DIRECT_COLOR :
                                        printf("(Direct color)");
                                        break;
        case VESA_MODE_YCbCr :
                                        printf("(YCbCr)");
                                        break;
        default :
                    if (VesaModeInfo.MemoryModel < 0x10)
                        printf("(VESA reserved)");
                    else
                        printf("(OEM defined)");

    }

    printf("\nDisplayVesaModeInfo: Bank size == %u",
                                      VesaModeInfo.BankSize);

    printf("\nDisplayVesaModeInfo: Number of image pages == %u",
                                      VesaModeInfo.NumberOfImagePages);

    if (VesaModeInfo.MemoryModel == VESA_MODE_DIRECT_COLOR)
    {
        printf("\nDisplayVesaModeInfo: Mask sizes Red(%u), Green(%u)",
                       VesaModeInfo.RedMaskSize,
                       VesaModeInfo.GreenMaskSize);

        printf("\nDisplayVesaModeInfo: Mask sizes Blue(%u), Rsvd(%u)",
                       VesaModeInfo.BlueMaskSize,
                       VesaModeInfo.RsvdMaskSize);

        printf("\nDisplayVesaModeInfo: Field lsb position Red(%u), Green(%u)",
                       VesaModeInfo.RedFieldPosition,
                       VesaModeInfo.GreenFieldPosition);

        printf("\nDisplayVesaModeInfo: Field lsb position Blue(%u), Rsvd(%u)",
                      VesaModeInfo.BlueFieldPosition,
                      VesaModeInfo.RsvdFieldPosition);

        printf("\nDisplayVesaModeInfo: DirectColorModeInfo");

        if (VesaModeInfo.DirectColorModeInfo &
            VESA_PROGRAMMABLE_DIRECT_COLOR_RAMP_TEST_BIT)
              printf("\nDisplayVesaModeInfo:  - Color ramp is programmable");
        else
              printf("\nDisplayVesaModeInfo:  - Color ramp is fixed");

        if (VesaModeInfo.DirectColorModeInfo &
            VESA_RESERVED_BITS_ARE_USABLE_TEST_BIT)
              printf("\nDisplayVesaModeInfo:  - Reserved bits are usable");
        else
              printf("\nDisplayVesaModeInfo:  - Do not use reserved bits");
     }
     if (VesaModeInfo.MemoryModel == VESA_MODE_YCbCr)
     {
        printf("\nDisplayVesaModeInfo: Mask sizes Y(%u), U(%u)",
                      VesaModeInfo.GreenMaskSize,
                      VesaModeInfo.BlueMaskSize);

        printf("\nDisplayVesaModeInfo: Mask sizes V(%u), Rsvd(%u)",
                        VesaModeInfo.RedMaskSize,
                        VesaModeInfo.RsvdMaskSize);

        printf("\nDisplayVesaModeInfo: Field lsb position Y(%u), U(%u)",
                      VesaModeInfo.GreenFieldPosition,
                      VesaModeInfo.BlueFieldPosition);

        printf("\nDisplayVesaModeInfo: Field lsb position V(%u), Rsvd(%u)",
                        VesaModeInfo.RedFieldPosition,
                        VesaModeInfo.RsvdFieldPosition);
     }

     return 0;
 }

int ExitLogo(void)
{
    WdsLogo();

    exit(1);

    return 0;
}

/*======================== End of File ========================*/


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Appendix B
RGB/HSV/HLS/HSI Color Model Conversion Routines

#define DEGREES_PER_RADIAN  (180.0 / 3.14159265358979)
#define UNDEFINED_HUE       -1.000
#define ZERO_SATURATION      0.0

struct RgbTriadStruct {
                          USGC Red;
                          USGC Green;
                          USGC Blue;
                       };

struct HsvTriadStruct  {
                          double Hue;
                          double Saturation;
                          double Value;
                       };

struct HlsTriadStruct  {
                          double Hue;
                          double Lightness;
                          double Saturation;
                       };

struct HsiTriadStruct  {
                          double Hue;
                          double Saturation;
                          double Intensity;
                       };

int RgbToHsv(struct RgbTriadStruct *RgbTriadStructPtr,
             struct HsvTriadStruct *HsvTriadStructPtr)
{

USGI MaxPrimaryValue, MinPrimaryValue, MaxMinDelta;

MaxPrimaryValue = (RgbTriadStructPtr->Red  > RgbTriadStructPtr->Green) ?
                 RgbTriadStructPtr->Red  : RgbTriadStructPtr->Green;

MaxPrimaryValue = (RgbTriadStructPtr->Blue > MaxPrimaryValue) ?
                 RgbTriadStructPtr->Blue : MaxPrimaryValue;

HsvTriadStructPtr->Value = ((double) MaxPrimaryValue) / 255.0;

if (!MaxPrimaryValue)
{
     HsvTriadStructPtr->Value      = 0.0;
     HsvTriadStructPtr->Hue        = UNDEFINED_HUE;
     HsvTriadStructPtr->Saturation = ZERO_SATURATION;

     return 0;
} else
{
        HsvTriadStructPtr->Value = ((double) MaxPrimaryValue) / 255.0;
}

MinPrimaryValue = (RgbTriadStructPtr->Red  < RgbTriadStructPtr->Green) ?
                  RgbTriadStructPtr->Red  : RgbTriadStructPtr->Green;

MinPrimaryValue = (RgbTriadStructPtr->Blue < MinPrimaryValue) ?
                  RgbTriadStructPtr->Blue : MinPrimaryValue;

if (!(MaxMinDelta = MaxPrimaryValue - MinPrimaryValue))
{
      HsvTriadStructPtr->Hue        = UNDEFINED_HUE;
      HsvTriadStructPtr->Saturation = ZERO_SATURATION;

      return 0;
}

HsvTriadStructPtr->Saturation =
            ((double) MaxMinDelta) / ((double) MaxPrimaryValue);

if (RgbTriadStructPtr->Red == MaxPrimaryValue)
     HsvTriadStructPtr->Hue =  0.0 +
          ((double) (RgbTriadStructPtr->Green - RgbTriadStructPtr->Blue))
                                                /
                                   ((double) MaxMinDelta);

else if (RgbTriadStructPtr->Green == MaxPrimaryValue)
          HsvTriadStructPtr->Hue =  2.0 +
             ((double) (RgbTriadStructPtr->Blue - RgbTriadStructPtr->Red))
                                                /
                                 ((double) MaxMinDelta);

     else if (RgbTriadStructPtr->Blue == MaxPrimaryValue)
               HsvTriadStructPtr->Hue =  4.0 +
                  ((double) (RgbTriadStructPtr->Red - RgbTriadStructPtr->Green))
                                                /
                                      ((double) MaxMinDelta);

    HsvTriadStructPtr->Hue *= 60.0;

    if (HsvTriadStructPtr->Hue < 0)
        HsvTriadStructPtr->Hue += 360.0;

    return 0;
}

int HsvToRgb(struct HsvTriadStruct *HsvTriadStructPtr,
             struct RgbTriadStruct *RgbTriadStructPtr)
{
    USGI   IntValue;
    double OneSixtythHue, DoubleDecimal, P, Q, T, V;

    if (HsvTriadStructPtr->Saturation == ZERO_SATURATION)
    {
        if (HsvTriadStructPtr->Hue == UNDEFINED_HUE)
        {
            RgbTriadStructPtr->Red   =
            RgbTriadStructPtr->Green =
            RgbTriadStructPtr->Blue  =
                         HsvTriadStructPtr->Value * 255.0 + 0.5;
        } else
        {
            printf("\n\nHsvToRgb: Illegal HSV hue value (saturation == 0)");
            printf("\nHue == %5.3f, Saturation = %5.3f, Value == %5.3f",
                      HsvTriadStructPtr->Hue,
                      HsvTriadStructPtr->Saturation,
                      HsvTriadStructPtr->Value);
        }
        return 0;
    }
    if (HsvTriadStructPtr->Hue > 359.9)
        HsvTriadStructPtr->Hue = 0.0;

    OneSixtythHue = (HsvTriadStructPtr->Hue / 60.0);

    IntValue     = (USGI) OneSixtythHue;
    DoubleDecimal = OneSixtythHue - ((double) IntValue);

    P = HsvTriadStructPtr->Value *
        (1.0 - HsvTriadStructPtr->Saturation);

    Q = HsvTriadStructPtr->Value *
        (1.0 - (HsvTriadStructPtr->Saturation * DoubleDecimal));

    T = HsvTriadStructPtr->Value *
        (1.0 - (HsvTriadStructPtr->Saturation * (1.0 - DoubleDecimal)));

    V = HsvTriadStructPtr->Value;

    switch (IntValue)
    {
        case 0:
                RgbTriadStructPtr->Red   = (USGI) (V * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (T * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (P * 255.0 + 0.5);

                break;

        case 1:
                RgbTriadStructPtr->Red   = (USGI) (Q * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (V * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (P * 255.0 + 0.5);

                break;

        case 2:
                RgbTriadStructPtr->Red   = (USGI) (P * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (V * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (T * 255.0 + 0.5);

                break;

        case 3:
                RgbTriadStructPtr->Red   = (USGI) (P * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (Q * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (V * 255.0 + 0.5);

                break;

        case 4:
                RgbTriadStructPtr->Red   = (USGI) (T * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (P * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (V * 255.0 + 0.5);

                break;

        case 5:
                RgbTriadStructPtr->Red   = (USGI) (V * 255.0 + 0.5);
                RgbTriadStructPtr->Green = (USGI) (P * 255.0 + 0.5);
                RgbTriadStructPtr->Blue  = (USGI) (Q * 255.0 + 0.5);

                break;
    }

    return 0;
}

int RgbToHls(struct RgbTriadStruct *RgbTriadStructPtr,
             struct HlsTriadStruct *HlsTriadStructPtr)
{
    USGI   MaxPrimaryValue, MinPrimaryValue, MaxMinDelta;
    double H;

    MaxPrimaryValue = (RgbTriadStructPtr->Red  > RgbTriadStructPtr->Green)?
                     RgbTriadStructPtr->Red  : RgbTriadStructPtr->Green;

    MaxPrimaryValue = (RgbTriadStructPtr->Blue > MaxPrimaryValue) ?
                     RgbTriadStructPtr->Blue : MaxPrimaryValue;

    MinPrimaryValue = (RgbTriadStructPtr->Red  < RgbTriadStructPtr->Green)?
                     RgbTriadStructPtr->Red  : RgbTriadStructPtr->Green;

    MinPrimaryValue = (RgbTriadStructPtr->Blue < MinPrimaryValue) ?
                     RgbTriadStructPtr->Blue : MinPrimaryValue;

    HlsTriadStructPtr->Lightness =
                ((double) (MaxPrimaryValue + MinPrimaryValue))
                                          /
                                     (2.0 * 255.0);

    if (MaxPrimaryValue == MinPrimaryValue)
    {
        HlsTriadStructPtr->Hue        = UNDEFINED_HUE;
        HlsTriadStructPtr->Saturation = ZERO_SATURATION;

        return 0;
    }

    MaxMinDelta = MaxPrimaryValue - MinPrimaryValue;

    if (HlsTriadStructPtr->Lightness < 0.5)
        HlsTriadStructPtr->Saturation =
                                        ((double) MaxMinDelta)
                                                /
                   ((double) (MaxPrimaryValue + MinPrimaryValue));
    else
        HlsTriadStructPtr->Saturation =
                                     ((double) MaxMinDelta)
                                                /
                   ((double) (2 * 255 - (MaxPrimaryValue + MinPrimaryValue)));

    if (RgbTriadStructPtr->Red == MaxPrimaryValue)
        HlsTriadStructPtr->Hue =  0.0 +
             ((double) (RgbTriadStructPtr->Green - RgbTriadStructPtr->Blue))
                                                /
                                      ((double) MaxMinDelta);

    else if (RgbTriadStructPtr->Green == MaxPrimaryValue)
              HlsTriadStructPtr->Hue =  2.0 +
                ((double) (RgbTriadStructPtr->Blue - RgbTriadStructPtr->Red))
                                                /
                                    ((double) MaxMinDelta);

    else if (RgbTriadStructPtr->Blue == MaxPrimaryValue)
             HlsTriadStructPtr->Hue =  4.0 +
               ((double) (RgbTriadStructPtr->Red - RgbTriadStructPtr->Green))
                                                /
                                    ((double) MaxMinDelta);

    HlsTriadStructPtr->Hue *= 60.0;

    if (HlsTriadStructPtr->Hue < 0)
        HlsTriadStructPtr->Hue += 360.0;

    return 0;
}

int HlsToRgb(struct HlsTriadStruct *HlsTriadStructPtr,
            struct RgbTriadStruct *RgbTriadStructPtr)
{
    double M1, M2;

    if (HlsTriadStructPtr->Lightness < 0.5)
      M2 = HlsTriadStructPtr->Lightness *
           (1.0 + HlsTriadStructPtr->Saturation);
    else
      M2 = HlsTriadStructPtr->Lightness  +
           HlsTriadStructPtr->Saturation -
          (HlsTriadStructPtr->Lightness * HlsTriadStructPtr->Saturation);

    M1 = 2.0 * HlsTriadStructPtr->Lightness - M2;

    if (HlsTriadStructPtr->Saturation == ZERO_SATURATION)
    {
        if (HlsTriadStructPtr->Hue == UNDEFINED_HUE)
        {
            RgbTriadStructPtr->Red   =
            RgbTriadStructPtr->Green =
            RgbTriadStructPtr->Blue  =
                       HlsTriadStructPtr->Lightness * 255.0 + 0.5;
        } else
        {
            printf("\n\nHlsToRgb: Illegal Hls hue value (saturation == 0)");
            printf("\nHue == %5.3f, Lightness = %5.3f, Saturation == %5.3f",
                      HlsTriadStructPtr->Hue,
                      HlsTriadStructPtr->Lightness,
                      HlsTriadStructPtr->Saturation);
        }

        return 0;
    }

    RgbTriadStructPtr->Red   = RgbCubeValue(M1, M2,
                                  HlsTriadStructPtr->Hue + 120.0);
    RgbTriadStructPtr->Green = RgbCubeValue(M1, M2,
                                  HlsTriadStructPtr->Hue);
    RgbTriadStructPtr->Blue  = RgbCubeValue(M1, M2,
                                  HlsTriadStructPtr->Hue - 120.0);

    return 0;
}

int RgbCubeValue(double N1, double N2, double Hue)
{
    double Value;

    if (Hue > 360.0) Hue -= 360.0;

    if (Hue <   0.0) Hue += 360.0;

    if      (Hue <  60.0) Value = N1 + (N2 - N1) * Hue / 60.0;
    else if (Hue < 180.0) Value = N2;
    else if (Hue < 240.0) Value = N1 + (N2 - N1) * (240 - Hue) / 60.0;
    else                  Value = N1;

    return (USGI) (Value * 255.0 + 0.5);
}

int HlsToRgb2(struct HlsTriadStruct *HlsTriadStructPtr,
              struct RgbTriadStruct *RgbTriadStructPtr)
{
    int    Sextant;
    double Min, V, Sv, H, Vsf;
    double Mid1, Mid2, Fract;

    if (HlsTriadStructPtr->Lightness <= 0.5)
      V = HlsTriadStructPtr->Lightness *
          (1.0 + HlsTriadStructPtr->Saturation);
    else
      V = HlsTriadStructPtr->Lightness  +
          HlsTriadStructPtr->Saturation -
         (HlsTriadStructPtr->Lightness * HlsTriadStructPtr->Saturation);

    if (V <= 0.0)
    {
        RgbTriadStructPtr->Red   =
        RgbTriadStructPtr->Green =
        RgbTriadStructPtr->Blue  = (USGC) 0;
    } else
    {
        Min     = HlsTriadStructPtr->Lightness +
                  HlsTriadStructPtr->Lightness - V;
        Sv      = (V - Min) / V;
        H       = HlsTriadStructPtr->Hue / 60.0;
        Sextant = (int) H;
        Fract   = H - (double) Sextant;
        Vsf     = V * Sv * Fract;
        Mid1    = Min + Vsf;
        Mid2    = V   - Vsf;

        V    *= 255.0; if (V    > 255.0) V    = 255.0;
        Min  *= 255.0; if (Min  > 255.0) Min  = 255.0;
        Mid1 *= 255.0; if (Mid1 > 255.0) Mid1 = 255.0;
        Mid2 *= 255.0; if (Mid2 > 255.0) Mid2 = 255.0;

        switch (Sextant)
        {
            case 0 :    RgbTriadStructPtr->Red   = (USGC) V;
                        RgbTriadStructPtr->Green = (USGC) Mid1;
                        RgbTriadStructPtr->Blue  = (USGC) Min;
                        break;

            case 1 :    RgbTriadStructPtr->Red   = (USGC) Mid2;
                        RgbTriadStructPtr->Green = (USGC) V;
                        RgbTriadStructPtr->Blue  = (USGC) Min;
                        break;

            case 2 :    RgbTriadStructPtr->Red   = (USGC) Min;
                        RgbTriadStructPtr->Green = (USGC) V;
                        RgbTriadStructPtr->Blue  = (USGC) Mid1;
                        break;

            case 3 :    RgbTriadStructPtr->Red   = (USGC) Min;
                        RgbTriadStructPtr->Green = (USGC) Mid2;
                        RgbTriadStructPtr->Blue  = (USGC) V;
                        break;

            case 4 :    RgbTriadStructPtr->Red   = (USGC) Mid1;
                        RgbTriadStructPtr->Green = (USGC) Min;
                        RgbTriadStructPtr->Blue  = (USGC) V;
                        break;

            case 5 :    RgbTriadStructPtr->Red   = (USGC) V;
                        RgbTriadStructPtr->Green = (USGC) Min;
                        RgbTriadStructPtr->Blue  = (USGC) Mid2;
                        break;

            default:    RgbTriadStructPtr->Red   = (USGC) 255;
                        RgbTriadStructPtr->Green = (USGC) 0;
                        RgbTriadStructPtr->Blue  = (USGC) 0;
                        break;
        }

    }

    return 0;
}

int RgbToHsi(struct RgbTriadStruct *RgbTriadStructPtr,
             struct HsiTriadStruct *HsiTriadStructPtr)
{
    double R, G, B, Sum, Quotient;
    double Radians, Angle, MinValue, MaxValue, TempDouble1, TempDouble2;

    R = ((double) RgbTriadStructPtr->Red)   / 255.0;
    G = ((double) RgbTriadStructPtr->Green) / 255.0;
    B = ((double) RgbTriadStructPtr->Blue)  / 255.0;

    Sum = R + G + B;

    HsiTriadStructPtr->Intensity = Sum / 3.0;

    MinValue = (R < G) ? R : G;
    MinValue = (B < MinValue) ? B : MinValue;

    MaxValue = (R > G) ? R : G;
    MaxValue = (B > MaxValue) ? B : MaxValue;

    if (HsiTriadStructPtr->Intensity < 0.00001)
        HsiTriadStructPtr->Saturation = ZERO_SATURATION;
    else
        HsiTriadStructPtr->Saturation = 1.0 - (3.0 * MinValue) / Sum;

    if (MinValue == MaxValue)
    {
        HsiTriadStructPtr->Hue        = UNDEFINED_HUE;
        HsiTriadStructPtr->Saturation = ZERO_SATURATION;

        return 0;
    }

    TempDouble1 = (((R - G) + (R - B)) / 2.0);
    TempDouble2 = (R - G)*(R - G) + (R - B)*(G - B);
    Quotient = (TempDouble1 / sqrt(TempDouble2));

    if (Quotient > 0.99999999999999999)
        Radians = 0.0;
    else if (Quotient < -0.99999999999999999)
        Radians = 3.1415926535;
    else
        Radians = acos(TempDouble1 / sqrt(TempDouble2));

    Angle = Radians * DEGREES_PER_RADIAN;

    if (B > G)
        HsiTriadStructPtr->Hue = 360.0 - Angle;
    else
        HsiTriadStructPtr->Hue = Angle;

    return 0;
}

int HsiToRgb(struct HsiTriadStruct *HsiTriadStructPtr,
             struct RgbTriadStruct *RgbTriadStructPtr)
{
    double r, g, b, Hue, Fraction, MaxDouble;

    if (HsiTriadStructPtr->Saturation == ZERO_SATURATION)
    {
        if (HsiTriadStructPtr->Hue == UNDEFINED_HUE)
        {
            RgbTriadStructPtr->Red   =
            RgbTriadStructPtr->Green =
            RgbTriadStructPtr->Blue  =
                       HsiTriadStructPtr->Intensity * 255.0 + 0.5;
        } else
        {
            printf("\n\nHsiToRgb: Illegal HSI hue value (saturation == 0)");
            printf("\nHue == %5.3f, Saturation = %5.3f, Intensity == %5.3f",
                      HsiTriadStructPtr->Hue,
                      HsiTriadStructPtr->Saturation,
                      HsiTriadStructPtr->Intensity);
        }

        return 0;
    }

    if (HsiTriadStructPtr->Hue >    0.0  &&
        HsiTriadStructPtr->Hue <= 120.0     )
    {
        b = (1.0 - HsiTriadStructPtr->Saturation) / 3.0;

        Hue = HsiTriadStructPtr->Hue;

        Fraction = (HsiTriadStructPtr->Saturation *
                             cos(Hue / DEGREES_PER_RADIAN))
                                     /
                    cos((60.0 - Hue) / DEGREES_PER_RADIAN);

        r = (1.0 + Fraction) / 3.0;

        g = 1.0 - (r + b);

    } else if (HsiTriadStructPtr->Hue >  120.0  &&
               HsiTriadStructPtr->Hue <= 240.0     )
    {

        Hue = HsiTriadStructPtr->Hue - 120.0;

        r = (1.0 - HsiTriadStructPtr->Saturation) / 3.0;

        Fraction =  (HsiTriadStructPtr->Saturation *
                                       cos(Hue / DEGREES_PER_RADIAN))
                                               /
                              cos((60.0 - Hue) / DEGREES_PER_RADIAN);

        g = (1.0 + Fraction) / 3.0;

        b = 1.0 - (r + g);

    } else
    {
                                                                            Hue=
HsiTriadStructPtr->Hue - 240.0;

        g = (1.0 - HsiTriadStructPtr->Saturation) / 3.0;

        Fraction =  (HsiTriadStructPtr->Saturation *
                         cos(Hue / DEGREES_PER_RADIAN))
                                 /
                cos((60.0 - Hue) / DEGREES_PER_RADIAN);

        b = (1.0 + Fraction) / 3.0;

        r = 1.0 - (g + b);

    }

    MaxDouble = ((r > g)         ? r         : g);
    MaxDouble = ((MaxDouble > b) ? MaxDouble : b);

    if (3.0 * HsiTriadStructPtr->Intensity * MaxDouble > 1.0)
        HsiTriadStructPtr->Intensity = (1.0 / (3.0 * MaxDouble));

    r *= 255.0;
    g *= 255.0;
    b *= 255.0;

    RgbTriadStructPtr->Red   =
          (USGC) ((3.0 * HsiTriadStructPtr->Intensity * r) + 0.5);

    RgbTriadStructPtr->Green =
          (USGC) ((3.0 * HsiTriadStructPtr->Intensity * g) + 0.5);

    RgbTriadStructPtr->Blue  =
          (USGC) ((3.0 * HsiTriadStructPtr->Intensity * b) + 0.5);

    return 0;
}

/*=================== End of File ===================*/


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Appendix C
MATH.C Sample Program

/*---------------------------------------------------------------
-----------*/
/*                                                                         */
/*               Copyright (C) 1995, W. David Schwaderer                    */
/*           Some rights reserved - see WdsLogo() for details               */
/*                                                                         */
/*   Function: This program displays sample image processing output          */
/*             for number sequences.  See the text description in:          */
/*                                                                         */
/*            Digital Imaging in C and the World Wide Web                   */
/*                          W. David Schwaderer                             */
/*                     Wordware Publishing, Inc., 1997                     */
/*                                                                        */
/*    Compiler: Borland C++ 3.1   Compile command: bcc -ml Math.C          */
/*                                                                        */
/*    This is a large model PC MS-DOS(TM) program.                               */
/*                                                                        */
/*   Call to programmers...                                                */
/*                                                                        */
/*     If you are interested enough to read this code, your name            */
/*     probably should be in it.  I will place contributing programmer's */
/*     names in source code when their suggestion(s) results in bug fixes       */
/*     or program enhancements.  Contributions must be new. Send written         */
/*     suggestions on paper to W. David Schwaderer via Wordware Publishing,     */
/*     Inc., 1997.                                                          */
/*                                                                          */
/*     Complete or massive rewrites are not helpful (though possibly         */
/*     warranted!).  Contributions become the property of W. David           */
/*     Schwaderer who reserves suggestion final disposition determination.  */
/*                                                                         */
/*     Thanks in advance for the consideration.                             */
/*                                                W. David Schwaderer        */
/*                                                                          */
/*   Version 1.0 3/15/95                                                     */
/*                                                                          */
/*       Contributor                          Enhancement                    */
/*   ----------------------------
  -----------------------------------------     */
/*    Your name here...            Your edited suggestion here...           */
/*    Hometown, country            Your edited suggestion here...            */
/*                                                                          */
/*----------------------------------------------------------------
----------*/

#include <stdio.h>

int InputArray[  ] = {
                       0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 8,
                             2, 2, 4, 4, 6, 6, 8, 8, 8,
                       8,    6,    4,    2, 2, 2,
                       8, 8, 8, 2, 2, 2,
                       8, 8, 8, 4, 4, 4,
                       8, 8, 8, 6, 6, 6,
                       8,    6,    4,    2,    0,
                       8, 8, 8, 7, 6, 5, 4, 3, 2, 1, 0, };

#define MAX_GRAPH_ROW 15

#define MAX_MASK_WIDTH 3

#define ARRAY_OFFSET (MAX_MASK_WIDTH / 2)

int OutputArray   [(sizeof(InputArray) / sizeof(int))];
int SmoothingArray[(sizeof(InputArray) / sizeof(int))];

int StagingArray[(sizeof(InputArray) / sizeof(int)) + (2 * ARRAY_OFFSET)];

#define INPUT_ELEMENT_COUNT      ((sizeof(InputArray)   / sizeof(int)))
#define STAGING_ARRAY_SIZE ((sizeof(StagingArray) / sizeof(int)))

void LoadStagingArray(void);
void PerformSmoothing(void);
void PerformSharpening(void);
void PerformUnsharpMasking(void);
void GraphArray(int *IntArrayPtr);
void DisplayInputOutputDifference(void);

int main()
{

    printf("\nmain: INPUT_ELEMENT_COUNT == %d, ARRAY_OFFSET == %d",
                    INPUT_ELEMENT_COUNT      , ARRAY_OFFSET         );

    printf("\nmain: sizeof(StagingArray) == %d",
                    sizeof(StagingArray)         );

    LoadStagingArray();

    printf("\n\fmain: InputArray[] follows...");

    GraphArray(InputArray);

    PerformSmoothing();
    PerformSharpening();
    PerformUnsharpMasking();

    printf("\n\fmain: Returning...\n");

    return 0;
}

void LoadStagingArray(void)
{
    unsigned Index;

    for (Index = 0; Index < ARRAY_OFFSET;  Index++)
        StagingArray[Index]                = InputArray[0];

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
        StagingArray[Index + ARRAY_OFFSET] = InputArray[Index];

    for (Index = 0; Index < ARRAY_OFFSET;  Index++)
        StagingArray[Index + INPUT_ELEMENT_COUNT + 1] =
                  InputArray[INPUT_ELEMENT_COUNT - 1];

    return;
}

void DisplayStagingArray(void)
{
    unsigned Index;

    putchar('\n');

    for (Index = 0; Index < STAGING_ARRAY_SIZE; Index++)
    {
        printf("\nStagingArray[%3d] = %d",
                      Index, StagingArray[Index]);
    }

    return;
}

void GraphArray(int *IntArrayPtr)
{
    int Row, Column;

    printf("\n");

    for (Row = 0; Row < MAX_GRAPH_ROW -1 ; Row++)
    {
        printf("\n%4d |", (MAX_GRAPH_ROW - Row - 1));

        for (Column = 0; Column < INPUT_ELEMENT_COUNT; Column++)
            if ((MAX_GRAPH_ROW - Row - 1) > IntArrayPtr[Column])
                putchar('-');
            else
                putchar('*');

        printf("|%3u", (MAX_GRAPH_ROW - Row - 1));
    }

    printf("\n%4d +", 0);

    for (Column = 0; Column < INPUT_ELEMENT_COUNT; Column++)
        putchar('-');

    printf("+%3u", 0);

    printf("\nInput:");

    for (Column = 0; Column < INPUT_ELEMENT_COUNT; Column++)
        printf("%1u", InputArray[Column]);

    printf("\n      ");

    for (Column = 0; Column < INPUT_ELEMENT_COUNT; Column++)
        if (IntArrayPtr[Column] == InputArray[Column])
            putchar('=');
        else
            if (IntArrayPtr[Column] > InputArray[Column])
                putchar('+');
            else
                putchar('-');

    return;
}

void DisplayInputOutputDifference(void)
{
    unsigned Index;

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
    {
        printf("\nGraphArray: InputArray[%3d] == %3d -> %3d",
                      Index, InputArray[Index], OutputArray[Index]);

        if (InputArray[Index] != OutputArray[Index])
          printf(" * (%3d)",(OutputArray[Index] - InputArray[Index]));
    }
}

void PerformSmoothing(void)
{
    unsigned Index;

    printf("\n\f");

    printf("PerformSmoothing: 1/3 [+1,+1,+1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET + INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] =
                  (StagingArray[Index - 1] +
                   StagingArray[Index    ] +
                   StagingArray[Index + 1])
                              /
                              3;
    }

    GraphArray(OutputArray);

    printf("\n\nPerformSmoothing: 1/4 [+1,+2,+1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET + INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] =
                  (StagingArray[Index - 1] +
               2 * StagingArray[Index    ] +  StagingArray[Index + 1])
                              /
                              4;
    }

    GraphArray(OutputArray);

    printf("\n\nPerformSmoothing: 1/2 [+1,0,+1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET + INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] =
                 (StagingArray[Index - 1] + StagingArray[Index + 1])
                                          /
                                          2;
    }

    GraphArray(OutputArray);

    return;
}

void PerformSharpening(void)
{
    unsigned Index;

    printf("\n\f");

    printf("PerformSharpening: 1/2 [-1,+2,+1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET + INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] =
                                   StagingArray[Index] +
                     ((StagingArray[Index+1] - StagingArray[Index-1]) / 2);

    }

    GraphArray(OutputArray);

    printf("\n\nPerformSharpening: 1/2 [+1,+2,-1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET+ INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] =
                                 StagingArray[Index] +
                             ((StagingArray[Index-1] - StagingArray[Index+1]) / 2);
    }

    GraphArray(OutputArray);

    printf("\n\fPerformSharpening: [+1,+1,-1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET+ INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] = StagingArray[Index-1] +
                                         StagingArray[Index  ] -
                                         StagingArray[Index+1];
    }

    GraphArray(OutputArray);

    printf("\n\nPerformSharpening: [-1,+1,+1] OutputArray[]...");

    for (Index = ARRAY_OFFSET;
         Index < ARRAY_OFFSET+ INPUT_ELEMENT_COUNT;
         Index++)
    {
        OutputArray[Index - ARRAY_OFFSET] = StagingArray[Index+1] +
                                         StagingArray[Index  ] -
                                         StagingArray[Index-1];
    }

    GraphArray(OutputArray);

    return;
}

void PerformUnsharpMasking(void)
{
    unsigned Index;

    printf("\n\f");

    printf("PerformUnsharpMasking: 1/3 [+1,+1,+1] OutputArray[]...");

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
    {
        SmoothingArray[Index] =
                 (StagingArray[Index - 1 + ARRAY_OFFSET] +
                  StagingArray[Index     + ARRAY_OFFSET] +
                  StagingArray[Index + 1 + ARRAY_OFFSET])
                            /
                            3;
    }

    for (Index = 0;
         Index < INPUT_ELEMENT_COUNT;
         Index++)
        OutputArray[Index] = 2 * InputArray[Index] - SmoothingArray[Index];

    GraphArray(OutputArray);

    printf("\n\nPerformUnsharpMasking: 1/4 [+1,+2,+1] OutputArray[]...");

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
    {
        SmoothingArray[Index] =
                 (StagingArray[Index - 1 + ARRAY_OFFSET] +
              2 * StagingArray[Index     + ARRAY_OFFSET] +
                  StagingArray[Index + 1 + ARRAY_OFFSET])
                            /
                            4;
    }

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
        OutputArray[Index] = 2 * InputArray[Index] - SmoothingArray[Index];

    GraphArray(OutputArray);

    printf("\n\nPerformUnsharpMasking: 1/4 [+1,+2,+1] OutputArray[]...");

    for (Index = 0; Index < INPUT_ELEMENT_COUNT; Index++)
        OutputArray[Index] =
              (2 * InputArray[Index]) -
              (   (StagingArray[Index - 1 + ARRAY_OFFSET] +
               2 * StagingArray[Index     + ARRAY_OFFSET] +
                   StagingArray[Index + 1 + ARRAY_OFFSET])
                            /
                            4);

    GraphArray(OutputArray);

    return;
}

/*============================  End of File
============================*/


Digital Imaging in C and the World Wide Web

 Digital Imaging in C and the World Wide Web - David Schwaderer

Appendix D
VESA.H and PCX.H Headers

VESA.H

#define VESA_MODES_ONLY 1

#define VIDEO_INT     0x10
#define VIDEO_MODE_0x03 0x03
#define VESA_FUNCTION  0x4F

#define VESA_GRAPHICS_640x400x256     0x0100
#define VESA_GRAPHICS_640x480x256     0x0101
#define VESA_GRAPHICS_800x600x16      0x0102
#define VESA_GRAPHICS_800x600x256     0x0103
#define VESA_GRAPHICS_1024x768x16     0x0104
#define VESA_GRAPHICS_1024x768x256    0x0105
#define VESA_GRAPHICS_1280x1024x16    0x0106
#define VESA_GRAPHICS_1280x1024x256   0x0107
#define VESA_TEXT_80x60             0x0108
#define VESA_TEXT_132x25            0x0109
#define VESA_TEXT_132x43            0x010A
#define VESA_TEXT_132x50            0x010B
#define VESA_TEXT_132x60            0x010C
#define VESA_DIRECT_320x200_1_5_5_5   0x010D
#define VESA_DIRECT_320x200_0_5_6_5   0x010E
#define VESA_DIRECT_320x200_0_8_8_8   0x010F
#define VESA_DIRECT_640x480_1_5_5_5   0x010D
#define VESA_DIRECT_640x480_0_5_6_5   0x010E
#define VESA_DIRECT_640x480_0_8_8_8   0x010F
#define VESA_DIRECT_800x600_1_5_5_5   0x0110
#define VESA_DIRECT_800x600_0_5_6_5   0x0111
#define VESA_DIRECT_800x600_0_8_8_8   0x0112
#define VESA_DIRECT_1024x768_1_5_5_5  0x0113
#define VESA_DIRECT_1024x768_0_5_6_5  0x0114
#define VESA_DIRECT_1024x768_0_8_8_8  0x0115
#define VESA_DIRECT_1280x1024_1_5_5_5 0x0116
#define VESA_DIRECT_1280x1024_0_5_6_5 0x0117
#define VESA_DIRECT_1280x1024_0_8_8_8 0x0118

#define VESA_WINDOW_A 0
#define VESA_WINDOW_B 1

#define VESA_CLEAR_VIDEO_MEMORY_FLAG       0x0000
#define VESA_DO_NOT_CLEAR_VIDEO_MEMORY_FLAG 0x8000

#define VESA_RETURN_INFO              0x00
#define VESA_RETURN_MODE_INFO          0x01
#define VESA_SET_MODE                 0x02
#define VESA_RETURN_CURRENT_MODE        0x03
#define VESA_SAVE_OR_RESTORE_STATE     0x04
#define VESA_WINDOW_CONTROL            0x05
#define VESA_LOGICAL_SCAN_LINE_LENGTH  0x06
#define VESA_SET_OR_GET_DISPLAY_START 0x07
#define VESA_SET_OR_GET_DAC_CONTROL    0x08

#define VESA_SELECT_DISPLAY_START  0x00
#define VESA_RETURN_DISPLAY_START   0x01

#define VESA_SET_DAC_PALETTE_WIDTH 0x00
#define VESA_GET_DAC_PALETTE_WIDTH 0x01

#define VESA_CALL_SUCCESSFUL       0x00
#define VESA_FUNCTION_SUPPORTED     0x4F
#define VESA_DAC_WIDTH_TEST_BIT   0x01
#define VESA_VIDEO_MODE_TEST_BIT  0x0100
#define VESA_SELECT_WINDOW_GRANULE  0x00
#define VESA_RETURN_WINDOW_GRANULE  0x01

#define VESA_MODE_AVAILABLE_TEST_BIT           0x0001
#define VESA_MODE_EXTENDED_INFO_AVAIL_TEST_BIT  0x0002
#define VESA_MODE_OUTPUT_FUN_TEST_BIT           0x0004
#define VESA_MODE_MONO_COLOR_TEST_BIT            0x0008
#define VESA_MODE_MONO_TYPE_TEST_BIT            0x0010

#define VESA_WINDOW_SUPPORTED_TEST_BIT   0x01
#define VESA_WINDOW_READABLE_TEST_BIT   0x02
#define VESA_WINDOW_WRITABLE_TEST_BIT  0x04

#define VESA_MODE_TEXT                  0x00
#define VESA_MODE_CGA_GRAPHICS           0x01
#define VESA_MODE_HERCULES_GRAPHICS       0x02
#define VESA_MODE_4_PLANE_PLANAR         0x03
#define VESA_MODE_PACKED_PIXEL          0x04
#define VESA_MODE_NON_CHAIN_4_256_COLOR  0x05
#define VESA_MODE_DIRECT_COLOR          0x06
#define VESA_MODE_YCbCr                0x07

#define VESA_PROGRAMMABLE_DIRECT_COLOR_RAMP_TEST_BIT 0x01
#define VESA_RESERVED_BITS_ARE_USABLE_TEST_BIT      0x02

struct VesaInfoBlock
    {
        USGC      VesaSignature[4];  /* 'VESA'           */
        USGI      VesaVersion;      /* eg. 0x0102 == 1.2  */
        char far *OemStringPtr;     /* who dat?          */
        USGC      Capabilities[4];
        USGI far *VideoModePtr;
        USGI      Total64kMemoryBlocks;
        char      Reserved[256]; /* more than enough space */
    }

struct VesaModeInfoBlock
    {
        USGI ModeAttributes;
        USGC WinAAttributes;
        USGC WinBAttributes;
        USGI WinGranularity;
        USGI WinSize;
        USGI WinASegment;
        USGI WinBSegment;
        void ((far *WinFuncPtr)());
        USGI BytesPerScanLine;
        USGI XResolution;
        USGI YResolution;
        USGC XCharSize;
        USGC YCharSize;
        USGC NumberOfPlanes;
        USGC BitsPerPixel;
        USGC NumberOfBanks;
        USGC MemoryModel;
        USGC BankSize;
        USGC NumberOfImagePages;
        USGC ReservedForPageFunction;
        USGC RedMaskSize;
        USGC RedFieldPosition;
        USGC GreenMaskSize;
        USGC GreenFieldPosition;
        USGC BlueMaskSize;
        USGC BlueFieldPosition;
        USGC RsvdMaskSize;
        USGC RsvdFieldPosition;
        USGC DirectColorModeInfo;

        char Reserved[256]; /* more than enough space */
    };

PCX.H

#define PCX_FILE_FLAG             0x0A
#define PCX_FILE_RUN_LENGTH_ENCODING 0x01

#define PCX_FILE_VERSION_2_5               0x00
#define PCX_FILE_VERSION_2_8_WITH_PALETTE   0x02
#define PCX_FILE_VERSION_2_8_WITHOUT_PALETTE 0x03
#define PCX_FILE_PC_PAINTBRUSH_FOR_WINDOWS    0x04
#define PCX_FILE_VERSION_3_0               0x05

#define PCX_FILE_COLOR_BW_PALETTE  0x01
#define PCX_FILE_GRAYSCALE_PALETTE 0x02

#define PCX_FILE_IMAGE_DATA_OFFSET             (128L)
#define PCX_FILE_256_COLOR_PALETTE_FLAG           0x0C
#define PCX_FILE_256_COLOR_PALETTE_FLAG_OFFSET   (-769L)

#define PCX_FILE_16_COLOR_PALETTE_SIZE  (3 * 16)
#define PCX_FILE_16_COLOR_PALETTE_OFFSET  (16)

#define PCX_RUN_LENGTH_INDICATOR_MASK    0x00C0

struct PcxHeaderStruct {
                        USGC ManufacturerFlag;
                        USGC FileFormatVersionNumber;
                        USGC EncodingMethod;
                        USGC BitsPerPixelPerPlane;
                        USGI XMin, YMin, XMax, YMax;
                        USGI SuggestedHorizontalDPI;
                        USGI SuggestedVerticalDPI;
                        USGC SixteenColorPalette[48];
                        USGC Reserved;
                        USGC NumberOfColorPlanes;
                        USGI BytesPerScanLine;
                        USGI PaletteInformation;
                        USGI ScreenHorizontalPixelCount;
                        USGI ScreenVerticalPixelCount;
                        USGC Filler[54];
                       };


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held
Table of Contents

About the CD

The CD-ROM included with this book contains the example programs discussed in the book. Most of the programs are included as BASIC text files (.BAS), C++ text files (.CPP), and executable programs (.EXE).

To copy the software to your hard drive, simply drag and drop the two directories to your drive. As these files require 4.2 megabytes of free space on your hard drive, you may wish to access the files you need directly from the CD, rather than copying them to your hard drive.

See Appendix A for more information about the files on the CD.

Notice: Opening the CD package makes this book NONRETURNABLE.

Table of Contents


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held
Table of Contents

Appendix A
Companion CD-ROM Files

This appendix lists the names of each of the files contained on the companion CD-ROM, as well as a description of the function of each file. To facilitate readers obtaining additional information concerning the operation and utilization of each program, files are listed in this appendix in the order in which they are described in each chapter, by chapter. The .EXE files, which are executable versions of the BASIC and C++ programs, are not listed in the book; however, they are listed below in the same chapter in which the BASIC or C++ program is described.

There are two directories on the CD: BASIC and C. The BASIC directory contains BASIC language source and executable programs, while the C directory contains C++ language source and executable programs. In the chart below, the .EXE file is listed only once with the name of the corresponding program followed by .xxx. The .CPP executable is in the C directory, and the .BAS executable is in the BASIC directory.

Chapter | Filename | Description
2 | SHIFT.BAS | This program creates a sequence of shifted alphabets.
 | SHIFT.CPP | This program is the C++ version of SHIFT.BAS.
 | SHIFT.EXE | This is the executable version of SHIFT.CPP.
 | CIPHER1.BAS | This program creates a shifted alphabet based upon the entry of an alphabetic shift key character.
 | CIPHER1.CPP | This program is the C++ version of CIPHER1.BAS.
 | CIPHER1.EXE | This is the executable version of CIPHER1.xxx.
 | CIPHER2.BAS | This program enciphers a one-line message using a cipher alphabet formed by shifting the plaintext alphabet through the use of an alphabetic shift key.
 | CIPHER2.CPP | This program is the C++ version of CIPHER2.BAS.
 | CIPHER2.EXE | This is the executable version of CIPHER2.xxx.
 | CIPHER3.BAS | This program expands upon the functionality of CIPHER2.BAS by adding a routine which displays the resulting enciphered text in groups of five characters.
 | CIPHER3.CPP | This program is the C++ version of CIPHER3.BAS.
 | CIPHER3.EXE | This is the executable version of CIPHER3.xxx.
 | CIPHER4.BAS | This program expands upon the functionality of CIPHER3.BAS by permitting spaces between words in a plaintext message and selectively enciphering the message using a simple monoalphabetic substitution process.
 | CIPHER4.CPP | This program is the C++ version of CIPHER4.BAS.
 | CIPHER4.EXE | This is the executable version of CIPHER4.xxx.
 | DCIPHER4.BAS | This program deciphers a message previously enciphered through the use of CIPHER4.BAS.
 | DCIPHER4.CPP | This program is the C++ version of DCIPHER4.BAS.
 | DCIPHER4.EXE | This is the executable version of DCIPHER4.xxx.
3 | WORD.BAS | This program develops an alphabet based upon the entry of a keyword or keyword phrase.
 | WORD.CPP | This program is the C++ version of WORD.BAS.
 | WORD.EXE | This is the executable version of WORD.xxx.
 | CIPHER5.BAS | This program enciphers a message based upon the use of a keyword or keyword phrase and an alphabetic shift key using a monoalphabetic substitution process.
 | CIPHER5.CPP | This program is the C++ version of CIPHER5.BAS.
 | CIPHER5.EXE | This is the executable version of CIPHER5.xxx.
 | DCIPHER5.BAS | This program deciphers a message enciphered using the program DCIPHER5.BAS.
 | DCIPHER5.CPP | This program is the C++ version of DCIPHER5.BAS.
 | DCIPHER5.EXE | This is the executable version of DCIPHER5.xxx.
4 | CIPHERTR.BAS | This program enciphers a message using a transposition matrix and a monoalphabetic substitution process based upon a keyword or keyword phrase and an alphabetic shift key. This program displays the composition of different alphabets to illustrate the operation of the enciphering process.
 | CIPHERTR.CPP | This program is the C++ version of CIPHERTR.BAS.
 | CIPHERTR.EXE | This is the executable version of CIPHERTR.xxx.
 | TRANSPORT.CPP | This program is a C++ expansion of BASIC's TRANSPORT subroutine.
 | TRANSPORT.EXE | This is the executable version of TRANSPORT.CPP.
 | CIPHER6.BAS | This program adds an interval extraction capability to CIPHERTR.BAS.
 | CIPHER6.CPP | This program is the C++ version of CIPHER6.BAS.
 | CIPHER6.EXE | This is the executable version of CIPHER6.xxx.
 | DCIPHER6.BAS | This program deciphers messages enciphered using the program CIPHER6.BAS.
 | DCIPHER6.CPP | This program is the C++ version of DCIPHER6.BAS.
 | DCIPHER6.EXE | This is the executable version of DCIPHER6.xxx.
5 | POLY1.BAS | This program creates a Vigenére tableau.
 | POLY1.CPP | This program is the C++ version of POLY1.BAS.
 | POLY1.EXE | This is the executable version of POLY1.xxx.
 | COUNT.CPP | This program performs a frequency count of the contents of an encrypted file.
 | COUNT.EXE | This is the executable version of COUNT.CPP.
 | POLY2.BAS | This program enciphers a message based upon the use of two keywords or keyword phrases using a polyalphabetic substitution process and 26 cipher alphabets.
 | POLY2.CPP | This program is the C++ version of POLY2.BAS.
 | POLY2.EXE | This is the executable version of POLY2.xxx.
 | DPOLY2.BAS | This program deciphers messages previously enciphered using the program POLY2.BAS.
 | DPOLY2.CPP | This program is the C++ version of DPOLY2.BAS.
 | DPOLY2.EXE | This is the executable version of DPOLY2.xxx.
6 | RANDOM1.BAS | This program generates random numbers between 0 and 25.
 | RANDOM1.CPP | This program is the C++ version of RANDOM1.BAS.
 | RANDOM1.EXE | This is the executable version of RANDOM1.xxx.
 | RANDOM2.BAS | This program demonstrates the use of random numbers to encipher a one-line message.
 | RANDOM2.CPP | This program is the C++ version of RANDOM2.BAS.
 | RANDOM2.EXE | This is the executable version of RANDOM2.xxx.
 | DRANDOM2.BAS | This programs deciphers a one-line message previously enciphered using the program RANDOM2.BAS.
 | DRANDOM2.CPP | This program is the C++ version of DRANDOM2.BAS.
 | DRANDOM2.EXE | This is the executable version of DRANDOM2.xxx.
 | ASCII.CPP | This program displays the ASCII value of characters.
 | ASCII.EXE | This is the executable version of ASCII.CPP.
 | POSITION.BAS | This program demonstrates the positioning to a place in the BASIC random number generator based upon a "secret" code up to six characters in length.
 | POSITION.CPP | This program is the C++ version of POSITION.BAS.
 | POSITION.EXE | This is the executable version of POSITION.xxx.
 | RTEST.BAS | This program scans a random number seed to locate any sequence of five repeated numbers.
 | RTEST.EXE | This is the executable version of RTEST.BAS.
 | RANDOM3.BAS | This program enciphers messages using the BASIC random number generator and a six-position "secret" code to locate a starting point in the random number generator.
 | RANDOM3.CPP | This program is the C++ version of RANDOM3.BAS.
 | RANDOM3.EXE | This is the executable version of RANDOM3.xxx.
 | DRANDOM3.BAS | This program deciphers messages previously enciphered using the program RANDOM3.BAS.
 | DRANDOM3.CPP | This program is the C++ version of DRANDOM3.BAS.
 | DRANDOM3.EXE | This is the executable version of DRANDOM3.xxx.
7 | ENCIPHER.BAS | This program performs encipherment using seven previously developed routines.
 | ENCIPHER.EXE | This is the executable version of ENCIPHER.BAS.
 | DECIPHER.BAS | This program deciphers data previously enciphered using the ENCIPHER.BAS program.
 | DECIPHER.EXE | This is the executable version of DECIPHER.BAS.
 | SECRET.MSG | This file contains the challenge message that readers can attempt to decipher.

Table of Contents


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held
Table of Contents

Index

A

alphabet,
cipher, 230
ciphertext, 18, 25
displacement, 27
keyword-based mixed, 95
numeric-based transposition, 142
plaintext, 18, 25
reverse ciphertext, 31
alphabetic relationships, 25-27
alternative, 133
common, 134
alphabetic shift key, 30
combining with keyword-based mixed alphabet, 104
using, 33
authentication, 370-371

B

backslash character, (\), 58
BASIC subroutines, 354-355
Baudot code, 274-275
binary random number sequences, 274

C

C++ language, 19
Caesar cipher, 16, 24
challenge message, 366
checkInput function, 100
cipher alphabet, 230
cipher disk, 222
cipher system, 2-4
cipher systems,
substitution, 15-17
transposition, 15, 17-18
types of, 15-18
cipher,
Caesar, 16, 24
Playfair, 218-219
Vigenere, 223-229
ciphers, 2-4
ciphertext, 6
converting to, 39-47
ciphertext alphabet, 18, 25
cleartext, see plaintext
code positions, 307
code system, 2-4
codes, 2-4
CONVERTSTORE subroutine, 58-60, 354
createAlphabet function, 100
createCipher function, 42, 94
cryptology, 1-2

D

DCONVERTSTORE subroutine, 129-130, 354
decipher function, 94
DECIPHER subroutine, 84, 354
DECIPHER.EXE execution, 364-366
decipherment, 3, 6-8, 32-33
reducing probability of, 232-233
decipherment process, 6-8
decoding, 3
decrypt function, 335
decryptFile function, 335
decryptKeyboardText function, 335
Diffie-Hellman concept, 372-373
digital signature, 370
displacement, 26-27, 31
displacement alphabet, 27
DMSGFILE subroutine, 82-84, 128-129, 264, 354
DO UNTIL loop, 59
DPCONVERTSTORE subroutine, 264-265
DPRTOUT subroutine, 85, 354

E

electronic mail systems, 18-19
ENCIPHER subroutine, 282
ENCIPHER.EXE execution, 362-364
encipherment, 3, 5, 31-32
RSA, 386-387
encipherment process, 5-6
encoding, 3
encryptFile function, 322
encryption systems,
private key based, 10-12
public key based, 10-12
types of, 10
encryptKeyboardText function, 322
errors, limiting, 47
Euclidean algorithm, 382-384
exponentiation, 387-390

F

Fermat's theorem, 391
formatData function, 94
FORMCIPHER subroutine, 34, 354
formCipheredMessage function, 42
FOR-NEXT statement, 35, 41
frequency analysis, 136, 217, 221

G

getFileToDecipher function, 94
getKeyword function, 100
getMessage function, 42
getShiftKey function, 42, 94
GOSUB statement, 67
greatest common divisor, 382
GROUPBY5 subroutine, 47-48, 354
modified, 63-64

I

INITIALIZE subroutine, 33, 96, 354
modified, 105, 356-358
INPUT statement, 34
INT function, 279
interval extraction, 143-144
interval function, 187
INTERVAL subroutine, 151-152, 354

K

Kerckhoffs method, 234
key, 4, 8-10
distribution, 11, 13-14
generation, 11, 13, 391
management, 12-14
key space, 8-10
keyword phrase, 96
KEYWORD subroutine, 96, 98-99, 354, 358-359
keyword-based mixed alphabet, 95
combining with alphabetic shift key, 104
constructing, 95-96
keyword-based monoalphabetic substitution,
deciphering process, 125-133
enciphering process, 106-125
weaknesses of, 135-136

L

LEN statement, 40
LTRIM$ statement, 58

M

MAGIC, 17
Manhattan Project, 143
matrix-based systems,
automating, 144-216
decipherment of, 211-216
encipherment of, 144-211
matrix-based transposition, 137-144
mnemonic key, 98
modular addition, 375-377
modular arithmetic, 375-380
finding multiplicative inverse in, 380
modular multiplication, 378-380
modular subtraction, 378
modulo arithmetic, 6-8
modulo 9 arithmetic, 376
modulo 26 arithmetic, 291
encipherment/decipherment, 291-292
monoalphabetic substitution, 23
transposition-based, 137-144
monoalphabetic unilateral substitution, 23
MSGDCIPHER subroutine, 354
MSGENCIPHER subroutine, 40-41, 354
MSGFILE subroutine, 57-58, 354
multiple-line messages, processing, 56-57
multiplicative inverses, 380

N

numeric transposition, 141-143
numeric-based transposition alphabet, 142
numericExtraction function, 187

O

Oite, 48
one-time pad, 277-278

P

PA-K2, 48
PCONVERTSTORE subroutine, 237-238
periodic polyalphabetic substitution, 230-233
PFORMCIPHER subroutine, 236-237
modified, 360
plaintext, 3, 5
plaintext alphabet, 18, 25
Playfair cipher, 218-219
pocket cipher, 229
polyalphabetic substitution systems, 221-234
periodic, 230-233
simple, 223-229
weaknesses of, 234
prime numbers, 381
large, 391-392
locating, 391-392
Mersenne primes, 381
Sophie Germain primes, 381
twin primes, 381
PRINTIT subroutine, 224
printResults function, 42
private key based encryption, 10-12
private key encryption, 371
private key generation, RSA, 386
probability of decipherment, reducing, 232-233
probable word attack, 229
program listings
ASCII.CPP, 309-310
CIPHER1.BAS, 33-34
CIPHER1.CPP, 35-39
CIPHER2.BAS, 39-40
CIPHER2.CPP, 42-47
CIPHER3.BAS, 49-50
CIPHER3.CPP, 51-56
CIPHER4.BAS, 64-67
CIPHER4.CPP, 70-81
CIPHER5.BAS (main portion), 105
CIPHER5.CPP, 109-125
CIPHER6.BAS, 184-185
CIPHER6.CPP, 187-211
CIPHERTR.BAS (main portion), 148-149
CIPHERTR.CPP, 161-183
combining random number sequence and three-numeric key, 304
COUNT.CPP, 240-245
DCIPHER4.BAS (main portion), 86
DCIPHER4.CPP, 88-94
DCIPHER5.BAS (main portion), 127-128
DCIPHER5.CPP, 132-133
DCIPHER6.BAS (main portion), 212
DCIPHER6.CPP (decipher function),215-216
DPOLY2.BAS (main portion), 263
DPOLY2.CPP, 267-269
DPOLY2.CPP (decipher and getFileToDecipher functions), 269-271
DRANDOM2.BAS, 292-293
DRANDOM2.CPP, 295-299
DRANDOM3.BAS (main portion), 332
DRANDOM3.CPP, 336-345
ENCIPHER.BAS (main portion), 356
POLY1.BAS, 224-225
POLY1.CPP, 226-228
POLY2.BAS (main portion), 235-236
POLY2.CPP, 246-262
POSITION.BAS, 313
POSITION.CPP, 314-317
random number generator, 301-302
RANDOM1.BAS, 280
RANDOM1.CPP, 283-285
RANDOM2.BAS, 281
RANDOM2.CPP, 286-291
RANDOM3.BAS (statements in main portion), 319
RANDOM3.CPP, 322-331
ROTOR.BAS, 347-348
RTEST.BAS, 318
SHIFT.BAS, 28
SHIFT.CPP, 29-30
TRANSPORT.CPP, 153-161
using ASCII character values, 308
WORD.BAS, 96-97
WORD.CPP, 100-103
PRTOUT subroutine, 35, 60-63, 354
pseudo-random number generator, creating, 305-306
pseudo-random number sequences, 273, 276
generating, 276-279
pseudo-random sequence, 16
public key based encryption, 10-12
public key encryption, 369-370
advantages, 373
disadvantages, 373-374
general operation of, 369-370
public key generation, RSA, 386
PURPLE, 17

Q

QuickBASIC language, 19

R

random number generation, 277-279
alternative process for, 345-351
random number generator, creating, 305
random number sequences, 273
generating, 275
random numbers, 273
extending security of, 301, 303
RANDOMIZE SEED1 statement, 349
RANDOMIZE SEED2 statement, 349
RANDOMIZE statement, 278-279
RCONVERTSTORE subroutine, 319-321, 354
modified, 360-361
RDCONVERTSTORE subroutine, 332-333, 355
REM statement, 98
repetition period, 231
RESTORE statement, 106
reverse ciphertext alphabet, 31
RND function, 276-277
ROTOR subroutine, 350
RSA, 385-387

S

SETUP subroutine, 354, 361-362
simple transposition, 140-141
six-position code, 310-312
slash character (/), 58
subroutines,
CONVERTSTORE, 58-60, 354
DCONVERTSTORE, 129-130, 354
DECIPHER, 84, 354
DMSGFILE, 82-84, 128-129, 264, 354
DPRTOUT, 85, 354
ENCIPHER, 282
FORMCIPHER, 34-35, 354
GROUPBY5, 47-48, 354
GROUPBY5, modified, 63-64
INITIALIZE, 33, 96, 354
INITIALIZE, modified, 105, 356-358
INTERVAL, 151-152, 354
KEYWORD, 96, 98-99, 354, 358-359
MSGDCIPHER, 354
MSGENCIPHER, 40-41, 354
MSGFILE, 57-58, 354
PCONVERTSTORE, 237-238
PFORMCIPHER, 236-237
PFORMCIPHER, modified, 360
PRINTIT, 224
PRTOUT, 35, 60-63, 354
RCONVERTSTORE, 319-321, 354
RCONVERTSTORE, modified, 360-361
RDCONVERTSTORE, 332-333, 355
ROTOR, 350
SETUP, 354, 361-362
SPCONVERTSTORE, 264-265
TABLE, 224
TRANSPORT, 145-148, 354
substitution cipher systems, 15-17
symmetric key system, see private key based encryption

T

TABLE subroutine, 224
totient function, 384-385
TRANSPORT subroutine, 145-148, 354
transposition,
matrix-based, 137-144
simple, 140-141
transposition cipher systems, 15, 17-18
transposition-based monoalphabetic substitution, 137-144

U

unilateral substitution, 23-24
UUDECODE, 18
UUENCODE, 18

V

Vigenére cipher system, 223-229
basic tableau, 223-224
encipherment, 228-229

W

WHILE-WEND loop, 62

Table of Contents


Learn Encryption Techniques with BASIC and C++

 [Spisak knjiga]

Learn Encryption Techniques with BASIC and C++
Author: Gilbert Held

Preface
About the CD

Chapter 1—Technology and Terminology
Ciphers Versus Codes
Cipher Terminology
Plaintext and Enciphered Text
Encipherment
Decipherment
Keys and Key Space
Types of Encryption Systems
Key Generation, Management, and Distribution
Types of Ciphers
Substitution Cipher Systems
Transposition Cipher Systems
Electronic Mail Transmission Considerations
Subroutines and Programs
File Naming Conventions
Chapter 2—Monoalphabetic Substitution Concepts
Evolution
Alphabets
Alphabetic Relationships
Displacement Alphabets
Encipherment
Decipherment
Automating Operations
Using an Alphabetic Shift Key
Converting to Ciphertext
The MSGENCIPHER Subroutine
The CIPHER2.CPP Program
Limiting the Effect of Errors
The GROUPBY5 Subroutine
The CIPHER3.BAS Program
The CIPHER3.CPP Program
Processing Multiple-Line Messages
The MSGFILE Subroutine
The CONVERTSTORE Subroutine
The PRTOUT Subroutine
Modifying the GROUPBY5 Subroutine
The CIPHER4.BAS Program
The CIPHER4.CPP Program
The DCIPHER4.BAS Program
The DMSGFILE Subroutine
The DECIPHER Subroutine
The DPRTOUT Subroutine
The DCIPHER4.CPP Program
Chapter 3—Keyword-Based Monoalphabetic Substitution
Keyword-Based Mixed Alphabets
Construction
Automating Keyword Construction
The KEYWORD Subroutine
Incorporating an Alphabetic Shift Key
The CIPHER5.BAS Program
The Encipherment Process
The CIPHER5.CPP Program
Deciphering Operations
The Decipherment Process
The DCIPHER5.BAS Program
The DCIPHER5.CPP Program
Alternative Relationships
Weaknesses
Chapter 4—Transposition-based Monoalphabetic Substitution
Matrix-Based Transposition
Simple Transposition
Encipherment
Decipherment
Numeric Transposition
Encipherment and Decipherment
Other Variations
Interval Extraction
Encipherment and Decipherment
Automating Matrix-Based Systems
Encipherment
The TRANSPORT Subroutine
The CIPHERTR.BAS Program
The INTERVAL Subroutine
The TRANSPORT.CPP Program
The CIPHERTR.CPP Program
The CIPHER6.BAS Program
The CIPHER6.CPP Program
Decipherment
The DCIPHER6.BAS Program
The DCIPHER6.CPP Program
Monoalphabetic Combinations
Chapter 5—Polyalphabetic Substitution
Simple Polyalphabetic Substitution Systems
The Vigenére Cipher
The POLY1.BAS Program
The POLY1.CPP Program
Encipherment
Other Polyalphabetic Substitution Methods
Periodic Polyalphabetic Substitution
Operation
Reducing Probability of Decipherment
The POLY2.BAS Program
The PFORMCIPHER Subroutine
The PCONVERTSTORE Subroutine
Program Execution
The COUNT.CPP Program
The POLY2.CPP Program
The DPOLY2.BAS Program
The DMSGFILE Subroutine
The DPCONVERTSTORE Subroutine
Program Execution
The DPOLY2.CPP Program
Chapter 6—Using Random Numbers
Random Numbers and Random Number Sequences
Generating Random Number Sequences
Pseudo-random Number Sequences
The RND Function
RANDOMIZE Statement
Working with Random Numbers
The RANDOM1.BAS Program
The RANDOM2.BAS Program
The RANDOM1.CPP Program
The RANDOM2.CPP Program
Modulo 26 Arithmetic
The DRANDOM2.BAS Program
The DRANDOM2.CPP Program
Constructing an Enciphering Program
Extending the Random Process
Extending the Combinations
Creating Your Own Random Number Generator
Program Development
The POSITION.BAS Program
The POSITION.CPP Program
The RTEST.BAS Program
The RANDOM3.BAS Program
The RCONVERTSTORE Subroutine
The RANDOM3.CPP Program
The DRANDOM3.BAS Program
The RDCONVERTSTORE Subroutine
The DRANDOM3.CPP Program
An Alternative Random Process
The ROTOR.BAS Program
Chapter 7—Developing Practical Programs
Module Utilization
The ENCIPHER.EXE Program
The DECIPHER.EXE Program
The Challenge
Chapter 8—Public Key Encryption
General Operation
Authentication Issues
Public vs. Private Key Encryption
Advantages and Disadvantages
Understanding the Mathematics
Modular Arithmetic
Modular Addition
Modular Subtraction
Modular Multiplication
Finding the Multiplicative Inverse
Prime Numbers
The Euclidean Algorithm
The Totient Function
RSA
Public Key Generation
Private Key Generation
Message Encipherment
Exponentiation Operations
Facilitating the Key Generation Process
Locating Large Primes
Summary
Appendix A
Index


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held
Table of Contents

Preface

"Bid $3.62 million on the 46th Street project," "potential corporate acquisition requires suspension of training activities through the end of month," and "meet me at 7 P.M. in the Radisson lobby to discuss…" are but a few examples of personal communications that for one reason or another should be confidential.

Today, we live in a world whose "communications dimensions" have rapidly shrunk due to the explosive growth in the use of computer-based networks. Messages that took hours or days to reach a recipient a decade ago are now delivered near instantaneously to most areas on the globe. Although advances in communications technology have considerably enhanced our quality of life, it has also provided the con artist, computer hacker, and persons many would classify as being in the lower elements of society with the ability to electronically take advantage of others. An unscrupulous person may commit electronic fraud by ordering items and billing the cost to others, or read personal messages and take actions that are harmful to other persons. In other situations, the inadvertent viewing of a personal message concerning a potential corporate reorganization, pending personnel action, or similar activity can result in an exponential increase in the rumor mill, while employee productivity decreases in an inverse proportion to these rumors.

While it is doubtful if fraud or rumors will ever cease as long as humans populate the earth, there are steps you can consider to keep personal communications personal. Some steps, such as routing messages correctly, avoid the embarrassment of an unintentional recipient. Other steps, such as converting your message into a sequence of characters that is not meaningful to an unintended recipient or casual observer, provide a mechanism to protect the content of your message. It is this area that is the focus of this book, providing practical methods readers can use to obtain a degree of privacy and security for messages that are transmitted over internal corporate communications networks or via public electronic messaging systems.

The focus of this book is upon the use of cryptology to encipher and decipher messages. As we examine different encipherment methods, we will also focus our attention upon the development of programs written in the C++ and BASIC programming languages that can be used to automate the encipherment and decipherment process.

Many readers by now will realize that the techniques covered in this book are available to any person who obtains a copy of this book. While this fact means the techniques to obtain message privacy are in the public domain, it does not mean that the use of those techniques results in public messages.

Most techniques covered in this book are based upon the use of keywords, phrases, or the selection of a random sequence of characters. Although almost any enciphering technique can be broken given a long enough message and time, the objective of this book is to provide readers with a mechanism to prevent the inadvertent reading of messages as well as delay the translation of messages intercepted by hackers, rascals, and scoundrels. Thus, a message concerning a bid due on Tuesday may be worthless to a person who reads the contents of your message on Thursday. By carefully considering the use of different enciphering techniques and keywords, phrases, or random character sequences to govern the encipherment process, you can keep personal communications personal as well as obtain a degree of message security that will enable you to conduct your business in a more efficient and effective manner.

As a professional author, I consider reader feedback extremely important and encourage you to send me your comments. Let me know if the information and techniques presented in this book satisfied your requirements or if there are other areas you would like me to tackle in a future edition. You can write to me via e-mail at 235-8068@mcimail.com or through my publisher whose address is on this book.

Gilbert Held
Macon, GA

Table of Contents


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 1
Technology and Terminology

The primary purpose of this chapter is to acquaint you with a core set of terms associated with the field of cryptology. You should note that your primary objective is to obtain a level of knowledge which provides the ability to develop and use practical encipherment techniques to hide the meaning of messages transmitted over different types of electronic mail systems. Many of the techniques we will cover are not unbreakable to a trained analyst; however, they will provide varying levels of message protection that can make it difficult and, in many cases, very time consuming to understand the meaning of a message. In fact, some practical techniques we will cover in this book may require more than 60 billion trials to correctly understand the meaning of a message. Because each trial could require the printing of an intercepted message, a 10-line message could conceivably require a person to scan 600 billion printed lines!

Although the United States National Security Agency (NSA), the United Kingdom's MI5, and Russia's modern version of the KGB have probably programmed supercomputers to use artificial intelligence in an attempt to understand the meaning of intercepted messages, few, if any, commercial organizations have the financial resources to obtain the required hardware to develop programs to perform similar operations. Thus, the more sophisticated techniques presented in this book should provide you with a mechanism to protect the meaning of your communications from the casual observer, the inadvertent observer, and most, if not all, intentional illicit intercepters.

To acquaint readers with the terminology associated with the field of cryptology, we will both discuss different terms as well as their meaning through the use of several examples. In doing so we will review the main types of ciphers that are covered in detail in succeeding chapters. When appropriate, we will discuss the use of different ciphers from a historical perspective; however, our intention is not to provide readers with information on the historical evolution of ciphers but to show some of their practical uses from a historical basis.

A secondary objective of this chapter is to discuss the subroutines and programs we will develop and include via program listings throughout this book. This discussion will acquaint readers with the rationale for selecting the C++ and BASIC programming languages that are used to develop subroutines and programs and file naming conventions which can facilitate the use of the subroutines and programs included in this book.

Ciphers Versus Codes

One of the major areas of confusion for many readers is the difference between a cipher system and a code system. Although both systems are designed to conceal information while it is transmitted, they do so using different substitution techniques.

If you're a video buff and remember such war films as The Longest Day, Tora, Tora, Tora, and similar films, you noted the transmission of messages whose contents appear to be meaningless. Such messages as "Paul has a long arm," "The soup is in the kitchen," and "Ralph needs shoes," for example, represent coded messages in which words and phrases are substituted for other words or phrases to disguise the meaning of the message. In other films, you may remember a soldier or sailor using equipment to intercept a message and preparing a punched paper tape containing the contents of the message. The paper tape was then placed into a special tape reader that read the tape and produced a new tape. The soldier or sailor would then read the message on the new tape and yell across the room, "Let's get this message to the Captain" or a similar line.

Those movies illustrated the use of specially constructed decipherment machines which were designed to read enciphered messages and reproduce the original text of the message known as plaintext or cleartext. The intercepted message punched on paper tape consisted of a series of what appeared to be randomly picked characters, such as the sequence "QAFRT…" That character sequence represented an enciphered message in which each plaintext character in the message was replaced by a ciphertext character according to a predefined algorithm.

In a cipher system, the plaintext, which represents a set of data prior to encipherment, is operated upon without regard to their meaning. In a code system, words or phrases are replaced by other words or phrases to hide the meaning of a message. Thus, the message FIRE THE CORPORATE LAWYER might be transmitted as GJSF UIF OPSQPSBUF MBXZFS using a simple cipher discussed later in this chapter. When transmitted in a code, the message could become SELL THE CORPORATE DONKEY, assuming SELL was the code for FIRE and DONKEY was the code for LAWYER.

Because many code systems normally do not change the meaning of more than a few terms in a message, such systems immediately provide a hint as to the contents of a message. In addition, the observation of a series of messages can provide a reasonable set of clues that will enable someone to determine the meaning of coded messages. Another obstacle to the use of codes, especially when they are only used infrequently, is the requirement to have prior knowledge of all of the substitutions. Some code systems used by military and diplomatic personnel are based upon the creation of code books that may have tens of thousands of terms and their equivalent codes. Not only are such code books difficult and time consuming to prepare but, in addition, their distribution and replacement by a new code book if an existing book should become lost or compromised can create a logistical nightmare. Although still popular for military and diplomatic use in certain situations, the efficient use of a code system requires the development of the previously mentioned code book which provides the basis for encoding and decoding messages. In comparison, many cipher systems only require knowledge of one or a few items of information known as keys to encipher and decipher a message. Thus, from a practical point of view, encipherment and decipherment operations can be easier to perform than encoding and decoding operations associated with conventional code book-based systems, especially as the length of a message increases. In addition, many cipher systems can be expected to provide a higher level of message protection than obtained from the use of a code system. Due to the preceding, the focus of this book is upon different encipherment techniques.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Cipher Terminology

As previously described, a cipher system is a system in which a substitution process occurs on individual characters or groups of characters without regard to their meaning. The actual enciphering process can be performed by hardware, software, or manually by the brainpower of one or more persons.

Plaintext and Enciphered Text

The use of a cipher system requires a message or text to operate upon. The original, unaltered contents of the message or text is known as plaintext or cleartext. Through the use of a cipher system, the meaning of the plaintext message is hidden. The process of hiding the meaning of the plaintext is known as enciphering, while the resultant text is referred to as enciphered text.

Encipherment

Figure 1.1 illustrates the encipherment process in a block diagram format. In examining Figure 1.1, note that plaintext (x) is converted to ciphertext (y) by an enciphering process (E), such that y=Ek(x), where k is a key. The enciphering process can be considered as an algorithm which operates upon the plaintext based upon the value of a key (k). Thus, the key defines the operation of the encipherment algorithm and different keys result in different ciphertext being created from a fixed plaintext.

Figure 1.1 The encipherment process results in the creation of ciphertext (y) by applying an encipherment algorithm (E) controlled by a key (k) against plaintext (x) so that y=Ek(x)

An example of the use of a modern key can be obtained by constructing a modulo 2 encipherment process. As a bit of review for readers who may not remember modulo arithmetic, when adding numbers using that arithmetic for base n you divide n into the added numbers and retain the remainder as the result of the modulo arithmetic process. That is, for our familiar base 10 the addition of 6 and 8 under modulo 10 results in a value of 4. Table 1.1 illustrates the results of modulo 2 for the four possibilities that can occur per bit position.

Table 1.1 Modulo 2 Addition
0 | 0 | 1 | 1
0 | 1 | 0 | 1
0 | 1 | 1 | 0

To illustrate the encipherment process using modulo 2 addition, let's encipher the word HELP, assuming it was first stored using the 7-bit American Standard Code for Information Interchange. Let's then assume the key was the word BURP. Table 1.2 illustrates the encipherment process using modulo 2 addition.

Table 1.2 Enciphering the word HELP with the key BURP using modulo 2 addition.
Plaintext | H | E | L | P
 | 1001000 | 1000101 | 1001100 | 1010000
⊕Key | B | U | R | P
 | 1000010 | 1010101 | 1010010 | 1010000
Enciphered text | 0001010 | 0010000 | 0011110 | 0000000
 | LF | DLE | RS | NUL

In examining the entries in Table 1.2, note that as a result of the modulo 2 encipherment process the plaintext HELP is converted into a sequence of 28 bits which represent four control characters—Line Feed (LF), Data Link Escape (DLE), Record Separator (RS), and Null (NUL). Thus, if a person intercepted the enciphered text they would observe the sequence LF, DLE, RS, NUL instead of the word HELP.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Decipherment

The process of converting or restoring ciphertext back into its original plaintext contents is known as decipherment. Figure 1.2 illustrates the decipherment process in a block diagram format. Decipherment (D) can be considered the inverse of the encipherment process, requiring the use of an algorithm which converts ciphertext back into its original plaintext. Decipherment is also controlled by a key (k), which is applied to ciphertext (y) to produce plaintext (x), such that x=Dk(y).

Figure 1.2 The decipherment process results in the creation of plaintext (x) by applying a decipherment algorithm (D) controlled by a key (k) against ciphertext (y) so that x=Dk(y).

You can obtain an appreciation for the decipherment process by deciphering the previously enciphered message. Assuming you have the key BURP and received the message LF DLE RS NUL, you would use modulo 2 subtraction to reconstruct the original plaintext message. Thus, an understanding of the decipherment process requires a brief overview of modulo subtraction.

Under modulo subtraction, if the minuend is greater than the subtrahend the result is obtained by subtracting the subtrahend from the minuend. If the minuend is less than the subtrahend, you would first add the value of the next base position to the minuend prior to subtracting the subtrahend. For example, under base 10 9-5 is 4, while 5-9 is 6 since a value of 10 is added to the minuend (5) prior to subtracting the subtrahend (9).

Since our previous encipherment was performed using modulo 2 addition, we will use modulo 2 subtraction to perform a decipherment operation. Table 1.3 illustrates the modulo 2 subtraction process for all four possible combinations per bit position.

Table 1.3 Modulo 2 Subtraction
0 | 0 | 1 | 1
0 | 1 | 0 | 1
0 | 1 | 1 | 0

Now that you have an appreciation for modulo 2 subtraction, let's turn our attention to deciphering the previously enciphered message. If you are the recipient of the message, you would receive the enciphered text LF DLE RS NUL. Assuming you have the key BURP, you would then perform a modulo 2 subtract process to decipher the enciphered text into its original plaintext. This decipherment process is illustrated in Table 1.4.

Table 1.4 Deciphering the message LF DLE RS NUL with the key BURP using modulo 2 subtraction.
Enciphered text | LF | DLE | RS | NUL
 | 0001010 | 0010000 | 0011110 | 0000000
8 Key | B | U | R | P
 | 1000010 | 1010101 | 1010010 | 1010000
Deciphered text | 1001000 | 1000101 | 1001100 | 1010000

Keys and Key Space

In examining Figure 1.1 and 1.2, the key—no pun intended—to enciphering and deciphering is the key that controls the operation of the enciphering and deciphering algorithms. Although a person may obtain a copy of an enciphered message and have knowledge of the algorithm used to encipher the message, you must use the correct key to successfully decipher the message. Thus, the number of possible values of a key, referred to as an algorithm's key space, is an important consideration in selecting an enciphering algorithm. This is because an algorithm with a limited key space is very susceptible to a trial-and-error process in which different keys are applied in an attempt to decipher an intercepted message. For example, if a sequence of nine decimal digits is used to form a key, its key space is 1010, or 10 billion, key values.

Although it is relatively easy to program a computer to generate a series of 10-digit sequences to use at different times, it is probably not realistic to expect a mortal to remember those sequences. Thus, a tradeoff will occur between key space and practicality if we wish to develop ciphers that can be easily used in a manual process. In addition, from a practical point of view, it is many times easier to remember words or phrases instead of a sequence of digits. Thus, many of the cipher systems discussed in this book are based upon the use of an alphabetic key in the form of a word or phrase. This usually provides an increased retention capability over the use of a sequence of digits. Even if you automate the enciphering process through the use of a program executed on a personal computer, under certain situations you may prefer to use a word or phrase in place of a digit sequence to facilitate the use of the program.

Another reason for using alphabetic or alphanumeric keys instead of a key restricted to digits is the significant increase in the key space afforded by the use of alphabetic or alphanumeric keys. For example, a two-digit key provides 10*10, or 100, unique keys. In comparison, the use of two uppercase or two lowercase alphabetic characters in a key provides 26*26, or 676, unique keys, while the use of two alphanumeric characters in which alphabetic characters are limited to a single case results in 36*36, or 1,296, unique keys.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Table 1.5 contains the key space obtained by the use of digits, letters, and alphanumeric characters for keys varying in length from 1 to 20 positions. The field width column indicates the number of characters in a key. You should note that the discussion of key space will be limited to the use of all uppercase or all lowercase letters when alphabetic or alphanumeric keys are used unless otherwise noted. Although the use of both uppercase and lowercase letters significantly increases the key space, it also increases the potential for confusion. That is, it becomes quite easy to remember a word or phrase but to forget which letters are uppercase and which letters are lowercase. For that reason we will normally restrict the use of keys in this book to letters of a single case. That is, if our key is limited to eight positions and we wish to use the word "computer," we would enter the key as "COMPUTER" or "computer," but not as "Computer," unless we develop a program which is case insensitive and converts all characters to one case.

Table 1.5 Key space based on characters used and field width.
Field Width | Digit Combinations | Letter Combinations | Alphanumeric Combinations
1 | 1.00000E+01 | 2.60000E+01 | 3.60000E+01
2 | 1.00000E+02 | 6.76000E+02 | 1.29600E+03
3 | 1.00000E+03 | 1.75760E+04 | 4.66560E+04
4 | 1.00000E+04 | 4.56976E+05 | 1.67962E+06
5 | 1.00000E+05 | 1.18814E+07 | 6.04662E+02
6 | 1.00000E+06 | 3.08916E+08 | 2.17678E+09
7 | 1.00000E+07 | 8.03181E+09 | 7.83642E+10
8 | 1.00000E+08 | 2.08827E+11 | 2.82111E+12
9 | 1.00000E+09 | 5.42950E+12 | 1.01560E+14
10 | 1.00000E+10 | 1.41167E+14 | 3.65616E+15
11 | 1.00000E+11 | 3.67034E+15 | 1.31622E+17
12 | 1.00000E+12 | 6.54290E+16 | 4.73838E+18
13 | 1.00000E+13 | 2.48115E+18 | 1.70582E+20
14 | 1.00000E+14 | 6.45100E+19 | 6.14094E+21
15 | 1.00000E+15 | 1.67726E+21 | 2.21074E+23
16 | 1.00000E+16 | 4.36087E+22 | 7.95866E+24
17 | 1.00000E+17 | 1.13383E+24 | 2.86512E+26
18 | 1.00000E+18 | 2.94795E+25 | 1.03144E+28
19 | 1.00000E+19 | 7.66467E+26 | 3.71319E+29
20 | 1.00000E+20 | 1.99281E+28 | 1.33675E+31

In examining the entries in Table 1.5, you should note that E+6 represents one million, while E+9 represents one billion. Thus, a six-character alphanumeric field produces 2.17 billion combinations even when we restrict our alphabet to all uppercase or all lowercase letters! Readers should also note that a nine-character field would be required to obtain just one billion key combinations if the key was restricted to digits. Thus, we can maintain a large key space with a lesser number of key characters or increase our key space by using alphanumeric characters instead of restricting our key to digits or letters.

Types of Encryption Systems

There are two types of encryption systems—private and public key based. The previously described method of encipherment and decipherment based upon the use of the same key for encryption and decryption is referred to as a private key based encryption system. That is, the key must remain private and cannot be disclosed to the person or persons who are intended recipients of an encrypted message. Otherwise the key is said to be compromised, meaning that any message encrypted using the key is potentially compromised. Another term commonly used to reference a private key based encryption system is a "symmetric" key system. Here, the term symmetric is used to reference the fact that the same key is used at both ends of the process.

One of the major problems associated with the use of a private key based encryption system is key generation and key distribution. For example, if you only need to send an encrypted message between two persons using a private key system, a single key used by each person becomes sufficient. If three people need to send encrypted messages to each other you could use one common key; however, if that key is compromised, then any messages transmitted between any of the three persons could become compromised. Thus, many organizations using a public key based encryption system commonly elect to use different keys for encryption between different persons. This means that the use of a public key based encryption system linking three persons or corporate offices would require three keys, while a system linking four persons or offices would require four keys.

This also means that for n users or organizational locations the number of unique keys required becomes 2n-2. While this may not be a problem for a small number of users or organizational locations, as those numbers increase they can reach a point where the administration process to include key generation and distribution becomes unmanageable. This is particularly true when you consider the use of modern electronic mail systems where employees may have to communicate with hundreds or thousands of persons. In this type of situation a private key based encryption system obviously becomes unsuitable. Similarly, it would be virtually impossible to use a private key based encryption system to secure electronic commerce from browsers to World Wide Web servers. Fortunately, the efforts of mathematicians resulted in the development of another type of encryption which significantly alleviates the previously described key generation and distribution problems associated with private key based encryption systems. This type of encryption is referred to as a public key based encryption system.

Public key based encryption dates to the late 1970s when mathematicians discovered certain mathematical relationships that resulted in a key used to encrypt a message being unable to perform decryption while another key in a matched pair of keys could be used for decryption. The mathematical relationship requires the use of two keys, one which remains private while the second can be provided to anybody, and can even be posted on a person's home page on the Web. That key is referred to as a public key.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Under public key encryption, a person wishing to send an encrypted message to another person or organization acquires the recipient's public key and uses that key to encrypt the message. The recipient uses their private key to decrypt the message. Since each user or organization in a public key system only has to maintain two keys, one public and one private, regardless of the number of persons or locations in the system the total number of keys required for n users becomes 2*n. Thus, for a large n the number of keys required is significantly reduced from the 2n-2 keys required for a private key based encryption system.

The use of a public key based encryption system results in keys providing one-way encryption and decryption based upon the type of key used, public for encryption, private for decryption. Thus, another term commonly used to reference a public key encryption system is an asymmetric encryption system.

From a mathematical perspective the ciphertext, Ek(x), is computed using a public key (kp) which has a mathematical relationship to the private key (kr). Unlike a private key system where Dk(Ek(x)) results in the reconstruction of the original plaintext, Dk(x) cannot be easily computed from kp without knowledge of kr. Instead, Dk(x) is easily computed when the private key (kr) is known. The key to the use of a public key system is the ability for individuals to generate their own public and private keys so that they can freely distribute their public key to persons they may wish to communicate with in a secure manner. Through the use of exponentiation and large prime numbers it becomes possible to generate relatively long public and private keys. While it is possible for other persons to use similar algorithms to generate public and private keys in an attempt to decrypt data encrypted by another person, the time required to factor relatively long primes would require the use of supercomputers for decades. Thus, you can be reasonably assured that properly developed public and private key pairs provide a high degree of protection for messages well into the future.

Key Generation, Management, and Distribution

An enciphering system that produces billions of possible key combinations may be extremely vulnerable if keys are selected in a manner in which their possible composition can be easily guessed. For example, a birth date or home address would not be advisable for use as a key as they represent one of the first trials in a trial-and-error process an unauthorized person will more than likely attempt to use as a key. Ideally, keys should be generated based upon some random process. However, if you're traveling and wish to use your laptop or notebook computer to transmit or receive private messages enciphered according to a predefined key, you would probably prefer to remember PCATV instead of Q4R51. Because PCATV is obviously an easier guess than Q4R51, you can consider using a random process to generate keys and store those keys for use over a predefined period of time on a computer file enciphered using a different key. Then, employees would only be tasked with remembering one key for a period of time to access a list of keys and the period of time each key is valid for use. This technique is one of many methods associated with key management.

Key management can be considered the process of generating, maintaining, and distributing keys. In addition, this activity is responsible for defining what action or actions to take in the event a cipher system or key to a cipher system is lost or compromised.

The key management process, as well as functions associated with this process, will vary from organization to organization based upon a variety of factors. Those factors may include the number of employees that require knowledge of an enciphering key, their geographical distribution, and the integrity of the firm's internal mail service. Additionally, the type of key system, public or private, will significantly affect the key distribution method. Some organizations post public keys on their Web servers or deliver those keys via internal interoffice mail in a manner similar to which computer passwords are distributed. Other organizations distribute keys via registered mail, through the use of the corporate electronic mail system enciphered using the current key, or in plaintext. Obviously, the repeated distribution of new private keys in plaintext or enciphered using the currently-in-use key opens the organization to a potentially compromising situation. However, you should remember that many organizations simply want a mechanism to keep personal communications personal and are not looking for nor do they seek an elaborate method to guarantee that their communications will never be broken. Although some sophisticated methods to distribute keys are discussed in succeeding chapters, our primary focus of attention will be placed on practical and efficient methods to distribute keys.

Regardless of the manner in which new keys are distributed, they should be distributed because employees will leave the organization, lose their keys, misplace a message containing a listing of keys, and do other things which in hindsight can compromise a system.

Perhaps one of the more interesting public compromises of security occurred when a president of the United States was photographed with a document folded in his suit pocket. The document was folded in such a manner that a keyword was prominently displayed in the photograph that appeared in newspapers. Fortunately for national security, the keyword denoted a classification associated with a special type of communications and did not reflect the key used to encipher communications. Unfortunately for many persons involved in national security, the display of the keyword resulted in a large amount of unpaid overtime required to rapidly change the keyword used to classify the type of communications exposed to the public.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

At some federal agencies, private keys are changed on a quarterly basis, while other agencies change keys on a monthly, weekly, and even a daily basis. The driving force behind the frequency of key changes should be the worth of the data whose contents are being hidden by encipherment and the potential threat to that data. The latter may be extremely hard to judge, since most persons will not know their electronic conversations have been targeted for interception until it is too late. In addition to the potential threat, you must balance key changes against the effort required to develop and distribute new keys on a periodic basis. If the system you devise is difficult to implement or requires too much effort to effectively use, many potential users may bypass the system you develop. Thus, the old military adage—Keep it simple—is worth remembering when you consider various methods for the generation and distribution of keys.

Types of Ciphers

A visit to a large university or public library will reward you with the ability to locate a large number of books which discuss the use of cipher systems. Some books, such as The Codebreakers (Macmillian, 1969) by David Kahn and Cryptanalysis (Dover Publications, 1956) by Helen F. Gaines, provide an excellent overview of a variety of cipher systems, primarily focusing upon the historical use of such systems. More recently published works, such as Security for Computer Networks by D.W. Davies and W.L. Price, can bring you up to date on recent developments in the construction of cipher systems from a mathematical basis. Reading these and other books will acquaint you with hundreds of types of cipher systems, ranging in scope from an elementary process that can be performed mentally to very complex systems that require the use of a computer. What each of those cipher systems has in common is the ability to be classified into one of two general categories—substitution and transposition.

The remainder of this section focuses on an initial overview of each cipher category and discusses several of the variants used to develop specific types of cipher systems. This is followed by a discussion of the constraints that must be employed in considering the algorithms and key space used to convert plaintext to ciphertext which will enable ciphertext to be carried by many electronic mail systems. Our examination of substitution and transposition cipher systems will be based upon the use of private keys. Although it is quite possible to develop public key based substitution and transposition cipher systems since the former dates from the 1970s while the latter can be traced to the time of Caesar, from a historical perspective the cipher systems we will examine were all developed based upon the use of private keys. Thus, we will respect the historical evolution of encryption and primarily examine the characteristics of substitution and transposition cipher systems via the use of private keys in this chapter.

Substitution Cipher Systems

In a substitution cipher system, each plaintext character is replaced by a ciphertext character using an algorithm which operates upon the plaintext in the sequence in which the plaintext characters appear. In a simple substitution cipher system, the replacement process is unaltered. That is, each plaintext character is replaced by the same ciphertext character. For example, the well-known Caesar cipher, in which the ciphertext represents a fixed displacement in the alphabet, would encipher the message KILL ALL THE LAWYERS as LJMM BMM UIF MBXZFST using a displacement of one character position as the encipherment algorithm.

You can denote the relationship between a plaintext alphabet and a ciphertext alphabet by listing one alphabet above the other. For example, restrict your plaintext and ciphertext alphabets to the 26 uppercase letters. Then, for the simple one-character Caesar cipher displacement, the relationship between the plaintext alphabet (P) and the ciphertext alphabet (C) is as appears below. In this example, the letters in the ciphertext alphabet are displaced by one character position from the letters in the plaintext alphabet. We can denote this relationship between the mapping of plaintext and ciphertext characters as PA=CB, which indicates that the letter A in the plaintext alphabet is aligned with the letter B in the ciphertext alphabet.

Plaintext alphabet (P): | ABCDEFGHIJKLMNOPQRSTUVWXYZ
Ciphertext alphabet (C): | BCDEFGHIJKLMNOPQRSTUVWXYZA


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

In a varying substitution cipher, the replacement of plaintext by ciphertext results in a very high probability that repeating plaintext characters are replaced by different ciphertext characters. To accomplish this requires the use of more than one "mapping" alphabet to convert plaintext to ciphertext, a term referred to as polyalphabetic substitution. As an alternative, a random or pseudo-random sequence can be used to generate a variable ciphertext replacement. Varying substitutions through the use of a pseudo-random sequence generated by a key is the basis for many commercially developed enciphering systems as well as the well-known data encryption standard (DES). In separate chapters in this book we will examine the construction of monoalphabetic, polyalphabetic, and pseudo-random number based cipher systems.

Prior to the start of World War II, the U.S. Army Signal Corps successfully constructed a cipher machine called PURPLE that duplicated the cryptographic operations of a Japanese machine. Intercepted Japanese transmissions were fed into PURPLE, producing plaintext information that was then translated into English and distributed under the code name MAGIC to President Roosevelt and top-level civilian and military officials within his administration. Although a series of urgent messages from Tokyo to the Japanese ambassador in Washington was intercepted and deciphered, this then-secret effort did not prevent the Pearl Harbor attack simply because Japan never transmitted a message stating they would attack Pearl Harbor! Whether intentional or not, the lack of an explicit attack message resulted in a degree of complacency within the U.S. government.

The ability to read enciphered messages, while important, does not necessarily mean you will always be able to understand the intentions of others. While effective in many situations, the ability to read enciphered messages is no substitute for the analysis of the contents of those messages. For example, in June of 1942, U.S. naval intelligence decoding of intercepted Japanese transmissions noted that one Japanese unit gave Midway as its post office address. By concentrating U.S. naval forces to meet the Japanese threat, the battle of Midway represented the turning point of the war in the Pacific.

Transposition Cipher Systems

As its name implies, a transposition cipher system rearranges the characters in a message. An elementary transposition system could simply swap plaintext character positions so that position n is mapped to n+1 and position n+1 is mapped to position n. Then, the message KILL ALL THE LAWYERS would become IKLL LAT LEH ALYWRES. In the preceding example, note that a visual observation immediately provides a good clue as to the meaning of the plaintext.

As an alternative to the transposition of plaintext characters, some cipher systems are based upon the use of an algorithm which transposes the characters of a mapping alphabet or series of mapping alphabets. A variety of transposition systems are covered in a later chapter of this book.

Electronic Mail Transmission Considerations

Until now, I have simply glossed over the contents of the plaintext alphabet and the resulting ciphertext alphabet. The plaintext alphabet represents all possible characters in the character set used to develop a plaintext message. The ciphertext alphabet represents all possible enciphered characters resulting from the enciphering process. It is the plaintext character set and the enciphering process that generates the ciphertext character set you must consider to successfully transmit an enciphered message through many electronic mail systems. The reason for this is the fact that most computer alphabets use eight bits to represent a character. This results in 28, or 256, unique characters that can be used to represent both plaintext and ciphertext alphabet sets. Unfortunately, several electronic mail systems are restricted to transmitting seven-bit characters, with the eighth bit in a transmitted byte used for parity. Such electronic mail systems are restricted to transmitting 27, or 128, characters. This means that many cipher systems, such as the DES algorithm, that are not restricted with respect to the ciphertext alphabet set it generates cannot be used to directly transfer data across certain electronic mail systems.

Transmitting randomly generated ciphertext that can represent any of the 256 unique characters in an 8-bit character set over a 7-bit electronic mail system requires a conversion of each 8-bit character. One of the earliest programs developed to perform this conversion is UUENCODE, which converts three 8-bit characters into four 7-bit characters. Another program known as UUDECODE is used to reconvert each group of four 7-bit characters transmitted via a 7-bit electronic mail system back into their original three 8-bit characters. Although you can use the UUENCODE/UUDECODE program pair or similar software products to transmit a message created using an 8-bit character set via a 7-bit electronic mail system, all recipients must have the decoding program associated with the encoding program.

To obtain the capability to transmit ciphertext via certain electronic mail systems without requiring recipients to have a 7-bit to 8-bit conversion program, you need to place limits on both the plaintext alphabet set as well as the algorithm used to create ciphertext. Several methods you can use to limit the plaintext alphabet set and the algorithm used to generate a ciphertext alphabet are discussed at appropriate points in succeeding chapters.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Subroutines and Programs

To extend the practical use of encipherment, numerous routines will be developed in this book to perform different enciphering and deciphering related operations. To provide code that can be utilized by a maximum number of readers, program modules are developed using both C++ and QuickBASIC. Concerning the development of C++ modules, this author and his son used the Microsoft Visual C++ compiler to create and execute C++ routines used in this book. Microsoft Corporation's QuickBASIC compiler was used to develop the QuickBASIC subroutines and programs contained in this book.

The selection of C++ as one of two programming languages to use in the development of programming modules was based upon several factors. First, C++ is a very powerful programming language which provides programmers with the ability to perform data manipulations frequently required when using different encryption algorithms. Secondly, C++ is standardized and compilers are readily available, which means that the modules created by this author using Microsoft's Visual C++ can be used by persons with C++ compilers developed by other software vendors. However, since it is possible that some persons may not have a C++ compiler, each program module was also coded using QuickBASIC. In addition, program modules created in both C++ and QuickBASIC are stored on the CD-ROM accompanying this book in both source and binary executable form. This will provide you with a significant degree of flexibility as you can elect to incorporate source modules into Visual C++ or Visual Basic to create Windows-compliant programs, execute the binary modules under DOS to experiment with the creation of ciphertext through the use of different encryption techniques, or modify source code to tailor one or more modules to your specific requirements.

QuickBASIC was also selected for several reasons. First and foremost, every version of PC-DOS and MS-DOS contains a BASIC language interpreter on the DOS distribution diskette. This means that most readers can easily adapt the subroutines and programs presented in this book to execute through the use of a BASIC language interpreter at no additional cost. Readers can use the programs as is, modify one or more programs, or select a number of subroutines that can be viewed similar to a construction set for the design of a program tailored to a specific reader requirement.

A second reason for the selection of QuickBASIC is the availability of QBASIC on all DOS distribution diskettes commencing with release 5.0 of that operating system. QBASIC can execute all subroutines and programs presented in this book without modification, facilitating the use of those subroutines and programs.

The large number of persons with BASIC programming knowledge enables more persons to be able to put theory to practice. The capability of the QBASIC compiler to produce object code permits the inclusion of ready-to-run, executable program files on the CD-ROM. This allows you to directly execute a program without having to modify the source code, as would be required if a BASIC interpreter is used. Of course, you cannot modify the executable program, which may take some of the fun out of the use of the program if you would like to attempt to modify one or more functions.

File Naming Conventions

To facilitate the use of the CD-ROM, a set of file naming conventions was developed and used to reference program listings in this book as well as files on the CD-ROM. Files with the extension .CPP represent C++ language source code files. In this book, those files appear as subroutine and program listings. Files with the extension .BAS are BASIC language source code files. In this book, those files also appear as subroutine and program listings. Files with the extension .DAT reference data files. These files contain plaintext and ciphertext messages which facilitate demonstrating the operation and utilization of the programs developed in this book and are also contained on the CD-ROM. The fourth type of file has the extension .EXE. Files with that extension represent executable programs that readers can directly run on any Windows and DOS-based personal computer.

To enable the largest number of potential readers to use this book, files are stored under two directories on the CD-ROM. The C directory contains C++ language source and executable language programs, while the BASIC directory contains BASIC language source and executable programs. Although it is easy to distinguish BASIC and C++ source programs from one another by their extensions, both programs use the extension .EXE for executable programs. Thus, it is important to remember to note the directory under which an executable program is stored as it serves as a mechanism to distinguish BASIC from C++ executable programs. Refer to Appendix A for a list of files contained on the CD, including their names and description. Table 1.6 summarizes the file naming conventions used in this book for the mythical file XXXX.

Table 1.6 File naming conventions
File Name | Description
XXXX.BAS | The BASIC source code for the program XXXX.
XXXX.CPP | The C++ source code for the program XXXX.
XXXX.EXE | The BASIC version is in the BASIC directory, and the C++ version is in the C directory.
XXXX.DAT | A sample data file.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 2
Monoalphabetic Substitution Concepts

At first glance, the title of this chapter may evoke a vision of coverage of a medical disease. Thus, let's clarify the scope and content of this chapter by discussing what we intend to cover.

The focus of this chapter is on the use of single letters or characters contained in one alphabet to replace plaintext letters and characters in a message. Because replacement letters and characters are restricted to those included in one alphabet, we use the term "monoalphabet." In recognition of the fact that characters in the alphabet are substituted for plaintext characters, we refer to the encipherment process as a monoalphabetic substitution process.

In some books you may note the term "uniliteral substitution" or "monoalphabetic uniliteral substitution." Here, the word "uniliteral" denotes the fact that the replacement process occurs on a character-by-character basis with one character from the replacement alphabet substituted for each character in the plaintext message.

Evolution

Monoalphabetic uniliteral substitution represents one of the earliest methods of cryptology, with the technique employed by Julius Caesar approximately 20 centuries ago to hide the contents of messages he sent by courier. Although Caesar's method of encipherment was relatively elementary, his name has been associated with a class of displacement enciphering techniques and the term "Caesar cipher" is used to denote encipherment by displacement.

According to historical records, Julius Caesar wrote to Cicero and other friends using a cipher system in which the plaintext letters in his message were replaced by letters three positions further down the alphabet. Thus, in the English language, INVADE would be enciphered as LQYDGH and ENGLAND would be enciphered as HQJODQG.

Augustus Caesar, the nephew of Julius and first emperor of Rome, modified his uncle's technique. Augustus substituted for each plaintext character, reducing the displacement from three positions to one. Whether Augustus had difficulty in counting, arthritis which precluded counting by the use of his fingers, or just wanted to make deciphering easier is unknown. What is known is that any enciphering system in which plaintext characters are replaced by characters in an alphabet displaced from the plaintext alphabet is referred to as a Caesar cipher, while the displaced alphabet is called a Caesar alphabet.

Although a Caesar cipher is a rather elementary enciphering technique, it provided a foundation for the development of more advanced monoalphabetic substitution techniques. Thus, an understanding of that class of ciphers provides you with the ability to recognize the advantages and disadvantages of other techniques whose foundation can be traced to the Caesar cipher.

Prior to examining the use of uniliteral substitution systems, let us define a few terms that will be used throughout this chapter. In doing so, let us restrict our alphabet to the uppercase letters A through Z and ignore lowercase letters, punctuation characters, and numerics for simplicity of illustration (numerics and punctuation characters can be spelled out—ZERO, COMMA, and so on). In fact, in many military systems the English alphabet is restricted to the 26 uppercase characters, so we can be considered as "borrowing" military techniques by initially restricting the alphabet we will use to uppercase letters. However, you can easily expand the examples presented in this chapter to incorporate the use of larger character sets if you so desire.

Alphabets

There are two basic types of alphabets you must consider in developing a monoalphabetic uniliteral substitution system—a plaintext alphabet and a ciphertext alphabet. The plaintext alphabet represents the alphabet from which the characters of a message are constructed, while the ciphertext alphabet represents the sequence of characters used to replace the plaintext characters during the uniliteral substitution process. Because the characters in the ciphertext alphabet are used to replace the characters in the plaintext alphabet, the ciphertext alphabet is also commonly known as a substitution alphabet.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Alphabetic Relationships

The top portion of Figure 2.1 illustrates the elementary relationship between a direct-sequence plaintext alphabet and a direct-sequence ciphertext alphabet. In this example, the ciphertext alphabet was shifted or wrapped five character positions to the right with respect to the plaintext alphabet if the wraparound of each character is considered to represent one positional shift. As you will shortly note, the relationship between the numeric positions representing a ciphertext character (C) and a plaintext character (P) can be expressed as C=(P+5) modulo 26.

Figure 2.1 Plaintext-cipertext relationship.

To illustrate the relationship between ciphertext and plaintext characters, assume each of the letters of the plaintext alphabet is sequentially assigned the numbers 0 through 25. Then, the process of encipherment illustrated in the top portion of Figure 2.1 can be expressed as C=(P+5) modulo 26, where C represents the ciphertext character's numeric value, P represents the plaintext character's numeric value, and the modulo 26 process requires 26 to be subtracted when the result of the addition exceeds 25.

For example, 25 mod 26 has a value of 25, while 26 mod 26 and 27 mod 26 have values of 0 and 1, respectively. To verify this relationship, the lower portion of Figure 2.1 indicates the numerical relationships between the characters in the plaintext and ciphertext alphabets. The numeric relationship is based upon the assignment of the values 0 through 25 to the letters in the plaintext alphabet and the shifting of the ciphertext alphabet by five positions with respect to the plaintext alphabet.

From an examination of Figure 2.1, it becomes obvious that the ciphertext alphabet can be shifted up to 25 positions to the right or it can be reversed and shifted to the left. When directly shifted to the right, a series of 25 cipher alphabets can be developed, since the 26th shift results in the ciphertext matching the plaintext. Perhaps the most famous example of the use of a shifted cipher alphabet is the one credited to Julius Caesar in which the original displacement between plaintext and ciphertext was three places to the right. This well-known cipher, as well as other displacement ciphers, is very weak because the reconstruction of plaintext may be reduced to a simple trial-and-error process. However, because you need to start somewhere, this simple displacement process represents a good starting point for our discussion of monoalphabetic uniliteral substitution systems.

The recipient of a message encoded as a monoalphabetic uniliteral substitution system requires a mechanism to convert ciphertext to plaintext. That mechanism is obtained by knowing the displacement used to encode the ciphertext and using that displacement to reverse the process. Mathematically the process can be expressed as P=(C-n) mod 26, where P represents the plaintext character's numeric value, C represents the ciphertext character's numeric value, n represents the displacement used to encode the ciphertext, and the modulo 26 process requires 26 to be subtracted when the result of the addition exceeds 25.

You can use Figure 2.1 to illustrate encipherment and decipherment. For example, assume the message HELP was encoded using a displacement of 5. The resulting ciphertext you would receive would be MJQU. You could convert the characters to their numeric equivalents, resulting in the sequence 12, 09, 21, 25. As previously noted, P=(C-n) mod 26. Since the displacement is 5 positions (P=(C-5) mod 26), this results in the received sequence of 12, 09, 21, 25 being converted to 07, 04, 16, 20. In the plaintext line in Figure 2.1 the numeric sequence 07, 04, 16, 20 represents the characters HELP which represents the deciphered message.

Displacement Alphabets

As previously illustrated in Figure 2.1, you can develop a series of 25 direct ciphertext alphabets with respect to a plaintext alphabet when both are restricted to uppercase English letters. The top portion of Listing 2.1 contains a simple program named SHIFT.BAS which displays all possible displaced alphabets from the standard uppercase English alphabet. In this program, the string array PLAINTXT$ is used to store each letter of the alphabet through the use of a READ statement contained in the first FOR-NEXT loop. The following pair of FOR-NEXT loops causes 25 shifted alphabets to be displayed with respect to the original alphabet which is displayed without any shift occurring. The bottom portion of Listing 2.1 illustrates the result obtained from the execution of SHIFT.BAS.

Listing 2.1 The SHIFT.BAS program listing.

Rem PROGRAM SHIFT.BAS
Rem This program produces a sequence of shifted alphabets
Cls
Dim PLAINTXT$(26)
For I = 0 To 25
READ PLAINTXT$(I)
Next I
Data "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K"
Data "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W",
"X", "Y", "Z"
For J = 0 To 25
For I = 0 To 25
Print PLAINTXT$((I + J) Mod 26);
Next I
Print
Next J

ABCDEFGHIJKLMNOPQRSTUVWXYZ
BCDEFGHIJKLMNOPQRSTUVWXYZA
CDEFGHIJKLMNOPQRSTUVWXYZAB
DEFGHIJKLMNOPQRSTUVWXYZABC
EFGHIJKLMNOPQRSTUVWXYZABCD
FGHIJKLMNOPQRSTUVWXYZABCDE
GHIJKLMNOPQRSTUVWXYZABCDEF
HIJKLMNOPQRSTUVWXYZABCDEFG
IJKLMNOPQRSTUVWXYZABCDEFGH
JKLMNOPQRSTUVWXYZABCDEFGHI
KLMNOPQRSTUVWXYZABCDEFGHIJ
LMNOPQRSTUVWXYZABCDEFGHIJK
MNOPQRSTUVWXYZABCDEFGHIJKL
NOPQRSTUVWXYZABCDEFGHIJKLM
OPQRSTUVWXYZABCDEFGHIJKLMN
PQRSTUVWXYZABCDEFGHIJKLMNO
QRSTUVWXYZABCDEFGHIJKLMNOP
RSTUVWXYZABCDEFGHIJKLMNOPQ
STUVWXYZABCDEFGHIJKLMNOPQR
TUVWXYZABCDEFGHIJKLMNOPQRS
UVWXYZABCDEFGHIJKLMNOPQRST
VWXYZABCDEFGHIJKLMNOPQRSTU
WXYZABCDEFGHIJKLMNOPQRSTUV
XYZABCDEFGHIJKLMNOPQRSTUVW
YZABCDEFGHIJKLMNOPQRSTUVWX
ZABCDEFGHIJKLMNOPQRSTUVWXY


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Since a goal of this book is to provide you with both BASIC and C++ modules you can incorporate into a program of your choice, the BASIC program SHIFT.BAS is also presented as a C++ program. Listing 2.2 provides the C++ program, which when executed produces the same results as those shown in the lower portion of Listing 2.1. As previously mentioned, C++ programs presented in this book were developed using Microsoft's Visual C++ program development package. Although Visual C++ can be considered to represent a "standardized" version of C++, it is probably worthwhile to review some items in the program listing. First, the array PLAINTEXT is used as a character array to contain the uppercase letters A through Z. Those characters are stored in elements 0 through 25 in the array. The increment operators (++) are positioned after each integer used for looping (i and j), which results in each integer being incremented after it is used in the loop. The first printf statement displays a sequence of 25 characters controlled by the i loop with the percent (%) sign used to represent modulo arithmetic. Once 25 characters are printed, the i loop terminates and the second printf statement simply generates a line feed. Both the program SHIFT.CPP and an executable version of the program stored under the filename SHIFT.EXE are contained on the CD-ROM accompanying this book. Source BASIC language programs and executable files are located in the BASIC directory, while source C++ language programs and executable files are located in the C directory.

Listing 2.2 The SHIFT.CPP program listing.

/*       PROGRAM SHIFT.C */
/*      This program produces a sequence of shifted alphabets */

#include <stdio.h>

int main()
{
   int i,j;

   char PLAINTEXT[26] = {'A','B','C','D','E',
           'F','G','H','I','J',
           'K','L','M','N','O',
           'P','Q','R','S','T',
           'U','V','W','X','Y','Z'};
    for ( j=0; j<=25; j++)
    {for (i=0; i<=25; i++)
    { printf("%c",PLAINTEXT[(i+j)%26]);
    } printf("\n");
    }

return (0);

}

In addition to a direct relationship between plaintext and ciphertext alphabets, you can reverse the relationship. That is, you can reverse and displace the ciphertext alphabet with respect to the plaintext alphabet. In doing so, the ciphertext can be applied to the plaintext alphabet at any of 26 points of coincidence between the two alphabets. Thus, you can develop a series of 25 reverse ciphertext alphabets with respect to a plaintext alphabet when both are restricted to uppercase English letters. Figure 2.2 illustrates an example of a reverse ciphertext alphabet in which the first letter of both the plaintext and ciphertext alphabets coincide. Note that the reverse ciphertext alphabet can also be shifted or displaced a total of 25 times prior to the relationship between plaintext and ciphertext characters repeats.

Figure 2.2 Reverse ciphertext alphabet.

The relationship between a plaintext and ciphertext alphabet can be direct or reversed. For either situation, the characters in one alphabet can be shifted n positions, or 0<n<25, with respect to the other alphabet prior to the alphabet repeating.

You can denote the relationship between plaintext and ciphertext alphabets through the location where the ciphertext alphabet coincides with the first letter (A) of the plaintext alphabet. This location is commonly referred to as the alphabetic shift key. For the ciphertext alphabet illustrated in Figure 2.1, the shift key can be expressed as PA=CF, where the P and C denote plaintext and ciphertext alphabets, and the subscript letters indicate the relationship between the two alphabets with respect to the beginning of the plaintext and ciphertext alphabets. For the reverse ciphertext alphabet illustrated in Figure 2.2, the alphabetic shift key can be expressed as PA=R(CA), where R indicates a reversal of the ciphertext alphabet beginning with the letter A.

Encipherment

To illustrate encipherment and decipherment operations, I'll use the simple displacement relationship between ciphertext and plaintext alphabets previously illustrated in Figure 2.1. In that example, the alphabetic shift key, PA=CF, defined the relationship between the two alphabets.

Suppose you wish to encipher the message MEET ME IN ST LOUIS. You would first locate M in the plaintext and note that it is replaced by R in the ciphertext alphabet. Next, each plaintext E would be replaced by a ciphertext J, and so on. The plaintext message's completed encipherment would yield RJJY RJ NS XY QTZNX.

In military systems, the resulting enciphered message would more than likely be rewritten into groups of five characters for transmission to minimize the effect of a transmission error and facilitate the transmission and reconstruction of the original message. Thus, if you were transmitting the message you might consider sending it as:

       RJJYR JNSXY QTZNX

Note that in placing the enciphered message into groups of five characters you remove spaces between words. This is usually done to eliminate obvious word groupings that would facilitate the decipherment of the message if it fell into the hands of the wrong party. Because this message has exactly 15 characters, it fills up three groups of five characters. This is not always the case, however, and unfilled groups are normally filled with one or more predefined letters known as message nulls. The most common message null is the letter X and should not be confused with the ASCII null character which represents a blank.

Care should be taken in selecting characters to fill ending message groups and terminating messages. In many military units, messages were terminated with the abbreviation FTA or one or more of those characters was used to terminate an ending message group. Not only was the abbreviation an obscene message concerning the Army, but more significantly‹, its inclusion in an enciphered message provided an enemy intercepting the message with the ability to rapidly gain knowledge concerning a portion of the substitution process used to encipher the message. Given that insight, with some additional effort, the entire contents of the message becomes vulnerable to decipherment.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Decipherment

Upon receipt of the enciphered message, you must know the relationship between the plaintext and ciphertext to perform our decipherment operation. Although you can use the alphabet relationship previously illustrated in Figure 2.1, that relationship is easier to use for encipherment than decipherment. For decipherment operations, you may wish to reverse the alphabet relationship shown in Figure 2.1, placing the ciphertext alphabet at the top. Then you would look up each letter from the enciphered message in the ciphertext alphabet and replace it with its plaintext alphabet equivalent. This produces the message MEETM EINST LOUIS, which you can rewrite to reflect the correct spacing between words, restoring the original form of the message.

Similar to the coding in the programs SHIFT.BAS and SHIFT.CPP, we can easily test any enciphered message to determine if a simple monoalphabetic substitution process was used. Because only 26 plaintext-ciphertext relationships exist, you can simply displace each character in the enciphered message by a uniform amount and vary that amount 25 times to discover the contents of the plaintext. Thus, a simple monoalphabetic substitution process does not offer a significant level of protection. However, the simple monoalphabetic substitution process represents an excellent starting point for a discussion of more sophisticated substitution processes as well as the development of a series of subroutines and programs that can be used to automate encipherment and decipherment operations. In the remainder of this chapter, I'll create a series of subroutines and programs that will form the basis for coding examples constructed in the rest of this book.

Automating Operations

The process of converting plaintext to ciphertext and the reverse process can be automated to facilitate monoalphabetic operations. As previously explained in Chapter 1, I'll use QuickBASIC and C++ to develop the subroutines and programs presented in this chapter and the remainder of this book.

Using an Alphabetic Shift Key

Listing 2.3 contains the program CIPHER1.BAS which can be used to create a ciphertext alphabet based upon the use of a defined alphabetic shift key. In this program, the string array PLAINTEXT$ is used to store the standard sequence of uppercase letters contained in the English alphabet, while the string array CIPHERTEXT$ is used to store the resulting cipher alphabet shifted according to an alphabetic shift key. Note that the subroutine INITIALIZE is used to initialize the plaintext values into the array PLAINTEXT$. In fact, in this and subsequent programs presented in this book, I'll use common subroutines, when possible, to facilitate the construction of specific programs or program segments developed to perform predefined operations. This will provide you with a "construction set" of routines that can be used to tailor the development of programs to your specific requirements.

Listing 2.3 The CIPHER1.BAS program listing.

REM PROGRAM CIPHER1.BAS
DIM PLAINTEXT$(25), CIPHERTEXT$(25)
GOSUB INITIALIZE
1       INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
       FOR I = 0 TO 25
       IF K$ = PLAINTEXT$(I) GOTO 2
       NEXT I
       PRINT "You must enter a letter from A to Z"
       GOTO 1
2       REM Position I represents shift key letter
GOSUB FORMCIPHER
GOSUB PRTOUT
STOP
INITIALIZE:
     REM Initialize plaintext values
     FOR I = 0 TO 25
     READ PLAINTEXT$(I)
     NEXT I
     DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
     DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
FORMCIPHER:
     REM Routine to form CIPHERTEXT alphabet based upon defined shift key
     J = I + 1
     FOR K = 0 TO 25
     CIPHERTEXT$(K) = PLAINTEXT$((K + J) MOD 26)
     NEXT K
RETURN
PRTOUT:
     REM Print results
     FOR I = 0 TO 25
     PRINT CIPHERTEXT$(I);
     NEXT I
     PRINT
RETURN
END

Returning to Listing 2.3, after the PLAINTEXT$ array is initialized, the INPUT statement displays the message "Enter UPPERCASE Alphabetic Shift Key" which is read into the string labeled K$. The value of K$ is compared against the alphabet to ensure that an uppercase letter was entered. If this is not the case, an error message is displayed and the program branches back to the statement labeled 1, causing the message "Enter UPPERCASE Alphabetic Shift Key" to be redisplayed. Assuming an uppercase letter is entered, the subroutine FORMCIPHER is invoked. In this subroutine, the position in the plaintext alphabet in which the shift key character equaled a plaintext character plus one is saved by assigning I+1 to the variable J. The reason I+1 is assigned to J is because the alphabet needs to be rotated through the position of the alphabetic shift key. For example, assume C was entered as the alphabetic shift key. Its position in the array PLAINTEXT$ is 2 since the index starts at 0. To shift the alphabet so it starts at the letter D you must add 1 to the position of the shift character.

The FOR-NEXT statement in the subroutine FORMCIPHER assigns 26 characters to the array CIPHERTEXT$ by adding the index K in the loop (which varies in value from 0 to 25) to the value of J, using the mod 26 operator. Thus, when K is 0 and the shift key is C, (K+J) mod 26 is 3, which results in D being assigned to CIPHERTEXT$(0). Similarly, when the shift key is C and K has a value of 25, (K+J) mod 26 has a value of 2, which results in the assignment of C to CIPHERTEXT$(25).

Once the FORMCIPHER subroutine is completed, the main part of the program invokes the subroutine PRTOUT. This subroutine prints the new alphabet based upon the entered alphabetic shift key.

Figure 2.3 illustrates several examples of the execution of CIPHER1.BAS. Note that the use of Z as the alphabetic shift key results in a full rotation of the alphabet.

Figure 2.3 CIPHER1.BAS execution examples.

The C++ version of the CIPHER1.BAS program can be located on the CD-ROM under the C directory. That program has the filename CIPHER1.CPP and is contained in Listing 2.4.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 2.4 The CIPHER1.CPP program listing.

/* PROGRAM :CIPHER1.CPP
  All C++ code written by Jonathan Held, January 20, 1998,
  using Microsoft's Visual C++ version 5.1.
*/
//standard include files
#include <iostream.h>

//function prototypes
void check(char &);
void findKey(char, const char [], int &);
void formCipher(const char[], char[], int);
void printResults(const char[], const char[]);

//main program
int main(){

    char PLAINTEXT[26] = {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I',
    'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U',
    'V', 'W', 'X', 'Y', 'Z'};

    char CIPHERTEXT[26], key;

    int key_location;

    cout << "Enter UPPERCASE Alphabetic Shift Key: ";
    cin >> key;
    check(key);

    findKey(key, PLAINTEXT, key_location);
    formCipher(PLAINTEXT, CIPHERTEXT, key_location);
    printResults(PLAINTEXT, CIPHERTEXT);

    return (0);
}//end main()

//-----------------------------------------------------
//Function: check()
//Parameters: key - alphabetic key entered by user
//Return type: none
//Purpose: Ensure key is a valid alphabetic character.  If it is
//a lowercase letter, this procedure will automatically convert it
//to an uppercase one.  This procedure also handles all error
//checking!
//-----------------------------------------------------
void check(char &key){
  int key_value;
  bool error = false;

  do {

       //only change came in this line, where I removed the call
       //to the function toupper()

       key_value = static_cast<int>(key);

       if ((key_value < 65) || (key_value >90)){
            cerr << "\n\aPlease enter an UPPERCASE Alphabetic Shift Key
                     (A-Z): ";
            cin >> key;
       }
       else {
       error = true;
       }
  } while (!(error));

  key = static_cast<char>(key_value);
  cout << "\nKey entered was: " << key << endl;

  return;
}//end check()

//-----------------------------------------------------
//Function: findkey()
//Parameters: key - alphabetic key entered by user
//      PLAINTEXT - the plaintext alphabet
//      key_location - the location of the key in the
//      PLAINTEXT alphabet
//Return type: none
//Purpose: Find the key in the PLAINTEXT alphabet.  This position
//will be used when we form the cipher.
//-----------------------------------------------------
void findKey(char key, const char PLAINTEXT[],
int &key_location){

   for (int ix=0; ix<26; ix++){
      if (key == PLAINTEXT[ix]){
          key_location = ix + 1;
          break;
      }
    }

    return;
}//end findKey()

//-----------------------------------------------------
//Function: formcipher()
//Parameters: PLAINTEXT - the plaintext alphabet
//      CIPHERTEXT - the cipher alphabet we will create
//      loc - the key location in the plaintext
//Return type: none
//Purpose: Create the ciphertext using modulo arithmetic.
//-----------------------------------------------------
void formCipher(const char PLAINTEXT[], char CIPHERTEXT[],
int loc){

   for (int ix=0; ix<26; ix++){
       CIPHERTEXT[ix] = PLAINTEXT[(ix + loc) % 26];
   }

   return;
}//end formCipher()

//-----------------------------------------------------
//Function: printresults()
//Parameters: PLAINTEXT - the plaintext alphabet
//      CIPHERTEXT - the ciphertext
//Return type: none
//Purpose: Print the plaintext and corresponding ciphertext based
//on the key the user selected.
//-----------------------------------------------------
void printResults(const char PLAINTEXT[],
const char CIPHERTEXT[]){

   cout << "PLAINTEXT:  ";

   for (int ix=0; ix<26; ix++){
         cout << PLAINTEXT[ix] << " ";
   }

   cout << "\nCIPHERTEXT: ";

   for (int jx=0; jx<26; jx++){
         cout << CIPHERTEXT[jx] << " ";
   }

   cout << endl;

   return;
}//end printResults()

In examining the C++ program listing for CIPHER1.CPP, note that four functions were used—check, findKey, formCipher, and printResults. Each function is documented within the program listing to include a description of their parameter(s), return type, and purpose. The execution of CIPHER1.CPP will produce the same results as the execution of CIPHER1.BAS, which was previously illustrated in Listing 2.3. The executable version of CIPHER1.CPP is named CIPHER1.EXE and is located under the C directory on the CD-ROM included with this book.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Converting to Ciphertext

We can build upon the programs CIPHER1.BAS and CIPHER1.CPP to expand their capability so that they can convert plaintext to ciphertext. Listing 2.5 contains the program listing for this new program which we named CIPHER2.BAS. This elementary program enciphers a one-line message based upon an alphabetic shift key.

Listing 2.5 The CIPHER2.BAS program listing.

REM PROGRAM CIPHER2.BAS
    PRINT "This program enciphers a one-line message based upon an
alphabetic shift key"
DIM PLAINTEXT$(26), CIPHERTEXT$(26)
GOSUB INITIALIZE
1   INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
    FOR I = 0 TO 25
    IF K$ = PLAINTEXT$(I) GOTO 2
    NEXT I
    PRINT "You must enter a letter from A to Z"
    GOTO 1
2   REM Position I represents shift key letter
GOSUB FORMCIPHER
    PRINT "Enter your message in UPPERCASE:"
    INPUT TEXT$
    MSGLEN = LEN(TEXT$)
GOSUB MSGENCIPHER
GOSUB PRTOUT
STOP
INITIALIZE:
    REM Initialize plaintext values
    FOR I = 0 TO 25
    READ PLAINTEXT$(I)
    NEXT I
    DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
    DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
FORMCIPHER:
    REM Routine to form CIPHERTEXT alphabet based upon defined shift key
    J = I + 1
    FOR K = 0 TO 25
    CIPHERTEXT$(K) = PLAINTEXT$((K + J) MOD 26)
    NEXT K
RETURN
MSGENCIPHER:
    REM Convert plaintext to ciphertext
    FOR I = 1 TO MSGLEN
    FOR J = 0 TO 25
    IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO 5
    NEXT J
5   MID$(TEXT$, I, 1) = CIPHERTEXT$(J)
    NEXT I
RETURN
PRTOUT:
    REM Print results
    PRINT "Resulting enciphered message is:"
    PRINT TEXT$
RETURN
END

In examining Listing 2.5, let us focus our attention upon the new additions, specifically the lines from PRINT "Enter your message in UPPERCASE" to STOP, and the subroutine MSGENCIPHER.

The statements following the referenced PRINT statement simply accept a one-line message and store it in the string variable TEXT$, obtain the length of the message through the use of the LEN statement and store the length of the message in the variable MSGLEN, invoke the subroutine MSGENCIPHER, and output or print the results obtained from an encipherment of the message by invoking the subroutine PRTOUT. Thus, the key to encipherment once the ciphertext is developed through the use of an alphabetic shift key is the subroutine MSGENCIPHER, which we will now discuss.

The MSGENCIPHER Subroutine

The subroutine MSGENCIPHER contains a nested pair of FOR-NEXT loops in which I is varied in the outer loop from 1 to the last character of the message denoted by MSGLEN. In the inner loop, J is varied from 0 to 25 and is used to match each plaintext character in the message to its equivalent PLAINTEXT array character position through the use of the MID function. For example, when I has a value of 1, MID$(TEXT$, I, 1) extracts the first character from the string TEXT$ which, when used in the IF statement, is compared to the value of the string variable PLAINTEXT$(J). As J is varied from 0 to 25, a value is reached in which the extracted character from TEXT$ equals a character in the PLAINTEXT$ string array and a branch to the statement label 5 occurs. At that location, the character in the string TEXT$ is replaced by the character in the CIPHERTEXT$ array located at position J. Thus, each character in the plaintext message will be replaced by a character from the shifted alphabet whose position coincides with the unshifted alphabet plaintext character.

Figure 2.4 illustrates the execution of the program CIPHER2.BAS using B as the alphabetic shift key to form the cipher alphabet. Note that at this point in time you have to eliminate spaces between words as there is no space character in our plaintext or ciphertext alphabet. Later in this book, I'll develop a routine which will remove spaces between words, enabling a user to enter a plaintext message in a more natural manner.

Figure 2.4 A CIPHER2.BAS execution example.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHER2.CPP Program

In developing a C++ version of CIPHER2.BAS, a considerable visual improvement occurred by allowing spaces between words. Listing 2.6 contains a listing of the program CIPHER2.CPP. Since this program is well defined through the use of a significant number of comments concerning its coding, our comments on its coding will be minimized. As noted from the function prototypes section, the program uses five key functions. The getShiftKey function checks for and accepts an uppercase letter as the shift key to be used by the program. The createCipher function creates the ciphertext alphabet based upon the shift key previously entered. The third function, getMessage, retrieves the user's message which can be up to 256 uppercase characters in length. The fourth function, formCipheredMessage, enciphers the previously entered message while the fifth function, printResults, displays the corresponding enciphered message.

Listing 2.6 The CIPHER2.CPP program listing.

/*
Program Cipher2.cpp
C++ Code written by Jonathan Held, January 21, 1998, using
Microsoft Visual C++, version 5.0

211 total lines of code.
*/

//standard include files
#include <iostream.h>
#include <string.h>

//function prototypes
void getShiftKey(char &);
void createCipher(const char [], char[], const char);
void getMessage(char []);
void formCipheredMessage(const char[], const char [], char []);
void printResults(const char[], const char[], const char[],
const char []);

//main program
int main(){

  //initialize plaintext
  char plaintext[26] = {'A','B','C','D','E','F','G','H','I',
                        'J','K','L','M','N','O','P','Q','R',
                        'S','T','U', 'V','W','X','Y','Z'};

  //other variables we will use
  char ciphertext[26], message_to_cipher[256],
  enciphered_message[256], key;

  //function calls

  do {

     //get the uppercase key
     getShiftKey(key);

     //create the cipher key
     createCipher(plaintext, ciphertext, key);

     //get the users message
     getMessage(message_to_cipher);

     //form the ciphered message
     formCipheredMessage(ciphertext, message_to_cipher,
     enciphered_message);

     //print off the results
     printResults(plaintext, ciphertext, message_to_cipher,
                    enciphered_message);

  } while (true);

  return (0);

}//end main()

//-----------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//-----------------------------------------------------
void getShiftKey(char &key_desired){

  bool error = true;

  do {
     //prompt user to enter an uppercase shift key
     cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
     cin >> key_desired;

     int key_value = static_cast<int>(key_desired);

     //do some error checking
     if ((key_value < 65) || (key_value > 90)){
        cerr << "\nYou must enter a letter
        from A to Z!" << endl << endl;
     }
     else {
        cout << endl;
        error = false;
     }
  } while (error);

  return;
}//end getShiftKey()

//-----------------------------------------------------
//Function: createCipher()
//Parameters:  PTEXT - the plaintext alphabet
//       ctext - the cipher alphabet we are going to create
//       user_key - the key the user entered
//Return Type: None
//Purpose: Create the cipher stream we will use later to encode the
//user's message.
//-----------------------------------------------------
void createCipher(const char PTEXT[], char ctext[], const char
user_key){

  int location;
  //find the location of the key in the plaintext
  for (int ix=0; ix<26; ix++){
     if (user_key == PTEXT[ix]){
        //location is one more than ix
        location = ix + 1;
        break;
     }
  }

  //create the cipher text
  for (int jx=0; jx<26; jx++){
     ctext[jx] = PTEXT[(jx + location) % 26];
  }
  return;
}//end createCipher();

//-----------------------------------------------------

//Function: getMessage()
//Parameters:  msg_to_cipher[] - the message entered by the user
//Return Type: None
//Purpose: Get the user's message and find out how long it is using
//the standard string function strlen including in the string.h
//library file.
//-----------------------------------------------------
void getMessage(char msg_to_cipher[]){

  //get the newline character off of the input stream
  cin.get();

  cout << "Enter the message in UPPERCASE characters: " << endl;

  //get the entire line, up to 256 characters
  cin.getline(msg_to_cipher, 256, '\n');

  return;
}//end getMessage()

//-----------------------------------------------------
//Function: formCipheredMessage()
//Parameters: CTEXT - the cipher alphabet we will use for substitution
//      MESSAGETOCIPHER - the user's message
//      enc_message - the enciphered message to be determined
//Return Type: None
//Purpose: Encipher the user's one-line message.
//-----------------------------------------------------
void formCipheredMessage(const char CTEXT[], const char MESSAGETOCIPHER[],
            char enc_message[]){

  int length = strlen(MESSAGETOCIPHER)+1;

  for (int ix=0; ix<length; ix++){
     //get character from MESSAGETOCIPHER
     char encode = MESSAGETOCIPHER[ix];
     int encode_value = static_cast<int>(encode);

     //handle the case where an input character is not a letter
     //A - Z
     if ((encode_value < 65) || (encode_value > 90)){
            enc_message[ix] = MESSAGETOCIPHER[ix];
     }
     else {
        //valid character - the easy way to calculate the ciphered
        //character is based on the plain text's ascii character value;
        //since it has to be a capital letter, it must be in the range
        //from 65 to 90, with A represented by 65, Z by 90. By simply
        //subtracting 65 from the encode_value (the integer representation
        //of the plaintext character), we now know what cipher character
        //to use.
        enc_message[ix] = CTEXT[encode_value-65];
     }
  }

    return;
}//end formCipheredMessage()

//-----------------------------------------------------
//Function: printResults()
//Parameters: PTEXT - the plaintext alphabet
//       CTEXT - the cipher alphabet we used for subsitution
//       MESSAGETOCIPHER - the user's message
//       ENCIPHEREDMESSAGE - the corresponding enciphered message
//Return Type: None
//Purpose: Display the values of all passed-in parameters.
//-----------------------------------------------------
void printResults(const char PTEXT[], const char CTEXT[],
                      const char MESSAGETOCIPHER[],
const char ENCIPHEREDMESSAGE[]){

  cout << "\nPLAINTEXT:  ";
  for (int ix=0; ix<26; ix++){
     cout << PTEXT[ix] << " ";
  }

  cout << endl;
  cout << "CIPHERTEXT: ";
  for (int jx=0; jx<26; jx++){
     cout << CTEXT[jx] << " ";
  }

  cout << "\n\nORIGINAL MESSAGE: " << MESSAGETOCIPHER
<< endl;
  cout << "\nCIPHERED MESSAGE: " << ENCIPHEREDMESSAGE
<< endl << endl;

  return;
}//end printResults()
//end file cipher5.cpp

Key differences between the BASIC and C++ versions of the CIPHER2 program are in the validation of uppercase letters and termination of the program. The C++ version uses ASCII values from 65 through 90 as a check for uppercase A through Z and recognizes the CTRL-C key combination as a mechanism to quit the program.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Limiting the Effect of Errors

The meaning of an enciphered message can be adversely affected by a human typing error or a transmission or transcription error. In such situations, a mistake which adds or omits a character would jumble the decipherment of the meaning of a message. Thus, we need a mechanism to limit the effect of errors caused by humans or technology. One common mechanism has its roots dating to the development of military systems and is known as character grouping—the most common method being the placement of enciphered text into groups of five characters for transmission.

To place ciphertext into groups of five characters, we will modify the CIPHER2.BAS and CIPHER2.CPP programs by the inclusion of a new subroutine. That subroutine, appropriately named GROUPBY5, replaces the subroutine PRTOUT contained in the CIPHER2.BAS program.

The GROUPBY5 Subroutine

Listing 2.7 lists the contents of the subroutine GROUPBY5. That subroutine uses the variable L as a group counter. When invoked, the subroutine sets the value of L to 1. The FOR-NEXT loop in the subroutine prints one character at a time through the use of the MID$ function, which extracts a character from the string variable TEXT$. After a character is printed, the value of the FOR-NEXT loop index (I) divided by 5 is compared to the value of L. If the value of I/5 equals the value of L, five characters have been printed and a branch to the label 6 occurs. At that label, the PRINT statement generates a space. If the value of I/5 does not equal the value of L, a branch to the label 7 occurs and the FOR-NEXT loop terminates the current index value of the loop.

Listing 2.7 The GROUPBY5 subroutine.

GROUPBY5:
  L = 1
  FOR I = 1 TO MSGLEN
  PRINT MID$(TEXT$, I, 1)
  IF I / 5 = L THEN GOTO 6
  GOTO 7
6 PRINT ' ';
  L = L + 1
7 NEXT I
RETURN

One of the more famous messages transmitted using groupings of five characters was sent by Takeo Yoshikawa, a Japanese naval ensign assigned as a consulate secretary to the Japanese consulate in Honolulu. At 6 P.M. on December 6, 1941, he sent his final message using what was known as Oite to the Japanese and PA-K2 to American codebreakers. This message contained 44 groups of five characters and was transmitted to Tokyo via RCA communications at a cost of $6.82.

The message transmitted by ensign Yoshikawa reported the arrival of an American battleship and mine sweeper into port and provided a summary of the number of ships at anchor and ships in dock in Honolulu by category—battleships, light cruisers, destroyers, and mine sweepers. The message also noted that it appeared that no air reconnaissance was being conducted by the U.S. fleet's air arm.

Although American cryptanalysts were able to crack messages transmitted in a PA-K2 code, doing so required an average of three days. Unfortunately, the contents of this message were deciphered well after the attack on Pearl Harbor had begun.

The CIPHER3.BAS Program

Listing 2.8 contains the listing of the main body of the program CIPHER3.BAS. That program includes the subroutine GROUPBY5 which groups a line of enciphered text into groups of five characters. Figure 2.5 illustrates an example of the execution of the CIPHER3.BAS program using the alphabetic shift key B, and shows the five-character grouping of the enciphered message. If you are familiar with military systems, you may wonder why the GROUPBY5 subroutine leaves the last group with less than five characters instead of filling the group by adding X's or some similar group terminator character. The reason for not doing so at this time is because this program only operates on one message line at a time. Thus, before you terminate an unfilled group, you should first modify our program to read and process multiple lines of plaintext.

Listing 2.8 The CIPHER3.BAS program listing.

REM PROGRAM CIPHER3.BAS
DIM PLAINTEXT$(25), CIPHERTEXT$(25)
CLS
       GOSUB INITIALIZE
1       INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
       FOR I = 0 TO 25
       IF K$ = PLAINTEXT$(I) GOTO 2
       NEXT I
       PRINT "You must enter a letter from A to Z"
       GOTO 1
2       REM Position I represents shift key letter
GOSUB FORMCIPHER
       PRINT "Enter your message in UPPERCASE:"
       INPUT TEXT$
       MSGLEN = LEN(TEXT$)
GOSUB MSGENCIPHER
       REM Print results
       PRINT "Resulting enciphered message is:"
GOSUB GROUPBY5
STOP
INITIALIZE:
       REM Initialize plaintext values
       FOR I = 0 TO 25
       READ PLAINTEXT$(I)
       NEXT I
       DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
       DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
FORMCIPHER:
       REM Routine to form CIPHERTEXT alphabet based upon defined shift key
       J = I + 1
       FOR K = 0 TO 25
       CIPHERTEXT$(K) = PLAINTEXT$((K + J) MOD 26)
       NEXT K
RETURN
MSGENCIPHER:
       REM Convert plaintext to ciphertext
       FOR I = 1 TO MSGLEN
       FOR J = 0 TO 25
       IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO 5
       NEXT J
5       MID$(TEXT$, I, 1) = CIPHERTEXT$(J)
       NEXT I
RETURN
GROUPBY5:
       L = 1
       FOR I = 1 TO MSGLEN
       PRINT MID$(TEXT$, I, 1);
       IF I / 5 = L THEN GOTO 6
       GOTO 7
6       PRINT " ";
       L = L + 1
7       NEXT I
RETURN
END

Figure 2.5 A sample execution of the program CIPHER3.BAS.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHER3.CPP Program

The C++ version of CIPHER3.BAS follows our naming convention and is stored in the C directory under the filename CIPHER3.CPP. This program is similar to the BASIC language version in that it first asks for the entry of an uppercase alphabetic shift key character. Next, it displays the plaintext and ciphertext alphabet resulting from the shift and prompts you to enter your "original message." Using the shift key the program generates an enciphered message in which the resulting enciphered text is displayed in military fashion in groups of five characters.

Listing 2.9 contains the CIPHER3.CPP program listing. Note that the function printResults places the enciphered text into groups of five characters and adds a space when the variable "counter" does not equal 0 and the value of counter mod 5 equals 0.

Listing 2.9 The CIPHER3.CPP program listing.

/*
Program Cipher3.cpp
C++ Code written by Jonathan Held, January 21, 1998, using
Microsoft Visual C++, version 5.0.

248 total lines of code.
*/

//standard include files
#include <iostream.h>
#include <string.h>
//function prototypes
void getShiftKey(char &);
void createCipher(const char [], char[], const char);
void getMessage(char []);
void formCipheredMessage(const char[], const char [], char []);
void printResults(const char[], const char[],
const char[], const char []);
void printCipher(const char[]);

//main program
int main(){

  //initialize plaintext
  char plaintext[26] = {'A','B','C','D','E','F','G','H','I',
               J','K','L','M','N','O','P','Q','R',
               S','T','U', 'V','W','X','Y','Z'};

  //other variables we will use
  char ciphertext[26], message_to_cipher[256],
enciphered_message[256], key;
  //function calls

  do {

     //get the uppercase key
     getShiftKey(key);

     //create the cipher key
     createCipher(plaintext, ciphertext, key);

     //get the users message
     getMessage(message_to_cipher);

     //form the ciphered message
     formCipheredMessage(ciphertext, message_to_cipher,
     enciphered_message);

     //print off the results
     printResults(plaintext, ciphertext, message_to_cipher,
                       enciphered_message);

  } while (true);

  return (0);

}//end main()

//-----------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//-----------------------------------------------------
void getShiftKey(char &key_desired){

  bool error = true;

  do {
     //prompt user to enter an uppercase shift key
     cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
     cin >> key_desired;

     int key_value = static_cast<int>(key_desired);
     //do some error checking
     if ((key_value < 65) || (key_value > 90)){
        cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
     }
     else {
        cout << endl;
        error = false;
     }
   } while (error);
   return;
}//end getShiftKey()

//-----------------------------------------------------
//Function: createCipher()
//Parameters: PTEXT - the plaintext alphabet
//            ctext - the cipher alphabet we are going to create
//            user_key - the key the user entered
//Return Type: none
//Purpose: Create the cipher stream we will use later to encode the
//user's message.
//-----------------------------------------------------
void createCipher(const char PTEXT[], char ctext[],
const char user_key){

  int location;

  //find the location of the key in the plaintext
  for (int ix=0; ix<26; ix++){
     if (user_key == PTEXT[ix]){
         //location is one more than ix
        location = ix + 1;
        break;
     }
  }

  //create the cipher text
  for (int jx=0; jx<26; jx++){
     ctext[jx] = PTEXT[(jx + location) % 26];
  }

  return;
}//end createCipher();
//-----------------------------------------------------
//Function: getMessage()
//Parameters:  msg_to_cipher[] - the message entered by the user
//Return Type: None
//Purpose: Get the user's message and find out how long it is using
//the standard string function strlen including in the string.h
//library file.
//-----------------------------------------------------
void getMessage(char msg_to_cipher[]){

   //get the newline character off of the input stream
  cin.get();

  cout << "Enter the message in UPPERCASE characters: " << endl;

  //get the entire line, up to 256 characters
  cin.getline(msg_to_cipher, 256, '\n');

  return;
}//end getMessage()

//-----------------------------------------------------
//Function: formCipheredMessage()
//Parameters:  CTEXT - the cipher alphabet we will use for substitution
//             MESSAGETOCIPHER - the user's message
//             enc_message - the enciphered message to be determined
//Return Type: None
//Purpose: Encipher the user's one-line message.
//-----------------------------------------------------
void formCipheredMessage(const char CTEXT[], const char MESSAGETOCIPHER[],
                        char enc_message[]){

  int length = strlen(MESSAGETOCIPHER)+1;

  for (int ix=0; ix<length; ix++){

     //get character from MESSAGETOCIPHER
     char encode = MESSAGETOCIPHER[ix];
     int encode_value = static_cast<int>(encode);

     //handle the case where an input character is not a letter
     //A - Z
     if ((encode_value < 65) || (encode_value > 90)){
        enc_message[ix] = MESSAGETOCIPHER[ix];
     }
     else {
        //valid character - the easy way to calculate the ciphered
        //character is based on the plain text's ascii character value;
        //since it has to be a capital letter, it must be in the range
        //from 65 to 90, with A represented by 65, Z by 90.  By simply
        //subtracting 65 from the encode_value (the integer representation
        //of the plaintext character), we now know what cipher character
        //to use.
        enc_message[ix] = CTEXT[encode_value-65];
    }
  }

  return;
}//end formCipheredMessage()

//-----------------------------------------------------
//Function: printResults()
//Parameters:  PTEXT - the plaintext alphabet
//             CTEXT - the cipher alphabet we used for subsitution
//             MESSAGETOCIPHER - the user's message
//             ENCIPHEREDMESSAGE - the corresponding enciphered message
//Return Type: None
//Purpose: Display the values of all passed-in parameters.
//-----------------------------------------------------
void printResults(const char PTEXT[], const char CTEXT[],
const char
MESSAGETOCIPHER[],
                 const char ENCIPHEREDMESSAGE[]){

  cout << "\nPLAINTEXT:  ";
  for (int ix=0; ix<26; ix++){
     cout << PTEXT[ix] << " ";
  }

  cout << endl;
  cout << "CIPHERTEXT: ";
  for (int jx=0; jx<26; jx++){
     cout << CTEXT[jx] << " ";
  }

  cout << "\n\nOriginal Message: " << endl << MESSAGETOCIPHER << endl;
  cout << "\nEnciphered Message: " << endl;
  printCipher(ENCIPHEREDMESSAGE);

  return;
}//end printResults()

//-----------------------------------------------------
//Function: printCipher()
//Parameters:  MSG - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-block characters
//-----------------------------------------------------
void printCipher(const char MSG[]){

  int counter = 0;

  int length = strlen(MSG)+1;

  for (int kx=0;kx<length;kx++){

     if (*(MSG+kx) != ' '){

        if ((counter != 0) && (counter%5 == 0)) {
           cout << " ";
        }

        cout << *(MSG+kx);
        counter++;
     }
  }

  cout << endl << endl;

  return;
}//end printCipher()
//end file cipher3.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Processing Multiple-Line Messages

Since you want the ability to read a message containing multiple lines of text, let's modify our previously constructed program by the addition of several subroutines. One subroutine will permit a message to be input from the keyboard or a previously created file. To provide you with the ability to enter text more naturally, this subroutine will also permit you to enter words with spaces between words, since the program will become responsible for their removal. Other subroutines will convert the plaintext message into ciphertext and position the enciphered text into groups of five characters for printing, terminating the last group of characters with an appropriate number of X's to fill any terminating group that has less than five characters.

The MSGFILE Subroutine

Listing 2.10 shows the contents of the subroutine labeled MSGFILE. This subroutine performs three major functions. First, it permits the user to assign filenames for the file used to store a plaintext and resulting ciphertext message or to select the default filenames MESSAGE.DAT for storing the plaintext message and CIPHERTX.DAT for storing the enciphered message. The second function performed by MSGFILE is to remove spaces between words. The third function performed by this subroutine is to store in a file the resulting plaintext message in which any spaces between words were removed.

Listing 2.10 The MSGFILE subroutine.

MSGFILE:
   REM Routine to assign I/O files and accept keyboard or file input
   REM and remove spaces between words
       INPUT "Enter filename to store plaintext message,
                        default=MESSAGE.DAT", INFILE$
       IF INFILE$ = "" THEN INFILE$ = "MESSAGE.DAT"
       INPUT "Enter filename to store enciphered message,
                         default=CIPHERTX.DAT", OUTFILE$
       IF OUTFILE$ = "" THEN OUTFILE$ = "CIPHERTX.DAT"
       INPUT "Select keyboard (k) or file (f) message input: ", IN$
       IF IN$ = "F" OR IN$ = "f" THEN RETURN
       OPEN INFILE$ FOR OUTPUT AS #1
   REM Routine to place message on a file removing spaces between words
       PRINT "Enter your message - place a / at the beginning of
                         each line"
       PRINT "that should remain in plaintext and a \ on a
                         separate line"
       PRINT "to indicate the end of the enciphered message"
       PRINT
AGN:     LINE INPUT TEXT$
         IF MID$(TEXT$, 1, 1) = "/" THEN GOTO XT
         NTEXT$ = ""
         FOR I = 1 TO LEN(TEXT$)
         NTEXT$ = NTEXT$ + LTRIM$(MID$(TEXT$, I, 1))
         NEXT I
         WRITE #1, NTEXT$
         IF MID$(TEXT$, 1, 1) = "\" GOTO DONE
         GOTO AGN
XT:       WRITE #1, TEXT$
         GOTO AGN
DONE:      CLOSE #1
RETURN

The first portion of the subroutine MSGFILE prompts the user to enter filenames for storing a plaintext message and its resulting enciphered message. If the user simply presses the Return key, the default filenames MESSAGE.DAT and CIPHERTX.DAT are used. Otherwise, the subroutine prompts the user to enter the appropriate plaintext and ciphertext filenames.

The second major routine in MSGFILE performs the actual placement of keyboard-entered plaintext on a file and removes spaces between words. To facilitate the transmission of enciphered messages, this portion of the subroutine permits users to prefix each heading message line with a forward slash (/) character if they do not want that message line to be enciphered. This allows a header portion of a message to be ignored during the encipherment process. The transmission of the enciphered text with a cleartext header facilitates its distribution to the appropriate recipient. To indicate the end of the message, a backslash (\) character must be entered by itself on a separate line.

The FOR-NEXT loop in the second half of the subroutine MSGFILE uses the LTRIM$ statement to remove spaces between words. This FOR-NEXT loop is invoked only when the beginning of a line does not contain a forward slash character. Since LTRIM$ removes leading spaces in a string, its use in the FOR-NEXT loop places a line of plaintext in which all spaces have been removed into the string variable NTEXT$.

The CONVERTSTORE Subroutine

The following subroutine reads the contents of the previously stored messages a line at a time, converts each message into its enciphered text, and stores the enciphered text on a line-by-line basis in the file whose name is assigned to the string variable OUTFILE$. Listing 2.11 shows the statements in the subroutine CONVERTSTORE which perform the previously mentioned functions.

Listing 2.11 The CONVERTSTORE subroutine.

CONVERTSTORE:
   REM Routine to convert and store ciphertext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM Convert plaintext to ciphertext
          FOR I = 1 TO MSGLEN
          FOR J = 0 TO 25
          IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO GOTIT
          NEXT J
GOTIT:      MID$(TEXT$, I, 1) = CIPHERTEXT$(J)
          NEXT I
CLEARTXT:     WRITE #2, TEXT$
   LOOP
DONE1:    CLOSE #2
RETURN

After opening the file whose name was assigned to the string variable INFILE$ for input and the file whose name was assigned to OUTFILE$ for output, the subroutine uses a DO UNTIL loop to read each message line from the file containing the plaintext message. Within the DO UNTIL loop, each line is processed based upon the first character in the line.

If the first character in a line is a forward slash, a branch to the label CLEARTXT occurs and the conversion of plaintext to enciphered text is bypassed. If the first character in a line is a backslash, the end of the previously stored message has been reached and a branch to the label DONE1 occurs. This terminates the loop, closes the previously opened file #2, and causes an exit from the subroutine to occur.

If the first character in each line is neither a forward slash nor a backslash character, the subroutine converts each plaintext character in the line into its equivalent ciphertext character. This is accomplished through the use of a pair of nested FOR-NEXT loops. The outer loop cycles through each of the characters in a line retrieved from the file containing the plaintext. The inner loop compares on a character-by-character basis the plaintext character to each character in the string array PLAINTEXT$. When a match occurs, a branch to the label GOTIT occurs. At that label, the plaintext character is replaced by the ciphertext character whose position in the array CIPHERTEXT$ is the same as the position of the plaintext character in the array PLAINTEXT$.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The PRTOUT Subroutine

Now that subroutines were developed to place a message on a file and process and convert the message to enciphered text which is stored on another file, I am ready to print the results. To do this I will develop a subroutine named PRTOUT which will process the file containing the enciphered text so that no more than 25 characters or five groups of five characters are displayed on a line. We have limited the printout of enciphered text to 25 characters per line to facilitate manual operations for readers who wish to use the resulting display of the enciphered message in place of the file containing the enciphered text for sending a message.

Listing 2.12 shows the contents of the subroutine PRTOUT. After using an INPUT statement to display the message informing a user to press the Return key, the subroutine clears the display and sets the string variable TEMP$ to a null value. Next, the message "Resulting enciphered message is:" is displayed and the file assigned to the variable OUTFILE$ is opened for input. Note that the enciphered text was previously stored on the file assigned to the variable OUTFILE$; thus, this subroutine will process the enciphered message.

Listing 2.12 The PRTOUT subroutine.

PRTOUT: REM Subroutine to print results
    INPUT "Press Return key to display resulting enciphered message", p$
    CLS: TEMP$ = ""
    PRINT "Resulting enciphered message is:"
    OPEN OUTFILE$ FOR INPUT AS #2
    DO UNTIL EOF(2)
        INPUT #2, TEXT$
        IF MID$(TEXT$, 1, 1) = "/" THEN PRINT RIGHT$(TEXT$,
                           LEN(TEXT$) - 1)
        IF MID$(TEXT$, 1, 1) = "/" THEN GOTO NOGROUP
        IF TEMP$ = "" GOTO BLANK         'blank string
        TEMP$ = TEMP$ + TEXT$            'concatenate
        TEXT$ = TEMP$
        TEMP$ = ""
BLANK:     MSGLEN = LEN(TEXT$)
        IF MSGLEN >= 25 THEN GOTO BIG
        TEMP$ = TEXT$
        GOTO NOGROUP               'less than 25 characters
BIG:     FOR I = 26 TO MSGLEN          'place remainder of string
                                                    'into temp
        TEMP$ = TEMP$ + MID$(TEXT$, I, 1)
        NEXT I
        MSGLEN = 25
        GOSUB GROUPBY5:                  'print 25 characters
NOGROUP: LOOP
        TEXT$ = TEMP$                    'print remainder
        MSGLEN = LEN(TEXT$)
        IF MSGLEN <= 25 THEN GOSUB GROUPBY5
        IF MSGLEN <= 25 THEN RETURN      'done printing
     WHILE MSGLEN > 25
        TEMP$ = LEFT$(TEXT$, 25)         'get first 25 characters
                                                         'in string
        TEMPR$ = RIGHT$(TEXT$, MSGLEN - 25)
        TEXT$ = TEMP$
        MSGLEN = LEN(TEXT$)
        GOSUB GROUPBY5
        TEXT$ = TEMPR$
        MSGLEN = LEN(TEXT$)
     WEND
        TEXT$ = TEMPR$
        MSGLEN = LEN(TEXT$)
        GOSUB GROUPBY5
RETURN

The DO UNTIL loop processes the lines on the enciphered file. First, each line is read into the string variable TEXT$. The first two IF statements check for the presence of a forward slash. If a forward slash occurs in the first character position in a line, the first IF statement causes the line to be printed without the slash. Since a forward slash indicates the line should be treated as cleartext, the second IF statement simply skips the remaining statements in the loop by branching to the label NOGROUP.

The first time PRTOUT is invoked, the string TEMP$ is set to the null string. Thus, the first time through the DO UNTIL loop, the third IF statement causes a branch to the label BLANK to occur. At that label the length of the retrieved line of ciphertext is determined through the use of the LEN function. If the length of the line equals or exceeds 25 characters, a branch to the label BIG occurs. If the length of the line did not exceed 25 characters, there were not enough characters for printing 25 characters on one line. At this point, the characters assigned to the variable TEXT$ are assigned to the variable TEMP$ and a branch to the label NOGROUP occurs to terminate one cycle through the loop.

When the value of TEMP$ is not a null string, the contents of the next line of ciphertext are concatenated, or joined, to the contents of TEMP$. Then, the contents of TEMP$ are assigned to TEXT$ and TEMP$ is reset to a null string. In this manner, you can continue to cycle through the contents of the enciphered file until you either reach the end of the file or the length of the variable TEXT$ equals or exceeds 25 characters.

When the length of TEXT$ equals or exceeds 25 characters, a branch to the label BIG occurs. At that location, the FOR-NEXT loop places the remainder of the contents of TEXT$ that exceed 25 characters into the variable TEMP$. Next, the message length variable MSGLEN is set to 25 and the subroutine GROUPBY5 is called. That subroutine, which was previously discussed and which will be discussed in more detail in the next section in this chapter, actually prints the characters in groups of fives and assigns X's to fill the last group, if that group contains less than five characters.

When all lines in the file assigned to OUTFILE$ are read, the DO UNTIL loop is exited. At this point, you need to print the remaining enciphered text stored in the variable TEMP$. To do this, assign the contents of TEMP$ to TEXT$ and obtain its length. If the length of TEXT$ is less than or equal to 25 characters, invoke the subroutine GROUPBY5 to print the last 25 or less characters in the enciphered message and then exit the subroutine. If there are more than 25 characters in TEXT$, use a WHILE-WEND loop to print the contents of TEXT$. Assign the first 25 characters in TEXT$ to TEMP$ through the use of the LEFT$ function. Next, the remaining characters in the string TEXT$ are assigned to the variable TEMPR$. After the contents of TEMP$ are assigned to TEXT$, the message length is determined and the subroutine GROUPBY5 is invoked. The contents of TEMPR$ are assigned to TEXT$, the length of TEXT$ is determined, and the subroutine GROUPBY5 is again invoked, this time printing the remaining portion of characters that exceeded 25 in length. When less than 25 characters remain in TEMPR$, the WHILE-WEND loop terminates. At that point the contents of TEMPR$ are assigned to TEXT$, its length is determined, and the subroutine GROUPBY5 is invoked for the last time.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Modifying the GROUPBY5 Subroutine

As previously constructed, this subroutine performs the actual printing of characters in groups of five. Now we will modify this subroutine so that it fills the last group with X's if there are less than five characters in the group. Listing 2.13 shows the statement in this modified subroutine.

Listing 2.13 The modified GROUPBY5 subroutine.

GROUPBY5:
   L = 1
   FOR I = 1 TO MSGLEN
       PRINT MID$(TEXT$, I, 1);
       IF I / 5 = L THEN GOTO AGROUP
       GOTO NGROUP
AGROUP:   PRINT " ";
       L = L + 1
NGROUP: NEXT I
   NUMX = 5 - MSGLEN MOD 5
   IF NUMX = 5 OR NUMX = 0 THEN GOTO PRTBLK
   FOR I = 1 TO NUMX
   PRINT "X";
   NEXT I
PRTBLK: PRINT
RETURN

When the subroutine is invoked, the variable L is set to 1 and the FOR-NEXT loop cycles through the contents of the string variable TEXT$, printing one character at a time. If the index I of the FOR-NEXT loop divided by 5 equals L, five characters have been printed and a branch to the label AGROUP occurs. At that location, the PRINT statement prints a space, providing a space between groups of five characters. Next, the variable L is incremented and the loop ends. If a group of five characters was not printed, a branch to the label NGROUP occurs, bypassing the printing of the space.

Once the FOR-NEXT loop is completed, you must determine if a group did not have five characters, a condition that can only occur when the last line in the enciphered message is processed. Here, the value of the variable NUMX determines the number of X's that must be added to fill the last group. Only if NUMX is greater than 0 or less than 5 do we invoke the next FOR-NEXT loop to print the terminating X's. Otherwise, a branch to the label PRTBLK occurs, which terminates a previous PRINT statement ending with a semicolon so that the next cycle through the subroutine GROUPBY5 will have succeeding groups placed on a new line.

The CIPHER4.BAS Program

The entire program previously developed as modules will be listed to facilitate recognizing how the previously mentioned subroutines and calling program relate to each other. This program is contained in the directory BASIC in the file CIPHER4.BAS on the CD-ROM and is given in Listing 2.14.

Listing 2.14 The CIPHER4.BAS program listing.

REM PROGRAM CIPHER4.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26)
CLS
    GOSUB INITIALIZE
    PRINT "CIPHER4.BAS PROGRAM accepts spaces between words in a
                plaintext "
    PRINT "message and selectively enciphers the message using a
           simple"
    PRINT "monoalphabetic substitution process."
1    INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
    FOR I = 0 TO 25
    IF K$ = PLAINTEXT$(I) GOTO 2
    NEXT I
    PRINT "You must enter a letter from A to Z"
    GOTO 1
2    REM Position I represents shift key letter
GOSUB FORMCIPHER          'create cipher alphabet
GOSUB MSGFILE          'assign I/O files, place message on a file
GOSUB CONVERTSTORE         'convert and store ciphertext on a file
GOSUB PRTOUT           'print results
STOP
INITIALIZE:
    REM Initialize plaintext values
    FOR I = 0 TO 25
    READ PLAINTEXT$(I)
    NEXT I
    DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
    DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
FORMCIPHER:
    REM Routine to form CIPHERTEXT alphabet based upon defined shift key
    J = I + 1
    FOR K = 0 TO 25
    CIPHERTEXT$(K) = PLAINTEXT$((K + J) MOD 26)
    NEXT K
RETURN
MSGFILE:
    REM Routine to assign I/O files and accept keyboard or file input
    REM and remove spaces between words
        INPUT "Enter filename to store plaintext message,
                          default=MESSAGE.DAT", INFILE$
        IF INFILE$ = "" THEN INFILE$ = "MESSAGE.DAT"
        INPUT "Enter filename to store enciphered message,
                          default=CIPHERTX.DAT", OUTFILE$
        IF OUTFILE$ = "" THEN OUTFILE$ = "CIPHERTX.DAT"
        INPUT "Select keyboard (k) or file (f) message input: ", IN$
        IF IN$ = "F" OR IN$ = "f" THEN RETURN
               OPEN INFILE$ FOR OUTPUT AS #1
    REM Routine to place message on a file removing spaces between words
       PRINT "Enter your message - place a / at the beginning of
                         each line"
       PRINT "that should remain in plaintext and a \ on a
                                 separate line"
       PRINT "to indicate the end of the enciphered message"
       PRINT
AGN:     LINE INPUT TEXT$
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO XT
       NTEXT$ = ""
       FOR I = 1 TO LEN(TEXT$)
       NTEXT$ = NTEXT$ + LTRIM$(MID$(TEXT$, I, 1))
       NEXT I
       WRITE #1, NTEXT$
       IF MID$(TEXT$, 1, 1) = "\" GOTO DONE
       GOTO AGN
XT:     WRITE #1, TEXT$
       GOTO AGN
DONE:    CLOSE #1
RETURN
CONVERTSTORE:
      REM Routine to convert and store ciphertext on a file
      OPEN INFILE$ FOR INPUT AS #1
      OPEN OUTFILE$ FOR OUTPUT AS #2
      DO UNTIL EOF(1)
          INPUT #1, TEXT$
          MSGLEN = LEN(TEXT$)
          IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
          IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
          REM Convert plaintext to ciphertext
             FOR I = 1 TO MSGLEN
             FOR J = 0 TO 25
             IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO GOTIT
             NEXT J
GOTIT:         MID$(TEXT$, I, 1) = CIPHERTEXT$(J)
             NEXT I
CLEARTXT         WRITE #2, TEXT$
   LOOP
DONE1:         CLOSE #2
RETURN
PRTOUT: REM Subroutine to print results
   INPUT "Press Return key to display resulting enciphered message", p$
   CLS: TEMP$ = ""
   PRINT "Resulting enciphered message is:"
   OPEN OUTFILE$ FOR INPUT AS #2
   DO UNTIL EOF(2)
      INPUT #2, TEXT$
      IF MID$(TEXT$, 1, 1) = "/" THEN PRINT RIGHT$(TEXT$,
                         LEN(TEXT$) - 1)
      IF MID$(TEXT$, 1, 1) = "/" THEN GOTO NOGROUP
      IF TEMP$ = "" GOTO BLANK        'blank string
      TEMP$ = TEMP$ + TEXT$       'concatenate
      TEXT$ = TEMP$
      TEMP$ = ""
BLANK:  MSGLEN = LEN(TEXT$)
      IF MSGLEN >= 25 THEN GOTO BIG
      TEMP$ = TEXT$
      GOTO NOGROUP                     'less than 25 characters
BIG:            FOR I = 26 TO MSGLEN       'place remainder of string
                                                    'into temp
       TEMP$ = TEMP$ + MID$(TEXT$, I, 1)
       NEXT I
       MSGLEN = 25
       GOSUB GROUPBY5:                  'print 25 characters
NOGROUP: LOOP
       TEXT$ = TEMP$                    'print remainder
       MSGLEN = LEN(TEXT$)
       IF MSGLEN <= 25 THEN GOSUB GROUPBY5
       IF MSGLEN <= 25 THEN RETURN      'done printing
     WHILE MSGLEN > 25
       TEMP$ = LEFT$(TEXT$, 25)         'get first 25 characters
                                                         'in string

       TEMPR$ = RIGHT$(TEXT$, MSGLEN - 25)
       TEXT$ = TEMP$
       MSGLEN = LEN(TEXT$)
       GOSUB GROUPBY5
       TEXT$ = TEMPR$
       MSGLEN = LEN(TEXT$)
     WEND
       TEXT$ = TEMPR$
       MSGLEN = LEN(TEXT$)
       GOSUB GROUPBY5
RETURN
GROUPBY5:
   L = 1
   FOR I = 1 TO MSGLEN
       PRINT MID$(TEXT$, I, 1);
       IF I / 5 = L THEN GOTO AGROUP
       GOTO NGROUP
AGROUP:   PRINT " ";
       L = L + 1
NGROUP:NEXT I
       NUMX = 5 - MSGLEN MOD 5
       IF NUMX = 5 OR NUMX = 0 THEN GOTO PRTBLK
       FOR I = 1 TO NUMX
       PRINT "X";
       NEXT I
PRTBLK:  PRINT
RETURN
END

Note that the main portion of CIPHER4.BAS very closely resembles CIPHER3.BAS. However, calls using the GOSUB statement were added to access the subroutines MSGFILE, CONVERTSTORE, and PRTOUT—the latter subroutine accesses the subroutine GROUPBY5 as previously discussed. You should also note the inclusion of several PRINT statements at the beginning of the program to better describe its operation.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Program Execution

To illustrate the use of CIPHER4.BAS, execute the program just developed. The top portion of Figure 2.6 illustrates the execution of the program which displays program information and enables the user to enter his or her message. In this example, the first two lines of the message were entered with the forward slash as a prefix character to ensure those lines remained in cleartext. After the two-line heading, the next four lines required encipherment and were entered without a forward slash character used as a prefix. Finally, the backslash character was entered by itself on a separate line to indicate the termination of the message.

Figure 2.6 The execution of CIPHER4.BAS

The lower portion of Figure 2.6 illustrates the resulting enciphered message printed in groups of five characters. Note that the first two lines of heading information are simply printed without alteration, while the body of the message is both enciphered and printed in groups of five characters. Also note that the last group contained three characters and required the addition of two X's to complete the group of five. You can easily confirm this visually by noting that the message ended with the word STOP. Because a simple alphabetic shift was used for enciphering using the character B as the shift key, each character of plaintext is shifted up two positions in the alphabet. Thus, STOP becomes UVQR. Because VQR are the last three characters in the message, two X's were added to complete the group.

Program Default Files

To understand the contents of the default files created by CIPHER4.BAS, let us examine their contents after the program is executed. Figure 2.7 lists the contents of MESSAGE.DAT and CIPHERTX.DAT based upon the message illustrated in Figure 2.6 being entered when CIPHER4.BAS was executed. As indicated in Figure 2.7, the heading is placed on a file as is, while the body of the message is placed on each file with spaces between words being removed.

Figure 2.7 The CIPHER4.BAS file contents.

The CIPHER4.CPP Program

The C++ version of CIPHER4.BAS contains a number of programming improvements which facilitate the use of the program. The program listing of CIPHER4.CPP is contained in Listing 2.15.

Listing 2.15 The CIPHER4.CPP program listing.

/*
Cipher4.cpp
C++ Code written by Jonathan Held, January 23, 1998, using
Microsoft Visual C++, version 5.0.

486 total lines of code.
*/

//standard include files
#include <iostream.h>    //standard i/o operations
#include <string.h>      //used to find the length of a string
#include <ctype.h>       //for character handling
#include <fstream.h>     //file stream processing
#include <stdlib.h>      //standard library functions

//function prototypes - see function headers for more information
void getFileNames(char* &, char* &);
int getInputType(void);
void getShiftKey(char &);
void createCipher(const char [], char[], const char);
bool getMessage(char*, char*, char [], const char[],
const char[]); char* formCipheredMessage(const char[],
const char [], char []); void printResults(const char[],
const char[], const char[], const char [],
                    const int);
void printCipherToFile(ofstream, const char[]);
bool encryptText(char *, char *, const char [],
const char[], char[]);

//-----------------------------------------------------
//Function: main()
//Parameters:  argc - the number of command line arguments passed to
//       main
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//-----------------------------------------------------
int main(int argc, char *argv[]){
  //initialize plaintext
  char plaintext[26] = {'A','B','C','D','E','F','G','H','I',
               'J','K','L','M','N','O','P','Q','R',
               'S','T','U', 'V','W','X','Y','Z'};

  //other variables we will use
  char ciphertext[26], message_to_cipher[256],
enciphered_message[256], key;
  char *infile, *outfile;
  int input_type;
  bool success = false;

  //function calls

  //this code allows the user to run the program from the
  //dos prompt with command line arguments, i.e. the user
  //can run the program at the dos prompt by typing
  //cipher <input filename> <output filename> <key>
  if (argc >= 4){

     infile = new char[strlen(argv[1])+1];
     strcpy(infile,argv[1]);

     outfile = new char[strlen(argv[2])+1];
     strcpy(outfile,argv[2]);

     key = toupper(*argv[3]);
     int key_value = static_cast<int>(key);

     if ((key_value < 65) || (key_value > 90)){
        //print error message
        cerr << "Error: Invalid key used. Goodbye." << endl;
        //force the program to terminate due to error
        exit(EXIT_FAILURE);
      }

     input_type = 1;
  }
  //user tried to run from the dos prompt but made a mistake
  else if (argc >=2 && argc <4){
     cout << "Usage: cipher10 <inputfile> <outputfile> <key>";

     //force the program to terminate due to error
     exit(EXIT_FAILURE);
  }
  else {
     //user wants to manually enter information

     //get file information
         getFileNames(infile, outfile);
     input_type = getInputType();

     //get the uppercase key
          getShiftKey(key);
       }

  //create the cipher key
  createCipher(plaintext, ciphertext, key);

  //process file input
  if (input_type){
     success = encryptText(infile, outfile, plaintext, ciphertext,
                     enciphered_message);
  }
  else {
     cout << "Use a \'\\' to leave a line in plaintext." << endl
          << "Use a \'\\' to indicate end of message input. " << endl;
     success = getMessage(infile, outfile, message_to_cipher, plaintext,
                                ciphertext);
  }

  //report success of operation
  if (!success){
     cerr << "Error: Invalid filename specified. Goodbye." << endl;
  }

  //delete dynamically allocated memory
  delete [] infile;
  delete [] outfile;

  return (EXIT_SUCCESS);

}//end main()

//-----------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
//       outfile_name - the output file we will write the
//       enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//-----------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
  char buffer[256];
  int length;

  cout << "Enter filename to store/retrieve plaintext message: ";

  cin >> buffer;
  length = strlen(buffer);

  infile_name = new char[length + 1];
  strcpy(infile_name, buffer);

  cout << "\nEnter filename to store enciphered message: ";

  cin >> buffer;
  length = strlen(buffer);
  outfile_name = new char[length + 1];
  strcpy(outfile_name, buffer);

  cout << endl;

  return;
}//end getFileNames()

//-----------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicated keyboard input, 1 indicates file
         input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//-----------------------------------------------------
int getInputType(void)
{
  char type;
  bool error = false;
  int value;
  do {
     //prompt user for input from file or keyboard
     cout << "Is file input from keyboard (K, k) or file (F, f): ";
     cin >> type;

     //make type an uppercase letter
    type = static_cast<char>(toupper(static_cast<int>(type)));

 //check for an invalid character
 if ((type != 'K') && (type != 'F')){
    cerr << "You have entered an invalid character!" << endl << endl;
    error = true;
}
else {
  if (type == 'K')
     value = 0;        //value of 0 represents keyboard input
     else value = 1;   //value of 1 represents file input
  error = false;
 }

} while (error);

  cout << endl;

  return value;

}//end getInputType()

//-----------------------------------------------------
//Function: encryptText()
//Parameters:  inp_file - the name of the input plaintext file
//     outp_file - the name of the output ciphertext file
//     PTEXT[] - the plaintext alphabet
//     CTEXT[] - the ciphertext alphabet
//     encoded_msg[] - the message to be encoded
//Return Type: bool, indicating success of operation
//Purpose: Used to encrypt file input.  Takes each line of the input
//file, encrypts it, and saves the result to the specified output
//file.
//-----------------------------------------------------
bool encryptText(char * inp_file, char * outp_file, const char PTEXT[],
          const char CTEXT[], char encoded_msg[])
{
  bool success = false;
  char buffer[256];
  char nextChar;

  //declare file stream objects
  ifstream input(inp_file, ios::in);
  ofstream output(outp_file, ios::app);

  if ((!input) || (!output)){
     //do nothing - I/O error; user will be notified upon
     //procedure's return to main()
}
else {

 success = true;

 //print plaintext and ciphertext alphabets to the
 //output file
 output << "PLAINTEXT:  " << PTEXT << endl;
 output << "CIPHERTEXT: " << CTEXT << endl << endl;

 while (input.getline(buffer, 256, '\n')){

  //look at the next character
  nextChar = input.peek();

  //if we've grabbed 256 characters, and the very next character
  //is the newline delimeter, we want to ignore it
  if (nextChar == '\n')
     input.ignore();

  //check to see if the user wants the line to appear in plain text
  if (buffer[0] == '/'){
     output << buffer << endl;
  }
  else {
    //encipher the line
    char *msg = formCipheredMessage(CTEXT, buffer, encoded_msg);
    //print the cipher in groups of five to the ouput file
    printCipherToFile(output, msg);
   } }

   //notify user where plaintext and ciphertext files are
     cout << "Plaintext file is: " << inp_file << endl;
     cout << "Encrypted file is: " << outp_file << endl << endl;

  }

  //don't forget to close the files
  input.close();
  output.close();

  //return success of the operation
  return success;
}//end encryptText()

//-----------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//-----------------------------------------------------
void getShiftKey(char &key_desired){

  bool error = true;

  do {
   //prompt user to enter an uppercase shift key
   cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
   cin >> key_desired;

   int key_value = static_cast<int>(key_desired);

   //do some error checking
   if ((key_value < 65) || (key_value > 90)){
      cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
    }
    else {
     error = false;
    }
  } while (error);

  cout << endl;

  return;
}//end getShiftKey()

//-----------------------------------------------------
//Function: createCipher()
//Parameters: PTEXT - the plaintext alphabet
//      ctext - the cipher alphabet we are going to create
//      user_key - the key the user entered
//Return Type: None
//Purpose: Create the cipher stream we will use later to encode the
//user's message.
//-----------------------------------------------------
void createCipher(const char PTEXT[], char ctext[], const char user_key){

 int location;

  //find the location of the key in the plaintext
  for (int ix=0; ix<26; ix++){
   if (user_key == PTEXT[ix]){
      //location is one more than ix
      location = ix + 1;
      break;
    }
  }

  //create the cipher text
  for (int jx=0; jx<26; jx++){
    ctext[jx] = PTEXT[(jx + location) % 26];
 }

  return;
}//end createCipher();

//-----------------------------------------------------
//Function: getMessage()
//Parameters:  input - the name of the input plaintext file
//       output the name of the output ciphertext file
//       msg_to_cipher - the essage to be encoded
//       PTEXT[] - the plaintext alphabet
//       CTEXT[] - the ciphertext alphabet
//Return Type: bool, indicating success of operation
//Purpose: Allow the user to manually input text from the keyboard.
//Save the text in plaintext to the input file; encrypt the text
//and save it to the specified output file for later retrieval.
//-----------------------------------------------------
bool getMessage(char* input, char* output,
char msg_to_cipher[], const char
                  PTEXT[], const
char CTEXT[]){

  bool go_on = true, success = false;

  ofstream textFile(input, ios::app);
  ofstream cipherFile(output, ios::app);

  if ((!textFile) || (!cipherFile)){
     //do nothing - error will be noted to user later
  }
  else {

   success = true;

   textFile << "PLAINTEXT:  " << PTEXT << endl;
   textFile << "CIPHERTEXT: " << CTEXT << endl << endl;

   //get the newline character off of the input stream
     cin.get();

   cout << "Enter the message in UPPERCASE characters: " << endl;

   while (go_on) {

    //get the entire line, up to 256 characters
    cin.getline(msg_to_cipher, 256, '\n');

   //case user doesn't want the text to be encrypted
   if (msg_to_cipher[0] == '/'){
      textFile << msg_to_cipher << endl;
      cipherFile << msg_to_cipher << endl;
   }

    //case user is done entering text (ASCII value is 92
    //for the newline delimeter
    else if (static_cast<int>(msg_to_cipher[0]) == 92){
      go_on = false;
   }

    //encrypt the text
      else {
       textFile << msg_to_cipher << endl;

       char enciphered_msg[256];
       formCipheredMessage(CTEXT,msg_to_cipher,enciphered_msg);
       printCipherToFile(cipherFile,enciphered_msg);
    }
  }
}

  //close the files
  textFile.close();
  cipherFile.close();

  //notify user where plaintext and ciphertext files are
  cout << "\nPlaintext file is: " << input << endl;
  cout << "Encrypted file is: " << output << endl << endl;

  return success;
}//end getMessage()

//-----------------------------------------------------
//Function: formCipheredMessage()
//Parameters:  CTEXT - the cipher alphabet we will use for substitution
//      MESSAGETOCIPHER - the user's message
//      enc_message - the enciphered message to be determined
//Return Type: char* - a pointer to the encoded information.
//Purpose: Encipher the user's message.
//-----------------------------------------------------
char* formCipheredMessage(const char CTEXT[], const char MESSAGETOCIPHER[],
             char enc_message[]){

  int length = strlen(MESSAGETOCIPHER)+1;

  int encode_value;

  for (int ix=0; ix<length; ix++){

     //test to see if we have an alphabetic charac‚ter; if not,
     //simply copy it to our encrypted message - this preserves
     //characters such as ', ! etc...
     if (!isalpha(static_cast<int>(MESSAGETOCIPHER[ix]))){
        enc_message[ix] = MESSAGETOCIPHER[ix];
     }
     els{
        //valid character - the easy way to calculate the ciphered
        //character is based on the plain text's ascii character value;
        //since it has to be a capital letter, it must be in the range
        //from 65 to 90, with A represented by 65, Z by 90.  By simply
        //subtracting 65 from the encode_value (the integer representation
        //of the plaintext character), we now know what cipher character
        //to use.

        encode_value = toupper(static_cast<int>(MESSAGETOCIPHER[ix]));
        enc_message[ix] = CTEXT[encode_value-65];
     }
  }

  //return a reference to the encoded message
  return enc_message;
}//end formCipheredMessage()

//-----------------------------------------------------
//Function: printCipherToFile()
//Parameters: op - the output file we are writing to
//      MSG - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-character blocks in the
//specified output file.
//-----------------------------------------------------
void printCipherToFile(ofstream op, const char MSG[]){

  int len = strlen(MSG) + 1;

  //keep track of number of characters
  int counter = 0;

  for (int kx=0;kx<len;kx++){

    if (*(MSG+kx) != ' '){

    //test to see if we need to print a space
    if ((counter != 0) && (counter%5 == 0)) {
       op << " ";
       }
    //if not, we have a character to print
    op << *(MSG+kx);

    //increment the count of characters
    counter++;
   }
  }

  //force a carriage return in the output file
  op << endl;

  return;
}//end printCipherToFile()
//end file cipher4.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

You can execute the program from the DOS prompt by typing:

      CIPHER4 <input file><output file><K>

Here the input and output files can use any extension; however, for consistency it is suggested that the extension .DAT be used. K represents the shift key you want to use, similar to our previous examples. To practice using CIPHER4.CPP, create a text file using a word processor or line editor. For example:

     THIS IS A TEST RUN OF THE PROGRAM
     /THIS LINE WILL BE IN CLEARTEXT
     THIS LINE WILL NOT

The C++ version of the program does not require the ending of file input with a backslash (\) character. Once you save your file as ASCII text, at the DOS prompt enter the previously referenced command line. Then, you can list the contents of the file OUTPUT.DAT by typing TYPE OUTPUT.DAT or by the use of a word processor. Due to the extensive documentation contained in the program we will not duplicate its contents by examining its modules.

Although simple monoalphabetic substitution produces ciphertext that offers little protection to a message, it provides you with a foundation for creating a program with many subroutines. Prior to moving on to more sophisticated methods of encipherment, I'll conclude this chapter by automating the decipherment process. Although you could create several new subroutines and incorporate them into our encipherment program through appropriate logic to form one program that both enciphers and deciphers messages, it may be more useful to illustrate the decipherment process by creating a separate program. That program will be named DCIPHER4.BAS to correspond to CIPHER4.BAS, because they both use the same simple monoalphabetic substitution process to encipher and decipher messages.

The DCIPHER4.BAS Program

The process of deciphering a message is the inverse of the encipherment process. For a simple monoalphabetic substitution process, you would first initialize the plaintext and ciphertext alphabets using the previously developed INITIALIZE and FORMCIPHER subroutines. Once the two alphabets are constructed, you would use a new subroutine to decipher the ciphertext characters by comparing each ciphertext character in the message to the ciphertext alphabet. When a match occurs, use the position of the character in the ciphertext alphabet as a pointer to extract a corresponding character from the plaintext alphabet. In addition to creating this new subroutine which is named DECIPHER, you must modify the previously created MSGFILE and PRTOUT subroutines. As you will recall, MSGFILE is used to assign I/O files and accept keyboard or file input, while PRTOUT is used to print the enciphered message. To reflect the new functions performed by those subroutines their names will be changed to DMSGFILE and DPRTOUT, respectively.

The DMSGFILE Subroutine

Listing 2.16 contains the statements in the subroutine DMSGFILE which assigns I/O files and accepts the entry of an enciphered message from either the keyboard or from a file. Because enciphered text was grouped into five-character groups with spaces between groups, the DMSGFILE subroutine is similar to the MSGFILE subroutine in that it removes spaces between groups, while MSGFILE removed spaces between words. Also note that the variable INFILE$ is used to store the name of the file containing the enciphered message, while the variable OUTFILE$ is used to store the resulting plaintext deciphered message. Thus, DMSGFILE assigns files opposite to the manner in which I/O files are assigned by the subroutine MSGFILE.

Listing 2.16 The DMSGFILE subroutine.

DMSGFILE:
   REM Routine to assign I/O files and accept keyboard or file
input
   REM and remove spaces between words
       INPUT "Enter filename to store enciphered message,
                        default=CIPHERTX.DAT",
INFILE$
       IF INFILE$ = "" THEN INFILE$ = "CIPHERTX.DAT"
       INPUT "Enter filename to store deciphered message,
                                 default=MESSAGE.DAT",
OUTFILE$
       IF OUTFILE$ = "" THEN OUTFILE$ = "MESSAGE.DAT"
       INPUT "Select keyboard (k) or file (f) ciphertext message
                        input: ", IN$
       IF IN$ = "F" OR IN$ = "f" THEN RETURN
       OPEN INFILE$ FOR OUTPUT AS #1
   REM Routine to place enciphered message
on a file removing spaces between groups
       PRINT "Enter your message - place a /
at the beginning of each line"
       PRINT "that should remain in plaintext and a \
on a separate line"
       PRINT "to indicate the end of the enciphered message"
       PRINT
AGN:     LINE INPUT TEXT$
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO XT
       NTEXT$ = ""
       FOR I = 1 TO LEN(TEXT$)
       NTEXT$ = NTEXT$ + LTRIM$(MID$(TEXT$, I, 1))
       NEXT I
       WRITE #1, NTEXT$
       IF MID$(TEXT$, 1, 1) = "\" GOTO DONE
       GOTO AGN
XT:     WRITE #1, TEXT$
       GOTO AGN
DONE:    CLOSE #1
RETURN

Similar to the manner in which MSGFILE was created, DMSGFILE uses the forward slash and backslash characters to indicate to the program that it should bypass conversion (/) or the end of the message is reached (\). Thus, the second part of DMSGFILE, with the exception of wording in PRINT statements, functions in the same manner as the subroutine MSGFILE used in the CIPHER4.BAS program.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DECIPHER Subroutine

The actual deciphering process is performed by the subroutine DECIPHER whose contents are listed in Listing 2.17. In examining the statements in the subroutine DECIPHER, readers should recognize its close resemblance to the subroutine CONVERTSTORE in the CIPHER4.BAS program. The only difference between the two subroutines is the search mechanisms bounded by the nested FOR-NEXT loops. In the DECIPHER subroutine, each character in the enciphered message is compared against the ciphertext alphabet. When a match occurs, the position of the character in the ciphertext alphabet is used as a pointer to extract a character from the plaintext alphabet. Also note that the first character of each line of data retrieved from the ciphertext file is examined to determine if a forward slash or backslash character is in the first position of the line. Similar to the encipherment process, the decipherment process uses those characters to bypass the decipherment process (/) or to terminate the decipherment process (\).

Listing 2.17 The DECIPHER subroutine.

DECIPHER:
   REM Routine to decipher and store plaintext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM Convert ciphertext to plaintext
          FOR I = 1 TO MSGLEN
          FOR J = 0 TO 25
          IF MID$(TEXT$, I, 1) = CIPHERTEXT$(J) THEN GOTO GOTIT
          NEXT J
GOTIT:      MID$(TEXT$, I, 1) = PLAINTEXT$(J)
          NEXT I
CLEARTXT:     WRITE #2, TEXT$
   LOOP
DONE1:   CLOSE #2
RETURN

The DPRTOUT Subroutine

The last subroutine required for decipherment is appropriately labeled DPRTOUT to reflect the fact that it prints the results of the decipherment process. Unlike the subroutine PRTOUT which limited the number of characters to 25 per line and required extensive logic to operate on strings and invoke the GROUPBY5 subroutine, DPRTOUT is very simple. DPRTOUT simply reads each line in the file containing the deciphered message, eliminates any forward slash character used as a line prefix, and prints each line as is. Listing 2.18 lists the contents of that subroutine.

Listing 2.18 The DPRTOUT subroutine.

DPRTOUT: REM Subroutine to print results
   INPUT "Press Return key to display resulting enciphered
message", p$
   CLS
   PRINT "Resulting deciphered message is:"
   OPEN OUTFILE$ FOR INPUT AS #2
   DO UNTIL EOF(2)
      INPUT #2, TEXT$
      IF MID$(TEXT$, 1, 1) = "/" THEN PRINT RIGHT$(TEXT$,
                         LEN(TEXT$) - 1)
      IF MID$(TEXT$, 1, 1) <> "/" THEN PRINT TEXT$
   LOOP
RETURN

To illustrate the composition of the resulting decipherment program, Listing 2.19 lists the main portion of the program DCIPHER4.BAS. You should note that two of the five subroutines, INITIALIZE and FORMCIPHER, are exactly the same as those subroutines used in CIPHER4.BAS. The three remaining subroutines, DMSGFILE, DECIPHER, and DPRTOUT, are modified versions of the subroutines MSGFILE, CONVERTSTORE, and PRTOUT. The entire decipherment program which deciphers a message enciphered using a simple monoalphabetic substitution process is stored on the file DCIPHER4.BAS on the CD-ROM under the BASIC directory. The executable program in that directory is named DCIPHER4.EXE.

Listing 2.19 The main portion of the DCIPHER4.BAS program.

REM PROGRAM DCIPHER4.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26)
CLS
GOSUB INITIALIZE
   PRINT "DCIPHER4.BAS PROGRAM deciphers a message using a simple"
   PRINT "monoalphabetic substitution process."
1   INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
   FOR I = 0 TO 25
   IF K$ = PLAINTEXT$(I) GOTO 2
   NEXT I
   PRINT "You must enter a letter from A to Z"
   GOTO 1
2  REM Position I represents shift key letter
GOSUB FORMCIPHER         'create cipher alphabet
GOSUB DMSGFILE         'assign I/O files, place message on a file
GOSUB DECIPHER         'convert enciphered message to plaintext
GOSUB DPRTOUT          'print results
STOP

Program Execution

To illustrate the operation of the DCIPHER4.BAS program, you can execute the program with input from the keyboard and from a file. To facilitate a comparison of the deciphered message to the original plaintext message, you can use the enciphered message contained at the bottom of Figure 2.6 as input to DCIPHER4.BAS.

Figure 2.8 illustrates the operation of DCIPHER4.BAS in which keyboard input was selected. In this example, you must use the keyboard to enter the ciphertext message in the same manner as it was displayed as a result of the execution of CIPHER4.BAS. That is, you must prefix each cleartext line with a forward slash, enter each line using the five-character groups contained on the line, and terminate the enciphered message with a backslash character on a separate line.

Figure 2.8 The execution of DCIPHER4.BAS using keyboard input.

The execution of DCIPHER4.BAS using keyboard input will display up to 25 characters per line because input was limited to five groups of five characters per line. In comparison, the use of file input when DCIPHER4.BAS is executed deciphers each full line contained on the file whose name was assigned to the variable OUTFILE$ (Figure 2.9). The use of file input may provide a more natural conversion to plaintext since words do not have to be split among two lines if they do not fit into a five-character group terminating a line.

Figure 2.9 The execution of DCIPHER4.BAS using file input.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DCIPHER4.CPP Program

In concluding this chapter we will turn our attention to the C++ version of DCIPHER4.BAS. This program, which is appropriately named DCIPHER4.CPP, is included on the accompanying CD-ROM under the C directory. In addition, the executable version of the program, named DCIPHER4.EXE, is also included in that directory.

Listing 2.20 contains the contents of the DCIPHER4.CPP program. This program is very similar to previously developed C++ programs, using functions to perform specific tasks. In this program you will note that constants were defined using mnemonics that are associated with constant values. This coding technique was used to facilitate any changes in coding readers may wish to effect.

Listing 2.20 The DCIPHER4.CPP program listing.

/*
dcipher.cpp   C++ code written by Jonathan Held
using Microsoft Visual C++, version5.0, on February 22, 1998.
*/

//standard include files
#include<iostream.h>
#include<fstream.h>
#include<stdlib.h>
#include<string.h>
//function prototypes
void getShiftKey(char &);
void createCipher(const char[], char[], const char);
void formatData(char []);
void getFileToDecipher(const char[], char[]);
void decipher(ifstream, ofstream, const char[], char[]);

//constants we will use
#define FIVE          5
#define TWENTYFIVE   25
#define TWENTYSIX    26
#define TWENTYSEVEN  27
#define SIXTYFIVE    65
#define NINETY       90
#define SIZE        256

//----------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------
int main()
{
  char plaintext[TWENTYSEVEN]=
 {'A','B','C','D','E','F','G','H',
                    'I','J','K','L','M','N','O','P',
                        'Q','R','S','T','U','V','W','X',
                        'Y','Z'};

  char ciphertext[TWENTYSEVEN], key;

  //get the shift key that was used
  getShiftKey(key);
  //create the cipher text based on the shift key
  createCipher(plaintext, ciphertext, key);

  //decipher the file
  getFileToDecipher(plaintext, ciphertext);

  return (0);

}

//----------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//----------------------------------------------------
void getShiftKey(char &key_desired){

  bool error = true;

  do {
     //prompt user to enter an uppercase shift key
     cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
     cin >> key_desired;

     int key_value = static_cast<int>(key_desired);

     //do some error checking
     if ((key_value < SIXTYFIVE) || (key_value > NINETY)){
        cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
     }
     else {
        cout << endl;
        error = false;
     }
  } while (error);

  return;
}//end getShiftKey()

//----------------------------------------------------
//Function: createCipher()
//Parameters: PTEXT - the plaintext alphabet
//      ctext - the cipher alphabet we are going to create
//      user_key - the key the user entered
//Return Type: none
//Purpose: Create the cipher stream we will use later to encode the
//user's message.
//----------------------------------------------------
void createCipher(const char PTEXT[], char ctext[], const char USER_KEY){
  int location;

  //find the location of the key in the plaintext
  for (int ix=0; ix<TWENTYSIX; ix++){
     if (USER_KEY == PTEXT[ix]){
         //location is one more than ix
        location = ix + 1;
        break;
     }
  }

  //create the cipher text
  for (int jx=0; jx<TWENTYSIX; jx++){
     ctext[jx] = PTEXT[(jx + location) % TWENTYSIX];
  }

  return;
}//end createCipher();

//-----------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//-----------------------------------------------------
void formatData(char data[]){

  for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
     if (*(data+nx) == ' '){
        //do nothing - skip over the space in the data
     }
     else {
        *(data+mx++) = *(data+nx);
     }
  }

  //don't forget to add the null terminator
  *(data+mx) = '\0';

  return;
}//end formatData()
//-----------------------------------------------------
//Function: getFileToDecipher()
//Parameters: PTEXT - the plaintext alphabet
//            ctext - the corresponding cipher text
//Return Type: None
//Purpose: Prompt the user for the name of the encrypted file
//and the file the user wants to store the decrypted text to.
//-----------------------------------------------------
void getFileToDecipher(const char PTEXT[], char ctext[])
{
  char fileinput[SIZE], fileoutput[SIZE];

  cout << "Enter name of file to decipher: ";
  cin >> fileinput;

  ifstream input(fileinput, ios::in);

  if (!(input)){
     cerr << "Input file not available. Exiting program." << endl;
     exit(EXIT_FAILURE);
  }

  cout << "Enter name of file for output: ";
  cin >> fileoutput;

  ofstream output(fileoutput, ios::out);

  if (!(output)){
     cerr << "Output file not created.  Exiting program." << endl;
     exit(EXIT_FAILURE);
  }

  decipher(input, output, PTEXT, ctext);

  //don't forget to close the files
  input.close();
  output.close();

  cout << "\nDeciphered text is in " << fileoutput << endl;

  return;
}//end getFileToDecipher()

//-----------------------------------------------------
//Function: decipher()
//Parameters: in - the input file we are deciphering
//      out - the output file we are writing the deciphered
//        text to
//      PTEXT - the plaintext alphabet (used to decipher)
//      ctext - the ciphertext
//Return Type: None
//Purpose: Decipher the input file and write the contents to the
//output file specified by the user.
//-----------------------------------------------------
void decipher(ifstream in, ofstream out, const char PTEXT[],
char ctext[])
{

  char enc_file_data[SIZE];

  //continue this process until we get to the end of the file
  while (in.getline(enc_file_data, SIZE, '\n')){
     if (enc_file_data[0] == '/'){
        out << enc_file_data << endl;
     }
     else {
        //format the data - i.e. get rid of all spaces
     formatData(enc_file_data);

     //dump data to file
     for (int ix=0; ix<strlen(enc_file_data); ix++){

        //used to keep track of what plaintext character
      //we are going to use
     int jx;

     for (jx=0; jx<TWENTYSIX; jx++){
     //find where the encrypted data is in the
     //ciphertext - this location corresponds to
     //the plaintext character location
       if (enc_file_data[ix] == ctext[jx])
       break;
       }

     //conditionals for grouping by five and inserting
  //new lines
  if (!(ix%TWENTYFIVE))
           out << endl;
        if ((ix!=0) && (!(ix%FIVE))){
           out << " " << PTEXT[jx];
        }
        else {
           out << PTEXT[jx];

        }
     }
     }
  }

  return;
}//end decipher()
//end file dcipher.cpp

The DCIPHER4.CPP program includes five functions as indicated in the function prototype section in the listing. getShiftKey obtains the desired uppercase key entered by the user and createCipher creates the cipher stream that would normally be used to encode a message. The function getFileToDecipher, as its name implies, prompts you for the name of the encrypted file and the file to be used to store the results of the decryption operation. The function decipher deciphers the contents of the input file and writes the results to the specified output file. This function invokes the function formatData for data formatting. In addition, the decipher function uses a for jx statement to locate the encrypted data in the ciphertext since the location corresponds to the plaintext character location. This operation requires the use of a pointer into the ciphertext array and explains why the function createCipher is used to create a cipher alphabet. DCIPHER4.CPP is similar to the execution of DCIPHER4.BAS where, with the exception of an option for typing input from the keyboard, the execution of the C++ program is left for you to perform.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 3
Keyword-Based Monoalphabetic Substitution

Using the information presented in Chapter 2, Monoalphabetic Substitution Concepts, as a base, this chapter continues the examination of monoalphabetic substitution techniques. This discussion opens with the development and use of a keyword-based mixed alphabet to add a degree of randomness to the development of plaintext and ciphertext alphabets. This will be followed by a discussion of several additional techniques that can be used to develop a ciphertext alphabet. Similar to the material presented in Chapter 2, you will examine different techniques and develop a series of subroutines and programs in Microsoft Corporation's QuickBASIC and C++ programing languages to automate the operation of each technique.

Keyword-Based Mixed Alphabets

The relationship between plaintext and ciphertext alphabets in which one alphabet is shifted from the other by n positions is easy to construct. Unfortunately, it also represents an encipherment method that offers a very limited degree of protection to a message that falls into the hands of the wrong party. To add an additional level of protection to messages, keyword mixed alphabets were developed.

Construction

In a keyword mixed alphabet, a word or phrase is selected as the keyword and is used for the formation of the letters of the alphabet, with repeated letters omitted after their first occurrence. At the end of the word or phrase, the remaining letters of the alphabet are used in their normal sequence, omitting letters previously used in the keyword or phrase. For example, suppose you use the phrase GOD SAVE THE QUEEN as a keyword phrase. The first step in this process is to group the phrase to form a continuous word and eliminate duplicate letters in this newly formed word. This would result in the keyword GODSAVETHQUN, since there were duplicate E's in the keyword. Next, you would add the letters of the alphabet to the keyword you just reduced to contain only one occurrence of each letter. This would result in the keyword-based mixed alphabet GODSAVETHQUNBCFIJKLMPRWXYZ.

Automating Keyword Construction

Listing 3.1 illustrates the statements in the program WORD.BAS which, when executed, produces a keyword-based mixed sequence alphabet. This program contains two subroutines—INITIALIZE and KEYWORD. The first subroutine is the same subroutine previously used in the CIPHER series of programs developed in Chapter 2 to initialize the PLAINTEXT array. The second subroutine was developed to produce a keyword-based mixed sequence alphabet. In this section, the focus is upon the subroutine labeled KEYWORD which actually forms the keyword-based mixed sequence alphabet. After a review of the operation of this subroutine, I'll show you how to incorporate it into a new series of programs which will enable you to create both keyword-based mixed alphabets as plaintext and ciphertext alphabets, and to incorporate a shift key character to encipher a message.

Listing 3.1 The WORD.BAS program listing.

REM PROGRAM WORD.BAS TO DEVELOP ALPHABET BASED ON A KEYWORD
DIM PLAINTEXT$(26), CIPHERTEXT$(26), KEY$(26)
  GOSUB INITIALIZE
  INPUT "Enter keyword in CAPS", TEXT$
  GOSUB KEYWORD
  PRINT "Keyword-based alphabet is:"; X$
  STOP
INITIALIZE:
  REM Initialize plaintext values
  FOR I = 0 TO 25
  READ PLAINTEXT$(I)
  NEXT I
  DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
  DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
KEYWORD:
  REM Place entered keyword into KEY$ array 1 character per position
    MSGLEN = LEN(TEXT$)
    FOR I = 1 TO MSGLEN
    KEY$(I) = MID$(TEXT$, I, 1)
    NEXT I
  REM ELIMINATE DUPLICATE LETTERS, REPLACE WITH NULLS
    K = 2
    FOR I = 1 TO MSGLEN
    FOR J = K TO MSGLEN
    IF KEY$(I)  KEY$(J) GOTO NOTDUP
    KEY$(J) = " "
NOTDUP:    NEXT J
    K = K + 1
    NEXT I
  REM REMOVE NULLS IN STRING
    X$ = ""
    FOR I = 1 TO MSGLEN
    X$ = X$ + LTRIM$(KEY$(I))
    NEXT I
  REM PLACE REVISED KEYWORD WITH NO DUPLICATE LETTERS BACK IN KEY$
    FOR I = 1 TO LEN(X$)
    KEY$(I) = MID$(X$, I, 1)
    NEXT I
  REM COMPARE KEY$ & PLAINTEXT$ ARRAYS, BLANK PLAINTEXT$ WHEN MATCHED
    FOR J = 1 TO LEN(X$)
    FOR K = 0 TO 25
    IF KEY$(J) = PLAINTEXT$(K) THEN PLAINTEXT$(K) = " "
    NEXT K
    NEXT J
  REM CREATE ONE STRING
    FOR I = 0 TO 25
    X$ = X$ + LTRIM$(PLAINTEXT$(I))
    NEXT I
  REM PLACE SEQUENCE BACK INTO PLAINTEXT$ ARRAY
    FOR I = 0 TO 25
    PLAINTEXT$(I) = MID$(X$, I + 1, 1)
    NEXT I
RETURN
END

The use of a mnemonic key to mix a cipher alphabet can be traced to a family of cryptologists that lived in Rome during the sixteenth century. One member of this family, Matteo Argenti, served as the papal secretary of ciphers under Pope Gregory XIV and five succeeding popes. Argenti taught cryptology to his younger brother, Marcello, who served as a cipher secretary to a cardinal and wrote a cryptology manual which contained the first written reference to the use of a mnemonic key to mix a cipher alphabet.

The first keyword mixed alphabet developed by the Argenti family used numerics in place of ciphertext characters. To increase the difficulty of unauthorized decipherment operations, the letters "q" and "u" were merged into a single unit for encipherment and rules prohibiting word separations and punctuation were developed.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The KEYWORD Subroutine

It may be useful to review the operation of the subroutine KEYWORD based upon each group of statements bounded by a REM statement. The first program module simply stores each character in the keyword into an element of the KEY$ string array. The second program module uses a nested pair of FOR-NEXT loops to eliminate duplicate letters in the keyword, replacing each duplicate letter with a null character. To illustrate the operation of this program module, assume that the keyword ALPHA, which has a string length of five, was used. In the program module, the inner loop (J) operates from 2 to 5, while the outer loop operates from 1 to 5. Thus, when I is 1, J varies from 2 to 5. This results in A being compared to L, P, H, and A. If KEY$(I) does not equal KEY$(J), a branch to the label NOTDUP occurs and J is incremented. If KEY$(I) equals KEY$(J) the statement KEY$(J)=" " sets KEY$(J) equal to a null character. After the value of J increments, K is incremented, which results in the second character in the keyword (L) being compared to the third (P) through the fifth (A) characters of the keyword.

The third program module removes the nulls previously inserted in place of duplicate characters. The statement X$=X$+LTRIM$(KEY$(I)) concatenates each element of KEY$ that is a non-null with the previous value of X$ because the string function LTRIM$ removes leading nulls from a string.

The fourth program module takes the recently formed string variable X$ which contains the keyword without duplicate characters and places those characters back into the array KEY$. The fifth program module then compares each character in the KEY$ array to each character in the PLAINTEXT$ array. If they match, the plaintext character is then set to a null. The sixth program module simply adds each non-blank element position of the PLAINTEXT array to the string X$, which now contains the keyword-based mixed alphabet. The seventh and last program module places the keyword-based alphabet into the array PLAINTEXT$.

The use of the previously described WORD.BAS program is illustrated in Figure 3.1, which indicates the QuickBASIC execution screen after the program WORD.BAS was executed three times in succession. The first time WORD.BAS was executed the keyword phrase SEVENBRIDES was used to develop the keyword-based mixed alphabet. The second time the program was executed the keyword phrase BROOKLYNBRIDGE was used, while the third keyword phrase was MICROSOFTWINDOWSISGREAT.

Figure 3.1 Results obtained from executing WORD.BAS several times.

The C++ version of WORD.BAS, which is appropriately named WORD.CPP, is contained in the C directory on the CD-ROM. Following our file naming conventions, the file WORD.EXE in the C directory is the executable version of the C++ program, while the file WORD.EXE located in the BASIC directory represents the executable version of the BASIC program.

Listing 3.2 provides the C++ version of WORD.BAS which was assigned the filename WORD.CPP. When you execute this program you will note several improvements. Those improvements include the ability to repetitively execute the program and to use the CTRL+C multi-key combination to terminate the program execution. Similar to all C++ programs developed for this book, WORD.CPP was written by subdividing activities into functions. Three key functions were developed—getKeyword, checkInput, and createAlphabet. The getKeyword function, as its name implies, prompts you for a keyword and continues prompting until a valid keyword without spaces or non-alphabetic characters is entered. The checkInput function is called by the getKeyword function and performs the actual verification of user input. The third function, createAlphabet, uses the verified keyword to construct the keyword alphabet. Since the program is documented through the use of a liberal amount of comment statements, you are referred to the program listing for a detailed explanation of the manner by which the code functions.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 3.2 The WORD.CPP program listing.

/*
word.cpp
C++ Code written by Jonathan Held, February 27, 1998, using
Microsoft Visual C++, version 5.0.  Please read file for
detailed description of how the program works.
*/

#include<iostream.h>
#include<iomanip.h>
#include<string.h>

//function prototypes
void getKeyword(char * &);
bool checkInput(char * &);
void createAlphabet(char *, char []);

//constants
#define TWENTYSIX  26
#define TWENTYSEVEN 27
#define SIXTYFIVE  65
#define NINETY    90
#define SIZE    256

//------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 if program terminated normally
//Purpose: Runs the main part of the program.
//------------------------------------------------
int main(){

  char *keyword;
  char ciphertext[TWENTYSEVEN]= {'\0'};

  cout << "*** AUTOMATING KEYWORD CONSTRUCTION ***" << endl
     << "***   Hit CTRL-C to quit program   ***" << endl << endl;

  do {

   getKeyword(keyword);
   cout << "\nYou entered\t\t" << keyword << endl;
   createAlphabet(keyword, ciphertext);
   cout << "Keyword alphabet is: " << setw(29) << ciphertext
                << endl << endl;
   delete [] keyword;
  }while(true);

  return (0);
}//end main()

//------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: None
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.
//------------------------------------------------
void getKeyword(char * &text)
{
  bool error = false;

  do {
    char buffer[SIZE];
    cout << "Enter keyword in CAPS (do not use" << endl
     << "spaces or non-alphabetic characters): ";

    cin.getline(buffer, SIZE, '\n');
    text = new char[strlen(buffer) + 1];
    strcpy(text, buffer);

    error = checkInput(text);

  } while (error);

  return;
}//end getKeyword()

//------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//------------------------------------------------
bool checkInput(char * &input)
{
  bool error = false;
  int count = strlen(input);

  for (int ix=0; ix<count; ix++){

    int char_value = static_cast<int>(*(input+ix));

    //determine if the user did not enter an uppercase character
    if ((char_value < SIXTYFIVE) || (char_value > NINETY)){
       error = true;
       cerr << "You entered an invalid keyword!" << endl << endl;
       break;
     }
  }

  return error;
}//end checkInput()

//------------------------------------------------
//Function: createAlphabet()
//Parameters: input - the keyword the user entered
//      cipher - the keyword alphabet that will be constructed
//Return Type: None
//Purpose: Creates the keyword alphabet.
//------------------------------------------------
void createAlphabet(char *input, char cipher[])
{
  bool used[TWENTYSIX];
  int index = 0,
     count = strlen(input);

  //no characters are initially used
  for (int ix=0; ix<TWENTYSIX; ix++){
    used[ix] = false;
  }

  //keep track of each character used, start forming the keyword
  //alphabet
  for (int jx=0; jx<count; jx++){

    //get each character of the input string (integer value)
    int char_value = static_cast<int>(*(input+jx));

    if (used[char_value-SIXTYFIVE]){
       //do nothing - the character was already used
    }
    else {
      //mark as used and add to the keyword alphabet
      used[char_value-SIXTYFIVE] = true;
      *(cipher+index++) = static_cast<char>(char_value);
    }
  }

  //go through the list of characters used - those which weren't
  //used should be added to the keyword alphabet
  for (int kx=0; kx<TWENTYSIX; kx++){

    if (!(used[kx])){
       *(cipher+index++) = static_cast<char>(SIXTYFIVE+kx);
    }
  }

  return;
}//end createAlphabet()
//end file word.cpp

In examining the automated creation of three keyword-based alphabets illustrated in Figure 3.1, you will note that the trailing portion of the resulting alphabets is very similar. In fact, the characters U and Z are positioned at the same locations in all three alphabets. If you simply positioned a keyword-based alphabet directly under a plaintext alphabet, there is a high degree of probability that one or more plaintext and ciphertext characters would equate to one another. This situation would obviously provide a valuable clue to the decipherment of an intercepted message. To eliminate or reduce the number of coincidental characters between a plaintext alphabet and a keyword-based ciphertext alphabet, you need to apply an alphabetic shift key to one or both alphabets.

Incorporating an Alphabetic Shift Key

In Chapter 2 you developed a program labeled CIPHER4.BAS that creates enciphered text based upon the use of an alphabetic shift key. In this chapter the discussion of monoalphabetic substitution is continued by examining the development of a keyword-based mixed alphabet as well as the statements in a subroutine contained in the program WORD.BAS to automate the keyword-based mixed alphabet creation process. Combining the use of a keyword-based mixed alphabet with an alphabetic shift key should result in a higher level of enciphered data protection than the separate use of either method. Thus, this section focuses on combining both techniques for enciphering and deciphering messages.

By placing the KEYWORD subroutine contained in WORD.BAS or the createAlphabet function from WORD.CPP into the CIPHER4.BAS or CIPHER4.CPP program, you gain the ability to encipher a message based upon a keyword or keyword phrase and an alphabetic shift. I'll call these new programs CIPHER5.BAS and CIPHER5.CPP.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHER5.BAS Program

Listing 3.3 illustrates the statements contained in the main portion of CIPHER5.BAS as well as the subroutine INITIALIZE, which required a slight modification from its prior form. You should note that lines terminating with five asterisks (*) as a comment indicate lines that were added to the main program to facilitate an explanation of its operation. Those lines can be removed if you do not wish to burden a user who has no desire to understand the enciphering process with this extraneous information.

Listing 3.3 The main portion of the program CIPHER5.BAS and the modified subroutine INITIALIZE.

REM PROGRAM CIPHER5.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26), KEY$(26)
CLS
GOSUB INITIALIZE
   PRINT "CIPHER5.BAS PROGRAM enciphers text based upon the use of
               a keyword or keyword"
   PRINT "phrase and an alphabetic shift key using a monoalphabetic
               substitution process."
   PRINT
   INPUT "Enter keyword or keyword phrase in UPPERCASE: ", TEXT$
   PRINT "Plaintext based alphabet is : ";    '*****
   FOR I = 0 TO 25: PRINT PLAINTEXT$(I);: NEXT I  '*****
   GOSUB KEYWORD             'form keyword-based mixed alphabet
   PRINT "Keyword-based alphabet is     : "; X$  '*****
1  INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
   FOR I = 0 TO 25
   IF K$ = PLAINTEXT$(I) GOTO 2
   NEXT I
   PRINT "You must enter a letter from A to Z"
   GOTO 1
2  REM Position I represents shift key letter

GOSUB FORMCIPHER       'create cipher alphabet
   PRINT "Shifted keyword mixed alphabet is  : ";   '*****
   FOR I = 0 TO 25: PRINT CIPHERTEXT$(I);: NEXT I: PRINT    '*****
GOSUB INITIALIZE       'reinitialize plaintext array
GOSUB MSGFILE          'assign I/O files, place message on a file
GOSUB CONVERTSTORE     'convert and store ciphertext on a file
GOSUB PRTOUT           'print results
STOP
INITIALIZE:
     RESTORE
     REM Initialize plaintext values
     FOR I = 0 TO 25
     READ PLAINTEXT$(I)
     NEXT I
     DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
     DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
 RETURN

As indicated in Listing 3.3, first initialize the string array PLAINTEXT$ and then display information about the program. After the keyword or keyword phrase is entered and assigned to the string variable TEXT$, the plaintext alphabet is displayed. Next, the subroutine KEYWORD is invoked to form a keyword-based mixed alphabet. That subroutine is the same as the subroutine previously developed in the program WORD.BAS.

After the subroutine KEYWORD is invoked, the keyword-based alphabet is displayed via the use of another optional statement. The user is then prompted to enter an alphabetic shift character, and the subroutine FORMCIPHER is invoked to create a cipher alphabet. That alphabet is displayed through the use of a pair of optional statements.

Because the subroutine FORMCIPHER creates the cipher alphabet by shifting the previously created keyword-based mixed alphabet, you must restore the plaintext alphabet to its original sequence to correctly convert plaintext characters to their equivalent ciphertext characters. This is accomplished by again invoking the subroutine INITIALIZE. However, because that subroutine previously read the normal letter sequence into the array PLAINTEXT, you must place a RESTORE statement at the beginning of the subroutine. Otherwise, you would receive an "out-of-data" error message.

The Encipherment Process

Figure 3.2 illustrates the execution of CIPHER5.BAS to encipher a short but important message. To understand the encipherment process, examine the composition of the different alphabets displayed by the program.

Figure 3.2 A sample execution of CIPHER5.BAS

The plaintext based alphabet is simply the character sequence A through Z. After a keyword or keyword phrase is entered, the plaintext sequence is modified based upon the keyword or keyword phrase. For example, entering the keyword MICROSOFTWINDOWS results in the keyword-based alphabet MICROSFTWNDABEGHJKLPQUVXYZ because all duplicate characters in the keyword or keyword phrase are eliminated prior to the addition of the characters in the plaintext alphabet that are not in the modified keyword or keyword phrase. The entry of an alphabetic shift key causes the cycling of the keyword-based alphabet so that all characters to and including the alphabetic shift key character are rotated. Thus, the shifted keyword mixed alphabet becomes EGHJKLPQUVXYZMICROSFTWNDAB.

The encipherment process requires each character in the message to be located in the array PLAINTEXT$, which contains the normal alphabetic sequence. When a match is found, the ciphertext character located in the same position in the shifted keyword mixed alphabet in the CIPHERTEXT$ array is extracted and substituted for the plaintext character. Note the positional relationship of characters between the plaintext based alphabet and the shifted keyword mixed alphabet in Figure 3.2. The first character in the message is B, located in the second position in the plaintext alphabet. The second character in the shifted keyword mixed alphabet is G. Hence, B was replaced by G. The second letter in the message is I, located in the ninth position in the plaintext alphabet. In the ninth position in the shifted keyword mixed alphabet, extract a U. Similarly, D is replaced by the character J and the first word in the plaintext message, BID, is enciphered as GUJ.

The CIPHER5.CPP Program

The C++ version of the previously described CIPHER5.BAS program is appropriately named CIPHER5.CPP and is contained on the CD-ROM under the C directory. The executable version of this C++ program follows our naming convention and is named CIPHER5.EXE. To distinguish the BASIC executable version of CIPHER5 from the C++ version of that program, it is important to remember that those versions are located in the BASIC and C directories on the CD-ROM.

Listing 3.4 lists the contents of the program CIPHER5.CPP. As you examine the program listing you will note that the C++ software author, Jonathan Held, replaced the #define statements with the use of integer constants to demonstrate another method you can use to create constants for a C++ program. Similar to other C++ program listings, the listing of CIPHER5.CPP includes a detailed description via embedded comments that indicates how the program operates. The only function that is completely new in Listing 3.4 is the welcome function whose statements are listed at the end of the program. Although the other functions in CIPHER5.CPP were developed for previously covered C++ programs, the entire program is given in Listing 3.4 to provide you with the manner by which functions interact with one another as well as to provide an example of the use of integer constants instead of magic numbers.

Listing 3.4 The CIPHER5.CPP program listing.

/*
cipher5.cpp
C++ Code written by Jonathan Held, February 28, 1998, using
Microsoft Visual C++, version 5.0.  Please read file for
detailed description of how the program works.
*/

//standard include files
#include <iostream.h>      //standard i/o operations
#include <string.h>        //used to find the length of a string
#include <ctype.h>         //for character handling
#include <fstream.h>       //file stream processing
#include <stdlib.h>        //standard library functions
#include <string.h>        //string handling functions

//Constants - we don't want to use any magic numbers!
//Notice how now we aren't using #defines as we did before -
//Just another way to create constants for our program.
const int TWO=2, THREE = 3, FOUR = 4, FIVE = 5, TWENTYFIVE = 25,
      TWENTYSIX = 26, TWENTYSEVEN = 27, SIXTYFIVE = 65,
      NINETY = 90, NINETYTWO = 92, SIZE = 256, BIGSIZE = 1000;

//function prototypes - see function headers for more information
void welcome(void);              //only function that is
                                 //new to this file

void getFileNames(char* &, char* &);   //all other functions have
int getInputType(void);                //been used in previous programs

//keyword or keyword phrase functions:
void getKeyword(char * &);
bool checkInput(char * &);
void createAlphabet(char *, char []);
bool getMessage(char*, char*, char [], const char[], const char[]);

//monoalphabetic substitution functions:
void getShiftKey(char &);
void shiftKeyword(char [], char); //shiftKeyword modified for this
                                  //program - see function for details

//other functions:
char* formCipheredMessage(const char[], const char [], char []);
void printResults(const char[], const char[], const char[], const char [],
                    const int);
void printCipherToFile(ofstream, char[]);
void formatData(char []);
void groupBUFFER(ofstream, int);
bool encryptText(char *, char *, const char [], const char[], char []);
void display(char *);

char BUFFER[BIGSIZE] = {'\0'};

//------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//------------------------------------------------
int main(){

  //initialize plaintext
  char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H','I',
                      'J','K','L','M','N','O','P','Q','R',
                      'S','T','U', 'V','W','X','Y','Z', '\0'};

  //other variables we will use
  char ciphertext[TWENTYSEVEN]= {'\0'};
  char message_to_cipher[SIZE], key;
  char *infile, *outfile, *keyword;
  int input_type;
  bool success = false;

  welcome();
  getKeyword(keyword);
  createAlphabet(keyword, ciphertext);
  cout << "\nPlaintext-based alphabet is: " << plaintext << endl
     << "Keyword-based alphabet is:   " << ciphertext << endl << endl;
  getShiftKey(key);
  shiftKeyword(ciphertext, key);
  cout << "Shifted keyword mixed alphabet is: " << endl << ciphertext
                  << endl << endl;
  getFileNames(infile, outfile);
  input_type = getInputType();
  //check for manual entry of data
  if (!(input_type)){
     success = getMessage(infile, outfile, message_to_cipher, plaintext,
                  ciphertext);
  }
  //user wants to encipher a file
  else {
    success = encryptText(infile, outfile, plaintext, ciphertext,
                  message_to_cipher);
  }

  //display the output file only if we were successful
  if (success){
     cout << "Press return to display resulting enciphered message."
                   << endl;
     //get the newlines off the current input stream
     cin.get();
     cin.get();
     display(outfile);
  }
  else {
    cerr << "Error executing program!" << endl;
  }

  //delete all dynamically allocated memory
  delete [] keyword;
  delete [] infile;
  delete [] outfile;

  return (static_cast<int>(!(success)));

}//end main()

//------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//        false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//------------------------------------------------
bool checkInput(char * &input)
{
  bool error = false;
  int count = strlen(input);

  for (int ix=0; ix<count; ix++){

    int char_value = static_cast<int>(*(input+ix));

    //determine if the user did not enter an uppercase character
    if ((char_value < SIXTYFIVE) || (char_value > NINETY)){
       error = true;
       cerr << "You entered an invalid keyword!" << endl << endl;
       break;
     }
  }

  return error;
}//end checkInput()

//-------------------------------------------------
//Function: createAlphabet()
//Parameters: input - the keyword the user entered
//      cipher - the keyword alphabet that will be constructed
//Return Type: None
//Purpose: Creates the keyword alphabet.
//-------------------------------------------------
void createAlphabet(char *input, char cipher[])
{
  bool used[TWENTYSIX];
  int index = 0,
      count = strlen(input);

  //no characters are initially used
  for (int ix=0; ix<TWENTYSIX; ix++){
    used[ix] = false;
  }

  //keep track of each character used, start forming the keyword
  //alphabet
  for (int jx=0; jx<count; jx++){

    //get each character of the input string (integer value)
    int char_value = static_cast<int>(*(input+jx));
    if (used[char_value-SIXTYFIVE]){
       //do nothing - the character was already used
    }
    else {
      //mark as used and add to the keyword alphabet
      used[char_value-SIXTYFIVE] = true;
      *(cipher+index++) = static_cast<char>(char_value);
    }
  }

  //go through the list of characters used - those which weren't
  //used should be added to the keyword alphabet
  for (int kx=0; kx<TWENTYSIX; kx++){

    if (!(used[kx])){
       *(cipher+index++) = static_cast<char>(SIXTYFIVE+kx);
    }
  }

  return;
}//end createAlphabet()

//------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echos the resulting output file to the screen.
//------------------------------------------------
void display(char *name)
{
  ifstream infile(name, ios::in);
  char input[SIZE];

  if (!(infile)){
     cerr << "Unable to open input file for display." << endl;
  }
  else {
    while (infile.getline(input, SIZE, '\n')){
     cout << input << endl;
    }
  }
  return;
}//end display()

//------------------------------------------------
//Function: encryptText()
//Parameters: inp_file - the name of the input plaintext file
//     outp_file - the name of the output ciphertext file
//     PTEXT[] - the plaintext alphabet
//     CTEXT[] - the ciphertext alphabet
//     encoded_msg[] - the message to be encoded
//Return Type: bool, indicating success of operation
//Purpose: Used to encrypt file input.  Takes each line of the input
//file, encrypts it, and saves the result to the specified output
//file.
//------------------------------------------------
bool encryptText(char * inp_file, char * outp_file, const char PTEXT[],
          const char CTEXT[], char encoded_msg[])
{
  bool success = false;
  char ip[SIZE];

  //declare file stream objects
  ifstream input(inp_file, ios::in);
  ofstream output(outp_file, ios::app);

  if ((!input) || (!output)){
    //do nothing - I/O error; user will be notified upon
    //procedure's return to main()
  }
  else {

    success = true;

    //print plaintext and ciphertext alphabets to the
    //output file
    output << "PLAINTEXT:  " << PTEXT << endl;
    output << "CIPHERTEXT: " << CTEXT << endl << endl;

    while (input.getline(ip, BIGSIZE, '\n')){

     //check to see if the user wants the line to appear in plain text
     if (ip[0] == '/'){
        if (strlen(BUFFER)>0){

        //empty whatever is in the buffer
        groupBUFFER(output, strlen(BUFFER));
        //adjust the buffer
        strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
        //output plaintext
      }

      output << ip << endl;

      }
      else {
        //encipher the line
        char *msg = formCipheredMessage(CTEXT, ip, encoded_msg);
        //print the cipher in groups of five to the ouput file
        printCipherToFile(output, msg);
      }
    }

    //empty the rest of the buffer
    groupBUFFER(output, strlen(BUFFER));

    //notify user where plaintext and ciphertext files are
    cout << "Plaintext file is: " << inp_file << endl;
    cout << "Encrypted file is: " << outp_file << endl << endl;
  }

  //don't forget to close the files
  input.close();
  output.close();

  //return success of the operation
  return success;
}//end encryptText()

//------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//------------------------------------------------
void formatData(char data[]){

  for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
    if (*(data+nx) == ' '){
       //do nothing - skip over the space in the data
    }
    else {
      *(data+mx++) = *(data+nx);
    }
  }

  //don't forget to add the null terminator
  *(data+mx) = '\0';

  return;
}//end formatData()

//------------------------------------------------
//Function: formCipheredMessage()
//Parameters:  CTEXT - the cipher alphabet we will use for substitution
//       MESSAGETOCIPHER - the user's message
//       enc_message - the enciphered message to be determined
//Return Type: char* - a pointer to the encoded information.
//Purpose: Encipher the user's message.
//------------------------------------------------
char* formCipheredMessage(const char CTEXT[], const char MESSAGETOCIPHER[],
               char enc_message[]){

  int length = strlen(MESSAGETOCIPHER)+1;

  int encode_value;

  for (int ix=0; ix<length; ix++){

    //test to see if we have an alphabetic character; if not,
    //simply copy it to our encrypted message - this preserves
    //characters such as ', ! etc...
    if (!isalpha(static_cast<int>(MESSAGETOCIPHER[ix]))){
      enc_message[ix] = MESSAGETOCIPHER[ix];
    }
    else {
       //valid character - the easy way to calculate the ciphered
       //character is based on the plain text's ascii character value;
       //since it has to be a capital letter, it must be in the range
       //from 65 to 90, with A represented by 65, Z by 90.  By simply
       //subtracting 65 from the encode_value (the integer representation
       //of the plaintext character), we now know what cipher character
           //to use.

       encode_value = toupper(static_cast<int>(MESSAGETOCIPHER[ix]));
       enc_message[ix] = CTEXT[encode_value-SIXTYFIVE];
     }
  }

  //return a reference to the encoded message
  return enc_message;
}//end formCipheredMessage()

//------------------------------------------------
//Function: getFileNames()
//Parameters: infile_name - the input file
//       outfile_name - the output file we will write the
//       enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
  char data[SIZE];

  cout << "Enter filename to store/retrieve plaintext message: ";

  cin >> data;

  infile_name = new char[strlen(data) + 1];
  strcpy(infile_name, data);

  cout << "Enter filename to store enciphered message: ";

  cin >> data;

  outfile_name = new char[strlen(data) + 1];
  strcpy(outfile_name, data);

  cout << endl;

  return;
}//end getFileNames()

//------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//------------------------------------------------
int getInputType(void)
{
  char type;
  bool error = false;
  int value;

  do {
    //prompt user for input from file or keyboard
    cout << "Is file input from keyboard (K, k) or file (F, f): ";
    cin >> type;

    //make type an uppercase letter
    type = static_cast<char>(toupper(static_cast<int>(type)));

    //check for an invalid character
    if((type != 'K') && (type != 'F')){
      cerr << "You have entered an invalid character!" << endl << endl;
      error = true;
    }
    else {
      if (type == 'K')
         value = 0;       //value of 0 represents keyboard input
         else value = 1;    //value of 1 represents file input
      error = false;
    }

  } while (error);

  cout << endl;

  return value;

}//end getInputType()

//------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: None
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.
//------------------------------------------------
void getKeyword(char * &text)
{
  bool error = false;

  do {
    char buffer[SIZE];
    cout << "Enter keyword or keyword phrase in UPPERCASE" << endl
      << "do not use spaces or non-alphabetic characters): ";

    cin.getline(buffer, SIZE, '\n');
    text = new char[strlen(buffer) + 1];
    strcpy(text, buffer);

    error = checkInput(text);

  } while (error);

  return;
}//end getKeyword()

//------------------------------------------------
//Function: getMessage()
//Parameters:  input - the name of the input plaintext file
//       output the name of the output ciphertext file
//       msg_to_cipher - the message to be encoded
//       PTEXT[] - the plaintext alphabet
//       CTEXT[] - the ciphertext alphabet
//Return Type: bool, indicating success of operation
//Purpose: Allow the user to manually input text from the keyboard.
//Save the text in plaintext to the input file; encrypt the text
//and save it to the specified output file for later retrieval.
//------------------------------------------------
bool getMessage(char* input, char* output, char msg_to_cipher[],
                  const char PTEXT[], const
char CTEXT[]){

  bool go_on = true, success = false;

  ofstream textFile(input, ios::app);
  ofstream cipherFile(output, ios::app);

  if((!textFile) || (!cipherFile)){
    //do nothing - error will be noted to user later
  }
  else {

    success = true;

    textFile << "PLAINTEXT:  " << PTEXT << endl;
    textFile << "CIPHERTEXT: " << CTEXT << endl << endl;

    //get the newline character off of the input stream
    cin.get();

    cout << "Enter the message in UPPERCASE or lowercase characters. "
                  << endl
         << "Non-alphabetic characters may be entered but are ignored."
                  << endl
         << "Use a / at the beginning of each line that should remain"
                  << endl
         << "in plaintext and a \\ on a separate line to indicate the"
                  << endl
         << "end of an enciphered message." << endl << endl;

    while (go_on) {

     //get the entire line, up to 256 characters
     cin.getline(msg_to_cipher, SIZE, '\n');

     //case user doesn't want the text to be encrypted
     if(msg_to_cipher[0] == '/'){

       if(strlen(BUFFER)>0){
         //empty whatever is in the buffer
         groupBUFFER(cipherFile, strlen(BUFFER));
         //adjust the buffer
         strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
       }

       //output plaintext
       textFile << msg_to_cipher << endl;
       cipherFile << msg_to_cipher << endl;
    }

     //case user is done entering text
     else if (static_cast<int>(msg_to_cipher[0]) == NINETYTWO){
       go_on = false;
    }

     //encrypt the text
     else {
       textFile << msg_to_cipher << endl;
       char enciphered_msg[BIGSIZE];
       formCipheredMessage(CTEXT,msg_to_cipher,enciphered_msg);
       printCipherToFile(cipherFile,enciphered_msg);
    }
    }

  //empty the rest of the buffer
  groupBUFFER(cipherFile, strlen(BUFFER));
  }

  //close the files
  textFile.close();
  cipherFile.close();

  //notify user where plaintext and ciphertext files are
  cout << "\nPlaintext file is: " << input << endl;
  cout << "Encrypted file is: " << output << endl << endl;

  return success;
}//end getMessage()

//------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//------------------------------------------------
void getShiftKey(char &key_desired){

  bool error = true;

  do {
    //prompt user to enter an uppercase shift key
    cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
    cin >> key_desired;

    int key_value = static_cast<int>(key_desired);

    //do some error checking
    if((key_value < SIXTYFIVE) || (key_value > NINETY)){
      cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
    }
    else {
      error = false;
    }
  } while (error);

  cout << endl;

  return;
}//end getShiftKey()

//------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
//      num - the number of characters we want to output
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//------------------------------------------------
void groupBUFFER(ofstream out, int num)
{

  for (int kx=0;kx<num;kx++){

    if((kx!=0) && (kx%TWENTYFIVE==0)){
      out << endl;
    }

    if((kx!=0) && (kx%FIVE == 0) && (kx%TWENTYFIVE!=0)){
      out << " " << *(BUFFER+kx);
    }
    else {
    out << *(BUFFER+kx);
    }
  }

  out << endl;

  return;
}//end groupBUFFER()

//---------------------------------------------
//Function: printCipherToFile()
//Parameters: op - the output file we are writing to
//      msg - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-block characters in the
//specified output file.
//---------------------------------------------
void printCipherToFile(ofstream op, char msg[]){

  formatData(msg);

  //check to see if there are more than 25 characters
  //in the buffer; if so, print out as many groups of
  //25 as possible
  if(strlen(BUFFER) >= TWENTYFIVE){

    int numchars = (strlen(BUFFER)/TWENTYFIVE)*TWENTYFIVE;
    //print the contents of the buffer to the output stream
    groupBUFFER(op, numchars);
    //shift whatever is left in the buffer
    strcpy(BUFFER, (BUFFER+numchars));
    //append data to the buffer
    strcat(BUFFER, msg);
  }

  //if buffer contents are less than 25, simply append the new
  //data to the buffer
  else if ((strlen(BUFFER) >= 0) && (strlen(BUFFER) < TWENTYFIVE)){
    strcat(BUFFER, msg);
  }

  return;
}//end printCipherToFile()

//------------------------------------------------
//Function: shiftKeyword()
//Parameters: ctext - the cipher alphabet we are going to shift
//      user_key - the key the user entered
//Return Type: None
//Purpose: Shift the keyword or keyword phrase we will use later to
//encode the user's message.
//------------------------------------------------
void shiftKeyword(char ctext[], const char USER_KEY){

  int location;
  char temp[TWENTYSIX] = {'\0'};

  //find the location of the key in the plaintext
  for (int ix=0; ix<TWENTYSIX; ix++){
    if(USER_KEY == ctext[ix]){
      location = ix;
      break;
     }
  }

  if(location == TWENTYFIVE){
     //do nothing
  }
  else {

    //put into temp all the characters up to and including the shift key
    //location now indicated how many characters, so we must increment by
    //one since the array uses a zero-based index
    strncpy(temp, ctext, location+1);
    //shift all remaining characters in the ciphertext
    strcpy(ctext, (ctext+location+1));
    //concatenate temp back to ctext
    strcat(ctext, temp);
  }
  return;
}//end shiftKeyword();

//------------------------------------------------
//Function: welcome()
//Parameters: None
//Return Type: None
//Purpose: Prints the program's greeting message.
//------------------------------------------------
void welcome(void)
{
  cout << "CIPHER5.EXE PROGRAM enciphers text based upon the use of a"
                 << endl
       << "keyword or keyword phrase and an alphabetic shift key using a"
                 << endl
       << "monoalphabetic substitution process." << endl << endl;
  return;
}//end welcome()
//end file cipher5.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Deciphering Operations

After developing CIPHER5.BAS and CIPHER5.CPP, it's only logical that a mechanism is required to restore a received message to its original plaintext content. In this section, several previously developed subroutines are modified and incorporated into programs appropriately named DCIPHER5.BAS and DCIPHER5.CPP as they perform the inverse function of CIPHER5.BAS and CIPHER5.CPP. Although you could incorporate deciphering operations into CIPHER5.BAS and CIPHER5.CPP, it is easier to explain deciphering coding operations as a separate entity. In addition, some organizations may prefer to limit non-headquarters or certain corporate locations' capability to deciphering messages and preclude those locations from originating enciphered messages. Thus, the separation of enciphering and deciphering operations may be preferred by some users. For other users who prefer a program that combines both operations, I'll develop such programs using more sophisticated enciphering techniques later in this book. However, because the focus is on the development of modular routines, I'll show you the ease in which modules from both programs can be combined.

Prior to developing the coding required to decipher a message created through the use of a keyword or keyword phrase and an alphabetic shift, it may be useful to review the manual process required to decipher a message enciphered in this manner.

The Decipherment Process

To correctly decipher an enciphered message developed using a keyword or keyword phrase and shift key requires knowledge of both elements. Once both elements are known, you can construct a keyword-based mixed alphabet and shift the alphabet to obtain the cipher alphabet. To initiate the deciphering process, place the cipher alphabet above the plaintext alphabet. For each letter in the enciphered message, first locate its position in the cipher alphabet and extract the plaintext character at the position in the alphabet which equals the position of the enciphered character in the cipher alphabet. For example, assume the keyword phrase was GODSAVETHEQUEEN and that the shift key is the character B. In this case the keyword-based alphabet becomes GODSAVETHQUNBCFIJKLMPRWXYZ. Because the shift key is the character B, shift the keyword-based alphabet until it is positioned so that the character B is on the extreme right. Remember that the alphabetic shift key was previously defined as an uppercase character in the plaintext alphabet that defines the location where the right end of the resulting shifted alphabet ends. Thus, using the shift key character B results in the cipher alphabet becoming: CFIJKLMPRWXYZGODSAVETHQUNB.

Now that you have constructed the cipher alphabet based upon the predefined keyword phrase and shift key, place that alphabet above the plaintext alphabet as shown below:

      Cipher alphabet:      CFIJKLMPRWXYZGODSAVETHQUNB
      Plaintext alphabet:   ABCDEFGHIJKLMNOPQRSTUVWXYZ

Now suppose the first two groups of five enciphered characters in a received message are LRAKW OVKDP. Searching the cipher alphabet, you would first locate the character L and read down the same position into the plaintext alphabet to extract the character F. Similarly, locating R in the cipher alphabet results in extracting I from the plaintext, A results in extracting R from the plaintext alphabet, K in the cipher alphabet produces E in the plaintext alphabet, and so on. Thus, LRAKW OVKDP in the enciphered message is equivalent to FIREJOSEPH in plaintext. Now that you have a feeling for the manual process required to decipher an enciphered message, let's construct a program to automate this process.

The DCIPHER5.BAS Program

To automate deciphering of a previously enciphered message based upon the use of a keyword or keyword phrase and an alphabetic shift key, I'll first construct a program appropriately named DCIPHER5.BAS. In doing so, I'll use all of the previously developed subroutines included in CIPHER5.BAS as is, or in a slightly modified form. To indicate that a change was made to a previously developed subroutine, I'll prefix its name with the character D to indicate its modification for deciphering a received message.

Listing 3.5 contains the statements that make up the main portion of DCIPHER5.BAS. Similar to the statements in CIPHER5.BAS, a comment of five asterisks is used to denote a PRINT statement included for illustrative purposes and whose elimination will not affect the operation of the program.

Listing 3.5 The main portion of the DCIPHER5.BAS program.

REM PROGRAM DCIPHER5.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26), KEY$(26)
CLS
GOSUB INITIALIZE
    PRINT "DCIPHER5.BAS PROGRAM deciphers text based upon the use
                of a keyword or keyword"
    PRINT "phrase and an alphabetic shift key using a monoalphabetic
                 substitution process."
    PRINT
    INPUT "Enter keyword or keyword phrase in UPPERCASE: ", TEXT$
    PRINT "Plaintext based alphabet is : ";    '*****
    FOR I = 0 TO 25: PRINT PLAINTEXT$(I);: NEXT I   '*****
    GOSUB KEYWORD             'form keyword-based mixed alphabet
    PRINT "Keyword-based alphabet is     : "; X$  '*****
1   INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
    FOR I = 0 TO 25
    IF K$ = PLAINTEXT$(I) GOTO 2
    NEXT I
    PRINT "You must enter a letter from A to Z"
    GOTO 1
2   REM Position I represents shift key letter

GOSUB FORMCIPHER         'create cipher alphabet
    PRINT "Shifted keyword mixed alphabet is   : ";   '*****
    FOR I = 0 TO 25: PRINT CIPHERTEXT$(I);: NEXT I: PRINT  '*****
GOSUB INITIALIZE         'reinitialize plaintext array
GOSUB DMSGFILE           'assign I/O files, place message on a file
GOSUB DCONVERTSTORE      'convert and store plaintext on a file
GOSUB PRTOUT             'print results
STOP

It should be noted that the only difference between the main portion of CIPHER5.BAS and DCIPHER5.BAS is the subroutine names DMSGFILE and DCONVERTSTORE. Since you noted that the character D would be used as a prefix to indicate a change to a previously developed subroutine for decipherment operations, let us focus our attention on those two subroutines.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DMSGFILE Subroutine

The process of assigning I/O files must be reversed for decipherment. That is, the filename used for the input of the plaintext message in the CIPHER5.BAS program becomes the output file. Similarly, the filename used for the storage of the enciphered message in the CIPHER5.BAS program becomes the input file for decipherment operations. To provide those modifications, you must change the use of the string variables OUTFILE$ and INFILE$ previously used in the subroutine MSGFILE. To change I/O file assignments, simply reverse the use of the previously mentioned string variables.

Listing 3.6 lists the statements in the new subroutine named DMSGFILE, which contains the reversal of I/O file assignments. If you compare this subroutine to the previously developed subroutine MSGFILE, you should note that the difference between the two is limited to the previously mentioned reversal of I/O filename assignments.

Listing 3.6 The DMSGFILE subroutine.

DMSGFILE:
   REM Routine to assign I/O files and accept keyboard or file input
   REM and remove spaces between words
       INPUT "Enter filename to store plaintext message,
                         default=MESSAGE.DAT", OUTFILE$
       IF OUTFILE$ = "" THEN OUTFILE$ = "MESSAGE.DAT"
       INPUT "Enter filename for enciphered message,
                         default=CIPHERTX.DAT", INFILE$
       IF INFILE$ = "" THEN INFILE$ = "CIPHERTX.DAT"
       INPUT "Select keyboard (k) or file (f) message input: ",
                         IN$
       IF IN$ = "F" OR IN$ = "f" THEN RETURN
       OPEN INFILE$ FOR OUTPUT AS #1
   REM Routine to place message on a file removing spaces between
                         words
       PRINT "Enter your message - place a / at the beginning of
                         each line"
       PRINT "that should remain in plaintext and a \ on a
                         separate line"
       PRINT "to indicate the end of the enciphered message"
       PRINT
AGN:     LINE INPUT TEXT$
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO XT
       NTEXT$ = ""
       FOR I = 1 TO LEN(TEXT$)
       NTEXT$ = NTEXT$ + LTRIM$(MID$(TEXT$, I, 1))
       NEXT I
       WRITE #1, NTEXT$
       IF MID$(TEXT$, 1, 1) = "\" GOTO DONE
       GOTO AGN
XT:     WRITE #1, TEXT$
       GOTO AGN
DONE:     CLOSE #1
RETURN

The DCONVERTSTORE Subroutine

The second subroutine that required modification for deciphering is CONVERTSTORE. As previously discussed, the modified subroutine was renamed DCONVERTSTORE to reflect its modification for decipherment operations.

Lisitng 3.7 gives the statements contained in the subroutine DCONVERTSTORE. Other than changing two REM statements to reflect the fact that the routine now converts ciphertext to plaintext, the only additional changes relate to search and replacement operations. The IF statement in the nested FOR-NEXT loop now compares each character in the string variable TEXT$ to each character in the CIPHERTEXT$ array instead of the PLAINTEXT$ array. When a match occurs, the character in TEXT$ is replaced by a character in the array PLAINTEXT$ located at the same position in the array where the character in TEXT$ matched the character in the array CIPHERTEXT$. Thus, DCONVERTSTORE simply makes use of the plaintext alphabet stored in the array PLAINTEXT$ and the ciphertext alphabet stored in the array CIPHERTEXT$ in the reverse manner in which they were used in the subroutine CONVERTSTORE.

Listing 3.7 THE DCONVERTSTORE subroutine.

DCONVERTSTORE:
   REM Routine to convert and store plaintext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM  Convert ciphertext to plaintext
            FOR I = 1 TO MSGLEN
            FOR J = 0 TO 25
            IF MID$(TEXT$, I, 1) = CIPHERTEXT$(J) THEN GOTO GOTIT
            NEXT J
GOTIT:        MID$(TEXT$, I, 1) = PLAINTEXT$(J)
            NEXT I
CLEARTXT:       WRITE #2, TEXT$
   LOOP
DONE1:     CLOSE #2
RETURN

Program Operation

To illustrate the operation of the DCIPHER5.BAS program, you can execute it using the previously enciphered message to the fictional JOHN P. BIDDER as input. Regardless of whether you use the keyboard or specify a file for input of the ciphered messaged, the program produces the same result. Figure 3.3 illustrates the execution of DCIPHER5.BAS in which the keyboard is used to enter the received enciphered message. Because the previously developed subroutine PRTOUT was used to display the resulting deciphered message, that message is displayed in groups of five characters as indicated in the lower portion of Figure 3.3. Although the program converted the ciphertext message back into its plaintext character representation, it did not restore the message to its original format with spaces between words and words rebuilt from two or more five-character groupings when it crossed such groupings. Those functions cannot be performed when the encipherment method does not substitute a character for the space character and results in the human recipient of the deciphered message being responsible for understanding how to correctly read the deciphered message.

Figure 3.3 The execution of DCIPHERS.BAS


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DCIPHER5.CPP Program

Once again a C++ version of a previously constructed BASIC language program is presented for readers who prefer this programming language. Maintaining our naming conventions, the C++ version of DCIPHER5.BAS is appropriately named DCIPHER5.CPP, while the executable version of the program is named DCIPHER5.EXE. To distinguish the executable version of DCIPHER5.BAS from DCIPHER5.CPP, remember that there are two directories on the CD-ROM accompanying this book and that source and executable BASIC programs are stored in the BASIC directory while C++ source and executable programs are stored in the C directory.

Listing 3.8 contains the complete listing of DCIPHER5.CPP. As you read through the code you will note that, similar to the BASIC program, large segments of code were reused. Of particular interest is the function appropriately named decipher, which locates where the encrypted data is in the ciphertext.

Listing 3.8 The DCIPHER5.CPP program listing.

/*
dcipher5.cpp
C++ code written by Jonathan Held
using Microsoft Visual C++, version 5.0, on March 12, 1998.
*/

//standard include files
#include<iostream.h>
#include<fstream.h>
#include<stdlib.h>
#include<string.h>
#include<ctype.h>

//function prototypes
bool checkInput(char * &);
void createAlphabet(char *, char[]);
void decipher(ifstream, ofstream, const char[], char[]);
void formatData(char []);
void getFileNames(char *&, char *&);
void getFileToDecipher(const char[], char[]);
bool getMessage(char* input, char* output, char msg_to_decipher[],
                  const char PTEXT[], const char CTEXT[]);
int getInputType();
void getKeyword(char *&);
void getShiftKey(char &);
void shiftKeyword(char [], char);

//constants we will use
const int FIVE = 5, TWENTYFIVE = 25, TWENTYSIX = 26,
    TWENTYSEVEN = 27, SIXTYFIVE = 65, NINETY = 90,
    NINETYTWO = 92, SIZE = 256;

//------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//------------------------------------------------
int main()
{
char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',

Alternative Relationships

Up to this point the focus has been on using the development of a keyword-based mixed alphabet to create a ciphertext alphabet. Once that was accomplished, an alphabetic shift key was used to further mix the relationship between plaintext and ciphertext alphabets. In performing those tasks, the only information a message recipient requires is the keyword or keyword phrase and the alphabetic shift key to decipher a message through the use of a program. With a little knowledge of the method used to create a keyword-based mixed alphabet and the use of a shift key, most people can learn how to manually decipher a message. The major benefit of the previously described encipherment method is the ability of most persons to manually perform encipherment and decipherment with a minimum of information.

You can extend the use of keyword or keyword phrase mixed alphabets and alphabetic shifting to the plaintext alphabet. In fact, you can develop nine common alphabetic relationships that are listed in Table 3.1. You could also reverse the resulting plaintext, ciphertext, or both plaintext and ciphertext alphabets to further increase the number of alphabetic relationships. However, this may add a degree of confusion to both encipherment and decipherment due to the large number of options users could be forced to consider and remember. To keep your techniques practical, I'll simply illustrate the use of two keyword-based mixed sequence alphabets for enciphering a message and leave it to you to modify CIPHER5.BAS to obtain other plaintext-ciphertext relationships.

Table 3.1 Common alphabetic relationships
plaintext alphabet in normal sequence | ciphertext alphabet in normal sequence and shifted
plaintext alphabet in normal sequence | ciphertext alphabet in keyword mixed sequence
plaintext alphabet in normal sequence | ciphertext alphabet in keyword mixed sequence and shifted
plaintext alphabet in keyword mixed sequence | ciphertext alphabet in normal sequence and shifted
plaintext alphabet in keyword mixed sequence | ciphertext alphabet in keyword mixed sequence
plaintext alphabet in keyword mixed sequence | ciphertext alphabet in keyword mixed sequence and shifted
plaintext alphabet in keyword mixed sequence and shifted | ciphertext alphabet in normal sequence and shifted
plaintext alphabet in keyword mixed sequence and shifted | ciphertext alphabet in keyword mixed sequence
plaintext alphabet in keyword mixed sequence and shifted | ciphertext alphabet in keyword mixed sequence and shifted


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

To illustrate the use of two keyword-based mixed alphabets, assume the keyword SANDIEGO is used to form the plaintext alphabet and the keyword BALTIMORE is used to form the ciphertext alphabet. Figure 3.4 illustrates the resulting mixed plaintext and ciphertext alphabets.

Figure 3.4 Dual mixed alphabet relationship.

Suppose you wish to encode BID. To do so, you would perform the same operation as previously discussed. That is, you would first locate B in the mixed plaintext alphabet and note its position in that alphabet. Next, you would extract the character in the ciphertext alphabet that corresponds to the location of B in the plaintext alphabet, resulting in B being replaced by E. Next, the letter I would be located in the mixed plaintext alphabet; the letter I would be extracted from the ciphertext alphabet. Similarly, D would be replaced by T.

Sometimes luck works both ways. During World War I the British War Office developed a cipher device which they planned to distribute to British forces as a field cipher. So highly regarded was the device that one argument against its adoption for field use was its possible capture and use by the Germans—a situation many high-level personnel in the Allied forces thought would preclude the ability of the Allies to decipher enemy messages. To verify the security afforded by their device, the British submitted five short enciphered messages to the only formal cryptologic organization in the United States.

Located on a 500-acre estate named Riverbank at Geneva, IL, the Department of Ciphers at Riverbank was headed by William Frederick Friedman, considered by some as America's greatest cryptologist. Although Friedman was able to determine the keyword of one of the mixed alphabets, apparently he hit the proverbial "blank wall" in attempting to determine the other keyword. Turning to his wife, he asked her to make her mind a blank and tell him the first word that came to mind when he said a word. When he said the first keyword, "CIPHER," she replied, "MACHINE." The intuitive guess of his wife was correct, and Friedman cabled the plaintext solution of the five messages to London, dooming the adoption of the device many persons were afraid would, if captured by the Germans, preclude the Allies from solving enemy messages.

Weaknesses

Note that although two different keywords were used, many characters in the ciphertext alphabet are the same as their plaintext equivalents, especially towards the end of each alphabet. This is one of several weaknesses of mixed alphabets, as the encipherment of a long message may provide numerous hints concerning the contents of a message when ciphertext and plaintext characters coincide. It is more practical to use one keyword-based alphabet and an appropriate alphabetic shift key to eliminate the overlapping of plaintext and ciphertext characters than to simply use two keyword mixed alphabets.

The second major weakness of all keyword-based monoalphabetic substitution techniques is the fact that a continuous one-for-one character replacement forms the basis of this category of enciphering techniques. This means that the techniques are very vulnerable to decipherment by a person performing a frequency analysis of characters in intercepted enciphered messages. For example, as Samuel Morse noted in developing the assignments of dots and dashes to characters in the English language in forming the code that bears his name, the letter E is the most frequently occurring character, followed by T. Thus, Morse assigned a dot to denote E and a dash to denote T prior to assigning sequences of dots and dashes to denote other characters in his code.

Suppose, for example, that you intercepted a message enciphered using the plaintext-ciphertext relationship illustrated in Figure 3.4. Further assume that you performed a frequency analysis of the characters in the enciphered message and discovered that the letter M occurred most frequently, followed by the character S. In this case, your first attempt to reconstruct the plaintext-ciphertext alphabet relationship in order to decipher the message would result in placing the character E in the plaintext alphabet over the character M in the ciphertext alphabet, followed by placing the character T in the plaintext alphabet over the character S in the ciphertext alphabet. Although a frequency analysis of characters may not provide an exact relationship between plaintext and ciphertext characters, the analysis normally provides enough intelligent clues to conduct a successful decipherment if you gain access to one long message or a few short messages enciphered using the same alphabetic relationship.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 4
Transposition-based Monoalphabetic Substitution

This chapter focuses on three commonly used methods to develop a mixed alphabet. The first two methods are related to the use of arrays, or matrices, while the third method can be considered a position and extraction technique. After you obtain an understanding of the manual process required for the formation of a transposition mixed alphabet and the encipherment of a message using that alphabet, I'll turn your attention to automating the process. I'll show you how to develop a series of subroutines you can use to perform different types of transpositions and then incorporate those subroutines into a program. When possible, previously created subroutines will be used to take advantage of your previous efforts.

Matrix-Based Transposition

By placing an alphabet into an NxN matrix, or array, you can obtain the ability to extract characters in many different ways. For example, consider the sequential plaintext alphabet placed into a 7x4 matrix as illustrated in Figure 4.1. Each character in the alphabet is referenced by a row and column position and can be considered an element in the array or matrix. By extracting matrix elements in a predefined sequence, you can develop a transposed alphabet.

Figure 4.1 Placing the plaintext alphabet into a matrix. Each character in the matrix is identified by its row and column position.

Perhaps the earliest use of a matrix with a cryptographic alphabet can be traced to the Greek writer Polybius. He arranged the Greek alphabet into a square and numbered the rows and columns whose values identified the location of each letter. This enabled each letter to be identified by a pair of numbers that represented the intersection of a row and column in the matrix.

In addition to devising the use of a matrix for cryptographic applications, Polybius suggested the use of his matrix as a mechanism to transmit information by torches. Polybius proposed that letters in the alphabet could be transmitted over relatively long distances by persons holding a different number of torches. For example, a person holding one torch at his right and two to his left would signal the letter located at the intersection of the first row and second column in the matrix.

The formation of an alphabet into a matrix was rediscovered by modern cryptographers and forms the basis for developing a large number of encipherment systems. Some systems convert letters into numbers, while other systems use the matrix to form different types of alphabets based upon the extraction of row and column entries using a predefined algorithm.

If you extract the matrix elements in each column commencing with column seven and then work backwards, extracting the elements in column six, column five, and so on, the transposed alphabet GNUFMTELSZDKRYCJQXBIPWAHOV is created. If we extracted the matrix elements in a progressive column order, we would obtain the transposed alphabet AHOVBIPWCJQXDKRYELSZFMTGNU.

Unfortunately, the placement of a plaintext alphabet into a matrix results in the development of a transposed alphabet in which several sequences of letters are displaced by fixed positions, making the job of a cryptanalyst easier if he or she attempts to reconstruct the transposed alphabet used to encipher messages. To add a degree of difficulty to any reconstruction attempt, these alphabets are normally developed through the use of a keyword or keyword phrase.

Figure 4.2 illustrates the construction of a keyword-based alphabet fitted into a 7x4 matrix, using KEYWORD as the keyword. Similar to the keyword-based alphabets discussed in Chapter 3, Keyword-Based Monoalphabetic Substitution, you can use either a keyword or keyword phrase to develop a mixed alphabet. Then, by placing the resulting alphabet into a matrix you can develop a transposed keyword-based mixed alphabet. The construction of this alphabet is easy to duplicate, providing the message recipient with an easy mechanism for preparing to decipher a received message. In addition, using a keyword or keyword phrase adds an additional degree of difficulty to a cryptanalyst attempting to decipher a message that intentionally or unintentionally falls into his or her hands.

Figure 4.2 A keyword-based mixed alphabet placed into a matrix. In this example, the keyword KEYWORD was used to form the keyword-based mixed alphabet.

In examining Figure 4.2, note that the keyword used is seven positions in length and defines the number of columns in the matrix. When developing the keyword-based mixed alphabet, remember to first remove any duplicate letters in the keyword or keyword phrase and then add the remaining letters of the alphabet that are not in the keyword or keyword phrase in the sequence in which they appear in the alphabet. For example, the keyword ALPHA would be reduced to ALPH to eliminate the duplicate A. This would result in the creation of a matrix with four columns.

Simple Transposition

There are several methods that can be used to develop a transposed keyword-based mixed alphabet. Perhaps the most obvious method, referred to as simple transposition, extracts column entries in their column order. The simple transposition of the keyword-based alphabet illustrated in Figure 4.2, in which the keyword is KEYWORD, produces the following keyword-based simple transposed alphabet: KAJTEBLUYCMVWFNXOGPZRHQDIS.

After you obtain the transposed alphabet, you can use it for the plaintext or ciphertext alphabet. In addition, you could create two different transposed alphabets and use one for the plaintext alphabet and the second for the ciphertext alphabet. The examples presented in this chapter restrict the use of transposed alphabets to the ciphertext alphabet.

Encipherment

Similar to previously described manual encipherment processes, you should list the plaintext alphabet above the ciphertext alphabet. For each character in the plaintext message, you would first locate that character in the plaintext alphabet and extract the corresponding character by position in the ciphertext alphabet. Figure 4.3 illustrates the encipherment of the first two characters of a plaintext message using the previously developed keyword-based simple transposed alphabet.

Figure 4.3 Encipherment using a keyword-based simple trasposed alphabet.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Decipherment

The decipherment process can be considered the reverse of the previously illustrated encipherment process. That is, you would rearrange your sheet of paper to place the enciphered message at the top of the page. Next, you would list the ciphertext alphabet above the plaintext alphabet. Then, you would take each character in the enciphered message and locate it in the ciphertext alphabet, extracting the plaintext alphabet character in the equivalent position where the ciphertext character was located in the ciphertext alphabet.

Numeric Transposition

Another popular method used to develop a transposed alphabet through the use of a matrix involves the selection of rows based upon the alphabetic position of the character in the keyword. This technique is known as numeric transposition, because a numeric is normally assigned to each column based upon the location of the first character in the column relative to the characters in the adjusted keyword. The characters in each column are then extracted to form the transposed alphabet based upon the numeric value assigned to each column.

Figure 4.4 illustrates the construction of a numeric transposition-based alphabet using the keyword KEYWORD. Note that a numeric is first assigned to each column based upon the position of the first character in each column in the first row of characters. Simply extract the characters in each column based upon the numeric sequence of each column.

Figure 4.4 Constructing a numeric transposition-based alphabet.

Encipherment and Decipherment

After you create a numeric-based transposition alphabet, you can use that alphabet as the ciphertext alphabet. You can then encipher and decipher messages in the same manner as previously described for the simple transposition alphabet.

Other Variations

Once a keyword- or keyword phrase-based alphabet is placed into a matrix, you can use several additional extraction techniques to form a transposition alphabet. Other variations you can consider include reversed simple and reversed numeric transpositions as well as different diagonally formed transpositions. Because the simple and conventional numeric transpositions are the primary methods used for the formation of transposed alphabets, I'll leave it as an exercise for readers to develop other types of transposed alphabets.

The use of a matrix for enciphering messages provides a mechanism for supplementing or circumventing the use of more secure systems. During World War II, General Leslie Groves who was in charge of the Manhattan Project, which resulted in the development of the atomic bomb, needed a mechanism to discuss matters of high secrecy over the telephone. General Groves developed a series of 10x10 matrices in which each matrix contained both letters of the alphabet as well as code word characters that represented different laboratory or manufacturing sites.

To reduce the possibility of an overheard telephone conversation being analyzed, General Groves took two actions. First, he developed a different matrix for each person he needed to talk to concerning matters of national security. This reduced the potential effect of the loss of a matrix and made it more difficult for a series of overheard conversations to be analyzed. Secondly, within each matrix, General Groves located multiple copies of frequently occurring letters in the English language, such as the letters A, E, I, O, U, and T. The number of repetitions of a character depended upon the frequency of occurrence of the letter in the alphabet. For example, in one of the general's matrices, he included nine E's and seven T's. This technique considerably reduced the possibility and effectiveness of a frequency analysis being used to decipher an overheard conversation.

Interval Extraction

The last technique discussed in this chapter that is used to create a program to automate its operation involves the extraction of characters from the plaintext alphabet based upon a predefined interval. As each letter in the alphabet is extracted, it is removed from the plaintext alphabet and entered into the interval extracted alphabet.

Figure 4.5 illustrates two examples of the formation of interval extracted alphabets based upon using an interval of three positions. In the example at the top portion of Figure 4.5, the interval 3 extraction process is applied against a sequential plaintext alphabet. In the example at the bottom of Figure 4.5, the interval 3 extraction process is applied against a keyword-based mixed alphabet using KEYWORD as the keyword. Note the significant differences between the resulting extracted alphabets even though the extraction interval was the same for each example.

Figure 4.5 Developing interval extracted alphabet.

Encipherment and Decipherment

Once an interval extracted alphabet is formed, it can be used in the same manner as previously described for the simple and numeric-based transposition alphabets. You could use the interval 3 extracted alphabet produced in Figure 4.5 as the ciphertext alphabet for enciphering a message as previously illustrated in Figure 4.3. Similarly, you can use that alphabet and its relationship to the plaintext alphabet to decipher an enciphered message.

Automating Matrix-Based Systems

Encipherment

To facilitate the automation process required to encipher messages based upon transposition-based mixed alphabets, you can create two new subroutines. One subroutine, which I'll label TRANSPORT, can be used to form either a simple or numeric transposition mixed alphabet sequence based upon the user's preference. The second subroutine, which I'll label INTERVAL, can be used to create an extracted alphabet based upon a predefined interval or position. After you create those two subroutines, you can add them to the previously created CIPHER5.BAS program and change the main portion of the program to permit the newly created subroutines to be invoked. The resulting program, which I'll call CIPHER6.BAS, permits users to dynamically enter the name of the files they wish to use to store an input plaintext message as well as the resulting enciphered message.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The TRANSPORT Subroutine

Listing 4.1 shows the contents of the subroutine labeled TRANSPORT, which creates either a simple or numeric transposition mixed sequence alphabet. This subroutine includes several unnecessary PRINT statements to illustrate how the program operates. Similar to the previous use of extraneous PRINT statements, I have identified those statements with comments consisting of five asterisks to facilitate their removal.

Listing 4.1 The TRANSPORT subroutine.

TRANSPORT:
   REM Routine to form simple or numeric transposition mixed sequence
   REM Initialize 26 by 26 matrix to nulls
       POSITION = 0
       FOR I = 0 TO 26
       FOR J = 0 TO 16
       TRANS$(I, J) = ""
       NEXT J, I
   REM Determine actual matrix size
       COLUMN = LEN(KEYCOL$)
       ROW = 26 / COLUMN - INT(26 / COLUMN)
       IF ROW > 0 THEN ROW = (INT(26 / COLUMN) + 1) ELSE ROW = 26
                       / COLUMN
   REM Place keyword-based alphabet into matrix elements
       PRINT "MATRIX IS:": PRINT
       FOR I = 0 TO ROW - 1
       FOR J = 0 TO COLUMN - 1
       IF POSITION > 25 THEN GOTO ALLDONE
       TRANS$(I, J) = PLAINTEXT$(POSITION)
       PRINT TRANS$(I, J);        '*****
       POSITION = POSITION + 1
       NEXT J
       PRINT                      '*****
ALLDONE:   NEXT I
       PRINT: PRINT               '*****
   IF ORDER$ = "NUMERIC" GOTO NUMERIC
   REM Form simple transposition-based mixed alphabet
       X$ = ""
       FOR I = 0 TO COLUMN - 1
       FOR J = 0 TO ROW - 1
       X$ = X$ + LTRIM$(TRANS$(J, I))
       NEXT J
       NEXT I
   REM Place into PLAINTEXT array
       FOR I = 0 TO 25
       PLAINTEXT$(I) = MID$(X$, I + 1, 1)
       NEXT I
   RETURN
   REM Form numeric transposition-based mixed alphabet
NUMERIC:   FOR I = 0 TO COLUMN - 1
       TEMP$(I) = TRANS$(0, I)'place in temporary storage first row
       NEXT I
       FOR I = 0 TO COLUMN - 1'begin sort
       FOR J = I + 1 TO COLUMN - 1
       IF TEMP$(I) > TEMP$(J) THEN SWAP TEMP$(I), TEMP$(J)
       NEXT J
       NEXT I 'TEMP$ array now contains sorted order of characters
   REM Compare sorted keyword to first column in matrix, extract
   REM when they match by getting row contents
       X$ = ""
       FOR I = 0 TO COLUMN - 1'
       FOR J = 0 TO COLUMN - 1
       IF TEMP$(I) <> TRANS$(0, J) GOTO NMATCH
       FOR K = 0 TO ROW - 1        'match, get row contents
       X$ = X$ + LTRIM$(TRANS$(K, J))
       NEXT K
NMATCH:    NEXT J
       NEXT I
   REM Place into PLAINTEXT array
       FOR I = 0 TO 25
       PLAINTEXT$(I) = MID$(X$, I + 1, 1)
       NEXT I
RETURN

The first group of statements in the subroutine TRANSPORT initializes a 26x26 string matrix labeled TRANS$ to nulls. This matrix permits a keyword or keyword phrase that can be up to 26 characters in length. The second group of statements determines the actual size of the matrix. Here, the value of COLUMN is the length of the keyword in characters. Thus, a keyword consisting of five characters would require a matrix of five columns and six rows for all 26 letters of the alphabet to fit into the matrix.

The third group of statements places the characters of a keyword-based alphabet into the string array TRANS$. Note that the values for the indices for the nested FOR-NEXT loops are adjusted by subtracting 1 from the values of ROW and COLUMN because the matrix elements commence at position 0,0. Also, note values are assigned to TRANS$ from the string array PLAINTEXT$. The array PLAINTEXT$ contains the keyword developed alphabet which was filled by the prior invoking of the previously described KEYWORD subroutine. You must invoke KEYWORD prior to invoking TRANSPORT. To facilitate passing information to determine the number of columns in the matrix, the statement KEYCOL$=X$ was added to the end of the third group of statements in the KEYWORD subroutine.

The PRINT statements included in the third group of statements simply display the matrix for your review and will be removed from the subroutine later in this chapter. The IF statement permits the selection of a simple or numeric-based mixed alphabet. If the string variable ORDER$ does not equal "NUMERIC," the simple transposition-based mixed alphabet is formed. This occurs in the fourth statement group which simply adds each character in every row of each column to the string X$ in column sequence. The resulting simple transposition-based mixed alphabet is placed into the string array PLAINTEXT$ and the subroutine ends.

If the user wants a numeric transposition-based mixed alphabet, a branch to the label NUMERIC in the subroutine occurs. At this location, the first character in each column is placed into temporary storage in the string array TEMP$. Next, the contents of TEMP$ are sorted, resulting in that array containing the characters of the keyword in sorted order. This is followed by another group of statements which cycles through the elements of the TEMP$ array. Each element of that array is matched against the first character in each column of the TRANS$ array. When a match occurs, the contents of the column are extracted and added to the string X$, which results in the formation of a numeric transposition mixed sequence alphabet. The last group of statements places the resulting alphabet into the string array PLAINTEXT$.

Listing 4.2 illustrates the modified calling sequence used to invoke the subroutine TRANSPORT. This calling sequence will display the formed matrix, ciphertext alphabet, and plaintext alphabet to enable you to verify the operation of the subroutine. To enable you to use the calling sequence and the TRANSPORT subroutine, the code has been stored on a file labeled CIPHERTR.BAS on the companion CD. The PRINT statements that display the matrix and alphabets will be removed when you develop the program CIPHER6.BAS.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 4.2 The main part of CIPHERTR.BAS and the calling sequence to invoke the TRANSPORT subroutine.

REM PROGRAM CIPHERTR.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26), KEY$(26), TRANS$(26, 26), TEMP$(26)
    CLS
    PRINT "PROGRAM CIPHERTR.BAS enciphers a message using a
               transposition matrix"
    PRINT "and a monoalphabetic substitution process based upon a
               keyword"
    PRINT "or keyword phrase and an alphabetic shift key."
    PRINT
GOSUB INITIALIZE
    INPUT "Enter keyword or keyword phrase in UPPERCASE:", TEXT$
    PRINT "Plaintext based alphabet is :";   '*****
    FOR I = 0 TO 25: PRINT PLAINTEXT$(I);: NEXT I   '*****
GOSUB KEYWORD            'form keyword-based mixed alphabet
    PRINT "Keyword-based alphabet is    :"; X$  '*****
    INPUT "Enter transposition matrix method - SIMPLE or NUMERIC :",
           ORDER$
GOSUB TRANSPORT
    PRINT ORDER$; " Transposition Alphabet is    :"; X$
1   INPUT "Enter UPPERCASE Alphabetic Shift Key:", K$
    FOR I = 0 TO 25
    IF K$ = PLAINTEXT$(I) GOTO 2
    NEXT I
    PRINT "You must enter a letter from A to Z"
    GOTO 1
2   REM Position I represents shift key letter

GOSUB FORMCIPHER          'create cipher alphabet
    PRINT "Shifted keyword mixed alphabet is     :";    '*****
    FOR I = 0 TO 25: PRINT CIPHERTEXT$(I);: NEXT I: PRINT      '*****
GOSUB INITIALIZE    'reinitialize plaintext array
GOSUB MSGFILE       'assign I/O files, place message on a file
GOSUB CONVERTSTORE       'convert and store ciphertext on a file
GOSUB PRTOUT        'print results
STOP

The CIPHERTR.BAS Program

Figure 4.6 illustrates the execution of the program CIPHERTR.BAS to encipher a message based upon the use of a simple transposition mixed sequence alphabet and an alphabetic shift key. This example uses the keyword KEYWORD and the alphabetic shift character C. Note the display of the 4-row-by-7-column matrix formed by the TRANSPORT subroutine. Because a simple transposition method was selected, the letters in the matrix are extracted in column order, KAJT followed by EBLU, and so on. This is verified by the line after the end of the matrix display.

Figure 4.6 The execution of CIPHERTR.BAS program using a simple transposition keyword mixed alphabet.

As in previous examples, an alphabetic shift key character results in the shift of a previously developed alphabet so that the position of the shift key letter is rotated to the right of the alphabet. Entering the shift key C rotates the simple transposition alphabet by ten positions.

The plaintext alphabet should be compared to the shifted keyword mixed alphabet for the encipherment of a message, because the latter alphabet is the ciphertext alphabet. In this example the message to be enciphered is MEET ME IN ST LOUIS ON FRIDAY FEBRUARY THIRTEEN AT NINE PM. The M in the plaintext alphabet corresponds to the letter Q in the shifted keyword mixed alphabet. Thus, the first character in the enciphered message is Q. The character E in the message is located in the plaintext alphabet. Because its position corresponds to the position of the character N in the shifted keyword mixed alphabet, replace E with N and continue the process to verify the operation of the portion of the program which enciphers a message based upon a simple transposition mixed alphabet and alphabetic shift key.

Figure 4.7 illustrates the execution of the program CIPHERTR.BAS when a numeric transposition mixed alphabet was selected. Because D is the lowest character in the keyword KEYWORD, the characters in the column headed by D are extracted first. Thus, DIS are the first three letters displayed for the numeric transposition alphabet. Because E is the next lowest letter in the keyword, the contents of the column headed by E are added to the alphabet. Thus, EBLU follows the characters DIS. You can examine the remainder of Figure 4.7 to verify that the message is indeed enciphered correctly by using the relationship between the plaintext and shifted keyword mixed alphabets displayed in that illustration.

Figure 4.7 The execution of CIPHERTR.BAS program using a numeric transposition keyword mixed alphabet.

The INTERVAL Subroutine

Listing 4.3 contains the contents of the subroutine INTERVAL, which forms an interval extracted alphabet based upon a predefined interval. That interval is passed to the subroutine in the variable INTERVAL.

Listing 4.3 The INTERVAL subroutine used to form an interval extracted alphabet.

INTERVAL:
    REM Subroutine to form interval extracted alphabet
    COUNT = 0
    I = INTERVAL - 1  'adjust since array starts at 0
    X$ = ""
    DO UNTIL COUNT >= 26
        X$ = X$ + PLAINTEXT$(I)
        COUNT = COUNT + 1
        I = I + INTERVAL
        IF I >= 26 THEN I = I - 26
    LOOP
    REM Put back into PLAINTEXT$ array
    FOR I = 0 TO 25
    PLAINTEXT$(I) = MID$(X$, I + 1, 1)
    NEXT I
RETURN

Note in Listing 4.3 that the interval value is decreased by 1 since the array starts at element 0. The DO UNTIL COUNT loop extracts each character from the string array PLAINTEXT$ until all of the characters are extracted. Note that whenever I equals or exceeds 26, you should reset I to I-26. This recirculates the sequence. Also note that since you are using the string array PLAINTEXT$, you must invoke the subroutine KEYWORD prior to invoking the subroutine INTERVAL if you wish to use an interval extraction based upon a keyword mixed alphabet. Similar to the TRANSPORT subroutine, conclude the INTERVAL subroutine by placing the interval extracted alphabet back into the array PLAINTEXT$.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The TRANSPORT.CPP Program

Prior to developing the program CIPHER6 that will encipher messages based upon any one of the three alphabetic formation techniques previously presented in this chapter, let's turn our attention to C++ coding of the subroutine TRANSPORT and the program CIPHERTR. The C++ versions of both are included on the CD-ROM in the C directory as both source and executable files.

Listing 4.4 lists the statements in the C++ version of TRANSPORT, which is stored on the CD-ROM under the filename TRANSPORT.CPP. The executable version of TRANSPORT.CPP is stored on the CD-ROM in the C directory under the filename TRANSPORT.EXE.

Listing 4.4 The TRANSPORT.CPP program listing.

/*
transport.cpp
C++ code written by Jonathan Held using Microsoft Visual C++, version
5.0, on March 21, 1998.
*/

//standard include files
#include<iostream.h>
#include<assert.h>
#include<string.h>
#include<ctype.h>

//function prototypes
bool checkInput(char * &);
void createCipherStream(char *, char[]);
void createMatrix(char ** &, const char [], const int, const int);
int findLowestValue(int *&, const int);
int getKeyword(char *&);
void printMatrix(char ** &, const int, const int);
void reviseCipherStream(char **&, char [], const int, const int);
void deleteMatrix(char **&, const int, const int);

//constants we will use
const int TWENTYFIVE = 25, TWENTYSIX = 26,
      TWENTYSEVEN = 27, SIXTYFIVE = 65, NINETY = 90,
      NINETYTWO = 92, SIZE = 256;

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main()
{
  char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                   'I','J','K','L','M','N','O','P',
                   'Q','R','S','T','U','V','W','X',
                   'Y','Z', '\0'};

  char cipherStream[TWENTYSEVEN] = {'\0'};
  char ** cMatrix;
  char *keyword;
  int rows, columns;

  //get the keyword we are going to use, determine number of rows
  //and columns of our matrix
  columns = getKeyword(keyword);
  rows = TWENTYSIX/columns;

  //integer division requires that we check and see if there is a
  //remainder; if so, we need to add one extra row to our matrix
  if (TWENTYSIX%columns){
     rows++;
  }

  //create the initial ciphertext stream
  createCipherStream(keyword, cipherStream);

  //insert the stream into our matrix
  createMatrix(cMatrix, cipherStream, rows, columns);

  reviseCipherStream(cMatrix, cipherStream, rows, columns);

  //echo to the user what we have
  cout << "\nPlaintext-based alphabet is:" << plaintext << endl
     << "Keyword-based alphabet is: " << cipherStream << endl << endl;

  //delete the dynamically allocated matrix
  deleteMatrix(cMatrix, rows, columns);

  delete keyword;

  return 0;
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
  bool error = false;
  int count = strlen(input);

  for (int ix=0; ix<count; ix++){

   int char_value = static_cast<int>(*(input+ix));
   //determine if the user did not enter an uppercase character
   if ((char_value < SIXTYFIVE) || (char_value > NINETY)){
      error = true;
      cerr << "You entered an invalid keyword!" << endl << endl;
      break;
   }
  }

  return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: createCipherStream()
//Parameters: input - the keyword the user entered
//      cipher - the keyword alphabet that will be constructed
//Return Type: None
//Purpose: Creates a preliminary cipher stream that will be used to
//form the cipher matrix.
//----------------------------------------------------------------
void createCipherStream(char *input, char stream[])
{
 bool used[TWENTYSIX];
 int index = 0,
   count = strlen(input);

 //no characters are initially used
 for (int ix=0; ix<TWENTYSIX; ix++){
   used[ix] = false;
 }

 //keep track of each character used, start forming the keyword
 //alphabet
 for (int jx=0; jx<count; jx++){

   //get each character of the input string (integer value)
   int char_value = static_cast<int>(*(input+jx));
   if (used[char_value-SIXTYFIVE]){
      //do nothing - the character was already used
   }
   else {
     //mark as used and add to the keyword alphabet
     used[char_value-SIXTYFIVE] = true;
     *(stream+index++) = static_cast<char>(char_value);
   }
 }

 //go through the list of characters used - those which weren't
 //used should be added to the keyword alphabet
 for (int kx=0; kx<TWENTYSIX; kx++){

   if (!(used[kx])){
      *(stream+index++) = static_cast<char>(SIXTYFIVE+kx);
   }
 }

 return;
}//end createCipherStream()

//----------------------------------------------------------------
//Function: createMatrix()
//Parameters: matrix - the matrix we are going to create
//      CSTREAM - the initial cipher stream based on the
//                      user's keyword
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Creates a numeric key transposed matrix, identical to
//figure 4.4.
//----------------------------------------------------------------
void createMatrix(char ** &matrix, const char CSTREAM[], const int ROWS,
                    const int COLS)
{
  int count = 0;

  //dynamically allocate memory for the RxC matrix
  //we use assert to ensure that memory was allocated;
  //if not, then the program will terminate abnormally
  assert(matrix = new char*[ROWS]);
  for (int ix=0; ix<ROWS; ix++){

    *(matrix + ix) = new char[COLS];
  }

  //fill in the matrix
  for (int jx=0; jx<ROWS; jx++){
    for (int kx=0; kx<COLS; kx++){
    //we only want to enter a character into the matrix
    //twenty-six times - most of the time we allocate
    //a matrix larger than what we will need to use; when
    //we have more than 26 characters, we simply insert a
    //null terminator into the matrix
      if (count < TWENTYSIX)
         *(*(matrix+jx)+kx) = CSTREAM[count++];
     else
         *(*(matrix+jx)+kx) = '\0';
     }
  }

  //print the resulting matrix
  printMatrix(matrix, ROWS, COLS);

  return;
}//end createMatrix()

//----------------------------------------------------------------
//Function: deleteMatrix()
//Parameters: matrix - the matrix we are going to destroy
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Destroys the dynamically allocated matrix!
//----------------------------------------------------------------
void deleteMatrix(char **&matrix, const int R, const int C)
{
 for(int ix=0; ix<R; ix++)
   delete [] *(matrix+ix);

 delete [] matrix;

 return;
}//end deleteMatrix()

//----------------------------------------------------------------
//Function: findLowestValue()
//Parameters: values - the ASCII values of all characters in the
//            top row of the matrix
//      COLS - the number of columns in the matrix
//Return Type: int - the column we want.
//Purpose: Determines what column we are going to extract from
//the matrix and put into the cipher stream.
//----------------------------------------------------------------
int findLowestValue(int *&values, const int COLS)
{
  int loc=0, lowest=999;

  for (int ix = 0; ix < COLS; ix++){
    if (*(values+ix) != 999){
       if (*(values+ix) < lowest){
          lowest = *(values+ix);
          loc = ix;
       }
    }
 }

 *(values+loc) = 999;

 return loc;
}//end findLowestValue()

//----------------------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: int - the length of the keyword
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.  Returns the length of the
//keyword.
//----------------------------------------------------------------
int getKeyword(char * &text)
{
  bool error = false;
  char buffer[SIZE];

  do {
     cout << "Enter keyword or keyword phrase in UPPERCASE" << endl
        << "do not use spaces or non-alphabetic characters): ";
     cin.getline(buffer, SIZE, '\n');
     assert(text = new char[strlen(buffer) + 1]);
     strcpy(text, buffer);

     error = checkInput(text);

    //delete text if there was an error
    if (error){
       delete [] text;
    }

  } while (error);

  cout << endl;

  return strlen(buffer);
}//end getKeyword()

//----------------------------------------------------------------
//Function: printMatrix()
//Parameters: the_matrix - the matrix we want to display
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Displays the matrix.
//----------------------------------------------------------------
void printMatrix(char ** &the_matrix, const int ROWS, const int COLUMNS)
{
  cout << "The matrix is:" << endl << endl;

  for (int ix=0; ix<ROWS; ix++){

     for (int kx=0; kx<COLUMNS; kx++){
       cout << *(*(the_matrix+ix)+kx) << " ";
     }

     cout << endl;
  }

  return;
}//end printMatrix()

//----------------------------------------------------------------
//Function: reviseCipherStream()
//Parameters: MATRIX - the numeric keyed transposed matrix
//      cipher - the cipher stream that was originally created
//                     when the user entered a keyword
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Overwrites the cipher stream initially created with the
//one we get by taking columns out of the matrix.
//----------------------------------------------------------------
void reviseCipherStream(char ** &MATRIX, char cipher[], const int R,
                       const int C)
{
  bool done = false;
  int counter = 0;

  //place ASCII values of first row's characters into a table;
  //we will use these values to determine the order in which
  //we pull columns out of the matrix
  int *top_row_ASCII_values = new int[C];
  assert(top_row_ASCII_values);

  //no cast to integer type required since this is already taken
  //care of for us
  for (int ix=0; ix<C; ix++){
      *(top_row_ASCII_values + ix) = *((*MATRIX)+ix);
  }

  for (int jx=0; jx<C; jx++){

    //find out what column we want
    int col = findLowestValue(top_row_ASCII_values, C);

    //put contents of the column into the cipher stream, but
    //only do this when we have a character!
    for (int kx=0; kx<R; kx++){
      if ((*(*(MATRIX+kx)+col)) != '\0'){
         cipher[counter++] = (*(*(MATRIX+kx)+col));
      }
    }
  }
  //destroy dynamically allocated memory
  delete [] top_row_ASCII_values;

  return;
}//end reviseCipherStream()
//end file transport.cpp

In examining the TRANSPORT.CPP listing shown in Listing 4.4, note that the function createMatrix is used to create a numeric key transposed matrix which is identical to Figure 4.4. This subroutine dynamically allocates memory based upon the rows and columns. Note that the function deleteMatrix is included in the program to remove any previously allocated matrix. This is good programming practice.

Two additional functions that warrant a bit of attention are findLowestValue and reviseCipherStream. The function findLowestValue is used to determine the column that is going to be used for extraction and placed into the cipher stream, while the function reviseCipherStream is used to overwrite the cipher stream initially created with the one obtained by taking columns out of the matrix.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHERTR.CPP Program

Listing 4.5 lists the statements in the C++ program CIPHERTR.CPP, which provides a similar capability to CIPHERTR.BAS. That is, this program enciphers a message based upon the use of a simple transposition mixed-sequence alphabet and an alphabet shift key. Although the listing is lengthy and the program is included on the CD-ROM, it is included in its entirety here as doing so provides you with the ability to note the relationship between functions and the order by which they are invoked. Due to the extensive documentation contained within the program listing, it is left for you to examine the listing and the remarks which illustrate how the program operates.

Listing 4.5 The CIPHERTR.CPP program listing.

/*
ciphertr.cpp
C++ code written by Jonathan Held
using Microsoft Visual C++, version 5.0, on March 23, 1998.
*/

//standard include files
#include<iostream.h>
#include<assert.h>
#include<string.h>
#include<ctype.h>
#include<fstream.h>

//function prototypes
bool checkInput(char * &);
void createCipherStream(char *, char[]);
void createMatrix(char ** &, const char [], const int, const int);
void display(char *);
bool encryptText(char *, char *, const char [], const char[], char []);
void deleteMatrix(char **&, const int, const int);
int findLowestValue(int *&, const int);
void formatData(char []);
char* formCipheredMessage(const char[], const char [], char []);
void getFileNames(char *&, char *&);
int getInputType(void);
int getKeyword(char *&);
bool getMessage(char*, char*, char [], const char[], const char[]);
void getShiftKey(char &);
void groupBUFFER(ofstream, int);
void numericExtraction(char **&, char [], const int, const int);
void printCipherToFile(ofstream, char[]);
void printMatrix(char ** &, const int, const int);
void reviseCipherStream(char **&, char [], const int, const int);
void shiftKeyword(char [], const char);
void simpleExtraction(char **&, char [], const int, const int);
int typeOfExtraction(void);

//constants we will use
const int FIVE = 5, TWENTYFIVE = 25, TWENTYSIX = 26,
      TWENTYSEVEN = 27, SIXTYFIVE = 65, NINETY = 90,
      NINETYTWO = 92, SIZE = 256, BIGSIZE = 1000;

char BUFFER[BIGSIZE] = {'\0'};
//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main()
{
 char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                  'I','J','K','L','M','N','O','P',
                  'Q','R','S','T','U','V','W','X',
                  'Y','Z', '\0'};

 char cipherStream[TWENTYSEVEN] = {'\0'};
 char message_to_cipher[SIZE], enciphered_message[SIZE];

 char ** cMatrix;
 char *keyword, key, *infile, *outfile;
 int rows, columns, input_type;
 bool success;

 //get the keyword we are going to use, determine number of rows
 //and columns of our matrix
 columns = getKeyword(keyword);
 rows = TWENTYSIX/columns;

 //integer division requires that we check and see if there is a
 //remainder; if so, we need to add one extra row to our matrix
 if (TWENTYSIX%columns){
    rows++;
 }

 //create the initial ciphertext stream
 createCipherStream(keyword, cipherStream);

 //echo to the user what we have
 cout << "Plaintext-based alphabet is: " << plaintext  << endl
    << "Keyword-based alphabet is:   " << cipherStream  << endl << endl;

 //insert the stream into our matrix
 createMatrix(cMatrix, cipherStream, rows, columns);

 reviseCipherStream(cMatrix, cipherStream, rows, columns);

 getShiftKey(key);
 shiftKeyword(cipherStream, key);

 getFileNames(infile, outfile);

 //query the user as to whether we are deciphering keyboard
 //or file input
 input_type = getInputType();
 //process file input
 if(input_type){
   success = encryptText(infile, outfile, plaintext, cipherStream,
                 enciphered_message);
 }
 else {
   cout << "Use a \'/\' to leave a line in plaintext." << endl
      << "Use a \'\\' to indicate end of message input. " << endl;
   success = getMessage(infile, outfile, message_to_cipher, plaintext,
    cipherStream);
 }

 //report success of operation
 if (!success){
    cerr << "Error: Invalid filename specified. Goodbye."  << endl;
 }
 else {
   cout << "Press return to display resulting enciphered message."
                      << endl;
   //get the newlines off the current input stream
   cin.get();
   if(input_type == 1){
     cin.get();
   }
   display(outfile);
 }

 //delete the dynamically allocated matrix
 deleteMatrix(cMatrix, rows, columns);

 delete keyword;

 return (!success);
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
 bool error = false;
 int count = strlen(input);

 for (int ix=0; ix<count; ix++){

   int char_value = static_cast<int>(*(input+ix));
   //determine if the user did not enter an uppercase character
   if ((char_value < SIXTYFIVE) || (char_value > NINETY)){
      error = true;
      cerr << "You entered an invalid keyword!" << endl << endl;
      break;
   }
 }

 return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: createCipherStream()
//Parameters: input - the keyword the user entered
//      cipher - the keyword alphabet that will be constructed
//Return Type: None
//Purpose: Creates a preliminary cipher stream that will be used to
//form the cipher matrix.
//----------------------------------------------------------------
void createCipherStream(char *input, char stream[])
{
 bool used[TWENTYSIX];
 int index = 0,
   count = strlen(input);

 //no characters are initially used
 for(int ix=0; ix<TWENTYSIX; ix++){
  used[ix] = false;
 }

 //keep track of each character used, start forming the keyword
 //alphabet
 for (int jx=0; jx<count; jx++){

   //get each character of the input string (integer value)
   int char_value = static_cast<int>(*(input+jx));

   if(used[char_value-SIXTYFIVE]){
     //do nothing - the character was already used
   }
   else {
     //mark as used and add to the keyword alphabet
     used[char_value-SIXTYFIVE] = true;
     *(stream+index++) = static_cast<char>(char_value);
   }
}

//go through the list of characters used - those which weren't
//used should be added to the keyword alphabet
 for(int kx=0; kx<TWENTYSIX; kx++){

   if(!(used[kx])){
     *(stream+index++) = static_cast<char>(SIXTYFIVE+kx);
   }
 }

 return;
}//end createCipherStream()

//----------------------------------------------------------------
//Function: createMatrix()
//Parameters: matrix - the matrix we are going to create
//      CSTREAM - the initial cipher stream based on the
//           user's keyword
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Creates a numeric key transposed matrix, identical to
//figure 4.4.
//----------------------------------------------------------------
void createMatrix(char ** &matrix, const char CSTREAM[], const int ROWS,
                    const int COLS)
{
 int count = 0;

 //dynamically allocate memory for the RxC matrix
 //we use assert to ensure that memory was allocated;
 //if not, then the program will terminate abnormally
 assert(matrix = new char*[ROWS]);
 for(int ix=0; ix<ROWS; ix++){
   *(matrix + ix) = new char[COLS];
 }

 //fill in the matrix
 for(int jx=0; jx<ROWS; jx++){
   for (int kx=0; kx<COLS; kx++){
   //we only want to enter a character into the matrix
   //twenty-six times - most of the time we allocate
   //a matrix larger than what we will need to use; when
   //we have more than 26 characters, we simply insert a
   //null terminator into the matrix
     if(count < TWENTYSIX)
       *(*(matrix+jx)+kx) = CSTREAM[count++];
   else
       *(*(matrix+jx)+kx) = '\0';
   }
 }

 return;
}//end createMatrix()

//----------------------------------------------------------------
//Function: encryptText()
//Parameters:  inp_file - the name of the input plaintext file
//     outp_file - the name of the output ciphertext file
//     PTEXT[] - the plaintext alphabet
//     CTEXT[] - the ciphertext alphabet
//     encoded_msg[] - the message to be encoded
//Return Type: bool, indicating success of operation
//Purpose: Used to encrypt file input.  Takes each line of the input
//file, encrypts it, and saves the result to the specified output
//file.
//----------------------------------------------------------------
bool encryptText(char * inp_file, char * outp_file, const char PTEXT[],
          const char CTEXT[], char encoded_msg[])
{
 bool success = false;
 char ip[SIZE];

 //declare file stream objects
 ifstream input(inp_file, ios::in);
 ofstream output(outp_file, ios::app);

 if((!input) || (!output)){
   //do nothing - I/O error; user will be notified upon
   //procedure's return to main()
 }
 else {

   success = true;

   //print plaintext and ciphertext alphabets to the
   //output file
   output << "PLAINTEXT:  " << PTEXT << endl;
   output << "CIPHERTEXT: " << CTEXT << endl << endl;

   while (input.getline(ip, BIGSIZE, '\n')){

    //check to see if the user wants the line to appear in plain text
    if(ip[0] == '/'){

      if (strlen(BUFFER)>0){

      //empty whatever is in the buffer
      groupBUFFER(output, strlen(BUFFER));
      //adjust the buffer
      strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
      //output plaintext
    }

    output << ip << endl;

    }
    else {
      //encipher the line
      char *msg = formCipheredMessage(CTEXT, ip, encoded_msg);
      //print the cipher in groups of five to the ouput file
      printCipherToFile(output, msg);
    }
 }

  //empty the rest of the buffer
  groupBUFFER(output, strlen(BUFFER));
  //notify user where plaintext and ciphertext files are
  cout << "Plaintext file is: " << inp_file << endl;
  cout << "Encrypted file is: " << outp_file << endl << endl;
 }

 //don't forget to close the files
 input.close();
 output.close();

 //return success of the operation
 return success;
}//end encryptText()

//----------------------------------------------------------------
//Function: deleteMatrix()
//Parameters: matrix - the matrix we are going to destroy
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Destroys the dynamically allocated matrix!
//----------------------------------------------------------------
void deleteMatrix(char **&matrix, const int R, const int C)
{
 for(int ix=0; ix<R; ix++)
   delete [] *(matrix+ix);

  delete [] matrix;

  return;
}//end deleteMatrix()

//----------------------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echoes the resulting output file to the screen.
//----------------------------------------------------------------
void display(char *name)
{
 ifstream infile(name, ios::in);
 char input[SIZE];

 if(!(infile)){
   cerr << "Unable to open input file for display." << endl;
 }
 else {
   while (infile.getline(input, SIZE, '\n')){
    cout << input << endl;
   }
 }

 return;
}//end display()

//----------------------------------------------------------------
//Function: findLowestValue()
//Parameters: values - the ASCII values of all characters in the
//            top row of the matrix
//      COLS - the number of columns in the matrix
//Return Type: int - the column we want.
//Purpose: Determines what column we are going to extract from
//the matrix and put into the cipher stream.
//----------------------------------------------------------------
int findLowestValue(int *&values, const int COLS)
{
 int loc=0, lowest=999;

 for(int ix = 0; ix < COLS; ix++){
   if(*(values+ix) != 999){
     if(*(values+ix) < lowest){
       lowest = *(values+ix);
       loc = ix;
     }
   }
 }

 *(values+loc) = 999;

 return loc;
}//end findLowestValue()

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[]){

 for(int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
   if(*(data+nx) == ' '){
      //do nothing - skip over the space in the data
   }
   else {
     *(data+mx++) = *(data+nx);
   }
 }

 //don't forget to add the null terminator
 *(data+mx) = '\0';

 return;
}//end formatData()

//----------------------------------------------------------------
//Function: formCipheredMessage()
//Parameters:  CTEXT - the cipher alphabet we will use for substitution
//      MESSAGETOCIPHER - the user's message
//      enc_message - the enciphered message to be determined
//Return Type: char* - a pointer to the encoded information.
//Purpose: Encipher the user's message.
//----------------------------------------------------------------
char* formCipheredMessage(const char CTEXT[], const char
              MESSAGETOCIPHER[], char enc_message[]){

 int length = strlen(MESSAGETOCIPHER)+1;

 int encode_value;

 for (int ix=0; ix<length; ix++){

  //test to see if we have an alphabetic character; if not,
  //simply copy it to our encrypted message - this preserves
  //characters such as ', ! etc...
  if(!isalpha(static_cast<int>(MESSAGETOCIPHER[ix]))){
    enc_message[ix] = MESSAGETOCIPHER[ix];
  }
  else {
    //valid character - the easy way to calculate the ciphered
    //character is based on the plain text's ascii character value;
    //since it has to be a capital letter, it must be in the range
    //from 65 to 90, with A represented by 65, Z by 90.  By simply
    //subtracting 65 from the encode_value (the integer representation
    //of the plaintext character), we now know what cipher character
    //to use.

    encode_value = toupper(static_cast<int>(MESSAGETOCIPHER[ix]));
    enc_message[ix] = CTEXT[encode_value-SIXTYFIVE];
  }
 }

 //return a reference to the encoded message
 return enc_message;
}//end formCipheredMessage()

//----------------------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//----------------------------------------------------------------
int getInputType(void)
{
 char type;
 bool error = false;
 int value;

 do {
  //prompt user for input from file or keyboard
  cout << "Is file input from keyboard (K, k) or file (F, f): ";
  cin >> type;

  //make type an uppercase letter
  type = static_cast<char>(toupper(static_cast<int>(type)));
  //check for an invalid character
  if((type != 'K') && (type != 'F')){
    cerr << "You have entered an invalid character!" << endl << endl;
    error = true;
  }
  else {
    if(type == 'K')
      value = 0;        //value of 0 represents keyboard input
      else value = 1;   //value of 1 represents file input
    error = false;
  }

 } while (error);

 cout << endl;

 return value;

}//end getInputType()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
//      outfile_name - the output file we will write the
//      enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//----------------------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
 char data[SIZE];

 cout << "Enter filename to store/retrieve plaintext message: ";

 cin >> data;

 infile_name = new char[strlen(data) + 1];
 strcpy(infile_name, data);

 cout << "Enter filename to store enciphered message: ";

 cin >> data;
 outfile_name = new char[strlen(data) + 1];
 strcpy(outfile_name, data);

 cout << endl;

 return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: int - the length of the keyword
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.  Returns the length of the
//keyword.
//----------------------------------------------------------------
int getKeyword(char * &text)
{
 bool error = false;
 char buffer[SIZE];

 do {
  cout << "Enter keyword or keyword phrase in UPPERCASE" << endl
     << "do not use spaces or non-alphabetic characters): ";

  cin.getline(buffer, SIZE, '\n');
  assert(text = new char[strlen(buffer) + 1]);
  strcpy(text, buffer);

  error = checkInput(text);

  //delete text if there was an error
  if(error){
    delete [] text;
  }

 } while (error);

 cout << endl;

 return strlen(buffer);
}//end getKeyword()

//----------------------------------------------------------------
//Function: getMessage()
//Parameters:  input - the name of the input plaintext file
//      output the name of the output ciphertext file
//      msg_to_cipher - the message to be encoded
//      PTEXT[] - the plaintext alphabet
//      CTEXT[] - the ciphertext alphabet
//Return Type: bool, indicating success of operation
//Purpose: Allow the user to manually input text from the keyboard.
//Save the text in plaintext to the input file; encrypt the text
//and save it to the specified output file for later retrieval.
//----------------------------------------------------------------
bool getMessage(char* input, char* output, char msg_to_cipher[],
         const char PTEXT[], const char CTEXT[])
{
 bool go_on = true, success = false;

 ofstream textFile(input, ios::app);
 ofstream cipherFile(output, ios::app);

 if((!textFile) || (!cipherFile)){
   //do nothing - error will be noted to user later
 }
 else {

   success = true;

   textFile << "PLAINTEXT:  " << PTEXT << endl;
   textFile << "CIPHERTEXT: " << CTEXT << endl << endl;

   //get the newline character off of the input stream
   cin.get();

   cout << "Enter the message in UPPERCASE characters: " << endl;

   while (go_on) {

    //get the entire line, up to 256 characters
    cin.getline(msg_to_cipher, SIZE, '\n');

    //case user doesn't want the text to be encrypted
    if(msg_to_cipher[0] == '/'){

      if(strlen(BUFFER)>0){
        //empty whatever is in the buffer
        groupBUFFER(cipherFile, strlen(BUFFER));
        //adjust the buffer
        strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
      }

      //output plaintext
      textFile << msg_to_cipher << endl;
      cipherFile << msg_to_cipher << endl;
  }

   //case user is done entering text
   else if (static_cast<int>(msg_to_cipher[0]) == NINETYTWO){
   go_on = false;
  }

   //encrypt the text
   else {
     textFile << msg_to_cipher << endl;
     char enciphered_msg[BIGSIZE];
     formCipheredMessage(CTEXT,msg_to_cipher,enciphered_msg);
     printCipherToFile(cipherFile,enciphered_msg);
  }
  }

 //empty the rest of the buffer
 groupBUFFER(cipherFile, strlen(BUFFER));
 }

 //close the files
 textFile.close();
 cipherFile.close();

 //notify user where plaintext and ciphertext files are
 cout << "\nPlaintext file is: " << input << endl;
 cout << "Encrypted file is: " << output << endl << endl;

 return success;
}//end getMessage()

//----------------------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//----------------------------------------------------------------
void getShiftKey(char &key_desired)
{
 bool error = true;

 do {
  //prompt user to enter an uppercase shift key
  cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
  cin >> key_desired;

  int key_value = static_cast<int>(key_desired);

  //do some error checking
  if((key_value < SIXTYFIVE) || (key_value > NINETY)){
    cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
  }
  else {
    cout << endl;

    error = false;
  }
 } while (error);

 return;
}//end getShiftKey()

//----------------------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
//      num - the number of characters we want to output
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//----------------------------------------------------------------
void groupBUFFER(ofstream out, int num)
{

 for(int kx=0;kx<num;kx++){

   if((kx!=0) && (kx%TWENTYFIVE==0)){
     out << endl;
   }

   if((kx!=0) && (kx%FIVE == 0) && (kx%TWENTYFIVE!=0)){
     out << " " << *(BUFFER+kx);
   }
   else {
   out << *(BUFFER+kx);
   }
 }

 out << endl;

 return;
}//end groupBUFFER()

//----------------------------------------------------------------
//Function: numericExtraction()
//Parameters: MATRIX - the matrix we want to form the cipher from
//      cipher - the cipher stream we will create
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Creates the cipher stream we will use to encrypt files
//by pulling off the columns from the matrix in alphabetical order
//based on the character in the top row, and inserting them into
//the cipher array.
//----------------------------------------------------------------
void numericExtraction(char **& MATRIX, char cipher[], const int R,
                            const int C)
{
 int counter = 0;

 //place ASCII values of first row's characters into a table;
 //we will use these values to determine the order in which
 //we pull columns out of the matrix
 int *top_row_ASCII_values = new int[C];
 assert(top_row_ASCII_values);

 //no cast to integer type required since this is already taken
 //care of for us
 for(int ix=0; ix<C; ix++){
   *(top_row_ASCII_values + ix) = *((*MATRIX)+ix);
 }

 for(int jx=0; jx<C; jx++){

   //find out what column we want
   int col = findLowestValue(top_row_ASCII_values, C);

   //put contents of the column into the cipher stream, but
   //only do this when we have a character!
   for(int kx=0; kx<R; kx++){
     if((*(*(MATRIX+kx)+col)) != '\0'){
       cipher[counter++] = (*(*(MATRIX+kx)+col));
     }
   }
 }

 //destroy dynamically allocated memory
 delete [] top_row_ASCII_values;

 return;
}//end numericExtraction()

//----------------------------------------------------------------
//Function: printCipherToFile()
//Parameters: op - the output file we are writing to
//      msg - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-block characters in the
//specified output file.
//----------------------------------------------------------------
void printCipherToFile(ofstream op, char msg[])
{
 formatData(msg);

 //check to see if there are more than 25 characters
 //in the buffer; if so, print out as many groups of
 //25 as possible
 if(strlen(BUFFER) >= TWENTYFIVE){

   int numchars = (strlen(BUFFER)/TWENTYFIVE)*TWENTYFIVE;
   //print the contents of the buffer to the output stream
   groupBUFFER(op, numchars);
   //shift whatever is left in the buffer
   strcpy(BUFFER, (BUFFER+numchars));
   //append data to the buffer
   strcat(BUFFER, msg);
 }

 //if buffer contents are less than 25, simply append the new
 //data to the buffer
 else if ((strlen(BUFFER) >= 0) && (strlen(BUFFER) < TWENTYFIVE)){
   strcat(BUFFER, msg);
 }

 return;
}//end printCipherToFile()

//----------------------------------------------------------------
//Function: printMatrix()
//Parameters: the_matrix - the matrix we want to display
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Displays the matrix.
//----------------------------------------------------------------
void printMatrix(char ** &the_matrix, const int ROWS, const int COLUMNS)
{
 cout << "\nThe matrix is:" << endl << endl;

 for(int ix=0; ix<ROWS; ix++){

   for(int kx=0; kx<COLUMNS; kx++){
     cout << *(*(the_matrix+ix)+kx) << " ";
   }

   cout << endl;
 }

  return;
}//end printMatrix()

//----------------------------------------------------------------
//Function: reviseCipherStream()
//Parameters: MATRIX - the numeric keyed transposed matrix
//      cipher - the cipher stream that was originally created
//           when the user entered a keyword
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Overwrites the cipher stream initially created with the
//one we get by taking columns out of the matrix.
//----------------------------------------------------------------
void reviseCipherStream(char ** &MATRIX, char cipher[], const int R,
                            const int C)
{
 int extract_type = typeOfExtraction();

 if(!(extract_type)){
   simpleExtraction(MATRIX, cipher, R, C);
 }
 else {
   numericExtraction(MATRIX, cipher, R, C);
 }

 //print the resulting matrix
 printMatrix(MATRIX, R, C);

 if(!(extract_type))
   cout << "\nSIMPLE Transposition alphabet is: " << cipher << endl
                      << endl;
 else
   cout << "\nNUMERIC Transposition alphabet is: " << cipher << endl
                      << endl;

 return;
}//end reviseCipherStream()

//----------------------------------------------------------------
//Function: shiftKeyword-()
//Parameters: ctext - the cipher alphabet we are going to shift
//      user_key - the key the user entered
//Return Type: None
//Purpose: Shift the keyword or keyword phrase we will use later to
//encode the user's message.
//----------------------------------------------------------------
void shiftKeyword(char ctext[], const char USER_KEY)
{
 int location;
 char temp[TWENTYSIX] = {'\0'};

 //find the location of the key in the plaintext
 for(int ix=0; ix<TWENTYSIX; ix++){
   if(USER_KEY == ctext[ix]){
     location = ix;
     break;
   }
 }

 if(location == TWENTYFIVE){
    //do nothing
 }
 else {

   //put into temp all the characters up to and including the shift key
   //location now indicated how many characters, so we must increment by
   //one since the array uses a zero-based index
   strncpy(temp, ctext, location+1);
   //shift all remaining characters in the ciphertext
   strcpy(ctext, (ctext+location+1));
   //concatenate temp back to ctext
   strcat(ctext, temp);
 }

 cout << "Shifted keyword-mixed alphabet is: " << ctext << endl << endl;

 return;
}//end shiftKeyword();

//----------------------------------------------------------------
//Function: simpleExtraction()
//Parameters: MATRIX - the matrix we want to form the cipher from
//      cipher - the cipher stream we will create
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Creates the cipher stream we will use to encrypt files
//by pulling off the columns from left to right and inserting them
//into the cipher array.
//----------------------------------------------------------------
void simpleExtraction(char **& MATRIX, char cipher[], const int R,
                           const int C)
{
 int counter = 0;

 for(int ix=0; ix<C; ix++){
   for(int kx=0; kx<R; kx++){
     if((*(*(MATRIX+kx)+ix)) != '\0'){
       cipher[counter++] = (*(*(MATRIX+kx)+ix));
   }
   }
 }

 return;
}//end simpleExtraction()

//----------------------------------------------------------------
//Function: typeOfExtraction()
//Parameters:  None
//Return Type: int - 0 denotes SIMPLE extraction, 1 NUMERIC
//Purpose: Determines what type of extraction the user wants to
//perform.
//----------------------------------------------------------------
int typeOfExtraction()
{
 char type[SIZE];
 int ret_value;

 do {
  cout << "Enter transposition matrix method - SIMPLE or NUMERIC: ";
  cin >> type;
 } while (strcmp(type, "SIMPLE") && strcmp(type, "NUMERIC"));

 if(!strcmp(type, "SIMPLE"))
   ret_value = 0;
 else
   ret_value = 1;

 return ret_value;
}//end typeOfExtraction
//end file ciphertr.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHER6.BAS Program

To illustrate the execution of the interval extraction subroutine, I'll incorporate that routine into a program which enciphers messages based upon any one of the three alphabet formation techniques presented earlier in this chapter—interval extraction, simple transposition, and numeric transposition. This program, which is contained on the file CIPHER6.BAS on the companion CD-ROM, contains extraneous PRINT statements highlighted with five asterisks as comments which were included to facilitate explaining the operation of the program. You may want to remove those statements from the program if you wish to use it or distribute it to associates for use.

Listing 4.6 contains the statements that make up the main portion of the CIPHER6.BAS program. The first group of statements starting with the label "START" simply displays the program's options, accepts the user-specified enciphering method, and assigns a value to the string variable ORDER$ based upon the type of enciphering technique selected.

Listing 4.6 The main portion of the CIPHER6.BAS program.

REM PROGRAM CIPHER6.BAS
DIM PLAINTEXT$(26), CIPHERTEXT$(26), KEY$(26), TRANS$(26, 26), TEMP$(26)
START:  CLS
    PRINT "PROGRAM CIPHER6.BAS ENCIPHERS A MESSAGE USING ONE"
    PRINT "OF THE FOLLOWING TECHNIQUES TO FORM A CIPHER ALPHABET"
    PRINT " (1) INTERVAL EXTRACTION SEQUENCE"
    PRINT " (2) SIMPLE TRANSPOSITION SEQUENCE"
    PRINT " (3) NUMERIC TRANSPOSITION SEQUENCE"
    PRINT
    INPUT "Enter method to form enciphered message 1, 2 or 3: ", X
    IF X < 1 OR X > 3 THEN GOTO START
    IF X = 2 THEN ORDER$ = "SIMPLE"
    IF X = 3 THEN ORDER$ = "NUMERIC"
    IF X > 1 THEN GOTO TRAN
    INPUT "Enter the interval as a numeric between 1 and 25: ", INTERVAL
TRAN:   INPUT "Enter keyword or keyword phrase in UPPERCASE: ", TEXT$
    PRINT
GOSUB INITIALIZE
    PRINT "Plaintext based alphabet is   : ";        '*****
    FOR I = 0 TO 25: PRINT PLAINTEXT$(I);: NEXT I    '*****
GOSUB KEYWORD                'form keyword based mixed alphabet
    PRINT "Keyword based alphabet is     : "; X$     '*****
ON X GOSUB INTERVAL, TRANSPORT, TRANSPORT
    IF X = 1 THEN PRINT "Interval Alphabet is              : "; X$
    IF X > 1 THEN PRINT ORDER$; " Transposition Alphabet is :"; X$
1   INPUT "Enter UPPERCASE Alphabetic Shift Key: ", K$
    FOR I = 0 TO 25
    IF K$ = PLAINTEXT$(I) GOTO 2
    NEXT I
    PRINT "You must enter a letter from A to Z"
    GOTO 1
2   REM Position I represents shift key letter

GOSUB FORMCIPHER          'create cipher alphabet
    PRINT "Shifted keyword mixed alphabet is      : ";        '*****
    FOR I = 0 TO 25: PRINT CIPHERTEXT$(I);: NEXT I: PRINT     '*****
GOSUB INITIALIZE          'reinitialize plaintext array
GOSUB MSGFILE             'assign I/O files, place message on a file
GOSUB CONVERTSTORE        'convert and store ciphertext on a file
GOSUB PRTOUT              'print results
STOP

If an interval extraction technique is selected, the user is prompted to enter the interval to be used. Regardless of the enciphering method selected, the user is then prompted to enter a keyword or keyword phrase that will be used to form a keyword-based mixed alphabet. Both the plaintext alphabet and keyword-based mixed alphabet are displayed through the use of extraneous PRINT statements you can remove. You should note that the subroutines INITIALIZE and KEYWORD, as well as the other subroutines invoked from the main portion of CIPHER6.BAS, contain the statements previously described in this book and will not be reviewed again.

The ON X GOSUB statement causes a branch to the INTERVAL or TRANSPORT subroutine based upon the user-selected encipherment method that was assigned to the variable X. After one of those subroutines is invoked, the appropriate alphabet is displayed by the use of extraneous IF-THEN PRINT statements that can be removed from the program. The program accepts an alphabetic shift key, invokes the FORMCIPHER subroutine, and displays the shifted keyword mixed alphabet which, in effect, is the cipher alphabet. Again, you can remove the extraneous PRINT statements if you so desire. Thereafter, the program invokes four more subroutines that reinitialize the plaintext array, assign I/O files and place a message on a file, convert and store the ciphertext on a file, and print the results of the encipherment operation.

Figure 4.8 illustrates the execution of CIPHER6.BAS when an interval extraction technique is used to form a cipher alphabet. In this example, the keyword MICROSOFT was used to form the keyword-based alphabet. It is that alphabet that is used as the basis for forming the interval alphabet. Because 3 was entered as the extraction interval, every third character in the keyword-based alphabet is removed to form the interval alphabet. Thus, C is extracted, followed by S, and so on from the keyword-based alphabet.

Figure 4.8 The execution of CIPHER6.BAS.

The selection of B as the shift key causes the interval alphabet to be rotated so that the character B in the alphabet is positioned at the extreme right. The resulting shifted keyword mixed alphabet formed using an interval extraction method represents the ciphertext alphabet. Thus, first locate each character in the message in the plaintext alphabet and then extract the character in the equivalent position in the shifted-keyword mixed alphabet.

Note that the message WE MUST MEET… results in the enciphered message MYWUJ NWYYN… You can verify the accuracy of the encipherment process by noting that the position of W in the plaintext message is equivalent to M in the shifted keyword mixed alphabet, E has the same position as Y, and so on, verifying the accuracy of the enciphering process.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The CIPHER6.CPP Program

Continuing our presentation of C++ versions of each BASIC language version, Listing 4.7 contains the program listing of CIPHER6.CPP. As you review the program listing, you will note that an extensive effort was made to make the program as "crash-free" as possible, as well as to incorporate the functionality of the BASIC language version of the program. When you examine the program listing, you may wish to focus your attention upon the functions interval, which creates the cipher stream based on interval extraction, and numericExtraction, which creates the cipher stream used to encrypt files by pulling off the columns from the matrix in alphabetical order based on the character in the top row and inserting them into the cipher array. Although many of the previously developed C++ functions are included once again in this program listing, the listing is presented in its entirety to facilitate reference to the relationship between functions as well as the manner by which they operate.

Listing 4.7 The CIPHER6.CPP program listing.

/*
cipher6.cpp
C++ code written by Jonathan Held using Microsoft Visual C++,
version 5.0, on March 24, 1998.
*/

//standard include files
#include<iostream.h>
#include<assert.h>
#include<string.h>
#include<ctype.h>
#include<fstream.h>

//function prototypes
bool checkInput(char * &);
void createCipherStream(char *, char[]);
void createMatrix(char ** &, const char [], const int, const int);
void display(char *);
bool encryptText(char *, char *, const char [], const char[], char []);
void deleteMatrix(char **&, const int, const int);
int findLowestValue(int *&, const int);
void formatData(char []);
char* formCipheredMessage(const char[], const char [], char []);
void getFileNames(char *&, char *&);
int getInputType(void);
int getKeyword(char *&);
bool getMessage(char*, char*, char [], const char[], const char[]);
void getShiftKey(char &);
void groupBUFFER(ofstream, int);
void interval(char[]);
void numericExtraction(char **&, char [], const int, const int);
void printCipherToFile(ofstream, char[]);
void printMatrix(char ** &, const int, const int);
void reviseCipherStream(char **&, char [], const int, const int,
                           const int);
void shiftKeyword(char [], const char);
void simpleExtraction(char **&, char [], const int, const int);
int welcome(void);

//constants we will use
const int ONE = 1, TWO = 2, THREE = 3, FIVE = 5, TWENTYFIVE = 25,
      TWENTYSIX = 26, TWENTYSEVEN = 27, SIXTYFIVE = 65,
      NINETY = 90, NINETYTWO = 92, SIZE = 256, DUMMY_VAL = 999,
      BIGSIZE = 1000;
char BUFFER[BIGSIZE] = {'\0'};
//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 execution is normal, 1 abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main()
{
 char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                  'I','J','K','L','M','N','O','P',
                        'Q','R','S','T','U','V','W','X',
                                        'Y','Z', '\0'};

 char cipherStream[TWENTYSEVEN] = {'\0'};
 char message_to_cipher[SIZE], enciphered_message[SIZE];

 char ** cMatrix;
 char *keyword, key, *infile, *outfile;
 int rows, columns, input_type, encipher_type;
 bool success;

 encipher_type = welcome();

 if(encipher_type == ONE){
    getKeyword(keyword);
    createCipherStream(keyword, cipherStream);
    cout << "Keyword-based alphabet is: "
         << cipherStream << endl << endl;
    interval(cipherStream);
 }
 else {
   //get the keyword we are going to use, determine number of rows
   //and columns of our matrix
   columns = getKeyword(keyword);
   rows = TWENTYSIX/columns;

   //integer division requires that we check and see if there is a
   //remainder; if so, we need to add one extra row to our matrix
   if(TWENTYSIX%columns){
     rows++;
   }

   //create the initial ciphertext stream
   createCipherStream(keyword, cipherStream);

   //echo to the user what we have
   cout << "Plaintext-based alphabet is: " << plaintext << endl
        << "Keyword-based alphabet is:   " << cipherStream << endl
                       << endl;

   //insert the stream into our matrix
   createMatrix(cMatrix, cipherStream, rows, columns);

   reviseCipherStream(cMatrix, cipherStream, rows, columns,
                   encipher_type);

 }

 getShiftKey(key);
 shiftKeyword(cipherStream, key);

 getFileNames(infile, outfile);

 input_type = getInputType();

 //process file input
 if(input_type){
   success = encryptText(infile, outfile, plaintext, cipherStream,
                 enciphered_message);
 }
 else {
   cout << "Use a \'/\' to leave a line in plaintext." << endl
     << "Use a \'\\' to indicate end of message input. " << endl;
   success = getMessage(infile, outfile, message_to_cipher, plaintext,
    cipherStream);
 }

 //report success of operation
 if(!success){
   cerr << "Error: Invalid filename specified. Goodbye." << endl;
 }
 else {
   cout << "Press return to display resulting enciphered message."
                    << endl;
   //get the newlines off the current input stream
   cin.get();
   if(input_type == ONE){
     cin.get();
   }
   display(outfile);
 }

 //delete dynamically allocated memory accordingly
 if(encipher_type != ONE)
   deleteMatrix(cMatrix, rows, columns);

 delete [] infile;
 delete [] outfile;
 delete [] keyword;

 return (!success);
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
 bool error = false;
 int count = strlen(input);

 for(int ix=0; ix<count; ix++){

   int char_value = static_cast<int>(*(input+ix));
   //determine if the user did not enter an uppercase character
   if((char_value < SIXTYFIVE) || (char_value > NINETY)){
     error = true;
     cerr << "You entered an invalid keyword!" << endl << endl;
     break;
   }
 }

 if(count == 0){
   cerr << "You entered an invalid keyword!" << endl << endl;
   error = true;
 }

 return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: createCipherStream()
//Parameters: input - the keyword the user entered
//      cipher - the keyword alphabet that will be constructed
//Return Type: None
//Purpose: Creates a preliminary cipher stream that will be used to
//form the cipher matrix.
//----------------------------------------------------------------
void createCipherStream(char *input, char stream[])
{
 bool used[TWENTYSIX];
 int index = 0,
   count = strlen(input);

 //no characters are initially used
 for(int ix=0; ix<TWENTYSIX; ix++){
   used[ix] = false;
 }

 //keep track of each character used, start forming the keyword
 //alphabet
 for(int jx=0; jx<count; jx++){

   //get each character of the input string (integer value)
   int char_value = static_cast<int>(*(input+jx));

   if(used[char_value-SIXTYFIVE]){
     //do nothing - the character was already used
   }
   else {
     //mark as used and add to the keyword alphabet
     used[char_value-SIXTYFIVE] = true;
     *(stream+index++) = static_cast<char>(char_value);
   }
 }

 //go through the list of characters used - those which weren't
 //used should be added to the keyword alphabet
 for(int kx=0; kx<TWENTYSIX; kx++){

   if(!(used[kx])){
     *(stream+index++) = static_cast<char>(SIXTYFIVE+kx);
   }
 }

 return;
}//end createCipherStream()

//----------------------------------------------------------------
//Function: createMatrix()
//Parameters: matrix - the matrix we are going to create
//      CSTREAM - the initial cipher stream based on the
//          user's keyword
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Creates a numeric key transposed matrix, identical to
//figure 4.4.
//----------------------------------------------------------------
void createMatrix(char ** &matrix, const char CSTREAM[], const int ROWS,
                    const int COLS)
{
 int count = 0;

 //dynamically allocate memory for the RxC matrix
 //we use assert to ensure that memory was allocated;
 //if not, then the program will terminate abnormally
  assert(matrix = new char*[ROWS]);
  for(int ix=0; ix<ROWS; ix++){
    *(matrix + ix) = new char[COLS];
  }

  //fill in the matrix
  for(int jx=0; jx<ROWS; jx++){
    for (int kx=0; kx<COLS; kx++){
    //we only want to enter a character into the matrix
    //twenty-six times - most of the time we allocate
    //a matrix larger than what we will need to use; when
    //we have more than 26 characters, we simply insert a
    //null terminator into the matrix
      if(count < TWENTYSIX)
        *(*(matrix+jx)+kx) = CSTREAM[count++];
            else
        *(*(matrix+jx)+kx) = '\0';
    }
 }

 return;
}//end createMatrix()

//----------------------------------------------------------------
//Function: encryptText()
//Parameters:  inp_file - the name of the input plaintext file
//       outp_file - the name of the output ciphertext file
//       PTEXT[] - the plaintext alphabet
//       CTEXT[] - the ciphertext alphabet
//       encoded_msg[] - the message to be encoded
//Return Type: bool, indicating success of operation
//Purpose: Used to encrypt file input.  Takes each line of the input
//file, encrypts it, and saves the result to the specified output
//file.
//----------------------------------------------------------------
bool encryptText(char * inp_file, char * outp_file, const char PTEXT[],
          const char CTEXT[], char encoded_msg[])
{
 bool success = false;
 char ip[SIZE];

 //declare file stream objects
 ifstream input(inp_file, ios::in);
 ofstream output(outp_file, ios::app);

 if((!input) || (!output)){
   //do nothing - I/O error; user will be notified upon
   //procedure's return to main()
 }
 else {

   success = true;

   //print plaintext and ciphertext alphabets to the
   //output file
   output << "PLAINTEXT:  " << PTEXT << endl;
   output << "CIPHERTEXT: " << CTEXT << endl << endl;

   while (input.getline(ip, BIGSIZE, '\n')){
   //check to see if the user wants the line to appear in plain text
   if(ip[0] == '/'){

     if (strlen(BUFFER)>0){

         //empty whatever is in the buffer
         groupBUFFER(output, strlen(BUFFER));
     //adjust the buffer
        strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
     //output plaintext
     }

   output << ip << endl;

   }
   else {
     //encipher the line
     char *msg = formCipheredMessage(CTEXT, ip, encoded_msg);
     //print the cipher in groups of five to the ouput file
     printCipherToFile(output, msg);
    }
   }

   //empty the rest of the buffer
   groupBUFFER(output, strlen(BUFFER));

   //notify user where plaintext and ciphertext files are
   cout << "Plaintext file is: " << inp_file << endl;
   cout << "Encrypted file is: " << outp_file << endl << endl;
 }

 //don't forget to close the files
 input.close();
 output.close();

 //return success of the operation
 return success;
}//end encryptText()

//----------------------------------------------------------------
//Function: deleteMatrix()
//Parameters: matrix - the matrix we are going to destroy
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Destroys the dynamically allocated matrix!
//----------------------------------------------------------------
void deleteMatrix(char **&matrix, const int R, const int C)
{
 for(int ix=0; ix<R; ix++)
   delete [] *(matrix+ix);

 delete [] matrix;

 return;
}//end deleteMatrix()

//----------------------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echoes the resulting output file to the screen.
//----------------------------------------------------------------
void display(char *name)
{
 ifstream infile(name, ios::in);
 char input[SIZE];

 if(!(infile)){
   cerr << "Unable to open input file for display." << endl;
 }
 else {
   while (infile.getline(input, SIZE, '\n')){
    cout << input << endl;
   }
 }

 return;
}//end display()

//----------------------------------------------------------------
//Function: findLowestValue()
//Parameters: values - the ASCII values of all characters in the
//            top row of the matrix
//      COLS - the number of columns in the matrix
//Return Type: int - the column we want.
//Purpose: Determines what column we are going to extract from
//the matrix and put into the cipher stream.
//----------------------------------------------------------------
int findLowestValue(int *&values, const int COLS)
{
 int loc=0, lowest = DUMMY_VAL;

 for(int ix = 0; ix < COLS; ix++){
   if(*(values+ix) != DUMMY_VAL){
     if(*(values+ix) < lowest){
       lowest = *(values+ix);
       loc = ix;
     }
   }
 }

 *(values+loc) = DUMMY_VAL;

 return loc;
}//end findLowestValue()

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[]){

 for(int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
   if(*(data+nx) == ' '){
      //do nothing - skip over the space in the data
   }
   else {
     *(data+mx++) = *(data+nx);
   }
 }

 //don't forget to add the null terminator
 *(data+mx) = '\0';

 return;
}//end formatData()

//----------------------------------------------------------------
//Function: formCipheredMessage()
//Parameters:  CTEXT - the cipher alphabet we will use for substitution
//      MESSAGETOCIPHER - the user's message
//      enc_message - the enciphered message to be determined
//Return Type: char* - a pointer to the encoded information.
//Purpose: Encipher the user's message.
//----------------------------------------------------------------
char* formCipheredMessage(const char CTEXT[], const char
               MESSAGETOCIPHER[], char enc_message[])
{
 int length = strlen(MESSAGETOCIPHER)+1;

 int encode_value;

 for(int ix=0; ix<length; ix++){

   //test to see if we have an alphabetic character; if not,
   //simply copy it to our encrypted message - this preserves
   //characters such as ', ! etc...
   if(!isalpha(static_cast<int>(MESSAGETOCIPHER[ix]))){
     enc_message[ix] = MESSAGETOCIPHER[ix];
   }
   else {
     //valid character - the easy way to calculate the ciphered
     //character is based on the plain text's ascii character value;
     //since it has to be a capital letter, it must be in the range
     //from 65 to 90, with A represented by 65, Z by 90.  By simply
     //subtracting 65 from the encode_value (the integer representation
     //of the plaintext character), we now know what cipher character
     //to use.

     encode_value = toupper(static_cast<int>(MESSAGETOCIPHER[ix]));
     enc_message[ix] = CTEXT[encode_value-SIXTYFIVE];
   }
 }

 //return a reference to the encoded message
 return enc_message;
}//end formCipheredMessage()

//----------------------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//----------------------------------------------------------------
int getInputType(void)
{
 char type;
 bool error = false;
 int value;

 do {
  //prompt user for input from file or keyboard
  cout << "Is file input from keyboard (K, k) or file (F, f): ";
  cin >> type;

  //make type an uppercase letter
  type = static_cast<char>(toupper(static_cast<int>(type)));

  //check for an invalid character
  if((type != 'K') && (type != 'F')){
     cerr << "You have entered an invalid character!" << endl << endl;
     error = true;
  }
  else {
    if(type == 'K')
      value = 0;        //value of 0 represents keyboard input
      else value = 1;   //value of 1 represents file input
    error = false;
  }

 } while (error);

 cout << endl;

 return value;
}//end getInputType()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
//      outfile_name - the output file we will write the
//      enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//----------------------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
 char data[SIZE];

 cout << "Enter filename to store/retrieve plaintext message: ";

 cin >> data;

 infile_name = new char[strlen(data) + 1];
 strcpy(infile_name, data);

 cout << "Enter filename to store enciphered message: ";

 cin >> data;

 outfile_name = new char[strlen(data) + 1];
 strcpy(outfile_name, data);

 cout << endl;

 return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: int - the length of the keyword
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.  Returns the length of the
//keyword.
//----------------------------------------------------------------
int getKeyword(char * &text)
{
 bool error = false;
 char buffer[SIZE];

 do {
   cout << "Enter keyword or keyword phrase in UPPERCASE" << endl
            << "do not use spaces or non-alphabetic characters): ";

   cin.getline(buffer, SIZE, '\n');
   assert(text = new char[strlen(buffer) + 1]);
   strcpy(text, buffer);

   error = checkInput(text);

   //delete text if there was an error
   if(error){
     delete [] text;
   }

 } while (error);

 cout << endl;

 return strlen(buffer);
}//end getKeyword()

//----------------------------------------------------------------
//Function: getMessage()
//Parameters:  input - the name of the input plaintext file
//      output the name of the output ciphertext file
//      msg_to_cipher - the message to be encoded
//      PTEXT[] - the plaintext alphabet
//      CTEXT[] - the ciphertext alphabet
//Return Type: bool, indicating success of operation
//Purpose: Allow the user to manually input text from the keyboard.
//Save the text in plaintext to the input file; encrypt the text
//and save it to the specified output file for later retrieval.
//----------------------------------------------------------------
bool getMessage(char* input, char* output, char msg_to_cipher[],
                  const char PTEXT[],
         const char CTEXT[])
{
 bool go_on = true, success = false;

 ofstream textFile(input, ios::app);
 ofstream cipherFile(output, ios::app);
 if((!textFile) || (!cipherFile)){
   //do nothing - error will be noted to user later
 }
 else {

   success = true;

   textFile << "PLAINTEXT:  " << PTEXT << endl;
   textFile << "CIPHERTEXT: " << CTEXT << endl << endl;

   //get the newline character off of the input stream
   cin.get();

   cout << "Enter the message in UPPERCASE characters: " << endl;

   while (go_on) {

    //get the entire line, up to 256 characters
    cin.getline(msg_to_cipher, SIZE, '\n');

    //case user doesn't want the text to be encrypted
    if(msg_to_cipher[0] == '/'){

      if(strlen(BUFFER)>0){
        //empty whatever is in the buffer
        groupBUFFER(cipherFile, strlen(BUFFER));
        //adjust the buffer
        strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
      }

      //output plaintext
      textFile << msg_to_cipher << endl;
      cipherFile << msg_to_cipher << endl;
      }

    //case user is done entering text
    else if (static_cast<int>(msg_to_cipher[0]) == NINETYTWO){
      go_on = false;
      }

    //encrypt the text
    else {
      textFile << msg_to_cipher << endl;
      char enciphered_msg[BIGSIZE];
      formCipheredMessage(CTEXT,msg_to_cipher,enciphered_msg);
      printCipherToFile(cipherFile,enciphered_msg);
      }
    }

 //empty the rest of the buffer
 groupBUFFER(cipherFile, strlen(BUFFER));
 }

 //close the files
 textFile.close();
 cipherFile.close();

 //notify user where plaintext and ciphertext files are
 cout << "\nPlaintext file is: " << input << endl;
 cout << "Encrypted file is: " << output << endl << endl;

 return success;
}//end getMessage()

//----------------------------------------------------------------
//Function: getShiftKey()
//Parameters:  key_desired - uppercase key entered by the user
//Return Type: None
//Purpose: Get the key the user enters; error checking performed
//until user enters a valid value.
//----------------------------------------------------------------
void getShiftKey(char &key_desired)
{
 bool error = true;

 do {
  //prompt user to enter an uppercase shift key
  cout << "Enter UPPERCASE Alphabetic Shift Key (CTRL-C to quit): ";
  cin >> key_desired;

  int key_value = static_cast<int>(key_desired);

  //do some error checking
  if((key_value < SIXTYFIVE) || (key_value > NINETY)){
    cerr << "\nYou must enter a letter from A to Z!" << endl << endl;
  }
  else {
    cout << endl;
    error = false;
   }
  } while (error);

  return;
}//end getShiftKey()

//----------------------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
//      num - the number of characters we want to output
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//----------------------------------------------------------------
void groupBUFFER(ofstream out, int num)
{

 for(int kx=0;kx<num;kx++){

   if((kx!=0) && (kx%TWENTYFIVE==0)){
     out << endl;
   }

   if((kx!=0) && (kx%FIVE == 0) && (kx%TWENTYFIVE!=0)){
      out << " " << *(BUFFER+kx);
   }
   else {
        out << *(BUFFER+kx);
   }
 }

 out << endl;

 return;
}//end groupBUFFER()

//----------------------------------------------------------------
//Function: interval()
//Parameters:  ctext - the cipher stream we are modifying
//Return Type: None
//Purpose: Create the cipher stream based on interval extraction.
//----------------------------------------------------------------
void interval(char ctext[])
{
  int interval, pos;
  char temp[TWENTYSEVEN];

  cout << "Enter the interval as a numeric between 1 and 25: ";
  cin >> interval;

  while ((interval < ONE) || (interval > TWENTYFIVE)){
   cerr << "\aInterval must be between 1 and 25." << endl;
   cerr << "Enter interval: ";
   cin >> interval;
  }

  for(int ix=0; ix<TWENTYSIX; ix++)
    temp[ix] = ctext[ix];

  for(int jx=0; jx<TWENTYSIX; jx++){
    pos = (((jx+1)*interval)-1)%TWENTYSIX;
    ctext[jx] = temp[pos];
  }

  cout << "\nInterval-based alphabet is " << ctext << endl << endl;

  return;
}//end interval()

//----------------------------------------------------------------
//Function: numericExtraction()
//Parameters: MATRIX - the matrix we want to form the cipher from
//      cipher - the cipher stream we will create
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Creates the cipher stream we will use to encrypt files
//by pulling off the columns from the matrix in alphabetical order
//based on the character in the top row, and inserting them into
//the cipher array.
//----------------------------------------------------------------
void numericExtraction(char **& MATRIX, char cipher[], const int R,
                           const int C)
{
  int counter = 0;

  //place ASCII values of first row's characters into a table;
  //we will use these values to determine the order in which
  //we pull columns out of the matrix
  int *top_row_ASCII_values = new int[C];
  assert(top_row_ASCII_values);

  //no cast to integer type required since this is already taken
  //care of for us
  for (int ix=0; ix<C; ix++){
    *(top_row_ASCII_values + ix) = *((*MATRIX)+ix);
  }

  for(int jx=0; jx<C; jx++){

    //find out what column we want
    int col = findLowestValue(top_row_ASCII_values, C);

    //put contents of the column into the cipher stream, but
    //only do this when we have a character!
    for(int kx=0; kx<R; kx++){
       if((*(*(MATRIX+kx)+col)) != '\0'){
         cipher[counter++] = (*(*(MATRIX+kx)+col));
       }
     }
  }

  //destroy dynamically allocated memory
  delete [] top_row_ASCII_values;

  return;
}//end numericExtraction()

//----------------------------------------------------------------
//Function: printCipherToFile()
//Parameters: op - the output file we are writing to
//            msg - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-block characters in the
//specified output file.
//----------------------------------------------------------------
void printCipherToFile(ofstream op, char msg[]){

  formatData(msg);

  //check to see if there are more than 25 characters
  //in the buffer; if so, print out as many groups of
  //25 as possible
  if(strlen(BUFFER) >= TWENTYFIVE){

     int numchars = (strlen(BUFFER)/TWENTYFIVE)*TWENTYFIVE;
     //print the contents of the buffer to the output stream
     groupBUFFER(op, numchars);
     //shift whatever is left in the buffer
     strcpy(BUFFER, (BUFFER+numchars));
     //append data to the buffer
     strcat(BUFFER, msg);
  }

  //if buffer contents are less than 25, simply append the new
  //data to the buffer
  else if ((strlen(BUFFER) >= 0) && (strlen(BUFFER) < TWENTYFIVE)){
    strcat(BUFFER, msg);
  }

  return;
}//end printCipherToFile()

//----------------------------------------------------------------
//Function: printMatrix()
//Parameters: the_matrix - the matrix we want to display
//      ROWS - the number of rows in the matrix
//      COLS - the number of columns in the matrix
//Return Type: None
//Purpose: Displays the matrix.
//----------------------------------------------------------------
void printMatrix(char ** &the_matrix, const int ROWS, const int COLUMNS)
{
  cout << "\nThe matrix is:" << endl << endl;

  for(int ix=0; ix<ROWS; ix++){

    for(int kx=0; kx<COLUMNS; kx++){
      cout << *(*(the_matrix+ix)+kx) << " ";
    }

    cout << endl;
  }

  return;
}//end printMatrix()

//----------------------------------------------------------------
//Function: reviseCipherStream()
//Parameters: MATRIX - the numeric keyed transposed matrix
//      cipher - the cipher stream that was originally created
//           when the user entered a keyword
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Overwrites the cipher stream initially created with the
//one we get by taking columns out of the matrix.
//----------------------------------------------------------------
void reviseCipherStream(char ** &MATRIX, char cipher[], const int R,
                const int C, const int enc_type)
{
  if(enc_type == TWO){
     simpleExtraction(MATRIX, cipher, R, C);
  }
  else {
    numericExtraction(MATRIX, cipher, R, C);
  }

  //print the resulting matrix
  printMatrix(MATRIX, R, C);

  if(enc_type == TWO)
    cout << "\nSIMPLE Transposition alphabet is: " << cipher << endl
              << endl;
  else
    cout << "\nNUMERIC Transposition alphabet is: " << cipher << endl
                      << endl;
  return;
}//end reviseCipherStream()

//----------------------------------------------------------------
//Function: shiftKeyword()
//Parameters: ctext - the cipher alphabet we are going to shift
//      user_key - the key the user entered
//Return Type: None
//Purpose: Shift the keyword or keyword phrase we will use later to
//encode the user's message.
//----------------------------------------------------------------
void shiftKeyword(char ctext[], const char USER_KEY)
{
  int location;
  char temp[TWENTYSIX] = {'\0'};

  //find the location of the key in the plaintext
  for(int ix=0; ix<TWENTYSIX; ix++){
     if(USER_KEY == ctext[ix]){
       location = ix;
       break;
     }
  }

  if (location == TWENTYFIVE){
          //do nothing
  }
  else {

     //put into temp all the characters up to and including the shift key
     //location now indicated how many characters, so we must increment by
     //one since the array uses a zero-based index
     strncpy(temp, ctext, location+1);
     //shift all remaining characters in the ciphertext
     strcpy(ctext, (ctext+location+1));
     //concatenate temp back to ctext
     strcat(ctext, temp);
  }

  cout << "Shifted keyword-mixed alphabet is: " << ctext << endl << endl;

  return;
}//end shiftKeyword();

//----------------------------------------------------------------
//Function: simpleExtraction()
//Parameters: MATRIX - the matrix we want to form the cipher from
//      cipher - the cipher stream we will create
//      R - the number of rows in the matrix
//      C - the number of columns in the matrix
//Return Type: None
//Purpose: Creates the cipher stream we will use to encrypt files
//by pulling off the columns from left to right and inserting them
//into the cipher array.
//----------------------------------------------------------------
void simpleExtraction(char **& MATRIX, char cipher[], const int R,
                           const int C)
{
  int counter = 0;

  for (int ix=0; ix<C; ix++){
     for (int kx=0; kx<R; kx++){
        if ((*(*(MATRIX+kx)+ix)) != '\0'){
           cipher[counter++] = (*(*(MATRIX+kx)+ix));
           }
     }
  }

  return;
}//end simpleExtraction()

//----------------------------------------------------------------
//Function: welcome()
//Parameters:  None
//Return Type: int - method the cipher stream will be created;
//1 is interval, 2 is simple, 3 is numeric
//Purpose: Allow the user to select which method he/she wants
//to use to create the cipher stream.
//----------------------------------------------------------------
int welcome()
{
  int type;

  cout << "Program cipher6.cpp enciphers a message using one" << endl
     << "of the following techniques to form a cipher alphabet:"
                        << endl << endl
        << "(1) Interval Extraction Sequence" << endl
        << "(2) Simple Transposition Sequence" << endl
        << "(3) Numeric Transposition Sequence" << endl << endl
        << "Enter method to form enciphered message 1, 2 or 3: ";

  cin >> type;

  while ((type <ONE) || (type > THREE)){
   cerr << "\aYou must enter a number between 1 and 3!" << endl;
   cerr << "Choice: ";
   cin >> type;
  }

  //get the newline off the stream
  cin.get();
  cout << endl;

  return type;
}//end welcome()
//end file cipher6.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Decipherment

You can easily construct a program to decipher messages previously enciphered through the use of an interval extraction, simple transposition, or numeric transposition sequence used to form a cipher alphabet. In doing so, you can use the subroutines contained in the previously created DCIPHER5.BAS program, change the main portion of the program to reflect the selection of one of the three decipherment techniques under consideration, and add the subroutines INTERVAL and TRANSPORT. The resulting programs, which are contained on the files DCIPHER6.BAS and DCIPHER6.CPP on the companion CD-ROM, are named as such since they are to be used to decipher messages previously enciphered through the use of the CIPHER6.BAS and CIPHER6.CPP programs.

The DCIPHER6.BAS Program

Listing 4.8 lists the statements in the main program of the DCIPHER6.BAS program. You should note that the main portion of DCIPHER6.BAS is almost an exact duplicate of CIPHER6.BAS, with the differences between programs related to the use of the term "decipher" in place of "encipher" and the inclusion of the subroutines DMSGFILE and DCONVERTSTORE in place of MSGFILE and CONVERTSTORE. As explained previously, the prefix D on a subroutine is used to indicate a modification to a previously developed subroutine which results in a deciphering operation in place of an enciphering operation. Similar to other programs in this chapter, a comment consisting of five asterisks is used to indicate an extraneous PRINT statement included for explanation purposes but which can be removed from the program if you so desire.

Listing 4.8 The main portion of the DCIPHER6.BAS program.

REM PROGRAM DECIPHER.BAS
DIM PLAINTEXT$(92), CIPHERTEXT$(36, 92), KEY$(92)
    CLS
    PRINT "DECIPHER.BAS PROGRAM deciphers text based upon the use"
    PRINT "of enciphering techniques contained in the book:"
    PRINT "ELECTRONIC MAIL PRIVACY:USING PRACTICAL ENCIPHERING
                TECHNIQUES"
    PRINT
    PRINT "This program supports the use of upper and lower case
                letters,"
    PRINT "digits, punctuation characters and other characters whose
                ASCII"
    PRINT "values range between 32 and 128, but EXCLUDES the use of the"
    PRINT "foward slash (/), backslash (\) and double quote characters."
    PRINT
    PRINT
AGN1:  INPUT "Enter your secret code (6 characters required): "; CODE$
    IF LEN(CODE$) = 6 THEN GOTO OK
    CLS
    PRINT "Your secret code must be 6 characters - please try again"
    PRINT
    GOTO AGN1
OK:    CLS
GOSUB INITIALIZE    'initialize plaintext values
GOSUB SETUP        'obtain random seed and position in seed
GOSUB DMSGFILE       'assign I/O files, place message on a file
GOSUB KEYWORD         'form keyword based alphabet of 96 characters
GOSUB PFORMCIPHER       'create 36 cipher alphabets
GOSUB RDCONVERTSTORE       'convert and store ciphertext on a file
GOSUB PRTOUT       'print results
STOP

To illustrate the use of DCIPHER6.BAS, let's execute CIPHER6.BAS. Figure 4.9 illustrates the plaintext message and its resulting enciphered message when an interval extraction sequence of 3, keyword of MAYDECEMBER, and alphabetic shift key of B are used to form the shifted keyword mixed alphabet.

Figure 4.9 Enciphering a sample message using the CIPHER.BAS program.

Figure 4.10 illustrates the use of the DCIPHER6.BAS program to decipher our previously enciphered message. Note that the message recipient must know the type of technique used to encipher the message, the keyword or keyword phrase, and the alphabetic shift key. In addition, when an interval extraction technique is used, the recipient must also know the interval used for extraction.

Figure 4.10 Deciphering the message created using CIPHER6.BAS through the use of the DCIPHER.BAS program.

Because the author of this book is not a good typist, file input was selected in place of keyboard input. Because the program was created using the same PRTOUT subroutine used by enciphering programs to display the resulting deciphered message, that message is displayed in groups of five characters.

The inclusion of PRINT statements to display the four alphabets developed by the program can be used to trace the operation of the program. In addition, you can use the positional relationship between the plaintext alphabet characters and the characters in the shifted keyword mixed alphabet to verify the deciphering of the message contained in Figure 4.9 into the plaintext shown at the bottom of Figure 4.10. For example, the first group of five enciphered characters in Figure 4.9 is YOUGI. Locating Y in the shifted keyword mixed alphabet in Figure 4.10 and reading upward to the plaintext alphabet results in the extraction of the character P. Similarly, O becomes L, U becomes E, G becomes A, and I becomes S. Thus, if you desire you can easily verify the operation of the program. Once you are satisfied with its operation or do not wish to probe its operation, you can remove the extraneous print statements that display the different alphabets.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DCIPHER6.CPP Program

Similar to the program DCIPHER6.BAS, its C++ version which is stored on the CD-ROM under the filename DCIPHER6.CPP has a large number of functions used in the same manner as its encipher version. Thus, in this section we will focus our attention on examining the function that provides the program with its decipherment capability. That function is the decipher function whose statements are given in Listing 4.9. Listing 4.9 includes a number of built-in comments that define the logic of the program. You can find the complete program and the executable version on the CD-ROM in the C directory.

Listing 4.9 The C++ decipher function used in the program DCIPHER6.CPP.

//----------------------------------------------------------------
//Function: decipher()
//Parameters: in - the input file we are deciphering
//      out - the output file we are writing the deciphered
//         text to
//      PTEXT - the plaintext alphabet (used to decipher)
//      ctext - the ciphertext
//Return Type: None
//Purpose: Decipher the input file and write the contents to the
//output file specified by the user.
//----------------------------------------------------------------
void decipher(ifstream in, ofstream out, const char PTEXT[], char ctext[])
{
  char enc_file_data[SIZE];

  //continue this process until we get to the end of the file
  while (in.getline(enc_file_data, SIZE, '\n')){
     if (enc_file_data[0] == '/'){
        out << enc_file_data << endl;
     }
     else {

     //format the data - i.e., get rid of all spaces
          formatData(enc_file_data);

        //dump data to file
        for (int ix=0; ix<strlen(enc_file_data); ix++){

        //used to keep track of what plaintext character
          //we are going to use
          int jx;

          for (jx=0; jx<TWENTYSIX; jx++){
          //find where the encrypted data is in the
          //ciphertext - this location corresponds to
          //the plaintext character location
            if (enc_file_data[ix] == ctext[jx])
            break;
          }

     //conditionals for grouping by five and inserting
        //new lines
        if (!(ix%TWENTYFIVE))
           out << endl;

        if ((ix!=0) && (!(ix%FIVE))){
           out << " " << PTEXT[jx];
        }
        else {
          out << PTEXT[jx];
        }
       }
    }
  }
  return;
}//end decipher()

Monoalphabetic Combinations

Regardless of the technique or techniques used to form a monoalphabetic cipher alphabet, the maximum number of trials necessary to correctly decipher a message remains the same. That number is 26!, which is approximately 4.0329E+26—a very large number. However, that large number can be very deceptive, as it is relatively easy for a trained cryptanalyst to decipher a lengthy message by a frequency analysis of the characters in the enciphered message. For example, E is the most common letter in the English language, followed by the letter T, which has the next highest frequency of occurrence. By performing a frequency analysis of the characters in an enciphered message a trained analyst may be able to note that E was replaced by one character, T by another, and so on. By deciphering a few characters correctly, patterns may become visually identifiable. For example, deciphering three enciphered characters as EET may provide the analyst with a clue that the character prefixing the first deciphered E is the plaintext M. This monoalphabetic substitution weakness means you should keep messages relatively short to hinder the possibility of a frequency analysis being used as a wedge by a cryptanalyst to initiate the deciphering of your message. This weakness also resulted in the development of polyalphabetic substitution encipherment and pseudo-random encipherment techniques—topics I will cover in the next two chapters of this book.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

One of the more interesting matrix-based enciphering techniques operates upon pairs of characters to reduce the potential for a frequency analysis providing an easy mechanism to decipher messages. Although referred to as the Playfair cipher after Lyon Playfair, a scientist of Victorian England and the Deputy Speaker of the House of Commons, in actuality, the cipher was developed by Playfair's friend and close associate Charles Wheatstone.

The Playfair cipher system is based upon the formation of a keyword mixed alphabet that is contained in a 5x5 matrix, where the letters I and J are regarded as identical and combined to reduce the English alphabet to 25 letters. Thus, the use of the keyword QUALE would result in the following alphabetic matrix being used for encipherment.

Q   U     A   L   E
B   C     D   F   G
H   I/J   K   M   N
O   P     R   S   T
V   W     X   Y   Z

The encipherment of messages occurs using pairs of letters—a technique referred to as digraphic encipherment. The rules governing the encipherment of each pair of characters is based upon the relationship of the letters to one another in the keyword-based matrix. Because there are only three possible relationships between letters in the matrix, there are three rules that govern encipherment.

If a pair of letters falls within the same row, they are replaced by the letters to their right, with each row considered cyclical so that the letter to the right of the last letter in a row becomes the first letter in the row. Thus, the pair of letters ST would be replaced by TO, while QU would be replaced by UA.

The second rule governing the encipherment of pairs of letters is applicable to letters that appear in the same column. Similar to the rule governing the replacement of pairs of letters contained in the same row, letters that appear in the same column are replaced by the letter beneath each letter. Here, the cyclical provision results in the letter below the last letter in the column being the letter at the top of the column. Thus, DR would be enciphered as KX, while ZE would be enciphered as EG.

The third rule governing encipherment is applicable to the situation in which a pair of plaintext letters is not located in a common row or column. When this occurs, each letter is replaced by the letter located in its own row at the column position occupied by the other plaintext letter. Thus, under this rule DE would be replaced by GA, while TH would be replaced by ON.

To further reduce the ability of frequency analysis as a decipherment tool, any double letter occurring in a pair is separated by the use of an X. Thus, HOLLY would become HOLXLY if the pair of L letters was on a pair boundary.

The use of digraphs by the Playfair cipher extends the cipher combinations from 26 letters when a one-for-one monoalphabetic substitution process is used to either 625 or 676 digraphs, depending upon the manner in which the I/J letter combination is used. Not only does this technique provide a substantial expanded base of characters for a frequency analysis, but it results in forcing an analysis to be performed on letter pairs whose frequency of occurrence has substantially less divergence than single letters. For example, the two most common letters in the English language, E and T, have an average frequency of approximately 12 percent and 9 percent, respectively. In comparison, the two most common letter pairs, TH and HE, have a frequency of occurrence of 3.5 percent and 2.5 percent.

During World War II, the Playfair cipher was used by Australian coast watchers hiding on many Japanese occupied islands in the South Pacific. Using the keys ROYAL NEW ZEALAND NAVY and PHYSICAL EXAMINATION, messages were transmitted concerning the loss of PT 109 which led to the rescue of Lieutenant John F. Kennedy and his PT boat crew.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 5
Polyalphabetic Substitution

The conclusion of Chapter 4, Transposition-Based Monoalphabetic Substitution, briefly discussed a major weakness of monoalphabetic substitution-based enciphering techniques. That weakness is its susceptibility to frequency analysis: A relatively long message can be used as a guide for a trial-and-error process in which the most commonly occurring letters in a plaintext alphabet are substituted for the most frequently occurring characters in the enciphered message. A trained cryptanalyst can use this technique as a wedge to further deciphering operations because the correct substitution of one plaintext character for an enciphered character is carried through the entire enciphered message when a monoalphabetic substitution process is employed.

One method used to overcome the weakness of a monoalphabetic substitution system is the use of a polyalphabetic substitution system. Unlike a monoalphabetic substitution system in which each plaintext character is mapped to a fixed ciphertext character, a polyalphabetic substitution system permits each plaintext character to be mapped into a different ciphertext character for a specified number of occurrences prior to being mapped back into its original ciphertext character. The number of different ciphertext characters each plaintext character can be mapped into depends upon the number of ciphertext alphabets used. The number of ciphertext alphabets used defines the period or cyclic redundancy of repetition of the mapping process.

Although a manual enciphering process becomes more difficult to perform as the number of ciphertext alphabets increases, the use of computers significantly reduces this difficulty. This chapter first examines the construction and utilization of several polyalphabetic substitution systems. Similar to previous chapters, this chapter shows you how to construct enciphering and deciphering programs to automate the use of polyalphabetic substitution systems to encipher and decipher messages. This chapter also discusses several variations you may wish to consider in developing different types of polyalphabetic substitution systems.

Lawyer and architect Leon Battista Alberti is considered by many to be the father of Western cryptology. The author of the most comprehensive manuscript on cryptanalysis for its time, Alberti is also credited with the invention of polyalphabetic substitution and enciphered code in the fifteenth century.

Alberti's efforts in the development of a polyalphabetic substitution system took the form of a disk constructed through the use of two copper plates. The circumference of each plate was divided into 24 equal parts known as cells. The larger copper plate, which was stationary, contained the letters of the alphabet entered into each cell in their alphabetic sequence. The letters H, K, and Y were not included as Alberti did not feel they were necessary. Because the letters J, U, and W were not in his alphabet, he inscribed the numbers 1 to 4 to complete the entries in the cells of the fixed plate.

The second plate, whose diameter was smaller than the stationary plate, had 24 cells into which the letters of the Latin alphabet were inscribed in a random order. This plate was movable and was placed over the larger plate, with both plates attached to one another by a needle affixed through the center of both plates. This resulted in the needle providing a relationship between the movable and fixed plate cell entries.

Alberti's cipher disk enabled the relationship between the plaintext alphabet contained on the circumference on the fixed plate and the ciphertext alphabet contained on the circumference on the movable plate to be altered. To do so simply required the placement of a letter in the movable disk so it was positioned toward a letter on the outer disk. Because each new setting of Alberti's disk created a new cipher alphabet, each setting resulted in the altering of the plaintext-ciphertext encipherment relationship and enabled encipherment to occur using a polyalphabetic substitution process. All that was required to successfully encipher and decipher messages was for each party to have identical disks and agree upon the initial index letter of the movable disk and a method to change the use of the index.

Simple Polyalphabetic Substitution Systems

One of the earliest polyalphabetic substitution systems is known as the Vigenére cipher after its inventor, Blaise de Vigénere. First published in 1586, the Vigenére cipher is based upon the development of a tableau in which all possible displaced alphabets are positioned one under another, commencing with the original alphabet in the top row.

The Vigenére Cipher

Figure 5.1 illustrates the basic Vigenére tableau which consists of 26 rows, with each row containing 26 letters. Note that the Vigenére tableau can be represented as a 26x26 matrix consisting of 676 elements. In fact, the tableau contained in Figure 5.1 was created by the execution of the program POLY1.BAS whose contents are shown in Listing 5.1.

Figure 5.1 Basic Vigenére tableau


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The POLY1.BAS Program

The POLY1.BAS program uses the previously developed INITIALIZE subroutine to initialize the letters of the alphabet into the string array PLAINTEXT$. The subroutine TABLE is used to fill the two-dimensional string array labeled TABLEAU$ with a sequence of 26 alphabets. Through the use of a nested pair of FOR-NEXT loops and a mod 26 operator, each row in the string array labeled TABLEAU$ is assigned an alphabet in which row n has a displacement of one character position with respect to row n-1. To illustrate the operation of the subroutine TABLE, note that when J is 0, I will vary from 0 to 25. Thus, the use of TABLEAU$(J,I) results in row 0 being assigned the values of PLAINTEXT$((I+0) mod 26), the non-displaced alphabet. Once the inner I loop is completed, J is incremented by 1 to a value of 1 and the I loop is again varied from 0 to 25. This results in TABLEAU$(J,I) having Row 1 assigned the alphabet displaced by one position, since PLAINTEXT$((I+J) mod 26) results in a value of 1 for J shifting the assignment of plaintext letters to TABLEAU$ by one position.

The subroutine PRINTIT was created to print the tableau as well as the row and column labels that represent vertically and horizontally positioned alphabets. This subroutine was developed simply to display the tableau, and its row and column positioning elements were used for the extraction of characters from the tableau during the enciphering process as we will shortly observe.

Listing 5.1 The POLY1.BAS program listing.

REM PROGRAM POLY1.BAS
DIM PLAINTEXT$(26), TABLEAU$(26, 26)
    CLS
GOSUB INITIALIZE
GOSUB TABLE
GOSUB PRINTIT
STOP
INITIALIZE:
    RESTORE
    REM Initialize plaintext values
    FOR I = 0 TO 25
    READ PLAINTEXT$(I)
    NEXT I
    DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
    DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
TABLE:
    FOR J = 0 TO 25
    FOR I = 0 TO 25
    TABLEAU$(J, I) = PLAINTEXT$((I + J) MOD 26)
    NEXT I
    NEXT J
RETURN
PRINTIT:
    PRINT "  ";
    FOR I = 0 TO 25: PRINT PLAINTEXT$(I);: NEXT I: PRINT: PRINT
    FOR J = 0 TO 25
    PRINT PLAINTEXT$(J); " ";
    FOR I = 0 TO 25
    PRINT TABLEAU$(J, I);
    NEXT I
    PRINT
    NEXT J
RETURN

The POLY1.CPP Program

To add a bit of diversity, the C++ program version of POLY1.BAS was written to display one line at a time, requiring you to hit Enter at the end of each line to proceed to the next line. Listing 5.2 contains the listing of the program POLY1.CPP. Both that program and the executable version of the program named POLY1.EXE are stored in the C directory on the companion CD-ROM.

Listing 5.2 The POLY1.CPP program listing.

/*poly1.cpp
C++ code written by Jonathan Held, March 26, 1998,
using Microsoft's Visual C++ version 5.0.
*/

//standard include files
#include <iostream.h>

//constant we will use
const int TWENTYSIX = 26, TWENTYSEVEN = 27, SIZE = 256;

//function prototypes
void initialize(char [][TWENTYSEVEN], const char []);
void printHeader(const char[]);
void printTable(const char [][TWENTYSEVEN]);

//----------------------------------------------------------------
//Function: main()
//Parameters:  None
//Return type: None
//Purpose: Demonstrates a simple Vigenere tableau.
//----------------------------------------------------------------
int main()
{
  char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                   'I','J','K','L','M','N','O','P',
                   'Q','R','S','T','U','V','W','X',
                                 'Y','Z','\0'};

  char table[TWENTYSIX][TWENTYSEVEN];

  initialize(table, plaintext);
  printHeader(plaintext);
  printTable(table);

  return 0;
}//end main()

//----------------------------------------------------------------
//Function: initialize()
//Parameters:  tbl - the two-dimensional Vigenere tableau
//       PTEXT - the plaintext alphabet
//Return type: None
//Purpose: Creates the Vigenere tableau by performing a shift on
//the plaintext alphabet for each Vigenere row entry.
//----------------------------------------------------------------
void initialize(char tbl[][TWENTYSEVEN], const char PTEXT[])
{
  for (int ix=0; ix<TWENTYSIX; ix++){
    for (int jx=0; jx<TWENTYSIX; jx++){
      tbl[ix][jx] = PTEXT[(ix+jx)%TWENTYSIX];
     }
     tbl[ix][TWENTYSIX] = '\0';
  }

  return;
}//end initialize()

//----------------------------------------------------------------
//Function: printHeader()
//Parameters:  PTEXT - the plaintext alphabet (also serves as the
//       Vigenere column header
//Return type: None
//Purpose: Prints the Vigenere column header and informs the user
//to hit enter after each line of the table is printed.
//----------------------------------------------------------------
void printHeader(const char PTEXT[])
{
  cout << "Hit enter at the end of each line" << endl
       << "to proceed to the next. " << endl << endl;

  cout << "\t";

  for (int ix=0; ix<TWENTYSIX; ix++)
    cout << PTEXT[ix];

  cout << endl << endl;
  return;
}//end printHeader()

//----------------------------------------------------------------
//Function: printTable()
//Parameters:  TABLE - the two-dimensional Vigenere tableau
//Return type: None
//Purpose: Creates the Vigenere tableau by performing a shift on
//the plaintext alphabet for each Vigenere row entry.
//----------------------------------------------------------------
void printTable(const char TABLE[][TWENTYSEVEN])
{
  char junk[SIZE];

  for (int ix=0; ix<TWENTYSIX; ix++){
    cout << TABLE[ix][0] << "\t";
    cout << TABLE[ix];

     //get any characters the user may have types off
     //the input stream and discard them
     cin.getline(junk, SIZE, '\n');
  }

  return;
}//end printTable()
//end file poly1.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Encipherment

To use the Vigenére cipher system, first select a keyword or keyword phrase. This keyword or keyword phrase, unlike those used in a keyword-based mixed alphabet, can retain repeating characters. For example, SECRET is a valid Vigenére cipher keyword, although it would be modified to SECRT if used to develop a keyword-based mixed alphabet.

Once the Vigenére keyword or keyword phrase is selected, it is repeated in tandem with the characters in the plaintext message as the plaintext message is encoded. On a repeating basis, one character from the plaintext message and one character from the keyword are used as row and column header indices. The location within the tableau where the indices intersect points to the ciphertext character which is extracted from the tableau.

Figure 5.2 illustrates an example of a Vigenére encipherment using the keyword SECRET. Due to the symmetrical structure of the tableau, you can use either the row or column for the plaintext character pointer and the opposite row or column for the keyword character pointer. For example, using the plaintext character for the row position and the keyword character for the column position results in the extraction of the ciphertext character E when the plaintext character is M and the keyword character is S. If you reverse the assignment of rows and columns, the use of the column character M and the row character of S as pointers also results in the extraction of the ciphertext character E.

Figure 5.2 A Vigenére encipherment

Although the use of a Vigenére cipher negates the use of a letter frequency count as a decipherment wedge by a cryptanalyst, this cipher system has one serious deficiency. This deficiency is known as the "probable word" attack in which a cryptanalyst first selects a word considered likely to be in the plaintext message. Then, the cryptanalyst performs a modulo 26 subtraction of each character in that word from all possible locations in the ciphertext, examining the sequence of characters produced by the modulo 26 subtraction process for a recognizable word or word fragment.

During World War I, the Italians used an encipher system known as cifrario tascabile, or "pocket cipher," that was a modified Vigenére cipher with the digits added to the end of the plaintext alphabet, while the cipher alphabets consisted of the digits 10 to 45 in sequence. Radiograms intercepted by the Austrians that were enciphered using the cifrario tascabile usually required less than half a day for trained cryptanalysts to decipher, and provided them with a wealth of information, including the ability to respond to an Italian offensive before the offensive was launched!

Other Polyalphabetic Substitution Methods

As previously explained, a major weakness of the Vigenére cipher system is its susceptibility to a probable word attack. This susceptibility results from the use of a repeating keyword whose characters are continuously used as pointers with each plaintext character for the extraction of a ciphertext character from the tableau. To counter this susceptibility you can develop a large number of different polyalphabetic substitution methods—a few of which we will investigate in this section. However, prior to doing so let us examine the major elements of a periodic polyalphabetic substitution system.

Periodic Polyalphabetic Substitution

A periodic polyalphabetic substitution system contains one plaintext alphabet and two or more cipher alphabets. The plaintext alphabet is considered to be the primary alphabet because it serves as an extraction pointer to a column location within each cipher alphabet row. The cipher alphabets provide the characters for extraction that yield the ciphertext and are referred to as secondary alphabets.

Figure 5.3 illustrates a simple example of a periodic polyalphabetic substitution system. In this example, the plaintext alphabet is the simple sequential English alphabet A through Z, with position one commencing with the letter A. The secondary cipher alphabets are a sequence of five plaintext alphabets shifted by one, two, three, four, and five character positions. The first cipher alphabet is indicated by the row header C1, while succeeding cipher alphabets are indicated by row headers C2, C3, C4, and C5.

Figure 5.3 Periodic polyalphabetic substitution system

Operation

The operation of a periodic polyalphabetic substitution system involves locating the plaintext character in the plaintext alphabet. Next, the match position is located and serves as a pointer to the extraction of a cipher character from the first cipher alphabet. The second plaintext character is then matched against the plaintext alphabet, and the match position serves as a mechanism to extract the second cipher character from the second cipher alphabet. This process continues until a character is extracted from the last cipher alphabet. At this point the process begins anew with the result of the next plaintext match causing an extraction to occur from the first cipher alphabet.

To illustrate the use of a periodic polyalphabetic substitution system, encipher the message FIRE FRED using the system illustrated in Figure 5.3. Locating F in the plaintext and using that character location as a pointer to the first cipher alphabet (C1) results in the extraction of the cipher character G. Next, locating I in the plaintext and using its position as a pointer to the second cipher alphabet (C2) results in the extraction of the cipher character K. Similarly, R is enciphered as U, E is enciphered as I, and F is enciphered as K. Because you have now used the last cipher alphabet, encipher R, which is the sixth character in the plaintext message, using the first cipher alphabet, resulting in R being enciphered as S. Similarly, E is enciphered using the second cipher alphabet as G, while D is enciphered using the third cipher alphabet to obtain its replacement by G. Thus, the message FIRE FRED is enciphered as GKUIKSGG.

In the example shown in Figure 5.3, there was a total of five cipher alphabets. This means that the use of each cipher alphabet is repeated for every five characters in the plaintext—this polyalphabetic substitution system is then said to have a period of five. Unfortunately, this also means that if a cryptanalyst can determine the repetition period of the cipher alphabets, he or she can perform a series of character frequency analyses that could eventually result in the decipherment of an intercepted message. To add a degree of difficulty to the life of a cryptanalyst, you can consider a variety of techniques to reduce the potential of analysis and decipherment.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Reducing Probability of Decipherment

One method that can be used to reduce the potential threat of analysis and decipherment is to extend its repetition period. Although this is time consuming when performed manually and subject to an increased probability of an enciphering error as the number of cipher alphabets increases, you can easily accomplish this through automation via a computer. Because you would also want to eliminate the use of a simple displacement for creating the cipher alphabets, you can use several methods to accomplish this. First, you can base the creation of the plaintext and each ciphertext alphabet upon the use of a keyword or keyword phrase. For example, assume you use the keyword MICROSOFT for the plaintext alphabet and the keyword phrase WINDOWS ARE FUN for the ciphertext alphabets. You can use the first keyword in its normal manner, and the second keyword can be used vertically as a mechanism to establish shifted cipher alphabets.

Figure 5.4 illustrates the use of a keyword-based periodic polyalphabetic substitution system using the keyword MICROSOFT to form the plaintext alphabet, while the keyword phrase WINDOWS ARE FUN is used to form shifted cipher alphabets. You should note that the plaintext keyword-based alphabet at the top of Figure 5.4 serves as a position pointer to each cipher alphabet. Concerning the cipher alphabet, you should note that since I will shortly develop a program using 26 cipher alphabets incorporated into a matrix, I labeled the cipher alphabets 0 to 25 to correspond to the array index that will be used. You should also note that the use of a vertical keyword or keyword phrase to form cipher alphabets positions the keyword or keyword phrase characters to the extreme right of the matrix in the last column. For example, in cipher alphabet 0 W is rotated to the last column; hence, the first column commences with the character X. Similarly, in Row 1 (cipher alphabet 1) the character I in the keyword phrase WINDOWSAREFUN is positioned in the last column, which makes the character J the first character in Row 1.

Figure 5.4 A plaintext keyword-based alphabet and polyalphabetic ciphertext matrix using a shifted keyword-based alphabet

One weakness of polyalphabetic substitution systems was discovered by Auguste Kerckhoffs, a Dutchman born in 1835 who became a naturalized citizen of France. Author of La Cryptographie militaire, considered one of the most extensive works on cryptanalysis, Kerckhoffs developed a general solution for polyalphabetic substitution systems that has deciphered many "unbreakable" messages.

Under the Kerckhoffs method, a group of several intercepted messages is aligned so that the position of each enciphered character in each message is aligned by column. If the same key is used for the encipherment of each message, the letters in each column represent an encipherment process in which the same ciphertext alphabet was used. Thus, each column can be attacked as if it represented a monoalphabetic substitution by performing a frequency analysis of the letters in each column. Of course, if you change the key used to govern the enciphering process, the frequency of key changes and the number of messages enciphered by any one key determines the ability of the Kerckhoffs process to be used against your messages. If you transmit only a few messages with each key prior to changing the key, a frequency analysis of your messages on a column-by-column basis will more than likely produce meaningless results. If you transmit a large number of messages but change keys frequently, your message groups enciphered by a common key will probably be sufficient for a cryptanalyst to apply Kerckhoffs' method. Thus, you must balance both the change in keys and the number of messages enciphered using a key.

The POLY2.BAS Program

Now that you have reviewed a few techniques to construct a series of cipher alphabets, I'll show you how to develop a program to encipher a message using a polyalphabetic substitution technique. This program is stored in the file POLY2.BAS on the companion CD-ROM, and, for simplicity of illustration, will use 26 cipher alphabets.

Listing 5.3 contains the main portion of POLY2.BAS. After using four PRINT statements to display information about the program, the first INPUT statement assigns the plaintext keyword or keyword phrase to the string variable TEXT$. The subroutine INITIALIZE is then invoked to form the plaintext alphabet and the subroutine KEYWORD is invoked to form a keyword-based mixed alphabet. This is followed by a FOR-NEXT loop which assigns the keyword-based alphabet to the string array PLAIN$, since the string array PLAINTEXT$ will be reused. Once the ciphertext keyword or keyword phrase is entered, it is assigned to the string variable TEXT$ and the subroutines INITIALIZE and KEYWORD are invoked again to form the ciphertext keyword-based mixed alphabet. This alphabet is placed into the array CIPHER$ through the use of a FOR-NEXT loop. Then, a series of four subroutines is invoked. Subroutines PFORMCIPHER and PCONVERTSTORE are modified versions of the subroutines FORMCIPHER and CONVERTSTORE previously developed, with the prefix P used to denote its modification for polyalphabetic use. Subroutines MSGFILE and PRTOUT remain the same and will not be discussed further.

Listing 5.3 The main portion of the POLY2.BAS program.

REM PROGRAM POLY2.BAS
DIM PLAINTEXT$(26), PLAIN$(26), CIPHER$(26), CIPHERTEXT$(26), KEY$(26)
    CLS
    PRINT "POLY2.BAS PROGRAM enciphers text based upon the use of
                two keywords"
    PRINT "or keyword phrases using a polyalphabetic substitution
                 process using"
    PRINT "26 cipher alphabets."
    PRINT
    INPUT "Enter plaintext keyword or keyword phrase in UPPERCASE: ",
                 TEXT$
GOSUB INITIALIZE             'initialize plaintext alphabet
GOSUB KEYWORD                'form keyword based mixed alphabet
   FOR I = 0 TO 25: PLAIN$(I) = PLAINTEXT$(I): NEXT I
   INPUT "Enter ciphertext keyword or keyword phrase in UPPERCASE:",
                TEXT$
GOSUB INITIALIZE             're-initialize plaintext alphabet
GOSUB KEYWORD
   FOR I = 0 TO 25: CIPHER$(I) = PLAINTEXT$(I): NEXT I
GOSUB PFORMCIPHER            'create 26 cipher alphabets
GOSUB MSGFILE                'assign I/O files, place message on a file
GOSUB PCONVERTSTORE          'convert and store ciphertext on a file
GOSUB PRTOUT                 'print results
STOP


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The PFORMCIPHER Subroutine

Listing 5.4 illustrates the statements contained in the subroutine PFORMCIPHER. As noted from the REM statement at the beginning of this subroutine, its function is to form 26 ciphertext alphabets using the first keyword character as a basis for generating each alphabet.

Listing 5.4 The PFORMCIPHER subroutine.

PFORMCIPHER:
    REM routine to form 26 CIPHERTEXT alphabets based upon the first
                 keyword character
    FOR ZI = 0 TO 25
    GOSUB INITIALIZE
    FOR J = 0 TO 25
    IF CIPHER$(ZI) = PLAINTEXT$(J) THEN GOTO GOTYA
    NEXT J
GOTYA:  J = J + 1
    FOR K = 0 TO 25
    CIPHERTEXT$(ZI) = CIPHERTEXT$(ZI) + PLAINTEXT$((K + J) MOD 26)
    NEXT K
    NEXT ZI
    LPRINT TAB(5);
RETURN

The subroutine PFORMCIPHER contains three FOR-NEXT loops. The outer loop uses the index ZI which varies from 0 to 25 and is used to develop 26 ciphertext alphabets. After the first FOR statement, invoke the subroutine INITIALIZE to set the plaintext alphabet in its sequence into the array PLAINTEXT$. The FOR-NEXT loop using the index J then compares the character in the appropriate location of the string array CIPHER$ to each character in PLAINTEXT$ until a match occurs. When a match occurs, the character in the string array CIPHER$, which represents the first character in the ciphertext alphabet ZI, is matched to position J in the plaintext alphabet and a branch to the label GOTYA occurs. Because you want to shift the matched character to the last column, increment J by 1 prior to using the FOR-NEXT loop with K as the index to form the ciphertext alphabet. The execution of the subroutine PFORMCIPHER produces the 26x26-character matrix previously illustrated in Figure 5.4, as CIPHERTEXT$(ZI) varies from 0 to 25 when WINDOWSAREFUN is used as the ciphertext keyword phrase.

The PCONVERTSTORE Subroutine

The subroutine PCONVERTSTORE is similar to the subroutine CONVERTSTORE, although it is modified to sequentially pick a ciphertext character from each cipher alphabet in sequence until the last alphabet is reached. The ciphertext character extraction process begins anew with the first cipher alphabet.

Listing 5.5 contains the statements in the subroutine PCONVERTSTORE. The statement that sets the variable C to a value of 0 also sets the pointer to the first cipher alphabet. The next modification to CONVERTSTORE is the replacement of the string array PLAINTEXT$ by the string array PLAIN$ in the first IF statement in the subroutine. This statement mates a character from the plaintext stored in the string variable TEXT$ to a character in the string variable PLAIN$ which contains the keyword-based mixed plaintext alphabet. When a match occurs, J points to the location in the keyword-based mixed alphabet that serves as a pointer to each cipher alphabet for the extraction to a cipher character. However, when you branch to the label GOTIT, you must increment J by 1 because you incremented J by that amount in the subroutine PFORMCIPHER.

Listing 5.5 The PCONVERTSTORE subroutine.

PCONVERTSTORE:
   REM Routine to convert and store ciphertext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   C = 0              'first alphabet pointer
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
           REM Convert plaintext to ciphertext
               FOR I = 1 TO MSGLEN
               FOR J = 0 TO 25
               IF MID$(TEXT$, I, 1) = PLAIN$(J) THEN GOTO GOTIT
               NEXT J
GOTIT:           MID$(TEXT$, I, 1) = MID$(CIPHERTEXT$(C), J + 1, 1)
               C = C + 1
               IF C = 26 THEN C = 0     'reset to first alphabet
               NEXT I
CLEARTXT:          WRITE #2, TEXT$
   LOOP
DONE1:         CLOSE #2
RETURN

Once a match occurs and you branch to the label GOTIT, the character in the string TEXT$ is replaced by a character from the cipher alphabet whose index value is C at position J+1 through the use of the MID$ statement. C is then incremented by 1 to point to the next cipher alphabet. However, if C equals 26 it is reset to a value of 0 to point back to the first cipher alphabet.

Program Execution

Figure 5.5 illustrates an example of the execution of POLY2.BAS using MICROSOFT as the plaintext keyword and WINDOWSAREGREAT as the ciphertext keyword phrase. I purposely changed the ciphertext keyword from WINDOWSAREFUN to WINDOWSAREGREAT to illustrate one deficiency of keyword-based polyalphabetic substitution systems—common keywords or keyword phrases. (The POLY2.BAS program enciphers text based on the use of two keywords or keyword phrases using a polyalphabetic substitution process using 26 cipher alphabets.)

Figure 5.5 The execution of POLY2.BAS

At this point it may be useful to manually encipher a portion of the cleartext message YOU CONFUSED… to develop the enciphered message VNIGT KHMKR EUEIX… shown in the lower portion of Figure 5.5. Using the keyword-based mixed plaintext alphabet at the top of Figure 5.4, locate Y and extract the character in the Y column from the first cipher alphabet, obtaining the character V. Next, locate the character O in the keyword-based mixed plaintext alphabet and extract the character in the plaintext O column from the second cipher alphabet to obtain the character N. Similarly, U in the plaintext message results in I being extracted from the third cipher alphabet, and so on.

In Figure 5.4, the ciphertext keyword phrase WINDOWSAREFUN was used, while the ciphertext keyword phrase WINDOWSAREGREAT was used when the program POLY2.BAS was executed. Because both keyword phrases have the first 10 characters in common, of which 9 are used to form cipher alphabets due to a duplicate W, the first 9 cipher alphabets are the same for both keyword phrases. This means that the use of a polyalphabetic substitution system will be more secure if you avoid industry-specific keyword or keyword phrases as well as other terms someone may be able to correctly guess based upon your or other employees' work habits.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The COUNT.CPP Program

As a "bonus" for readers who have followed the alphabets in this chapter, I will provide a bonus program in this section in addition to the customary C++ version of a previously explained BASIC program. The bonus C++ program is appropriately named COUNT.CPP as it counts the characters in a file, performing a frequency analysis.

Listing 5.6 contains a listing of the statements in the C++ bonus program. Although the program COUNT.CPP was written to analyze encrypted files that contain uppercase alphabetic characters, you can easily modify it to operate on all characters in the character set by changing its computeFrequencyCount function. This modification is left as an exercise for you to perform.

Listing 5.6 The COUNT.CPP program listing.

/*
count.cpp
C++ code written by Jonathan Held, March 27, 1998,
using Microsoft's Visual C++ version 5.0.
*/

//standard include files
#include <assert.h>
#include <ctype.h>
#include <iostream.h>
#include <iomanip.h>
#include <fstream.h>
#include <string.h>

//function prototypes
bool computeFrequencyCount(const char *&, const char*&);
void displayOutput(ofstream, const char *&);
void getFileNames(char *&, char *&);
void initialize(void);
void welcome(void);

//constants
const int TWENTYSIX = 26, SIXTYFIVE = 65, NINETY = 90,
      ONE_HUNDRED = 100, SIZE = 256;

//globals
int count[TWENTYSIX];
int total_characters = 0;

//----------------------------------------------------------------
//Function: main()
//Parameters:  None
//Return type: int
//Purpose: Performs a frequency analysis on encrypted text that
//was enciphered using monoalphabetic substitution techniques.
//----------------------------------------------------------------
int main()
{
  bool success = false;
  char *inp_file, *op_file;

  welcome();
  initialize();
  getFileNames(inp_file, op_file);
  success = computeFrequencyCount(inp_file, op_file);

  if (!success){
     cerr << "\a\nProblem opening input file and/or "
        << "writing to output file!" << endl
        << "Program terminating abnormally." << endl;
  }

  delete [] inp_file;
  delete [] op_file;

  return (success);
}//end main

//----------------------------------------------------------------
//Function: computeFrequencyCount()
//Parameters:  INP - the name of the input file
//       OP - the name of the output file
//Return type: bool, true if we are successful, false otherwise
//Purpose: Computes the frequency count of an encrypted file by
//reading a line of text, analyzing the line one character at a
//time, indexing into the global count array and incrementing the
//count of that character, and continuing until it reaches the
//end of the line.  This process is continued until the eof is
//reached.  Since all characters are uppercase, we index into the
//count array on the ASCII value of the character minus 65.  Hence,
//an A is at index 0 (65-65=0), and Z is at index 25 (90-65=25).
//----------------------------------------------------------------
bool computeFrequencyCount(const char *&INP, const char *&OP)
{
 bool success = false;
 char temp[SIZE];

 ifstream input(INP, ios:: in);
 ofstream output(OP, ios:: out);

 //if we can't open the input file, we need to report the
 //error to the user
 if ((!input) || (!output)){
    //do nothing
 }
 else {
   success = true;

   //read each character
   char one_character;

   //character's ASCII value
   int value;

   //perform frequency count on each line
   while (input.getline(temp, SIZE, '\n')){

    for (int ix=0; ix<strlen(temp); ix++){

      one_character = *(temp+ix);
      value = static_cast<int>(one_character);
      //check for a valid character
      if ((value >= SIXTYFIVE) || (value <= NINETY)){

      //increment the number of characters
        total_characters++;

         //add the count of the character to the array
         count[value - SIXTYFIVE] += 1;
         }
      }
   }

  //check for no characters in the input file
  if (total_characters == 0){
     success = false;
  }
  else {
    //echo the output to the screen and save it to a file
    displayOutput(output, OP);
  }
}

return success;
}//end computeFrequencyCount()

//----------------------------------------------------------------
//Function: displayOutput()
//Parameters:  OUTFILE - the file we are writing the results to
//       FILE_NAME - the name of the output file
//Return type: None
//Purpose: Displays the results to the screen and saves them to a
//file named FILE_NAME.
//----------------------------------------------------------------
void displayOutput(ofstream OUTFILE, const char*& FILE_NAME)
{
  cout << "Frequency count is: " << endl << endl;
  OUTFILE << "Frequency count is: " << endl <<
  endl;

  cout << "Character" << "\t" << "Count" << "\t" << "Frequency" << endl;
  OUTFILE << "Character" << "\t" << "Count" << "\t" << "Frequency" << endl;

  for (int ix=0; ix<TWENTYSIX; ix++){
    cout << setw(4) << static_cast<char>(ix + SIXTYFIVE) << "\t\t"
             << setw(3) << count[ix] << "\t"
             << static_cast<float>(count[ix])/total_characters*ONE_HUNDRED
             << "%" << endl;
  OUTFILE << setw(4) << static_cast<char>(ix + SIXTYFIVE) << "\t\t"
              << setw(3) << count[ix] << "\t"
             << static_cast<float>(count[ix])/total_characters*ONE_HUNDRED
             << "%" << endl;

  }

  cout << "\nTotal number of characters is: " << total_characters << endl;
  cout << "Data saved to file " << FILE_NAME << endl;
  OUTFILE << "\nTotal number of characters is: " << total_characters <<
  endl;

  return;
}//end displayOutput()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  input - the input file
//       output - the output file where the frequency data
//            will be saved
//Return type: None
//Purpose: Queries the user for the input and output file names.
//----------------------------------------------------------------
void getFileNames(char *&input, char *&output)
{
  char temp[SIZE];
  int char_count = 0;

  cout << "Input file (the file you want to perform" << endl
       << "the frequency count on): ";

  cin.getline(temp, SIZE, '\n');
  char_count = strlen(temp);

  assert(input = new char[char_count+1]);
  strcpy(input, temp);

  cout << "\nOutput file: ";
  cin.getline(temp, SIZE, '\n');
  char_count = strlen(temp);
  assert(output = new char[char_count+1]);
  strcpy(output, temp);
  return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: initialize()
//Parameters:  None
//Return type: None
//Purpose: Initializes the character count to 0 for each character
//in the global count array.
//----------------------------------------------------------------
void initialize()
{
  for (int ix=0; ix<TWENTYSIX; ix++){
    count[ix] = 0;
  }

  return;
}//end initialize

//----------------------------------------------------------------
//Function: welcome()
//Parameters:  None
//Return type: None
//Purpose: Describes the count.cpp program.
//----------------------------------------------------------------
void welcome(void)
{
 cout << "Program count.cpp performs a frequency count "
       << "on an encrypted file." << endl
       << "Results can be used to help decipher messages "
       << "that used a mono-" << endl
       << "alphabetic substitution scheme." << endl << endl;

  return;
}//end welcome()
//end file count.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The POLY2.CPP Program

The program POLY2.CPP, which represents the C++ language version of the POLY2.BAS program, was developed as a continuation of our strategy of providing encryption examples in both languages. Listing 5.7 includes the POLY2.CPP program listing. The actual program, as well as the executable version of the program, are located in the C directory on the CD-ROM. Concerning the executable version of the program, its filename follows the naming convention used in this book, resulting in the filename POLY2.EXE. To distinguish the executable versions of the BASIC and C++ programs from one another, it is important to remember that although they have the same filename, they are located in different directories. That is, the file POLY2.EXE located in the directory BASIC represents the executable version of the program POLY2.BAS while the file POLY2.EXE located in the C directory represents the executable version of the program POLY2.CPP.

Listing 5.7 The POLY2.CPP program listing.

/*poly2.cpp
C++ code written by Jonathan Held, April 1, 1998,
using Microsoft's Visual C++ version 5.0.
*/

//standard includes
#include<iostream.h>
#include<assert.h>
#include<string.h>
#include<ctype.h>
#include<fstream.h>

//constants we will use
const int FIVE = 5, TWENTYFIVE = 25, TWENTYSIX = 26,
      TWENTYSEVEN = 27, SIXTYFIVE = 65, NINETY = 90,
      NINETYTWO = 92, SIZE = 256, BIGSIZE = 1000;

//function prototypes
bool checkInput(char * &);
void createStream(char *, char []);
void createTable(char [][TWENTYSEVEN], const char [], const char []);
void display(char *);
void formatData(char []);
bool encryptText(char *, char *, const char PTEXT[],
                         const char [][TWENTYSEVEN], char []);
char* formCipheredMessage(const char [], const char []
               [TWENTYSEVEN],
               const char[], char []);

void getFileNames(char *&, char *&);
int getInputType(void);
int getKeyword(char *&);

bool getMessage(char *, char *, const char [],
              const char [][TWENTYSEVEN], char []);

void groupBUFFER(ofstream, int);
void printCipherToFile(ofstream, char[]);
bool writeMatrixToFile(const char [], const char [][TWENTYSEVEN]);

char BUFFER[BIGSIZE] = {'\0'};

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main()
{

  char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                   'I','J','K','L','M','N','O','P',
                   'Q','R','S','T','U','V','W','X',
                   'Y','Z','\0'};

  char *ptext_keyword, *ctext_keyword, *infile, *outfile;

  //p_stream is used to help form the alphabetic-shifted cipher
  //alphabets
  char p_stream[TWENTYSIX] = {'\0'};

  //c_stream represented the top row of the polyalphabetic ciphertext
  //matrix
  char c_stream[TWENTYSIX] = {'\0'};
  //table is the actual polyalphabetic ciphertext matrix
  char table[TWENTYSIX][TWENTYSEVEN], message_to_cipher[SIZE];

  int type_of_input;
  bool success = false;

  cout << "Enter plaintext keyword or keyword phrase in UPPERCASE: ";
  getKeyword(ptext_keyword);
  cout << "Enter ciphertext keyword or keyword phrase in UPPERCASE: ";
  getKeyword(ctext_keyword);
  createStream(ptext_keyword, p_stream);
  createStream(ctext_keyword, c_stream);
  createTable(table, plaintext, c_stream);
  getFileNames(infile, outfile);
  type_of_input = getInputType();

  //handle file input
  if (type_of_input){
     success = encryptText(infile, outfile, p_stream, table,
                                 message_to_cipher);
  }
  //keyboard input
  else {
    success = getMessage(infile, outfile, p_stream, table,
                              message_to_cipher);
  }

  //report success of operation
  if (!success){
     cerr << "Error: Invalid filename specified. Goodbye." << endl;
  }
  else {
     cout << "Press return to display resulting enciphered message."
                    << endl;
     //get the newlines off the current input stream
     cin.get();

     if (type_of_input){
        cin.get();
     }

     display(outfile);
  }
  writeMatrixToFile(p_stream, table);

  delete [] ptext_keyword;
  delete [] ctext_keyword;

  return (0);
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the keyword the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
 bool error = false;
 int count = strlen(input);

 for (int ix=0; ix<count; ix++){

   int char_value = static_cast<int>(*(input+ix));
   //determine if the user did not enter an uppercase character
   if ((char_value < SIXTYFIVE) || (char_value > NINETY)){
      error = true;
      cerr << "\aYou entered an invalid keyword!" << endl << endl
      << "Re-enter keyword: ";
      break;
   }
}

//check for just the newline and no other characters entered
if (count == 0){
   cerr << "\aYou entered an invalid keyword!" << endl << endl
   << "Re-enter keyword: ";
   error = true;
}

return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: createStream()
//Parameters: input - the keyword the user entered
//      stream - streams used in the polyalphabetic cipher
//           matrix.
//Return Type: None
//Purpose: Creates a preliminary stream that will be used in
//conjunction with the polyalphabetic cipher matrix.
//----------------------------------------------------------------
void createStream(char *input, char stream[])
{
  bool used[TWENTYSIX];
  int index = 0,
     count = strlen(input);

  //no characters are initially used
  for (int ix=0; ix<TWENTYSIX; ix++){
    used[ix] = false;
  }

//keep track of each character used, start forming the keyword
//alphabet
for (int jx=0; jx<count; jx++){

  //get each character of the input string (integer value)
  int char_value = static_cast<int>(*(input+jx));

  if (used[char_value-SIXTYFIVE]){
     //do nothing - the character was already used
  }
  else {
    //mark as used and add to the keyword alphabet
    used[char_value-SIXTYFIVE] = true;
    *(stream+index++) = static_cast<char>(char_value);
  }
}

//go through the list of characters used - those which weren't
//used should be added to the keyword alphabet
for (int kx=0; kx<TWENTYSIX; kx++){

  if (!(used[kx])){
     *(stream+index++) = static_cast<char>(SIXTYFIVE+kx);
}
}

stream[TWENTYSIX] = '\0';

return;
}//end createStream()

//----------------------------------------------------------------
//Function: createTable()
//Parameters: tbl - the polyalphabetic ciphertext matrix we are
//                  creating
//      PTEXT - the plaintext alphabet we use to form
//          shifted keyword alphabets; starting position
//          within the plaintext alphabet is determined by
//          the cipher stream
//      CSTREAM - the cipher stream that represents the top
//           row of the polyalphabetic ciphertext matrix
//Return Type: None
//Purpose: Creates a polyalphabetic ciphertext matrix.  Each character
//in the cipherstream represents a row in the polyalphabetic ciphertext
//matrix; this row is a shifted alphabet that starts at the character
//subsequent to the cstream character.
//----------------------------------------------------------------
void createTable(char tbl[][TWENTYSEVEN], const char PTEXT[],
                const char CSTREAM[])
{
  for (int ix=0; ix<TWENTYSIX; ix++){
    int start_pos = (static_cast<int>(CSTREAM[ix]) - SIXTYFIVE + 1) %
                             TWENTYSIX;
     for (int jx=0; jx<TWENTYSIX; jx++){
          tbl[ix][jx] = PTEXT[(start_pos + jx) % TWENTYSIX];
     }
    tbl[ix][TWENTYSIX] = '\0';
   }

 return;
}//end createTable()

//----------------------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echoes the resulting output file to the screen.
//----------------------------------------------------------------
void display(char *name)
{
  ifstream infile(name, ios::in);
  char input[SIZE];

  if (!(infile)){
     cerr << "Unable to open input file for display." << endl;
  }
  else {
    while (infile.getline(input, SIZE, '\n')){
     cout << input << endl;
    }
  }

  cout << endl;

  return;
}//end display()

//----------------------------------------------------------------
//Function: encryptText()
//Parameters: inp_file - the name of the input plaintext file
/       outp_file - the name of the output file to save the
/              encrypted text
/       PSTREAM[] - the top row of the polyalphabetic ciphertext
/             matrix
/       TBL[][] - the polyalphabetic ciphertext matrix
/        encoded_msg[] - the message to be encoded
//Return Type: bool, indicating success of operation
//Purpose: Used to encrypt file input.  Takes each line of the input
//file, encrypts it, and saves the result to the specified output
//file.
//----------------------------------------------------------------
bool encryptText(char * inp_file, char * outp_file, const char PSTREAM[],
          const char TBL[][TWENTYSEVEN], char encoded_msg[])
{
  bool success = false;
  char ip[SIZE];

  //declare file stream objects
  ifstream input(inp_file, ios::in);
  ofstream output(outp_file, ios::app);

  if ((!input) || (!output)){
     //do nothing - I/O error; user will be notified upon
     //procedure's return to main()
  }
  else {

    success = true;

    while (input.getline(ip, BIGSIZE, '\n')){

     //check to see if the user wants the line to appear in plaintext
     if (ip[0] == '/'){

      if (strlen(BUFFER)>0){

         //empty whatever is in the buffer
         groupBUFFER(output, strlen(BUFFER));
         //adjust the buffer
         strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
         //output plaintext
       }

     output << ip << endl;

     }
     else {
      //encipher the line
      char *msg = formCipheredMessage(PSTREAM, TBL, ip, encoded_msg);
      //print the cipher in groups of five to the ouput file
      printCipherToFile(output, msg);
     }
  }

 //empty the rest of the buffer
 groupBUFFER(output, strlen(BUFFER));

 //notify user where plaintext and ciphertext files are
 cout << "Plaintext file is: " << inp_file << endl;
 cout << "Encrypted file is: " << outp_file <<  endl << endl;
}

 //don't forget to close the files
 input.close();
 output.close();

 //return success of the operation
 return success;
}//end encryptText()

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[])
{
  for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
    if (*(data+nx) == ' '){
       //do nothing - skip over the space in the data
    }
    else {
      *(data+mx++) = *(data+nx);
    }
  }

  //don't forget to add the null terminator
  *(data+mx) = '\0';

  return;
}//end formatData()

//----------------------------------------------------------------
//Function: formCipheredMessage()
//Parameters: PSTREAM - the top row of the polyalphabetic cipher matrix
//      TBL - the actual polyalphabetic cipher matrix
//      MESSAGETOCIPHER - the message we want to encipher
//      enc_message - the enciphered message
//Return Type: char* - a pointer to the encoded information.
//Purpose: Encipher the user's message.
//----------------------------------------------------------------
char* formCipheredMessage(const char PSTREAM[], const char TBL[]
               [TWENTYSEVEN],
               const char MESSAGETOCIPHER[], char enc_message[])
{
 int length = strlen(MESSAGETOCIPHER)+1;

 //location identifies where in the CSTREAM character the plaintext
 //character is
 int location = 0;

 //use this variable to keep track of which cipher alphabet
 //we are using - making it a static ensures that its value
 //is preserved when we make subsequent function calls
 static int which_cipher_alphabet = 0;

 for (int ix=0; ix<length; ix++){

  //test to see if we have an alphabetic character; if not,
  //simply copy it to our encrypted message - this preserves
  //characters such as ', ! etc...
  if (!isalpha(static_cast<int>(MESSAGETOCIPHER[ix]))){
        enc_message[ix] = MESSAGETOCIPHER[ix];
  }
  else {

    //find the location of the character we want to
       //encipher in the CSTREAM
       for (int jx=0; jx<TWENTYSIX; jx++){
         if (MESSAGETOCIPHER[ix] == PSTREAM[jx]){
           location = jx;
           break;
         }
       }

       enc_message[ix] = TBL[which_cipher_alphabet%TWENTYSIX][location];

       //go to the next cipher alphabet
         which_cipher_alphabet++;
    }
 }

 //return a reference to the encoded message
 return enc_message;
}//end formCipheredMessage()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
         outfile_name - the output file we will write the
         enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//----------------------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
  char data[SIZE];

  cout << "Enter filename to store/retrieve plaintext message: ";

  cin >> data;

  infile_name = new char[strlen(data) + 1];
  strcpy(infile_name, data);

  cout << "Enter filename to store enciphered message: ";

  cin >> data;

  outfile_name = new char[strlen(data) + 1];
  strcpy(outfile_name, data);

  cout << endl;

  return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//----------------------------------------------------------------
int getInputType(void)
{
  char type;
  bool error = false;
  int value;
  do {
     //prompt user for input from file or keyboard
     cout << "Is file input from keyboard (K, k) or file (F, f): ";
     cin >> type;

     //make type an uppercase letter
     type = static_cast<char>(toupper(static_cast<int>(type)));

     //check for an invalid character
     if ((type != 'K') && (type != 'F')){
        cerr << "You have entered an invalid character!" << endl << endl;
        error = true;
     }
     else {
        if (type == 'K')
           value = 0;        //value of 0 represents keyboard input
           else value = 1;   //value of 1 represents file input
        error = false;
     }

  } while (error);

  cout << endl;

  return value;
}//end getInputType()

//----------------------------------------------------------------
//Function: getKeyword()
//Parameters: text - the keyword that the user enters
//Return Type: int - the length of the keyword
//Purpose: Prompts the user for a keyword and continues until
//a valid keyword has been entered.  Returns the length of the
//keyword.
//----------------------------------------------------------------
int getKeyword(char * &text)
{
  bool error = false;
  char buffer[SIZE];
  do {

   cin.getline(buffer, SIZE, '\n');
   assert(text = new char[strlen(buffer) + 1]);
   strcpy(text, buffer);

   error = checkInput(text);

   //delete text if there was an error
   if (error){
      delete [] text;
     }

  } while (error);

  cout << endl;

  return strlen(buffer);
}//end getKeyword()

//----------------------------------------------------------------
//Function: getMessage()
//Parameters:  input - the name of the input plaintext file
         output the name of the output ciphertext file
         msg_to_cipher - the message to be encoded
//       PSTREAM - the top row of the polyalphabetic ciphertext
//            matrix; used to index into the actual table
//       TBL - the polyalphabetic ciphertext matrix
//Return Type: bool, indicating success of operation
//Purpose: Allow the user to manually input text from the keyboard.
//Save the text in plaintext to the input file; encrypt the text
//and save it to the specified output file for later retrieval.
//----------------------------------------------------------------
bool getMessage(char* input, char* output, const char PSTREAM[],
               const char TBL[][TWENTYSEVEN], char msg_to_cipher[])
{
  bool go_on = true, success = false;

  ofstream textFile(input, ios::app);
  ofstream cipherFile(output, ios::app);

  if ((!textFile) || (!cipherFile)){
     //do nothing - error will be noted to user later
  }
  else {

    success = true;

    //get the newline character off of the input stream
    cin.get();

    cout << "Enter the message in UPPERCASE or lowercase characters. "
                       << endl
    << "Non-alphabetic characters may be entered but are ignored."
                       << endl
    << "Use a / at the beginning of each line that should remain"
                       << endl
    << "in plaintext and a \\ on a separate line to indicate the"
                       << endl
    << "end of an enciphered message." << endl < <endl;

while (go_on) {

 //get the entire line, up to 256 characters
 cin.getline(msg_to_cipher, SIZE, '\n');

 //case user doesn't want the text to be encrypted
 if (msg_to_cipher[0] == '/'){

    if (strlen(BUFFER)>0){
       //empty whatever is in the buffer
       groupBUFFER(cipherFile, strlen(BUFFER));
       //adjust the buffer
       strcpy(BUFFER, (BUFFER+strlen(BUFFER)));
    }

    //output plaintext
    textFile << msg_to_cipher << endl;
    cipherFile << msg_to_cipher << endl;
     }

   //case user is done entering text
   else if (static_cast<int>(msg_to_cipher[0]) == NINETYTWO){
     go_on = false;
     }

     //encrypt the text
     else {
       textFile << msg_to_cipher << endl;
       char enciphered_msg[BIGSIZE];
       formCipheredMessage(PSTREAM, TBL, msg_to_cipher,enciphered_msg);
       printCipherToFile(cipherFile,enciphered_msg);
       }
  }

 //empty the rest of the buffer
 groupBUFFER(cipherFile, strlen(BUFFER));
 }

 //close the files
 textFile.close();
 cipherFile.close();

 //notify user where plaintext and ciphertext files are
 cout << "\nPlaintext file is: " << input << endl;
 cout << "Encrypted file is: " << output << endl << endl;

 return success;
}//end getMessage()

//----------------------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
        num - the number of characters we want to output
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//----------------------------------------------------------------
void groupBUFFER(ofstream out, int num)
{
  for (int kx=0;kx<num;kx++){

    if ((kx!=0) && (kx%TWENTYFIVE==0)){
       out << endl;
    }

    if ((kx!=0) && (kx%FIVE == 0) && (kx%TWENTYFIVE!=0)){
       out << " " << *(BUFFER+kx);
    }
    else {
       out << *(BUFFER+kx);
    }
  }

  out << endl;

  return;
}//end groupBUFFER()

//----------------------------------------------------------------
//Function: printCipherToFile()
//Parameters: op - the output file we are writing to
//      msg - the cipher text we are displaying
//Return Type: None
//Purpose: Group the cipher in 5-block characters in the
//specified output file.
//----------------------------------------------------------------
void printCipherToFile(ofstream op, char msg[])
{
  formatData(msg);

  //check to see if there are more than 25 characters
  //in the buffer; if so, print out as many groups of
  //25 as possible
  if (strlen(BUFFER) >= TWENTYFIVE){

     int numchars = (strlen(BUFFER)/TWENTYFIVE)*TWENTYFIVE;
     //print the contents of the buffer to the output stream
     groupBUFFER(op, numchars);
     //shift whatever is left in the buffer
     strcpy(BUFFER, (BUFFER+numchars));
     //append data to the buffer
     strcat(BUFFER, msg);
  }

  //if buffer contents are less than 25, simply append the new
  //data to the buffer
  else if ((strlen(BUFFER) >= 0) && (strlen(BUFFER) < TWENTYFIVE)){
  strcat(BUFFER, msg);
  }

  return;
}//end printCipherToFile()

//----------------------------------------------------------------
//Function: writeMatrixToFile()
//Parameters: CSTREAM - the top row of the polyalphabetic cipher matrix
//      TBL - the actual matrix
//Return Type: bool - indicates success of the operation
//Purpose: Prints the matrix to a file "table.dat" for future
//reference.
//----------------------------------------------------------------
bool writeMatrixToFile(const char PSTREAM[], const char TBL[][TWENTYSEVEN])
{
  ofstream output("table.dat", ios::out);
  bool success = false;

  if (output){
     success = true;
     output << PSTREAM << endl << endl;
     for (int ix=0; ix<TWENTYSIX; ix++)
       output << TBL[ix] << endl;
  }

  cout << "Polyalphabetic matrix was written to file table.dat"
       << endl << endl;

  output.close();

  return success;
}//end writeMatrixToFile()
//end file poly2.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DPOLY2.BAS Program

Earlier in this chapter you developed the programs POLY2.BAS and POLY2.CPP to encipher messages using a polyalphabetic substitution technique. Because you need a mechanism to automate the decipherment of such messages, I will now focus your attention upon the development of a program to reverse the encipherment process. I will label these two programs DPOLY2.BAS and DPOLY2.CPP to denote that they decipher messages enciphered using the POLY2.BAS and POLY2.CPP programs.

In this section you will first examine the operation of the BASIC program DPOLY2.BAS. Once this is accomplished I will turn your attention to the statements in the C++ version of the program.

Listing 5.8 shows the statements in the main portion of the program DPOLY2.BAS developed to decipher messages previously enciphered using the program POLY2.BAS. When you examine Listing 5.8, you will note that the only differences between the main portion of the programs are changes to PRINT statements that essentially replace "cipher" with "decipher" and the prefix of two subroutines by the character D to denote major changes to previously developed subroutines without that prefix. Because the only subroutines that were significantly changed were MSGFILE and PCONVERTSTORE, I will focus on these two subroutines.

Listing 5.8. The main portion of the DPOLY2.BAS program.

REM PROGRAM DPOLY2.BAS
DIM PLAINTEXT$(26), PLAIN$(26), CIPHER$(26), CIPHERTEXT$(26), KEY$(26)
    CLS
    PRINT "DPOLY2.BAS PROGRAM deciphers text previously enciphered
                using two keywords"
    PRINT "or keyword phrases and a polyalphabetic substitution process
    using"
    PRINT "26 cipher alphabets."
    PRINT
    INPUT "Enter plaintext keyword or keyword phrase in UPPERCASE: ",
           TEXT$
GOSUB INITIALIZE           'initialize plaintext alphabet
GOSUB KEYWORD                'form keyword based mixed alphabet
   FOR I = 0 TO 25: PLAIN$(I) = PLAINTEXT$(I): NEXT I
   INPUT "Enter ciphertext keyword or keyword phrase in UPPERCASE:",
                TEXT$
GOSUB INITIALIZE           're-initialize plaintext alphabet
GOSUB KEYWORD
   FOR I = 0 TO 25: CIPHER$(I) = PLAINTEXT$(I): NEXT I
GOSUB PFORMCIPHER           'create 26 cipher alphabets
GOSUB DMSGFILE            'assign I/O files, place message on a file
GOSUB DPCONVERTSTORE           'convert and store ciphertext on a file
GOSUB PRTOUT             'print results
STOP

The DMSGFILE Subroutine

The major differences between the previously developed subroutine MSGFILE and the subroutine DMSGFILE are related to the assignment of the string variables OUTFILE$ and INFILE$. In the subroutine DMSGFILE, these assignments are reversed from their assignment in MSGFILE. Because the contents of the subroutine DMSGFILE were described as part of our description of the program DCIPHER6.BAS and other programs in Chapter 4, you should refer to that chapter for information concerning this subroutine and the program listing.

The DPCONVERTSTORE Subroutine

The statements used to form the subroutine DPCONVERTSTORE are contained in Listing 5.9. Unlike the subroutine PCONVERTSTORE, which converts a plaintext message to ciphertext and stores the message in ciphertext on a disk file, DPCONVERTSTORE performs a reverse process. That is, the subroutine DPCONVERTSTORE converts an enciphered message to plaintext and stores the plaintext message on a disk file.

Listing 5.9. The DPCONVERTSTORE subroutine listing.

DPCONVERTSTORE:
   REM Routine to convert and store ciphertext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   C = 0              'first alphabet pointer
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM Convert plaintext to ciphertext
           FOR I = 1 TO MSGLEN
           FOR J = 0 TO 25
           IF MID$(TEXT$, I, 1) = MID$(CIPHERTEXT$(C), J + 1,
                                  1) THEN GOTO GOTIT
           NEXT J
GOTIT:       MID$(TEXT$, I, 1) = PLAIN$(J)
           C = C + 1
           IF C = 26 THEN C = 0   'reset to first alphabet
           NEXT I
CLEARTXT:               WRITE #2, TEXT$
   LOOP
DONE1:         CLOSE #2
RETURN

When you compare the subroutine DPCONVERTSTORE to the subroutine PCONVERTSTORE, you should note that the difference between the two subroutines is limited to the structure of an IF statement in each subroutine and the statement at label GOTIT. In PCONVERTSTORE, the first IF statement is used to match each character in the string variable TEXT$ to a character in the keyword-based alphabet stored in the array PLAIN$ and use the location of the match as a pointer into a cipher alphabet. In the subroutine DPCONVERTSTORE, each character is matched to the cipher alphabet whose value is C at position J+1.

In the subroutine PCONVERTSTORE, a branch to the label GOTIT results in the replacement of a plaintext character (MID$(TEXT$,I,1)) by a ciphertext character (MID$(CIPHERTEXT$(C),J+1,1)) in the cipher alphabet C at position J+1. In the subroutine DPCONVERTSTORE, you should replace each enciphered character with its equivalent plaintext character. Because a match using the previously described IF statement results in the location of the correct alphabet and location within the alphabet, we then set each character in TEXT$ to the array element PLAIN$(J) to extract the appropriate plaintext character. This allows you to convert the ciphertext back into its equivalent plaintext character.

Program Execution

To verify the operation of the program DPOLY2.BAS, execute it using the same plaintext and ciphertext keywords used to encipher your message using the POLY2.BAS program. Figure 5.6 illustrates the execution of the DPOLY2.BAS program using the plaintext keyword MICROSOFT and the ciphertext keyword phrase WINDOWSAREGREAT. The lower portion of this example shows the resulting deciphered message. You will note that this message is the same message that was entered as plaintext when the program POLY2.BAS was executed in Figure 5.5.

Figure 5.6 The execution of the program DPOLY2.BAS.

Other differences between DPOLY2.BAS and POLY2.BAS relate to PRINT statements that were changed to display "deciphered" instead of "enciphered" in the subroutine PRTOUT. Because this was an insignificant change, I did not change the name of the subroutine; however, the PRINT statements in the subroutine were changed when the program was stored on the file DPOLY2.BAS on the companion CD-ROM.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DPOLY2.CPP Program

In concluding this chapter, dual coverage of BASIC and C++ programs will be continued by turning your attention to the C++ version of DPOLY2. That program, which has the filename DPOLY2.CPP, is contained on the CD-ROM in the C directory.

Listing 5.10 contains a listing of the statements in the header and the main function of DPOLY2.CPP. In examining the listing shown in Listing 5.10 you will note that many functions used in the program POLY2.CPP are reused, such as checkInput, createStream, and createTable. Thus, in this section I will focus your attention on the key difference between the two programs. That difference is primarily in the inclusion of the functions decipher and getFileToDecipher. Listing 5.11 lists the statements in those two functions. Since each function is explicitly documented through the use of comments, I will leave it to you to review the statements for each function.

Listing 5.10 Header and function main statements in DPOLY2.CPP.

/*DPOLY2.CPP
C++ code written by Jonathan Held
Using Microsoft's Visual C++ Version 5.0
*/

//standard includes
#include<iostream.h>
#include<assert.h>
#include<string.h>
#include<ctype.h>
#include<fstream.h>
#include<stdlib.h>

//constants we will use
const int FIVE = 5, TWENTYFIVE = 25, TWENTYSIX = 26,
      TWENTYSEVEN = 27, SIXTYFIVE = 65, NINETY = 90,
      NINETYTWO = 92, SIZE = 256, BIGSIZE = 1000;

//function prototypes
bool checkInput(char * &);
void createStream(char *, char []);
void createTable(char [][TWENTYSEVEN], const char [], const char []);
void decipher(ifstream, ofstream, const char[], char[]);
void display(char *);
void formatData(char []);
void getFileNames(char *&, char *&);
void getFileToDecipher(const char[], char[][TWENTYSEVEN]);
int getInputType(void);
int getKeyword(char *&);
bool getMessage(char*, char*, char [], const char[], const
                  char[][TWENTYSEVEN]);

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main()
{

  char plaintext[TWENTYSEVEN] = {'A','B','C','D','E','F','G','H',
                       'I','J','K','L','M','N','O','P',
                       'Q','R','S','T','U','V','W','X',
                       'Y','Z','\0'};

 char *ptext_keyword, *ctext_keyword, *infile, *outfile;

  //p_stream is used to help form the alphabetic-shifted cipher
  //alphabets
  char p_stream[TWENTYSIX] = {'\0'};

  //c_stream represented the top row of the polyalphabetic ciphertext
  //matrix
  char c_stream[TWENTYSIX] = {'\0'};

  //table is the actual polyalphabetic ciphertext matrix
  char table[TWENTYSIX][TWENTYSEVEN];

  char enc_msg[SIZE];

  int type_of_input;
  bool success = false;

  cout << "Enter plaintext keyword or keyword phrase in UPPERCASE: ";
  getKeyword(ptext_keyword);
  cout << "Enter ciphertext keyword or keyword phrase in UPPERCASE: ";
  getKeyword(ctext_keyword);
  createStream(ptext_keyword, p_stream);
  createStream(ctext_keyword, c_stream);
  createTable(table, plaintext, c_stream);
  type_of_input = getInputType();

  if (type_of_input){
     //decipher accordingly
     getFileToDecipher(p_stream, table);
  }
  else {
     getFileNames(infile, outfile);
     getMessage(infile, outfile, enc_msg, p_stream, table);
     cout << "Press return to display the deciphered text." << endl;
     display(outfile);
  }

  return 0;
}

Listing 5.11 Statements in the decipher and getFileToDecipher functions from the program DPOLY2.CPP.

//----------------------------------------------------------------
//Function: decipher()
//Parameters: in - the input file we are deciphering
//      out - the output file we are writing the deciphered
//                  text to
//      PSTREAM - the plaintext alphabet (used to decipher)
//      TBL - the polyalphabetic substitution table
//Return Type: None
//Purpose: Decipher the input file and write the contents to the
//output file specified by the user.
//----------------------------------------------------------------
void decipher(ifstream in, ofstream out, const char PSTREAM[], const
           char TBL[][TWENTYSEVEN])
{
  char enc_file_data[SIZE];
  static int which_cipher_alphabet = 0;

  //continue this process until we get to the end of the file
  while (in.getline(enc_file_data, SIZE, '\n')){

     if (enc_file_data[0] == '/'){
        out << enc_file_data << endl;
     }
     else {
        //format the data - i.e., get rid of all spaces
               formatData(enc_file_data);

        //dump data to file
        for (int ix=0; ix<strlen(enc_file_data); ix++){

       //used to keep track of what plaintext character
         //we are going to use
         int jx;
         for (jx=0; jx<TWENTYSIX; jx++){
         //find where the encrypted data is in the
         //ciphertext - this location corresponds to
         //the plaintext character location
         if (enc_file_data[ix] == TBL[which_cipher_alphabet%TWENTYSIX][jx])
            break;
    }

        //conditionals for grouping by five and inserting
          //new lines
          if (!(ix%TWENTYFIVE))
             out << endl;

          if ((ix!=0) && (!(ix%FIVE))){
             out << " " << PSTREAM[jx];
          }
          else {
            out << PSTREAM[jx];
          }

            which_cipher_alphabet++;
        }
     }
  }

  return;
}//end decipher()

//----------------------------------------------------------------
///----------------------------------------------------------------
//Function: getFileToDecipher()
//Parameters: PSTREAM - the plaintext stream at the top of the
//      polyalphabetic substitution table
//      TBL - the polyalphabetic table
//Return Type: None
//Purpose: Prompt the user for the name of the encrypted file
//and the file the user wants to store the decrypted text to.
//----------------------------------------------------------------
void getFileToDecipher(const char PSTREAM[], char TBL[][TWENTYSEVEN])
{
  char fileinput[SIZE], fileoutput[SIZE];

  cout << "Enter name of file to decipher: ";
  cin >> fileinput;

  ifstream input(fileinput, ios::in);

  if (!(input)){
     cerr << "Input file not available. Exiting program." << endl;
     exit(EXIT_FAILURE);
  }

  cout << "Enter name of file for output: ";
  cin >> fileoutput;
  ofstream output(fileoutput, ios::out);

  if (!(output)){
     cerr << "Output file not created.  Exiting program." << endl;
     exit(EXIT_FAILURE);
  }

  decipher(input, output, PSTREAM, TBL);

  //don't forget to close the files
  input.close();
  output.close();

  cout << "\nDeciphered text is in " << fileoutput << endl;
  cout << "\nPress return to display deciphered text." << endl;
  cin.get();
  cin.get();

  display(fileoutput);

  return;
}//end getFileToDecipher()

//----------------------------------------------------------------


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 6
Using Random Numbers

Through the use of random numbers you can decrease the decipherment capability of people who illicitly gain access to an enciphered message. In fact, the use of a true random number sequence as an enciphering key theoretically makes it impossible for a person to decipher that message. Unfortunately, the use of a true random number sequence is beyond the capability of most organizations due to the time, cost, and the effort required to provide the sequence to others who must use it to decipher messages. Thus, most random number sequences are actually pseudo-random number sequences that, with varying degrees of difficulty, can be duplicated.

This chapter first discusses the ways in which true random number sequences are developed and why they are impractical for most persons and organizations. Next, you will examine the built-in random number generator included in most programming languages and learn how to use that facility to create a pseudo-random number sequence that can serve as an enciphering and deciphering key. This discussion is followed by the construction of a series of programs that illustrate the use of computer-generated random number sequences to encipher and decipher messages.

Random Numbers and Random Number Sequences

Random numbers are unordered numbers that are independent of preceding and succeeding numbers. Thus, a random number sequence is an unordered series of random numbers.

The use of random number sequences in cryptology provides a mechanism to create an unbreakable enciphered message. To illustrate this, assume the random number sequence you obtain (through a process described later in this chapter) is converted into a binary sequence. Also assume that each plaintext character in your message is converted into a binary sequence. You could use modulo 2 addition to add the plaintext and random number binary values to produce an enciphered binary sequence. If the recipient has the same random number sequence, he or she would use modulo 2 subtraction to subtract the binary values of the random numbers from the enciphered binary sequence to reproduce the plaintext message. Because the binary random number sequence functions as a unique key known only to the message encipherer and message decipherer, any person who illicitly obtains a copy of the enciphered message cannot decipher the message.

The father of the application of modulo 2 arithmetic to encipherment systems was Gilbert Vernam. Working in downtown Manhattan for American Telephone & Telegraph Company during 1917, Vernam was assigned to a project to investigate the security of the telegraph.

In 1917, the Baudot code was used for the transmission of information via telegraph. In this code, characters were represented by a predefined sequence of a combination of five marks and spaces which represented different levels of current that would be placed on the telegraph line to identify a character. Automated telegraph operations used punched paper tape for both offline message preparation as well as message transmission and reception. Marks were represented by holes, and spaces were represented by leaving the tape intact.

After studying telegraph operations, Vernam proposed punching a "key" tape consisting of randomly selected characters literally pulled from a hat. He proposed electromechanically adding the pulses on the keytape to the pulses of plaintext characters as follows:

plaintext key ciphertext
mark + mark = space
mark + space = mark
space + mark = mark
space + space = space

With Vernam's method, if you assign a binary 1 to a mark and a binary 0 to a space, you have modulo 2 addition forming the ciphertext. Thus, it should not be surprising that Vernam's method for reconstructing the plaintext characters was based upon modulo 2 subtraction.

Vernam's encipherment ideas were converted from paper to hardware by the construction of two devices during 1918. These devices were connected to two teletype- writers and their successful use represents the beginning of automated cryptographic operations.

Generating Random Number Sequences

A true random number sequence is not only an unordered sequence of numbers but also a non-reproducible sequence. According to rumors, certain spy agencies monitor natural processes, such as the height of solar flares on the sun or the number of gamma rays emitted by decaying radioactive material per unit period of time, to develop a sequence of non-reproducible random numbers. These numbers are converted to a binary sequence for use as a key by equipment manufactured to encipher and decipher messages. Diskettes or tapes containing random binary sequences that serve as an enciphering and deciphering key are then prepared for use during different time periods and those disks are sent by courier to government locations where communications security (COMSEC) equipment that uses such disks is located. COMSEC personnel then load the appropriate disks at predefined times to change the keys, since the duplicate use of a binary key sequence would eliminate the reproducibility of the key and make the messages enciphered using a repeating key susceptible to decipherment by foreign governments monitoring communications.

The time, effort, and cost associated with developing true random number sequences and distributing those sequences for use normally restricts its use to government institutions. Commercial products that encipher messages and data files normally do so through the use of pseudo-random number sequences.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Pseudo-random Number Sequences

A pseudo-random number sequence is based on the use of an algorithm to generate a sequence of numbers. Any person who has access to the algorithm can exercise the algorithm and will eventually be able to reproduce the pseudo-random number sequence used by another person. Although this means that the use of pseudo-random number sequences makes an enciphered message decipherable by an unauthorized party, the time and effort required to do so may preclude this from actually happening. Some algorithms may generate millions to tens or hundreds of billions of unique key sequences, forcing those who illicitly obtain access to an enciphered message to use powerful mainframe computers for months or years in an attempt to decipher a message. The selection of an algorithm to generate pseudo-random numbers can provide a very high level of security even though the resulting enciphered message based on the use of that algorithm is breakable. One algorithm which provides a reasonable number of random number sequences is built into the BASIC programming language in the form of the RND function.

The RND Function

The BASIC RND function invokes a routine which generates pseudo-random decimal numbers in the range 0 to 1. In most versions of BASIC, including QuickBASIC, the format of the RND function is RND[(x)], where the value of x determines how the function generates the next random number. Table 6.1 indicates the number returned by the function based upon different argument values.

Table 6.1. Returned RND function number.
Argument Value | Number Returned
x<0 | Always returns the same number for any given x.
x>0 or x omitted | Returns the next random number in the sequence.
x=0 | Returns the last number generated.

When using the RND function in a program with a value of x>0, the same sequence of random numbers is generated each time you execute the program. This generation is illustrated in Listing 6.1 in which the top portion lists a simple program which prints five random numbers through the use of the RND function, while the lower portion shows the results obtained by executing the program three times. Note that each sequence of random numbers generated by the program is exactly the same.

Listing 6.1. Using the RND function.

FOR I = 1 TO 5
PRINT RND(I);
NEXT I
PRINT

.7055475       .533424         .5795186 .2895625 .301948
.7055475       .533424         .5795186 .2895625 .301948
.7055475       .533424         .5795186 .2895625 .301948

Using random numbers for enciphering and deciphering messages dates to the modern spy. During World War II, Nazi agents, as well as Allied spies working in occupied Europe and in neutral countries, enciphered and deciphered messages based upon the use of what is known as a one-time pad.

Each one-time pad consists of a series of pages with each page containing a sequence of random numbers, usually blocked into groups of five digits for convenience. Each page of random numbers was to be used one time, providing the basis for the name of the pad.

To use this method, a spy first prepares his or her message in plaintext, positioning the characters in the message into groups of five letters. Then, the value of each letter in the group is added (using modulo 26 arithmetic) to the value of each random digit to produce the enciphered message. The following example illustrates the enciphering operation attributed to agent Maxwell Smart.

plaintext message | SENDM | OREMO | NEYNO | WXXXX
random digits | 15829 | 31276 | 50341 | 92371
modulo 26 addition
enciphered message | TJVFV | RSGTU | SEBRP | FZAEY

At the spy's home base, the received message is decoded through the use of a second one-time pad whose contents match the contents of the pad used by the spy. For example, the received enciphered message would be deciphered as follows:

received message | TJVFV | RSGTU | SEBRP | FZAEY
random digits | 15829 | 31276 | 50341 | 92371
modulo 26 subtraction | SENDM | OREMO | NEYNO | WXXXX

Unless the one-time pad falls into the hands of enemy agents, the resulting enciphered message is unbreakable. This technique is so secure that its use has been documented in many spy trials occurring during the Cold War, although the capture of the spies was attributed to factors other than the ability of counterspies to read enciphered messages.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

RANDOMIZE Statement

The use of an easily duplicated random number sequence clearly limits its usefulness as an encryption and decryption key. Fortunately, BASIC contains a statement that can be used to reseed the random number generator and produce a different sequence of random numbers. The statement that reseeds the random number generator is the RANDOMIZE statement whose format is RANDOMIZE[expression]. Note that the random number sequence uses the seed to generate a new sequence of random numbers, with the first number becoming a new seed. That new seed is then used to generate a newer seed, and so on, based upon the algorithm used by BASIC to generate random numbers. Thus, the seed generated by the use of the RANDOMIZE statement governs the location where the random number generation process begins.

If the optional expression in the RANDOMIZE statement is omitted, BASIC will pause and display the message "Random Number Seed (-32768 to 32767)?" To change the sequence of random numbers generated by the use of a RND function, you can place a RANDOMIZE statement at the beginning of a program. Listing 6.2 illustrates this concept, with the top portion containing a small program listing that includes the RANDOMIZE statement, while the lower portion shows the results of executing the program three times. Note that the use of a different seed results in the generation of a new random number sequence.

Listing 6.2 Using the RANDOMIZE statement.

RANDOMIZE
FOR I = 1 TO 5
PRINT RND(I);
NEXT I
PRINT

Random-number seed (-32768 to 32767)? -5
.4362451       .3257061.2495473.1388514.1759562
Random-number seed (-32768 to 32767)? -5
.4362451       .3257061.2495473.1388514.1759562
Random-number seed (-32768 to 32767)? 0
.7641413       .3576428.1068624.70753124.804176E-02

Because the value of the random number seed can range from -32678 to 32767, you can obtain the ability to generate 65,536 different random number sequences through the use of the RANDOMIZE statement. You can consider using a variable with a RANDOMIZE statement. Reading the value for the variable from the keyboard serves as a mechanism to provide a program user with the ability to specify the random number sequence to be used to encipher and decipher messages. The seed value then becomes the "code" or "key" for the enciphering and deciphering process.

Working with Random Numbers

Because a majority of the applications for random numbers are for numbers in some range other than 0 to 1, you need a mechanism to change those numbers to integers. That mechanism is obtained by multiplying the value of RND by 10n, where n defines the number of integers you need from 1 to 6. You can then use the BASIC INT function to obtain the integer value of the resulting computation. For example, INT(RND*100) would produce integers whose values are between 0 and 99, while INT(RND*1000) would produce integers whose values are between 0 and 999.

The RANDOM1.BAS Program

The top portion of Listing 6.3 contains the program listing of RANDOM1.BAS. The lower portion shows the results obtained when the program is executed three times. This program generates random numbers between 0 and 25 based upon the use of the RND function and reseeds the random number generator by specifying a reseeding position. You should note that the program simply discards any number exceeding 25 to smooth the resulting random numbers to the range 0 to 25.

Listing 6.3 The RANDOM1.BAS program listing and sample execution with different values assigned to the RANDOMIZE statement.

REM Program RANDOM1.BAS
REM routine to generate random numbers between 0 and 25
       CLS
start: INPUT "Enter a number, 0 to terminate: ", y
       IF y = 0 THEN STOP
       RANDOMIZE (y)
       FOR I = 1 TO 75
       x = INT(RND(I) * 100)
       IF x > 25 THEN GOTO skip
       PRINT x;
skip:  NEXT I
       PRINT
       GOTO start

Enter a number, 0 to terminate: 5
23 7 19 18 22 6 17 15 1 3 17 23 20 13 14
Enter a number, 0 to terminate: 7
11 5 16 20 16 3 15 4 4 23 4 4 24 23 21 19 8 3 11 6 3
Enter a number, 0 to terminate: 9
1 14 14 22 7 22 7 19 23 4 20 9 2 23 5 15 11
Enter a number, 0 to terminate: 0


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The RANDOM2.BAS Program

Listing 6.4 contains the program listing and sample execution of the program RANDOM2.BAS. This simple program was developed to illustrate how you can use random numbers to encipher messages. In this example, our plaintext alphabet is restricted to the 26 uppercase letters. This alphabet is initialized by the INITIALIZE subroutine. Next, the program uses a user-entered number to seed the random number generator and accepts a one-line message read into the string variable TEXT$. The characters in that message are then placed into the string array KEY$.

Listing 6.4 The RANDOM2.BAS program listing and its repeated execution using different random seed numbers and the same plaintext message.

REM Program RANDOM2.BAS
REM Sample program to demonstrate use of random numbers for enciphering
DIM KEY$(80), PLAINTEXT$(26)
CLS
   PRINT "Program RANDOM2.BAS to demonstrate use of random numbers and"
   PRINT "random number seed in enciphering operations"
GOSUB INITIALIZE
START: PRINT
   INPUT "Enter a random seed number, 0 to terminate ", Y
   IF Y = 0 THEN STOP
   RANDOMIZE (Y)
   INPUT "Enter a one line message in UPPERCASE: ", TEXT$
   FOR I = 1 TO LEN(TEXT$)
   KEY$(I) = MID$(TEXT$, I, 1)
   NEXT I
REM Encipher
   PRINT "Enciphered message is               : ";
   FOR I = 1 TO LEN(TEXT$)
   FOR J = 0 TO 25
   IF PLAINTEXT$(J) = KEY$(I) GOTO GOTIT   'locate character position
   NEXT J                                  ' in plaintext array
GOTIT: X = INT(RND * 100)                      'get random number
   IF X > 25 THEN GOTO GOTIT               ' less than 25
   Z = (J + X) MOD 26                      'use MOD 26 addition to add
   PRINT PLAINTEXT$(Z);                    'display enciphered character
   NEXT I
   GOTO START
INITIALIZE:
   RESTORE
   REM Initialize plaintext values
   FOR I = 0 TO 25
   READ PLAINTEXT$(I)
   NEXT I
   DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
   DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN
Program RANDOM2.BAS to demonstrate use of random numbers and
random number seed in enciphering operations

Enter a random seed number, 0 to terminate 5
Enter a one-line message in UPPERCASE: FIREFREDNOW
Enciphered message is: CPKWBXVSORN
Enter a random seed number, 0 to terminate 8
Enter a one-line message in UPPERCASE: FIREFREDNOW
Enciphered message is: RDMXNQVCJXE
Enter a random seed number, 0 to terminate 51
Enter a one-line message in UPPERCASE: FIREFREDNOW
Enciphered message is: ZXDDMCBNHGZ
Enter a random seed number, 0 to terminate 0

The routine labeled ENCIPHER matches each character in the message that was placed in the string array KEY$ to a plaintext character. When a match is found, a branch to the label GOTIT occurs, and a two-digit random number is obtained. Next, the program discards the random number if it exceeded 25 and obtains a new random number, forcing x to be between 0 and 25.

Once an acceptable value of x is obtained, the index value of J, which represents the plaintext character location in the alphabet, is added via modulo 26 addition to the random number. By restricting the largest random number to 25 and the plaintext character position to 26, the largest number produced by adding the two becomes 51. Performing modulo 26 addition prevents a resulting number occurring from different additions, such as would be the case if a random number added to the index resulted in a value of 52 or more. For example, 1 mod 26 is 1, but so is 53 mod 26 and 79 mod 26. By limiting the maximum value of the sum of the index and random number to 51, you ensure you can correctly decipher a message that was enciphered using the technique previously described.

The RANDOM1.CPP Program

Since the purpose of this book is to acquaint you with both BASIC and C++ encryption and decryption coding methods, let's turn our attention to two C++ random number programs. The first program, RANDOM1.CPP, prompts you for a seed and uses that seed to generate a sequence of random numbers between 1 and 100, only displaying those numbers that are less than 25.

The top portion of Listing 6.5 contains the statement in the program RANDOM1.CPP, while the lower portion shows three examples of its execution. Although this program was written to attempt to duplicate the program RANDOM1.BAS, the lack of a C++ RANDOMIZE statement resulted in the development of the function getSeed which is used to return a user-entered seed value. That value is used with the C++ function SRAND to generate 75 random numbers, with only those numbers whose values do not exceed 25 being displayed.

If you compare the random numbers shown at the bottom of Listing 6.5 to those produced by the execution of the BASIC version of the program, you will note different results even though the same seed values were used. These differences result from the fact that most programming languages use a different algorithm to generate random numbers.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 6.5 The RANDOM1.CPP program listing and its repeated execution using different random seed numbers.

/*
random1.cpp
C++ code written by Jonathan Held on April 26, 1998,
using Microsoft Visual C++ version 5.0

Author's note: In C++, if you wish to randomize without
having to enter a seed each time, you can use the
statement:

 srand(time(NULL));

This causes the computer to read its clock to obtain the
value for the seed automatically.  The time function (found
in the library time.h) returns the current "calendar time"
in seconds.
*/

#include <iostream.h>
#include <stdlib.h>

//function prototypes
unsigned getSeed(void);
void generateNumbers(const unsigned);

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 if program terminated normally
//Purpose: driver that makes calls to various procedures defined
//in random1.cpp
//----------------------------------------------------------------
int main(){

  unsigned seed;

  while (seed = getSeed())
   generateNumbers(seed);

 return 0;
}//end main()

//----------------------------------------------------------------
//Function: getSeed()
//Parameters: none
//Return Type: unsigned int - the seed the client entered
//Purpose: gets the seed for the random number generator.  If the
//user enters a negative number, it causes the seed to underflow to
//a very high non-negative number.
//----------------------------------------------------------------
unsigned getSeed(){

 unsigned seed_Number;

  cout << "Enter seed, 0 to terminate: ";
  cin >> seed_Number;

 return seed_Number;
}//end getSeed()

//----------------------------------------------------------------
//Function: generateNumbers()
//Parameters: unsigned int - the seed the client entered
//Return Type: None
//Purpose: Generates random numbers between 1 and 100 and prints
//only those numbers that are less than 25.  Note that if you
//enter the same seed number at different times, you get the same
//"random" number sequence.  Using the srand(time(NULL)) will
//lessen the chance that the same seed is used twice to generate
//random numbers.
//----------------------------------------------------------------
void generateNumbers(const unsigned SEED){

 int ran_num;

 srand(SEED);

 for (int ix=0; ix < 75; ix++){
   ran_num = 1 + rand() % 100;
   if (ran_num < 25)
      cout << ran_num << " ";
  }

  cout << endl << endl;

  return;
}//end generateNumbers()
//end file random1.cpp

A:\c>random1
Enter seed, 0 to terminate: 5
15 22 8 8 18 12 23 11 16 23 7 16 20 16 13 6 5 18 2 11 7

Enter seed, 0 to terminate: 7
23 16 22 17 19 10 2 16 13 13 2 14 4 12 20 18 20 4 20 20

Enter seed, 0 to terminate: 9
10 23 12 18 3 13 17 18 2 5 11 20 9 5 15 22

Enter seed, 0 to terminate:

The RANDOM2.CPP Program

Continuing our exploration of C++ and random numbers, the RANDOM2.CPP program listing is shown in the top portion of Listing 6.6. This program is equivalent to RANDOM2.BAS to encode your message using random numbers. The lower portion of Listing 6.6 illustrates several examples of the execution of the program. When comparing the results to the results obtained by the execution of RANDOM2.BAS, you will note that the use of the same seed values produces different results based upon differences between the two programs in their built-in random number generators. As you will note, RANDOM2.CPP uses random numbers greater than 25 and adds that number to the location of each character in the plaintext array. The sum is then divided by 26 in the same manner as RANDOM2.BAS.

Both RANDOM1.CPP and RANDOM2.CPP and their executable files are contained on the CD-ROM accompanying this book. The executable versions are named RANDOM1.EXE and RANDOM2.EXE. All four programs are located in the C directory on the CD-ROM.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 6.6 The RANDOM2.CPP program listing and its repeated execution using different random seed numbers.

/*
random2.cpp
C++ code written by Jonathan Held on April 28, 1998,
using Microsoft Visual C++ version 5.0
*/

#include <iostream.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

//function prototypes
bool checkInput(char *&);
char* encipher(char *&);
void formatData(char []);
void getMessage(char *&);
unsigned int getSeed(void);
void gotit(const char [], char *&, const int, int);

const int BUFFER_SIZE = 256;

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 if program terminated normally
//Purpose: driver that makes calls to various procedures defined
//in random1.cpp
//----------------------------------------------------------------
int main(){
   unsigned int seed;
   char *message, *result;

  while (seed = getSeed()) {
   srand(seed);
   getMessage(message);
   cout << "\nYou entered: " << message << endl;
   result = encipher(message);
   cout << "Enciphered message is " << result << endl << endl;
   delete message;
 }

 return 0;
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the message the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
 bool error = false;
 int count = strlen(input);

 for (int ix=0; ix<count; ix++){

  int char_value = static_cast<int>(*(input+ix));
  //determine if the user did not enter an uppercase character
  if ((char_value < 65) || (char_value > 90)){
      error = true;
      cerr << "\aYou entered an invalid message!" << endl << endl;
       break;
   }
}

if (count == 0){
  cerr << "\aYou entered an invalid message!" << endl << endl;
  error = true;
}
 return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: encipher()
//Parameters: inp - the user's input that will be enciphered
//Return Type: char * - a pointer to the encrypted string
//Purpose: returns a pointer to the encrypted text
//----------------------------------------------------------------
char* encipher(char *&inp){

 //yet another, and easier way to represent the plaintext
 //array
 const char plaintext[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
 int index = 0;

 char *encrypted = new char[strlen(inp)+1];
 assert(encrypted);

 for (int ix=0; ix < strlen(inp); ix++){
   for (int jx=0; jx <26; jx++){
    if (plaintext[jx] == inp[ix]){
          gotit(plaintext, encrypted, jx, index++);
          break;
      }
   }
}

//don't forget to insert the null terminator
encrypted[strlen(inp)] = '\0';

 return encrypted;
}//end encipher();

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[]){
  for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
   if (*(data+nx) == ' '){
      //do nothing - skip over the space in the data
   }
   else {
     *(data+mx++) = *(data+nx);
   }
 }

 //don't forget to add the null terminator
 *(data+mx) = '\0';

 return;
}//end formatData()

//----------------------------------------------------------------
//Function: getMessage()
//Parameters: input - user's one-line message
//Return Type: None
//Purpose: Gets a one-line message from the user.
//----------------------------------------------------------------
void getMessage(char *&input){

  char buffer[BUFFER_SIZE] = {'\0'};
  bool error = false;

  do{
   cout << "Enter a one-line message (no spaces) in UPPERCASE: ";
   cin.getline(buffer, BUFFER_SIZE, '\n');
   assert(input = new char[strlen(buffer) + 1]);
   strcpy(input, buffer);
   formatData(input);
   error = checkInput(input);

   if (error)
      delete input;

 } while (error);

 return;
}//getMessage()
//----------------------------------------------------------------
//Function: getSeed()
//Parameters: none
//Return Type: unsigned int - the seed the client entered
//Purpose: gets the seed for the random number generator.  If the
//user enters a negative number, it causes the seed to underflow to
//a very high non-negative number.  If the user enters an
//alphanumeric sequence, e.g., "A123", then seed_Number is set to
//zero.
//----------------------------------------------------------------
unsigned getSeed(){

 unsigned int seed_Number;

 cout << "Enter seed, 0 (or non-digit) to terminate: ";
 cin >> seed_Number;
 cin.ignore();

 return seed_Number;
}//end getSeed()

//----------------------------------------------------------------
//Function: gotit()
//Parameters: p_text - the plaintext array, i.e., characters "A-Z"
//      enc_msg - a pointer to the message that will be encoded
//      loc - the location of each encoded character in the
//      plaintext array
//Return Type: None
//Purpose: encodes the user's input using random numbers
//----------------------------------------------------------------
void gotit(const char p_text[], char *&enc_msg, int loc, int index){

 int random_num, z;

 do {
  random_num = rand() % 100;
 } while (random_num > 25);

 z = (loc + random_num) % 26;
 enc_msg[index] = p_text[z];

 return;
}//end gotit()
//end file random2.cpp
A:\c>RANDOM2
Enter seed, 0 (or non-digit) to terminate: 5
Enter a one-line message (no spaces) in UPPERCASE: FIREFREDNOW

You entered: FIREFREDNOW
Enciphered message is TDYLWCANCKC

Enter seed, 0 (or non-digit) to terminate: 8
Enter a one-line message (no spaces) in UPPERCASE: FIREFREDNOW

You entered: FIREFREDNOW
Enciphered message is QLEGOKYPGRV

Enter seed, 0 (or non-digit) to terminate: 51
Enter a one-line message (no spaces) in UPPERCASE: FIREFREDNOW

You entered: FIREFREDNOW
Enciphered message is KIYOSXAWQTW

Enter seed, 0 (or non-digit) to terminate:


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Modulo 26 Arithmetic

To better understand the routine necessary to decipher the previously enciphered series of messages that used a random number-based enciphering program requires a review of modulo 26 addition and modulo 26 subtraction. Suppose the position values of the first four characters in a plaintext message with respect to their location in the sequential plaintext alphabet are 6, 2, 4, and 18, while a randomly generated number sequence based upon a defined seed was 12, 14, 22, and 17.

The top of Figure 6.1 illustrates the enciphering process performed through the use of modulo 26 addition. Once the enciphered characters reach their destination, the same sequence of random numbers is used to reconstruct the original characters. However, this time modulo 26 subtraction is performed as illustrated in the bottom portion of Figure 6.1. When performing modulo 26 subtraction, if the numerator is smaller than the subtrahend you "borrow" 1, which in effect has a decimal value of 26, prior to performing the required subtraction. Thus, 9-17 becomes (26+9)-17, or 18, when modulo 26 subtraction is performed.

Figure 6.1 Enciphering and deciphering using modulo 26 addition and subtraction.

Now that you have an appreciation of the modulo subtraction process, let's focus our attention upon a small program developed to decipher a previously enciphered one-line message based upon a specified seed for the BASIC random number generator.

The DRANDOM2.BAS Program

Listing 6.7 contains the statements in the program DRANDOM2.BAS and three examples of its execution. In keeping with the file naming conventions previously established, I labeled the program DRANDOM2.BAS to indicate that it deciphers a message previously enciphered using the program RANDOM2.BAS.

Listing 6.7 The DRANDOM2.BAS program listing and its repeated execution using different random seed numbers and different ciphertext messages to re-create a common plaintext message.

REM Program DRANDOM2.BAS
DIM KEY$(80), PLAINTEXT$(26)
CLS
   PRINT "Program DRANDOM2.BAS to demonstrate use of random numbers
                and"
   PRINT "random number seed in deciphering operations"
GOSUB INITIALIZE
START: PRINT
   INPUT "Enter a random seed number, 0 to terminate ", Y
   IF Y = 0 THEN STOP
   RANDOMIZE (Y)
   INPUT "Enter a one line message in UPPERCASE: ", TEXT$
   FOR I = 1 TO LEN(TEXT$)
   KEY$(I) = MID$(TEXT$, I, 1)
   NEXT I
REM Decipher
   PRINT "Deciphered message is               : ";
   FOR I = 1 TO LEN(TEXT$)
   FOR J = 0 TO 25
   IF PLAINTEXT$(J) = KEY$(I) GOTO GOTIT
   NEXT J
GOTIT: X = INT(RND * 100)
   IF X > 25 THEN GOTO GOTIT
   IF J < X THEN J = J + 26
   Z = J - X
   PRINT PLAINTEXT$(Z);
   NEXT I
   GOTO START
INITIALIZE:
   RESTORE
   REM Initialize plaintext values
   FOR I = 0 TO 25
   READ PLAINTEXT$(I)
   NEXT I
   DATA "A","B","C","D","E","F","G","H","I","J","K","L","M","N"
   DATA "O","P","Q","R","S","T","U","V","W","X","Y","Z"
RETURN

Program DRANDOM2.BAS to demonstrate use of random numbers and
random number seed in deciphering operations

Enter a random seed number, 0 to terminate 5
Enter a one line message in UPPERCASE: CPKWBXVSORN
Deciphered message is          : FIREFREDNOW
Enter a random seed number, 0 to terminate 8
Enter a one line message in UPPERCASE: RDMXNQVCJXE
Deciphered message is          : FIREFREDNOW
Enter a random seed number, 0 to terminate 51
Enter a one line message in UPPERCASE: ZXDDMCBNHGZ
Deciphered message is          : FIREFREDNOW
Enter a random seed number, 0 to terminate 0

After entering the random number generator's seed number and the one-line enciphered message, the decipher routine matches each character in the message against the plaintext alphabet stored in the array PLAINTEXT$. Once a match occurs, you have located the correct position value or location of the ciphertext character in the plaintext character in the plaintext alphabet—a value assigned to the variable J. After the value of J is obtained, a branch to the label GOTIT occurs and a random number between 0 and 25 is extracted. Because you must perform modulo 26 subtraction to reconstruct the plaintext character, compare the value of J to the value of x and increase J by 26 if the value of J is less than x. Then we subtract the value of x from the value of J, in effect performing modulo 26 subtraction. As indicated in the lower portion of Listing 6.7, this process enables you to reconstruct the plaintext message.

A comparison of the execution of RANDOM2.BAS and DRANDOM2.BAS contained in the lower portions of Listings 6.4 and 6.7 verifies the correctness of the deciphering process. That is, plaintext messages enciphered using a defined random number seed value in Listing 6.4 are entered as ciphertext messages in Listing 6.7 with the same seed number. This results in the correct decipherment of each message into its original plaintext contents.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DRANDOM2.CPP Program

Listing 6.8 lists the statements in the C++ program DRANDOM2.CPP. This program represents the C++ version of DRANDOM2.BAS and can be considered to represent the decipher version of RANDOM2.CPP. That is, a message enciphered using RANDOM2.CPP can be deciphered using DRANDOM2.CPP as long as you enter the same seed for each program. Figure 6.2 illustrates the execution of RANDOM2.EXE and DRANDOM2.EXE using the same seed number. As indicated in Figure 6.2, the message HELP was enciphered as QAJA by RANDOM2.EXE based upon the use of 9 for the seed. Then DRANDOM2.EXE was executed using the same seed value; however, the one-line message input to the program is QAJA, which results in the deciphered message HELP.

Figure 6.2 Using the C++ executable file DRANDOM2.EXE to decipher a one-line message enciphered using RANDOM2.EXE.

Listing 6.8 The DRANDOM2.CPP program listing.

/*
drandom2.cpp
C++ code written by Jonathan Held on May 1, 1998,
using Microsoft Visual C++ version 5.0
*/

#include <iostream.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

//function prototypes
bool checkInput(char *&);
char* decipher(char *&);
void formatData(char []);
void getMessage(char *&);
unsigned int getSeed(void);
void gotit(const char [], char *&, const int, int);

const int BUFFER_SIZE = 256;

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 if program terminated normally
//Purpose: driver that makes calls to various procedures defined
//in random1.cpp
//----------------------------------------------------------------
int main(){

  unsigned int seed;
  char *message, *result;

  while (seed = getSeed()) {
   srand(seed);
   getMessage(message);
   cout << "\nYou entered: " << message << endl;
   result = decipher(message);
   cout << "Deciphered message is " << result << endl << endl;
   delete message;
}

 return 0;
}//end main()

//----------------------------------------------------------------
//Function: checkInput()
//Parameters: input - the message the user entered
//Return Type: bool - true if the input string contains an error,
//       false otherwise
//Purpose: Checks the user's keyword for invalid characters.
//----------------------------------------------------------------
bool checkInput(char * &input)
{
 bool error = false;
 int count = strlen(input);

 for (int ix=0; ix<count; ix++){

  int char_value = static_cast<int>(*(input+ix));
  //determine if the user did not enter an uppercase character
  if ((char_value < 65) || (char_value > 90)){
    error = true;
    cerr << "\aYou entered an invalid message!" << endl << endl;
    break;
  }
 }
if (count == 0){
   cerr << "\aYou entered an invalid message!" << endl << endl;
   error = true;
 }

 return error;
}//end checkInput()

//----------------------------------------------------------------
//Function: decipher()
//Parameters: inp - the user's input that will be deciphered
//Return Type: char * - a pointer to the decrypted string
//Purpose: returns a pointer to the decrypted text
//----------------------------------------------------------------
char* decipher(char *&inp){
//yet another, and easier way to represent the plaintext
//array
const char plaintext[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
int index = 0;

char *decrypted = new char[strlen(inp)+1];
assert(decrypted);

for (int ix=0; ix < strlen(inp); ix++){
  for (int jx=0; jx <26; jx++){
   if (plaintext[jx] == inp[ix]){
         gotit(plaintext, decrypted, jx, index++);
         break;
   }
  }
 }

//don't forget to insert the null terminator
decrypted[strlen(inp)] = '\0';

return decrypted;
}//end encipher();

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[]){

 for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
  if (*(data+nx) == ' '){
     //do nothing - skip over the space in the data
  }
  else {
   *(data+mx++) = *(data+nx);
  }
}

//don't forget to add the null terminator
*(data+mx) = '\0';
 return;
}//end formatData()

//----------------------------------------------------------------
//Function: getMessage()
//Parameters: input - user's one-line message
//Return Type: None
//Purpose: Gets a one-line message from the user.
//----------------------------------------------------------------
void getMessage(char *&input){

 char buffer[BUFFER_SIZE] = {'\0'};
 bool error = false;

 do{
  cout << "Enter a one-line message (no spaces) in UPPERCASE: ";
  cin.getline(buffer, BUFFER_SIZE, '\n');
  assert(input = new char[strlen(buffer) + 1]);
  strcpy(input, buffer);
  formatData(input);
  error = checkInput(input);

  if (error)
     delete input;

 } while (error);

 return;
}//getMessage()

//----------------------------------------------------------------
//Function: getSeed()
//Parameters: none
//Return Type: unsigned int - the seed the client entered
//Purpose: gets the seed for the random number generator.  If the
//user enters a negative number, it causes the seed to underflow to
//a very high non-negative number.  If the user enters an
//alphanumeric sequence, e.g., "A123", then seed_Number is set to
//zero.
//----------------------------------------------------------------
unsigned getSeed(){
 unsigned int seed_Number;

 cout << "Enter seed, 0 (or non-digit) to terminate: ";
 cin >> seed_Number;
 cin.ignore();

 return seed_Number;
}//end getSeed()

//----------------------------------------------------------------
//Function: gotit()
//Parameters: p_text - the plaintext array, i.e., characters "A-Z"
//      enc_msg - a pointer to the message that will be encoded
//      loc - the location of each encoded character in the
//      plaintext array
//Return Type: None
//Purpose: encodes the user's input using random numbers
//----------------------------------------------------------------
void gotit(const char p_text[], char *&dec_msg, int loc, int index){

 int random_num, z;

 do {
   random_num = rand() % 100;
 } while (random_num > 25);

 if (loc < random_num)
         loc+= 26;

 z = loc - random_num;
 dec_msg[index] = p_text[z];

 return;
}//end gotit()
//end file drandom2.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Constructing an Enciphering Program

Now that you have reviewed the operation and utilization of the BASIC RANDOMIZE statement, the RND function, and the equivalent C++ statement, you have the foundation that enables you to construct enciphering programs which will use the built-in BASIC random number generator and the equivalent C++ statements. However, since that random number generator is available to anyone who has a computer and appropriate software, an intercepted enciphered message could be easily deciphered if you simply use a numeric value for selecting an available random number seed. This is because there are a limited number of random number seeds from which you can select in comparison to the number of combinations obtained by mixing up an alphabet. Prior to creating your random number-based enciphering program, first focus your attention upon extending the random process to obtain a much larger series of possible enciphering combinations.

Extending the Random Process

As previously mentioned in this chapter, the BASIC random number seed varies from -32768 to 32767, providing a total of 65,536 random number sequences (remember that a seed value of 0 also produces a sequence). Therefore, a person who intercepts or otherwise obtains a copy of your enciphered message can decipher that message by trying each random number seed. This obviously takes some time; however, it subjects the contents of your enciphered message to decipherment through the use of a well-known maximum number of decipherment attempts.

There are numerous methods you can consider to extend the level of security in using BASIC's and C++'s random number generators. For example, to obtain an additional level of security you can consider using two key numbers in your encipherment and decipherment process. The first key number, as previously explained, selects the random number seed. The second number would either directly or indirectly set the beginning location in the random number sequence selected. This process could theoretically result in an infinite number of combinations an unauthorized person might be faced with during an illegal decipherment process.

The top of Listing 6.9 contains a routine designed to generate random numbers between 0 and 25 based upon a defined seed and location within the selected seed. The bottom portion of Listing 6.9 shows the results obtained by executing the program three times. In examining Listing 6.9, note that each time the program was executed, the same seed was executed, but the position in the seed was changed by the use of a FOR-NEXT loop in which the statement dummy=RND is executed x times, where x represents the value of the location in the seed. The first random number considered for use by the program is located at position x+1 in the selected seed.

Listing 6.9 The routine to generate a random number sequence based upon a defined seed and a location in the seed.

REM routine to generate random numbers between 0 and 25
REM based on a defined seed and location in the seed

start: INPUT "Enter your key numbers separated by commas: ", y, x
       RANDOMIZE (y)
       FOR i = 1 to x: dummy = RND: NEXT I
       FOR i = 1 to 75
       x = INT(RND(i) * 100)
       IF x > 25 THEN GOTO skip
       PRINT x;
skip:  NEXT I
       PRINT

Enter your key numbers separated by commas: 1,1
 10 10 10 5 21 22 11 4 5 18 21 15 17 8 15 13 15 9
Enter your key numbers separated by commas: 1,50
 17 8 15 13 15 9 10 16 12 18 6 0 15 9 23 5 11
Enter your key numbers separated by commas: 1,100
 15 9 23 5 11 3 14 2 10 13 6 4 10 10 23 22 5 11 13 8 2 21

The use of a FOR-NEXT statement to select a location in a random number sequence places a limit on the location within the sequence you can begin your encipherment process. This is because each version of BASIC and C++ has a limit on the maximum value that can be assigned to a variable. For example, the maximum value of an integer is 32,767 in QuickBASIC, while a long integer can have a value up to 2,147,483,647 in that language. You can also use single- or double-precision numeric variables to extend the maximum value allowed by the language and the number of combinations allowed in selecting a position within a random number sequence. However, the use of high numbers, whether they are integers or numerics, can result in significant user waiting periods while the computer cycles through a large number of random numbers within a particular random number seed to locate the desired position within a random number sequence. For example, the execution of the FOR-NEXT loop FOR I = 1 TO x shown at the top of Listing 6.9 could result in the user of the program waiting for several minutes if an extremely large value was entered for x and the program was executed on an old 8088-based computer.

In addition to a long waiting period, the use of a large number is difficult to remember and might restrict the use of the preceding example to a simple low value or easy-to-remember number, such as 500, 1,000, and so on. The use of a number limited to 1,000 or so still significantly extends the number of combinations resulting from the selection of a random number sequence and position within the sequence. For 1,000 positions within a sequence, the total number of combinations an unauthorized person would need to try to decipher a message would be extended to 65,536,000 since there are 65,536 different random number seeds. Even if the right combination was discovered halfway through the search sequence, this could result in over 32 million combinations being used for the attempted decipherment. For a message such as BID THIRTY TWO MILLION ON TUESDAY AT THE AUCTION, any decipherment that occurs after Tuesday may be worthless. The use of two keys that select a random number sequence and position within the sequence may provide a significant level of message protection for many users. However, recognizing the need of most users for an extra level of protection, I'll demonstrate a few easy-to-implement techniques that can be used to obtain that desired additional level of protection.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Extending the Combinations

You can substantially increase both the number of combinations available for starting the random number sequence to be used in a program as well as the speed in which the user of the program obtains access to the sequence. To accomplish this, use a third key to increment the FOR-NEXT loop to both speed up the positioning process as well as increase the number of combinations an unauthorized party must try in an attempt to decipher an enciphered message.

The top portion of Listing 6.10 contains the program listing for entering a three-number sequence to select a position within a random number sequence defined by a random number seed from which to commence the encipherment process at the next random number position. In this example, the first number is assigned to the variable x, and is used to select the random number seed sequence. The second number entered is assigned to the variable y and defines the limit of the FOR-NEXT loop. The value of the third number entered is assigned to the variable z and defines the increment used in the FOR-NEXT loop. Each of the numbers can be considered to represent a key. You can consider this elementary program a random number generator based upon the use of a three-numeric key sequence.

Listing 6.10 Sample program that uses three numbers to rapidly position the starting point within a random number sequence of a defined random number seed.

INPUT "Enter your three key numbers, each separated by a comma: ",x,y,z
RANDOMIZE (x)
FOR i = 1 to y STEP z
dummy = RND
NEXT I
FOR I = 1 TO 10
PRINT INT(RND * 100)
NEXT I
PRINT

Enter your three key numbers, each separated by a  comma: 12,24,4
 1 87 10 94 95 82 26 71 21 57
Enter your three key numbers, each separated by a  comma: 12,24,9
 50 27 89 1 87 10 94 95 82 26
Enter your three key numbers, each separated by a  comma: 12,240,9
 31 88 52 70 32 72 70 37 95 12
Enter your three key numbers, each separated by a  comma: 12,110543,43
 3 3 17 51 70 18 51 35 79 27
Enter your three key numbers, each separated by a  comma: 110543,1993,12
 65 34 48 63 89 66 63 2 41 29

The lower portion of Listing 6.10 illustrates the results obtained from executing the program five times in sequence using different key combinations.

The first two examples illustrated at the bottom of Listing 6.10 could have used the month and day of the month for the first two keys and an hour for the third key. The third example could have used the Julian day for the day instead of the day of the month. In the fourth example, the second key could be a person's date of birth. In the fifth example, perhaps the first key is the person's date of birth, while the second and third keys are the current year and month.

Regardless of the manner in which values are assigned to each key, the use of three keys can provide an extremely large number of combinations. However, the key (no pun intended) to obtaining a large number of combinations is the mechanism used to generate key values. By selecting easy-to-remember but relatively large values for each of the three keys, you can force an unauthorized person with access to a deciphering program to obtain the power of a mainframe or modern Pentium II computer to decipher your message. For example, using a date in the form of mmddyy for the first key, a year in the form of yyyy for the second key, and a number in the form of dd for the third key could result in up to 10 billion possible combinations an unauthorized person obtaining access to an enciphered message might have to try in an attempt to decipher your message! Now that's a significant order of magnitude beyond the 65,536 random number seed sequences and adds a degree of decipherment difficulty that corresponds to a level of protection afforded by many commercially available systems whose purchase price significantly exceeds the cost of this book.

Creating Your Own Random Number Generator

Although the preceding three key examples indicated a series of approximately 10 billion combinations would be required, when you use a built-in random number generator you simplify the life of code-breakers. This is because if they have a bit of knowledge concerning the random number generation scheme you are using, it is a relatively simple process to program a computer to cycle through every possible combination. Because of this, the use of a built-in random number generator entails a bit of a risk. Although it could take a long time to lock onto your random number sequence, who would have expected desktop computers to operate at 400 MHZ a few years ago? This means that the months, days, or hours of attack required today may be reduced to days, hours, or minutes tomorrow. Recognizing this fact, let's examine how you can easily create your own pseudo-random number generator. Because there are an infinite number of equations you could use to create your own generator, doing so can make the life of a code-breaker significantly more difficult.

Because a pseudo-random number generator requires an equation, let's define one. Let's assume the following equation is your magic formula for generating a random number:

      INTEGER  MYRANDOM * 37 + 462
              (        42         )

Although it may appear the preceding formula represents a randomly developed equation, let me explain its operation to shed light on how you can develop your own random number generator. If you assume that the variable MYRANDOM ranges between 0 and 100, then the maximum value of the preceding formula becomes:

      INTEGER    100 * 37 + 462   or 99
                (       42     )

Thus, your magic formula would return values from 11 to 99, as when MYRANDOM has a value of zero the formula returns a value of 11.

To use a magic formula, you could accept a number from the keyboard and assign its value to the variable MYRANDOM. After computing a pseudo-random number, you could both use that number and assign its value to MYRANDOM to generate a second pseudo-random number. For example, assume you entered 10 from the keyboard. The first time the magic formula is computed you would obtain:

      INTEGER    10 * 37 + 462    or 19
                (       42    )

Using 19 as input to the formula, you would obtain:

      INTEGER    19 * 37 + 462    or 27
                (       42    )

Although the preceding magic formula was rather simplistic, it illustrates how you can create your own pseudo-random number generator. In concluding this chapter, I'll illustrate the creation of a program to generate pseudo-random numbers without using a language's built-in random number generator function.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Program Development

In this section, you will construct a program for enciphering and deciphering based upon the use of Microsoft's QuickBASIC random number generator, as well as a C++ equivalent. To do so, you will develop a mechanism to obtain a position in the random number generator which is difficult to duplicate.

To develop a subroutine to obtain a difficult-to-duplicate position within the random number generator, you must consider both the number of character positions you will use in a code your program will operate upon to obtain a location in the random number generator as well as the characters that can be used for each position. For example, a two-position code restricted to numerics is limited to containing values from 00 to 99, or 100 (102) combinations. If those two positions can contain alphanumeric characters, each position can contain 36 combinations, while the two positions represent a total of 362, or 1,296, combinations.

Table 6.2 lists the number of combinations resulting from the use of code lengths varying from 1 to 10 positions, with each position containing up to 10 (numeric only), 26 (alphabetic only), 36 (alphanumeric), and 128 (any seven-bit ASCII) permissible characters. As indicated in Table 6.2, an expansion of the number of code positions and/or the number of characters permitted to be entered in each position significantly increases the possible number of code combinations. Because most people can easily remember a six-position code, use this type of code to develop a mechanism to obtain a difficult-to-duplicate position within the built-in BASIC random number generator. Using six positions and allowing alphanumeric characters to be placed in each position results in 2,176,782,336 possible combinations! If you extend the characters that can be placed into each code position to the normal ASCII code, you would obtain 439*1010 possible combinations. Because characters should be converted to their ASCII numeric representation to use for positioning purposes, you need a mechanism to perform this operation.

Table 6.2 Code combinations based upon code length and characters allowed.
X | 10^X | 26^X | 36^X | 128^X
1 | 100.00E-01 | 260.00E-01 | 360.00E-01 | 128.00E+00
2 | 100.00E+00 | 676.00E+00 | 129.60E+01 | 163.84E+02
3 | 100.00E+01 | 175.76E+02 | 466.56E+02 | 209.72E+04
4 | 100.00E+02 | 456.98E+03 | 167.96E+04 | 268.44E+06
5 | 100.00E+03 | 118.81E+05 | 604.66E+05 | 343.60E+08
6 | 100.00E+04 | 308.92E+06 | 217.68E+07 | 439.80E+10
7 | 100.00E+05 | 803.18E+07 | 783.64E+08 | 562.95E+12
8 | 100.00E+06 | 208.83E+09 | 282.11E+10 | 720.58E+14
9 | 100.00E+07 | 542.95E+10 | 101.56E+12 | 922.34E+16
10 | 100.00E+08 | 141.17E+12 | 365.62E+13 | 118.06E+19

Listing 6.11 illustrates a short segment of code to read characters and display their ASCII values. In this example, the program executes continuously until a break occurs. After reading a string that is assigned to the string variable A$, the program obtains the length of the string. Next, a FOR-NEXT loop extracts each character from the string through the use of the MID$ function and prints the ASCII value of each character through the use of the ASC function. The lower portion of Listing 6.11 illustrates the resulting ASCII values of each character from six strings entered as codes. Note that the code can be all alphabetic (GODAWGS), numeric (362436), alphanumeric (NINE9), or include special characters such as the exclamation mark that are included in the normal ASCII character set (POWER!).

Listing 6.11 Using ASCII character values.

CLS
START:
       INPUT A$
       X = LEN(A$)
       FOR I = 1 TO X
       PRINT ASC(MID$(A$, I, 1)); " ";
       NEXT I
       PRINT
GOTO START

? GODAWGS
 71    79    68    65    87    71    83
? YEATIGERS
 89    69    65    84    73    71    69    82    83
? 362436
 51    54    50    52    51    54
? POWER!
 80    78    87    69    82    33
? NINE9
 78    73    78    69    57
? $39.99
 36    51    57    46    57    57
?

At the top of Listing 6.12 you will find a C++ version of the BASIC program in Listing 6.11. The C++ version looks at each character entered. This can result in an interesting display that may require a bit of interpretation. Thus, let's examine the output of the program shown in the lower portion of Listing 6.12.

In examining the execution of the program ASCII.CPP, note that if you enter characters separated by a space, such as "1 2 4 6," the ASCII value of 32 which represents a space will be displayed. Thus, the ASCII values for the sequence "1 2 4 6" is displayed as "49 32 50 32 52 32 54." The preceding is true for both alphabetic and numeric characters as indicated by the entry of the character sequence "A B C D" in the lower portion of Listing 6.12.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 6.12 The ASCII.CPP program listing and sample execution.

/*
ASCII.cpp
C++ code written by Jonathan Held, May 8, 1998, using Microsoft's
Visual C++ version 5.0.
*/

#include <iostream.h>
#include <string.h>
#include <iomanip.h>

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally, any other
//value indicated abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main(){
  char buffer[1000];
  const char *spacing = "              ";
  cout << "Use CTRL-C to terminate program." << endl << endl;
  while(true){

   cout << "Enter a sequence of characters: ";
   cin.getline(buffer, 1000, '\n');
   cout << endl << "ASCII values: ";
   int ix = strlen(buffer);
    for (int jx=0; jx<ix; jx++){
     if ((jx!=0) && (!(jx%10)))
        cout << endl << spacing;
     cout << setw(3) << (int) buffer[jx] << " ";
    }
    cout << endl << endl;
  }
 return 0;
}//end main()
//end file listing

C:\>ASCII
Use CTRL-C to terminate program.

Enter a sequence of characters: 12

ASCII values:  49  50

Enter a sequence of characters: 1 2 4 6

ASCII values:  49  32  50  32  52  32  54

Enter a sequence of characters: A B C D

ASCII values:  65  32  66  32  67  32  68

At this point it may be useful to examine a six-position code used to perform a series of manipulations to obtain a position within a random number seed. Although I will review one series of manipulations, a six-character code provides 6!, or 720, possible methods by which the characters can be used if they are used one character at a time. If you desire, you can change the method of code character manipulations to fit your specific requirements.

To manipulate the six-position code, first multiply the ASCII value of each of the first two characters in the code to use as the RANDOMIZE seed setting. Next, multiply the ASCII values of the third and fourth characters in the code to serve as a first limit for entry into a position within the selected seed. Use the ASCII value of the fifth code character to increment towards the limit. As a further offset into the random number sequence, multiply the ASCII values of the fifth and sixth code characters and use the result as a second limit. Then, sequence towards that limit using the ASCII value of the fourth code character as a mechanism for incrementing towards that limit.

The use of the six-character grouping we previously discussed is illustrated graphically in Figure 6.3. An analysis of the combinations obtained from the use of that code is warranted, as it indicates how good intentions can go astray and why many persons throughout the history of mankind have become so involved in developing "unbreakable" techniques that they literally couldn't see the trees in the forest and did not understand the weakness they created in their quest for increasing the complexity of their code design.

Figure 6.3 Using the code characters.

In Figure 6.3 the use of a two-character position to generate a seed setting when each position is restricted to alphanumeric characters reduces the number of seed combinations to 362, or 1,296, which is significantly less than the maximum number of seed settings supported in BASIC. The entry into the selected seed is based upon the multiplication of the value of code positions three and four, which are added to the product of the values of code positions five and six. This means that the maximum value of the displacement into the selected seed is 2,592 positions when code positions are limited to alphanumeric characters. Thus, the maximum number of combinations one has to try in an attempt to locate the position within a selected seed for enciphering a message becomes 1,296 seeds * 2,592 positions, or 3,359,232 combinations! Although this is still a formidable number, it is certainly less formidable than the maximum number of combinations that can be obtained from a six-character position code. You may wish to consider alternative methods to use the characters in a code to initiate a random number sequence within a seed at a defined position. Later in this chapter I will discuss an alternative method you can consider for the use of the characters in a code to initiate a sequence of random numbers. However, for the moment I will focus on the way you can code the previously described code character manipulations for use in programs that encipher and decipher messages based upon the use of the built-in BASIC random number generator. I will also develop programs that will accept any ASCII character in each code position. This will significantly extend the number of possible combinations others must try in an attempt to decipher a message enciphered through the use of the program RANDOM3.BAS developed later in this chapter. For example, with each character code position capable of containing any conventional ASCII character, the use of code positions one and two for selecting the seed increases the number of seeds capable of being used to 127 * 127, or 16,129. Once you have an appreciation for the coding and operation of "secret" code-based programs, you will learn how you can make those "secret" codes even less vulnerable. Following our customary method, once I describe a BASIC program, I will then indicate the appropriate C++ program.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The POSITION.BAS Program

The top of Listing 6.13 gives the statements in the demonstration program POSITION.BAS which accepts a "secret" code up to six characters in length and uses the ASCII values of those characters in the manner previously discussed as a mechanism for positioning into a place in the random number generator. In this program, the subroutine SETUP actually performs the previously described manipulation of the ASCII value of each code character. The lower portion of Listing 6.13 illustrates the execution of the program using five different "secret" codes. The program displays the numeric (ASCII) value of each code character and the first five random numbers following the execution of the positioning subroutine. Any ASCII character can be used for each code character; however, control characters that are non-printable are obviously not printed when used as code characters.

Listing 6.13 The POSITION.BAS program listing and examples of its repeated execution.

REM Program POSITION.BAS demonstrates the positioning to a place in the
REM random number generator based upon a secret code up to 6 characters
AGN:
   INPUT "Enter your secret code (6 characters maximum): "; CODE$
   IF LEN(CODE$) > 6 THEN GOTO AGN
GOSUB SETUP
   PRINT "Numeric value of code characters: ";
   FOR I = 1 TO 6
   PRINT CODE(I); " ";
   NEXT I
   PRINT: PRINT "1st five random numbers: ";
   FOR I = 1 TO 5: PRINT RND;: NEXT I: PRINT: PRINT
   STOP
SETUP:
   REM routine to position random number generator
   FOR I = 1 TO LEN(CODE$)
   CODE(I) = ASC(MID$(CODE$, I, 1))
   NEXT I
   SEED = CODE(1) * CODE2
   MAX = CODE(3) * CODE(4)
   RANDOMIZE SEED
   FOR I = 1 TO MAX STEP CODE(5)        'position into sequence
   DUMMY = RND
   NEXT I
   FOR I = 1 TO CODE(5) * CODE(6) STEP CODE(4) 'further offset into
    sequence
   DUMMY = RND
   NEXT I
RETURN

Enter your secret code (6 characters maximum): ? HELPME
Numeric value of code characters:  72   69   76   80   77   69
1st five random numbers: .8671363 .3378422 .4237103 .6096486  .9087062

Enter your secret code (6 characters maximum): ? 9NINES
Numeric value of code characters:  57   78   73   78   69   83
1st five random numbers:  .4766153 .8707944 .8031864 .3487226 .6214768

Enter your secret code (6 characters maximum): ? 382438
Numeric value of code characters:  51   56   50   52   51   56
1st five random numbers: .944414 .949271 .5596389 .7912182 9.593111E-02

Enter your secret code (6 characters maximum): ? 3CARDS
Numeric value of code characters:  51   67   65   82   68   83
1st five random numbers: .4237103 .6096486 .9087062 .8784156 .0561406

Enter your secret code (6 characters maximum): ? helpme
Numeric value of code characters:  104   101   108   112   109   101
1st five random numbers: .5051253 .7871563 .9859242 .1925381 .3514438

The POSITION.CPP Program

Continuing our dual language approach to illustrate major encryption coding techniques, the top portion of Listing 6.14 contains the program listing for the C++ version of POSITION.BAS. This program, which has the filename POSITION.CPP, is contained on the CD-ROM in the C directory. Similarly, the executable version of this program is stored under the name POSITION.EXE in the same directory. The lower portion of Listing 6.14 illustrates two examples of the execution of POSITION.CPP. If you compare the execution of POSITION.CPP to the execution of POSITION.BAS, you will note that the use of the same secret code produces different results. As previously explained, the random number generators used in BASIC and C++ differ and result in the difference between the results display when executing POSITION.BAS and POSITION.CPP.

Listing 6.14 The POSITION.CPP program listing and examples of the execution of the program.

/*
position.cpp
C++ code written by Jonathan Held, May 8, 1998, using Microsoft's
Visual C++ version 5.0.
*/

//standard includes
#include <iostream.h>
#include <string.h>
#include <stdlib.h>
#include <iomanip.h>

//function prototypes
void firstFiveRndNumbers(char *);

void getCode(char *&);
void numericValues(char *);
//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally, any other
//value means abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main(){

  cout << "\aHit CTRL-C to terminate this progam." << endl << endl;
  while (true){
   char * the_Code;
   getCode(the_Code);
   numericValues(the_Code);
   firstFiveRndNumbers(the_Code);
   cout << endl;
 }
 return 0;
}//end main()

//----------------------------------------------------------------
//Function: firstFiveRndNumbers()
//Parameters: the_Code - a pointer to the keyword entered by the
//client
//Return Type: None
//Purpose: Generates the first five random numbers in the sequence.
//Random number generation is done based on a six-character code,
//entered by the client.
//----------------------------------------------------------------
void firstFiveRndNumbers(char *the_Code){

 int seed = (int)*the_Code * (int)(*(the_Code+1));
 int max = (int)(*(the_Code+2)) * (int)(*(the_Code+3));
 float dummy;
 int increment;
 srand(seed);
 cout << "1st five random numbers: ";
 increment = (int) (*(the_Code + 4));

for (int ix=1; ix<max; ix+=increment)
   dummy = ((float) rand() )/RAND_MAX;
        max = (int)(*(the_Code + 4)) * (int) (*(the_Code+5));

   for (int jx=1; jx<max; jx+=increment)
      dummy = ((float) rand() )/RAND_MAX;

   for (int kx=1; kx<=5; kx++)
      cout << ((float) rand() )/RAND_MAX << " ";

   cout << endl;
   return;
}//end firstFiveRndNumbers()
//----------------------------------------------------------------
//Function: getCode()
//Parameters: the_Code - the code that will be entered by the client
//Return Type: None
//Purpose: Prompts the client for a keyword of at least six characters.
//Checks the length of the keyword entered to make sure it has the
//required number of characters.  If it doesn't, it prints an error
//message and prompts for a keyword again.
//----------------------------------------------------------------
void getCode(char *&the_Code){

 int length;
 bool go_on = true;
 char buffer[1000];
 while(go_on){
  cout << "Enter your secret code (6 characters minimum): ";
  cin >> buffer;
  length = strlen(buffer);
  if (length < 6) {
     cerr << "\aCode must be 6 characters or more!"
          << endl << endl;
 }
 else {
      the_Code = new char[length];
      strcpy(the_Code, buffer);
      go_on = false;
  }
 }
 return;
}//end getCode()
//----------------------------------------------------------------
//Function: numericValues()
//Parameters: the_Code - the code entered by the client
//Return Type: None
//Purpose: Displays the ASCII values of the characters in the
//keyword.
//----------------------------------------------------------------
void numericValues(char *the_Code){

  char *alias = the_Code;
  int count = 0;
  char *spacing = "                               ";
  cout <<  "Numeric value of code characters: ";
  while (*alias){
   if ((count !=0) && (!(count%10)))
      cout << endl << spacing;
      cout << setw(4) << (int) (*alias++);
        count++;
   }

  cout << endl;
  return;
}//end numericValues()
//end file position.cpp

A:\c>position
Hit CTRL-C to terminate this progam.

Enter your secret code (6 characters minimum): HELPME
Numeric value of code characters:   72  69  76  80  77  69
1st five random numbers: 0.856807 0.441908 0.621845 0.638844 0.32

Enter your secret code (6 characters minimum): 9NINES
Numeric value of code characters:   57  78  73  78  69  83
1st five random numbers: 0.516404 0.737236 0.561724 0.396344 0.16


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The RTEST.BAS Program

One of the concerns you may have at this point is the possibility that each random seed results in a sequence of random numbers that repeat. To determine if this is a potential problem, a program was developed to generate the first five random numbers in a seed as two-digit integers and cycle through the seed looking for a match.

Listing 6.15 gives the statements in this program that was labeled RTEST.BAS. Figure 6.4 illustrates the results of the execution of RTEST.BAS when 1 was entered as the seed number. After a period of waiting, I terminated the execution of the program by entering a CTRL-BREAK key sequence because more than 5.7 million numbers had been tested without encountering a duplicate sequence of five numbers. With over 64,000 possible seeds and over 5.7 million non-repeating random number sequences per seed, it is obvious that it will be very difficult to duplicate a position selected by a well-thought-out code word that is manipulated in a well-thought-out manner.

Figure 6.4 Sample execution of the RTEST.BAS program.

Listing 6.15 The RTEST.BAS program listing, which tests for a repeated sequence within a random number seed.

REM Random sequence test program RTEST.BAS
    CLS
    PRINT "TEST FOR 5 REPEATED DIGITS AT BEGINNING OF SEQUENCE"
    POSITION = 3
    INPUT "Enter random generator seed number "; x
    RANDOMIZE x
    FOR I = 1 TO 5  'get first five random numbers as 2-digit integers
    A(I) = INT(RND * 100)
    NEXT I
    k = 1
again: FOR I = 1 TO 5   'get next group of five random numbers as
                  2-digit integers
    b(I) = INT(RND * 100)
    NEXT I
    FOR I = 1 TO 5    'do they match?
    IF A(I) <> b(I) THEN GOTO skip
    NEXT I
    LOCATE POSITION, 1
    PRINT "match occured at number =", k
    POSITION = POSITION + 1
    GOTO again
skip: k = k + (6 - I)
     LOCATE 1, 60
     PRINT "search at #"; k
     GOTO again

Now that you have a subroutine that can be used for positioning with the BASIC and C++ random number generation routines, you can develop enciphering programs.

The RANDOM3.BAS Program

Listing 6.16 contains the statements in the main portion of the enciphering program stored in the file RANDOM3.BAS on the BASIC directory on the companion CD-ROM. This program was constructed to encipher messages using the built-in BASIC random number generator using the previously described subroutine SETUP to obtain a difficult-to-duplicate position within the random number generator.

Listing 6.16 Statements in the main portion of the RANDOM3.BAS program.

REM Program RANDOM3.BAS
    CLS
    DIM PLAINTEXT$(25)
    PRINT "RANDOM3.BAS - A program which enciphers messages"
    PRINT "using the built-in BASIC random number generator"
    PRINT
RTN:
    INPUT "Enter your secret code (6 characters maximum): "; CODE$
    IF LEN(CODE$) > 6 THEN GOTO RTN
GOSUB INITIALIZE          'initialize plaintext values
GOSUB SETUP              'obtain random seed and position in seed
GOSUB MSGFILE          'assign I/O files, place message on a file
GOSUB RCONVERTSTORE         'convert and store ciphertext on a file
GOSUB PRTOUT           'print results
STOP

When you examine the statements contained in Listing 6.16, note that the only new subroutine is RCONVERTSTORE, with the prefix R in the name of the subroutine used to identify its modification for use in our random program operation. Thus, in discussing the operation of this program, I will focus attention upon the new subroutine developed for use in the program.

The RCONVERTSTORE Subroutine

Listing 6.17 lists the statements in the subroutine RCONVERTSTORE. The difference between this subroutine and the previously described CONVERTSTORE subroutine are the statements bounded by the FOR-NEXT loop.

Listing 6.17 Statements in the RCONVERTSTORE subroutine.

RCONVERTSTORE:
   REM Routine to convert and store ciphertext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM Convert plaintext to ciphertext
           FOR I = 1 TO MSGLEN
           X = INT(RND * 100)       'get 2-digit integer
           X = INT(X / 3.85)        'smooth to 0 to 25
           FOR J = 0 TO 25
           IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO
                                                   GOTIT
           NEXT J
GOTIT:       MID$(TEXT$, I, 1) = PLAINTEXT$((X + J) MOD 26)
           NEXT I
CLEARTXT:      WRITE #2, TEXT$
   LOOP
DONE1:     CLOSE #2
RETURN

When converting the subroutine for operation with random numbers, the statement X=INT(RND*100) obtains a one- or two-digit number between 0 and 99. The next statement lowers the maximum resulting value of the modified random number by dividing the number by 3.85 and taking the integer of the result. This smooths the random number to a value between 0 and 25, which is equivalent to the index values in the array PLAINTEXT in which the plaintext alphabet is stored.

The inner FOR-NEXT loop which increments J from 0 to 25 compares each character from a line of input from file #1 to a letter in the string array PLAINTEXT$. When a match occurs, J represents the position in the array PLAINTEXT$ where the match occurred. The branch to the label GOTIT results in the mod 26 addition of the value of J to the smoothed value of the extracted random number. The mod 26 addition of X and J is then used as an index to extract a new character from the string array PLAINTEXT$, which is used to replace the plaintext character in TEXT$. The subroutine RCONVERTSTORE uses the plaintext character set as a mechanism to generate ciphertext based upon the previously described algorithm.

Figure 6.5 illustrates the execution of the program RANDOM3.BAS. In this example, the code 9LIVES was used to obtain a position in the BASIC random number generator. Because the program uses the previously developed PRTOUT subroutine, the resulting enciphered message is printed in groups of five characters as illustrated at the bottom of Figure 6.5.

Figure 6.5 Execution of RANDOM3.BAS.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The RANDOM3.CPP Program

Continuing a presentation of C++ coding examples, Listing 6.18 lists the statements in the file RANDOM3.CPP. This file, along with its executable version named RANDOM3.EXE, is stored in the C directory on the companion CD-ROM.

In examining the program listing contained in Listing 6.18, note the functions encryptKeyboardText and encryptFile. The first function writes text entered via the keyboard to an input file and encrypts it to an output file. The second function, encryptFile, encrypts the contents of an input file. Two additional functions that warrant your attention include getCode and setupSeed as a pointer into the C++ random number generator. Since the executable version of RANDOM3.CPP produces similar results to the execution of RANDOM3.BAS, I will leave it to you to use the program.

Listing 6.18 The RANDOM3.CPP program listing.

/*
random3.cpp
C++ code written by Jonathan Held, May 14, 1998, using Microsoft's
Visual C++ version 5.0.
*/

//standard includes
#include <iostream.h>
#include <string.h>
#include <stdlib.h>
#include <iomanip.h>
#include <fstream.h>

//function prototypes
void setupSeed(char *);
void display(char *);
void encrypt(ofstream, char []);
void encryptKeyboardText(char *, char *);
bool encryptFile(char *, char *);
void formatData(char []);
void getCode(char *&);
void getFileNames(char * &, char * &);
int getInputType(void);
void groupBUFFER(ofstream, char [], int);

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally, any other
//value means abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main(){

  char * the_Code, *infile, *outfile;
  int input_type;

  getCode(the_Code);
  setupSeed(the_Code);
  getFileNames(infile, outfile);
  input_type = getInputType();

  //do something with file input first
  if (input_type){
     encryptFile(infile, outfile);
     cin.get();
     display(outfile);
  }
  else {
    encryptKeyboardText(infile, outfile);
    display(outfile);
  }

  return 0;
}//end main()

//----------------------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echoes the resulting output file to the screen.
//----------------------------------------------------------------
void display(char *name)
{
  const int SIZE = 256;
  ifstream infile(name, ios::in);
  char input[SIZE];

  cout << "Press return to display the encrypted file.";
  cin.getline(input, SIZE, '\n');
  strcpy(input, input+strlen(input));

  if (!(infile)){
    cerr << "Unable to open input file for display." << endl;
  }
  else {
    while (infile.getline(input, SIZE, '\n')){
     cout << input << endl;
    }
  }

  cout << endl;

  infile.close();

  return;
}//end display()

//----------------------------------------------------------------
//Function: encrypt()
//Parameters: wt - the output stream
//      text - the text that needs to be encrypted
//Return Type: None
//Purpose: Performs the encryption of the client's text based on
//random numbers.
//----------------------------------------------------------------
void encrypt(ofstream wt, char text[]){

  const int SIZE = 256;

  char plaintext[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
  int x, ix, jx, location;

  for (ix=0; ix < strlen(text); ix++){
    x = (rand() % 26);

  for (jx=0; jx < strlen(plaintext); jx++){
    if (text[ix] == plaintext[jx])
        location = jx;
  }

  text[ix] = plaintext[(x + location)%26];
 }

 groupBUFFER(wt, text, strlen(text));
 return;
}//end encrypt()

//----------------------------------------------------------------
//Function: encryptKeyboardText
//Parameters: in - the input file name
//      op  - the output file name
//Return Type: None
//Purpose: Writes the client's text to an input file and encrypts
//it to an output file.
//----------------------------------------------------------------
void encryptKeyboardText(char *in, char *op){

  const int SIZE = 256;
  const int NEW_LINE = 92;
  char buffer[SIZE];
  bool go_on = true;

  ofstream input(in, ios:: out);
  ofstream output(op, ios::out);

  if ((!input) || (!output)){
     cerr << "Unable to open file " << in << " or " << op
              << "." << endl;
  }
  else {
    cout << "Enter the message in UPPERCASE or lowercase characters. "
                     << endl
          << "Non-alphabetic characters may be entered but are
                     ignored." << endl
          << "Use a / at the beginning of each line that should
                     remain" << endl
          << "in plaintext and a \\ on a separate line to indicate
                    the" << endl
          << "end of an enciphered message." << endl << endl;

   while (go_on){

    cin.getline(buffer, SIZE, '\n');

   input << buffer << endl;

   if (buffer[0] == '/'){
   output << buffer << endl;
 }
 else if (((int) buffer[0]) == NEW_LINE){
  go_on = false;
 }
 else {
     formatData(buffer);
      encrypt(output, buffer);
     }
  }
 }

 input.close();
 output.close();

 cout << "Plaintext saved in " << in << endl
   << "Encrypted text in  " << op << endl << endl;

 return;
}//end encryptKeyboardText()

//----------------------------------------------------------------
//Function: encryptFile()
//Parameters: inp - the input file name
//      op  - the output file name
//Return Type: bool indicating success of the operation
//Purpose: Encrypts the contents of a file.
//----------------------------------------------------------------
bool encryptFile(char *inp, char *op){

 const int SIZE = 256;
 char buffer[256];
 bool success = false;

 //declare file stream objects
 ifstream input(inp, ios::in);
 ofstream output(op, ios::out);

 if ((!input) || (!output)){
         cerr << "Unable to open/write to files " << inp << " or "
              << op << "." << endl;
 }
 else {
   success = true;
   while (input.getline(buffer, SIZE, '\n')){

    if (buffer[0] == '/'){
        output << buffer << endl;
        strcpy(buffer, buffer+strlen(buffer));
     }
      else {
       formatData(buffer);
       encrypt(output, buffer);
     }
    }
  }

  input.close();
  output.close();

  return success;
}//end encryptFile()

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[])
{
 for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
  if (*(data+nx) == ' '){
     //do nothing - skip over the space in the data
  }
  else {
    *(data+mx++) = *(data+nx);
  }
 }

 //don't forget to add the null terminator
 *(data+mx) = '\0';

 return;
}//end formatData()

//----------------------------------------------------------------
//Function: getCode()
//Parameters: the_Code - the code that will be entered by the client
//Return Type: None
//Purpose: Prompts the client for a keyword of at least six characters.
//Checks the length of the keyword entered to make sure it has the
//required number of characters.  If it doesn't, it prints an error
//message and prompts for a keyword again.
//----------------------------------------------------------------
void getCode(char *&the_Code){

 int length;
 bool go_on = true;
 char buffer[1000];

 while(go_on){

  cout << "Enter your secret code (6 characters minimum): ";
  cin >> buffer;
  length = strlen(buffer);

  if (length < 6) {
  cerr << "\aCode must be 6 characters or more!"
      << endl << endl;
  }
  else {
   the_Code = new char[length];
   strcpy(the_Code, buffer);
   go_on = false;
  }
 }

 return;
}//end getCode()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
//      outfile_name - the output file we will write the
//      enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//----------------------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
  const int SIZE = 256;
  char data[SIZE];

  cout << "Enter filename to store/retrieve plaintext message: ";

  cin >> data;

  infile_name = new char[strlen(data) + 1];
  strcpy(infile_name, data);

  cout << "Enter filename to store enciphered message: ";

  cin >> data;

  outfile_name = new char[strlen(data) + 1];
  strcpy(outfile_name, data);

  return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//----------------------------------------------------------------
int getInputType(void)
{
  char type;
  bool error = false;
  int value;

  do {
    //prompt user for input from file or keyboard
    cout << "Is file input from keyboard (K, k) or file (F, f): ";
    cin >> type;

    //make type an uppercase letter
    type =static_cast<char>(toupper(static_cast<int>(type)));

     //check for an invalid character
     if ((type != 'K') && (type != 'F')){
       cerr << "You have entered an invalid character!" << endl << endl;
       error = true;
     }
     else {
       if (type == 'K')
          value = 0;        //value of 0 represents keyboard input
          else value = 1;   //value of 1 represents file input
       error = false;
    }

  } while (error);

  cout << endl;

  return value;
}//end getInputType()

//----------------------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
//      num - the number of characters we want to output
//      buffer - the buffer to be written to the stream
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//----------------------------------------------------------------
void groupBUFFER(ofstream out, char buffer[], int num)
{
  for (int kx=0;kx<num;kx++){

   if ((kx!=0) && (kx%25==0)){
      out << endl;
   }

   if ((kx!=0) && (kx%5 == 0) && (kx%25!=0)){
      out << " " << *(buffer+kx);
   }
   else {
      out << *(buffer+kx);
  }
}
 out << endl;

 return;
}//end groupBUFFER()

//----------------------------------------------------------------
//Function: setupSeed()
//Parameters: the_Code - a pointer to the keyword entered by the
//client
//Return Type: None
//Purpose: Sets up the random number seed based on the keyword
//code entered by the client.
//----------------------------------------------------------------
void setupSeed(char *the_Code){

 int seed = (int)*the_Code * (int)(*(the_Code+1));
 int max = (int)(*(the_Code+2)) * (int)(*(the_Code+3));
 float dummy;
 int increment;

 srand(seed);

 increment = (int) (*(the_Code + 4));

 for (int ix=1; ix<max; ix+=increment)
   dummy = ((float) rand() )/RAND_MAX;

 max = (int)(*(the_Code + 4)) * (int) (*(the_Code+5));

 for (int jx=1; jx<max; jx+=increment)
  dummy = ((float) rand() )/RAND_MAX;

 return;
}//end setupSeed()
//end file random3.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DRANDOM3.BAS Program

To use consistent naming conventions, I labeled the programs that decipher messages enciphered using RANDOM3.BAS and RANDOM3.CPP as DRANDOM3.BAS and DRANDOM3.CPP. Listing 6.19 contains the statements in the main part of that program that are very similar to RANDOM3.BAS. In fact, the only change in addition to changes in the words in strings located in PRINT statements is the substitution of the subroutines DMSGFILE for MSGFILE and RDCONVERTSTORE for RCONVERTSTORE.

Listing 6.19 Statements in the main portion of the DRANDOM3.BAS program.

REM Program DRANDOM3.BAS
   CLS
   DIM PLAINTEXT$(25)
   PRINT "DRANDOM3.BAS - A program which deciphers messages"
   PRINT "using the built-in BASIC random number generator"
   PRINT "that were enciphered using the RANDOM3.BAS program"
   PRINT
RTN:
   INPUT "Enter your secret code (6 characters maximum): "; CODE$
   IF LEN(CODE$) > 6 THEN GOTO RTN
GOSUB INITIALIZE         'initialize plaintext values
GOSUB SETUP          'obtain random seed and position in seed
GOSUB DMSGFILE         'assign I/O files, place message on a file
GOSUB RDCONVERTSTORE         'convert and store plaintext on a file
GOSUB PRTOUT         'print results
STOP

The subroutine DMSGFILE essentially adjusts I/O file references to the opposite of the subroutine MSGFILE. The subroutine RDCONVERTSTORE is very similar to the subroutine RCONVERTSTORE, but converts ciphertext to plaintext instead of plaintext to ciphertext.

The RDCONVERTSTORE Subroutine

Listing 6.20 lists the statements in the subroutine RDCONVERTSTORE. In this subroutine a random number is extracted and smoothed to a value between 0 and 25 in the same manner as performed by the subroutine RCONVERTSTORE. Next, each character in the string variable TEXT$ is matched to a character in the string array PLAINTEXT$, with the position of the match assigned to the variable J.

Listing 6.20 Statements in the RDCONVERTSTORE subroutine.

RDCONVERTSTORE:
   REM Routine to convert and store plaintext on a file
   OPEN INFILE$ FOR INPUT AS #1
   OPEN OUTFILE$ FOR OUTPUT AS #2
   DO UNTIL EOF(1)
       INPUT #1, TEXT$
       MSGLEN = LEN(TEXT$)
       IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT
       IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE1
       REM Convert ciphertext to plaintext
           FOR I = 1 TO MSGLEN
           X = INT(RND * 100)       'get 2 digit integer
           X = INT(X / 3.85)        'smooth to 0 to 25
           FOR J = 0 TO 25
           IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO
                                                    GOTIT
           NEXT J
GOTIT:       IF J < X THEN J = J + 26
           MID$(TEXT$, I, 1) = PLAINTEXT$((J - X) MOD 26)
           NEXT I
CLEARTXT:      WRITE #2, TEXT$
   LOOP
DONE1:     CLOSE #2
RETURN

To perform modulo 26 subtraction, the value of J is compared to the value of X. If J is less than X, J is incremented by 26. The value of X is then subtracted from J via modulo 26 arithmetic and the resulting value is used as an index into the string array PLAINTEXT$. The character at the index position represents the plaintext character and it is used to replace the ciphertext character in the variable TEXT$.

Figure 6.6 illustrates the operation of the program DRANDOM3.BAS. In this example the user simply enters a "secret" code and selects the default filenames and file input as the mechanism for reading the previously created enciphered message. The resulting deciphered message in the lower portion of Figure 6.6 is displayed in groups of five characters since the subroutine PRTOUT was used in the program.

Figure 6.6 Execution of the DRANDOM3.BAS program.

Readers may be curious as to the effect of entering a different "secret" code during the execution of the program DRANDOM3.BAS. To illustrate the effect, I used the code 9lives instead of 9LIVES. What may appear to be a small code change is a significant change to the mechanism developed for positioning to a place into the BASIC random number generator.

Figure 6.7 illustrates the effect of using the program DRANDOM3.BAS with an invalid secret code. If you compare the decipherment illustrated in Figure 6.6 to the decipherment illustrated in Figure 6.7, you will note that the use of an incorrect code results in a meaningless deciphered message. Thus, it is as important to keep your secret code secure as it is not to lose your code.

Figure 6.7 Using DRANDOM3.BAS with an incorrect code.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DRANDOM3.CPP Program

Listing 6.22 lists the statements in the program DRANDOM3.CPP, which represents the C++ version of DRANDOM3.BAS. Similar to previously described programs, both the source code and executable version of the program are stored in the C directory on the companion CD-ROM. The source file is named DRANDOM3.CPP and the executable file is named DRANDOM3.EXE.

In examining the program listing, note that although it is very similar to RANDOM3.CPP, it contains three new functions used to perform decryption operations. Those functions are decrypt, decryptKeyboardText, and decryptFile. The decrypt function performs the decryption of text based on the code used to generate a random number sequence. The decryptKeyboardText function writes text entered from the keyboard to an input file and decrypts the text to an output file. The third new function, decryptFile, performs the actual decryption of the contents of a file. The use of the executable version of DRANDOM.CPP should only be used with encrypted data generated by the executable version of RANDOM.CPP. This is because the BASIC language versions of those programs use a different random number generator, in effect precluding interoperability between the C++ and BASIC language versions of those programs.

Listing 6.22 The DRANDOM3.CPP program listing.

/*
drandom3.cpp
C++ code written by Jonathan Held, May 15, 1998, using Microsoft's
Visual C++ version 5.0.
*/

//standard includes
#include <iostream.h>
#include <string.h>
#include <stdlib.h>
#include <iomanip.h>
#include <fstream.h>

//function prototypes
void setupSeed(char *);
void display(char *);
void decrypt(ofstream, char []);
void decryptKeyboardText(char *, char *);
bool decryptFile(char *, char *);
void formatData(char []);
void getCode(char *&);
void getFileNames(char * &, char * &);
int getInputType(void);
void groupBUFFER(ofstream, char [], int);

//----------------------------------------------------------------
//Function: main()
//Parameters: None
//Return Type: int - 0 means program terminated normally, any other
//value means abnormal termination
//Purpose: Runs the main part of the program.
//----------------------------------------------------------------
int main(){

  char * the_Code, *infile, *outfile;
  int input_type;

  getCode(the_Code);
  setupSeed(the_Code);
  getFileNames(infile, outfile);
  input_type = getInputType();

  //do something with file input first
  if (input_type){
     decryptFile(infile, outfile);
     cin.get();
     display(outfile);
  }
  else {
    decryptKeyboardText(infile, outfile);
    display(outfile);
  }

  return 0;
}//end main()

//----------------------------------------------------------------
//Function: display()
//Parameters: name - the name of the file the user wants displayed
//Return Type: None
//Purpose: Echoes the resulting output file to the screen.
//----------------------------------------------------------------
void display(char *name)
{
  const int SIZE = 256;
  ifstream infile(name, ios::in);
  char input[SIZE];

  cout << "Press return to display the encrypted file.";
  cin.getline(input, SIZE, '\n');
  strcpy(input, input+strlen(input));

  if (!(infile)){
     cerr << "Unable to open input file for display." << endl;
  }
  else {
    while (infile.getline(input, SIZE, '\n')){
     cout << input << endl;
    }
  }
  cout << endl;

  infile.close();

  return;
}//end display()

//----------------------------------------------------------------
//Function: decrypt()
//Parameters: wt - the output stream
//     text - the text that needs to be decrypted
//Return Type: None
//Purpose: Performs the decryption of the client's text based on
//random numbers.
//----------------------------------------------------------------
void decrypt(ofstream wt, char text[]){

  const int SIZE = 256;

  char plaintext[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
  int x, location, ix, jx;

  for (ix=0; ix < strlen(text); ix++){
   x = (rand() % 26);

   for (jx=0; jx < strlen(plaintext); jx++){
    if (text[ix] == plaintext[jx])
          location = jx;
   }

   if (location < x)
     location += 26;

    text[ix] = plaintext[(location-x)%26];

  }

  groupBUFFER(wt, text, strlen(text));
  return;
}//end encrypt()

//----------------------------------------------------------------
//Function: decryptKeyboardText
//Parameters: in - the input file name
//     op  - the output file name
//Return Type: None
//Purpose: Writes the client's text to an input file and decrypts
//it to an output file.
//----------------------------------------------------------------
void decryptKeyboardText(char *in, char *op){

  const int SIZE = 256;
  const int NEW_LINE = 92;
  char buffer[SIZE];
  bool go_on = true;

  ofstream input(in, ios:: out);
  ofstream output(op, ios::out);

  if ((!input) || (!output)){
     cerr << "Unable to open file " << in << " or " << op
               << "." << endl;
  }
  else {
    cout << "Enter the message in UPPERCASE or lowercase characters.
                     " << endl
               << "Non-alphabetic characters may be entered but are
                      ignored." << endl
               << "Use a / at the beginning of each line that should
                       remain" << endl
               << "in plaintext and a \\ on a separate line to indicate
                       the" << endl
               << "end of an enciphered message." << endl << endl;

   while (go_on){

    cin.getline(buffer, SIZE, '\n');

   input << buffer << endl;

   if (buffer[0] == '/'){
        output << buffer << endl;
      }
      else if (((int) buffer[0]) == NEW_LINE){
        go_on = false;
      }
             else {
            formatData(buffer);
               decrypt(output, buffer);
            }
      }
   }
   input.close();
   output.close();

   cout << "Plaintext saved in " << in << endl
     << "Encrypted text in  " << op << endl << endl;

 return;
}//end decryptKeyboardText()

//----------------------------------------------------------------
//Function: decryptFile()
//Parameters: inp - the input file name
//      op  - the output file name
//Return Type: bool indicating success of the operation
//Purpose: decrypts the contents of a file.
//----------------------------------------------------------------
bool decryptFile(char *inp, char *op){

 const int SIZE = 256;
 char buffer[256];
 bool success = false;

 //declare file stream objects
 ifstream input(inp, ios::in);
 ofstream output(op, ios::out);

 if ((!input) || (!output)){
          cerr << "Unable to open/write to files " << inp << " or "
               << op << "." << endl;
 }
 else {
   success = true;
   while (input.getline(buffer, SIZE, '\n')){

    if (buffer[0] == '/'){
       output << buffer << endl;
              strcpy(buffer, buffer+strlen(buffer));
         }
            else {
              formatData(buffer);
              decrypt(output, buffer);
            }
      }
   }

   input.close();
   output.close();

   return success;
}//end decryptFile()

//----------------------------------------------------------------
//Function: formatData()
//Parameters: data - the array we want to format
//Return Type: None
//Purpose: Get rid of all spaces in the array.
//----------------------------------------------------------------
void formatData(char data[])
{
  for (int mx=0, nx=0; (*(data+nx) != '\0'); nx++){
    if (*(data+nx) == ' '){
     //do nothing - skip over the space in the data
    }
    else {
      *(data+mx++) = *(data+nx);
      }
   }

  //don't forget to add the null terminator
  *(data+mx) = '\0';

  return;
}//end formatData()

//----------------------------------------------------------------
//Function: getCode()
//Parameters: the_Code - the code that will be entered by the client
//Return Type: None
//Purpose: Prompts the client for a keyword of at least six characters.
//Checks the length of the keyword entered to make sure it has the
//required number of characters.  If it doesn't, it prints an error
//message and prompts for a keyword again.
//----------------------------------------------------------------
void getCode(char *&the_Code){

  int length;
  bool go_on = true;
  char buffer[1000];

  while(go_on){

   cout << "Enter your secret code (6 characters minimum): ";
   cin >> buffer;
   length = strlen(buffer);

   if (length < 6) {
   cerr << "\aCode must be 6 characters or more!"
        << endl << endl;
   }
   else {
     the_Code = new char[length];
     strcpy(the_Code, buffer);
     go_on = false;
    }
  }

 return;
}//end getCode()

//----------------------------------------------------------------
//Function: getFileNames()
//Parameters:  infile_name - the input file
//       outfile_name - the output file we will write the
//       enciphered text to
//Return Type: None
//Purpose: Get file information from the user.
//----------------------------------------------------------------
void getFileNames(char * &infile_name, char * &outfile_name)
{
  const int SIZE = 256;
  char data[SIZE];

  cout << "Enter filename to store/retrieve plaintext message: ";

  cin >> data;

  infile_name = new char[strlen(data) + 1];
  strcpy(infile_name, data);

  cout << "Enter filename to store enciphered message: ";

  cin >> data;

  outfile_name = new char[strlen(data) + 1];
  strcpy(outfile_name, data);
  return;
}//end getFileNames()

//----------------------------------------------------------------
//Function: getInputType()
//Parameters: None
//Return Type: int - 0 indicates keyboard input, 1 indicates file
//       input
//Purpose: Determines if the user will be manually entering text to
//be enciphered or if the user wants a file to be enciphered.
//----------------------------------------------------------------
int getInputType(void)
{
  char type;
  bool error = false;
  int value;

  do {
    //prompt user for input from file or keyboard
    cout << "Is file input from keyboard (K, k) or file (F, f): ";
    cin >> type;

    //make type an uppercase letter
    type =static_cast<char>(toupper(static_cast<int>(type)));
      //check for an invalid character
      if ((type != 'K') && (type != 'F')){
         cerr << "You have entered an invalid character!"
         << endl << endl;
         error = true;
      }
      else {
        if (type == 'K')
          value = 0;     //value of 0 represents keyboard input
          else value = 1; //value of 1 represents file input
         error = false;
      }

   } while (error);

   cout << endl;

   return value;
}//end getInputType()

//----------------------------------------------------------------
//Function: groupBUFFER()
//Parameters: out - the output stream we are writing to
//      num - the number of characters we want to output
//      buffer - the buffer to be written to the stream
//Return Type: None
//Purpose: Output the buffer in groups of five characters at a
//time.
//----------------------------------------------------------------
void groupBUFFER(ofstream out, char buffer[], int num)
{
  for (int kx=0;kx<num;kx++){

   if ((kx!=0) && (kx%25==0)){
      out << endl;
   }

   if ((kx!=0) && (kx%5 == 0) && (kx%25!=0)){
      out << " " << *(buffer+kx);
   }
   else {
      out << *(buffer+kx);
   }
 }
   out << endl;

   return;
}//end groupBUFFER()

//----------------------------------------------------------------
//Function: setupSeed()
//Parameters: the_Code - a pointer to the keyword entered by the
//client
//Return Type: None
//Purpose: Sets up the random number seed based on the keyword
//code entered by the client.
//----------------------------------------------------------------
void setupSeed(char *the_Code){

  int seed = (int)*the_Code * (int)(*(the_Code+1));
  int max = (int)(*(the_Code+2)) * (int)(*(the_Code+3));
  float dummy;
  int increment;
  srand(seed);

  increment = (int) (*(the_Code + 4));

  for (int ix=1; ix<max; ix+=increment)
    dummy = ((float) rand() )/RAND_MAX;

  max = (int)(*(the_Code + 4)) * (int) (*(the_Code+5));

  for (int jx=1; jx<max; jx+=increment)
    dummy = ((float) rand() )/RAND_MAX;

  return;
}//end setupSeed()
//end file drandom3.cpp


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

An Alternative Random Process

A second process readers can consider for generating random numbers for use in enciphering operations is based upon the work of Gilbert Vernam (see the shaded box near the beginning of this chapter). Using Vernam's method, which he employed to develop a long random key from two relatively short random keys, as a foundation, you can perform a similar operation through the use of two random seed numbers.

To develop this alternative process, assume that you retain the use of a six-character "secret" code word. You can use the ASCII values of the code word as a mechanism to select two random number seeds. For each seed selected, you can position yourselves into the random number sequence and extract 1,000 random numbers from the first sequence and 999 from the second. You can then develop a routine I will call ROTOR that operates upon the two random number sequences similar to Mr. Vernam's dual paper tape reader.

When Gilbert Vernam developed the first automatic enciphering system, random characters were literally pulled from a hat. This system was time-consuming, and it limited the number of random characters on the tape to 1,000. Vernam realized that looping the tape repeated the random character sequence used for encipherment operations every 1,000 characters. To overcome this weakness, Vernam designed a special tape reader that read two tapes consisting of randomly selected characters. The contents of each tape were added to one another using modulo addition. When a cycle of the shorter tape was completed, the longer tape was repositioned by one character position and the sequence of modulo addition by character position was repeated. The longer tape contained 1,000 randomly selected characters, and the shorter tape contained 999 randomly selected characters, which resulted in the use of two tapes extending the random character key sequence to 999,000 before the sequence repeated.

By extending the random cipher key from 1,000 to 999,000 characters, the security of Vernam's automated enciphering system was significantly enhanced. This extension permitted hundreds to thousands of messages to be enciphered before a new pair of tapes was necessary to prevent the key from repeating.

The ROTOR.BAS Program

Listing 6.21 contains the statements in the program ROTOR.BAS which is used to illustrate how you can create your own random number sequence generating routing that can be considerably different from that contained in BASIC but is based upon the use of the BASIC random number generator.

Listing 6.21 The ROTOR.BAS program listing.

REM PROGRAM ROTOR.BAS
REM This program constructs a new random number sequence of numbers that
REM range in value from 0 to 99 based upon the extraction of random numbers
REM from two seeds whose location are based on the composition of a
REM six-position code word. This sequence repeats after 999,000 characters
REM with the same secret code.
CLS
DIM R1(1000), R2(999)
RTN:
      INPUT "Enter your secret code (6 characters maximum): "; CODE$
      IF LEN(CODE$) > 6 THEN GOTO RTN
GOSUB SETUP
      P1 = 0
      P2 = 1
      FOR K=1 TO 500
GOSUB ROTOR
      NEXT K
STOP
SETUP:
      REM Routine to extract two random numbers sequences based on
      REM The composition of the secret code
      FOR I = 1 TO LEN(CODE$)
      CODE(I) = ASC(MID$(CODE$, I, 1)
      NEXT I
      SEED1 = CODE(1)*CODE(3)
      X1 = CODE(5)
      DO UNTIL X1<= 4
                   X1 = X1 / 2
      LOOP
      SEED1 = SEED * X1
      SEED2 = CODE(2) * CODE(4)
      X2 = CODE(6)
      DO UNTIL X2 <= 4
                    X2 = X2 / 2
      LOOP
      SEED2 = SEED2 * X2
      RANDOMIZE SEED1
      MAX = CODE(1) * CODE(3) * CODE(5)
      FOR I = 1 TO MAX STEP CODE(2)       'position into sequence
            DUMMY = RND
      NEXT I
      FOR I = 1 TO CODE(1) * 500 STEP CODE(3) 'go further into sequence
            DUMMY = RND
      NEXT I
      FOR I = 1 TO 1000                        'obtain first sequence
            R1(i) = RND         'of random numbers
       NEXT I
       RANDOMIZE SEED2
       MAX = CODE(2) * CODE(4) * CODE(6)
       FOR I = 1 TO MAX STEP CODE(4)    'position into next sequence
             DUMMY = RND
       NEXT I
       FOR I = 1 TO CODE(4) * 8307 STEP CODE(2) 'go further into
                                                               sequence
             DUMMY = RND
        NEXT I
        FOR I = 1 TO 999                        'get second sequence
              R2(I) = RND
        NEXT I
RETURN
ROTOR:
         R = INT(((R1(P1) + R2(P2)) * 100) MOD 99)
         PRINT USING "## "; R;
         P1 = P1+ 1
         IF P1 = 1000 THEN P2 = P2 + 1
         IF P1 = 1000 THEN P1 = 0
RETURN

The two arrays in the program, R1 and R2, function similar to Mr. Vernam's two punched paper tapes. That is, they contain sequences of 1,000 and 999 random numbers, respectively. You can perform a modulo addition operation on the numbers in the two sequences as well as rotate the relationship between the numbers in each sequence to generate a new random number sequence.

After you enter your "secret" code, it is assigned to the string variable CODE$, similar to the previously developed RANDOM3.BAS program. The subroutine SETUP is then invoked. This subroutine obtains a position in a random number sequence similar to the previously developed routine of the same name. However, this routine uses the ASCII value of the "secret" code to obtain two different seed numbers as well as the location for the extraction of random numbers from each seed.

The first seed, specified by the variable SEED1, has its value determined by first multiplying the ASCII value of the first and third characters in the "secret" code. This produces a maximum value of 127*127, or 16,129. Because the number of seeds supported by BASIC is considerably higher, you then extract the ASCII value of the fifth character in the "secret" code and continuously divide it by 2 until it is less than or equal to 4. This results in the extension of the seed to a maximum value of 64,516. Next, the value of a second seed is computed. First, the ASCII values of the second and fourth characters in the "secret" code are multiplied by one another. Then divide the ASCII value of the sixth character in the code by 2 until it is less than or equal to 4, and use that value as a multiplier of the product of the ASCII value of the second and fourth characters. Once the two seeds are computed, you are ready to select each seed and position yourself into the random number sequence in each seed prior to extracting the random numbers and placing them into the R1 and R2 arrays.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

After the first seed is selected using the RANDOMIZE SEED1 statement, use a two-step approach to position yourself into the random number sequence within the selected seed. First multiply the ASCII values of the first, third, and fifth characters in the "secret" code to serve as the limit for the first FOR-NEXT loop, assigning the value of the three products to MAX. The ASCII value of the second character in the "secret" code is used as the stepping increment through the FOR-NEXT loop.

To go further into the random number sequence, we use a second FOR-NEXT loop in which the ASCII value of the first character multiplied by 500 serves as the limit of the loop, and the ASCII value of the third character in the "secret" code functions as the stepping mechanism through the loop. Once your positioning is completed, extract 1,000 random numbers which are placed into the array R1.

Using the value computed for the second seed, use the statement RANDOMIZE SEED2 to select that seed. Next, use the ASCII value of the characters in the "secret" code to position yourself into a random number sequence in that seed. Multiply the values of the second, fourth, and sixth characters in the "secret" code to serve as a limit for the FOR-NEXT loop and use the value of the fourth character to step through the loop. To go further into the sequence, another FOR-NEXT loop multiplies the ASCII value of the fourth character by 8,307 and uses the ASCII value of the second character as a mechanism to step through the loop. This is followed by extracting 999 random numbers and placing them into the array R2.

When you examine the contents of the routine SETUP, note that both the ASCII value of the characters in the "secret" code word and the numerics entered in the program to govern the extraction process were used. You can use this subroutine as a guide to develop a random number extraction process that differs from the techniques presented in this book if you wish to obtain an additional degree of message security.

Once the subroutine SETUP is invoked, set the "paper tape" rotation scheme into operation by assigning the value 0 to the variable P1 and 1 to the variable P2. Then, for illustrative purposes, execute the subroutine ROTOR 500 times. If you were to use this subroutine to encipher a message, each time you invoke the subroutine, up to 999,000 times, you would receive a newly created random number whose value would be between 0 and 99. You must consider transmitting messages with less than 999,000 characters as well as the effect of adding a number with that possible variance to the set of values of plaintext characters you wish to encipher. For example, if 32 of the possible resulting character values are between 0 and 31, those enciphered characters become control characters that cannot be transmitted over 7-bit electronic mail systems. Another concern you should think about is the necessity to print copies of enciphered messages, because you cannot print control characters. You may wish to modify the ROTOR subroutine to produce random numbers within a range of values which, when added to the ASCII value of the supported set of plaintext characters, permits transmission over 7-bit electronic mail systems and printing of enciphered messages.

The first statement in the subroutine ROTOR adds the random numbers in the arrays R1 (0) and R2 (1), multiplies the result by 100, performs a modulo 99 operation, and extracts the integer result. After the value of R is printed, the value of P1 which represents the position in the first random number sequence is incremented by 1. Only after all 1,000 random numbers in the array R1 have been added using the first random number in R2 do you re-sequence or rotor the "paper tape." Here, when P1 equals 1,000 we increment P2 by 1 and reset P1 to 0. Each time the subroutine ROTOR is invoked you would continue this process.

To illustrate the result of the previously described set of routines, Figure 6.8 contains the random numbers generated from the execution of ROTOR.BAS. In this example, the secret code ROUTE9 was used to generate the sequence of 500 random numbers generated by the program.

Figure 6.8 Execution of ROTOR.BAS using the secret code ROUTE9.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 7
Developing Practical Programs

This chapter will first discuss the utilization of previously developed subroutines and concepts into programs for enciphering and deciphering operations. Because you may prefer to use a ready-to-run set of enciphering and deciphering programs, this chapter also presents an overview of the use of a special pair of programs which are contained on the CD-ROM that accompanies this book.

Module Utilization

In previous chapters in this book I developed a number of subroutines that perform predefined operations. As I developed specific programs, those programs were constructed based upon the use of a number of subroutines that provided us with the ability to perform modular operations and link those operations together. You should consider using the previously developed series of subroutines, as well as coding some of the concepts discussed in this book to modify those subroutines, as a basis for developing programs to encipher and decipher messages to satisfy specific user requirements. For example, you may require the use of an enciphering program that restricts the permissible range of enciphered characters. This restriction may result from the ability of an electronic mail system to only accept a limited character set or another limitation that affects the range of resulting enciphered characters. By tailoring previously developed routines you can create enciphering and deciphering programs to satisfy those requirements. To assist you in reviewing the functions performed by the BASIC language subroutines developed and contained in this book, Table 7.1 summarizes their primary function.

Table 7.1 BASIC language subroutine functionality
Subroutine | Function Performed
INITIALIZE | This subroutine sets the elements in a string array to the uppercase letters of the alphabet.
FORMCIPHER | This subroutine forms a cipher alphabet by using an uppercase shift key character to rotate the plaintext alphabet until the shift key character is on the extreme right.
MSGENCIPHER | This subroutine converts plaintext to ciphertext using a simple monoalphabetic substitution.
GROUPBY5 | This subroutine prints a space after the display of each group of five enciphered characters.
MSGFILE | This subroutine assigns I/O files and accepts either keyboard or file input of messages.
CONVERTSTORE | This subroutine reads the contents of a plaintext message from a file, enciphers the message using a monoalphabetic substitution process, and outputs the resulting enciphered message to a file.
PRTOUT | This subroutine prints an enciphered message in groups of five characters, placing a space between each group and padding the last group with X's until it contains five characters.
KEYWORD | This subroutine creates a keyword-based mixed alphabet based upon a keyword or keyword phrase.
TRANSPORT | This subroutine creates a simple or numeric transposition mixed sequence alphabet.
INTERVAL | This subroutine creates an interval extracted mixed alphabet.
MSGDCIPHER | This subroutine uses a monoalphabetic substitution process to convert ciphertext to plaintext.
DMSGFILE | This subroutine is the reverse of the subroutine MSGFILE with respect to I/O file assignment names as the routine accepts files containing a ciphertext and assigns an output file to store plaintext.
DECIPHER | This routine can be considered as the reverse of the subroutine CONVERTSTORE as it converts ciphertext to plaintext.
DPRTOUT | This subroutine prints the resulting deciphered message on a line-by-line basis without character groupings.
DCONVERTSTORE | This subroutine can be considered as the reverse of the subroutine CONVERTSTORE as it converts ciphertext to plaintext.
RCONVERTSTORE | This is the modified version of the subroutine CONVERTSTORE used for random number-based encipherment. The subroutine smooths random numbers to between 0 and 25 for modulo 26 addition to alphabetic plaintext characters to produce enciphered text.
SETUP | This subroutine selects a random number seed and location in the seed based upon the ASCII value of the characters in a "secret" code.
RDCONVERTSTORE | This is the modified version of the subroutine CONVERTSTORE used for deciphering of messages enciphered through the use of the subroutine RCONVERTSTORE.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The ENCIPHER.EXE Program

Up to this point, the programs developed in this book had significant restrictions concerning the character set available for encipherment. For example, most programs restricted the composition of a message to the uppercase alphabet, forcing users to spell out punctuation characters, such as COMMA for a "," and PERIOD or STOP to signify the end of a sentence.

Because you may prefer to type a message following a minimum of constraints, a new set of programs was developed to perform message enciphering and deciphering operations. The enciphering program is contained in the file ENCIPHER.EXE on the CD-ROM in the form of a directly executable file. The source version of the program is stored on the file ENCIPHER.BAS, with both files located in the BASIC directory.

In a previous version of this book, only the executable version of the ENCIPHER program was provided. If you are curious as to why no source version of this program was provided, if you will let your curiosity wait a few minutes, the rationale for only providing executable versions of the pair of programs whose use is described in this chapter will become apparent.

The program ENCIPHER.EXE was created using seven previously developed subroutines as a foundation for enciphering. Listing 7.1 contains the main module of ENCIPHER.BAS, illustrating the sequence in which different subroutines are called. Our examination of the coding used in several subroutines will provide you with the ability to modify one or more subroutines or use different groups of subroutines to tailor an enciphering program to your specific requirements. This will ensure your program differs from this program and will make it much more difficult for someone to decipher a message they may gain access to. In fact, the author recommends that you develop your own program rather than rely upon any program available for use by the general public, since such a program is equivalent to one government providing a cipher machine to another government, especially if the latter is unfriendly. When this occurs, the unfriendly government will use the machine to attempt to decipher messages illicitly obtained by putting the machine through its paces, attempting one combination after another. The unfriendly government will more than likely also take the machine apart to determine what weaknesses exist and attempt to exploit those weaknesses. Due to the preceding, not all of the subroutines in the program ENCIPHER.BAS were discussed in the earlier version of this book.

Listing 7.1 The main portion of the ENCIPHER.BAS program.

REM PROGRAM ENCIPHER.BAS
DIM PLAINTEXT$(128), CIPHERTEXT$(36, 92), KEY$(128)
  CLS
  PRINT "ENCIPHER.EXE PROGRAM enciphers text based upon the use"
  PRINT "of enciphering techniques contained in the book TOP SECRET:"
  PRINT "TRANSMITTING ELECTRONIC MAIL USING PRACTICAL ENCIPHERING
TECHNIQUES"
  PRINT
  PRINT "This program supports the use of upper and lower case letters,"
  PRINT "digits, punctuation characters and other characters whose ASCII"
  PRINT "values range between 32 and 128, but EXCLUDES the use of the"
  PRINT "foward slash (/), backslash (\) and double quote characters."
  PRINT
  PRINT
AGN1:   INPUT "Enter your secret code (6 characters required): "; CODE$
  IF LEN(CODE$) = 6 THEN GOTO OK
  CLS
  PRINT "Your secret code must be 6 characters - please try again"
  PRINT
  GOTO AGN1
OK:  CLS
GOSUB INITIALIZE         'initialize plaintext values
GOSUB SETUP              'obtain random seed and position in seed
GOSUB MSGFILE            'assign I/O files, place message on a file
GOSUB KEYWORD            'form keyword based alphabet of 96 characters
GOSUB PFORMCIPHER        'create 36 cipher alphabets
GOSUB RCONVERTSTORE      'convert and store ciphertext on a file
GOSUB PRTOUT             'print results
STOP

The subroutine INITIALIZE was modified to expand the ability of ENCIPHER.BAS to accept a wider mixture of characters. The subroutine, whose coding is listed in Listing 7.2, was modified to initialize a string array labeled PLAINTEXT$ to all characters whose ASCII values are between 32 and 127, except those characters whose ASCII values equal 34, 47, and 92. The latter restriction eliminates the forward and backslash characters as well as the quote (") character from use in a message, except the first two characters can be used to denote a message header or the end of a message. In addition, those characters whose ASCII values equal or exceed 128 or are below 32 should not be used in your plaintext message. ASCII characters whose values are below 32 are control characters that are either non-printable or whose printing results in some strange effects, such as horizontal and vertical tabs, form feeds, and so on. This can significantly alter the display of an enciphered message, and the exclusion of those characters enables an enciphered message to more easily be input via the keyboard if this should become necessary. ASCII characters whose values exceed 127 are known as extended ASCII. Several popular electronic mail systems will not accept those characters, hence, I have excluded their use in both the plaintext message and the resulting ciphertext. The latter is accomplished by structuring the creation of a modulo addition process that results in ciphertext characters whose ASCII values are between 32 and 127. In spite of these restrictions, the program opens up the vast majority of your keyboard for use in creating messages. You can use all uppercase and lowercase alphabetic characters, the space character, all digits, and most punctuation characters to prepare your message.

Listing 7.2 The modified INITIALIZE subroutine.

INITIALIZE:
     RESTORE
     REM Initialize plaintext values
     I = 0
     K = 32
GOOD:   IF K = 47 OR K = 92 OR K = 34 THEN GOTO NOGD'skip \, /  and "
     PLAINTEXT$(I) = CHR$(K)
     I = I + 1
NOGD:    K = K + 1
     IF I < 128 GOTO GOOD
RETURN

When you examine the statements in the modified subroutine INITIALIZE, you may note that the variable I serves as the array index, while the variable K serves as the ASCII character value. Thus, when PLAINTEXT$(0)=CHR$(32), the 0 element of the string array PLAINTEXT$ is set to a space character. Thus, the coding in Listing 7.2 replaces the use of READ and DATA statements used in previous INITIALIZE subroutines.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Because the subroutine SETUP provides a mechanism to select a random seed and positions you into a seed, in an earlier version of this book I purposely skipped a discussion of the code used in that subroutine (discussing the mechanism used to obtain the seed and position within the seed is similar to giving a burglar the keys to the store). The earlier version offered a reward of $1,000 to the first person who could decode an encrypted file. Although the prize was won, it required approximately a year of effort on the part of many persons until a satisfactory solution was found. Because no prize is now offered, the entire source file is included on the CD-ROM to permit you to experiment in creating different length keys.

The code used in the subroutine KEYWORD is contained in Listing 7.3. This subroutine is most interesting as it uses the "secret" code to develop a 30-character keyword. Note that the expansion of the 6-character "secret" code to a 30-character keyword substitutes the string "9" for the characters "\", "/", and the double quote whose ASCII value is 34. To preclude any extended ASCII characters occurring in the keyword, any character in the expansion process that has an ASCII value above 127 is set to the string "Q." The remaining modules in the subroutine are similar to modules I developed in the original KEYWORD subroutine. The main difference between the remainder of the subroutines concerns the length of the array PLAINTEXT$, which now contains 93 characters and whose element value now ranges from 0 to 92.

Listing 7.3 The KEYWORD subroutine.

KEYWORD:
   REM Place entered keyword into KEY$ array 1 character per position
   REM but mix word thru expansion to 30 characters
     MSGLEN = 30
     J = 1
     FOR I = 1 TO 6
AGN2:  IF J >= 31 THEN GOTO NULLIT
     KEY$(J) = MID$(CODE$, I, 1)
     FOR J = J + 1 TO J + 4
     KEY$(J) = CHR$(ASC(MID$(CODE$, I, 1)) + (J - 1))
     IF KEY$(J) = "\" OR KEY$(J) = "/" OR KEY$(J) = CHR$(34) THEN
KEY$(J) = "9"
     IF ASC(KEY$(J)) > 127 THEN KEY$(J) = "Q"
     NEXT J
INCIT: NEXT I
   REM ELIMINATE DUPLICATE LETTERS, REPLACE WITH CARRIAGE RETURNS
NULLIT: K = 2
     FOR I = 1 TO MSGLEN
     FOR J = K TO MSGLEN
     IF KEY$(I) <> KEY$(J) GOTO NOTDUP
     KEY$(J) = CHR$(13)
NOTDUP:  NEXT J
     K = K + 1
     NEXT I
   REM REMOVE CARRIAGE RETURNS IN STRING
     X$ = ""
     FOR I = 1 TO MSGLEN
     IF KEY$(I) = CHR$(13) THEN GOTO ASKP
     X$ = X$ + KEY$(I)
ASKP:  NEXT I
   REM PLACE REVISED KEYWORD WITH NO DUPLICATE LETTERS BACK IN KEY$
     FOR I = 1 TO LEN(X$)
     KEY$(I) = MID$(X$, I, 1)
     NEXT I
   REM COMPARE KEY$ & PLAINTEXT$ ARRAYS, SET PLAINTEXT$ ELEMENT TO CR WHEN
MATCHED
     FOR J = 1 TO LEN(X$)
     FOR K = 0 TO 92
     IF KEY$(J) = PLAINTEXT$(K) THEN PLAINTEXT$(K) = CHR$(13)
     NEXT K
     NEXT J
   REM CREATE ONE STRING
     FOR I = 0 TO 92
     IF PLAINTEXT$(I) = CHR$(13) THEN GOTO SKIP
     X$ = X$ + PLAINTEXT$(I)
SKIP:  NEXT I
   REM PLACE SEQUENCE BACK INTO PLAINTEXT$ ARRAY
     FOR I = 0 TO 92
     PLAINTEXT$(I) = MID$(X$, I + 1, 1)
     NEXT I
RETURN

Listing 7.4 contains the code from the modified subroutine PFORMCIPHER. In examining this subroutine, note that it forms a two-dimensional ciphertext alphabet. The index JJ varies from 0 to 35 and forms 36 cipher alphabets that are based upon the keyword exploded from the "secret" code. By now you probably recognize that ENCIPHER.BAS uses a combination of random numbers and a rotating ciphertext alphabet to perform encipherment. Exactly how this occurs is illustrated by the code in the modified subroutine RCONVERTSTORE.

Listing 7.4 The modified PFORMCIPHER subroutine.

PFORMCIPHER:
    REM routine to form 35 CIPHERTEXT alphabets based upon the
                 characters in the code
    FOR JJ = 0 TO 35
    FOR KK = 0 TO 92
    CIPHERTEXT$(JJ, KK) = PLAINTEXT$((KK + JJ) MOD 92)
    NEXT KK
    NEXT JJ
RETURN

Listing 7.5 contains the code in the modified subroutine RCONVERTSTORE. In examining the code in the subroutine RCONVERTSTORE, note that after extracting a two-digit random number using X=INT(RND*100), I divide the result by 1.087 and take the integer portion of the result. This smooths the random number to a value between 0 and 92 which corresponds to the 93 elements in the array PLAINTEXT$. Then I compare each character in each line of the plaintext to a character in the array PLAINTEXT$. When a match occurs, I add the index value of the match position in PLAINTEXT$ to the smoothed value of X using mod 92 addition. The result, stored in the variable Z, is used to extract a character from one of the ciphertext alphabets contained in the string array CIPHERTEXT$. Thus, ENCIPHER.BAS uses random numbers to extract characters from a two-dimensional alphabet to obtain enciphered text. The routines used for enciphering illustrate how you can combine two or more techniques to develop your own enciphering program if you wish to do so.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Listing 7.5 The modified RCONVERTSTORE subroutine.

RCONVERTSTORE:
  REM Routine to convert and store ciphertext on a file
  OPEN INFILE$ FOR INPUT AS #1
  OPEN OUTFILE$ FOR OUTPUT AS #2
  ALPHA = 0     'first alphabetic pointer
  PRINT "Enciphering operation in progress ";
  DO UNTIL EOF(1)
     INPUT #1, TEXT$
     PRINT ".";
     MSGLEN = LEN(TEXT$)
     IF MID$(TEXT$, 1, 1) = "/" THEN GOTO CLEARTXT1
     IF MID$(TEXT$, 1, 1) = "\" THEN GOTO DONE2
     REM Convert plaintext to ciphertext
        FOR I = 1 TO MSGLEN
        X = INT(RND * 100)       'get 2-digit integer
        X = INT(X / 1.087)       'smooth to 0 to 92
        FOR J = 0 TO 92
        IF MID$(TEXT$, I, 1) = PLAINTEXT$(J) THEN GOTO MATCH
        NEXT J
MATCH:     Z = (J + X) MOD 92
        MID$(TEXT$, I, 1) = CIPHERTEXT$(ALPHA, Z)
        NEXT I
        ALPHA = ALPHA + 1
        IF ALPHA = 35 THEN ALPHA = 0'reset alphapet pointer
CLEARTXT1:    WRITE #2, TEXT$
  LOOP
DONE2:  CLOSE #2
     CLOSE #1
RETURN
PRTOUT: REM Subroutine to print results
  PRINT
  INPUT "Press Return key to display resulting enciphered message", p$
  CLS: TEMP$ = ""
  PRINT "Resulting enciphered message is:"
  OPEN OUTFILE$ FOR INPUT AS #2
  DO UNTIL EOF(2)
     INPUT #2, TEXT$
     IF MID$(TEXT$, 1, 1) = "/" THEN PRINT RIGHT$(TEXT$, LEN
(TEXT$) - 1)
     IF MID$(TEXT$, 1, 1) = "/" THEN GOTO NOGROUP
     PRINT TEXT$
NOGROUP: LOOP
  CLOSE #1: CLOSE #2
RETURN

To provide you with an indication of how to position the random number generator based upon the composition of a secret code, Listing 7.6 gives the contents of the subroutine SETUP. In examining the statements in the subroutine, note that the seed is set to the value of the product of the first and sixth characters in the secret code. Next, the integer value of the product of the ASCII values of the even characters in the six-position code is used as an initial offset, and the product of the ASCII values of the first and third characters in the code are used as a delimiter in a FOR-NEXT loop, with the value of the second character used to step through the loop.

Listing 7.6 The SETUP subroutine positions the random number generator based upon the composition of the secret code.

SETUP:
   REM routine to position the random number generator based upon
   REM the composition of the secret code
   FOR I = 1 TO LEN(CODE$)
   CODE(I) = ASC(MID$(CODE$, I, 1))
   NEXT I
   SEED = CODE(1) * CODE(6)
   MAX = INT(CODE(2) * CODE(4) * CODE(6))
   RANDOMIZE SEED
   FOR I = 1 TO MAX STEP (CODE(5) * CODE(1))   'position into sequence
   DUMMY = RND
   NEXT I
   FOR I = 1 TO CODE(1) * CODE(3) STEP CODE(2) 'further offset into
                                                               sequence
   DUMMY = RND
   NEXT I
RETURN

The execution of the program ENCIPHER.EXE is illustrated in Figures 7.1 through 7.3. Figure 7.1 illustrates the display of the first screen of the program which first displays information about the program and then prompts you to enter a six-digit "secret" code. Unlike the restrictions on the plaintext characters in a message, the code can consist of most characters in the ASCII character set; however, as discussed in this book, you should select a code that is not difficult to repeat. Thus, I recommend the use of printable characters.

Figure 7.1 The first screen display of the ENCHIPHER.EXE program.

Figure 7.2 illustrates the second screen displayed when the program ENCIPHER.EXE is executed. This screen first displays information concerning the filenames to be used to store the plaintext and ciphertext messages. Because the keyboard was selected for entering the plaintext message, the program then displays information concerning the use of the forward and backslash characters. As in our previously developed programs, the forward slash is used to prefix headers that should remain as plaintext, while the backslash is used to indicate the termination of the message.

Figure 7.2 The second screen displayed upon execution of ENCHIPHER.EXE.

After the previously discussed information is displayed, you can enter your message. After the backslash character is entered, the program displays the message "Enciphering operation in progress…" Because the program can require from a few seconds to half a minute or more depending upon the computer you use to encipher your message, dots are displayed while the program executes so you will know it is working. Once the encipherment operation is completed, the message "Press Return key to display resulting enciphered message" is displayed. Once you press the Return key, the resulting enciphered message is displayed as illustrated in Figure 7.3. In addition, the enciphered message is also written onto the file CIPHERTX.DAT, so you can transfer that file via a communications program if you so desire.

Figure 7.3 The enciphered message created ENCIPHER.BAS.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The DECIPHER.EXE Program

The program DECIPHER.BAS is provided in both source and executable versions on the CD-ROM accompanying this book. Similar to the ENCIPHER.EXE program, the program DECIPHER.EXE operates through the use of three screens. The execution of the program DECIPHER.EXE is illustrated in Figures 7.4 through 7.6.

To illustrate the use of the deciphering program I will use the program to operate upon the previously enciphered message contained on the file CIPHERTX.DAT. Figure 7.4 illustrates the entry of the same "secret" code that was used to encipher our previously created plaintext message.

Figure 7.4 The first screen displayed upon execution of DECIPHER.EXE.

Once your "secret" code is entered, the program displays a second screen which, similar to ENCIPHER.EXE, enables you to assign filenames for storing the plaintext and enciphered messages and select keyboard or file input (Figure 7.5). In this example, the default filenames were accepted and once the deciphering operation commenced, an appropriate message was displayed. After the deciphering operation is completed and you press the Return key, the resulting deciphered message is displayed. This is illustrated in Figure 7.6. In comparing the plaintext message in Figure 7.2 to the deciphered message contained in Figure 7.6, you will note they are exactly the same.

Figure 7.5 The second screen displayed upon execution of DECIPHER.EXE.

Figure 7.6 The resulting deciphered message.

The Challenge

In concluding this chapter, an enciphered message is presented in Figure 7.7. Unlike the task faced by most cryptanalysts who do not have access to an enciphering system used to create a message, I have considerably simplified the task of those who may wish to decipher this message. This message was enciphered using the directly executable program ENCIPHER.EXE which is contained on the CD-ROM in the BASIC directory. The enciphered message, which is stored on the file CIPHERTX.DAT on the CD-ROM, can be deciphered using the directly executable program DECIPHER.EXE, which is also contained on the CD-ROM. To facilitate your effort, unlike the original version of this book, this new edition contains both source and executable versions of the programs ENCIPHER and DECIPHER.

Figure 7.7 The "challenge" message.

Providing the deciphering program may simplify the task of deciphering the message. You can consider using a trial-and-error approach by using different "secret" codes in an attempt to decipher the message. As you cycle through various combinations of "secret" codes, one of the points raised a number of times in this book will become apparent—the program provides a level of privacy for electronic transmission that will deter the casual observer of a message from understanding the contents of the message.

The reason for originally providing only directly executable copies of the two programs whose use was described in this chapter was to hide the technique employed to use the "secret" code used by the enciphering program to generate a random number sequence. Although I originally left the door unlocked, I purposely decided not to turn on the lights!

In a future edition of this book, I anticipate extending the coverage of topics to codebreaking. Thus, if you feel you are successful in deciphering the message illustrated in Figure 7.7 and contained on the file SECRET.MSG, please send a description of the approach you used to decipher the message as well as the deciphered message to me via e-mail at 235-8068@mcimail.com.

I would like to consider describing techniques used by you to decipher the message illustrated in Figure 7.7 or even approaches you took that were not successful. Thus, I welcome all reader comments concerning not only the specific message you are challenged to decipher but all aspects of this book.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Chapter 8
Public Key Encryption

In this concluding chapter, I turn your attention to a relatively new field of encryption when examined against the evolution of encryption techniques dating to before Caesar's time. That field of encryption is referred to as public key encryption, the focus of this chapter.

Unlike the previous coverage of encryption techniques which included a minimal amount of mathematics, any discussion of public key encryption requires an understanding of a considerable number of mathematical operations. Those operations are crucial to understand how public key encryption works. A second difference between this chapter and previous chapters presented in this book concerns the development of programs which will be conspicuous by their omission in this chapter. The reason for this omission is based upon current United States government restrictions on the export of certain types of encryption software. Because neither I nor my publisher can police the sale of this book nor its distribution, it is safer to simply present concepts in this chapter and leave it to you to develop public key software programs if you so desire.

General Operation

Figure 8.1 illustrates the general operation of a public key-based encryption system. In this example, an encryption algorithm (E) is used with a public encryption key (ke) to encipher input (x), producing ciphertext (Eke(x)). Decryption employs a decryption algorithm (D) which operates upon a secret decryption key (kd) such that D(Eke(x)) restores the encrypted input to its original form. Note that the algorithms E and D are public; however, the decryption key (kd) must be kept secret. When public key-based secure communications occur in the opposite direction, another key pair must be created, with the recipient becoming the originator and using an encryption key, while the originator becomes the recipient and uses a secret decryption key. Thus, communication in each direction requires the use of a public-private key pair.

Figure 8.1 Public key encryption.

Authentication Issues

Because the public encryption key is published, anyone can use it to encrypt a message. This means that it becomes possible for a person to use a public key to send a false message which, when deciphered, could appear as a legitimate message. This problem is not normally applicable to conventional private key systems since the distribution of private keys are on a controlled basis. This also means that a public key system normally requires a method of authentication, typically accomplished by the use of a digital signature. However, it is also possible for two persons using a public key system to authenticate one another by placing a random string into their message and requesting the party they are communicating with to return the string in their initial reply. Because the recipient is the only person who can decipher the message, when they encrypt the message in another public key associated with the originator's private key, the originator can check the message to ensure it was correctly received. Then, this receipt indicates that the recipient is the party they claim to be. Figure 8.2 illustrates how two parties can authenticate one another via the use of a public key system.

Figure 8.2 Authentication via the exchange of a trial message.

Public vs. Private Key Encryption

In a classic private key cryptosystem, both the sender and receiver of a message must know and use the same secret key. This means both parties must have access to the same key, and if either party should lose or divulge the key, both parties are compromised. Although the use of secret key cryptography dates to before the time of Caesar and is still frequently used in electronic commerce and other business applications, it has several weaknesses in addition to the disclosure of the key compromising messages sent between parties. First, the management of keys can become a daunting task, especially when you have many persons or locations that must communicate with one another and wish to minimize the potential compromise of a key on messages by assigning different keys to each pair of persons or locations that requires secure communications. A second problem associated with secret keys is their distribution. Some organizations initially distribute keys via registered mail, which considerably reduces your ability to promptly respond to changing user requirements. Similarly, if a key should become compromised, the ability to distribute a new key will be dependent upon a telephone call, a courier, or an Express Mail letter, all subject to a security breach.

The Work of Diffie and Hellman

In 1976 the concept of public cryptography was introduced by Whitfield Diffie and Martin Hellman as a mechanism to solve the key management and key distribution problems associated with private key cryptography. In their now- classic paper, they discussed the features of a public key system and noted that such a system should have the following four properties for a given message (M) based upon encryption (E) and decryption (D) procedures:

1. D(E(M)) = M
2. Both E and D are easy to compute.
3. It is computationally unfeasible to derive D from E.
4. E(D(M)) = M

The first property simply states that once a message is encrypted the application of a decryption procedure will restore it, in effect the same as the property for a private key system. The second property means that for the cryptosystem to be practical, both encryption and decryption should be computationally fast, which also holds for private key cryptology. It is the third property which signifies the key difference between public and private key cryptology, as this property means that encryption is a one-way function and it is not feasible to invert E unless you already know D. This also means that the encryption procedure E can be made public, and only the person who knows D can decrypt the message.

The fourth property indicates that the mapping is one-to-one, meaning that the application of the decryption procedure to a message (M) can be used to verify the person who sent the message, in effect permitting a digital signature.

Under the Diffie-Hellman concept, each person who requires secure communications obtains a pair of keys. One key is referred to as the public key and can be distributed to any person who requires the ability to communicate securely with the person who holds the secret key. The key, no pun intended, to public key encryption is the fact that no third party can determine the composition of the private key from the public key.

It is important to note that the paper presented by Diffie and Hellman provided a concept or framework for the development of public key encryption techniques. However, Diffie-Hellman does not actually represent a public key encryption technique. Although not an actual technique, the work of Diffie and Hellman considerably influenced the development of public key encryption techniques and represents a significant milestone in the field of encryption.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Advantages and Disadvantages

The primary advantage associated with the use of public key cryptology is the increased level of security and convenience it provides, which reduces the effort involved in key management and key distribution associated with a private key system. Since the private key associated with the public key system does not have to be distributed, this both simplifies key management and reduces the possibility of key interception.

The ability to distribute a public key that can encrypt messages that can only be decrypted by the corresponding private key depends upon the selection of an appropriate algorithm. The algorithm must be a one-way function, which from a computational perspective is significantly easier to perform in one direction than from the reverse. For example, the one-way function might be able to be performed in seconds or minutes in the forward direction, but require months or years of trial and error to perform in the opposite direction.

As an example of a one-way function, consider the following equation:

     y = 5x4 + 27x3 + 14x2 + 12

While it is relatively easy and fast to compute y given x, it is much more difficult to compute x given y. Although the time required to execute the one-way function in the forward direction is relatively fast in comparison to the process to perform the reverse, it requires significantly more than the amount of time required to perform private key encryption. This means that public key encryption may not be suitable for certain types of applications, such as high-speed encrypted communications. In such situations, public key cryptology can be used as a supplement to private key cryptology. For example, you could use a public key system to distribute private keys. In fact, a popular application for the use of a public key system by government, industry, and financial organizations is the distribution of private keys via a public key system.

There was a young fellow named Ben
Who could only count to modulo ten.
He said, 'When I go
past my last little toe
I shall have to start over again.'

-Anonymous

Understanding the Mathematics

The ability to obtain an appreciation for the manner by which public key cryptology operates requires an understanding of a few areas of mathematics that are used to create one-way functions. Thus, prior to turning your attention to the manner by which a popular public key system operates, let me turn your attention to a review of some mathematical concepts.

Most public key systems are based upon finite arithmetic in which numbers are manipulated according to rules that are different from ordinary arithmetic. Because one-way functions used in public key systems are based upon finite arithmetic, and modular arithmetic results in finite sets of numbers, let's commence our review of mathematics with this topic. Since I previously discussed modular arithmetic when developing private key algorithms in this book, I will not turn to this topic in detail.

Modular Arithmetic

Modular arithmetic is a branch of mathematics in which you operate upon a finite set of numbers. For example, the finite set of numbers {0, 1, 2,. .. n-1} are referred to as modulo n arithmetic. Here, n can take any positive integer value except 1 and the result of any arithmetic operation belongs to the same set of n numbers.

Counting in modulo arithmetic goes from 0 to n-1, with the next numbers in the series being 0, 1, . . ., so that the sequence continues and repeats. In fact, when you perform modulo arithmetic, such operations as addition and multiplication are performed as with ordinary arithmetic, with the answer obtained by dividing by n and using the remainder as the result. The result is said to be modulo n, abbreviated as mod n. Thus writing x mod n references the remainder of x when divided by n. To obtain an appreciation for the operation of modulo arithmetic, let me turn your attention to modular addition and modular multiplication.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Modular Addition

When you perform modular addition, the value you obtain for a+b is the same as in normal arithmetic when a+b<n. Otherwise, when a+b>n, you would subtract n from the sum. If the result is less than n, you would obtain the modulo sum. Otherwise, you would continue to subtract n from the previous result until a value less than n is obtained. This action is equivalent to dividing by n and using the remainder as the result.

For example:

    8 + 6 + 3 + 9 + 2 + 6 (modulo 9) = 7

because the sum in normal arithmetic is 34. When divided by 9, you obtain a remainder of 7.

Now it can be told!

Many years ago during the era of communism, I was selected to represent the United States at an engineering conference in Moscow. At that time most home telephones of dissidents and refuseniks were monitored by the KGB; however, most public telephones were not, as there were too many even for the resources of the Soviet spy agency. After meeting with dissidents and refuseniks, I agreed to carry a list of public telephone numbers and calling times out of Moscow so those assisting these harassed persons could communicate with them. Because it wouldn't be smart to simply carry out a list of telephone numbers, those numbers were encoded using modulo 9 arithmetic, and an extra digit was added so that the numbers appeared to represent the combination to a safe or lock. For example:

original telephone number: 4770293
modulo 9 number: 5229706
"safe combination" 522-97-063

Upon return to the free world it was a relatively simple process to convert the "safe combination" back into its telephone number. To do so, all that was required was to add the appropriate digit to each number to obtain nine and discard the last digit. For example, 5 + 4 = 9, so the first digit is restored to its original value of 4. Similarly, 2 + 7 = 9, resulting in the second digit of the restored telephone number reconstructed to its original value of 7.

For several years after I visited Moscow, my "modulo 9" number scheme was transported via tourists and in postcards to inform persons in the United States, the United Kingdom, France, and Israel of public telephone numbers they could call, and the date and time to call.

If you perform mod 10 addition, you can simply use the last digit of the answer. Similarly, if you perform mod 100 addition, you can use the last two digits. For example:

    6 + 13 mod 10 = 9,
    114 + 8 mod 100 = 2, and
    9 + 8 mod 10 = 7

Note that 9, 19, and 29 all represent the same mod 10 number, while 33, 133, and 233 all represent the same mod 100 number. In such situations it is common to use the symbol "@" for equivalence for each series of numbers. Under modular arithmetic two equivalent numbers can be considered as different names for the same mod n number.

Figure 8.3 illustrates results obtained by the construction of a mod 10 addition table. Note that, similar to the sidebar story about my use of modulo 9 addition to encrypt telephone numbers, you can also use a constant modulo 10 as a scheme for encrypting digits. This is because the use of a constant modulo 10 maps each decimal digit into a different decimal digit, such that the process is reversible. In doing so, the use of a constant modulo 10 becomes your "secret" key and decryption is performed by modulo 10 subtraction. Thus, let me turn your attention briefly to modular subtraction.

Figure 8.3 Modulo 10 addition.

Modular Subtraction

There are several methods you can use to perform modular subtraction. To illustrate these methods, assume you wish to perform the operation (a-b) modulo n. If a>b, you would simply subtract b from a. If a<b, you would add n to a prior to subtracting b. For example, (7–8) modulo 10 is equal to 7+10–8, or 9.

A second method you can use to perform modular subtraction is by adding –b, which is referred to as b's additive inverse. Returning to our previous example and noting that the additive inverse of a number is the number you have to add to it to obtain zero, you have:

    (7–8) modulo 10 becomes equivalent to 7–(–2) = 9

In the preceding example note that 8's inverse is 2, because in mod 10 arithmetic 8 + 2 = 0. Thus, if the secret key was 8 for encryption, you would add 8 mod 10, while decryption would be performed by adding 2 mod 10.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Modular Multiplication

To continue your examination of modular arithmetic, let me turn your attention to modular multiplication. Figure 8.4 contains a modular 10 multiplication table, indicating the mod 10 values obtained by multiplying each row value by each column value, mod 10. For example, 8*8 mod 10 is 64 mod 10, which is 6 with a remainder of 4 when divided by 10, producing a result of 4.

Figure 8.4 Modulo 10 multiplication.

If you examine the multiplication table illustrated in Figure 8.4, you will note that multiplication by 1, 3, 7, and 9 will work as an encryption key since each multiplication results in a one-to-one substitution of the digits. You will also note that multiplication by 0, 2, 4, 5, 6, and 8 will not work as an encryption key. For example, if you attempted to encrypt by multiplying by 2, half the numbers would be duplicates and result in the loss of information. Thus, you would not be able to correctly decrypt using a multiplier of 2.

If you select an appropriate multiplier using modular 10 multiplication, it becomes possible to both encrypt and correctly decrypt data. Concerning the decryption process, similar to modular arithmetic addition, you can undo the effect of modular multiplication by multiplying by an inverse. Here the inverse is known as the multiplicative inverse.

In ordinary arithmetic the multiplicative inverse of x is 1/x, which results in a fraction when x is an integer. In modular arithmetic, the set of numbers you work with are all integers. Thus, the multiplicative inverse of x, which is written as x-1, is the number by which you multiply x to obtain 1.

For example, consider 5-1 mod 4. This means what number multiplied by 5 mod 4 results in a value of 1. Because 5 mod 4 has a value of 1, this is a relatively simple problem whose answer is 1. However, as you will note later in this chapter, finding the multiplicative inverse can become a daunting task.

Returning to Figure 8.4 which contained the modulo 10 multiplication table, only the numbers 1, 3, 7, and 9 have multiplicative inverses modulo 10. For example, 7 is the multiplicative inverse of 3, while 9 is the multiplicative inverse of 1. Thus, encryption could be performed by multiplying by 3 while decryption could be performed by multiplying by 7. Note that both 1 and 9 are their own inverses. Thus, similar to modular addition, modular multiplication could be used to encrypt and decrypt, although you do have to be careful of the manner by which you select digits.

Finding the Multiplicative Inverse

The ability to find a multiplicative inverse in modular arithmetic can be difficult, especially when n is large. For example, if n was a 50-digit number you could spend a considerable period of trial and error time searching for an inverse. Fortunately, the Euclidean algorithm can be used to find the inverse as, given x and n, the algorithm finds the number y such that x*y mod n = 1.

The reason multiplicative inverses are important is the fact that they are the only numbers that do not share any common factor other than 1. This means they are in mathematical terms relatively prime numbers. For example, from Figure 8.4 the numbers 1, 3, 7, 9 are the only ones with multiplicative inverses. That is, the largest integer that divides both 3 and 7 is 1. Similarly, the largest integer that divides both 7 and 9 is also 1. In comparison, 2, 4, 6, and 8 do not have a multiplicative inverse modulo 10, and are not relatively prime to 10. For example, 2 divides 4, 6, and 8. This means that you can use mod n multiplication by any number x relatively prime to n as an encryption key since you can multiply x to encrypt and x-1 to decrypt. Although whether or not the scheme is secure remains to be determined, this means encryption can occur via multiplication by x mod n, while decryption can occur by multiplying the enciphered data by x-1 mod n.

The key, again no pun intended, to the use of numbers that have multiplicative inverses is to use relatively large prime numbers. Thus, prior to moving forward, let's review prime numbers.

Prime Numbers

A prime number is a number greater than 1 that can be divided evenly only by itself and 1. The first 20 prime numbers are 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, and 71. The fundamental Theorem of Arithmetic indicates that primes are the building blocks of the positive integers. In fact, every positive integer is a product of prime numbers. Such products are referred to as composites. Thus, there are three types of natural numbers: primes, composites, and unity.

The ancient Greeks proved mathematically that there are an infinite number of primes and that they are irregularly spaced. During the late 1990s the use of primes in public key cryptology resulted in thousands of persons searching for larger primes. Through the use of specialized software, a prime consisting of 895,932 digits was discovered in 1997.

Three of the more interesting types of primes are Mersenne primes, twin primes, and Sophie Germain primes. Mersenne primes are primes of the form 2⊥p-, and are the easiest type of number to check for on a binary computer. Although 36 Mersenne primes have been discovered, several regions of numbers between previously identified primes have not been completely searched, and it is not known if the largest is the 36th.

Twin primes are primes of the form p and p=2, and differ by two. A Sophie Germain prime is an odd prime p for which 2p+1 is also a prime. This category of primes was named after Sophie Germain who provided the first case of Fermat's Last Theorem, i.e., xn + yn = zn has no solutions in non-zero integers for n>2 for exponents divisible by such primes.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

The Euclidean Algorithm

It was briefly mentioned previously that the Euclidean algorithm provides a mechanism to find the multiplicative inverse mod n. Both the Euclidean algorithm and a function developed by Euler known as the totient function play an important role in public key encryption. Thus, prior to directing your attention to a public key algorithm, an additional review of an algorithm and function is warranted.

As you might expect, the Euclidean algorithm was invented by Euclid. While it is conjectured that Euclid was attempting to determine a method to find the greatest common divisor (gcd) of two integers, resulting in the algorithm holding his name, his algorithm can also be used to determine the multiplicative inverse mod n. Thus, it holds an important role in several public key encryption methods.

As a review, the greatest common divisor of two integers is the largest integer that evenly divides both. If the greatest common divisor of two numbers is 1, then those numbers are relatively prime. This means that an algorithm that enables you to efficiently find the gcd of two integers can also determine if the gcd of the two integers is relatively prime. As you will shortly note, if the gcd of two integers is relatively prime, you can find their multiplicative inverse. For example, if you want to find the multiplicative inverse of m mod n, this means you need to find a number μ such that μm = 1 mod n. This means that μm differs from 1 by a multiple of n. This also means that there is an integer v, such that μn + vm = 1. As you will soon note, you can use Euclid's algorithm to find μ and v provided gcd(m,n) = 1. If m and n are not relatively prime, you will not be able to determine μ and v, which means that m does not have a multiplicative inverse mod n.

To illustrate the Euclidean algorithm, let's assume you want to compute the gcd of 432 and 252. If you assume the pair x,y with y<x, then the smaller number y will have a divisor D such that x-y, x-2y, etc., also have that divisor unless one of them is zero, a situation which indicates that y divides x and y is the gcd. Thus, you can take x-ny where n is the integer result of dividing x by y, and x-ny is the remainder, which is less than x.

The top portion of Figure 8.5 illustrates generic coding for finding the gcd using the Euclidean algorithm. The lower portion of Figure 8.5 illustrates the use of the Euclidian algorithm for finding the gcd of 432 and 252. Starting with 432 (a) and 252 (b), you divide a by b and store the remainder of 180 (c). Next you divide b by c and take the remainder of 72 as d. You repeat this sequence of operations until you obtain a zero remainder. The last number before zero, which is 2, is the gcd. If the two numbers have no other common divisor, the number 1 will be the result, indicating that they are relatively prime. An example of the use of the Euclidean algorithm to determine that two numbers have no other common divisor than 1 is shown in Figure 8.6.

Figure 8.5 Finding the greatest common divisor using the Euclidean algorithm.

Figure 8.6 Using the Euclidean algorithm to determine that 19 and 3 are relatively prime to one another.

The Totient Function

The symbol 0 is known as the totient function, supposedly a contraction of the terms total and quotient. Through the use of the totient function, you can determine how many numbers less than n are relatively prime to n. For example, if n is prime, then the set of integers {1, 2, … n-1} are relatively prime to n, resulting in 0(n) = n-1. If you assume n is a product of two distinct primes, p and q, then there are (p-1)*(q-1) numbers relatively prime to n, resulting in 0(n) = (p-1)(q-1).

As previously indicated, when you construct a table of modulo exponentiation you will note that the results in certain columns are the same. Based upon the research of mathematicians, it turns out that xy mod n is the same as x(y mod 0 (n)) mod n. For example, assume x = 2 and y = 2. The numbers relatively prime to 10 are {1, 3, 7, 9}, therefore 0(10) = 4. Then:

    xy mod n = 22 mod 10 = 4

and

    x(y mod 0(n)) mod n = 2(2 mod 4) mod 10 = 4

and

    y = 3.

Then:

    xy mod 10 = 53 mod 10 = 5

and

    x(y mod 0(n)) mod n = 5(3 mod 4) mod 10 = 5

The use of the preceding formulas as equivalency is true for all values of n that are primes or any product of distinct primes. One important special case of the preceding is where y = 1 mod 0(n). In this situation, for any number x, xy = x mod n, which forms the basis for the RSA public key encryption system.

Once you determine a number is a prime through the use of the Euclidean algorithm, you can use the totient function to determine the multiplicative inverse. For example, if n is a prime, then 0(n) = n-1. If x is not a multiple of n, then:

    x0(n) mod n = 1

Then, the multiplicative inverse x-1 mod n becomes:

    Inverse = x0(n)-1 mod n

To illustrate this, let's assume you want to determine 5-1 mod 3. Since 3 is prime, you would then compute:

    multiplicative inverse = 50(3)-1 mod 3

Because:

    0(3) = 3-1 = 2

Then:

    multiplicative inverse = 52-1 mod 3 = 5 mod 3 = 2

Check the preceding as follows:

    2*5 mod 3 = 10 mod 3 = 1

Sure enough, 2 is the multiplicative inverse of 5 mod 3.

Now that you have an appreciation for the general mathematics behind public key cryptology, let me turn your attention to the operation of a popular public key encryption system.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

RSA

RSA is a public key encryption system invented by Rivest, Shamir, and Adleman in 1978. Under this technique, both encryption and decryption are performed using exponentiation. RSA obtains security from the difficulty associated with factoring large prime numbers. As you will shortly note, public and private keys are functions of a pair of large prime numbers. Thus, the ability to decipher plaintext from the use of a public key and ciphertext is the equivalent of factoring two large primes.

Using the RSA system, each party generates a public key and a corresponding private key. To do so, two large primes of equal length are selected, p and q. To facilitate security, each prime should be a minimum of 32 bytes, or 256 bits, in length. Once the two primes are selected, you multiply them together and find their product n, which is referred to as the modulus of pq.

Public Key Generation

To generate a public key, each person using the RSA algorithm generates their encryption (e) and decryption (d) keys. The encryption key, e, is selected such that it is less than n and relatively prime to (p-1)(q-1), which represents the totient function 0(n). This means that e and (p-1)(q-1) have no common factors other than 1. Your public key then becomes the pair <e,n>.

Private Key Generation

To compute your private key, you will find another number d, such that (ed-1) is divisible by (p-1)(q-1). That is, d represents the multiplicative inverse of e mod 0(n). Then, the private key becomes the pair <d,n>. Once your public and private keys are selected, the primes p and q are no longer needed. Although it is safe to discard them, they should not be revealed.

Message Encipherment

To encipher a message m you would first subdivide it into blocks smaller than n (m<n). When working with binary data this would be equivalent to selecting the largest power of 2 less than n. You would then use your public key (e) to compute ciphertext (c) as follows:

    c = mie mod n

where mi represents block I of message m.

To decrypt a message you would use your private key (d) on each ciphertext block ci as follows:

    mi = cid mod n

To illustrate the use of RSA, let's use a few small primes to facilitate observing the relevant computations. First, let's assume you selected p = 53 and q = 61. Then:

    n = p*q = 53*61 = 3233

Next you would randomly select an encryption key, e, such that it has no common factors with 3233. For our example, let's select e to be 41. Then, the private key d would be:

    d = 41-1 mod 3233

To compute d, you must obtain the multiplicative inverse of 41 mod 3233. That is, you must determine the number which when multiplied by 41 mod 3233 yields a result of unity. Based upon Euler's application of the totient function, if n is a prime, the 0(n) = n-1. If p and q are primes and n = pq, then 0(n) = (p-1)(q-1). Thus, if the gcd (a,n) = 1, where x is not a multiple of n, then:

    x0(n) mod n = 1

This means you can compute the multiplicative inverse x-1 mod n as follows:

    X = x0(n)-1 mod n

Returning to your computation, you need to determine the private key d, such that:

    d = 41-1 mod 3233

Since 3233 is prime, then 0(3233) = 3233-1 = 3232. Thus, the inverse of 41 mod 3233 becomes:

    413232-1 mod 3233 = 413231 mod 3233


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Exponentiation Operations

While it is possible to attempt to raise a number to a large power, doing so will more than likely exhaust the capacity of your computer as there is a finite limit to the size of integers on each computer. Instead of directly raising a number to a power, you can do the modular reduction after each multiply operation. For example, assume you want to compute 1237 mod 456. Instead of computing 123 multiplied by itself 7 times and then dividing by 456 to obtain the remainder, let's do modular reduction after each multiply. That is:

    1232 = 123*123 = 15129 mod 456 = 81 mod 456
    1233 = 123*81  mod 456 = 9963  mod 456 = 387 mod 456
    1234 = 123*387 mod 456 = 47601 mod 456 = 177 mod 456
    1235 = 123*177 mod 456 = 21771 mod 456 = 339 mod 456
    1236 = 123*339 mod 456 = 41697 mod 456 = 201 mod 456
    1237 = 123*201 mod 456 = 24723 mod 456 = 99  mod 456

While the preceding method reduces the computation to a series of seven small multiplies and seven small divides, most exponents used with RSA are considerably larger. This means that you would more than likely want to obtain a more efficient method to compute exponents.

To raise a number x to an exponent, you can perform a series of squaring operations. For example:

    1232 = 123*123 = 15129 mod 456 = 81  mod 456
    1234 = 81*81   = 6561  mod 456 = 177 mod 456

While you could continue and square 123 again, obtaining the value of 1238, unfortunately the 1237 that you require is not a power of 2. However, if you know what 123x is, then it is relatively easy to compute 1232x, since you obtain that by squaring 123x. Similarly, you can compute 1232x+1, as all you need to do is multiply the result obtained by squaring 123x (1232x) by 123. This means you can perform a sequence of squaring operations to obtain the nearest power of two to the exponent you require, and then multiply the result n times, where n is the difference between the exponent you seek and the nearest computed power of 2. For example:

    1237 mod 456 = 1232 * 1232 * 123 * 123 * 123 mod 456
    1234 mod 456 = 177 mod 456

Then:

    1237 mod 456   = 123*123*123*177 mod 456
                   = 123*123*339 mod 456
                   = 123*201 mod 456
                   = 99 mod 456

Note that the preceding method required two squaring and three multiplication operations. You can reduce the number of operations further if you note that 123a*123b = 123a+b. This means given:

    1232 mod 456 = 81  mod 456
    1237 mod 456 = 387 mod 456

you would compute the following:

    1236 = 1232*1234 = 81.387 mod 456 = 339 mod 456
    1237 = 123*1236 = 123.339 mod 456 = 99 mod 456

Note that in this example you would compute 1232 mod 456, 1234 mod 456, 1236 mod 456, and then multiply the last result by 123. This reduces the number of computations to two squares and two multiplies.

If you convert the exponent to binary, you can compute 123 raised to a sequence of powers. For example, 7 is 1112 which represents the sequence 12, 112, 1112. Note that each successive power concatenates one additional bit of the desired exponent. Also note that each successive power (from the most significant bit position) is either twice the preceding power or once or more than twice the preceding power. Thus, you can raise 123 to the first, third, and seventh power by repeated squaring together with multiplication by 123 for the bits that are 1. For example, commencing with the first power you start with:

    123 mod 456

To obtain the third power, you would multiply 123 by its square, i.e., 123 (1232). To then obtain the seventh power, you would obtain the sixth power (1233)2 and multiply by 123. Thus, raising 123 to the seventh power can be accomplished by repeated squaring and, as required, multiplying by 123 for the bits that are 1. Focusing on exponents:

    7 = (((1)2 + 1)2 +1)

Then you obtain:

    1237 = (((123)2 * 123)2 * 123)

Using the preceding method you can perform exponentiation of a base to an exponent as follows:

1. Commence by setting an initial value to 1.
2. Convert the exponent to binary.
3. Read the binary value of the exponent bit by bit from high order to low order and square your value for each position. If the value of the bit position is 1, then multiply by the base. After each operation, perform modular reduction to maintain a relatively small intermediate result.

Now that you have an appreciation for techniques to facilitate exponentiation, let's return to the key computation process. It was noted previously that the inverse of 41 mod 3233 is:

    413231 mod 3233

Note that 3231 is 1100100111112. This means you can compute 41 raised to the 3231 power by squaring 12 times and multiplying 8 times. Although I will leave it as an exercise for you to compute, let's assume the answer was x. You would then publish e and n and keep the value x which represents your secret decryption key. Suppose you want to encrypt the famous message ABADABA… Then, the ASCII value of your message m is:

    m = 64656467646564…

To facilitate computations you would break the message into small blocks. For example:

    m1 = 6465
    m2 = 6467
    m3 = 6465

and so on.

Using the encryption key of 41, the first block would be encrypted as:

    c1 = 646541 mod 3233

Similarly, the second block would be encrypted as follows:

    c2 = 646741 mod 3233

You would continue the same operation on the remaining blocks until you generated the entire encrypted message. As previously discussed, decryption would be performed on a block by block (c1, c2…) basis, performing the same exponentiation but using the decryption key whose value is x. Thus:

    m1 = c1x mod 3233
    m2 = c2x mod 3233

and so on.


Learn Encryption Techniques with BASIC and C++

 Learn Encryption Techniques with BASIC and C++ - Gil Held

Facilitating the Key Generation Process

Based upon the preceding, the use of RSA depends upon selecting appropriate values for n, d, and e. As previously noted, selecting those values to obtain an RSA key can be significantly computational intensive. In fact, the selection of values for n, d, and e can be substantially more computational intensive than using its public and private keys. Fortunately there are a few methods you can apply to reduce your computations.

Locating Large Primes

Although there are an infinite number of primes, you will more than likely select them randomly. This means you will require a mechanism to determine if your random selection is a prime.

One method that can be used to determine if a number n is a prime is to divide it by all integers >2 and <= √n and note whether or not the division comes out even. For example, consider the prime 11. The integer of its square root is 3. Then 11/2 and 11/3 do not provide an even number. Thus, 11 is a prime. Now consider 16. Its square root is 4. Then, 16/2, 16/3, and 16/4 provide at least one even number, which means 16 is a non-prime.

While the preceding method provides a mechanism for testing whether or not a number is a prime, as primes get rather lengthy, the time required for such testing becomes impractical. A more practical method to test for a prime is obtained by the use of Euler's totient function. As previously noted, if n is prime, then 0(n) = n-1. When this situation occurs, the function takes on a simpler form and is known as Fermat's theorem.

Fermat's theorem says that if p is prime and 0<x<p, where x is relative prime to n, then:

    xp-1 = 1 mod p

Based on Fermat's theorem you would pick a number x such that x<n and compute xn-1 mod n. If the answer is not 1, then n is not a prime. If the answer is 1, there is a very minute possibility that n is not a prime; however, the probability is so small that you can safely consider the number to be a prime.

The steps necessary to locate an appropriate prime can be summarized as follows:

1. Select random number n in an appropriate length. Note that, according to RSA Laboratories, the recommended module sizes are 768 bits for personal use, 1,024 bits for corporate use, and 2,048 bits for extremely important keys, such as the key pair of a certifying authority. Because the two primes (p and q) should be roughly equal in length, the recommendation of a 768-bit modulus for personal use results in each prime having a length of approximately 384 bits.
2. Select a random value for x, such that x<n, and compute xc mod n, where c represents an odd number for which n-1=26c and b is the number of times 2 divides n -1, i.e., 2b is the largest power of 2 that divides n -1. If the computation of xc mod n is +1 or -1, n has passed the primary test for the selected value of x. If the computation does not result in +1 or -1, replace the result by its square and determine if the computation produces +1 or -1. If the result is +1, n is not prime as the previous result is a square root of 1 different from +1, -1. If the result is -1, n passes the primary test for x. You would repeat the preceding b -1 times.
3. If n is not a prime, return to step 1.

Summary

The intention of this chapter was to acquaint you with the concept behind public key cryptology to include its mathematics. Although the examples should provide a foundation sufficient for developing programs, a word of caution is in order. Currently, government regulations concerning the export of encryption software with lengthy keys is prohibited. In fact, an office in the government called Munitions Control passes judgment on the ability to export certain types of encryption software and hardware. Because the government has made life difficult for those who have simply posted software on their Web site, you are cautioned to carefully consider how any software you plan to develop that uses keys in excess of 40 bits will be distributed. Otherwise, there is a chance you could experience the necessity to incur a significant amount of legal fees in attempting to explain your actions.