Newsgroups: sci.crypt
Path: msuinfo!caen!sdd.hp.com!elroy.jpl.nasa.gov!nntp-server.caltech.edu!heathh
From: heathh@cco.caltech.edu (Heath Ian Hunnicutt)
Subject: Re: Non-Randomness in DES-Encrypted Files
Message-ID: <1992Jan22.031638.5203@cco.caltech.edu>
Organization: California Institute of Technology, Pasadena
References: <3238@wet.UUCP>
Distribution: usa
Date: Wed, 22 Jan 1992 03:16:38 GMT

The following posting is a flame against Peter Davidson, and Dolphin Software. 
Those of you who are not interested in bitter tirades against stupid people
should skip the rest of this article.


Also, if anyone could mail me instructions for insert the [new-page]
character which more recognizes into my posts using vi, I would never
again leabe all these blank lines.

BTW: The best part is at the very end, if you get bored, skip to that.








naga@wet.UUCP (Peter Davidson) writes:
^^^^
Still playing D&D, guys?

> 
>   Some Observations Concerning the Non-Randomness of DES-Encrypted Files
Nice title.  Makes you sound like an expert, all right.

>                      Copyright 1992 Dolphin Software
Wrong, bucko.  Your article is entirely in the public domain, and it
can be quoted and criticized, not to mention reproduced without your
permission.  In fact, most of the rights you claim with your 
"Copyright" crap are usurped by the owner of the InterNet backbone,
that is, the U.S. government.

>It is believed by many that DES randomizes plaintext, a view expressed by
>Dr Peter Wayner, of the Department of Computer Science at Cornell
>University, when he writes in a recent sci.crypt posting: "...  DES has
>generally been found to be a relatively good random number generator.  It
	I don't think is it especially cool of you to pick on Dr. Wayner,
a man who has already shown that he is a good deal smarter than you 
yay-hoos.

>However, applying frequency-analysis programs to DES-encrypted files often
>reveals startling regularities, a fact which seems inconsistent with the
>claim that DES is a good random number generator.  Not all DES-encrypted
>files exhibit such startling regularities, but many do.
	Applying even a modicum of common sense to the test cases you give
reveals _startling_ *ahem* examples of your own poor reasoning.

>What follows is a report describing some observations which cast doubt
>on the randomizing qualities of the DES algorithm.
	Why not let me decide if they cast doubt on the randomness of the
DES?  Prove your point, don't bludgeon me into believing it.

> 
>MC1.GEN is a documentation file which consists mostly of standard English
>text.  It was written from Word Perfect as a generic text file, meaning
>that the Word Perfect header and all Word Perfect formatting was thrown
>away, and CR/LF byte pairs occur only at the ends of paragraphs (not at the
>ends of all the lines).
> 
>When MOSTFREQ.EXE, a frequency analysis program, is run on MC1.GEN it
>produces the following results for the 24 most-frequent bytes:
> 
>    File MC1.GEN contains 52882 bytes and 89 different byte values.
>      Rank     Byte      Frequency   ASCII value
> 
>        1    32 = 0x20       12470      space
>        2   101 = 0x65        3882      e
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Who are you trying to kid?  Right here, your whole argument breaks down.
With that many spaces, there are definitely several cases in your test
where blocks of 8 spaces occur on exactly an 8-byte boundary.  These
blocks will always encode to the same ciphertext blocks.  Therefore,
an unusual number of strings of spaces will skew the stats
drammatically.

>This is, of course, consistent with the usual comparative frequency of
>occurrence of letters in English.
	It is? Approx 3 times as many spaces as e's?  I doubt it.
I'll believe your claims when all the strings of spaces are condensed
to exactly one space.  i.e., no two spaces are adjacent.

> 
>MC1.GEN was first encrypted using Central Point Software's implementation
>of DES, PCSECURE (Version 5.1) using full DES encryption in expert mode.
>The hexadecimal encryption key used was FB 19 9D 02 A4 83 2F 85.  MOSTFREQ,
>when run on MC1.SEC, produces the following results for the 24 most-frequent
>bytes:
> 
>    File MC1.SEC contains 52956 bytes and 256 different byte values.
>      Rank     Byte      Frequency   ASCII value
> 
>        1    33 = 0x21         471      !
>        2    83 = 0x53         471      S
>        3    51 = 0x33         464      3
>        4   186 = 0xBA         462
>        5   192 = 0xC0         455
>        6   200 = 0xC8         455
>        7     5 = 0x05         451
>        8   165 = 0xA5         451
> 
> 
>It will be seen that in MC1.SEC there are eight byte values which occur far
>more frequently than the rest.  In fact, the least-frequent of these byte
>values (05 and A5, occurring 451 times each) occur 80% more frequently than
>any of the other 248 byte values (the most frequent of which is FA, which
>occurs 250 times).
	I am not surprised at this, considering that file you fed it.

> 
>To disprove the hypothesis that this might be a peculiarity of PCSECURE
	I don't think you've proved anything.  If you want to be so
darn sure that the encryption is done correctly, write your own
DES program.  It's not that tough, for most people.

>(which, it will be noted, does not preserve the size of the file during
>encryption) 
	Your ignorance is showing.  Any DES algorithm must round the
number of bytes in the plaintext message up to a multiple of 8.  This
is usually done by padding the end with 0x00 or such.  I believe
differences in this padding cause some of the differences between
output in your next two examples.

>the same file, MC1.GEN, was encrypted using The Private Line
>(version 6.03, in Electronic Code Book mode) using the same hexadecimal
>password, to produce a file MC1.TPL.  The frequency results obtained by
 ^^^^^^^^
	Password? Try "Key".
>MOSTFREQ are:
> 
[Omitted boring table]
> 
>It will be seen that in MC1.TPL there are again eight byte values which
>occur far more frequently than the rest.  The least-frequent of these byte
>values (C5, occurring 449 times) occurs over 62% more frequently than any
>of the other 248 byte values (the most frequent of which is 16, which
>occurs 276 times).
> 
>Thus both encryption programs produce ciphertext in which exactly eight byte
>values stand out from the others in a glaring fashion.  However, they are
>not the same eight bytes values.  In fact, the two sets of eight byte
>values have no byte value in common.  This leads one to suspect that one
>of these programs has not implemented the DES algorithm exactly.
	Or, perhaps, PCSECURE reads the key in some different order
than you expected, and BTW, did you make sure that you set it up for
"Full DES" rather than "quick encryption".  Also, if you doubt the
validity of your results, why cite them?
> 
>The file MC1.GEN was then encrypted using a third implementation of DES,
>a program called MERLIN, using ECB mode and the same hexadecimal key.  The
>frequency results for the resulting file, MC1.MER, are:
> 
>It will be seen that the results are almost the same as for the file
>resulting from encryption of MC1.GEN with The Private Line, namely, the
>same 24 most-frequent bytes occurring in the same order.  (MERLIN also,
>like PCSECURE, does not preserve the size of the file during encryption,
>the difference being 6 bytes.)  There are, however, some minor frequency
>count differences (e.g. byte 22 occurs 276 times in MC1.TPL but only 274
>times in MC1.MER).  Since the ciphertext produced by The Private Line and
>by MERLIN are almost the same it thus seems that the PCSECURE implementation
>of DES departed from the standard.
	So what?  I thought your point was that the DES is not working right,
not that PCSecure is buggy...  Once again, I'm sure that the differences
you noticed between the two similar outputs were due to the choice of 
padding for the last 6 bytes.

> 
>In any case these frequency observations show that in the case of a fairly
>typical file consisting of English text (and these observations can be
>repeated with other text files) a startlingly non-random distribution of
>byte values results from encryption using three different software
>implementations of DES.  This must lead us to have grave doubts concerning
>the claim that DES always, or even usually, randomizes plaintext.
	This post leads me to have grave doubts about the brainpower
at Dolphin Software.  I do not feel that your test file was in any way
'typical', unless you just had a lot of single spaces in the file, but
I doubt that.  Rather, I suspect that Word Perfect turned out quite a few
space-filled lines.  This alone has skewed your results.  And your
cryptic comments about various implementations not "preserving the original
file sizes" give me reason to believe that you are not especially familiar 
with the DES.
> 
>----------------------------------------------------------------------------
>|Dolphin Software publishes MS-DOS programming tools and data encryption   |
>|software which is an alternative to DES.  Information about their products|
>|may be obtained by writing to 48 Shattuck Square #147, Berkeley, CA 94704.|
>----------------------------------------------------------------------------
Aha!
The motive revealed!  First, good effort -- I'll never buy anything you
sell.

Second, and most important to me is the fact that your are more or less
_ADVERTISING_ a for-profit concern on the Internet.  This is ILLEGAL.  When
you gained access, you should have been asked to sign an agreement that
requires you to not transact or solicit business via the InterNet.  This
is taxpayer's money that you and I are using, here.  (For the most part...)

	Right now, I want a response from readers of sci.crypt as to 
whether or not they consider this type of low-key advertising to be against
the policies of the net.  I'll post a summary of what support or derision
I receive in this regard.

-- 
Heath Hunnicutt              | X-Wing Pilot: "Ahhh! I'm gonna die!"
heathh@cobalt.cco.caltech.edu| Rebel Command: "Stay on target..."
131.215.48.30                | ... my life at Caltech
---------------------------------------------------------------------------
