Pardon The Decimal Dust

by The Metric Maven

Early in my time as an undergraduate at a Midwestern university, I discovered that having “too many decimal places” on my homework was a terrible and ignorant trespass on rationality and technical competence. I was told that the decimal values in my calculation, that were only a few places past the decimal marker, were meaningless. As a graduate student, my TA had the “experience” to understand this problem, and the importance of properly rounding numbers. What I’ve generally settled on these days, is to use three places past a decimal point so that I can easily change a metric number to an integer expression, should I want to change the chosen metric prefix. I also have enough years behind me to realize that whatever number of places were chosen by my TA, they were not the product of an error analysis, they were the product of personal preference.

I’ve heard critics of “too many decimal places” call values they believe are insignificant “decimal dust.” There is no clear definition of this rubric. It has been defined as:


DECIMAL DUST: An inconsequential numerical amount.

What I’ve come to realize, is that a sort of meaningless bifurcated “intellectual” tug-of-war occurs between those who are concerned about “the excessive use of decimals,” and those who see “too much precision.” Historians have noticed that Newton would compute answers out to an excessively unnecessary number of decimal places. Their conclusion is that “he just liked doing the calculation.” This is very probably true. Quite possibly the best engineer I’ve ever shared an office with was Michel. He was able to take the abstract equations of electromagnetism and turn them into useful computer code. We were, of course, banned from presenting any results that interfaced with the Aerospace world in metric, but inside of every computer was, to my knowledge, metric computations hidden away in ones and zeros, so as not to offend delicate Olde English sensibilities.

I was fascinated with understanding the details of how Michel implemented his computer code and verified it. What I noticed immediately was that Michel would hand compute each line of code, and compare it with the computer code’s output at each line. I was surprised that he was carrying the hand calculation out to perhaps ten decimal places! I had always harbored a secret desire to compute out to that many places. It gave me some strange reassurance that my code was right when I did, despite the admonishment I’d received at my University against creating and propagating meaningless “garbage numbers.”

One day I could clearly see that something was bothering Michel, and I just had to know what it was, as what he worked on was usually very interesting. He had some computer code he had not written, that had been used in-house for sometime. He was told to use it to predict the outcome of a measurement. Michel had derived a formula that should have been equivalent to what was implemented in the computer code, but about four or five decimal places out, the values were different. Michel showed me the hand calculation (he checked it three times) and the computer output. In his French accent he said “They should be the same, should they not?” I agreed. We checked the value of physical constants, such as the speed of light. They were all the same. Finally Michel saw the problem, and it was in the code. At certain extremes it would introduce a considerable error into the computation. That was the day I began to always check my hand and computer code computations to at least 5-6 places, minimum. I would learn that one man’s decimal dust is another man’s gold dust.

Indeed, decimal dust can be a source of new scientific knowledge. In the 1960s, Edward Lorenz (1917-2008) had noticed a very interesting output from a non-linear mathematical computer model he was using. He wanted to repeat the computation and input the initial conditions by hand as a short-cut. Lorenz rounded the original input value of 0.506127 to 0.506, a number of decimal places expected to be insignificant, and plenty accurate. When he ran it again, the computation output was nothing like the previous computation after a short period. Changing the input value at the level of “decimal dust” was expected to have no effect on the computation, clearly for the mostly non-linear world we live in, this is not the case. It was Lorenz that coined the now ubiquitous term “the butterfly effect” for sensitivity to initial conditions, and ushered along the science of chaos theory into what it is today. The tiny pressure changes caused from a butterfly flapping its wings in Africa, has the potential to be the seed for a hurricane in the Atlantic ocean. There are cases where non-linear deterministic equations need an infinite number of decimal places for a computation to repeat over all time.

In the early 1980s, British scientists using ground-based measurements reported a large ozone hole had appeared above Antarctica. This was quite surprising as satellite data had not noted the same problem. The computer code for the satellite had “data quality control algorithms” that rejected such values as “unreasonable.” Assumptions about what values are important, and those that are not, are assumptions, and should be understood as such. Another example is “filtered viruses.” It was assumed viruses had to be smaller than a certain dimension, so all other microbes above that size were removed with filter paper. It took decades for researchers to realize that monster size viruses exist. I’ve written about this in my essay Don’t Assume What You Don’t Know.

The a priori assumption of what is important is used as a rhetorical cudgel to suppress “excessive” information. When I’ve argued that human height should be represented in meters or millimeters (preferably millimeters), there is a vast outcry that only the traditional atavistic pseudo-inch known as the centimeter should be used. To use millimeters is, harrumph!, “too much precision.” It is also a possible lost opportunity for researchers as information has been suppressed from the introduction of a capricious assumption. One can always round the offending values down, but obtaining better precision after-the-fact is not an option. In my view, those who use the term decimal dust in a manner other than as a metaphor for tiny, are lazy in their criticisms, and assume they know how many decimal places matter without any familiarity with subject and the values involved.

When long and thoughtful effort is expended, one can introduce the astonishing simplification of using integers, which eschew decimal points entirely. As has been pointed out ad nausium in this blog, using millimeters for housing construction is a measurement environment that is partitioned in a way that allows for this incredible simplification. Pat Naughtin noted that integer values in grams should be used for measuring babies. This produces an intuitive understanding of the amount of weight that a baby gains or loses compared with Kilograms. Grams, millimeters and milliliters are efficient for everyday use. Integer values are the most instinctual numerals for comparison tables. The metric system is beautiful in its ability to provide the most intuitive ways of expressing the values of nature. It is up to us to use it wisely and thoughtfully, instead of dogmatically. In my view, this measurement introspection is sorely lacking in our modern society, and definitely in the community of science writers and researchers.

If you liked this essay and wish to support the work of The Metric Maven, please visit his Patreon Page.

Pushing The Envelope

Guest Post

By James W. Way

Besides our slow adoption of the metric system, the United States differs from the rest of the industrialized world in another way.  We use US Letter (8½ × 11 inch paper), while the vast majority of other nations use A4, a paper size created in Germany in the 1920’s.  Converting to inches, the dimensions for A4 are approximately 8¼ × 1111/16 (US Letter is slightly wider, but not as tall).  A4 is officially defined as 210 mm × 297 mm; after converting to millimeters, US Letter is 216 mm × 279 mm.

The U. S. has a national standard for metric paper, ANSI/ASME Y14.1M, which gives identical dimensions for the A sizes, but does not go smaller than A4.  There exists an A5 size, common for notepads, with A6 used for postcards (as the numbers get larger the sizes become smaller).  A4 is part of a whole series of A and B paper sizes defined by ISO 216 (the International Organization for Standardization).  I will summarize the advantages of these metric sizes, even though the Metric Maven has written on this topic before (see The Metric Paper Tiger from 2014-02-10).

Two sheets of 8½ × 11 inch paper equal one 11 × 17.  Enlarging an image from the former onto the latter, however, results in different margins.

Combining two sheets of A4 side by side equals one A3.  Also, the margins will be correct when enlarging an image from A4 onto A3.  Why is this so?  Because metric sizes use the only height to width ratio where this will work:  H = W × √2 (height = width × the square root of two, or 1.4142).

After the French Revolution, some sizes with this aspect ratio were created, but never became widely known.  In 1911, an institute called Die Brücke (The Bridge) was founded in Munich, which attempted to standardize paper formats.  Sixteen sizes were created, for everything from postage stamps to books:  size I – 1 cm × 1.41 cm, size II – 1.41 cm × 2 cm, etc.

Die Brücke only lasted a few years before going bankrupt.  After World War I, a former associate named Dr. Walter Porstmann improved on the original concept, numbering the sizes in the opposite direction.  A0 is a sheet with an area of one square meter (but a 1: √2 aspect ratio).  Dividing this in half results in two A1 sheets, and so on.  Thus, A4 is one sixteenth of a square meter; if the listed weight on a ream of paper is 80 g/m2, one A4 sheet is 5 g.

In 1922, these sizes became a DIN standard (Deutsches Institut für Normung, or the German Institute for Standardization) and gradually spread throughout the world.

An unfolded A4 sheet fits in a corresponding C4 envelope.  The B sizes are mostly used for books, and don’t have their own envelopes, as shown below.

Another resource for information on ISO paper and envelopes is Markus Kuhn’s excellent webpage, International Standard Paper Sizes.  He writes:

The ISO paper sizes are based on the metric system.  The square-root-of-two ratio does not permit both the height and width of the pages to be nicely rounded metric lengths.

This can be seen in the figure above.  With A and B paper, at least one side is a nice metric length, but this is not the case with C envelopes.

Envelope Name                Common Use                     Dimensions (mm)

C4A4 unfolded229 × 324
C5A4 folded in half162 × 229
C6A5 folded in half114 × 162

However, in a strange coincidence, the three most common C sizes do convert nicely to inches:

Name      Inches                          Exact Conversion            Rounded to mm

C49 × 12 ¾228.6 × 323.85229 × 324
C56 ⅜ × 9161.925 × 228.6162 × 229
C64 ½ × 6 ⅜114.3 × 161.925114 × 162

A manufacturer would be justified in labeling these by their correct metric dimensions even if they used inches in-house.  The US already has 9 × 12 and 6 × 9 envelopes; making C4 and C5 would not pose much of a problem.

Of course, a letter is most commonly folded in thirds and sent in a business envelope.  How big is an A4 sheet when folded in exact thirds?  The height is 297 mm, so 297 ÷ 3 = 99, with a width of 210 mm.

The most common metric commercial envelope is DL (110 mm × 220 mm); there is 11 mm of room above if folded in perfect thirds.  DL originally stood for DIN Lang (DIN length); it is separate from the A, B, and C sizes.

Now, how much extra room is there for US Letter in a #10 envelope?  First, let’s divide 11 inches by 3 and …if you’re a teacher, watch your students fumble around with this one!  Of course, things become much easier when inches are converted to millimeters.

US Letter is 279 mm high, so 279 ÷ 3 = 93.  The height of #10 is 4⅛ inches:  4.125 × 25.4 = 104.775 mm.  After rounding this up:  105 – 93 = 12 mm.

The two sizes leave a similar amount of extra space relative to the size of paper that is being used.

When sending a business letter, however, most people do not fold it in exact thirds.  There will be some space between the top (folded down) edge of the letter and the bottom crease.  If you have difficulty estimating this by sight, the following method is a good option.

Put the bottom of an A4 letter against the inside flap of a DL envelope, as shown in the photo below.

The section not resting on the envelope is 187 mm high (297 – 110 = 187).  Fold the top third of the page (at the bottom in the photo) up to the lower edge of the envelope.  To calculate the size of this fold:  187 ÷ 2 = 93.5 mm.

What is the height of the remaining two thirds of the paper when we rotate it right side up?  297 – 93.5 = 203.5 mm.  Now, fold the bottom of the page up to a few millimeters below the first fold, and the letter will fit nicely.  A similar method will work for US Letter inside #10.

DL leaves only 5 mm of room on either side for an A4 sheet – quite narrow for automatic insertion.  Since US Letter is 216 mm wide, DL (at 220 mm) cannot be used as an envelope for both.

ISO 269 (Correspondence envelopes – Designation and sizes) contains the following note about Universal Postal Union regulations:

When processing size A4 documents in inserting machines, the size of DL envelopes may be insufficient.  To satisfy the needs for automatic insertion, an envelope size larger than DL may be used as long as the size can be considered standardized according to UPU regulations.  (Upper limit is at present 120 mm × 235 mm.)

In Australia, the upper limit above corresponds exactly to the DLX size, allowing DL to fit inside as a reply envelope.  The “X” probably stands for maximum, but also brings to mind “XL” as an abbreviation for extra-large, (even though the two letters are reversed).

Between the two is an intermediate DLE size (114 mm × 225 mm) that also fits inside DLX, though the smaller DL is used with automatic machines.  DLE converts to inches quite easily.

Inches                                 Exact Conversion               Rounded to mm

4 ½ × 8 ⅞114.3 × 225.425114 × 225

The above dimensions, when compared with American envelopes, are equal to the short side of #11 and the long side of #9.  A DLE envelope can be used for both US Letter and A4, being slightly wider than DL.

Yes, these conversion tables conflict with Pat Naughtin’s philosophy of “don’t duel with dual.”  But it helps U.S. manufacturers to know that some international sizes have similar dimensions to what they already produce.

Here are some other sizes worth mentioning.

In Germany, C6/5 is popular, using the short side of C6 (114 mm) and the long side of C5 (229 mm).  The U.K. prefers to name this size DL+.  Italians use an envelope 10 mm wider than DL (110 mm × 230 mm).  These are each fine by themselves, but can’t work together as a reply/outer envelope like the three Australian sizes.

Statistics compiled by the Envelope Manufacturers Association (EMA) show that U. S. sales peaked in 2005.  In our electronic age, this market has declined, with total sales now similar to the mid 1980’s.  Here are some places to buy in the U. S., if you are so inclined.

ISO envelopes are sold by, but only the most common sizes:  C4, C5, C6, DL, as well as a variety of metric paper.

Another excellent resource is Empire Imports.  While they do not sell envelopes, they specialize in metric paper and related products, stocking items such as hole punches, folders, binders, etc.

Finally, some fountain pen dealers stock a limited number of metric sizes, since these types of pens work best with high quality European and Japanese stationery.  A good example is The Goulet Pen Company.

These are my personal observations; I have no financial relationship with any of these sellers.

If you would like to support the work of The Metric Maven, please visit his Patreon Page.