In my article “Floating-Point Questions Are Endless on stackoverflow.com” I showed examples of the many questions asked that demonstrate lack of knowledge of the most basic property of floating-point — that not all decimal values are representable in binary. In response to a reader’s comment on my article I wrote:
It would be interesting to know how it’s taught today (it’s been a very long time since I was taught it). I can’t imagine though that the person teaching it wouldn’t say — within a sentence or two of saying “floating-point” — that it “can’t represent all decimal numbers accurately”.
That prompted me to look through my box of thirty plus year old college (undergraduate) notebooks. I found notebooks for four classes in which I was taught floating-point. The notes from three of those classes confirm what I thought — that we were warned early of the decimal/binary mismatch. But in the first class of the four — the beginner’s class — it’s less clear what we were told. I’ll show you images of the relevant excerpts from my notes. (I notice I had some elements of cursive in my handwriting back then.)
“Computer Applications” Class
The first class I used floating-point in was called “Computer Applications”. (Don’t be fooled by the name; we programmed mathematical formulas — in APL!) In my notebook, I didn’t find the explicit warnings I was looking for — just hints a few weeks into the semester. For example:
The highlighted excerpt says
“will round off in printing…only in printing, not internally”
That is a clue, but not hard evidence. In the section of notes that defines the floating-point format, I found no warning. On the other hand, this class was taught by the same professor that taught my “Seminumerical Algorithms” class (see below), and we were warned there. In any case, it’s hard to imagine programming in a mathematical language like APL and not being told this.
“Numerical Methods” Class
In my “Numerical Methods” class, we were warned on the second day of class:
Note the highlighted excerpt:
“* decimal #’s terminating in one base may not terminate in another”
“Electronic Computers” Class
In my “Electronic Computers” class, which was about computer architecture, we were warned on the second day of class:
The highlighted excerpt implies that all integers can be represented exactly in binary, but that only some fractional values can be. (From notes on a prior page, m is defined as the number of fractional digits.)
“Seminumerical Algorithms” Class
In my “Seminumerical Algorithms” class, which was based on Knuth’s book of the same name, we were warned in the second lecture on floating-point:
The diagram says it all: the round-trip through binary may change the decimal value.
I especially like this note:
“for banking … work with integers”
That advice alone could answer many a question on Stack Overflow.
Conclusion
So did I know about this basic fact of binary floating-point on day one, or would I have been asking the same type of question on Stack Overflow — had it existed in 1982? I think I knew, but I guess I can’t really be sure.
