College Notebook: When I Was Taught Floating-Point

In my article “Floating-Point Questions Are Endless on stackoverflow.com” I showed examples of the many questions asked that demonstrate lack of knowledge of the most basic property of floating-point — that not all decimal values are representable in binary. In response to a reader’s comment on my article I wrote:

It would be interesting to know how it’s taught today (it’s been a very long time since I was taught it). I can’t imagine though that the person teaching it wouldn’t say — within a sentence or two of saying “floating-point” — that it “can’t represent all decimal numbers accurately”.

That prompted me to look through my box of thirty plus year old college (undergraduate) notebooks. I found notebooks for four classes in which I was taught floating-point. The notes from three of those classes confirm what I thought — that we were warned early of the decimal/binary mismatch. But in the first class of the four — the beginner’s class — it’s less clear what we were told. I’ll show you images of the relevant excerpts from my notes. (I notice I had some elements of cursive in my handwriting back then.)

“Computer Applications” Class

The first class I used floating-point in was called “Computer Applications”. (Don’t be fooled by the name; we programmed mathematical formulas — in APL!) In my notebook, I didn’t find the explicit warnings I was looking for — just hints a few weeks into the semester. For example:

An excerpt from my “Computer Applications” class notebook (January 26, 1982)
An excerpt from my “Computer Applications” class notebook (January 26, 1982)

The highlighted excerpt says

“will round off in printing…only in printing, not internally”

That is a clue, but not hard evidence. In the section of notes that defines the floating-point format, I found no warning. On the other hand, this class was taught by the same professor that taught my “Seminumerical Algorithms” class (see below), and we were warned there. In any case, it’s hard to imagine programming in a mathematical language like APL and not being told this.

“Numerical Methods” Class

In my “Numerical Methods” class, we were warned on the second day of class:

An excerpt from my “Numerical Methods” class notebook (January 18, 1983)
An excerpt from my “Numerical Methods” class notebook (January 18, 1983)

Note the highlighted excerpt:

“* decimal #’s terminating in one base may not terminate in another”

“Electronic Computers” Class

In my “Electronic Computers” class, which was about computer architecture, we were warned on the second day of class:

An excerpt from my “Electronic Computers” class notebook (September 12, 1983)
An excerpt from my “Electronic Computers” class notebook (September 12, 1983)

The highlighted excerpt implies that all integers can be represented exactly in binary, but that only some fractional values can be. (From notes on a prior page, m is defined as the number of fractional digits.)

“Seminumerical Algorithms” Class

In my “Seminumerical Algorithms” class, which was based on Knuth’s book of the same name, we were warned in the second lecture on floating-point:

An excerpt from my “Seminumerical Algorithms” class notebook (October 25, 1984)
An excerpt from my “Seminumerical Algorithms” class notebook (October 25, 1984)

The diagram says it all: the round-trip through binary may change the decimal value.

I especially like this note:

“for banking … work with integers”

That advice alone could answer many a question on Stack Overflow.

Conclusion

So did I know about this basic fact of binary floating-point on day one, or would I have been asking the same type of question on Stack Overflow — had it existed in 1982? I think I knew, but I guess I can’t really be sure.

Dingbat
RSS feed icon
RSS e-mail icon

Leave a Reply

Your email address will not be published. Required fields are marked *

(Cookies must be enabled to leave a comment...it reduces spam.)