Incorrectly Rounded Subnormal Conversions in Java

While verifying the fix to the Java 2.2250738585072012e-308 bug I found an OpenJDK testcase for verifying conversions of edge case subnormal double-precision numbers. I ran the testcase, expecting it to work — but it failed! I determined it fails because Java converts some subnormal numbers incorrectly.

(By the way, this bug exists in prior versions of Java — it has nothing to do with the fix.)

The OpenJDK Testcase

The OpenJDK testcase is a Java class that tries to verify that a decimal number “just above” or “just below” a subnormal power of two rounds to that subnormal power of two (the subnormal powers of two are 2^-1023 through 2^-1074). Precisely, what it tests are numbers that are within ±2^-1075 of each of those negative powers of two. 2^-1075 is a half a unit in the last place (ULP) — half of 2^-1074, the value of a ULP for all subnormal numbers.

The testcase uses Math.scalb() to generate a double representing each power of two. From each double, it uses the BigDecimal class to generate the decimal strings representing the numbers 1/2 ULP above and 1/2 ULP below the power of two. It then calls Double.parseDouble() to convert each pair of strings, making sure they both convert to the original power of two.

Subnormal double-precision numbers range from 1023 to 1074 decimal places. The BigDecimal strings, which are expressed in scientific notation, represent numbers with 1075 decimal places. They are one decimal place too long, and must be rounded to 1074 places.

In terms of binary, subnormal numbers have 1 to 52 significant bits, unlike normalized numbers, which have 53. The BigDecimal strings, when converted to binary, are 53 significant bits long, with bit 53 always 1; they must be rounded to 52 bits. These are tough halfway cases, that are sometimes rounded incorrectly by Java.

My Testcase

I modified the OpenJDK testcase to generate 1/2 ULP tests for randomly generated subnormal numbers, not just powers of two. I discovered that any decimal number halfway between any two subnormal numbers may be rounded incorrectly.

Examples of Incorrect Conversions

I ran my testcase on a 32-bit Windows XP system, using Java SE 6 — both updates 23 and 24. Below, I’ll show four examples of incorrectly rounded conversions.

Example 1: Half-ULP Above a Power of Two

Java converts this 1075-digit decimal number (315 leading zeros plus 760 significant digits) incorrectly:

6.6312368714697582767853966302759672433990999473553031442499717587
362866301392654396180682007880487441059604205526018528897150063763
256665955396033303618005191075917832333584923372080578494993608994
251286407188566165030934449228547591599881603044399098682919739314
266256986631577498362522745234853124423586512070512924530832781161
439325697279187097860044978723221938561502254152119972830784963194
121246401117772161481107528151017752957198119743384519360959074196
224175384736794951486324803914359317679811223967034438033355297560
033532098300718322306892013830155987921841729099279241763393155074
022348361207309147831684007154624400538175927027662135590421159867
638194826541287705957668068727833491469671712939495988506756821156
96218943412532098591327667236328125E-316

This is the number 2^-1047 + 2^-1075. Shown in unnormalized binary scientific notation, it looks like

0.00000000000000000000000010000000000000000000000000001 x 2^-1022.

(Bit 53, the rounding bit, is highlighted.)

By the round-half-to-even rule, it should round down to 2^-1047, which equals

0.0000000000000000000000001000000000000000000000000000 x 2^-1022.

(I included superfluous trailing zeros to pad everything out to 52 bits so that the alignment is obvious.)

Instead, Java converts it to this number, one ULP above its correctly rounded value:

0.0000000000000000000000001000000000000000000000000001 x 2^-1022.

Example 2: Half-ULP Below a Power of Two

Java converts this 1075-digit decimal number (318 leading zeros plus 757 significant digits) incorrectly:

3.2378839133029012895883524125015321748630376694231080599012970495
523019706706765657868357425877995578606157765598382834355143910841
531692526891905643964595773946180389283653051434639551003566966656
292020173313440317300443693602052583458034314716600326995807313009
548483639755486900107515300188817581841745696521731104736960227499
346384253806233697747365600089974040609674980283891918789639685754
392222064169814626901133425240027243859416510512935526014211553334
302252372915238433223313261384314778235911424088000307751706259156
707286570031519536642607698224949379518458015308952384398197084033
899378732414634842056080000272705311068273879077914449185347715987
501628125488627684932015189916680282517302999531439241685457086639
13273994694463908672332763671875E-319

This is the number 2^-1058 – 2^-1075:

0.00000000000000000000000000000000000011111111111111111 x 2^-1022.

Again, by half-to-even rounding, it should round up to 2^-1058, which equals

0.0000000000000000000000000000000000010000000000000000 x 2^-1022.

Instead, Java converts it to this number, one ULP below its correctly rounded value:

0.0000000000000000000000000000000000001111111111111111 x 2^-1022.

Example 3: Half-ULP Above a Non Power of Two

Java converts this 1075-digit decimal number (309 leading zeros plus 766 significant digits) incorrectly:

6.9533558078476771059728052155218916902221198171459507544162056079
800301315496366888061157263994418800653863998640286912755395394146
528315847956685600829998895513577849614468960421131982842131079351
102171626549398024160346762138294097205837595404767869364138165416
212878432484332023692099166122496760055730227032447997146221165421
888377703760223711720795591258533828013962195524188394697705149041
926576270603193728475623010741404426602378441141744972109554498963
891803958271916028866544881824524095839813894427833770015054620157
450178487545746683421617594966617660200287528887833870748507731929
971029979366198762266880963149896457660004790090837317365857503352
620998601508967187744019647968271662832256419920407478943826987518
09812609536720628966577351093292236328125E-310

This is the number (2^-1027 + 2^-1066) + 2^-1075:

0.00001000000000000000000000000000000000000001000000001 x 2^-1022.

It should round down, converting to 2^-1027 + 2^-1066, which equals

0.0000100000000000000000000000000000000000000100000000 x 2^-1022.

Instead, Java converts it to this number, one ULP above its correctly rounded value:

0.0000100000000000000000000000000000000000000100000001 x 2^-1022.

Example 4: Half-ULP Below a Non Power of Two

Java converts this 1075-digit decimal number (318 leading zeros plus 757 significant digits) incorrectly:

3.3390685575711885818357137012809439119234019169985217716556569973
284403145596153181688491490746626090999981130094655664268081703784
340657229916596426194677060348844249897410807907667784563321682004
646515939958173717821250106683466529959122339932545844611258684816
333436749050742710644097630907080178565840197768788124253120088123
262603630354748115322368533599053346255754042160606228586332807443
018924703005556787346899784768703698535494132771566221702458461669
916553215355296238706468887866375289955928004361779017462862722733
744717014529914330472578638646014242520247915673681950560773208853
293843223323915646452641434007986196650406080775491621739636492640
497383622906068758834568265867109610417379088720358034812416003767
05491726170293986797332763671875E-319

This is the number (2^-1058 + 2^-1063) – 2^-1075:

0.00000000000000000000000000000000000100000111111111111 x 2^-1022.

It should round up, converting to 2^-1058 + 2^-1063, which equals

0.0000000000000000000000000000000000010000100000000000 x 2^-1022.

Instead, Java converts it to this number, one ULP below its correctly rounded value:

0.0000000000000000000000000000000000010000011111111111 x 2^-1022.

Bug Report

I wrote a Java bug report for this problem: Bug ID 7019078: Double.parseDouble() converts some subnormal numbers incorrectly.

A Related Existing Bug

Java Bug 4396272 reports an error in the conversion of 2^-1075: it should round to zero (by half-to-even rounding), but instead rounds to 2^-1074.

Discussion

All of the incorrect conversions I found were off by one ULP. This is consistent with other languages — like Visual C++ and GCC C — that don’t use David Gay’s correctly rounding strtod() function. However, Java’s FloatingDecimal class is clearly modeled on David Gay’s code, so I assume it was Java’s intent to do all its conversions correctly.