Incorrectly Rounded Subnormal Conversions in Java

While verifying the fix to the Java 2.2250738585072012e-308 bug I found an OpenJDK testcase for verifying conversions of edge case subnormal double-precision numbers. I ran the testcase, expecting it to work — but it failed! I determined it fails because Java converts some subnormal numbers incorrectly.

(By the way, this bug exists in prior versions of Java — it has nothing to do with the fix.)

The OpenJDK Testcase

The OpenJDK testcase is a Java class that tries to verify that a decimal number “just above” or “just below” a subnormal power of two rounds to that subnormal power of two (the subnormal powers of two are 2^-1023 through 2^-1074). Precisely, what it tests are numbers that are within ±2^-1075 of each of those negative powers of two. 2^-1075 is a half a unit in the last place (ULP) — half of 2^-1074, the value of a ULP for all subnormal numbers.

The testcase uses Math.scalb() to generate a double representing each power of two. From each double, it uses the BigDecimal class to generate the decimal strings representing the numbers 1/2 ULP above and 1/2 ULP below the power of two. It then calls Double.parseDouble() to convert each pair of strings, making sure they both convert to the original power of two.

Subnormal double-precision numbers range from 1023 to 1074 decimal places. The BigDecimal strings, which are expressed in scientific notation, represent numbers with 1075 decimal places. They are one decimal place too long, and must be rounded to 1074 places.

In terms of binary, subnormal numbers have 1 to 52 significant bits, unlike normalized numbers, which have 53. The BigDecimal strings, when converted to binary, are 53 significant bits long, with bit 53 always 1; they must be rounded to 52 bits. These are tough halfway cases, that are sometimes rounded incorrectly by Java.

My Testcase

I modified the OpenJDK testcase to generate 1/2 ULP tests for randomly generated subnormal numbers, not just powers of two. I discovered that any decimal number halfway between any two subnormal numbers may be rounded incorrectly.

Examples of Incorrect Conversions

I ran my testcase on a 32-bit Windows XP system, using Java SE 6 — both updates 23 and 24. Below, I’ll show four examples of incorrectly rounded conversions.

Example 1: Half-ULP Above a Power of Two

Java converts this 1075-digit decimal number (315 leading zeros plus 760 significant digits) incorrectly:

6.631236871469758276785396630275967243399099947355303144249971758736286630139265439618068200788048744105960420552601852889715006376325666595539603330361800519107591783233358492337208057849499360899425128640718856616503093444922854759159988160304439909868291973931426625698663157749836252274523485312442358651207051292453083278116143932569727918709786004497872322193856150225415211997283078496319412124640111777216148110752815101775295719811974338451936095907419622417538473679495148632480391435931767981122396703443803335529756003353209830071832230689201383015598792184172909927924176339315507402234836120730914783168400715462440053817592702766213559042115986763819482654128770595766806872783349146967171293949598850675682115696218943412532098591327667236328125E-316

This is the number 2^-1047 + 2^-1075. Shown in unnormalized binary scientific notation, it looks like

0.00000000000000000000000010000000000000000000000000001 x 2^-1022.

(Bit 53, the rounding bit, is highlighted.)

By the round-half-to-even rule, it should round down to 2^-1047, which equals

0.0000000000000000000000001000000000000000000000000000 x 2^-1022.

(I included superfluous trailing zeros to pad everything out to 52 bits so that the alignment is obvious.)

Instead, Java converts it to this number, one ULP above its correctly rounded value:

0.0000000000000000000000001000000000000000000000000001 x 2^-1022.

Example 2: Half-ULP Below a Power of Two

Java converts this 1075-digit decimal number (318 leading zeros plus 757 significant digits) incorrectly:

3.237883913302901289588352412501532174863037669423108059901297049552301970670676565786835742587799557860615776559838283435514391084153169252689190564396459577394618038928365305143463955100356696665629202017331344031730044369360205258345803431471660032699580731300954848363975548690010751530018881758184174569652173110473696022749934638425380623369774736560008997404060967498028389191878963968575439222206416981462690113342524002724385941651051293552601421155333430225237291523843322331326138431477823591142408800030775170625915670728657003151953664260769822494937951845801530895238439819708403389937873241463484205608000027270531106827387907791444918534771598750162812548862768493201518991668028251730299953143924168545708663913273994694463908672332763671875E-319

This is the number 2^-1058 – 2^-1075:

0.00000000000000000000000000000000000011111111111111111 x 2^-1022.

Again, by half-to-even rounding, it should round up to 2^-1058, which equals

0.0000000000000000000000000000000000010000000000000000 x 2^-1022.

Instead, Java converts it to this number, one ULP below its correctly rounded value:

0.0000000000000000000000000000000000001111111111111111 x 2^-1022.

Example 3: Half-ULP Above a Non Power of Two

Java converts this 1075-digit decimal number (309 leading zeros plus 766 significant digits) incorrectly:

6.953355807847677105972805215521891690222119817145950754416205607980030131549636688806115726399441880065386399864028691275539539414652831584795668560082999889551357784961446896042113198284213107935110217162654939802416034676213829409720583759540476786936413816541621287843248433202369209916612249676005573022703244799714622116542188837770376022371172079559125853382801396219552418839469770514904192657627060319372847562301074140442660237844114174497210955449896389180395827191602886654488182452409583981389442783377001505462015745017848754574668342161759496661766020028752888783387074850773192997102997936619876226688096314989645766000479009083731736585750335262099860150896718774401964796827166283225641992040747894382698751809812609536720628966577351093292236328125E-310

This is the number (2^-1027 + 2^-1066) + 2^-1075:

0.00001000000000000000000000000000000000000001000000001 x 2^-1022.

It should round down, converting to 2^-1027 + 2^-1066, which equals

0.0000100000000000000000000000000000000000000100000000 x 2^-1022.

Instead, Java converts it to this number, one ULP above its correctly rounded value:

0.0000100000000000000000000000000000000000000100000001 x 2^-1022.

Example 4: Half-ULP Below a Non Power of Two

Java converts this 1075-digit decimal number (318 leading zeros plus 757 significant digits) incorrectly:

3.339068557571188581835713701280943911923401916998521771655656997328440314559615318168849149074662609099998113009465566426808170378434065722991659642619467706034884424989741080790766778456332168200464651593995817371782125010668346652995912233993254584461125868481633343674905074271064409763090708017856584019776878812425312008812326260363035474811532236853359905334625575404216060622858633280744301892470300555678734689978476870369853549413277156622170245846166991655321535529623870646888786637528995592800436177901746286272273374471701452991433047257863864601424252024791567368195056077320885329384322332391564645264143400798619665040608077549162173963649264049738362290606875883456826586710961041737908872035803481241600376705491726170293986797332763671875E-319

This is the number (2^-1058 + 2^-1063) – 2^-1075:

0.00000000000000000000000000000000000100000111111111111 x 2^-1022.

It should round up, converting to 2^-1058 + 2^-1063, which equals

0.0000000000000000000000000000000000010000100000000000 x 2^-1022.

Instead, Java converts it to this number, one ULP below its correctly rounded value:

0.0000000000000000000000000000000000010000011111111111 x 2^-1022.

Bug Report

I wrote a Java bug report for this problem: Bug ID 7019078: Double.parseDouble() converts some subnormal numbers incorrectly.

A Related Existing Bug

Java Bug 4396272 reports an error in the conversion of 2^-1075: it should round to zero (by half-to-even rounding), but instead rounds to 2^-1074.

Discussion

All of the incorrect conversions I found were off by one ULP. This is consistent with other languages — like Visual C++ and GCC C — that don’t use David Gay’s correctly rounding strtod() function. However, Java’s FloatingDecimal class is clearly modeled on David Gay’s code, so I assume it was Java’s intent to do all its conversions correctly.