Anomalies In IntelliJ Kotlin Floating-Point Literal Inspection

IntelliJ IDEA has a code inspection for Kotlin that will warn you if a decimal floating-point literal exceeds the precision of its type (Float or Double). It will suggest an equivalent literal (one that maps to the same binary floating-point number) that has fewer digits, or has the same number of digits but is closer to the floating-point number.

Screenshot in IntelliJ IDEA of hovering over a flagged 17-digit literal with a suggested 10-digit replacement
Hovering over a flagged 17-digit literal suggests a 10-digit replacement.

For Doubles for example, every literal over 17-digits should be flagged, since it never takes more than 17 digits to specify any double-precision binary floating-point value. Literals with 16 or 17 digits should be flagged if there is a replacement that is shorter or closer. And no literal with 15 digits or fewer should ever be flagged, since doubles have of 15-digits of precision.

But IntelliJ doesn’t always adhere to that, like when it suggests an 18-digit replacement for a 13-digit literal!

Screenshot of IntelliJ IDEA suggesting an 18-digit replacement for a 13-digit literal
An 18-digit replacement suggested for a 13-digit literal!

Continue reading “Anomalies In IntelliJ Kotlin Floating-Point Literal Inspection”

Another NaN In the Wild

I see these from time to time, but I don’t always capture them; here’s one I saw recently while playing a podcast:

A NaN in an ad in the app Castbox (partial image)
A NaN in an ad in the app Castbox (click for full image).

(According to Castbox, this is an error in the ad and is out of their control.)

Visual C++ strtod(): Still Broken

About a year ago Bruce Dawson informed me that Microsoft is fixing their decimal to floating-point conversion routine in the next release of Visual Studio; I finally made the time to test the new code. I installed Visual Studio Community 2015 Release Candidate and ran my old C++ testcases. The good news: all of the individual conversion errors that I wrote about are fixed. The bad news: many errors remain.

Continue reading “Visual C++ strtod(): Still Broken”

PHP Converts 2.2250738585072012e-308 Incorrectly

While testing my new decimal to floating-point converter I discovered a bug in old territory: PHP incorrectly converts the number 2.2250738585072012e-308.

<?php printf("%.17g",2.2250738585072012e-308); ?>

This prints 2.2250738585072009E-308; it should print 2.2250738585072014e-308. (I verified that the internal double value is wrong; the printed value correctly represents it.)

Continue reading “PHP Converts 2.2250738585072012e-308 Incorrectly”

Gay’s strtod() Returns Zero For Inputs Just Above 2^-1075

While running some of GCC’s string to double conversion testcases I discovered a bug in David Gay’s strtod(): it converts some very small subnormal numbers incorrectly. Unlike numbers 2-1075 or smaller, which should convert to zero under round-to-nearest/ties-to-even rounding, numbers between 2-1075 and 2-1074 should convert to 2-1074, the smallest number representable in double-precision binary floating-point. strtod() correctly converts the former to 0, but it incorrectly converts the latter to 0 as well.

(Update 11/25/13: This bug has been fixed.)

Continue reading “Gay’s strtod() Returns Zero For Inputs Just Above 2^-1075”

GLIBC strtod() Incorrectly Converts 2^-1075

A reader of my blog, Water Qian, reported a bug to me after reading my article “How GLIBC’s strtod() Works”. I recently tested strtod(), which was was fixed to do correct rounding in glibc 2.17; I had found no incorrect conversions.

Water tested the conversion of 2-1075 — in retrospect an obvious corner case I should have tried — and found that it converted incorrectly to 0x0.0000000000001p-1022. That’s 2-1074, the smallest double-precision value. It should have converted to 0, under round-to-nearest/ties-to-even rounding.

(Update 11/13/13: This bug has been fixed for version 2.19.)

Continue reading “GLIBC strtod() Incorrectly Converts 2^-1075”

GCC Conversions Are Incorrect, Architecture or Otherwise

Recently I wrote about my retesting of the gcc C compiler’s string to double conversions and how it appeared that its incorrect conversions were due to an architecture-dependent bug. My examples converted incorrectly on 32-bit systems, but worked on 64-bit systems — at least most of them. I decided to dig into gcc’s source code and trace its execution, and I found the architecture dependency I was looking for. But I found more than that: due to limited precision, gcc will do incorrect conversions on any system. I’ve constructed an example to demonstrate this.

Continue reading “GCC Conversions Are Incorrect, Architecture or Otherwise”

Incorrect Round-Trip Conversions in Visual C++

Paul Bristow, a Boost.Math library author and reader of my blog, recently alerted me to a problem he discovered many years ago in Visual C++: some double-precision floating-point values fail to round-trip through a stringstream as a 17-digit decimal string. Interestingly, the 17-digit strings that C++ generates are not the problem; they are correctly rounded. The problem is that the conversion of those strings to floating-point is sometimes incorrect, off by one binary ULP.

I’ve previously discovered that Visual Studio makes incorrect decimal to floating-point conversions, and that Microsoft is OK with it — at least based on their response to my now deleted bug report. But incorrect decimal to floating-point conversions in this context seems like a problem that needs fixing. When you serialize a double to a 17-digit decimal string, shouldn’t you get the same double back later? Apparently Microsoft doesn’t think so, because Paul’s bug report has also been deleted.

Continue reading “Incorrect Round-Trip Conversions in Visual C++”

A Bug in the Bigcomp Function of David Gay’s strtod()

Last week, a reader of my blog, Geza Herman, told me about a bug he found in David Gay’s strtod() function. In random testing of decimal numbers nearly halfway between double-precision floating-point numbers, he discovered this 53-digit number, which converts incorrectly:

1.8254370818746402660437411213933955878019332885742187

As Geza noted, the problem is in the bigcomp() function, an optimization that kicks in for long decimal inputs. I traced his example through bigcomp() — I’ll show you what’s going on.

Continue reading “A Bug in the Bigcomp Function of David Gay’s strtod()”

A Better Fix for the PHP 2.2250738585072011e-308 Bug

Recently I discovered a bug in PHP’s decimal to floating-point conversion routine, zend_strtod(): it went into an infinite loop trying to convert the decimal string 2.2250738585072011e-308 to floating-point. zend_strtod() is based on David Gay’s strtod() function in dtoa.c, as are the decimal to floating-point conversion routines of many other open source projects. So why hasn’t this bug affected these other projects?

zend_strtod() is based on a very old copy of dtoa.c. The current version of dtoa.c is immune to the 2.2250738585072011e-308 bug — and has been since 1997 by my reckoning. So while the ‘volatile’ keyword fixes the PHP problem, I think there’s a better solution: upgrade zend_strtod() to the latest dtoa.c.

Continue reading “A Better Fix for the PHP 2.2250738585072011e-308 Bug”

Copyright © 2008-2023 Exploring Binary

Privacy policy

Powered by WordPress

css.php