From 31650fd46556268d8bba93d0f41835306a065517 Mon Sep 17 00:00:00 2001 From: John Scipione Date: Wed, 26 Jun 2013 18:55:48 -0400 Subject: [PATCH] DeskCalc: Limit precision of large magnitude results This targets a problem where a numbers with large numbers of non-decimal significant digits took a long time to round after converting to scientific notation because they are rounded one character a time. To solve this, after converting to scientific notation lop off everything after 40 characters greatly reducing the amount of further rounding needed. An example I used to test this was to calculate 10,000! which gives a result with 35660 significant non-decimal digits (aka a lot). By loping off numbers after 40 characters before rounding to fit the operation goes from ~10 seconds to complete to under a second. I chose 40 as a max as it is large enough to ensure that the result will get rounded with some leeway provided for font width variations. Worse-case scenario is the result is off by 1 in the last place. Numbers with large numbers of significant decimal digits get rounded by MAPM so aren't a problem. --- src/apps/deskcalc/ExpressionTextView.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/apps/deskcalc/ExpressionTextView.cpp b/src/apps/deskcalc/ExpressionTextView.cpp index ae27e55553..f100efd074 100644 --- a/src/apps/deskcalc/ExpressionTextView.cpp +++ b/src/apps/deskcalc/ExpressionTextView.cpp @@ -244,10 +244,15 @@ ExpressionTextView::SetValue(BString value) } } - // add the exponent - offset = value.CountChars() - 1; - if (exponent != 0) + if (exponent != 0) { + value.Truncate(40); + // truncate to a reasonable precision + // while ensuring result will be rounded + offset = value.CountChars() - 1; value << "E" << exponent; + // add the exponent + } else + offset = value.CountChars() - 1; // reduce the number of digits until the string fits or can not be // made any shorter