DeskCalc: Limit precision of large magnitude results
This targets a problem where a numbers with large numbers of non-decimal significant digits took a long time to round after converting to scientific notation because they are rounded one character a time. To solve this, after converting to scientific notation lop off everything after 40 characters greatly reducing the amount of further rounding needed. An example I used to test this was to calculate 10,000! which gives a result with 35660 significant non-decimal digits (aka a lot). By loping off numbers after 40 characters before rounding to fit the operation goes from ~10 seconds to complete to under a second. I chose 40 as a max as it is large enough to ensure that the result will get rounded with some leeway provided for font width variations. Worse-case scenario is the result is off by 1 in the last place. Numbers with large numbers of significant decimal digits get rounded by MAPM so aren't a problem.
This commit is contained in:
@@ -244,10 +244,15 @@ ExpressionTextView::SetValue(BString value)
|
||||
}
|
||||
}
|
||||
|
||||
// add the exponent
|
||||
offset = value.CountChars() - 1;
|
||||
if (exponent != 0)
|
||||
if (exponent != 0) {
|
||||
value.Truncate(40);
|
||||
// truncate to a reasonable precision
|
||||
// while ensuring result will be rounded
|
||||
offset = value.CountChars() - 1;
|
||||
value << "E" << exponent;
|
||||
// add the exponent
|
||||
} else
|
||||
offset = value.CountChars() - 1;
|
||||
|
||||
// reduce the number of digits until the string fits or can not be
|
||||
// made any shorter
|
||||
|
||||
Reference in New Issue
Block a user