Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I want to round number:
=num(round(39965.666666666999, 0.1), '#,###.###############')
My expectation is to get 39965.7, but the reality is 39,965.700000000004000
Is it bug or feature?
OC: Win Server 2008 EE
QV: 9.00.7440.8 SR3 x64
Does anybody have any idea how to get the proper result for the round?
Calculate your calendar DateHour the same way, and they'll match.
In reality:
1 * 3 / 3 = 1
1 / 3 * 3 = 1
In a computer:
1 * 3 / 3 = 1.00000000
1 / 3 * 3 = 0.99999999
That's exactly what you're doing. You're calculating the "same" value in several different ways, and thinking that somehow they're going to match. They're not going to match. You're using a computer. Computers have limited precision regardless of what base they're using, be that binary or decimal or hexadecimal or whatever. Any fraction that can only be represented as a repeating decimal in your chosen base is not going to have an exact representation in a computer.
To solve your problem, you have two options that I can think of. First, you can stick with integer arithmetic for 100% reliability. You say you can't do that, but you can. Seconds are integers. That's what your source data was in. You can preserve that, and only calculate things like timestamps on the fly using the 100% accurate seconds. That's very likely overkill for your situation, though. If your problem is simply that you have multiple storage values for the same fundamental time, you just have to calculate all your times the same way. If you're going to have a Timestamp field, and a DateHour field, and another DateHour field in a calendar, calculate them the same way, and the "incorrect" values will all match, and will therefore not cause you this problem, or any other practical problems that I've thought of.
See attached.
It works.
Thank you very much