Weird decimal behavior in numeric fields
2831 4 1- 8BitBeard
- Member
- 26 posts
- Joined: June 2016
- Offline
- peteski
- Member
- 517 posts
- Joined: Dec. 2013
- Offline
- BabaJ
- Member
- 2024 posts
- Joined: Sept. 2015
- Offline
It's not a precision error. The solution is to understand why you are seeing what you have.
There are a number of sources that explain it but here is one:
https://docs.python.org/2/tutorial/floatingpoint.html [docs.python.org]
There are a number of sources that explain it but here is one:
https://docs.python.org/2/tutorial/floatingpoint.html [docs.python.org]
- anon_user_37409885
- Member
- 4189 posts
- Joined: June 2012
- Offline
It's always helpful to know 1 and 0.999… are equal:
https://www.physicsforums.com/insights/why-do-people-say-that-1-and-999-are-equal/ [physicsforums.com]
Edit: For those interested in this:
“9.999… reasons that .999… = 1”
https://www.physicsforums.com/insights/why-do-people-say-that-1-and-999-are-equal/ [physicsforums.com]
Edit: For those interested in this:
“9.999… reasons that .999… = 1”
Edited by anon_user_37409885 - Jan. 13, 2017 20:24:14
- BabaJ
- Member
- 2024 posts
- Joined: Sept. 2015
- Offline
Actualy 1 and 0.999…. are not equal and there is no proof, contrary to the assumed proof.
The reason is because one hasn't yet defined 0.999…
By writting 0.999… as is, one is saying it goes on infinitely.
If it's going on infinitely, one has not yet established what that number actually is;
If one doesn't know yet what the number IS, one cannot make a true comparison, whether it is equal or not.
I know I'm being a bit pedantic, but to say both are equal is simply not true and can lead to wrong assumptions about results one gets when writing equations.
1 divided by 3 gives a number that goes on infinitely, one will never get a true finite number from that operation.
It's good to know that there are many instances in which you don't get finite numbers, and in those cases knowing how your software/hardware will handle those situations eg. number of decimal place precision for floating point numbers before they become rounded, or if desired truncated - This, I think is a much more helpful way to look/understand the situation.
The reason is because one hasn't yet defined 0.999…
By writting 0.999… as is, one is saying it goes on infinitely.
If it's going on infinitely, one has not yet established what that number actually is;
If one doesn't know yet what the number IS, one cannot make a true comparison, whether it is equal or not.
I know I'm being a bit pedantic, but to say both are equal is simply not true and can lead to wrong assumptions about results one gets when writing equations.
1 divided by 3 gives a number that goes on infinitely, one will never get a true finite number from that operation.
It's good to know that there are many instances in which you don't get finite numbers, and in those cases knowing how your software/hardware will handle those situations eg. number of decimal place precision for floating point numbers before they become rounded, or if desired truncated - This, I think is a much more helpful way to look/understand the situation.
Edited by BabaJ - Jan. 13, 2017 15:59:14
-
- Quick Links