VEX Precision Issues

   2668   4   2
User Avatar
Member
12 posts
Joined: May 2015
Offline
I'm in desperate need of help here. The problem is simple, Houdini is giving some wildly inaccurate results with simple math problems. I first ran into this when using the modulo operator, but now I'm noticing it with stuff as basic as addition and subtraction. There are two things I've observed with this issue. Firstly, the problem when the numbers are variables.Secondly, the issues appears completely random. I can't find any patterns or anything.

One example is the equation: (vspacing*vpt)+voffset) - vpos.

(vspacing*vpt)+voffset) gives the correct result. vpos, is correct. But when I subtract vpos from (vspacing*vpt)+voffset) the result is simply not as it should be. For example, when the answer should be 0, the result is actually 7.45058e-9.

This is really screwing me up. I've trying trying to write some simple tools in VEX but this problem is making half the stuff I want to do absolutely impossible. Can anyone help? Does anyone know why this is happening?
User Avatar
Member
4189 posts
Joined: June 2012
Offline
What Every Computer Scientist Should Know About Floating-Point Arithmetic

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html [docs.oracle.com]
User Avatar
Member
7771 posts
Joined: Sept. 2011
Online
another helpful article on the matter:

https://en.wikipedia.org/wiki/Loss_of_significance [en.wikipedia.org]
User Avatar
Member
12 posts
Joined: May 2015
Offline
Never even heard of floating point arithmetic before. Thanks for those articles!
User Avatar
Member
806 posts
Joined: Oct. 2016
Offline
Moin,

I switched the (work in progress, to be seriously edited later on) chapter on “Floating Points” in my “Learn to program if that is the only thing you never ever wanted to do in your life” book that I am creating with my Patr(e)ons …

https://www.patreon.com/posts/15093066 [www.patreon.com]

Marc
---
Out of here. Being called a dick after having supported Houdini users for years is over my paygrade.
I will work for money, but NOT for "you have to provide people with free products" Indie-artists.
Good bye.
https://www.marc-albrecht.de [www.marc-albrecht.de]
  • Quick Links