Forums Technical Discussion Floating point error

Floating point error

   1447   4   0
User Avatar
nuclear_polygon
Member
20 posts
Joined: Nov. 2018
Offline
 
see attached clip. why does this happen? why it can not be just 0.9, why 0.900000000002? I want to put the channel value to the font sop, so I want it to look readable.

Attachments:
s01.mp4 (109.1 KB)

 Report
User Avatar
BabaJ
Member
2036 posts
Joined: Sept. 2015
Offline
 
Left click on your parameter label to alternate between the two display options.
 Report
User Avatar
jsmack
Member
7737 posts
Joined: Sept. 2011
Offline
 
To use floats as text in a Font sop, it's best to format them first to the desired readable precision. Use sprintf() in VEX or format() in python to convert the float to a string before referencing in the font sop's text field.
 Report
User Avatar
nuclear_polygon
Member
20 posts
Joined: Nov. 2018
Offline
 
thanx a lot! that's generally what I wanted to know
but does HScript have such expression?
Edited by nuclear_polygon - Aug. 28, 2020 10:20:29
 Report
User Avatar
BabaJ
Member
2036 posts
Joined: Sept. 2015
Offline
 
Well…in the font text parameter field you could use Hscript:

`ftoa(trunc(ch('your_string_parameter_reference'))) + '.' + trunc(frac(ch('your_string_parameter_reference')) * pow(10,change_this_to_number_of_desired_decimal_places))`
 Report
  • Quick Links