OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 10:30 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: Weird behaving floats/doubles
PostPosted: Sun Nov 21, 2021 12:01 pm 
Offline
Member
Member

Joined: Tue Aug 31, 2021 7:25 am
Posts: 67
For some reason when doing maths on floats/doubles they start behaving rather weirdly.
For instance, when multiplying 0.235 by 1000, the result is 234, not 235.

I also wrote a script that simply to convert a fraction to an integer, it first extracts the decimal points `(x * 1e17 - (uint32_t(x)) * 1e17) / 1e17` then keeps multiplying the number by 10 until the condition dec == 0 equals to true.
Problem is with many points (can't pinpoint which ones) for example 0.295, everything freezes which I presume is due to the condition never evaluating to true. Could it be an issue with the FPU configuration?

The initiation of the FPU is pretty much the exact from the documentation and set to word 0x37F with the CR4 or'd with 0x200.

I am running in protected mode.

Thanks!


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Sun Nov 21, 2021 12:56 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
YDeeps1 wrote:
everything freezes which I presume is due to the condition never evaluating to true.

Have you checked with a debugger?

YDeeps1 wrote:
The initiation of the FPU is pretty much the exact from the documentation and set to word 0x37F with the CR4 or'd with 0x200.

Which documentation? Which word is set to 0x37F? Why aren't you setting OSXMMEXCPT in CR4?

Which floating-point instruction sets did you tell your compiler to use?


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Sun Nov 21, 2021 2:51 pm 
Offline
Member
Member

Joined: Tue Aug 31, 2021 7:25 am
Posts: 67
Octocontrabass wrote:
YDeeps1 wrote:
everything freezes which I presume is due to the condition never evaluating to true.

Have you checked with a debugger?

YDeeps1 wrote:
The initiation of the FPU is pretty much the exact from the documentation and set to word 0x37F with the CR4 or'd with 0x200.

Which documentation? Which word is set to 0x37F? Why aren't you setting OSXMMEXCPT in CR4?

Which floating-point instruction sets did you tell your compiler to use?


Have you checked with a debugger?:
I have checked and did confirm that is the case. The three digits did become integers (with the weird precision) followed by random numbers causing it to never finish and eventually overflow to a negative value.

Which documentation?:
I got the documentation confused with another one, ignore that.

Which word is set to 0x37F?:
I set the control word with fldcw to a memory location containing 0x37F.

Why aren't you setting OSXMMEXCPT in CR4?:
I have set it now (10th bit starting at 0 in CR4).

Which floating-point instruction sets did you tell your compiler to use?:
Not sure how or where to set this. I assume the defaults in a g++ cross-compiler?


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Sun Nov 21, 2021 3:12 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
YDeeps1 wrote:
I have checked and did confirm that is the case. The three digits did become integers (with the weird precision) followed by random numbers causing it to never finish and eventually overflow to a negative value.

It sounds like the algorithm you've chosen isn't stable. Stable algorithms take a bit of effort, but they do exist. Of course, you must still initialize the floating point environment correctly for these algorithms to work as expected.

YDeeps1 wrote:
I set the control word with fldcw to a memory location containing 0x37F.

Did you initialize the rest of the x87 registers using FNINIT? Did you initialize MXCSR?

YDeeps1 wrote:
Not sure how or where to set this. I assume the defaults in a g++ cross-compiler?

The defaults might depend on how you built your cross-compiler. Here are the options to control which instructions are used.


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Sun Nov 21, 2021 10:27 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
YDeeps1 wrote:
For instance, when multiplying 0.235 by 1000, the result is 234, not 235.
Are you rounding the result correctly or are you just truncating it?
235/1000 = 47/200. Since even the completely reduced fraction does not have a power of two as denominator, 47/200 cannot be represented as a finite binary fraction. Therefore any floating-point representation of it must be rounded, so either a little above or a little below. And indeed this can depend on the precision used to save the number (e.g. 0.1 as a single precision number was a bit below, and as a double it was slightly above 0.1). So if you save the number in a precision that is slightly below the target, then your calculation may result in 234.999... and if you then get the result by truncating instead of rounding, yes, you get 234 out of it.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Mon Nov 22, 2021 6:48 am 
Offline
Member
Member

Joined: Tue Aug 31, 2021 7:25 am
Posts: 67
nullplan wrote:
YDeeps1 wrote:
For instance, when multiplying 0.235 by 1000, the result is 234, not 235.
Are you rounding the result correctly or are you just truncating it?
235/1000 = 47/200. Since even the completely reduced fraction does not have a power of two as denominator, 47/200 cannot be represented as a finite binary fraction. Therefore any floating-point representation of it must be rounded, so either a little above or a little below. And indeed this can depend on the precision used to save the number (e.g. 0.1 as a single precision number was a bit below, and as a double it was slightly above 0.1). So if you save the number in a precision that is slightly below the target, then your calculation may result in 234.999... and if you then get the result by truncating instead of rounding, yes, you get 234 out of it.


That would actually make a lot of sense. The algorithm does not try to round at all and was made without real thought. I should add a precision limit first and then implement an algorithm to round nasty numbers or round to x decimal places. If I remember correctly, the numbers after were .99999 or something similar which would make sense.


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Mon Nov 22, 2021 6:52 am 
Offline
Member
Member

Joined: Tue Aug 31, 2021 7:25 am
Posts: 67
Octocontrabass wrote:
YDeeps1 wrote:
I have checked and did confirm that is the case. The three digits did become integers (with the weird precision) followed by random numbers causing it to never finish and eventually overflow to a negative value.

It sounds like the algorithm you've chosen isn't stable. Stable algorithms take a bit of effort, but they do exist. Of course, you must still initialize the floating point environment correctly for these algorithms to work as expected.

YDeeps1 wrote:
I set the control word with fldcw to a memory location containing 0x37F.

Did you initialize the rest of the x87 registers using FNINIT? Did you initialize MXCSR?

YDeeps1 wrote:
Not sure how or where to set this. I assume the defaults in a g++ cross-compiler?

The defaults might depend on how you built your cross-compiler. Here are the options to control which instructions are used.


Yeah I figured my algorithm is not stable and does not take into account lack of precision.

Did you initialize the rest of the x87 registers using FNINIT? Did you initialize MXCSR?:
I thought running fninit initialized all of the registers. Regarding MXCSR I have not, first time hearing about it actually.


Top
 Profile  
 
 Post subject: Re: Weird behaving floats/doubles
PostPosted: Mon Nov 22, 2021 9:41 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
YDeeps1 wrote:
I thought running fninit initialized all of the registers. Regarding MXCSR I have not, first time hearing about it actually.

FNINIT only initializes the x87 registers. MXCSR isn't an x87 register, so you need to initialize it separately if you want to use SSE or AVX. (You don't need to initialize MXCSR if you're not using SSE/AVX.)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot], Majestic-12 [Bot] and 56 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group