comp.lang.ada
 help / color / mirror / Atom feed
From: "Björn Lundin" <b.f.lundin@gmail.com>
Subject: Re: Fixed vs float and precision and conversions
Date: Thu, 9 Jul 2020 09:23:30 +0200	[thread overview]
Message-ID: <re6glj$bs1$1@dont-email.me> (raw)
In-Reply-To: <re6cvd$1udg$1@gioia.aioe.org>

Den 2020-07-09 kl. 08:20, skrev Dmitry A. Kazakov:
> On 08/07/2020 21:41, Björn Lundin wrote:
> 
>> As I replied to Niklas:
>> It is at least not certain that the number expressed in fixed_type is 
>> one of them in the array below.
> 
> The array has varying step 0.01 to 10.0, your test has step 0.001. What 
> do you expect? Of course it must fail at a very high rate from 9/1 to 
> 999/1, practically always.
> 

welll, in practice not always.
It only fails when - like my example -

Fixed_Type(float(5.10)) becomes 5.099

and its been running with that construct since 2014,
and I only noticed now. So - not always, actually very seldom.

I did expect floating comparison error to go away using fixed type,
but I forgot about 'garbage in -> garbage out'

I don't have that kind of rounding errors once the values are converted 
to fixed_type but I did not realize that I had them in the conversion.

-- 
Björn

  reply	other threads:[~2020-07-09  7:23 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-07 21:10 Fixed vs float and precision and conversions Björn Lundin
2020-07-07 21:30 ` Niklas Holsti
2020-07-08  8:17   ` Björn Lundin
2020-07-07 21:58 ` Shark8
2020-07-08  8:15   ` Björn Lundin
2020-07-08 16:16     ` Shark8
2020-07-08 18:08       ` Björn Lundin
2020-07-08 18:10         ` Björn Lundin
2020-07-08 18:21           ` Niklas Holsti
2020-07-08 19:39             ` Björn Lundin
2020-07-08 20:34               ` Niklas Holsti
2020-07-08 21:24                 ` Björn Lundin
2020-07-09  7:11                   ` Niklas Holsti
2020-07-08 18:36           ` Dmitry A. Kazakov
2020-07-08 19:41             ` Björn Lundin
2020-07-09  6:20               ` Dmitry A. Kazakov
2020-07-09  7:23                 ` Björn Lundin [this message]
2020-07-09  7:49                   ` Dmitry A. Kazakov
2020-07-08 21:16         ` Shark8
2020-07-08 21:47           ` Björn Lundin
replies disabled

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox