From: "Björn Lundin" <b.f.lundin@gmail.com>
Subject: Re: Fixed vs float and precision and conversions
Date: Wed, 8 Jul 2020 23:24:55 +0200 [thread overview]
Message-ID: <re5dj7$4fn$1@dont-email.me> (raw)
In-Reply-To: <hmmov7FkvneU1@mid.individual.net>
Den 2020-07-08 kl. 22:34, skrev Niklas Holsti:
>
> No, see below. You are confusing decimal (base-10) reals with binary
> (base-2) floats.
Yes.
>> It is at least not certain that the number expressed in fixed_type is
>> one of them in the array below.
>> And that is my real problem - I think.
>
>
> That is certainly true -- if you don't know anything about the Float
> number, then you won't know much about the result of Fixed_Type'Round on
> that Float number. If you want to be sure that the result is one of the
> numbers in your table, you have to check.
>
yes - I realize that
>
> But the values in the Global_Odds table are all given with two decimals,
> not three, so why not define Fixed_Type with delta 0.01, which avoids
> this possible Float precision problem, as well as being more apt?
Because I wanted to express 6.5% -> 0.065 in fixed_type as well.
I think I mentioned that in a post.
Fixed_type is used generally here (in this system) instead of float, not
only for that array.
This may have been a bad design choise though.
--
Björn
next prev parent reply other threads:[~2020-07-08 21:24 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-07 21:10 Fixed vs float and precision and conversions Björn Lundin
2020-07-07 21:30 ` Niklas Holsti
2020-07-08 8:17 ` Björn Lundin
2020-07-07 21:58 ` Shark8
2020-07-08 8:15 ` Björn Lundin
2020-07-08 16:16 ` Shark8
2020-07-08 18:08 ` Björn Lundin
2020-07-08 18:10 ` Björn Lundin
2020-07-08 18:21 ` Niklas Holsti
2020-07-08 19:39 ` Björn Lundin
2020-07-08 20:34 ` Niklas Holsti
2020-07-08 21:24 ` Björn Lundin [this message]
2020-07-09 7:11 ` Niklas Holsti
2020-07-08 18:36 ` Dmitry A. Kazakov
2020-07-08 19:41 ` Björn Lundin
2020-07-09 6:20 ` Dmitry A. Kazakov
2020-07-09 7:23 ` Björn Lundin
2020-07-09 7:49 ` Dmitry A. Kazakov
2020-07-08 21:16 ` Shark8
2020-07-08 21:47 ` Björn Lundin
replies disabled
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox