From: Niklas Holsti <niklas.holsti@tidorum.invalid>
Subject: Re: Fixed vs float and precision and conversions
Date: Wed, 8 Jul 2020 23:34:46 +0300 [thread overview]
Message-ID: <hmmov7FkvneU1@mid.individual.net> (raw)
In-Reply-To: <re57d9$qv4$1@dont-email.me>
On 2020-07-08 22:39, Björn Lundin wrote:
> Den 2020-07-08 kl. 20:21, skrev Niklas Holsti:
>> On 2020-07-08 21:10, Björn Lundin wrote:
>>> Den 2020-07-08 kl. 20:08, skrev Björn Lundin:
>>>
>>> The main reason for asking was to see if I got the whole concept
>>> of fixed types wrong or not.
>>> I did expect
>>> 'You should do this or that one-liner' as Niklas proposed.
>>> I did not get that to work though
>>
>> Oh. What happened when you tried? How did it fail?
>>
>
> I did a test routine like below,
> but realized that Float(5.10) - which was converted to Fixed_Type(5.099)
> is a valid fixed_type of course.
No, see below. You are confusing decimal (base-10) reals with binary
(base-2) floats.
> so
> Fix1 := Fixed_Type(Flt);
> or
> Fix2 := Fixed_Type'Round(Flt);
>
> does not really matter, since both may return 5.099
> when given 5.10
Wrong. The point is that Float'(5.10) is not exactly 5.10, because
base-2 floats cannot represent decimal fractions exactly. Since the
result, as you showed in your first post, of converting (with
truncation) Float'(5.10) to Fixed_Type is 5.099, the actual (binary)
value of Float'(5.10) is a little less than 5.10, so the truncation
gives 5.099 instead of 5.100.
But Fixed_Type'Round (Float'(5.10)) will always give 5.100.
> It is at least not certain that the number expressed in fixed_type is
> one of them in the array below.
> And that is my real problem - I think.
That is certainly true -- if you don't know anything about the Float
number, then you won't know much about the result of Fixed_Type'Round on
that Float number. If you want to be sure that the result is one of the
numbers in your table, you have to check.
> ------------------------------
> type Fixed_Type is delta 0.001 digits 18;
> Bad_odds : exception;
> type Tics_Type is new integer range 1 .. 350;
>
> ---------------------------
> Global_Odds_Table : array(Tics_Type'Range) of Fixed_Type :=
> (
> 1.01, 1.02, 1.03, 1.04,
> 1.05, 1.06, 1.07, 1.08, 1.09,
[snip]
> 950.00, 960.00, 970.00, 980.00, 990.00,
> 1000.00);
If your Float numbers are as high as that (1000), you are approaching
the limit of Float precision (about 7 decimal digits) compared to the
delta of 0.001 for the Fixed_Type. That is, the Float representation of
a Fixed_Type value close to 1000.0 might just be off by more than
0.0005, in which case Fixed_Type'Round would give the corresponding (but
wrong) value.
However, I tried Fixed_Type'Round on Float values of 999.998, 999.999,
1000.001 and 1000.002, and got the corresponding Fixed_Type values, so
it may be ok.
But the values in the Global_Odds table are all given with two decimals,
not three, so why not define Fixed_Type with delta 0.01, which avoids
this possible Float precision problem, as well as being more apt?
--
Niklas Holsti
niklas holsti tidorum fi
. @ .
next prev parent reply other threads:[~2020-07-08 20:34 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-07 21:10 Fixed vs float and precision and conversions Björn Lundin
2020-07-07 21:30 ` Niklas Holsti
2020-07-08 8:17 ` Björn Lundin
2020-07-07 21:58 ` Shark8
2020-07-08 8:15 ` Björn Lundin
2020-07-08 16:16 ` Shark8
2020-07-08 18:08 ` Björn Lundin
2020-07-08 18:10 ` Björn Lundin
2020-07-08 18:21 ` Niklas Holsti
2020-07-08 19:39 ` Björn Lundin
2020-07-08 20:34 ` Niklas Holsti [this message]
2020-07-08 21:24 ` Björn Lundin
2020-07-09 7:11 ` Niklas Holsti
2020-07-08 18:36 ` Dmitry A. Kazakov
2020-07-08 19:41 ` Björn Lundin
2020-07-09 6:20 ` Dmitry A. Kazakov
2020-07-09 7:23 ` Björn Lundin
2020-07-09 7:49 ` Dmitry A. Kazakov
2020-07-08 21:16 ` Shark8
2020-07-08 21:47 ` Björn Lundin
replies disabled
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox