Help with designing data structure for novice Ada programmer

Help with designing data structure for novice Ada programmer

[Aleksy Grabowski]

Hi all, I'm new here.

I'm experienced embedded C developer and I've started 2 years ago an 
implementation of one of the specifications kind of popular in the
payment industry. I also hope to make it open source one day.

I've implemented most of it in C, but now I have a problem, because I
have to guarantee consistency of a huge configurable data structure,
that can be changed by an entity which in theory may not be in my
control. And adding all checks quickly became nightmarish and then I've
found an Ada language.

I already did some experiments, but it doesn't look right, because my
experience in Ada is around 1 week. And I want to quickly assess if it
is worth doing at all.

So my requirements:

  1. Important: Syntax to accessing, setting, comparing, allocating
     values should be as simple as possible, because the body of code is
     generated from diagrams and it should be understandable by a non
     programmer, ie payments expert. In C I've wrote some macros for it.
  2. Memory management should be unobtrusive to the visual look of
     diagrams. In C I've used a memory pool, so I just have to do a
     single de-allocation after each transaction.
  3. It should support optional, mandatory and conditional fields. In C
     modeled as pointers.
  4. It should support bit fields that may have different meaning if some
     flag is set (in C a union of bitfields)
  5. Consistency between elements, if a particular flag is set to True
     then some other fields must have a certain value as well. Probably
     doable by Dynamic_Predicate.
  6. Data elements should be accessible by name `db.processingStatus' or
     by it's tag value like `db (16#CA#)' and also it should support any
     additional tag that can be supplied by the card, even if its
     symbolic name isn't known (in C it's void*).
  7. I have to guarantee consistency of a different views of the same
     value eg. amount represented as BCD array – usual representation
     or 32 bit unsigned integer when requested by tag 16#81#.
  8. It should be possible to modify data elements both symbolically or
     using binary arrays with encoded data. Either because it is
     specified this way or I have to set it to the unspecified value. In
     C I've achieved this by having a union of char[] and a bitfield.
  9. And as a bonus it should be serializable to EMV's variant of TLV
     which isn't the same as you have in ASN.1, because EMV specification
     was standardized before ASN.1 BER and those committees had little to
     no cooperation.

I'm asking Ada wizards if it is at all doable in Ada? Or maybe some
links to opensource project where something similar was already achieved
so I can look it up?

--
Best Regards,
Alex

P.S. My code is available on github if someone is interested.

Re: Help with designing data structure for novice Ada programmer

[Paul Rubin]

Aleksy Grabowski <hurufu@gmail.com> writes:
> I'm experienced embedded C developer and I've started 2 years ago an
> implementation of one of the specifications kind of popular in the
> payment industry. I also hope to make it open source one day.

Can you say what specification it is?

> have to guarantee consistency of a huge configurable data structure,
> that can be changed by an entity which in theory may not be in my
> control. And adding all checks quickly became nightmarish and then I've
> found an Ada language.

It sounds like you waht to define a datatype for this structure, with
access and update procedures (OOP is not necessary but it's the same
idea) that make sure all the rules are followed.  Is there more to it
than that?

Ada sounds like a reasonable choice, C sounds terrible, other
possibilities depend on the hardware and software environment.  Would
this have to run on a smart card cpu or anything like that?

Re: Help with designing data structure for novice Ada programmer

[Aleksy Grabowski]

First of all thanks for an answer.

On 1/9/22 03:36, Paul Rubin wrote:
> Can you say what specification it is?

Yeah sure, it's called nexo. It is an open specification and it is
available for free after a registration on nexo-standards.org. It is
basically a high level description of an abstract payment terminal. They
specify all the data structures and full flow of control. Basically they
try to combine all EMV books and all regional payment schemes into a one
huge beast, to make a universal terminal.

> It sounds like you waht to define a datatype for this structure, with
> access and update procedures (OOP is not necessary but it's the same
> idea) that make sure all the rules are followed.  Is there more to it
> than that?

Something more. The first thing is update procedures, Right now I have
two modules trusted and non-trusted. Trusted part just does whatever it
wants to and non-trusted uses geters and setters. But the other thing is
that I want to model data as close to the spec as possible, so if it is
set to update some enum to a value 16#3F00# I do this. The another part
is because they tried to unify the whole payment zoo into single spec
there a lot of bizarre things there.

I also have some concrete question, how to properly implement optional
element? Right now I have something like this:

     generic
         type T is private;
     package TypeUtils is
         type Optional(exists : Boolean) is
         record
             case exists is
                 when True => value : T;
                 when False => null;
             end case;
         end record;
     end TypeUtils;

But it looks like it doesn't work, because it depends on the 
discriminant `exists' but it is set once in .ads file and I can't modify
it in runtime.

> Ada sounds like a reasonable choice, C sounds terrible, other
> possibilities depend on the hardware and software environment.  Would
> this have to run on a smart card cpu or anything like that?

Not really, arm CPU is enough. The one I have is MAX32590. I'm also not
really worried about memory consumption. Those embedded platforms have a
plenty of RAM nowadays.

How large this project can be? I'm already two years into it :)

Re: Help with designing data structure for novice Ada programmer

[Dmitry A. Kazakov]

On 2022-01-09 10:49, Aleksy Grabowski wrote:

> I also have some concrete question, how to properly implement optional
> element? Right now I have something like this:
> 
>      generic
>          type T is private;
>      package TypeUtils is
>          type Optional(exists : Boolean) is
>          record
>              case exists is
>                  when True => value : T;
>                  when False => null;
>              end case;
>          end record;
>      end TypeUtils;
> 
> But it looks like it doesn't work, because it depends on the 
> discriminant `exists' but it is set once in .ads file and I can't modify
> it in runtime.

Provide a default value for the discriminant:

    type Optional (Defined : Boolean := False) is record
        case Present is
           when True =>
              Value : T;
           when False =>
              null;
        end case;
    end record;

Now it is a definite type and you can place it into another record type:

    type Container is record
       ...
       Optional_Field : Optional;
       ...
    end record;

and you can always update it:

    X : Container;

    X.Optional_Field := (True, Value_1);
    ...
    X.Optional_Field := (Defined => False);
    ...
    X.Optional_Field := (True, Value_2);
    ...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Help with designing data structure for novice Ada programmer

[1.AAC0832]

On 1/8/22 9:36 PM, Paul Rubin wrote:
> Aleksy Grabowski <hurufu@gmail.com> writes:
>> I'm experienced embedded C developer and I've started 2 years ago an
>> implementation of one of the specifications kind of popular in the
>> payment industry. I also hope to make it open source one day.
> 
> Can you say what specification it is?
> 
>> have to guarantee consistency of a huge configurable data structure,
>> that can be changed by an entity which in theory may not be in my
>> control. And adding all checks quickly became nightmarish and then I've
>> found an Ada language.
> 
> It sounds like you waht to define a datatype for this structure, with
> access and update procedures (OOP is not necessary but it's the same
> idea) that make sure all the rules are followed.  Is there more to it
> than that?
> 
> Ada sounds like a reasonable choice, C sounds terrible, other
> possibilities depend on the hardware and software environment.  Would
> this have to run on a smart card cpu or anything like that?

   Having just implemented a small app using interlinked
   doubly-linked lists that are allocated on demand, Ada
   can do at least that much quite easily and cleanly without
   any OOP BS. "Records"/"Structs" are much as in Pascal,
   but they don't call pointers "pointers"  :-)

   You DO have to bring in a special proc to FREE memory allocated
   on the heap however. Kinda weird - you'd think making and freeing
   would naturally be implemented together. This "free()" is very
   type-specific, multiple incarnation of the proc will be needed
   if you have multiple kinds of records to free.

   The other (extensive) requirements ... I'll leave that to the
   more experienced.

   (am I off one level here ?)

Re: Help with designing data structure for novice Ada programmer

[Dennis Lee Bieber]

On Fri, 14 Jan 2022 00:01:22 -0500, "1.AAC0832" <z24ba7.net> declaimed the
following:


>   You DO have to bring in a special proc to FREE memory allocated
>   on the heap however. Kinda weird - you'd think making and freeing
>   would naturally be implemented together. This "free()" is very
>   type-specific, multiple incarnation of the proc will be needed
>   if you have multiple kinds of records to free.
>
	Take into account the origins for Ada. Many embedded/real-time
applications follow the practice of pre-allocating all memory during
application initialization before transitioning into "running" mode. They
never deallocate memory. If something goes wrong, the entire application
system is rebooted, resetting all memory.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

Re: Help with designing data structure for novice Ada programmer

[Dmitry A. Kazakov]

On 2022-01-14 17:05, Dennis Lee Bieber wrote:
> On Fri, 14 Jan 2022 00:01:22 -0500, "1.AAC0832" <z24ba7.net> declaimed the
> following:
> 
> 
>>    You DO have to bring in a special proc to FREE memory allocated
>>    on the heap however. Kinda weird - you'd think making and freeing
>>    would naturally be implemented together. This "free()" is very
>>    type-specific, multiple incarnation of the proc will be needed
>>    if you have multiple kinds of records to free.
>>
> 	Take into account the origins for Ada. Many embedded/real-time
> applications follow the practice of pre-allocating all memory during
> application initialization before transitioning into "running" mode. They
> never deallocate memory. If something goes wrong, the entire application
> system is rebooted, resetting all memory.

They may use some special management of resources that do not involve 
heap and explicit deallocation. E.g. a linked list can be allocated in 
an arena. There would be no free(), the whole list gets killed when the 
arena is erased.

The arena itself may sit in a local/static storage array or a 
pre-allocated upon start array.

Ada provides user memory pools for that kind of implementations. As an 
example consider a network protocol implementation. There would be a 
fixed-size storage per connection where entities of an incoming packet 
are stored upon parsing it. After processing the packet the storage is 
emptied.

No buffer overflow or network attacks possible. If the packet has too 
much data, Storage_Error is propagated from new and the connection gets 
dropped.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de