Re: Custom Storage Pool questions

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On 2021-10-15 02:36, Randy Brukardt wrote:
> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
> news:sk8mbv$15ca$1@gioia.aioe.org...
>> On 2021-10-14 03:21, Randy Brukardt wrote:
>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>>> news:sjbq96$3cl$1@gioia.aioe.org...
>>>> On 2021-10-03 06:33, Randy Brukardt wrote:
>>> ...
>>>> Yes, but static monolithic linking is even more fragile. Typically a
>>>> customer orders software off the shelf. It means that he says I need,
>>>> e.g.
>>>> HTTP client, ModBus master, CANOpen etc. It is simply impossible to
>>>> re-link everything for each customer and run integration tests.
>>>
>>> ??? When you are dynamically loading stuff, you simply are assuming
>>> everything is OK.
>>
>> A relocatable DLL is tested with a test application.
> 
> Testing cannot ensure that a contract hasn't been violated, especially the
> implicit ones that get created by the runtime behavior of a library. At
> best, you can test a few percent of the way a library can be used (and
> people are good at finding unanticipated ways to use a library).

Yes, high integrity system would likely have a monolithic design, but 
this too is changing because the size of systems keeps on growing.

>>> (Which is usually nonsense, but for the sake of argument,
>>> assume that it is OK to do.) When you statically link, you surely can
>>> make
>>> the same assumption.
>>
>> A static library is tested same way, true, but integration of a static
>> library is different and testing that is not possible without developing
>> some massive tool-chain, like Linux distributions had in early days.
> 
> ??? Your "test application" is just a way of running unit tests against a
> library. You can surely do exactly the same testing with a statically-linked
> library, it's hard to imagine how a library is packaged would make any
> difference.

It is linking static stuff alternately that need to be tested. If you 
use a subset of statically linked components you need to change the code 
that uses them correspondingly, unless you do an equivalent of 
dynamically loaded components without any advantages of.

As an example, consider switching between GNUTLS and OpenSSL for 
encryption of say MQTT connections.

>> Yes, but maintainability trumps everything.
> 
> I agree with the sentiment, but the only way to get any sort of
> maintenability is with strong contracts and lots of static analysis.

Yes, contracts is a weak part of dynamically loaded stuff. In our case a 
component registers itself after its library is loaded by providing an 
instance of tagged object.

> Otherwise, subtle changes in a library will break the users and there will
> be no way to find where the dependency is. Nothing I've ever worked on has
> ever been close to maintainable because there is so much that Ada cannot
> describe (even though Ada itself is certainly a help in this area). You just
> have to re-test to make sure that no major problems have been introduced
> (there's a reason that compiler writer's rerun a huge test suite every day).

Yes, but it is not economically viable anymore. Nobody would pay for that.

> Only if you don't use unit tests. But then how you can test a dynamic
> library escapes me. (I've never used unit tests with Janus/Ada because it is
> too hard to set up the initial conditions for a meaningful test. The easiest
> way to do that is to compile something, but of course you no longer can do
> unit tests as you have the entire rest of the system dragged along.)

We test Ada packages statically linked and we have 
semi-unit/semi-integration tests that load the library first. It is not 
a big deal.

>> Dynamic libraries flatten that. Yes, this requires normalization of
>> plug-in interfaces etc.
> 
> As noted above, I don't see how. If testing a dynamic library is possible,
> surely running the same tests against a static library would give the same
> results (and assurances).

Only if you create some equivalent of "static" plug-in with all 
disadvantages of proper plug-in and none of the advantages.

>>> There's no use to an Ada dynamic library -- if it's only for your
>>> organization's use, static linking is way better.
>>
>> You compare static vs. import library. The case I am talking about is
>> static vs. late dynamic loading, i.e. dlopen/dlsym stuff. And, yes, we do
>> dlsym on entries with Ada calling conventions. No C stuff.
> 
> That sort of stuff is just plain evil. :-)

Yes! (:-))

> I don't see any way that such loading could work with Ada semantics; there
> is an assumption that all of your ancestors exist before you can do
> anything. The elaboration checks were intended to check that.

We have a core libraries which are import libraries for the plug-in. 
When a plug-in is loaded, the core libraries are elaborated unless 
already loaded, the plug-in library itself is not elaborated, because 
automatic elaboration would deadlock under Windows. Then a dedicated 
entry point is called in the plug-in library. The first thing it does is 
a call to the plug-in elaboration code. GNAT generates an 
<library-name>init entry for that. After this the plug-in registers 
itself providing a tagged object, which primitive operations are 
basically the library's true interface.

I know it sounds horrific, but it works pretty well.

> ??? The dispatching tables are defined statically by the compiler, and never
> change. What I'd do for dynamically loaded libraries is use a wrapper that
> indirectly calls the dynamically loaded libraries' subprograms. So loading
> the library (actually, declaring the extension) simply has to set up an
> array of pointers to the dynamically loaded subprograms. (You can't call
> them statically because you don't know where they'll be.) The dispatch
> tables never change.

And how do you dispatch? Consider the case:

The core library:

    package A is
       type T is tagged ...;
       procedure Foo (X : in out T);

       procedure Trill_Me (X : in out T'Class);
    end A;

    package body A is
       procedure Trill_Me (X : in out T'Class) is
       begin
          X.Foo; -- Dispatches to Foo overridden in a loadable library
       end Trill_Me;
   end A;

Inside the loadable library:

   type S is new T with ...;
   overriding procedure Foo (X : in out S);
   ...
   X : B;
   ...
   Trill_Me (X);

Do you keep a pointer to the dispatching table inside the object, like 
C++ does? Because I had a more general model in mind, when dispatching 
tables were attached to the primitive operations rather than objects.

>>> The problem comes about when you have things whose lifetime is limited
>>> and
>>> need to have a static link or display to access them. Managing that is a
>>> nightmare, no matter how you try to do it.
>>
>> The lifetime of library objects in a dynamically loaded library is limited
>> by loading/unloading of the library.
> 
> They're still treated as library-level,

Right, and this is the problem, because semantically anything inside a 
dynamically loaded library is not just nested, worse, it is more like 
new/Unchecked_Deallocation, but with things like types etc.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de