Re: Custom Storage Pool questions

["Randy Brukardt" <randy@rrsoftware.com>]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:skbdb5$u50$1@gioia.aioe.org...
> On 2021-10-15 02:36, Randy Brukardt wrote:
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>> news:sk8mbv$15ca$1@gioia.aioe.org...
>>> On 2021-10-14 03:21, Randy Brukardt wrote:
>>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>>>> news:sjbq96$3cl$1@gioia.aioe.org...
>>>>> On 2021-10-03 06:33, Randy Brukardt wrote:
>>>> ...
>>> A static library is tested same way, true, but integration of a static
>>> library is different and testing that is not possible without developing
>>> some massive tool-chain, like Linux distributions had in early days.
>>
>> ??? Your "test application" is just a way of running unit tests against a
>> library. You can surely do exactly the same testing with a 
>> statically-linked
>> library, it's hard to imagine how a library is packaged would make any
>> difference.
>
> It is linking static stuff alternately that need to be tested. If you use 
> a subset of statically linked components you need to change the code that 
> uses them correspondingly, unless you do an equivalent of dynamically 
> loaded components without any advantages of.
>
> As an example, consider switching between GNUTLS and OpenSSL for 
> encryption of say MQTT connections.

I guess I don't follow. The connections between units is static (in Ada, and 
any dynamic loading in Ada has to follow the same model), so any unit 
testing on a unit tests all of its dependencies as well. If you don't use a 
unit at all, it isn't included in the closure, and whether or not it passes 
any tests is irrelevant.

In the case you describe, you'd have a binding that abstracts the two 
underlying libraries, and you'd unit test that. Assuming it passes with both 
implementations, it shouldn't matter which is used in a particular program. 
How the foreign langauge code is implemented (static or dynamic binding, 
programming language, etc.) is irrelevant to the Ada program. So again I 
don't see the problem.

....
>> Otherwise, subtle changes in a library will break the users and there 
>> will
>> be no way to find where the dependency is. Nothing I've ever worked on 
>> has
>> ever been close to maintainable because there is so much that Ada cannot
>> describe (even though Ada itself is certainly a help in this area). You 
>> just
>> have to re-test to make sure that no major problems have been introduced
>> (there's a reason that compiler writer's rerun a huge test suite every 
>> day).
>
> Yes, but it is not economically viable anymore. Nobody would pay for that.

Really? They have no choice -- otherwise, your product will fail 
periodically without any way to find out why. Has software gotten so bad 
that no one cares about that? I certainly would never do that to our 
customers (and I hate support anyway, I want to reduce it as much as 
possible).

>> Only if you don't use unit tests. But then how you can test a dynamic
>> library escapes me. (I've never used unit tests with Janus/Ada because it 
>> is
>> too hard to set up the initial conditions for a meaningful test. The 
>> easiest
>> way to do that is to compile something, but of course you no longer can 
>> do
>> unit tests as you have the entire rest of the system dragged along.)
>
> We test Ada packages statically linked and we have 
> semi-unit/semi-integration tests that load the library first. It is not a 
> big deal.

Which sounds like there is no reason to use dynamic linking except to make 
your applications far more fragile. I suppose it doesn't matter as much if 
the libraries are all under your control, but that is rarely the case in the 
real world. (Your example of connection encryption is a good one; changes to 
the underlying stuff tends to break existing programs. Not much that can be 
done about it, of course, but those updates don't really fix anything by 
themselves, the using programs tend to need to be repaired.)

>>> Dynamic libraries flatten that. Yes, this requires normalization of
>>> plug-in interfaces etc.
>>
>> As noted above, I don't see how. If testing a dynamic library is 
>> possible,
>> surely running the same tests against a static library would give the 
>> same
>> results (and assurances).
>
> Only if you create some equivalent of "static" plug-in with all 
> disadvantages of proper plug-in and none of the advantages.

There's no plug-ins in a statically linked system. What would be the point? 
I'm assuming that we're talking about Ada interfaced libraries here (C is a 
different kettle of fish), so you're talking about switching implementations 
of a single Ada spec. We've done that going back to the beginning of Ada 
time; it's managed by decent build tools and has gotten pretty simple in 
most Ada compilers. So what would a plug-in buy?

>>>> There's no use to an Ada dynamic library -- if it's only for your
>>>> organization's use, static linking is way better.
>>>
>>> You compare static vs. import library. The case I am talking about is
>>> static vs. late dynamic loading, i.e. dlopen/dlsym stuff. And, yes, we 
>>> do
>>> dlsym on entries with Ada calling conventions. No C stuff.
>>
>> That sort of stuff is just plain evil. :-)
>
> Yes! (:-))
>
>> I don't see any way that such loading could work with Ada semantics; 
>> there
>> is an assumption that all of your ancestors exist before you can do
>> anything. The elaboration checks were intended to check that.
>
> We have a core libraries which are import libraries for the plug-in. When 
> a plug-in is loaded, the core libraries are elaborated unless already 
> loaded, the plug-in library itself is not elaborated, because automatic 
> elaboration would deadlock under Windows. Then a dedicated entry point is 
> called in the plug-in library. The first thing it does is a call to the 
> plug-in elaboration code. GNAT generates an <library-name>init entry for 
> that. After this the plug-in registers itself providing a tagged object, 
> which primitive operations are basically the library's true interface.
>
> I know it sounds horrific, but it works pretty well.

It does sound horrific, and it doesn't seem to buy much.

>> ??? The dispatching tables are defined statically by the compiler, and 
>> never
>> change. What I'd do for dynamically loaded libraries is use a wrapper 
>> that
>> indirectly calls the dynamically loaded libraries' subprograms. So 
>> loading
>> the library (actually, declaring the extension) simply has to set up an
>> array of pointers to the dynamically loaded subprograms. (You can't call
>> them statically because you don't know where they'll be.) The dispatch
>> tables never change.
>
> And how do you dispatch? Consider the case:
>
> The core library:
>
>    package A is
>       type T is tagged ...;
>       procedure Foo (X : in out T);
>
>       procedure Trill_Me (X : in out T'Class);
>    end A;
>
>    package body A is
>       procedure Trill_Me (X : in out T'Class) is
>       begin
>          X.Foo; -- Dispatches to Foo overridden in a loadable library
>       end Trill_Me;
>   end A;
>
> Inside the loadable library:
>
>   type S is new T with ...;
>   overriding procedure Foo (X : in out S);
>   ...
>   X : B;
>   ...
>   Trill_Me (X);
>
> Do you keep a pointer to the dispatching table inside the object, like C++ 
> does? Because I had a more general model in mind, when dispatching tables 
> were attached to the primitive operations rather than objects.

A tag is a property of a type in Ada, and it includes the dispatch table. 
You could have a model where the dispatch table didn't live in the object 
(but that's not Ada, you have to be able to recover the original tag of the 
object), but that wouldn't change anything about the structure of the 
tables. I don't see any model that makes sense associated with the 
operations. The whole point of a tagged type is that it is a set of 
operations called in a consistent way, breaking that up makes no sense.

....
>>>> The problem comes about when you have things whose lifetime is limited
>>>> and
>>>> need to have a static link or display to access them. Managing that is 
>>>> a
>>>> nightmare, no matter how you try to do it.
>>>
>>> The lifetime of library objects in a dynamically loaded library is 
>>> limited
>>> by loading/unloading of the library.
>>
>> They're still treated as library-level,
>
> Right, and this is the problem, because semantically anything inside a 
> dynamically loaded library is not just nested, worse, it is more like 
> new/Unchecked_Deallocation, but with things like types etc.

You can't unload a library until all of the things that depend upon it have 
been unloaded, so from the perspective of a compiler, it acts like 
library-level. The whole mess about loading/unloading is on the user (which 
is what I meant about "unsafe" yesterday), and if you get it wrong, your 
program is erroneous and can do any manner of things. It's no more worth it 
for a compiler to worry about bad unloading than it is to worry about 
dangling pointers. At best, those sorts of things have to be handled 
dynamically (and the compiler doesn't care much about dynamic behavior).

                            Randy.