gprbuild and package renames (testsuites)

gprbuild and package renames (testsuites)

[Per Jakobsen]

Hi

I'm using gprbuilds "Naming" package to make Stub-packages for the system to test:

   package Naming is
      for Specification ("B") use "stub_b.ads";
      for Body          ("B") use "stub_b.adb";
   end Naming;

This works fine for a single package, but then I wonder if testing multiple packages may be linked into the same executable:

The package structure is Package A depends on Package B which depends on Package C, and I want to write unit-tests for each package.

To test Package A, a stub must be made for Package B.
To test Package B, a stub must be made for Package C.
Package C can be tested without stubs.

Is it possible and/or reasonable to link in tests for multiple packages into the same testrunner?

Test suite A: Package A -> Package Stub_B
Test suite B: Package B -> Package Stub_C
Test suite C: Package C

Thing can get even more "interesting" if there are circular dependencies ("limited with")...

How are you guys handling this?

~Per

Re: gprbuild and package renames (testsuites)

[briot.emmanuel]

I don't think you'll be able to run all three testsuites from the same executable. Instead, you could use a "scenario variable" to configure mode1, mode2 and mode3. Some case statements would be used to select which packages are stubs, as well as a mode-specific executable name and likely mode-specific object dirs. Then you run all three executables independently.

Things get more interesting when you have multiple projects involved. In this case, that would mean having to duplicate the case statements in all projects (and things get even more complicated when you want to mock various combinations -- for instance mock postgresql and Kafka for one test, but not mock postgresql in another one).

In the end, the case statement approach doesn't scale really well. We ended up using "extending" and "extending all" projects. For instance:

   project Postgresql is
         ...  --   used to build production executables
   end Postgresql;

   with "postgresql.gpr";
   with "b.gpr";
   project Build is
        ...  --  used to build production executables
   end Build;
 
   project Postgresql_Mock extends "postgresql.gpr" is
       --  override any file you need
       for Source_Dirs use ("mocks");
   end Postgresql_Mock;

  
   with "postgresql_mock.gpr";
   with "b.gpr";
   project Unittest extends all "build.gpr" is
        --  any project imported by build.gpr or b.gpr that was
        --  also importing "postgresql.gpr" will now instead
        --  import "postgresql_mock.gpr" and thus see the mocks
   end Unittest;

This solution is not so easy to setup, and is kind of fragile because it frequently happens that gprbuild doesn't properly recompile all the files (though I could never reproduce systematically and thus report it). In general
this happens when the timestamps of the mocks are similar to the timestamps of
the original files, so by "touch"-ing some files we get around the gprbuild
issues...

Emmanuel

Re: gprbuild and package renames (testsuites)

[Per Jakobsen]

> I don't think you'll be able to run all three testsuites from the same executable.

No, it's likely not feasible today. Dependency Injection is likely a better way in most cases.

> In the end, the case statement approach doesn't scale really well. We ended up using "extending" and "extending all" projects. For instance: ...

> This solution is not so easy to setup, and is kind of fragile because it frequently happens that gprbuild doesn't properly recompile all the files (though I could never reproduce systematically and thus report it).

Yikes, that reminds me of the old Borland C++ Builder - Two rebuilds in order to be sure that everything was up to date :-S

Re: gprbuild and package renames (testsuites)

[Mark Lorenzen]

On Monday, June 24, 2019 at 9:47:05 AM UTC+2, Per Jakobsen wrote:
> Hi
> 
> I'm using gprbuilds "Naming" package to make Stub-packages for the system to test:
> 
>    package Naming is
>       for Specification ("B") use "stub_b.ads";
>       for Body          ("B") use "stub_b.adb";
>    end Naming;
> 
> This works fine for a single package, but then I wonder if testing multiple packages may be linked into the same executable:
> 
> The package structure is Package A depends on Package B which depends on Package C, and I want to write unit-tests for each package.
> 
> To test Package A, a stub must be made for Package B.
> To test Package B, a stub must be made for Package C.
> Package C can be tested without stubs.
> 
> Is it possible and/or reasonable to link in tests for multiple packages into the same testrunner?
> 
> Test suite A: Package A -> Package Stub_B
> Test suite B: Package B -> Package Stub_C
> Test suite C: Package C
> 
> Thing can get even more "interesting" if there are circular dependencies ("limited with")...
> 
> How are you guys handling this?
> 
> ~Per

I can't give you an answer to your problem, but when I have had the need to provide stubbed packages for unit testing, I have made a unit test project that *extends* the original project. In the *extending* project, a new implementation of a package overrides the original. I thus have a project per unit test.

Regards,

Mark