ANN: Simple components for Ada v.4.42

ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

The new version provides an implementation of JSON (RFC 7158)

    http://www.dmitry-kazakov.de/ada/components.htm

The JSON implementation supports both parsing and output of JSON 
objects. The parser allocates parts of the JSON object in a 
user-provided storage pool that can be an arena stack allowing both 
performance and safety by limiting the overall size of the object.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[briot.emmanuel]

On Monday, September 16, 2019 at 8:48:36 AM UTC+2, Dmitry A. Kazakov wrote:
> The new version provides an implementation of JSON (RFC 7158)


Hi Dmitry,
I would be curious to know the differences compared to what GNATCOLL.JSON provides, if you know ? It would be nice if the Ada world did not have too many packages competing here.
(Although I was the GNATCOLL maintainer for a long time, I did not write the JSON package, so this is really an open question :-)

Various things which I find limiting in GNATCOLL.JSON: performance is not really good because there are a lot of memory allocations (your short description seems to indicate this might be an area where your package improves things); gnatcoll treats arrays and objects completely differently, one without tagged types and the other with; simple types systematically have to be converted explicitly to a json object with Create;...

The GNATCOLL.JSON parser can be extended to support YAML (I have some local patches to support comments and optional quotes for strings, but not the indentation-based structures), but it would be nice if this was provided out of the box...

Thoughts ?

Emmanuel

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-09-16 08:58, briot.emmanuel@gmail.com wrote:
> On Monday, September 16, 2019 at 8:48:36 AM UTC+2, Dmitry A. Kazakov wrote:
>> The new version provides an implementation of JSON (RFC 7158)
> 
> I would be curious to know the differences compared to what GNATCOLL.JSON provides, if you know ? It would be nice if the Ada world did not have too many packages competing here.
> (Although I was the GNATCOLL maintainer for a long time, I did not write the JSON package, so this is really an open question :-)
> 
> Various things which I find limiting in GNATCOLL.JSON: performance is not really good because there are a lot of memory allocations (your short description seems to indicate this might be an area where your package improves things); gnatcoll treats arrays and objects completely differently, one without tagged types and the other with; simple types systematically have to be converted explicitly to a json object with Create;...

The difference as far as I can tell:

1. My JSON parser sits on top of table-driven parser with does all work. 
The implementation merely defines a table with JSON operations ("[]", 
"{}", ":").

2. Parts of the input object are allocated in an external pool. The pool 
allocation policy is strictly LIFO, so an arena can be used to free JSON 
object once consumed. IMO there is no reason to keep JSON objects any 
longer. No tagged objects used.

3. No mutable JSON objects and construction methods are provided as I 
see no reason for having them for such a primitive thing. It is far more 
easier just to output JSON stuff using Ada I/O facilities or 
Strings_Edit. The only less trivial thing is JSON-escaped strings. For 
them a procedure Put and function Image are provided.

> The GNATCOLL.JSON parser can be extended to support YAML (I have some local patches to support comments and optional quotes for strings, but not the indentation-based structures), but it would be nice if this was provided out of the box...

YAML looks a completely different thing. Any relation to JSON seems 
purely historical.

It would be no problem to write a table-driven parser for YAML, if there 
were demand.

I am kind of sceptical regarding YAML usefulness. Not that JSON has much 
merits either, but at least it is actively used with HTTP REST API and 
JavaScript.

YAML is like XML a wrong solution to a non-existent problem.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[Olivier Henley]

> Not that JSON has much merits either

I want to know more on why you think so? 

Since virtually every web 'distributed architecture' since 2010 decided to go 'string based', what could be more appropriate than Json? 

Is it the 'string based' approach that itches you? 

...

I know protobuf is serialization of structured primitive types and grpc handle these as a form of rpc over http2.

I know grpc is available for many languages and therefore makes a viable alternative to Json in an heterogeneous network of services.

I also know Ada has no implementation of protobuf and/or grpc.

Where do we go from there?

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-10-03 20:58, Olivier Henley wrote:
>> Not that JSON has much merits either
> 
> I want to know more on why you think so?
> 
> Since virtually every web 'distributed architecture' since 2010 decided to go 'string based', what could be more appropriate than Json?
> 
> Is it the 'string based' approach that itches you?

It is many things, not just one. Before talking about a solution I would 
like know the problem.

Why there should a data definition language there. Why this language 
must lack any decent language features? You could not parse it 
otherwise, then why a language again? Why it must pretend to be 
readable, while nobody ever reads it. Why it should be weakly typed. Why 
it should be non-portable (e.g. JSON numbers) etc.

> I know protobuf is serialization of structured primitive types and grpc handle these as a form of rpc over http2.
> 
> I know grpc is available for many languages and therefore makes a viable alternative to Json in an heterogeneous network of services.

RPC is not a good idea either. But even so, there is no reason to use 
something like JSON for marshaling RPC parameters.

> I also know Ada has no implementation of protobuf and/or grpc.

Well, Ada community has very limited resources available. It must treat 
such things pragmatically.

I would say, if you need communication do it in Ada. If you need 
interoperability with some legacy mess, OK, then there is a demand and 
one could consider some Ada bindings/adapter to that mess. So far nobody 
voiced it.

> Where do we go from there?

We sit and wait. Bad ideas tend to drown themselves in countless useless 
implementations, JSON, XML, YAML ... There is no rush. Next year 
somebody will invent something no less worse.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[Olivier Henley]

First I want to say that I totally agree with you.

> Before talking about a solution I would like know the problem.

The problem is that Ada is nowhere to be seen on things such as the Tiobe programming index and therefore communicating Ada-Ada does not make the cut for a lot of projects. There is just no public Ada API out there.

This fact, for sure, puts off a lot of adoption where many new projects probably have 'cloud communication' needs to non Ada services.

> OK, then there is a demand and one could consider some Ada bindings/adapter to that mess. So far nobody voiced it.

I think nobody voiced it because they are actively building business using another language that provides the 'fashionist' way of communicating. 

Its a chicken-egg problem and I am not sure we should sit and wait though.

Re: ANN: Simple components for Ada v.4.42

[Olivier Henley]

In essence what I am trying to say is that 'Ada is not in the game'.

Do not get me wrong, every company using Ada deserves uttermost respect but they are no Google, Apple, Microsoft etc of this world. 

Those are setting the rules, you like it or not (http2?), and if I can use an analogy, we are on the bench for the whole 'cloud' segment.

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-10-03 23:29, Olivier Henley wrote:
> In essence what I am trying to say is that 'Ada is not in the game'.

There are different games.

> Do not get me wrong, every company using Ada deserves uttermost respect but they are no Google, Apple, Microsoft etc of this world.

True, but Google, Apple and Microsoft are not companies. They are so big 
that in fact they are cartels.

> Those are setting the rules, you like it or not (http2?), and if I can use an analogy, we are on the bench for the whole 'cloud' segment.

That depends on what your company does. Usually the pressure to use 
something you do not like comes from the customer, not from Microsoft et al.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[J-P. Rosen]

Le 16/09/2019 à 08:58, briot.emmanuel@gmail.com a écrit :
> It would be nice if the Ada world did not have too many packages competing here.
Sure. See https://xkcd.com/927/ :-)

-- 
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00
http://www.adalog.fr

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-09-16 08:58, briot.emmanuel@gmail.com wrote:

> The GNATCOLL.JSON parser can be extended to support YAML (I have some local patches to support comments and optional quotes for strings, but not the indentation-based structures), but it would be nice if this was provided out of the box...

I forgot, there is an Ada YAML implementation:

    https://ada.yaml.io
    https://github.com/yaml/AdaYaml

Maybe I would do it rather naked without IMO unnecessary DOM, 
presentation etc, yet it is there!

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[onox]

On Monday, September 16, 2019 at 8:58:41 AM UTC+2, briot....@gmail.com wrote:
> On Monday, September 16, 2019 at 8:48:36 AM UTC+2, Dmitry A. Kazakov wrote:
> > The new version provides an implementation of JSON (RFC 7158)
> 
> 
> Hi Dmitry,
> I would be curious to know the differences compared to what GNATCOLL.JSON provides, if you know ? It would be nice if the Ada world did not have too many packages competing here.

Uh oh, I've written a JSON parser ([0]) too! Sorry :) It's quite fast, only about 1K SLOC, supports Ada 2012's iterator and indexing syntax, Apache 2.0 license, but it does not handle UTF-8 yet (patches welcome though)

> Various things which I find limiting in GNATCOLL.JSON: performance is not really good because there are a lot of memory allocations

I did some benchmarking of some JSON parsers using a 110 M large .json file from [1]:

- Parsers.JSON choked on it (I used Parsers.Multiline_Source.Stream_IO)

- GNATCOLL.JSON needs about ~ 17 seconds

- json-ada needs about ~ 3 seconds (should be ~ 1 second to be competitive with other languages, patches/advice welcome)

[0] https://github.com/onox/json-ada

[1] https://github.com/kostya/benchmarks#Json

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-10-04 01:53, onox wrote:

> Uh oh, I've written a JSON parser ([0]) too! Sorry :) It's quite fast, only about 1K SLOC, supports Ada 2012's iterator and indexing syntax, Apache 2.0 license, but it does not handle UTF-8 yet (patches welcome though)

Good to know, thanks.

>> Various things which I find limiting in GNATCOLL.JSON: performance is not really good because there are a lot of memory allocations
> 
> I did some benchmarking of some JSON parsers using a 110 M large .json file from [1]:

It do not find the JSON file there. Can you give a link how to download it?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[Olivier Henley]

On Friday, October 4, 2019 at 3:57:00 AM UTC-4, Dmitry A. Kazakov wrote:

> It do not find the JSON file there. Can you give a link how to download it?

you need to generate it using ruby 
https://github.com/kostya/benchmarks/tree/master/json

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-10-04 12:52, Olivier Henley wrote:
> On Friday, October 4, 2019 at 3:57:00 AM UTC-4, Dmitry A. Kazakov wrote:
> 
>> It do not find the JSON file there. Can you give a link how to download it?
> 
> you need to generate it using ruby
> https://github.com/kostya/benchmarks/tree/master/json

I see. The reason why this is not parsed is the stack size. The source 
contains a JSON array of 1000000 elements. In order to be parsed there 
should be possible to pass an array of 1M arguments. If I correctly 
remember it could be done like this:

    gprbuild ... -largs -Wl,--stack=0x8000000

How relevant such samples are is another question.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ANN: Simple components for Ada v.4.42

[Olivier Henley]

On your simple components documentation page we can read:

"A stack pool consists of contiguous segments allocated dynamically as necessary. The discriminants control the stack segments allocation policy. Initial_Size determines the initial default size of a newly allocated segment. If this size is less than the size of the object being allocated the default size is set to the object size multiplied to Items_Number."

Given this:

Arena : aliased Stack_Storage.Pool (200, 512); -- Pool to keep JSON object 
...
Data : constant JSON_Value := Parse (Input'Access, Arena'Access); -- Parse the file

From that, correct me if I am wrong, if I throw a json document (Input) bigger than 200*512, the stack pool will grow dynamically to accomodate?


   

Re: ANN: Simple components for Ada v.4.42

[Dmitry A. Kazakov]

On 2019-10-03 17:43, Olivier Henley wrote:
> On your simple components documentation page we can read:
> 
> "A stack pool consists of contiguous segments allocated dynamically as necessary. The discriminants control the stack segments allocation policy. Initial_Size determines the initial default size of a newly allocated segment. If this size is less than the size of the object being allocated the default size is set to the object size multiplied to Items_Number."
> 
> Given this:
> 
> Arena : aliased Stack_Storage.Pool (200, 512); -- Pool to keep JSON object
> ...
> Data : constant JSON_Value := Parse (Input'Access, Arena'Access); -- Parse the file
> 
>  From that, correct me if I am wrong, if I throw a json document (Input) bigger than 200*512, the stack pool will grow dynamically to accomodate?

Yes

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de