gnatcol json vs network read

gnatcol json vs network read

[Stephen Leake]

I'm reading json over a network, with code like:

   function Check_Ack (DB : not null access Database) return   
      GNATCOLL.JSON.JSON_Value
   is
      use GNATCOLL.JSON;
      Msg : constant String := String (Network_String'Input (DB.Stream));
   begin
      if DB.Verbosity > 1  then
         Ada.Text_IO.Put_Line ("Remote: " & Msg);
      end if;

      declare
         Response : constant JSON_Value := Read (Msg);
      begin
         if Response.Get ("Status") /= Ack_Nack'Image (Ack) then
            raise SAL.Invalid_Operation with Response.Get ("Message");
         end if;
         return Response.Get ("Data");
      end;
   end Check_Ack;

Occasionally, this will throw INVALID_JSON_STREAM <data>:1:155: empty stream

Apparently this is because the network is sending a data packet with less then a whole JSON object; in the case I'm currently debugging, it's missing two closing }.

So I need to check Msg for complete syntax before calling Read. However, GNATCOLL.JSON does not provide a function for that, and event the lower level Read in gnatcol-json.adb raises the exception.

So I guess I'll write a {} counter, and wait for more data from the network if the first object is not complete.

Has anyone else dealt with this problem?

-- Stephe

Re: gnatcol json vs network read

[Olivier Henley]

> Occasionally, this will throw INVALID_JSON_STREAM <data>:1:155: empty stream

side note:

make sure the source of your json stream does not contain capital letters for keys. If you have such anomalies, perform a to_lower() before trying to parse/extract the json. It happened to me before and the error message was not helping... the kind you got.

olivier

Re: gnatcol json vs network read

[Stephen Leake]

On Friday, December 14, 2018 at 9:50:10 AM UTC-8, Stephen Leake wrote:

> 
> So I guess I'll write a {} counter, and wait for more data from the network if the first object is not complete.

Arg. The problem turned out to be the client (an Android app) is sending ' as UTF-8 16#e28099# instead of ASCII ', throwing off the byte count. sigh.

There should still be an incomplete object problem, but apparently I haven't actually seen it yet.

-- Stephe

Re: gnatcol json vs network read

[Per Sandberg]

Well this all depends on the definition of Network_String'Input and 
Network_String'Output.

Will Network_String'Output always send the whole string in such a format 
that Network_String'Input is able to reconstruct it regardless of 
fragmentation on the transport.

/P


On 12/14/18 6:50 PM, Stephen Leake wrote:
> I'm reading json over a network, with code like:
> 
>     function Check_Ack (DB : not null access Database) return
>        GNATCOLL.JSON.JSON_Value
>     is
>        use GNATCOLL.JSON;
>        Msg : constant String := String (Network_String'Input (DB.Stream));
>     begin
>        if DB.Verbosity > 1  then
>           Ada.Text_IO.Put_Line ("Remote: " & Msg);
>        end if;
> 
>        declare
>           Response : constant JSON_Value := Read (Msg);
>        begin
>           if Response.Get ("Status") /= Ack_Nack'Image (Ack) then
>              raise SAL.Invalid_Operation with Response.Get ("Message");
>           end if;
>           return Response.Get ("Data");
>        end;
>     end Check_Ack;
> 
> Occasionally, this will throw INVALID_JSON_STREAM <data>:1:155: empty stream
> 
> Apparently this is because the network is sending a data packet with less then a whole JSON object; in the case I'm currently debugging, it's missing two closing }.
> 
> So I need to check Msg for complete syntax before calling Read. However, GNATCOLL.JSON does not provide a function for that, and event the lower level Read in gnatcol-json.adb raises the exception.
> 
> So I guess I'll write a {} counter, and wait for more data from the network if the first object is not complete.
> 
> Has anyone else dealt with this problem?
> 
> -- Stephe
> 

Re: gnatcol json vs network read

[Stephen Leake]

On Friday, December 14, 2018 at 1:58:05 PM UTC-8, Per Sandberg wrote:
> Well this all depends on the definition of Network_String'Input and 
> Network_String'Output.
> 
> Will Network_String'Output always send the whole string in such a format 
> that Network_String'Input is able to reconstruct it regardless of 
> fragmentation on the transport.

Sigh, you are right. I use this for 'Input:

      Str      : Stream_Element_Array (1 .. 8);
      Str_Last : Stream_Element_Offset;
   begin
      Read (Stream.all, Str, Str_Last);
      if Str_Last < Str'Last then
         raise Ada.IO_Exceptions.End_Error;
      end if;

So 'Input either returns all the bytes sent, or raises End_Error.

Thanks for the nudge :).

-- Stephe