Re: GNAT/Ada95 Streams Performance Issue

Re: GNAT/Ada95 Streams Performance Issue

[Waldek Hebisch]

Warren W. Gay VE3WWG (ve3wwg@cogeco.ca) wrote:
: 3. The individual characters that make up Msg are written
:      out to stream S, using individual calls to Character'Write.

: Step # 3 is the performance killer.

: To help in this regard, APQ 1.2 (not released yet) now supports
: two new functions for performance reasons:

:       String_Output to substitute for String'Output
:       String_Input to substitute for String'Input

: These routines deal with the array of characters as
: one block of data.

: But this is really just a patch over part of a much
: larger problem.

: How have other people dealt with this type of streams
: performance issue?  Or do people generally avoid
: streams when performance is important?

Well, I just toyed with a simple cat program:

        loop
                Ada.Text_IO.Get_Line( Line, Last );
                Ada.Text_IO.Put_Line( Line(1..Last) );
        end loop;

Using GNAT (gcc 3.1.1), I get very bad performance (about 10 MB/s, raw 
system calls (in C) give 450MB/s). I looked at the sources, and while 
Put_Line essentialy outputs the block (it makes a copy and modifies it 
slightly) Get_Line works with single characters. The problem is that to 
cooperate nicely with C Ada has to go trough C stdio interface. Since Ada 
and C semantics differ Ada cannot use C block input. The problem is magnified 
becouse each C stdio function performs locking. 

--
                              Waldek Hebisch
hebisch@math.uni.wroc.pl    or hebisch@hera.math.uni.wroc.pl 

Re: GNAT/Ada95 Streams Performance Issue

[Robert Dewar]

hebisch@math.uni.wroc.pl (Waldek Hebisch) wrote in message news:<aj8j3t$bmn$1@panorama.wcss.wroc.pl>...

>         loop
>                 Ada.Text_IO.Get_Line( Line, Last );
>                 Ada.Text_IO.Put_Line( Line(1..Last) );
>         end loop;

This is nothing *like* a "raw cat" program, it has quite
subtle semantics that you likely do not intend :-)

The proper way to do a big copy is with Stream_IO. The
design of Text_IO is definitely not suitable for large
scale file operations.

Re: GNAT/Ada95 Streams Performance Issue

[Larry Hazel]

Waldek Hebisch wrote:
> Warren W. Gay VE3WWG (ve3wwg@cogeco.ca) wrote:
> : 3. The individual characters that make up Msg are written
> :      out to stream S, using individual calls to Character'Write.
> 
> : Step # 3 is the performance killer.
> 
> : To help in this regard, APQ 1.2 (not released yet) now supports
> : two new functions for performance reasons:
> 
> :       String_Output to substitute for String'Output
> :       String_Input to substitute for String'Input
> 
> : These routines deal with the array of characters as
> : one block of data.
> 
> : But this is really just a patch over part of a much
> : larger problem.
> 
> : How have other people dealt with this type of streams
> : performance issue?  Or do people generally avoid
> : streams when performance is important?
> 
> Well, I just toyed with a simple cat program:
> 
>         loop
>                 Ada.Text_IO.Get_Line( Line, Last );
>                 Ada.Text_IO.Put_Line( Line(1..Last) );
>         end loop;
> 
> Using GNAT (gcc 3.1.1), I get very bad performance (about 10 MB/s, raw 
> system calls (in C) give 450MB/s). I looked at the sources, and while 
> Put_Line essentialy outputs the block (it makes a copy and modifies it 
> slightly) Get_Line works with single characters. The problem is that to 
> cooperate nicely with C Ada has to go trough C stdio interface. Since Ada 
> and C semantics differ Ada cannot use C block input. The problem is magnified 
> becouse each C stdio function performs locking. 
> 
Somewhere around 1990, I wrote a simple file copy program using Sequential_IO on 
files of bytes.
This was using the Verdix Ada 83 compiler on a Sun.  I did this as a test 
because C people kept telling me how slow Ada was.  I wanted to see for myself. 
    It was consistently faster than the unix cp command.  I suspect there was a 
lot of buffering going on behind the scenes.

Larry

Re: GNAT/Ada95 Streams Performance Issue

[Robert Dewar]

Larry Hazel <lhhazel@otelco.net> wrote in message news:<3D585F63.6010205@otelco.net>...

> Somewhere around 1990, I wrote a simple file copy program 
> using Sequential_IO on 
> files of bytes.

Note that there is no portable way of doing this in either
Ada 83 or Ada 95. In practice, instantiating SIO for type
Character would probably work.

> This was using the Verdix Ada 83 compiler on a Sun.  I 
> did this as a test  because C people kept telling me how 
> slow Ada was.  I wanted to see for myself. 
> It was consistently faster than the unix cp command.

Hard to believe, and if true, simply a comment on a truly
appalling implementation of cp.

> I suspect there was a 
> lot of buffering going on behind the scenes.

That's not enough to account for this surprising result!

Re: GNAT/Ada95 Streams Performance Issue

[Larry Hazel]

Robert Dewar wrote:
> Larry Hazel <lhhazel@otelco.net> wrote in message news:<3D585F63.6010205@otelco.net>...
> 
> 
>>Somewhere around 1990, I wrote a simple file copy program 
>>using Sequential_IO on 
>>files of bytes.
> 
> 
> Note that there is no portable way of doing this in either
> Ada 83 or Ada 95. In practice, instantiating SIO for type
> Character would probably work.
> 
> 
>>This was using the Verdix Ada 83 compiler on a Sun.  I 
>>did this as a test  because C people kept telling me how 
>>slow Ada was.  I wanted to see for myself. 
>>It was consistently faster than the unix cp command.
> 
> 
> Hard to believe, and if true, simply a comment on a truly
> appalling implementation of cp.
> 
> 
>>I suspect there was a 
>>lot of buffering going on behind the scenes.
> 
> 
> That's not enough to account for this surprising result!

Robert,
After reading your reply and trying to remember what I did, I think the byte SIO 
  program was the first attempt and not very fast.  I vaguely remember doing 
something with direct io and large arrays of bytes for the program that was 
faster than cp.
Larry

Re: GNAT/Ada95 Streams Performance Issue

[Robert Dewar]

Larry Hazel <lhhazel@otelco.net> wrote in message news:<3D590D80.201@otelco.net>...
> Robert,
> After reading your reply and trying to remember what I did, I think the byte SIO 
>   program was the first attempt and not very fast.  I vaguely remember doing 
> something with direct io and large arrays of bytes for the program that was 
> faster than cp.
> Larry


That certainly makes more sense. The trouble is of course
that with Direct_IO you can't deal nicely with the last
partial block (you are also depending very much on impl
dependent choices in how Direct_IO works).

Stream_IO should have the speed advantage without these
disadvantages.

Re: GNAT/Ada95 Streams Performance Issue

[Lutz Donnerhacke]

* Robert Dewar wrote:
>That certainly makes more sense. The trouble is of course
>that with Direct_IO you can't deal nicely with the last
>partial block (you are also depending very much on impl
>dependent choices in how Direct_IO works).
>
>Stream_IO should have the speed advantage without these
>disadvantages.

Sorry, don't take this to serious:

...
   function copy (
     in_fd, out_fd : OS_File_T;
     offset        : access OS_Off_T;
     count         : OS_Size_T
   ) return OS_SSize_T;
   pragma Import(C, copy, "sendfile");
   offset : aliased OS_Off_T := OS_Off_T'First;
   send   : OS_SSize_T :=
     copy (in_fd, out_fd, offset'Access, OS_Size_T'Last);
begin
   if send = -1 then
      Put_Line ("Error ...");
   else
      Put_Line ("Copied" & OS_SSize_T'Image (send) & " Bytes.");
   end if;
end;   

Re: GNAT/Ada95 Streams Performance Issue

[Waldek Hebisch]

Robert Dewar (dewar@gnat.com) wrote:
: hebisch@math.uni.wroc.pl (Waldek Hebisch) wrote in message news:<aj8j3t$bmn$1@panorama.wcss.wroc.pl>...

: >         loop
: >                 Ada.Text_IO.Get_Line( Line, Last );
: >                 Ada.Text_IO.Put_Line( Line(1..Last) );
: >         end loop;

: This is nothing *like* a "raw cat" program, it has quite
: subtle semantics that you likely do not intend :-)

The goal is NOT to copy files, it is just a skeleton for a 
program which works with a file line-by-line. I know that 
the program mishandles long lines, and that Ada treats control 
chars differently then C. Is there anything more subtle going 
on? And is there a better standard way to read a line?

I find it somewhat disturbing that reading a line take more 
CPU time then simple processing. 
--
                              Waldek Hebisch
hebisch@math.uni.wroc.pl    or hebisch@hera.math.uni.wroc.pl 

Re: GNAT/Ada95 Streams Performance Issue

[Mike Brenner]

Hi,

An Ada program that copies a text file from standard input to standard output using streams would be a very useful example to post. 

I don't know how to do it because I don't know how to connect a TEXT_stream to standard input using the standard Ada stream packages.

Mike


Robert Dewar wrote:
> That certainly makes more sense. The trouble is of course
> that with Direct_IO you can't deal nicely with the last
> partial block (you are also depending very much on impl
> dependent choices in how Direct_IO works).
> 
> Stream_IO should have the speed advantage without these
> disadvantages.


Larry Hazel <lhhazel@otelco.net> wrote in message news:<3D590D80.201@otelco.net>...
>> ... something with direct io and large arrays of bytes 
>> for the program that was faster than cp.