Re: Streams and Concurrency

Re: Streams and Concurrency

[David Botton]

Would you be able to post a small sample demonstrating this.

David Botton

James S. Rogers wrote in message <76c3tv$acs@bgtnsc02.worldnet.att.net>...
>A question was recently raised about task safe use of streams.

ETC....

Re: Streams and Concurrency

[dennison]

In article <76c3tv$acs@bgtnsc02.worldnet.att.net>,
  "James S. Rogers" <jimmaureenrogers@worldnet.att.net> wrote:
> would be the typical Ada 83 approach.  The more modern, and
> more efficient approach in Ada 95 is to have encapsulate the stream
> in a protected object.  All access will be correctly and effectively
> controlled by the protected object mechanism.

That's the first thing I thought of too. This solution protects the stream
itself from corruption just fine. The problem is that it provides absolutely
no protection for the *data* in the stream. If I do a 'Write of a record,
that gets translated into *several* "Write" calls to the stream, not just
one. There's nothing stopping another task from making a call to the stream's
Write subprogram in the middle of my record's 'Write. There's nothing
stopping the stream service task from calling the stream's Read procedure in
the middle of my 'Write either. Making the stream a proteced object does
nothing for me here.

The design of streams makes this problem basicly intractable at any level but
the client level. It can't be done just by adding code to the stream
implementation.

The sad thing is that this isn't even the biggest problem. The fun *really*
starts when you want to read the data back in safely without blocking...

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dennison]

In article <76cat4$2ldc$1@news.gate.net>,
  "David Botton" <dbotton@hotmail.com> wrote:
> Would you be able to post a small sample demonstrating this.
>
> David Botton
>
> James S. Rogers wrote in message <76c3tv$acs@bgtnsc02.worldnet.att.net>...
> >A question was recently raised about task safe use of streams.
>
> ETC....
>
>

Actually, I already have code that does that and probably could post it if
you really want to see it. Unfortunately, it doesn't really solve the problem
unless don't mind interleaved data on your stream. Also, if your reader task
does any kind of extra blocking or protocol on the data, he could concievably
end up doing that on every single non-composite field that was in your
structure. (Ewww!)

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[James S. Rogers]

Following is a modest example of what I mean about
encapsulating a stream in a protected object.

The example uses Ada.Streams.Stream_Io for simplicity.  You can
reasonably substitute your own stream implementation as needed.

Note that the file name for the log file is passed as a discriminant to
the protected type.

I believe this solution solves the interleaving problem.  The read and
write operations will be atomic.  The write operation will not complete
until all 'write operations for the tagged type complete.  The key is
that the task calling the protected write and read operations is
ignorant of the use of a stream.  No stream operations occur outside
the protection of the protected type.

--------------------------------------------------------------------------
-- Simple Example of using streams in a protected object for access by
-- multiple tasks.
--------------------------------------------------------------------------
   package Type_Hierarchy is
      type Base_Type is tagged private;

   -- Add dispatching operations to the Base_type

      type First_Child is new Base_Type with private;

   -- Add or override operations for First_Child

   private

      type Base_Type is tagged record
            Date : String(1..10);
            Time : String(1..8);
         end record;

      type First_Child is new Base_type with record
            Voltage : Float;
         end record;
   end Type_Hierarchy;


   with Type_Hierarchy;
   use Type_Hierarchy;
   with Ada.Steams.Stream_Io;

   package Multi_Task_Logging is

      protected type Log_Stream (Log_Name : String) is
         Procedure Write(Item : in Base_Type'Class);
         Entry Read(Item : out Base_Type'Class);
      private
         File : Ada.Streams.Stream_Io.File_Type;
         Log_Access : Ada.Streams.Stream_Io.Stream_Access :=
            Ada.Streams.Stream_Io.Stream(File => File);
      end Log_Stream;
   end Multi_Tasking_Logging;

   package body Multi_Tasking_Logging is

      protected body Log_Stream is
         Procedure Write(Item : in Base_Type'Class) is
         begin
            if not Ada.Streams.Stream_Io.Is_Open(File) then
               Ada.Streams.Stream_Io.Open(File => File,
                                          Name => Log_Name,
                                          Mode =>
Ada.Streams.Stream_Io.Append_File);
            else
               Ada.Streams.Stream_Io.Reset(File => File,
                                           Mode =>
Ada.Streams.Stream_Io.Append_File);
            end if;
            Base_Type'Class'Write(Stream => Log_Access, Item => Item);
         end Write;

         Entry Read(Item : out Base_Type'Class) when not
         Ada.Streams.Stream_Io.End_of_File(File) is
         begin
            if not Ada.Streams.Stream_Io.is_Open(File) then
               Ada.Streams.Stream_Io.Open(File => File,
                                          Name => Log_Name,
                                          Mode =>
Ada.Streams.Stream_Io.In_File);
            else
               Ada.Streams.Stream_Io.Reset(File => File,
                                           Mode =>
Ada.Streams.Stream_Io.In_File);
            end if;
            Base_Type'Class'Read( Stream => Log_Access, Item => Item);
         end Read;
      end Log_Stream;

   End Multi_Tasking_Logging;

-----------------------------------

Jim Rogers
Colorado Springs, Colorado USA

Re: Streams and Concurrency

[Tucker Taft]

It seems clear that the stream attributes do not directly support the 
logging from multiple tasks, with interleaving occurring only
between "top-level" objects, as desired by this application.
Furthermore, they don't support reading without waiting.

However, it should be simple enough to build a generic package
which provides the desired functionality.

Each current "top-level" use of a stream attribute in the application
would be changed to be a call on a "read/write" or "input/output" routine of an 
appropriate instance of this generic package.  Such a routine would 
acquire a lock on the stream, invoke the corresponding stream attribute,
and release the lock.  

Furthermore, to support non-blocking read, you would need to add some 
kind of marker into the stream itself, I suspect, to be placed between 
"top-level" objects in the stream.  Then, when you call the non-blocking
"read" or "input" routine of the instance of this generic package,
it would call a routine of the underlying stream which would indicate
whether there is an "entire" top-level object already buffered up.
If not, the non-blocking read/input routine of the generic would
return a "not available" indicator.  If there is an entire top-level
object buffered up, the read/input routine would use the corresponding
stream attribute to read the data from the stream, confident that it
wouldn't block or get an end-of-file indication during the read.

--
-Tucker Taft   stt@averstar.com   http://www.averstar.com/~stt/
Technical Director, Distributed IT Solutions  (www.averstar.com/tools)
AverStar (formerly Intermetrics, Inc.)   Burlington, MA  USA

Re: Streams and Concurrency

[dennison]

In article <F4sFr8.HHG.0.-s@inmet.camb.inmet.com>,
  stt@houdini.camb.inmet.com (Tucker Taft) wrote:
> It seems clear that the stream attributes do not directly support the
> logging from multiple tasks, with interleaving occurring only
> between "top-level" objects, as desired by this application.
> Furthermore, they don't support reading without waiting.
>
> However, it should be simple enough to build a generic package
> which provides the desired functionality.
>
> Each current "top-level" use of a stream attribute in the application
> would be changed to be a call on a "read/write" or "input/output" routine of
an
> appropriate instance of this generic package.  Such a routine would
> acquire a lock on the stream, invoke the corresponding stream attribute,
> and release the lock.

Ahhh. Now that might do the trick. Thanks a lot! It hadn't occurred to me
that generics can call the attributes too. Of course if your clients would
prefer the attribute interface this is a problem. But I don't see that being
too much of an issue.

> Furthermore, to support non-blocking read, you would need to add some
> kind of marker into the stream itself, I suspect, to be placed between
> "top-level" objects in the stream.  Then, when you call the non-blocking
> "read" or "input" routine of the instance of this generic package,

Yes, exactly. The problem is there's no way for the reader to know where
"between" is without first trying to read the entire object. Even within the
same data structure, the size could change depending on the value of a tag or
variant (or some other user-defined critera). The only way I could see to do
it is to have the writer put the length of the object in the stream *at the
front* of the object *after* the object is written. Doing this requires some
kind of direct access into the stream, and a way of blocking reads until the
whole object is written. Otherwise the reader could read that size data out
of the stream before the writer has a chance to fill it in.

My problem with doing the above was that it also requires the whole-object
locking of the stream (which you  have just explained one way to handle).

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dennison]

In article <76dn7b$a35@bgtnsc03.worldnet.att.net>,
  "James S. Rogers" <jimmaureenrogers@worldnet.att.net> wrote:
> Following is a modest example of what I mean about
> encapsulating a stream in a protected object.
>
> I believe this solution solves the interleaving problem.  The read and
> write operations will be atomic.  The write operation will not complete
> until all 'write operations for the tagged type complete.  The key is
> that the task calling the protected write and read operations is
> ignorant of the use of a stream.  No stream operations occur outside
> the protection of the protected type.

I see. You (probably Robert too) were talking about wrapping the client's
'Read and 'Write calls themselves. So what you have here is a multi-taksing
logging package, rather than a multitasking stream. I was thinking about a
pure stream implementation. There appears to be no way to write the stream
itself to handle this. Perhaps that's not as bad as I first thought it was...

But this solution only works for objects that are part of the same class
hiearchy. That's too restrictive. I'd want to be able to use any type. It
sounds like Tuck's generic suggestion is the way to go.

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[James S. Rogers]

dennison@telepath.com wrote in message <76e4rg$bsr$1@nnrp1.dejanews.com>...
>I see. You (probably Robert too) were talking about wrapping the client's
>'Read and 'Write calls themselves. So what you have here is a multi-taksing
>logging package, rather than a multitasking stream. I was thinking about a
>pure stream implementation. There appears to be no way to write the stream
>itself to handle this. Perhaps that's not as bad as I first thought it
was...
>
>But this solution only works for objects that are part of the same class
>hiearchy. That's too restrictive. I'd want to be able to use any type. It
>sounds like Tuck's generic suggestion is the way to go.


Yes, that is a distinct restriction.  On the other hand, if you adopted
a programming style from another language, such as Java, you could
have all your data types inherit from an abstract null tagged record.
The result would be some more overhead, but a very generally
useful approach to using streams with multiple tasks.  It would also
simplify the problem of reading and understanding the log file
contents.  The abstract base type could have a Print or Display
procedure which must be implemented for each type in the
hierarchy. Displaying the contents of the file would then rely on the
dispatching mechanism of tagged types.

Jim Rogers
Colorado Springs, Colorado USA

Re: Streams and Concurrency

[Robert I. Eachus]

In article <76dgdl$qle$1@nnrp1.dejanews.com> dennison@telepath.com writes:

 > That's the first thing I thought of too. This solution protects the
 > stream itself from corruption just fine. The problem is that it
 > provides absolutely no protection for the *data* in the stream. If
 > I do a 'Write of a record, that gets translated into *several*
 > "Write" calls to the stream, not just one. There's nothing stopping
 > another task from making a call to the stream's Write subprogram in
 > the middle of my record's 'Write. There's nothing stopping the
 > stream service task from calling the stream's Read procedure in the
 > middle of my 'Write either...

    This is wrong.  The encapsulation as illustrated by Jim Rogers
does it right.  The protected object doesn't wrap every call to 'Read
or 'Write, just the highest level calls.  Even if one of those calls
causes many lower level calls all happen inside a single call to the
protected object.

 > Making the stream a proteced object does nothing for me here.
    
    This is trying to solve the wrong problem.  Putting calls to the
protected action inside 'Read or 'Write can't work right.  You have to
have a single protected object which encapsulates the particular
stream.  If you need to read and write different types of values to
the same stream, you either have to derive them all from a single
tagged type, or have several sets of read and write entries to the
protected type, one set for each type of object.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Streams and Concurrency

[Robert I. Eachus]

In article <F4sFr8.HHG.0.-s@inmet.camb.inmet.com> stt@houdini.camb.inmet.com (Tucker Taft) writes:

 > Furthermore, to support non-blocking read, you would need to add some 
 > kind of marker into the stream itself, I suspect, to be placed between 
 > "top-level" objects in the stream.  Then, when you call the non-blocking
 > "read" or "input" routine of the instance of this generic package,
 > it would call a routine of the underlying stream which would indicate
 > whether there is an "entire" top-level object already buffered up.
 > If not, the non-blocking read/input routine of the generic would
 > return a "not available" indicator.  If there is an entire top-level
 > object buffered up, the read/input routine would use the corresponding
 > stream attribute to read the data from the stream, confident that it
 > wouldn't block or get an end-of-file indication during the read.

    Since the non-blocking read would have to do read ahead to find
the marker, the marker doesn't really add anything.  So you really
only need two sets of entries, one which does blocking read and
writes, and one set which is non-blocking.  HOWEVER, the routines
underlying the Read operations for interprocessor communication and
I/O streams will have to be defined in terms of Stream_Elements,
because you will have to not only do read ahead, but you will have to
save the read ahead material in another (in memory) stream.

    Note that this is an somewhat special pattern.  It is only of
interest in the presence of tasking and either multiple processors or
the like.  And while it can easily be written in portable Ada 95, it
is one where low-level implementation support can significantly
improve performance.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Streams and Concurrency

[Tucker Taft]

Robert I. Eachus (eachus@spectre.mitre.org) wrote:
: In article <F4sFr8.HHG.0.-s@inmet.camb.inmet.com> stt@houdini.camb.inmet.com (Tucker Taft) writes:

:  > Furthermore, to support non-blocking read, you would need to add some 
:  > kind of marker into the stream itself, I suspect, to be placed between 
:  > "top-level" objects in the stream.  Then, when you call the non-blocking
:  > "read" or "input" routine of the instance of this generic package,
:  > it would call a routine of the underlying stream which would indicate
:  > whether there is an "entire" top-level object already buffered up.
:  > If not, the non-blocking read/input routine of the generic would
:  > return a "not available" indicator.  If there is an entire top-level
:  > object buffered up, the read/input routine would use the corresponding
:  > stream attribute to read the data from the stream, confident that it
:  > wouldn't block or get an end-of-file indication during the read.

:     Since the non-blocking read would have to do read ahead to find
: the marker, the marker doesn't really add anything.  So you really
: only need two sets of entries, one which does blocking read and
: writes, and one set which is non-blocking.  

Perhaps I need to explain more.  There are *three* kinds of "read"s in
this picture:

   1) A "read" routine that is part of the generic package.
      This is the non-blocking one.  It reads "top-level" objects
      only.

   2) A "read" attribute of the (formal) type (plus all of the
       "read" attributes of the subcomponent types).

   3) The (dispatching) read procedure of the stream type.
      This reads stream element arrays.

A single call on (1) can result will result in a call of the
Read attribute (2), which will then result, at least implicitly,
in calls on the Read attributes of the subcomponents.  Ultimately,
one or more calls on (3) will occur.

To support non-blocking (1), the stream type will need to be 
augmented with the notion of a marker of some sort to delineate 
"top-level" objects, and a procedure (let's call it 
"Lock_If_OK_To_Read") which can check whether an entire top-level 
object has been read in, by seeing whether an end-object marker is in 
the buffer, and if so, lock the stream object (all in a non-blocking 
fashion) and return an indication of success.  

If this "Lock_If_OK_To_Read" indicates success, then the non-blocking
read (1) routine will turn around and call the read (2) attribute,
which will in turn result in one or more calls to the stream read (3) 
routine.

It is presumably possible for the lock-if-OK-to-read routine
to do non-blocking read-ahead, since it "knows" what kind of
stream device it is dealing with.  However, the read (2) attribute
cannot do non-blocking read-ahead, since it knows nothing about
the underlying stream type.  Furthermore, the expectation is that
the system-provided read attribute(s) is (are) being used.
Hence, before read (1) invokes read (2), it must use the
lock-if-OK-to-read routine.  The generic package routines would 
require a stream of a type that supports this additional procedure.

: ...
: 					Robert I. Eachus

--
-Tucker Taft   stt@averstar.com   http://www.averstar.com/~stt/
Technical Director, Distributed IT Solutions  (www.averstar.com/tools)
AverStar (formerly Intermetrics, Inc.)   Burlington, MA  USA

Streams and Concurrency

[James S. Rogers]

A question was recently raised about task safe use of streams.
A stream is just as safe in a concurrent design as any other shared
resource.  That is, unless it is properly protected, it is not at all safe.
The same may be said of shared memory, or shared files.

There are two classical solutions to this problem.  The first is to
elminate the shared resource, and have a single, unshared resource
for each task.  The second is to protect the shared resource by having
only one task at a time access the resource.

Classical database engines control concurrent access to the
database by forcing all tasks to access the database through a
single database engine.  The same sort of thing can be achieved
in Ada in a couple of ways.  One way is to have all tasks send
messages to a single controlling task through a rendezvous.  This
would be the typical Ada 83 approach.  The more modern, and
more efficient approach in Ada 95 is to have encapsulate the stream
in a protected object.  All access will be correctly and effectively
controlled by the protected object mechanism.

When you use a protected object the "read" and "write" procedures
or entries must have a parameter of a class-wide type.  The
class-wide type parameter will allow the protected object to
read and write a wide range of otherwise heterogeneous data items
to and from the stream.  The stream may simply be an instance of
Ada.Streams.Stream_Io, or it may be a custom-made stream of
your own choosing.  The implementation will be irrelevant to the
tasks accessing the protected object.  It will also be irrelevant to the
protected object itself.

Jim Rogers
Colorado Springs, Colorado USA

Re: Streams and Concurrency

[robert_dewar]

Well I wrote my previous message before reading all the
followups, but I see they basically all agree with what
I suggested, namely that it makes more sense to create
the logging abstraction at a higher level.

Clearly there is no conceivable way for the implementation
of the streams themselves to provide automatic locking, so
the thought that they should seems fundamentally flawed.
The stream implementation *could* provide the primitives
so that attribute implementations could easily provide the
necessary locking, but I still think it makes sense to
abstract at a higher level.


-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[Jean-Pierre Rosen]

James S. Rogers a �crit dans le message
<76dn7b$a35@bgtnsc03.worldnet.att.net>...
>Following is a modest example of what I mean about
>encapsulating a stream in a protected object.
>[snip]

Unfortunately, all IO operations are potentially blocking, and
therefore not allowed from within a protected operation.
To get the effect of mutual exclusion for procedures requiring
potentially blocking operations, have a look at package Protection
available from Adalog's components page
(http://perso.wanadoo.fr/adalog/compo2.htm)
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://perso.wanadoo.fr/adalog

Re: Streams and Concurrency

[dennison]

In article <EACHUS.98Dec30172549@spectre.mitre.org>,
  eachus@spectre.mitre.org (Robert I. Eachus) wrote:
> In article <F4sFr8.HHG.0.-s@inmet.camb.inmet.com> stt@houdini.camb.inmet.com
(Tucker Taft) writes:
>
>  > Furthermore, to support non-blocking read, you would need to add some
>  > kind of marker into the stream itself, I suspect, to be placed between
>  > "top-level" objects in the stream.  Then, when you call the non-blocking

>     Since the non-blocking read would have to do read ahead to find
> the marker, the marker doesn't really add anything.  So you really
> only need two sets of entries, one which does blocking read and
> writes, and one set which is non-blocking.  HOWEVER, the routines
> underlying the Read operations for interprocessor communication and
> I/O streams will have to be defined in terms of Stream_Elements,
> because you will have to not only do read ahead, but you will have to
> save the read ahead material in another (in memory) stream.

Ahhh. I had thought of that approach. I discarded it because I'm talking
real-time processes here. I don't have time to go copying large chuncks of
data around in the context of the real-time process. It does solve the
problem of handling objects w/ limited private fields, though.

>     Note that this is an somewhat special pattern.  It is only of
> interest in the presence of tasking and either multiple processors or
> the like.  And while it can easily be written in portable Ada 95, it
> is one where low-level implementation support can significantly
> improve performance.

"Special" perhaps, but not all that odd. It can occur in *any* tasking
program. In my 10 years of Ada use I've only worked on two programs that had
no tasks. It doesn't take a multiprocessor to cause a task switch in the
middle of a non-blocking operation. If you have tasks with different
priorites it can happen. If your scheduling policy supports any kind of time
slicing it can happen. Plus if the stream's Write procedure performs any I/O,
its possible that a 'Write itself is a blocking operation.


--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dewar]

In article <76fe92$46c$1@platane.wanadoo.fr>,
  "Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> wrote:
> Unfortunately, all IO operations are potentially
> blocking, and therefore not allowed from within a
> protected operation. To get the effect of mutual
> exclusion for procedures requiring potentially blocking
> operations, have a look at package Protection
> available from Adalog's components page
> (http://perso.wanadoo.fr/adalog/compo2.htm)


To me the abstraction violation that comes from the
restriction on blocking operations in protected objects
is an intolerable concession to one particular low level
model of implementation, which in fact as far as I know
virtually no Ada 95 compiler uses. I think that this
optimization should have been achieved with a restrictions
pragma

   pragma Restrictions (No_Blocking_Operations, type-name);

or some such,

In any case, GNAT guarantees that blocking operations CAN
be done within protected objects, of course you may get
deadlock if you do obviously bad things, but certainly
doing IO within a protected object is not such a bad thing,
and this will work fine. We have many customers who depend
on this guarantee.

Yes, yes, of course this is potentially non-portable, but
it makes protected objects enormously more useful. Remember
that there are two advantages of protected objects:

1) possibility of more efficient implementation (I say
possibility here because in fact there are many
complications which result in less efficiency than you
might have hoped for in the general case, and in any
case some cases of agent tasks can be optimized in a
corresponding efficient manner.

2) preferable abstraction to a task in some situations

For my taste 2) is much more important than 1), and the
restriction on blocking operations badly damages the use
of 2).

We have had more than one user who was plain incredulous
on being informed of this restriction, and indeed it was
in talking with one of these users that we decided to
guarantee that GNAT does NOT have this restriction.

If we ever find a real need to implement this restriction,
for efficiency purposes or for any other purpose, we can
add a restrictions identifier.

By the way, another way around this is to do the actual
IO with an interface to C. There is nothing in the RM that
says that a C routine doing I/O is potentially blocking in
the formal sense in the RM (remember that the notion of
potentially blocking is not isomorphic to actual blocking,
there can be blocking operations that are not potentially
blocking, and vice versa).

So an implementation is free to define whether interfacing
to C I/O routines is or is not blocking, and for sure a
compiler cannot in fact check, so this is likely to work
on most Ada 95 compilers.

Robert Dewar
Ada Core Technologies

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dewar]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2393 bytes --]

In article <76fe92$46c$1@platane.wanadoo.fr>,
  "Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> wrote:
>
> James S. Rogers a �crit dans le message
> <76dn7b$a35@bgtnsc03.worldnet.att.net>...
> >Following is a modest example of what I mean about
> >encapsulating a stream in a protected object.
> >[snip]
>
> Unfortunately, all IO operations are potentially
> blocking, and therefore not allowed from within a
> protected operation.
> To get the effect of mutual exclusion for procedures
> requiring potentially blocking operations, have a look at
> package Protection available from Adalog's components
> page (http://perso.wanadoo.fr/adalog/compo2.htm)

By the way, JPR here seems to be making the mistake of
assuming that stream = stream_io, a common confusion. The
original question here related to a particular
implementation of streams that you have not seen, and you
do not know if it does any potentially blocking operations.

Yes, it had to do with file logging, but that does not
mean that it does I/O. For example, a perfectly reasonable
organization is to have a stream implementation that fills
a buffer that is part of a protected type, and a consumer
task that empties the buffer and does I/O on it. This
organization violates absolutely nothing with regards to
doing potentially blocking operations.

In Ada 95, if you are following the official design, every
single abstraction must be labeled in its spec as to
whether it does or does not do PB operations, so that you
know if it can be called from a protected operation. Yes,
that's frightful from an abstractrion point of view, but
that's the way the language is, due to the excessive
influence of one particular efficiency-obsessed specific
design for possible implementation of protected types.

A given implementation of streams (for logging or any
other purpose) may or may not do PB operations, and that
must be part of its spec.

As I commented before, we have essentially extended the
language for GNAT by defining that the implementation
dependent behavior for PB operations, i.e. the behavior
of the compiler for this bounded error, is NOT to reject
the PB operation, but to go ahead and do it, with the
expected consequences.

Robert Dewar
Ada Core Technologies

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[robert_dewar]

In article <76g0nu$nsq$1@nnrp1.dejanews.com>,
  dennison@telepath.com wrote:
> "Special" perhaps, but not all that odd. It can occur in
> *any* tasking program. In my 10 years of Ada use I've
> only worked on two programs that had no tasks. It doesn't
> take a multiprocessor to cause a task switch in the
> middle of a non-blocking operation. If you have tasks
> with different priorites it can happen. If your
> scheduling policy supports any kind of time slicing it
> can happen. Plus if the stream's Write procedure performs
> any I/O, its possible that a 'Write itself is a blocking
> operation.

Note that of course if you have protected objects, it is
not generally possible for a higher priority task to
interrupt in a problematic way, that is what ceiling
priorities are all about. And of course with
FIFO_Within_Priorities, time slicing is forbidden for
very good reasons!

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dennison]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

In article <76g9ds$ura$1@nnrp1.dejanews.com>,
  dewar@gnat.com wrote:
> In article <76fe92$46c$1@platane.wanadoo.fr>,
>   "Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> wrote:
> >
> > James S. Rogers a �crit dans le message
> > <76dn7b$a35@bgtnsc03.worldnet.att.net>...
> > >Following is a modest example of what I mean about
> > >encapsulating a stream in a protected object.
> > >[snip]
> >
> > Unfortunately, all IO operations are potentially
> > blocking, and therefore not allowed from within a
> > protected operation.
> > To get the effect of mutual exclusion for procedures
> > requiring potentially blocking operations, have a look at
> > package Protection available from Adalog's components
> > page (http://perso.wanadoo.fr/adalog/compo2.htm)
>
> By the way, JPR here seems to be making the mistake of
> assuming that stream = stream_io, a common confusion. The
> original question here related to a particular
> implementation of streams that you have not seen, and you
> do not know if it does any potentially blocking operations.

Quite true. What started all this is that I was trying to use streams as a
typless buffer between two tasks. One of these tasks is Real-Time, and *can't*
wait for another task or I/O (thus the buffer).

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[dennison]

In article <76g9jh$us1$1@nnrp1.dejanews.com>,
  robert_dewar@my-dejanews.com wrote:
> In article <76g0nu$nsq$1@nnrp1.dejanews.com>,
>   dennison@telepath.com wrote:
> > "Special" perhaps, but not all that odd. It can occur in
> > *any* tasking program. In my 10 years of Ada use I've
> > only worked on two programs that had no tasks. It doesn't
> > take a multiprocessor to cause a task switch in the
> > middle of a non-blocking operation. If you have tasks
> > with different priorites it can happen. If your
> > scheduling policy supports any kind of time slicing it
> > can happen. Plus if the stream's Write procedure performs
> > any I/O, its possible that a 'Write itself is a blocking
> > operation.
>
> Note that of course if you have protected objects, it is
> not generally possible for a higher priority task to
> interrupt in a problematic way, that is what ceiling
> priorities are all about. And of course with
> FIFO_Within_Priorities, time slicing is forbidden for
> very good reasons!

True. Of course if we move our system to a dual Pentium architecture, that
won't help. Folks here may want to dismiss that as an odball setup, but your
days of being able to do that are numbered.

--
T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Ada-G, was Re: Streams and Concurrency

[Tom Moran]

>In any case, GNAT guarantees that blocking operations CAN
>be done within protected objects, 
>...
>we decided to
>guarantee that GNAT does NOT have this restriction
"Many projects used their own languages or special variations of
existing languages.  This virtually locked the (DOD) into contracting
the original developers of a system for all maintenance work arising
during the life of that system."  Cohen, "Ada as a Second Language",
1986, p. 2 on the history of Ada.
   As a person who spends a fair amount of time testing portability of
new code, and working around the bugs and limitations of 5 different
"Ada 95" compilers, I'd suggest that anyone thinking of locking
themselves into a vendor-specific "special variations of existing
languages" look very carefully before they leap.

Re: Streams and Concurrency

[robert_dewar]

In article <76gecs$2sq$1@nnrp1.dejanews.com>,
  dennison@telepath.com wrote:
> True. Of course if we move our system to a dual Pentium
> architecture, that won't help. Folks here may want to
> dismiss that as an odball setup, but your
> days of being able to do that are numbered.


This is a misunderstanding. Time slicing is STILL not
allowed in a multi-processor environment, and the ceiling
priority protocol guarantees that a protected operation
cannot be interrupted by another higher priority task
attempting to enter the protected region, even in the
multi-processing case.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Ada-G, was Re: Streams and Concurrency

[Larry Kilgallen]

In article <76hqhk$56h$1@nnrp1.dejanews.com>, dewar@gnat.com writes:

> However, many real world projects are in fact pretty much
> committed to a single compiler, and even those that don't
> think they are often are. Indeed, consider this case, the
> user in question might never have even realized they were
> doing something that was not legitimate if we had not
> pointed it out. A lot of code depends on erroneous or
> implementation dependent constructs without realizing it.

DEC Ada for VMS, at least on Alpha, produces a "non-portability"
summary at the end of the listing. I don't know if it gets down
to this level of detail, but that sounds like the right approach
for indicating dependence on a particular implementation.  Of
course this might be viewed by a compiler vendor as having some
negative business impact as well as some positive business impact,
and that probably moves it down a few notches on the priority
scale.

Larry Kilgallen

Re: Ada-G, was Re: Streams and Concurrency

[Brian Rogoff]

On Thu, 31 Dec 1998, Tom Moran wrote:
>    As a person who spends a fair amount of time testing portability of
> new code, and working around the bugs and limitations of 5 different
> "Ada 95" compilers, I'd suggest that anyone thinking of locking
> themselves into a vendor-specific "special variations of existing
> languages" look very carefully before they leap.

I agree completely, but the message I get from this interesting discussion
is that vendors should agree on Ada fixes (based on customer input amongst 
other things) a bit faster than once a decade.

-- Brian

Re: Ada-G, was Re: Streams and Concurrency

[dewar]

In article <368bdf3c.3097724@news.pacbell.net>,
  tmoran@bix.com (Tom Moran) wrote:
> As a person who spends a fair amount of time testing
> portability of new code, and working around the bugs and
> limitations of 5 different "Ada 95" compilers, I'd
> suggest that anyone thinking of locking themselves into a
> vendor-specific "special variations of existing
> languages" look very carefully before they leap.


That of course is good advice. It is worth emphasizing that
the particular case here was a matter of a large customer
insisting that

(a) they absolutely needed this feature
(b) all versions of GNAT seemed to work the way they
    expected.
(c) they would not be able to use any version of GNAT that
    did not work that way.

Most certainly, they realized that they were committing
themselves to compilers that worked this way, but they
took the position that any compiler not working this way
was broken and would not be acceptable for use.

Tom is in the business of writing portable library code
whose value depends on its being able to run with many
different Ada compilers, and there of course, portability
is paramount.

However, many real world projects are in fact pretty much
committed to a single compiler, and even those that don't
think they are often are. Indeed, consider this case, the
user in question might never have even realized they were
doing something that was not legitimate if we had not
pointed it out. A lot of code depends on erroneous or
implementation dependent constructs without realizing it.

Actually one of the most significant cases of dependence
on particular compilers arises with the use of specialized
library packages. Certainly it is a good thing for
compilers to provide useful stuff, but it can be a trap if
portability turns out to be important. Note that this is
particularly the case if the specs of the routines in
question are protected by copyright.

Robert Dewar
Ada Core Technologies

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[Jean-Pierre Rosen]

dewar@gnat.com a �crit dans le message
<76g9ds$ura$1@nnrp1.dejanews.com>...
>In article <76fe92$46c$1@platane.wanadoo.fr>,
>  "Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> wrote:
>> Unfortunately, all IO operations are potentially
>> blocking, and therefore not allowed from within a
>> protected operation.
>> To get the effect of mutual exclusion for procedures
>> requiring potentially blocking operations, have a look at
>> package Protection available from Adalog's components
>> page (http://perso.wanadoo.fr/adalog/compo2.htm)
>
>By the way, JPR here seems to be making the mistake of
>assuming that stream = stream_io, a common confusion. The
>original question here related to a particular
>implementation of streams that you have not seen, and you
>do not know if it does any potentially blocking operations.
>
Of course not, but that particular exemple used Text_Streams and I
took this opportunity to remind people of this often overlooked
problem.
Why, if it works on most compilers ? Because I hate concealed design
decisions. Choosing between portability and efficiency (or in this
case, ease of programming) is a trade-off, and it is perfectly
acceptable to disregard portability as long as it is a concious design
decision. But as you noticed, often people make this choice without
being aware of it, and *this* is (IMHO) very bad.
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://perso.wanadoo.fr/adalog

Re: Streams and Concurrency

[robert_dewar]

In article <76qb23$ba3$1@platane.wanadoo.fr>,
  "Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> wrote:
> Of course not, but that particular exemple used
> Text_Streams and I took this opportunity to remind people
> of this often overlooked problem. Why, if it works on
> most compilers ? Because I hate concealed design
> decisions. Choosing between portability and efficiency
> (or in this case, ease of programming) is a trade-off,
> and it is perfectly acceptable to disregard portability
> as long as it is a concious design decision. But as you
> noticed, often people make this choice without
> being aware of it, and *this* is (IMHO) very bad.

But of course when you call a foreign language routine of
any kind AT ALL, you are implementation dependent. This is
the only issue we are talking about here!

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Streams and Concurrency

[Robert I. Eachus]

In article <76g0nu$nsq$1@nnrp1.dejanews.com> dennison@telepath.com writes:

 > "Special" perhaps, but not all that odd. It can occur in *any* tasking
 > program. In my 10 years of Ada use I've only worked on two programs that had
 > no tasks.

   My use of special was referring to the fact that this pattern
requires either Unchecked_Conversion or its moral equivalent, but is
still completely portable!  You have to move the same data as both
records and as arrays of Storage_Elements, but the mapping between the
two does not affect the operation of the program. (At least as long as
it is reversible.)
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Streams and Concurrency

[Robert I. Eachus]

In article <76e4rg$bsr$1@nnrp1.dejanews.com> dennison@telepath.com writes:

 > But this solution only works for objects that are part of the same class
 > hiearchy. That's too restrictive. I'd want to be able to use any type. It
 > sounds like Tuck's generic suggestion is the way to go.

   Wonderful opertunity for generic children. ;-)  Create a (generic)
package that encapsulates the stream and the locking mechanism.  This
package would then have a generic child package that you instantiate
once for each type you want to write to the stream.  (And now for the
really tricky part...)

   If you want to be able to read the mixed stream without knowing
what type of value appears next, you have to have a parent type in the
parent generic, or a parent access type, and a read operation declared
there which returns objects of the class rooted at this type.  You can
then use a fairly common Ada idiom (a declare block with a class-wide
object initialized by a function) to make the contents of the stream
available.

   The case where you know the structure of the stream, or as Tucker
suggested, have markers in the stream, is much easier to program.

      
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Ada-G, was Re: Streams and Concurrency

[Robert I. Eachus]

In article <Pine.BSF.4.05.9901011003090.10526-100000@shell5.ba.best.com> Brian Rogoff <bpr@shell5.ba.best.com> writes:

 > I agree completely, but the message I get from this interesting discussion
 > is that vendors should agree on Ada fixes (based on customer input amongst 
 > other things) a bit faster than once a decade.

   They do, but the process is still slow and deliberate.  During the
(officially just over eleven years) between Ada 83 and Ada 95, there
were a half-dozen cases where the Ada Rapporteur Group issued Nonbinding
interpretations, one of them on raising Contstraint_Error instead of
Program_Error.  The goal of these NBIs was to allow implementors to
move in the direction that the new language revision would move
without forcing any particular schedule.  In cases where the language
was unclear but all compilers behaved consistantly, we enforced the de
facto standard with binding interpretations.

   The official process with Ada 95 may be somewhat different since we
intend to issue corrigenda consolidating the AIs, but the results
should be the same--a slow evolution in the definition of the
language.

--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Streams and Concurrency

[Mats Weber]

dewar@gnat.com wrote:

> In any case, GNAT guarantees that blocking operations CAN
> be done within protected objects, of course you may get
> deadlock if you do obviously bad things, but certainly
> doing IO within a protected object is not such a bad thing,
> and this will work fine. We have many customers who depend
> on this guarantee.

I agree totally.

> Yes, yes, of course this is potentially non-portable, but
> it makes protected objects enormously more useful. Remember
> that there are two advantages of protected objects:
> 
> 1) possibility of more efficient implementation (I say
> possibility here because in fact there are many
> complications which result in less efficiency than you
> might have hoped for in the general case, and in any
> case some cases of agent tasks can be optimized in a
> corresponding efficient manner.

Are you talking about what DEC Ada and Verdix call passive tasks ?

> 2) preferable abstraction to a task in some situations
> 
> For my taste 2) is much more important than 1), and the
> restriction on blocking operations badly damages the use
> of 2).

I also see other reasons:

3) In order to protect objects with potentially blocking operations (a
socket, a pipe, etc.), you must either use a semaphore implemented as a
protected object and call the semaphore _twice_, once for locking and
once for unlocking, or you must use a task. Thus the promised
performance gain with protected objects is lost.

4) It is another case where the standard enforces things that no
compiler can check, neither at compile time nor at run time. Ada 83 had
incorrect order dependences in this category, where it was allowed to
raise Program_Error if it found one. Fortunately, this was removed from
Ada 95 and I think the restriction on blocking protected operations
falls in the same category.

Re: Ada-G, was Re: Streams and Concurrency

[Mats Weber]

"Robert I. Eachus" wrote:

>    They do, but the process is still slow and deliberate.  During the
> (officially just over eleven years) between Ada 83 and Ada 95, there
> were a half-dozen cases where the Ada Rapporteur Group issued Nonbinding
> interpretations, one of them on raising Contstraint_Error instead of
> Program_Error.
  ^^^^^^^^^^^^^
You mean Numeric_Error here, right ?

Re: Streams and Concurrency

[Tucker Taft]

Mats Weber (Mats.Weber@elca-matrix.ch) wrote:

.. [discussion of restriction against potentially-blocking operations
   in protected operations]
: ...
: 4) It is another case where the standard enforces things that no
: compiler can check, neither at compile time nor at run time. Ada 83 had
: incorrect order dependences in this category, where it was allowed to
: raise Program_Error if it found one. Fortunately, this was removed from
: Ada 95 and I think the restriction on blocking protected operations
: falls in the same category.

Checking for explicit use of entry calls or task activation "statically"
within the protected operation is not difficult.  Checking for
such uses outside the protected body is of course more difficult (though not
impossible if willing to do it at link-time).  Detecting incorrect order 
dependences, by contrast, involves solving the halting problem ;-).

The restriction against performing other *entry* calls (or activating
new tasks) while in a protected operation seems entirely appropriate, and 
I presume that GNAT does not support that.  I agree that things which
the programmer "knows" won't suspend the Ada task ("suspend" in the sense
of waiting indefinitely for some other Ada task of potentially lower 
priority to wake it up) are safe and generally portable to do, so in
this sense the standard was probably overly conservative.

--
-Tucker Taft   stt@averstar.com   http://www.averstar.com/~stt/
Technical Director, Distributed IT Solutions  (www.averstar.com/tools)
AverStar (formerly Intermetrics, Inc.)   Burlington, MA  USA

Re: Ada-G, was Re: Streams and Concurrency

[Robert I. Eachus]

In article <36962BBC.23B129F0@elca-matrix.ch> Mats Weber <Mats.Weber@elca-matrix.ch> writes:

 > >    They do, but the process is still slow and deliberate.  During the
 > > (officially just over eleven years) between Ada 83 and Ada 95, there
 > > were a half-dozen cases where the Ada Rapporteur Group issued Nonbinding
 > > interpretations, one of them on raising Contstraint_Error instead of
 > > Program_Error.
     ^^^^^^^^^^^^^
 >  You mean Numeric_Error here, right ?

   Oops!  You are right.  (And I mispelled Constraint_Error too.  I'll
have to take my fingers in for a 50,000 message checkup.)

   However...Note that in going from Ada 83 to Ada 95, several cases
where Constraint_Error or Program_Error error could be raised were
tightened up.  This is a good thing, but it may mean your code now
raises Program_Error where before you saw Constraint_Error.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Bulk synchronous model for IO (was Re: Streams and Concurrency)

[Kevin Rigotti]

Interesting thread ...

In the more general case of lots of parallel activity requiring IO to happen
in a controlled way then it might be interesting to use Streams as the basis
for doing 'BSP'-like things in Ada, where IO is initially buffered locally
and only completed between tasks at the end of the next 'superstep'

The BSP home page has links to details of a C library (BSPlib) that provides
primitives for doing this, but it might be fun to see how the same thing
might be done in Ada.

Just a thought, maybe a nice little project for a student :-)

Kevin Rigotti
Senior Scientist, ATC Systems Group


begin 666 BSP Worldwide Home Page.url
M6TEN=&5R;F5T4VAO<G1C=71=#0I54DP]:'1T<#HO+W=W=RYB<W M=V]R;&1W
G:61E+F]R9R\-"DUO9&EF:65D/44P030P1C5".#$S1$)%,#$V0@T*
`
end