comp.lang.ada
 help / color / mirror / Atom feed
* Broadcast / iterate to all Connection objects via Simple Components?
@ 2023-02-07 20:29 A.J.
  2023-02-08  8:55 ` Dmitry A. Kazakov
                   ` (2 more replies)
  0 siblings, 3 replies; 22+ messages in thread
From: A.J. @ 2023-02-07 20:29 UTC (permalink / raw)


Hello everyone,

In an effort to better learn network programming in Ada, I've been working through the Protohacker Challenges (https://protohackers.com/), and the current challenge (number 3) is to create a chat server.

I am using a TCP Connections Server with Simple Components, specifically a Connection_State_Machine, but I've run into a problem.  I'm trying to send a message received via "procedure Process_Packet (Client : in out Server_Connection)" to all connected Clients.

My (potentially incorrect) thought on how to accomplish this is to iterate through all of the clients currently connected, and use Send to send the message received to those clients.  I've been struggling with how to actually do this though, since I couldn't use "function Get_Clients_Count (Listener : Connections_Server) return Natural" from within Process_Packets.

Another thought I had could be to just place every message received in a central queue, and then once all of the packets have been received, to then process that queue and send the results to every connected client.

I tried overriding "procedure On_Worker_Start (Listener : in out Connections_Server)", thinking that I could use it to read such a queue, but it never seemed to be called from within my program and I'm still unsure how to iterate through the Connection objects anyway.

Am I approaching this the right way, or am I missing something very obvious?  I've read the test files that came with Simple Components, including the data server but couldn't see a way to get each client to interact with each other.  If I didn't explain this well enough, please let me know, I'll be happy to clarify.

Kind regards,
AJ.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-07 20:29 Broadcast / iterate to all Connection objects via Simple Components? A.J.
@ 2023-02-08  8:55 ` Dmitry A. Kazakov
  2023-02-08  9:55 ` Jeffrey R.Carter
  2023-02-13 20:33 ` Daniel Norte de Moraes
  2 siblings, 0 replies; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-08  8:55 UTC (permalink / raw)


On 2023-02-07 21:29, A.J. wrote:

> In an effort to better learn network programming in Ada, I've been working through the Protohacker Challenges (https://protohackers.com/), and the current challenge (number 3) is to create a chat server.
> 
> I am using a TCP Connections Server with Simple Components, specifically a Connection_State_Machine, but I've run into a problem.  I'm trying to send a message received via "procedure Process_Packet (Client : in out Server_Connection)" to all connected Clients.
> 
> My (potentially incorrect) thought on how to accomplish this is to iterate through all of the clients currently connected, and use Send to send the message received to those clients.  I've been struggling with how to actually do this though, since I couldn't use "function Get_Clients_Count (Listener : Connections_Server) return Natural" from within Process_Packets.
> 
> Another thought I had could be to just place every message received in a central queue, and then once all of the packets have been received, to then process that queue and send the results to every connected client.
> 
> I tried overriding "procedure On_Worker_Start (Listener : in out Connections_Server)", thinking that I could use it to read such a queue, but it never seemed to be called from within my program and I'm still unsure how to iterate through the Connection objects anyway.
> 
> Am I approaching this the right way, or am I missing something very obvious?  I've read the test files that came with Simple Components, including the data server but couldn't see a way to get each client to interact with each other.  If I didn't explain this well enough, please let me know, I'll be happy to clarify.

Short answer, clients never interact, but nothing prevents you from 
maintaining a list of clients, e.g. in the factory of.

Long answer. If you have to create a messages board, that is lot more 
than mere TCP/IP clients. Clients are only a carrier of messages. I 
presume you would need:

- Some archive of messages, persistent or not maintained by the server.

- Identification of recipients, which come and go. Usually a client 
authenticates itself upon connection to the server via one of countless 
schemas user/password/hash/challenge etc.

- Thus you need some user authentication store/database, again, 
persistent or not. Or use an external one, like LDAP etc.

- The messages stored by the server need unique identifiers and likely 
time stamp, and likely poster identification, and the list of the 
recipients. E.g. in the authentication store each user would have the 
list of received message identifiers.

- The server runs a task (or tasks) dispatching messages from the store 
to all users connected through currently active clients. The task walks 
messages list and the list of all clients, for each client it looks if 
the message was successfully sent to the client's user. If not, the task 
sends it. Since all communication is asynchronous the message receipt is 
too asynchronous to sending. On confirmation of receipt, the user 
messages list is updated. You might need more than Boolean receipt flag. 
E.g. "sending in progress" to prevent multiple sending and/or repeated 
messages filter on the client side.

And

   server /= socket listener, there could be many
   client /= user, client plays a role of some authenticated user

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-07 20:29 Broadcast / iterate to all Connection objects via Simple Components? A.J.
  2023-02-08  8:55 ` Dmitry A. Kazakov
@ 2023-02-08  9:55 ` Jeffrey R.Carter
  2023-02-13  7:28   ` Emmanuel Briot
  2023-02-13 20:33 ` Daniel Norte de Moraes
  2 siblings, 1 reply; 22+ messages in thread
From: Jeffrey R.Carter @ 2023-02-08  9:55 UTC (permalink / raw)


On 2023-02-07 21:29, A.J. wrote:
> 
> In an effort to better learn network programming in Ada, I've been working through the Protohacker Challenges (https://protohackers.com/), and the current challenge (number 3) is to create a chat server.

For an example of this, see the Chattanooga demo that comes with Gnoga 
(https://sourceforge.net/projects/gnoga/). A screenshot and intermittently 
working (not right now) on-line version are available at 
https://sourceforge.net/p/gnoga/wiki/Gnoga-Gallery/.

-- 
Jeff Carter
"I'm a vicious jungle beast!"
Play It Again, Sam
131

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-08  9:55 ` Jeffrey R.Carter
@ 2023-02-13  7:28   ` Emmanuel Briot
  2023-02-13  8:30     ` Dmitry A. Kazakov
  0 siblings, 1 reply; 22+ messages in thread
From: Emmanuel Briot @ 2023-02-13  7:28 UTC (permalink / raw)


I am not sure how familiar you are with Network programming in general (not just as it would be done in Ada).
Using a blocking Send could actually kill your performance. 
You mentioned you would be sending a message to one client after another.  Imaging one of the client has small socket buffers, and is
busy doing something else at the moment so not reading your message immediately.  If you are sending a large message, your server
would only be able to send part of the message, then it would block until the client has read enough that there is space again in the
socket buffers to send the rest of the message.  That could take ... days ?  In the meantime, you server is not doing anything else, and
no other client gets send anything...

Instead, you need to use non-blocking sockets.  When Send returns, it has sent whatever it could for the moment.  You then need to
monitor the socket (and all other similar ones) using something like  select (which is limited to sockets < 1024, so pretty useless
for an actual server in practice),  poll (better version of select) or epoll (the best in my opinion).   I have written a similar server that
has 25000 concurrent clients, and serving them all with 10 worker tasks.  That would never fly with blocking sockets.

A similar approach when receiving messages from clients, by the way.  The message might have sent only part of its message, so you
need to give up temporarily, and come back to it when poll tells you there is something new to read.

Emmanuel

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13  7:28   ` Emmanuel Briot
@ 2023-02-13  8:30     ` Dmitry A. Kazakov
  2023-02-13  8:44       ` Emmanuel Briot
  0 siblings, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-13  8:30 UTC (permalink / raw)


On 2023-02-13 08:28, Emmanuel Briot wrote:
> I am not sure how familiar you are with Network programming in general (not just as it would be done in Ada).
> Using a blocking Send could actually kill your performance.
[...]
> A similar approach when receiving messages from clients, by the way.  The message might have sent only part of its message, so you
> need to give up temporarily, and come back to it when poll tells you there is something new to read.

Yes. All networking in Simple components is built on non-blocking 
sockets (socket select).

P.S. This poses difficulties for users, who see all communication turned 
upside down being driven by arbitrary socket events rather than by the 
protocol logic. This was a reason I argued for introducing co-routines 
with task interface in Ada.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13  8:30     ` Dmitry A. Kazakov
@ 2023-02-13  8:44       ` Emmanuel Briot
  2023-02-13 10:55         ` Dmitry A. Kazakov
  0 siblings, 1 reply; 22+ messages in thread
From: Emmanuel Briot @ 2023-02-13  8:44 UTC (permalink / raw)


> sockets (socket select). 

Have you taken a look at epoll(), on linux ?  It is so much more natural to use, and so much more efficient in practice.
The example I mentioned above (a server with 25_000 concurrent connections) cannot work with select (which only
accepts file descriptors up to 1024), and is slow with poll (since the result of the latter is the number of events, and we
need to iterate over all registered sockets every time).

> P.S. This poses difficulties for users, who see all communication turned 
> upside down being driven by arbitrary socket events rather than by the 
> protocol logic. This was a reason I argued for introducing co-routines 
> with task interface in Ada.

In my own code, I basically provide an epoll-based generic framework.  One of the formal parameters is a `Job_Type`
with one primitive operation `Execute`.
The latter is initially called when a new connection is established, and is expected to do as much non-blocking work
as it can (Execute is run in one of the worker tasks).  When it cannot make progress, it returns a tuple (file_descriptor, type_of_event_to_wait_for)
to indicate when it needs to be called again in the future, for instance some data became available to read on the
specified file_descriptor.
The intent is that the `Job_Type` is implemented as a state machine internally.

Of course, a state machine is one of the two ways I know (along with a task) to implement the equivalent of
a co-routine in Ada.  So I 100% agree with you that co-routines would be very useful in simplifying user code,
in particular in the scenario we are discussing here !

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13  8:44       ` Emmanuel Briot
@ 2023-02-13 10:55         ` Dmitry A. Kazakov
  2023-02-13 11:07           ` Emmanuel Briot
  0 siblings, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-13 10:55 UTC (permalink / raw)


On 2023-02-13 09:44, Emmanuel Briot wrote:
>> sockets (socket select).
> 
> Have you taken a look at epoll(), on linux ?

The implementation is on top of GNAT.Sockets, so no.

> It is so much more natural to use, and so much more efficient in practice.
> The example I mentioned above (a server with 25_000 concurrent connections) cannot work with select (which only
> accepts file descriptors up to 1024), and is slow with poll (since the result of the latter is the number of events, and we
> need to iterate over all registered sockets every time).

Well, if there is Linux kernel level support why it is not used in 
socket select as it is in epoll? I would expect them do that at some 
point or drop epoll... (:-))

>> P.S. This poses difficulties for users, who see all communication turned
>> upside down being driven by arbitrary socket events rather than by the
>> protocol logic. This was a reason I argued for introducing co-routines
>> with task interface in Ada.
> 
> In my own code, I basically provide an epoll-based generic framework.  One of the formal parameters is a `Job_Type`
> with one primitive operation `Execute`.
> The latter is initially called when a new connection is established, and is expected to do as much non-blocking work
> as it can (Execute is run in one of the worker tasks).  When it cannot make progress, it returns a tuple (file_descriptor, type_of_event_to_wait_for)
> to indicate when it needs to be called again in the future, for instance some data became available to read on the
> specified file_descriptor.
> The intent is that the `Job_Type` is implemented as a state machine internally.

Yes, state machine is what I want to avoid. With complex layered 
protocols it imposes incredible difficulties requiring auxiliary stacks 
and buffers with errors almost intractable either by testing or by 
formal proofs.

> Of course, a state machine is one of the two ways I know (along with a task) to implement the equivalent of
> a co-routine in Ada.  So I 100% agree with you that co-routines would be very useful in simplifying user code,
> in particular in the scenario we are discussing here !

I'd like to have special Ada "tasks" acting as co-routines on top of 
proper tasks yielding when the socket buffer is empty or full.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 10:55         ` Dmitry A. Kazakov
@ 2023-02-13 11:07           ` Emmanuel Briot
  2023-02-13 11:57             ` Dmitry A. Kazakov
  2023-02-13 16:40             ` Jeremy Grosser <jeremy@synack.me>
  0 siblings, 2 replies; 22+ messages in thread
From: Emmanuel Briot @ 2023-02-13 11:07 UTC (permalink / raw)


On Monday, February 13, 2023 at 11:55:10 AM UTC+1, Dmitry A. Kazakov wrote:
> Well, if there is Linux kernel level support why it is not used in 
> socket select as it is in epoll? I would expect them do that at some 
> point or drop epoll... (:-))

Because in practice the linux developers don't get to modify such APIs, which
are mandated by Posix, or Unix, or some RFC.
So the API for select and poll will *never* change.

epoll is definitely the modern approach on linux, until of course someone finds
something even better.  epoll is fully thread safe too, which is very nice when
used from Ada.
Using select() is totally outdated at this point, and means you can never handle
more than 1000 simultaneous clients, and that only if you do not have other
file descriptors open (database, files,...)

The person who developed GNAT.Sockets has left AdaCore a while ago, so "they" (which
I assume is what your message was referring to) are actually unlikely to update that.
They also have strong concerns about platform-agnostic support, and epoll is linux-specific
at this point (likely also BSD).  There exists multiple libraries out there that provide an API
common to multiple platforms, and that use epoll on linux.  Maybe that's what would make
sense, but nowadays with Alire, I would expect someone to build a crate there rather than
AdaCore modify GNAT.Sockets.


> Yes, state machine is what I want to avoid. With complex layered 
> protocols it imposes incredible difficulties requiring auxiliary stacks 
> and buffers with errors almost intractable either by testing or by 
> formal proofs.

tell me about auxiliary stacks :-)  In practice, in my experience, you can have a single
incoming buffer which is used by one state, and then another when the first state is no
longer active,... so we do not need to have too many buffers, but that definitely is not
trivial.  Currently, I have a stack of iterators reading from a socket, buffering on top of that,
then decompressing LZ4 data, then decoding our binary encoding to Ada values.


> I'd like to have special Ada "tasks" acting as co-routines on top of 
> proper tasks yielding when the socket buffer is empty or full.

This is an approach we had discussed at AdaCore before I left.  There are multiple drawbacks
here: the limited stack size for tasks by default (2MB), the fact that entries cannot return indefinite
types, the fact that currently those tasks are assigned to OS threads (so too many of them does
impact resource usage),...

A colleague had found an external library that would provide several stacks and thus let people
implement actual co-routines.  We did not do much more work on that, but it was a nice proof
of concept, and efficient.  I think things are mostly blocked now, as the ARG has been discussing
these topics for quite a few years now.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 11:07           ` Emmanuel Briot
@ 2023-02-13 11:57             ` Dmitry A. Kazakov
  2023-02-13 13:22               ` Niklas Holsti
  2023-02-13 16:40             ` Jeremy Grosser <jeremy@synack.me>
  1 sibling, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-13 11:57 UTC (permalink / raw)


On 2023-02-13 12:07, Emmanuel Briot wrote:
> On Monday, February 13, 2023 at 11:55:10 AM UTC+1, Dmitry A. Kazakov wrote:
>> Well, if there is Linux kernel level support why it is not used in
>> socket select as it is in epoll? I would expect them do that at some
>> point or drop epoll... (:-))
> 
> Because in practice the linux developers don't get to modify such APIs, which
> are mandated by Posix, or Unix, or some RFC.
> So the API for select and poll will *never* change.

Well, the API does not really prescribe the actual implementation of 
fd_set. If you need that many elements in the set then using a bitmask 
and not having some bookkeeping to balance load will bit you at some 
point anyway.

> epoll is definitely the modern approach on linux, until of course someone finds
> something even better.  epoll is fully thread safe too, which is very nice when
> used from Ada.
> Using select() is totally outdated at this point, and means you can never handle
> more than 1000 simultaneous clients, and that only if you do not have other
> file descriptors open (database, files,...)

Well, it is indicative how outdated Linux always was. Asynchronous I/O 
was old news long *before* Linus even started his project... (:-))

> The person who developed GNAT.Sockets has left AdaCore a while ago, so "they" (which
> I assume is what your message was referring to) are actually unlikely to update that.

Yes, that is what I expect, a face lifting of Linux implementation of 
GNAT.Sockets.

> They also have strong concerns about platform-agnostic support, and epoll is linux-specific
> at this point (likely also BSD).

GNAT.Sockets is no thin bindings, I see little problem here. AFAIK BSD 
allows increasing FD_SETSIZE, but I am not sure.

> There exists multiple libraries out there that provide an API
> common to multiple platforms, and that use epoll on linux.  Maybe that's what would make
> sense, but nowadays with Alire, I would expect someone to build a crate there rather than
> AdaCore modify GNAT.Sockets.

That is an option too. However presently I cannot use Alire because last 
time I looked it did not support automated uploading, versioning and 
target dependencies through the GPR.

>> I'd like to have special Ada "tasks" acting as co-routines on top of
>> proper tasks yielding when the socket buffer is empty or full.
> 
> This is an approach we had discussed at AdaCore before I left.  There are multiple drawbacks
> here: the limited stack size for tasks by default (2MB), the fact that entries cannot return indefinite
> types, the fact that currently those tasks are assigned to OS threads (so too many of them does
> impact resource usage),...

My idea is to have these as pseudo-tasks scheduled by the Ada run-time 
and not mapped onto any OS threads. A proper thread would pick up such a 
task and run it until it yields. The crucial point is to use the stack 
of the pseudo-task in place of the thread's stack or backing it up and 
cleaning the portion of the stack at the point of yielding, whatever.

> A colleague had found an external library that would provide several stacks and thus let people
> implement actual co-routines.  We did not do much more work on that, but it was a nice proof
> of concept, and efficient.  I think things are mostly blocked now, as the ARG has been discussing
> these topics for quite a few years now.

I have an impression that ARG's view on co-routines totally ignores the 
use case of communication stacks and other cases state machines show 
their ugly faces...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 11:57             ` Dmitry A. Kazakov
@ 2023-02-13 13:22               ` Niklas Holsti
  2023-02-13 15:10                 ` Dmitry A. Kazakov
  2023-02-13 15:43                 ` J-P. Rosen
  0 siblings, 2 replies; 22+ messages in thread
From: Niklas Holsti @ 2023-02-13 13:22 UTC (permalink / raw)


On 2023-02-13 13:57, Dmitry A. Kazakov wrote:
> On 2023-02-13 12:07, Emmanuel Briot wrote:
>> On Monday, February 13, 2023 at 11:55:10 AM UTC+1, Dmitry A. Kazakov 
>> wrote:


    [ snip discussion of network programming details, retain
      discussion about co-routines ]


>>> I'd like to have special Ada "tasks" acting as co-routines on top of
>>> proper tasks yielding when the socket buffer is empty or full.
>>
>> This is an approach we had discussed at AdaCore before I left.  There 
>> are multiple drawbacks
>> here: the limited stack size for tasks by default (2MB), the fact that 
>> entries cannot return indefinite
>> types, the fact that currently those tasks are assigned to OS threads 
>> (so too many of them does
>> impact resource usage),...
> 
> My idea is to have these as pseudo-tasks scheduled by the Ada run-time 
> and not mapped onto any OS threads. A proper thread would pick up such a 
> task and run it until it yields. The crucial point is to use the stack 
> of the pseudo-task in place of the thread's stack or backing it up and 
> cleaning the portion of the stack at the point of yielding, whatever.
> 
>> A colleague had found an external library that would provide several 
>> stacks and thus let people
>> implement actual co-routines.  We did not do much more work on that, 
>> but it was a nice proof
>> of concept, and efficient.  I think things are mostly blocked now, as 
>> the ARG has been discussing
>> these topics for quite a few years now.
> 
> I have an impression that ARG's view on co-routines totally ignores the 
> use case of communication stacks and other cases state machines show 
> their ugly faces...


So your co-routines would (1) have their own stack and (2) be 
independently schedulable, which implies (3) having their own execution 
context (register values, instruction pointer, etc.) How is that 
different from the Ada concept of a "task"? How could the ARG separate 
between a "task" and a "co-routine" in the Ada RM?

There exist Ada compilers and run-times where the tasking concept is 
implemented entirely in the run-time system, without involving the 
underlying OS (if there even is one). That approach was mostly abandoned 
in favour of mapping tasks to OS threads, which makes it easier to use 
potentially blocking OS services from tasks without blocking the entire 
Ada program.

So is your problem only that using OS threads is less "efficient" than 
switching and scheduling threads of control in the run-time system? If 
so, that seems to be a quality-of-implementation issue that could be 
solved in a compiler-specific way, and not an issue with the Ada 
language itself.

The point (from Emmanuel) that task entries cannot return indefinite 
types is certainly a language limitation, but seems to have little to do 
with the possible differences between tasks and co-routines, and could 
be addressed on its own if Ada users so desire.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 13:22               ` Niklas Holsti
@ 2023-02-13 15:10                 ` Dmitry A. Kazakov
  2023-02-13 16:26                   ` Niklas Holsti
  2023-02-13 15:43                 ` J-P. Rosen
  1 sibling, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-13 15:10 UTC (permalink / raw)


On 2023-02-13 14:22, Niklas Holsti wrote:
> On 2023-02-13 13:57, Dmitry A. Kazakov wrote:

>> I have an impression that ARG's view on co-routines totally ignores 
>> the use case of communication stacks and other cases state machines 
>> show their ugly faces...
> 
> So your co-routines would (1) have their own stack and (2) be 
> independently schedulable, which implies (3) having their own execution 
> context (register values, instruction pointer, etc.)

Sure. You should be able to implement communication logic in a natural way:

1. Read n bytes [block until finished]
2. Do things
3. Write m bytes [block until finished]
4. Repeat

> How is that 
> different from the Ada concept of a "task"?

It is no different, that the whole point of deploying high level 
abstraction: task instead of low level one: state machine.

> How could the ARG separate 
> between a "task" and a "co-routine" in the Ada RM?

Syntax sugar does not bother me. I trust ARG to introduce a couple of 
reserved words in the most annoying way... (:-))

> So is your problem only that using OS threads is less "efficient" than 
> switching and scheduling threads of control in the run-time system?

This too. However the main purpose is control inversion caused by 
callback architectures. A huge number of libraries are built on that 
pattern. This OK for the library provider because it is the most natural 
and efficient way. For the user implementing his own logic, be it 
communication protocol, GUI etc, it is a huge architectural problem as 
it distorts the problem space logic. So the goal is to convert a 
callback/event driven architecture into plain control flow.

> If 
> so, that seems to be a quality-of-implementation issue that could be 
> solved in a compiler-specific way, and not an issue with the Ada 
> language itself.

In Ada 83 there was no way to pass a procedure as a parameter. We used a 
task instead... (:-))

But sure, a possibility to delegate a callback to an entry call without 
intermediates is certainly welcome.

> The point (from Emmanuel) that task entries cannot return indefinite 
> types is certainly a language limitation, but seems to have little to do 
> with the possible differences between tasks and co-routines, and could 
> be addressed on its own if Ada users so desire.

Yes.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 13:22               ` Niklas Holsti
  2023-02-13 15:10                 ` Dmitry A. Kazakov
@ 2023-02-13 15:43                 ` J-P. Rosen
  1 sibling, 0 replies; 22+ messages in thread
From: J-P. Rosen @ 2023-02-13 15:43 UTC (permalink / raw)


Le 13/02/2023 à 14:22, Niklas Holsti a écrit :
> The point (from Emmanuel) that task entries cannot return indefinite 
> types is certainly a language limitation, but seems to have little to do 
> with the possible differences between tasks and co-routines, and could 
> be addressed on its own if Ada users so desire.

That's what Holders are intended for... (changing indefinite types into a definite one)
-- 
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
https://www.adalog.fr https://www.adacontrol.fr

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 15:10                 ` Dmitry A. Kazakov
@ 2023-02-13 16:26                   ` Niklas Holsti
  2023-02-13 19:48                     ` Dmitry A. Kazakov
  0 siblings, 1 reply; 22+ messages in thread
From: Niklas Holsti @ 2023-02-13 16:26 UTC (permalink / raw)


On 2023-02-13 17:10, Dmitry A. Kazakov wrote:
> On 2023-02-13 14:22, Niklas Holsti wrote:
>> On 2023-02-13 13:57, Dmitry A. Kazakov wrote:
> 
>>> I have an impression that ARG's view on co-routines totally ignores 
>>> the use case of communication stacks and other cases state machines 
>>> show their ugly faces...
>>
>> So your co-routines would (1) have their own stack and (2) be 
>> independently schedulable, which implies (3) having their own 
>> execution context (register values, instruction pointer, etc.)
> 
> Sure. You should be able to implement communication logic in a natural way:
> 
> 1. Read n bytes [block until finished]
> 2. Do things
> 3. Write m bytes [block until finished]
> 4. Repeat


Agreed.


>> How is that different from the Ada concept of a "task"?
> 
> It is no different, that the whole point of deploying high level 
> abstraction: task instead of low level one: state machine.
> 
>> How could the ARG separate between a "task" and a "co-routine" in the 
>> Ada RM?
> 
> Syntax sugar does not bother me. I trust ARG to introduce a couple of 
> reserved words in the most annoying way... (:-))


But why should the RM define two "task" concepts that have exactly the 
same properties? Or should co-routines have some different properties? 
That was the point of my question, apologies if I was unclear.

For example, co-routines could have the ability to form some kind of 
"mutually exclusive groups" such that there would be no need for 
mutual-exclusion controls such as protected objects between co-routines 
in the same group. That could be one way to identify the co-routines 
that are to be executed by one and the same "real" task, as you propose.

What properties do you propose for co-routines that would not hold for 
ordinary tasks?


>> So is your problem only that using OS threads is less "efficient" than 
>> switching and scheduling threads of control in the run-time system?
> 
> This too.


But you cannot ask the ARG to define co-routines as being "like tasks 
but more efficient". :-)


> However the main purpose is control inversion caused by 
> callback architectures. A huge number of libraries are built on that 
> pattern. This OK for the library provider because it is the most natural 
> and efficient way. For the user implementing his own logic, be it 
> communication protocol, GUI etc, it is a huge architectural problem as 
> it distorts the problem space logic. So the goal is to convert a 
> callback/event driven architecture into plain control flow.


Yes, yes. But tasks can do that. How would co-routines help, aside from 
perhaps being "more efficient"?

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 11:07           ` Emmanuel Briot
  2023-02-13 11:57             ` Dmitry A. Kazakov
@ 2023-02-13 16:40             ` Jeremy Grosser <jeremy@synack.me>
  1 sibling, 0 replies; 22+ messages in thread
From: Jeremy Grosser <jeremy@synack.me> @ 2023-02-13 16:40 UTC (permalink / raw)


> epoll is definitely the modern approach on linux, until of course someone finds 
> something even better. epoll is fully thread safe too, which is very nice when 
> used from Ada. 

For high performance networking, io_uring [1] is the new kid on the block, but the API involves a scary amount of pointer manipulation, so I'm not convinced that it's safe to use yet.

While epoll is thread safe, there are some subtleties. If you register a listening socket with epoll, then call epoll_wait from multiple threads, more than one thread may be woken up when the socket has a waiting incoming connection to be accepted. Only one thread will get a successful return from accept(), the others will return EAGAIN. This wastes cycles if your server handles lots of incoming connections. The recently added (kernel >=4.5) EPOLLEXCLUSIVE flag enables a mutex that ensures the event is only delivered to a single thread.

> They also have strong concerns about platform-agnostic support, and epoll is linux-specific 
> at this point (likely also BSD). There exists multiple libraries out there that provide an API 
> common to multiple platforms, and that use epoll on linux. Maybe that's what would make 
> sense, but nowadays with Alire, I would expect someone to build a crate there rather than 
> AdaCore modify GNAT.Sockets.

On BSD, the kqueue [2] API provides similar functionality to epoll. I believe kqueue is a better design, but you use what your platform supports.

libev [3] is the library I see used most commonly for cross-platform evented I/O. It will use the best available polling syscalls on whatever platform it's compiled for. Unfortunately, it's composed mostly of C preprocessor macros.

I've already written an epoll binding [5] that's in the Alire index. GNAT.Sockets provides the types and bindings for the portable syscalls.

For the Protohackers puzzles, I've written a small evented I/O server using those bindings [6]. Note that this server does not use events for the send() calls yet, which may block, though in practice it isn't an issue with the size of the payloads used in this application. I do plan to refactor this to buffer data to be sent when the Writable (EPOLLOUT) event is ready.

So far, I've found tasks and coroutines to be unnecessary for these servers, though coroutines would make it possible to implement Ada.Streams compatible Read and Write procedures, providing a cleaner interface that doesn't expose callbacks to the user.

[1] https://lwn.net/Articles/776703/
[2] https://people.freebsd.org/~jlemon/papers/kqueue.pdf
[3] https://linux.die.net/man/3/ev
[4]  https://github.com/JeremyGrosser/epoll-ada
[5] https://github.com/JeremyGrosser/protohackers/blob/master/src/mini.adb

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 16:26                   ` Niklas Holsti
@ 2023-02-13 19:48                     ` Dmitry A. Kazakov
  2023-02-15  9:54                       ` Niklas Holsti
  0 siblings, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-13 19:48 UTC (permalink / raw)


On 2023-02-13 17:26, Niklas Holsti wrote:
> On 2023-02-13 17:10, Dmitry A. Kazakov wrote:

> But why should the RM define two "task" concepts that have exactly the 
> same properties? Or should co-routines have some different properties? 
> That was the point of my question, apologies if I was unclear.

Ada has floating-point, fixed-point, universal real implementations of 
the same concept of real number. Why?

> For example, co-routines could have the ability to form some kind of 
> "mutually exclusive groups" such that there would be no need for 
> mutual-exclusion controls such as protected objects between co-routines 
> in the same group.

When exclusion is ensured by the logic of I/O processing, then there is 
no difference. In all other cases a pool of threads could service a pool 
of co-routines.

> What properties do you propose for co-routines that would not hold for 
> ordinary tasks?

I do not see it that way. The objective is being able to reuse blocking 
code as-is with non-blocking I/O and conversely.

As a simplest case consider a pipeline. There are two tasks at its ends. 
That is a blocking implementation. Now, I want to take both tasks and 
run them as co-routines on the context of a single thread.

> But you cannot ask the ARG to define co-routines as being "like tasks 
> but more efficient". :-)

Rendezvous is asymmetric client-server. In a callback scenario parties 
are equivalent. Each can wait for another. With tasks a protected object 
is usually thrown in to work it around.

The objective is twofold:

1. To simplify "pipeline" stuff. Each side need not to know what the 
other side does or each other. They know the pipeline. The programmer 
should use a call or a timed entry call or a selective accept. I leave 
that open for now.

2. To allow sharing context of a single OS thread. So the term "co-routine."

3. As I said above, the goal is to take plain I/O code (the client) and 
reuse it with no adjustments with a server/provider/consumer in place of 
the OS I/O subsystem.

>> However the main purpose is control inversion caused by callback 
>> architectures. A huge number of libraries are built on that pattern. 
>> This OK for the library provider because it is the most natural and 
>> efficient way. For the user implementing his own logic, be it 
>> communication protocol, GUI etc, it is a huge architectural problem as 
>> it distorts the problem space logic. So the goal is to convert a 
>> callback/event driven architecture into plain control flow.
> 
> Yes, yes. But tasks can do that. How would co-routines help, aside from 
> perhaps being "more efficient"?

It is way too complicated and not reusable (see #1). #2 requires 
language magic. Finally, it is not about more efficient code it is about 
prohibitively inefficient code.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-07 20:29 Broadcast / iterate to all Connection objects via Simple Components? A.J.
  2023-02-08  8:55 ` Dmitry A. Kazakov
  2023-02-08  9:55 ` Jeffrey R.Carter
@ 2023-02-13 20:33 ` Daniel Norte de Moraes
  2 siblings, 0 replies; 22+ messages in thread
From: Daniel Norte de Moraes @ 2023-02-13 20:33 UTC (permalink / raw)


Em Tue, 7 Feb 2023 12:29:39 -0800 (PST), A.J. escreveu:

> Hello everyone,

Hi! Wellcome!

> 
> In an effort to better learn network programming in Ada, I've been
> working through the Protohacker Challenges (https://protohackers.com/),
> and the current challenge (number 3) is to create a chat server.
> 
> I am using a TCP Connections Server with Simple Components, specifically
> a Connection_State_Machine, but I've run into a problem.  I'm trying to
> send a message received via "procedure Process_Packet (Client : in out
> Server_Connection)" to all connected Clients.
> 
> My (potentially incorrect) thought on how to accomplish this is to
> iterate through all of the clients currently connected, and use Send to
> send the message received to those clients.  I've been struggling with
> how to actually do this though, since I couldn't use "function
> Get_Clients_Count (Listener : Connections_Server) return Natural" from
> within Process_Packets.
> 
> Another thought I had could be to just place every message received in a
> central queue, and then once all of the packets have been received, to
> then process that queue and send the results to every connected client.
> 
> I tried overriding "procedure On_Worker_Start (Listener : in out
> Connections_Server)", thinking that I could use it to read such a queue,
> but it never seemed to be called from within my program and I'm still
> unsure how to iterate through the Connection objects anyway.
> 
> Am I approaching this the right way, or am I missing something very
> obvious?  I've read the test files that came with Simple Components,
> including the data server but couldn't see a way to get each client to
> interact with each other.  If I didn't explain this well enough, please
> let me know, I'll be happy to clarify.
> 
> Kind regards,
> AJ.

 Do you is obliged to use gnat-sockets ?

 Adare_Net has support to syncronous IO multiplex; 
 With Adare_Net you can create virtually infinite groups of network pools 
to get and send data on demand. and you can discover if peers closed 
connections and others things too. you can use syncronous i/o 
multiplexing  groups of network pools in dedicate ada tasks (or similar)  
to get truly real-time network programming and apps.

Thanks!

https://github.com/danieagle/adare-net

https://github.com/danieagle/adare-net/blob/main/adare_net/src/adare_net-
sockets-polls.ads

Each poll_type has a limit of 255 entries (more than that can be very 
slower) But you can have as many poll_types as you want.

Thanks!

Best Whishes,
GrateFull, Dani.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-13 19:48                     ` Dmitry A. Kazakov
@ 2023-02-15  9:54                       ` Niklas Holsti
  2023-02-15 10:57                         ` Dmitry A. Kazakov
  0 siblings, 1 reply; 22+ messages in thread
From: Niklas Holsti @ 2023-02-15  9:54 UTC (permalink / raw)


On 2023-02-13 21:48, Dmitry A. Kazakov wrote:
> On 2023-02-13 17:26, Niklas Holsti wrote:
>> On 2023-02-13 17:10, Dmitry A. Kazakov wrote:
> 
>> But why should the RM define two "task" concepts that have exactly the 
>> same properties? Or should co-routines have some different properties? 
>> That was the point of my question, apologies if I was unclear.
> 
> Ada has floating-point, fixed-point, universal real implementations of 
> the same concept of real number. Why?


Of course because these implementations have /different/ properties, 
suitable for different uses. They are /not/ the same, and their 
differences are defined in the Ada RM.

I am asking you what differences you propose between Ada tasks and your 
Ada co-routines, as they would be defined in the Ada RM (and not as they 
would be implemented in some Ada programming system).

Perhaps you think that those differences are implicit in the term 
"co-routine", but I don't find it so, sorry.


> Rendezvous is asymmetric client-server. In a callback scenario parties 
> are equivalent. Each can wait for another. With tasks a protected object 
> is usually thrown in to work it around.


Aha, so you are proposing that co-routines would have some other means, 
not protected objects, for symmetric inter-co-routine communication. 
What would that be, perhaps directly shared (unprotected) variables? 
Then you have to ensure that those co-routines are implicitly mutually 
exclusive, right? Or are you suggesting some other form of 
inter-co-routine communication?


> The objective is twofold:


(Actually threefold, it seems :-) )


> 1. To simplify "pipeline" stuff. Each side need not to know what the 
> other side does or each other. They know the pipeline. The programmer 
> should use a call or a timed entry call or a selective accept. I leave 
> that open for now.


By "pipeline", do you mean an order-preserving stream of data from a 
producer to a consumer? If so, a simple protected queue or buffer 
between producer and consumer tasks implements it.

By leaving the details open, you leave your proposal fuzzy and hard to 
understand.


> 2. To allow sharing context of a single OS thread. So the term 
> "co-routine."


I don't think that property is implicit in the term "co-routine", 
because co-routines can exist even without OS threads.

But most importantly, this property cannot be used to /define/ the 
"co-routine" concept in the Ada RM. That definition may of course be 
/chosen/ so as to /allow/ this implementation.

However, this property is already satisfied for "tasks" in those Ada 
systems that implement tasking in the RTS, without using multiple OS 
threads. So the current definition of Ada "tasks" already allows such 
sharing.


> 3. As I said above, the goal is to take plain I/O code (the client) and 
> reuse it with no adjustments with a server/provider/consumer in place of 
> the OS I/O subsystem.


And that server/provider/consumer would be what? Ada code in the 
application? Or part of the Ada RTS? Would it have to be aware of all of 
the OS I/O subsystem's services? For example, would it have to know 
about the socket interface to networking?

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-15  9:54                       ` Niklas Holsti
@ 2023-02-15 10:57                         ` Dmitry A. Kazakov
  2023-02-15 18:37                           ` Niklas Holsti
  0 siblings, 1 reply; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-15 10:57 UTC (permalink / raw)


On 2023-02-15 10:54, Niklas Holsti wrote:
> On 2023-02-13 21:48, Dmitry A. Kazakov wrote:
>> On 2023-02-13 17:26, Niklas Holsti wrote:
>>> On 2023-02-13 17:10, Dmitry A. Kazakov wrote:
>>
>>> But why should the RM define two "task" concepts that have exactly 
>>> the same properties? Or should co-routines have some different 
>>> properties? That was the point of my question, apologies if I was 
>>> unclear.
>>
>> Ada has floating-point, fixed-point, universal real implementations of 
>> the same concept of real number. Why?
> 
> Of course because these implementations have /different/ properties, 
> suitable for different uses. They are /not/ the same, and their 
> differences are defined in the Ada RM.

The concept here is R, the set of real numbers.

> Perhaps you think that those differences are implicit in the term 
> "co-routine", but I don't find it so, sorry.

Clearly co-routine and OS thread have different properties as 
implementations of same concept of a "thread of control."

>> Rendezvous is asymmetric client-server. In a callback scenario parties 
>> are equivalent. Each can wait for another. With tasks a protected 
>> object is usually thrown in to work it around.
> 
> Aha, so you are proposing that co-routines would have some other means, 
> not protected objects, for symmetric inter-co-routine communication.

When you do blocking I/O you do not [explicitly] communicate with the 
system PCI bus or whatever.

> What would that be, perhaps directly shared (unprotected) variables?

Nothing. When you call Ada.Text_IO.Put_Line, you just do that, File_Type 
is not shared, not explicitly.

> Then you have to ensure that those co-routines are implicitly mutually 
> exclusive, right?

Exclusive to what?

>> 1. To simplify "pipeline" stuff. Each side need not to know what the 
>> other side does or each other. They know the pipeline. The programmer 
>> should use a call or a timed entry call or a selective accept. I leave 
>> that open for now.
> 
> By "pipeline", do you mean an order-preserving stream of data from a 
> producer to a consumer? If so, a simple protected queue or buffer 
> between producer and consumer tasks implements it.

No it does not. I was talking about the pipeline ends, not about the 
pipeline itself.

> By leaving the details open, you leave your proposal fuzzy and hard to 
> understand.

You need an implementation writing a buffer of n bytes into a FIFO queue?

procedure Write (S : in out FIFO; X : Stream_Element_Array) is
    From : Stream_Element_Offset := X'First;
begin
    loop
       Queue_Immediate (S, X (This..X'Last), From);
       exit when From >= X'Last;
       Wait_For_Not_Full (S);
       From := From + 1;
    end loop;
end Write;

The client code calls Write, which appears as a blocking call.

Should I provide another end too? (:-))

>> 2. To allow sharing context of a single OS thread. So the term 
>> "co-routine."
> 
> I don't think that property is implicit in the term "co-routine", 
> because co-routines can exist even without OS threads.

In the same sense how System.Address can exist without a processor? (:-))

> But most importantly, this property cannot be used to /define/ the 
> "co-routine" concept in the Ada RM. That definition may of course be 
> /chosen/ so as to /allow/ this implementation.

See above. The concept here is thread of control.

> However, this property is already satisfied for "tasks" in those Ada 
> systems that implement tasking in the RTS, without using multiple OS 
> threads. So the current definition of Ada "tasks" already allows such 
> sharing.

It does not support user implementations of tasks coexisting with the 
OS-backed tasks. Certainly one possible way would be like it was done 
with the access types and storage pools to provide an abstraction layer 
for tasks. However, I think that could be a bit extreme.

>> 3. As I said above, the goal is to take plain I/O code (the client) 
>> and reuse it with no adjustments with a server/provider/consumer in 
>> place of the OS I/O subsystem.
> 
> And that server/provider/consumer would be what?

See the pipeline example.

> Ada code in the 
> application? Or part of the Ada RTS? Would it have to be aware of all of 
> the OS I/O subsystem's services? For example, would it have to know 
> about the socket interface to networking?

The library supplied implementation of Send (see Write above) will. The 
client will call Send which in the case of non-blocking I/O will yield 
in Wait_For_Not_Full above and resume upon a callback or explicitly by 
the handler of the socket set. The co-routine case is having the handler 
and many callers to Send ran by the same OS thread (or by a much 
narrower pool of OS threads).

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-15 10:57                         ` Dmitry A. Kazakov
@ 2023-02-15 18:37                           ` Niklas Holsti
  2023-02-19  1:27                             ` A.J.
  0 siblings, 1 reply; 22+ messages in thread
From: Niklas Holsti @ 2023-02-15 18:37 UTC (permalink / raw)


On 2023-02-15 12:57, Dmitry A. Kazakov wrote:


> Clearly co-routine and OS thread have different properties as 
> implementations of same concept of a "thread of control."


I've been asking you to explain how those different properties would be 
visible to an Ada programmer who wants to use co-routines. That is, how 
would Ada co-routines differ from Ada tasks *for an Ada programmer*, but 
so far I've seen no answer from you that I find understandable. I think 
we can stop here.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-15 18:37                           ` Niklas Holsti
@ 2023-02-19  1:27                             ` A.J.
  2023-02-19  8:29                               ` Dmitry A. Kazakov
  2023-02-19 14:37                               ` Niklas Holsti
  0 siblings, 2 replies; 22+ messages in thread
From: A.J. @ 2023-02-19  1:27 UTC (permalink / raw)


Thank you for all of the responses and discussion, it pointed me in the right direction!  The "chat server"[1] (if you could call it that) does work, and my friends and I were able to telnet into it and chat.  One of my friends even tried throwing things at the server to break it, but it didn't crash!

Dmitry, maintaining a list of clients was the vital part I was missing.  I played around with using synchronized queues and tasks, but ended up defaulting to an ordered map with a UID as the key and wrapped it in a protected type.  I couldn't get Send() to send more data than Available_To_Send (after calling it, Available_To_Send ended up returning 0, and continued to do so despite wrapping Send() in a loop), but increasing the send buffer to 8kb per connection worked fine.  I would simply loop through that ordered map each time I needed to send something to all of the clients.

I really like simple components, and it would be neat if the GNAT maintainers implement epoll in the backend for linux systems, kqueue for BSD and MacOS.  Any server I write will be for linux though anyway.  I'm also interested in trying to benchmark Simple Component's connections server (both pooled and standard) against epoll to see how it fares.  Perhaps the clever tasking that the Connections Server utilizes can keep up with epoll despite what GNAT.Sockets utilizes!

Regarding coroutines vs tasks, I think at a high level it's hard to differentiate, but at a lower level, when I think of tasks vs what a coroutine would be, I think of Go, and their "goroutines."[2]  Creating a task in Ada, at least on linux, ends up creating a pthread, and you get all of the overhead that comes with threading (it's initialized in the kernel).  coroutines are managed by the go runtime (I believe in user space) and have much less overhead to create or manage, since it's not creating a specific thread.

Ada 202x supports the "parallel" block[3] though I understand no runtime has utilized it yet-- would that end up being a coroutine or is it meant for something else?

[1] https://github.com/AJ-Ianozi/protohackers/tree/main/budget_chat/src
[2] https://www.geeksforgeeks.org/golang-goroutine-vs-thread/
[3] http://www.ada-auth.org/standards/22rm/html/RM-5-6-1.html

Kind regards,
AJ.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-19  1:27                             ` A.J.
@ 2023-02-19  8:29                               ` Dmitry A. Kazakov
  2023-02-19 14:37                               ` Niklas Holsti
  1 sibling, 0 replies; 22+ messages in thread
From: Dmitry A. Kazakov @ 2023-02-19  8:29 UTC (permalink / raw)


On 2023-02-19 02:27, A.J. wrote:

> I couldn't get Send() to send more data than Available_To_Send (after calling it, Available_To_Send ended up returning 0, and continued to do so despite wrapping Send() in a loop),

As it must because the outgoing buffer is full. You must save the 
client's state and return (=yield) at this point. When a portion of data 
is sent (taken from the buffer) the client's Sent primitive procedure 
will be called and from there the client can restore its state and 
continue. You cannot loop, it is pointless. You must yield.

This was the essence of the whole discussion about the state machines 
vs. co-routines. Writing clients is *difficult*.

> but increasing the send buffer to 8kb per connection worked fine.

Sure in many cases there exist buffer size that guarantees sending [*]. 
This is not always the case. When not, the client must drop connection 
if it cannot sent. So, you will have unreliably working clients.

--------------------
[*] Many industrial protocols have bounded the packet size and work in a 
query-answer manner. For these the output buffer size is roughly the 
packet size.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: Broadcast / iterate to all Connection objects via Simple Components?
  2023-02-19  1:27                             ` A.J.
  2023-02-19  8:29                               ` Dmitry A. Kazakov
@ 2023-02-19 14:37                               ` Niklas Holsti
  1 sibling, 0 replies; 22+ messages in thread
From: Niklas Holsti @ 2023-02-19 14:37 UTC (permalink / raw)


On 2023-02-19 3:27, A.J. wrote:


> Creating a task in Ada, at least on linux, ends up creating a
> pthread,

With the current GNAT compiler, yes. But not necessarily with all Ada 
compilers, even on Linux.


> coroutines are managed by the go runtime (I believe in user space) 
> and have much less overhead to create or manage, since it's not
> creating a specific thread.

Some Ada compilers may have run-times that implement Ada tasks within 
the run-time, with minimal or no kernel/OS interaction.


> Ada 202x supports the "parallel" block[3] though I understand no 
> runtime has utilized it yet-- would that end up being a coroutine or
>  is it meant for something else?
> 
> [3] http://www.ada-auth.org/standards/22rm/html/RM-5-6-1.html

As I understand it, the parallel execution constructs (parallel blocks 
and parallel loops) in Ada 2022 are meant to parallelize computations 
using multiple cores -- that is, real parallelism, not just concurrency.

The Ada2022 RM describes each parallel computation in such a parallel 
construct as its own thread of control, but all operating within the 
same task, and all meant to be /independent/ of each other. For example, 
a computation on a vector that divides the vector into non-overlapping 
chunks and allocates one core to each chunk.

Within a parallel construct (in any of the parallel threads) it is a 
bounded error to invoke an operation that is potentially blocking. So 
the independent computations are not expected to suspend themselves, 
thus they are not co-routines.

The parallelism in parallel blocks and parallel loops is a "fork-join" 
parallelism. In other words, when the block or loop is entered all the 
parallel threads are created, and all those threads are destroyed when 
the block or loop is exited.

So they are independent threads running "in" the same task, as Dmitry 
wants, but they are not scheduled by that task in any sense. The task 
"splits" into these separate threads, and only these, until the end of 
the parallel construct.

Moreover, there are rules and checks on data-flow between the 
independent computations, meant to exclude data races. So it is not 
intended that the parallel computations (within the same parallel 
construct) should form pipes or have other inter-computation data flows.

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2023-02-19 14:37 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-07 20:29 Broadcast / iterate to all Connection objects via Simple Components? A.J.
2023-02-08  8:55 ` Dmitry A. Kazakov
2023-02-08  9:55 ` Jeffrey R.Carter
2023-02-13  7:28   ` Emmanuel Briot
2023-02-13  8:30     ` Dmitry A. Kazakov
2023-02-13  8:44       ` Emmanuel Briot
2023-02-13 10:55         ` Dmitry A. Kazakov
2023-02-13 11:07           ` Emmanuel Briot
2023-02-13 11:57             ` Dmitry A. Kazakov
2023-02-13 13:22               ` Niklas Holsti
2023-02-13 15:10                 ` Dmitry A. Kazakov
2023-02-13 16:26                   ` Niklas Holsti
2023-02-13 19:48                     ` Dmitry A. Kazakov
2023-02-15  9:54                       ` Niklas Holsti
2023-02-15 10:57                         ` Dmitry A. Kazakov
2023-02-15 18:37                           ` Niklas Holsti
2023-02-19  1:27                             ` A.J.
2023-02-19  8:29                               ` Dmitry A. Kazakov
2023-02-19 14:37                               ` Niklas Holsti
2023-02-13 15:43                 ` J-P. Rosen
2023-02-13 16:40             ` Jeremy Grosser <jeremy@synack.me>
2023-02-13 20:33 ` Daniel Norte de Moraes

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox