IOCP: Sockets and Async RPC

[Hector Santos]

Folks, I revamping our internet hosting session-based RPC
client/server framework around IOCP and Async RPC and I have some
questions in create the new framework.

Background:

The current framework is a traditional client/server model where there
is main thread RPC client as a hosting listing server and each
connection spawns a new RPC client session thread. The RPC Server
offers 250+ functions API DLL library for RPC clients to use.

I have the socket IOCP layout worked out for socket connections and to
dispatch commands for the particular hosting server.

To use one command example, SEARCH, the dispatch is:

BOOL iocpDisp_Search(LPCLIENT_CTX lpCtx, char *szText)
{
//
// illustrating sync RPC call
//
SOCKET t = lpCtx->hClientSocket;
TMsgHeader msg = {0};
while (wcMailSearch(szText,msg)) {
// do something with msg
Sendf(t,"%u,%u\r\n",msg.Id, msg.Number);
msg.Id++;
}
return TRUE;
}

The goal is to make the above async RPC using IOCP notification.

I guess I am not sure of:

1) If I should have two I/O complete ports, one for sockets,
one for handling Async RPC calls.

2) How to keep/pass/separate/link the RPC_ASYNC_STATE instance with
the socket LPCLIENT_TXT instance. In other words, do I need to
initialize a unique RPC_ASYNC_STATE per async RPC function?

Overall, how should I prepare the structures for what is otherwise two
IOCP "streams"?

Would it be better to use Async RPC with a callback notification?

Thanks in advance

[m]

In my experience, a single pool of worker threads (IOCP) for both IO and
processing is usually the most efficient design. Having said that, there
are many reasons why separate pools might be better including better
modularity of code, and easier support for NUMA and processor groups. One
can also implement multiple pools to support class of service etc. and
session quotas. The key is to remember that no matter how many threads,
there are still only X CPUs in the system and each context switch adds
overhead. Conversely, a simple design that minimizes context switches, can
leave you exposes to denial of sevice.

Folks, I revamping our internet hosting session-based RPC client/server
framework around IOCP and Async RPC and I have some questions in create
the new framework.
The current framework is a traditional client/server model where there is
main thread RPC client as a hosting listing server and each connection
spawns a new RPC client session thread. The RPC Server offers 250+
functions API DLL library for RPC clients to use.
I have the socket IOCP layout worked out for socket connections and to
dispatch commands for the particular hosting server.
BOOL iocpDisp_Search(LPCLIENT_CTX lpCtx, char *szText)
{
//
// illustrating sync RPC call
//
SOCKET t = lpCtx->hClientSocket;
TMsgHeader msg = {0};
while (wcMailSearch(szText,msg)) {
// do something with msg
Sendf(t,"%u,%u\r\n",msg.Id, msg.Number);
msg.Id++;
}
return TRUE;
}
The goal is to make the above async RPC using IOCP notification.
1) If I should have two I/O complete ports, one for sockets,
one for handling Async RPC calls.
2) How to keep/pass/separate/link the RPC_ASYNC_STATE instance with
the socket LPCLIENT_TXT instance. In other words, do I need to
initialize a unique RPC_ASYNC_STATE per async RPC function?
Overall, how should I prepare the structures for what is otherwise two
IOCP "streams"?
Would it be better to use Async RPC with a callback notification?
Thanks in advance
--
HLS

[Hector Santos]

Post by m
In my experience, a single pool of worker threads (IOCP) for both IO and
processing is usually the most efficient design. Having said that,
there are many reasons why separate pools might be better including
better modularity of code, and easier support for NUMA and processor
groups. One can also implement multiple pools to support class of
service etc. and session quotas. The key is to remember that no matter
how many threads, there are still only X CPUs in the system and each
context switch adds overhead. Conversely, a simple design that
minimizes context switches, can leave you exposes to denial of sevice.

There was two reasons: separation of I/O channels including one that
already exist (ncacn_ip_tcp) and not seeing how to associate an async
RPC "file handle object" with an iocp.

I'm not referring to RPC_ASYNC_STATE.u.IOC.hIOport but with
CreateIOCompletionPort(). I was presuming the RPC system manager has
logic to making the association and bind.

In other words, in this case, the RPC bind is ncacn_ip_tcp (socket) or
ncalrpc (some unknown kernel port, shared memory) and it is probably
already using an IOCP internally.

So far its working very well, but I'm still far from any real full
blown testing.

Thanks

[m]

AFAIK, there is no way ;) This is one of those 'other' reasons!

I am glad that you have it working at least for test

Post by Hector Santos

Post by m
In my experience, a single pool of worker threads (IOCP) for both IO and
processing is usually the most efficient design. Having said that, there
are many reasons why separate pools might be better including better
modularity of code, and easier support for NUMA and processor groups.
One can also implement multiple pools to support class of service etc.
and session quotas. The key is to remember that no matter how many
threads, there are still only X CPUs in the system and each context
switch adds overhead. Conversely, a simple design that minimizes context
switches, can leave you exposes to denial of sevice.

There was two reasons: separation of I/O channels including one that
already exist (ncacn_ip_tcp) and not seeing how to associate an async RPC
"file handle object" with an iocp.
I'm not referring to RPC_ASYNC_STATE.u.IOC.hIOport but with
CreateIOCompletionPort(). I was presuming the RPC system manager has logic
to making the association and bind.
In other words, in this case, the RPC bind is ncacn_ip_tcp (socket) or
ncalrpc (some unknown kernel port, shared memory) and it is probably
already using an IOCP internally.
So far its working very well, but I'm still far from any real full blown
testing.
Thanks
--
HLS